i am a newbie to scraping and selenium. The page which i want to scrape uses a js script on a button to get to the next page. I found part of a code (Click a Button in Scrapy) on SO but i cant seem to make it work.
from selenium import webdriver
import scrapy
class chSpider(scrapy.Spider):
name = 'spidypy'
allowed_domains = ['117.145.177.252']
start_urls = ['http://117.145.177.252/login.do?method=enterPdamccx']
def __init__(self):
self.driver = webdriver.Firefox()
def parse(self,response):
self.driver.get('http://117.145.177.252/login.do?method=enterPdamccx')
while True:
try:
next = self.driver.find_element_by_xpath('/html/body/form/div[3]/div/div/a')
url = 'http://117.145.177.252/login.do?method=enterPdamccx'
yield Request(url,callback=self.parse2)
next.click()
except:
break
self.driver.close()
def parse2(self,response):
print('you are here!')
I recieve following error messages several times:
selenium.common.exceptions.WebDriverException: Message: connection refused
JavaScript questions and answers, JavaScript questions pdf, JavaScript question bank, JavaScript questions and answers pdf, mcq on JavaScript pdf, JavaScript questions and solutions, JavaScript mcq Test , Interview JavaScript questions, JavaScript Questions for Interview, JavaScript MCQ (Multiple Choice Questions)
