I will write web scraping script/spider/crawler using python, you just need to click on the script and rest will be take care by script itself.
Libraries i use to write the script:
- Scrapy
- BeautifulSoup
- Requests
- Lxml
- Pandas
- Selenium
- RegEx
- csv
- json
Features:
- Multithreaded (process of executing multiple threads simultaneously)
- Write the data into any database (csv, json, xml, mongoDB, google sheets)
- Scrape heavy JS pages using selenium (headless)
- Proxy rotation
- AJAX page scraping using POST
You will receive a well commented code/script with advanced error handling.