I have been scraping websites since last 2 years. I have done scraping using scrapy framework. I am experienced in crawling of anti scraping or scraping friendly websites. I have almost used to all features of scrapy framework like proxy use, concurrent requests etc. I have used aws s3 and mysql database to store the scraped data.