I had been doing scrapping from last 3-4 years. Already done with around 15000 websites so far with scrappy, celenium , browser automation in many real estate,webshop, news sites .