Web Scraping using Selenium

Web Scraping is a popular methodology to extract data from websites. This is often done to derive insights for Sentiment Analysis, Predicting User preferences, Cross-Selling products, etc. Some of the real-life examples of web scraping include – extracting data for pricing analysis, user ratings for movie sentiment analysis, corporate admin tasks to read and classify log files in an HTML, search bots trying to make sense of a results page. While web scraping activity does not provide intelligence of its own, as we have seen above the data extracted can be useful in multiple ways. A more common use case would be a start-up eCommerce website trying to set a price on its products based on market research on competitors.