Using Parsel instead of Beautiful Soup for Web Scraping

Mon, 28 Nov 2022 00:00:00 +0000

Web scraping is an automated process to extract data from web page, and since Python is one the most popular programming languages it’s common to see people use it for doing web scraping tasks like me :)

For a long time, I have been using Beautiful Soup 4 to extract data from web pages’ HTML markup, it’s popular, easy, robust, and battle-tested library for navigating, searching, and modifying the DOM tree. But, recently I came across Parsel, another HTML parsing library that supports XPath selectors, which is missing in Beautiful Soup, and I was in need of using something that can extract data from HTML using XPath (rather than Scrapy, funny enough, later I knew that Scrapy uses Parsel under the hood :D), so I decided to get it a try.

Debugging and fixing Selenium's send_keys() wrong text input

Mon, 28 Sep 2020 00:00:00 +0000

There are many kinds of bugs you may face while programming, but without a doubt, ghost bugs are the worse!

I recently faced a weird problem while working on a freelance Selenium (A portable framework for testing web applications) project in Python which is: send_keys method is sending random wrong input. Here are the details of the problem, how did I debug it, and how I managed to fix after hours of investigating!

Python on yshalsager

Using Parsel instead of Beautiful Soup for Web Scraping

Debugging and fixing Selenium's send_keys() wrong text input