Selenium In Python

A Beginner’s Guide To Browser Automations

Photo by Aideal Hwa on Unsplash

Selenium is an open-source tool that can automate tasks on a web browser. Its main purpose is testing, however, it’s a powerful tool for a developer/data scientist because it has many more applications like scraping and automating boring tasks like posting on pages on Facebook. In this post, we will show you the basics of selenium and how to use it on a simple example in python.

First things first, you need to install selenium:

pip install selenium

Also, you need to download chromedriver for your google chrome browser version which can be found here, and add it to your working directory.

Selenium works just like a human. We need to identify where the buttons or forms are and then perform an action like typing or pressing a button. This is all you need to know to understand its logic.

Example: How to automatically search on Google

Let’s start with this simple example. Firstly we need to import the libraries we need and set the cromedriver.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
import pandas as pd
import pickle
import time
 
driver = webdriver.Chrome(os.path.abspath( “chromedriver” ))

Now, if we run the following, a web browser will open at google.com.

driver.get(‘https://www.google.com/')

In order to continue, we need to press the “I agree” button so we need to identify its element in the HTML code of the website. This is because we are going to use its XPath(don’t worry if you don’t know what it is) to Selenium.

If you right-click in the browser and press inspect, the Dev-Tools will appear. Then, click on the top left corner of the Dev-Tools on a pointer-like symbol as you can see in the screenshot below. Now, if you hover over the element you want to find (in our case the “I agree” button) and click on it, its element should be highlighted in…