Selenium with Python Tutorial : A Detailed Guide for Automation

What is Selenium WebDriver?

A web page consists of different web elements, such as text boxes, checkboxes, buttons, etc. Web automation testing involves automating the tasks that have to be performed on those web elements. Selenium WebDriver is a popular web-based automation testing framework that is primarily used for automating tasks related to Web UI testing.

Selenium WebDriver does not interact directly with the web elements on a page. A browser-specific Selenium WebDriver acts as the bridge between the test script and the web browser. Selenium WebDriver is the main component that communicates with the web browser. Selenium locators are used for locating elements on the page so that appropriate methods can be used for interacting with the element.

Selenium WebDriver supports most of the popular programming languages used by developers and testers, namely – Python, Java, C#, Ruby, and more. It supports popular operating systems such as Windows, Mac OS, Linux, and Solaris.

Mozilla Firefox is the default web browser of Selenium WebDriver.

Selenium WebDriver Architecture

Understanding the communication between different blocks of Selenium is essential before looking into Selenium with Python. Selenium WebDriver APIs are used for communicating between programming languages and web browsers.

The Selenium WebDriver architecture comprises of the following blocks:

• Selenium Client Libraries
• JSON Wire Protocol
• Browser Drivers
• Browsers

Let’s take a detailed look into each of these components:

Selenium Client Libraries

As mentioned earlier, developers can use Selenium for performing automation testing with popular programming languages. Selenium Client Libraries or Selenium Language Bindings make this multi-language support possible in Selenium.

The focus of this Selenium Python tutorial is using Selenium with Python. Hence, we would require language bindings for Python. Language drivers for programming languages, including Python, can be downloaded from the official Selenium location for Client Libraries.

JSON Wire Protocol

JSON stands for JavaScript Object Notation. JSON Wire Protocol is used for the transfer of data between the server and client on the web. It is a REST (Representational State Transfer) API that facilitates information transfer between the HTTP Server.

Each web browser, namely – Chrome, Firefox, Internet Explorer, etc., has its own browser driver (or HTTP Server).

Browser Drivers

Browser drivers are primarily responsible for communicating with the corresponding web browser. Every browser has its own browser driver, and the same needs to be installed on the machine where automation testing will be performed.

Since communication with the web browser happens via the browser driver, the browser’s internal logic is not revealed. The browser driver adds the much-needed level of abstraction to the interaction with the browser.

When the browser driver receives any command (or request), it is executed on the respective browser, and the response of execution is sent to the web driver as an HTTP response.

Browsers

Selenium can be used with popular browsers such as Chrome, Firefox, Internet Explorer, Microsoft Edge, etc. The framework cannot be used for browsers whose browser driver is not available.

Selenium Suite of Tools

Selenium version v1 consisted of IDE, RC, and Selenium Grid. The latest stable version of Selenium is 3.141.59, and the Alpha version of Selenium 4 is 4.0.0-alpha-6. All the Selenium Client Libraries are compatible with Selenium 4.

Selenium WebDriver was introduced in Selenium v2, and Selenium RC was deprecated in Selenium 3. Selenium RC has a more complex architecture and also lacks in performance.

Selenium Suite consists of the following components:

• Selenium Integrated Development Environment (IDE)
• Selenium Remote Control (RC)
• Selenium WebDriver
• Selenium Grid

Let’s look into each of these components in greater detail in this section of the Selenium Python Tutorial:

Selenium IDE

Selenium IDE is a popular tool for playback and record testing. It was earlier available only as a Firefox plugin, but now Selenium IDE is also available as a Chrome add-on.

The current version of Selenium IDE has the provision of a command-line tool (SIDE Runner) that lets you run your .side project on a Node.js platform. Selenium WebDriver is required if you want to create test scripts using Selenium IDE.

Selenium RC

Selenium RC was considered the main component in Selenium until the introduction of Selenium WebDriver in Selenium v2. Selenium RC was widely appreciated, as it could overcome the same-origin policy, which caused major concerns when performing web automation testing. The same-origin policy was introduced for security purposes, and it made sure that contents on a web page are not accessible to a script from another domain (or site).

Selenium RC Server, which is an HTTP proxy server, was introduced for overcoming the same-origin policy. Hence, Selenium RC comprises the Selenium Client and Selenium RC Server.

Selenium RC Server was designed for tricking the browser such that it believes that the web application being tested and Selenium Core belong to the same domain.

To overcome JavaScript Security restriction (or same-origin policy), proxy injection mode was used. Here, Selenium RC Server sits between the browser & website(or app) under test and masks the test candidate (website or app) under a fictional URL.

These changes not only complicated the architecture but also resulted in elongating the test execution time. Hence Selenium RC is officially deprecated.

Selenium WebDriver

Selenium WebDriver is an automated testing framework used for the validation of websites (and web applications). It supports popular programming languages such as Python, C#, Java, Ruby, and more.

Selenium WebDriver was introduced in Selenium v2. As Selenium WebDriver communicates with a web browser using its corresponding browser driver, it does not require a component like Selenium RC Server (as in Selenium RC).

Selenium WebDriver for popular browsers can be downloaded from the links mentioned below:

In further sections of this Selenium WebDriver tutorial, we would look at using Selenium with Python framework such as PyTest.

Selenium Grid

Selenium Grid lets you run tests across different combinations of browsers, operating systems, and devices. Parallel test execution can be performed using a Selenium Grid, as tests can be executed in parallel against different browser and OS combinations in parallel.

Selenium Grid uses a Hub and Node architecture, where the Hub is the master and one or more Nodes are its slaves. Selenium Grid 4, which is still under Alpha, supports Standalone Mode, Fully Distributed Mode, and the traditional Hub & Node Mode.

TestMu AI offers an online Selenium Grid, which can be used to perform live interactive cross-browser testing of your public or locally hosted websites and web apps on 3000+ real mobile and desktop browsers running on a real operating system.

2M+ Devs and QAs rely on TestMu AI

Deliver immersive digital experiences with Next-Generation Mobile Apps and Cross Browser Testing Cloud

Sign up with googleSign up with Email

This certification is for professionals looking to develop advanced, hands-on expertise in Python automation testing with Selenium and take their career to the next level.

Here’s a short glimpse of the Selenium Python 101 certification from TestMu AI:

If you are preparing for an interview you can learn more through selenium interview questions.

Difference between Selenium WebDriver and Selenium RC

Though Selenium RC (Remote Control) and Selenium WebDriver are test automation tools that support different programming languages, they differ a lot in many aspects. The major difference lies in the architecture of both these tools.

1. Architecture

Selenium RC works in a similar way irrespective of the browser on which tests are performed. The following operations are performed when a test script is executed on Selenium RC:

• Selenium RC Server injects JavaScript Code, also called Selenium Core, into the web browser.
• On receipt of commands from Selenium RC Server, the Selenium Core executes those instructions as JavaScript commands.
• The browser on which the test is performed executes the commands from the Selenium Core and returns the summary of test execution to the Selenium RC Server.

This complicated the architecture of Selenium RC. On the other hand, Selenium WebDriver does not inject any custom script but works directly with the web browsers by using corresponding browser drivers. Selenium WebDriver is a successor to Selenium RC.

2. Speed of Test Execution

Selenium WebDriver makes direct calls to the web browser using the browser drivers for that particular browser. Hence, Selenium WebDriver is much faster in terms of test execution speed, as it follows a simplified architecture (i.e., Hub and Node model) compared to Selenium RC.

3. Built-in reporting mechanism

Selenium RC provides an automated HTML file that contains the test results, whereas the reporting feature is not available in Selenium WebDriver by default.

However, test reports can be generated easily with Selenium WebDriver when used with test automation frameworks such as TestNG, PyTest, etc.

4. Headless browser automation

Selenium WebDriver supports testing on HTMLUnit browsers (or headless browsers), whereas Selenium RC does not support it.

5. Ease of Use

Selenium WebDriver has a wide-range of commands that are user-friendly & very easy to use. On the other hand, Selenium RC offers a limited set of commands that are not so user-friendly as compared to Selenium WebDriver.

6. Support for new browsers

Selenium WebDriver interacts with a web browser through its corresponding Browser Driver. It uses the browser’s native support for automation. Hence, Selenium WebDriver works on those web browsers whose browser drivers (e.g., ChromeDriver for Chrome, GeckoDriver for Firefox, etc.) are available.

Configuration for a new browser has to be implemented as a part of a Selenium WebDriver release. Selenium RC does not have this limitation, as there is no such component as a browser driver involved in its architecture.

Automation Testing with Selenium Python

Now that you have a detailed understanding of Selenium WebDriver, we move to the part where we would demonstrate how to use Selenium with Python. We start this section of Selenium Python Tutorial by setting up Selenium, Python, etc., on Windows for Python web automation.

Setting up Selenium with Python

Follow the below-mentioned steps for setting up the development environment for Selenium with Python:

• Download Python for Windows and then install the same. Skip this step if you already have Python installed on your machine.
• For installing and managing any package in Python, PIP has to be installed on the machine. PIP is the package management system in Python. To install pip on Windows, download get-pip.py and save it in your machine.Now, navigate to the directory where get-pip.py is downloaded and saved – and execute the following command on the terminal to install it:

python get-pip.py

pip --version

• PyTest is more widely used than PyUnit (the default test framework with Python). Hence, we have used PyTest for demonstration purposes in this article. For installing the PyTest framework, execute the following command on the terminal:

pip install –U pytest

You can verify whether PyTest installation is successful or not by executing the below command:

pytest --version

Shown below is the output of the above command.

• If Selenium framework is not installed on the machine, execute the below command for installing Selenium framework:

pip install -U selenium

python -c "import selenium; print(selenium.__version__)"

The following command helps in retrieving the version of Selenium installed for Python:

Here is the output of the above command that confirms the version of Selenium installed on the machine:

• The test would be performed on the Chrome browser. Hence, you should download the Chrome WebDriver that matches the ‘major’ version of Chrome on your PC. We have Chrome version 84.0.4147.105 in our system; hence, we downloaded Chrome WebDriver with the same version.

  It is always a good practice to download the browser WebDriver (i.e., Chrome WebDriver in our case) in the location where the corresponding browser is installed. By doing so, you would not be required to specify it’s the path when instantiating Chrome WebDriver.

  

  The prerequisites for executing the Selenium test automation script with Python and PyTest are complete.

Demonstration of Selenium Automation Testing with Python

In this Selenium Python Tutorial, we have used the following test scenario for demonstration:

• Navigate to the URL https://lambdatest.github.io/sample-todo-app/
• Select the first two checkboxes
• Send ‘Happy Testing at TestMu AI’ to the textbox with id = sampletodotext
• Click the Add Button and verify whether the text has been added or not

Implementation

#Implementation of Selenium WebDriver with Python using PyTest
 
import pytest
from selenium import webdriver
import sys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from time import sleep
 
def test_lambdatest_todo_app():
    chrome_driver = webdriver.Chrome()
    
    chrome_driver.get('https://lambdatest.github.io/sample-todo-app/')
    chrome_driver.maximize_window()
 
    chrome_driver.find_element_by_name("li1").click()
    chrome_driver.find_element_by_name("li2").click()
 
    title = "Sample page - lambdatest.com"
    assert title == chrome_driver.title
 
    sample_text = "Happy Testing at LambdaTest"
    email_text_field = chrome_driver.find_element_by_id("sampletodotext")
    email_text_field.send_keys(sample_text)
    sleep(5)
 
    chrome_driver.find_element_by_id("addbutton").click()
    sleep(5)
 
    output_str = chrome_driver.find_element_by_name("li6").text
    sys.stderr.write(output_str)
    
    sleep(2)
    chrome_driver.close()

Code Walkthrough

Lines (1-6) : Modules such as pytest, sys, selenium, etc. are imported before the implementation of the test method.

#Implementation of Selenium with Python using PyTest
 
import pytest
from selenium import webdriver
import sys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from time import sleep

Lines (9-10) : The test method is test_lambdatest_todo_app(). Selenium WebDriver for Chrome is instantiated with webdriver.Chrome() thereby launching the Chrome browser.

def test_lambdatest_todo_app():
    chrome_driver = webdriver.Chrome()

Lines (12) : driver.get() method in Selenium is used for opening the URL under test, i.e., https://lambdatest.github.io/sample-todo-app/

chrome_driver.get('https://lambdatest.github.io/sample-todo-app/')

Lines (13) : driver.maximize_window() method is used for maximizing the browser window.

chrome_driver.maximize_window()

Lines (15-16) : The ‘Inspect Tool’ in Chrome is used for locating the required element on the web page.

Once the elements are located, the click() method is used for performing the necessary operation.

chrome_driver.find_element_by_name("li1").click()
chrome_driver.find_element_by_name("li2").click()

Lines (18-19): The driver.title() method is used for retrieving the title of the web page under test. Assert is raised if the expected title does not match the title of the web page displayed in the browser window.

title = "Sample page - lambdatest.com"
assert title == chrome_driver.title

Lines (21-23) : The web element (i.e., text box) where the text ‘Happy Testing at TestMu AI’ has to be entered is located using ‘Inspect Tool’ in Chrome.

The find_element_by_id() method is used with input as the ID of the element which we located using ‘Inspect Tool.’ The driver.send_keys() method is used for entering the sample text ‘Happy Testing at TestMu AI‘ in the text box with id ‘sampletodotext.’

sample_text = "Happy Testing at LambdaTest"
email_text_field = chrome_driver.find_element_by_id("sampletodotext")
email_text_field.send_keys(sample_text)

Line (26) : The ‘add’ button on the page is located using its ID property. Selenium click() method is performed to add the text ‘Happy Testing at TestMu AI’ to the list.

chrome_driver.find_element_by_id("addbutton").click()

Line (33) : The driver.close() Selenium method is used for closing the browser window under focus. This will release the resources held by the WebDriver instance.

chrome_driver.close()

Execution

Navigate to the directory where the test code is located and execute the following command on the terminal

pytest test_lambdatest_todo.py --verbose --capture=no

The –verbose option is used for setting the verbosity level to default. The –capture option is used to capture only stderr and not stdout.

Here is the snapshot of the test execution, which indicates that the test has PASSED.