DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
The Latest "Software Integration: The Intersection of APIs, Microservices, and Cloud-Based Systems" Trend Report
Get the report
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. PyTest Tutorial — Parallel Testing With Selenium Grid

PyTest Tutorial — Parallel Testing With Selenium Grid

Save time spent on Selenium test automation by running tests on parallel. Learn how to run parallel testing.

Himanshu Sheth user avatar by
Himanshu Sheth
CORE ·
Jul. 30, 20 · Tutorial
Like (15)
Save
Tweet
Share
8.81K Views

Join the DZone community and get the full member experience.

Join For Free

Selenium is one of the widely used test automation frameworks for automated browser testing. Selenium test automation is really helpful in testing websites or web apps on different combinations of browsers, operating systems, and devices. Giving better functional test coverage as the code is tested on a wide range of combinations.

Running these tests in sequence can be time-consuming, as you’d wait for one test to complete before running other tests. You can save a lot of time by performing tests in parallel, thus improving the scalability of your Selenium test automation. Parallel testing helps in performing tests on browsers simultaneously, providing better test coverage in a shorter time. 

In this Selenium Python tutorial, I’ll show. you how to run parallel tests in pytest using Selenium Grid. The Selenium Grid to run can either be local or cloud-based. For more information on setting up the local Selenium Grid, we recommend having a look at our detailed blog on Setting up Selenium Grid for automation testing.

How To Run Selenium Tests In Parallel With Python Using Pytest-xdist?

By default, PyTest does not support parallel testing which is extremely essential for scenarios such as automated browser testing. Parallel testing is a must-have to achieve continuous integration as tests can be executed at a rapid pace. To run Selenium tests in parallel with Python, you need to install the pytest-xdist plugin.

Features (or Execution Modes) of Pytest-xdist

It is a PyTest distributed testing plugin that extends python PyTest  with some unique execution modes mentioned below in this Selenium Python tutorial :

  • Multi-process load balancing — Multiple CPUs or hosts can be used for a combined test run. This aids in speeding up the development along with using special resources of machines.

  • LooponFail — Tests can be executed repeatedly in a sub-process. After every test run, the pytest re-runs all the tests that have failed before. This process is repeated until all the tests pass. This is considered the end of the test.

  • Multi-platform coverage — Different Python interpreters (e.g. PyTest, PyUnit, etc.) or platforms can be specified and tests can be performed parallelly on all of them.

Installation Of pytest-xdist To Run Selenium Tests In Parallel With Python

The xdist plugin can be installed by executing either of the following commands on the terminal:

pip install pytest-xdist

easy_install pytest-xdist

Shown below in this Selenium Python tutorial is the installation screenshot of xdist plugin to run Selenium tests in parallel with Python.

python tests

Command-Line Options for Parallel Testing

The pytest-xdist plugin provides command-line options for sending tests to multiple CPUs. The number of CPUs is passed after the option –n.

pytest -n <num-of-cpus>

The option speeds up the parallel execution for lengthy tests, as well as, tests that have frequent I/O (Input/Output) operations in the implementation. pytest-xdist (or xdist-plugin) has many other options, details of the same are available in the xdist-plugin documentation

Run Selenium Tests in Parallel With Python Using pytest-xdist Plugin

To demonstrate the usage of the pytest-xdist plugin to run Selenium tests in parallel with Python, I’ll take four test scenarios for this Selenium Python tutorial which I’ll execute on Chrome and Firefox web browsers. The four test scenarios are divided across two python files and the fixtures are shared via conftest.py. For more detailed information on PyTest fixtures and usage of conftest.py, you can refer to a previous Selenium Python tutorial on PyTest fixtures.

Test Scenario — 1 (Run Selenium tests in parallel with Python on Chrome Browser)

Test Case — 1

1. Navigate to the URL https://lambdatest.github.io/sample-todo-app/

2. Select the first two checkboxes

3. Send ‘Happy Testing at LambdaTest’ to the textbox with id = sampletodotext

4. Click the Add Button and verify whether the text has been added or not

Test Case — 2

1. Navigate to the URL https://lambdatest.com/

2. Compare the window title with the expected title

3. Raise assert if titles do not match

Test Scenario — 2 ((Run Selenium tests in parallel with Python on Firefox Browser)

Test Case — 1

1. Navigate to the URL https://www.google.com

2. Search for “LambdaTest”

3. Click on the first test result

4. Raise an Assert if the Page Title does not match the expected title

 Test Case — 2

1. Navigate to the URL https://lambdatest.com/blog

2. Compare the window title with the expected title

3. Raise assert if titles do not match

Implementation

Fixtures for invoking the Chrome and Firefox browser are added in Conftest.py. The file is placed in the root folder where the files that contain implementation for the test cases are also located.

Conftest.py

As different URLs are used for automated browser testing, the scope is set to class (instead of a session) and a new browser instance is loaded for each test.

Python
 




xxxxxxxxxx
1
17


 
1
#Run Selenium tests in parallel with Python for Selenium Python tutorial
2
import pytest
3
from selenium import webdriver
4
@pytest.fixture(scope="class")
5
def driver_init_1(request):
6
    web_driver = webdriver.Chrome()
7
    request.cls.driver = web_driver
8
    yield
9
    web_driver.close()
10
    
11
@pytest.fixture(scope="class")
12
def driver_init_2(request):
13
    web_driver = webdriver.Firefox()
14
    request.cls.driver = web_driver
15
    yield
16
    web_driver.close()
17

          



test_pytest_1.py

This file contains the implementation of Test Scenario — 1 (Run Selenium tests in parallel with Python on Chrome Browser). The @pytest.mark.usefixtures decorator is included for using the fixture driver_chrome_init().

It contains two test cases to run Selenium tests in parallel with Python:

  • test_lambdatest_todo_app() — The required web elements are located using Selenium methods such as find_element_by_name(), driver.find_element_by_id(). Once the elements are located, necessary commands such as click(), etc. are used to perform the required operation.

  • test_lambdatest_load() — The URL under test https://www.lambdatest.com is loaded and the window title is compared with the expected title. Assert is raised if the titles do not match.

close() command in Selenium WebDriver is used to close the browser window once the test execution is completed.

Python
 




xxxxxxxxxx
1
44


 
1
import pytest
2
import pytest_html
3
from selenium import webdriver
4
from selenium.webdriver.chrome.options import Options
5
from selenium.webdriver.common.keys import Keys
6
import time
7
from time import sleep
8
import sys
9
 
10
@pytest.mark.usefixtures("driver_init_1")
11
class BasicTest:
12
    pass
13
class Test_URL_Chrome(BasicTest):
14
    def test_lambdatest_todo_app(self):
15
        self.driver.get('https://lambdatest.github.io/sample-todo-app/')
16
        self.driver.maximize_window()
17
 
18
        self.driver.find_element_by_name("li1").click()
19
        self.driver.find_element_by_name("li2").click()
20
 
21
        title = "Sample page - lambdatest.com"
22
        assert title ==  self.driver.title
23
 
24
        sample_text = "Happy Testing at LambdaTest"
25
        email_text_field =  self.driver.find_element_by_id("sampletodotext")
26
        email_text_field.send_keys(sample_text)
27
        time.sleep(5)
28
 
29
        self.driver.find_element_by_id("addbutton").click()
30
        time.sleep(5)
31
 
32
        output_str =  self.driver.find_element_by_name("li6").text
33
        sys.stderr.write(output_str)
34
 
35
        time.sleep(2)
36
        
37
    def test_lambdatest_load(self):
38
        self.driver.get('https://www.lambdatest.com/')
39
        self.driver.maximize_window()
40
 
41
        expected_title = "cross-browser Testing Tools | Free Automated Website Testing | LambdaTest"
42
        assert expected_title ==  self.driver.title
43
        time.sleep(5)
44

          



test_pytest_2.py

This file contains the implementation of Test Scenario — 2 (Execution on Firefox Browser). The @pytest.mark.usefixtures decorator is included for using the fixture driver_firefox_init().

It contains two test cases to run Selenium tests in parallel with Python:

  • test_google_search() — The search box on Google homepage is located using find_element_by_xpath method of Selenium WebDriver. Once the element is located, search text (i.e. LambdaTest) is passed to the search box. After the search operation is performed, the first search result is located using its XPath. Click method of Selenium WebDriver is used to click on the first result, post which the window title is compared to check the success of the test.

  • test_lambdatest_blog_load() — Test URL https://www.lambdatest.comm/blog  is loaded and the window title is compared with the expected title. Assert is raised if the titles do not match.

close() command in Selenium WebDriver is used to close the browser window once the test execution is completed.

Python
 




xxxxxxxxxx
1
53


 
1
import pytest
2
import pytest_html
3
from selenium import webdriver
4
from selenium.webdriver.chrome.options import Options
5
from selenium.webdriver.common.keys import Keys
6
import time
7
from time import sleep
8
import sys
9
 
10
@pytest.mark.usefixtures("driver_init_2")
11
class BasicTest:
12
    pass
13
class Test_URL_Firefox(BasicTest):
14
    def test_google_search(self):
15
        self.driver.get('https://www.google.com/')
16
        self.driver.maximize_window()
17
        title = "Google"
18
        assert title == self.driver.title
19
 
20
        search_text = "LambdaTest"
21
        search_box = self.driver.find_element_by_xpath("//input[@name='q']")
22
        search_box.send_keys(search_text)
23
 
24
        time.sleep(5)
25
 
26
        # Option 1 - To Submit the search
27
        # search_box.submit()
28
 
29
        # Option 2 - To Submit the search
30
        search_box.send_keys(Keys.ARROW_DOWN)
31
        search_box.send_keys(Keys.ARROW_UP)
32
        time.sleep(2)
33
        search_box.send_keys(Keys.RETURN)
34
 
35
        time.sleep(5)
36
 
37
        # Click on the LambdaTest HomePage Link
38
        title = "cross-browser Testing Tools | Free Automated Website Testing | LambdaTest"
39
        lt_link = self.driver.find_element_by_xpath("//h3[.='LambdaTest: cross-browser Testing Tools | Free Automated ...']")
40
        lt_link.click()
41
 
42
        time.sleep(10)
43
        assert title == self.driver.title   
44
        time.sleep(2)   
45
 
46
    def test_lambdatest_blog_load(self):
47
        self.driver.get('https://www.lambdatest.com/blog/')
48
        self.driver.maximize_window()
49
 
50
        expected_title = "LambdaTest | A cross-browser Testing Blog"
51
        assert expected_title ==  self.driver.title
52
        time.sleep(5)
53

          



Two tests are performed in parallel. Hence, -n option (i.e. num_of_cpus) is set to 2. Execution is performed by invoking the following command on the terminal

pytest -s -v -n=2

Here is the screenshot of the execution which indicates that two tests are performed simultaneously (i.e. in parallel).

pytest

As seen below in this Selenium Python tutorial, all the four tests have passed to run Selenium tests in parallel with Python in the pytest.

pytest

Challenges With In-House Selenium Grid

There is a limitation when it comes to testing on a local machine as you can only load certain browsers (and their versions) on the local machine. For example, it is not feasible to keep different versions of a particular browser on the same machine. If you own a machine with Mac OS, you may find it difficult to perform automation browser testing on Internet Explorer or Edge browsers.

The ideal execution environment should be a network of different interconnected machines that have different browser environments so that tests can be executed in parallel. This is what Selenium Grid is built for. Selenium Grid was created as part of the Selenium Suite by Phillipe Hanrigou in 2008.

Time to Market (TTM) is particularly important for consumer-facing products. As a test manager, a conscious decision has to be taken on whether it is viable to set up an in-house testing infrastructure (or local Selenium Grids) with different VMs. Setting up the local infrastructure for rare (yet important) combinations such as cross-browser testing with IE on Mac OS can turn out to be a big task.

Below in this Selenium Python tutorial are some of the major challenges that testers might face when performing cross-browser testing on a local Selenium Grid:

1. Scalability — With every new browser release, the local Selenium Grid also needs to be scaled up. Scaling up will require a significant investment from an infrastructure point of view and the ROI (Return on Investment) might not be very high.

2. Limitations on cross-browser testing — The focus of testing your product against different browser combinations should be on improving the product quality. With 2,000+ browsers to choose from, parallel test execution is almost limitless. Along with the latest browsers (and platform combinations), testing has to be performed on legacy browsers as well.

By shifting cross-browser testing to an online Selenium grid, the test team can focus squarely on the task at hand, rather than being worried about the upkeep of the in-house test infrastructure.

3.  Reliability at a price — Development of a device lab, maintaining VMs and their licenses, and ensuring that they are functional (round the clock) can be a daunting and expensive task.

4. Keeping up with the latest Selenium releases — Online Selenium Grid is normally kept up to date with the latest Selenium releases, including Selenium Alpha releases. For example, Selenium 4 has a host of new features that should be made available to the test automation at the earliest. With the local Selenium grid, it would be a challenging task to update the Selenium infrastructure with every new release (especially the Alpha, Beta releases.)

Irrespective of the size of infrastructure, you would still need an in-house team to look after the reliability and maintenance of the Selenium grid infrastructure. The actual costs incurred in setting up the infrastructure can turn out to be significantly higher than the ballpark estimates.

Shifting the cross-browser testing activity to a fast and reliable online Selenium Grid such as LambdaTest helps in accelerating the automated browser testing process. It also leads to improving the team’s productivity as all activities related to Selenium automation testing are centralized in one place.

Getting Started With Parallel Testing on an Online Selenium Grid

The major advantage of using a reliable online Selenium Grid infrastructure is there is no need to install any additional software (or plugin) on your machine. The browsers on VMs have pre-installed versions of Adobe Flash, Adobe Shockwave, Adobe Reader, Microsoft Silverlight, etc. hence, making it easy to test RIA (Rich Internet Applications). To get started with cross-browser testing on LambdaTest, perform the following steps:

  1. Create an account on LambdaTest. The user-name and access key available in the profile section is required for accessing the Selenium Grid on LambdaTest.

  2. Depending on the test requirements, select the appropriate subscription plan. For starters, there is a Lite Plan which is free for lifetime and offers free 100 automation minutes for 15 days.

Once you have logged in to the platform, use the Selenium Desired Capabilities Generator for configuring your Selenium tests on the Selenium Grid.

Porting PyTest Automation Tests To Online Selenium Grid

Before porting the existing implementation to Selenium Grid, we generate the required browser capabilities using a capabilities generator. For the demonstration, we use the same examples that were demonstrated earlier.

Test Scenario — 1 (Browser: Chrome 80.0, Platform: Windows 10)

Test Case – 1

1. Navigate to the URL https://lambdatest.github.io/sample-todo-app/

2. Select the first two checkboxes

3. Send ‘Happy Testing at LambdaTest’ to the textbox with id = sampletodotext

4. Click the Add Button and verify whether the text has been added or not

Test Case — 2

1. Navigate to the URL https://lambdatest.com/

2. Compare the window title with the expected title

3. Raise assert if titles do not match

Test Scenario — 2 (Browser: Safari 12.0, Platform: macOS Mojave)

Test Case — 1

1. Navigate to the URL https://www.google.com

2. Search for “LambdaTest”

3. Click on the first test result

4. Raise an Assert if the Page Title does not match the expected title

Test Case — 2

1. Navigate to the URL https://lambdatest.com/blog

2. Compare the window title with the expected title

3. Raise assert if titles do not match

The Capabilities generator is used to generate capabilities for (browser + OS combinations). As the test framework being used is PyTest, we choose the language as Python.

Device Capabilities for Test Scenario — 1 (Browser: Chrome 80.0, Platform: Windows 10)

Java
 




xxxxxxxxxx
1


 
1
capabilities = {
2
        "build" : "Porting test to LambdaTest Selenium Grid (Chrome)",
3
        "name" : "Porting test to LambdaTest Selenium Grid (Chrome)",
4
        "platform" : "Windows 10",
5
        "browserName" : "Chrome",
6
        "version" : "80.0"
7
}
8

          



Device Capabilities for Test Scenario — 2 (Browser: Safari 12.0, Platform: macOS Mojave)

Java
 




xxxxxxxxxx
1


 
1
capabilities = {
2
        "build" : "Porting test to LambdaTest Selenium Grid",
3
        "name" : "Porting test to LambdaTest Selenium Grid",
4
        "platform" : "macOS Mojave",
5
        "browserName" : "Safari",
6
        "version" : "12.0"
7
}
8

          



The parameters supplied to the desired capabilities to run Selenium tests in parallel with Python are below in this Selenium Python tutorial:

Parameter

Description

Example

Build

Build name with which you can identify the build

Porting test to LambdaTest Selenium Grid

Name

Test name to identify the test being performed

Porting test to LambdaTest Selenium Grid

Platform

Platform/Operating System on which you intend the test to be performed

Windows 10, macOS Mojave, etc.

BrowserName

The browser on which the automation test would be performed

Firefox, Chrome, Microsoft Edge, Internet Explorer

version

Particular browser version on which the test would be performed

Firefox version 64.0, Chrome version 70.0, etc.

selenium_version

A version of Selenium which would be used for the testing

3.13.0

firefox.driver

Remote WebDriver version for Firefox

2.42

You can also enable the headless option to perform automated browser testing on web browsers without the GUI (Graphical User Interface).

lambdatest

As the tests are executed on the online Selenium Grid, credentials consisting of a combination of user-name and the access token is used to access the LambdaTest Grid URL — @hub.lambdatest.com/wd/hub.

Python
 




xxxxxxxxxx
1


 
1
remote_url = "https://" + user_name + ":" + app_key + "@hub.lambdatest.com/wd/hub"
2

          



The remote WebDriver API uses the remote URL and browser capabilities (ch_capabilities/saf_capabilities) which was generated using the LambdaTest capabilities generator.

Python
 




xxxxxxxxxx
1


 
1
web_driver = webdriver.Remote(command_executor = remote_url, desired_capabilities = saf_capabilities)



Conftest.py is used to share fixtures across files. Changes for porting from local Selenium Grid to Online Selenium Grid are only done in conftest.py. These are infrastructure changes to make the code work on the Online Selenium Grid.

Online Selenium Grid

Conftest.py

The changes are only related to porting the test to the online Selenium Grid.

Python
 




xxxxxxxxxx
1
41


 
1
# Import the 'modules' that are required for execution to run Selenium tests in parallel with Python
2
import pytest
3
from selenium import webdriver
4
import urllib3
5
import warnings
6
 
7
ch_capabilities = {
8
        "build" : "Porting test to LambdaTest Selenium Grid (Chrome)",
9
        "name" : "Porting test to LambdaTest Selenium Grid (Chrome)",
10
        "platform" : "Windows 10",
11
        "browserName" : "Chrome",
12
        "version" : "80.0"
13
}
14
 
15
saf_capabilities = {
16
        "build" : "Porting test to LambdaTest Selenium Grid",
17
        "name" : "Porting test to LambdaTest Selenium Grid",
18
        "platform" : "macOS Mojave",
19
        "browserName" : "Safari",
20
        "version" : "12.0"
21
}
22
 
23
user_name = "user-name"
24
app_key = "app_key"
25
@pytest.fixture(scope="class")
26
def driver_init_1(request):
27
    urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
28
    remote_url = "https://" + user_name + ":" + app_key + "@hub.lambdatest.com/wd/hub"
29
    # web_driver = webdriver.Chrome()
30
    web_driver = webdriver.Remote(command_executor = remote_url, desired_capabilities = ch_capabilities)
31
.........................
32
.........................
33
@pytest.fixture(scope="class")
34
def driver_init_2(request):
35
    urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
36
    remote_url = "https://" + user_name + ":" + app_key + "@hub.lambdatest.com/wd/hub"
37
    # web_driver = webdriver.Firefox()
38
    web_driver = webdriver.Remote(command_executor = remote_url, desired_capabilities = saf_capabilities)
39
.........................
40
.........................
41

          



test_pytest_1.py and test_pytest_2.py

There are no changes required in the core implementation as the changes are only related to the infrastructure. Hence, implementation in test_pytest_1.py and test_pytest_2.py remains the same as the one used for cross-browser testing on local Selenium Grid, even though we have used a different browser and OS combinations for cross-browser testing.

For executing the code on the online Selenium Grid on LambdaTest, we use the pytest-xdist plugin with several parallel tests set to 4. The reason for selecting 4 is because my current billing plan on LambdaTest enables the execution of 5 tests in parallel.

lambdatest 0/5 rating

Execute the following command on the terminal to run Selenium tests in parallel with Python:

pytest -s -v -n=4

Here is the screenshot of the e

xecution which indicates that four tests are running in parallel on the online Selenium Grid:

Here is the terminal execution screenshot indicating that all the four tests to run Selenium tests in parallel with Python have passed.

selenium testing

lambdatest chrome

As seen below in this Selenium Python tutorial, the tests have executed successfully on the online Selenium Grid.

porting test

All In All

In this Selenium Python tutorial, I looked at using the PyTest framework for local and online Selenium Grid to run Selenium tests in parallel with Python. Automated browser testing using the local Selenium Grid can hit a roadblock at any point in time as it is not scalable and requires a significant investment in setting up and maintaining the overall infrastructure. An online Selenium Grid helps in speeding up the testing process as tests can be executed in parallel on a reliable Selenium Grid. 

Testing on an online Selenium Grid also aids in improving the test coverage as testing can be performed on varied combinations of browsers, platforms, and devices. With minimal efforts (and code changes), existing test implementations can be ported to work with an online Selenium grid.

I hope you found this article informative and found it helpful. In case of any doubts, do reach out in the comment section down below. Also, I’d love it if you could share this Selenium Python tutorial with your peers and colleagues, this might help them if they're facing any challenges to run Selenium tests in parallel with Python. That’s all for now!

Happy Testing!

Testing Python (language) Test case Infrastructure Execution (computing) Implementation Test automation Continuous Integration/Deployment

Published at DZone with permission of Himanshu Sheth. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Configure Kubernetes Health Checks
  • Host Hack Attempt Detection Using ELK
  • Demystifying the Infrastructure as Code Landscape
  • What “The Rings of Power” Taught Me About a Career in Tech

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: