5 Tips Of GUI Tests With Python + Selenium

I have been using Python + Selenium for years. Honestly, I'm far from a frontend expert or a QA expert. Here are my lessons learned from this journey.

If you're also a Python user and need GUI testing, check this out!

1. Use Docker to Setup Selenium Headless Environments
Setting up GUI testkit takes effort. Especially in the old days! Ever heard of something like Xvfb[1], Xephyr, PyVirtualDisplay[2], etc?

Too much information, isn't it? It will blow your mind. Not to mention the maintenance effort. And yes, the troubleshooting!

#1 Tip: Use Docker to Setup TestKit.

Check the dockerfile in GitHub.

# Start container
docker run -d --restart=always \
    -v /tmp/screenshot:/tmp/screenshot \
    -h selenium --name selenium denny/python-selenium:v1

# Run A Sample Test.
# With customized test scripts, you can do more!
docker exec selenium \
   python /home/seluser/selenium_load_page.py \
   --page_url https://www.dennyzhang.com --should_save_screenshot

# Check Screenshot
ls -lth /tmp/screenshot

With the above simple commands, your remote Xvfb server should be up-and-running now.

If it had crashed, your container will crash too. Consequently, it will be restarted automatically. Beautiful, isn't it?

2. Deal With Async Loading for JavaScript and CSS

contact$dennyzhang.com docker exec -it selenium ps -ef | grep -i xvfb
seluser     12     1  0 Sep04 ?        00:00:00 /bin/sh /usr/bin/xvfb-run -n 99
seluser     23    12  0 Sep04 ?        00:00:00 Xvfb :99 -screen 0 1360x1020x24

In web programming, async loading is quite common. Especially with Node.js. So before we check UI functionalities, we need to wait until for the elements to be shown.[3]

from selenium import webdriver

driver = webdriver.Firefox()
driver.implicitly_wait(10) # seconds
driver.get("http://somedomain/url_that_delays_loading")
myDynamicElement = driver.find_element_by_id("myDynamicElement")

3. Raise Alerts, if Page Load Is Too Slow
Apparently, the first priority is making sure the major functionalities work.

But also remember to watch out for slowness. This helps people to detect performance issues.

import time

print("Open page: %s" % (page_url))
start_clock = time.clock()

gui_login_webpage(driver, page_url, username, password)

end_clock = time.clock()
elapsed_seconds = ((end_clock - start_clock) * 1000)
is_ok = True
if elapsed_seconds > max_load_seconds:
    print("ERROR: page load is too slow. It took %s seconds, more than %d seconds." \
          % ("{:.2f}".format(elapsed_seconds), max_load_seconds))
    is_ok = False
else:
    print("Page load took: %s seconds." % ("{:.2f}".format(elapsed_seconds)))

4. Raise Alerts if Any 5XX or 4XX Errors Are Detected.
One URL request will launch lots of HTTP sub-requests. If any requests run into issues, we need to notify the developers.

IGNORE_ERROR_LIST = ["favicon.ico"]

all_warnings = driver.get_log('browser')
critical_errors = []
for warning in all_warnings:
    if warning['level'] == 'SEVERE':
        has_error = True
        for ignore_err in IGNORE_ERROR_LIST:
            if ignore_err in warning['message']:
                has_error = False
                break
        if has_error is True:
            critical_errors.append(warning)

5. Wrap GUI Test as Jenkins Jobs
People only accept solutions which are easy to use. So let's use Jenkins to achieve visualOps.

Quick Python Selenium CheatSheet[4]