Browserium - An Open Source Contribution to Py Pi

Hello readers, this is an article that I will be posting today for a module that I have contributed to PyPi (Python Package Index). The module name is called "Browserium." Let's see what problem the module would aim to resolve.

Problem Statement

With the very fast pace of development, it has now become very important to have a regular release cycle and it should be also kept in mind that we do a quality release. For this reason, we have to have our tests automated as well so that we can have centralized reports for regressions and other flaws in the system at the end of each build.

Now, for a stable build, we have to check that our application is compatible with different browsers and platforms. When we start implementing a framework based out of Selenium WebDriver, for the code to get executed in different browsers we have to configure each of the browsers separately and make a call to the browser based on the requirement. That is a large amount of code written.

Idea

The problem statement that has been defined above was the reason I took out time to ease this entire process of setting up the browser drivers and the respective configurations for each browser. What if this process can be reduced down into a few steps of execution? That is how I ended up with the idea and the implementation of Browserium.

The intended audience for this module is for Selenium enthusiasts and joining the dots which would help reduce lines of code and come out with better approachable functionalities.

What Does Browserium Provide?

• Single-step download the required browser drivers.
• Single-step update the required browser drivers.
• Create a single instance for your required browser object. No more code required to configure your browsers separately.
• A set of browser related generic functions that can be utilized for debugging as well as for achieving the required functionalities. So, we are reducing quite an effort over here as well!!
• You can run browsers Chrome and Firefox using the headless option as well so that it is comfortable running your framework on the server as well.

Browserium by Functionality

There are two ways in which you can use Browserium.

• Download the required browser driver, create an instance for the specific browser driver class.
• Download the required browser driver, create an instance for the specific browser driver class, create an instance for the browser controller class and use the generic functions to get started with your framework.

You can refer to the above diagram for reference.

Installing and Updating driver packages

Install Browserium Module

• To install Browserium using PiP run the command:

pip install browserium

• To install Browserium from GitHub run the command:

pip install git+git://github.com/browserium/Browserium.git

Modules Installed with Browserium

• requests
• selenium
• wget
• python-daemon

Make sure you have ssh configured in GitHub. You can also use https as well to install the module. But preferable would be if you have ssh configured in GitHub.

To install Browserium from GitHub using HTTPS run the command:

pip install git+https://github.com/browserium/Browserium.git

• To download chrome driver run the command browserium download --driver=chromedriver
• To download gecko driver run the command browserium download --driver=geckodriver
• To download opera driver run the command browserium download --driver=operadriver

Note: There might be a possibility that you have to provide write permission to the "/usr/local/bin" directory. Just run the command:

sudo chmod 757 /usr/local/bin/

Update Drivers

• To update Chrome driver run the command browserium update --driver=chromedriver
• To update Gecko driver run the command browserium update --driver=geckodriver
• To update Opera driver run the command browserium update --driver=operadriver

Get Started with Browserium

Browser Controller Class by Functionality

The Browser Controller class provides you with some eccentric methods that can be utilized to achieve the required functions.

• get_url (driver, URL): request the required URL entered. Pass the required driver object and the "URL" as parameters.
• implicit_wait_time (driver, time): Apply implicit wait before the dom loads. Pass the required driver object and the time as parameters.
• set_window_size (driver, height, width): Set the window size for the current running browser. Pass the required driver object, height and the width of the window.
• get_current_url (driver): Get the current URL. Pass the required driver object as a parameter.
• get_network_requests (driver): Get all the network requests for the current page. Pass the required driver object as a parameter.
• performance_metrics (driver): Get required page performance data. Pass the required driver object as a parameter.
• check_console_logs (driver): Get all console logs. Pass the required driver object as a parameter.
• get_page_source (driver): Get the current page source. Pass the required driver object as a parameter.
• get_site_cookies (driver): Get all the site cookies. Pass the required driver object as a parameter.

Creating an Instance of Chrome Using Browserium

• Create an instance for the ChromeDriverObject class
• Use the instance for ChromeDriverObject class to call the set_chromedriver_object method.
• Create an instance for the Browser_controller class to use the generic methods.

from browserium.generic_functions.chrome_object import ChromeDriverObject
from browserium.generic_functions.browser_controller import Browser_controller
from time import sleep

class Test_1():
def test_chromedriver_type1(self):
      chromedriver = ChromeDriverObject()
      controller = Browser_controller()
      driver = chromedriver.set_chromedriver_object()
      controller.get_url(driver, "https://www.google.co.in")
      controller.implicit_wait_time(driver, 4)
      current_url = controller.get_current_url(driver)
      print current_url

• To run chrome driver using the headless feature you have to pass the argument '--headless' in the set_chromedriver_object() method.

from browserium.generic_functions.chrome_object import ChromeDriverObject
from browserium.generic_functions.browser_controller import Browser_controller
class Test_1():
  def test_chromedriver_type1(self):
    chromedriver = ChromeDriverObject()
    controller = Browser_controller()
    driver = chromedriver.set_chromedriver_object('--headless')
    controller.get_url(driver, "https://www.google.co.in")
    controller.implicit_wait_time(driver, 4)
    current_url = controller.get_current_url(driver)
    print current_url

Create an Instance of Firefox Using Browserium

• Create an instance for the GeckoDriverObject class
• Use the instance for GeckoDriverObject class to call the set_geckodriver_object method.
• Create an instance for the Browser_controller class to use the generic methods.

from browserium.generic_functions.gecko_object import GeckoDriverObject
from browserium.generic_functions.browser_controller import Browser_controller
class Test1():
  def test_geckodriver_type1(self):
    geckodriver = GeckoDriverObject()
    controller = Browser_controller()
    driver = geckodriver.set_geckodriver_object()
    controller.implicit_wait_time(driver, 4)
    controller.get_url(driver, "https://www.google.co.in")
    current_url = controller.get_current_url(driver)
    print current_url
    print driver.title

from browserium.generic_functions.opera_object import OperaDriverObject
from browserium.generic_functions.browser_controller import Browser_controller
class Test1():
  def test_operadriver_type1(self):
    operadriver = OperaDriverObject()
    controller = Browser_controller()
    driver = operadriver.set_operadriver_object()
    controller.implicit_wait_time(driver, 4)
    controller.get_url(driver, "https://www.google.co.in")
    current_url = controller.get_current_url(driver)
    print current_url
    print driver.title

With the latest version of Browserium (v1.1.1) I have introduced a feature where you can stash all your logs in real time using the services of Logstash. The reason for using Elastic Search, Logstash and Kibana is to give the users a real time dashboard which can give an idea of the statuses for the tests executed in real time.

Logstash is an open source server-side data processing pipeline that ingests data from multiple resources and sends it to the respective stash, in this case, it would be Elasticsearch.

Elasticsearch is a restful service that helps store and uncover the respective data. Kibana is your dashboard to the answer for the data stored using Elasticsearch. Kibana lets you visualize your Elasticsearch data and navigate the Elastic Stack.

To know more about the ELK stack please follow this link. The ELK stack is entirely open sourced and can be used to set up your analytics stack.

To install ELK stack please follow the respective steps as mentioned in this link

How to Submit Log Events to Logstash?

To send log events to Logstash using Python, I am using a module called "Python Logstash Async". Python Logstash async helps to collect log events and transmit the collected events in a separate worker thread to Logstash. Browserium provides the codebase to log the required events through Python Logstash async as a wrapper. The basic configuration to run the build for Logstash is also provided by default and is executed as a background job.

But this process is optional, and is at users discretion to utilize it. If the user wants to run the Logstash build then the user has to run this command

browserium logstash --driver=logstash_build

To stash the events using Logstash please follow the required codebase

from browserium.generic_functions.logger import Logstash
class Test_1():
    def test_1(self):
      log = Logstash()
      log.logstash_type("INFO","Test")
      log.logstash_type("ERROR","Test")
      log.logstash_type("WARNING","Test")

So basically, you have to call the module Logstash and pass your required log type ("INFO", "ERROR", "WARNING") and log message as parameters to the logstash_type method.

Once, the stashing process is complete then basically you need to go to the Kibana dashboard. The services of Kibana runs on the port 5601.

Get Started with Kibana

To get started with Kibana please follow the required steps.

1. First, visit the management tab as shown in the picture.

2. Click on "Index Patterns"

3. Click on the button "Create Index Pattern"

4. You will be able to see the "indices" listed below. Please check the screenshot for more reference.

5. Filter the "index pattern" that has to be defined and click on the "Next Step" button. Please check the screenshot for more reference.

6. This lists every field in the Logstash indices and the fields recorded using elastic search.

7. Once the Logstash indices are populated you can create a decisive dashboard using the raw metric data. Please check the screenshot for more information.

8. This is the Logstash data that has been recorded using Elasticsearch. Please check the screenshot for more reference.

Upcoming Progress to Be Made in The Module

Here is some of the progress that I would like to release in the upcoming versions of Browserium.

1. Desired capabilities for browsers to be dynamically passed as object for all respective browsers.

2. Currently, there is no support for Microsoft Edge that has been provided. Later versions to include support for Microsoft Edge as well.

Note

• The module is built on python version 2.7
• The module is OSI approved MIT licensed

References

PyPi - https://pypi.org/project/browserium/

GitHub - https://github.com/browserium/Browserium