DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. Test Automation Analytics Using Cube.js

Test Automation Analytics Using Cube.js

A developer walks us through a use case in which he found Cube.js to be really helpful.

Shreyas Chaudhari user avatar by
Shreyas Chaudhari
·
May. 22, 19 · Tutorial
Like (7)
Save
Tweet
Share
9.32K Views

Join the DZone community and get the full member experience.

Join For Free

Location: Thiksay Monastery, Leh Manali Highway, India. Photographer: Self. It's been a decade in Quality Engineering and Test Automation, and my observation has been that visibility is generally missing when it comes to Test Automation. Yes! You heard it right. Visibility!

It is often the case that the technical aspects do not matter much, as much as the non-technical ones.

At times, I even got a feeling that Test Automation was becoming the responsibility of only the QAs when every member of the team is responsible for the Quality aspect. And the reason for that was also missing visibility of Test Automation at times. Since you don’t see what's going on on a day to day basis, you don’t hold yourself responsible for it. And, at times, don’t see much of value in it as well.

Out of Sight, Out of Mind.

One of the first and the foremost things that the team practicing Agile believes in is Visibility. Visibility in the form of Story Walls, Barometer (To track overall sprint progress), Build Monitors, Burndown Charts, Ice-cream Meters (Who is attending Stand ups on time and who is not) is very common among the team practicing Agile.

Visibility of automation is not only important for Quality Analysts or Test automation engineers but for every member of the Scrum team, i.e. BAs, Devs, UX Designers, Product Owners, Product Managers, and Business Stakeholders. Test automation metrics are direct indicators of the health of the product and hence hold a very high significance. Metrics help in making effective decisions.

The first Test Automation dashboard I had created was exactly a decade ago using JSPs and MySQL. After that, I used Spring Boot, MySQL, and a UI framework from time to time. But, with the rise of Node.js and the mammoth rate at which JS frameworks are coming up, this time I thought of exploring something beyond the traditional stack.

As I was exploring various options, something that grabbed my attention was Cube.js. I looked at some of the examples on the Cube.js website. Looking at the live demos, I realized that this is what I was looking for. I had a quick glance at the documentation and found it to be neatly organized, detailed, and well done. I joined the Slack channel to see how the support is and it was pretty good. Almost all questions were answered with a decent turn around time. In myy personal experience , I asked two questions on the Slack channel and both were resolved within minutes.

So, to be precise, what Cube.js needs from its user is a solid understanding of your database. And it does the heavy lifting in terms of wiring or chaining the components. A bit of plumbing gets the show up and running in a few hours or less. In less than two days I created this Test Automation Dashboard. The database that I have used is MySQL. The database schema is as in the snapshot below.

Database Schema

test automation results database schema

Let us have a look at each of these tables:

  • TbExecution — This table holds the details about the test automation execution. Every time that test automation execution is triggered, a new Execution entry is added to this table.
  • TbFeature — This table holds the details of all the features in a particular product.
  • TbTestplan — This table holds the details of the testplan in all the features. The relationship here is many to one, i.e. one feature will have multiple testplans.
  • TbTestcase — This table consists of all the testcases that belong to all the testplans. The relationship here is many to one, i.e. one testplan will have multiple testcases.
  • TbExecutionDetails — This is the table that holds the execution results of all automation runs. It has the execution status of every testcase in a testplan across features executed during a specific test automation execution.

test automation results database

One of the basic things about this entire set up of MySQL and Cube.js is that it is test automation framework agnostic. Meaning, irrespective of you using TestNG-Selenium or TestNG-RestAssured or Jasmine-Protractor or Mocha-SuperTest or any other framework of your choice, all you have to focus on is updating the MySQL database after the test automation has run. As long as the underlying language used for test automation is able to connect to the MySQL database and update the results post-execution, the desired charts/metric will be displayed by Cube.js.

Test Automation Framework and Cube.js

Test Automation Framework and Cube.js

The goal behind creating this test automation dashboard is to answer some of the questions regarding the overall test coverage. Questions like:

  1. In the last five automation executions, what is the overall test case pass percentage? Test Case Pass Percentage
  2. In the last five automation executions, what is the overall pass/fail ratio?Pass Fail Test Case Count
  3. What are the total test cases executed, pass/fail rates, and pass percentage for the last five executions? Test Case Count Per Execution
  4. What are the total test cases executed and test passed/failed in a test plan in every feature? Test Case Count Per Feature Per TestPlan

Now that we understand the goals behind coming up with this dashboard, let's take a look at its folder structure. The database schema and the folder structure can be cloned from here. The folder structure is as in the snapshot below.

test automation Cube.js folder structure

In the cloned git repository you'll see an additional folder — mysql-schema — which holds the MySQL database schema. That is the sample test data along with the database schema that we will be using to generate the charts as in the snapshot above.

Pre-Requisites

  1. Node
  2. npm
  3. MySQL

Step by Step Guide

Step 1: Verify if Pre-Requisites are properly installed.

a. Node.js and npm Node.js and npm version

b. MySQLMySQL Installation

Step 2: Set up the “testautomationresults” database using the “testautomationresults.sql” file from “mysql-schema” folder.

mysql -u root -p

drop database testautomationresults;

create database testautomationresults;

use testautomationresults;

SOURCE C:\testautomationresults.sql

Database setup

Step 3: Git clone the repository using the command below:

git clone https://github.com/Shreyasc27/testautomationdashboard-cubejs.git

Step 4:

cd testautomationdashboard-cubejs

Edit the “.env” with the appropriate MySQL details :

Cube.js Environment File

Step 5:

cd testautomationdashboard-cubejs

npm install

npm run dev

Development Server

As in the cmd above, open the URL: “http://localhost:4000”. The evelopment environment, as shown below, will be displayed.

Cube.js Development Environment

For further exploring this development environment, refer to the document on the Cube.js website.

Step 6:

Running the commands below will generate a “build” folder in “dashboard-app”.

cd testautomationdashboard-cubejs\dashboard-app

npm install

npm run-script

npm run-script build

Create build

Step 7: Copy the generated “build” folder to any other directory.

cd build

npm install http-server -g

http-server

Copy generated build folder

Step 8: Click on any of the URLs listed. the Test Automation Dashboard should be up and running.

Test Automation Dashboard

Test Automation Dashboard

Test Automation Dashboard

References

The git repository here can be cloned to use (as is), updated, or enhanced with the Test Automation Analytics Dashboard using Cube.js.

Test automation Test data Analytics

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Utilize OpenAI API to Extract Information From PDF Files
  • AWS Cloud Migration: Best Practices and Pitfalls to Avoid
  • Event Driven 2.0
  • Building a Scalable Search Architecture

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: