View Squish Test Results of Merge Requests in GitLab
Testing is always easier when it can be directly integrated. Take a look at this demonstration of how to use Merge with GitLab
Join the DZone community and get the full member experience.Join For Free
It’s very important to make automated GUI tests a part of a Continuous Integration (CI) process. Squish offers add-ons for many popular CI tools like Jenkins, Bamboo, and TeamCity. The full list is here. If your tool is not on the list, it can be still integrated as Squish tests because we support a command-line interface for all of our tools. Moreover, Squish can generate JUnit reports, which can be parsed easily by many CI or Test Management tools.
This article demonstrates how to run Squish tests on merge requests in GitLab. Squish will generate a JUnit report and we will configure a job in GitLab to use this report. Later GitLab will display a test summary on the merge request page. It contains information about newly failed tests, existing failures and resolved failures. Based on the above data, a decision about merge is made.
The first step is to configure GitLab Runner. We will configure the following environment variables:
- SQUISH_DIR – Squish Installation directory
- SQUISH_LICENSEKEY_DIR – Squish license location
- SQUISH_SERVER_PORT – port at which squishserver process is started
- DISPLAY – display used to display an application GUI. We use VNC Server to provide a headless display
The next step is to define a pipeline in
.gitlab-ci.yml file. Pipelines are defined by specifying jobs that run in stages. In our example, we define job named “squish-tests1” which is run in a test stage.
squish-tests1: stage: test script: - echo DISPLAY=$DISPLAY - echo $SQUISH_DIR - echo "Starting squishserver..." - $SQUISH_DIR/bin/squishserver --port $SQUISH_SERVER_PORT 1>server.log 2>&1 & - sleep 5 - echo "Register AUT..." - $SQUISH_DIR/bin/squishserver --port $SQUISH_SERVER_PORT --config addAUT addressbook /home/name/Squish/squish-for-qt-6.4.0/examples/qt/addressbook - $SQUISH_DIR/bin/squishrunner --port $SQUISH_SERVER_PORT --testsuite tests/suite_addressbook --reportgen junit,report.xml --reportgen stdout - echo "Stopping squishserver..." - $SQUISH_DIR/bin/squishserver --stop --port $SQUISH_SERVER_PORT & artifacts: reports: junit: report.xml paths: - server.log
The job performs the following actions:
squishserverand redirect its stdout and stderr output to a
- Register the Application Under Test (AUT)
squishrunnerto run the Test Suite and generate JUnit report and stdout report (so results are also printed to a screen)
- Collect artifacts
While working on a dev branch, we introduced a defect that is causing a failure in a test case. When committing a change, Squish Tests are executed.
Later in GitLab, we create a merge request to the master branch. On the Merge Request Page, a Test Summary shows a difference between test results on the master versus the merged branch.
In the picture above, a Test Summary shows that there is one new failed test. Based on this information, a decision about the merge can be made.
Published at DZone with permission of Tomasz Pawlowski. See the original article here.
Opinions expressed by DZone contributors are their own.