Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Introducing PICQT for Writing Cucumber Tests With Iridium: Part II

DZone's Guide to

Introducing PICQT for Writing Cucumber Tests With Iridium: Part II

Here is Part II of this series on PICQT for writing Cucumber tests with Iridium.

· Web Dev Zone
Free Resource

Start coding today to experience the powerful engine that drives data application’s development, brought to you in partnership with Qlik.

Here is Part II of this series on PICQT for writing Cucumber tests with Iridium. Did you miss Part I? Read it here!

The “P” in PICQT

When running unit tests, you will typically mock out requests to dependent services like databases or REST interfaces, removing the uncertainty of network requests or the availability of external servers. This is not a luxury you’re not likely to have when running end to end tests run in Iridium, meaning you are at the mercy of network speed fluctuations, the backend services, 3rd party scripts that you inject into your web application and all of the dozens of other factors that determine the performance and reliability of a web application.

What this means is that there is going to be a trade-off between speed and reliability when running tests in Iridium.

When running tests in your CI system, you’ll want to favor reliability over speed. There is nothing worse than breaking builds because of unreliable end to end tests. Tests that produce too many false negatives are of little value, as people will just learn to ignore them.

When running tests locally, or using tests as a way to jump you to the point in the web application where you are actively developing, speed is more important. When running locally you may well be mocking backend services, and if the test fails 2 screens before the screen you wanted to get to, it is no big deal to manually fill in the last screens yourself.

Iridium has a number of steps that are used to configure the performance characteristics of a test. Steps like When I set the default wait time between steps to "2" will result in each step that interacts with elements on the page waiting for 2 seconds before moving to the next step, while steps like And I set the default wait for elements to be available to "10" seconds mean that steps will wait up to 10 seconds for the element they need to interact with to become available. Increasing these values allows Iridium to be more tolerant of slow web applications, while decreasing the values means the script will execute faster.

Steps that define how a test is run (as opposed to what actions a step will perform) are addressed with scenarios tagged with the @profile tag. In the example above, we have tagged a scenario with @profile @quick-profile, and the steps in the scenario reduce the amount of time that Iridium waits between each step that interacts with elements on the page.

To run the test with this quick profile, we use the following command:

java \
    -DmonochromeOutput=false \
    -DtagsOverride="~@detailed-test,@rebuild-cost-detailed-test;~@exit-step,@rebuild-cost-detailed-test;~@profile,@quick-profile" \
    -DtestDestination=Chrome \
    -DenableScenarioScreenshots=true \
    -DtestSource=./test.feature \
    -DsaveReportsInHomeDir=true \
    -DleaveWindowsOpen=true \
    -DstartInternalProxy=-browserMob \
    -DappURLOverride=https://application.com \
    -jar IridiumApplicationTesting.jar

The only change in this command from the one that was demonstrated in the section “The “I” in PICQT” is that we now run any scenario tagged with @quick-profile. The result is that the default wait time between steps is reduced to half a second, meaning the test will take significantly less time to execute and drop the developer to the screen that they are working on.

Other steps that you may want to include in a profile are steps like And I block access to the URL regex ".*?thumbnail.*" with response "500", which will use BrowserMob to block requests to thumbnail images. Blocking these kinds of requests can significantly reduce the amount of time a page takes to load.

Just be aware the when BrowserMob is disabled (as it is in the command above with startInternalProxy set to  -browserMob) these network modification steps will have no effect. Modifying network requests like this can be useful for quick tests run by your CI system (see The “Q” in PICQT), but will be of little value when returning control of the browser back to the developer.

The “T” in PICQT

Not all journeys through a web application leave the application in a state where you can continue the test. Let’s assume that after 4 failed attempts to enter a postcode the application assumes that you are scraping the site and kills your session. Once the session is dead, there is no way to continue testing.

This is an example of a terminal test. It is a dead end in the journey, and the only option we have after reaching this dead end is ending the test.

   @terminal-detailed-test @capture-address-detailed-test
    Scenario Outline: Test antiscraping security on postcode input
        And I clear the element found by "PostcodeInput"
        And I populate the element found by "PostcodeInput" with "<postcode>"
        Then I verify that the page contains the text "Invalid postcode"
        Examples:
          | postcode |
          | 1000     |
          | 2699     |
          | 0799     |
          | 3212     |
          | 4212     |

    @terminal-detailed-test @capture-address-detailed-test
    Scenario Outline: Exit after triggering a terminal condition
        And I skip all remaining steps

You can run this terminal test with the following command:

java \
    -DmonochromeOutput=false \
    -DtagsOverride="~@terminal-detailed-test,@capture-address-detailed-test;~@detailed-test;~@exit-step;~@profile" \
    -DtestDestination=Chrome \
    -DenableScenarioScreenshots=true \
    -DtestSource=./test.feature \
    -DsaveReportsInHomeDir=true \
    -DleaveWindowsOpen=true \
    -DstartInternalProxy=-browserMob \
    -DappURLOverride=https://application.com \
    -jar IridiumApplicationTesting.jar

The tag set passed to Iridium here results in detailed tests and exit steps being skipped, and all terminal detailed steps being skipped except for the ones tagged with @capture-address-detailed-test. This means that our terminal test can take advantage of all the previous scenarios which set up the state of the application, the terminal test can then run, and the test is exited.

The “C” in PICQT

Now our test has a mixture of scenarios that are either detailed tests that probe the edge cases or simply click and populate whatever elements are needed to complete a journey.

Running all scenarios (excluding terminal ones) provides us with a complete regression test of our web application. The test will take its time as it validates every widget and input looking for the slightest issue. By nature, these kinds of tests are time-consuming (expect them to take hours in large applications), and will most likely be run overnight to find any errors introduced the day before.

Run a complete test with the following command:

java \
    -DmonochromeOutput=false \
    -DtagsOverride="~@terminal-detailed-test;~@exit-step;~@profile" \
    -DtestDestination=Chrome \
    -DenableScenarioScreenshots=true \
    -DtestSource=./test.feature \
    -DsaveReportsInHomeDir=true \
    -DleaveWindowsOpen=false \
    -DappURLOverride=https://application.com \
    -jar IridiumApplicationTesting.jar

The tag set here excludes any terminal tests and scenarios that exit after a detailed test, and any special profile scenarios. Unlike all the previous examples, it does not exclude detailed test scenarios, meaning they will all be run.

Because this test is designed to run on a CI system overnight, there is no need to keep the browser window open, so the leaveWindowsOpen system property has been removed. Also, because we don’t need to have a functional browser after the tests are complete, we remove the startInternalProxy system property, which means that BrowserMob will be enabled by default.

The “Q” in PICQT

As you may have guessed, quick tests are the opposite of complete tests. These tests don’t delve into the detailed validation of every widget, which means they will complete relatively quickly. Quick tests can be run with every commit, giving developers timely feedback on any obvious errors that have been introduced.

Run a quick test with the command:

java \
    -DmonochromeOutput=false \
    -DtagsOverride="~@terminal-detailed-test;~@detailed-test;~@exit-step;~@profile" \
    -DtestDestination=Chrome \
    -DenableScenarioScreenshots=true \
    -DtestSource=./test.feature \
    -DsaveReportsInHomeDir=true \
    -DleaveWindowsOpen=false \
    -DappURLOverride=https://application.com \
    -jar IridiumApplicationTesting.jar

The tag set used here is the same as the complete test, while also excluding any scenario tagged with @detailed-test.

You may also want to enable a quick profile with quick tests:

java \
    -DmonochromeOutput=false \
    -DtagsOverride="~@terminal-detailed-test;~@detailed-test;~@exit-step;~@profile,@quick-profile" \
    -DtestDestination=Chrome \
    -DenableScenarioScreenshots=true \
    -DtestSource=./test.feature \
    -DsaveReportsInHomeDir=true \
    -DleaveWindowsOpen=false \
    -DappURLOverride=https://application.com \
    -jar IridiumApplicationTesting.jar

Conclusion

The PICQT standard provides a consistent set of tags that can be applied to Iridium scripts to give them maximum value and versatility. With one script developers can jump to the page they are working on instead of manually clicking through a web application, teams can be quickly notified of significant errors that are introduced during the day, while detailed regression tests can be run overnight to pick up more subtle errors.

Create data driven applications in Qlik’s free and easy to use coding environment, brought to you in partnership with Qlik.

Topics:
test ,web application ,tag ,application ,scenario ,iridium

Published at DZone with permission of Matthew Casperson, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}