Capturing a Screenshot on Test Failure for Parallel Execution With Appium and TestNG

This is a tutorial post. If you need just the solution you can jump at this topic, but I strongly recommend you to read the introduction.

Introduction

One of the basic test automation architecture items is ti know when your test fails and have  evidence of it. For front-end tests, we commonly use a screenshot to show where the error occurred.

Usually, the screenshot name is the test name. Some people add the date and time to track this information, but for parallel execution, we need to add more information on the screenshot name.

The example will show an Appium test script developed with Java and the use of TestNG to facilitate the approach to capture a screenshot.

The Basics

How to Capture Screenshots With Appium

In my case, I'll save a file in a directory; I need to call the getScreenShotAs method from driver, ansd pass the parameter OutputType.FILE.

After that, I use the FileUtils class from Commons IO to copy our file (screenshot) to a physical directory.

File file = driver.getScreenshotAs(OutputType.FILE);
FileUtils.copyFile(file, new File("myScreenshot.png");

How to Add It to Your Test

Basically, you'll add the lines above to some strategy on test failure. The basic one, it adds it to a try-catch block. It's "ugly," but it works.

// some code ommited
try {
  // add the interactions
} catch (Throwable t) {
  // capture the screenshot
  // fail the testcase
}

The problem with this approach is that you need to add this block in every test case (automated script) and, depending how many tests you have, it is a lot of work. And remember, DRY.

The Problem With Parallel Test Execution

If you are running your test in parallel, there's a solution to how you will differentiate the screenshot file name. Using the previous approach to give the test class name or test name will cause a file override. Adding the date and time will cause a manual work to see which device that screenshot belongs.

The Solution

Create a Test Listener

In TestNG, you can implement some listeners to modify TestNG's behavior. The ITestListener (doc | javadoc) add a capability to be notified, in real-time, about test starts, passes, fail, etc…

This interface (ITestListener) has a method, onTestFailure, that will be called every time your test fails. So, we have the listener to add the screenshot generation.

Simply create a class and implements ITestListener:

public class TestListener implements ITestListener {

    @Override
    public void onTestStart(ITestResult iTestResult) {}

    @Override
    public void onTestSuccess(ITestResult iTestResult) {}

    @Override
    public void onTestFailure(ITestResult iTestResult) {}

    @Override
    public void onTestSkipped(ITestResult iTestResult) {}

    @Override
    public void onTestFailedButWithinSuccessPercentage(ITestResult iTestResult) {}

    @Override
    public void onStart(ITestContext iTestContext) {}

    @Override
    public void onFinish(ITestContext iTestContext) {}

}

Add the Screenshot Capture on Test Failure

It's easy to add a screenshot on test failure. Just add the screenshot code (you have seen this above) in the onTestFailure method!

But, you'll face the same problem I had: how to capture the driver object to use in the getScreenshotAs method.

The simple snippet is below:

@Override
public void onTestFailure(ITestResult iTestResult) {
    File file = driver.getScreenshotAs(OutputType.FILE);
    FileUtils.copyFile(file, new File("myScreenshot.png");
}

How to Get the Driver Object in the TestListener Class

With the iTestResult information (coming from the parameter) you can access the test class where the error occurred. Basically, it's a reflection.

You need to get the class, field (the driver), and value of this field (driver instantiated). The driver is on the test, and not on a base class or something related, because of the parallelism.

The basic snippet for this is here:

@Override
public void onTestFailure(ITestResult iTestResult) {
    Class clazz = iTestResult.getTestClass().getRealClass();
    Field field = clazz.getDeclaredField("driver");

    field.setAccessible(true);

    AppiumDriver<?> driver = (AppiumDriver<?>) field.get(iTestResult.getInstance());
}

Now you just need to add the code for capturing the screenshot. We're almost there!

How to Associate the Listener With the Test

Simply add an annotation @Listeners passing as parameter you custom listener class.

@Listeners(TestListener.class)
public class MyTest {
}

Now every time your test fails, and the screenshot will be taken.

How to Solve the Screenshot Name Problem

Now, for the complete example, we need to solve this problem. As we use parameters on the test to execute the same test on different devices, a trick is to add these parameters in the screenshot file name. It's also a good practice (or recommended practice) to add the test class and test name in the screenshot file name.

I've created a helper method to generate the filename:

private String composeTestName(ITestResult iTestResult) {
    StringBuffer completeFileName = new StringBuffer();
    completeFileName.append(iTestResult.getTestClass().getRealClass().getSimpleName()); // simplified class name
    completeFileName.append("_");
    completeFileName.append(iTestResult.getName()); // method name

    // all the parameters information
    Object[] parameters = iTestResult.getParameters();
    for(Object parameter : parameters) {
        completeFileName.append("_");
        completeFileName.append(parameter);
    }

    // return the complete name and replace : by - (in the case the emulator have port as device name)
    return completeFileName.toString().replace(":", "-");
}

An example of a screenshot file name with this code is: MyTest_myTest_android_emulator-5554_7.0.1.png

A Complete Example

You can see a complete example, with the basic architecture to execute parallel tests with Appium and this trick to create screenshots on test failure.

The complete test listener can be found here.

Any questions? Write a comment!