Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

How to Make Simple End-To-End Tests With JavaTea

DZone 's Guide to

How to Make Simple End-To-End Tests With JavaTea

Try combining the power of Java and the simple syntax of Tea script to make pairwise and parallel testing easier.

· Performance Zone ·
Free Resource

Image title


Although automated testing is an effective way to validate web applications, creating automated tests is time-consuming and maintaining the tests is painful. End-to-end testing is especially complex and taking time to execute.

To make it easier, this article proposes to use JavaTea, which is an end-to-end functional automation framework. It is built on Selenium WebDriver but it allows you to describe tests with less code compared to Selenium. The following three topics are focused in this article:

  • Simple syntax for ease of code
  • Pairwise technique to reduce the number of test cases
  • Parallel execution to save the execution time

Simple Syntax

JavaTea enables users to describe scripting statements in Java code, called Tea script. When writing tests with Selenium, elements may be found by using locators such as XPath, ID and name. Tea script supports the same locators but also an additional locator called "text locator" which identifies an element using a text string displayed on a page. Let’s see how the text locator works using a sample web site shown in Figure 1. There are two pages, Page 1 and 2.


Image title

Figure 1. Two-page web site to be tested

Test scenario to be tested here is this:

  1. Open Page 1.
  2. Enter a name "John."
  3. Enter his email "john@email."
  4. Select "Tokyo" as his place.
  5. Click on Next. (go to next page)
  6. Validate values displayed on Page 2.

The user operations on Page 1 (step 2 to 5) can be coded, as shown in Listing 1, by using Tea script:

Listing 1. Tea script code snippet to set values and click Next on Page 1

'Name'> = 'John'
          'john@email'
          'Tokyo'
          true


Listing 2 is a code snippet of Selenium which executes the same operations.


Listing 2. Selenium code snippet

WebElement name = driver.findElement(By.id("name"));
name.sendKeys("John");

WebElement email = driver.findElement(By.id("email"));
email.sendKeys("john@email");

WebElement place = driver.findElement(By.id("place"));
Select placeSel = new Select(place);
placeSel.selectByVisibleText("Tokyo");

WebElement next = driver.findElement(By.id("nextBtn"));
next.click();


How can Tea script be so simple? Let's take a look at the code one by one. The first thing to do is to find a text element where it's displayed close to the target input element. In this sample, the Name input field is the target element, hence the "Name" label displayed left next to it is a good candidate to start with. To find the text element, a text locator can be used. As shown in Listing 3, it can be simply described as a test string enclosed in quotations.


Listing 3. A text locator to find a text element of "Name"

'Name'


The "Name" returns a text element object but our target is the input element displayed right next to it. To get the input element, a ">" operator can be used as shown in Listing 4.


Listing 4. A shift ">" operator finds the element displayed next to the text

'Name'>


The ">" operator shifts the element position pointing to the right. Likewise, elements on Page 1 can be reached by using text locators as shown in Figure 2.


Image title

Figure 2. Text locators on Page 1


The next thing we need to do is to send a test value "John" into the input element. In Tea script, sending a value can be done by using an equal assignment as shown in Listing 5:


Listing 5. Send a value to a web element

'Name'> = 'John'


The value on the right-hand side is set to the element on the left-hand side. Likewise, the equal assignment can be applied on other input and button elements as shown in Listing 6:


Listing 6. Tea script code snippet to set values and click Next on Page 1

'Name'> = 'John'
'Email'> = 'john@email'
'Place'> = 'Tokyo'
'Next' = true


The last statement,  "Next" = true , represents a mouse click on the Next button. Thus, it navigates to the next page when executed.

As you saw, Tea script allows access to other elements based on the element found. That is because JavaTea internally has an element index and holds elements around the Name text element. Figure 3 shows the index numbers starting with Name label element.


Image title

Figure 3. An element index started from Name text element


The equal assignment does one more job. It increments the element index and makes current element point to the next element object. For example, when you get the Name input element by using "Name">, the element index is set to 1 as you can see in Figure 3. After an assignment statement shown in Listing 5 is performed, the element index becomes 2 and it points to Email input element. Because of this you do not have to describe the rest of text label. You can simply describe a list of values that you want to set. In this sample, "Email"> =, "Place"> = and "Next" = can be eliminated from the script.


Listing 7. The second and after the second left hand sides are optional

'Name'> = 'John'
/* 'Email'> = */ 'john@email'
/* 'Place'> = */ 'Tokyo'
/* 'Next' =   */ true


Go back to the Listing 1 and compare with the Listing 7. It is the code after eliminated the optional left-hand sides.

Listing 8 shows a complete Java code containing the Tea script and assertion methods to validate the values entered.


Listing 8. WizardTest.javat

import static tea.TeaAssert.assertEquals;
import org.junit.jupiter.api.Test;

public class WizardTest extends tea.TeaBase {
  @Test
  public void test() {
    createDriver('chrome');
    driver.get(new java.io.File('Page1.html').toURI().toString());

    #
      'Name'> = 'John'
                'john@email'
                'Tokyo'
                true
    #

    assertEquals(#'NAME'@>, 'John', 'Name entered');
    assertEquals(#'EMAIL'@>, 'john@email', 'Email entered');
    assertEquals(#'PLACE'@>, 'Tokyo', 'Place selected');

    driver.quit();
  }
}


A pom.xml is available to download from JavaTea web site. With the XML, you can compile and execute the javat by running the commands below: 

C:> mvn exec:java

C:> mvn test



Pairwise Testing

JavaTea has a feature to apply a pairwise (a.k.a. all-pairs) testing technique. It saves a lot of time when testing various combinations of input values. JavaTea allows you to specify multiple input values, and generates test cases designed to execute all possible combinations of each pair of the input values. The advantage of using pairwise technique is that it dramatically reduces the number of test cases so that we can execute the tests in real time. I will not explain the details but focus on how it works in JavaTea.

In Figure 1, there are three input parameters labeled Name, Email and Place on the Page 1. I picked two input values as representative values for each as shown in Table 1:


Table 1. Possible input values

Parameter

Value 1

Value 2

Name

John

<EMPTY>

Email

john@email

<EMPTY>

Place

New York

Tokyo


In order to cover all possible combinations of the input values, eight cases need to be tested (2 x 2 x 2 = 8 combinations). By applying the pairwise algorithm, we can reduce the number of test cases into four. Table 2 shows the four test cases generated by JavaTea:


Table 2. Test cases created by the pairwise algorithm

Test Case #

Name

Email

Place

1

John

john@email

New York

2

<EMPTY>

<EMPTY>

New York

3

<EMPTY>

john@email

Tokyo

4

John

<EMPTY>

Tokyo


Even though there are only four test cases, they cover all combinations of each pair of parameters. In this example, we could eliminate four cases (from 8 to 4 test cases). You may think it isn’t a big difference, but in the real world we could have more input parameters and pages which produce more possible combinations of input values. Suppose you have five pages and each page has three input parameters. To cover all possible combinations, you need to run 32,768 scenarios which are impossible to execute within a limited time. The number of combinations is easily increased by adding elements and pages. This is known as a combinatorial explosion (Figure 4).

However, if we focus on a pair of input parameters, the number of possible combinations can be reduced to nine. This is where pairwise testing is extremely useful. Note that the number of combinations generated by pairwise depends on its algorithm.


Image title

Figure 4. Combinatorial explosion


To apply pairwise technique, Tea script allows describing multiple input values between {* and *}.


Listing 9. Multiple input values

'Name'> = {* 'John', '' *}
          {* 'john@email', '' *}
          {* 'New York', 'Tokyo' *}
          true


The above script represents possible values listed in Table 1. The complete code is shown in Listing 10.


Listing 10. PairwiseTest.javat

import static tea.TeaAssert.assertEquals;
import org.junit.jupiter.api.Test;

public class PairwiseTest extends tea.TeaBase {
  @Test
  public void test() {
    createDriver('chrome');
    driver.get(new java.io.File('Page1.html').toURI().toString());

    #
      'Name'> = {* 'John', '' *}
                {* 'john@email', '' *}
                {* 'New York', 'Tokyo' *}
                true
    #

    assertEquals(#'NAME'@>, toActualValue({***0***}), 'Name entered');
    assertEquals(#'EMAIL'@>, toActualValue({***1***}), 'Email entered');
    assertEquals(#'PLACE'@>, {***2***}, 'Place selected');

    driver.quit();
  }

  private String toActualValue(String value) {
    return value.length() > 0 ? value : "No Value";
  }
}


"{***0***}" is a placeholder to be replaced with the first input value chosen by JavaTea at the compile time. In this example, it will be replaced with "John" or "" (an empty value). You can compile the javat with the same command as shown below:


 C:> mvn exec:java

 


t-wise degree of thoroughness: 2

Possible combinations: 8

Test cases created: 4

1: [0, 0, 0]

2: [1, 1, 0]

3: [1, 0, 1]

4: [0, 1, 1]


This time, JavaTea detects the multiple input values and generates multiple Java files based on a pairwise algorithm. Thus, the command shows pairwise related information where the total number of possible combinations is 8 and the number of test cases actually created is 4. After the execution, you will see four Java files created, which implemented the test cases defined in Table 2.


Image title

Figure 5.  Java files generated from javat


Parallel Execution

Although pairwise reduces the number of test cases, it still generates multiple tests and takes time to execute. Thus, you may want to run the tests in parallel. This can be done by using existing Java Testing libraries such as JUnit or TestNG. For example, JUnit allows you to change execution mode by annotating @Execution on your test class as shown in Listing 11.


Listing 11. Parallel execution with JUnit

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.parallel.Execution;
import org.junit.jupiter.api.parallel.ExecutionMode;

@Execution(ExecutionMode.CONCURRENT)
public class ParallelExecutionTest extends tea.TeaBase {


When you run the tests, this time four browsers are opened and a test is executed in each browser as shown in Figure 6.


Image title

Figure 6.  Execute four tests in parallel


On the console (Figure 7), you will see each test took about 6 seconds but the total time is less than 10 seconds. It shows that the tests were executed at the almost same time.


Image title

Figure 7.  Console logs of parallel execution


If you want to run with using TestNG, you can simply change to use org.testng.annotations.Test annotation instead of JUnit one.


Listing 12. Parallel execution with TestNG

import org.testng.annotations.Test;

public class ParallelExecutionTest extends tea.TeaBase {
  @Test
  public void test() {


JavaTea has an option –X to generate a sample testng.xml file (Listing 13) for parallel execution:


Listing 13. testng.xml generated by –X option

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestSuite" parallel="classes" thread-count="4" preserve-order="true">
        <parameter name="browser" value="chrome"/>
        <parameter name="verbose" value="false"/>
                <test name="ParallelExecutionTest">
                <classes>
                        <class name="ParallelExecutionTest1"/>
                        <class name="ParallelExecutionTest2"/>
                        <class name="ParallelExecutionTest3"/>
                        <class name="ParallelExecutionTest4"/>
                </classes>
        </test>
</suite>


Once you run the XML, TestNG generates a test results report as shown in Figure 8. Although it says the total running time is 25 seconds, it actually took less than 10 seconds to execute the four tests.


Image title

Figure 8.  TestNG Test results report


Conclusions

This article introduced JavaTea end-to-end testing framework. It offers simple syntax for ease of code and pairwise technique to reduce the number of test cases. Also, the generated tests can be executed in parallel with using JUnit and TestNG. These techniques allow us to test various combinations of input values in a real time and save a lot of time in execution.

This article mentioned only the basic features of JavaTea. To learn more, please visit JavaTea web site and play around with the samples.


Resources



Topics:
test automation framework ,selenium automation testing ,java ,parallel execution ,junit ,testng ,pairwise error ,performance

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}