As developers, we all know the value of testable code. The ability to click a button and have a reasonable level of confidence that the changes you just committed didn’t break anything is a huge productivity booster.
Wouldn’t it be nice to have that same level of confidence in your documentation?
It sounds like a nice idea, but how do you create testable documentation? How can written words possibly be tested in an automated way?
To answer these questions, we first need to take a step back and understand what kind of content goes into technical documentation. DITA provides a handy reference for breaking down technical documentation into several standard kinds of information. There are 5 types, but I want to focus on 3: concepts, references, and tasks.
A concept is a general, high-level explanation of what something is. A concept would answer questions like “what is a database” or “what is a web browser.” You can, and indeed usually should answer these questions without going into a lot of details about specific products or implementations. Concepts are the plain English explanation of what something is.
Concepts don’t need a lot of ongoing testing. A thing is what it is, and unless your terminology is constantly being muddled up with each new release of your software, concepts don’t change much at all. Well written concepts can express a high-level idea without getting bogged down in the kind of implementation details that are subject to change, so once you have accurately and succinctly described what something is, that description shouldn’t need to change.
Reference information is things like tables and lists of facts. They might be things like the list of valid inputs for a field, command line options, or API endpoints.
You very much want to be generating reference information automatically. It is nearly impossible to accurately maintain this sort of information manually, and most of the time the information is defined by the system you are documenting. Once this information is being generated automatically, there is little need to test the results because the information is known to be generated from an authoritative source.
Tasks are instructions that end users follow to achieve some goal. They are usually a list of steps that need to be taken and represent the work that your customers actually do with your product.
Tasks will change frequently, and cannot be generated automatically. If you could automatically generate the list of steps that the end user needed to complete a task, then the end user would not need to perform the list of steps!
Identifying the Documentation Burden
Of these three types of information, tasks are the ones that you will take up the majority of your time to create and maintain.
How many times have you referred to some technical documentation only to find that the menu item or button it refers to has moved since the documentation was written? Keeping tasks up to date with new releases of software is a constant battle and one that a lot of vendors are losing.
The reason for this is actually quite simple: the testing of documentation is typically not automated. Those list of steps and accompanying screenshots have to be manually followed and verified with each software release to ensure that they are still valid. And in all my years working as a technical writer and developer, I have never seen this validation process done consistently. I am, however, frequently frustrated by documentation that has drifted from the software it was written for.
Writing Testable Tasks
Writing testable documentation need not be that difficult. The Gherkin Given/And/When/Then syntax used in software tests has the advantage of exposing automated tests in a readable fashion.
Iridium has taken advantage of the Gherkin syntax through the Cucumber library and combined it with Selenium WebDriver to provide the ability to write readable and verifiable end-to-end tests.
Given that end-to-end tests will often replicate the tasks that end users would perform with a web application, why not use the tests as documentation? With version 1.0.34 of Iridium, you can now do exactly that.
Version 1.0.34 adds two important features to Iridium that make this possible.
The first feature is to enable mouse movement between the fields that Selenium is interacting with.
Normally when using the Selenium API to interact with a web page, elements like links, buttons, and textboxes are clickable or populated with no visual indication that they are about to be used. If you watch a web browser being automated by Selenium, it can be hard to follow exactly what is going on.
To allow end-to-end tests to also serve as a visual guide, Iridium introduces the experimental moveCursorToElement system property. When set to true, Iridium will move the mouse cursor over the element Selenium is about to interact with, which gives the illusion that Selenium is using the mouse the way a human operator would.
The second feature provides information windows that pop up over the web browser.
If you were demonstrating a web application to another person, often you would provide additional information about the steps you were taking as you interacted with the page. Iridium allows you to embed these hints into the test script itself with steps like:
When I display the help message "The Students screen allows student information to be managed."
An Example of Testable Tasks
All of this comes together to allow you to write a test script like this:
Feature: Editing a student surname Scenario: Create aliases Given I set the alias mappings | First Student Edit | /html/body/div/table/tbody/tr/td/a | | Last Name | LastName | Scenario: Open the app Given I open the page "https://contoso.azurewebsites.net" Then I wait "30" seconds for the page to contain the text "Contoso University" Scenario: Open the Students screen When I display the help message "The Students screen allows student information to be managed." And I click the "Students" link Then I wait "30" seconds for the page to contain the text "Find by name" Scenario: Edit a student When I display the help message "The Edit link allows student details to be edited" And I click the "First Student Edit" link And I clear the "Last Name" text box And I populate the "Last Name" text box with "Smith" And I click the "Save" button Then I display the help message "The student's name has been updated in the database."
This is a very simple example of an Iridium test script that opens up a web application used to manage student enrollment in a university. The test demonstrates how to edit a student’s name. Even without knowing how the web application that is being tested operates, the steps in this test are easy enough to understand because they are written in plain English.
You can run this test for yourself via this webstart launcher. Just make sure you have Java 8 and Chrome installed on your PC.
In addition, this is a test script that can be automatically run and verified. Indeed verifying the functionality of a web application via scripts like these is the primary function of Iridium. If any of the elements in the web page have been removed or renamed, the test will fail and you know that this task needs to be updated.
Right now the mouse movement only works when a browser is in full-screen mode. This is because the
org.openqa.selenium.interactions.internal.Coordinates.onScreen() method has not been implemented in Selenium, and so the only way to map the position of elements in a web page to their positions on the screen is to remove all the browser’s toolbars and borders.
There are also issues with screen scaling, so it is best to run the tests on a regular (i.e. not HiDPI) screen.
By writing tasks as Iridium tests, you can easily create testable documentation that can also be used to generate screencasts. And because these tests can be automatically verified, much of the maintenance burden is removed.
So not only will you have a decent selection of end-to-end tests, you will also have a library of high-quality documentation.
If you are interested in writing end-to-end tests like the one shown above, check out the Iridium documentation page, and take a look at the courses available on Udemy. Iridium itself is a free and open source project on GitHub.