Selenium and Section 508

Selenium is a popular test tool for web applications. It was developed from 2004 onwards, originally to acceptance-test a time-and-expense-reporting application for ThoughtWorks. Members of ThoughtWorks are still involved in its development, as are others, on the OpenQA web site.

Selenium is written in JavaScript and runs within a browser. There are several versions of the tool: the easiest to deploy and try out is the Selenium IDE. This is available as a Firefox extension here. Once installed, the extension is very easy to use. It allows users to record tests by simply clicking and typing through them and to edit and replay them as needed. You can also build up tests manually, if you wish.

Possibly easiest to use is the editor interface:

[img_assist|nid=6954|title=Selenium IDE|desc=|link=none|align=center|width=400|height=520]

There are three lines that may be filled in for each command in each test, shown in the editor interface as Command, Target, and Value. Not all commands require all three; all commands, of course, require the Command line to be filled in. Each command has its own set of parameters. These are stored as HTML files by the IDE, with a command structure that looks like this:

  <tr>
    <td>Command</td>
    <td>Target</td>
    <td>Value</td>
  </tr>

I actually prefer the HTML form of the tests and often build tests in HTML directly (which can be done in the Source tab of the interface or in your favorite text editor). Your mileage, of course, may vary, but I shall be using the HTML form of the tests in this article.

Selenium logs the state of its testing (JavaScript can't, of course, write to files) in the Log Console. This is just a table of results built up at runtime. Some Selenium tests give less-than-ideal feedback about exactly what it was that went wrong, but even limited error messages can often be useful.

The downside of using the Selenium IDE is that it only runs in Firefox. I rarely find this a limitation, and it is not one for the current article. Most of the tests herein use XPath to identify things that should or should not exist in the web application; if the Firefox tests succeed, the web application is fine, in whatever browser it is actually viewed. Should Selenium solutions be required in other browsers, it is not difficult to use the IDE to convert the tests to other languages, for example Java (where they are converted to JUnit tests); in those languages, typically, other browsers are supported, but the IDE cannot be used.
One extremely useful aspect of Selenium it that it allows you to extend the functions it provides by adding your own. I do this quite frequently; there are a number of user-contributed extensions here. We shall see two examples of this later on.
Selenium has become an absolutely necessary part of my development and testing regime. It makes test-driven development possible in a web environment without a lot of fuss and bother.

Section 508

Section 508 is a part of the Rehabilitation Act of 1973, as amended in 1998 (29 U.S.C. § 794d). People with disabilities often found the web forbidding and inaccessible, since it often didn't even minimally take into account the needs of people whose approach to the web is not visual or mouse based. The idea of Section 508 was to make such information (not only web applications, but that is our focus here) accessible by removing barriers that held users back and to encourage the development of technologies that would make accessibility easier over time. The law applies to Federal agencies, but as a standard is a good thing to keep in mind whenever developing for the web: nothing in Section 508 requires making anything more difficult to use for any user and making information available to more users simply increases our user base.

Purpose of this article

In many cases, the adaptations required to meet Section 508 standards are not onerous. Rather, they are things we tend to forget when we don't ourselves use them routinely.

The purpose of the present article is to show that Selenium testing can be used effectively as an aide-mémoire, ensuring that we don't forget to put in features that will make our web sites more accessible and that we don't put in features that will make them less accessible.

I personally create a test script for my applications that use JSPs that simply navigates to (and therefore compiles) all JSPs automatically, testing for Section 508 compliance along the way. Automated as part of my build process, I find this a useful way to avoid the "hang" time that JSPs can sometimes (and irritatingly) cause, and at the same time to ensure that I haven't missed a chance for Section 508 compliance that costs me little if anything and enables access.

Script tags

While external style sheets (using the link tag) can be overridden by users with specific issues, enabling those users to modify the appearance of an HTML page to satisfy their specific needs, inline CSS styles defined within HTML cannot be so overridden. The test itself to catch cases of this is a simple check of the number of style elements in the HTML page, using XPath (the verify option, in Selenium, does not stop the test case, even if it must report a failure of the test):

  <tr>
    <td>verifyXpathCount</td>
    <td>//style</td>
    <td>0</td>
  </tr>

 

Alt attributes (round 1)

Each content-carrying image should have an alt textual description for non-visual users. Were it this simple, the test would be relatively straightforward:

  <tr>
    <td>verifyXpathCount</td>
    <td>//img[not(normalize-space(@alt))]</td>
    <td>0</td>
  </tr>

As we shall see later, this rule comes with an exception, which may or may not apply to your site.

The same logic applies to various animation types (Java applets, Flash files, video files, audio files, and other plug-ins):

  <tr>
    <td>verifyXpathCount</td>
    <td>//applet[not(normalize-space(@alt))]</td>
    <td>0</td>
  </tr>
  <tr>
    <td>verifyXpathCount</td>
    <td>//object[not(normalize-space())]</td>
    <td>0</td>
  </tr>

Again, your specific cases may differ slightly, but this should point the way towards a solution.

Tables

Table columns should be headed by "th" tags to make the logical organization of data clear. Here the test is trivial:

  <tr>
    <td>verifyXpathCount</td>
    <td>//table[not(descendant::th)]</td>
    <td>0</td>
  </tr>

Of course, this assumes proper HTML. If you are still using tables to do page layout (shame on the browser developers that force you to do this with their buggy CSS implementations!), the situation is more complex. In this case, I would suggest using a class attribute to distinguish non-layout tables (say, "class='layout'"). Then the solution is not too much worse:

  <tr>
    <td>verifyXpathCount</td>
    <td>//table[not(@class='layout') and not(descendant::th)]</td>
    <td>0</td>
  </tr>
  <tr>
    <td>verifyXpathCount</td>
    <td>//table[@class='layout' and (descendant::th or descendant::td/@headers)]</td>
    <td>0</td>
  </tr>

Of course, the second check ensures that there are no "th" elements (or "td" elements using the headers attribute) in purely layout tables, which might well confuse, especially with audio screen readers.
The scope attribute (equal to "col") or the headers attribute should be used for columns:

  <tr>
    <td>verifyXpathCount</td>
    <td>//table[descendant::th[normalize-space(@scope)!='col' and not(normalize-space(@headers))]]</td>
    <td>0</td>
  </tr>

If you use layout tables, column groups, or what the HTML specification charmingly calls "axes in an n-dimensional space" (i.e., the axis attribute), the XPath will have to be adjusted, of course.
The scope attribute (equal to "row") or headers attribute should also be used for the first cell in rows:

  <tr>
    <td>verifyXpathCount</td>
    <td>//table[descendant::td[position()=1  and normalize-space(@scope)!='row' and not(normalize-space(@headers))]]</td>
    <td>0</td>
  </tr>

 Labels

The use of labels for various form data input fields is another important aspect of the logical organization of information. The label for attribute must match up exactly with the id attribute of the field to which it refers:

  <tr>
    <td>verifyXpathCount</td>
    <td>//input[not(//label/@for=./@id) and normalize-space(@type)!='hidden' and normalize-space(@type)!='button' and normalize-space(@type)!='submit' and normalize-space(@type)!='reset' and normalize-space(@type)!='image']</td>
    <td>0</td>
  </tr>
  <tr>
    <td>verifyXpathCount</td>
    <td>//select[not(//label/@for=./@id)]</td>
    <td>0</td>
  </tr>
  <tr>
    <td>verifyXpathCount</td>
    <td>//textarea[not(//label/@for=./@id)]</td>
    <td>0</td>
  </tr>

This can get a bit more complex with tables including checkboxes or radio buttons, which may legitimately not have labels.

Adding a class (possibly "radioTable") to the table and then adding

and not(normalize-space(@type)='radio' and ancestor::table[@class='radioTable'])

to the first of the above XPath expressions (just before the last square bracket) should help. You may, of course, use another predicate to identify the table, such as an id or other attribute.

Frames

In a perfect world, we wouldn't need frames at all, but in this fallen one, they are sometimes useful. They should always have titles describing their contents:

  <tr>
    <td>verifyXpathCount</td>
    <td>//frame[not(normalize-space(@title))]</td>
    <td>0</td>
  </tr>

Image maps

Server-side image maps are rarely used nowadays and for good reason: they make it far more difficult for the user to determine whether or not they are in the right area of the image (the principle of least surprise applies here). They should not be used at all in Section-508-compliant websites unless equivalent text links are provided (which pretty much obviates the image map in the first place).

The test for them, however, is simple enough:

  <tr>
    <td>verifyXpathCount</td>
    <td>//img[ancestor::a and @ismap]</td>
    <td>0</td>
  </tr>
  <tr>
    <td>verifyXpathCount</td>
    <td>//input[normalize-space(@type)='image' and @ismap]</td>
    <td>0</td>
  </tr>

Alt attributes (round 2)

As we said above, the requirement for alt attributes on image tags comes with an exception: where an image alt description would be redundant, use "alt=''". The sort of image envisioned here might, for example, depict a paginating arrow. Even for such images, the alt attribute can be useful unless there is already equivalent text right next to the image. (Of course, if there is such text already, why are you using the image? Use a link instead!)

For the latter case, the solution is necessarily more complex. Selenium cannot be expected to read your mind: you must know what you expect your page to look like. However, it is not difficult to determine whether it does in fact look like what you expect.

We will implement our first Selenium extension to test for this. The Selenium IDE allows you to define a set of extensions in a JavaScript file. By convention, this is called "user-extensions.js", but it can be called anything you like. More than one file can be applied if desired, enabling modularization if it is appropriate for your situation. Apply the files you want using "Options->Selenium Core Extensions" in the "Options" menu in the Selenium IDE.

Our extension, doStoreXPathInfo, is shown below. Selenium will ignore the "do" part of the function name; the command will appear in the Selenium IDE as "storeXPathInfo". It takes two arguments: the XPath expression to be evaluated and the variable within which to store the information retrieved about each node selected by the XPath expression. This variable may be used later in the same Selenium test.

The XPath expression may return elements or attributes, and may return a single node or multiple nodes. The information retrieved depends upon the node: for attributes, the value is retrieved; for elements, the inner HTML is retrieved. If none of these has succeeded, the id attribute is retrieved; failing even this, the name attribute of the node is retrieved. None of these guarantees that information is retrieved; if no information is retrieved about a node, the string "no info" is substituted. Multiple retrieved values are comma-separated.

The function deletes older copies of the variable. For convenient comparison, if no nodes are retrieved, the function stores an empty string as the variable value.

Selenium.prototype.doStoreXPathInfo = function(xpath, variableName)
{
  delete storedVars[variableName];
  var bbot = this.browserbot;
  var result = eval_xpath(xpath, bbot.getDocument(), {
    ignoreAttributesWithoutValue: bbot.ignoreAttributesWithoutValue,
    allowNativeXpath: bbot.allowNativeXpath,
    xpathLibrary: bbot.xpathLibrary,
    namespaceResolver: bbot._namespaceResolver
  });
  if (result.length == 0)
  {
    this.doEcho('No nodes were found corresponding to the XPath expression ' + xpath);
    this.doStore('', variableName);
    return;
  }
  var values = '';
  for (var i = 0; i < result.length; ++i)
  {
    if (values)
    {
      values += ',';
    }
    var value = '';
    var attributesToBeTried = new Array('value', 'innerHTML', 'id', 'name');
    for (var j = 0; j < attributesToBeTried.length && ! value; j++)
    {
      value = result[i].wrappedJSObject[attributesToBeTried[j]];
    }
    values += (value ? value : 'no info');
  }
  if (values)
  {
    this.doStore(values, variableName);
  }
  else
  {
    this.doEcho('No information was retrieved to store to the variable ' + variableName);
  }
}

With this addition, the test is not too complicated. First, there should be no images without alt attributes at all:

  <tr>
    <td>verifyXpathCount</td>
    <td>//img[not(@alt)]</td>
    <td>0</td>
  </tr>

Second, you need to ensure that the correct images have empty alt attributes. You could simply count the number of such images and compare it to what you expect:

  <tr>
    <td>verifyXpathCount</td>
    <td>//img[@alt and not(normalize-space(@alt))]</td>
    <td>yourCountHere</td>
  </tr>

Of course, you would have to replace "yourCountHere" with the number of images you expect to have empty alt attributes.

However, as you are a responsible developer, you will want to check the source URLs to ensure that only the correct images have empty alt attributes. You, therefore, will use the new command instead:

  <tr>
    <td>storeXPathInfo</td>
    <td>//img[@alt and not(normalize-space(@alt))]/@src</td>
    <td>noAltImages</td>
  </tr>
  <tr>
    <td>verifyEval</td>
    <td>'${noAltImages}'=='yourValueHere'</td>
    <td>exact:true</td>
  </tr>

Replace "yourValueHere" with the value you expect the page to return, that is, the source URLs of the images without the alt attribute, in document order and comma-separated (don't add extra spaces!). If you don't expect to have images without the alt attribute, you may, as noted above, use an empty string (e.g., "'${noAltImages}'==''"), but better yet is not to use this test at all; the test under Alt attribute (round 1) is quite sufficient.

Font size

Font sizes and other sizes that may be changed to assist readers on Section-508-compliant sites should be denominated in ems rather than pixels or points. This enables browser controls to be used to resize the actual appearance of fonts.

The term "em" originally referred, in early typography, to the height of the metal base from which the cast letter arose (it takes its name from the fact that in older typefaces the capital M was frequently made exactly this width). The term has been abstracted to mean "the number of points specifying a size for a particular typeface", so for a 9-point typeface, 1 em would equal 9 points. However, the em scales when the font size is changed by the user, unlike pixels or points, which are fixed in size.

While Selenium may not be the ideal testing tool for this (a simple text search can discover whether there are any appearances of "px" or "pt" in your CSS files), it can still be useful as a backstop. The test requires a somewhat more complex Selenium extension than we saw earlier:

Selenium.prototype.getFontsInPixelsCount = function()
{
  var errorList = new Array();
  var index = 0;
  var errors = 0;
  var regex = /[0-9]+[ ]*pt|[0-9]+[ ]*px/i;
  var stylesheets = this.browserbot.getDocument().styleSheets;
  var propertiesToBeTested = new Array('font', 'fontSize', 'letterSpacing', 'lineHeight', 'textIndent', 'wordSpacing');
  for (var i = 0; i < stylesheets.length; i++)
  {
    var stylesheet = stylesheets[i];
    if (! stylesheet.disabled && (! stylesheet.href || stylesheet.href.substring(0, 9) != 'chrome://'))
    {
      var rules = this.getAllStyleRules(stylesheet.cssRules);
      for (var j = 0; j < rules.length; j++)
      {
        if (rules[j].type == CSSRule.STYLE_RULE)
        {
          var style = rules[j].style;
          for (var k = 0; k < propertiesToBeTested.length; k++)
          {
            var text = style[propertiesToBeTested[k]];
            if (text && regex.test(text))
            {
              errorList[index++] = (stylesheet.href ? stylesheet.href : 'inline-style') +
                '@' + rules[j].selectorText;
              errors++;
            }
          }
        }
      }
    }
  }
  if (errors)
  {
    var errorStr = '';
    for (var i = 0; i < errorList.length; i++)
    {
      if (errorStr)
      {
        errorStr += ',';
      }
      errorStr += errorList[i];
    }
    this.doEcho('errors: ' + errorStr);
  }
  return errors;
}

Selenium.prototype.getAllStyleRules = function(rules)
{
  var result = new Array();
  var index = 0;
  for (var i = 0; i < rules.length; i++)
  {
    if (rules[i].type == CSSRule.STYLE_RULE)
    {
      result[index++] = rules[i];
    }
    else if (rules[i].type == CSSRule.IMPORT_RULE)
    {
      var inner = this.getAllStyleRules(rules[i].styleSheet.cssRules);
      result = result.concat(inner);
      index = result.length;
    }
  }
  return result;
}

This extension ignores the default style sheets supplied with Firefox, since these must use point-defined fonts. Again, with this extension in place, the test is quite simple:

  <tr>
    <td>verifyFontsInPixelsCount</td>
    <td>0</td>
    <td></td>
  </tr>

Note that, like our previous extension, Selenium does not directly use the name of the extension function. Because this is a "getXXX" function, Selenium automatically generates the command "verifyFontsInPixelsCount" (also "assertFontsInPixelsCount" and "waitForFontsInPixelsCount"). It also automatically takes care of doing the comparison, marking the command as failed if the count doesn't match for the verify and assert commands (which differ in that the verify will continue after failure, while the assert will stop), or waiting for the configured maximum time period for the waitFor command (not terribly useful in this particular case).

Conclusion

This review doesn't by any means fully exhaust Section 508 requirements. There are many other important features that are needed on accessible websites. However, Selenium can be used to test the features covered here easily and there are few if any downsides to accommodating them. There is also no reason not to test new or existing sites for these features or not to add them where it is possible to do so.

Making all your sites available to as wide an audience as possible is easy with Selenium!