Testing JavaFX UIs: Part 1 of ?

This is the first part of a series of blog entries where I plan to document our progress testing JavaFX UIs with FEST.

Although I had fun, my first attempt failed miserably. The main reason is that my assumptions were wrong. I thought that, since JavaFX uses Swing in its "desktop profile," testing a JavaFX UI that uses Swing would be similar to testing a regular Swing UI created in Java. This is partially right. In my previous try, I could successfully find a JButton using FEST-Swing. The problem was simulating a user clicking the found JButton. It seemed that FEST's Robot could not find the location of the component on the screen. That was pretty weird, since click simulation works perfectly for Swing-based apps. I couldn't understand why FEST could not find the coordinates of a #@$! JButton that in front of me!

Before I continue, I'd like to clarify that:

• I'm just trying to figure out how to test JavaFX UIs
• I only have access to the source code distributed with JavaFX
• I'm not reverse-engineering/decompiling any of the JavaFX classes

I used the excellent tool Swing Explorer to visually inspect the JavaFX UI I wanted to test. Here is a screenshot of the JavaFX calculator example loaded in Swing Explorer:

Swing Explorer saved me a lot of time and effort! This wonderful tool can show you where Swing components are actually displayed in a JFrame (Swing Explorer can do a lot more than that, but this is the functionality I used.) I tried to find if the JButton that I wanted to click with FEST was being displayed in the JavaFX UI. For my surprise, the JButton (and actually any Swing component) is hidden! It seems that JavaFX paints a "fake" button instead. I don't know how exactly this works since Sun does not distribute the source code of the javafx-swing jar file.

Since the JButton is hidden, there should be a way JavaFX knows about the location where to paint it on the screen. In order to simulate a user clicking this JButton using FEST, I had to figure out how to obtain such location.

Figuring out some of the internals of JavaFX was a lot easier than what I expected (actually, I think I was lucky.) I started by writing a test. I first obtained a reference to the com.sun.javafx.scene.JSGPanelImpl, which hosts all the Swing components in the UI. Then, I found the method getScene, which returns a com.sun.scenario.scenegraph.SGNode. After that, it was a matter of looping through the children of its parent (and the children of the children, and so forth.) At last I found something useful: com.sun.scenario.scenegraph.fx.FXNode, a subclass of SGNode, provides the method getLeaf that returns another SGNode. This "leaf" node can be an instance of com.sun.scenario.scenegraph.SGComponent. And, finally, a SGComponent has a reference to the Swing component I've been looking for, through the method getComponent.

The following is the (ugly and hacky) code to find a node that has the button I'm looking for:

private static FXNode nodeWithButton(String text, SGParent root) 
{
  for (SGNode child : root.getChildren()) 
  {
     if (child instanceof FXNode) {
      FXNode fxNode = (FXNode) child;
      SGNode leaf = fxNode.getLeaf();
      if (leaf instanceof SGComponent) {
        SGComponent componentNode = (SGComponent) leaf;
        Component component = componentNode.getComponent();
        if (component instanceof JButton) {
          JButton button = (JButton) component;
          if (text.equals(button.getText())) return fxNode;
        }
      }
    }
    if (child instanceof SGParent) {
      SGParent newRoot = (SGParent) child;
      FXNode fxNode = nodeWithButton(text, newRoot);
      if (fxNode != null) return fxNode;
    }
  }
  return null;
}

At this point I found the nodes that has Swing components as leaf nodes, but I still haven't found the location on the screen of the JButton. I took a closer look at the properties of FXNode and found the method getBoundsInScene, which has all the information I need!

Finally I found a way to obtain the location on the screen and click the JavaFX node containing the Swing component I was looking for:

private void click(FXNode node) 
{  
  moveMouseTo(node);
  realRobot.mousePress(BUTTON1_MASK);
  realRobot.mouseRelease(BUTTON1_MASK);
  robot.waitForIdle();
}

private void moveMouseTo(FXNode fxNode) 
{  
  Rectangle2D boundsInScene = fxNode.getBoundsInScene();
  int centerX = (int)boundsInScene.getCenterX();
  int centerY = (int)boundsInScene.getCenterY();
  Point p = fxNode.getPanel().getLocationOnScreen();
  p.translate(centerX, centerY);
  realRobot.mouseMove(p.x, p.y);
}

The whole test can be found here.

Unfortunately, I confirmed that is not possible to test JavaFX UIs with FEST-Swing as it is today. In order to do so, we need to provide JavaFX-specific support, which IMHO will require the following work:

1. Create a new component hierarchy based on JavaFX nodes, not on Swing components
2. Create a new Robot that knows how to simulate user input on JavaFX nodes (we will still use the AWT Robot under the hood)
3. Create new fixtures for JavaFX nodes

There are a couple of things that makes me uncomfortable:

• We depend on classes that belong to com.sun.* packages, not javafx.* ones. This means that Sun can change implementation details at any time and our tool will stop working.
• It is not clear (at least to me) the license for some jars provided with JavaFX. The worst-case scenario is that we might have to publish our code as GPL, instead of Apache 2.

Feedback is always appreciated

From http://www.jroller.com/alexRuiz