Top 7 Mistakes When Testing JavaFX Applications

JavaFX is a versatile tool for creating rich enterprise-grade GUI applications. Testing these applications is an integral part of the development lifecycle. However, Internet sources are very scarce when it comes to defining best practices and guidelines for testing JavaFX apps. Therefore, developers must rely on commercial offerings for JavaFX testing services or write their test suites following trial-and-error approaches.

This article summarises the seven most common mistakes programmers make when testing JavaFX applications and ways to avoid them.

Scope and Baseline

Two projects were used for demonstrating JavaFX testing capabilities: RaffleFX and SolfeggioFX. The latter uses Spring Boot in addition to JavaFX.

Note that these projects don’t contain JavaFX dependencies because they are developed based on open source Liberica JDK with integrated JavaFX support.

JDK version: 21

TestFX was used as a testing framework. It is actively developed, open-source, and with a wide variety of features. RobotFX, a TestFX class, was used for interacting with the UI.

Other libraries and tools used: JUnit5,  AssertJ, JavaFX Monocle for headless testing in CI.

Mistake 1: Updating UI Off the FX Thread

JavaFX creates an application thread upon application start, and only this thread can render the UI elements. This is one of the most common pitfalls in JavaFX testing because the tests run on the JUnit thread, not on the FX application thread, and it is easy to forget to perform specific actions explicitly on the FX thread, such as writing to or reading from the UI. Take a look at this code snippet:

List<String> names = List.of("Alice", "Mike", "Linda");
TextArea area = fxRobot.lookup("#text")
                .queryAs(TextArea.class);
area.setText(String.join(System.lineSeparator(), names));

Here, we are trying to update the UI off the application thread. As a result, another thread is created and tries to perform actions on UI elements. This results in

• Thrown java.lang.IllegalStateException: Not on FX application thread;
• Random NPEs inside skins,
• Deadlocks,
• States that never update.

What can we do?

Write to the UI to mutate controls or fire handlers on the FX thread. If you use the FxRobot class, you can achieve that by wrapping mutations in robot.interact(() -> { ... }).

List<String> names = List.of("Alice", "Mike", "Linda");
TextArea area = fxRobot.lookup("#text")
                .queryAs(TextArea.class);
fxRobot.interact(() ->
                area.setText(String.join(System.lineSeparator(), names)));

Read from the UI to get text, snapshot pixels, or query layout on the FX thread and return a value:

   private static Color samplePixel(Canvas canvas, Point2D p) throws Exception {

        return WaitForAsyncUtils.asyncFx(() -> {

            WritableImage img = canvas.snapshot(new SnapshotParameters(), null);
            PixelReader pr = img.getPixelReader();
            int x = (int) Math.round(p.getX());
            int y = (int) Math.round(p.getY());
            x = Math.max(0, Math.min(x, (int) canvas.getWidth() - 1));
            y = Math.max(0, Math.min(y, (int) canvas.getHeight() - 1));
            return pr.getColor(x, y);
        }).get();
    }

On the other hand, the input, such as pressing, clicking, or releasing, should happen on the test thread. Do not wrap it in robot.interact():

robot.press(KeyCode.Q);

Mistake 2: Bootstrapping Tests and FXML ClassLoader Incorrectly

When you combine JavaFX/TestFX with a framework such as Spring Boot, it is easy to boot the application the wrong way. The thing is that TestFX owns the Stage, but Spring owns the beans. So, if you boot Spring without giving it the TestFX Stage, the beans will not be able to use it. On the other hand, if you call Application.start(...) directly, you can end up with two contexts.

Another mistake is related to the same situation of using JavaFX with Spring. FXMLLoader uses a different classloader than Spring. Therefore, controllers Spring creates aren’t the same “type” as the ones FXML asks for.

Incorrect bootstrapping results in:

• NoSuchBeanDefinitionException: ...Controller even though it’s a @Component.
• Random NPEs from the custom FxmlLoader because applicationContext is null.
• Stack traces mention exceptions related to ClassLoader or “can’t find bean for controller X”.

What can we do?

Make FXMLoader use the same class loader as Spring in the application code:

public Parent load(String fxmlPath) throws IOException {
        FXMLLoader loader = new FXMLLoader();
        loader.setLocation(getClass().getResource(fxmlPath));
        loader.setClassLoader(getClass().getClassLoader());
        return loader.load();
    }

Use @Start to wire up a real Stage, and Dependency Injection to inject fakes.

Don’t call new FxApplication.start(stage) if this code boots Spring internally.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE)
@ExtendWith(ApplicationExtension.class)
class PianoKeyboardColorTest {

    @Autowired
    private ConfigurableApplicationContext context;

    @Start
    public void start(Stage stage) throws Exception {
        FxmlLoader loader = new FxmlLoader(context);
        Parent rootNode = loader.load("/fxml/in-build-keyboard.fxml");

        stage.setScene(new Scene(rootNode, 800, 600));
        stage.show();
        WaitForAsyncUtils.waitForFxEvents();
    }

Mistake 3: Confusing Handler Wiring with Real User Input

When you try to trigger the UI behavior by calling the controller methods directly, you are testing the code wiring, but not the real event path the user takes, such as focusing, clicking, pressing, etc. As a result, your tests may pass but miss bugs. Alternatively, the test may hang and fail because the input never fired.

Another side of this coin is triggering the UI event that can’t happen, such as going full screen in the headless mode somewhere in CI. In this case, the assertions will time out waiting for the event that will never happen.

For example, we can trigger the button action with robot.clickOn() and button.fire(), but these methods are not equivalent. The robot.clickOn() simulates a real mouse click by moving the mouse, pressing, and releasing. The button.fire() triggers the button’s action programmatically and skips the mouse events entirely.

What can we do?

Don’t mix integration and interaction tests, i.e., avoid calling controller methods directly in the UI tests.

Use robot.clickOn() or similar FxRobot’s methods to test user interaction and UI behaviour: pressed/hover visuals, etc. Note that this method runs on the test thread, so you don’t have to wrap it in interact(): 

Canvas canvas = robot.lookup("#keyboard").queryAs(Canvas.class);
robot.interact(canvas::requestFocus);
robot.press(KeyCode.Q);

Use button.fire() or similar control methods to assert handler effects without relying on real pointer semantics. Note that these methods run on the FX thread, so they must be wrapped in interact():

Button btn = fxRobot.lookup("#startButton").queryButton();
fxRobot.interact(btn::fire);

Assert by changes in the UI, such as the presence of a node in the new scene, label text change, button visibility mode, not by assuming the service call succeeded.

WaitForAsyncUtils.waitFor(3, SECONDS,
    () -> robot.lookup("#startPane").tryQuery().isPresent());

In headless mode, if the platform can’t do something like going full screen, assert a proxy signal (pseudo-classes, button state).

Mistake 4: Racing the FX Event Queue

As JavaFX is a single-thread kit, all UI events happen on the FX Application Thread, and so, events like animations, layout, etc., get queued. If you assert in tests before the queue is drained, you are testing the UI that doesn’t exist yet:

• You fire an action and immediately assert. As a result, your check runs before the handler executes.
• You query the scene right after a scene switch when the new nodes aren’t attached yet.
• You read pixels or control state from the test thread while JavaFX is mid-layout.

Therefore, tests pass or fail unpredictably depending on CPU, CI, and whatnot.

What can we do?

In the case of simple changes, use WaitForAsyncUtils.waitForFxEvents() for the event queue of the JavaFX Application Thread to be completed:

@Start
public void start(Stage stage) throws Exception {
    FxmlLoader loader = new FxmlLoader(context);
    Parent rootNode = loader.load("/fxml/in-build-keyboard.fxml");

    stage.setScene(new Scene(rootNode, 800, 600));
    stage.show();
    WaitForAsyncUtils.waitForFxEvents();
}

In the case you are waiting for observable outcomes, use WaitForAsyncUtils.waitFor() to wait for some conditions to be met:

@Test
void shouldChangeSceneWhenContinueButtonIsClicked(FxRobot fxRobot) throws TimeoutException {
    Parent oldRoot = stage.getScene().getRoot();

    Button btn = fxRobot.lookup("#continueButton").queryButton();
    fxRobot.interact(btn::fire);

    WaitForAsyncUtils.waitFor(3, TimeUnit.SECONDS,
            () -> stage.getScene().getRoot() != oldRoot);

    assertThat(stage.getScene().getRoot()).isNotSameAs(oldRoot);
    assertThat(
            fxRobot.lookup("#startButton")
                    .queryAs(Button.class)).isNotNull();

}

The same approach should be applied when dealing with animations. Wait for the state to change, not the duration the animation is supposed to run:

@Test
void shouldHideAndDisableButtonsWhenRaffling(FxRobot fxRobot) throws TimeoutException {

    Button start = fxRobot.lookup("#startButton").queryButton();
    Button repeat = fxRobot.lookup("#repeatButton").queryButton();

    fxRobot.interact(start::fire);

    WaitForAsyncUtils.waitFor(5, TimeUnit.SECONDS, () ->
            WaitForAsyncUtils.asyncFx(() ->
                    repeat.isVisible() && !repeat.isDisabled()
            ).get()
    );
    assertThat(repeat.isVisible()).isFalse();
    assertThat(repeat.isDisabled()).isTrue();

}

Mistake 5: Assuming Pixel-Perfect Equality Across Platforms

The pixel colors in the JavaFX applications may differ slightly on various platforms due to various reasons: CI uses Monocle, whereas Prism SW and the laptop use a GPU pipeline, or one machine uses LCD subpixel text and another uses grayscale. If the tests assess exact RGB equality on all platforms, the tests may pass locally and fail in CI or on another local machine.

What exactly happens?

• JavaFX apps can run with different DPI scaling on various displays / in various environments: see release notes, bugs, javadoc proving that. On HiDPI and retina displays, JavaFX renders at a scale >1, so logical coordinates don’t map 1:1 to physical pixels. As a result, antialiasing and rounding shift colors slightly, breaking pixel-perfect assertions.
• Headless Monocle uses software Prism, not the desktop GPU, leading to slightly different composites.
• The FontSmoothingType enum in JavaFX specifies the preferred mechanism for smoothing the edges of fonts: sub-pixel LCD or GRAY. Due to this fact, the pixels may vary depending on the mode used by the system. Even if the mode is set in the application, JavaFx may fall back on a different mode if the first one is not supported by the system. See the proof for macOS and Linux as an example.

What can we do?

Don’t assert the exact color. Compare baseline vs changed and allow for some tolerance in color and pixel density.

For example, in SolfeggioFX, to test that the key color on the virtual piano has changed when the corresponding key was pressed, we can calculate pixel indices using Math.round() to tolerate the fractional positions in the case of HiDPI and Math.max()/min() to avoid sampling outside the image in case the Point2D value is near the edge:

private static Color samplePixel(Canvas canvas, Point2D p) throws Exception {

    return WaitForAsyncUtils.asyncFx(() -> {

        WritableImage img = canvas.snapshot(new SnapshotParameters(), null);
        PixelReader pr = img.getPixelReader();
        int x = (int) Math.round(p.getX());
        int y = (int) Math.round(p.getY());
        x = Math.max(0, Math.min(x, (int) canvas.getWidth() - 1));
        y = Math.max(0, Math.min(y, (int) canvas.getHeight() - 1));
        return pr.getColor(x, y);
    }).get();
}

In addition, we can allow for a small absolute difference when comparing colors:

private static boolean colorsClose(Color a, Color b) {
    double eps = 0.02; // tolerate small AA differences (~2%)
    return Math.abs(a.getRed() - b.getRed())   < eps &&
            Math.abs(a.getGreen() - b.getGreen()) < eps &&
            Math.abs(a.getBlue() - b.getBlue())  < eps;
}

@Test
void shouldHighLightPressedKey(FxRobot robot) throws Exception {
    Point2D point = Objects.requireNonNull(centers.get('Q'));
    Color before = samplePixel(canvas, point);

    robot.press(KeyCode.Q);

    WaitForAsyncUtils.waitFor(1500, TimeUnit.MILLISECONDS,
            () -> !colorsClose(WaitForAsyncUtils.asyncFx(() -> samplePixel(canvas, point)).get(), before));

    Color duringPress = samplePixel(canvas, point);
    assertThat(before.equals(duringPress)).isFalse();

}

Sample pixels inside the shape, not near borders, to avoid having a different color if borders blend with the background. 

In SolfeggioFX, we stored per-key centers in the Canvas properties when drawing the virtual piano, and used this data in the tests to sample pixels near the key center:

// production code

canvas.getProperties().put("keyCenters", Map<Character, Point2D> centers);

// tests

Point2D point = Objects.requireNonNull(centers.get('Q'));

Mistake 6: Misconfiguring Headless CI

Running JavaFX tests in CI differs from the standard testing process. The tests must run in headless mode and be backed by Monocle, an implementation of the Glass windowing component of JavaFX for embedded systems. But simply adding the dependency on Monocle won’t help much, and tests that pass locally may fail in CI due to multiple factors:

• UI tests run in parallel.
• Required modules are locked down, but Monocle uses com.sun.glass.ui reflectively. As a result, you get exceptions like IllegalAccessError: module javafx.graphics does not export com.sun.glass.ui or InaccessibleObjectException: … does not "opens com.sun.glass.ui"
• Tests assert platform features that don’t exist in headless, for instance, Stage.setFullScreen(true). So, the tests hang and finally fail with the TimeoutException

What can we do?

Add the Monocle dependency and set all necessary flags to run the tests in headless mode. In addition, open the required modules with --add-opens.

Add the Monocle dependency first:

<dependency>
    <groupId>org.pdfsam</groupId>
    <artifactId>javafx-monocle</artifactId>
    <version>21</version>
    <scope>test</scope>
</dependency>

Then, specify all required flags in a separate plugin: set the headless mode, disable parallelism, etc. Note that the --add-opens are specific to the RaffleFX application used for demonstration, in your case, the modules may be different.

This application is developed and compiled in CI using a Java runtime with bundled JavaFX modules, but if you add the dependencies on JavaFX modules manually, you may have to use the additional --add-exports flag that allows compile-time access to Glass internals:

<profile>
    <id>headless-ci</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.5.3</version>
                <configuration>
                    <forkCount>1</forkCount>
                    <reuseForks>true</reuseForks>
                    <argLine>
                        --add-opens=javafx.graphics/com.sun.javafx.application=ALL-UNNAMED
                        --add-opens=javafx.graphics/com.sun.glass.ui=ALL-UNNAMED
                        --add-opens=javafx.graphics/com.sun.javafx.util=ALL-UNNAMED
                        --add-opens=javafx.base/com.sun.javafx.logging=ALL-UNNAMED
                        --add-opens=javafx.graphics/com.sun.glass.ui.monocle=ALL-UNNAMED
                        -Dtestfx.robot=glass
                        -Dtestfx.headless=true
                        -Dglass.platform=Monocle
                        -Dmonocle.platform=Headless
                        -Dprism.order=sw
                        -Dprism.text=t2k
                        -Djava.awt.headless=true
                    </argLine>
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

Adjust the tests that wait for stage.isFullScreen() and assert a proxy signal or skip these tests in CI. 

In the workflow file, make sure to install all necessary native libraries for JavaFX and run the tests with the correct profile. The file below uses Liberica JDK 21 with JavaFX in the setup-java action, so no additional dependencies on FX are required:

name: Tests
on:
  push:
    paths-ignore:
      - 'docs/**'
      - '**/*.md'
    branches: [ main ]

jobs:
  test_linux_headless:
    name: UI tests (Ubuntu + Monocle)
    runs-on: ubuntu-latest
    steps:
      - name: Install Linux packages for JavaFX
        run: |
          sudo apt-get update
          sudo apt-get install -y \
            libasound2-dev libavcodec-dev libavformat-dev libavutil-dev \
            libgl-dev libgtk-3-dev libpango1.0-dev libxtst-dev
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v5
        with:
          distribution: 'liberica'
          java-version: '21'
          java-package: 'jdk+fx'
          cache: maven
      - name: Run tests (headless with Monocle)
        run: ./mvnw -B -Pheadless-ci test
      - name: Upload surefire reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: surefire-reports
          path: |
            **/target/surefire-reports/*
            **/target/failsafe-reports/*

Mistake 7: Entangling Business Logic with UI (Non-Determinism)

Last but not least, testing business logic with UI is not the best practice. Just as you separate controllers and service tests for web apps, the domain logic tests should not coexist with UI tests in one class.

In the worst-case scenario, the tests become slow and yield inconsistent results.

What can we do?

The best solution would be to move business logic to ViewModels and test it with plain JUnit. This way, you don’t depend on animations and other UI events, and make sure that your tests are always deterministic.

Conclusion

JavaFX applications need testing just like any other program. On the one hand, you verify that the application functions exactly as expected. On the other hand, you make it more maintainable in the long term.

Nevertheless, the unfamiliar process of JavaFX testing may result in numerous exceptions during test runs or ‘mysterious’ test failures. Luckily, developers can navigate these unknown waters safely, keeping an eye on the following waymarks:

• FX thread vs test thread: Mutate UI and read from UI on the FX Application thread, send input from the test thread.
• Correct bootstrap: If you use frameworks such as Spring, make sure to start Spring/TestFX in the right order and make FXMLLoader use Spring’s class loader.
• FX event queue: Wait until the FX queue is drained before making assertions and assert by state, not duration.
• No pixel-perfect assertions: Keep in mind that the environment and platform may affect the visuals slightly, so allow for tolerance when testing colors and take samples closer to the element center. 
• CI headless configuration: Configure the headless testing with Monocle, open required Glass internals, and avoid asserting platform features Monocle can’t emulate.

Testing JavaFX may seem complicated, and this article covers the most common pitfalls. But following these pieces of advice, you will be able to build a reliable testing foundation for your JavaFX program.