The JavaFX Scenegraph Dilemma
The Performance Zone is presented by AppDynamics. Scalability and better performance are constant concerns for the developer and operations manager. Try AppDynamics' fully-featured performance tool for Java, .NET, PHP, & Node.js.
"A dilemma is a problem offering at least two solutions or possibilities, of which none are practically acceptable; one in this position has been traditionally described as "being on the horns of a dilemma", neither horn being comfortable; or "being between a rock and a hard place", since both objects or metaphorical choices being rough." - Wikipedia
As an application designer developing a complex graphic solution (equivalent to an electronic circuit schematic diagram with hundred of symbols), I looked forward to using JavaFX, provided that it actually had the right stuff to meet the needs of my application. Presently, after having learnt much about JavaFX and Java 2D, hitting roadblocks and making seemingly endless design compromises, I remain stilted with the limitations inherent in JavaFX for realizing my graphic requirements.
I’ve chosen to write this post to bring this problem to light and to create the opportunity to discuss possible solutions in an open forum.
Java’s Graphic Evolution
Java 1.2 introduced the Java 2D API. This immediate mode API allows code to render graphics immediately via graphics context method calls. With a bit of work, some truly excellent graphic compositions can be created, as demonstrated in Vincent Hardy’s excellent Graphic Layer Framework book.
Competing against immediate mode is the concept of a scenegraph and its retained mode approach to rendering graphics via nodes, which are built up into a data structure (model) called a scene, and rendered via Java 2D. Instead of rendering graphics immediately, rendering takes place via a scenegraph manager in response to various scenegraph changes.
One of Java’s earliest (if not the earliest) scenegraphs was Piccolo, developed at the University of Maryland. While developing F3 (JavaFX’s predecessor), Chris Oliver used Piccolo as the original scenegraph API. For unknown (possibly political or legal) reasons, Sun replaced Piccolo with the somewhat inferior (less optimized) Scenario API.
MVC Coupling Scenarios
Most application user interfaces fit into the basic model, view and controller pattern (MVC), and it is desirable to see and build a user interface in such a consistent, logical and loosely coupled way. Where this simple pattern loses its power is when the view becomes too tightly coupled with the model. This is the danger with a scenegraph; it is a model that is a view that also has controllers. A scenegraph packages the MVC pattern in a specific way, which is suitable for some things, but not for others. In other words, it assumes too many roles, making it unsuitable for more elaborate rendering tasks in the long run.
The primary purpose of a scenegraph in a given complex rendering is to be a graphical representation of the data objects in a model. The first thing to notice when using a scenegraph is that it requires you to create nodes, which extend data objects that your model may already have. For example, the electronic circuit aspects mentioned above may already be expressed and structured as objects, in their own efficient model. If, instead, the model is expressed with nodes in a scenegraph, it can quickly consume a lot of memory and can bog down performance (if there are a lot of them). For instance, an electronic circuit schematic representation with its many symbols requires very many nodes. It gets even worse when you may want to have alternative graphic representations of the same model. Really, what a scenegraph wants you to do is use its nodes as your model, but this does not always work, except for relevantly straightforward things.
As I was considering the disadvantages of coupling the MVC too tightly, it gradually dawned on me that I actually have my own ‘graph’ of objects in the model suited to my needs, which I had already created, and that the redundant, memory hungry and poor performance results of using a scenegraph are not worth it. So here I am now, in the heart of a dilemma, where a scenegraph seems like an excellent solution, but really is not because I already have an optimal model, and its many data objects and their variables would consume way to many nodes to render using a scenegraph.
Okay, this sucks—where do I go next for my rendering needs? Should I go back to using only Java 2D? Is the JavaFX scenegraph really that inappropriate for my application’s needs? These are some of the questions that tear me apart, giving me a beautiful nightmare somewhere in limbo between JavaFX and Java 2D, trying to keep the joy in having a great project that has no easy graphic solution. It’s the kind of situation where I really want and would expect JavaFX to be perfectly suited for my needs, but this is just not the case.
After living the nightmare long enough, it has become increasingly clear to me that it’s best to assume my own responsibilities for most of what the JavaFX scenegraph would do for me. This includes data structures, picking/hitting, and layout/collision detection duties, as well as all rendering using only Java 2D. On the bright side, my application will be fast, not suffer from memory bloat, and have no quasi-redundant tiers. Manipulating and maintaining my model will also be streamlined, and graphic representation has become just a chore to be delegated to some off to the side process. Furthermore, it’s easier to have multiple graphic representations of my singular model. Hey, maybe the fog of JavaFX is lifted off my eyes and I’m really onto something better now!
JavaFX Performance Issues
Besides my application having its own design needs, which is making JavaFX not as useful to work with as I had hoped, there has also been a growing consensus in the JavaFX community that JavaFX’s performance is not all it could be. Here are some of the things that have come up:
- Markus Kohler's Memory Analytics
- Jim Connor's Node Count
- The JavaFX Journey's Flattened Node
- JavaJeff's Canvas Node
- JavaJeff's Painter's Canvas
- Chris Oliver's Instancing (Lens) System
- Osvaldo Doederlein's Benchmark Test
- Charles Ditzel's What's Wrong With JavaFX?
The blogs above propose different solutions: creating ImageViews instead of using many nodes, and a Lens solution for rubberstamping nodes in a highly efficient way, which may never be implemented in JavaFX (why was this not in from the start?). Besides these items, some blogs also point out benchmark tests that have been conducted, and which are usually concerned with scenegraph speed over scenegraph design.
Hybrid Design Options
For my own application, I’m beginning to see how JavaFX and Java 2D can each work to make it happen, but this is only due to solutions like the Painter’s Canvas. For the model, I use special arrays (JavaFX sequences will not suffice) that contain POJO’s as data members. My view is composed in layers, which are essentially JavaJeff’s Painter Canvas nodes that render the model directly to an offscreen image (this is done with Java 2D directly, not with JavaFX in a tiled fashion), which is bound to an ImageView node in the scenegraph.
My own hit system is used to know where on the ImageView node I am editing, with my controllers using temporary inplace JavaFX nodes while editing. Editing changes are wired to my model, which records the changes and updates the ImageView node again via the offscreen Java 2D rendering mechanism. Pretty straightforward, just not what I had expected to have to do now that JavaFX exists.
Scaling The Challenge
What motivated me to write this post is Chris Oliver’s post on Instancing, where he presented his research on the Lens solution. We may never see this excellent solution, and it is a shame that it is not part of the scenegraph already. I can understand how it is a real challenge for the JavaFX engineers to create a flexible scenegraph that can serve different usages, including UI controls, games, animations and demanding graphic scenes like mine that feature too many symbols to be feasibly done with standard nodes only.
Some of us are really trying to put JavaFX to worthwhile use. However, to succeed, we need a flexible, adaptable and scalable scenegraph. We also need more insight into how and where to use the combination of JavaFX and Java 2D because not everyone wants to build only video or twitter applications. We should also have an open-sourced scenegraph API, and know more about where JavaFX is heading in its subsequent versions, instead of having to work in the dark.
Always hopeful…yet discontented,
Thom the inventor