Discussions all over the place attest that each of these frameworks is different enough that transferring from one to the other is not a painless process. Therefore making the right choice up front is pretty important. Assuming you've arrived at the point where you're spending more time working on your in house framework than on the application/s built on top of it, it's time to look around for one of the existing frameworks to handle your infrastructural concerns. In the web world, no serious application works with plain old JSPs and servlets—instead, the developer/s choose a framework as the basis of their application, since it provides the underlying requirements common to all web applications. So it is too, increasingly, in the Java desktop space, where the developer seldom has the time, interest, and/or skillset to develop an application's "plumbing"—lifecycle management, docking framework, actions system, etc. But which specific framework to choose? By means of which criteria?
So, without further ado, here are what are, in my personal opinion, the most important criteria to consider. I'd be interested to hear others and then we can perhaps pick up each of the existing desktop frameworks and see how they stack up.
- Architecture. How scaleable is it? I.e., can it be extended from the outside, via plugins? Is it based on generally accepted standards? E.g., Swing is the standard UI Toolkit; OSGi is the standard bundle format. On the other hand, do "generally accepted standards" matter to you? And does it matter if the application can't be extended? How well can existing frameworks integrate into the framework? E.g., if the framework offers a windowing system, is it possible to swap in a different windowing system offered by competing vendors? Or are you locked into all of the framework's native features? How modern is the architecture? E.g., is there a strong reliance on XML or is it possible to use annotations instead? How flexible is the architecture for these kind of modifications?
- Feature Set. Are the features very extensive or only very basic? How much of a very large application's boilerplate code and typical "plumbing" can you let the framework handle for you? Does the framework provide you with little more than an action system? Or is there also a windowing system? And how well does it play with others? And how extendable is it? Can it, in fact, be extended and, if so, how easily and how well supported is it to do so? If you were to write down all the things that you'd like your perfect framework to have, which of the existing frameworks matches your needs?
- Open Source. Is it open source? Does it matter? What are the pros and cons when it comes to your specific needs? Are the fears you, or someone else, might have in relation to open source in this context balanced out by other factors?
- Community. Is it large? If it is large, who does it consist of—lots of one-person developers or large companies with distributed developers working on the same project? If it is small, is it growing? If it is small, are there good reasons? Is the community passionate? Are there few/many examples of the framework being used "for real"? A long list of screenshots? Testimonials? Blurbs on various websites attesting to the wonderfulness of the framework underpinning their app? Or are they too embarrassed to mention it?
- Learning Curve. How hard is it to get started? If it is hard, are there good reasons, such as the complexity of the framework? In that case, is the complexity justified or is it an early warning sign of a badly designed framework? If the learning curve is long, is the end point worth reaching? If the learning curve is small, could that be because the framework doesn't have much to offer? If the learning curve is long, is that because the architecture has introduced many new concepts that are known in other forms elsewhere, instead of simply using the standard approaches? Does it rely very heavily on existing approaches elsewhere, thus making the learning curve pleasantly short? Does the framework offer a training course?
- Tooling. What kind of tools are there for the framework? Do the major IDE's support it? Or maybe it is so good that there are very few specific artifacts requiring special support? How much can be generated/automated? What are the specific pain points where tooling is needed and could those pain points be solved without tooling, i.e., by implementing the framework in a different way?
- Documentation. Is it reliable? Up to date? Are there contributions from the comunity? Does it cover the absolute basics? Does it also cover all the advanced topics that the framework concerns itself with? How many unexpectedly undocumented "gotchas" does one typicaly come across? Are there recently published books about the framework? (That's a good sign because it means that the framework lasted at least as long as the time it took for the book to be written and published.) How many books? How up to date? Only in English? Have they been translated? If not translated, is there interest in doing so?
- Support. Considering the documentation and the community, as well as anything else the framework might offer (mailing lists, helpdesk), how quickly/efficiently can your problems be solved when you have them? If you were to have a deadline tomorrow with a big problem today, which framework's support would be the one you could most rely on to help you out? Do the mailing lists have, on balance, more complainers than helpful people? How responsive are the core contributors to the framework to questions that are asked by newbies? How quickly and thoroughly are bug reports handled? Are there patches or do you have to wait for the next release when you encounter a bug? Can bugs be escalated and under which circumstances?
- Future. What's the framework's future? Is there a company behind it that is invested heavily in it? How does it communicate with its users about its future, if at all? What's the framework's past been like? E.g., has it dropped out of the air without any explanation and has it suddenly reappeared on the scene without much of a plan going forward? Has it existed consistently and has it shown solid progress over its lifetime? If it were a piece of ice in the middle of a lake, would you feel safe jumping on it?
I suppose many of these criteria could apply to anything. However, here I'm trying to provide someone evaluating desktop frameworks with some criteria that they might have overlooked while shopping for a solution that could, potentially, have a very big impact on the success of their project. Interested to know which of these spark debate and also to see whether there are others that should be added!