I asked NetBeans Dream Team member Tom Wheeler this question, in relation to his contribution to advice given during a training session described here. Below follows his response, which could apply to any modular system (based on OSGi bundles, NetBeans modules, or anything comparable).

There's really no right answer for knowing how to split your application into multiple modules.  In general, it's better to have lots of small modules instead of a few big ones. In general, I definitely prefer to have lots of little modules than a few big ones. If you're designing a new app from scratch, it's easy to achieve "smallness" in your modules and you'll have a better design (less coupling) because of it.  If you're porting an existing application to the NetBeans platform, then you'll probably just have one or two big modules at first and you'll have to achieve smallness as you make things more modular over time.

One rule of thumb is that you can often split the application into modules based on package boundaries.  Thus, if you have the following packages:

    com.tomwheeler.app.gui
    com.tomwheeler.app.data
    com.tomwheeler.app.logic

You might refactor this into a GUI module, a data module and a logic module.  As you continue development, you might find that these could be split even further.  For example, your data module might have objects representing customer data and product data, so you could create a customer data module and a product data module.

Making your application modular actually makes refactoring easier because it's clear exactly what you've exposed as a public API.  For any given feature, I almost always strive to separate API, implementation and client code into separate modules. This allows me to develop iteratively -- I can first concentrate on the API, then write a very basic implementation of it.  I can then develop additional (or better) implementations of that API as time allows.

One example might be a reporting subsystem for an application. I could create an API that defines what's needed to generate a report, but the API won't focus on details like file format.  I can then create the simplest possible implementation of that API; it might create a plain text version of the report.  I can later create a new module which creates a better report, perhaps in Excel or PDF format.  While I could simply replace the first version, I could also easily add a little code to let the user select the desired report format.

In addition to reducing coupling in your application's design, having smaller modules gives you more flexibility.  When your application is modular, it's very easy to build different versions of the application with different sets of features, kind of like the "basic", "standard" and "enterprise" versions of software you sometimes see.  Doing that with a monolithic application is usually pretty difficult.

If you, reading this article, have experience in this area, i.e., modularization, please share your thoughts on this topic in the comments below, since doing so would be very useful to others starting on this path!