Over a million developers have joined DZone.

The day after JSON: Spearal and mobile apps

· Mobile Zone

Visually compose APIs with easy-to-use tooling. Learn how IBM API Connect provides near-universal access to data and services both on-premises and in the cloud, brought to you in partnership with IBM.

The goal of the Spearal project is to provide an open-source serialization library that just works, whatever the complexity of your data structures, and that can be used as a simple replacement for JSON in most cases. We are here going to show an example with a Spring-based server and an Android app.

If you want to get more comprehensive information on the project, it would be a good idea to first have a look at the introduction to Spearal by Franck.

Almost all applications are now using JSON as a 'choice by default' serialization format to transfer data between mobile/Web clients and servers. Franck demonstrates in his article some common issues with JSON and how they are addressed in Spearal:

  • Support for object references and circular graphs
  • Use of property filters to get different levels of details
  • Built-in support for JPA lazy associations
  • Support for differential updates with partial objects

Handling these issues with JSON is possible at the cost of a lot of manual written boilerplate code, such as DTO classes, copying of properties between objects (often with tools like Dozer), additional methods or parameters on the server resources...

All this adds a significant burden on the development and maintenance of the application and Spearal aims at eliminating all this useless code and providing a more data-friendly serialization format.

In this article I will show an example of Spearal in a mobile environment with a Spring MVC REST server, an AngularJS application and a Spring Android REST application. This is a Spring port of the JAX-RS example from Franck's article which was itself forked from the nice Java EE 7 / AngularJS example by Roberto Cortes.

Clone, build and start the demo

Execute the following commands (you will need Git and Maven):

git clone https://github.com/spearal-examples/spring-angular.git
cd spring-angular
mvn clean install tomcat7:run

When Tomcat is started, point your browser to http://localhost:8080/spring-angular/.  The AngularJS application is quite simple and provides basic CRUD operations using an AngularJS REST/JSON resource. If you have already tried the javaee7-angular example, it is the exact same AngularJS application.

You can easily switch to Spearal serialization by browsing the alternate url http://localhost:8080/spring-angular/index-spearal.html.

I won't go into deep details on how to configure the AngularJS application to use Spearal because it has already been described in Franck's article but to summarize it requires the following 3 steps:

  1. Import the traceur runtime and compiled JS6 Spearal library
  2. Setup the AngularJS resource actions with a binary payload and the Spearal encoder/decoder
  3. Setup the AngularJS resource actions with the Spearal Accept/Content-Type headers

In a perfect world there would be a Spearal/AngularJS module which would further simplify this setup, for now it's a little bit of manual work.

Nice, but what else can Spearal do for me ?

A common requirement in many applications is to be able to serialize the same data object with various levels of details. For exemple here we would like to get the full object in the edit form but only a few properties to display in the table.

With JSON you would do this by writing different DTOs, for example we could have Person and PersonSmall where PersonSmall would have only 3 properties id, name and description. For a responsive UI where the table adapts to the available size of the screen, we would need even more versions of the object like PersonSmall, PersonMedium, etc. You would also have to write different methods that copy the query results to these diffent objects or use tools like Dozer. This is not only tedious but also quite inflexible because we have to think in advance to various use cases, and we have to build the server code accordingly.

Maintaining these DTOs is a pain and one of our main goals for Spearal is to eliminate DTOs once and for all. Why not just let the serializer extract only a subset of the object properties, much like a 'SELECT id, name, description FROM person' in a SQL query (yes, SQL is old-fashioned but this example would be far less understandable with a Map/Reduce function). This is exactly what Spearal property filters are made for, so let's enable a filter for the table in the example index-spearal.html:

if (method === "query") {
    action.headers[Spearal.PROPERTY_FILTER_HEADER] = Spearal.filterHeader(
        "id", "name", "description"

The query action is the one used to fetch data in the table, we just pass the filter in a specific HTTP header that will be retrieved by Spearal on the server. You can also note that we have to use the server-side class name to indicate on which object we want to apply the filter.

Stop Tomcat, rebuild and restart the application with this command:

mvn clean install tomcat7:run

Refresh the index-spearal.html page. Everything is hopefully working as before, but as the property url is quite long, we have significantly reduced the size of the encoded data (as can be verified with the Chrome developer tools).

The client can control exactly the amount of data it needs, without any change on the server application. We could have different sizes for the table with different properties, like a compact mobile version and a full desktop version. For example we will show later with the Android client how we can adapt the display and the fetched properties depending on running on a phone or a tablet.

It is important to note that not all properties have been encoded so we get only partial objects on the client. That means that some properties of the JavaScript objects that we receive from filtered requests are undefined (which is not null, but effectively unknown). This is definitely what we want on the client but what happens when we send this partial object back to the server ?

There is no concept of an 'undefined' property in Java, so partial objects are instantiated as Javassist proxies on the server, and access to an undefined property will trigger an UndefinedPropertyException. While this is not an issue in many cases, this would prevent us from using directly these partial objects with a JPA EntityManager for example and lead us back to the problem of DTOs with which you constantly have to copy properties between objects. Fortunately that is exactly why the Spearal/JPA integration provides an EntityManager wrapper that supports the merge of partial objects. This integration is very simple to configure, in particular when using the Spring framework.

Some explanations on the Spring configuration

The Spearal integration with Spring consists in different parts:

  • A message converter that encodes/decodes the Spearal format
  • A body interceptor that retrieves the property filters from http headers and applies them to the encoded response
  • A Spearal configurator that allows using Spring beans for all pluggable elements in Spearal
  • A comprehensive JPA support that allows proper encoding of lazy associations and merge of partial objects

The configuration is very simple and can be done using either XML or Java config. Here is an example of a XML config:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	<bean id="spearalFactory" class="org.spearal.DefaultSpearalFactory"/>

And with Java config:

public class PersonWebConfig extends WebMvcConfigurerAdapter {
    public SpearalFactory spearalFactory() {
        return new DefaultSpearalFactory();
    public RestResource restResource() {
        return new RestResource();
public class PersonJpaConfig {
    public LocalContainerEntityManagerFactoryBean entityManagerFactory() {

This is just an additional encoder configuration, so your server is still able to respond to JSON-encoded requests but now it can also respond to Spearal-encoded requests.

The Android client

Now that we have a working Web application, let's build a native Android application. We already use Spring on the server, so why not use it also on the client. The Spring Android REST client uses the same low-level architecture than Spring MVC REST, so we can reuse the Spearal/Spring library for our Android app.

You will need Android Studio to run the example. In the menu VCS / Checkout from  Version Control / Git, select https://github.com/spearal-examples/spearal-example-android.git to clone the example in Android Studio.

By default the app will connect to the example server application hosted at https://examples-spearal.rhcloud.com/spring-angular/ so you don't need to have your own running server.

The application itself is quite simple and has been created using the master/details flow template proposed by Android Studio when creating a new project.

Let's just see what we have changed to consume Spearal REST resources instead of dummy local data.

The first step is to define the client data model which consists in the two classes Person and PaginatedListWrapper corresponding to what the server exposes. Here is for example the class Person:

public class Person implements Serializable {

    private static final long serialVersionUID = 1L;
    private Long id;
    private String name;
    private String description;
    private String imageUrl;

    public Long getId() {
        return id;
    public void setId(Long id) {
        this.id = id;

    public String getName() {
        return name;
    public void setName(String name) {
        this.name = name;

    public String getDescription() {
        return description;
    public void setDescription(String description) {
        this.description = description;

    public String getImageUrl() {
        return imageUrl;
    public void setImageUrl(String link) {
        this.imageUrl = link;

It's the same bean as the one on the server, only stripped from the JPA annotations.

Now that we have a model, we need to be able to call the REST service. To implement this we are going to call Spring RestTemplate methods from an Android AsyncTask to avoid blocking the UI during the remote call. We create a simple base task class which encapsulates the common configuration and usage of the RestTemplate:

public abstract class AbstractRestAsyncTask<Params, Progress, Result> extends AsyncTask {

    private final RestTemplate restTemplate;
    protected boolean success = false;

    private static String url(String path) {
        return PersonListActivity.baseUrl + path;

    protected AbstractRestAsyncTask() {
        restTemplate = new RestTemplate();
        restTemplate.getMessageConverters().add(new SpearalMessageConverter(SpearalFactoryHolder.getInstance()));

    protected Result doInBackground(Params... params) {
        try {
            Result result = doRestCall(params);
            success = true;
            return result;
        catch (Exception e) {
            Log.e(getClass().getSimpleName(), e.getMessage(), e);
        return null;

    protected void onPostExecute(final Result result) {
        if (success)

    protected final <R> R getForObject(String path, Class<R> resultClass, Object... params) {
        return restTemplate.getForObject(url(path), resultClass, params);

    protected final <R> R getFiltered(String path, Class<R> resultClass, SpearalPropertyFilterBuilder filter, Object... params) {
        SpearalEntity<Params> filterEntity = new SpearalEntity<Params>(SpearalFactoryHolder.getInstance(), null, null, filter);
        ResponseEntity<R> responseEntity = restTemplate.exchange(url(path), HttpMethod.GET, filterEntity, resultClass, params);
        return responseEntity.getBody();

    protected final Result postForObject(String path, Class<Result> resultClass, Params object) {
        return restTemplate.postForObject(url(path), object, resultClass);

    protected final void delete(String path, Object... params) {
        restTemplate.delete(url(path), params);

    protected abstract Result doRestCall(Params... params);

    protected abstract void onRestSuccess(Result result);

The task first initializes a RestTemplate and configures the SpearalMessageConverter. This converter will be able to encode and decode the Spearal format, and set the correct Accept/Content-Type http headers for the message payload.

In addition to wrapping the default RestTemplate methods we add a getFiltered method which will pass a property filter header to the server and that we will use to fetch the list of persons to display in the main table.

You may have noticed that most Spearal related objects require a SpearalFactory object which holds Spearal internal configuration. In an Android app, it's a common practice to store global configuration objects in static singletons. This is what we do here in the SpearalFactoryHolder class.

public class SpearalFactoryHolder {
    private static SpearalFactory spearalFactory = null;
    public static SpearalFactory getInstance() {
        if (spearalFactory != null)
            return spearalFactory;
        spearalFactory = new DefaultSpearalFactory();
        spearalFactory.getContext().configure(new PackageTranslatorAliasStrategy(
            "org.spearal.examples.android", "org.spearal.samples.springangular"));
        return spearalFactory;

This is also the good place to configure the SpearalFactory. Here we define a particular aliasing strategy to convert the package names of the server model to the package names of the client model. The PackageTranslatorAliasStrategy used here implies that the class names and package structures of the domain model are the same between client and server but it's usually an acceptable tradeoff and anyway we could define any kind of aliasing strategy for more complex requirements.

The application is now correctly configured and we have everything needed to call our remote service in the view fragments. The application includes two activities and two fragments, one for the main list and one for the detail, the detail activity being used only for large/tablet displays.

First the list fragment: we execute a load task in onStart() to fetch the data to display in the list:

public void onStart() {

    new LoadPersonsTask().execute();

The task first determines a property filter depending on the size of the display. isTwoPane() indicates that the application runs on a large screen device such as a tablet so we will add a second column to the table, and add the property description to the filter.

protected List<Person> doRestCall(Void... params) {
    SpearalPropertyFilterBuilder filter = mCallbacks.isTwoPane()
       ? SpearalPropertyFilterBuilder.of(Person.class, "name", "description")
       : SpearalPropertyFilterBuilder.of(Person.class, "name");
   return getFiltered("/persons?pageSize=100", PaginatedListWrapper.class, filter).getList();

In the success callback, we then set the list adapter to display the items in the table and show one or two columns:

protected void onRestSuccess(final List<Person> persons) {
    if (mCallbacks.isTwoPane()) {
        setListAdapter(new ArrayAdapter<Person>(getActivity(),
            R.layout.list_item_large, android.R.id.text1, persons) {
            public View getView(int position, View convertView, ViewGroup parent) {
                View view = super.getView(position, convertView, parent);
               return view;

    else {
        setListAdapter(new ArrayAdapter<Person>(getActivity(),
            android.R.layout.simple_list_item_activated_1, android.R.id.text1, persons) {
             public View getView(int position, View convertView, ViewGroup parent) {
                 View view = super.getView(position, convertView, parent);
                 return view;

    getListAdapter().registerDataSetObserver(new DataSetObserver() {
        public void onChanged() {
            new LoadPersonsTask().execute();

Finally we register a dataset observer on the list adapter so other fragments can notify that the list content has changed and that we should refresh the data.

The detail fragment is very similar, we just trigger asynchronous tasks when we have to interact with the server resource.

The following screenshots show how the app appears in phone and tablet modes:

In conclusion, there is very little difference in the resource access code between an Android app using JSON and an Android app using Spearal. Replacing JSON by Spearal in an existing or new application is very easy and will buy you a lot of additional power: object references, circular graphs, better serialization of numeric types, partial objects, support for JPA...

Working with partial objects

As explained earlier, the effect of using server-side property filters is that the client receives only partial objects. We have not yet found a reasonable way to implement class proxies in Android (though DexMaker looks like a good option) so Spearal falls back to creating plain objects and just letting default values (usually null) for undefined properties.

While this is perfectly acceptable for read-only objects (like those displayed in the table), it is not if we want to send partial objects back to the server, because a null value cannot be distinguished from an undefined value and we risk to overwrite existing values.

This would be the case for example if we get an undefined lazy association from a JPA entity, which would be translated as null and then sent back to the server as null and corrupting the database if we merge it as it.

To avoid this issue, we will use here an option provided by Spearal that consists in defining the data model as an interface and let Spearal use Java dynamic proxies to implement the actual data object. We will thus transform the class Person into an interface with only getters and setters:

public interface Person extends Serializable {

    public Long getId();
    public void setId(Long id);

    public String getName();
    public void setName(String name);

    public String getDescription();
    public void setDescription(String description);

    public String getImageUrl();
    public void setImageUrl(String imageUrl);

Obviously we cannot do new Person() anymore so we have to use a small utility method in the SpearalFactory:


This is not an ideal solution, but it works fine and even makes simpler to write the client data model. The main drawback is that you don't have control on the model class and cannot extend it or put methods in it. It can be perfectly fine for the aficionados of a pure domain model without business logic methods, this may not be totally satisfying for others (us included) and we will take care of this issue in a next version.

To see these new partial objects in action, we have to make some changes on the server, for example add two new properties to the server model that demonstrate the support of JPA lazy associations and data model evolution:

private Person bestFriend;

public Person getBestFriend() {
   return bestFriend;
public void setBestFriend(Person bestFriend) {
   this.bestFriend = bestFriend;

private Person worstEnemy;

public Person getWorstEnemy() {
    return worstEnemy;
public void setWorstEnemy(Person worstEnemy) {
   this.worstEnemy = worstEnemy;

Theses changes are already deployed at the hosted location https://examples-spearal.rhcloud.com/spring-angular-v2/index-spearal.html where you should see the additional column Worst Enemy (don't try the JSON version at /index.html, it does not work with lazy associations).

To point the Android app to this url, just use the arrow icon on the top right before the plus icon and select Example v2.

You can use the same 'outdated' Android application (not upgraded to the new model) with both the old version of the server and the new one without any change, the existence or absence of new properties in the client model being properly handled by the server-side Spearal deserialization and JPA integration.

Note: if you want, you can of course add your own local server url (with your real IP address, the device cannot access localhost) to the menu.

Differential updates

Having these partial objects on the client has another nice use: the client can send only some of the properties and not the complete object. For example we could send only the properties that the user has changed. We can see how this works in the method applyTextValue:

applyTextValue(person, "name", getView(), R.id.form_name);
private static void applyTextValue(Object object, String propertyName, View rootView, int textViewId) {
    try {
        Method getter = object.getClass().getMethod("get" + propertyName.substring(0, 1).toUpperCase() + propertyName.substring(1));
        Object currentValue = getter.invoke(object);
        String value = ((TextView)rootView.findViewById(textViewId)).getText().toString();
        if (value.equals(currentValue))
            Spearal.undefine(object, propertyName);
        else {
            Method setter = object.getClass().getMethod("set" + propertyName.substring(0, 1).toUpperCase() + propertyName.substring(1), String.class);
            setter.invoke(object, value);
    catch (Exception e) {
        // Should probably do something

Basically we get the existing value on the object and compare it to the value of the TextView. If it has changed we update the object with the setter method and if not we 'undefine' the value of the partial object so it won't be sent on the network. At the end what will be sent over the wire are just a small object containing only the changed properties. On the other side, the server merge will be able to handle this correctly.


We have seen how Spearal can make our REST resources more data friendly. We have also seen how we can remove the need for DTOs in lots of usual cases.

Spearal has mostly been thought and designed for two use cases:

  • Controlled environments where you build both the server and the clients
  • External APIs where you want to provide your users with a strongly-typed data model and library and not only a set of REST/JSON resources

Of course that does not mean that you can't use it for other cases !

The next important step is the iOS implementation which first release should be ready soon.

This is just the beginning of the project, so feedback is very appreciated, and obviously contributions and bug reports are welcome.

Follow us on Twitter: @Spearal and join our user forum.

The Mobile Zone is brought to you in partnership with Strongloop and IBM.  Visually compose APIs with easy-to-use tooling. Learn how IBM API Connect provides near-universal access to data and services both on-premises and in the cloud.


The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}