Handling Complex Objects Persistency and Messaging on Mobile

Data persistency and messaging are very common tasks that you almost certainly need in almost all of your apps. Mobile platforms have come a long way in supporting data persistency, mostly through the SQLite engine. It has become the standard on all mobile platforms. However, it is still a very lightweight engine and does not give you all the capabilities of an SQL server. This should not even be the case on a mobile device where persisting data is intended to be mostly for caching until the data reaches its ultimate destination on a server. With this in mind, how do we go about persisting complex objects on the mobile platform? How do we handle complex objects messaging? And this is exactly what we will discuss in this blog post. This is not an introduction on how to use an ORM to store some objects in the database, but rather how to handle the complex relationships between a data object and its children or siblings when storing or messaging that piece of data.

SQLite.NET PCL

I have been developing for mobile platforms for quite some time, and I have been really enjoying the use of SQLite.Net PCL. This is a very simple, lightweight ORM that was built on the initial SQLite.NET with the added support for PCLs (Portable Class Library). This library has made it very easy to store data in the database on a mobile device and it gives you a unified API on all platforms to do all data persistency related tasks, so if you have not used this before, I would highly encourage you to have a look at it, and this blog post will assume the use of this framework for storing data locally.

Versioning

If your mobile application is intended for building a to-do list, then it might not be a troubling thing to lose one entry here or there, or to resolve conflicting items by taking the latest one (I am saying this and knowing that you might lose some users if you do that). However, what if your mobile app was concerned with emergency management, or used by health practitioners? This makes it a requirement to pre-define the logic for how items are stored, conflict is resolved, and you are required to always keep all bits of information on all devices. I had a similar requirement where I needed to maintain the versioning of all items and define how a conflict will be resolved. Thus, I decided to use versioning for my data objects. This design makes the following decisions:

1. Any change will be applied to the data objects as a change log.
2. Change logs are globally and uniquely identifiable via unique IDs.
3. Data objects will have a VERSION attribute, which is unique. This VERSION attribute will refer to the most recent Change Log Id.
4. Each data object will have a list of Change Logs in an ordered list. The ordering of these logs represents the timeline of when these change logs were applied.
5. For the sake of simplicity, we will assume that a data object has a list of properties/attributes that will be represented as table of key/value pairs.
6. Other decisions/assumptions of the design will be ignored for the sake of this blog post. Such decision could include storing the type of change logs (change created by a user, result of data merge, etc), storing other security (authentication/authorisation) data on each data item, storing other meta data on each data object like who changed what and when.

With that in mind, our data object diagram should look something like the following:

Enforcing Version-based Changes

Now that we have put together our basic design, we need to implement it in a way that is safe for a team of developers to work on. We cannot assume that we will ask the team members, “Hey, can you please use this method when you try to apply some changes, because it is necessary?” Think that would work? I know you must be thinking now that this is absurd, but I have seen this in some teams, or if they do not say it this way, they would rely on some comments in the code or some other wiki/documentation. My approach is to make it fool-proof and to let the design document itself. It should not be required to explain this for people. Developers (my team members) should be able to use this without worrying about the internal implementation. So to do that, we need the following:

1. Read-Only Properties

To ensure that we are not going to change any property of our data object without using change logs, properties need to be read-only. This makes sure that we cannot create a new version of the item without either using the applyChangeLog(log) method or using a constructor.

2. Fully Defined Constructor(s)

We cannot provide a constructor that would allow the consumer of our framework to create/instantiate a data object without specifying all its attributes. Therefore, our constructors should define all properties of an object at the creation time.

3. Easy Composition

Our framework (data-object) need to have an easy way to construct objects from a serialized version or from another version of the item. This is very necessary for when storing these data objects to the database or when trying to message them over the network.

With all that out of the way, our basic object implementation could look something like this:

// our change log first
public class ChangeLog 
{
   public string Id {get;set;}
   public int Order {get;set;}
   public string CreatedBy {get;set;}
   public ChangeLogType Type {get;set;}
   public DateTime CreatedOn {get;set;}
   public string ParentId { get; set; }
   public Dictionary<string, object> ChangingAttributes {get;set;}
 }

 public class RegisterItem : ModelBase
 {
   public RegisterItem (string id, string name) 
           : this(id, name, string.Empty, new Dictionary<string,object>(), new List<ChangeLog>())
   {
   }

   public RegisterItem (string id, string name, 
                        string version, Dictionary<string, object> attributes, List<ChangeLog> changeLogs)
   {
      Id = id;
      name = name;
      _version = version;
      _attributes = attributes;
      _changeLogs = changeLogs;
   }

   // This is needed for the internal use (serialisation/De-serialisation, and db storage).
   // Bcoz this ctor is Obsolete, it will through a warning and the warn will be escalated to an error if used.     
   [Obsolete("This ctor is only for the deserialiser and the db layer. Use other ctor with full params")]
   public RegisterItem ()
   {          
   }

   public string Version { get {return _version;} private set { _version = value;}  }
   private string _version {get; set;}

   public string Name { get{ return _name; } private set { _name = value;}  }
   private string _name { get; set; }

   [Ignore]
   public List<ChangeLog> ChangeLogs { get { return _changeLogs; } private set { _changeLogs = value;}  }
   private List<ChangeLog> _changeLogs { get; set; }

   [Ignore]
   public Dictionary<string, object> Attributes {get{return _attributes; } private set { _attributes = value;}}
   private Dictionary<string, object> _attributes { get; set;}
 }

So far, so good; thus far, we have implemented our data objects with its basic versioning for its children objects. Now the question is how we store this in the database and how to serialize/deserialize the object to send it over the network. This is actually the second tricky part, because if you have worked with SQLite.Net before, you know that it is designed to enable mobile developers to store simple objects and basic typed attributes to the database. For our scenario, we have complex objects with children objects and other complex attributes (Dictionary).

Storing in the SQLite database

Our database will store basic information about the data objects (name, id, version) along with a full copy of the object that is serialized to a basic type like string (or a binary, if you like). To make this smooth and simple to our consumers (developers who use this API), we added a property to the data object that is called AsJson. This property will serialize and store the full copy of the object when the object is stored in the database, and when the object is constructed from its basic attributes, it will populate the other properties (like children objects and other complex properties (i.e., Dictionary). A simple implementation of this property could be something like this:

[JsonIgnore]
public string AsJson 
{
  get
  {
     var json = MySerialiser.ToJson(this);
     return json;
  } 
  set
  {
     var json = value;
     if (!string.IsNullOrEmpty(json))
     {
         var originalObject = MySerialiser.LoadFromJson<RegisterItem>(json);
         if (originalObject != null)
         {
            //We could use something like AutoMapper here
            _changeLogs = originalObject.ChangeLogs;
            _attributes = originalObject.Attributes;
         }
      }                                 
  }
}

As you can see, the property itself (AsJson) is ignored when serializing to JSON because it will be a circular-dependency and it would not work. Plus, our developers would need to do anything when storing or pulling from to/from the database. Our AsJson property would do the work and get our items saved/constructed to/from the database. Also, notice how we had the [Ignore] attribute on our complex children objects. This belongs to our ORM (SQLite.Net) which will be understood as no need to store these objects to the database.

Messaging

A couple of months ago, I gave a talk at DDD Melbourne regarding messaging in Peer-2-Peer scenarios on mobile devices; you can find the slide deck here. For this exact project, I needed to be able to serialize/deserialize my data objects to send them over my P2P connections to the other party. This has been made much easier by the property that we discussed earlier, called AsJson. The only tricky part is that when serializing, you need to modify the default settings of your JSON serializer as it needs to be able to serialize/deserialize private properties of the data objects. Assuming that we use something like Newtowonsoft.Json, our serializer will be something like this:

var resolver = new PrivateSetterContractResolver();
var settings = new JsonSerializerSettings{ ContractResolver = resolver };
var obj = JsonConvert.DeserializeObject<T>(input, settings);

And that’s it. Hope you find this useful and you have picked up a few ideas on how to handle storing and messaging of complex data objects on mobile. If you have a question, comments, or maybe a suggestion to do things in a different/better way, I would love to hear from you, so get in touch.