Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Data Interchange Formats for IoT, Mobile, and Cloud Apps

DZone's Guide to

Data Interchange Formats for IoT, Mobile, and Cloud Apps

This excellent compilation of data interchange formats and associated tools can serve as a cheatsheet to have when building your IoT, mobile, or cloud apps.

· IoT Zone ·
Free Resource

As a technologist interested in reducing network traffic while applications communicate together,  reducing battery consumption in mobile applications, and the enablement of IoT, I have been trying over the past year to work out suitable data formats for IoT and mobile applications.

It was key to me that the formats be available in the Java and Android platforms and usable in any XML- or JSON-based application, including the existing ones, with virtually no impact. Why start with Java and Android? The reason is simple: They are my preferred platforms for building applications. Secondly, the data formats will be made available on the .NET platform (C# language) following the same principles.

Data Interchange Formats Suited for IoT and Mobile Applications: The BDB Formats

BDB stands for binary data buffer. It is a family of binary data formats designed to have the following characteristics:

  1. Lightweight

  2. Compact

  3. Fast

  4. Easily traversable. A less easily traversable version is a possible alternative

  5. Schema-less

  6. Self-describing

  7. Efficient. Encoding and decoding are performed very quickly

  8. Text-editable. Viewing and editing in a text editor of choice (preferably open source) is one of the requirements

The BDB data formats are TLV-like data formats that have built-in support for the following data types:

  1. int8

  2. int16

  3. int32

  4. int64

  5. integer

  6. float32

  7. float64

  8. decimal

  9. boolean

  10. string

  11. enumerable-string

  12. uri

  13. binary

  14. instant (date, datetime, timestamp and time; with or without timezone info)

  15. duration

  16. time-period

  17. array

  18. object

  19. GUID

  20. ObjectId

  21. int32 array

  22. int16 array

  23. int8 array

  24. int64 array

  25. boolean array

  26. float32 array

  27. float64 array

  28. string array

  29. uri array

  30. cstring — latin/8-bit string; may be localized

  31. IETF date-time

  32. cstring array

The following encodings are supported for string values:

  1. UTF-8

  2. UTF-16BE

  3. UTF-16LE

  4. US-ASCII

  5. ISO-8859-1

  6. UTF-32BE

  7. UTF-32LE

  8. windows-1252 (or CP1252)

  9. ISO-8859-15

  10. x-utf-24 — compressed UTF-32BE

  11. x-utf-24le — compressed UTF-32LE

  12. windows-1251

  13. ISO-8859-5

  14. ISO-8859-6

  15. ISO-8859-9

  16. ISO-8859-11

BDB data formats started with six flavors:

  1. BDB: The generic BDB data format

  2. XBDD: The BDB data format for the XML data model; comments and CDATA are supported 

  3. JSONB: The BDB data format for the JSON data model

  4. CSVB: The BDB data format for CSV data model — does not support array or object data types

  5. YAMLB: The BDB data format for the YAML data model

  6. HBDB: The BDB data format for the HTML data model — comments and CDATA are supported

  7. XHBDB: The BDB data format for the HTML data model — comments and CDATA are supported

XBDB

Full support of xml data model is a MUST.

  • Attributes

  • Namespaces

  • Elements

  • Comments

  • CHARACTERS

  • CDATA

  • DOCTYPE

Availability of the javax.xml.XMLStreamReader and javax.xml.XMLStreamWriter java streaming APIs in Java is a must-have.

Availability of an XmlPullParser Android XML streaming API for the Android platform is a must as well.

Meanwhile, availability of system.j.core.xml.IXMLStreamReader for the Java/Android platforms is strongly recommended.

Availability of C#'s standard XML streaming APIs for use of XBDB is also strongly recommended.

The open source JEdit is going to be tweaked to provide for viewing and editing of XDBD files as if they were text XML files.

Compactness

The encoding of the data of a =n inv100.xml file (97230 bytes) that comes along with fastinfoset, using the javax.xml.XMLStreamWriter API through a class built for supporting XDBD transparently in the standard Java platform fashion yields a compression factor of 90%. No schema has been used, and no non-javax.xml.XMLStreamWriter features have been used.

JSONB

Full support of the JSON data model:

  • Number, String, Boolean, Null

  • Array

  • Object

Support for the BDB data types that are standard JSON data types is a MUST.

Availability of the javax.json.stream.JsonParser Java streaming APIs is a MUST.

Availability of the javax.json.stream.JsonGenerator Java streaming APIs is a MUST.

Availability of system.j.core.json.IJSONStreamReader for the Java/Android platforms is strongly recommended.

Availability of C# standard JSON streaming APIs for use of JSONB is strongly recommended.

JEdit open source will be tweaked to provide for viewing and editing of XDBD files as if they were text XML files.

HBDB and XHBDB

These are the same as XBDB with the difference that elements/attributes are not extensible. They are limited to HTML/XHTML elements/attributes.

CSVB

CSVB is the BDB flavor corresponding to the CSV data format as specified in RFC4180: 

  • Starts with a header that specified the names and the data types of the fields

  • The header is optionally followed by the indicator of the field separator, which is only useful in viewing/editing the data in a text editor

  • Then comes the records

The support of all the BDB data types, of course except array and object, is a MUST. JEdit will be tweaked to provide viewing and editing of XDBD files as if they were text XML files.

Registered MIME Types

The following MIME types have been registered at IANA for the BDB data interchange formats: 

  • application/vnd.xdbd

  • application/vnd.jsonb

  • application/vnd.csvb

  • application/vnd.bdb

The BDB data formats will be governed by a standard organization later. Once this is done, new registrations with the "vnd." prefix dropped from MIME sub-types will be requested from IANA while the MIME types having the "vnd." prefix will be kept for compatibility's sake.

More BDB Flavors Are Available or Soon to Come

Below is just an excerpt: 

  • CSSB (for Cascading Stylesheet)

  • DTDXB (for Document Type Definition)

  • XSDB (for XML schema)

  • DOCBOOKB (for DocBook)

  • XBRLB (for xbrl)

  • RNGB (for RelaxNG)

  • FPMLB (for fpML)

  • OPENOFFB (for Open Office)

  • CXMLB (for cXML)

  • XSLTB (for XSLT)

  • XSLFOB (for XSL-FO)

  • IFXB (for IFX)

  • SWIFTMXB (for SWIFT XML-based standards)

Topics:
iot ,iot development ,mobile development ,cloud development ,data interchange

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}