What Is Recon? How We Augmented XML and JSON For Streaming Data

Ever since applications started moving data records, we’ve needed ways to annotate those records with formatting instructions. Many of these record notation formats are familiar to developers. For example, according to the IETF, JSON “defines a small set of formatting rules for the portable representation of structured data.” In practical terms, JSON makes it possible to describe value pairs, arrays, or a series of values as a human-readable document.

Similarly, the prolific XML markup language makes it possible to encode data into a format that is both human and machine-readable. Without formatting instructions provided by JSON and XML, machines would lack the context necessary to express and analyze documents. However, what happens when data cannot be expressed as a document?

For example, streaming data cannot be adequately expressed as a document. Documents are historical artifacts, a snapshot of data at a defined point in time. Conversely, streaming data is an ongoing and continuously updating series of values; representing streaming data as a document is like trying to capture all the water from a fire hose with water buckets (without losing water pressure!). Instead of trying to represent streaming data as a document, developers need a way to annotate streaming data and include relevant state context for every event. In other words, there’s a need for a language that combines object notation with attributes. This is the motivation for the Recon notation language.

What is Record Notation (Recon)?

The name Recon is shorthand for record notation. Recon is object notation with attributes, sort of like if JSON and XML had a baby. Recon attributes facilitate uniformly disambiguating polymorphic structures. And first-class language extensions for selectors, expressions, and functions make Recon an expressive format for domain-specific languages. Recon is part of the open-source swimOS framework, which can be found here.

Recon combines the simplicity of JSON with the expressiveness of XML. As shown in the example below, Recon looks a bit like a hybrid of the two. Yet, Recon is deceptively simple: the grammar for Recon is scarcely larger than the grammar for JSON. This underlying uniformity makes Recon more expressive and more consistent to work with than either XML or JSON.

Here's a Recon example:

@html {
  @head {
    @title "Greetings"
  }
  @body {
    @h1 "Introduction"
    @p [I have @a(href:"https://en.wikipedia.org/wiki/Markup_language")[markup syntax]
        for when you need it.  But I'm not a text chauvinist.  I'm a structured object
        notation first and foremost.  The numbers {1, 2, 3} are parsed as numbers,
        not strings.  Any my attributes make it easy to define, embed, and
        disambiguate microformats and domain specific languages.]
    @p [Need a microformat for time?  You'll find it falls out naturally after
        {{10 @minutes}} of using Recon.  Need to build a DSL for real-time GUI
        widgets?  Recon helps you do so cleanly and concisely, like this:]
    @pie {
      title: "Events"
      linkStats: @link(host: "warp://traffic.swim.services", node: "swim:meta:mesh", lane: "linkStats", type: value)
      @slice {
        value: $max(0.1, $rate($linkStats.downMessageCount))
        label: @text($percent($value, $total))
        legend: @text([Down ({$round($value)}/s)])
        innerRadius: 10 + 7.5 * $value / $max($value) @pct
        outerRadius: 20 + 7.5 * $value / $max($value) @pct
      }
      @slice {
        value: $max(0.1, $rate($linkStats.upMessageCount))
        label: @text($percent($value, $total))
        legend: @text([Up ({$round($value)}/s)])
        innerRadius: 10 + 7.5 * $value / $max($value) @pct
        outerRadius: 20 + 7.5 * $value / $max($value) @pct
      }
    }
  }
}

Data Types in Recon

Record Notation has six primitive data types: text, data, num, bool, extant, and absent and one aggregate data type: record. Below, I’ve included a brief description of each data type. You can read more about Recon data types here on GitHub.

• Text values: Text values take one of two forms, a quoted string or an unquoted identifier.

• Data values: Binary data encoded as a leading % symbol, followed by a base64 literal.

• Num values: Numbers serialized as decimal literals.

• Bool values: Booleans are represented by true  and false  identifiers.

• Extant values: Extant symbolizes something that is defined, but which has no specific value. Extant is represented by an empty token where a value is expected.

• Absent values: Extant symbolizes a thing that is defined, but which has no specific value. 

• Record values: Records play the combined role of an array and associative array. Think of a record as a partially keyed list — a sequence where some items may have keys and other items may lack keys. An array is a record in which no items have keys. An associative array is a record in which every item has a key. An object is a record where every item has a text key.

For users trying out Recon for the first time, I recommend starting with the swimOS tutorial. You can also read more about Recon on the swimOS developer site and on GitHub.

Learn More

Let us know what you're building using the open source swimOS platform. You can get started with swimOS here and make sure to STAR us on GitHub.