Introducing MapNeat, a JVM JSON Transformation Library
Given Kotlin's high interoperability with most of the JVM languages, MapNeat is easy to use in any Java project without any particular hassle.
MapNeat is a JVM library written in Kotlin, that provides an easy to use DSL (Domain Specific Language) for manipulating and transforming existing JSONs, XMLs, or POJOs into new JSONs. The trick is that no intermediary model classes are needed, and all the changes are done using a mostly descriptive approach.
The library can be particularly useful when integrating various systems that need to exchange messages in different formats, creating DTOs, etc.
Given Kotlin's high interoperability with most of the JVM languages, MapNeat is easy to use in any Java project without any particular hassle.
How it Works
A typical transformation starts with the source input (JSON, XML, or any Java Object), and then it contains a series of Operations applied in order:
val jsonValue : String = "..."
val transformedJson = json(fromJson(jsonValue)) {
/* operation1 */
/* operation2 */
/* operation3 */
/* conditional block */
/* operation4 */
}.getPrettyString() // Transformed output
If the source is XML, "fromXML(xmlValue: String)" can be used. In this case, the "xmlValue" is automatically converted to JSON using JSON In Java.
If the source is a POJO, "fromObject(object)" can be used. In this case, the object is automatically converted to JSON using Jackson.
The First Example
As a rule, multiple inputs (sources) can be used inside a transformation. So, given two JSON objects JSON1 and JSON2:
JSON1
xxxxxxxxxx
{
"id": 380557,
"first_name": "Gary",
"last_name": "Young",
"photo": "http://srcimg.com/100/150",
"married": false,
"visits" : [
{
"country" : "Romania",
"date" : "2020-10-10"
},
{
"country" : "Romania",
"date" : "2019-07-21"
},
{
"country" : "Italy",
"date" : "2019-12-21"
},
{
"country" : "France",
"date" : "2019-02-21"
}
]
}
JSON2
xxxxxxxxxx
{
"citizenship" : [ "Romanian", "French" ]
}
We write the transformation as:
xxxxxxxxxx
val transform = json(fromJson(JSON1)) {
"person.id" /= 100
"person.firstName" *= "$.first_name"
"person.lastName" *= "$.last_name"
// We can using a nested json assignment instead of using the "." notation
"person.meta" /= json {
"information1" /= "ABC"
"information2" /= "ABC2"
}
// We can assign a value from a lambda expression
"person.maritalStatus" /= {
if(sourceCtx().read("$.married"))
"married"
else
"unmarried"
}
"person.visited" *= {
// We select only the country name from the visits array
expression = "$.visits[*].country"
processor = { countries ->
// We don't allow duplications so we create a Set
(countries as List<String>).toMutableSet()
}
}
// We add a new country using the "[+]" notation
"person.visited[+]" /= "Ireland"
// We merge another array into the visited[] array
"person.visited[++]" /= mutableListOf("Israel", "Japan")
// We look into a secondary json source - JSON2
// Assigning the citizenship array to a temporary path (person._tmp.array)
"person._tmp" /= json(fromJson(JSON2)) {
"array" *= "$.citizenship"
}
// We copy the content of temporary array into the path were we want to keep it
"person._tmp.array" % "person.citizenships"
// We remove the temporary path
- "person._tmp"
// We rename "citizenships" to "citizenship" because we don't like typos
"person.citizenships" %= "person.citizenship"
}
After all the operations are performed step by step, the output looks like this:
xxxxxxxxxx
{
"person" : {
"id" : 100,
"firstName" : "Gary",
"lastName" : "Young",
"meta" : {
"information1" : "ABC",
"information2" : "ABC2"
},
"maritalStatus" : "unmarried",
"visited" : [ "Romania", "Italy", "France", "Ireland", "Israel", "Japan" ],
"citizenship" : [ "Romanian", "French" ]
}
}
Operations
In the previous example, you might wonder what the operators /=, *=, %, %=, - are doing.
Those are actually shortcuts methods for the operations we are performing:
| Operator | Operation | Description |
|---|---|---|
/= |
assign |
Assigns a given constant or a value computed in a lambda expression to a certain path in the target JSON (the result). |
*= |
shift |
Shifts a portion from the source JSON based on a JSON Path expression. |
% |
copy |
Copies a path from the target JSON to another path. |
%= |
move |
Moves a path from the target JSON to another path. |
- |
delete |
Deletes a path from the target JSON. |
Additionally, the paths from the target JSON can be "decorated" with "array notation":
| Array Notation | Description |
|---|---|
path[] |
A new array will be created through the assign and shift operations. |
path[+] |
An append will be performed through the assign and shift operations. |
path[++] |
A merge will be performed through the assign and shift operations. |
If you prefer, instead of using the operators you can use their equivalent methods.
For example, assign (/=):
"person.name" /= "Andrei"
Can be written as ("assign"):
xxxxxxxxxx
"person.name" assign "Andrei"
Or (*=):
xxxxxxxxxx
"person.name" *= "$.user.full_name"
Can be written as:
xxxxxxxxxx
"person.name" shift "$.user.full_name"
Personally, I prefer the operator notation (/=, *=, etc.), but some people consider the methods (assign, shift) more readable.
For the rest of the examples, the operator notation will be used.
Assign (/=)
The Assign Operation is used to assign a value to a path in the resulting JSON (target).
The value can be a constant object, or a lambda (()-> Any).
Example:
xxxxxxxxxx
import net.andreinc.mapneat.dsl.json
const val A_SRC_1 = """
{
"id": 380557,
"first_name": "Gary",
"last_name": "Young"
}
"""
const val A_SRC_2 = """
{
"photo": "http://srcimg.com/100/150",
"married": false
}
"""
fun main() {
val transformed = json(A_SRC_1) {
// Assigning a constant
"user.user_name" /= "neo2020"
// Assigning value from a lambda expression
"user.first_name" /= { sourceCtx().read("$.first_name") }
// Assigning value from another JSON source
"more_info" /= json(A_SRC_2) {
"married" /= { sourceCtx().read("$.married") }
}
// Assigning an inner JSON with the same source as the parent
"more_info2" /= json {
"last_name" /= { sourceCtx().read("$.last_name") }
}
}
println(transformed)
}
The target JSON looks like:
xxxxxxxxxx
{
"user" : {
"user_name" : "neo2020",
"first_name" : "Gary"
},
"more_info" : {
"married" : false
},
"more_info2" : {
"last_name" : "Young"
}
}
In the lambda method we can access the following methods from the "outer" context:
sourceCtx()which represents theReadContextof the source. We can use this to read JSON Paths just like in the example above;targetCtx()which represents theReacContextof the target. This is calculated each time we call the method. So, it contains only the changes that were made up until that point. In most cases, this shouldn't be called.
In case we are using an inner JSON structure, we also have reference to the parent source and target contexts:
parent.sourceCtx()parent.targetCtx()
parent() returns a Nullable value, so it called with !! (double bang) in Kotlin.
x
... {
"something" /= "Something Value"
"person" /= json {
"innerSomething" /= { parent()!!.targetCtx().read("$.something") }
}
}
For more information about ReadContext please check the JSON-path's documentation.
The Assign operation can also be used in conjunction with left-side array notations ([], [+], [++]):
xxxxxxxxxx
fun main() {
val transformed = json("{}") {
println("Simple array creation:")
"a" /= 1
"b" /= 1
println(this)
println("Adds a new value in the array:")
"a[+]" /= 2
"b[+]" /= true
println(this)
println("Merge in an existing array:")
"b[++]" /= arrayOf("a", "b", "c")
println(this)
}
}
Output:
xxxxxxxxxx
Simple array creation:
{
"a" : 1,
"b" : 1
}
Adds a new value in the array
{
"a" : [ 1, 2 ],
"b" : [ 1, true ]
}
Merge in an existing array:
{
"a" : [ 1, 2 ],
"b" : [ 1, true, "a", "b", "c" ]
}
Shift (*=)
The Shift operation is very similar to the Assign operation, but it provides an easier way to query the source JSON using JSON-path.
Example:
xxxxxxxxxx
package net.andreinc.mapneat.examples
import net.andreinc.mapneat.dsl.json
import java.time.LocalDate
import java.time.format.DateTimeFormatter
val JSON_VAL = """
{
"id": 380557,
"first_name": "Gary",
"last_name": "Young",
"photo": "http://srcimg.com/100/150",
"married": false,
"visits" : [
{
"country" : "Romania",
"date" : "2020-10-10"
},
{
"country" : "Romania",
"date" : "2019-07-21"
},
{
"country" : "Italy",
"date" : "2019-12-21"
},
{
"country" : "France",
"date" : "2019-02-21"
}
]
}
"""
fun main() {
val transformed = json(JSON_VAL) {
"user.name.first" *= "$.first_name"
// We use an additional processor to capitalise the last Name
"user.name.last" *= {
expression = "$.last_name"
processor = { (it as String).toUpperCase() }
}
// We add the photo directly into an array
"user.photos[]" *= "$.photo"
// We don't allow duplicates
"user.visits.countries" *= {
expression = "$.visits[*].country"
processor = { (it as MutableList<String>).toSet().toMutableList() }
}
// We keep only the last visit
"user.visits.lastVisit" *= {
expression = "$.visits[*].date"
processor = {
(it as MutableList<String>)
.stream()
.map { LocalDate.parse(it, DateTimeFormatter.ISO_DATE) }
.max(LocalDate::compareTo)
.get()
.toString()
}
}
}
println(transformed)
}
Output:
xxxxxxxxxx
{
"user" : {
"name" : {
"first" : "Gary",
"last" : "YOUNG"
},
"photos" : [ "http://srcimg.com/100/150" ],
"visits" : {
"countries" : [ "Romania", "Italy", "France" ],
"lastVisit" : "2020-10-10"
}
}
}
As you can see in the above example, each expression can be accompanied by an additional processor method that allows developers to refine the results provided by the JSON path expression.
Similar to the Assign lambdas, sourceCtx(), targetCtx(), parent!!.sourceCtx(), parent!!.targetCtx() are also available to the method context and can be used.
Copy (%)
The Copy Operation moves a certain path from the target JSON to another path in the target JSON.
Example:
xxxxxxxxxx
import net.andreinc.mapneat.dsl.json
fun main() {
val transformed = json("{}") {
"some.long.path" /= mutableListOf("A, B, C")
"some.long.path" % "copy"
println(this)
}
}
Output:
xxxxxxxxxx
{
"some" : {
"long" : {
"path" : [ "A, B, C" ]
}
},
"copy" : [ "A, B, C" ]
}
Move (%=)
The Move operation moves a certain path from the target JSON to a new path in the target JSON.
Example:
xxxxxxxxxx
package net.andreinc.mapneat.examples
import net.andreinc.mapneat.dsl.json
fun main() {
json("{}") {
"array" /= intArrayOf(1,2,3)
"array" %= "a.b.c.d"
println(this)
}
}
Output:
xxxxxxxxxx
{
"a" : {
"b" : {
"c" : {
"d" : [ 1, 2, 3 ]
}
}
}
}
Delete (-)
The Delete operation deletes a certain path from the target JSON.
Example:
xxxxxxxxxx
package net.andreinc.mapneat.examples
import net.andreinc.mapneat.dsl.json
fun main() {
json("{}") {
"a.b.c" /= mutableListOf(1,2,3,4,true)
"a.b.d" /= "a"
// deletes the array from "a.b.c"
- "a.b.c"
println(this)
}
}
Output:
xxxxxxxxxx
{
"a" : {
"b" : {
"d" : "a"
}
}
}
Using MapNeat From Java
Given Kotlin's high level of interoperability with Java, MapNeat can be used in any Java application.
The DSL file should remain kotlin, but it can be called from any Java program, as simple as:
xxxxxxxxxx
@file : JvmName("Sample")
package kotlinPrograms
import net.andreinc.mapneat.dsl.json
fun personTransform(input: String) : String {
return json(input) {
"person.name" /= "Andrei"
"person.age" /= 13
}.getPrettyString()
}
In the Java file:
xxxxxxxxxx
import static kotlinPrograms.Sample.personTransform;
public class Main {
public static void main(String[] args) {
// personTransform(String) is the method from Kotlin
String person = personTransform("{}");
System.out.println(person);
}
}
PS: Configuring the Java application to be Kotlin-enabled it's quite simple, usually IntelliJ is doing this automatically without any developer intervention.
Opinions expressed by DZone contributors are their own.