DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Related

  • Production Database Migration or Modernization: A Comprehensive Planning Guide [Part 1]
  • Why Database Migrations Take Months and How to Speed Them Up
  • Data Migration from AWS DocumentDB to Atlas on AWS
  • Open-Source SPL That Can Execute SQL Without RDB

Trending

  • Zero-Downtime Deployments for Java Apps on Kubernetes
  • Building a DevOps-Ready Internal Developer Platform: A Hands-On Guide to Golden Paths, Self-Service, and Automated Delivery Pipelines
  • MuleSoft IDP: Enhancing Efficiency and Accuracy in Data Extraction
  • LLM-Powered Deep Parsing for Industrial Inventory Search
  1. DZone
  2. Data Engineering
  3. Data
  4. JSON data migration

JSON data migration

By 
Ladislav Gažo user avatar
Ladislav Gažo
·
Sep. 09, 11 · Interview
Likes (0)
Comment
Save
Tweet
Share
12.5K Views

Join the DZone community and get the full member experience.

Join For Free

JSON data format is simple and still powerful. Nowadays you can encounter more and more web applications communicating using JSON format then a couple of years ago. It is simple for a developer to read the format, it is effective for a web browser to parse the format and there are databases using it as its primary data format.

But what happens when the data structure changes? You need to migrate.

And that is where acris-json-migration might help you!

Example situation

Let's shed a light into it and assume we have a data like this:

{
        "firstName":"John"
        "secondName":"Doe"
        "street":"Over the rainbow"
        "streetNr":21
}

 

Such data can be represented by following Java domain object:

public class Person {
        String firstName;
        String secondName;
        String street;
        Integer streetNr;

        // ... and getters and setters...
}

 

Well, this seems like data about a person named John Doe. We stored it in a database and you can clearly see, that secondName is probably not the field name we really like to have. But a developer made a mistake and in second version of our domain model we are going to fix it:

{
        "firstName":"John"
        "surname":"Doe"
        "street":"Over the rainbow"
        "streetNr":21
}

 

Now you can see the point - thousands of data stored in the format defined by Person class in its version #1 but our program communicating in version #2 with changed secondName to surname in Person class. Clients can wonder why the don't see surnames, can't they? ;)

One thing to remember (for the following context) - the class Person changed and there is only Person class in version #2.

Simple migration script

In this situation I would like to write a script:

public class PersonV1toV2Script extends JacksonTransformationScript<ObjectNode> {

        @Override
        public void process(ObjectNode node) {
                rename(node, "secondName", "surname");
        }

}

From the above example it is clear that the script will do the job. And you can do pretty anything with the whole tree of JSON data - adding new nodes, removing existing ones, transforming here and there - all thanks to Jackson's tree model.

How can I execute it?

There is a Transformer abstract class representing a transofmer responsible for passing JSON data to a script and writing it back.

Currently there are tow kinds of transformers:

  • Jackson-based
  • JSONT-based

Jackson-based is the preferred one and is more developed then JSONT-based.

So to execute a transformation on a data set you have to specify only two lines of code:

JacksonTransformer t = new JacksonTransformer(input, output);
t.transform(PersonV1toV2Script.class.getName());

... where input and output represent directories. In the input directory all files are treated as files containing JSON data and are transformed and written to the output directory. For a detailed test you can look into TransformerTest in the project.

Conclusion

The script's helper API is evolving and provides you with nice methods like removeIfExists or addNonExistent methods. We would like to hear about your use-cases which are not handled by acris-json-migration yet so the project can generally serve the purpose of JSON data migration.

file IO Data (computing) Data migration Database

Opinions expressed by DZone contributors are their own.

Related

  • Production Database Migration or Modernization: A Comprehensive Planning Guide [Part 1]
  • Why Database Migrations Take Months and How to Speed Them Up
  • Data Migration from AWS DocumentDB to Atlas on AWS
  • Open-Source SPL That Can Execute SQL Without RDB

Partner Resources

×

Comments

The likes didn't load as expected. Please refresh the page and try again.

  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook