Over a million developers have joined DZone.

MongoDB Incremental Migration Scripts

· Java Zone

Microservices! They are everywhere, or at least, the term is. When should you use a microservice architecture? What factors should be considered when making that decision? Do the benefits outweigh the costs? Why is everyone so excited about them, anyway?  Brought to you in partnership with IBM.

An incremental software development process requires an incremental database migration strategy.

I remember working on an enterprise application where the hibernate.hbm2ddl.autowas the default data migration tool.

Updating the production environment required intensive preparation and the migration scripts were only created on-the-spot. An unforeseen error could have led production data corruption.

Incremental updates to the rescue

The incremental database update is a technical feature that needs to be addressed in the very first application development iterations.

We used to develop our own custom data migration implementations and spending time on writing/supporting frameworks is always working against your current project budget.

A project must be packed with both application code and all associated database schema/data updates scripts. Using incremental migration scripts allows us to automate the deployment process and to take advantage of continuous delivery.

Nowadays you don’t have to implement data migration tools, Flyway does a better job than all our previous custom frameworks. All database schema and data changes have to be recorded in incremental update scripts following a well-defined naming convention.

RDBMS migration plan addresses both schema and data changes. It’s always good to separate schema and data changes. Integration tests might only use the schema migration scripts in conjunction with test-time related data .

Flyway supports all major relation database systems but for NoSQL (e.g. MongoDB) your need to to look somewhere else.


Mongeez is an open-source project aiming to automate MongoDB data migration. MongoDB is schema-less, so migration scripts are only targeting data updates only.

Integrating mongeez

First you have to define a mongeez configuration file:


<file path="v1_1__initial_data.js"/>
<file path="v1_2__update_products.js"/>

Then you add the actual migrate scripts:


//mongeez formatted javascript
//changeset system:v1_1
"_id": 1,
"name" : "TV",
"price" : 199.99,
"currency" : 'USD',
"quantity" : 5,
"version" : 1
"_id": 2,
"name" : "Radio",
"price" : 29.99,
"currency" : 'USD',
"quantity" : 3,
"version" : 1


//mongeez formatted javascript
//changeset system:v1_2
name : 'TV'
$inc : {
price : -10,
version : 1
multi: true

And you need to add the MongeezRunner too:

<bean id="mongeez" class="org.mongeez.MongeezRunner" depends-on="mongo">
<property name="mongo" ref="mongo"/>
<property name="executeEnabled" value="true"/>
<property name="dbName" value="${mongo.dbname}"/>
<property name="file" value="classpath:mongodb/migration/mongeez.xml"/>

Running mongeez

When the application first starts, the incremental scripts will be analyzed and only run if necessary:

INFO  [main]: o.m.r.FilesetXMLReader - Num of changefiles 2
INFO  [main]: o.m.ChangeSetExecutor - ChangeSet v1_1 has been executed
INFO  [main]: o.m.ChangeSetExecutor - ChangeSet v1_2 has been executed

Mongeez uses a separate MongoDB collection to record previously run scripts:

"_id" : ObjectId("543b69eeaac7e436b2ce142d"),
"type" : "configuration",
"supportResourcePath" : true
"_id" : ObjectId("543b69efaac7e436b2ce142e"),
"type" : "changeSetExecution",
"file" : "v1_1__initial_data.js",
"changeId" : "v1_1",
"author" : "system",
"resourcePath" : "mongodb/migration/v1_1__initial_data.js",
"date" : "2014-10-13T08:58:07+03:00"
"_id" : ObjectId("543b69efaac7e436b2ce142f"),
"type" : "changeSetExecution",
"file" : "v1_2__update_products.js",
"changeId" : "v1_2",
"author" : "system",
"resourcePath" : "mongodb/migration/v1_2__update_products.js",
"date" : "2014-10-13T08:58:07+03:00"


To automate the deployment process you need to create self-sufficient packs, containing both bytecode and all associated configuration (xml files, resource bundles and data migration scripts). Before starting writing your own custom framework, you should always investigate for available open-source alternatives.

Code available on GitHub.

Discover how the Watson team is further developing SDKs in Java, Node.js, Python, iOS, and Android to access these services and make programming easy. Brought to you in partnership with IBM.


Published at DZone with permission of Vlad Mihalcea, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}