DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Databases
  4. Move Documents From MongoDB to Couchbase
Content provided by Couchbase logo

Move Documents From MongoDB to Couchbase

Here's how to move your data out of MongoDB to Couchbase.

Laurent Doguin user avatar by
Laurent Doguin
·
Dec. 01, 15 · Tutorial
Like (4)
Save
Tweet
Share
4.83K Views

Maybe you remember my post about moving data from CouchDB to Couchbase. If you are not using CouchDB but MongoDB you might have felt let down. I am sorry. Let's fix this. Here's how to move your data out of MongoDB to Couchbase.

One of the great things about using RxJava is that this API is becoming more widespread. We made that choice a while ago and are happy to see more people join that wagon, like Mongo did recently.

The fact that we share a stream API is going to make things really easy. My goal is to get an Observable of Mongo Document, Map it to Couchbase Documents and then write them to Couchbase. As you can see this process is really easy to model with RxJava. In pseudo code it would look like:

getMongoCollectionStream().fromMongoDocumentToCouchbaseDocuments().writeCBdocToCouchbase()

Again this fits very well with RxJava.

MongoDB allows you to open a collection and return it as an Observable of Document. This is exactly what we need and can happen in a couple of lines:

    MongoClient client = MongoClients.create(connectionString);
    MongoDatabase db = client.getDatabase(dbName);
    Observable<org.bson.Document> mongoDocs = db.getCollection(collectionName).find();

Then we need to transform documents coming in into Couchbase documents. We can easily use the Rx map operator to do so. When writing a document on couchbase you need a key or an id. In a Mongo document it is stored in the _id field using the ObjectID abstraction. It's generated by Mongo based on several criteria. Once we get that id all we have to do is get the Doc as a JSON string and create a RawJsonDocument based on that and the id.

 db.getCollection(collectionName).find().toObservable()
                .map(new Func1<org.bson.Document, Document>() {
                    public Document call(org.bson.Document mongoDoc) {
                        mongoDoc.put(typeField, type);
                        RawJsonDocument d = RawJsonDocument.create(mongoDoc
                                .getObjectId("_id").toHexString(), mongoDoc
                                .toJson());
                        return d;
                    };
                });

After this map.operation we are left with an observable of couchbase document. The last step is to write it to Couchbase.

db.getCollection(collectionName).find().toObservable()
                .map(new Func1<org.bson.Document, Document>() {
                    public Document call(org.bson.Document mongoDoc) {
                        mongoDoc.put(typeField, type);
                        RawJsonDocument d = RawJsonDocument.create(mongoDoc
                                .getObjectId("_id").toHexString(), mongoDoc
                                .toJson());
                        return d;
                    };
                }).map(asyncBucket.upsert(doc));;

I have included all this code and made it configurable in the couchbase-java-importer. It's available on Github here.

If you don't need to do anything other than importing the JSON documents from a collection straight into Couchbase, this should be sufficient. Just download the binary and execute it making sure the yml configuration file is in your classpath and contains the right info.

This is one particular way to move data from MongoDB to Couchbase. There are others. Both databases support CSV import for instance and Mongo has a CSV exporter. So, once you have a CSV file you can use cbtransfer or couchbase-java-importer to get its content to Couchbase.

All these examples are assuming you don't want to do any transformation to the data before importing it. Transformation steps would be really easy to plug in the RxJava importer. The code is very simple so please feel free to fork it or send pull requests! Adding more importers to that project can be a great way to enter the CB community by the way.


Comments

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: