Over a million developers have joined DZone.

Mongolastic

Mongolastic is an handy tool written in Java 8 which converts a MongoDB JSON dataset to the applicative ElasticSearch JSON Bulk file, ready for the environment.

· Database Zone

Learn NoSQL for free with hands-on sample code, example queries, tutorials, and more.  Brought to you in partnership with Couchbase.


Build Status

 license-MIT-blue mongo.java.driver-3.0.4-brightgreen es.java.driver-2.1.1-brightgreen


What data you have in your MongoDB can also be included into your ElasticSearch and vice versa using this tool. You can perform the aforementioned facilities by providing a file having a yaml characteristic.



How it works


First, download the latest mongolastic.jar file.


Second, give a yaml file which must contains the following structure:


misc:
    dindex:
        name: <string>   (1)
        as: <string>     (2)
    ctype:
        name: <string>   (3)
        as: <string>     (4)
    direction: <em | me> (5)
mongo:
    host: <string>       (6)
    port: <number>       (7)
    query: "mongo-query" (8)
elastic:
    host: <string>       (9)
    port: <number>       (10)


  1. the database/index name to connect to.
  2. another database/index name in which documents will be located in the target service (Optional)
  3. the collection/type name to export.
  4. another collection/type name in which indexed/collected documents will reside in the target service (Optional)
  5. direction of the data transfer. the default direction is me (that is, mongo to elasticsearch). You can skip this option if your data move from mongo to es.
  6. the name of the host machine where the mongod is running.
  7. the port where the mongod instance is listening.
  8. data will be transferred based on a json mongodb query (Optional)
  9. the name of the host machine where the elastic node is running.
  10. the transport port where the transport module will communicate with the running elastic node. E.g. 9300 for node-to-node communication.


Here is an example of a configuration file:


misc:
    dindex:
        name: twitter
        as: kodcu
    ctype:
        name: tweets
        as: posts
mongo:
    host: localhost
    port: 27017
    query: "{ 'user.name' : 'kodcu.com'}"
elastic:
    host: localhost
    port: 9300


the config says that the transfer direction is from mongodb to elasticsearch, mongolastic first looks at the tweets collection, where the user name is kodcu.com, of the twitter database located on a mongod server running on default host interface and port number. If It finds the corresponding data, It will start copying those into an elasticsearch environment running on default host and transport number. After all, you should see a type called "posts" in an index called "kodcu" in the current elastic node. Why the index and type are different is because "dindex.as" and "ctype.as" options were set, these indicates that your data being transferred exist in posts type of the kodcu index.


After downloading the jar and providing a conf file, you can run the tool as below:


$ java -jar mongolastic.jar -f config.file


Note Every attempt of running the tool drops the mentioned db/index in the target environment.



License


Mongolastic is released under MIT.




The Getting Started with NoSQL Guide will get you hands-on with NoSQL in minutes with no coding needed. Brought to you in partnership with Couchbase.

Topics:
nosql ,tool ,json ,mongodb ,dataset ,elasticsearch ,bulk ,database

Published at DZone with permission of Hakan Özler. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}