Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

The Modern Application Stack – Part 3: Building a REST API Using Express.js

DZone's Guide to

The Modern Application Stack – Part 3: Building a REST API Using Express.js

Learn from an expert how to get started using Express to build a REST API so that a remote client can work with MongoDB.

· Web Dev Zone
Free Resource

Learn how to build modern digital experience apps with Crafter CMS. Download this eBook now. Brought to you in partnership with Crafter Software

Introduction

This is the third in a series of blog posts examining the technologies that are driving the development of modern web and mobile applications.

Part 1: Introducing The MEAN Stack (and the young MERN upstart) introduced the technologies making up the MEAN (MongoDB, Express, Angular, Node.js) and MERN (MongoDB, Express, React, Node.js) Stacks, why you might want to use them, and how to combine them to build your web application (or your native mobile or desktop app).

The remainder of the series is focused on working through the end-to-end steps of building a real (albeit simple) application: MongoPop. Part 2: Using MongoDB With Node.js created an environment where we could work with a MongoDB database from Node.js; it also created a simplified interface to the MongoDB Node.js Driver.

This post builds on from those first posts by using Express to build a REST API so that a remote client can work with MongoDB. You will be missing out on a lot in the way of context if you have skipped those posts – it's recommended to follow through those first.

The REST API

A Representational State Transfer (REST) interface provides a set of operations that can be invoked by a remote client (which could be another service) over a network, using the HTTP protocol. The client will typically provide parameters such as a string for which to search, or the name of a resource to be deleted.

Many services provide a REST API so that clients (their own and those of 3rd parties) and other services can use the service in a well defined, loosely coupled manner. As an example, the Google Places API can be used to search for information about a specific location:

curl "https://maps.googleapis.com/maps/api/place/details/json?placeid=ChIJKxSwWSZgAUgR0tWM0zAkZBc&key=AIzaSyC53qhhXAmPOsxc34WManoorp7SVN_Qezo" {
        "html_attributions": [],
        "result": {
            "address_components": [{
                    "long_name": "19",
                    "short_name": "19",
                    "types": ["street_number"]
                },
                {
                    "long_name": "Route des Allassins",
                    "short_name": "Route des Allassins",
                    "types": ["route"]
                },
                {
                    "long_name": "Le Grand-Village-Plage",
                    "short_name": "Le Grand-Village-Plage",
                    "types": ["locality", "political"]
                },
                {
                    "long_name": "Charente-Maritime",
                    "short_name": "Charente-Maritime",
                    "types": ["administrative_area_level_2", "political"]
                },
                {
                    "long_name": "Nouvelle-Aquitaine",
                    "short_name": "Nouvelle-Aquitaine",
                    "types": ["administrative_area_level_1", "political"]
                },
                {
                    "long_name": "France",
                    "short_name": "FR",
                    "types": ["country", "political"]
                },
                {
                    "long_name": "17370",
                    "short_name": "17370",
                    "types": ["postal_code"]
                }
            ],

            ...
            "utc_offset": 60,
            "vicinity": "19 Route des Allassins, Le Grand-Village-Plage",
            "website": "http://www.oleronvilla.com/"
        },
        "status": "OK"



Breaking down the URI used in that curl request:

  • No method is specified and so the curl command defaults to an HTTP GET.
  • maps.googleapis.com is the address of the Google APIs service.
  • /maps/api/place/details/json is the route path to the specific operation that's being requested.
  • placeid=ChIJKxSwWSZgAUgR0tWM0zAkZBc is a parameter (passed to the function bound to this route path), identifying which place we want to read the data for.
  • key=AIzaSyC53qhhXAmPOsxc34WManoorp7SVN_Qezo is a parameter indicating the Google API key, verifying that it's a registered application making the request (Google will also cap, or bill for, the number of requests made using this key)

There's a convention as to which HTTP method should be used for which types of operation:

  • GET: Fetches data
  • POST: Adds new data
  • PUT: Updates data
  • DELETE: Removes data

Mongopop's REST API breaks this convention and uses POST for some read requests (as it's simpler passing arguments than with GET).

These are the REST operations that will be implemented in Express for Mongopop:


Express routes implemented for the Mongopop REST API
Route Path HTTP Method Parameters Response Purpose
/pop/
GET
{
"AppName": "MongoPop",
"Version": 1.0
}

Returns the version of the API.
/pop/ip
GET
{"ip": string}
Fetches the IP Address of the server running the Mongopop backend.
/pop/config
GET
{
mongodb: {
    defaultDatabase: string,
    defaultCollection: string,
    defaultUri: string
},
mockarooUrl: string
}

Fetches client-side defaults from the back-end config file.
/pop/addDocs
POST
{
MongoDBURI: string;
collectionName: string;
dataSource: string;
numberDocs: number;
unique: boolean;
}

{
success: boolean;
count: number;
error: string;
}

Add numberDocs batches of documents, using documents fetched from dataSource
/pop/sampleDocs
POST
{
MongoDBURI: string;
collectionName: string;
numberDocs: number;
}

{
success: boolean;documents: string;
error: string;
}

Read a sample of the documents from a collection.
/pop/countDocs
POST
{
MongoDBURI: string; 
collectionName: string;
}

{
success: boolean;count: number;
error: string;
}

Counts the number of documents in the collection.
/pop/updateDocs
POST
{
MongoDBURI: string;
collectionName: string;
matchPattern: Object;
dataChange: Object;
threads: number;
}

{
success: boolean;
count: number;
error: string;
}

Apply an update to all documents in a collection which match a given pattern


Express

Express is the web application framework that runs your back-end application (JavaScript) code. Express runs as a module within the Node.js environment.

Express can handle the routing of requests to the right functions within your application (or to different apps running in the same environment).

You can run the app's full business logic within Express and even use an optional view engine to generate the final HTML to be rendered by the user's browser. At the other extreme, Express can be used to simply provide a REST API – giving the front-end app access to the resources it needs e.g., the database.

The Mongopop application uses Express to perform two functions:

  • Send the front-end application code to the remote client when the user browses to our app.
  • Provide a REST API that the front-end can access using HTTP network calls, in order to access the database.

Downloading, Running, and Using the Application

The application's Express code is included as part of the Mongopop package installed in Part 2: Using MongoDB With Node.js.

What Are a All of These Files?

A reminder of the files described in Part 2:

  • package.json: Instructs the Node.js package manager (npm) on what it needs to do; including which dependency packages should be installed.
  • node_modues: Directory where npm will install packages.
  • node_modues/mongodb: The MongoDB driver for Node.js
  • node_modues/mongodb-core: Low-level MongoDB driver library; available for framework developers (application developers should avoid using it directly).
  • javascripts/db.js: A JavaScript module we've created for use by our Node.js apps (in this series, it will be Express) to access MongoDB; this module, in turn, uses the MongoDB Node.js driver.

Other files and directories that are relevant to our Express application:

  • config.js: Contains the application–specific configuration options.
  • bin/www: The script that starts an Express application; this is invoked by the npm start script within the package.json file. Starts the HTTP server, pointing it to the appmodule in app.js
  • app.js: Defines the main application module (app). Configures:
    • That the application will be run by Express
    • Which routes there will be, and where they are located in the file system (routesdirectory)
    • What view engine to use (Jade in this case)
    • Where to find the /views/ to be used by the view engine (views directory)
    • What middleware to use (e.g. to parse the JSON received in requests)
    • Where the static files (which can be read by the remote client) are located (public directory)
    • Error handler for queries sent to an undefined route
  • views: Directory containing the templates that will be used by the Jade view engine to create the HTML for any pages generated by the Express application (for this application, this is just the error page that's used in cases such as mistyped routes ("404 Page not found")
  • routes: Directory containing one JavaScript file for each Express route
    • routes/pop.js: Contains the Express application for the /pop route; this is the implementation of the Mongopop REST API. This defines methods for all of the supported route paths.
  • public: Contains all of the static files that must be accessible via a remote client (e.g., our Angular to React apps). This is not used for the REST API and so can be ignored until Parts 4 and 5.

The rest of the files and directories can be ignored for now – they will be covered in later posts in this series.

Architecture

REST AIP implemented in Express.js

The new REST API (implemented in routes/pop.js) uses the javascripts/db.js database layer implemented in Part 2 to access the MongoDB database via the MongoDB Node.js Driver. As we don't yet have either the Angular or React clients, we will user the curlcommand-line tool to manually test the REST API.

Code Highlights

Config.js

var config = {
    expressPort: 3000,
    client: {
        mongodb: {
            defaultDatabase: "mongopop",
            defaultCollection: "simples",
            defaultUri: "mongodb://localhost:27017"
        },
        mockarooUrl: "http://www.mockaroo.com/536ecbc0/download?count=1000&key=48da1ee0"
    }
};
module.exports = config;


The config module can be imported by other parts of the application so that your preferences can be taken into account.

expressPort is used by bin/www to decide what port the web server should listen on; change this if that port is already in use.

client contains defaults to be used by the client (Angular or React). It's important to create your own schema at Mockaroo.com and replace client.mockarooUrl with your custom URL (the one included here will fail if used too often).

Bin/Www

This is mostly boiler-plate code to start Express with your application. This code ensures that it is our application, app.js, that is run by the Express server:

// Location of the main express application module (`app.js`)
var app = require('../app');
var server = http.createServer(app);


This code uses the expressPort from config.js as the port for the server to listen on; it will be overruled if the user sets the PORT environment variable:

var config = require('../config.js');
var port = normalizePort(process.env.PORT || config.expressPort);
app.set('port', port);


App.js

This file defines the app module; much of the contents are boilerplate (and covered by comments in the code) but we look here at a few of the lines that are particular to this application.

Make this an Express application:

var express = require('express');
var app = express();


Define where the views (templates used by the Jade view engine to generate the HTML code) and static files (files that must be accessible by a remote client) are located:

app.set('views', path.join(__dirname, 'views'));
app.use(express.static(path.join(__dirname, 'public')));


Create the /pop route and associate it with the file containing its code (routes/pop.js):

var pop = require('./routes/pop');
app.use('/pop', pop);


Routes/Pop.js

This file implements each of the operations provided by the Mongopop REST API. Because of the /pop route defined in app.js Express will direct any URL of the form http://<mongopop-server>:3000/pop/X here. Within this file a route handler is created in order direct incoming requests to http://<mongopop-server>:3000/pop/X to the appropriate function:

var router = express.Router();


As the /pop route is only intended for the REST API, end users shouldn't be browsing here but we create a top-level handler for the GET method in case they do:

router.get('/', function(req, res, next) {
    // This isn't part of API and is just used from a browser or curl to test that
    // "/pop" is being routed correctly.

    var testObject = {
        "AppName": "MongoPop",
        "Version": 1.0
    }
    res.json(testObject);
});



Results of browsing to the top-route for the Mongopop MongoDB application

This is the first time that we see how to send a response to a request; res.json(testObject);converts testObject into a JSON document and sends it back to the requesting client as part of the response message.

The simplest useful route path is the GET method on /pop/ip which sends a response containing the IP address of the back-end server. This is useful to the Mongopop client as it means the user can see it and add it to the MongoDB Atlas whitelist. The code to determine and store publicIP is left out here but can be found in the full source file for pop.js.

router.get('/ip', function(req, res, next) {
    // Sends a response with the IP address of the server running this service.

    res.json({
        "ip": publicIP
    });
});



Fetching the IP address for the MongoDB Mongopop back-end using REST API

We've seen that it's possible to test GET methods from a browser's address bar; that isn't possible for POST methods and so it's useful to be able to test using the curl command-line command:

curl localhost: 3000 / pop / ip {
    "ip": "92.237.136.220"
}


The GET method for /pop/config is just as simple – responding with the client-specific configuration data:

var config = require('../config.js');
router.get('/config', function(req, res, next) {
    res.json(config.client);
})



curl http: //localhost:3000/pop/config
    {
        "mongodb": {
            "defaultDatabase": "mongopop",
            "defaultCollection": "simples",
            "defaultUri": "mongodb://localhost:27017"
        },
        "mockarooUrl": "http://www.mockaroo.com/536ecbc0/download?count=1000&key=48da1ee0"
    }



The results of the request are still very simple, but the output from curl is already starting to get messy; piping it through python -mjson.tool makes it easier to read:

curl http: //localhost:3000/pop/config | python -mjson.tool
    %
    Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 195 100 195 0 0 28727 0--: --: -- --: --: -- --: --: --32500 {
    "mockarooUrl": "http://www.mockaroo.com/536ecbc0/download?count=1000&key=48da1ee0",
    "mongodb": {
        "defaultCollection": "simples",
        "defaultDatabase": "mongopop",
        "defaultUri": "mongodb://localhost:27017"
    }
}



The simplest operation that actually accesses the database is the POST method for the /pop/countDocs route path:

var DB = require('../javascripts/db');
router.post('/countDocs', function(req, res, next) {
    /* Request from client to count the number of documents in a
    collection; the request should be of the form:
    {
    MongoDBURI: string; // Connect string for MongoDB instance
    collectionName: string;
    }
    The response will contain:
    {
    success: boolean;
    count: number; // The number of documents in the collection
    error: string;
    }
    */

    var requestBody = req.body;
    var database = new DB;
    database.connect(requestBody.MongoDBURI)
        .then(
            function() {
                return database.countDocuments(requestBody.collectionName)
            })
        .then(
            function(count) {
                return {
                    "success": true,
                    "count": count,
                    "error": ""
                };
            },
            function(err) {
                console.log("Failed to count the documents: " + err);
                return {
                    "success": false,
                    "count": 0,
                    "error": "Failed to count the documents: " + err
                };
            })
        .then(
            function(resultObject) {
                database.close();
                res.json(resultObject);
            })
})



database
is an instance of the object prototype defined in javascripts/db (see The Modern Application Stack – Part 2: Using MongoDB With Node.js) and so all this method needs to do is use that object to:view rawcountRoute.js hosted with ❤ by GitHub.

  • Connect to the database (using the address of the MongoDB server provided in the request body). The results from the promise returned by database.connect is passed to the function(s) in the first .then clause. Refer back to Part 2: Using MongoDB With Node.js if you need a recap on using promises.
  • The function in the .then clause handles the case where the database.connect promise is resolved (success). This function requests a count of the documents – the database connection information is now stored within the database object and so only the collection name needs to be passed. The promise returned by database.countDocuments is passed to the next .then clause. Note that there is no second (error) function provided, and so if the promise from database.connect is rejected, then that failure passes through to the next .then clause in the chain.
  • The second .then clause has two functions:
    • The first is invoked if and when the promise is resolved (success) and it returns a success response (which is automatically converted into a resolved promise that it passed to the final .then clause in the chain). count is the value returned when the promise from the call to database.countDocuments was resolved.
    • The second function handles the failure case (could be from either database.connect or database.countDocuments) by returning an error response object (which is converted to a resolved promise).
  • The final .then clause closes the database connection and then sends the HTTP response back to the client; the response is built by converting the resultObject(which could represent success or failure) to a JSON string.

Once more, curl can be used from the command-line to test this operation; as this is a POST request, the --data option is used to pass the JSON document to be included in the request:

curl - g - X POST--data '{
"MongoDBURI": "mongodb://localhost:27017/mongopop?authSource=admin&socketTimeoutMS=30000&maxPoolSize=20",
"collectionName": "simples"
}
' -i "localhost:3000/pop/countDocs" --header "Content-Type:application/json"
HTTP / 1.1 200 OK
X - Powered - By: Express
Content - Type: application / json;
charset = utf - 8
Content - Length: 43
ETag: W / "2b-+oXLqLmOvVK01tCeNbBDyg"
Date: Mon, 30 Jan 2017 08: 35: 16 GMT
Connection: keep - alive {
    "success": true,
    "count": 2828000,
    "error": ""
}



curl
can also be used to test the error paths. Cause the database connection to fail by using the wrong port number in the MongoDB URI:

curl - g - X POST--data '{
"MongoDBURI": "mongodb://localhost:27018/mongopop?authSource=admin&socketTimeoutMS=30000&maxPoolSize=20",
"collectionName": "simples"
}
' -i "localhost:3000/pop/countDocs" --header "Content-Type:application/json"
HTTP / 1.1 200 OK
X - Powered - By: Express
Content - Type: application / json;
charset = utf - 8
Content - Length: 131
ETag: W / "83-h7jQCGc5BTG8dXp61IP4yA"
Date: Mon, 30 Jan 2017 08: 37: 44 GMT
Connection: keep - alive {
    "success": false,
    "count": 0,
    "error": "Failed to count the documents: failed to connect to server [localhost:27018] on first connect"
}



Cause the count to fail by using the name of a non-existent collection:

curl - g - X POST--data '{
"MongoDBURI": "mongodb://localhost:27017/mongopop?authSource=admin&socketTimeoutMS=30000&maxPoolSize=20",
"collectionName": "missing"
}
' -i "localhost:3000/pop/countDocs" --header "Content-Type:application/json"
HTTP / 1.1 200 OK
X - Powered - By: Express
Content - Type: application / json;
charset = utf - 8
Content - Length: 129
ETag: W / "81-ct+QBrPOtMmSpo0mjFJ3BA"
Date: Mon, 30 Jan 2017 08: 40: 49 GMT
Connection: keep - alive {
    "success": false,
    "count": 0,
    "error": "Failed to count the documents: Collection missing does not exist. Currently in strict mode."
}



The POST method for the pop/sampleDocs route path works in a very similar way:

router.post('/sampleDocs', function(req, res, next) {
    /* Request from client to read a sample of the documents from a collection; the request
    should be of the form:
    {
    MongoDBURI: string; // Connect string for MongoDB instance
    collectionName: string;
    numberDocs: number; // How many documents should be in the result set
    }
    The response will contain:
    {
    success: boolean;
    documents: string; // Sample of documents from collection
    error: string;
    }
    */

    var requestBody = req.body;
    var database = new DB;
    database.connect(requestBody.MongoDBURI)
        .then(
            function() {
                // Returning will pass the promise returned by sampleCollection to
                // the next.then in the chain
                return database.sampleCollection(
                    requestBody.collectionName,
                    requestBody.numberDocs)
            }) // No function is provided to handle the connection failing and so that
        // error will flow through to the next.then
        .then(
            function(docs) {
                return {
                    "success": true,
                    "documents": docs,
                    "error": ""
                };
            },
            function(error) {
                console.log('Failed to retrieve sample data: ' + error);
                return {
                    "success": false,
                    "documents": null,
                    "error": "Failed to retrieve sample data: " + error
                };
            })
        .then(
            function(resultObject) {
                database.close();
                res.json(resultObject);
            }
        )
})



Testing this new operation:

curl - g - X POST--data '{
"MongoDBURI": "mongodb://localhost:27017/mongopop?authSource=admin&socketTimeoutMS=30000&maxPoolSize=20",
"collectionName": "simples",
"numberDocs": "2"
}
' "localhost:3000/pop/sampleDocs" --header "Content-Type:application/json" | python -mjson.tool %
Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 669 100 509 100 160 188 59 0: 00: 02 0: 00: 02--: --: --188 {
    "documents": [{
            "_id": "5859696971c490c4cacab431",
            "email": "dwilliams8h@arstechnica.com",
            "first_name": "Donald",
            "gender": "Male",
            "id": 306,
            "ip_address": "223.153.70.35",
            "last_name": "Williams",
            "mongopopComment": "MongoPop has been here",
            "mongopopCounter": 50
        },
        {
            "_id": "586d36b95c6401b247e4e83d",
            "email": "bgrantat@deliciousdays.com",
            "first_name": "Brian",
            "gender": "Male",
            "id": 390,
            "ip_address": "152.116.123.169",
            "last_name": "Grant",
            "mongopopComment": "MongoPop has been here",
            "mongopopCounter": 8
        }
    ],
    "error": "",
    "success": true
}



The POST method for pop/updateDocs is a little more complex as the caller can request multiple update operations be performed. The simplest way to process multiple asynchronous, promise-returning function calls in parallel is to build an array of the tasks and pass it to the Promise.all method which returns a promise that either resolves after all of the tasks have succeeded or is rejected if any of the tasks fail:

router.post('/updateDocs', function(req, res, next) {
    /* Request from client to apply an update to all documents in a collection
    which match a given pattern; the request should be of the form:
    {
    MongoDBURI: string; // Connect string for MongoDB instance
    collectionName: string;
    matchPattern: Object; // Filter to determine which documents should
    // be updated (e.g. '{"gender": "Male"}'')
    dataChange: Object; // Change to be applied to each matching change
    // (e.g. '{"$set": {"mmyComment": "This is a
    // man"}, "$inc": {"myCounter": 1}}')
    threads: number; // How many times to repeat (in parallel) the operation
    }
    The response will contain:
    {
    success: boolean;
    count: number; // The number of documents updated (should be the
    // the number of documents matching the pattern
    // multiplied by the number of threads)
    error: string;
    }
    */

    var requestBody = req.body;
    var database = new DB;
    database.connect(requestBody.MongoDBURI)
        .then(
            function() {
                // Build up a list of the operations to be performed

                var taskList = []
                for (var i = 0; i < requestBody.threads; i++) {
                    taskList.push(database.updateCollection(
                        requestBody.collectionName,
                        requestBody.matchPattern,
                        requestBody.dataChange));
                }
                // Asynchronously run all of the operations

                var allPromise = Promise.all(taskList);
                allPromise
                    .then(
                        function(values) {
                            documentsUpdated = values.reduce(add, 0);
                            return {
                                "success": true,
                                "count": documentsUpdated,
                                "error": {}
                            };
                        },
                        function(error) {
                            console.log("Error updating documents" + error);
                            return {
                                "success": false,
                                "count": 0,
                                "error": "Error updating documents: " + error
                            };
                        }
                    )
                    .then(
                        function(resultObject) {
                            database.close();
                            res.json(resultObject);
                        }
                    )
            },
            function(error) {
                console.log("Failed to connect to the database: " + error);
                resultObject = {
                    "success": false,
                    "count": 0,
                    "error": "Failed to connect to the database: " + error
                };
                res.json(resultObject);
            }
        );
})



Testing with curl:

curl - g - X POST--data '{
"MongoDBURI": "mongodb://localhost:27017/mongopop?authSource=admin&socketTimeoutMS=30000&maxPoolSize=20",
"collectionName": "simples",
"matchPattern": {
    "gender": "Male"
},
"dataChange": {
    "$set": {
        "mongopopComment": "This is a man"
    },
    "$inc": {
        "mongopopCounter": 1
    }
},
"threads": "5"
}
' "localhost:3000/pop/updateDocs" --header "Content-Type:application/json" {
    "success": true,
    "count": 6951670,
    "error": {}
}



The final method uses example data from a service such as Mockaroo to populate a MongoDB collection. A helper function is created that makes the call to that external service:

var request = require("request");
function requestJSON(requestURL) {
    // Retrieve an array of example JSON documents from an external source
    // e.g. mockaroo.com. Returns a promise that either resolves to the results
    // from the JSON service or rejects with the received error.

    return new Promise(function(resolve, reject) {
        // Mockaroo can have problems with https - this is random sample data so by
        // definition shouldn't need to be private
        finalDocURL = requestURL.replace('https', 'http');
        request({
            url: finalDocURL,
            json: true
        }, function(error, response, body) {
            if (error || response.statusCode != 200) {
                console.log("Failed to fetch documents: " + error.message);
                reject(error.message);
            } else {
                resolve(body);
            }
        })
    })
}



That function is then used in the POST method for /pop/addDocs:

router.post('/addDocs', function(req, res, next) {
    /* Request from client to add a number of documents to a collection; the request
    should be of the form:
    {
    MongoDBURI: string; // Connect string for MongoDB instance
    collectionName: string;
    dataSource: string; // e.g. a Mockaroo.com URL to produce example docs
    numberDocs: number; // How many (in thousands) documents sould be added
    unique: boolean; // Whether each batch of 1,000 documents should be distinct
    // from the others (much slower if set to true)
    }
    The response will contain:
    {
    success: boolean;
    count: number; // How many documents were added (in thousands)
    error: string;
    }
    */

    var requestBody = req.body;
    var uniqueDocs = req.body.unique;
    var batchesCompleted = 0;
    var database = new DB;
    var docURL = requestBody.dataSource;
    database.connect(requestBody.MongoDBURI)
        .then(
            function() {
                if (uniqueDocs) {
                    // Need to fetch another batch of unique documents for each batch
                    // of 1,000 docs

                    for (i = 0; i < requestBody.numberDocs; i++) {
                        // Fetch the example documents (based on the caller's source URI)

                        requestJSON(docURL)
                            .then(
                                function(docs) {
                                    // The first function provided as a parameter to "then"
                                    // is called if the promise is resolved successfully. The
                                    // "requestJSON" method returns the retrieved documents
                                    // which the code in this function sees as the "docs"
                                    // parameter. Write these docs to the database:

                                    database.popCollection(requestBody.collectionName, docs)
                                        .then(
                                            function(results) {
                                                return batchesCompleted++;
                                            },
                                            function(error) {
                                                // The second function provided as a parameter to "then"
                                                // is called if the promise is rejected. "err" is set to
                                                // to the error passed by popCollection

                                                database.close();
                                                resultObject = {
                                                    "success": false,
                                                    "count": batchesCompleted,
                                                    "error": "Failed to write mock data: " + error
                                                };
                                                res.json(resultObject);
                                                throw (false);
                                            }
                                        )
                                        .then(
                                            function() {
                                                // If all off the batches have been (successfully) added
                                                // then build and send the response.

                                                if (batchesCompleted == requestBody.numberDocs) {
                                                    database.close();
                                                    console.log('Wrote all Mock data');
                                                    resultObject = {
                                                        "success": true,
                                                        "count": batchesCompleted,
                                                        "error": ""
                                                    };
                                                    res.json(resultObject);
                                                }
                                            },
                                            function(error) {}
                                        )
                                },
                                function(error) {
                                    database.close();
                                    resultObject = {
                                        "success": false,
                                        "count": batchesCompleted,
                                        "error": "Failed to fetch mock data: " + error
                                    };
                                    res.json(resultObject);
                                }
                            )
                    }
                } else {
                    // Fetch one set of sample data and then use for repeated batches of writes

                    requestJSON(docURL)
                        .then(
                            function(docs) {
                                // Build an array of popCollection calls (not being executed at this point)

                                var taskList = [];
                                for (i = 0; i < requestBody.numberDocs; i++) {
                                    taskList.push(database.popCollection(requestBody.collectionName, docs))
                                }
                                // Promise.all executes all of the tasks in the provided array asynchronously (i.e.
                                // they can run in parallel).

                                var allPromise = Promise.all(taskList);
                                allPromise
                                    .then(
                                        function(result) {
                                            database.close();
                                            resultObject = {
                                                "success": true,
                                                "count": requestBody.numberDocs,
                                                "error": ""
                                            };
                                            res.json(resultObject);
                                        },
                                        function(error) {
                                            database.close();
                                            resultObject = {
                                                "success": false,
                                                "count": 0, // If some writes succeeded then the real count may be > 0
                                                "error": "Failed to write data: " + error
                                            };
                                            res.json(resultObject);
                                        }
                                    )
                            },
                            function(error) {
                                database.close();
                                resultObject = {
                                    "success": false,
                                    "count": 0,
                                    "error": "Failed to fetch mock data: " + error
                                };
                                res.json(resultObject);
                            }
                        )
                }
            },
            function(error) {
                resultObject = {
                    "success": false,
                    "count": 0,
                    "error": "Failed to connect to database: " + error
                };
                res.json(resultObject);
            }
        )
})



This method is longer than the previous ones – mostly because there are two paths:view 

  • In the first path, the client has requested that a fresh set of 1,000 example documents be used for each pass at adding a batch of documents. This path is much slower and will eat through your Mockaroo quota much faster.
  • In the second path, just one batch of 1,000 example documents is fetched from Mockaroo and then those same documents are repeatedly added. This path is faster but it results in duplicate documents (apart from a MongoDB-created _id field). This path cannot be used if the _id is part of the example documents generated by Mockaroo.

So far, we've used the Chrome browser and the curl command-line tool to test the REST API. A third approach is to use the Postman Chrome app:

Testing MongoDB Mongopop REST API with Postman Chrome app

Debugging Tips

One way to debug a Node.js application is to liberally sprinkle console.log messages throughout your code but that takes extra effort and bloats your code base. Every time you want to understand something new, you must add extra logging to your code and then restart your application.

Developers working with browser-side JavaScript benefit from the excellent tools built into modern browsers – for example, Google's Chrome Developer Tools which let you:

  • Browse code (e.g. HTML and JavaScript)
  • Add breakpoints
  • View & alter contents of variables
  • View and modify css styles
  • View network messages
  • Access the console (view output and issue commands)
  • Check security details
  • Audit memory use, CPU, etc.

You open the Chrome DevTools within the Chrome browser using "View/Developer/Developer Tools".

Fortunately, you can use the node-debug command of node-inspector to get a very similar experience for Node.js back-end applications. To install node-inspector:

npm install - g node - inspector



node-inspector
can be used to debug the Mongopop Express application by starting it with node-debug via the express-debug script in package.json:

"scripts": {
    "start": "cd public && npm run tsc && cd.. && node./bin/www",
    "debug": "cd public && npm run tsc && cd.. && node-debug./bin/www",
    "tsc": "cd public && npm run tsc",
    "tsc:w": "cd public && npm run tsc:w",
    "express": "node./bin/www",
    "express-debug": "node-debug./bin/www",
    "postinstall": "cd public && npm install"
},



To run the Mongopop REST API with node-debug, kill the Express app if it's already running and then execute:

npm run express - debug


Note that this automatically adds a breakpoint at the start of the app and so you will need to skip over that to run the application.view rawexpress-debug.bash hosted with ❤ by GitHub.

Using Chrome Developer Tools with MongoDB Express Node.js application

Depending on your version of Node.js, you may see this error:

node - debug bin / www
Node Inspector v0 .12 .8
Visit http: //127.0.0.1:8080/?port=5858 to start debugging.
    Debugging `bin/www`
Debugger listening on 127.0 .0 .1: 5858 /
    usr / local / lib / node_modules / node - inspector / lib / InjectorClient.js: 111
cb(error, NM[0].ref); ^

TypeError: Cannot read property 'ref' of undefined
at InjectorClient. < anonymous > (/usr/local / lib / node_modules / node - inspector / lib / InjectorClient.js: 111: 22)
at / usr / local / lib / node_modules / node - inspector / lib / DebuggerClient.js: 121: 7
at Object.value(/usr/local / lib / node_modules / node - inspector / lib / callback.js: 23: 20)
at Debugger._processResponse(/usr/local / lib / node_modules / node - inspector / lib / debugger.js: 95: 21)
at Protocol.execute(_debugger.js: 121: 14)
at emitOne(events.js: 96: 13)
at Socket.emit(events.js: 188: 7)
at readableAddChunk(_stream_readable.js: 176: 18)
at Socket.Readable.push(_stream_readable.js: 134: 10)
at TCP.onread(net.js: 551: 20)



If you do, apply this patch to /usr/local/lib/node_modules/node-inspector/lib/InjectorClient.js.

Summary & What's Next in the Series

Part 1: Introducing The MEAN Stack provided an overview of the technologies that are used by modern application developers – in particular, the MERN and MEAN stacks. Part 2: Using MongoDB With Node.js set up Node.js and the MongoDB Driver and then used them to build a new Node.js module to provide a simplified interface to the database.

This post built upon the first two of the series by stepping through how to implement a REST API using Express. We also looked at three different ways to test this API and how to debug Node.js applications. This REST API is required by both the Angular (Part 4) and React (Part 5) web app clients, as well as by the alternative UIs explored in Part 6.

The next part of this series implements the Angular client that makes use of the REST API – at the end of that post, you will understand the end-to-end steps required to implement an application using the MEAN stack.

Continue to follow this blog series to step through building the remaining stages of the MongoPop application:

Crafter is a modern CMS platform for building modern websites and content-rich digital experiences. Download this eBook now. Brought to you in partnership with Crafter Software.

Topics:
express.js ,mongodb ,node js tutorial ,rest api

Published at DZone with permission of Andrew Morgan, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}