Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

An Intro to Offline Data Storage and Sync With PouchBD and Couchbase

DZone's Guide to

An Intro to Offline Data Storage and Sync With PouchBD and Couchbase

What happens to your networked data storage pipeline when the network fails? Consider an offline-first approach, which erases the hassle of connection failures.

· Database Zone
Free Resource

Learn how to move from MongoDB to Couchbase Server for consistent high performance in distributed environments at any scale.

Offline-first is an approach to software development that is different from the traditional approach – where lack of network connection is treated as an error, thereby affecting the overall user experience. With the offline-first approach, you start with the most constrained environment and later respond to the user experience of your application by progressively enhancing it as the functionalities become available. For offline-first, it is assumed that lack of network connection creates an unsatisfactory user experience; therefore, offline-first seeks to provide the best possible user experience in all conditions.

The offline-first approach comes with some concerns such as, how to store data or manage transactions when offline and sync it with the server, how to keep the offline data secure, or how to resolve data conflicts if two users make changes to the same record while offline, etc. For this post, we’ll look at offline data storage and synchronization by building a web phonebook app to store contacts.

Building the Sample App

Our sample app will be a web app that’ll be built with Bootstrap, jQuery, PouchDB, Hoodie store-client plugin for PouchDB, and Couchbase Sync Gateway. To start, we’ll lay out the page, which will include a form to enter the contact’s name, email and phone, and also display a list of saved contacts.

Image title

Laying Out the Page

Create a folder for the application, then download bootstrap and jQuery. Unzip the files and put them in a new folder called, asset. Add a new file called index.html to your root folder and copy the snippet below to it.

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="utf-8">
    <title>My Hoodie App</title>
    <link rel="stylesheet" href="assets/bootstrap/bootstrap.min.css">
</head>

<body>

    <nav class="navbar navbar-default">
        <div class="container-fluid">
            <div class="navbar-header">
                <a class="navbar-brand" href="#"> Phonebook</a>
            </div>
        </div>
    </nav>
    <div class="container">

        <div class="row">
            <div class="col-md-10">
                <h2>Add new contact </h2>
                <hr />
                <form id="contactForm" class="form-horizontal">
                    <div class="form-group">
                        <label for="name" class="col-sm-2 control-label">Name</label>
                        <div class="col-sm-10">
                            <input type="text" class="form-control" id="name" placeholder="Name">
                        </div>
                    </div>
                    <div class="form-group">
                        <label for="mobile" class="col-sm-2 control-label">Mobile</label>
                        <div class="col-sm-10">
                            <input type="text" class="form-control" id="mobile" placeholder="Mobile">
                        </div>
                    </div>
                    <div class="form-group">
                        <label for="email" class="col-sm-2 control-label">Email</label>
                        <div class="col-sm-10">
                            <input type="email" class="form-control" id="email" placeholder="Email">
                        </div>
                    </div>
                    <div class="form-group">
                        <div class="col-sm-offset-2 col-sm-10">
                            <button type="submit" class="btn btn-default">Save Contact</button>
                        </div>
                    </div>
                </form>
                <hr />
            </div>
        </div>

        <div class="row">
            <div class="col-md-10">
                <h2>Contact List</h2>
                <hr />
                <table id="contactList" class="table table-bordered">
                    <thead>
                        <tr>
                            <th>Name</th>
                            <th>Mobile</th>
                            <th>Email</th>
                            <th></th>
                        </tr>
                    </thead>
                    <tbody>

                    </tbody>
                </table>
            </div>
        </div>


        <script src="assets/jquery-2.1.0.min.js"></script>
        <script src="assets/bootstrap/bootstrap.min.js"></script>
</body>

</html>


What we have created is a page with a form to enter and save contacts, and also display a list of saved contacts.

Image title

Offline Data Access/Storage

Data becomes one of the key concerns of an offline-first app. Any application that works offline has to deal with offline data access and storage, which will be handled by storing data on the client, and dealing with the issue of syncing data reliably. There are various client-side databases for mobile and web apps, including Couchbase Lite and Cloudant Sync for mobile and desktop clients, and IndexedDB and Web SQL for browsers. For our sample we’ll use PouchDB, a JavaScript client-side database API modeled after the CouchDB API. PouchDB abstracts away the different browser-supported database and their different programming interface. It was created to help build applications that work as well offline as they do online, by storing data locally while offline and synchronizing to the server and other connected clients when online.

I’ll also use hoodie store-client, a PouchDB plugin for data persistence and offline sync. I prefer to work with this API because of the API available for working with data and synchronization – it automatically adds _id and other timestamps when new data is added. I’ll add this plugin, as well as PouchDB, using the npm with the following command or download from this: store-client and PouchBD. Note that I’m using PouchDB 6.1.2 and hoodie-store-client 7.0.1.

npm install --save pouchdb

npm install --save @hoodie/store-client

 

Now include these files to the page.

<script src="node_modules/pouchdb/dist/pouchdb.min.js"></script>
<script src="node_modules/@hoodie/store-client/dist/hoodie-store-client.js"></script>


With that done, we add a new file called index.js in the asset folder and add a link to this file in the page. Inside this file, the first thing we do is initialize a Store object with the name of the database and URL to the server for synchronization.

$(function(){ 
    var store = new Store('example', { remote: 'http://localhost:4984/example', PouchDB: PouchDB });
});


Add the following code to save the value entered in the form and also add it to the list of contacts on the page.

$('#contactForm').submit(function(event) {
    event.preventDefault();

    var name = $('#name').val();
    var email = $('#email').val();
    var mobile = $('#mobile').val();

    // Save the contact to the database
    store.add({
        name: name,
        mobile: mobile,
        email: email
    });

    $('#contactForm')[0].reset();
    });

    //add new contact to the page
    function addNewContactToList(contact) {
    var newContact = '<tr><td>' + contact.name + '</td><td>' + contact.mobile + '</td><td>' + contact.email + '</td></tr>'
    $("#contactList tbody").append(newContact);
    }

    //when a new entry is added to the database, run the corresponding function
    store.on('add', addNewContactToList);

    function loadContacts() {
    store.findAll().then(function(contacts) {
      var tbody = '';
      $.each(contacts, function (i, contact) {
        var row = '<tr><td>' + contact.name + '</td><td>' + contact.mobile + '</td><td>' + contact.email + '</td></tr>';
        tbody += row;
      });

      $("#contactList tbody").html('').html(tbody);
    });
  }

  // when the site loads in the browser,
  // we load all previously saved contacts from hoodie
  loadContacts();


I used store.add to insert to the local database, and then listen to events using store.on, particularly the add event. This will be useful when we start synchronizing data and will display new data after a complete synchronization with remote databases. I also added a function to display local data when the page loads or refreshes. We can check to see that we can save data locally.

Image title

We now have offline data access/storage working. With what we’ve done so far, you should begin to see the shift in mindset for offline-first – storing data locally first, and afterwards pushing changes to the server when online. Our next step is to make it sync this data with a Couchbase server. To make this work, we need Couchbase Sync Gateway.

Couchbase Sync Gateway

Sync Gateway is a secure web gateway application with synchronization, REST, stream, batch, and event APIs for accessing and synchronizing data over the web. Sync Gateway enables, among other things, secure data replication between Couchbase Server and

Couchbase Lite and/or PouchDB. For a quick guide on installation see: [guide](https://blog.couchbase.com/getting-comfortable-couchbase-mobile-installing-sync-gateway/). If Sync Gateway is installed, it’ll be accessible from the http://localhost:4984 when started. As you may recall, we had a similar URL when initializing the Store object to use PouchDB, but with an extra /exampleappended to it. The extra appendix specifies the name of the database we sync with. To get our app using offline data storage and synchronization, let’s start the Sync Gateway service and update the page’s JavaScript for automatic sync. I’ll use the configuration setting for Sync Gateway.

{
    "log":["*"],
    "databases": {
        "example": {
            "server":"walrus:",
            "users": {
                "GUEST": {
                    "disabled": false,
                    "admin_channels": ["*"]
                }
            }
        }
    },
    "CORS": {
        "Origin": ["http://127.0.0.1:8801"],
        "LoginOrigin": ["http://127.0.0.1:8801"],
        "Headers": ["Content-Type"],
        "MaxAge": 17280000
    }
}


We have a basic configuration which will establish a connection to the example database and use the in-memory storage option (walrus), which in production we should change to point to a Couchbase server. We also added settings to allow cross-origin resource sharing for our app which is on a different port. We now have to start the Sync Gateway service running the following command in the terminal:

$ ./bin/sync_gateway my-config/phonebook-config.json

 

Image title

 

With Sync Gateway started, we have to update the code for continuous synchronization. Add the following code to index.js:

store.connect();


The code above uses the hoodie-store connect method to tell PouchDB to start a continuous replication with the remote database. Once we reload our database, the data we added in the previous step will automatically be synced to the server. You can see this using the Sync Gateway admin URL

Wrap Up

At this point the app works well with offline data storage and real-time cross-device synchronization. Since we have a web app and users are likely to open the page when offline or on a flaky connection, we can make the page load during these situations, and load even faster using Service Worker, Cache API, and Fetch API by caching the app’s asset and intercepting request to these and returning the cached responses. This is beyond the scope of this blog post, but I believe I’ve demonstrated how easy it is to make data available when offline, and keep all connected browser clients in sync, while making the app load data fast, and addressing the mindsetof developing offline-first.  

Here are two GIFs that show it working.

You can grab the source code here and give it a spin!

Want to deliver a whole new level of customer experience? Learn how to make your move from MongoDB to Couchbase Server.

Topics:
couchbase ,pouchdb ,offline data storage ,tutorial ,database

Published at DZone with permission of Laura Czajkowski, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}