Over a million developers have joined DZone.

New to Couchbase 4.5: AtPlus

Scan consistency options in a distributed database let you choose whether or not to wait for index updates to settle before returning results.

· Database Zone

Sign up for the Couchbase Community Newsletter to stay ahead of the curve on the latest NoSQL news, events, and webinars. Brought to you in partnership with Coucbase.

Couchbase 4.5 introduces a new Scan Consistency option: AtPlus. This joins the existing options to make a total of three:

  • NotBounded (default)
  • AtPlus (new to Couchbase Server 4.5)
  • RequestPlus

This blog post is going to review NotBounded and RequestPlus, as well as introduce AtPlus. I’ll also provide a code sample that you can use to try out all three yourself.

Not Bounded

This is the default behavior.

A N1QL query that is using "Not Bounded" Scan Consistency means that it will not wait for any indexes to finish updating before running the query and returning results. Let's say you have documents A,B, and C in a bucket. At the time of the query, only A and B are indexed. With Not Bounded, only documents A and B will be returned.

Since the query is not waiting on any indexing, this is the best option for performance. Document C will show up once it has been indexed. This shouldn’t take long, but if you have just created C, for instance, and then immediately query for a full list of documents, C may not show up. To demonstrate with the travel-sample bucket:

        private static void NotBoundedExample()
        {
            Console.WriteLine("========= NonBounded (default)");
            // get the current count
            var result1 =
                _bucket.Query<dynamic>("SELECT COUNT(1) as airportCount FROM `travel-sample` WHERE type='airport'")
                    .Rows.First();
            Console.WriteLine($"Initial count: {result1.airportCount}");

            // insert a new airport
            var doc = new Document<dynamic>
            {
                Id = "ScanConsistency::airport::" + _random.Next(10000),
                Content = new
                {
                    type = "airport"
                }
            };
            _bucket.Insert(doc);

            // get the count again
            var result2 =
                _bucket.Query<dynamic>("SELECT COUNT(1) as airportCount FROM `travel-sample` WHERE type='airport'")
                    .Rows.First();
            Console.WriteLine($"Count after insert: {result2.airportCount}");

            // wait a few seconds and get the count again
            Console.Write("Waiting for 5 seconds...");
            Thread.Sleep(5000);
            var result3 =
                _bucket.Query<dynamic>("SELECT COUNT(1) as airportCount FROM `travel-sample` WHERE type='airport'")
                    .Rows.First();
            Console.WriteLine($"Count after waiting: {result3.airportCount}");
        }


What I would expect to see when running this code is:

  1. Initial count: N
  2. Count after insert: N (still)
  3. Waiting
  4. Count after waiting: N+1

I set the wait time at 5 seconds, which is probably overkill. I use Thread.Sleep just for demonstration purposes, but obviously that's a clunky tactic for a real app. That brings us to RequestPlus.

RequestPlus

This scan consistency option provides nearly the opposite of Not Bounded. It will wait until all document changes and index updates (up until the query was run) are processed before it runs the query.

Here's a simplified example sequence of events when using RequestPlus

  1. Document C is created.
  2. N1QL query to get all documents is executed (A,B, and C all exist in the bucket).
  3. N1QL query is put on hold because at least one document needs to be indexed (document C).
  4. Indexing process is completed. Document C is now indexed.
  5. N1QL query executes, returning A,B,C.
        private static void RequestPlusExample()
        {
            Console.WriteLine("========= RequestPlus");

            // get the current count
            var result1 =
                _bucket.Query<dynamic>("SELECT COUNT(1) as airportCount FROM `travel-sample` WHERE type='airport'")
                    .Rows.First();
            Console.WriteLine($"Initial count: {result1.airportCount}");

            // insert a new airport
            var doc = new Document<dynamic>
            {
                Id = "ScanConsistency::airport::" + _random.Next(10000),
                Content = new
                {
                    type = "airport"
                }
            };
            _bucket.Insert(doc);

            // get the count again
            var request =
                QueryRequest.Create("SELECT COUNT(1) as airportCount FROM `travel-sample` WHERE type='airport'");
            request.ScanConsistency(ScanConsistency.RequestPlus);
            var result2 = _bucket.Query<dynamic>(request).Rows.First();
            Console.WriteLine($"Count after insert with RequestPlus: {result2.airportCount}");
        }


This gives us completeness in the query results, at the expense of performance. In some cases, it doesn’t make sense to wait for everything to be indexed. And that brings us to AtPlus.

AtPlus (New to Couchbase 4.5)

This scan consistency option provides something of a middle ground between RequestPlus and Not Bounded. This is also a brand new scan consistency option for Couchbase 4.5.

With AtPlus, you'll have to do a little more work in your code, but in return, you'll get better performance than using RequestPlus. Instead of waiting for an entire index to complete (which could be multiple documents), it will just wait for the documents that you specify to be indexed before running the query. This is sometimes known as "read your own write" or RYOW.

        private static void AtPlusExample()
        {
            Console.WriteLine("========= AtPlus");

            // get the current count
            var result1 = _bucket.Query<dynamic>("SELECT COUNT(1) as airportCount FROM `travel-sample` WHERE type='airport'")
                    .Rows.First();
            Console.WriteLine($"Initial count: {result1.airportCount}");

            // insert a new airport
            var doc = new Document<dynamic>
            {
                Id = "ScanConsistency::airport::" + _random.Next(10000),
                Content = new
                {
                    type = "airport"
                }
            };
            var insertResult = _bucket.Insert(doc);

            // get the count again
            var state = MutationState.From(insertResult.Document);
            var request = new QueryRequest("SELECT COUNT(1) as airportCount FROM `travel-sample` WHERE type='airport'");
            var t = request.ConsistentWith(state);
            var result2 = _bucket.Query<dynamic>(t).Rows.First();
            Console.WriteLine($"Count after insert with AtPlus: {result2.airportCount}");
        }


With this example, the N1QL query will wait only for the new document to be indexed. It will not wait for anything else. In a high-volume system, this could provide a good balance of increased performance and completeness of results.

Note that at the time of writing this post, you'll need to make sure that you explicitly set UseEnhancedDurability to true when setting up your ClientConfiguration:

            config.BucketConfigs = new Dictionary<string, BucketConfiguration> {
                {
                    "travel-sample", new BucketConfiguration
                    {
                        UseEnhancedDurability = true
                    }
                }
            };


Conclusion

With N1QL, there's a spectrum of speed and completeness. With Couchbase Server 4.5, you now have three options. NotBounded for raw speed, RequestPlus for up-to-now completeness, and AtPlus sits in between.

You can download the complete source code for this blog post on Github.

Please leave a comment, ping me on Twitter, or email me (matthew.groves AT couchbase DOT com).

The Getting Started with NoSQL Guide will get you hands-on with NoSQL in minutes with no coding needed. Brought to you in partnership with Couchbase.

Topics:
couchbase ,distributed database

Published at DZone with permission of Matthew Groves, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}