Couchbase Java SDK 2.0.0 Developer Preview 1

This new major version, planned to be 2.0, is a complete rewrite of the old 1.x series (currently 1.4.*). Leaving no stone unturned, it is built on top of RxJava for reactive and streaming-based programming and uses Netty as a consolidated IO layer for the best performance possible.

Before we get started, here is a mild warning: this first developer preview is released for developers to try our newest approach, and very importantly, give us feedback as early as possible in the process. APIs will change, also based on your input and feedback. Also note that rebalance support is only partially implemented, so make sure to add nodes to the cluster and then keep it as-is while playing around. Finally, this is only a small subset of the final API and functionality we will roll out in the coming weeks and months, so don't expect it yet to be on-par with the current SDK.

Motivation

The timing is perfect for a new SDK. With the release of Couchbase Server 2.0, the server has made a significant shift from a key-value oriented usage pattern to a query oriented one (mainly through the addition of Views). The current development of our next-generation query language (N1QL) only accelerates this movement. As a result, it is time that the SDKs also adopt a different programming model to aid the developer in terms of productivity and scalability of their systems.

In addition, while the current Java SDK is tried and true, it becomes harder to add new features that we ship with new server releases. The current SDK is built on top of the spymemcached library, which provides excellent support for our key-value based operations, but most of the other infrastructure that we provide (rebalance support, views,...) need to be worked around it. So, we decided that rather than just spruce up the living room, what we really needed was to rebuild from the foundations up.

What you see in this developer preview is only the first of many release which eventually will end up in a brand-new, stable and performant Couchbase Server SDK for not only Java but for the whole JVM. We fully recognize that more and more languages on the JVM are becoming popular (like Scala, JRuby or Clojure) and instead of providing wrappers around the Java SDK we wanted to go a different route. The SDK is split up into a core module which implements all of the common infrastructure like rebalance support, IO, retry mechanisms, backpressure and so forth, all wrapped by a completely message oriented API. It literally looks like this:

public interface Cluster {

    <R extends CouchbaseResponse> Observable<R> send(CouchbaseRequest request);

}

Now this is of course not the API that we expose to the average user, but it allows us to build 100% language specific SDKs on top of it without having to reimplement the hard bits over and over again. We will start out with the Java wrapper first, but are actively looking to add Scala support as well and my colleague Sergey Avseyev is already working on a JRuby client. It also makes community-driven wrappers much easier to develop.

You may have also noticed that the send method not only returns R but an Observable<R>. An Observable is like a Iterable, but push instead of pull based. All methods on the Java SDK as well use those type of return values (the examples a bit later will show this). This allows you to build fully reactive and asynchronous applications on top of the SDK, without having to work with thread pools directly. Observables are part of Rx (Reactive Extensions), available in a multitude of languages thanks to different libraries. We are using RxJava, which is the Java variant. It is very function oriented, so people coming from languages like Scala, Erlang or others will feel right at home. At first glance it can seem strange but you'll get used to it in no time - and you never want to go back (we promise!). And remember, you can always block on an Observable and transform it into an Iterable.

Inside the core package, we also unified the IO layer. It is now completely powered by Netty, one of the fastest NIO frameworks around on the JVM and used by many high-profile companies like Twitter, Facebook or Redhat in key deployments. This means it is much easier to configure and manage, providing the same characteristics no matter if you are doing a key-based lookup or running a N1QL query.

Hello SDK!

You need to download the developer preview first. You can do that either through a direct download of all the JARs, or use maven:

   <dependencies>
        <dependency>
            <groupId>com.couchbase.client</groupId>
            <artifactId>couchbase-client</artifactId>
            <version>2.0.0-dp</version>
        </dependency>
    </dependencies>

    <repositories>
        <repository>
            <id>couchbase</id>
            <name>couchbase repo</name>
            <url>http://files.couchbase.com/maven2</url>
            <snapshots><enabled>false</enabled></snapshots>
        </repository>
    </repositories>

Once it's on your classpath, you can connect to a cluster and then open the bucket (here to localhost with the default bucket).

Cluster cluster = new CouchbaseCluster();
Bucket bucket = cluster.openBucket().toBlockingObservable().single();

Since we want to block the current thread until our bucket is connected, we convert it into a blocking observable and then call `single()` to return our bucket instance. Now we can upsert a document (which has the same semantics as the previous `set` command) and then retreive it and print it out on the console:

Document doc = JsonDocument.create("hello", JsonObject.empty().put("Hello", "World"));
bucket.upsert(doc).toBlockingObservable().single();

// Prints JsonDocument{id='hello', cas=122278124461233, expiry=0, status=SUCCESS, content={Hello=World}}
System.out.println(bucket.get("hello").toBlockingObservable().single());

If you check your web UI, you can see a JSON document with the key "hello" created. Note how naturally you can now work with JSON documents compared to the current SDK.

While in the previous example we did block two times (once for the upsert and once for the get), we can use Observables to model this in a reactive way. Here is Java 8:

Document doc = JsonDocument.create("hello", JsonObject.empty().put("Hello", "World"));
bucket
    .upsert(doc)
    .flatMap(document -> bucket.get("hello"))
    .doOnNext(System.out::println)
    .flatMap(document -> cluster.disconnect())
    .subscribe();

It is also possible with Java 6 and 7, but looks a little more verbose since they do not have lambda support:

Document doc = JsonDocument.create("hello", JsonObject.empty().put("Hello", "World"));
bucket
    .upsert(doc)
    .flatMap(new Func1<Document, Observable<JsonDocument>>() {
        @Override
        public Observable<JsonDocument> call(Document document) {
            return bucket.get("hello");
        }
    })
    .doOnNext(new Action1<JsonDocument>() {
        @Override
        public void call(JsonDocument document) {
            System.out.println(document);
        }
     })
    .flatMap(new Func1<JsonDocument, Observable<Boolean>>() {
        @Override
        public Observable<Boolean> call(JsonDocument document) {
            return cluster.disconnect();
        }
    })
    .subscribe();

This code is fully asynchronous, and can be fanned out to many threads if needed. There is a multitude of options regarding scheduling and error handling available, which will be covered in subsequent blog posts.

Querying Views

The real power with Observables comes when you need to retreive more than one value. This is the case with nearly all query responses, be it views or N1QL queries. Here is a synchronous example on view querying, you can see that we are converting the Observable into a Iterable (which is its blocking cousin):

Iterator<ViewResult> iterator = bucket.query(ViewQuery.from("design", "view")).toBlockingObservable().getIterator();

while(iterator.hasNext()) {
    ViewResult result = iterator.next();
    System.out.println(result.id());
}

Or, we can use the asynchronous variant again:

bucket.query(ViewQuery.from("design", "view")).subscribe(viewResult ->
    System.out.println(viewResult.id())
);

Both examples print the IDs found for each document in the view query.

Querying N1QL

If you have the N1QL server running, you can also query your documents like this:

System.setProperty("com.couchbase.client.queryEnabled", "true");

Cluster cluster = new CouchbaseCluster();
Bucket bucket = cluster.openBucket().toBlockingObservable().single();

bucket.query(Query.raw("select * from default")).subscribe(result -> {
        System.out.println(result);
});

Make sure to set the system property so that querying gets enabled. You can then work on the resulting `JsonObject` instances.  This is just the basic interface. More on that below.

Next Steps

We truly believe that with this first developer preview, the groundwork has been laid for something really great. This release already contains functionality which will be provided with some of the next server versions, so if you dig in the sources you may find a hint or two. The next developer previews will add full rebalance support and getting the API closer to completion (that is, fluent DSLs for both View and N1QL Querying, replica read, durability constraints, …). Also, we will provide more transcoders, especially one for legacy codebases and other serialization purposes (so you don't have to use JSON all the time). Based on your feedback we will tweak the current APIs, fix bugs as they come up and make sure the whole thing gets more stable as we approach a beta release.

Please raise any issues you find or enhancements you'd like to see in our issue tracker. You can also ask questions on our communities page, or even get in touch with me directly over twitter.

Enjoy!