Create a Production-Grade Java Microservice Architecture With JHipster and Couchbase

Introduction

Every successful developer needs to keep up to date with cutting-edge new technologies and emerging trends by trying them, playing with them, seeing if they can fit into a project, and seeing how they can improve upon them.

But how can one achieve that in a fast-changing world where new technologies and frameworks are born daily?

JHipster builds fully tested applications quickly and easily, using best practices, methodologies, and strategies. It proposes a lot of options (Architecture — monolith or microservices; Database — SQL or NoSQL; Security — session, JWT, or OAuth2 authentication; Other — WebSocket, caches, Docker, Kubernetes, etc.), which can help you try the best technologies out there and choose the most suitable combinations for your use case.

Today, we are going to focus on some of the most important new architectural concepts: microservices and NoSQL with Couchbase. 

Why Couchbase?

Couchbase is a NoSQL document-oriented database. With document databases, you have a schemaless design which allows you to change your data freely and easily. You can also store the whole structure in a single document, avoiding lots of unnecessary joins, which means naturally faster read and write operations.

Couchbase exposes a fast key-value store and a powerful query engine along with built-in indexers to query data with N1QL, a SQL-like language for JSON documents. With its masterless distributed architecture, it is very easy to scale. It can deliver millions of operations per second without the need of a third-party cache to perform. Couchbase also comes with built-in full-text search, an analytics engine, and a complete mobile solution.

Why Microservices?

Microservice architecture is a pattern of developing software systems that focus on building applications as a set of small, modular, and loosely coupled services, enabling easier continuous delivery, better testability, scalability, and improved fault isolation. Each service can be written in different languages and may use different data storage techniques, which enables the ability to organize development around multiple feature teams.

But microservices comes with some challenges: decomposing the application into services can be very complicated, and is very much an art. Developers also have to deal with the additional complexity of a distributed system.

JHipster handles most of the complexities of microservices: Service discovery and configuration with or JHipster Registry (Netflix Eureka, Spring Cloud Config Server, and monitoring dashboard), load balancing with Netflix Ribbon , fault tolerance with Netflix Hystrix, and centralized logging and monitoring with jhipster-console (Elasticsearch, Logstash, and Kibana personalized stack), and more.

Creating a Brewery Microservice

Prerequisites:

yarn global add generator-jhipster

Generate an API Gateway

In order to access our different services in a microservice architecture, we're going to need an API Gateway. It is the entrance to your microservices. It provides HTTP routing (Netflix Zuul) and load balancing (Netflix Ribbon), quality of services (Netflix Hystrix), security (Spring Security), and API documentation ( ) for all microservices.

In a terminal window:

mkdir couchbase-jhipster-microservices-example
cd couchbase-jhipster-microservices-example
mkdir gateway
cd gateway
jhipster

JHipster asks about the type of application you want to create and what features you want to include. You can find all the details about available options on JHipster Website. Use the following answers in order to generate a gateway with Couchbase support.

Generate a Brewery Microservice Application

In couchbase-jhipster-microservices-example, create a brewery directory, then run jhipster to generate a microservice with Couchbase database with the following answers:

Generate Brewery Entities

Create a file brewery.jh in couchbase-jhipster-microservices-example directory with the following (JHipster Domain Language):

entity Beer {
   name String required,
   category String required,
   description String,
   style String,
   brewery String,
   abv Float,
   ibu Integer,
   srm Integer,
   upc Integer,
   updated LocalDate
}

entity Brewery {
   name String required,
   description String,
   address String required maxlength(200),
   city String,
   code String,
   country String,
   phone String pattern(/[0-9- .]+/),
   state String,
   website String,
   updated LocalDate
}

paginate Beer with pager paginate Brewery with infinite-scroll

In a terminal window, run the following commands:

cd brewery
jhipster import-jdl ../brewery.jh
cd ../gateway
jhipster entity brewery
jhipster entity beer

When asked to overwrite files, always answer a.

Run the Microservice Architecture

In order to run our architecture, we need to start the following:

1. JHipster Registry
2. Keycloack: an open source identity and access management solution
3. Couchbase Server
4. Brewery microservice
5. Gateway

Fortunately, JHipster has a docker-compose sub-generator that will start all necessary services without headache. But first, we need to build our applications. In couchbase-jhipster-microservices-example:

cd gateway
./mvnw package -Pprod dockerfile:build -DskipTests
cd ../brewery
./mvnw package -Pprod dockerfile:build -DskipTests
cd ..
mkdir docker-compose && cd docker-compose

Now, we can generate docker-compose.yml using the following command and answers:

jhipster docker-compose

In order to make our app work with a local keycloak, we need to add to your hosts file (Windows: C:\Windows\System32\drivers\etc\hosts, Mac/Linux: /etc/hosts ) the following line:

127.0.0.1 keycloak

Before starting everything, make sure you have configured Docker with enough memory and CPUs, then run:

docker-compose up

Once everything finishes starting, open a browser to Gateway (http://localhost:8080/), click on account, then sign in.

You should be redirected to keycloak, log in using admin for both user and password.

You'll return to the gateway where you can see administration interfaces, change language, view, and edit your entities...

JHipster-registry

Apart from being the backbone of your microservice application, as it is a discovery server that handles routing, load balancing, scalability, and a configuration server with Spring Cloud Config Server that provides runtime configuration of your applications, JHipster registry is also an administration server where you can visualize your application instances, health, metrics, and logs.

How Does It Work?

Accessing the Database

First, you need access to brewery-couchbase instance. In order to do so, update docker-compose/docker-compose.yml to publish its administration interface port with the following:

brewery-couchbase:
   build:
   context: ../brewery/src/main/docker
   dockerfile: couchbase/Couchbase.Dockerfile
   environment:
       - BUCKET=brewery
   ports:
       - 8091:8091

Let's apply our changes on docker-compose directory:

docker-compose up -d

When everything is up, open Couchbase administration interface on http://localhost:8091/, then log in using "Administrator" as user and "password" as the password. Open brewery bucket Documents.

As you can see, your bucket is already populated with some documents: JHipster populates bootstrap documents using a Java data migration library for Couchbase, as it strongly favors simplicity and convention over configuration.

You can take a look at changelog files in:

brewery/src/main/resources/config/couchmove/changelog directory:

• V0.1__initial_setup directory: contains user and authority documents, used by Spring Security to authenticate users.
• V0__create_indexes.n1ql: creates needed indexes.

As Flyway or for SQL databases, Couchmove maintains changelog documents, keeping track of executed changelogs.

Spring Boot and Spring Data Couchbase

JHipster generates a Spring Boot 2 application and uses Spring Data to access relational and non-relational databases. In our case, it is using Spring Data Couchbase to provide integration with the Couchbase Server database. But it goes much further by personalizing how it works:

By default, Spring Data Couchbase uses N1qlCouchbaseRepository.java,  which leverages N1QL only for paging or sorting findAll operations. All other operations are using views as you can see in the implementation of SimpleCouchbaseRepository.java. Since view indexes are always accessed from disk, they are not quite as performant.

Let's see how JHipster improves this behavior:

@Configuration
@Profile("!" + JHipsterConstants.SPRING_PROFILE_CLOUD)
@EnableCouchbaseRepositories(repositoryBaseClass = CustomN1qlCouchbaseRepository.class, basePackages = "com.couchbase.example.brewery.repository")
@Import(value = CouchbaseAutoConfiguration.class)
@EnableCouchbaseAuditing(auditorAwareRef = "springSecurityAuditorAware")
public class DatabaseConfiguration {
   ...
   @Bean
   public Couchmove couchmove(Bucket couchbaseBucket) {
       log.debug("Configuring Couchmove");
       Couchmove couchMove = new Couchmove(couchbaseBucket, "config/couchmove/changelog");
       couchMove.migrate();
       return couchMove;
   }
   ...
}                                                           

DatabaseConfiguration class configures and starts Couchmove migrations and also enables Couchbase Repositories, but with a custom base class: CustomN1qlCouchbaseRepository

public class CustomN1qlCouchbaseRepository<T, ID extends Serializable> extends N1qlCouchbaseRepository<T, ID> {
   ...
   @Override
   public <S extends T> S save(S entity) {
       return super.save(populateIdIfNecessary(entity));
   }
   ...
}

This class extends the default repository to auto-generate document ID before saving and enables usage of N1QL for all operations by implementing CustomN1qlRespository interface:

@NoRepositoryBean
public interface N1qlCouchbaseRepository<T, ID extends Serializable> extends CouchbasePagingAndSortingRepository<T, ID> {

   @Query("#{#n1ql.selectEntity} WHERE #{#n1ql.filter}")
   List<T> findAll();

   @Query("SELECT count(*) FROM #{#n1ql.bucket} WHERE #{#n1ql.filter}")
   long count();

   @Query("DELETE FROM #{#n1ql.bucket} WHERE #{#n1ql.filter} returning #{#n1ql.fields}")
   T removeAll();

   default void deleteAll() {
       removeAll();
   }
}

Ids are auto-generated using GeneratedValue annotation with a prefix using annotation, which by default is the name of the class, separated by __ delimiter.

@Document
public class Brewery implements Serializable {

   private static final long serialVersionUID = 1L;

   public static final String PREFIX = "brewery";

   @SuppressWarnings("unused")
   @IdPrefix
   private String prefix = PREFIX;

   @Id
   @GeneratedValue(strategy = UNIQUE, delimiter = ID_DELIMITER)
   private String id;
   ...
}

Tests

We said earlier that JHipster-generated applications are fully tested with unit and integration tests, but how does it work for integration ones?

For Couchbase, it uses Couchbase testContainers, which is a module extending testcontainers, a Java library that makes it easy to launch any Docker containers for the duration of JUnit tests in order to automatically start an instance of Couchbase Server and configure it with the necessary services, users, buckets, and indexes.

In order to run the tests, run the following command in a terminal window:

./mvnw clean test

Populate Some Data

In order to populate some data, we will take advantage of Couchbase Sample buckets. Open Settings tab, Sample Buckets, and then select beer-sample and click on Load Sample Data.

You need to shape data in Spring Data Couchbase serialization format. To do so, open Query tab, and execute the following queries:

INSERT INTO brewery (KEY k, VALUE v)
  SELECT type || "__" || meta().id as k,
  {
     "name": name,
     "category": category,
     "description": description,
     "style": style,
     "brewery": brewery_id,
     "abv": abv,
     "ibu": ibu,
     "srm": srm,
     "upc": upc,
     "updated": STR_TO_MILLIS(updated),
     "_class" : "com.couchbase.example.brewery.domain.Beer"
  } as v
  FROM `beer-sample`
  WHERE type = 'beer';

INSERT INTO brewery (KEY k, VALUE v)
  SELECT type || "__" || meta().id as k,
  {
     "name": name,
     "description": description,
     "address": address[0],
     "country": country,
     "website": website,
     "code": code,
     "city": city,
     "phone": phone,
     "state": state,
     "updated": STR_TO_MILLIS(updated),
     "_class" : "com.couchbase.example.brewery.domain.Brewery"
  } as v
  FROM `beer-sample`
  WHERE type = 'brewery';

Here is an example of an inserted beer document:

Now let's see how JHipster handles newly inserted documents. Navigate to Gateway, Entities, then open Brewery. JHipster first loads 20 breweries, and if you scroll down, it loads more! It proposes page navigation for Beer entity because we choose this behavior when generating entities with brewery.jh JDL.

paginate Beer with pager
paginate Brewery with infinite-scroll

Source Code

The source code for generated applications is available at tchlyah/couchbase-jhipster-microservices-example.

What's Next?

JHipster, with the help of Elasticsearch, proposes an option that adds search capabilities on top of the database. But Couchbase has an out-of-the-box Full-Text Search feature that provides extensive capabilities for natural-language querying. Why not implement that support into JHipster? If you can help, don't hesitate to contribute!

If you have any questions, leave a comment below.