LiquiCouch: A Liquibase-Like Framework to Automate Schema Changes
Let's take a look what LiquiCouch is and how it works with Maven, with Gradle, and using it with and without Spring.
Join the DZone community and get the full member experience.
Join For FreeA lot has already been said about all the advantages of schema-less databases. They make a bunch of cumbersome tasks simpler and sometimes, much faster. But any relevant system cannot be totally "schema-less"; they expect at least some core structure to work with.
But is it really necessary to "validate" your schema from both your application and database? Sometimes, it sounds like overkill to me as most of the side effects of schema-less databases can be easily handled in the application code itself or in the query language (N1QL in our case). Of course, as your model evolves, it might get a little bit trickier to handle every single possibility since the project start.
In those cases, we commonly run data migrations (aka: execute Updates, Inserts or deletes) to migrate the "old schema" to the new one. Ideally, those migrations need to be executed automatically, sequentially, and if something goes wrong, the whole application start should fail.
Well, turns out that it is exactly what LiquiCouch does! It is a liquibase-like Java implementation for Couchbase. Let's see how it works:
With Maven:
<dependency>
<groupId>com.github.deniswsrosa</groupId>
<artifactId>liquicouch</artifactId>
<version>0.6.4</version>
</dependency>
With Gradle:
compile 'org.javassist:javassist:3.18.2-GA' // workaround for ${javassist.version} placeholder issue*
compile 'com.github.liquicouch:liquicouch:0.6.4'
Usage With Spring
You need to instantiate the Liquicouch object and provide some configuration. If you use Spring, it can be instantiated as a singleton bean in the Spring context. In this case, the migration process will be executed automatically upon startup.
@Autowired
private ApplicationContext context;
@Bean
public LiquiCouch liquicouch(){
LiquiCouch runner = new LiquiCouch(context); //It will grab all the data needed from the application.properties file
runner.setChangeLogsScanPackage(
"com.example.yourapp.changelogs"); // the package to be scanned for changesets
return runner;
}
Usage Without Spring
Using LiquiCouch without a spring context has a similar configuration but you have to remember to run execute() method to start the migration process.
LiquiCouch runner = new LiquiCouch("couchbase://SOME_IP_ADDRESS", "yourBucketName", "bucketPasword");
runner.setChangeLogsScanPackage(
"com.example.yourapp.changelogs"); // package to scan for changesets
runner.execute(); // ------> starts migration changesets
The examples above provide minimal configuration. However, the Liquicouch object provides some other possibilities (setters) to make the tool more flexible:
runner.setEnabled(shouldBeEnabled); // default is true, migration won't start if set to false
Creating Change Logs
ChangeLog contains a bunch of ChangeSets. ChangeSet is a single task (set of instructions made on a database). In other words, ChangeLog is a class annotated with @ChangeLog and contains methods annotated with @ChangeSet.
package com.example.yourapp.changelogs;
@ChangeLog
public class DatabaseChangelog {
@ChangeSet(order = "1", id = "someChangeId", author = "testAuthor")
public void importantWorkToDo(){
// task implementation
}
}
@ChangeLog
A class with change sets must be annotated by @ChangeLog. There can be more than one change log class but in that case, an order argument must be provided:
@ChangeLog(order = "001")
public class DatabaseChangelog {
//...
}
ChangeLogs are sorted alphabetically (that is why it is a good practice to start the order with zeros) by order argument, and change sets are applied due to this order.
@ChangeSet
A method annotated by @ChangeSet is taken and applied to the database. The history of applied change sets is stored in a dbChangeLog type document.

Annotation Parameters:
- order — a string for sorting change sets in one changelog. It sorts alphabetically in an ascending order. It can be a number, a date etc.
- ID — a name of a change set that must be unique for all change logs in a database
- author — the author of a changeSet
- runAlways — [optional, default: false] changeset will always be executed but only the first execution event will be stored as a document.
- recounts — [optional, default: 0] [Only applied when changSet returns a ParameterizedN1qlQuery] if you want to be sure that all documents have been updated, you can return a ParameterizedN1qlQuery. This query expects a result called size. If size is not zero, the query will be executed again according to the number of recounts specified. If none of the recounts returns zero, an exception will be thrown, and the application will fail to start.
- retries — [optional, default: 0] [Only applied when changSet returns a ParameterizedN1qlQuery] if the recount operation fails (the count result isn't zero), it will rerun the changeSet in an attempt to update the remaining documents (Your changeSet should be able to run multiple times without any side effects). If all retries fail, an exception will the thrown and the application will fail to start.

Defining ChangeSet Methods
Methods annotated by @ChangeSet
can have one of the following definition:
/**
* If you are using Spring, you can Autowire your Services or Repositories
*/
@Component
@ChangeLog(order = "001")
public class Migration1 {
@Autowired // Yes, You can a
private UserService userService;
@Autowired
private UserRepository userRepository;
@ChangeSet(order = "001", id = "someChangeId1", author = "testAuthor")
public void importantWorkToDo(Bucket bucket){
System.out.println("----------Migration1 - Method1");
}
@ChangeSet(order = "002", id = "someChangeId2", author = "testAuthor")
public void method2(Bucket bucket){
System.out.println("----------Migration1 - Method2");
}
@ChangeSet(order = "003", id = "someChangeId3", author = "testAuthor")
public void method3(Bucket bucket){
System.out.println("----------Migration1 - Method3");
}
@ChangeSet(order = "004", id = "someChangeId4", author = "testAuthor")
public void method4(Bucket bucket){
System.out.println("----------Migration1 - Method4");
}
@ChangeSet(order = "005", id = "someChangeId5", author = "testAuthor")
public void method5(){
System.out.println("----------Migration1 - Method5 (The bucket parameter is not necessary here)");
}
/**
* Here is an example of how you can check if your update has run successfully, all you need to do is to
* return a ParameterizedN1qlQuery.
* @return
*/
@ChangeSet(order = "006", id = "someChangeId6", author = "testAuthor", recounts = "2", retries = "1")
public ParameterizedN1qlQuery method6(){
//adding some data as an example
userService.save(new User("someUserIdForTesting", "user1", new Address(), new ArrayList<>(), Arrays.asList("admin", "manager")));
Iterable<User> users = userRepository.findAll(new PageRequest(0, 100));//we just care about the first 100 records
users.forEach( e-> {
//rename admin to adm
if(e.getSecurityRoles().contains("admin")) {
e.getSecurityRoles().remove("admin");
e.getSecurityRoles().add("adm");
}
userRepository.save(e);
});
//IMPORTANT: The query MUST have an attribute called *size*
String queryString = "Select count(userRole) as size from test t unnest t.securityRoles as userRole " +
" where t._class='com.cb.springdata.sample.entities.User' " +
" and userRole = 'admin'";
N1qlParams params = N1qlParams.build().consistency(ScanConsistency.REQUEST_PLUS).adhoc(true);
ParameterizedN1qlQuery query = N1qlQuery.parameterized(queryString, JsonObject.create(), params);
return query;
}
}
Without Spring
/**
* This is an example of how to use it without Spring, in this case you can execute all the queries via the Bucket argument.
*/
@ChangeLog(order = "2")
public class Migration2 {
@ChangeSet(order = "1", id = "someChangeId21", author = "testAuthor")
public void importantWorkToDo(){
System.out.println("----------Migration2 - Method1");
}
@ChangeSet(order = "2", id = "someChangeId22", author = "testAuthor")
public void method2(){
System.out.println("----------Migration2 - Method2");
}
@ChangeSet(order = "3", id = "someChangeId23", author = "testAuthor")
public void method3(){
System.out.println("----------Migration2 - Method3");
}
@ChangeSet(order = "4", id = "someChangeId24", author = "testAuthor")
public void method4(){
System.out.println("----------Migration2 - Method4");
}
@ChangeSet(order = "5", id = "someChangeId25", author = "testAuthor")
public void method5(Bucket bucket){
System.out.println("----------Migration2 - Method5");
}
@ChangeSet(order = "6", id = "someChangeId256", author = "testAuthor", runAlways=true)
public void method6(Bucket bucket){
System.out.println("----------Migration2 - Method6 - THIS SHOULD ALWAYS RUN "+bucket.name());
}
}
Demo Project
You can clone the sample project here https://github.com/deniswsrosa/liquicouch-demo
Documentation
https://github.com/deniswsrosa/liquicouch
Support
This is a community project mainly supported by me. If you have any issues or questions, just ping me at @deniswsrosa
Published at DZone with permission of Denis W S Rosa, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments