Spring Data: Data Auditing Using JaVers and MongoDB

Recently I was looking for some simple options to track all changes to entities in all my microservices. After researching, I discovered the Java library named JaVers. It is designed as a framework for auditing changes in object-oriented data.

In this post, we’ll see how to use JaVers in a simple Spring Boot application and MongoDB environment to track changes of entities.

Prerequistes

• Spring Boot 2.
• Maven 3.6.1.
• JAVA 8.
• Mongo 4.4.

What is JaVers

JaVers is an audit log framework that helps to track changes of entities in the application. 

The usage of this tool is not limited to debugging and auditing only. It can be successfully applied to perform analysis, force security policies and maintaining the event log, too.

Why use JaVers

• It is open source and free to use.
• JaVers is written in Java 8 and can be run on JRE 8 or higher.
• It is compatible with both traditional relational relational database systems as well as NoSQL systems. This makes migration between RDBMS and NoSQL much smoother.
• Easy integration with Spring and Spring Boot.

Setup Spring Boot Project

To integrate JaVers in your application you just need to include the JaVers dependency in you project’s build.gradle or pom.xml file. 

xxxxxxxxxx

<dependency>

    <groupId>org.javers</groupId>

    <artifactId>javers-spring-boot-starter-mongo</artifactId>

    <version>5.13.2</version>

</dependency>

xxxxxxxxxx

compile 'org.javers:javers-spring-boot-starter-mongo:5.13.2'

JaversRepository Configuration

JaVers starters rely on Spring Data starters.  it offers an option to use a dedicated Mongo database for JaVers data.

xxxxxxxxxx

javers:

  mongodb:

    host: localhost

    port: 27017

    authentication-database: admin

    database: <<databasemane>>

    username: <<dbuser>>

    password: <<dbpassword>>

Or

xxxxxxxxxx

javers:

  mongodb:

    uri: mongodb://<<dbuser>>:<<dbpassword>>@localhost:27017/<<databasemane>>&authSource=admin

If javers.mongodb property is defined, either host or uri has to set. If so, an application’s data and JaVers data are stored in different databases. 

Audit Annotations

JaVers is based on annotations: @JaversSpringDataAuditable and @JaversAuditable. 

@JaversSpringDataAuditable annotation is used to audit JPA repositories.

xxxxxxxxxx

@JaversSpringDataAuditable

    public interface EmployeeRepository extends MongoRepository<Employee, String> {

}

By adding this annotation, you create an audit entry for each create, update or delete operation you perform in the database.

In case of a custom repository or If you aren't using Spring Data repositories, annotate all data-changing methods with @JaversAuditable.

xxxxxxxxxx

@JaversAuditable

public User save(User user) {

    ...//

}

@JaversAuditable

public User update(User user) {

    ...//

}

@JaversAuditable

public void delete(User user) {

    ...//

}

Trying it Out

Now let's try to change the data. for this, we use the Postman tool to run the endpoints.

we notice that JaVers added two new collections in our MongoDB database after saving the employee document.

• jv_head_id — one document with the last CommitId,
• jv_snapshots — domain object snapshots. Each document contains snapshot data and commit metadata.

In the jv_snapshots collection, there are three types of documents for auditing a document.

• INITIAL: Create operation in the database.
• UPDATE: Update operation.
• TERMINATE: Delete operation has been performed.

When creating the employee document, we can see in jv_snapshots:

• commitMetadata: This includes valuable information such as the author of the change, the time of the event in local (commitDate) and UTC (commitDateInstant). 
• globalId: Contains information about the entity being modified and its database id.
• state: Contains all the field values of the entity being modified. Please note that this is the state of the object after the change has been performed.
• changedProperties: This includes the fields that have been modified in this audit log entry. Since this is a creation operation, all fields have been included in the changed properties array.

Let's start the update operation:

Notice that a new document has been created with type UPDATE and the version has been incremented to 2. 

Finally, let's run the delete operation:

Since the object has been deleted, the entry log type here is TERMINATE. Notice that the changed properties array is empty.

Author Provider

Each committed change in JaVers should have its author.  i.e. the user who made the change.  Moreover, JaVers supports Spring Security out of the box. You need to register an implementation of the AuthorProvider interface, which should return a current user name. 

xxxxxxxxxx

@Configuration

public class JaversConfiguration {

    private static final String AUTHOR = "aek author";

    @Bean

    public AuthorProvider provideJaversAuthor() {

        return new SimpleAuthorProvider();

    }

    private static class SimpleAuthorProvider implements AuthorProvider {

        @Override

        public String provide() {

            return AUTHOR;

        }

    }

}

In this article, I created a simple custom implementation of the AuthorProvider interface.

Retrieving Audit Information From JaVers Snapshots Repository

 JaVers provides its own JaVers Query Language (JQL). It is a simple, fluent API which allows you to query JaversRepository for changes of a given class, object or property.

Data history can be fetched from JaversRepository using javers.find*() methods in one of three views: Shadows, Changes, and Snapshots.

• Shadow is a historical version of a domain object restored from a snapshot.
• Change represents an atomic difference between two objects.
• Snapshot is a historical state of a domain object captured as the property:value map.

Let's take a look at some examples:

When I want to retrieve all the changes made to the Employee collection. 

curl --location --request GET 'http://localhost:8080/audit/employees'

xxxxxxxxxx

<pre>Changes:

        Commit 5.00 done by aek author at 14 nov 2020, 14:30:11 :

        * changes on com.java.audit.springdatamongojavers.domain.Employee/5fafdbb15be81677928fbbb7 :

        - 'email' value changed from 'johndoe.emal.com' to 'john.email.com'

        - 'firstName' value changed from 'John' to 'Doe'

        - 'lastName' value changed from 'Doe' to 'John'

        Commit 4.00 done by aek author at 14 nov 2020, 14:29:21 :

        * new object: com.java.audit.springdatamongojavers.domain.Employee/5fafdbb15be81677928fbbb7

        * changes on com.java.audit.springdatamongojavers.domain.Employee/5fafdbb15be81677928fbbb7 :

        - 'email' value changed from '' to 'johndoe.emal.com'

        - 'firstName' value changed from '' to 'John'

        - 'id' value changed from '' to '5fafdbb15be81677928fbbb7'

        - 'lastName' value changed from '' to 'Doe'

        Commit 3.00 done by aek author at 14 nov 2020, 13:48:51 :

        * object removed: com.java.audit.springdatamongojavers.domain.Employee/5fafbbc7c3f606329bd51e7c

        Commit 2.00 done by aek author at 14 nov 2020, 13:37:02 :

        * changes on com.java.audit.springdatamongojavers.domain.Employee/5fafbbc7c3f606329bd51e7c :

        - 'email' value changed from 'johndoe.emal.com' to 'john.email.com'

        - 'firstName' value changed from 'John' to 'Doe'

        - 'lastName' value changed from 'Doe' to 'John'

        Commit 1.00 done by aek author at 14 nov 2020, 12:13:11 :

        * new object: com.java.audit.springdatamongojavers.domain.Employee/5fafbbc7c3f606329bd51e7c

        * changes on com.java.audit.springdatamongojavers.domain.Employee/5fafbbc7c3f606329bd51e7c :

        - 'email' value changed from '' to 'johndoe.emal.com'

        - 'firstName' value changed from '' to 'John'

        - 'id' value changed from '' to '5fafbbc7c3f606329bd51e7c'

        - 'lastName' value changed from '' to 'Doe'

</pre>

Properties are matched by name, and their values are compared, without paying much attention to the actual Employee class.

JaVers can have different applications, from debugging to complex analysis. The official documentation presents several examples of use.

The complete source code can be found in my GitHub repository.