Exposing Microservices Over REST Protocol Buffers

Today, exposing a RESTful API with JSON protocol is the most common standard. We can find many articles describing advantages and disadvantages of JSON versus XML. Both of these protocols exchange messages in text format. If an important aspect affecting the choice of communication protocol in your systems is a performance you should definitely pay attention to Protocol Buffers. It is a binary format created by Google as "a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more."

Protocol Buffers, sometimes referred to as Protobuf, are not only a message format but also a set of language rules that define the structure of messages. It is extremely useful in service to service communication, a described in the article "Beating JSON Performance With Protobuf." In that example, Protobuf was about five times faster than JSON for tests based on the Spring Boot framework.

An introduction to Protocol Buffers can be found here. My sample is similar to previous samples from my weblog – it is based on two microservices, account and customer, which calls one of account’s endpoints. Let’s begin with a message types definition provided inside .proto file. Place your .proto file in the src/main/proto directory. Here’s account.proto defined in account service. We set  java_package  and  java_outer_classname  to define the package and name of the Java generated class. Message definition syntax is pretty intuitive. The Account object generated from that file has three properties: id, customerId, and number. There is also anAccounts object, which wraps the list of Account objects.

syntax = "proto3";

package model;

option java_package = "pl.piomin.services.protobuf.account.model";
option java_outer_classname = "AccountProto";

message Accounts {
    repeated Account account = 1;
}

message Account {

    int32 id = 1;
    string number = 2;
    int32 customer_id = 3;

}

Here’s the .proto file definition from customer service. It is a little bit more complicated than the previous one from account service. In addition to its definitions, it contains definitions of account service messages, because they are used by the @Feign client.

syntax = "proto3";

package model;

option java_package = "pl.piomin.services.protobuf.customer.model";
option java_outer_classname = "CustomerProto";

message Accounts {
    repeated Account account = 1;
}

message Account {

    int32 id = 1;
    string number = 2;
    int32 customer_id = 3;

}

message Customers {
    repeated Customer customers = 1;
}

message Customer {

    int32 id = 1;
    string pesel = 2;
    string name = 3;
    CustomerType type = 4;
    repeated Account accounts = 5;

    enum CustomerType {
        INDIVIDUAL = 0;
        COMPANY = 1;
    }

}

We generate source code from the message definitions above by using the protobuf-maven-plugin Maven plugin. The plugin needs to have a protocExecutable file location set. This executable file can be downloaded from Google’s Protocol Buffer download site.

<plugin>
    <groupId>org.xolstice.maven.plugins</groupId>
    <artifactId>protobuf-maven-plugin</artifactId>
    <version>0.5.0</version>
    <executions>
        <execution>
            <id>protobuf-compile</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>compile</goal>
            </goals>
            <configuration>
                <outputDirectory>src/main/generated</outputDirectory>
                <protocExecutable>${proto.executable}</protocExecutable>
            </configuration>
        </execution>
    </executions>
</plugin>

Protobuf classes are generated into the src/main/generated output directory. Let’s add that source directory to Maven sources with build-helper-maven-plugin.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <executions>
        <execution>
            <id>add-source</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>add-source</goal>
            </goals>
            <configuration>
                <sources>
                    <source>src/main/generated</source>
                </sources>
            </configuration>
        </execution>
    </executions>
</plugin>

The sample application source code is available on GitHub. Before proceeding to the next steps build application using mvn clean install command. Generated classes are available under src/main/generated and our microservices are ready to run. Now, let me describe some implementation details. We need two dependencies in Maven pom.xml to use Protobuf.

<dependency>
    <groupId>com.google.protobuf</groupId>
    <artifactId>protobuf-java</artifactId>
    <version>3.3.1</version>
</dependency>
<dependency>
    <groupId>com.googlecode.protobuf-java-format</groupId>
    <artifactId>protobuf-java-format</artifactId>
    <version>1.4</version>
</dependency>

Then, we need to declare the default HttpMessageConverter@Bean and inject it into RestTemplate@Bean.

@Bean
@Primary
ProtobufHttpMessageConverter protobufHttpMessageConverter() {
    return new ProtobufHttpMessageConverter();
}

@Bean
RestTemplate restTemplate(ProtobufHttpMessageConverter hmc) {
    return new RestTemplate(Arrays.asList(hmc));
}

Here’s the REST @Controller code. Account and Accounts from the AccountProto generated class are returned as a response body in all three API methods visible below. All objects generated from .proto files have the newBuilder method used for creating new object instances. I also set application/x-protobuf as the default response content type.

@RestController
public class AccountController {

    @Autowired
    AccountRepository repository;

    protected Logger logger = Logger.getLogger(AccountController.class.getName());

    @RequestMapping(value = "/accounts/{number}", produces = "application/x-protobuf")
    public Account findByNumber(@PathVariable("number") String number) {
        logger.info(String.format("Account.findByNumber(%s)", number));
        return repository.findByNumber(number);
    }

    @RequestMapping(value = "/accounts/customer/{customer}", produces = "application/x-protobuf")
    public Accounts findByCustomer(@PathVariable("customer") Integer customerId) {
        logger.info(String.format("Account.findByCustomer(%s)", customerId));
        return Accounts.newBuilder().addAllAccount(repository.findByCustomer(customerId)).build();
    }

    @RequestMapping(value = "/accounts", produces = "application/x-protobuf")
    public Accounts findAll() {
        logger.info("Account.findAll()");
        return Accounts.newBuilder().addAllAccount(repository.findAll()).build();
    }

}

The method GET /accounts/customer/{customer} is called from customer service using the @Feign client.

@FeignClient(value = "account-service")
public interface AccountClient {

    @RequestMapping(method = RequestMethod.GET, value = "/accounts/customer/{customerId}")
    Accounts getAccounts(@PathVariable("customerId") Integer customerId);

}

We can easily test the described configuration using the JUnit test class visible below.

@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
@RunWith(SpringRunner.class)
public class AccountApplicationTest {

    protected Logger logger = Logger.getLogger(AccountApplicationTest.class.getName());

    @Autowired
    TestRestTemplate template;

    @Test
    public void testFindByNumber() {
        Account a = this.template.getForObject("/accounts/{id}", Account.class, "111111");
        logger.info("Account[\n" + a + "]");
    }

    @Test
    public void testFindByCustomer() {
        Accounts a = this.template.getForObject("/accounts/customer/{customer}", Accounts.class, "2");
        logger.info("Accounts[\n" + a + "]");
    }

    @Test
    public void testFindAll() {
        Accounts a = this.template.getForObject("/accounts", Accounts.class);
        logger.info("Accounts[\n" + a + "]");
    }

    @TestConfiguration
    static class Config {

        @Bean
        public RestTemplateBuilder restTemplateBuilder() {
            return new RestTemplateBuilder().additionalMessageConverters(new ProtobufHttpMessageConverter());
        }

    }

}

This article shows how to enable Protocol Buffers for microservices project based on Spring Boot. Protocol Buffer is an alternative to text-based protocols like XML or JSON and surpasses them in terms of performance. Adapt to this protocol using in Spring Boot application is pretty simple. For microservices, we can still use Spring Cloud components like Feign or Ribbon in combination with Protocol Buffers, as with REST over JSON or XML.