Written by Josh Long on the Spring blog This week I’m in São Paulo, Brazil presenting at QCon SP. I had an interesting discussion with someone who loves Spring’s REST stack, but wondered if there was something more efficient than plain-ol’ JSON. Indeed, there is! I often get asked about Spring’s support for high-speed binary based encoding of messages. Spring’s long supported RPC encoding with the likes of Hessian, Burlap, etc., and Spring Framework 4.1 introduced support for Google Protocol Buffers which can be used with REST services as well. From the Google Protocol Buffer website: Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages… Google uses Protocol Buffers extensively in their own, internal, service-centric architecture. A .proto document describes the types (_messages_) to be encoded and contains a definition language that should be familiar to anyone who’s used C structs. In the document, you define types, fields in those types, and their ordering (memory offsets!) in the type relative to each other. The .proto files aren’t implementations - they’re declarative descriptions of messages that may be conveyed over the wire. They can prescribe and validate constraints - the type of a given field, or the cardinatlity of that field - on the messages that are encoded and decoded. You must use the Protobuf compiler to generate the appropriate client for your language of choice. You can use Google Protocol Buffers anyway you like, but in this post we’ll look at using it as a way to encode REST service payloads. This approach is powerful: you can use content-negotiation to serve high speed Protocol Buffer payloads to the clients (in any number of languages) that accept it, and something more conventional like JSON for those that don’t. Protocol Buffer messages offer a number of improvements over typical JSON-encoded messages, particularly in a polyglot system where microservices are implemented in various technologies but need to be able to reason about communication between services in a consistant, long-term manner. Protocol Buffers are several nice features that promote stable APIs: Protocol Buffers offer backward compatibility for free. Each field is numbered in a Protocol Buffer, so you don’t have to change the behavior of the code going forward to maintain backward compatability with older clients. Clients that don’t know about new fields won’t bother trying to parse them. Protocol Buffers provide a natural place to specify validation using the required,optional, and repeated keywords. Each client enforces these constraints in their own way. Protocol Buffers are polyglot, and work with all manner of technologies. In the example code for this blog alone there is a Ruby, Python and Java client for the Java service demonstrated. It’s just a matter of using one of the numerous supported compilers. You might think that you could just use Java’s inbuilt serialization mechanism in a homogeneous service environment but, as the Protocol Buffers team were quick to point out whent hey first introduced the technology, there are some problems even with that. Java language luminary Josh Bloch’s epic tome, Effective Java, on page 213, provides further details. Let’s first look at our .proto document: package demo; option java_package = "demo"; option java_outer_classname = "CustomerProtos"; message Customer { required int32 id = 1; required string firstName = 2; required string lastName = 3; enum EmailType { PRIVATE = 1; PROFESSIONAL = 2; } message EmailAddress { required string email = 1; optional EmailType type = 2 [default = PROFESSIONAL]; } repeated EmailAddress email = 5; } message Organization { required string name = 1; repeated Customer customer = 2; } You then pass this definition to the protoc compiler and specify the output type, like this: protoc -I=$IN_DIR --java_out=$OUT_DIR $IN_DIR/customer.proto Here’s the little Bash script I put together to code-generate my various clients: #!/usr/bin/env bash SRC_DIR=`pwd` DST_DIR=`pwd`/../src/main/ echo source: $SRC_DIR echo destination root: $DST_DIR function ensure_implementations(){ # Ruby and Go aren't natively supported it seems # Java and Python are gem list | grep ruby-protocol-buffers || sudo gem install ruby-protocol-buffers go get -u github.com/golang/protobuf/{proto,protoc-gen-go} } function gen(){ D=$1 echo $D OUT=$DST_DIR/$D mkdir -p $OUT protoc -I=$SRC_DIR --${D}_out=$OUT $SRC_DIR/customer.proto } ensure_implementations gen java gen python gen ruby This will generate the appropriate client classes in the src/main/{java,ruby,python}folders. Let’s first look at the Spring MVC REST service itself. A Spring MVC REST Service In our example, we’ll register an instance of Spring framework 4.1’s org.springframework.http.converter.protobuf.ProtobufHttpMessageConverter. This type is an HttpMessageConverter. HttpMessageConverters encode and decode the requests and responses in REST service calls. They’re usually activated after some sort of content negotiation has occurred: if the client specifies Accept: application/x-protobuf, for example, then our REST service will send back the Protocol Buffer-encoded response. package demo; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.context.annotation.Bean; import org.springframework.http.converter.protobuf.ProtobufHttpMessageConverter; import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; import java.util.Arrays; import java.util.Collection; import java.util.Map; import java.util.concurrent.ConcurrentHashMap; import java.util.stream.Collectors; @SpringBootApplication public class DemoApplication { public static void main(String[] args) { SpringApplication.run(DemoApplication.class, args); } @Bean ProtobufHttpMessageConverter protobufHttpMessageConverter() { return new ProtobufHttpMessageConverter(); } private CustomerProtos.Customer customer(int id, String f, String l, Collection emails) { Collection emailAddresses = emails.stream().map(e -> CustomerProtos.Customer.EmailAddress.newBuilder() .setType(CustomerProtos.Customer.EmailType.PROFESSIONAL) .setEmail(e).build()) .collect(Collectors.toList()); return CustomerProtos.Customer.newBuilder() .setFirstName(f) .setLastName(l) .setId(id) .addAllEmail(emailAddresses) .build(); } @Bean CustomerRepository customerRepository() { Map customers = new ConcurrentHashMap<>(); // populate with some dummy data Arrays.asList( customer(1, "Chris", "Richardson", Arrays.asList("[email protected]")), customer(2, "Josh", "Long", Arrays.asList("[email protected]")), customer(3, "Matt", "Stine", Arrays.asList("[email protected]")), customer(4, "Russ", "Miles", Arrays.asList("[email protected]")) ).forEach(c -> customers.put(c.getId(), c)); // our lambda just gets forwarded to Map#get(Integer) return customers::get; } } interface CustomerRepository { CustomerProtos.Customer findById(int id); } @RestController class CustomerRestController { @Autowired private CustomerRepository customerRepository; @RequestMapping("/customers/{id}") CustomerProtos.Customer customer(@PathVariable Integer id) { return this.customerRepository.findById(id); } } Most of this code is pretty straightforward. It’s a Spring Boot application. Spring Boot automatically registers HttpMessageConverter beans so we need only define the ProtobufHttpMessageConverter bean and it gets configured appropriately. The @Configuration class seeds some dummy date and a mock CustomerRepository object. I won’t reproduce the Java type for our Protocol Buffer, demo/CustomerProtos.java, here as it is code-generated bit twiddling and parsing code; not all that interesting to read. One convenience is that the Java implementation automatically provides builder methods for quickly creating instances of these types in Java. The code-generated types are dumb struct like objects. They’re suitable for use as DTOs, but should not be used as the basis for your API. Do not extend them using Java inheritance to introduce new functionality; it’ll break the implementation and it’s bad OOP practice, anyway. If you want to keep things cleaner, simply wrapt and adapt them as appropriate, perhaps handling conversion from an ORM entity to the Protocol Buffer client type as appropriate in that wrapper. HttpMessageConverters may also be used with Spring’s REST client, the RestTemplate. Here’s the appropriate Java-language unit test: package demo; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.IntegrationTest; import org.springframework.boot.test.SpringApplicationConfiguration; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.http.ResponseEntity; import org.springframework.http.converter.protobuf.ProtobufHttpMessageConverter; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.test.context.web.WebAppConfiguration; import org.springframework.web.client.RestTemplate; import java.util.Arrays; @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = DemoApplication.class) @WebAppConfiguration @IntegrationTest public class DemoApplicationTests { @Configuration public static class RestClientConfiguration { @Bean RestTemplate restTemplate(ProtobufHttpMessageConverter hmc) { return new RestTemplate(Arrays.asList(hmc)); } @Bean ProtobufHttpMessageConverter protobufHttpMessageConverter() { return new ProtobufHttpMessageConverter(); } } @Autowired private RestTemplate restTemplate; private int port = 8080; @Test public void contextLoaded() { ResponseEntity customer = restTemplate.getForEntity( "http://127.0.0.1:" + port + "/customers/2", CustomerProtos.Customer.class); System.out.println("customer retrieved: " + customer.toString()); } } Things just work as you’d expect, not only in Java and Spring, but also in Ruby and Python. For completeness, here is a simple client using Ruby (client types omitted): #!/usr/bin/env ruby require './customer.pb' require 'net/http' require 'uri' uri = URI.parse('http://localhost:8080/customers/3') body = Net::HTTP.get(uri) puts Demo::Customer.parse(body) ..and here’s a client in Python (client types omitted): #!/usr/bin/env python import urllib import customer_pb2 if __name__ == '__main__': customer = customer_pb2.Customer() customers_read = urllib.urlopen('http://localhost:8080/customers/1').read() customer.ParseFromString(customers_read) print customer Where to go from Here If you want very high speed message encoding that works with multiple languages, Protocol Buffers are a compelling option. There are other encoding technologies like Avro or Thrift, but none nearly so mature and entrenched as Protocol Buffers. You don’t necessarily need to use Protocol Buffers with REST, either. You could plug it into some sort of RPC service, if that’s your style. There are almost as many client implementations as there are buildpacks for Cloud Foundry - so you could run almost anything on Cloud Foundry and enjoy the same high speed, consistent messaging across all your services! The code for this example is available online, as well, so don’t hesitate to check it out! Also.. Hi gang, in 2015, I’ve been trying to do a random tech-tip style post every week based on things that I see garnering interest in the community, either here or on the Pivotal blog. I use these weekly-_ish_ (OK! OK! - it’s not been easy doing them as regularly as This Week in Spring, but so far I haven’t missed a week! :-) ) posts as a chance to focus not on a specific new release, per se, but on the application of Spring in service to some community use case that might be cross-cutting or just might benefit from having a spotlight shined on it. So far we’ve looked at all manner of things - Vaadin, Activiti, 12-Factor App Style Configuration, Smarter Service to Service Invocations, Couchbase, and much more, etc. - and we’ve got some interesting stuff lined up, too. I wondered what else you want to see talked about, however. If you’ve got some ideas about what you’d like to see covered, or a community post of your own to contribute, reach out to me on Twitter (@starbuxman) or via email (jlong [at] pivotal [dot] io). I remain, as always, at your service.

This post is a guest post by Activiti co-founder and community member Joram Barrez (@jbarrez) who works for Alfresco. Thanks Joram! I’d like to see more of these community guest posts, so - as usual - don’t hesitate to ping me (@starbuxman) with ideas and contributions! -Josh Introduction Activiti is an Apache-licensed business process management (BPM) engine. Such an engine has as core goal to take a process definition comprised of human tasks and service calls and execute those in a certain order, while exposing various API’s to start, manage and query data about process instances for that definition. Contrary to many of its competitors, Activiti is lightweight and integrates easily with any Java technology or project. All that, and it works at any scale - from just a few dozen to many thousands or even millions of process executions. The source code of Activiti can be found on Github. The project was founded and is sponsored by Alfresco, but enjoys contributions from all across the globe and industries. A process definition is typically visualized as a flow-chart-like diagram. In recent years, the BPMN 2.0 standard (an OMG standard, like UML) has become the de-facto ‘language’ of these diagrams. This standard defines how a certain shape on the diagram should be interpreted, both technically and business-wise and how it is stored, as a not-so-hipster XML file.. but luckily most of the tooling hides this for you. This is a standard, and you can use any number of compliant tools to design (and even run) your BPMN processes. That said, if you’re asking me, there is no better choice than Activiti! Spring Boot integration Activiti and Spring play nicely together. The convention-over-configuration approach in Spring Boot works nicely with Activiti’s process engine is setup and use. Out of the box, you only need a database, as process executions can span anywhere from a few seconds to a couple of years. Obviously, as an intrinsic part of a process definition is calling and consuming data to and from various systems with all kinds of technologies. The simplicity of adding the needed dependencies and integrating various pieces of (boiler-plate) logic with Spring Boot really makes this child’s play. Using Spring Boot and Activiti in a microservice approach also makes a lot of sense. Spring Boot makes it easy to get a production-ready service up and running in no time and - in a distributed microservice architecture - Activiti processes can glue together various microservices while also weaving in human workflow (tasks and forms) to achieve a certain goal. The Spring Boot integration in Activiti was created by Spring expert Josh Long. Josh and I did a webinar a couple of months ago that should give you a good insight into the basics of the Activiti integration for Spring Boot. The Activiti user guide section on Spring Boot is also a great starting place to get more information. Getting Started The code for this example can be found in my Github repository. The process we’ll implement here is a hiring process for a developer. It’s simplified of course (as it needs to fit on this web page), but you should get the core concepts. Here’s the diagram: As said in the introduction, all shapes here have a very specific interpretation thanks to the BPMN 2.0 standard. But even without knowledge of BPMN, the process is pretty easy to understand: When the process starts, the resume of the job applicant is stored in an external system. The process then waits until a telephone interview has been conducted. This is done by a user (see the little icon of a person in the corner). If the telephone interview wasn’t all that, a polite rejection email is sent. Otherwise, both a tech interview and financial negotiation should happen. Note that at any point, the applicant can cancel. That’s shown in the diagram as the event on the boundary of the big rectangle. When the event happens, everything inside will be killed and the process halts. If all goes well, a welcome email is sent. This is the BPMN for this process Let’s create a new Maven project, and add the dependencies needed to get Spring Boot, Activiti and a database. We’ll use an in memory database to keep things simple. org.activiti spring-boot-starter-basic ${activiti.version} com.h2database h2 1.4.185 So only two dependencies is what is needed to create a very first Spring Boot + Activiti application: @SpringBootApplication public class MyApp { public static void main(String[] args) { SpringApplication.run(MyApp.class, args); } } You could already run this application, it won’t do anything functionally but behind the scenes it already creates an in-memory H2 database creates an Activiti process engine using that database exposes all Activiti services as Spring Beans configures tidbits here and there such as the Activiti async job executor, mail server, etc. Let’s get something running. Drop the BPMN 2.0 process definition into thesrc/main/resources/processes folder. All processes placed here will automatically be deployed (ie. parsed and made to be executable) to the Activiti engine. Let’s keep things simple to start, and create a CommandLineRunner that will be executed when the app boots up: @Bean CommandLineRunner init( final RepositoryService repositoryService, final RuntimeService runtimeService, final TaskService taskService) { return new CommandLineRunner() { public void run(String... strings) throws Exception { Map variables = new HashMap(); variables.put("applicantName", "John Doe"); variables.put("email", "[email protected]"); variables.put("phoneNumber", "123456789"); runtimeService.startProcessInstanceByKey("hireProcess", variables); } }; } So what’s happening here is that we create a map of all the variables needed to run the process and pass it when starting process. If you’d check the process definition you’ll see we reference those variables using ${variableName} in many places (such as the task description). The first step of the process is an automatic step (see the little cogwheel icon), implemented using an expression that uses a Spring Bean: which is implemented with activiti:expression="${resumeService.storeResume()}" Of course, we need that bean or the process would not start. So let’s create it: @Component public class ResumeService { public void storeResume() { System.out.println("Storing resume ..."); } } When running the application now, you’ll see that the bean is called: . ____ _ __ _ _ /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \ ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \ \\/ ___)| |_)| | | | | || (_| | ) ) ) ) ' |____| .__|_| |_|_| |_\__, | / / / / =========|_|==============|___/=/_/_/_/ :: Spring Boot :: (v1.2.0.RELEASE) 2015-02-16 11:55:11.129 INFO 304 --- [ main] MyApp : Starting MyApp on The-Activiti-Machine.local with PID 304 ... Storing resume ... 2015-02-16 11:55:13.662 INFO 304 --- [ main] MyApp : Started MyApp in 2.788 seconds (JVM running for 3.067) And that’s it! Congrats with running your first process instance using Activiti in Spring Boot! Let’s spice things up a bit, and add following dependency to our pom.xml: org.activiti spring-boot-starter-rest-api ${activiti.version} Having this on the classpath does a nifty thing: it takes the Activiti REST API (which is written in Spring MVC) and exposes this fully in your application. The REST API of Activiti is fully documented in the Activiti User Guide. The REST API is secured by basic auth, and won’t have any users by default. Let’s add an admin user to the system as shown below (add this to the MyApp class). Don’t do this in a production system of course, there you’ll want to hook in the authentication to LDAP or something else. @Bean InitializingBean usersAndGroupsInitializer(final IdentityService identityService) { return new InitializingBean() { public void afterPropertiesSet() throws Exception { Group group = identityService.newGroup("user"); group.setName("users"); group.setType("security-role"); identityService.saveGroup(group); User admin = identityService.newUser("admin"); admin.setPassword("admin"); identityService.saveUser(admin); } }; } Start the application. We can now start a process instance as we did in the CommandLineRunner, but now using REST: curl -u admin:admin -H "Content-Type: application/json" -d '{"processDefinitionKey":"hireProcess", "variables": [ {"name":"applicantName", "value":"John Doe"}, {"name":"email", "value":"[email protected]"}, {"name":"phoneNumber", "value":"1234567"} ]}' http://localhost:8080/runtime/process-instances Which returns us the json representation of the process instance: { "tenantId": "", "url": "http://localhost:8080/runtime/process-instances/5", "activityId": "sid-42BAE58A-8FFB-4B02-AAED-E0D8EA5A7E39", "id": "5", "processDefinitionUrl": "http://localhost:8080/repository/process-definitions/hireProcess:1:4", "suspended": false, "completed": false, "ended": false, "businessKey": null, "variables": [], "processDefinitionId": "hireProcess:1:4" } I just want to stand still for a moment how cool this is. Just by adding one dependency, you’re getting the whole Activiti REST API embedded in your application! Let’s make it even cooler, and add following dependency org.activiti spring-boot-starter-actuator ${activiti.version} This adds a Spring Boot actuator endpoint for Activiti. If we restart the application, and hithttp://localhost:8080/activiti/, we get some basic stats about our processes. With some imagination that in a live system you’ve got many more process definitions deployed and executing, you can see how this is useful. The same actuator is also registered as a JMX bean exposing similar information. { completedTaskCountToday: 0, deployedProcessDefinitions: [ "hireProcess (v1)" ], processDefinitionCount: 1, cachedProcessDefinitionCount: 1, runningProcessInstanceCount: { hireProcess (v1): 0 }, completedTaskCount: 0, completedActivities: 0, completedProcessInstanceCount: { hireProcess (v1): 0 }, openTaskCount: 0 } To finish our coding, let’s create a dedicated REST endpoint for our hire process, that could be consumed by for example a javascript web application (out of scope for this article). So most likely, we’ll have a form for the applicant to fill in the details we’ve been passing programmatically above. And while we’re at it, let’s store the applicant information as a JPA entity. In that case, the data won’t be stored in Activiti anymore, but in a separate table and referenced by Activiti when needed. You probably guessed it by now, JPA support is enabled by adding a dependency: org.activiti spring-boot-starter-jpa ${activiti.version} and add the entity to the MyApp class: @Entity class Applicant { @Id @GeneratedValue private Long id; private String name; private String email; private String phoneNumber; // Getters and setters We’ll also need a Repository for this Entity (put this in a separate file or also in MyApp). No need for any methods, the Repository magic from Spring will generate the methods we need for us. public interface ApplicantRepository extends JpaRepository { // .. } And now we can create the dedicated REST endpoint: @RestController public class MyRestController { @Autowired private RuntimeService runtimeService; @Autowired private ApplicantRepository applicantRepository; @RequestMapping(value="/start-hire-process", method= RequestMethod.POST, produces= MediaType.APPLICATION_JSON_VALUE) public void startHireProcess(@RequestBody Map data) { Applicant applicant = new Applicant(data.get("name"), data.get("email"), data.get("phoneNumber")); applicantRepository.save(applicant); Map variables = new HashMap(); variables.put("applicant", applicant); runtimeService.startProcessInstanceByKey("hireProcessWithJpa", variables); } } Note we’re now using a slightly different process called ‘hireProcessWithJpa’, which has a few tweaks in it to cope with the fact the data is now in a JPA entity. So for example, we can’t use ${applicantName} anymore, but we now have to use ${applicant.name}. Let’s restart the application and start a new process instance: curl -u admin:admin -H "Content-Type: application/json" -d '{"name":"John Doe", "email": "[email protected]", "phoneNumber":"123456789"}' http://localhost:8080/start-hire-process We can now go through our process. You could create a custom endpoints for this too, exposing different task queries with different forms … but I’ll leave this to your imagination and use the default Activiti REST end points to walk through the process. Let’s see which task the process instance currently is at (you could pass in more detailed parameters here, for example the ‘processInstanceId’ for better filtering): curl -u admin:admin -H "Content-Type: application/json" http://localhost:8080/runtime/tasks which returns { "order": "asc", "size": 1, "sort": "id", "total": 1, "data": [{ "id": "14", "processInstanceId": "8", "createTime": "2015-02-16T13:11:26.078+01:00", "description": "Conduct a telephone interview with John Doe. Phone number = 123456789", "name": "Telephone interview" ... }], "start": 0 } So, our process is now at the Telephone interview. In a realistic application, there would be a task list and a form that could be filled in to complete this task. Let’s complete this task (we have to set the telephoneInterviewOutcome variable as the exclusive gateway uses it to route the execution): curl -u admin:admin -H "Content-Type: application/json" -d '{"action" : "complete", "variables": [ {"name":"telephoneInterviewOutcome", "value":true} ]}' http://localhost:8080/runtime/tasks/14 When we get the tasks again now, the process instance will have moved on to the two tasks in parallel in the subprocess (big rectangle): { "order": "asc", "size": 2, "sort": "id", "total": 2, "data": [ { ... "name": "Tech interview" }, { ... "name": "Financial negotiation" } ], "start": 0 } We can now continue the rest of the process in a similar fashion, but I’ll leave that to you to play around with. Testing One of the strengths of using Activiti for creating business processes is that everything is simply Java. As a consequence, processes can be tested as regular Java code with unit tests. Spring Boot makes writing such test a breeze. Here’s how the unit test for the “happy path” looks like (while omitting @Autowired fields and test e-mail server setup). The code also shows the use of the Activiti API’s for querying tasks for a given group and process instance. @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = {MyApp.class}) @WebAppConfiguration @IntegrationTest public class HireProcessTest { @Test public void testHappyPath() { // Create test applicant Applicant applicant = new Applicant("John Doe", "[email protected]", "12344"); applicantRepository.save(applicant); // Start process instance Map variables = new HashMap(); variables.put("applicant", applicant); ProcessInstance processInstance = runtimeService.startProcessInstanceByKey("hireProcessWithJpa", variables); // First, the 'phone interview' should be active Task task = taskService.createTaskQuery() .processInstanceId(processInstance.getId()) .taskCandidateGroup("dev-managers") .singleResult(); Assert.assertEquals("Telephone interview", task.getName()); // Completing the phone interview with success should trigger two new tasks Map taskVariables = new HashMap(); taskVariables.put("telephoneInterviewOutcome", true); taskService.complete(task.getId(), taskVariables); List tasks = taskService.createTaskQuery() .processInstanceId(processInstance.getId()) .orderByTaskName().asc() .list(); Assert.assertEquals(2, tasks.size()); Assert.assertEquals("Financial negotiation", tasks.get(0).getName()); Assert.assertEquals("Tech interview", tasks.get(1).getName()); // Completing both should wrap up the subprocess, send out the 'welcome mail' and end the process instance taskVariables = new HashMap(); taskVariables.put("techOk", true); taskService.complete(tasks.get(0).getId(), taskVariables); taskVariables = new HashMap(); taskVariables.put("financialOk", true); taskService.complete(tasks.get(1).getId(), taskVariables); // Verify email Assert.assertEquals(1, wiser.getMessages().size()); // Verify process completed Assert.assertEquals(1, historyService.createHistoricProcessInstanceQuery().finished().count()); } Next steps We haven’t touched any of the tooling around Activiti. There is a bunch more than just the engine, like the Eclipse plugin to design processes, a free web editor in the cloud (also included in the .zip download you can get from Activiti's site, a web application that showcases many of the features of the engine, … The current release of Activiti (version 5.17.0) has integration with Spring Boot 1.1.6. However, the current master version is compatible with 1.2.1. Using Spring Boot 1.2.0 brings us sweet stuff like support for XA transactions with JTA. This means you can hook up your processes easily with JMS, JPA and Activiti logic all in the same transaction! ..Which brings us to the next point … In this example, we’ve focussed heavily on human interactions (and barely touched it). But there’s many things you can do around orchestrating systems too. The Spring Boot integration also has Spring Integration support you could leverage to do just that in a very neat way! And of course there is much much more about the BPMN 2.0 standard. Read more about itin the Activiti docs.

What does implementing microservices mean for a software developer? Especially, for the rookies, greenhorns, and newbs out there? I’m not talking about microservice software architecture here; this is about microservices software development. And not just that, the ultimate implementation goal should be “microservices done right”. For this post, I’ll go with Java. Yes, it’s wordy. Yes, it’s resource intensive (especially when used for the sole purpose of returning a single string). However the concept of classes and objects goes well with my intention of explaining how to do microservices correctly. Plus, it makes sense to use microservices in environments that are heavily biased towards Java. Anyway, please feel free to add your own “Hello, World!” microservice in your favorite language in the comments section below. Hello, monolith! As a prerequisite, you should be familiar with the following piece of code, what it does, and why it has to look the way it does (read this tutorial if you don’t): class Starter { public static void main(String[] args) { System.out.println(“Hello, World!”); } } This is a simple console application that yields the string “Hello, World!” This is not written in the microservice way. This is an example of when not to use the microservices approach: if all you need on your console is a single string, this is all you need. Hello, code duplication! In addition to this console application, I want this string to be available on the web by calling http://localhost:80/helloWorld.servlet from a browser. Here is the required code, implemented as plain HTTP servlet (yes, it’s wordy. Get over it.) class HelloWorldServlet extends HttpServlet { public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { response.getWriter().println(“Hello, World!”); } } The string “Hello, World!” has to be “implemented” again. Sure, this is no big deal. But this simple string could be so much more. It could be the result of a complex calculation or it could be the result of a time consuming search query. So, just imagine that the string “Hello, World!” is the result of a week’s worth of hard work (If you’re new to programming, it may very well be...). How should you go about making it available to apps and services that you create? Step 1: HelloWorldService.java To save yourself from duplicating a week’s worth of coding, allow me to introduce the HelloWorldService class: class HelloWorldService { public String greet() { return “Hello, World!”; } } You can re-use this fine piece of software craftmanship in all your apps and classes without re-implementing or duplicating code. Here’s our console application again: class Starter { HelloWorldService helloWorldService = new HelloWorldService(); public static void main(String[] args) { String message = helloWorldService.greet(); System.out.println(message); } } The same goes for servlets: class HelloWorldServlet extends HttpServlet { HelloWorldService helloWorldService = new HelloWorldService(); public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { String message = helloWorldService.greet(); response.getWriter().println(message); } } It also works great for Spring MVC controllers: @Controller class HelloWorldController { HelloWorldService helloWorldService = new HelloWorldService(); @RequestMapping("/helloWorld") public String greet() { String message = helloWorldService.greet(); return message; } } I could go on and show more examples, but I think you get the point (spoiler: it’s the bold lines that matter). Those of you who are familiar with microservices could point out that this may be fine for getting rid of code duplication, but this is no microservice. You’re right, but to get to “microservices done right,” you have to be able to separate you app’s concerns, which is what I did here in the most possible basic way: I separated the app’s frontend concerns from its backend concerns. The frontend is either a console app or a servlet, the backend is HelloWorldService. Serviceward, ho! To go down microservice lane from here, all we have to do is wrap HelloWorldService into some kind of web component that makes it accessible via HTTP, right? Let’s see… First, we could just use our servlet code from above, as it conveniently returns the string as a response to any HTTP request. But we won’t. Why? Because there’s something missing: fault tolerance. What could possibly fail when returning a simple string? That’s not the point. What matters is that the client side (the code that calls HelloWorldService) should be given enough information to effectively react to failures. We face two possible problems: The service as a whole may be unavailable The service may be unable to return a proper response The service is unavailable If a service is unavailable, it’s the client that is responsible for dealing with the situation. Frameworks like unirest.io save you the effort of writing many lines of code when dealing with HTTP requests. Future> future = Unirest.post("HTTP://helloworld.myservices.local/greet") .header("accept", "application/json") .asJsonAsync(new Callback() { public void failed(UnirestException e) { //tell them UI folks that the request went south } public void completed(HttpResponse response) { //extract data from response and fulfill it’s destiny } public void cancelled() { //shot a note to UI dept that the request got cancelled } } ); With this code, the client now knows when the service is not available or has timed out following no response. Wee can easily have an error message displayed in place of the string we expected to receive. Try/catch is probably the right solution here. Invalid responses however pose more of a challenge. The service fails If the service fails, we can just return a string with an appropriate error message. But how can you know if a message is an error message or a correct response? Yes, you can start every error message with [ERROR] or invent another “smart” (read: not-so-smart) workaround, but this won’t be a solution you’ll be proud of. And, there’s always the possibility that even valid responses may begin with ERROR because it’s simply part of the message. I’d go with JSON or XML for wrapping the answer. I prefer JSON because it’s a little less wordy than XML. And I really like using the JSON-HTML tool over at json.bloople.net for visualizing results during development. Of course, you might go for any of the numerous alternatives, like protobuf or a proprietary solution of your own. The main point is that you need to be able to apply structure to responses: { “status”:”ok”, ”message”:”Hello, World!” } By checking the status attribute, you can easily decide whether to handle an error or to display an appropriate message. { “status”:”error”, ”message”:”Invalid input parameter” } The possibilities are endless here. You can add an error code or additional properties. This all boils down to a single important point: apply structure to your responses. Structure, why? Because structure not only helps you keep your code maintainable, it also serves as the foundation of the API of your service. An API definition consists of more than a URL like this: GET HTTP://helloworld.myservices.local/greet API definitions also consist of the response structures that can be expected as a response (you know this already from a few lines back): { “status”:”ok”, ”message”:”Hello, World!” } Most important takeaway Keeping the API specifications of a service’s request and response stable is a key requirement for succeeding with microservices. Conclusion Are you (and your project) ready for microservices? If you read this and kept asking yourself, what good is all the overhead of microservices, then either your project won’t benefit from microservices or you’re just not there yet (for mindset perspective see my previous post about the value of microservices ). If you can’t stop thinking about microservices, then you probably are ready.

Retry-After is a lesser known HTTP response header.

I've been intrigued by JHipster ever since I first tried it last September. I'd worked with AngularJS and Spring Boot quite a bit, and I liked the idea that someone had combined them, adding some nifty features along the way. When I spoke about AngularJS earlier this month, I included a few slides on JHipster near the end of the presentation. This week, I received an email from someone who attended that presentation. Hey Matt, We met a few weeks back when you presented at DOSUG. You were talking about JHipster which I had been eyeing for a few months and wanted your quick .02 cents. I have built a pretty heavy application over the last 6 months that is using mostly the same tech as JHipster. Java Spring JPA AngularJS Compass Grunt It's ridiculously close for most of the tech stack. So, I was debating rolling it over into a JHipster app to make it a more familiar stack for folks. My concern is that it I will spend months trying to shoehorn it in for not much ROI. Any thoughts on going down this path? What are the biggest issues you've seen in using JHipster? It seems pretty straightforward except for the entity generators. I'm concerned they are totally different than what I am using. The main difference in what I'm doing compared to JHipster is my almost complete use of groovy instead of old school Java in the app. I would have to be forced into going back to regular java beans... Thoughts? I replied with the following advice: JHipster is great for starting a project, but I don't know that it buys you much value after the first few months. I would stick with your current setup and consider JHipster for your next project. I've only prototyped with it, I haven't created any client apps or put anything in production. I have with Spring Boot and AngularJS though, so I like that JHipster combines them for me. JHipster doesn't generate Scala or Groovy code, but you could still use them in a project as long as you had Maven/Gradle configured properly. You might try generating a new app with JHipster and examine how they're doing this. At the very least, it can be a good learning tool, even if you're not using it directly. Java Hipsters: Do you agree with this advice? Have you tried migrating an existing app to JHipster? Are any of you using Scala or Groovy in your JHipster projects?

Written by Dave Syer in the Spring blog In this article we continue our discussion of how to use Spring Security with Angular JS in a “single page application”. Here we show how to build an API Gateway to control the authentication and access to the backend resources using Spring Cloud. This is the fourth in a series of articles, and you can catch up on the basic building blocks of the application or build it from scratch by reading the first article, or you can just go straight to the source code in Github. In the last article we built a simple distributed application that used Spring Session to authenticate the backend resources. In this one we make the UI server into a reverse proxy to the backend resource server, fixing the issues with the last implementation (technical complexity introduced by custom token authentication), and giving us a lot of new options for controlling access from the browser client. Reminder: if you are working through this article with the sample application, be sure to clear your browser cache of cookies and HTTP Basic credentials. In Chrome the best way to do that for a single server is to open a new incognito window. Creating an API Gateway An API Gateway is a single point of entry (and control) for front end clients, which could be browser based (like the examples in this article) or mobile. The client only has to know the URL of one server, and the backend can be refactored at will with no change, which is a significant advantage. There are other advantages in terms of centralization and control: rate limiting, authentication, auditing and logging. And implementing a simple reverse proxy is really simple with Spring Cloud. If you were following along in the code, you will know that the application implementation at the end of the last article was a bit complicated, so it’s not a great place to iterate away from. There was, however, a halfway point which we could start from more easily, where the backend resource wasn’t yet secured with Spring Security. The source code for this is a separate project in Github so we are going to start from there. It has a UI server and a resource server and they are talking to each other. The resource server doesn’t have Spring Security yet so we can get the system working first and then add that layer. Declarative Reverse Proxy in One Line To turn it into an API Gateawy, the UI server needs one small tweak. Somewhere in the Spring configuration we need to add an @EnableZuulProxy annotation, e.g. in the main (only)application class: @SpringBootApplication @RestController @EnableZuulProxy public class UiApplication { ... } and in an external configuration file we need to map a local resource in the UI server to a remote one in the external configuration (“application.yml”): security: ... zuul: routes: resource: path: /resource/** url: http://localhost:9000 This says “map paths with the pattern /resource/** in this server to the same paths in the remote server at localhost:9000”. Simple and yet effective (OK so it’s 6 lines including the YAML, but you don’t always need that)! All we need to make this work is the right stuff on the classpath. For that purpose we have a few new lines in our Maven POM: org.springframework.cloud spring-cloud-starter-parent 1.0.0.BUILD-SNAPSHOT pom import org.springframework.cloud spring-cloud-starter-zuul ... Note the use of the “spring-cloud-starter-zuul” - it’s a starter POM just like the Spring Boot ones, but it governs the dependencies we need for this Zuul proxy. We are also using because we want to be able to depend on all the versions of transitive dependencies being correct. Consuming the Proxy in the Client With those changes in place our application still works, but we haven’t actually used the new proxy yet until we modify the client. Fortunately that’s trivial. We just need to go from this implementation of the “home” controller: angular.module('hello', [ 'ngRoute' ]) ... .controller('home', function($scope, $http) { $http.get('http://localhost:9000/').success(function(data) { $scope.greeting = data; }) }); to a local resource: angular.module('hello', [ 'ngRoute' ]) ... .controller('home', function($scope, $http) { $http.get('resource/').success(function(data) { $scope.greeting = data; }) }); Now when we fire up the servers everything is working and the requests are being proxied through the UI (API Gateway) to the resource server. Further Simplifications Even better: we don’t need the CORS filter any more in the resource server. We threw that one together pretty quickly anyway, and it should have been a red light that we had to do anything as technically focused by hand (especially where it concerns security). Fortunately it is now redundant, so we can just throw it away, and go back to sleeping at night! Securing the Resource Server You might remember in the intermediate state that we started from there is no security in place for the resource server. Aside: Lack of software security might not even be a problem if your network architecture mirrors the application architecture (you can just make the resource server physically inaccessible to anyone but the UI server). As a simple demonstration of that we can make the resource server only accessible on localhost. Just add this to application.properties in the resource server: server.address: 127.0.0.1 Wow, that was easy! Do that with a network address that’s only visible in your data center and you have a security solution that works for all resource servers and all user desktops. Suppose that we decide we do need security at the software level (quite likely for a number of reasons). That’s not going to be a problem, because all we need to do is add Spring Security as a dependency (in the resource server POM): org.springframework.boot spring-boot-starter-security That’s enough to get us a secure resource server, but it won’t get us a working application yet, for the same reason that it didn’t in Part III: there is no shared authentication state between the two servers. Sharing Authentication State We can use the same mechanism to share authentication (and CSRF) state as we did in the last, i.e. Spring Session. We add the dependency to both servers as before: org.springframework.session spring-session 1.0.0.RELEASE org.springframework.boot spring-boot-starter-redis but this time the configuration is much simpler because we can just add the same Filterdeclaration to both. First the UI server (adding @EnableRedisHttpSession): @SpringBootApplication @RestController @EnableZuulProxy @EnableRedisHttpSession public class UiApplication { ... } and then the resource server. There are two changes to make: one is adding@EnableRedisHttpSession and a HeaderHttpSessionStrategy bean to theResourceApplication: @SpringBootApplication @RestController @EnableRedisHttpSession class ResourceApplication { ... @Bean HeaderHttpSessionStrategy sessionStrategy() { new HeaderHttpSessionStrategy(); } } and the other is to explicitly ask for a non-stateless session creation policy inapplication.properties: security.sessions: NEVER As long as redis is still running in the background (use the fig.yml if you like to start it) then the system will work. Load the homepage for the UI at http://localhost:8080 and login and you will see the message from the backend rendered on the homepage. How Does it Work? What is going on behind the scenes now? First we can look at the HTTP requests in the UI server (and API Gateway): VERB PATH STATUS RESPONSE GET / 200 index.html GET /css/angular-bootstrap.css 200 Twitter bootstrap CSS GET /js/angular-bootstrap.js 200 Bootstrap and Angular JS GET /js/hello.js 200 Application logic GET /user 302 Redirect to login page GET /login 200 Whitelabel login page (ignored) GET /resource 302 Redirect to login page GET /login 200 Whitelabel login page (ignored) GET /login.html 200 Angular login form partial POST /login 302 Redirect to home page (ignored) GET /user 200 JSON authenticated user GET /resource 200 (Proxied) JSON greeting That’s identical to the sequence at the end of Part II except for the fact that the cookie names are slightly different (“SESSION” instead of “JSESSIONID”) because we are using Spring Session. But the architecture is different and that last request to “/resource” is special because it was proxied to the resource server. We can see the reverse proxy in action by looking at the “/trace” endpoint in the UI server (from Spring Boot Actuator, which we added with the Spring Cloud dependencies). Go tohttp://localhost:8080/trace in a browser and scroll to the end (if you don’t have one already get a JSON plugin for your browser to make it nice and readable). You will need to authenticate with HTTP Basic (browser popup), but the same credentials are valid as for your login form. At or near the end you should see a pair of requests something like this: { "timestamp": 1420558194546, "info": { "method": "GET", "path": "/", "query": "" "remote": true, "proxy": "resource", "headers": { "request": { "accept": "application/json, text/plain, */*", "x-xsrf-token": "542c7005-309c-4f50-8a1d-d6c74afe8260", "cookie": "SESSION=c18846b5-f805-4679-9820-cd13bd83be67; XSRF-TOKEN=542c7005-309c-4f50-8a1d-d6c74afe8260", "x-forwarded-prefix": "/resource", "x-forwarded-host": "localhost:8080" }, "response": { "Content-Type": "application/json;charset=UTF-8", "status": "200" } }, } }, { "timestamp": 1420558200232, "info": { "method": "GET", "path": "/resource/", "headers": { "request": { "host": "localhost:8080", "accept": "application/json, text/plain, */*", "x-xsrf-token": "542c7005-309c-4f50-8a1d-d6c74afe8260", "cookie": "SESSION=c18846b5-f805-4679-9820-cd13bd83be67; XSRF-TOKEN=542c7005-309c-4f50-8a1d-d6c74afe8260" }, "response": { "Content-Type": "application/json;charset=UTF-8", "status": "200" } } } }, The second entry there is the request from the client to the gateway on “/resource” and you can see the cookies (added by the browser) and the CSRF header (added by Angular as discussed inPart II). The first entry has remote: true and that means it’s tracing the call to the resource server. You can see it went out to a uri path “/” and you can see that (crucially) the cookies and CSRF headers have been sent too. Without Spring Session these headers would be meaningless to the resource server, but the way we have set it up it can now use those headers to re-constitute a session with authentication and CSRF token data. So the request is permitted and we are in business! Conclusion We covered quite a lot in this article but we got to a really nice place where there is a minimal amount of boilerplate code in our two servers, they are both nicely secure and the user experience isn’t compromised. That alone would be a reason to use the API Gateway pattern, but really we have only scratched the surface of what that might be used for (Netflix uses it for a lot of things). Read up on Spring Cloud to find out more on how to make it easy to add more features to the gateway. The next article in this series will extend the application architecture a bit by extracting the authentication responsibilities to a separate server (the Single Sign On pattern).

Microservices is a new software architecture and delivery paradigm, where applications are composed of several small runtime services. The current mainstream approach for software delivery is to build, integrate, and test entire applications as a monolith. This approach requires any software change, however small, to require a full test cycle of the entire application. With Microservices a software module is delivered as an independent runtime service with a well defined API. The Microservices approach allow faster delivery of smaller incremental changes to an application. There are several tradeoffs to consider with the Microservices architecture. On one hand, the Microservices approach builds on several best practices and patterns for software design, architecture, and DevOps style organization. On the other hand, Microservices requires expertise in distributed programming and can become an operational nightmare without proper tooling in place. There are several good posts that highlight the pros-and-cons of Microservices, and I have added in the references section. In the remainder of this post, I will define five architectural constraints (principles that drive desired properties) for the Microservices architectural style. To be a Microservice, a service must be: Elastic Resilient Composable Minimal, and; Complete Microservice Constraint #1 - Elastic A microservice must be able to scale, up or down, independently of other services in the same application. This constraint implies that based on load, or other factors, you can fine tune your applications performance, availability, and resource usage. This constraint can be realized in different ways, but a popular pattern is to architect the system so that you can run multiple stateless instances of each microservice, and there is a mechanism for Service naming, registration, and discovery along with routing and load-balancing of requests. Microservice Constraint #2 - Resilient A microservice must fail without impacting other services in the same application. A failure of a single service instance should have minimal impact on the application. A failure of all instances of a microservice, should only impact a single application function and users should be able to continue using the rest of the application without impact. Adrian Cockroft describes Microservices as loosely coupled service oriented architecture with bounded contexts [3]. To be resilient a service has to be loosely coupled with other services, and a bounded context limits a service’s failure domain. Microservice Constraint #3 - Composable A microservice must offer an interface that is uniform and is designed to support service composition. Microservice APIs should be designed with a common way of identifying, representing, and manipulating resources, describing the API schema and supported API operations. The ‘Uniform Interfaces constraint of the REST architectural style describes this in detail. Service Composition is a SOA principle that has fairly obvious benefits, but few guidelines on how it can be achieved. A Microservice interface should be designed to support composition patterns like aggregation, linking, and higher-level functions such as caching, proxies and gateways. I previously discussed REST constraints and elements in as two part blog post: REST is not about APIs Microservice Constraint #4 - Minimal A microservice must only contain highly cohesive entities In software, cohesion is a measure of whether things belong together. A module is said to have high cohesion if all objects and functions in it are focused on the same tasks. Higher cohesion leads to more maintainable software. A Microservice should perform a single business function, which implies that all of its components are highly cohesive. This is also an Single Responsibility Principle (SRP) of object-oriented design [5] Microservice Constraint #5 - Complete A microservice must be functionally complete Bjarne Stroustrup, the creator of C++, stated that a good interface must be, “minimal but complete” i.e. as small as possible, and no smaller. Similarly, a Microservice must offer a complete function, with minimal dependencies (loose coupling) to other services in the application. This is important, as otherwise its becomes impossible to version and upgrade individual services. This constraint is designed to oppose the minimal constraint. Put together a microservice must be “minimal but complete.” Conclusions Designing a Microservices application requires application of several principles, patterns, and best practices of modular design and service-oriented architectures. In this post, I've outlined five architectural constraints which can help guide and retain the key benefits of a Microservices-style architecture. For example, Microservices Constraint# 1 - Elastic steers implementations towards separating the data tier from the application tier, and leads to stateless services. At Nirmata we have built our solution, that makes it easy to deploy and operate microservices applications, using these very same principles. We believe that Microservices style applications, running in containers, will power the next generation of software innovation. If you are using, or interested in using microservices, I would love to hear from you. Jim Bugwadia Founder and CEO Nirmata -- For additional content and articles follow us at @NirmataCloud. -- If you are in the San Francisco Bay Area, come join our Microservices meetup group. References [1] Microservices, Martin Fowler and James Lewis, http://martinfowler.com/articles/microservices.html [2] Microservices Are Not a free lunch!, Benjamin Wootton, http://contino.co.uk/microservices-not-a-free-lunch/ [3] State of the Art in Microservices, Adrian Cockroft, http://thenewstack.io/dockercon-europe-adrian-cockcroft-on-the-state-of-microservices/ [4] The Principles of Object-Oriented Design, Robert C. Martin, http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod

I recently came across this requirement where a xsd:datetime in the payload is needed to be converted to a different date time format as follows, Original format : 2015-01-07T09:30:10+02:00 Required date: 2015/01/07 09:30:10 In WSO2 ESB, I found that this transformation can be achieved through a XSLT mediator, class mediator or a script mediator. In an overview, XSLT mediator uses a XSL stylesheet to format the xml payload passed to the mediator whereas in class mediator and script mediator we use java code and javascript code respectively to manipulate the message context. In this blog post I am going to present how this transformation can be achieved by means of the XSLT mediator. XSL Stylesheet Proxy configuration dateTime.xsl XLS style sheet is stored as an inline xml local entry in ESB. In the proxy, the original date is passed as an parameter ("date_time") to the XLS style sheet. I have used format-dateTime function, a function of XSL 2.0, to do the transformation. Sample request 2015-01-07T09:30:10+02:00 Console output 2015/01/07 09:30:10 GMT+2

Of late, I have been looking into Microservice containers that are available out there to help speed up the development. Although, Microservice is a generic term however there is some consensus with respect to what it means. Hence, we may conveniently refer to the definition Microservice as an "architectural design pattern, in which complex applications are composed of small, independent processes communicating with each other using language-agnostic APIs. These services are small, highly decoupled and focus on doing a small task." There are several Microservice containers out there. However, in my experience I have found Dropwizard and Spring-boot to have had received more attention and they appear to be widely used compared to the rest. In my current role, I was asked create a comparison matrix between the two, so it's here below. Dropwizard Spring-Boot What is it? Dropwizard pulls together stable, mature libraries from the Java ecosystem into a simple, light-weight package that lets you focus on getting things done. [more...] Takes an opinionated view of building production-ready Spring applications. Spring Boot favours convention over configuration and is designed to get you up and running as quickly as possible. [more...] Overview? Dropwizard straddles the line between being a library and a framework. Provide performant, reliable implementations of everything a production-ready web application needs. [more...] Spring-boot takes an opinionated view of the Spring platform and third-party libraries so you can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration. [more...] Out of the box features? Dropwizard has out-of-the-box support for sophisticated configuration, application metrics, logging, operational tools, and much more, allowing you and your team to ship a production-quality web service in the shortest time possible. [more...] Spring-boot provides a range of non-functional features that are common to large classes of projects (e.g. embedded servers, security, metrics, health checks, externalized configuration). [more...] Libraries Core: Jetty, Jersey, Jackson and Matrics Others: Guava, Liquibase and Joda Time. Spring, JUnit, Logback, Guava. There are several starter POM files covering various use cases, which can be included in the POM to get started. Dependency Injection? No built in Dependency Injection. Requires a 3rd party dependency injection framework such as Guice, CDI or Dagger. [Ref...] Built in Dependency Injection provided by Spring Dependency Injection container. [Ref...] Types of Services i.e. REST, SOAP Has some support for other types of services but primarily is designed for performant HTTP/REST LAYER. If ever need to integrate SOAP, there is a dropwizard bundle for building SOAP web services using JAX-WS API is provided here but it’s not official drop-wizard sub project. [more...] As well as supporting REST Spring-boot has support for other types of services such as JMS, Advanced Message Queuing Protocol, SOAP based Web Services to name a few. [more...] Deployment? How it creates the Executable Jar? Uses Shading to build executable fat jars, where a shaded jar spackages all classes, from all jars, into a single 'uber jar'. [Ref...] Spring-boot adopts a different approach and avoids shaded jars, as it becomes hard to see which libraries you are actually using in your application. It can also be problematic if the same filename is used in Shaded jars. Instead it uses “Nested Jar” approach where all classes from all jars do not need to be included into a single “uber jar” instead all dependent jars should be in the “lib” folder, spring loader loads them appropriately. [Ref...] Contract First Web Services? No built in support. Would have to refer to 3rd party library (CXF or any other JAX-WS implementation) if needed a solution for the Contract First SOAP based services. Contract First services support is available with the help of spring-boot-starter-ws starter application. [Ref...] Externalised Configuration for properties and YAML Supports both Properties and YAML Supports both Properties and YAML Concluding Remarks If dealing with only REST micro services, drop wizard is an excellent choice. Where Spring-boot shines is the types of services supported i.e. REST, JMS, Messaging, and Contract First Services. Not least a fully built in Dependency Injection container. Disclaimer: The matrix is purely based on my personal views and experiences, having tried both frameworks and is by no means an exhaustive guide. Readers are requested to do their own research before making a strategic decision between the two very formidable frameworks.

In this article I will attempt to run the Mule ESB community edition in Docker in order to see whether it is feasible without any greater inconvenience. My goal is to be able to use Docker both when testing as well as in a production environment in order to gain better control over the environment and to separate different types of environments. I imagine that most of the Docker-related information can be applied to other applications – I have used Mule since it is what I usually work with. The conclusion I have made after having completed my experiments is that it is possible to run Mule ESB in Docker without any inconvenience. In addition, Docker will indeed allow me to have better control over the different environments and also allow me to separate them as I find appropriate. Finally, I just want to mention that I have used Docker in an Ubuntu environment. I have not attempted any of the exercises in Docker running on Windows or Mac OS X. Docker Briefly In short, Docker allows for creating of images that serve as blueprints for containers. A Docker container is an instance of a Docker image in the same way a Java object is an instance of a Java class. FROM codingtony/java MAINTAINER tony(dot)bussieres(at)ticksmith(dot)com RUN wget https://repository.mulesoft.org/nexus/content/repositories/releases/org/mule/distributions/mule-standalone/3.5.0/mule-standalone-3.5.0.tar.gz RUN cd /opt && tar xvzf ~/mule-standalone-3.5.0.tar.gz RUN echo "4a94356f7401ac8be30a992a414ca9b9 /mule-standalone-3.5.0.tar.gz" | md5sum -c RUN rm ~/mule-standalone-3.5.0.tar.gz RUN ln -s /opt/mule-standalone-3.5.0 /opt/mule CMD [ "/opt/mule/bin/mule" ] The resource isolation features of Linux are used to create Docker containers, which are more lightweight than virtual machines and are separated from the environment in which Docker runs, the host. Using Docker an image can be created that, every time it is started has a known state. In order to remove any doubts about whether the environment has been altered in any way, the container can be stopped and a new container started. I can even run multiple Docker containers on one and the same computer to simulate a multi-server production environment. Applications can also be run in their own Docker containers, as shown in this figure. Three Docker containers, each containing a specific application, running in one host. A more detailed introduction to Docker is available here. The main entry point to the Docker documentation can be found here. Motivation Some of the motivations I have for using Docker in both testing and production environments are: The environment in which I test my application should be as similar as the final deployment environment as possible, if not identical. Making the deployment environment easy to scale up and down. If it is easy to start a new processing node when need arise and stop it if it is no longer used, I will be able to adapt to changes rather quickly and thus reduce errors caused by, for instance, load peaks. Maintain an increased number of nodes to which applications can be deployed. Instead of running one instance of some kind of application server, Mule ESB in my case, on a computer, I want multiple instances that are partitioned, for instance, according to importance. High-priority applications run on one separate instance, which have higher priority both as far as resources (CPU, memory, disk etc) are concerned but also as far as support is concerned. Applications which are less critical run on another instance. Enable quick replacement of instances in the deployment environment. Reasons for having to replace instances may be hardware failure etc. Better control over the contents of the different environments. The concept of an environment that, at any time, may be disposed (and restarted) discourages hacks in the environment, which are usually poorly documented and sometimes difficult to trace. Using Docker, I need to change the appropriate Docker image if I want to make changes to some application environment. The Docker image file, commonly known as Dockerfile, can be checked into any ordinary revision control system, such as Git, Subversion etc, making changes reversible and traceable. Automate the creation of a testing environment. An example could be a nightly job that runs on my build server which creates a test environment, deploys one or more applications to it and then performs tests, such as load-testing. Prerequisites To get the best possible experience when running Docker, I run it under Ubuntu. According to the current documentation, Docker is supported under the following versions of Ubuntu: 12.04 LTS (64-bit) 13.04 (64-bit) 13.10 (64-bit) 14.04 (64-bit) Against my usual conservative self, I chose Ubuntu 14.10, which at the time of writing this article is the latest version. While I haven’t run into any issues, I cannot promise anything regarding compatibility with Docker as far as this version of Ubuntu is concerned. Installing Docker Before we install anything, those who have the Docker version from the Ubuntu repository should remove this version before installing a newer version of Docker, since the Ubuntu repository does not contain the most recent version and the package does not have the same name as the Docker package we will install: sudo apt-get remove docker.io The simplest way to install Docker is to use an installation script made available at the Docker website: curl -sSL https://get.docker.com/ubuntu/ | sudo sh If you are not running Ubuntu or if you do not want to use the above way of installing Docker, please refer to this page containing instructions on how to install Docker on various platforms. To verify the Docker installation, open a terminal window and enter: sudo docker version Output similar to the following should appear: Client version: 1.4.1 Client API version: 1.16 Go version (client): go1.3.3 Git commit (client): 5bc2ff8 OS/Arch (client): linux/amd64 Server version: 1.4.1 Server API version: 1.16 Go version (server): go1.3.3 Git commit (server): 5bc2ff8 We are now ready to start a Mule instance in Docker. Running Mule in Docker One of the advantages with Docker is that there is a large repository of Docker images that are ready to be used, and even extended if one so wishes. ThisDocker image is the one that I will use in this article. It is well documented, there is a source repository and it contains a recent version of the Mule ESB Community Edition. Some additional details on the Docker image: Ubuntu 14.04. Oracle JavaSE 1.7.0_65. This version will change as the PPA containing the package is updated. Mule ESB CE 3.5.0 Note that the image may change at any time and the specifications above may have changed. If you intend to use Docker in your organization, I would suspect that the best alternative is to create your own Docker images that are totally under your control. The Docker image repository is an excellent source of inspiration and aid even in this case. Starting a Docker Container To start a Docker container using this image, open a terminal window and write: sudo docker run codingtony/mule The first time an image is used it needs to be downloaded and created. This usually takes quite some time, so I suggest a short break here – perhaps for a cup of coffee or tea. If you just want to download an image without starting it, exchange the Docker command “run” with “pull”. Once the container is started, you will see some output to the console. If you are familiar with Mule, you will recognize the log output: MULE_HOME is set to /opt/mule-standalone-3.5.0 Running in console (foreground) mode by default, use Ctrl-C to exit... MULE_HOME is set to /opt/mule-standalone-3.5.0 Running Mule... --> Wrapper Started as Console Launching a JVM... Starting the Mule Container... Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. INFO 2015-01-05 04:41:42,302 [WrapperListener_start_runner] org.mule.module.launcher.MuleContainer: ********************************************************************** * Mule ESB and Integration Platform * * Version: 3.5.0 Build: ff1df1f3 * * MuleSoft, Inc. * * For more information go to http://www.mulesoft.org * * * * Server started: 1/5/15 4:41 AM * * JDK: 1.7.0_65 (mixed mode) * * OS: Linux (3.16.0-28-generic, amd64) * * Host: f95698cfb796 (172.17.0.2) * ********************************************************************** Note that: In the text-box containing information about the Mule ESB and Integration Platform, there is a row which starts with “Host:”. The hexadecimal digit that follows is the Docker container id and the IP-address is the external IP-address of the Docker container in which Mule is running. Before we do anything with the Mule instance running in Docker, let’s take a look at Docker containers. Docker Containers We can verify that there is a Docker container running by opening another terminal window, or a tab in the first terminal window, and running the command: sudo docker ps As a result, you will see output similar to the following (I have edited the output in order for the columns to be aligned with the column titles): CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f95698cfb796 codingtony/mule:latest "/opt/mule/bin/mule" 7 min ago Up 7 min jolly_hopper From this output we can see that: The ID of the container is f95698cfb796. This ID can be used when performing operations on the container, such as stopping it, restarting it etc. The name of the image used to created the container. The command that is currently executing. If we look at the Dockerfile for the image, we can see that the last line in this file is: CMD [ “/opt/mule/bin/mule” ] This is the command that is executed whenever an instance of the Docker image is launched and it matches what we see in the COMMAND column for the Docker container. The CREATED column shows how much time has passed since the container was created. The STATUS column shows the current status of the image. When you have used Docker for a while, you can view all the containers using: sudo docker ps -a This will show you containers that are not running, in addition to the running ones. Containers that are not running can be restarted. The PORTS column shows any port mappings for the container. More about port mappings later. Finally, the NAMES column contain a more human-friendly container name. This container name can be used in the same way as the container id. Docker containers will consume disk-space and if you want to determine how much disk-space each of the containers on your computer use, issue the following command: sudo docker ps -a -s An additional column, SIZE, will be shown and in this column I see that my Mule container consumes 41,76kB. Note that this is in addition to the disk-space consumed by the Docker image. This number will grow if you use the container under a longer period of time, as the container retains any files written to disk. To completely remove a stopped Docker container, find the id or name of the container and use the command: sudo docker rm [container id or name here] Before going further, let’s stop the running container and remove it: sudo docker stop [container id or name here] sudo docker rm [container id or name here] Files and Docker Containers So far we have managed to start a Mule instance running inside a Docker container, but there were no Mule applications deployed to it and the logs that were generated were only visible in the terminal window. I want to be able to deploy my applications to the Mule instance and examine the logs in a convenient way. In this section I will show how to: Share one or more directories in the host file-system with a Docker container. Access the files in a Docker container from the host. As the first step in looking at sharing directories between the host operating system and a Docker container, we are going to look at Mule logs. As part of this exercise we also set up the directories in the host operating system that are going to be shared with the Docker container. In your home directory, create a directory named “mule-root”. In the “mule-root” directory, create three directories named “apps”, “conf” and “logs”. Download the Mule CE 3.5.0 standalone distribution from this link. From the Mule CE 3.5.0 distribution, copy the files in the “apps” directory to the “mule-root/apps” directory you just created. From the Mule CE 3.5.0 distribution, copy the files in the “conf” directory to the “mule-root/conf” directory you created. The resulting file- and directory-structure should look like this (shown using the tree command): ~/mule-root/ ├── apps │ └── default │ └── mule-config.xml ├── conf │ ├── log4j.properties │ ├── tls-default.conf │ ├── tls-fips140-2.conf │ ├── wrapper-additional.conf │ └── wrapper.conf └── logs Edit the log4j.properties file in the “mule-root/conf” directory and set the log-level on the last line in the file to “DEBUG”. This modification has nothing to do with sharing directories, but is in order for us to be able to see some more output from Mule when we run it later. The last two lines should now look like this: # Mule classes log4j.logger.org.mule=DEBUG Binding Volumes We are now ready to launch a new Docker container and when we do, we will tell Docker to map three directories in the Docker container to three directories in the host operating system. Three directories in a Docker container bound to three directories in the host. Launch the Docker container with the command below. The -v option tells Docker that we want to make the contents of a directory in the host available at a certain path in the Docker container file-system. The -d option runs the container in the background and the terminal prompt will be available as soon as the id of the newly launched Docker container has been printed. sudo docker run -d -v ~/mule-root/apps:/opt/mule/apps -v ~/mule-root/conf:/opt/mule/conf -v ~/mule-root/logs:/opt/mule/logs codingtony/mule Examine the “mule-root” directory and its subdirectories in the host, which should now look like below. The files on the highlighted rows have been created by Mule. mule-root/ ├── apps │ ├── default │ │ └── mule-config.xml │ └── default-anchor.txt ├── conf │ ├── log4j.properties │ ├── tls-default.conf │ ├── tls-fips140-2.conf │ ├── wrapper-additional.conf │ └── wrapper.conf └── logs ├── mule-app-default.log ├── mule-domain-default.log └── mule.log Examine the “mule.log” file using the command “tail -f ~/mule-root/logs/mule.log”. There should be periodic output written to the log file similar to the following: DEBUG 2015-01-05 12:05:37,216 [Mule.app.deployer.monitor.1.thread.1] org.mule.module.launcher.DeploymentDirectoryWatcher: Checking for changes... DEBUG 2015-01-05 12:05:37,216 [Mule.app.deployer.monitor.1.thread.1] org.mule.module.launcher.DeploymentDirectoryWatcher: Current anchors: default-anchor.txt DEBUG 2015-01-05 12:05:37,216 [Mule.app.deployer.monitor.1.thread.1] org.mule.module.launcher.DeploymentDirectoryWatcher: Deleted anchors: Stop and remove the container: sudo docker stop [container id or name here] sudo docker rm [container id or name here] Direct Access to Docker Container Files When running Docker under the Ubuntu OS it is also possible to access the file-system of a Docker container from the host file-system. It may be possible to do this under other operating systems too, but I haven’t had the opportunity to test this. This technique may come in handy during development or testing with Docker containers for which you haven’t bound any volumes. Note! If given the choice to use either volume binding, as seen above, or direct access to container files as we will look at in this section for something more than a temporary file access, I would chose to use volume binding. Direct access to Docker container files relies on implementation details that I suspect may change in future versions of Docker if the developers find it suitable. With all that said, lets get the action started: Start a new Docker container: sudo docker run -d codingtony/mule Find the id of the newly launched Docker container: sudo docker ps Examine low-level information about the newly launched Docker container: sudo docker inspect [container id or name here] Output similar to this will be printed to the console (portions removed to conserve space): [{ "AppArmorProfile": "", "Args": [], "Config": { ... }, "Created": "2015-01-12T07:58:47.913905369Z", "Driver": "aufs", "ExecDriver": "native-0.2", "HostConfig": { ... }, "HostnamePath": "/var/lib/docker/containers/68b40def7ad6a7f819bd654d5627ad1c3a0f40c84e0fb0f875760f1bd6790eef/hostname", "HostsPath": "/var/lib/docker/containers/68b40def7ad6a7f819bd654d5627ad1c3a0f40c84e0fb0f875760f1bd6790eef/hosts", "Id": "68b40def7ad6a7f819bd654d5627ad1c3a0f40c84e0fb0f875760f1bd6790eef", "Image": "bcd0f37d48d4501ad64bae941d95446b157a6f15e31251e26918dbac542d731f", "MountLabel": "", "Name": "/thirsty_darwin", "NetworkSettings": { ... }, "Path": "/opt/mule/bin/mule", "ProcessLabel": "", "ResolvConfPath": "/var/lib/docker/containers/68b40def7ad6a7f819bd654d5627ad1c3a0f40c84e0fb0f875760f1bd6790eef/resolv.conf", "State": { ... }, "Volumes": {}, "VolumesRW": {} }] Locate the “Driver” node (highlighted in the above output) and ensure that its value is “aufs”. If it is not, you may need to modify the directory paths below replacing “aufs” with the value of this node. Personally I have only seen the “aufs” value at this node so anything else is uncharted territory to me. Copy the long hexadecimal value that can be found at the “Id” node (also highlighted in the above output). This is the long id of the Docker container. In a terminal window, issue the following command, inserting the long id of your container where noted: sudo ls -al /var/lib/docker/aufs/mnt/[long container id here] You are now looking at the root of the volume used by the Docker container you just launched. In the same terminal window, issue the following command: sudo ls -al /var/lib/docker/aufs/mnt/[long container id here]/opt The output from this command should look like this: total 12 drwxr-xr-x 4 root root 4096 jan 12 15:58 . drwxr-xr-x 75 root root 4096 jan 12 15:58 .. lrwxrwxrwx 1 root root 26 aug 10 04:19 mule -> /opt/mule-standalone-3.5.0 drwxr-xr-x 17 409 409 4096 jan 12 15:58 mule-standalone-3.5.0 Examine this line in the Dockerfile:RUN ln -s /opt/mule-standalone-3.5.0 /opt/muleWe see that a symbolic link is created and that the directory name and the name of the symbolic link matches the output we saw earlier. This matches the directory output in the previous step. To examine the Mule log file that we looked at when binding volumes earlier, use the following command: sudo cat /var/lib/docker/aufs/mnt/[long container id here]/opt/mule-standalone-3.5.0/logs/mule.log Next we create a new file in the Docker container using vi: sudo vi /var/lib/docker/aufs/mnt/[long container id here]/opt/mule-standalone-3.5.0/test.txt Enter some text into the new file by first pressing i and the type the text. When you are finished entering the text, press the Escape key and write the file to disk by typing the characters “:wq” without quotes. This writes the new contents of the file to disk and quits the editor. Leave the Docker container running after you are finished. In the next section, we are going to look at the file we just created from inside the Docker container. We have seen that we can examine the file system of a Docker container without binding volumes. It is also possible to copy or move files from the host file-system to the container’s file system using the regular commands. Root privileges are required both when examining and writing to the Docker container’s file system. Entering a Docker Container In order to verify that the file we just created in the host was indeed written to the Docker container, we are going to start a bash shell in the running Docker container and examine the location where the new file is expected to be located and the contents of the file. In the process we will see how we can execute commands in a Docker container from the host. Issue the command below in a terminal window. The exec Docker command is used to run a command, bash in this case, in a running Docker container. The -i flags tell Docker to keep the input stream open while the command is being executed. In this example, it allows us to enter commands into the bash shell running inside the Docker container. The -t flag cause Docker to allocate a text terminal to which the output from the command execution is printed. sudo docker exec -i -t [container id or name here] bash Note the prompt, which should change to [user]@[Docker container id]. In my case it looks like this: root@3ea374a280da:/# Go to the Mule installation directory using this command: cd /opt/mule-standalone-3.5.0/ Examine the contents of the directory: ls -al Among the other files, you should see the “test.txt” file: -rw-r--r-- 1 root root 53 Jan 14 03:19 test.txt Examine the contents of the “text.txt” file. The contents of the file should match what you entered earlier. cat text.txt Exit to the host OS: exit Stop and remove the container: sudo docker stop [container id or name here] sudo docker rm [container id or name here] We have seen that we can execute commands in a running Docker container. In this particular example, we used it to execute the bash shell and examine a file. I draw the conclusion that I should be able to set up a Docker image that contains a very controlled environment for some type of test and then create a container from that image and start the test from the host. Deploying a Mule Application In this section we will look at deploying a Mule application to an instance of the Mule ESB running in a Docker container. We will use volume binding, that we looked at in the section on files and Docker containers, to share directories in the host with the Docker container in order to make it easy to deploy applications, modify running applications, examine logs etc. Preparations Before deploying the application, we need to make some preparations: First of all, we restore the original log-level that we changed earlier. In this example, there will be log output when the applications we will deploy is run and we can limit the log generated by Mule. Edit the log4j.properties file in the “mule-root/conf” directory in the host and set the log-level on the last line in the file back to “INFO” and add one line, as in the listing below. The last three lines should now look like this: # Mule classes log4j.logger.org.mule=INFO log4j.logger.org.mule.tck.functional=DEBUG Next, we create the Mule application which we will deploy to the Mule ESB running in Docker: In some directory, create a file named “mule-deploy.properties” with the following contents: redeployment.enabled=true encoding=UTF-8 domain=default config.resources=HelloWorld.xml In the same directory create a file named “HelloWorld.xml”. This file contains the Mule configuration for our example application: Create a zip-archive named “mule-hello.zip” containing the two files created above: zip mule-hello.zip mule-deploy.properties HelloWorld.xml Deploy the Mule Application Before you start the Docker container in which the Mule EBS will run, make sure that you have created and prepared the directories in the host as described in the section Files and Docker Containers above. Start a new Mule Docker container using the command that we used when binding volumes: sudo docker run -d -v ~/mule-root/apps:/opt/mule/apps -v ~/mule-root/conf:/opt/mule/conf -v ~/mule-root/logs:/opt/mule/logs codingtony/mule As before, the -v option tells Docker to bind three directories in the host to three locations in the Docker container’s file system. Find the IP-address of the Docker container: sudo docker inspect [container id or name here] | grep IPAddress In my case, I see the following line which reveals the IP-address of the Docker container: “IPAddress”: “172.0.17.2”, Open a terminal window or tab and examine the Mule log. Leave this window or tab open during the exercise, in order to be able to verify the output from Mule. tail -f ~/mule-root/logs/mule.log Copy the zip-archive “mule-hello.zip” created earlier to the host directory ~/mule-root/apps/. Verify that the application has been deployed without errors in the Mule log: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + Started app 'mule-hello' + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Leave the Docker container running after you are finished. In the next section we will look at how to access endpoints exposed by applications running in Docker containers. By binding directories in the host thus making them available in the Docker container, it becomes very simple to deploy Mule applications to an instance of Mule ESB running in a Docker container. I am considering this setup for a production environment as well, since it will enable me to perform backups of the directories containing Mule applications and configuration without having to access the Docker container’s file system. It is also in accord with the idea that a Docker container should be able to be quickly and easily restarted, which I feel it would not be if I had to deploy a number of Mule applications to it in order to recreate its previous state. Accessing Endpoints We now know that we can run the Mule ESB in a Docker container, we can deploy applications and examine the logs quite easily but one final, very important question remains to be answered; how to access endpoints exposed by applications running in a Docker container. This section assumes that the Mule application we deployed to Mule in the previous section is still running. In the host, open a web-browser and issue a request to the Docker container’s IP-address at port 8181. In my case, the URL is http://172.17.0.2:8181 Alternatively use the curl command in a terminal window. In my case I would write: curl 172.17.0.2:8181 The result should be a greeting in the following format: Hello World! It is now: 2015-01-14T07:39:03.942Z In addition, you should be able to see that a message was received in the Mule log. Now try the URL http://localhost:8181 You will get a message saying that the connection was refused, provided that you do not already have a service listening at that port. If you have another computer available that is connected to the same network as the host computer running Ubuntu, do the following: – Find the IP-address of the Ubuntu host computer using the ifconfigcommand. – In a web-browser on the other computer, try accessing port 8181 at the IP-address of the Ubuntu host computer. Again you will get a message saying that the connection was refused. Stop and remove the container: sudo docker stop [container id or name here] sudo docker rm [container id or name here] Without any particular measures taken, we see that we can access a service exposed in a Docker container from the Docker host but we did not succeed in accessing the service from another computer. To make a service exposed in a Docker container reachable from outside of the host, we need to tell Docker to publish a port from the Docker container to a port in the host using the -p flag: Launch a new Docker container using the following command: sudo docker run -d -p 8181:8181 -v ~/mule-root/apps:/opt/mule/apps -v ~/mule-root/conf:/opt/mule/conf -v ~/mule-root/logs:/opt/mule/logs codingtony/mule The added flag -p 8181:8181 makes the service exposed at port 8181 in the Docker container available at port 8181 in the host. Try accessing the URL http://localhost:8181 from a web-browser on the host computer.The result should be a greeting of the form we have seen earlier. Try accessing port 8181 at the IP-address of the Ubuntu host computer from another computer.This should also result in a greeting message. Stop and remove the container: sudo docker stop [container id or name here] sudo docker rm [container id or name here] Using the -p flag, we have seen that we can expose a service in a Docker container so that it becomes accessible from outside of the host computer. However, we also see that this information need to be supplied at the time of launching the Docker container. The conclusions that I draw from this is that: I can test and develop against a Mule ESB instance running in a Docker container without having to publish any ports, provided that my development computer is the Docker host computer. In a production environment or any other environment that need to expose services running in a Docker container to “the outside world” and where services will be added over time, I would consider deploying an Apache HTTP Server or NGINX on the Docker host computer and use it to proxy the services that are to be exposed. This way I can avoid re-launching the Docker container each time a new service is added and I can even (temporarily) redirect the proxy to some other computer if I need to perform some maintenance. Is There More? Of course! This article should only be considered an introduction and I am just a beginner with Docker. I hope I will have the time and inspiration to write more about Docker as I learn more.

An increasing number of systems are being built to handle the Volume, Velocity and Variety of Big Data, and hopefully help gain new insights and make better business decisions. Here, we will look at ways to deal with Big Data’s Volume and Velocity simultaneously, within a single architecture solution. Volume + Velocity Apache Hadoop provides both reliable storage (HDFS) and a processing system (MapReduce) for large data sets across clusters of computers. MapReduce is a batch query processor that is targeted at long-running background processes. Hadoop can handle Volume. But to handle Velocity, we need real-time processing tools that can compensate for the high-latency of batch systems, and serve the most recent data continuously, as new data arrives and older data is progressively integrated into the batch framework. Therefore we need both batch and real-time to run in parallel, and add a real-time computational system (e.g. Apache Storm) to our batch framework. This architectural combination of batch and real-time computation is referred to as a Lambda Architecture (λ). Generic Lambda λ has three layers: The Batch Layer manages the master data and precomputes the batch views The Speed Layer serves recent data only and increments the real-time views The Serving Layer is responsible for indexing and exposing the views so that they can be queried. The three layers are outlined in the below diagram along with a sample choice of technology stacks: Incoming data is dispatched to both Batch and Speed layers for processing. At the other end, queries are answered by merging both batch and real-time views. Note that real-time views are transient by nature and their data is discarded (making room for newer data) once propagated through the Batch and Serving layers. Most of the complexity is pushed onto the much smaller Speed layer where the results are only temporary, a process known as “complexity isolation“. We are indeed isolating the complexity of concurrent data updates in a layer that is regularly purged and kept small in size. λ is technology agnostic. The data pipeline is broken down into layers with clear demarcation of responsibilities, and at each layer, we can choose from a number of technologies. The Speed layer for instance could use either Apache Storm, or Apache Spark Streaming, or Spring “XD” ( eXtreme Data) etc. How do we recover from mistakes in λ ? Basically, we recompute the views. If that takes too long, we just revert to the previous, non-corrupted versions of our data. We can do that because of data immutability in the master dataset: data is never updated, only appended to (time-based ordering). The system is therefore Human Fault-Tolerant: if we write bad data, we can just remove that data altogether and recompute. Unified Lambda The downside of λ is its inherent complexity. Keeping in sync two already complex distributed systems is quite an implementation and maintenance challenge. People have started to look for simpler alternatives that would bring just about the same benefits and handle the full problem set. There are basically three approaches: 1) Adopt a pure streaming approach, and use a flexible framework such as Apache Samza to provide some type of batch processing. Although its distributed streaming layer is pluggable, Samza typically relies on Apache Kafka. Samza’s streams are replayable, ordered partitions. Samza can be configured for batching, i.e. consume several messages from the same stream partition in sequence. 2) Take the opposite approach, and choose a flexible Batch framework that would also allow micro-batches, small enough to be close to real-time, with Apache Spark/Spark Streaming or Storm’s Trident. Spark streaming is essentially a sequence of small batch processes that can reach latency as low as one second.Trident is a high-level abstraction on top of Storm that can process streams as small batches as well as do batch aggregation. 3) Use a technology stack already combining batch and real-time, such as Spring “XD”, Summingbird or Lambdoop. Summingbird (“Streaming MapReduce”) is a hybrid system where both batch/real-time workflows can be run at the same time and the results merged automatically.The Speed layer runs on Storm and the Batch layer on Hadoop, Lambdoop (Lambda-Hadoop, with HBase, Storm and Redis) also combines batch/real-time by offering a single API for both processing paradigms: The integrated approach (unified λ) seeks to handle Big Data’s Volume and Velocity by featuring a hybrid computation model, where both batch and real-time data processing are combined transparently. And with a unified framework, there would be only one system to learn, and one system to maintain.

as i said in resolving the conflict between software architecture and code , my focus for this year is representing a software architecture model as code. in simple sketches for diagramming your software architecture , i showed an example system context diagram for my techtribes.je website. it's a simple diagram that shows techtribes.je in the middle, surrounded by the key types of users and system dependencies. it's your typical "big picture" view. this diagram was created using omnigraffle (think microsoft visio for mac os x) and it's exactly that - a static diagram that needs to be manually kept up to date. instead, wouldn't it be great if this diagram was based upon a model that we could better version control, collaborate on and visualize? if you're not sure what i mean by a "model", take a look at models, sketches and everything in between . this is basically what the aim of structurizr is. it's a way to describe a software architecture model as code, and then visualize it in a simple way. the structurizr java library is available on github and you can download a prebuilt binary . just as a warning, this is very much a work in progress and so don't be surprised if things change! here's some java code to recreate the techtribes.je system context diagram. package com.structurizr.example; import com.structurizr.io.json.jsonwriter; import com.structurizr.model.location; import com.structurizr.model.model; import com.structurizr.model.person; import com.structurizr.model.softwaresystem; import com.structurizr.view.systemcontextview; import com.structurizr.view.viewset; import java.io.stringwriter; /** * this is a model of the system context for the techtribes.je system, * the code for which can be found at https://github.com/techtribesje/techtribesje */ public class techtribessystemcontext { public static void main(string[] args) throws exception { // create a model and the software system we want to describe model model = new model("techtribes.je", "this is a model of the system context for the techtribes.je system, the code for which can be found at https://github.com/techtribesje/techtribesje"); softwaresystem techtribes = model.addsoftwaresystem(location.internal, "techtribes.je", "techtribes.je is the only way to keep up to date with the it, tech and digital sector in jersey and guernsey, channel islands"); // create the various types of people (roles) that use the software system person anonymoususer = model.addperson(location.external, "anonymous user", "anybody on the web."); anonymoususer.uses(techtribes, "view people, tribes (businesses, communities and interest groups), content, events, jobs, etc from the local tech, digital and it sector."); person authenticateduser = model.addperson(location.external, "aggregated user", "a user or business with content that is aggregated into the website."); authenticateduser.uses(techtribes, "manage user profile and tribe membership."); person adminuser = model.addperson(location.external, "administration user", "a system administration user."); adminuser.uses(techtribes, "add people, add tribes and manage tribe membership."); // create the various software systems that techtribes.je has a dependency on softwaresystem twitter = model.addsoftwaresystem(location.external, "twitter", "twitter.com"); techtribes.uses(twitter, "gets profile information and tweets from."); softwaresystem github = model.addsoftwaresystem(location.external, "github", "github.com"); techtribes.uses(github, "gets information about public code repositories from."); softwaresystem blogs = model.addsoftwaresystem(location.external, "blogs", "rss and atom feeds"); techtribes.uses(blogs, "gets content using rss and atom feeds from."); // now create the system context view based upon the model viewset viewset = new viewset(model); systemcontextview contextview = viewset.createcontextview(techtribes); contextview.addallsoftwaresystems(); contextview.addallpeople(); // and output the model and view to json (so that we can render it using structurizr.com) jsonwriter jsonwriter = new jsonwriter(true); stringwriter stringwriter = new stringwriter(); jsonwriter.write(viewset, stringwriter); system.out.println(stringwriter.tostring()); } } executing this code creates this json , which you can then copy and paste into the try it page of structurizr. the result (if you move the boxes around) is something like this. don't worry, there will eventually be an api for uploading software architecture models and the diagrams will get some styling, but it proves the concept. what we have then is an api that implements the various levels in my c4 software architecture model, with a simple browser-based rendering tool. hopefully that's a nice simple introduction of how to represent a software architecture model as code, and gives you a flavour for the sort of direction i'm taking it. having the software architecture as code provides some interesting opportunities that you don't get with static diagrams from visio, etc and the ability to keep the models up to date automatically by scanning the codebase is what i find particularly exciting. if you have any thoughts on this, please do drop me a note.

My objective here is to recreate a similar set-up in a smaller unit test mode.

spring boot already comes with great, pre-configured logging system inside, but in real projects it's important to have an ability to search logs, aggregate them and access easy. one of the easiest option for it is http://papertrailapp.com/ . they provide logging service with syslog protocol and 100mb/mo free plan. lets prepare papertrail for our example: create logging group in papetrail dashboard ("create group" button). create log destination in papertrail dashboard. go to "account -> log destinations" click "create log destination" button. make sure your group is selected in " new systems will join" field. you can leave all others fields with their default values, just click "create". remember your log destination (will looks like logs2.papertrailapp:12345), we will use it later spring boot uses logback as default logging system. it's powerful tool for logging with many logging options. for our purposes we will use ch.qos.logback.classic.net.syslogappender . add "logback.xml" file to your "resources" folder with following content: ${papertrail_host} ${papertrail_port} user ${papertrail_app:-app} %highlight([%.-1level]) %35.35logger{35}:\t%m\t%cyan%ex{5} true i'm using environment variables ( (1), (2), and (3) ) for papertrail's credentials, it will allow you to configure different log destinations in different application environments. note excluded throwable at (4). we already have pattern for throwable in suffixpattern ( %cyan%ex{5} ). why we do this? because otherwise exception stacktraces will be printed after main log message line-by-line, it will increase traffic and also you will not be able to see stacktrace in search. i will demonstrate how it works with really simple spring boot application: package com.github.bsideup.spring.boot.example.papertrail; import org.slf4j.logger; import org.slf4j.loggerfactory; import org.springframework.boot.springapplication; import org.springframework.boot.autoconfigure.springbootapplication; import org.springframework.web.bind.annotation.requestmapping; import org.springframework.web.bind.annotation.restcontroller; @springbootapplication @restcontroller public class application { private static final logger log = loggerfactory.getlogger(application.class); public static void main(string[] args) { springapplication.run(application.class, args); } @requestmapping("/") public string index() { log.warn("i'm so tired to welcome everyone", new exception(new exception())); return "hello world!"; } } now, run your application with following environment variables: papertrail_host - host from your log destination (i.e. logs2.papertrailapp.com) papertrail_port - port from your log destination (i.e. 12345) [optional] papertrail_app - application name (default: app) your papertrails output will looks like mine: you can find sample project at github: https://github.com/bsideup/spring-boot-sample-papertrail

Recently at DevSKiller.com we've decided to move majority of our stuff to simple containers. It was pretty easy due to use of Spring Boot uber-jars, but the problem was in NewRelic agents which should have to be included separately. That caused uncomfortable situation so we decided to solve it by including NewRelic agent into our uber-jar applications. If you also want to simplify your life please follow provided instructions :) At first we have to add proper dependency into our pom.xml descriptor: com.newrelic.agent.java< newrelic-agent 3.12.1 provided Now since we have proper jar included into our project it's time to unpack the dependency to have all necessary classes in our application jar file: org.apache.maven.plugins maven-dependency-plugin 2.9 prepare-package unpack-dependencies newrelic-agent ${project.build.outputDirectory} After this step we've all agent related classes accessible directly from our jar. But still the file cannot be used as an agent jar. There are some important manifest entries that have to be present in every agent jar. The most important is the Premain-Class attribute specifying main agent class including premain() method. In case of NewRelic it's also important to include Can-Redefine-Classes and Can-Retransform-Classes attributes. The easiest way to do that is to extend maven-jar-plugin configuration: org.apache.maven.plugins maven-jar-plugin 2.5 com.newrelic.bootstrap.BootstrapAgent true true Now is coming the tricky part :) NewRelic agent also contains class with main() method which causes that Spring Boot repackager plugin is unable to find single main() method so build fails. It's not a problem but we have to remember to specify proper main class in spring-boot-maven-plugin (or in gradle plugin): my.custom.Application That's all! You can execute your application with following command: java -javaagent:myapp.jar -jar myapp.jar Last but not least: don't forget to include NewRelic configuration file (newrelic.yml) in the same directory as your application jar. The other solution is to set newrelic.config.file system property to point the fully qualified file name.

NEW EDIT: As of Spring Boot 1.4.0, faking of Spring Beans is supported natively via annotation @MockBean. Read Spring Boot docs for more info. OLD EDIT: Here is better example how to mock Spring bean. I've worked with Spring for several years. But I was always frustrated with how messy can XML configuration become. As various annotations and possibilities of Java configuration were popping up, I started to enjoy programming with Spring. That is why I strongly entourage using Java configuration. In my opinion, XML configuration is suitable only when you need to have visualized Spring Integration or Spring Batch flow. Hopefully Spring Tool Suite will be able to visualize Java configurations for these frameworks also. One of the nasty aspects of XML configuration is that it often leads to huge XML configuration files. Developers therefore often create test context configuration for integration testing. But what is the purpose of integration testing, when there isn’t production wiring tested? Such integration test has very little value. So I was always trying to design my production contexts in testable fashion. I except that when you are creating new project / module you would avoid XML configuration as much as possible. So with Java configuration you can create Spring configuration per module / package and scan them in main context (@Configuration is also candidate for component scanning). This way you can naturally create islands Spring beans. These islands can be easily tested in isolation. But I have to admit that it’s not always possible to test production Java configuration as is. Rarely you need to amend behavior or spy on certain beans. There is library for it called Springockito. To be honest I didn’t use it so far, because I always try to design Spring configuration to avoid need for mocking. Looking at Springockito pace of development and number of open issues, I would be little bit worried to introduce it into my test suite stack. Fact that last release was done before Spring 4 release brings up questions like “Is it possible to easily integrate it with Spring 4?”. I don’t know, because I didn’t try it. I prefer pure Spring approach if I need to mock Spring bean in integration test. Spring provides @Primary annotation for specifying which bean should be preferred in the case when two beans with same type are registered. This is handy because you can override production bean with fake bean in integration test. Let’s explore this approach and some pitfalls on examples. I chose this simplistic / dummy production code structure for demonstration: @Repository public class AddressDao { public String readAddress(String userName) { return "3 Dark Corner"; } } @Service public class AddressService { private AddressDao addressDao; @Autowired public AddressService(AddressDao addressDao) { this.addressDao = addressDao; } public String getAddressForUser(String userName){ return addressDao.readAddress(userName); } } @Service public class UserService { private AddressService addressService; @Autowired public UserService(AddressService addressService) { this.addressService = addressService; } public String getUserDetails(String userName){ String address = addressService.getAddressForUser(userName); return String.format("User %s, %s", userName, address); } } AddressDao singleton bean instance is injected into AddressService. AddressService is similarly used in UserService. I have to warn you at this stage. My approach is slightly invasive to production code. To be able to fake existing production beans, we have to register fake beans in integration test. But these fake beans are usually in the same package sub-tree as production beans (assuming you are using standard Maven files structure: “src/main/java” and “src/test/java”). So when they are in the same package sub-tree, they would be scanned during integration tests. But we don’t want to use all bean fakes in all integration tests. Fakes could break unrelated integration tests. So we need to have mechanism, how to tell the test to use only certain fake beans. This is done by excluding fake beans from component scanning completely. Integration test explicitly define which fake/s are being used (will show this later). Now let’s take a look at mechanism of excluding fake beans from component scanning. We define our own marker annotation: public @interface BeanMock { } And exclude @BeanMock annotation from component scanning in main Spring configuration. @Configuration @ComponentScan(excludeFilters = @Filter(BeanMock.class)) @EnableAutoConfiguration public class Application { } Root package of component scan is current package of Application class. So all above production beans needs to be in same package or sub-package. We are now need to create integration test forUserService. Let’s spy on address service bean. Of course such testing doesn’t make practical sense with this production code, but this is just example. So here is our spying bean: @Configuration @BeanMock public class AddressServiceSpy { @Bean @Primary public AddressService registerAddressServiceSpy(AddressService addressService) { return spy(addressService); } } Production AddressService bean is autowired from production context, wrapped into Mockito‘s spy and registered as primary bean for AddressService type. @Primary annotation makes sure that our fake bean will be used in integration test instead of production bean. @BeanMock annotation ensures that this bean can’t be scanned by Application component scanning. Let’s take a look at the integration test now: @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = { Application.class, AddressServiceSpy.class }) public class UserServiceITest { @Autowired private UserService userService; @Autowired private AddressService addressService; @Test public void testGetUserDetails() { // GIVEN - spring context defined by Application class // WHEN String actualUserDetails = userService.getUserDetails("john"); // THEN Assert.assertEquals("User john, 3 Dark Corner", actualUserDetails); verify(addressService, times(1)).getAddressForUser("john"); } } @SpringApplicationConfigration annotation has two parameters. First (Application.class) declares Spring configuration under test. Second parameter (AddressServiceSpy.class) specifies fake bean that will be loaded for our testing into Spring IoC container. It’s obvious that we can use as many bean fakes as needed, but you don’t want to have many bean fakes. This approach should be used rarely and if you observe yourself using such mocking often, you are probably having serious problem with tight coupling in your application or within your development team in general. TDD methodology should help you target this problem. Bear in mind: “Less mocking is always better!”. So consider production design changes that allow for lower usage of mocks. This applies also for unit testing. Within integration test we can autowire this spy bean and use it for various verifications. In this case we verified if testing method userService.getUserDetails called methodaddressService.getAddressForUser with parameter “john”. I have one more example. In this case we wouldn’t spy on production bean. We will mock it: @Configuration @BeanMock public class AddressDaoMock { @Bean @Primary public AddressDao registerAddressDaoMock() { return mock(AddressDao.class); } } Again we override production bean, but this time we replace it with Mockito’s mock. We can than record behavior for mock in our integration test: @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = { Application.class, AddressDaoMock.class }) public class AddressServiceITest { @Autowired private AddressService addressService; @Autowired private AddressDao addressDao; @Test public void testGetAddressForUser() { // GIVEN when(addressDao.readAddress("john")).thenReturn("5 Bright Corner"); // WHEN String actualAddress = addressService.getAddressForUser("john"); // THEN Assert.assertEquals("5 Bright Corner", actualAddress); } @After public void resetMock() { reset(addressDao); } } We load mocked bean via @SpringApplicationConfiguration‘s parameter. In test method, we stubaddressDao.readAddress method to return “5 Bright Corner” string when “john” is passed to it as parameter. But bear in mind that recorded behavior can be carried to different integration test via Spring context. We don’t want tests affecting each other. So you can avoid future problems in your test suite by reseting mocks after test. This is done in method resetMock. Source code is on Github.

Learn all about creating a microservices architecture on Java in this great tutorial.

In my previous articles I compared 4 frameworks commonly used in communicating with MongoDB from the JVM and found out that in that use-case, Spring Data for MongoDB was the easiest solution. However I did make the remark that it doesn’t use the GeoJSON format to store geolocation coordinates and geometries. I tried to add GeoJSON support before, but couldn’t get the conversion to work propertly. But after some extensive searching I found out that the reason for it not working was my use of Spring Boot: its autoconfiguration for MongoDB does not support custom conversion out of the box. Luckily, the solution was simple: provide an extra configuration that extends from AbstractMongoConfiguration and import that in the Boot application. In that configuration you can override the customConversions() and add your converters. When you compare the geo classes in Spring Data and GeoJSON, I noticed that only a subset of GeoJSON geometries can be mapped on Spring Data geo classes: Point and Polygon. Spring Boot does not support LineString, MultiLineString, MultiPolygon or MultiPoint. However, in your mapped domain classes, you won’t use these normally. Creating a converter that adheres to the GeoJSON format is quite straightforward. import com.mongodb.BasicDBObject import com.mongodb.DBObject import org.springframework.core.convert.converter.Converter import org.springframework.data.convert.ReadingConverter import org.springframework.data.convert.WritingConverter import org.springframework.data.geo.Point import org.springframework.data.geo.Polygon final class GeoJsonConverters { static List> getConvertersToRegister() { return [ GeoJsonDBObjectToPointConverter.INSTANCE, GeoJsonDBObjectToPolygonConverter.INSTANCE, GeoJsonPointToDBObjectConverter.INSTANCE, GeoJsonPolygonToDBObjectConverter.INSTANCE ] } @WritingConverter static enum GeoJsonPointToDBObjectConverter implements Converter { INSTANCE; @Override DBObject convert(Point source) { return new BasicDBObject([type: 'Point', coordinates: [source.x, source.y]]) } } @ReadingConverter static enum GeoJsonDBObjectToPointConverter implements Converter { INSTANCE; @Override Point convert(DBObject source) { def coordinates = source.coordinates as double[] return new Point(coordinates[0], coordinates[1]) } } @WritingConverter static enum GeoJsonPolygonToDBObjectConverter implements Converter { INSTANCE; @Override DBObject convert(Polygon source) { def coordinates = source.points.collect { [it.x, it.y] } return new BasicDBObject([type: 'Polygon', coordinates: coordinates]) } } @ReadingConverter static enum GeoJsonDBObjectToPolygonConverter implements Converter { INSTANCE; @Override Polygon convert(DBObject source) { def coordinates = source.coordinates as double[] return new Point(coordinates[0], coordinates[1]) } } } To add those converters to the Spring context, you’ll have to override some methods in your MongoDB spring configuration class. import com.mongodb.Mongo import org.springframework.beans.factory.annotation.* import org.springframework.boot.SpringApplication import org.springframework.boot.autoconfigure.EnableAutoConfiguration import org.springframework.context.annotation.* import org.springframework.data.mongodb.config.AbstractMongoConfiguration import org.springframework.data.mongodb.core.convert.* @EnableAutoConfiguration @ComponentScan @Configuration @Import([MongoComparisonMongoConfiguration]) class MongoComparison { static void main(String[] args) { SpringApplication.run(MongoComparison, args); } } @Configuration class MongoComparisonMongoConfiguration extends AbstractMongoConfiguration { @Autowired Mongo mongo; @Value("\${spring.data.mongodb.database}") String databaseName; @Override protected String getDatabaseName() { return databaseName } @Override Mongo mongo() throws Exception { return mongo } @Override CustomConversions customConversions() { def customConverters = [] customConverters << GeoJsonConverters.convertersToRegister return new CustomConversions(customConverters.flatten()) } } As Spring Boot already provides the configuration of the Mongo instance and the name of the database, we can reuse these in the MongoDB configuration class. The custom conversions take preference over the existing ones for Point and Polygon. I’ll be writing a library this weekend to add support for all GeoJSON geometries in Spring Data for MongoDB. However, I already noticed it’ll be very hard to provide support for those in generated query methods in repositories, but with annotated queries being possible, I don’t think this will be a big issue but we’ll see.

recently some pwc tech supremos wrote an article: agile coding in enterprise it: code small and local . subsections: moving away from the monolith why microservices? msa: a think-small approach for rapid development thinking the msa way: minimalism is a must where msa makes sense in msa, integration is the problem, not the solution conclusion msa is short for microservices architecture(s), in the above article. the article posits that microservices is the antidote to monoliths. it doesn’t mention cookie cutter scaling at all, which is another antidote to monoliths, with the right build infrastructure and devops. here’s a view of hypothetical architecture a company could deploy if they were doing microservices: w is web server. p and q don’t stand for anything in particular. here’s the same solution as cookie-cutter scaling, and the alternate (historical) choice of monolith to the right of it: the cookie cutter approach will often leverage components that are dependency injected into each other, and though monoliths might be the same today, pre 2004 they were probably hairballs of singletons (the design patten, not the springframework idiom). continuous delivery, agile? here’s one excerpt that confuses me: " … makes no sense to design and develop software over an 18-month process to accommodate all possible use cases when those use cases can change unexpectedly and the life span of code modules might be less than 18 months…. as i recall, the 18 month-delay problem was solved previously. agile methodologies principally, and continuous delivery/deployment in more recent times. it does not matter whether you’re compiling a monolith, a cookie-cutter solution, old soa services, or microservices, the 18-month fear isn’t real if you’re doing agile and/or cd. agile and cd were increasing the release cadence, and allowing the organization to pivot faster before microservices. it doesn’t matter whether you’ve got a monolith, something cookie-cutter scaled, or soa (micro or not), you’re going to be able to benefit from agile practices and devops setup that facilitates cd. in something like 30 thoughtworks client engagements since 2002, i have not seen the 18-month process at all. in fact i last encountered it in 1997 on an as/400 project, which was the last time i saw a waterfall process being championed. build(s) and trunk elsewhere there is a suggestion: “each microservice [has] its own build, to avoid trunk conflict”. that isn’t unique to microservices, of course. component based systems today also have a multiple build file (module) structure in a source tree. hopefully “trunk” mentioned is alluding to trunk based development, as i would recommend. build technologies this is a expansion on the above, and you can skip this paragraph if you want. hierarchical build systems like maven has allow you to have one build file per module (whether that’s a service or a simple jar destined for the classpath of a bigger thing). buck has a build grammar that allows for a build to grow/shrink/change based on what is being built (from implicitly shared source). maven is for the java ecosystem, while buck promises to be multi-language. both are doing multi-module builds for the sake of a composed or servicified deployment. both maven and buck are presently competing to draw the most reduced set of compile/test/deploy operations for the changes since last build for a hierarchy of modules. anyway, what is it we are striving for? what we want is to develop cheaply, and to deploy smoothly and often, without defect. we want the ability to deploy without large permanent or temporary headcount overseeing or participating in deployment. aside from development costs, and support/operation, deployment costs are a potentially big factor in total cost of ownership. what i like about cookie-cutter is the uniformity of the deployable things. the team size for deployment of such a thing doesn’t grow with the numbers of nodes that binary is being deployed to. at least, if you’re able to automate the deployment to those nodes, and have a strategy for handling the users connected to the stack at redeployment time somehow (sessions or stateless). the uniformity of the deployment is a cheapener, i think. when you have a number of dissimilar services, you might be able to minimize release personnel if you’re only doing one service. if more than one service is being updated in a particular deployment, you’re going to have to concentrate to make sure you don’t experience a multiplier effect for the participants. it is possible of course, to keep the headcount small, but the practice needed beforehand is bigger, which in turn allows for some calmness around the actual deployment. if we’ve stepped away from the project management office thinking that suggests three buggy releases a year (which is more usual than 18 month schedules of old), then we can employ continuous deployment to further eliminate personnel costs around going live. this is something that microservices does well at, but because the most adept proponents design forwards & backwards compatibility into the permutations most likely to co-exist in production. it is at least much quicker to redeploy and bounce one small service, n times than the the cookie-cutter uniform deployment.

Originally written by Sébastien Deluze on the SpringSource blog Spring Jackson support has been improved lately to be more flexible and powerful. This blog post gives you an update about the most useful Jackson related features available in Spring Framework 4.x and Spring Boot. All the code samples are coming from this spring-jackson-demo sample application, feel free to have a look at the code. JSON Views It can sometimes be useful to filter contextually objects serialized to the HTTP response body. In order to provide such capabilities, Spring MVC now has builtin support for Jackson’s Serialization Views. The following example illustrates how to use @JsonView to filter fields depending on the context of serialization - e.g. getting a "summary" view when dealing with collections, and getting a full representation when dealing with a single resource: public class View { interface Summary {} } public class User { @JsonView(View.Summary.class) private Long id; @JsonView(View.Summary.class) private String firstname; @JsonView(View.Summary.class) private String lastname; private String email; private String address; private String postalCode; private String city; private String country; } public class Message { @JsonView(View.Summary.class) private Long id; @JsonView(View.Summary.class) private LocalDate created; @JsonView(View.Summary.class) private String title; @JsonView(View.Summary.class) private User author; private List recipients; private String body; } Thanks to Spring MVC @JsonView support, it is possible to choose, on a per handler method basis, which field should be serialized: @RestController public class MessageController { @Autowired private MessageService messageService; @JsonView(View.Summary.class) @RequestMapping("/") public List getAllMessages() { return messageService.getAll(); } @RequestMapping("/{id}") public Message getMessage(@PathVariable Long id) { return messageService.get(id); } } In this example, if all messages are retrieved, only the most important fields are serialized thanks to the getAllMessages() method annotated with@JsonView(View.Summary.class): [ { "id" : 1, "created" : "2014-11-14", "title" : "Info", "author" : { "id" : 1, "firstname" : "Brian", "lastname" : "Clozel" } }, { "id" : 2, "created" : "2014-11-14", "title" : "Warning", "author" : { "id" : 2, "firstname" : "Stéphane", "lastname" : "Nicoll" } }, { "id" : 3, "created" : "2014-11-14", "title" : "Alert", "author" : { "id" : 3, "firstname" : "Rossen", "lastname" : "Stoyanchev" } } ] In Spring MVC default configuration, MapperFeature.DEFAULT_VIEW_INCLUSION is set tofalse. That means that when enabling a JSON View, non annotated fields or properties likebody or recipients are not serialized. When a specific Message is retrieved using the getMessage() handler method (no JSON View specified), all fields are serialized as expected: { "id" : 1, "created" : "2014-11-14", "title" : "Info", "body" : "This is an information message", "author" : { "id" : 1, "firstname" : "Brian", "lastname" : "Clozel", "email" : "[email protected]", "address" : "1 Jaures street", "postalCode" : "69003", "city" : "Lyon", "country" : "France" }, "recipients" : [ { "id" : 2, "firstname" : "Stéphane", "lastname" : "Nicoll", "email" : "[email protected]", "address" : "42 Obama street", "postalCode" : "1000", "city" : "Brussel", "country" : "Belgium" }, { "id" : 3, "firstname" : "Rossen", "lastname" : "Stoyanchev", "email" : "[email protected]", "address" : "3 Warren street", "postalCode" : "10011", "city" : "New York", "country" : "USA" } ] } Only one class or interface can be specified with the @JsonView annotation, but you can use inheritance to represent JSON View hierarchies (if a field is part of a JSON View, it will be also part of parent view). For example, this handler method will serialize fields annotated with@JsonView(View.Summary.class) and @JsonView(View.SummaryWithRecipients.class): public class View { interface Summary {} interface SummaryWithRecipients extends Summary {} } public class Message { @JsonView(View.Summary.class) private Long id; @JsonView(View.Summary.class) private LocalDate created; @JsonView(View.Summary.class) private String title; @JsonView(View.Summary.class) private User author; @JsonView(View.SummaryWithRecipients.class) private List recipients; private String body; } @RestController public class MessageController { @Autowired private MessageService messageService; @JsonView(View.SummaryWithRecipients.class) @RequestMapping("/with-recipients") public List getAllMessagesWithRecipients() { return messageService.getAll(); } } JSON Views could also be specified when using RestTemplate HTTP client orMappingJackson2JsonView by wrapping the value to serialize in a MappingJacksonValue as shown in this code sample. JSONP As described in the reference documentation, you can enable JSONP for @ResponseBody andResponseEntity methods by declaring an @ControllerAdvice bean that extendsAbstractJsonpResponseBodyAdvice as shown below: @ControllerAdvice public class JsonpAdvice extends AbstractJsonpResponseBodyAdvice { public JsonpAdvice() { super("callback"); } } With such @ControllerAdvice bean registered, it will be possible to request the JSON webservice from another domain using a In this example, the received payload would be: parseResponse({ "id" : 1, "created" : "2014-11-14", ... }); JSONP is also supported and automatically enabled when using MappingJackson2JsonViewwith a request that has a query parameter named jsonp or callback. The JSONP query parameter name(s) could be customized through the jsonpParameterNames property. XML support Since 2.0 release, Jackson provides first class support for some other data formats than JSON. Spring Framework and Spring Boot provide builtin support for Jackson based XML serialization/deserialization. As soon as you include the jackson-dataformat-xml dependency to your project, it is automatically used instead of JAXB2. Using Jackson XML extension has several advantages over JAXB2: Both Jackson and JAXB annotations are recognized JSON View are supported, allowing you to build easily REST Webservices with the same filtered output for both XML and JSON data formats No need to annotate your class with @XmlRootElement, each class serializable in JSON will serializable in XML You usually also want to make sure that the XML library in use is Woodstox since: It is faster than Stax implementation provided with the JDK It avoids some known issues like adding unnecessary namespace prefixes Some features like pretty print don't work without it In order to use it, simply add the latest woodstox-core-asl dependency available to your project. Customizing the Jackson ObjectMapper Prior to Spring Framework 4.1.1, Jackson HttpMessageConverters were usingObjectMapper default configuration. In order to provide a better and easily customizable default configuration, a new Jackson2ObjectMapperBuilder has been introduced. It is the JavaConfig equivalent of the well known Jackson2ObjectMapperFactoryBean used in XML configuration. Jackson2ObjectMapperBuilder provides a nice API to customize various Jackson settings while retaining Spring Framework provided default ones. It also allows to createObjectMapper and XmlMapper instances based on the same configuration. Both Jackson2ObjectMapperBuilder and Jackson2ObjectMapperFactoryBean define a better Jackson default configuration. For example, theDeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES property set to false, in order to allow deserialization of JSON objects with unmapped properties. Jackson support for Java 8 Date & Time API data types is automatically registered when Java 8 is used and jackson-datatype-jsr310 is on the classpath. Joda-Time support is registered as well when jackson-datatype-joda is part of your project dependencies. These classes also allow you to register easily Jackson mixins, modules, serializers or even property naming strategy like PropertyNamingStrategy.CAMEL_CASE_TO_LOWER_CASE_WITH_UNDERSCORES if you want to have your userName java property translated to user_name in JSON. With Spring Boot As described in the Spring Boot reference documentation, there are various ways tocustomize the Jackson ObjectMapper. You can for example enable/disable Jackson features easily by adding properties likespring.jackson.serialization.indent_output=true to application.properties. As an alternative, in the upcoming 1.2 release Spring Boot also allows to customize the Jackson configuration (JSON and XML) used by Spring MVC HttpMessageConverters by declaring a Jackson2ObjectMapperBuilder @Bean: @Bean public Jackson2ObjectMapperBuilder jacksonBuilder() { Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder(); builder.indentOutput(true).dateFormat(new SimpleDateFormat("yyyy-MM-dd")); return builder; } This is useful if you want to use advanced Jackson configuration not exposed through regular configuration keys. Without Spring Boot In a plain Spring Framework application, you can also use Jackson2ObjectMapperBuilder to customize the XML and JSON HttpMessageConverters as shown bellow: @Configuration @EnableWebMvc public class WebConfiguration extends WebMvcConfigurerAdapter { @Override public void configureMessageConverters(List> converters) { Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder(); builder.indentOutput(true).dateFormat(new SimpleDateFormat("yyyy-MM-dd")); converters.add(new MappingJackson2HttpMessageConverter(builder.build())); converters.add(new MappingJackson2XmlHttpMessageConverter(builder.createXmlMapper(true).build())); } } More to come With the upcoming Spring Framework 4.1.3 release, thanks to the addition of a Spring context aware HandlerInstantiator (see SPR-10768 for more details), you will be able to autowire Jackson handlers (serializers, deserializers, type and type id resolvers). This will allow you to build, for example, a custom deserializer that will replace a field containing only a reference in the JSON payload by the full Entity retrieved from the database.