Of late, I have been looking into Microservice containers that are available out there to help speed up the development. Although, Microservice is a generic term however there is some consensus with respect to what it means. Hence, we may conveniently refer to the definition Microservice as an "architectural design pattern, in which complex applications are composed of small, independent processes communicating with each other using language-agnostic APIs. These services are small, highly decoupled and focus on doing a small task." There are several Microservice containers out there. However, in my experience I have found Dropwizard and Spring-boot to have had received more attention and they appear to be widely used compared to the rest. In my current role, I was asked create a comparison matrix between the two, so it's here below. Dropwizard Spring-Boot What is it? Dropwizard pulls together stable, mature libraries from the Java ecosystem into a simple, light-weight package that lets you focus on getting things done. [more...] Takes an opinionated view of building production-ready Spring applications. Spring Boot favours convention over configuration and is designed to get you up and running as quickly as possible. [more...] Overview? Dropwizard straddles the line between being a library and a framework. Provide performant, reliable implementations of everything a production-ready web application needs. [more...] Spring-boot takes an opinionated view of the Spring platform and third-party libraries so you can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration. [more...] Out of the box features? Dropwizard has out-of-the-box support for sophisticated configuration, application metrics, logging, operational tools, and much more, allowing you and your team to ship a production-quality web service in the shortest time possible. [more...] Spring-boot provides a range of non-functional features that are common to large classes of projects (e.g. embedded servers, security, metrics, health checks, externalized configuration). [more...] Libraries Core: Jetty, Jersey, Jackson and Matrics Others: Guava, Liquibase and Joda Time. Spring, JUnit, Logback, Guava. There are several starter POM files covering various use cases, which can be included in the POM to get started. Dependency Injection? No built in Dependency Injection. Requires a 3rd party dependency injection framework such as Guice, CDI or Dagger. [Ref...] Built in Dependency Injection provided by Spring Dependency Injection container. [Ref...] Types of Services i.e. REST, SOAP Has some support for other types of services but primarily is designed for performant HTTP/REST LAYER. If ever need to integrate SOAP, there is a dropwizard bundle for building SOAP web services using JAX-WS API is provided here but it’s not official drop-wizard sub project. [more...] As well as supporting REST Spring-boot has support for other types of services such as JMS, Advanced Message Queuing Protocol, SOAP based Web Services to name a few. [more...] Deployment? How it creates the Executable Jar? Uses Shading to build executable fat jars, where a shaded jar spackages all classes, from all jars, into a single 'uber jar'. [Ref...] Spring-boot adopts a different approach and avoids shaded jars, as it becomes hard to see which libraries you are actually using in your application. It can also be problematic if the same filename is used in Shaded jars. Instead it uses “Nested Jar” approach where all classes from all jars do not need to be included into a single “uber jar” instead all dependent jars should be in the “lib” folder, spring loader loads them appropriately. [Ref...] Contract First Web Services? No built in support. Would have to refer to 3rd party library (CXF or any other JAX-WS implementation) if needed a solution for the Contract First SOAP based services. Contract First services support is available with the help of spring-boot-starter-ws starter application. [Ref...] Externalised Configuration for properties and YAML Supports both Properties and YAML Supports both Properties and YAML Concluding Remarks If dealing with only REST micro services, drop wizard is an excellent choice. Where Spring-boot shines is the types of services supported i.e. REST, JMS, Messaging, and Contract First Services. Not least a fully built in Dependency Injection container. Disclaimer: The matrix is purely based on my personal views and experiences, having tried both frameworks and is by no means an exhaustive guide. Readers are requested to do their own research before making a strategic decision between the two very formidable frameworks.

In this article I will attempt to run the Mule ESB community edition in Docker in order to see whether it is feasible without any greater inconvenience. My goal is to be able to use Docker both when testing as well as in a production environment in order to gain better control over the environment and to separate different types of environments. I imagine that most of the Docker-related information can be applied to other applications – I have used Mule since it is what I usually work with. The conclusion I have made after having completed my experiments is that it is possible to run Mule ESB in Docker without any inconvenience. In addition, Docker will indeed allow me to have better control over the different environments and also allow me to separate them as I find appropriate. Finally, I just want to mention that I have used Docker in an Ubuntu environment. I have not attempted any of the exercises in Docker running on Windows or Mac OS X. Docker Briefly In short, Docker allows for creating of images that serve as blueprints for containers. A Docker container is an instance of a Docker image in the same way a Java object is an instance of a Java class. FROM codingtony/java MAINTAINER tony(dot)bussieres(at)ticksmith(dot)com RUN wget https://repository.mulesoft.org/nexus/content/repositories/releases/org/mule/distributions/mule-standalone/3.5.0/mule-standalone-3.5.0.tar.gz RUN cd /opt && tar xvzf ~/mule-standalone-3.5.0.tar.gz RUN echo "4a94356f7401ac8be30a992a414ca9b9 /mule-standalone-3.5.0.tar.gz" | md5sum -c RUN rm ~/mule-standalone-3.5.0.tar.gz RUN ln -s /opt/mule-standalone-3.5.0 /opt/mule CMD [ "/opt/mule/bin/mule" ] The resource isolation features of Linux are used to create Docker containers, which are more lightweight than virtual machines and are separated from the environment in which Docker runs, the host. Using Docker an image can be created that, every time it is started has a known state. In order to remove any doubts about whether the environment has been altered in any way, the container can be stopped and a new container started. I can even run multiple Docker containers on one and the same computer to simulate a multi-server production environment. Applications can also be run in their own Docker containers, as shown in this figure. Three Docker containers, each containing a specific application, running in one host. A more detailed introduction to Docker is available here. The main entry point to the Docker documentation can be found here. Motivation Some of the motivations I have for using Docker in both testing and production environments are: The environment in which I test my application should be as similar as the final deployment environment as possible, if not identical. Making the deployment environment easy to scale up and down. If it is easy to start a new processing node when need arise and stop it if it is no longer used, I will be able to adapt to changes rather quickly and thus reduce errors caused by, for instance, load peaks. Maintain an increased number of nodes to which applications can be deployed. Instead of running one instance of some kind of application server, Mule ESB in my case, on a computer, I want multiple instances that are partitioned, for instance, according to importance. High-priority applications run on one separate instance, which have higher priority both as far as resources (CPU, memory, disk etc) are concerned but also as far as support is concerned. Applications which are less critical run on another instance. Enable quick replacement of instances in the deployment environment. Reasons for having to replace instances may be hardware failure etc. Better control over the contents of the different environments. The concept of an environment that, at any time, may be disposed (and restarted) discourages hacks in the environment, which are usually poorly documented and sometimes difficult to trace. Using Docker, I need to change the appropriate Docker image if I want to make changes to some application environment. The Docker image file, commonly known as Dockerfile, can be checked into any ordinary revision control system, such as Git, Subversion etc, making changes reversible and traceable. Automate the creation of a testing environment. An example could be a nightly job that runs on my build server which creates a test environment, deploys one or more applications to it and then performs tests, such as load-testing. Prerequisites To get the best possible experience when running Docker, I run it under Ubuntu. According to the current documentation, Docker is supported under the following versions of Ubuntu: 12.04 LTS (64-bit) 13.04 (64-bit) 13.10 (64-bit) 14.04 (64-bit) Against my usual conservative self, I chose Ubuntu 14.10, which at the time of writing this article is the latest version. While I haven’t run into any issues, I cannot promise anything regarding compatibility with Docker as far as this version of Ubuntu is concerned. Installing Docker Before we install anything, those who have the Docker version from the Ubuntu repository should remove this version before installing a newer version of Docker, since the Ubuntu repository does not contain the most recent version and the package does not have the same name as the Docker package we will install: sudo apt-get remove docker.io The simplest way to install Docker is to use an installation script made available at the Docker website: curl -sSL https://get.docker.com/ubuntu/ | sudo sh If you are not running Ubuntu or if you do not want to use the above way of installing Docker, please refer to this page containing instructions on how to install Docker on various platforms. To verify the Docker installation, open a terminal window and enter: sudo docker version Output similar to the following should appear: Client version: 1.4.1 Client API version: 1.16 Go version (client): go1.3.3 Git commit (client): 5bc2ff8 OS/Arch (client): linux/amd64 Server version: 1.4.1 Server API version: 1.16 Go version (server): go1.3.3 Git commit (server): 5bc2ff8 We are now ready to start a Mule instance in Docker. Running Mule in Docker One of the advantages with Docker is that there is a large repository of Docker images that are ready to be used, and even extended if one so wishes. ThisDocker image is the one that I will use in this article. It is well documented, there is a source repository and it contains a recent version of the Mule ESB Community Edition. Some additional details on the Docker image: Ubuntu 14.04. Oracle JavaSE 1.7.0_65. This version will change as the PPA containing the package is updated. Mule ESB CE 3.5.0 Note that the image may change at any time and the specifications above may have changed. If you intend to use Docker in your organization, I would suspect that the best alternative is to create your own Docker images that are totally under your control. The Docker image repository is an excellent source of inspiration and aid even in this case. Starting a Docker Container To start a Docker container using this image, open a terminal window and write: sudo docker run codingtony/mule The first time an image is used it needs to be downloaded and created. This usually takes quite some time, so I suggest a short break here – perhaps for a cup of coffee or tea. If you just want to download an image without starting it, exchange the Docker command “run” with “pull”. Once the container is started, you will see some output to the console. If you are familiar with Mule, you will recognize the log output: MULE_HOME is set to /opt/mule-standalone-3.5.0 Running in console (foreground) mode by default, use Ctrl-C to exit... MULE_HOME is set to /opt/mule-standalone-3.5.0 Running Mule... --> Wrapper Started as Console Launching a JVM... Starting the Mule Container... Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. INFO 2015-01-05 04:41:42,302 [WrapperListener_start_runner] org.mule.module.launcher.MuleContainer: ********************************************************************** * Mule ESB and Integration Platform * * Version: 3.5.0 Build: ff1df1f3 * * MuleSoft, Inc. * * For more information go to http://www.mulesoft.org * * * * Server started: 1/5/15 4:41 AM * * JDK: 1.7.0_65 (mixed mode) * * OS: Linux (3.16.0-28-generic, amd64) * * Host: f95698cfb796 (172.17.0.2) * ********************************************************************** Note that: In the text-box containing information about the Mule ESB and Integration Platform, there is a row which starts with “Host:”. The hexadecimal digit that follows is the Docker container id and the IP-address is the external IP-address of the Docker container in which Mule is running. Before we do anything with the Mule instance running in Docker, let’s take a look at Docker containers. Docker Containers We can verify that there is a Docker container running by opening another terminal window, or a tab in the first terminal window, and running the command: sudo docker ps As a result, you will see output similar to the following (I have edited the output in order for the columns to be aligned with the column titles): CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f95698cfb796 codingtony/mule:latest "/opt/mule/bin/mule" 7 min ago Up 7 min jolly_hopper From this output we can see that: The ID of the container is f95698cfb796. This ID can be used when performing operations on the container, such as stopping it, restarting it etc. The name of the image used to created the container. The command that is currently executing. If we look at the Dockerfile for the image, we can see that the last line in this file is: CMD [ “/opt/mule/bin/mule” ] This is the command that is executed whenever an instance of the Docker image is launched and it matches what we see in the COMMAND column for the Docker container. The CREATED column shows how much time has passed since the container was created. The STATUS column shows the current status of the image. When you have used Docker for a while, you can view all the containers using: sudo docker ps -a This will show you containers that are not running, in addition to the running ones. Containers that are not running can be restarted. The PORTS column shows any port mappings for the container. More about port mappings later. Finally, the NAMES column contain a more human-friendly container name. This container name can be used in the same way as the container id. Docker containers will consume disk-space and if you want to determine how much disk-space each of the containers on your computer use, issue the following command: sudo docker ps -a -s An additional column, SIZE, will be shown and in this column I see that my Mule container consumes 41,76kB. Note that this is in addition to the disk-space consumed by the Docker image. This number will grow if you use the container under a longer period of time, as the container retains any files written to disk. To completely remove a stopped Docker container, find the id or name of the container and use the command: sudo docker rm [container id or name here] Before going further, let’s stop the running container and remove it: sudo docker stop [container id or name here] sudo docker rm [container id or name here] Files and Docker Containers So far we have managed to start a Mule instance running inside a Docker container, but there were no Mule applications deployed to it and the logs that were generated were only visible in the terminal window. I want to be able to deploy my applications to the Mule instance and examine the logs in a convenient way. In this section I will show how to: Share one or more directories in the host file-system with a Docker container. Access the files in a Docker container from the host. As the first step in looking at sharing directories between the host operating system and a Docker container, we are going to look at Mule logs. As part of this exercise we also set up the directories in the host operating system that are going to be shared with the Docker container. In your home directory, create a directory named “mule-root”. In the “mule-root” directory, create three directories named “apps”, “conf” and “logs”. Download the Mule CE 3.5.0 standalone distribution from this link. From the Mule CE 3.5.0 distribution, copy the files in the “apps” directory to the “mule-root/apps” directory you just created. From the Mule CE 3.5.0 distribution, copy the files in the “conf” directory to the “mule-root/conf” directory you created. The resulting file- and directory-structure should look like this (shown using the tree command): ~/mule-root/ ├── apps │ └── default │ └── mule-config.xml ├── conf │ ├── log4j.properties │ ├── tls-default.conf │ ├── tls-fips140-2.conf │ ├── wrapper-additional.conf │ └── wrapper.conf └── logs Edit the log4j.properties file in the “mule-root/conf” directory and set the log-level on the last line in the file to “DEBUG”. This modification has nothing to do with sharing directories, but is in order for us to be able to see some more output from Mule when we run it later. The last two lines should now look like this: # Mule classes log4j.logger.org.mule=DEBUG Binding Volumes We are now ready to launch a new Docker container and when we do, we will tell Docker to map three directories in the Docker container to three directories in the host operating system. Three directories in a Docker container bound to three directories in the host. Launch the Docker container with the command below. The -v option tells Docker that we want to make the contents of a directory in the host available at a certain path in the Docker container file-system. The -d option runs the container in the background and the terminal prompt will be available as soon as the id of the newly launched Docker container has been printed. sudo docker run -d -v ~/mule-root/apps:/opt/mule/apps -v ~/mule-root/conf:/opt/mule/conf -v ~/mule-root/logs:/opt/mule/logs codingtony/mule Examine the “mule-root” directory and its subdirectories in the host, which should now look like below. The files on the highlighted rows have been created by Mule. mule-root/ ├── apps │ ├── default │ │ └── mule-config.xml │ └── default-anchor.txt ├── conf │ ├── log4j.properties │ ├── tls-default.conf │ ├── tls-fips140-2.conf │ ├── wrapper-additional.conf │ └── wrapper.conf └── logs ├── mule-app-default.log ├── mule-domain-default.log └── mule.log Examine the “mule.log” file using the command “tail -f ~/mule-root/logs/mule.log”. There should be periodic output written to the log file similar to the following: DEBUG 2015-01-05 12:05:37,216 [Mule.app.deployer.monitor.1.thread.1] org.mule.module.launcher.DeploymentDirectoryWatcher: Checking for changes... DEBUG 2015-01-05 12:05:37,216 [Mule.app.deployer.monitor.1.thread.1] org.mule.module.launcher.DeploymentDirectoryWatcher: Current anchors: default-anchor.txt DEBUG 2015-01-05 12:05:37,216 [Mule.app.deployer.monitor.1.thread.1] org.mule.module.launcher.DeploymentDirectoryWatcher: Deleted anchors: Stop and remove the container: sudo docker stop [container id or name here] sudo docker rm [container id or name here] Direct Access to Docker Container Files When running Docker under the Ubuntu OS it is also possible to access the file-system of a Docker container from the host file-system. It may be possible to do this under other operating systems too, but I haven’t had the opportunity to test this. This technique may come in handy during development or testing with Docker containers for which you haven’t bound any volumes. Note! If given the choice to use either volume binding, as seen above, or direct access to container files as we will look at in this section for something more than a temporary file access, I would chose to use volume binding. Direct access to Docker container files relies on implementation details that I suspect may change in future versions of Docker if the developers find it suitable. With all that said, lets get the action started: Start a new Docker container: sudo docker run -d codingtony/mule Find the id of the newly launched Docker container: sudo docker ps Examine low-level information about the newly launched Docker container: sudo docker inspect [container id or name here] Output similar to this will be printed to the console (portions removed to conserve space): [{ "AppArmorProfile": "", "Args": [], "Config": { ... }, "Created": "2015-01-12T07:58:47.913905369Z", "Driver": "aufs", "ExecDriver": "native-0.2", "HostConfig": { ... }, "HostnamePath": "/var/lib/docker/containers/68b40def7ad6a7f819bd654d5627ad1c3a0f40c84e0fb0f875760f1bd6790eef/hostname", "HostsPath": "/var/lib/docker/containers/68b40def7ad6a7f819bd654d5627ad1c3a0f40c84e0fb0f875760f1bd6790eef/hosts", "Id": "68b40def7ad6a7f819bd654d5627ad1c3a0f40c84e0fb0f875760f1bd6790eef", "Image": "bcd0f37d48d4501ad64bae941d95446b157a6f15e31251e26918dbac542d731f", "MountLabel": "", "Name": "/thirsty_darwin", "NetworkSettings": { ... }, "Path": "/opt/mule/bin/mule", "ProcessLabel": "", "ResolvConfPath": "/var/lib/docker/containers/68b40def7ad6a7f819bd654d5627ad1c3a0f40c84e0fb0f875760f1bd6790eef/resolv.conf", "State": { ... }, "Volumes": {}, "VolumesRW": {} }] Locate the “Driver” node (highlighted in the above output) and ensure that its value is “aufs”. If it is not, you may need to modify the directory paths below replacing “aufs” with the value of this node. Personally I have only seen the “aufs” value at this node so anything else is uncharted territory to me. Copy the long hexadecimal value that can be found at the “Id” node (also highlighted in the above output). This is the long id of the Docker container. In a terminal window, issue the following command, inserting the long id of your container where noted: sudo ls -al /var/lib/docker/aufs/mnt/[long container id here] You are now looking at the root of the volume used by the Docker container you just launched. In the same terminal window, issue the following command: sudo ls -al /var/lib/docker/aufs/mnt/[long container id here]/opt The output from this command should look like this: total 12 drwxr-xr-x 4 root root 4096 jan 12 15:58 . drwxr-xr-x 75 root root 4096 jan 12 15:58 .. lrwxrwxrwx 1 root root 26 aug 10 04:19 mule -> /opt/mule-standalone-3.5.0 drwxr-xr-x 17 409 409 4096 jan 12 15:58 mule-standalone-3.5.0 Examine this line in the Dockerfile:RUN ln -s /opt/mule-standalone-3.5.0 /opt/muleWe see that a symbolic link is created and that the directory name and the name of the symbolic link matches the output we saw earlier. This matches the directory output in the previous step. To examine the Mule log file that we looked at when binding volumes earlier, use the following command: sudo cat /var/lib/docker/aufs/mnt/[long container id here]/opt/mule-standalone-3.5.0/logs/mule.log Next we create a new file in the Docker container using vi: sudo vi /var/lib/docker/aufs/mnt/[long container id here]/opt/mule-standalone-3.5.0/test.txt Enter some text into the new file by first pressing i and the type the text. When you are finished entering the text, press the Escape key and write the file to disk by typing the characters “:wq” without quotes. This writes the new contents of the file to disk and quits the editor. Leave the Docker container running after you are finished. In the next section, we are going to look at the file we just created from inside the Docker container. We have seen that we can examine the file system of a Docker container without binding volumes. It is also possible to copy or move files from the host file-system to the container’s file system using the regular commands. Root privileges are required both when examining and writing to the Docker container’s file system. Entering a Docker Container In order to verify that the file we just created in the host was indeed written to the Docker container, we are going to start a bash shell in the running Docker container and examine the location where the new file is expected to be located and the contents of the file. In the process we will see how we can execute commands in a Docker container from the host. Issue the command below in a terminal window. The exec Docker command is used to run a command, bash in this case, in a running Docker container. The -i flags tell Docker to keep the input stream open while the command is being executed. In this example, it allows us to enter commands into the bash shell running inside the Docker container. The -t flag cause Docker to allocate a text terminal to which the output from the command execution is printed. sudo docker exec -i -t [container id or name here] bash Note the prompt, which should change to [user]@[Docker container id]. In my case it looks like this: root@3ea374a280da:/# Go to the Mule installation directory using this command: cd /opt/mule-standalone-3.5.0/ Examine the contents of the directory: ls -al Among the other files, you should see the “test.txt” file: -rw-r--r-- 1 root root 53 Jan 14 03:19 test.txt Examine the contents of the “text.txt” file. The contents of the file should match what you entered earlier. cat text.txt Exit to the host OS: exit Stop and remove the container: sudo docker stop [container id or name here] sudo docker rm [container id or name here] We have seen that we can execute commands in a running Docker container. In this particular example, we used it to execute the bash shell and examine a file. I draw the conclusion that I should be able to set up a Docker image that contains a very controlled environment for some type of test and then create a container from that image and start the test from the host. Deploying a Mule Application In this section we will look at deploying a Mule application to an instance of the Mule ESB running in a Docker container. We will use volume binding, that we looked at in the section on files and Docker containers, to share directories in the host with the Docker container in order to make it easy to deploy applications, modify running applications, examine logs etc. Preparations Before deploying the application, we need to make some preparations: First of all, we restore the original log-level that we changed earlier. In this example, there will be log output when the applications we will deploy is run and we can limit the log generated by Mule. Edit the log4j.properties file in the “mule-root/conf” directory in the host and set the log-level on the last line in the file back to “INFO” and add one line, as in the listing below. The last three lines should now look like this: # Mule classes log4j.logger.org.mule=INFO log4j.logger.org.mule.tck.functional=DEBUG Next, we create the Mule application which we will deploy to the Mule ESB running in Docker: In some directory, create a file named “mule-deploy.properties” with the following contents: redeployment.enabled=true encoding=UTF-8 domain=default config.resources=HelloWorld.xml In the same directory create a file named “HelloWorld.xml”. This file contains the Mule configuration for our example application: Create a zip-archive named “mule-hello.zip” containing the two files created above: zip mule-hello.zip mule-deploy.properties HelloWorld.xml Deploy the Mule Application Before you start the Docker container in which the Mule EBS will run, make sure that you have created and prepared the directories in the host as described in the section Files and Docker Containers above. Start a new Mule Docker container using the command that we used when binding volumes: sudo docker run -d -v ~/mule-root/apps:/opt/mule/apps -v ~/mule-root/conf:/opt/mule/conf -v ~/mule-root/logs:/opt/mule/logs codingtony/mule As before, the -v option tells Docker to bind three directories in the host to three locations in the Docker container’s file system. Find the IP-address of the Docker container: sudo docker inspect [container id or name here] | grep IPAddress In my case, I see the following line which reveals the IP-address of the Docker container: “IPAddress”: “172.0.17.2”, Open a terminal window or tab and examine the Mule log. Leave this window or tab open during the exercise, in order to be able to verify the output from Mule. tail -f ~/mule-root/logs/mule.log Copy the zip-archive “mule-hello.zip” created earlier to the host directory ~/mule-root/apps/. Verify that the application has been deployed without errors in the Mule log: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + Started app 'mule-hello' + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Leave the Docker container running after you are finished. In the next section we will look at how to access endpoints exposed by applications running in Docker containers. By binding directories in the host thus making them available in the Docker container, it becomes very simple to deploy Mule applications to an instance of Mule ESB running in a Docker container. I am considering this setup for a production environment as well, since it will enable me to perform backups of the directories containing Mule applications and configuration without having to access the Docker container’s file system. It is also in accord with the idea that a Docker container should be able to be quickly and easily restarted, which I feel it would not be if I had to deploy a number of Mule applications to it in order to recreate its previous state. Accessing Endpoints We now know that we can run the Mule ESB in a Docker container, we can deploy applications and examine the logs quite easily but one final, very important question remains to be answered; how to access endpoints exposed by applications running in a Docker container. This section assumes that the Mule application we deployed to Mule in the previous section is still running. In the host, open a web-browser and issue a request to the Docker container’s IP-address at port 8181. In my case, the URL is http://172.17.0.2:8181 Alternatively use the curl command in a terminal window. In my case I would write: curl 172.17.0.2:8181 The result should be a greeting in the following format: Hello World! It is now: 2015-01-14T07:39:03.942Z In addition, you should be able to see that a message was received in the Mule log. Now try the URL http://localhost:8181 You will get a message saying that the connection was refused, provided that you do not already have a service listening at that port. If you have another computer available that is connected to the same network as the host computer running Ubuntu, do the following: – Find the IP-address of the Ubuntu host computer using the ifconfigcommand. – In a web-browser on the other computer, try accessing port 8181 at the IP-address of the Ubuntu host computer. Again you will get a message saying that the connection was refused. Stop and remove the container: sudo docker stop [container id or name here] sudo docker rm [container id or name here] Without any particular measures taken, we see that we can access a service exposed in a Docker container from the Docker host but we did not succeed in accessing the service from another computer. To make a service exposed in a Docker container reachable from outside of the host, we need to tell Docker to publish a port from the Docker container to a port in the host using the -p flag: Launch a new Docker container using the following command: sudo docker run -d -p 8181:8181 -v ~/mule-root/apps:/opt/mule/apps -v ~/mule-root/conf:/opt/mule/conf -v ~/mule-root/logs:/opt/mule/logs codingtony/mule The added flag -p 8181:8181 makes the service exposed at port 8181 in the Docker container available at port 8181 in the host. Try accessing the URL http://localhost:8181 from a web-browser on the host computer.The result should be a greeting of the form we have seen earlier. Try accessing port 8181 at the IP-address of the Ubuntu host computer from another computer.This should also result in a greeting message. Stop and remove the container: sudo docker stop [container id or name here] sudo docker rm [container id or name here] Using the -p flag, we have seen that we can expose a service in a Docker container so that it becomes accessible from outside of the host computer. However, we also see that this information need to be supplied at the time of launching the Docker container. The conclusions that I draw from this is that: I can test and develop against a Mule ESB instance running in a Docker container without having to publish any ports, provided that my development computer is the Docker host computer. In a production environment or any other environment that need to expose services running in a Docker container to “the outside world” and where services will be added over time, I would consider deploying an Apache HTTP Server or NGINX on the Docker host computer and use it to proxy the services that are to be exposed. This way I can avoid re-launching the Docker container each time a new service is added and I can even (temporarily) redirect the proxy to some other computer if I need to perform some maintenance. Is There More? Of course! This article should only be considered an introduction and I am just a beginner with Docker. I hope I will have the time and inspiration to write more about Docker as I learn more.

An increasing number of systems are being built to handle the Volume, Velocity and Variety of Big Data, and hopefully help gain new insights and make better business decisions. Here, we will look at ways to deal with Big Data’s Volume and Velocity simultaneously, within a single architecture solution. Volume + Velocity Apache Hadoop provides both reliable storage (HDFS) and a processing system (MapReduce) for large data sets across clusters of computers. MapReduce is a batch query processor that is targeted at long-running background processes. Hadoop can handle Volume. But to handle Velocity, we need real-time processing tools that can compensate for the high-latency of batch systems, and serve the most recent data continuously, as new data arrives and older data is progressively integrated into the batch framework. Therefore we need both batch and real-time to run in parallel, and add a real-time computational system (e.g. Apache Storm) to our batch framework. This architectural combination of batch and real-time computation is referred to as a Lambda Architecture (λ). Generic Lambda λ has three layers: The Batch Layer manages the master data and precomputes the batch views The Speed Layer serves recent data only and increments the real-time views The Serving Layer is responsible for indexing and exposing the views so that they can be queried. The three layers are outlined in the below diagram along with a sample choice of technology stacks: Incoming data is dispatched to both Batch and Speed layers for processing. At the other end, queries are answered by merging both batch and real-time views. Note that real-time views are transient by nature and their data is discarded (making room for newer data) once propagated through the Batch and Serving layers. Most of the complexity is pushed onto the much smaller Speed layer where the results are only temporary, a process known as “complexity isolation“. We are indeed isolating the complexity of concurrent data updates in a layer that is regularly purged and kept small in size. λ is technology agnostic. The data pipeline is broken down into layers with clear demarcation of responsibilities, and at each layer, we can choose from a number of technologies. The Speed layer for instance could use either Apache Storm, or Apache Spark Streaming, or Spring “XD” ( eXtreme Data) etc. How do we recover from mistakes in λ ? Basically, we recompute the views. If that takes too long, we just revert to the previous, non-corrupted versions of our data. We can do that because of data immutability in the master dataset: data is never updated, only appended to (time-based ordering). The system is therefore Human Fault-Tolerant: if we write bad data, we can just remove that data altogether and recompute. Unified Lambda The downside of λ is its inherent complexity. Keeping in sync two already complex distributed systems is quite an implementation and maintenance challenge. People have started to look for simpler alternatives that would bring just about the same benefits and handle the full problem set. There are basically three approaches: 1) Adopt a pure streaming approach, and use a flexible framework such as Apache Samza to provide some type of batch processing. Although its distributed streaming layer is pluggable, Samza typically relies on Apache Kafka. Samza’s streams are replayable, ordered partitions. Samza can be configured for batching, i.e. consume several messages from the same stream partition in sequence. 2) Take the opposite approach, and choose a flexible Batch framework that would also allow micro-batches, small enough to be close to real-time, with Apache Spark/Spark Streaming or Storm’s Trident. Spark streaming is essentially a sequence of small batch processes that can reach latency as low as one second.Trident is a high-level abstraction on top of Storm that can process streams as small batches as well as do batch aggregation. 3) Use a technology stack already combining batch and real-time, such as Spring “XD”, Summingbird or Lambdoop. Summingbird (“Streaming MapReduce”) is a hybrid system where both batch/real-time workflows can be run at the same time and the results merged automatically.The Speed layer runs on Storm and the Batch layer on Hadoop, Lambdoop (Lambda-Hadoop, with HBase, Storm and Redis) also combines batch/real-time by offering a single API for both processing paradigms: The integrated approach (unified λ) seeks to handle Big Data’s Volume and Velocity by featuring a hybrid computation model, where both batch and real-time data processing are combined transparently. And with a unified framework, there would be only one system to learn, and one system to maintain.

as i said in resolving the conflict between software architecture and code , my focus for this year is representing a software architecture model as code. in simple sketches for diagramming your software architecture , i showed an example system context diagram for my techtribes.je website. it's a simple diagram that shows techtribes.je in the middle, surrounded by the key types of users and system dependencies. it's your typical "big picture" view. this diagram was created using omnigraffle (think microsoft visio for mac os x) and it's exactly that - a static diagram that needs to be manually kept up to date. instead, wouldn't it be great if this diagram was based upon a model that we could better version control, collaborate on and visualize? if you're not sure what i mean by a "model", take a look at models, sketches and everything in between . this is basically what the aim of structurizr is. it's a way to describe a software architecture model as code, and then visualize it in a simple way. the structurizr java library is available on github and you can download a prebuilt binary . just as a warning, this is very much a work in progress and so don't be surprised if things change! here's some java code to recreate the techtribes.je system context diagram. package com.structurizr.example; import com.structurizr.io.json.jsonwriter; import com.structurizr.model.location; import com.structurizr.model.model; import com.structurizr.model.person; import com.structurizr.model.softwaresystem; import com.structurizr.view.systemcontextview; import com.structurizr.view.viewset; import java.io.stringwriter; /** * this is a model of the system context for the techtribes.je system, * the code for which can be found at https://github.com/techtribesje/techtribesje */ public class techtribessystemcontext { public static void main(string[] args) throws exception { // create a model and the software system we want to describe model model = new model("techtribes.je", "this is a model of the system context for the techtribes.je system, the code for which can be found at https://github.com/techtribesje/techtribesje"); softwaresystem techtribes = model.addsoftwaresystem(location.internal, "techtribes.je", "techtribes.je is the only way to keep up to date with the it, tech and digital sector in jersey and guernsey, channel islands"); // create the various types of people (roles) that use the software system person anonymoususer = model.addperson(location.external, "anonymous user", "anybody on the web."); anonymoususer.uses(techtribes, "view people, tribes (businesses, communities and interest groups), content, events, jobs, etc from the local tech, digital and it sector."); person authenticateduser = model.addperson(location.external, "aggregated user", "a user or business with content that is aggregated into the website."); authenticateduser.uses(techtribes, "manage user profile and tribe membership."); person adminuser = model.addperson(location.external, "administration user", "a system administration user."); adminuser.uses(techtribes, "add people, add tribes and manage tribe membership."); // create the various software systems that techtribes.je has a dependency on softwaresystem twitter = model.addsoftwaresystem(location.external, "twitter", "twitter.com"); techtribes.uses(twitter, "gets profile information and tweets from."); softwaresystem github = model.addsoftwaresystem(location.external, "github", "github.com"); techtribes.uses(github, "gets information about public code repositories from."); softwaresystem blogs = model.addsoftwaresystem(location.external, "blogs", "rss and atom feeds"); techtribes.uses(blogs, "gets content using rss and atom feeds from."); // now create the system context view based upon the model viewset viewset = new viewset(model); systemcontextview contextview = viewset.createcontextview(techtribes); contextview.addallsoftwaresystems(); contextview.addallpeople(); // and output the model and view to json (so that we can render it using structurizr.com) jsonwriter jsonwriter = new jsonwriter(true); stringwriter stringwriter = new stringwriter(); jsonwriter.write(viewset, stringwriter); system.out.println(stringwriter.tostring()); } } executing this code creates this json , which you can then copy and paste into the try it page of structurizr. the result (if you move the boxes around) is something like this. don't worry, there will eventually be an api for uploading software architecture models and the diagrams will get some styling, but it proves the concept. what we have then is an api that implements the various levels in my c4 software architecture model, with a simple browser-based rendering tool. hopefully that's a nice simple introduction of how to represent a software architecture model as code, and gives you a flavour for the sort of direction i'm taking it. having the software architecture as code provides some interesting opportunities that you don't get with static diagrams from visio, etc and the ability to keep the models up to date automatically by scanning the codebase is what i find particularly exciting. if you have any thoughts on this, please do drop me a note.

My objective here is to recreate a similar set-up in a smaller unit test mode.

spring boot already comes with great, pre-configured logging system inside, but in real projects it's important to have an ability to search logs, aggregate them and access easy. one of the easiest option for it is http://papertrailapp.com/ . they provide logging service with syslog protocol and 100mb/mo free plan. lets prepare papertrail for our example: create logging group in papetrail dashboard ("create group" button). create log destination in papertrail dashboard. go to "account -> log destinations" click "create log destination" button. make sure your group is selected in " new systems will join" field. you can leave all others fields with their default values, just click "create". remember your log destination (will looks like logs2.papertrailapp:12345), we will use it later spring boot uses logback as default logging system. it's powerful tool for logging with many logging options. for our purposes we will use ch.qos.logback.classic.net.syslogappender . add "logback.xml" file to your "resources" folder with following content: ${papertrail_host} ${papertrail_port} user ${papertrail_app:-app} %highlight([%.-1level]) %35.35logger{35}:\t%m\t%cyan%ex{5} true i'm using environment variables ( (1), (2), and (3) ) for papertrail's credentials, it will allow you to configure different log destinations in different application environments. note excluded throwable at (4). we already have pattern for throwable in suffixpattern ( %cyan%ex{5} ). why we do this? because otherwise exception stacktraces will be printed after main log message line-by-line, it will increase traffic and also you will not be able to see stacktrace in search. i will demonstrate how it works with really simple spring boot application: package com.github.bsideup.spring.boot.example.papertrail; import org.slf4j.logger; import org.slf4j.loggerfactory; import org.springframework.boot.springapplication; import org.springframework.boot.autoconfigure.springbootapplication; import org.springframework.web.bind.annotation.requestmapping; import org.springframework.web.bind.annotation.restcontroller; @springbootapplication @restcontroller public class application { private static final logger log = loggerfactory.getlogger(application.class); public static void main(string[] args) { springapplication.run(application.class, args); } @requestmapping("/") public string index() { log.warn("i'm so tired to welcome everyone", new exception(new exception())); return "hello world!"; } } now, run your application with following environment variables: papertrail_host - host from your log destination (i.e. logs2.papertrailapp.com) papertrail_port - port from your log destination (i.e. 12345) [optional] papertrail_app - application name (default: app) your papertrails output will looks like mine: you can find sample project at github: https://github.com/bsideup/spring-boot-sample-papertrail

Recently at DevSKiller.com we've decided to move majority of our stuff to simple containers. It was pretty easy due to use of Spring Boot uber-jars, but the problem was in NewRelic agents which should have to be included separately. That caused uncomfortable situation so we decided to solve it by including NewRelic agent into our uber-jar applications. If you also want to simplify your life please follow provided instructions :) At first we have to add proper dependency into our pom.xml descriptor: com.newrelic.agent.java< newrelic-agent 3.12.1 provided Now since we have proper jar included into our project it's time to unpack the dependency to have all necessary classes in our application jar file: org.apache.maven.plugins maven-dependency-plugin 2.9 prepare-package unpack-dependencies newrelic-agent ${project.build.outputDirectory} After this step we've all agent related classes accessible directly from our jar. But still the file cannot be used as an agent jar. There are some important manifest entries that have to be present in every agent jar. The most important is the Premain-Class attribute specifying main agent class including premain() method. In case of NewRelic it's also important to include Can-Redefine-Classes and Can-Retransform-Classes attributes. The easiest way to do that is to extend maven-jar-plugin configuration: org.apache.maven.plugins maven-jar-plugin 2.5 com.newrelic.bootstrap.BootstrapAgent true true Now is coming the tricky part :) NewRelic agent also contains class with main() method which causes that Spring Boot repackager plugin is unable to find single main() method so build fails. It's not a problem but we have to remember to specify proper main class in spring-boot-maven-plugin (or in gradle plugin): my.custom.Application That's all! You can execute your application with following command: java -javaagent:myapp.jar -jar myapp.jar Last but not least: don't forget to include NewRelic configuration file (newrelic.yml) in the same directory as your application jar. The other solution is to set newrelic.config.file system property to point the fully qualified file name.

NEW EDIT: As of Spring Boot 1.4.0, faking of Spring Beans is supported natively via annotation @MockBean. Read Spring Boot docs for more info. OLD EDIT: Here is better example how to mock Spring bean. I've worked with Spring for several years. But I was always frustrated with how messy can XML configuration become. As various annotations and possibilities of Java configuration were popping up, I started to enjoy programming with Spring. That is why I strongly entourage using Java configuration. In my opinion, XML configuration is suitable only when you need to have visualized Spring Integration or Spring Batch flow. Hopefully Spring Tool Suite will be able to visualize Java configurations for these frameworks also. One of the nasty aspects of XML configuration is that it often leads to huge XML configuration files. Developers therefore often create test context configuration for integration testing. But what is the purpose of integration testing, when there isn’t production wiring tested? Such integration test has very little value. So I was always trying to design my production contexts in testable fashion. I except that when you are creating new project / module you would avoid XML configuration as much as possible. So with Java configuration you can create Spring configuration per module / package and scan them in main context (@Configuration is also candidate for component scanning). This way you can naturally create islands Spring beans. These islands can be easily tested in isolation. But I have to admit that it’s not always possible to test production Java configuration as is. Rarely you need to amend behavior or spy on certain beans. There is library for it called Springockito. To be honest I didn’t use it so far, because I always try to design Spring configuration to avoid need for mocking. Looking at Springockito pace of development and number of open issues, I would be little bit worried to introduce it into my test suite stack. Fact that last release was done before Spring 4 release brings up questions like “Is it possible to easily integrate it with Spring 4?”. I don’t know, because I didn’t try it. I prefer pure Spring approach if I need to mock Spring bean in integration test. Spring provides @Primary annotation for specifying which bean should be preferred in the case when two beans with same type are registered. This is handy because you can override production bean with fake bean in integration test. Let’s explore this approach and some pitfalls on examples. I chose this simplistic / dummy production code structure for demonstration: @Repository public class AddressDao { public String readAddress(String userName) { return "3 Dark Corner"; } } @Service public class AddressService { private AddressDao addressDao; @Autowired public AddressService(AddressDao addressDao) { this.addressDao = addressDao; } public String getAddressForUser(String userName){ return addressDao.readAddress(userName); } } @Service public class UserService { private AddressService addressService; @Autowired public UserService(AddressService addressService) { this.addressService = addressService; } public String getUserDetails(String userName){ String address = addressService.getAddressForUser(userName); return String.format("User %s, %s", userName, address); } } AddressDao singleton bean instance is injected into AddressService. AddressService is similarly used in UserService. I have to warn you at this stage. My approach is slightly invasive to production code. To be able to fake existing production beans, we have to register fake beans in integration test. But these fake beans are usually in the same package sub-tree as production beans (assuming you are using standard Maven files structure: “src/main/java” and “src/test/java”). So when they are in the same package sub-tree, they would be scanned during integration tests. But we don’t want to use all bean fakes in all integration tests. Fakes could break unrelated integration tests. So we need to have mechanism, how to tell the test to use only certain fake beans. This is done by excluding fake beans from component scanning completely. Integration test explicitly define which fake/s are being used (will show this later). Now let’s take a look at mechanism of excluding fake beans from component scanning. We define our own marker annotation: public @interface BeanMock { } And exclude @BeanMock annotation from component scanning in main Spring configuration. @Configuration @ComponentScan(excludeFilters = @Filter(BeanMock.class)) @EnableAutoConfiguration public class Application { } Root package of component scan is current package of Application class. So all above production beans needs to be in same package or sub-package. We are now need to create integration test forUserService. Let’s spy on address service bean. Of course such testing doesn’t make practical sense with this production code, but this is just example. So here is our spying bean: @Configuration @BeanMock public class AddressServiceSpy { @Bean @Primary public AddressService registerAddressServiceSpy(AddressService addressService) { return spy(addressService); } } Production AddressService bean is autowired from production context, wrapped into Mockito‘s spy and registered as primary bean for AddressService type. @Primary annotation makes sure that our fake bean will be used in integration test instead of production bean. @BeanMock annotation ensures that this bean can’t be scanned by Application component scanning. Let’s take a look at the integration test now: @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = { Application.class, AddressServiceSpy.class }) public class UserServiceITest { @Autowired private UserService userService; @Autowired private AddressService addressService; @Test public void testGetUserDetails() { // GIVEN - spring context defined by Application class // WHEN String actualUserDetails = userService.getUserDetails("john"); // THEN Assert.assertEquals("User john, 3 Dark Corner", actualUserDetails); verify(addressService, times(1)).getAddressForUser("john"); } } @SpringApplicationConfigration annotation has two parameters. First (Application.class) declares Spring configuration under test. Second parameter (AddressServiceSpy.class) specifies fake bean that will be loaded for our testing into Spring IoC container. It’s obvious that we can use as many bean fakes as needed, but you don’t want to have many bean fakes. This approach should be used rarely and if you observe yourself using such mocking often, you are probably having serious problem with tight coupling in your application or within your development team in general. TDD methodology should help you target this problem. Bear in mind: “Less mocking is always better!”. So consider production design changes that allow for lower usage of mocks. This applies also for unit testing. Within integration test we can autowire this spy bean and use it for various verifications. In this case we verified if testing method userService.getUserDetails called methodaddressService.getAddressForUser with parameter “john”. I have one more example. In this case we wouldn’t spy on production bean. We will mock it: @Configuration @BeanMock public class AddressDaoMock { @Bean @Primary public AddressDao registerAddressDaoMock() { return mock(AddressDao.class); } } Again we override production bean, but this time we replace it with Mockito’s mock. We can than record behavior for mock in our integration test: @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = { Application.class, AddressDaoMock.class }) public class AddressServiceITest { @Autowired private AddressService addressService; @Autowired private AddressDao addressDao; @Test public void testGetAddressForUser() { // GIVEN when(addressDao.readAddress("john")).thenReturn("5 Bright Corner"); // WHEN String actualAddress = addressService.getAddressForUser("john"); // THEN Assert.assertEquals("5 Bright Corner", actualAddress); } @After public void resetMock() { reset(addressDao); } } We load mocked bean via @SpringApplicationConfiguration‘s parameter. In test method, we stubaddressDao.readAddress method to return “5 Bright Corner” string when “john” is passed to it as parameter. But bear in mind that recorded behavior can be carried to different integration test via Spring context. We don’t want tests affecting each other. So you can avoid future problems in your test suite by reseting mocks after test. This is done in method resetMock. Source code is on Github.

Learn all about creating a microservices architecture on Java in this great tutorial.

In my previous articles I compared 4 frameworks commonly used in communicating with MongoDB from the JVM and found out that in that use-case, Spring Data for MongoDB was the easiest solution. However I did make the remark that it doesn’t use the GeoJSON format to store geolocation coordinates and geometries. I tried to add GeoJSON support before, but couldn’t get the conversion to work propertly. But after some extensive searching I found out that the reason for it not working was my use of Spring Boot: its autoconfiguration for MongoDB does not support custom conversion out of the box. Luckily, the solution was simple: provide an extra configuration that extends from AbstractMongoConfiguration and import that in the Boot application. In that configuration you can override the customConversions() and add your converters. When you compare the geo classes in Spring Data and GeoJSON, I noticed that only a subset of GeoJSON geometries can be mapped on Spring Data geo classes: Point and Polygon. Spring Boot does not support LineString, MultiLineString, MultiPolygon or MultiPoint. However, in your mapped domain classes, you won’t use these normally. Creating a converter that adheres to the GeoJSON format is quite straightforward. import com.mongodb.BasicDBObject import com.mongodb.DBObject import org.springframework.core.convert.converter.Converter import org.springframework.data.convert.ReadingConverter import org.springframework.data.convert.WritingConverter import org.springframework.data.geo.Point import org.springframework.data.geo.Polygon final class GeoJsonConverters { static List> getConvertersToRegister() { return [ GeoJsonDBObjectToPointConverter.INSTANCE, GeoJsonDBObjectToPolygonConverter.INSTANCE, GeoJsonPointToDBObjectConverter.INSTANCE, GeoJsonPolygonToDBObjectConverter.INSTANCE ] } @WritingConverter static enum GeoJsonPointToDBObjectConverter implements Converter { INSTANCE; @Override DBObject convert(Point source) { return new BasicDBObject([type: 'Point', coordinates: [source.x, source.y]]) } } @ReadingConverter static enum GeoJsonDBObjectToPointConverter implements Converter { INSTANCE; @Override Point convert(DBObject source) { def coordinates = source.coordinates as double[] return new Point(coordinates[0], coordinates[1]) } } @WritingConverter static enum GeoJsonPolygonToDBObjectConverter implements Converter { INSTANCE; @Override DBObject convert(Polygon source) { def coordinates = source.points.collect { [it.x, it.y] } return new BasicDBObject([type: 'Polygon', coordinates: coordinates]) } } @ReadingConverter static enum GeoJsonDBObjectToPolygonConverter implements Converter { INSTANCE; @Override Polygon convert(DBObject source) { def coordinates = source.coordinates as double[] return new Point(coordinates[0], coordinates[1]) } } } To add those converters to the Spring context, you’ll have to override some methods in your MongoDB spring configuration class. import com.mongodb.Mongo import org.springframework.beans.factory.annotation.* import org.springframework.boot.SpringApplication import org.springframework.boot.autoconfigure.EnableAutoConfiguration import org.springframework.context.annotation.* import org.springframework.data.mongodb.config.AbstractMongoConfiguration import org.springframework.data.mongodb.core.convert.* @EnableAutoConfiguration @ComponentScan @Configuration @Import([MongoComparisonMongoConfiguration]) class MongoComparison { static void main(String[] args) { SpringApplication.run(MongoComparison, args); } } @Configuration class MongoComparisonMongoConfiguration extends AbstractMongoConfiguration { @Autowired Mongo mongo; @Value("\${spring.data.mongodb.database}") String databaseName; @Override protected String getDatabaseName() { return databaseName } @Override Mongo mongo() throws Exception { return mongo } @Override CustomConversions customConversions() { def customConverters = [] customConverters << GeoJsonConverters.convertersToRegister return new CustomConversions(customConverters.flatten()) } } As Spring Boot already provides the configuration of the Mongo instance and the name of the database, we can reuse these in the MongoDB configuration class. The custom conversions take preference over the existing ones for Point and Polygon. I’ll be writing a library this weekend to add support for all GeoJSON geometries in Spring Data for MongoDB. However, I already noticed it’ll be very hard to provide support for those in generated query methods in repositories, but with annotated queries being possible, I don’t think this will be a big issue but we’ll see.

recently some pwc tech supremos wrote an article: agile coding in enterprise it: code small and local . subsections: moving away from the monolith why microservices? msa: a think-small approach for rapid development thinking the msa way: minimalism is a must where msa makes sense in msa, integration is the problem, not the solution conclusion msa is short for microservices architecture(s), in the above article. the article posits that microservices is the antidote to monoliths. it doesn’t mention cookie cutter scaling at all, which is another antidote to monoliths, with the right build infrastructure and devops. here’s a view of hypothetical architecture a company could deploy if they were doing microservices: w is web server. p and q don’t stand for anything in particular. here’s the same solution as cookie-cutter scaling, and the alternate (historical) choice of monolith to the right of it: the cookie cutter approach will often leverage components that are dependency injected into each other, and though monoliths might be the same today, pre 2004 they were probably hairballs of singletons (the design patten, not the springframework idiom). continuous delivery, agile? here’s one excerpt that confuses me: " … makes no sense to design and develop software over an 18-month process to accommodate all possible use cases when those use cases can change unexpectedly and the life span of code modules might be less than 18 months…. as i recall, the 18 month-delay problem was solved previously. agile methodologies principally, and continuous delivery/deployment in more recent times. it does not matter whether you’re compiling a monolith, a cookie-cutter solution, old soa services, or microservices, the 18-month fear isn’t real if you’re doing agile and/or cd. agile and cd were increasing the release cadence, and allowing the organization to pivot faster before microservices. it doesn’t matter whether you’ve got a monolith, something cookie-cutter scaled, or soa (micro or not), you’re going to be able to benefit from agile practices and devops setup that facilitates cd. in something like 30 thoughtworks client engagements since 2002, i have not seen the 18-month process at all. in fact i last encountered it in 1997 on an as/400 project, which was the last time i saw a waterfall process being championed. build(s) and trunk elsewhere there is a suggestion: “each microservice [has] its own build, to avoid trunk conflict”. that isn’t unique to microservices, of course. component based systems today also have a multiple build file (module) structure in a source tree. hopefully “trunk” mentioned is alluding to trunk based development, as i would recommend. build technologies this is a expansion on the above, and you can skip this paragraph if you want. hierarchical build systems like maven has allow you to have one build file per module (whether that’s a service or a simple jar destined for the classpath of a bigger thing). buck has a build grammar that allows for a build to grow/shrink/change based on what is being built (from implicitly shared source). maven is for the java ecosystem, while buck promises to be multi-language. both are doing multi-module builds for the sake of a composed or servicified deployment. both maven and buck are presently competing to draw the most reduced set of compile/test/deploy operations for the changes since last build for a hierarchy of modules. anyway, what is it we are striving for? what we want is to develop cheaply, and to deploy smoothly and often, without defect. we want the ability to deploy without large permanent or temporary headcount overseeing or participating in deployment. aside from development costs, and support/operation, deployment costs are a potentially big factor in total cost of ownership. what i like about cookie-cutter is the uniformity of the deployable things. the team size for deployment of such a thing doesn’t grow with the numbers of nodes that binary is being deployed to. at least, if you’re able to automate the deployment to those nodes, and have a strategy for handling the users connected to the stack at redeployment time somehow (sessions or stateless). the uniformity of the deployment is a cheapener, i think. when you have a number of dissimilar services, you might be able to minimize release personnel if you’re only doing one service. if more than one service is being updated in a particular deployment, you’re going to have to concentrate to make sure you don’t experience a multiplier effect for the participants. it is possible of course, to keep the headcount small, but the practice needed beforehand is bigger, which in turn allows for some calmness around the actual deployment. if we’ve stepped away from the project management office thinking that suggests three buggy releases a year (which is more usual than 18 month schedules of old), then we can employ continuous deployment to further eliminate personnel costs around going live. this is something that microservices does well at, but because the most adept proponents design forwards & backwards compatibility into the permutations most likely to co-exist in production. it is at least much quicker to redeploy and bounce one small service, n times than the the cookie-cutter uniform deployment.

Originally written by Sébastien Deluze on the SpringSource blog Spring Jackson support has been improved lately to be more flexible and powerful. This blog post gives you an update about the most useful Jackson related features available in Spring Framework 4.x and Spring Boot. All the code samples are coming from this spring-jackson-demo sample application, feel free to have a look at the code. JSON Views It can sometimes be useful to filter contextually objects serialized to the HTTP response body. In order to provide such capabilities, Spring MVC now has builtin support for Jackson’s Serialization Views. The following example illustrates how to use @JsonView to filter fields depending on the context of serialization - e.g. getting a "summary" view when dealing with collections, and getting a full representation when dealing with a single resource: public class View { interface Summary {} } public class User { @JsonView(View.Summary.class) private Long id; @JsonView(View.Summary.class) private String firstname; @JsonView(View.Summary.class) private String lastname; private String email; private String address; private String postalCode; private String city; private String country; } public class Message { @JsonView(View.Summary.class) private Long id; @JsonView(View.Summary.class) private LocalDate created; @JsonView(View.Summary.class) private String title; @JsonView(View.Summary.class) private User author; private List recipients; private String body; } Thanks to Spring MVC @JsonView support, it is possible to choose, on a per handler method basis, which field should be serialized: @RestController public class MessageController { @Autowired private MessageService messageService; @JsonView(View.Summary.class) @RequestMapping("/") public List getAllMessages() { return messageService.getAll(); } @RequestMapping("/{id}") public Message getMessage(@PathVariable Long id) { return messageService.get(id); } } In this example, if all messages are retrieved, only the most important fields are serialized thanks to the getAllMessages() method annotated with@JsonView(View.Summary.class): [ { "id" : 1, "created" : "2014-11-14", "title" : "Info", "author" : { "id" : 1, "firstname" : "Brian", "lastname" : "Clozel" } }, { "id" : 2, "created" : "2014-11-14", "title" : "Warning", "author" : { "id" : 2, "firstname" : "Stéphane", "lastname" : "Nicoll" } }, { "id" : 3, "created" : "2014-11-14", "title" : "Alert", "author" : { "id" : 3, "firstname" : "Rossen", "lastname" : "Stoyanchev" } } ] In Spring MVC default configuration, MapperFeature.DEFAULT_VIEW_INCLUSION is set tofalse. That means that when enabling a JSON View, non annotated fields or properties likebody or recipients are not serialized. When a specific Message is retrieved using the getMessage() handler method (no JSON View specified), all fields are serialized as expected: { "id" : 1, "created" : "2014-11-14", "title" : "Info", "body" : "This is an information message", "author" : { "id" : 1, "firstname" : "Brian", "lastname" : "Clozel", "email" : "[email protected]", "address" : "1 Jaures street", "postalCode" : "69003", "city" : "Lyon", "country" : "France" }, "recipients" : [ { "id" : 2, "firstname" : "Stéphane", "lastname" : "Nicoll", "email" : "[email protected]", "address" : "42 Obama street", "postalCode" : "1000", "city" : "Brussel", "country" : "Belgium" }, { "id" : 3, "firstname" : "Rossen", "lastname" : "Stoyanchev", "email" : "[email protected]", "address" : "3 Warren street", "postalCode" : "10011", "city" : "New York", "country" : "USA" } ] } Only one class or interface can be specified with the @JsonView annotation, but you can use inheritance to represent JSON View hierarchies (if a field is part of a JSON View, it will be also part of parent view). For example, this handler method will serialize fields annotated with@JsonView(View.Summary.class) and @JsonView(View.SummaryWithRecipients.class): public class View { interface Summary {} interface SummaryWithRecipients extends Summary {} } public class Message { @JsonView(View.Summary.class) private Long id; @JsonView(View.Summary.class) private LocalDate created; @JsonView(View.Summary.class) private String title; @JsonView(View.Summary.class) private User author; @JsonView(View.SummaryWithRecipients.class) private List recipients; private String body; } @RestController public class MessageController { @Autowired private MessageService messageService; @JsonView(View.SummaryWithRecipients.class) @RequestMapping("/with-recipients") public List getAllMessagesWithRecipients() { return messageService.getAll(); } } JSON Views could also be specified when using RestTemplate HTTP client orMappingJackson2JsonView by wrapping the value to serialize in a MappingJacksonValue as shown in this code sample. JSONP As described in the reference documentation, you can enable JSONP for @ResponseBody andResponseEntity methods by declaring an @ControllerAdvice bean that extendsAbstractJsonpResponseBodyAdvice as shown below: @ControllerAdvice public class JsonpAdvice extends AbstractJsonpResponseBodyAdvice { public JsonpAdvice() { super("callback"); } } With such @ControllerAdvice bean registered, it will be possible to request the JSON webservice from another domain using a In this example, the received payload would be: parseResponse({ "id" : 1, "created" : "2014-11-14", ... }); JSONP is also supported and automatically enabled when using MappingJackson2JsonViewwith a request that has a query parameter named jsonp or callback. The JSONP query parameter name(s) could be customized through the jsonpParameterNames property. XML support Since 2.0 release, Jackson provides first class support for some other data formats than JSON. Spring Framework and Spring Boot provide builtin support for Jackson based XML serialization/deserialization. As soon as you include the jackson-dataformat-xml dependency to your project, it is automatically used instead of JAXB2. Using Jackson XML extension has several advantages over JAXB2: Both Jackson and JAXB annotations are recognized JSON View are supported, allowing you to build easily REST Webservices with the same filtered output for both XML and JSON data formats No need to annotate your class with @XmlRootElement, each class serializable in JSON will serializable in XML You usually also want to make sure that the XML library in use is Woodstox since: It is faster than Stax implementation provided with the JDK It avoids some known issues like adding unnecessary namespace prefixes Some features like pretty print don't work without it In order to use it, simply add the latest woodstox-core-asl dependency available to your project. Customizing the Jackson ObjectMapper Prior to Spring Framework 4.1.1, Jackson HttpMessageConverters were usingObjectMapper default configuration. In order to provide a better and easily customizable default configuration, a new Jackson2ObjectMapperBuilder has been introduced. It is the JavaConfig equivalent of the well known Jackson2ObjectMapperFactoryBean used in XML configuration. Jackson2ObjectMapperBuilder provides a nice API to customize various Jackson settings while retaining Spring Framework provided default ones. It also allows to createObjectMapper and XmlMapper instances based on the same configuration. Both Jackson2ObjectMapperBuilder and Jackson2ObjectMapperFactoryBean define a better Jackson default configuration. For example, theDeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES property set to false, in order to allow deserialization of JSON objects with unmapped properties. Jackson support for Java 8 Date & Time API data types is automatically registered when Java 8 is used and jackson-datatype-jsr310 is on the classpath. Joda-Time support is registered as well when jackson-datatype-joda is part of your project dependencies. These classes also allow you to register easily Jackson mixins, modules, serializers or even property naming strategy like PropertyNamingStrategy.CAMEL_CASE_TO_LOWER_CASE_WITH_UNDERSCORES if you want to have your userName java property translated to user_name in JSON. With Spring Boot As described in the Spring Boot reference documentation, there are various ways tocustomize the Jackson ObjectMapper. You can for example enable/disable Jackson features easily by adding properties likespring.jackson.serialization.indent_output=true to application.properties. As an alternative, in the upcoming 1.2 release Spring Boot also allows to customize the Jackson configuration (JSON and XML) used by Spring MVC HttpMessageConverters by declaring a Jackson2ObjectMapperBuilder @Bean: @Bean public Jackson2ObjectMapperBuilder jacksonBuilder() { Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder(); builder.indentOutput(true).dateFormat(new SimpleDateFormat("yyyy-MM-dd")); return builder; } This is useful if you want to use advanced Jackson configuration not exposed through regular configuration keys. Without Spring Boot In a plain Spring Framework application, you can also use Jackson2ObjectMapperBuilder to customize the XML and JSON HttpMessageConverters as shown bellow: @Configuration @EnableWebMvc public class WebConfiguration extends WebMvcConfigurerAdapter { @Override public void configureMessageConverters(List> converters) { Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder(); builder.indentOutput(true).dateFormat(new SimpleDateFormat("yyyy-MM-dd")); converters.add(new MappingJackson2HttpMessageConverter(builder.build())); converters.add(new MappingJackson2XmlHttpMessageConverter(builder.createXmlMapper(true).build())); } } More to come With the upcoming Spring Framework 4.1.3 release, thanks to the addition of a Spring context aware HandlerInstantiator (see SPR-10768 for more details), you will be able to autowire Jackson handlers (serializers, deserializers, type and type id resolvers). This will allow you to build, for example, a custom deserializer that will replace a field containing only a reference in the JSON payload by the full Entity retrieved from the database.

Originally written by Artem Bilan on the SpringSource blog. Dear Spring Community! Recently we published the Spring Integration Java DSL: Line by line tutorial, which uses Java 8 Lambdas extensively. We received some feedback that this is good introduction to the DSL, but a similar tutorial is needed for those users, who can't move to the Java 8 or aren't yet familiar with Lambdas, but wish to take advantage So, to help those Spring Integration users who want to moved from XML configuration to Java & Annotation configuration, we provide this line-by-line tutorial to demonstrate that, even without Lambdas, we gain a lot from Spring Integration Java DSL usage. Although, most will agree that the lambda syntax provides for a more succinct definition. We analyse here the same Cafe Demo sample, but using the pre Java 8 variant for configuration. Many options are the same, so we just copy/paste their description here to achieve a complete picture. Since this Spring Integration Java DSL configuration is quite different to the Java 8 lambda style, it will be useful for all users to get a knowlage how we can achieve the same result with a rich variety of options provided by the Spring Integration Java DSL. The source code for our application is placed in a single class, which is a Boot application; significant lines are annotated with a number corresponding to the comments, which follow: @SpringBootApplication // 1 @IntegrationComponentScan // 2 public class Application { public static void main(String[] args) throws Exception { ConfigurableApplicationContext ctx = SpringApplication.run(Application.class, args); // 3 Cafe cafe = ctx.getBean(Cafe.class); // 4 for (int i = 1; i <= 100; i++) { // 5 Order order = new Order(i); order.addItem(DrinkType.LATTE, 2, false); order.addItem(DrinkType.MOCHA, 3, true); cafe.placeOrder(order); } System.out.println("Hit 'Enter' to terminate"); // 6 System.in.read(); ctx.close(); } @MessagingGateway // 7 public interface Cafe { @Gateway(requestChannel = "orders.input") // 8 void placeOrder(Order order); // 9 } private final AtomicInteger hotDrinkCounter = new AtomicInteger(); private final AtomicInteger coldDrinkCounter = new AtomicInteger(); // 10 @Autowired private CafeAggregator cafeAggregator; // 11 @Bean(name = PollerMetadata.DEFAULT_POLLER) public PollerMetadata poller() { // 12 return Pollers.fixedDelay(1000).get(); } @Bean @SuppressWarnings("unchecked") public IntegrationFlow orders() { // 13 return IntegrationFlows.from("orders.input") // 14 .split("payload.items", (Consumer) null) // 15 .channel(MessageChannels.executor(Executors.newCachedThreadPool()))// 16 .route("payload.iced", // 17 new Consumer>() { // 18 @Override public void accept(RouterSpec spec) { spec.channelMapping("true", "iced") .channelMapping("false", "hot"); // 19 } }) .get(); // 20 } @Bean public IntegrationFlow icedFlow() { // 21 return IntegrationFlows.from(MessageChannels.queue("iced", 10)) // 22 .handle(new GenericHandler() { // 23 @Override public Object handle(OrderItem payload, Map headers) { Uninterruptibles.sleepUninterruptibly(1, TimeUnit.SECONDS); System.out.println(Thread.currentThread().getName() + " prepared cold drink #" + coldDrinkCounter.incrementAndGet() + " for order #" + payload.getOrderNumber() + ": " + payload); return payload; // 24 } }) .channel("output") // 25 .get(); } @Bean public IntegrationFlow hotFlow() { // 26 return IntegrationFlows.from(MessageChannels.queue("hot", 10)) .handle(new GenericHandler() { @Override public Object handle(OrderItem payload, Map headers) { Uninterruptibles.sleepUninterruptibly(5, TimeUnit.SECONDS); // 27 System.out.println(Thread.currentThread().getName() + " prepared hot drink #" + hotDrinkCounter.incrementAndGet() + " for order #" + payload.getOrderNumber() + ": " + payload); return payload; } }) .channel("output") .get(); } @Bean public IntegrationFlow resultFlow() { // 28 return IntegrationFlows.from("output") // 29 .transform(new GenericTransformer() { // 30 @Override public Drink transform(OrderItem orderItem) { return new Drink(orderItem.getOrderNumber(), orderItem.getDrinkType(), orderItem.isIced(), orderItem.getShots()); // 31 } }) .aggregate(new Consumer() { // 32 @Override public void accept(AggregatorSpec aggregatorSpec) { aggregatorSpec.processor(cafeAggregator, null); // 33 } }, null) .handle(CharacterStreamWritingMessageHandler.stdout()) // 34 .get(); } @Component public static class CafeAggregator { // 35 @Aggregator // 36 public Delivery output(List drinks) { return new Delivery(drinks); } @CorrelationStrategy // 37 public Integer correlation(Drink drink) { return drink.getOrderNumber(); } } } Examining the code line by line... 1. @SpringBootApplication This new meta-annotation from Spring Boot 1.2. Includes @Configuration and@EnableAutoConfiguration. Since we are in a Spring Integration application and Spring Boot has auto-configuration for it, the @EnableIntegration is automatically applied, to initialize the Spring Integration infrastructure including an environment for the Java DSL -DslIntegrationConfigurationInitializer, which is picked up by theIntegrationConfigurationBeanFactoryPostProcessor from /META-INF/spring.factories. 2. @IntegrationComponentScan The Spring Integration analogue of @ComponentScan to scan components based on interfaces, (the Spring Framework's @ComponentScan only looks at classes). Spring Integration supports the discovery of interfaces annotated with @MessagingGateway (see #7 below). 3. ConfigurableApplicationContext ctx = SpringApplication.run(Application.class, args); The main method of our class is designed to start the Spring Boot application using the configuration from this class and starts an ApplicationContext via Spring Boot. In addition, it delegates command line arguments to the Spring Boot. For example you can specify --debug to see logs for the boot auto-configuration report. 4. Cafe cafe = ctx.getBean(Cafe.class); Since we already have an ApplicationContext we can start to interact with application. AndCafe is that entry point - in EIP terms a gateway. Gateways are simply interfaces and the application does not interact with the Messaging API; it simply deals with the domain (see #7 below). 5. for (int i = 1; i <= 100; i++) { To demonstrate the cafe "work" we intiate 100 orders with two drinks - one hot and one iced. And send the Order to the Cafe gateway. 6. System.out.println("Hit 'Enter' to terminate"); Typically Spring Integration application are asynchronous, hence to avoid early exit from themain Thread we block the main method until some end-user interaction through the command line. Non daemon threads will keep the application open but System.read()provides us with a mechanism to close the application cleanly. 7. @MessagingGateway The annotation to mark a business interface to indicate it is a gateway between the end-application and integration layer. It is an analogue of component from Spring Integration XML configuration. Spring Integration creates a Proxy for this interface and populates it as a bean in the application context. The purpose of this Proxy is to wrap parameters in a Message object and send it to the MessageChannel according to the provided options. 8. @Gateway(requestChannel = "orders.input") The method level annotation to distinct business logic by methods as well as by the target integration flows. In this sample we use a requestChannel reference of orders.input, which is a MessageChannel bean name of our IntegrationFlow input channel (see below #14). 9. void placeOrder(Order order); The interface method is a central point to interact from end-application with the integration layer. This method has a void return type. It means that our integration flow is one-wayand we just send messages to the integration flow, but don't wait for a reply. 10. private AtomicInteger hotDrinkCounter = new AtomicInteger(); private AtomicInteger coldDrinkCounter = new AtomicInteger(); Two counters to gather the information how our cafe works with drinks. 11. @Autowired private CafeAggregator cafeAggregator; The POJO for the Aggregator logic (see #33 and #35 below). Since it is a Spring bean, we can simply inject it even to the current @Configuration and use in any place below, e.g. from the .aggregate() EIP-method. 12. @Bean(name = PollerMetadata.DEFAULT_POLLER) public PollerMetadata poller() { The default poller bean. It is a analogue of component from Spring Integration XML configuration. Required for endpoints where the inputChannelis a PollableChannel. In this case, it is necessary for the two Cafe queues - hot and iced (see below #18). Here we use the Pollers factory from the DSL project and use its method-chain fluent API to build the poller metadata. Note that Pollers can be used directly from an IntegrationFlow definition, if a specific poller (rather than the default poller) is needed for an endpoint. 13. @Bean public IntegrationFlow orders() { The IntegrationFlow bean definition. It is the central component of the Spring Integration Java DSL, although it does not play any role at runtime, just during the bean registration phase. All other code below registers Spring Integration components (MessageChannel,MessageHandler, EventDrivenConsumer, MessageProducer, MessageSource etc.) in theIntegrationFlow object, which is parsed by the IntegrationFlowBeanPostProcessor to process those components and register them as beans in the application context as necessary (some elements, such as channels may already exist). 14. return IntegrationFlows.from("orders.input") The IntegrationFlows is the main factory class to start the IntegrationFlow. It provides a number of overloaded .from() methods to allow starting a flow from aSourcePollingChannelAdapter for a MessageSource implementations, e.g.JdbcPollingChannelAdapter; from a MessageProducer, e.g.WebSocketInboundChannelAdapter; or simply a MessageChannel. All ".from()" options have several convenient variants to configure the appropriate component for the start of theIntegrationFlow. Here we use just a channel name, which is converted to aDirectChannel bean definition during the bean definition phase while parsing theIntegrationFlow. In the Java 8 variant, we used here a Lambda definition - and thisMessageChannel has been implicitly created with the bean name based on theIntegrationFlow bean name. 15. .split("payload.items", (Consumer) null) Since our integration flow accepts messages through the orders.input channel, we are ready to consume and process them. The first EIP-method in our scenario is .split(). We know that the message payload from orders.input channel is an Order domain object, so we can simply use here a Spring (SpEL) Expression to return Collection. So, this performs the split EI pattern, and we send each collection entry as a separate message to the next channel. In the background, the .split() method registers aExpressionEvaluatingSplitter MessageHandler implementation and anEventDrivenConsumer for that MessageHandler, wiring in the orders.input channel as the inputChannel. The second argument for the .split() EIP-method is for an endpointConfigurer to customize options like autoStartup, requiresReply, adviceChain etc. We use herenull to show that we rely on the default options for the endpoint. Many of EIP-methods provide overloaded versions with and without endpointConfigurer. Currently.split(String expression) EIP-method without the endpointConfigurer argument is not available; this will be addressed in a future release. 16. .channel(MessageChannels.executor(Executors.newCachedThreadPool())) The .channel() EIP-method allows the specification of concrete MessageChannels between endpoints, as it is done via output-channel/input-channel attributes pair with Spring Integration XML configuration. By default, endpoints in the DSL integration flow definition are wired with DirectChannels, which get bean names based on theIntegrationFlow bean name and index in the flow chain. In this case we select a specificMessageChannel implementation from the Channels factory class; the selected channel here is an ExecutorChannel, to allow distribution of messages from the splitter to separate Threads, to process them in parallel in the downstream flow. 17. .route("payload.iced", The next EIP-method in our scenario is .route(), to send hot/iced order items to different Cafe kitchens. We again use here a SpEL expression to get the routingKey from the incoming message. In the Java 8 variant, we used a method-reference Lambda expression, but for pre Java 8 style we must use SpEL or an inline interface implementation. Many anonymous classes in a flow can make the flow difficult to read so we prefer SpEL in most cases. 18. new Consumer>() { The second argument of .route() EIP-method is a functional interface Consumer to specify ExpressionEvaluatingRouter options using a RouterSpec Builder. Since we don't have any choice with pre Java 8, we just provide here an inline implementation for this interface. 19. spec.channelMapping("true", "iced") .channelMapping("false", "hot"); With the Consumer>#accept()implementation we can provide desired AbstractMappingMessageRouter options. One of them is channelMappings, when we specify the routing logic by the result of router expresion and the target MessageChannel for the apropriate result. In this case iced andhot are MessageChannel names for IntegrationFlows below. 20. .get(); This finalizes the flow. Any IntegrationFlows.from() method returns anIntegrationFlowBuilder instance and this get() method extracts an IntegrationFlowobject from the IntegrationFlowBuilder configuration. Everything starting from the.from() and up to the method before the .get() is an IntegrationFlow definition. All defined components are stored in the IntegrationFlow and processed by theIntegrationFlowBeanPostProcessor during the bean creation phase. 21. @Bean public IntegrationFlow icedFlow() { This is the second IntegrationFlow bean definition - for iced drinks. Here we demonstrate that several IntegrationFlows can be wired together to create a single complex application. Note: it isn't recommended to inject one IntegrationFlow to another; it might cause unexpected behaviour. Since they provide Integration components for the bean registration and MessageChannels one of them, the best way to wire and inject is viaMessageChannel or @MessagingGateway interfaces. 22. return IntegrationFlows.from(MessageChannels.queue("iced", 10)) The iced IntegrationFlow starts from a QueueChannel that has a capacity of 10messages; it is registered as a bean with the name iced. As you remember we use this name as one of the route mappings (see above #19). In our sample, we use here a restricted QueueChannel to reflect the Cafe kitchen busy state from real life. And here is a place where we need that global poller for the next endpoint which is listening on this channel. 23. .handle(new GenericHandler() { The .handle() EIP-method of the iced flow demonstrates the concrete Cafe kitchen work. Since we can't minimize the code with something like Java 8 Lambda expression, we provide here an inline implementation for the GenericHandler functional interface with the expected payload type as the generic argument. With the Java 8 example, we distribute this.handle() between several subscriber subflows for a PublishSubscribeChannel. However in this case, the logic is all implemented in the one method. 24. Uninterruptibles.sleepUninterruptibly(1, TimeUnit.SECONDS); System.out.println(Thread.currentThread().getName() + " prepared cold drink #" + coldDrinkCounter.incrementAndGet() + " for order #" + payload.getOrderNumber() + ": " + payload); return payload; The business logic implementation for the current .handle() EIP-component. WithUninterruptibles.sleepUninterruptibly(1, TimeUnit.SECONDS); we just block the current Thread for some timeout to demonstrate how quickly the Cafe kitchen prepares a drink. After that we just report to STDOUT that the drink is ready and return the currentOrderItem from the GenericHandler for the next endpoint in our IntegrationFlow. In the background, the DSL framework registers a ServiceActivatingHandler for theMethodInvokingMessageProcessor to invoke the GenericHandler#handle at runtime. In addition, the framework registers a PollingConsumer endpoint for the QueueChannelabove. This endpoint relies on the default poller to poll messages from the queue. Of course, we always can use a specific poller for any concrete endpoint. In that case, we would have to provide a second endpointConfigurer argument to the .handle() EIP-method. 25. .channel("output") Since it is not the end of our Cafe scenario, we send the result of the current flow to theoutput channel using the convenient EIP-method .channel() and the name of theMessageChannel bean (see below #29). This is the logical end of the current iced drink subflow, so we use the .get() method to return the IntegrationFlow. Flows that end with a reply-producing handler that don't have a final .channel() will return the reply to the message replyChannel header. 26. @Bean public IntegrationFlow hotFlow() { The IntegrationFlow definition for hot drinks. It is similar to the previous iced drinks flow, but with specific hot business logic. It starts from the hot QueueChannel which is mapped from the router above. 27. Uninterruptibles.sleepUninterruptibly(5, TimeUnit.SECONDS); The sleepUninterruptibly for hot drinks. Right, we need more time to boil the water! 28. @Bean public IntegrationFlow resultFlow() { One more IntegrationFlow bean definition to prepare the Delivery for the Cafe client based on the Drinks. 29. return IntegrationFlows.from("output") The resultFlow starts from the DirectChannel, which is created during the bean definition phase with this provided name. You should remember that we use the outputchannel name from the Cafe kitchens flows in the last .channel() in those definitions. 30. .transform(new GenericTransformer() { The .transform() EIP-method is for the appropriate pattern implementation and expects some object to convert one payload to another. In our sample we use an inline implementation of the GenericTransformer functional interface to convert OrderItem to Drink and we specify that using generic arguments. In the background, the DSL framework registers aMessageTransformingHandler and an EventDrivenConsumer endpoint with default options to consume messages from the output MessageChannel. 31. public Drink transform(OrderItem orderItem) { return new Drink(orderItem.getOrderNumber(), orderItem.getDrinkType(), orderItem.isIced(), orderItem.getShots()); } The business-specific GenericTransformer#transform() implementation to demonstrate how we benefit from Java Generics to transform one payload to another. Note: Spring Integration uses ConversionService before any method invocation and if you provide some specific Converter implementation, some domain payload can be converted to another automatically, when the framework has an appropriate registered Converter. 32. .aggregate(new Consumer() { The .aggregate() EIP-method provides options to configure anAggregatingMessageHandler and its endpoint, similar to what we can do with the component when using Spring Integration XML configuration. Of course, with the Java DSL we have more power to configure the aggregator in place, without any other extra beans. However we demonstrate here an aggregator configuration with annotations (see below #35). From the Cafe business logic perspective we compose the Delivery for the initial Order, since we .split() the original order to the OrderItems near the beginning. 33. public void accept(AggregatorSpec aggregatorSpec) { aggregatorSpec.processor(cafeAggregator, null); } An inline implementation of the Consumer for the AggregatorSpec. Using theaggregatorSpec Builder we can provide desired options for the aggregator component, which will be registered as an AggregatingMessageHandler bean. Here we just provide theprocessor as a reference to the autowired (see #11 above) CafeAggregator component (see #35 below). The second argument of the .processor() option is methodName. Since we are relying on the aggregator annotation configuration for the POJO, we don't need to provide the method here and the framework will determine the correct POJO methods in the background. 34. .handle(CharacterStreamWritingMessageHandler.stdout()) It is the end of our flow - the Delivery is delivered to the client! We just print here the message payload to STDOUT using out-of-the-boxCharacterStreamWritingMessageHandler from Spring Integration Core. This is a case to show how existing components from Spring Integration Core (and its modules) can be used from the Java DSL. 35. @Component public static class CafeAggregator { The bean to specify the business logic for the aggregator above. This bean is picked up by the @ComponentScan, which is a part of the @SpringBootApplication meta-annotation (see above #1). So, this component becomes a bean and we can automatically wire (@Autowired) it to other components in the application context (see #11 above). 36. @Aggregator public Delivery output(List drinks) { return new Delivery(drinks); } The POJO-specific MessageGroupProcessor to build the output payload based on the payloads from aggregated messages. Since we mark this method with the @Aggregatorannotation, the target AggregatingMessageHandler can extract this method for theMethodInvokingMessageGroupProcessor. 37. @CorrelationStrategy public Integer correlation(Drink drink) { return drink.getOrderNumber(); } The POJO-specific CorrelationStrategy to extract the custom correlationKey from each inbound aggregator message. Since we mark this method with @CorrelationStrategyannotation the target AggregatingMessageHandler can extract this method for theMethodInvokingCorrelationStrategy. There is a similar self-explained@ReleaseStrategy annotation, but we rely in our Cafe sample just on the defaultSequenceSizeReleaseStrategy, which is based on the sequenceDetails message header populated by the splitter from the beginning of our integration flow. Well, we have finished describing the Cafe Demo sample based on the Spring Integration Java DSL when Java Lambda support is not available. Compare it with XML sample and also seeLambda support tutorial to get more information regarding Spring Integration. As you can see, using the DSL without lambdas is a little more verbose because you need to provide boilerplate code for inline anonymous implementations of functional interfaces. However, we believe it is important to support the use of the DSL for users who can't yet move to Java 8. Many of the DSL benefits (fluent API, compile-time validation etc) are available for all users. The use of lambdas continues the Spring Framework tradition of reducing or eliminating boilerplate code, so we encourage users to try Java 8 and lambdas and to encourage their organizations to consider allowing the use of Java 8 for Spring Integration applications. In addition see the Reference Manual for more information. As always, we look forward to your comments and feedback (StackOverflow (spring-integration tag), Spring JIRA, GitHub) and we very much welcome contributions! Thank you for your time and patience to read this!

When I needed to do prototyping, proof of concept or play with some new technology in free time, starting new project was always a little annoying barrier with Maven. Have to say that setting up Maven project is not hard and you can use Maven Archetypes. But Archetypes are often out of date. Who wants to play with old technologies? So I always end up wiring in dependencies I wanted to play with. Not very productive spent time. But than Spring Boot came to my way. I fell in love. In last few months I created at least 50 small playground projects, prototypes with Spring Boot. Also incorporated it at work. It’s just perfect for prototyping, learning, microservices, web, batch, enterprise, message flow or command line applications. You have to be dinosaur or be blind not to evaluate Spring Boot for your next Spring project. And when you finish evaluate it, you will go for it. I promise. I feel a need to highlight how easy is Black Box Testing of Spring Boot microservice. Black Box Testing refers to testing without any poking with application artifact. Such testing can be called also integration testing. You can also perform performance or stress testing way I am going to demonstrate. Spring Boot Microservice is usually web application with embedded Tomcat. So it is executed as JAR from command line. There is possibility to convert Spring Boot project into WAR artifact, that can be hosted on shared Servlet container. But we don’t want that now. It’s better when microservice has its own little embedded container. I used existing Spring’s REST service guide as testing target. Focus is mostly on testing project, so it is handy to use this “Hello World” REST application as example. I expect these two common tools are set up and installed on your machine: Maven 3 Git So we’ll need to download source code and install JAR artifact into our local repository. I am going to use command line to download and install the microservice. Let’s go to some directory where we download source code. Use these commands: git clone [email protected]:spring-guides/gs-rest-service.git cd gs-rest-service/complete mvn clean install If everything went OK, Spring Boot microservice JAR artifact is now installed in our local Maven repository. In serious Java development, it would be rather installed into shared repository (e.g. Artifactory, Nexus,… ). When our microservice is installed, we can focus on testing project. It is also Maven and Spring Boot based. Black box testing will be achieved by downloading the artifact from Maven repository (doesn’t matter if it is local or remote). Maven-dependency-plugin can help us this way: org.apache.maven.plugins maven-dependency-plugin copy-dependencies compile copy-dependencies gs-rest-service true It downloads microservice artifact into target/dependency directory by default. As you can see, it’s hooked to compile phase of Maven lifecycle, so that downloaded artifact is available during test phase. Artifact version is stripped from version information. We use latest version. It makes usage of JAR artifact easier during testing. Readers skilled with Maven may notice missing plugin version. Spring Boot driven project is inherited from parent Maven project called spring-boot-starter-parent. It contains versions of main Maven plugins. This is one of the Spring Boot’s opinionated aspects. I like it, because it provides stable dependencies matrix. You can change the version if you need. When we have artifact in our file system, we can start testing. We need to be able to execute JAR file from command line. I used standard JavaProcessBuilder this way: public class ProcessExecutor { public Process execute(String jarName) throws IOException { Process p = null; ProcessBuilder pb = new ProcessBuilder("java", "-jar", jarName); pb.directory(new File("target/dependency")); File log = new File("log"); pb.redirectErrorStream(true); pb.redirectOutput(Redirect.appendTo(log)); p = pb.start(); return p; } } This class executes given process JAR based on given file name. Location is hard-coded to target/dependency directory, where maven-dependency-plugin located our artifact. Standard and error outputs are redirected to file. Next class needed for testing is DTO (Data transfer object). It is simple POJO that will be used for deserialization from JSON. I use Lombok project to reduce boilerplate code needed for getters, setters, hashCode and equals. @Data @AllArgsConstructor @NoArgsConstructor public class Greeting { private long id; private String content; } Test itself looks like this: public class BlackBoxTest { private static final String RESOURCE_URL = "http://localhost:8080/greeting"; @Test public void contextLoads() throws InterruptedException, IOException { Process process = null; Greeting actualGreeting = null; try { process = new ProcessExecutor().execute("gs-rest-service.jar"); RestTemplate restTemplate = new RestTemplate(); waitForStart(restTemplate); actualGreeting = restTemplate.getForObject(RESOURCE_URL, Greeting.class); } finally { process.destroyForcibly(); } Assert.assertEquals(new Greeting(2L, "Hello, World!"), actualGreeting); } private void waitForStart(RestTemplate restTemplate) { while (true) { try { Thread.sleep(500); restTemplate.getForObject(RESOURCE_URL, String.class); return; } catch (Throwable throwable) { // ignoring errors } } } } It executes Spring Boot microservice process first and wait unit it starts. To verify if microservice is started, it sends HTTP request to URL where it’s expected. The service is ready for testing after first successful response. Microservice should send simple greeting JSON response for HTTP GET request. Deserialization from JSON into our Greeting DTO is verified at the end of the test. Source code is shared on Github.

[This article was written by Yaron Parasol.] Docker containers were created to help enable the fast, and reliable deployment of application components or tiers, by creating a container that holds a self-contained ready to deploy parts of applications, with the middleware and the app business logic needed to run them successfully. For example, a Spring application within a Tomcat container. By design, Docker is purposely an isolated self-contained part of the application, typically one tier or even one node in a tier. However, an application is typically multi-tier in its architecture and that means you have tiers with dependencies between them, where the nature of the dependencies can be anything from network connections and remote API invocations, to exchange of messages between application tiers. And hence an app is a set of different containers with specific configurations. This is why you need a way to glue the pieces of your app together. While, Docker has a basic solution for connecting containers using a Docker bridge, this solution is not always the preferred one, especially when deploying the container across different hosts and you need to take care of real network settings. Docker orchestration with TOSCA + Cloudify. Check it out. Go So, what role does the orchestrator play? The orchestrator will take care of two things: The timing of container creation - as containers need to be created by order of dependencies and Container configuration in order to allow containers to communicate with one another - and for that the orchestrator needs to pass runtime properties between containers. As a side note here: With Docker you need a special tweak here, as you typically don’t touch config files inside a container, you keep the container intact, so there is an interesting workaround for cases that this is required. One method to do this is by using a YAML-based orchestration plan to orchestrate the deployment of apps and post-deployment automation processes, which is the approach Cloudify employs. Based on TOSCA (topology and orchestration standard of cloud apps), this orchestration plan describes the components and their lifecycle, and the relationships between components, especially when it comes to complex topologies. This includes, what’s connected to what, what’s hosted on what, and other such considerations. TOSCA is able to describe the infrastructure, as well as, the middleware tier, and app layers on top of these. Cloudify basically takes this TOSCA orchestration plan (dubbed blueprints in Cloudify speak) and materializes these using workflows that traverse the graph of components, or this plan of components and issues commands to agents. These then create the app components and glue them together. The agents use extensions called plugins that are adaptors between the Cloudify configuration and the various infrastructure as a service (IaaS) and automation tools’ APIs. In our case, we created a plugin to interface with the Docker API. Introducing the Docker Cloudify Plugin The Cloudify-Docker plugin is quite straightforward, it installs the Docker API endpoint/server on the machine and then uses the Docker-Py binding to create, configure, and remove containers. TOSCA lifecycle events are: Create - installation of the app components Configure - configuration of the component Start - startup/running the component There is also stop & delete - for shutdown and removal We started by using the create - to create the container, we did not implement configure at the beginning, and start to run the application. But then we realized that for containers with dependencies we need to have runtime properties, such as IP import of the counterpart container in order to create the container for example. When we create an app server container, we need the port and IP of the database container. So, we pushed the creation of the container to the configure event, and used a TOSCA relationship pre-configure hook, to get the dependent container’s info at runtime. The way to expose the runtime info to the container with the dependencies is by setting them as environment variables. 01.interfaces: 02. cloudify.interfaces.lifecycle: 03. configure: 04. implementation: docker.docker_plugin.tasks.configure 05. inputs: 06. container_config: 07. command: mongod--rest--httpinterface --smallfiles 08. image: dockerfile/mongodb 09. start: 10. implementation: docker.docker_plugin.tasks.run 11. inputs: 12. container_start: 13. port_bindings: 14. 27017: 27017 15. 28017: 28017 Nodecellar Example I’d like to explain how this works by using our Nodecellar app as an example. The Nodecellar app is composed of two hosts that, in this case, Cloudify didn’t create but just SSHed into and then installed agents on. On one we have the MongoD container, with a MongoD process. On the other we have the Nodecellar container with NodeJS and the Nodecellar app within it. The Nodecellar container needs a connection to the MongoD container to run the app queries when the app starts. Ultimately, an orchestrator should not be limited to software deployment, the whole idea behind Docker Is to allow for agility, so we’d also like to use Docker in situations of auto-scale out and auto-heal, CD. In our next post we’ll show exactly that - how Cloudify can be used with Docker for post-deployment scenarios.

Originally authored by Artem Bilan on the SpringSource blog Dear Spring Community! Just after the Spring Integration Java DSL 1.0 GA release announcement I want to introduce the Spring Integration Java DSL to you as a line by line tutorial based on the classic Cafe Demo integration sample. We describe here Spring Boot support, Spring Framework Java and Annotation configuration, the IntegrationFlow feature and pay tribute to Java 8 Lambdasupport which was an inspiration for the DSL style. Of course, it is all backed by the Spring Integration Core project. But, before we launch into the description of the Cafe demonstration app here's a shorter example just to get started... @Configuration @EnableAutoConfiguration @IntegrationComponentScan public class Start { public static void main(String[] args) throws InterruptedException { ConfigurableApplicationContext ctx = SpringApplication.run(Start.class, args); List strings = Arrays.asList("foo", "bar"); System.out.println(ctx.getBean(Upcase.class).upcase(strings)); ctx.close(); } @MessagingGateway public interface Upcase { @Gateway(requestChannel = "upcase.input") Collection upcase(Collection strings); } @Bean public IntegrationFlow upcase() { return f -> f .split() // 1 .transform(String::toUpperCase) // 2 .aggregate(); // 3 } } We will leave the description of the infrastructure (annotations etc) to the main cafe flow description. Here, we want you to concentrate on the last @Bean, the IntegrationFlow as well as the gateway method which sends messages to that flow. In the main method we send a collection of strings to the gateway and print the results to STDOUT. The flow first splits the collection into individual Strings (1); each string is then transformed to upper case (2) and finally we re-aggregate them back into a collection (3) Since that's the end of the flow, the framework returns the result of the aggregation back to the gateway and the new payload becomes the return value from the gateway method. The equivalent XML configuration might be... or... Cafe Demo The purpose of the Cafe Demo application is to demonstrate how Enterprise Integration Patterns (EIP) can be used to reflect the order-delivery scenario in a real life cafe. With this application, we handle several drink orders - hot and iced. After running the application we can see in the standard output (System.out.println) how cold drinks are prepared quicker than hot. However the delivery for the whole order is postponed until the hot drink is ready. To reflect the domain model we have several classes: Order, OrderItem, Drink andDelivery. They all are mentioned in the integration scenario, but we won't analyze them here, because they are simple enough. The source code for our application is placed only in a single class; significant lines are annotated with a number corresponding to the comments, which follow: @SpringBootApplication // 1 @IntegrationComponentScan // 2 public class Application { public static void main(String[] args) throws Exception { ConfigurableApplicationContext ctx = SpringApplication.run(Application.class, args);// 3 Cafe cafe = ctx.getBean(Cafe.class); // 4 for (int i = 1; i <= 100; i++) { // 5 Order order = new Order(i); order.addItem(DrinkType.LATTE, 2, false); //hot order.addItem(DrinkType.MOCHA, 3, true); //iced cafe.placeOrder(order); } System.out.println("Hit 'Enter' to terminate"); // 6 System.in.read(); ctx.close(); } @MessagingGateway // 7 public interface Cafe { @Gateway(requestChannel = "orders.input") // 8 void placeOrder(Order order); // 9 } private AtomicInteger hotDrinkCounter = new AtomicInteger(); private AtomicInteger coldDrinkCounter = new AtomicInteger(); // 10 @Bean(name = PollerMetadata.DEFAULT_POLLER) public PollerMetadata poller() { // 11 return Pollers.fixedDelay(1000).get(); } @Bean public IntegrationFlow orders() { // 12 return f -> f // 13 .split(Order.class, Order::getItems) // 14 .channel(c -> c.executor(Executors.newCachedThreadPool()))// 15 .route(OrderItem::isIced, mapping -> mapping // 16 .subFlowMapping("true", sf -> sf // 17 .channel(c -> c.queue(10)) // 18 .publishSubscribeChannel(c -> c // 19 .subscribe(s -> // 20 s.handle(m -> sleepUninterruptibly(1, TimeUnit.SECONDS)))// 21 .subscribe(sub -> sub // 22 .transform(item -> Thread.currentThread().getName() + " prepared cold drink #" + this.coldDrinkCounter.incrementAndGet() + " for order #" + item.getOrderNumber() + ": " + item) // 23 .handle(m -> System.out.println(m.getPayload())))))// 24 .subFlowMapping("false", sf -> sf // 25 .channel(c -> c.queue(10)) .publishSubscribeChannel(c -> c .subscribe(s -> s.handle(m -> sleepUninterruptibly(5, TimeUnit.SECONDS)))// 26 .subscribe(sub -> sub .transform(item -> Thread.currentThread().getName() + " prepared hot drink #" + this.hotDrinkCounter.incrementAndGet() + " for order #" + item.getOrderNumber() + ": " + item) .handle(m -> System.out.println(m.getPayload())))))) .transform(orderItem -> new Drink(orderItem.getOrderNumber(), orderItem.getDrinkType(), orderItem.isIced(), orderItem.getShots())) // 27 .aggregate(aggregator -> aggregator // 28 .outputProcessor(group -> // 29 new Delivery(group.getMessages() .stream() .map(message -> (Drink) message.getPayload()) .collect(Collectors.toList()))) // 30 .correlationStrategy(m -> ((Drink) m.getPayload()).getOrderNumber()), null) // 31 .handle(CharacterStreamWritingMessageHandler.stdout()); // 32 } } Examining the code line by line... 1. @SpringBootApplication This new meta-annotation from Spring Boot 1.2. Includes @Configuration and@EnableAutoConfiguration. Since we are in a Spring Integration application and Spring Boot has auto-configuration for it, the @EnableIntegration is automatically applied, to initialize the Spring Integration infrastructure including an environment for the Java DSL -DslIntegrationConfigurationInitializer, which is picked up by theIntegrationConfigurationBeanFactoryPostProcessor from /META-INF/spring.factories. 2. @IntegrationComponentScan The Spring Integration analogue of @ComponentScan to scan components based on interfaces, (the Spring Framework's @ComponentScan only looks at classes). Spring Integration supports the discovery of interfaces annotated with @MessagingGateway (see #7 below). 3. ConfigurableApplicationContext ctx = SpringApplication.run(Application.class, args); The main method of our class is designed to start the Spring Boot application using the configuration from this class and starts an ApplicationContext via Spring Boot. In addition, it delegates command line arguments to the Spring Boot. For example you can specify --debug to see logs for the boot auto-configuration report. 4. Cafe cafe = ctx.getBean(Cafe.class); Since we already have an ApplicationContext we can start to interact with application. AndCafe is that entry point - in EIP terms a gateway. Gateways are simply interfaces and the application does not interact with the Messaging API; it simply deals with the domain (see #7 below). 5. for (int i = 1; i <= 100; i++) { To demonstrate the cafe "work" we intiate 100 orders with two drinks - one hot and one iced. And send the Order to the Cafe gateway. 6. System.out.println("Hit 'Enter' to terminate"); Typically Spring Integration application are asynchronous, hence to avoid early exit from themain Thread we block the main method until some end-user interaction through the command line. Non daemon threads will keep the application open but System.read()provides us with a mechanism to close the application cleanly. 7. @MessagingGateway The annotation to mark a business interface to indicate it is a gateway between the end-application and integration layer. It is an analogue of component from Spring Integration XML configuration. Spring Integration creates a Proxy for this interface and populates it as a bean in the application context. The purpose of this Proxy is to wrap parameters in a Message object and send it to the MessageChannel according to the provided options. 8. @Gateway(requestChannel = "orders.input") The method level annotation to distinct business logic by methods as well as by the target integration flows. In this sample we use a requestChannel reference of orders.input, which is a MessageChannel bean name of our IntegrationFlow input channel (see below #13). 9. void placeOrder(Order order); The interface method is a central point to interact from end-application with the integration layer. This method has a void return type. It means that our integration flow is one-wayand we just send messages to the integration flow, but don't wait for a reply. 10. private AtomicInteger hotDrinkCounter = new AtomicInteger(); private AtomicInteger coldDrinkCounter = new AtomicInteger(); Two counters to gather the information how our cafe works with drinks. 11. @Bean(name = PollerMetadata.DEFAULT_POLLER) public PollerMetadata poller() { The default poller bean. It is a analogue of component from Spring Integration XML configuration. Required for endpoints where the inputChannelis a PollableChannel. In this case, it is necessary for the two Cafe queues - hot and iced (see below #18). Here we use the Pollers factory from the DSL project and use its method-chain fluent API to build the poller metadata. Note that Pollers can be used directly from an IntegrationFlow definition, if a specific poller (rather than the default poller) is needed for an endpoint. 12. @Bean public IntegrationFlow orders() { The IntegrationFlow bean definition. It is the central component of the Spring Integration Java DSL, although it does not play any role at runtime, just during the bean registration phase. All other code below registers Spring Integration components (MessageChannel,MessageHandler, EventDrivenConsumer, MessageProducer, MessageSource etc.) in theIntegrationFlow object, which is parsed by the IntegrationFlowBeanPostProcessor to process those components and register them as beans in the application context as necessary (some elements, such as channels may already exist). 13. return f -> f The IntegrationFlow is a Consumer functional interface, so we can minimize our code and concentrate just only on the integration scenario requirements. Its Lambda acceptsIntegrationFlowDefinition as an argument. This class offers a comprehensive set of methods which can be composed to the chain. We call these EIP-methods, because they provide implementations for EI patterns and populate components from Spring Integration Core. During the bean registration phase, the IntegrationFlowBeanPostProcessor converts this inline (Lambda) IntegrationFlow to a StandardIntegrationFlow and processes its components. The same we can achieve using IntegrationFlows factory (e.g.IntegrationFlow.from("channelX"). ... .get()), but we find the Lambda definition more elegant. An IntegrationFlow definition using a Lambda populates DirectChannel as an inputChannel of the flow and it is registered in the application context as a bean with the name orders.input in this our sample (flow bean name + ".input"). That's why we use that name for the Cafe gateway. 14. .split(Order.class, Order::getItems) Since our integration flow accepts message through the orders.input channel, we are ready to consume and process them. The first EIP-method in our scenario is .split(). We know that the message payload from orders.input channel is an Order domain object, so we can simply use its type here and use the Java 8 method-reference feature. The first parameter is a type of message payload we expect, and the second is a method reference to the getItems() method, which returns Collection. So, this performs thesplit EI pattern, when we send each collection entry as a separate message to the next channel. In the background, the .split() method registers a MethodInvokingSplitterMessageHandler implementation and the EventDrivenConsumer for thatMessageHandler, and wiring in the orders.input channel as the inputChannel. 15. .channel(c -> c.executor(Executors.newCachedThreadPool())) The .channel() EIP-method allows the specification of concrete MessageChannels between endpoints, as it is done via output-channel/input-channel attributes pair with Spring Integration XML configuration. By default, endpoints in the DSL integration flow definition are wired with DirectChannels, which get the bean names based on theIntegrationFlow bean name and index in the flow chain. In this case we use anotherLambda expression, which selects a specific MessageChannel implementation from itsChannels factory and configures it with the fluent API. The current channel here is anExecutorChannel, to allow to distribute messages from the splitter to separateThreads, to process them in parallel in the downstream flow. 16. .route(OrderItem::isIced, mapping -> mapping The next EIP-method in our scenario is .route(), to send hot/iced order items to different Cafe kitchens. We again use here a method reference (isIced()) to get theroutingKey from the incoming message. The second Lambda parameter represents arouter mapping - something similar to sub-element for the component from Spring Integration XML configuration. However since we are using Java we can go a bit further with its Lambda support! The Spring Integration Java DSL introduced thesubflow definition for routers in addition to traditional channel mapping. Each subflow is executed depending on the routing and, if the subflow produces a result, it is passed to the next element in the flow definition after the router. 17. .subFlowMapping("true", sf -> sf Specifies the integration flow for the current router's mappingKey. We have in this samples two subflows - hot and iced. The subflow is the same IntegrationFlow functional interface, therefore we can use its Lambda exactly the same as we do on the top levelIntegrationFlow definition. The subflows don't have any runtime dependency with its parent, it's just a logical relationship. 18. .channel(c -> c.queue(10)) We already know that a Lambda definition for the IntegrationFlow starts from[FLOW_BEAN_NAME].input DirectChannel, so it may be a question "how does it work here if we specify .channel() again?". The DSL takes care of such a case and wires those two channels with a BridgeHandler and endpoint. In our sample, we use here a restrictedQueueChannel to reflect the Cafe kitchen busy state from real life. And here is a place where we need that global poller for the next endpoint which is listening on this channel. 19. .publishSubscribeChannel(c -> c The .publishSubscribeChannel() EIP-method is a variant of the .channel() for aMessageChannels.publishSubscribe(), but with the .subscribe() option when we can specify subflow as a subscriber to the channel. Right, subflow one more time! So, subflows can be specified to any depth. Independently of the presence .subscribe() subflows, the next endpoint in the parent flow is also a subscriber to this .publishSubscribeChannel(). Since we are in the .route() subflow already, the last subscriber is an implicit BridgeHandlerwhich just pops the message to the top level - to a similar implicit BridgeHandler to pop message to the next .transform() endpoint in the main flow. And one more note about this current position of our flow: the previous EIP-method is .channel(c -> c.queue(10)) and this one is for MessageChannel too. So, they are again tied with an implicit BridgeHandleras well. In a real application we could avoid this .publishSubscribeChannel() just with the single .handle() for the Cafe kitchen, but our goal here to cover DSL features as much as possible. That's why we distribute the kitchen work to several subflows for the samePublishSubscribeChannel. 20. .subscribe(s -> The .subscribe() method accepts an IntegrationFlow as parameter, which can be specified as Lambda to configure subscriber as subflow. We use here several subflow subscribers to avoid multi-line Lambdas and cover some DSL as we as Spring Integration capabilities. 21. s.handle(m -> sleepUninterruptibly(1, TimeUnit.SECONDS))) Here we use a simple .handle() EIP-method just to block the current Thread for some timeout to demonstrate how quickly the Cafe kitchen prepares a drink. Here we use Google Guava Uninterruptibles.sleepUninterruptibly, to avoid using a try...catch block within the Lambda expression, although you can do that and your Lambda will be multi-line. Or you can move that code to a separate method and use it here as method reference. Since we don't use any Executor on the .publishSubscribeChannel() all subscribers will beperformed sequentially on the same Thread; in our case it is one of TaskScheduler's Threads from poller on the previous QueueChannel. That's why this sleep blocks all downstream process and allows to demonstrate the busy state for that restricted to 10QueueChannel. 22. .subscribe(sub -> sub The next subflow subscriber which will be performed only after that sleep with 1 second foriced drink. We use here one more subflow because .handle() of previous one is one-way with the nature of the Lambda for MessageHandler. That's why, to go ahead with process of our whole flow, we have several subscribers: some of subflows finish after their work and don't return anything to the parent flow. 23. .transform(item -> Thread.currentThread().getName() + " prepared cold drink #" + this.coldDrinkCounter.incrementAndGet() + " for order #" + item.getOrderNumber() + ": " + item) The transformer in the current subscriber subflow is to convert the OrderItem to the friendly STDOUT message for the next .handle. Here we see the use of generics with the Lambda expression. This is implemented using the GenericTransformer functional interface. 24. .handle(m -> System.out.println(m.getPayload()))))) The .handle() here just to demonstrate how to use Lambda expression to print thepayload to STDOUT. It is a signal that our drink is ready. After that the final (implicit) subscriber to the PublishSubscribeChannel just sends the message with the OrderItemto the .transform() in the main flow. 25. .subFlowMapping("false", sf -> sf The .subFlowMapping() for the hot drinks. Actually it is similar to the previous iceddrinks subflow, but with specific hot business logic. 26. s.handle(m -> sleepUninterruptibly(5, TimeUnit.SECONDS))) The sleepUninterruptibly for hot drinks. Right, we need more time to boil the water! 27. .transform(orderItem -> new Drink(orderItem.getOrderNumber(), orderItem.getDrinkType(), orderItem.isIced(), orderItem.getShots())) The main OrderItem to Drink transformer, which is performed when the .route()subflow returns its result after the Cafe kitchen subscribers have finished preparing the drink. 28. .aggregate(aggregator -> aggregator The .aggregate() EIP-method provides similar options to configure anAggregatingMessageHandler and its endpoint, like we can do with the component when using Spring Integration XML configuration. Of course, with the Java DSL we have more power to configure the aggregator just in place, without any other extra beans. And Lambdas come to the rescue again! From the Cafe business logic perspective we compose theDelivery for the initial Order, since we .split() the original order to the OrderItems near the beginning. 29. .outputProcessor(group -> The .outputProcessor() of the AggregatorSpec allows us to emit a custom result after aggregator completes the group. It's an analogue of ref/method from the component or the @Aggregator annotation on a POJO method. Our goal here to compose aDelivery for all Drinks. 30. new Delivery(group.getMessages() .stream() .map(message -> (Drink) message.getPayload()) .collect(Collectors.toList()))) As you see we use here the Java 8 Stream feature for Collection. We iterate over messages from the released MessageGroup and convert (map) each of them to its Drinkpayload. The result of the Stream (.collect()) (a list of Drinks) is passed to theDelivery constructor. The Message with this new Delivery payload is sent to the next endpoint in our Cafe scenario. 31. .correlationStrategy(m -> ((Drink) m.getPayload()).getOrderNumber()), null) The .correlationStrategy() Lambda demonstrates how we can customize an aggregator behaviour. Of course, we can rely here just only on a built-in SequenceDetails from Spring Integration, which is populated by default from .split() in the beginning of our flow to each split message, but the Lambda sample for the CorrelationStrategy is included for illustration. (With XML, we could have used a correlation-expression or a customCorrelationStrategy). The second argument in this line for the .aggregate() EIP-method is for the endpointConfigurer to customize options like autoStartup,requiresReply, adviceChain etc. We use here null to show that we rely on the default options for the endpoint. Many of EIP-methods provide overloaded versions with and withoutendpointConfigurer, but .aggregate() requires an endpoint argument, to avoid an explicit cast for the AggregatorSpec Lambda argument. 32. .handle(CharacterStreamWritingMessageHandler.stdout()); It is the end of our flow - the Delivery is delivered to the client! We just print here the message payload to STDOUT using out-of-the-boxCharacterStreamWritingMessageHandler from Spring Integration Core. This is a case to show how existing components from Spring Integration Core (and its modules) can be used from the Java DSL. Well, we have finished describing the Cafe Demo sample based on the Spring Integration Java DSL. Compare it with XML sample to get more information regarding Spring Integration. This is not an overall tutorial to the DSL stuff. We don't review here theendpointConfigurer options, Transformers factory, the IntegrationComponentSpechierarchy, the NamespaceFactories, how we can specify several IntegrationFlow beans and wire them to a single application etc., see the Reference Manual for more information. At least this line-by-line tutorial should show you Spring Integration Java DSL basics and its seamless fusion between Spring Framework Java & Annotation configuration, Spring Integration foundation and Java 8 Lambda support! Also see the si4demo to see the evolution of Spring Integration including the Java DSL, as shown at the 2014 SpringOne/2GX Conference. (Video should be available soon). As always, we look forward to your comments and feedback (StackOverflow (spring-integration tag), Spring JIRA, GitHub) and we very much welcome contributions! P.S. Even if this tutorial is fully based on the Java 8 Lambda support, we don't want to miss pre Java 8 users, we are going to provide similar non-Lambda blog post. Stay tuned!

Thread Pools are very important to execute synchronous & asynchronous processes. This article shows how to develop and monitor Thread Pool Services by using Spring. Creating Thread Pool has been explained via two alternative methods. Used Technologies : JDK 1.6.0_21 Spring 3.0.5 Maven 3.0.2 STEP 1 : CREATE MAVEN PROJECT A maven project is created as below. (It can be created by using Maven or IDE Plug-in). STEP 2 : LIBRARIES Spring dependencies are added to Maven’ s pom.xml. ? org.springframework spring-core ${spring.version} org.springframework spring-context ${spring.version} For creating runnable-jar, below plugin can be used. ? org.apache.maven.plugins maven-shade-plugin 1.3.1 package shade com.otv.exe.Application META-INF/spring.handlers META-INF/spring.schemas STEP 3 : CREATE TASK CLASS A new TestTask Class is created by implementing Runnable Interface. This class shows to be executed tasks. ? package com.otv.task; import org.apache.log4j.Logger; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class TestTask implements Runnable { private static Logger log = Logger.getLogger(TestTask.class); String taskName; public TestTask() { } public TestTask(String taskName) { this.taskName = taskName; } public void run() { try { log.debug(this.taskName + " : is started."); Thread.sleep(10000); log.debug(this.taskName + " : is completed."); } catch (InterruptedException e) { log.error(this.taskName + " : is not completed!"); e.printStackTrace(); } } @Override public String toString() { return (getTaskName()); } public String getTaskName() { return taskName; } public void setTaskName(String taskName) { this.taskName = taskName; } } STEP 4 : CREATE TestRejectedExecutionHandler CLASS TestRejectedExecutionHandler Class is created by implementing RejectedExecutionHandler Interface. If there is no idle thread and queue overflows, tasks will be rejected. This class handles rejected tasks. ? package com.otv.handler; import java.util.concurrent.RejectedExecutionHandler; import java.util.concurrent.ThreadPoolExecutor; import org.apache.log4j.Logger; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class TestRejectedExecutionHandler implements RejectedExecutionHandler { private static Logger log = Logger.getLogger(TestRejectedExecutionHandler.class); public void rejectedExecution(Runnable runnable, ThreadPoolExecutor executor) { log.debug(runnable.toString() + " : has been rejected"); } } STEP 5 : CREATE ITestThreadPoolExecutorService INTERFACE ITestThreadPoolExecutorService Interface is created. ? package com.otv.srv; import java.util.concurrent.ThreadPoolExecutor; import com.otv.handler.TestRejectedExecutionHandler; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public interface ITestThreadPoolExecutorService { public ThreadPoolExecutor createNewThreadPool(); public int getCorePoolSize(); public void setCorePoolSize(int corePoolSize); public int getMaxPoolSize(); public void setMaxPoolSize(int maximumPoolSize); public long getKeepAliveTime(); public void setKeepAliveTime(long keepAliveTime); public int getQueueCapacity(); public void setQueueCapacity(int queueCapacity); public TestRejectedExecutionHandler getTestRejectedExecutionHandler(); public void setTestRejectedExecutionHandler(TestRejectedExecutionHandler testRejectedExecutionHandler); } STEP 6 : CREATE TestThreadPoolExecutorService CLASS TestThreadPoolExecutorService Class is created by implementing ITestThreadPoolExecutorService Interface. This class creates a new Thread Pool. ? package com.otv.srv; import java.util.concurrent.ArrayBlockingQueue; import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; import com.otv.handler.TestRejectedExecutionHandler; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class TestThreadPoolExecutorService implements ITestThreadPoolExecutorService { private int corePoolSize; private int maxPoolSize; private long keepAliveTime; private int queueCapacity; TestRejectedExecutionHandler testRejectedExecutionHandler; public ThreadPoolExecutor createNewThreadPool() { ThreadPoolExecutor executor = new ThreadPoolExecutor(getCorePoolSize(), getMaxPoolSize(), getKeepAliveTime(), TimeUnit.SECONDS, new ArrayBlockingQueue(getQueueCapacity()), getTestRejectedExecutionHandler()); return executor; } public int getCorePoolSize() { return corePoolSize; } public void setCorePoolSize(int corePoolSize) { this.corePoolSize = corePoolSize; } public int getMaxPoolSize() { return maxPoolSize; } public void setMaxPoolSize(int maxPoolSize) { this.maxPoolSize = maxPoolSize; } public long getKeepAliveTime() { return keepAliveTime; } public void setKeepAliveTime(long keepAliveTime) { this.keepAliveTime = keepAliveTime; } public int getQueueCapacity() { return queueCapacity; } public void setQueueCapacity(int queueCapacity) { this.queueCapacity = queueCapacity; } public TestRejectedExecutionHandler getTestRejectedExecutionHandler() { return testRejectedExecutionHandler; } public void setTestRejectedExecutionHandler(TestRejectedExecutionHandler testRejectedExecutionHandler) { this.testRejectedExecutionHandler = testRejectedExecutionHandler; } } STEP 7 : CREATE IThreadPoolMonitorService INTERFACE IThreadPoolMonitorService Interface is created. ? package com.otv.monitor.srv; import java.util.concurrent.ThreadPoolExecutor; public interface IThreadPoolMonitorService extends Runnable { public void monitorThreadPool(); public ThreadPoolExecutor getExecutor(); public void setExecutor(ThreadPoolExecutor executor); } STEP 8 : CREATE ThreadPoolMonitorService CLASS ThreadPoolMonitorService Class is created by implementing IThreadPoolMonitorService Interface. This class monitors created thread pool. ? package com.otv.monitor.srv; import java.util.concurrent.ThreadPoolExecutor; import org.apache.log4j.Logger; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class ThreadPoolMonitorService implements IThreadPoolMonitorService { private static Logger log = Logger.getLogger(ThreadPoolMonitorService.class); ThreadPoolExecutor executor; private long monitoringPeriod; public void run() { try { while (true){ monitorThreadPool(); Thread.sleep(monitoringPeriod*1000); } } catch (Exception e) { log.error(e.getMessage()); } } public void monitorThreadPool() { StringBuffer strBuff = new StringBuffer(); strBuff.append("CurrentPoolSize : ").append(executor.getPoolSize()); strBuff.append(" - CorePoolSize : ").append(executor.getCorePoolSize()); strBuff.append(" - MaximumPoolSize : ").append(executor.getMaximumPoolSize()); strBuff.append(" - ActiveTaskCount : ").append(executor.getActiveCount()); strBuff.append(" - CompletedTaskCount : ").append(executor.getCompletedTaskCount()); strBuff.append(" - TotalTaskCount : ").append(executor.getTaskCount()); strBuff.append(" - isTerminated : ").append(executor.isTerminated()); log.debug(strBuff.toString()); } public ThreadPoolExecutor getExecutor() { return executor; } public void setExecutor(ThreadPoolExecutor executor) { this.executor = executor; } public long getMonitoringPeriod() { return monitoringPeriod; } public void setMonitoringPeriod(long monitoringPeriod) { this.monitoringPeriod = monitoringPeriod; } } STEP 9 : CREATE Starter CLASS Starter Class is created. ? package com.otv.start; import java.util.concurrent.ThreadPoolExecutor; import org.apache.log4j.Logger; import com.otv.handler.TestRejectedExecutionHandler; import com.otv.monitor.srv.IThreadPoolMonitorService; import com.otv.monitor.srv.ThreadPoolMonitorService; import com.otv.srv.ITestThreadPoolExecutorService; import com.otv.srv.TestThreadPoolExecutorService; import com.otv.task.TestTask; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class Starter { private static Logger log = Logger.getLogger(TestRejectedExecutionHandler.class); IThreadPoolMonitorService threadPoolMonitorService; ITestThreadPoolExecutorService testThreadPoolExecutorService; public void start() { // A new thread pool is created... ThreadPoolExecutor executor = testThreadPoolExecutorService.createNewThreadPool(); executor.allowCoreThreadTimeOut(true); // Created executor is set to ThreadPoolMonitorService... threadPoolMonitorService.setExecutor(executor); // ThreadPoolMonitorService is started... Thread monitor = new Thread(threadPoolMonitorService); monitor.start(); // New tasks are executed... for(int i=1;i<10;i++) { executor.execute(new TestTask("Task"+i)); } try { Thread.sleep(40000); } catch (Exception e) { log.error(e.getMessage()); } for(int i=10;i<19;i++) { executor.execute(new TestTask("Task"+i)); } // executor is shutdown... executor.shutdown(); } public IThreadPoolMonitorService getThreadPoolMonitorService() { return threadPoolMonitorService; } public void setThreadPoolMonitorService(IThreadPoolMonitorService threadPoolMonitorService) { this.threadPoolMonitorService = threadPoolMonitorService; } public ITestThreadPoolExecutorService getTestThreadPoolExecutorService() { return testThreadPoolExecutorService; } public void setTestThreadPoolExecutorService(ITestThreadPoolExecutorService testThreadPoolExecutorService) { this.testThreadPoolExecutorService = testThreadPoolExecutorService; } } STEP 10 : CREATE Application CLASS Application Class is created. This class runs the application. ? package com.otv.start; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class Application { public static void main(String[] args) { ApplicationContext context = new ClassPathXmlApplicationContext("applicationContext.xml"); Starter starter = (Starter) context.getBean("Starter"); starter.start(); } } STEP 11 : CREATE applicationContext.xml applicationContext.xml is created. ? STEP 12 : ALTERNATIVE METHOD TO CREATE THREAD POOL ThreadPoolTaskExecutor Class provided by Spring can also be used to create Thread Pool. ? STEP 13 : BUILD PROJECT After OTV_Spring_ThreadPool Project is build, OTV_Spring_ThreadPool-0.0.1-SNAPSHOT.jar will be created. STEP 14 : RUN PROJECT After created OTV_Spring_ThreadPool-0.0.1-SNAPSHOT.jar file is run, below output logs will be shown : ? 18.10.2011 20:08:48 DEBUG (TestRejectedExecutionHandler.java:19) - Task7 : has been rejected 18.10.2011 20:08:48 DEBUG (TestRejectedExecutionHandler.java:19) - Task8 : has been rejected 18.10.2011 20:08:48 DEBUG (TestRejectedExecutionHandler.java:19) - Task9 : has been rejected 18.10.2011 20:08:48 DEBUG (TestTask.java:25) - Task1 : is started. 18.10.2011 20:08:48 DEBUG (TestTask.java:25) - Task6 : is started. 18.10.2011 20:08:48 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 2 - CompletedTaskCount : 0 - TotalTaskCount : 5 - isTerminated : false 18.10.2011 20:08:48 DEBUG (TestTask.java:25) - Task5 : is started. 18.10.2011 20:08:53 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 0 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:08:58 DEBUG (TestTask.java:27) - Task6 : is completed. 18.10.2011 20:08:58 DEBUG (TestTask.java:27) - Task1 : is completed. 18.10.2011 20:08:58 DEBUG (TestTask.java:25) - Task3 : is started. 18.10.2011 20:08:58 DEBUG (TestTask.java:25) - Task2 : is started. 18.10.2011 20:08:58 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 2 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:08:58 DEBUG (TestTask.java:27) - Task5 : is completed. 18.10.2011 20:08:58 DEBUG (TestTask.java:25) - Task4 : is started. 18.10.2011 20:09:03 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 3 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:09:08 DEBUG (TestTask.java:27) - Task2 : is completed. 18.10.2011 20:09:08 DEBUG (TestTask.java:27) - Task3 : is completed. 18.10.2011 20:09:08 DEBUG (TestTask.java:27) - Task4 : is completed. 18.10.2011 20:09:08 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 6 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:09:13 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 6 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:09:18 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 0 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 6 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:09:23 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 0 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 6 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:09:28 DEBUG (TestTask.java:25) - Task10 : is started. 18.10.2011 20:09:28 DEBUG (TestRejectedExecutionHandler.java:19) - Task16 : has been rejected 18.10.2011 20:09:28 DEBUG (TestRejectedExecutionHandler.java:19) - Task17 : has been rejected 18.10.2011 20:09:28 DEBUG (TestRejectedExecutionHandler.java:19) - Task18 : has been rejected 18.10.2011 20:09:28 DEBUG (TestTask.java:25) - Task14 : is started. 18.10.2011 20:09:28 DEBUG (TestTask.java:25) - Task15 : is started. 18.10.2011 20:09:28 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 6 - TotalTaskCount : 12 - isTerminated : false 18.10.2011 20:09:33 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 6 - TotalTaskCount : 12 - isTerminated : false 18.10.2011 20:09:38 DEBUG (TestTask.java:27) - Task10 : is completed. 18.10.2011 20:09:38 DEBUG (TestTask.java:25) - Task11 : is started. 18.10.2011 20:09:38 DEBUG (TestTask.java:27) - Task14 : is completed. 18.10.2011 20:09:38 DEBUG (TestTask.java:27) - Task15 : is completed. 18.10.2011 20:09:38 DEBUG (TestTask.java:25) - Task12 : is started. 18.10.2011 20:09:38 DEBUG (TestTask.java:25) - Task13 : is started. 18.10.2011 20:09:38 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 9 - TotalTaskCount : 12 - isTerminated : false 18.10.2011 20:09:43 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 9 - TotalTaskCount : 12 - isTerminated : false 18.10.2011 20:09:48 DEBUG (TestTask.java:27) - Task11 : is completed. 18.10.2011 20:09:48 DEBUG (TestTask.java:27) - Task13 : is completed. 18.10.2011 20:09:48 DEBUG (TestTask.java:27) - Task12 : is completed. 18.10.2011 20:09:48 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 0 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 12 - TotalTaskCount : 12 - isTerminated : true 18.10.2011 20:09:53 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 0 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 12 - TotalTaskCount : 12 - isTerminated : true STEP 15 : DOWNLOAD OTV_Spring_ThreadPool

Cookies set through the Owin API sometimes mysteriously disappear. The problem is that deep within System.Web, there has been a cookie monster sleeping since the dawn of time (well, at least since .NET and System.Web was released). The monster has been sleeping for all this time, but now, with the new times arriving with Owin, the monster is awake. Being starved from the long sleep, it eats cookies set through the Owin API for breakfast. Even if the cookies are properly set, they are eaten by the monster before the Set-Cookie headers are sent out to the client browser. This typically results in heisenbugsaffecting sign in and sign out functionality. TL;DR The problem is that System.Web has its own master source of cookie information and that isn’t the Set-Cookie header. Owin only knows about the Set-Cookie header. A workaround is to make sure that any cookies set by Owin are also set in the HttpContext.Current.Response.Cookies collection. This is exactly what my Kentor.OwinCookieSaver middleware does. It should be added in to the Owin pipeline (typically in Startup.Auth.cs), before any middleware that handles cookies. app.UseKentorOwinCookieSaver(); The cookie saver middleware preserves cookies set by other middleware. Unfortunately it is not reliable for cookies set by the application code (such as in MVC Actions). The reason is that the System.Web cookie handling code might be run after the application code, but before the middleware. For cookies set by the application code, the workaround by storing a dummy value in the sessions is more safe. The Reason The System.Web API has been around since the dawn of .NET. Back then, it was tightly coupled to IIS and the one and only API for web applications. As the one and only, it could assume that it was the master of all information. In HttpResponse.cs there is a check whether the cookie collection is changed (adding cookies doesn’t count as a change) and in that case it wipes the existing Set-Cookie header. if (_cookies.Changed || needToReset) { // delete all set cookie headers headers.Remove("Set-Cookie"); // write all the cookies again for(int c = 0; c < _cookies.Count; c++) { // Write the cookies, code removed for brevity. } } This is what a sleeping cookie monster looks like in the code. It’s sleeping, because there’s still nothing questioning the cooke collection being the master. But that all changes when Owin was introduced. The Owin API knows nothing aboutSystem.Web.HttpContext. In fact, that’s kind of the point with Owin, to break the dependency between .NET web applications and IIS. In Katana, cookies are (in most cases) added by a call toResponse.Cookies.Append() which adds a new Set-Cookie header. Effectively we have a system with conflicting views on where the master information is stored. Owin considers the actual header to be the master while System.Web considers the response cookie collection to be the master. Having conflicting masters is never a good idea. This is a known issue for Katana, classified as “High Impact”. The Workaround Middleware The conflict between the two distance relatives System.Web and Owin is a typical family conflict. The older one is wrong, but won’t change views just because someone young appears with new facts. When mediating such a conflict it’s usually easiest to get the younger generation to work around the older. Changing System.Web is not feasible, so the focus has to be on Owin. The workaround middleware I’ve created checks the Set-Cookie header and syncs its contents back to the cookie collection. By putting it before any cookie handling middleware in the pipeline it can save the cookies from the monster, before System.Web deletes the header. The core function of the workaround middleware is the Invoke method. public async override Task Invoke(IOwinContext context) { await Next.Invoke(context); var setCookie = context.Response.Headers.GetValues("Set-Cookie"); if(setCookie != null) { var cookies = CookieParser.Parse(setCookie); foreach(var c in cookies) { if(!HttpContext.Current.Response.Cookies.AllKeys.Contains(c.Name)) { HttpContext.Current.Response.Cookies.Add(c); } } } } The logic is quite straight forward. Parse each Set-Cookie header into a HttpCookie object and ensure that it is present in the response cookie collection. For the applications we’ve tested it works, but it is a workaround and not a real fix. Please leave a comment below if you find situation where this workaround does not work. That’s very valuable information for others having the same issue. A Permanent Fix I’m also looking into fixing this permanently by contributing to the System.Web host in Katana. The fix there would be to directly intercept any calls to set the Set-Cookie header and add them to the cookie to the collection too. That should be a much more stable solution as it prevents the problem rather than trying to fix it afterwards.

I’m a huge Vaadin fan and I’ve created a Github workshop I can demo at conferences. A common issue with such kind of workshops is that attendees have to prepare their workstations in advance… and there’s always a significant part of them that comes with not everything ready. At this point, two options are available to the speaker: either wait for each of the attendee to finish the preparation – too bad for the people who took the time at home to do that, or start anyway – and lose the not-ready part. Given the current buzz around Docker, I thought that could be a very good way to make the workshop preparation quicker – only one step, and hasslefree – no problem regarding the quirks of your operation system. The required steps I ask the attendees are the following: Install Git Install Java, Maven and Tomcat Clone the git repo Build the project (to prepare the Maven repository) Deploy the built webapp Start Tomcat These should directly be automated into Docker. As I wasted much time getting this to work, here’s the tale of my journey in achieving this (be warned, it’s quite long). If you’ve got similar use-cases, I hope it will be useful in you getting things done faster. Starting with Docker The first step was to get to know the basics about Docker. Fortunately, I had the chance to attend a Docker workshop by David Gageot at Duchess Swiss. This included both Docker installation and basics of Dockerfile. I assume readers have likewise a basic understanding of Docker. For those who don’t, I guess browsing the Docker’s official documentation is a nice idea: Installation Dockerfile reference Building my first Dockerfile The Docker image can be built with the following command ran into the directory of the Dockerfile: $ docker build -t vaadinworkshop . The first issues one can encounter when playing with Docker the first time, is to get the following error message: Get http:///var/run/docker.sock/v1.14/containers/json: dial unix /var/run/docker.sock: no such file or directory The reason is because one didn’t export the required environment variables displayed by the boot2docker information message. If you lost the exact data, no worry, just use the shellinit boot2docker parameter: $ boot2docker shellinit Writing /Users/i303869/.docker/boot2docker-vm/ca.pem: Writing /Users/i303869/.docker/boot2docker-vm/cert.pem: Writing /Users/i303869/.docker/boot2docker-vm/key.pem: export DOCKER_HOST=tcp://192.168.59.103:2376 export DOCKER_CERT_PATH=/Users/i303869/.docker/boot2docker-vm Copy-paste the export lines above will solve the issue. These can also be set in one’s .bashrc script as it seems these values seldom change. Next in line is the following error: Get http://192.168.59.103:2376/v1.14/containers/json: malformed HTTP response "x15x03x01x00x02x02" This error message seems to be because of a mismatch between versions of the client and the server. It seems it is because of a bug on Mac OSX when upgrading. For a long term solution, reinstall Docker from scratch; for a quick fix, use the --tls flag with the docker command. As it is quite cumbersome to type it everything, one can alias it: $ alias docker="docker --tls" My last mistake when building the image comes from building the Dockerfile from a not empty directory. Docker sends every file it finds in the directory of the Dockerfile to the Docker container for build: $ docker --tls build -t vaadinworkshop . Sending build context to Docker daemon Too many kB Fix: do not try this at home and start from a directory container the Dockerfile only. Starting from scratch Dockerfiles describe images – images are built as a layered list of instructions. Docker images are designed around single inheritance: one image has to be set a single parent. An image requiring no parent starts from scratch, but Docker provides 4 base official distributions: busybox, debian, ubuntu and centos (operating systems are generally a good start). Whatever you want to achieve, it is necessary to choose the right parent. Given the requirements I set for myself (Java, Maven, Tomcat and Git), I tried to find the right starting image. Many Dockerfiles are already available online on the Docker hub. The browsing app is quite good, but to be really honest, the search can really be improved. My intention was to use the image that matched the most of my requirements, then fill the gap. I could find no image providing Git, but I thought the dgageot/maven Dockerfile would be a nice starting point. The problem is that the base image is a busybox and provides no installer out-of-the-box (apt-get, yum, whatever). For this reason, David uses a lot of curl to get Java 8 and Maven in his Dockerfiles. I foolishly thought I could use a different flavor of busybox that provides the opkg installer. After a while, I accumulated many problems, resolving one heading to another. In the end, I finally decided to use the OS I was most comfortable with and to install everything myself: FROM ubuntu:utopic Scripting Java installation Installing git, maven and tomcat packages is very straightforward (if you don’t forget to use the non-interactive options) with RUN and apt-get: RUN apt-get update && \ apt-get install -y --force-yes git maven tomcat8 Java doesn’t fall into this nice pattern, as Oracle wants you to accept the license. Nice people did however publish it to a third-party repo. Steps are the following: Add the needed package repository Configure the system to automatically accept the license Configure the system to add un-certified packages Update the list of repositories At last, install the package Also add a package for Java 8 system configuration. RUN echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list && \ echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && \ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886 RUN apt-get update && \ apt-get install -y --force-yes oracle-java8-installer oracle-java8-set-default Building the sources Getting the workshop’s sources and building them is quite straightforward with the following instructions: RUN git clone https://github.com/nfrankel/vaadin7-workshop.git WORKDIR /vaadin7-workshop RUN mvn package The drawback of this approach is that Maven will start from a fresh repository, and thus download the Internet the first time it is launched. At first, I wanted to mount a volume from the host to the container to share the ~/.m2/repository folder to avoid this, but I noticed this could only be done at runtime through the -v option as the VOLUME instruction cannot point to a host directory. Starting the image The simplest command to start the created Docker image is the following: $ docker run -p 8080:8080 Do not forget the port forwarding from the container to the host, 8080 for the standard HTTP port. Also, note that it’s not necessary to run the container as a daemon (with the -d option). The added value of that is that the standard output of the CMD (see below) will be redirected to the host. When running as a daemon and wanting to check the logs, one has to execute bash in the container, which requires a sequence of cumbersome manipulations. Configuring and launching Tomcat Tomcat can be launched when starting the container by just adding the following instruction to the Dockerfile: CMD ["catalina.sh", "run"] However, trying to start the container at this point will result in the following error: Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/common/classes], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/common], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/server/classes], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/server], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/shared/classes], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/shared], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina initDirs SEVERE: Cannot find specified temporary folder at /usr/share/tomcat8/temp Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina load WARNING: Unable to load server configuration from [/usr/share/tomcat8/conf/server.xml] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina initDirs SEVERE: Cannot find specified temporary folder at /usr/share/tomcat8/temp Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina load WARNING: Unable to load server configuration from [/usr/share/tomcat8/conf/server.xml] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina start SEVERE: Cannot start server. Server instance is not configured. I have no idea why, but it seems Tomcat 8 on Ubuntu is not configured in any meaningful way. Everything is available but we need some symbolic links here and there as well as creating the temp directory. This translates into the following instruction in the Dockerfile: RUN ln -s /var/lib/tomcat8/common $CATALINA_HOME/common && \ ln -s /var/lib/tomcat8/server $CATALINA_HOME/server && \ ln -s /var/lib/tomcat8/shared $CATALINA_HOME/shared && \ ln -s /etc/tomcat8 $CATALINA_HOME/conf && \ mkdir $CATALINA_HOME/temp The final trick is to connect the exploded webapp folder created by Maven to Tomcat’s webapps folder, which it looks for deployments: RUN mkdir $CATALINA_HOME/webapps && \ ln -s /vaadin7-workshop/target/workshop-7.2-1.0-SNAPSHOT/ $CATALINA_HOME/webapps/vaadinworkshop At this point, the Holy Grail is not far away, you just have to browse the URL… if only we knew what the IP was. Since running on Mac, there’s an additional VM beside the host and the container that’s involved. To get this IP, type: $ boot2docker ip The VM's Host only interface IP address is: 192.168.59.103 Now, browsing http://192.168.59.103:8080/vaadinworkshop/ will bring us to the familiar workshop screen: Developing from there Everything works fine but didn’t we just forget about one important thing, like how workshop attendees are supposed to work on the sources? Easy enough, just mount the volume when starting the container: docker run -v /Users//vaadin7-workshop:/vaadin7-workshop -p 8080:8080 vaadinworkshop Note that the host volume must be part of /Users and if on OSX, it must use boot2docker v. 1.3+. Unfortunately, it seems now is the showstopper, as mounting an empty directory from the host to the container will not make the container’s directory available from the host. On the contrary, it will empty the container’s directory given that the host’s directory doesn’t exist… It seems there’s an issue in Docker on Mac. The installation of JHipster runs into the same problem, and proposes to use the Samba Docker folder sharing project. I’m afraid I was too lazy to go further at this point. However, this taught me much about Docker, its usages and use-cases (as well as OSX integration limitations). For those who are interested, you’ll find below the Docker file. Happy Docker! FROM ubuntu:utopic MAINTAINER Nicolas Frankel # Config to get to install Java 8 w/o interaction RUN echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list && echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886 RUN apt-get update && apt-get install -y --force-yes git oracle-java8-installer oracle-java8-set-default maven tomcat8 RUN git clone https://github.com/nfrankel/vaadin7-workshop.git WORKDIR /vaadin7-workshop RUN git checkout v7.2-1 RUN mvn package ENV JAVA_HOME /usr/lib/jvm/java-8-oracle ENV CATALINA_HOME /usr/share/tomcat8 ENV PATH $PATH:$CATALINA_HOME/bin # Configure Tomcat 8 directories RUN ln -s /var/lib/tomcat8/common $CATALINA_HOME/common && ln -s /var/lib/tomcat8/server $CATALINA_HOME/server && ln -s /var/lib/tomcat8/shared $CATALINA_HOME/shared && ln -s /etc/tomcat8 $CATALINA_HOME/conf && mkdir $CATALINA_HOME/temp && mkdir $CATALINA_HOME/webapps && ln -s /vaadin7-workshop/target/workshop-7.2-1.0-SNAPSHOT/ $CATALINA_HOME/webapps/vaadinworkshop VOLUME ["/vaadin7-workshop"] CMD ["catalina.sh", "run"] # docker build -t vaadinworkshop . # docker run -v ~/vaadin7-workshop training/webapp -p 8080:8080 vaadinworkshop

Spring-session is a very cool new project that aims to provide a simpler way of managing sessions in Java based web applications. One of the features that I explored with spring-session recently was the way it supports externalizing session state without needing to fiddle with the internals of specific web containers like Tomcat or Jetty. To test spring-session I have used a shopping cart type application(available here) which makes heavy use of session by keeping the items added to the cart as a session attribute, as can be seen from these screenshots: Consider first a scenario without Spring session. So this is how I have exposed my application: I am using nginx to load balance across two instances of this application. This set-up is very easy to run using Spring boot, I brought up two instances of the app up using two different server ports, this way: mvn spring-boot:run -Dserver.port=8080 mvn spring-boot:run -Dserver.port=8082 and this is my nginx.conf to load balance across these two instances: events { worker_connections 1024; } http { upstream sessionApp { server localhost:8080; server localhost:8082; } server { listen 80; location / { proxy_pass http://sessionApp; } } } I display the port number of the application in the footer just to show which instance is handling the request. If I were to do nothing to move the state of the session out the application then the behavior of the application would be erratic as the session established on one instance of the application would not be recognized by the other instance - specifically if Tomcat receives a session id it does not recognize then the behavior is to create a new session. Introducing Spring session into the application There are container specific ways to introduce a external session stores - One example is here, where Redis is configured as a store for Tomcat. PivotalGemfire provides a module to externalize Tomcat's session state. The advantage of using Spring-session is that there is no dependence on the container at all - maintaining session state becomes an application concern. The instructions on configuring an application to use Spring session is detailed very well at the Spring-session site, just to quickly summarize how I have configured my Spring Boot application, these are first the dependencies that I have pulled in: org.springframework.session spring-session 1.0.0.BUILD-SNAPSHOT org.springframework.session spring-session-data-redis 1.0.0.BUILD-SNAPSHOT org.springframework.data spring-data-redis 1.4.1.RELEASE redis.clients jedis 2.4.1 and my configuration to use Spring-session for session support, note the Spring Boot specific FilterRegistrationBean which is used to register the session repository filter: import org.springframework.boot.context.embedded.FilterRegistrationBean; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.core.annotation.Order; import org.springframework.data.redis.connection.jedis.JedisConnectionFactory; import org.springframework.session.data.redis.config.annotation.web.http.EnableRedisHttpSession; import org.springframework.session.web.http.SessionRepositoryFilter; import org.springframework.web.filter.DelegatingFilterProxy; import java.util.Arrays; @Configuration @EnableRedisHttpSession public class SessionRepositoryConfig { @Bean @Order(value = 0) public FilterRegistrationBean sessionRepositoryFilterRegistration(SessionRepositoryFilter springSessionRepositoryFilter) { FilterRegistrationBean filterRegistrationBean = new FilterRegistrationBean(); filterRegistrationBean.setFilter(new DelegatingFilterProxy(springSessionRepositoryFilter)); filterRegistrationBean.setUrlPatterns(Arrays.asList("/*")); return filterRegistrationBean; } @Bean public JedisConnectionFactory connectionFactory() { return new JedisConnectionFactory(); } } And that is it! magically now all session is handled by Spring-session, and neatly externalized to Redis. If I were to retry my previous configuration of using nginx to load balance two different Spring-Boot applications using the common Redis store, the application just works irrespective of the instance handling the request. I look forward to further enhancements to this excellent new project. The sample application which makes use of Spring-session is available here: https://github.com/bijukunjummen/shopping-cart-cf-app.git