In an earlier blog entry I had touched on a fictitious rube goldberg flow for capitalizing a string through a complicated series of steps, the premise of the article was to introduce Spring Integration Java DSL as an alternative to defining integration flows through xml configuration files. I learned a few new things after writing that blog entry, thanks to Artem Bilan and wanted to document those learnings here: So, first my original sample, here I have the following flow(the one's in bold): Take in a message of this type - "hello from spring integ" Split it up into individual words(hello, from, spring, integ) Send each word to a ActiveMQ queue Pick up the word fragments from the queue and capitalize each word Place the response back into a response queue Pick up the message, re-sequence based on the original sequence of the words Aggregate back into a sentence("HELLO FROM SPRING INTEG") and Return the sentence back to the calling application. EchoFlowOutbound.java: @Bean public DirectChannel sequenceChannel() { return new DirectChannel(); } @Bean public DirectChannel requestChannel() { return new DirectChannel(); } @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from(requestChannel()) .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .get(); } @Bean public IntegrationFlow flowOnReturnOfMessage() { return IntegrationFlows.from(sequenceChannel()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); jmsOutboundGateway.setReplyChannel(sequenceChannel()); return jmsOutboundGateway; } It turns out, based on Artem Bilan's feedback, that a few things can be optimized here. First notice how I have explicitly defined two direct channels, "requestChannel" for starting the flow that takes in the string message and the "sequenceChannel" to handle the message once it returns back from the jms message queue, these can actually be totally removed and the flow made a little more concise this way: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); return jmsOutboundGateway; } "requestChannel" is now being implicitly created just by declaring a name for it. The sequence channel is more interesting, quoting Artem Bilan - do not specify outputChannel for AbstractReplyProducingMessageHandler and rely on DSL , what it means is that here jmsOutboundGateway is a AbstractReplyProducingMessageHandler and its reply channel is implicitly derived by the DSL. Further, two methods which were earlier handling the flows for sending out the message to the queue and then continuing once the message is back, is collapsed into one. And IMHO it does read a little better because of this change. The second good change and the topic of this article is the introduction of the Jms namespace factories, when I had written the previous blog article, DSL had support for defining the AMQ inbound/outbound adapter/gateway, now there is support for Jms based inbound/adapter adapter/gateways also, this simplifies the flow even further, the flow now looks like this: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(Jms.outboundGateway(connectionFactory) .requestDestination("amq.outbound")) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } The inbound Jms part of the flow also simplifies to the following: @Bean public IntegrationFlow inboundFlow() { return IntegrationFlows.from(Jms.inboundGateway(connectionFactory) .destination("amq.outbound")) .transform((String s) -> s.toUpperCase()) .get(); } Thus, to conclude, Spring Integration Java DSL is an exciting new way to concisely configure Spring Integration flows. It is already very impressive in how it simplifies the readability of flows, the introduction of the Jms namespace factories takes it even further for JMS based flows.

some diagrams for the spring petclinic application following on from my previous post ( software architecture as code ) where i demonstrated how to create a software architecture model as code, i decided to throw together a quick implementation of a spring component finder that could be used to (mostly) automatically create a model of a spring mvc web application. spring has a bunch of annotations (e.g. @controller, @component, @service and @repository) and these are often/can be used to signify the major building blocks of a web application. to illustrate this, i took the spring petclinic application and produced some diagrams for it. first is a context diagram. next up are the containers, which in this case are just a web server (e.g. apache tomcat) and a database (hsqldb by default). and finally we have a diagram showing the components that make up the web application. these, and their dependencies, were found by scanning the compiled version of the application (i cloned the project from github and ran the maven build). here is the code that i used to generate the model behind the diagrams. package com.structurizr.example; import com.fasterxml.jackson.databind.objectmapper; import com.fasterxml.jackson.databind.serializationfeature; import com.structurizr.componentfinder.componentfinder; import com.structurizr.componentfinder.springcomponentfinderstrategy; import com.structurizr.model.*; import com.structurizr.view.componentview; import com.structurizr.view.containerview; import com.structurizr.view.contextview; /** * this is a c4 representation of the spring petclinic sample app (https://github.com/spring-projects/spring-petclinic/). */ public class springpetclinic { public static void main(string[] args) throws exception { model model = new model(); // create the basic model (the stuff we can't get from the code) softwaresystem springpetclinic = model.addsoftwaresystem(location.internal, "spring petclinic", ""); person user = model.addperson(location.external, "user", ""); user.uses(springpetclinic, "uses"); container webapplication = springpetclinic.addcontainer("web application", "", "apache tomcat 7.x"); container relationaldatabase = springpetclinic.addcontainer("relational database", "", "hsqldb"); user.uses(webapplication, "uses"); webapplication.uses(relationaldatabase, "reads from and writes to"); // and now automatically find all spring @controller, @component, @service and @repository components componentfinder componentfinder = new componentfinder(webapplication, "org.springframework.samples.petclinic", new springcomponentfinderstrategy()); componentfinder.findcomponents(); // finally create some views contextview contextview = model.createcontextview(springpetclinic); contextview.addallsoftwaresystems(); contextview.addallpeople(); containerview containerview = model.createcontainerview(springpetclinic); containerview.addallpeople(); containerview.addallsoftwaresystems(); containerview.addallcontainers(); componentview componentview = model.createcomponentview(springpetclinic, webapplication); componentview.addallcomponents(); objectmapper objectmapper = new objectmapper(); objectmapper.enable(serializationfeature.indent_output); string modelasjson = objectmapper.writevalueasstring(model); system.out.println(modelasjson); } } the resulting json representing the model was then copy-pasted across into my simple (and very much in progress) diagramming tool . admittedly the diagrams are lacking on some details (i.e. component responsibilities and arrow annotations, although those can be fixed), but this approach proves you can expend very little effort to get something that is relatively useful. as i've said before, it's all about getting the abstractions right.

We have published the results of the Eclipse Community Survey 2014. Thank you to everyone who participated in the survey this year. The complete results and data are available for anyone to download [xls] [ods]. As in other years, I think the results provide an interesting perspective on what tools software developers are using and the type of applications they are building. Here are some key highlights from the results this year: 1) Git #1 Code Management Tool. Git has finally surpassed Subversion to be the top code management tool used by software developers. A third of developers (33.3%) report they use Git as their primary code management tool compared to 30.7% using Subversion. Subversion continues to show a downward trend from previous years when it was used by more than half the developers. Of note, 9.6% claim GitHub is their primary code management tool so the prevalence of overall Git usage is becoming dominate. 2) Maven and Jenkins Key Tools. For Build and Release tools, Maven and Jenkins continue to be key tools used by developers. Of interest is the growth of Gradle from 2013 (4.5%) to 2014 (11%). 3) Top 3 Application Servers. Tomcat (32.6%), JBoss (11.8%) and Jetty (7.2%) continue to be the top 3 application servers. 4) Java 8 Adoption. Java 8 was released in March 2014 and already 9.2% of Java developers have migrated to Java 8 as their primary version of Java. 59.2% are using Java 7 but close to a quarter are using Java 6 or before. 5) Majority of Developers Use JavaScript. More and more software developers use multiple languages to develop software. Due to the Eclipse biased of the survey, Java is not surprisingly the top primary language. However, when asked what other languages developers might use, JavaScript stands out to be a popular language with a the majority of developers (56.2%) claiming it as a secondary language. 6) Developers Experimenting With Open Hardware. The Internet of Things (IoT) and Open Hardware have become important industry trends in the last couple of years. Over a third (35.7%) of software developers are spending their own personal time learning about devices like the BeagleBone, Arduino and Raspberry Pi. Thanks again to everyone who participated in the survey. I hope everyone finds the results of interest.

while writing my spring-hibernate integration post i realize that we (as java developers) have so many frameworks available which can make life easy by rapidly developing so many common things using frameworks. this ease of development is not available in java particularly when you considering ui development, we still have to struggle with old, bad looking jsp with combination of css, jquery. thank god we now have so many javascript toolkits, framework that make developers' lives easy by offering so many ready-to-use widgets. extjs is the most advanced among those client side ui frameworks. today i am going to demonstrate to you how you can leverage extjs 4 to create crud application. in the next post i will try to use the same js code with spring mvc as a backend . create a html page which include extjs library and aur books.js file. here is the extjs crud code, i have combined it in a single file. you can follow the extjs 4 mvc folder structure. ext.onready(function () { ext.define('techzoo.model.book', { extend: 'ext.data.model', fields: [ {name: 'title', type: 'string'}, {name: 'author', type: 'string'}, {name: 'price', type: 'int'}, {name: 'qty', type: 'int'} ] }); ext.define('techzoo.store.books', { extend : 'ext.data.store', model : 'techzoo.model.book', fields : ['title', 'author','price', 'qty'], data : [ { title: 'jdbc, servlet and jsp', author: 'santosh kumar', price: 300, qty : 12000 }, { title: 'head first java', author: 'kathy sierra', price: 550, qty : 2500 }, { title: 'java scjp certification', author: 'khalid mughal', price: 650, qty : 5500 }, { title: 'spring and hinernate', author: 'santosh kumar', price: 350, qty : 2500 }, { title: 'mastering c++', author: 'k. r. venugopal', price: 400, qty : 1200 } ] }); ext.define('techzoo.view.bookslist', { extend: 'ext.grid.panel', alias: 'widget.bookslist', title: 'books list - (extjs 4 crud example - @ tousif khan)', store: 'books', initcomponent: function () { this.tbar = [{ text : 'add book', action : 'add', iconcls : 'book-add' }]; this.columns = [ { header: 'title', dataindex: 'title', flex: 1 }, { header: 'author', dataindex: 'author' }, { header: 'price', dataindex: 'price' , width: 60 }, { header: 'quantity', dataindex: 'qty', width: 80 }, { header: 'action', width: 50, renderer: function (v, m, r) { var id = ext.id(); var max = 15; ext.defer(function () { ext.widget('image', { renderto: id, name: 'delete', src : 'images/book_delete.png', listeners : { afterrender: function (me) { me.getel().on('click', function() { var grid = ext.componentquery.query('bookslist')[0]; if (grid) { var sm = grid.getselectionmodel(); var rs = sm.getselection(); if (!rs.length) { ext.msg.alert('info', 'no book selected'); return; } ext.msg.confirm('remove book', 'are you sure you want to delete?', function (button) { if (button == 'yes') { grid.store.remove(rs[0]); } }); } }); } } }); }, 50); return ext.string.format('', id); } } ]; this.callparent(arguments); } }); ext.define('techzoo.view.booksform', { extend : 'ext.window.window', alias : 'widget.booksform', title : 'add book', width : 350, layout : 'fit', resizable: false, closeaction: 'hide', modal : true, config : { recordindex : 0, action : '' }, items : [{ xtype : 'form', layout: 'anchor', bodystyle: { background: 'none', padding: '10px', border: '0' }, defaults: { xtype : 'textfield', anchor: '100%' }, items : [{ name : 'title', fieldlabel: 'book title' },{ name: 'author', fieldlabel: 'author name' },{ name: 'price', fieldlabel: 'price' },{ name: 'qty', fieldlabel: 'quantity' }] }], buttons: [{ text: 'ok', action: 'add' },{ text : 'reset', handler : function () { this.up('window').down('form').getform().reset(); } },{ text : 'cancel', handler: function () { this.up('window').close(); } }] }); ext.define('techzoo.controller.books', { extend : 'ext.app.controller', stores : ['books'], views : ['bookslist', 'booksform'], refs : [{ ref : 'formwindow', xtype : 'booksform', selector: 'booksform', autocreate: true }], init: function () { this.control({ 'bookslist > toolbar > button[action=add]': { click: this.showaddform }, 'bookslist': { itemdblclick: this.onrowdblclick }, 'booksform button[action=add]': { click: this.doaddbook } }); }, onrowdblclick: function(me, record, item, index) { var win = this.getformwindow(); win.settitle('edit book'); win.setaction('edit'); win.setrecordindex(index); win.down('form').getform().setvalues(record.getdata()); win.show(); }, showaddform: function () { var win = this.getformwindow(); win.settitle('add book'); win.setaction('add'); win.down('form').getform().reset(); win.show(); }, doaddbook: function () { var win = this.getformwindow(); var store = this.getbooksstore(); var values = win.down('form').getvalues(); var action = win.getaction(); var book = ext.create('techzoo.model.book', values); if(action == 'edit') { store.removeat(win.getrecordindex()); store.insert(win.getrecordindex(), book); } else { store.add(book); } win.close(); } }); ext.application({ name : 'techzoo', controllers: ['books'], launch: function () { ext.widget('bookslist', { width : 500, height: 300, renderto: 'output' }); } } ); }); output: open the html file in any browser, the output will look similar to below. you can click to add new book record in grid. double click to any of the grid row to edit the record. view live demo

I recently saw Mike Wienser’s SpringOne2GX talk about Application Security Pitfalls. It is very informative and worth watching if you are using Spring’s stack on servlet container. It reminded me one serious Spring Security Misconfiguration I was facing once. Going to explain it on Spring’s Guide Project called Securing a Web Application. This project uses Spring Boot, Spring Integration and Spring MVC. Project uses these views: @Configuration public class MvcConfig extends WebMvcConfigurerAdapter { @Override public void addViewControllers(ViewControllerRegistry registry) { registry.addViewController("/home").setViewName("home"); registry.addViewController("/").setViewName("home"); registry.addViewController("/hello").setViewName("hello"); registry.addViewController("/login").setViewName("login"); } } Where “/home”, “/” and “/login” URLs should be publicly accessible and “/hello” should be accessible only to authenticated user. Here is original Spring Security configuration from Guide: @Configuration @EnableWebMvcSecurity public class WebSecurityConfig extends WebSecurityConfigurerAdapter { @Override protected void configure(HttpSecurity http) throws Exception { http .authorizeRequests() .antMatchers("/", "/home").permitAll() .anyRequest().authenticated(); http .formLogin() .loginPage("/login") .permitAll() .and() .logout() .permitAll(); } @Override protected void configure(AuthenticationManagerBuilder auth) throws Exception { auth .inMemoryAuthentication() .withUser("user").password("password").roles("USER"); } } Nice and explanatory as all Spring’s Guides are. First “configure” method registers “/” and “home” as public and specifies that everything else should be authenticated. It also registers login URL. Second “configure” method specifies authentication method for role “USER”. Of course you don’t want to use it like this in production :). Now I am going to slightly amend this code. @Override protected void configure(HttpSecurity http) throws Exception { //!!! Don't use this example !!! http .authorizeRequests() .antMatchers("/hello").hasRole("USER"); //... same as above ... } Everything is public and private endpoints have to be listed. You can see that my amended code have the same behavior as original. In fact it saved one line of code. But there is serious problem with this. What if my I need to introduce new private endpoint? Let’s say I am not aware of the fact that it needs to be registered in Spring Security configuration. My new endpoint would be public. Such misconfiguration is really hard to catch and can lead to unwanted exposure of URLs. So conclusion is: Always authenticate all endpoints by default.

One more of the many cool features of thymeleaf is the ability to render fragments of templates - I have found this to be an especially useful feature to use with AngularJs. AngularJS $routeProvider or AngularUI router can be configured to return partial views for different "paths", using thymeleaf to return these partial views works really well. Consider a simple CRUD flow, with the AngularUI router views defined this way: app.config(function ($stateProvider, $urlRouterProvider) { $urlRouterProvider.otherwise("list"); $stateProvider .state('list', { url:'/list', templateUrl: URLS.partialsList, controller: 'HotelCtrl' }) .state('edit', { url:'/edit/:hotelId', templateUrl: URLS.partialsEdit, controller: 'HotelEditCtrl' }) .state('create', { url:'/create', templateUrl: URLS.partialsCreate, controller: 'HotelCtrl' }); }); The templateUrl above is the partial view rendered when the appropriate state is activated, here these are defined using javascript variables and set using thymeleaf templates this way(to cleanly resolve the context path of the deployed application as the root path): Now, consider one of the fragment definitions, say the one handling the list: file: templates/hotels/partialList.html Hotels IDNameAddressZipAction{{hotel.id}{{hotel.name}{{hotel.address}{{hotel.zip}Edit | Delete New Hotel The great thing about thymeleaf here is that this view can be opened up in a browser and previewed. To return the part of the view, which in this case is the section which starts with "th:fragment="content"", all I have to do is to return the name of the view as "hotels/partialList::content"! The same approach can be followed for the update and the create views. One part which I have left open is about how the uri in the UI which is "/hotels/partialsList" maps to "hotels/partialList::content", with Spring MVC this can be easily done through a View Controller, which is essentially a way to return a view name without needing to go through a Controller and can be configured this way: @Configuration public class WebConfig extends WebMvcConfigurerAdapter { @Override public void addViewControllers(ViewControllerRegistry registry) { registry.addViewController("/hotels/partialsList").setViewName("hotels/partialsList::content"); registry.addViewController("/hotels/partialsCreate").setViewName("hotels/partialsCreate::content"); registry.addViewController("/hotels/partialsEdit").setViewName("hotels/partialsEdit::content"); } } So to summarize, you create a full html view using thymeleaf templates which can be previewed and any rendering issues fixed by opening the view in a browser during development time and then return the fragment of the view at runtime purely by referring to the relevant section of the html page. A sample which follows this pattern is available at this github location: https://github.com/bijukunjummen/spring-boot-mvc-test

With the web being used on so many different devices now it's very important that you can change your design to fit on different screen sizes. The best way of changing your display depending on screen size is to use media queries to find out the size viewport of the screen and allowing you to change the design depending on what screen size is on. You will mainly make these changes in the CSS file as you can define the media query break points to change the design on different devices like this. /* Smartphones (portrait) ----------- */ @media only screen and (max-width : 320px) { } /* Desktops and laptops ----------- */ @media only screen and (min-width : 1224px) { } /* Large screens ----------- */ @media only screen and (min-width : 1824px) { } The above code will allow you to make styling completely different on different devices, but what if you wanted to change the functionality of the site depending on the screen size? What if you needed to use some Javascript code on different screen sizes, for example to create a slide down navigation button. How Do You Use Media Queries With jQuery Media queries will be checking the width of the window to see what the size of the device is so you would think that you can use a method like .width() on the window object like this. if($(window).width() < 767) { // change functionality for smaller screens } else { // change functionality for larger screens } But this will not return the true value of the window as it takes into effect things like body padding and scroll bars on the window. The other option you have when checking the media size is to use a Javascript method of .matchMedia() on the window object. var window_size = window.matchMedia('(max-width: 768px)')); This works the same way as media queries and is supported on many browsers apart from IE9 and lower. Can I Use window.matchMedia To use matchMedia you need to pass in the min or max values you want to check (like media queries) and see if the viewport matches this. if (window.matchMedia('(max-width: 768px)').matches) { // do functionality on screens smaller than 768px } Now you can use this to add a click event on to a sub-menu for screens smaller than 768px. The below code is an example of how you can add some Javascript code which will only be run on screens smaller than 768px. if (window.matchMedia('(max-width: 768px)').matches) { $('.sub-menu-button').on('click', function(e) { var subMenu = $(this).next('.sub-navigation'); if(subMenu.is(':visible')) { subMenu.slideUp(); } else { subMenu.slideDown(); } return false; }); }

Wondering how Spring @Transactional works? Learn about what's really going on.

A common tactic to help speed up your website is to use a technique called lazy loading which means that instead of loading everything your page needs at the start it will only load resources as and when it needs them. For example you can lazy load images so you can start the page off only with the images you need to view the page correctly, then other images that are out of view you won't need to load straight away. As the user scrolls down the page you can then search to see if the images are about to come into view and lazy load the images when they are needed. You can do the same with other resources such as JavaScript or CSS files, you can make sure you only load in the script as and when they need them to be used. An example of this I have used in the past is loading Disqus comments on the click event of a button, this jQuery code will then load in the Disqus Javascript file and initialise the Disqus code on the selected div. $j=jQuery.noConflict(); $j(document).ready(function() { $j('.showDisqus').on('click', function(){ // click event of the show comments button var disqus_shortname = 'enter_your_disqus_user_name'; // Enter your disqus user name // ajax request to load the disqus javascript $j.ajax({ type: "GET", url: "http://" + disqus_shortname + ".disqus.com/embed.js", dataType: "script", cache: true }); $j(this).fadeOut(); // remove the show comments button }); }); Ajax Method As you can see from the code above we have a click event of the .showDisqus button, inside this using the jQuery method .ajax() which is making a GET request for the script of embedding Disqus into your application. The ajax method is normally used to make basic http requests to a server side script and return the output of the script. On this occasion we are making a GET request and setting the dataType to be a script. This tells jQuery to treat the return as if we are including a new JavaScript file, this will disable browser caching on the script by adding a timestamp parameter to the end of the script. If you would like to enable caching of the script then you need to make sure you include a cache: true parameter. $.ajax({ type: "GET", url: "http://test_script.js", dataType: "script", cache: true }); Get Script Method Another option to get the script via GET ajax is to use the method getScript() this is simply a wrapper for the above ajax method. $.ajax({ url: url, dataType: "script", success: success }); This allows you to reduce the amount of code you are writing by simply using this method. $.getScript( "http://test_script.js" ) .done(function( script, textStatus ) { alert('Successfully loaded script'); }) .fail(function( jqxhr, settings, exception ) { alert('Failed to load script'); }); The problem with using getScript() is that it will never cache the script as it will always add the timestamp querystring to the end of the JavaScript file. As the ajax() method allows you to choose if you cache the script or not it is better to use this method when loading in a script that will not change.

A new Java based DSL has now been introduced for Spring Integration which makes it possible to define the Spring Integration message flows using pure java based configuration instead of using the Spring XML based configuration. I tried the DSL for a sample Integration flow that I have - I call it the Rube Goldberg flow, for it follows a convoluted path in trying to capitalize a string passed in as input. The flow looks like this and does some crazy things to perform a simple task: It takes in a message of this type - "hello from spring integ" splits it up into individual words(hello, from, spring, integ) sends each word to a ActiveMQ queue from the queue the word fragments are picked up by a enricher to capitalize each word placing the response back into a response queue It is picked up, resequenced based on the original sequence of the words aggregated back into a sentence("HELLO FROM SPRING INTEG") and returned back to the application. To start with Spring Integration Java DSL, a simple Xml based configuration to capitalize a String would look like this: There is nothing much going on here, a messaging gateway takes in the message passed in from the application, capitalizes it in a transformer and this is returned back to the application. Expressing this in Spring Integration Java DSL: @Configuration @EnableIntegration @IntegrationComponentScan @ComponentScan public class EchoFlow { @Bean public DirectChannel requestChannel() { return new DirectChannel(); } @Bean public IntegrationFlow simpleEchoFlow() { return IntegrationFlows.from(requestChannel()) .transform((String s) -> s.toUpperCase()) .get(); } } @MessagingGateway public interface EchoGateway { @Gateway(requestChannel = "requestChannel") String echo(String message); } Do note that @MessagingGateway annotation is not a part of Spring Integration Java DSL, it is an existing component in Spring Integration and serves the same purpose as the gateway component in XML based configuration. I like the fact that the transformation can be expressed using typesafe Java 8 lambda expressions rather than the Spring-EL expression. Note that the transformation expression could have coded in quite few alternate ways: ??.transform((String s) -> s.toUpperCase()) Or: ??.transform(s -> s.toUpperCase()) Or using method references: ??.transform(String::toUpperCase) Moving onto the more complicated Rube Goldberg flow to accomplish the same task, again starting with XML based configuration. There are two configurations to express this flow: rube-1.xml: This configuration takes care of steps 1, 2, 3, 6, 7, 8 : It takes in a message of this type - "hello from spring integ" splits it up into individual words(hello, from, spring, integ) sends each word to a ActiveMQ queue from the queue the word fragments are picked up by a enricher to capitalize each word placing the response back into a response queue It is picked up, resequenced based on the original sequence of the words aggregated back into a sentence("HELLO FROM SPRING INTEG") and returned back to the application. and rube-2.xml for steps 4, 5: It takes in a message of this type - "hello from spring integ" splits it up into individual words(hello, from, spring, integ) sends each word to a ActiveMQ queue from the queue the word fragments are picked up by a enricher to capitalize each word placing the response back into a response queue It is picked up, resequenced based on the original sequence of the words aggregated back into a sentence("HELLO FROM SPRING INTEG") and returned back to the application. Now, expressing this Rube Goldberg flow using Spring Integration Java DSL, the configuration looks like this, again in two parts: EchoFlowOutbound.java: @Bean public DirectChannel sequenceChannel() { return new DirectChannel(); } @Bean public DirectChannel requestChannel() { return new DirectChannel(); } @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from(requestChannel()) .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .get(); } @Bean public IntegrationFlow flowOnReturnOfMessage() { return IntegrationFlows.from(sequenceChannel()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } and EchoFlowInbound.java: @Bean public JmsMessageDrivenEndpoint jmsInbound() { return new JmsMessageDrivenEndpoint(listenerContainer(), messageListener()); } @Bean public IntegrationFlow inboundFlow() { return IntegrationFlows.from(enhanceMessageChannel()) .transform((String s) -> s.toUpperCase()) .get(); } Again here the code is completely typesafe and is checked for any errors at development time rather than at runtime as with the XML based configuration. Again I like the fact that transformation, aggregation statements can be expressed concisely using Java 8 lamda expressions as opposed to Spring-EL expressions. What I have not displayed here is some of the support code, to set up the activemq test infrastructure, this configuration continues to remain as xml and I have included this code in a sample github project. All in all, I am very excited to see this new way of expressing the Spring Integration messaging flow using pure Java and I am looking forward to seeing its continuing evolution and may be even try and participate in its evolution in small ways. Here is the entire working code in a github repo: https://github.com/bijukunjummen/rg-si References and Acknowledgement: Spring Integration Java DSL introduction blog article by Artem Bilan: https://spring.io/blog/2014/05/08/spring-integration-java-dsl-milestone-1-released Spring Integration Java DSL website and wiki: https://github.com/spring-projects/spring-integration-extensions/wiki/Spring-Integration-Java-DSL-Reference. A lot of code has been shamelessly copied over from this wiki by me :-). Also, a big thanks to Artem for guidance on a question that I had Webinar by Gary Russell on Spring Integration 4.0 in which Spring Integration Java DSL is covered in great detail.

After attending Sam Newman’s microservice talks at Geecon last week I started to think more about what is most likely an essential feature of service-oriented/microservice platforms for monitoring, reporting and diagnostics: correlation ids. Correlation ids allow distributed tracing within complex service oriented platforms, where a single request into the application can often be dealt with by multiple downstream service. Without the ability to correlate downstream service requests it can be very difficult to understand how requests are being handled within your platform. I’ve seen the benefit of correlation ids in several recent SOA projects I have worked on, but as Sam mentioned in his talks, it’s often very easy to think this type of tracing won’t be needed when building the initial version of the application, but then very difficult to retrofit into the application when you do realise the benefits (and the need for!). I’ve not yet found the perfect way to implement correlation ids within a Java/Spring-based application, but after chatting to Sam via email he made several suggestions which I have now turned into a simple project using Spring Boot to demonstrate how this could be implemented. Why? During both of Sam’s Geecon talks he mentioned that in his experience correlation ids were very useful for diagnostic purposes. Correlation ids are essentially an id that is generated and associated with a single (typically user-driven) request into the application that is passed down through the stack and onto dependent services. In SOA or microservice platforms this type of id is very useful, as requests into the application typically are ‘fanned out’ or handled by multiple downstream services, and a correlation id allows all of the downstream requests (from the initial point of request) to be correlated or grouped based on the id. So called ‘distributed tracing’ can then be performed using the correlation ids by combining all the downstream service logs and matching the required id to see the trace of the request throughout your entire application stack (which is very easy if you are using a centralised logging framework such as logstash) The big players in the service-oriented field have been talking about the need for distributed tracing and correlating requests for quite some time, and as such Twitter have created their open source Zipkin framework (which often plugs into their RPC framework Finagle), and Netflix has open-sourced their Karyon web/microservice framework, both of which provide distributed tracing. There are of course commercial offering in this area, one such product being AppDynamics, which is very cool, but has a rather hefty price tag. Creating a proof-of-concept in Spring Boot As great as Zipkin and Karyon are, they are both relatively invasive, in that you have to build your services on top of the (often opinionated) frameworks. This might be fine for some use cases, but no so much for others, especially when you are building microservices. I’ve been enjoying experimenting with Spring Boot of late, and this framework builds on the much known and loved (at least by me :-) ) Spring framework by providing lots of preconfigured sensible defaults. This allows you to build microservices (especially ones that communicate via RESTful interfaces) very rapidly. The remainder of this blog pos explains how I implemented a (hopefully) non-invasive way of implementing correlation ids. Goals Allow a correlation id to be generated for a initial request into the application Enable the correlation id to be passed to downstream services, using as method that is as non-invasive into the code as possible Implementation I have created two projects on GitHub, one containing an implementation where all requests are being handled in a synchronous style (i.e. the traditional Spring approach of handling all request processing on a single thread), and also one for when an asynchronous (non-blocking) style of communication is being used (i.e., using the Servlet 3 asynchronous support combined with Spring’s DeferredResult and Java’s Futures/Callables). The majority of this article describes the asynchronous implementation, as this is more interesting: Spring Boot asynchronous (DeferredResult + Futures) communication correlation id Github repo The main work in both code bases is undertaken by the CorrelationHeaderFilter, which is a standard Java EE Filter that inspects the HttpServletRequest header for the presence of a correlationId. If one is found then we set a ThreadLocal variable in the RequestCorrelation Class (discussed later). If a correlation id is not found then one is generated and added to the RequestCorrelation Class: public class CorrelationHeaderFilter implements Filter { //... @Override public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException { final HttpServletRequest httpServletRequest = (HttpServletRequest) servletRequest; String currentCorrId = httpServletRequest.getHeader(RequestCorrelation.CORRELATION_ID_HEADER); if (!currentRequestIsAsyncDispatcher(httpServletRequest)) { if (currentCorrId == null) { currentCorrId = UUID.randomUUID().toString(); LOGGER.info("No correlationId found in Header. Generated : " + currentCorrId); } else { LOGGER.info("Found correlationId in Header : " + currentCorrId); } RequestCorrelation.setId(currentCorrId); } filterChain.doFilter(httpServletRequest, servletResponse); } //... private boolean currentRequestIsAsyncDispatcher(HttpServletRequest httpServletRequest) { return httpServletRequest.getDispatcherType().equals(DispatcherType.ASYNC); } The only thing is this code that may not instantly be obvious is the conditional checkcurrentRequestIsAsyncDispatcher(httpServletRequest), but this is here to guard against the correlation id code being executed when the Async Dispatcher thread is running to return the results (this is interesting to note, as I initially didn’t expect the Async Dispatcher to trigger the execution of the filter again?) Here is the RequestCorrelation Class, which contains a simple ThreadLocal static variable to hold the correlation id for the current Thread of execution (set via the CorrelationHeaderFilter above) public class RequestCorrelation { public static final String CORRELATION_ID = "correlationId"; private static final ThreadLocal id = new ThreadLocal(); public static String getId() { return id.get(); } public static void setId(String correlationId) { id.set(correlationId); } } Once the correlation id is stored in the RequestCorrelation Class it can be retrieved and added to downstream service requests (or data store access etc) as required by calling the static getId() method within RequestCorrelation. It is probably a good idea to encapsulate this behaviour away from your application services, and you can see an example of how to do this in a RestClient Class I have created, which composes Spring’s RestTemplate and handles the setting of the correlation id within the header transparently from the calling Class. @Component public class CorrelatingRestClient implements RestClient { private RestTemplate restTemplate = new RestTemplate(); @Override public String getForString(String uri) { String correlationId = RequestCorrelation.getId(); HttpHeaders httpHeaders = new HttpHeaders(); httpHeaders.set(RequestCorrelation.CORRELATION_ID, correlationId); LOGGER.info("start REST request to {} with correlationId {}", uri, correlationId); //TODO: error-handling and fault-tolerance in production ResponseEntity response = restTemplate.exchange(uri, HttpMethod.GET, new HttpEntity(httpHeaders), String.class); LOGGER.info("completed REST request to {} with correlationId {}", uri, correlationId); return response.getBody(); } } //... calling Class public String exampleMethod() { RestClient restClient = new CorrelatingRestClient(); return restClient.getForString(URI_LOCATION); //correlation id handling completely abstracted to RestClient impl } Making this work for asynchronous requests… The code included above works fine when you are handling all of your requests synchronously, but it is often a good idea in a SOA/microservice platform to handle requests in a non-blocking asynchronous manner. In Spring this can be achieved by using the DeferredResult Class in combination with the Servlet 3 asynchronous support. The problem with using ThreadLocal variables within the asynchronous approach is that the Thread that initially handles the request (and creates the DeferredResult/Future) will not be the Thread doing the actual processing. Accordingly, a bit of glue code is needed to ensure that the correlation id is propagated across the Threads. This can be achieved by extending Callable with the required functionality: (don’t worry if example Calling Class code doesn’t look intuitive – this adaption between DeferredResults and Futures is a necessary evil within Spring, and the full code including the boilerplate ListenableFutureAdapter is in my GitHub repo): public class CorrelationCallable implements Callable { private String correlationId; private Callable callable; public CorrelationCallable(Callable targetCallable) { correlationId = RequestCorrelation.getId(); callable = targetCallable; } @Override public V call() throws Exception { RequestCorrelation.setId(correlationId); return callable.call(); } } //... Calling Class @RequestMapping("externalNews") public DeferredResult externalNews() { return new ListenableFutureAdapter<>(service.submit(new CorrelationCallable<>(externalNewsService::getNews))); } And there we have it – the propagation of correlation id regardless of the synchronous/asynchronous nature of processing! You can clone the Github report containing my asynchronous example, and execute the application by running mvn spring-boot:run at the command line. If you access http://localhost:8080/externalNewsin your browser (or via curl) you will see something similar to the following in your Spring Boot console, which clearly demonstrates a correlation id being generated on the initial request, and then this being propagated through to a simulated external call (have a look in the ExternalNewsServiceRest Class to see how this has been implemented): [nio-8080-exec-1] u.c.t.e.c.w.f.CorrelationHeaderFilter : No correlationId found in Header. Generated : d205991b-c613-4acd-97b8-97112b2b2ad0 [pool-1-thread-1] u.c.t.e.c.w.c.CorrelatingRestClient : start REST request to http://localhost:8080/news with correlationId d205991b-c613-4acd-97b8-97112b2b2ad0 [nio-8080-exec-2] u.c.t.e.c.w.f.CorrelationHeaderFilter : Found correlationId in Header : d205991b-c613-4acd-97b8-97112b2b2ad0 [pool-1-thread-1] u.c.t.e.c.w.c.CorrelatingRestClient : completed REST request to http://localhost:8080/news with correlationId d205991b-c613-4acd-97b8-97112b2b2ad0 Conclusion I’m quite happy with this simple prototype, and it does meet the two goals I listed above. Future work will include writing some tests for this code (shame on me for not TDDing!), and also extend this functionality to a more realistic example. I would like to say a massive thanks to Sam, not only for sharing his knowledge at the great talks at Geecon, but also for taking time to respond to my emails. If you’re interested in microservices and related work I can highly recommend Sam’s Microservice book which is available in Early Access at O’Reilly. I’ve enjoyed reading the currently available chapters, and having implemented quite a few SOA projects recently I can relate to a lot of the good advice contained within. I’ll be following the development of this book with keen interest! If you have any comments or thoughts then please do share them via the comment below, or feel free to get in touch via the usual mechanisms! References I used Tomasz Nurkiewicz’s excellent blog several times for learning how best to wire up all of the DeferredResult/Future code in Spring: http://www.nurkiewicz.com/2013/03/deferredresult-asynchronous-processing.html

Utility Method for Spring REST public static HttpServletRequest getCurrentRequest() { RequestAttributes requestAttributes = RequestContextHolder.getRequestAttributes(); Assert.state(requestAttributes != null, "Could not find current request via RequestContextHolder"); Assert.isInstanceOf(ServletRequestAttributes.class, requestAttributes); HttpServletRequest servletRequest = ((ServletRequestAttributes) requestAttributes).getRequest(); Assert.state(servletRequest != null, "Could not find current HttpServletRequest"); return servletRequest; } Sometimes it’s easier to get the underlying Servlet request to get some headers or variables. final String userIpAddress = getCurrentRequest().getRemoteAddr(); final String userAgent = getCurrentRequest().getHeader("user-agent"); This is used in the simple REST service using HTTP Post verb @ the awesome CloudFoundry: (Source) Tool for Creating Your Test JSON. Spring Boot Documentation

After attending Sam Newman’s microservice talks at Geecon last week I started to think more about what is most likely an essential feature of service-oriented/microservice platforms for monitoring, reporting and diagnostics: correlation ids. Correlation ids allow distributed tracing within complex service oriented platforms, where a single request into the application can often be dealt with by multiple downstream service. Without the ability to correlate downstream service requests it can be very difficult to understand how requests are being handled within your platform. I’ve seen the benefit of correlation ids in several recent SOA projects I have worked on, but as Sam mentioned in his talks, it’s often very easy to think this type of tracing won’t be needed when building the initial version of the application, but then very difficult to retrofit into the application when you do realise the benefits (and the need for!). I’ve not yet found the perfect way to implement correlation ids within a Java/Spring-based application, but after chatting to Sam via email he made several suggestions which I have now turned into a simple project using Spring Boot to demonstrate how this could be implemented. Why? During both of Sam’s Geecon talks he mentioned that in his experience correlation ids were very useful for diagnostic purposes. Correlation ids are essentially an id that is generated and associated with a single (typically user-driven) request into the application that is passed down through the stack and onto dependent services. In SOA or microservice platforms this type of id is very useful, as requests into the application typically are ‘fanned out’ or handled by multiple downstream services, and a correlation id allows all of the downstream requests (from the initial point of request) to be correlated or grouped based on the id. So called ‘distributed tracing’ can then be performed using the correlation ids by combining all the downstream service logs and matching the required id to see the trace of the request throughout your entire application stack (which is very easy if you are using a centralised logging framework such as logstash) The big players in the service-oriented field have been talking about the need for distributed tracing and correlating requests for quite some time, and as such Twitter have created their open source Zipkin framework (which often plugs into their RPC framework Finagle), and Netflix has open-sourced their Karyon web/microservice framework, both of which provide distributed tracing. There are of course commercial offering in this area, one such product being AppDynamics, which is very cool, but has a rather hefty price tag. Creating a proof-of-concept in Spring Boot As great as Zipkin and Karyon are, they are both relatively invasive, in that you have to build your services on top of the (often opinionated) frameworks. This might be fine for some use cases, but no so much for others, especially when you are building microservices. I’ve been enjoying experimenting with Spring Boot of late, and this framework builds on the much known and loved (at least by me :-) ) Spring framework by providing lots of preconfigured sensible defaults. This allows you to build microservices (especially ones that communicate via RESTful interfaces) very rapidly. The remainder of this blog pos explains how I implemented a (hopefully) non-invasive way of implementing correlation ids. Goals Allow a correlation id to be generated for a initial request into the application Enable the correlation id to be passed to downstream services, using as method that is as non-invasive into the code as possible Implementation I have created two projects on GitHub, one containing an implementation where all requests are being handled in a synchronous style (i.e. the traditional Spring approach of handling all request processing on a single thread), and also one for when an asynchronous (non-blocking) style of communication is being used (i.e., using the Servlet 3 asynchronous support combined with Spring’s DeferredResult and Java’s Futures/Callables). The majority of this article describes the asynchronous implementation, as this is more interesting: Spring Boot asynchronous (DeferredResult + Futures) communication correlation id Github repo The main work in both code bases is undertaken by the CorrelationHeaderFilter, which is a standard Java EE Filter that inspects the HttpServletRequest header for the presence of a correlationId. If one is found then we set a ThreadLocal variable in the RequestCorrelation Class (discussed later). If a correlation id is not found then one is generated and added to the RequestCorrelation Class: public class CorrelationHeaderFilter implements Filter { //... @Override public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException { final HttpServletRequest httpServletRequest = (HttpServletRequest) servletRequest; String currentCorrId = httpServletRequest.getHeader(RequestCorrelation.CORRELATION_ID_HEADER); if (!currentRequestIsAsyncDispatcher(httpServletRequest)) { if (currentCorrId == null) { currentCorrId = UUID.randomUUID().toString(); LOGGER.info("No correlationId found in Header. Generated : " + currentCorrId); } else { LOGGER.info("Found correlationId in Header : " + currentCorrId); } RequestCorrelation.setId(currentCorrId); } filterChain.doFilter(httpServletRequest, servletResponse); } //... private boolean currentRequestIsAsyncDispatcher(HttpServletRequest httpServletRequest) { return httpServletRequest.getDispatcherType().equals(DispatcherType.ASYNC); } The only thing is this code that may not instantly be obvious is the conditional check currentRequestIsAsyncDispatcher(httpServletRequest), but this is here to guard against the correlation id code being executed when the Async Dispatcher thread is running to return the results (this is interesting to note, as I initially didn’t expect the Async Dispatcher to trigger the execution of the filter again?) Here is the RequestCorrelation Class, which contains a simple ThreadLocal static variable to hold the correlation id for the current Thread of execution (set via the CorrelationHeaderFilter above) public class RequestCorrelation { public static final String CORRELATION_ID = "correlationId"; private static final ThreadLocal id = new ThreadLocal(); public static String getId() { return id.get(); } public static void setId(String correlationId) { id.set(correlationId); } } Once the correlation id is stored in the RequestCorrelation Class it can be retrieved and added to downstream service requests (or data store access etc) as required by calling the static getId() method within RequestCorrelation. It is probably a good idea to encapsulate this behaviour away from your application services, and you can see an example of how to do this in a RestClient Class I have created, which composes Spring’s RestTemplate and handles the setting of the correlation id within the header transparently from the calling Class. @Component public class CorrelatingRestClient implements RestClient { private RestTemplate restTemplate = new RestTemplate(); @Override public String getForString(String uri) { String correlationId = RequestCorrelation.getId(); HttpHeaders httpHeaders = new HttpHeaders(); httpHeaders.set(RequestCorrelation.CORRELATION_ID, correlationId); LOGGER.info("start REST request to {} with correlationId {}", uri, correlationId); //TODO: error-handling and fault-tolerance in production ResponseEntity response = restTemplate.exchange(uri, HttpMethod.GET, new HttpEntity(httpHeaders), String.class); LOGGER.info("completed REST request to {} with correlationId {}", uri, correlationId); return response.getBody(); } } //... calling Class public String exampleMethod() { RestClient restClient = new CorrelatingRestClient(); return restClient.getForString(URI_LOCATION); //correlation id handling completely abstracted to RestClient impl } Making this work for asynchronous requests… The code included above works fine when you are handling all of your requests synchronously, but it is often a good idea in a SOA/microservice platform to handle requests in a non-blocking asynchronous manner. In Spring this can be achieved by using the DeferredResult Class in combination with the Servlet 3 asynchronous support. The problem with using ThreadLocal variables within the asynchronous approach is that the Thread that initially handles the request (and creates the DeferredResult/Future) will not be the Thread doing the actual processing. Accordingly, a bit of glue code is needed to ensure that the correlation id is propagated across the Threads. This can be achieved by extending Callable with the required functionality: (don’t worry if example Calling Class code doesn’t look intuitive – this adaption between DeferredResults and Futures is a necessary evil within Spring, and the full code including the boilerplate ListenableFutureAdapter is in my GitHub repo): public class CorrelationCallable implements Callable { private String correlationId; private Callable callable; public CorrelationCallable(Callable targetCallable) { correlationId = RequestCorrelation.getId(); callable = targetCallable; } @Override public V call() throws Exception { RequestCorrelation.setId(correlationId); return callable.call(); } } //... Calling Class @RequestMapping("externalNews") public DeferredResult externalNews() { return new ListenableFutureAdapter<>(service.submit(new CorrelationCallable<>(externalNewsService::getNews))); } And there we have it – the propagation of correlation id regardless of the synchronous/asynchronous nature of processing! You can clone the Github report containing my asynchronous example, and execute the application by running mvn spring-boot:run at the command line. If you access http://localhost:8080/externalNews in your browser (or via curl) you will see something similar to the following in your Spring Boot console, which clearly demonstrates a correlation id being generated on the initial request, and then this being propagated through to a simulated external call (have a look in the ExternalNewsServiceRest Class to see how this has been implemented): [nio-8080-exec-1] u.c.t.e.c.w.f.CorrelationHeaderFilter : No correlationId found in Header. Generated : d205991b-c613-4acd-97b8-97112b2b2ad0 [pool-1-thread-1] u.c.t.e.c.w.c.CorrelatingRestClient : start REST request to http://localhost:8080/news with correlationId d205991b-c613-4acd-97b8-97112b2b2ad0 [nio-8080-exec-2] u.c.t.e.c.w.f.CorrelationHeaderFilter : Found correlationId in Header : d205991b-c613-4acd-97b8-97112b2b2ad0 [pool-1-thread-1] u.c.t.e.c.w.c.CorrelatingRestClient : completed REST request to http://localhost:8080/news with correlationId d205991b-c613-4acd-97b8-97112b2b2ad0 Conclusion I’m quite happy with this simple prototype, and it does meet the two goals I listed above. Future work will include writing some tests for this code (shame on me for not TDDing!), and also extend this functionality to a more realistic example. I would like to say a massive thanks to Sam, not only for sharing his knowledge at the great talks at Geecon, but also for taking time to respond to my emails. If you’re interested in microservices and related work I can highly recommend Sam’s Microservice book which is available in Early Access at O’Reilly. I’ve enjoyed reading the currently available chapters, and having implemented quite a few SOA projects recently I can relate to a lot of the good advice contained within. I’ll be following the development of this book with keen interest! If you have any comments or thoughts then please do share them via the comment below, or feel free to get in touch via the usual mechanisms! References I used Tomasz Nurkiewicz’s excellent blog several times for learning how best to wire up all of the DeferredResult/Future code in Spring: http://www.nurkiewicz.com/2013/03/deferredresult-asynchronous-processing.html

last week i wrote a blog post how to load complete inheritance tree of spring beans into list . a similar feature can be used for autowiring all implementors of a certain interface. let’s have this structure of spring beans. notice that bear is abstract class, therefore it’s not a spring bean. so we have three beas: wolf, polarbear and grizzly. in following snippet are all implementors loaded into list with constructor injection: @service public class nature { list runners; @autowired public nature(list runners) { this.runners = runners; } public void showrunners() { runners.foreach(system.out::println); } } method showrunners is using java 8 foreach method that consumes method reference. this construct outputs loaded beans into console. you would find a lot of reading about these new java 8 features these days. spring context is loaded by this main class: public class main { public static void main(string[] args) { annotationconfigapplicationcontext context = new annotationconfigapplicationcontext(springcontext.class); nature nature = context.getbean(nature.class); nature.showrunners(); } } console output: polarbear [] wolf [] grizzly [] this feature can be handy sometimes. source code of this short example is on github .

We care a lot about the stuff that goes around Solr and Elasticsearch in our client’s infrastructure. One area that seems to always be being reinvented for-better-or-worse is the data ETL/data ingest path from data source X to the search engine. One tool we’ve enjoyed using for basic ETL these days is Apache Camel. Camel is an extremely feature-rich Java data integration framework for wiring up just about anything to anything else. And by anything I mean anything: file system, databases, HTTP, search engines, twitter, IRC, etc. One area I initially struggled with with Camel was exactly how to test my code. Lets say I have defined a simple Camel route like this: from("file:inbox") .unmarshall(csv) // parse as CSV .split() // now we're operating on individual CSV lines .bean("customTransformation") // do some random operation on the CSV line .to("solr://localhost:8983/solr/collection1/update") Great! Now if you’ve gotten into Camel testing, you may know there’s something called “AdviceWith“. What is this interesting sounding thing? Well I think its a way of saying “take these routes and muck with them” — stub out this, intercept that and don’t forward, etc. Exactly the kind of slicing and dicing I’d like to do in my unit tests! I definitely recommend reading up on the docs, but here’s the real step-by-step built around where you’re probably going to get stuck (cause its where I got stuck!) getting AdviceWith to work for your tests. 1. Use CamelTestSupport Ok most importantly, we need to actually define a test that uses CamelTestSupport. CamelTestSupport automatically creates and starts our camel context for us. public class ItGoesToSolrTest extends CamelTestSupport { ... } 2. Specify the route builder we’re testing In our test, we need to tell CamelTestSupport where it can access its routes: @Override protected RouteBuilder createRouteBuilder() { return new MyProductionRouteBuilder(); } 3. Specify any beans we’d like to register Its probably the case that you’re using Java beans with Camel. If you’re using the bean integration and referring to beans by name in your camel routes, you’ll need to register those names with an instance of your class. @Override protected Context createJndiContext() throws Exception { JndiContext context = new JndiContext(); context.bind("customTransformation", new CustomTransformation()); return context; } 4. Monkey with our production routes using advice with Second we need to actually use the AdviceWithRouteBuilder before each test: @Before public void mockEndpoints() throws Exception { AdviceWithRouteBuilder mockSolr = new AdviceWithRouteBuilder() { @Override public void configure() throws Exception { // mock the for testing interceptSendToEndpoint("solr://localhost:8983/solr/collection1/update") .skipSendToOriginalEndpoint() .to("mock:catchSolrMessages"); } }) context.getRouteDefinition(1). .adviceWith(context, mockSolr); } There’s a couple things to notice here: In configure we simply snag an endpoint (in this case Solr) and then we have complete freedom to do whatever we want. In this case, we’re rewiring it to a mock endpoint we can use for testing. Notice how we get a route definition by index (in this case 1) to snag the route we’re testing and that we’d like to monkey with. This is how I’ve seen it in most Camel examples, and its hard to guess how Camel is going to assign some index to your route. A better way would be to give our route definition a name: from(“file:inbox”) .routeId(“csvToSolrRoute”) .unmarshall(csv) // parse as CSV then we can refer to this name when retrieving our route: context.getRouteDefinition("csvToSolrRoute"). .adviceWith(context, mockSolr); 5. Tell CamelTestSupport you want to manually start/stop camel One problem you will run into with the normal tutorials is that CamelTestSupport may start routes before your mocks have taken hold. Thus your mocked routes won’t be part of what CamelTestSupport has actually started. You’ll be pulling your hair out wondering why Camel insists on attempting to forward documents to an actual Solr instance and not your test endpoint. To take matters into your own hands, luckily CamelTestSupport comes to the rescue with a simple method you need to override to communicate your intent to manually start/stop the camel context: @Override public boolean isUseAdviceWith() { return true; } Then in your test, you’ll need to be sure to do: @Test public void foo() { context.start(); // tests! context.stop(); } 6. Write a test! Now you’re equipped to try out a real test! @Test public void testWithRealFile() { MockEndpoint mockSolr = getMockEndpoint("mock:catchSolrMessages"); File testCsv = getTestfile(); context.start(); mockSolr.expectedMessageCount(1); FileUtils.copyFile(testCsv, "inbox"); mockSolr.assertIsSatisfied(); context.stop(); } And that’s just scratching the surface of Camel’s testing capabilities. Check out the camel docs for information on stimulating endpoints directly with the ProducerTemplate thus letting you avoid using real files — and all kinds of goodies. Anyway, hopefully my experiences with AdviceWith can help you get it up and running in your tests! I’d love to hear about your experiences or any tips I’m missing either in the comments or [via email][5]. If you’d love to utilize Solr or Elasticsearch for search and analytics, but can’t figure out how to integrate them with your data infrastructure — contact us! Maybe there’s a camel recipe we could cook up for you that could do just the trick.

If you’ve read the previous five blogs in this series, you’ll know that I’ve been building a Spring application that runs periodically to check a whole bunch of error logs for exceptions and then email you the results. Having written the code and the tests, and being fairly certain it’ll work the next and final step is to package the whole thing up and deploy it to a production machine. The actual deployment and packaging methods will depend upon your own organisation's processes and procedures. In this example, however, I’m going to choose the simplest way possible to create and deploy an executable JAR file. The first step was completed several weeks ago, and that’s defining our output as a JAR file in the Maven POM file, which, as you’ll probably already know, is done using the packaging element: jar It’s okay having a JAR file, but in this case there’s a further step involved: making it executable. To make a JAR file executable you need to add a MANIFEST.MF file and place it in a directory called META-INF. The manifest file is a file that describes the JAR file to both the JVM and human readers. As usual, there are a couple of ways of doing this, for example if you wanted to make life difficult for yourself, you could hand-craft your own file and place it in the META-INF directory inside the project’s src/main/resources directory. On the other hand, you could use themaven-jar plug-in and do it automatically. To do that, you need to to add the following to your POM file. org.apache.maven.plugins maven-jar-plugin 2.4 true com.captaindebug.errortrack.Main lib/ The interesting point here is the configuration element. It contains three sub-elements: addClasspath: this means that the plug-in will add the classpath to the MANIFEST.MF file so that the JVM can find all the support jars when running the app. mainClass: this tells the plug-in to add a Main-Class attribute to the MANIFEST.MF file, so that the JVM knows where to find the the entry point to the application. In this case it’s com.captaindebug.errortrack.Main classpathPrefix: this is really useful. It allows you to locate all the support jars in a different directory to the main part of the application. In this case I’ve chosen the very simple and short name of lib. If you run the build and then open up the resulting JAR file and extract and examine the /META-INF/MANIFEST.MFfile, you’ll find something rather like this: Manifest-Version: 1.0 Built-By: Roger Build-Jdk: 1.7.0_09 Class-Path: lib/spring-context-3.2.7.RELEASE.jar lib/spring-aop-3.2.7.RELEASE.jar lib/aopalliance-1.0.jar lib/spring-beans-3.2.7.RELEASE.jar lib/spring-core-3.2.7.RELEASE.jar lib/spring-expression-3.2.7.RELEASE.jar lib/slf4j-api-1.6.6.jar lib/slf4j-log4j12-1.6.6.jar lib/log4j-1.2.16.jar lib/guava-13.0.1.jar lib/commons-lang3-3.1.jar lib/commons-logging-1.1.3.jar lib/spring-context-support-3.2.7.RELEASE.jar lib/spring-tx-3.2.7.RELEASE.jar lib/quartz-1.8.6.jar lib/mail-1.4.jar lib/activation-1.1.jar Created-By: Apache Maven 3.0.4 Main-Class: com.captaindebug.errortrack.Main Archiver-Version: Plexus Archiver The last step is to marshall all the support jars into one directory, in this case the lib directory, so that the JVM can find them when you run the application. Again, there are two ways of approaching this: the easy way and the hard way. The hard way involves manually collecting together all the JAR files as defined by the POM (both direct and transient dependencies) and copying them to an output directory. The easy way involves getting the maven-dependency-plugin to do it for you. This involves adding the following to your POM file: org.apache.maven.plugins maven-dependency-plugin 2.5.1 copy-dependencies package copy-dependencies ${project.build.directory}/lib/ In this case you’re using the copy-dependencies goal executed in the package phase to copy all the project dependencies to the${project.build.directory}/lib/ directory - note that the final part of the directory path, lib, matches theclasspathPrefix setting from the previous step. In order to make life easier, I’ve also created a small run script: runme.sh: #!/bin/bash echo Running Error Tracking... java -jar error-track-1.0-SNAPSHOT.jar com.captaindebug.errortrack.Main And that’s about it. The application is just about complete. I’ve copied it to my build machine where it now monitors the Captain Debug Github sample apps and build. I could, and indeed may, add a few more features to the app. There are a few rough edges that need knocking off the code: for example is it best to run it as a separate app, or would it be a better idea to turn it into a web app? Furthermore, wouldn’t it be a good idea to ensure that the same errors aren’t reported twice? I may get around to thart soon... or maybe I'll talk about something else; so much to blog about so little time... The code for this blog is available on Github at: https://github.com/roghughe/captaindebug/tree/master/error-track. If you want to look at other blogs in this series take a look here… Tracking Application Exceptions With Spring Tracking Exceptions With Spring - Part 2 - Delegate Pattern Error Tracking Reports - Part 3 - Strategy and Package Private Tracking Exceptions - Part 4 - Spring's Mail Sender Tracking Exceptions - Part 5 - Scheduling With Spring

Spring Boot is a brand new application framework from Spring. It allows fabulously quick development and rapid prototyping (even including CLI). One of its main features is to work from single "uber jar" file. By "uber jar" I mean that all dependencies, even an application server like Tomcat or Jetty are packed into a single file. In that we can start web application by typing java -jar application.jar The only thing we're missing is the managing script. And now I want to dive into that topic. Of course to do anything more than starting our application we need to know its PID. Spring Boot has a solution named ApplicationPidListener. To use it we need to tell SpringApplication we want to include this listener. And there are to ways to achieve that. Easiest way it to create file META-INF/spring.factories containing lines: org.springframework.context.ApplicationListener=\ org.springframework.boot.actuate.system.ApplicationPidListener Second way allows us to customize listener by specifying own name or location for PID file. public class Application { public static void main(String[] args) { SpringApplication springApplication = new SpringApplication(Application.class); springApplication.addListeners( new ApplicationPidListener("app.pid")); springApplication.run(args); } } Now, when we already have our PID file we need bash script providing standard operations like stop, start, restart and status checking. Below you can find simple script solving that challenge. Of course remember to customize highlighted lines :) #!/bin/sh JARFile="application.jar" PIDFile="application.pid" SPRING_OPTS="-DLOG_FILE=application.log" function check_if_pid_file_exists { if [ ! -f $PIDFile ] then echo "PID file not found: $PIDFile" exit 1 fi } function check_if_process_is_running { if ps -p $(print_process) > /dev/null then return 0 else return 1 fi } function print_process { echo $(<"$PIDFile") } case "$1" in status) check_if_pid_file_exists if check_if_process_is_running then echo $(print_process)" is running" else echo "Process not running: $(print_process)" fi ;; stop) check_if_pid_file_exists if ! check_if_process_is_running then echo "Process $(print_process) already stopped" exit 0 fi kill -TERM $(print_process) echo -ne "Waiting for process to stop" NOT_KILLED=1 for i in {1..20}; do if check_if_process_is_running then echo -ne "." sleep 1 else NOT_KILLED=0 fi done echo if [ $NOT_KILLED = 1 ] then echo "Cannot kill process $(print_process)" exit 1 fi echo "Process stopped" ;; start) if [ -f $PIDFile ] && check_if_process_is_running then echo "Process $(print_process) already running" exit 1 fi nohup java $SPRING_OPTS -jar $JARFile & echo "Process started" ;; restart) $0 stop if [ $? = 1 ] then exit 1 fi $0 start ;; *) echo "Usage: $0 {start|stop|restart|status}" exit 1 esac exit 0 I'm sure that there are a lot of possibilities to tune that script, so comments are welcomed :)

I have been using Spring Scala for a toy project for the last few days and I have to say that it is a fantastic project, it simplifies Spring configuration even further when compared to the already simple configuration purely based on Spring Java Config. Let me demonstrate this by starting with the Cake Pattern based sample here: // ======================= // service interfaces trait OnOffDeviceComponent { val onOff: OnOffDevice trait OnOffDevice { def on: Unit def off: Unit } } trait SensorDeviceComponent { val sensor: SensorDevice trait SensorDevice { def isCoffeePresent: Boolean } } // ======================= // service implementations trait OnOffDeviceComponentImpl extends OnOffDeviceComponent { class Heater extends OnOffDevice { def on = println("heater.on") def off = println("heater.off") } } trait SensorDeviceComponentImpl extends SensorDeviceComponent { class PotSensor extends SensorDevice { def isCoffeePresent = true } } // ======================= // service declaring two dependencies that it wants injected trait WarmerComponentImpl { this: SensorDeviceComponent with OnOffDeviceComponent => class Warmer { def trigger = { if (sensor.isCoffeePresent) onOff.on else onOff.off } } } // ======================= // instantiate the services in a module object ComponentRegistry extends OnOffDeviceComponentImpl with SensorDeviceComponentImpl with WarmerComponentImpl { val onOff = new Heater val sensor = new PotSensor val warmer = new Warmer } // ======================= val warmer = ComponentRegistry.warmer warmer.trigger Cake pattern is a pure Scala way of specifying the dependencies. Now, if we were to specify this dependency using Spring's native Java config, but with Scala as the language, firs to define the components that need to be wired together: trait SensorDevice { def isCoffeePresent: Boolean } class PotSensor extends SensorDevice { def isCoffeePresent = true } trait OnOffDevice { def on: Unit def off: Unit } class Heater extends OnOffDevice { def on = println("heater.on") def off = println("heater.off") } class Warmer(s: SensorDevice, o: OnOffDevice) { def trigger = { if (s.isCoffeePresent) o.on else o.off } } and the configuration with Spring Java Config and a sample which makes use of this configuration: import org.springframework.context.annotation.Configuration import org.springframework.context.annotation.Bean @Configuration class WarmerConfig { @Bean def heater(): OnOffDevice = new Heater @Bean def potSensor(): SensorDevice = new PotSensor @Bean def warmer() = new Warmer(potSensor(), heater()) } import org.springframework.context.annotation.AnnotationConfigApplicationContext val ac = new AnnotationConfigApplicationContext(classOf[WarmerConfig]) val warmer = ac.getBean("warmer", classOf[Warmer]) warmer.trigger Taking this further to use Spring-Scala project to specify the dependencies, the configuration and a sample look like this: import org.springframework.context.annotation.Configuration import org.springframework.context.annotation.Bean @Configuration class WarmerConfig { @Bean def heater(): OnOffDevice = new Heater @Bean def potSensor(): SensorDevice = new PotSensor @Bean def warmer() = new Warmer(potSensor(), heater()) } import org.springframework.context.annotation.AnnotationConfigApplicationContext val ac = new AnnotationConfigApplicationContext(classOf[WarmerConfig]) val warmer = ac.getBean("warmer", classOf[Warmer]) warmer.trigger The essence of the Spring Scala project as explained in this wiki is the "bean" method derived from the `FunctionalConfiguration` trait, this method can be called to create a bean, passing in parameters to specify, if required, bean name, alias, scope and a function which returns the instantiated bean. This sample hopefully gives a good appreciation for how simple Spring Java Config is, and how much more simpler Spring-Scala project makes it for Scala based projects.

Earlier I had blogged about using Scala with Spring Boot and how the combination just works. There was one issue with the previous approach though - the only way to run the earlier configuration was to build the project into a jar file and run the jar file. ./gradlew build java -jar build/libs/spring-boot-scala-web-0.1.0.jar Spring boot comes with a gradle based plugin which should have allowed the project to run with a "gradle bootRun" command, this unfortunately gives an error for scala based projects. A good workaround is to use sbt for building and running Spring-boot based projects. The catch though is that with gradle and maven, the versions of the dependencies would have been managed through a parent pom, now these have to be explicitly specified. This is how a sample sbt build file with the dependencies spelled out looks: name := "spring-boot-scala-web" version := "1.0" scalaVersion := "2.10.4" sbtVersion := "0.13.1" seq(webSettings : _*) libraryDependencies ++= Seq( "org.springframework.boot" % "spring-boot-starter-web" % "1.0.2.RELEASE", "org.springframework.boot" % "spring-boot-starter-data-jpa" % "1.0.2.RELEASE", "org.webjars" % "bootstrap" % "3.1.1", "org.webjars" % "jquery" % "2.1.0-2", "org.thymeleaf" % "thymeleaf-spring4" % "2.1.2.RELEASE", "org.hibernate" % "hibernate-validator" % "5.0.2.Final", "nz.net.ultraq.thymeleaf" % "thymeleaf-layout-dialect" % "1.2.1", "org.hsqldb" % "hsqldb" % "2.3.1", "org.springframework.boot" % "spring-boot-starter-tomcat" % "1.0.2.RELEASE" % "provided", "javax.servlet" % "javax.servlet-api" % "3.0.1" % "provided" ) libraryDependencies ++= Seq( "org.apache.tomcat.embed" % "tomcat-embed-core" % "7.0.53" % "container", "org.apache.tomcat.embed" % "tomcat-embed-logging-juli" % "7.0.53" % "container", "org.apache.tomcat.embed" % "tomcat-embed-jasper" % "7.0.53" % "container" ) Here I am also using xsbt-web-plugin which is plugin for building scala web applications. xsbt-web-plugin also comes with commands to start-up tomcat or jetty based containers and run the applications within these containers, however I had difficulty in getting these to work. What worked is the runMain command to start up the Spring-boot main program through sbt: runMain mvctest.SampleWebApplication and xsbt-web-plugin allows the project to be packaged as a war file using the "package" command, this war deploys and runs without any issues in a standalone tomcat container. Here is a github project with these changes: https://github.com/bijukunjummen/spring-boot-scala-web.git

There are multiple ways to get hold of and use an Http session with a Spring based web application. This is a summarization based on an experience with a recent project. Approach 1 Just inject in HttpSession where it is required. @Service public class ShoppingCartService { @Autowired private HttpSession httpSession; ... } Though surprising, since the service above is a singleton, this works well. Spring intelligently injects in a proxy to the actual HttpSession and this proxy knows how to internally delegate to the right session for the request. The catch with handling session this way though is that the object being retrieved and saved back in the session will have to be managed by the user: public void removeFromCart(long productId) { ShoppingCart shoppingCart = getShoppingCartInSession(); shoppingCart.removeItemFromCart(productId); updateCartInSession(shoppingCart); } Approach 2 Accept it as a parameter, this will work only in the web tier though: @Controller public class ShoppingCartController { @RequestMapping("/addToCart") public String addToCart(long productId, HttpSession httpSession) { //do something with the httpSession } } Approach 3 Create a bean and scope it to the session this way: @Component @Scope(proxyMode=ScopedProxyMode.TARGET_CLASS, value="session") public class ShoppingCart implements Serializable{ ... } Spring creates a proxy for a session scoped bean and makes the proxy available to services which inject in this bean. An advantage of using this approach is that any state changes on this bean are handled by Spring, it would take care of retrieving this bean from the session and propagating any changes to the bean back to the session. Further if the bean were to have any Spring lifecycle methods(say @PostConstruct or @PreDestroy annotated methods), they would get called appropriately. Approach 4 Annotating Spring MVC model attributes with @SessionAttribute annotation: @SessionAttributes("shoppingCart") public class OrderFlowController { public String step1(@ModelAttribute("shoppingCart") ShoppingCart shoppingCart) { } public String step2(@ModelAttribute("shoppingCart") ShoppingCart shoppingCart) { } public String step3(@ModelAttribute("shoppingCart") ShoppingCart shoppingCart, SessionStatus status) { status.setComplete(); } } The use case for using SessionAttributes annotation is very specific, to hold state during a flow like above Given these approaches, I personally prefer Approach 3 of using session scoped beans, this way depending on Spring to manage the underlying details of retrieving and storing the object into session. Other approaches have value though based on the scenario that you may be faced with, ranging from requiring more control over raw Http Sessions to needing to handle temporary state like in Approach 4 above.