Restlet is a lightweight, comprehensive, open source REST framework for the Java platform. Restlet is suitable for both server and client Web applications.

A file poller is a very useful mechanism to solve common IT problems. Camel’s built-in file component is extremely flexible, and there are many options available for configuration. Let’s cover few common usages here. Polling a Directory for Input Files Here is a typical Camel Route used to poll a directory for input files every second. import org.slf4j.*; import org.apache.camel.*; import org.apache.camel.builder.*; import java.io.*; public class FileRouteBuilder extends RouteBuilder { static Logger LOG = LoggerFactory.getLogger(FileRouteBuilder.class); public void configure() { from("file://target/input?delay=1000") .process(new Processor() { public void process(Exchange msg) { File file = msg.getIn().getBody(File.class); LOG.info("Processing file: " + file); } }); } } Run this with following: mvn compile exec:java -Dexec.mainClass=org.apache.camel.main.Main -Dexec.args='-r camelcoredemo.FileRouteBuilder' The program will begin to poll your target/input folder under your current directory and wait for incoming files. To test with input files, you would need to open another terminal and then create some files like this: echo 'Hello 1' > target/input/test1.txt echo 'Hello 2' > target/input/test2.txt You should now see the first prompt window start picking up the files and passing them to the next Processor step. In the Processor, we obtain the File object from the message body. It then simply logs its file name. You may hit CTRL+C when you are done. There many configurable options from the file component. You may use this in the URL, but most of the default settings are enough to get you going as the simple case above shows us. Some of these default behaviors are such that if the input folder doesn’t exist, it will create it. And when the file is done processing by the Route, it will be moved into a .camel folder. If you don’t want the file at all after processing, then set delete=true in the URL. Reading in the File Content and Converting to Different Types By default, the file component will create a org.apache.camel.component.file.GenericFile object for each file found and pass it down your Route as message body. You may retrieve all your file information through this object. Alternatively, you may also use the Exchange API to auto convert the message body object to a type you expect to receive (eg: as with msg.getIn().getBody(File.class)). In the example above, the File is a type you expect to get from the message body, and Camel will try to convert it for you. Camel uses the context’s registry space to pre-register many TypeConverter's that can handle the conversion of most of the common data types (like Java primitives). These TypeConverters are a powerful way to make your Route and Processor more flexible and portable. Camel will not only convert just your File object from a message body, but it can also read the file content. If your files are character text based, then you can simply do this. from("file://target/input?charset=UTF-8") .process(new Processor() { public void process(Exchange msg) { String text = msg.getIn().getBody(String.class); LOG.info("Processing text: " + text); } }); That’s it! Simply specify that it is a String type, and Camel will read your file and pass in the entire file text content as a body message. You may even use the charset to change the encoding. If you are dealing with a binary file, then simply try byte[] bytes =msg.getIn().getBody(byte[].class); conversion instead. Pretty cool huh? Polling and Processing Large Files When working with large files, there are a few options in the file component that you might want to use to ensure proper handling. For example, you might want to move the input file into a staging folder before the Route starts the processing; and when it’s done, move it to a .completed folder. from("file://target/input?preMove=staging&move=.completed") .process(new Processor() { public void process(Exchange msg) { File file = msg.getIn().getBody(File.class); LOG.info("Processing file: " + file); } }); To feed input files properly into the polling folder, it’s best if the sender generates the input files in a temporary folder first, and only when it’s ready then move it into the polling folder. This will minimize reading an incomplete file by the Route if the input file might take time to generate. Another solution to this is to configure the file endpoint to only read the polling folder when there is a signal or when a ready-marker file exists. For example: from("file://target/input?preMove=staging&move=.completed&doneFileName=ReadyFile.txt") .process(new Processor() { public void process(Exchange msg) { File file = msg.getIn().getBody(File.class); LOG.info("Processing file: " + file); } }); The code above will only read the target/input folder when a ReadyFile.txt file exists. The marker file can just be an empty file, and it will be removed by Camel after polling. This solution would allow the sender to generate input files in no matter how long it takes. Another concern with large file processing is avoiding loading a file's entire content into memory for processing. To be more practical, you want to split the file into records (eg: per line) and process it one by one (this is called "streaming"). Here is how you would do that using Camel. from("file://target/input?preMove=staging&move=.completed") .split(body().tokenize("\n")) .streaming() .process(new Processor() { public void process(Exchange msg) { String line = msg.getIn().getBody(String.class); LOG.info("Processing line: " + line); } }); This Route will allow you to process large size file without consuming too much memory, and it will process it line-by-line very efficiently. Writing Messages Back into File The file component can also be used to write messages into files. Recall that we may use dataset components to generate sample messages. We will use that to feed the Route and send it to the file component so you can see that each message generated will be saved into a file. package camelcoredemo; import org.slf4j.*; import org.apache.camel.*; import org.apache.camel.builder.*; import org.apache.camel.main.Main; import org.apache.camel.component.dataset.*; public class FileDemoCamel extends Main { static Logger LOG = LoggerFactory.getLogger(FileDemoCamel.class); public static void main(String[] args) throws Exception { FileDemoCamel main = new FileDemoCamel(); main.enableHangupSupport(); main.addRouteBuilder(createRouteBuilder()); main.bind("sampleGenerator", createDataSet()); main.run(args); } static RouteBuilder createRouteBuilder() { return new RouteBuilder() { public void configure() { from("dataset://sampleGenerator") .to("file://target/output"); } }; } static DataSet createDataSet() { return new SimpleDataSet(); } } Compile and run it. mvn compile exec:java -Dexec.mainClass=camelcoredemo.FileDemoCamel Upon completion, you will see that 10 files would be generated in the target/output folder with the file name in ID--- format. There are more options availabe from File component that you may explore. Try it out with a Route and see for yourself.

Camel Timer is a simple and yet useful component. It brings the JDK’s timer functionality into your camel Route with very simple config. from("timer://mytimer?period=1000") .process(new Processor() { public void process(Exchange msg) { LOG.info("Processing {}", msg); } }); That will generate a timer event message every second. You may short hand 1000 with 1s instead. It supports mfor minutes, or h for hours as well. Pretty handy. Another useful timer feature is that it can limit (stop) the number of timer messages after a certain count. You simply need to add repeatCount option toward the url. Couple of properties from the event message would be useful when handling the timer message. Here is an example how to read them. from("timer://mytimer?period=1s&repeatCount=5") .process(new Processor() { public void process(Exchange msg) { java.util.Date fireTime = msg.getProperty(Exchange.TIMER_FIRED_TIME, java.util.Date.class); int eventCount = msg.getProperty(Exchange.TIMER_COUNTER, Integer.class); LOG.info("We received {}th timer event that was fired on {}", eventCount, fireTime); } }); There are more options availabe from Timer component that you may explore. Try it out with a Route and see it for yourself.

in this article i’ll introduce the basic coding that require to create jquery datatable using json passed by simple servlet. datatable is very powerful jquery based grid with advance features which can be build in short span of time with customize features. installation 1. download latest jquery datatable download 2. above download will provide two jquery plugin jquery.js and querytables.js 3. default stylesheet which shipped with latest datatable download package note: you can download full source code from github link creating the datatable we can write below code to create the basic datatable with data feedsummary.jsp ========================== $(document).ready will ready to execute the javascript and var otable = $(‘#tableid’).datatable says that write datatable on tableid place. datatables will adding sorting, filtering, paging and information to your table by default, providing the end user of your web-site with the ability to control the display of the table and find the information that they want from it as quickly as possible. the pointer tableid and column name will be defined in table tag as below feedsummary.jsp ===================== first namelast nameaddress 1address 2 above datatable code invoke feedservlet which will return json string as defined below feedservlet.java =============== protected void dopost(httpservletrequest request, httpservletresponse response) throws servletexception, ioexception { printwriter out = response.getwriter(); string json = "{ \"demo\":[[\"first name\",\"last name\","+ +\"address1\",\"address2\"],[\"first name\",\"last name\",\"address1\",\"address2\"]]}"; out.println(json); } now either we can use servlet annotation or web.xml as below to register above feedservlet web.xml ========= feedservlet feedservlet feedservlet feedservlet /feedservlet running incorporate the above point and deploy with server to view the result as follows: http://localhost:8080/exampledatatablejson/feedsummary.jsp jquery datatable image conclusion you can download full source code from github link and most welcome to fork or update the same. references: http://datatables.net/examples/

This post was originally authored by Rob Winch from SpringSource. This is my last post in a two part series on Spring Security 3.2.0.RC1. My previous post discussed Spring Security's CSRF protection. In this post we will discuss how to use Spring Security to add various response headers to help secure your application. SECURITY HEADERS Many of the new Spring Security features in 3.2.0.RC1 are implemented by adding headers to the response. The foundation for these features came from hard work from Marten Deinum. If the name sounds familiar, it may because one of his 10K+ posts on the Spring Forums has helped you out. If you are using XML configuration, you can add all of the default headers using Spring Security's element with no child elements to add all the default headers to the response: ... If you are using Spring Security's Java configuration, all of the default security headers are added by default. They can be disabled using the Java configuration below: @EnableWebSecurity @Configuration public class WebSecurityConfig extends WebSecurityConfigurerAdapter { @Override protected void configure(HttpSecurity http) throws Exception { http .headers().disable() ...; } } The remainder of this post will discuss each of the default headers in more detail: Cache Control Content Type Options HTTP Strict Transport Security X-Frame-Options X-XSS-PROTECTION Cache Control In the past Spring Security required you to provide your own cache control for your web application. This seemed reasonable at the time, but browser caches have evolved to include caches for secure connections as well. This means that a user may view an authenticated page, log out, and then a malicious user can use the browser history to view the cached page. To help mitigate this Spring Security has added cache control support which will insert the following headers into you response. Cache-Control: no-cache, no-store, max-age=0, must-revalidate Pragma: no-cache Simply adding the element with no child elements will automatically add Cache Control and quite a few other protections. However, if you only want cache control, you can enable this feature using Spring Security's XML namespace with the element. ... Similarly, you can enable only cache control within Java Configuration with the following: @EnableWebSecurity @Configuration public class WebSecurityConfig extends WebSecurityConfigurerAdapter { @Override protected void configure(HttpSecurity http) throws Exception { http .headers() .cacheControl() .and() ...; } } If you actually want to cache specific responses, your application can selectively invokeHttpServletResponse.setHeader(String,String) to override the header set by Spring Security. This is useful to ensure things like CSS, JavaScript, and images are properly cached. When using Spring Web MVC, this is typically done within your configuration. For example, the following configuration will ensure that the cache headers are set for all of your resources: @EnableWebMvc public class WebMvcConfiguration extends WebMvcConfigurerAdapter { @Override public void addResourceHandlers(ResourceHandlerRegistry registry) { registry .addResourceHandler("/resources/**") .addResourceLocations("/resources/") .setCachePeriod(31556926); } // ... } Content Type Options Uploading Files There are many additional things one should do (i.e. only display the document in a distinct domain, ensure Content-Type header is set, sanitize the document, etc) when allowing content to be uploaded. However, these measures are out of the scope of what Spring Security provides. It is also important to point out when disabling content sniffing, you must specify the content type in order for things to work properly. Historically browsers, including Internet Explorer, would try to guess the content type of a request using content sniffing. This allowed browsers to improve the user experience by guessing the content type on resources that had not specified the content type. For example, if a browser encountered a JavaScript file that did not have the content type specified, it would be able to guess the content type and then execute it. The problem with content sniffing is that this allowed malicious users to use polyglots (i.e. a file that is valid as multiple content types) to execute XSS attacks. For example, some sites may allow users to submit a valid postscript document to a website and view it. A malicious user might create a postscript document that is also a valid JavaScript file and execute a XSS attack with it. Content sniffing can be disabled by adding the following header to our response: X-Content-Type-Options: nosniff Just as with the cache control element, the nosniff directive is added by default when using the element with no child elements. However, if you want more control over which headers are added you can use the element as shown below: ... The X-Content-Type-Options header is added by default with Spring Security Java configuration. If you want more control over the headers, you can explicitly specify the content type options with the following: @EnableWebSecurity @Configuration public class WebSecurityConfig extends WebSecurityConfigurerAdapter { @Override protected void configure(HttpSecurity http) throws Exception { http .headers() .contentTypeOptions() .and() ...; } } HTTP Strict Transport Security (HSTS) When you type in your bank's website, do you enter mybank.example.com or do you enter https://mybank.example.com? If you omit the https protocol, you are potentially vulnerable toMan in the Middle attacks. Even if the website performs a redirect to https://mybank.example.com a malicious user could intercept the initial HTTP request and manipulate the response (i.e. redirect to https://mibank.example.com and steal their credentials). Many users omit the https protocol and this is why HTTP Strict Transport Security (HSTS)was created. Once mybank.example.com is added as a HSTS host, a browser can know ahead of time that any request to mybank.example.com should be interpreted as https://mybank.example.com. This greatly reduces the possibility of a Man in the Middle attack occurring. HSTS Notes In accordance with RFC6797, the HSTS header is only injected into HTTPS responses. In order for the browser to acknowledge the header, the browser must first trust the CA that signed the SSL certificate used to make the connection (not just the SSL certificate). One way for a site to be marked as a HSTS host is to have the host preloaded into the browser. Another is to add the "Strict-Transport-Security" header to the response. For example the following would instruct the browser to treat the domain as an HSTS host for a year (there are approximately 31536000 seconds in a year): Strict-Transport-Security: max-age=31536000 ; includeSubDomains The optional includeSubDomains directive instructs Spring Security that subdomains (i.e. secure.mybank.example.com) should also be treated as an HSTS domain. As with the other headers, Spring Security adds the previous header to the response when the element is specified with no child elements. It is also automatically added when you are using Java Configuration. You can also only use HSTS headers with the element as shown below: ... Similarly, you can enable only HSTS headers with Java Configuration: @EnableWebSecurity @Configuration public class WebSecurityConfig extends WebSecurityConfigurerAdapter { @Override protected void configure(HttpSecurity http) throws Exception { http .headers() .hsts() .and() ...; } } X-Frame-Options Content Security Policy Another modern approach to dealing with clickjacking is using a Content Security Policy. Spring Security does not provide support for this as the specification is not released and it is quite a bit more complicated. To stay up to date with this issue and to see how you can implement it with Spring Security refer to SEC-2117 Allowing your website to be added to a frame can be a security issue. For example, using clever CSS styling users could be tricked into clicking on something that they were not intending (video demo). For example, a user that is logged into their bank might click a button that grants access to other users. This sort of attack is known asClickjacking. There are a number ways to mitigate clickjacking attacks. For example, to protect legacy browsers from clickjacking attacks you can use frame breaking code. While not perfect, the frame breaking code is the best you can do for the legacy browsers. A more modern approach to address clickjacking is to use X-Frame-Options header: X-Frame-Options: DENY The X-Frame-Options response header instructs the browser to prevent any site with this header in the response from being rendered within a frame. As with the other response headers, this is automatically included when the element is specified with no child elements. You can also explicitly specify the element to control which headers are added to the response. ... Similarly, you can enable only frame options within Java Configuration with the following: @EnableWebSecurity @Configuration public class WebSecurityConfig extends WebSecurityConfigurerAdapter { @Override protected void configure(HttpSecurity http) throws Exception { http .headers() .frameOptions() .and() ...; } } X-XSS-Protection Some browsers have built in support for filtering out reflected XSS attacks. This is by no means full proof, but does assist in XSS protection. The filtering is typically enabled by default, so adding the header typically just ensures it is enabled and instructs the browser what to do when a XSS attack is detected. For example, the filter might try to change the content in the least invasive way to still render everything. At times, this type of replacement can become a XSS vulnerability in itself. Instead, it is best to block the content rather than attempt to fix it. To do this we can add the following header: X-XSS-Protection: 1; mode=block This header is included by default when the element is specified with no child elements. We can explicitly state it using the element as shown below: ... Similarly, you can enable only xss protection within Java Configuration with the following: @EnableWebSecurity @Configuration public class WebSecurityConfig extends WebSecurityConfigurerAdapter { @Override protected void configure(HttpSecurity http) throws Exception { http .headers() .xssProtection() .and() ...; } } FEEDBACK PLEASE If you encounter a bug, have an idea for improvement, etc please do not hesitate to bring it up! We want to hear your thoughts so we can ensure we get it right before the code is generally available. Trying out new features early is a good and simple way to give back to the community. This also ensures that the features you want are present and working as you think they should. Please log any issues or feature requests to the Spring Security JIRA. After logging a JIRA, we encourage (but do not require) you to submit your changes in a pull request. You can read more about how to do this in the Contributor Guidelines If you have questions on how to do something, please use the Spring Security forums orStack Overflow with the tag spring-security (I will be monitoring them closely). If you have specific comments questions about this blog, feel free to leave a comment. Using the appropriate tools will help make it easier for everyone. CONCLUSION You should have a good understanding of the new features present in Spring Security 3.2.RC1.

In the upcoming release of Apache Camel 2.12 we have introduced an SPI that allows users to plugin different schedulers for schedule-based consumers. The motivation for this feature came from the fact that some Camel components have scheduled consumers. Usually file and FTP consumers. By default, they use the scheduler from the JVM that can schedule based on a fixed period. Now, with the SPI, we allow a different scheduler to be used instead. We have two cron-based schedulers ready out-of-box in the camel-quartz2 and camel-spring components. So pick your favorite, or dive in and build your own scheduler. CRON expression If you want to pickup files during working hours (polling every 10th second) on weekdays, you can easily do this now (notice we use + as a space separator). Example with Spring: ... Example with Quartz: ... To implement a similar solution in older releases of Camel, you would need to use sort in order to use a route policy. There is a cron-based route policy that can be used to setup cron expressions when a route should be started and when it should be stopped. With this new functionality in Camel 2.12, it's even easier to just define the cron expression in the endpoint URI directly. There are more details in the Camel docs: - Polling Consumer - Quarz2 Component Camel-quartz2 is also a new component in the upcoming Apache Camel 2.12 release. In fact, we already have 14 new components. You can take a peek at the work-in-progress release notes to see what is coming down the road.

In my last post, we saw how to use ASP.NET SignalR and AngularJS together using a "Hello World" type of example. Now that we have some idea of how to use both technologies together, we will take a look at a more advanced scenario in order to make the frameworks work better together. I assume that you have already seen the SignalR Stock ticker sample. If not, download it from GitHub or add the NuGet Package to an existing ASP.NET web application. Make sure that you take some time to run the sample at least once on multiple browsers and have a glance at the code before you proceed any further. I hope you had a look at the code on both the server and the client side of the stock ticker sample. We will not make any modification to the server code and the layout of the page, but we will rewrite the JavaScript part using features of AngularJS. Since there is a lot of client-side code to convert, let’s do it in a two-part series: Creating a custom service to communicate with the hub on the server and using the service in a controller (this post) Performing UI changes on the page, like enabling/disabling buttons, scrolling stock values in a list, and adding animation effect to values in a table and list (next post) Make a copy of the StockTicker.html file and give it a name of your choice. Add two JavaScript files, controller.js and factory.js, to the project. We will add script to these files soon. Modify the script reference section of the page to include the following script files: Let’s start implementing the SignalR part inside a custom service. To keep the controller free from doing anything other than providing data to the view and handling view events, we are creating a custom service to handle the hub communication logic. The service is responsible for: Creating objects needed for communication Configuring client functions to proxy and to respond when a market is opened, closed, or reset, or when a stock value is updated Starting a hub connection Getting the current values of stocks and the current market statuses of the stocks once the connection is started Opening, closing or resetting markets on demand Following is the module and the factory that handles the functionality mentioned above: var app = angular.module('app', []); app.value('$', $); app.factory('stockTickerData', ['$', '$rootScope', function ($, $rootScope) { function stockTickerOperations() { //Objects needed for SignalR var connection; var proxy; //To set values to fields in the controller var setMarketState; var setValues; var updateStocks; //This function will be called by controller to set callback functions var setCallbacks = function (setMarketStateCallback, setValuesCallback, updateStocksCallback) { setMarketState = setMarketStateCallback; setValues = setValuesCallback; updateStocks = updateStocksCallback; }; var initializeClient = function () { //Creating connection and proxy objects connection = $.hubConnection(); proxy = connection.createHubProxy('stockTicker'); configureProxyClientFunctions(); start(); }; var configureProxyClientFunctions = function () { proxy.on('marketOpened', function () { //set market state as open $rootScope.$apply(setMarketState(true)); }); proxy.on('marketClosed', function () { //set market state as closed $rootScope.$apply(setMarketState(false)); }); proxy.on('marketReset', function () { //Reset stock values initializeStockMarket(); }); proxy.on('updateStockPrice', function (stock) { $rootScope.$apply(updateStocks(stock)); }); }; var initializeStockMarket = function () { //Getting values of stocks from the hub and setting it to controllers field proxy.invoke('getAllStocks').done(function (data) { $rootScope.$apply(setValues(data)); }).pipe(function () { //Setting market state to field in controller based on the current state proxy.invoke('getMarketState').done(function (state) { if (state == 'Open') $rootScope.$apply(setMarketState(true)); else $rootScope.$apply(setMarketState(false)); }); }); }; var start = function () { //Starting the connection and initializing market connection.start().pipe(function () { initializeStockMarket(); }); }; var openMarket = function () { proxy.invoke('openMarket'); }; var closeMarket = function () { proxy.invoke('closeMarket'); }; var reset = function () { proxy.invoke('reset'); }; return { initializeClient: initializeClient, openMarket: openMarket, closeMarket: closeMarket, reset: reset, setCallbacks: setCallbacks } }; return stockTickerOperations; } ]); We need a controller to start the work. The controller will have the following components: An array to store the current stock values and a Boolean value to store the current market state Setters to assign values to the fields A function to modify the value of an entry in the stocks array Functions to handle open, close and reset operations when the corresponding button is clicked Set callbacks to the service and ask the service to kick off the communication Following is the code in the controller: var StockTickerCtrl = function ($scope, stockTickerData) { $scope.stocks = []; $scope.marketIsOpen = false; $scope.openMarket = function () { ops.openMarket(); } $scope.closeMarket = function () { ops.closeMarket(); } $scope.reset = function () { ops.reset(); } function assignStocks(stocks) { $scope.stocks = stocks; } function replaceStock(stock) { for (var count = 0; count < $scope.stocks.length; count++) { if ($scope.stocks[count].Symbol == stock.Symbol) { $scope.stocks[count] = stock; } } } function setMarketState(isOpen) { $scope.marketIsOpen = isOpen; } var ops = stockTickerData(); ops.setCallbacks(setMarketState, assignStocks, replaceStock); ops.initializeClient(); } The layout of the HTML page will remain unchanged. But we need to change the way data is rendered on the screen. The stock ticker sample uses a poor man’s template technique to render content in the table and in the scrolling list. Since we are using AngularJS, let’s replace it with expressions. Following is the mark-up on the page in Angular’s style: ASP.NET SignalR Stock Ticker Sample Live Stock Table SymbolPriceOpenHighLowChange% loading... {{stock.Symbol} {{stock.Price | number:2} {{stock.DayOpen | number:2} {{stock.DayHigh | number:2} {{stock.DayLow | number:2} {{stock.Change} {{stock.PercentChange} Live Stock Ticker loading... {{stock.Symbol} {{stock.Price | number:2}{{stock.Change} ({{stock.PercentChange}) Open this page on a browser and the original stock ticker page on another browser window and play with the buttons. You will see that both screens have the same data at any given point in time. The only difference would be the state of the buttons, their color and a scrolling list. We will fix them in the next post. Happy coding!

Working Example on Github There's a small, self contained mavenised example project over on Github to accompany this post - check it out here:https://github.com/adrianmilne/jpa-lucene-spring-demo Running the Demo See the README file over on GitHub for details of running the demo. Essentially - it's just running the Unit Tests, with the usual maven build and test results output to the console - example below. This is the result of running the DBUnit test, which inserts Book data into the HSQL database using JPA, and then uses Lucene to query the data, testing that the expected Books are returned (i.e. only those int he SCI-FI category, containing the word 'Space', and ensuring that any with 'Space' in the title appear before those with 'Space' only in the description. The Book Entity Our simple example stores Books. The Book entity class below is a standard JPA Entity with a few additional annotations to identify it to Lucene: @Indexed - this identifies that the class will be added to the Lucene index. You can define a specific index by adding the 'index' attribute to the annotation. We're just choosing the simplest, minimal configuration for this example. In addition to this - you also need to specify which properties on the entity are to be indexed, and how they are to be indexed. For our example we are again going for the default option by just adding an @Field annotation with no extra parameters. We are adding one other annotation to the 'title' field - @Boost - this is just telling Lucene to give more weight to search term matches that appear in this field (than the same term appearing in the description field). This example is purposefully kept minimal in terms of the ins-and-outs of Lucene (I may cover that in a later post) - we're really just concentrating on the integration with JPA and Spring for now. package com.cor.demo.jpa.entity; import javax.persistence.Entity; import javax.persistence.EnumType; import javax.persistence.Enumerated; import javax.persistence.GeneratedValue; import javax.persistence.Id; import javax.persistence.Lob; import org.hibernate.search.annotations.Boost; import org.hibernate.search.annotations.Field; import org.hibernate.search.annotations.Indexed; /** * Book JPA Entity. */ @Entity @Indexed public class Book { @Id @GeneratedValue private Long id; @Field @Boost(value = 1.5f) private String title; @Field @Lob private String description; @Field @Enumerated(EnumType.STRING) private BookCategory category; public Book(){ } public Book(String title, BookCategory category, String description){ this.title = title; this.category = category; this.description = description; } public Long getId() { return id; } public void setId(Long id) { this.id = id; } public String getTitle() { return title; } public void setTitle(String title) { this.title = title; } public BookCategory getCategory() { return category; } public void setCategory(BookCategory category) { this.category = category; } public String getDescription() { return description; } public void setDescription(String description) { this.description = description; } @Override public String toString() { return "Book [id=" + id + ", title=" + title + ", description=" + description + ", category=" + category + "]"; } } The Book Manager The BookManager class acts as a simple service layer for the Book operations - used for adding books and searching books. As you can see, the JPA database resources are autowired in by Spring from the application-context.xml. We are just using an in-memory hsql database in this example. package com.cor.demo.jpa.manager; import java.util.List; import javax.persistence.EntityManager; import javax.persistence.PersistenceContext; import javax.persistence.PersistenceContextType; import javax.persistence.Query; import org.hibernate.search.jpa.FullTextEntityManager; import org.hibernate.search.jpa.Search; import org.hibernate.search.query.dsl.QueryBuilder; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.context.annotation.Scope; import org.springframework.stereotype.Component; import org.springframework.transaction.annotation.Transactional; import com.cor.demo.jpa.entity.Book; import com.cor.demo.jpa.entity.BookCategory; /** * Manager for persisting and searching on Books. Uses JPA and Lucene. */ @Component @Scope(value = "singleton") public class BookManager { /** Logger. */ private static Logger LOG = LoggerFactory.getLogger(BookManager.class); /** JPA Persistence Unit. */ @PersistenceContext(type = PersistenceContextType.EXTENDED, name = "booksPU") private EntityManager em; /** Hibernate Full Text Entity Manager. */ private FullTextEntityManager ftem; /** * Method to manually update the Full Text Index. This is not required if inserting entities * using this Manager as they will automatically be indexed. Useful though if you need to index * data inserted using a different method (e.g. pre-existing data, or test data inserted via * scripts or DbUnit). */ public void updateFullTextIndex() throws Exception { LOG.info("Updating Index"); getFullTextEntityManager().createIndexer().startAndWait(); } /** * Add a Book to the Database. */ @Transactional public Book addBook(Book book) { LOG.info("Adding Book : " + book); em.persist(book); return book; } /** * Delete All Books. */ @SuppressWarnings("unchecked") @Transactional public void deleteAllBooks() { LOG.info("Delete All Books"); Query allBooks = em.createQuery("select b from Book b"); List books = allBooks.getResultList(); // We need to delete individually (rather than a bulk delete) to ensure they are removed // from the Lucene index correctly for (Book b : books) { em.remove(b); } } @SuppressWarnings("unchecked") @Transactional public void listAllBooks() { LOG.info("List All Books"); LOG.info("------------------------------------------"); Query allBooks = em.createQuery("select b from Book b"); List books = allBooks.getResultList(); for (Book b : books) { LOG.info(b.toString()); getFullTextEntityManager().index(b); } } /** * Search for a Book. */ @SuppressWarnings("unchecked") @Transactional public List search(BookCategory category, String searchString) { LOG.info("------------------------------------------"); LOG.info("Searching Books in category '" + category + "' for phrase '" + searchString + "'"); // Create a Query Builder QueryBuilder qb = getFullTextEntityManager().getSearchFactory().buildQueryBuilder().forEntity(Book.class).get(); // Create a Lucene Full Text Query org.apache.lucene.search.Query luceneQuery = qb.bool() .must(qb.keyword().onFields("title", "description").matching(searchString).createQuery()) .must(qb.keyword().onField("category").matching(category).createQuery()).createQuery(); Query fullTextQuery = getFullTextEntityManager().createFullTextQuery(luceneQuery, Book.class); // Run Query and print out results to console List result = (List) fullTextQuery.getResultList(); // Log the Results LOG.info("Found Matching Books :" + result.size()); for (Book b : result) { LOG.info(" - " + b); } return result; } /** * Convenience method to get Full Test Entity Manager. Protected scope to assist mocking in Unit * Tests. * @return Full Text Entity Manager. */ protected FullTextEntityManager getFullTextEntityManager() { if (ftem == null) { ftem = Search.getFullTextEntityManager(em); } return ftem; } /** * Get the JPA Entity Manager (required for the DBUnit Tests). * @return Entity manager */ protected EntityManager getEntityManager() { return em; } /** * Sets the JPA Entity Manager (required to assist with mocking in Unit Test) * @param em EntityManager */ protected void setEntityManager(EntityManager em) { this.em = em; } } application-context.xml This is the Spring configuration file. You can see in the JPA Entity Manager configuration the key for 'hibernate.search.default.indexBase' is added to the jpaPropertyMap to tell Lucene where to create the index. We have also externalised the database login credentials to a properties file (as you may wish to change these for different environments), for example by updating the propertyConfigurer to look for and use a different external properties if it finds one on the file system). classpath:/system.properties Testing Using DBUnit In the project is an example of using DBUnit with Spring to test adding and searching against the database using DBUnit to populate the database with test data, exercise the Book Manager search operations and then clean the database down. This is a great way to test database functionality and can be easily integrated into maven and continuous build environments. Because DBUnit bypasses the standard JPA insertion calls - the data does not get automatically added to the Lucene index. We have a method exposed on the service interface to update the Full Text index 'updateFullTextIndex()' - calling this causes Lucene to update the index with the current data in the database. This can be useful when you are adding search to pre-populated databases to index the existing content. package com.cor.demo.jpa.manager; import java.io.InputStream; import java.util.List; import org.dbunit.DBTestCase; import org.dbunit.database.DatabaseConnection; import org.dbunit.database.IDatabaseConnection; import org.dbunit.dataset.IDataSet; import org.dbunit.dataset.xml.FlatXmlDataSetBuilder; import org.dbunit.operation.DatabaseOperation; import org.hibernate.impl.SessionImpl; import org.junit.After; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import com.cor.demo.jpa.entity.Book; import com.cor.demo.jpa.entity.BookCategory; /** * DBUnit Test - loads data defined in 'test-data-set.xml' into the database to run tests against the * BookManager. More thorough (and ultimately easier in this context) than using mocks. */ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = { "classpath:/application-context.xml" }) public class BookManagerDBUnitTest extends DBTestCase { /** Logger. */ private static Logger LOG = LoggerFactory.getLogger(BookManagerDBUnitTest.class); /** Book Manager Under Test. */ @Autowired private BookManager bookManager; @Before public void setup() throws Exception { DatabaseOperation.CLEAN_INSERT.execute(getDatabaseConnection(), getDataSet()); } @After public void tearDown() { deleteBooks(); } @Override protected IDataSet getDataSet() throws Exception { InputStream inputStream = this.getClass().getClassLoader().getResourceAsStream("test-data-set.xml"); FlatXmlDataSetBuilder builder = new FlatXmlDataSetBuilder(); return builder.build(inputStream); } /** * Get the underlying database connection from the JPA Entity Manager (DBUnit needs this connection). * @return Database Connection * @throws Exception */ private IDatabaseConnection getDatabaseConnection() throws Exception { return new DatabaseConnection(((SessionImpl) (bookManager.getEntityManager().getDelegate())).connection()); } /** * Tests the expected results for searching for 'Space' in SCF-FI books. */ @Test public void testSciFiBookSearch() throws Exception { bookManager.listAllBooks(); bookManager.updateFullTextIndex(); List results = bookManager.search(BookCategory.SCIFI, "Space"); assertEquals("Expected 2 results for SCI FI search for 'Space'", 2, results.size()); assertEquals("Expected 1st result to be '2001: A Space Oddysey'", "2001: A Space Oddysey", results.get(0).getTitle()); assertEquals("Expected 2nd result to be 'Apollo 13'", "Apollo 13", results.get(1).getTitle()); } private void deleteBooks() { LOG.info("Deleting Books...-"); bookManager.deleteAllBooks(); } } The source data for the test is defined in an xml file.

originally written by paul chapman there are two ways to generate output using spring mvc: you can use the restful @responsebody approach and http message converters, typically to return data-formats like json or xml. programmatic clients, mobile apps and ajax enabled browsers are the usual clients. alternatively you may use view resolution . although views are perfectly capable of generating json and xml if you wish (more on that in my next post), views are normally used to generate presentation formats like html for a traditional web-application. actually there is a third possibility – some applications require both, and spring mvc supports such combinations easily. we will come back to that right at the end. in either case you'll need to deal with multiple representations (or views) of the same data returned by the controller. working out which data format to return is called content negotiation . there are three situations where we need to know what type of data-format to send in the http response: httpmessageconverters: determine the right converter to use. request mappings: map an incoming http request to different methods that return different formats. view resolution: pick the right view to use. determining what format the user has requested relies on a contentnegotationstrategy . there are default implementations available out of the box, but you can also implement your own if you wish. in this post i want to discuss how to configure and use content negotiation with spring, mostly in terms of restful controllers using http message converters. in a later post i will show how to setup content negotiation specifically for use with views using spring's contentnegotiatingviewresolver . how does content negotiation work? getting the right content when making a request via http it is possible to specify what type of response you would like by setting the accept header property. web browsers have this preset to request html (among other things). in fact, if you look, you will see that browsers actually send very confusing accept headers, which makes relying on them impractical. see http://www.gethifi.com/blog/browser-rest-http-accept-headers for a nice discussion of this problem. bottom-line: accept headers are messed up and you can't normally change them either (unless you use javascript and ajax). so, for those situations where the accept header property is not desirable, spring offers some conventions to use instead. (this was one of the nice changes in spring 3.2 making a flexible content selection strategy available across all of spring mvc not just when using views). you can configure a content negotiation strategy centrally once and it will apply wherever different formats (media types) need to be determined. enabling content negotiation in spring mvc spring supports a couple of conventions for selecting the format required: url suffixes and/or a url parameter. these work alongside the use of accept headers. as a result, the content-type can be requested in any of three ways. by default they are checked in this order: add a path extension (suffix) in the url. so, if the incoming url is something like http://myserver/myapp/accounts/list.html then html is required. for a spreadsheet the url should be http://myserver/myapp/accounts/list.xls . the suffix to media-type mapping is automatically defined via the javabeans activation framework or jaf (so activation.jar must be on the class path). a url parameter like this: http://myserver/myapp/accounts/list?format=xls . the name of the parameter is format by default, but this may be changed. using a parameter is disabled by default, but when enabled, it is checked second. finally the accept http header property is checked. this is how http is actually defined to work, but, as previously mentioned, it can be problematic to use. the java configuration to set this up, looks like this. simply customize the predefined content negotiation manager via its configurer. note the mediatype helper class has predefined constants for most well-known media-types. @configuration @enablewebmvc public class webconfig extends webmvcconfigureradapter { /** * setup a simple strategy: use all the defaults and return xml by default when not sure. */ @override public void configurecontentnegotiation(contentnegotiationconfigurer configurer) { configurer.defaultcontenttype(mediatype.application_xml); } } when using xml configuration, the content negotiation strategy is most easily setup via the contentnegotiationmanagerfactorybean : the contentnegotiationmanager created by either setup is an implementation of contentnegotationstrategy that implements the ppa strategy (path extension, then parameter, then accept header) described above. additional configuration options in java configuration, the strategy can be fully customized using methods on the configurer: @configuration @enablewebmvc public class webconfig extends webmvcconfigureradapter { /** * total customization - see below for explanation. */ @override public void configurecontentnegotiation(contentnegotiationconfigurer configurer) { configurer.favorpathextension(false). favorparameter(true). parametername("mediatype"). ignoreacceptheader(true). usejaf(false). defaultcontenttype(mediatype.application_json). mediatype("xml", mediatype.application_xml). mediatype("json", mediatype.application_json); } } in xml, the strategy can be configured using methods on the factory bean: what we did, in both cases: disabled path extension. note that favor does not mean use one approach in preference to another, it just enables or disables it. the order of checking is always path extension, parameter, accept header. enable the use of the url parameter but instead of using the default parameter, format , we will use mediatype instead. ignore the accept header completely. this is often the best approach if most of your clients are actually web-browsers (typically making rest calls via ajax). don't use the jaf, instead specify the media type mappings manually – we only wish to support json and xml. listing user accounts example to demonstrate, i have put together a simple account listing application as our worked example – the screenshot shows a typical list of accounts in html. the complete code can be found at github: https://github.com/paulc4/mvc-content-neg . to return a list of accounts in json or xml, i need a controller like this. we will ignore the html generating methods for now. @controller class accountcontroller { @requestmapping(value="/accounts", method=requestmethod.get) @responsestatus(httpstatus.ok) public @responsebody list list(model model, principal principal) { return accountmanager.getaccounts(principal) ); } // other methods ... } here is the content-negotiation strategy setup: or, using java configuration, the code looks like this: @override public void configurecontentnegotiation( contentnegotiationconfigurer configurer) { // simple strategy: only path extension is taken into account configurer.favorpathextension(true). ignoreacceptheader(true). usejaf(false). defaultcontenttype(mediatype.text_html). mediatype("html", mediatype.text_html). mediatype("xml", mediatype.application_xml). mediatype("json", mediatype.application_json); } provided i have jaxb2 and jackson on my classpath, spring mvc will automatically setup the necessary httpmessageconverters . my domain classes must also be marked up with jaxb2 and jackson annotations to enable conversion (otherwise the message converters don't know what to do). in response to comments (below), the annotated account class is shown below . here is the json output from our accounts application (note path-extension in url). how does the system know whether to convert to xml or json? because of content negotiation – any one of the three ( ppa strategy ) options discussed above will be used depending on how the contentnegotiationmanager is configured. in this case the url ends in accounts.json because the path-extension is the only strategy enabled. in the sample code you can switch between xml or java configuration of mvc by setting an active profile in the web.xml . the profiles are "xml" and "javaconfig" respectively. combining data and presentation formats spring mvc's rest support builds on the existing mvc controller framework. so it is possible to have the same web-applications return information both as raw data (like json) and using a presentation format (like html). both techniques can easily be used side by side in the same controller, like this: @controller class accountcontroller { // restful method @requestmapping(value="/accounts", produces={"application/xml", "application/json"}) @responsestatus(httpstatus.ok) public @responsebody list listwithmarshalling(principal principal) { return accountmanager.getaccounts(principal); } // view-based method @requestmapping("/accounts") public string listwithview(model model, principal principal) { // call restful method to avoid repeating account lookup logic model.addattribute( listwithmarshalling(principal) ); // return the view to use for rendering the response return ¨accounts/list¨; } } there is a simple pattern here: the @responsebody method handles all data access and integration with the underlying service layer (the accountmanager ). the second method calls the first and sets up the response in the model for use by a view. this avoids duplicated logic. to determine which of the two @requestmapping methods to pick, we are again using our ppa content negotiation strategy. it allows the produces option to work. urls ending with accounts.xml or accounts.json map to the first method, any other urls ending in accounts.anything map to the second. another approach alternatively we could do the whole thing with just one method if we used views to generate all possible content-types. this is where the contentnegotiatingviewresolver comes in and that will be the subject of my next post . acknoweldgements i would like to thank rossen stoyanchev for his help in writing this post. any errors are my own. addendum: the annotated account class added 2 june 2013 . since there were some questions on how to annotate a class for jaxb, here is part of the account class. for brevity i have omitted the data-members, and all methods except the annotated getters. i could annotate the data-members directly if preferred (just like jpa annotations in fact). remember that jackson can marshal objects to json using these same annotations. /** * represents an account for a member of a financial institution. an account has * zero or more {@link transaction}s and belongs to a {@link customer}. an aggregate entity. */ @entity @table(name = "t_account") @xmlrootelement public class account { // data-members omitted ... public account(customer owner, string number, string type) { this.owner = owner; this.number = number; this.type = type; } /** * returns the number used to uniquely identify this account. */ @xmlattribute public string getnumber() { return number; } /** * get the account type. * * @return one of "credit", "savings", "check". */ @xmlattribute public string gettype() { return type; } /** * get the credit-card, if any, associated with this account. * * @return the credit-card number or null if there isn't one. */ @xmlattribute public string getcreditcardnumber() { return stringutils.hastext(creditcardnumber) ? creditcardnumber : null; } /** * get the balance of this account in local currency. * * @return current account balance. */ @xmlattribute public monetaryamount getbalance() { return balance; } /** * returns a single account transaction. callers should not attempt to hold * on or modify the returned object. this method should only be used * transitively; for example, called to facilitate reporting or testing. * * @param name * the name of the transaction account e.g "fred smith" * @return the beneficiary object */ @xmlelement // make these a nested element public set gettransactions() { return transactions; } // setters and other methods ... }

When you have a piece of code that often fails and must be retried, this Java 7/8 library provides rich and unobtrusive API with fast and scalable solution to this problem: ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor(); RetryExecutor executor = new AsyncRetryExecutor(scheduler). retryOn(SocketException.class). withExponentialBackoff(500, 2). //500ms times 2 after each retry withMaxDelay(10_000). //10 seconds withUniformJitter(). //add between +/- 100 ms randomly withMaxRetries(20); You can now run arbitrary block of code and the library will retry it for you in case it throws SocketException: final CompletableFuture future = executor.getWithRetry(() -> new Socket("localhost", 8080) ); future.thenAccept(socket -> System.out.println("Connected! " + socket) ); Please look carefully! getWithRetry() does not block. It returns CompletableFuture immediately and invokes given function asynchronously. You can listen for that Future or even for multiple futures at once and do other work in the meantime. So what this code does is: trying to connect to localhost:8080 and if it fails with SocketException it will retry after 500 milliseconds (with some random jitter), doubling delay after each retry, but not above 10 seconds. Equivalent but more concise syntax: executor. getWithRetry(() -> new Socket("localhost", 8080)). thenAccept(socket -> System.out.println("Connected! " + socket)); This is a sample output that you might expect: TRACE | Retry 0 failed after 3ms, scheduled next retry in 508ms (Sun Jul 21 21:01:12 CEST 2013) java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0-ea] //... TRACE | Retry 1 failed after 0ms, scheduled next retry in 934ms (Sun Jul 21 21:01:13 CEST 2013) java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0-ea] //... TRACE | Retry 2 failed after 0ms, scheduled next retry in 1919ms (Sun Jul 21 21:01:15 CEST 2013) java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0-ea] //... TRACE | Successful after 2 retries, took 0ms and returned: Socket[addr=localhost/127.0.0.1,port=8080,localport=46332] Connected! Socket[addr=localhost/127.0.0.1,port=8080,localport=46332] Imagine you connect to two different systems, one is slow, second unreliable and fails often: CompletableFuture stringFuture = executor.getWithRetry(ctx -> unreliable()); CompletableFuture intFuture = executor.getWithRetry(ctx -> slow()); stringFuture.thenAcceptBoth(intFuture, (String s, Integer i) -> { //both done after some retries }); thenAcceptBoth() callback is executed asynchronously when both slow and unreliable systems finally reply without any failure. Similarly (using CompletableFuture.acceptEither()) you can call two or more unreliable servers asynchronously at the same time and be notified when the first one succeeds after some number of retries. I can’t emphasize this enough - retries are executed asynchronously and effectively use thread pool, rather than sleeping blindly. Rationale Often we are forced to retry given piece of code because it failed and we must try again, typically with a small delay to spare CPU. This requirement is quite common and there are few ready-made generic implementations with retry support in Spring Batch through RetryTemplate class being best known. But there are few other, quite similar approaches ([1], [2]). All of these attempts (and I bet many of you implemented similar tool yourself!) suffer the same issue - they are blocking, thus wasting a lot of resources and not scaling well. This is not bad per se because it makes programming model much simpler - the library takes care of retrying and you simply have to wait for return value longer than usual. But not only it creates leaky abstraction (method that is typically very fast suddenly becomes slow due to retries and delay), but also wastes valuable threads since such facility will spend most of the time sleeping between retries. Therefore Async-Retry utility was created, targeting Java 8 (with Java 7 backport existing) and addressing issues above. The main abstraction is RetryExecutor that provides simple API: public interface RetryExecutor { CompletableFuture doWithRetry(RetryRunnable action); CompletableFuture getWithRetry(Callable task); CompletableFuture getWithRetry(RetryCallable task); CompletableFuture getFutureWithRetry(RetryCallable> task); } Don’t worry about RetryRunnable and RetryCallable - they allow checked exceptions for your convenience and most of the time we will use lambda expressions anyway. Please note that it returns CompletableFuture. We no longer pretend that calling faulty method is fast. If the library encounters an exception it will retry our block of code with preconfigured backoff delays. The invocation time will sky-rocket from milliseconds to several seconds. CompletableFuture clearly indicates that. Moreover it’s not a dumb java.util.concurrent.Future we all know - CompletableFuture in Java 8 is very powerful and most importantly - non-blocking by default. If you need blocking result after all, just call .get() on Future object. Basic API The API is very simple. You provide a block of code and the library will run it multiple times until it returns normally rather than throwing an exception. It may also put configurable delays between retries: RetryExecutor executor = //... executor.getWithRetry(() -> new Socket("localhost", 8080)); Returned CompletableFuture will be resolved once connecting to localhost:8080 succeeds. Optionally we can consume RetryContext to get extra context like which retry is currently being executed: executor. getWithRetry(ctx -> new Socket("localhost", 8080 + ctx.getRetryCount())). thenAccept(System.out::println); This code is more clever than it looks. During first execution ctx.getRetryCount() returns 0, therefore we try to connect to localhost:8080. If this fails, next retry will try localhost:8081 (8080 + 1) and so on. And if you realize that all of this happens asynchronously you can scan ports of several machines and be notified about first responding port on each host: Arrays.asList("host-one", "host-two", "host-three"). stream(). forEach(host -> executor. getWithRetry(ctx -> new Socket(host, 8080 + ctx.getRetryCount())). thenAccept(System.out::println) ); For each host RetryExecutor will attempt to connect to port 8080 and retry with higher ports. getFutureWithRetry() requires special attention. I you want to retry method that already returns CompletableFuture: e.g. result of asynchronous HTTP call: private CompletableFuture asyncHttp(URL url) { /*...*/} //... final CompletableFuture> response = executor.getWithRetry(ctx -> asyncHttp(new URL("http://example.com"))); Passing asyncHttp() to getWithRetry() will yield CompletableFuture>. Not only it’s awkward to work with, but also broken. The library will barely call asyncHttp() and retry only if it fails, but not if returned CompletableFuture fails. The solution is simple: final CompletableFuture response = executor.getFutureWithRetry(ctx -> asyncHttp(new URL("http://example.com"))); In this case RetryExecutor will understand that whatever was returned from asyncHttp() is the actually just a Future and will (asynchronously) wait for result or failure. This library is much more powerful, so let’s dive into: Configuration Options In general there are two important factors you can configure: RetryPolicy that controls whether next retry attempt should be made and Backoff - that optionally adds delay between subsequent retry attempts. By default RetryExecutor repeats user task infinitely on every Throwable and adds 1 second delay between retry attempts. Creating an Instance of RetryExecutor Default implementation of RetryExecutor is AsyncRetryExecutor which you can create directly: ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor(); RetryExecutor executor = new AsyncRetryExecutor(scheduler); //... scheduler.shutdownNow(); The only required dependency is standard ScheduledExecutorService from JDK. One thread is enough in many cases but if you want to concurrently handle retries of hundreds or more tasks, consider increasing the pool size. Notice that the AsyncRetryExecutor does not take care of shutting down the ScheduledExecutorService. This is a conscious design decision which will be explained later. AsyncRetryExecutor has few other constructors but most of the time altering the behaviour of this class is most convenient with calling chained with*() methods. You will see plenty of examples written this way. Later on we will simply use executor reference without defining it. Assume it’s of RetryExecutor type. Retrying Policy Exceptions By default every Throwable (except special AbortRetryException) thrown from user task causes retry. Obviously this is configurable. For example in JPA you may want to retry a transaction that failed due to OptimisticLockException - but every other exception should fail immediately: executor. retryOn(OptimisticLockException.class). withNoDelay(). getWithRetry(ctx -> dao.optimistic()); Where dao.optimistic() may throw OptimisticLockException. In that case you probably don’t want any delay between retries, more on that later. If you don’t like the default of retrying on every Throwable, just restrict that using retryOn(): executor.retryOn(Exception.class) Of course the opposite might also be desired - to abort retrying and fail immediately in case of certain exception being thrown rather than retrying. It’s that simple: executor. abortOn(NullPointerException.class). abortOn(IllegalArgumentException.class). getWithRetry(ctx -> dao.optimistic()); Clearly you don’t want to retry NullPointerException or IllegalArgumentException as they indicate programming bug rather than transient failure. And finally you can combine both retry and abort policies. User code will retry in case of any retryOn() exception (or subclass) unless it should abortOn() specified exception. For example we want to retry every IOException or SQLException but abort if FileNotFoundException or java.sql.DataTruncation is encountered (order is irrelevant): executor. retryOn(IOException.class). abortIf(FileNotFoundException.class). retryOn(SQLException.class). abortIf(DataTruncation.class). getWithRetry(ctx -> dao.load(42)); If this is not enough you can provide custom predicate that will be invoked on each failure: executor. abortIf(throwable -> throwable instanceof SQLException && throwable.getMessage().contains("ORA-00911") ); Max Number of Retries Another way of interrupting retrying “loop” (remember that this process is asynchronous, there is no blocking loop) is by specifying maximum number of retries: executor.withMaxRetries(5) In rare cases you may want to disable retries and barely take advantage from asynchronous execution. In that case try: executor.dontRetry() Delays Between Retries (backoff) Retrying immediately after failure is sometimes desired (see OptimisticLockException example) but in most cases it’s a bad idea. If you can’t connect to external system, waiting a little bit before next attempt sounds reasonably. You save CPU, bandwidth and other server’s resources. But there are quite a few options to consider: should we retry with constant intervals or increase delay after each failure? should there be a lower and upper limit on waiting time? should we add random “jitter” to delay times to spread retries of many tasks in time? This library answers all these questions. Fixed Interval Between Retries By default each retry is preceded by 1 second waiting time. So if initial attempt fails, first retry will be executed after 1 second. Of course we can change that default, e.g. to 200 milliseconds: executor.withFixedBackoff(200) If we are already here, by default backoff is applied after executing user task. If user task itself consumes some time, retries will be less frequent. For example with retry delay of 200ms and average time it takes before user task fails at about 50ms RetryExecutor will retry about 4 times per second (50ms + 200ms). However if you want to keep retry frequency at more predictable level you can use fixedRate flag: executor. withFixedBackoff(200). withFixedRate() This is similar to “fixed rate” vs. “fixed delay” approaches in ScheduledExecutorService. BTW don’t expect RetryExecutor to be very precise, it does it’s best but it heavily depends on aforementioned ScheduledExecutorService accuracy. Exponentially Growing Intervals Between Retries It’s probably an active research subject, but in general you may wish to expand retry delay over time, assuming that if the user task fails several times we should try less frequently. For example let’s say we start with 100ms delay until first retry attempt is made but if that one fails as well, we should wait two times more (200ms). And later 400ms, 800ms… You get the idea: executor.withExponentialBackoff(100, 2) This is an exponential function that can grow very fast. Thus it’s useful to set maximum backoff time at some reasonable level, e.g. 10 seconds: executor. withExponentialBackoff(100, 2). withMaxDelay(10_000) //10 seconds Random Jitter One phenomena often observed during major outages is that systems tend to synchronize. Imagine a busy system that suddenly stops responding. Hundreds or thousands of requests fail and are retried. It depends on your backoff but by default all these requests will retry exactly after one second producing huge wave of traffic at one point in time. Finally such failures are propagated to other systems that, in turn, synchronize as well. To avoid this problem it’s useful to spread retries over time, flattening the load. A simple solution is to add random jitter to delay time so that not all request are scheduled for retry at the exact same time. You have choice between uniform jitter (random value from -100ms to 100ms): executor.withUniformJitter(100) //ms …and proportional jitter, multiplying delay time by random factor, by default between 0.9 and 1.1 (10%): executor.withProportionalJitter(0.1) //10% You may also put hard lower limit on delay time to avoid to short retry times being scheduled: executor.withMinDelay(50) //ms Implementation Details This library was built with Java 8 in mind to take advantage of lambdas and new CompletableFuture abstraction (but Java 7 port with Guava dependency exists). It uses ScheduledExecutorService underneath to run tasks and schedule retries in the future - which allows best thread utilization. But what is really interesting is that the whole library is fully immutable, there is no single mutable field, at all. This might be counter-intuitive at first, take for example this trivial code sample: ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor(); AsyncRetryExecutor first = new AsyncRetryExecutor(scheduler). retryOn(Exception.class). withExponentialBackoff(500, 2); AsyncRetryExecutor second = first.abortOn(FileNotFoundException.class); AsyncRetryExecutor third = second.withMaxRetries(10); It might seem that all with*() methods or retryOn()/abortOn() mutate existing executor. But that’s not the case, each configuration change creates new instance, leaving the old one untouched. So for example while first executor will retry on FileNotFoundException, the second and third won’t. However they all share the same scheduler. This is the reason why AsyncRetryExecutor does not shut down ScheduledExecutorService (it doesn’t even have any close() method). Since we have no idea how many copies of AsyncRetryExecutor exist pointing to the same scheduler, we don’t even try to manage its lifecycle. However this is typically not a problem (see Spring integration below). You might be wondering, why such an awkward design decision? There are three reasons: when writing a concurrent code immutability can greatly reduce risk of multi-threading bugs. For example RetryContext holds number of retries. But instead of mutating it we simply create new instance (copy) with incremented but final counter. No race condition or visibility can ever occur. if you are given an existing RetryExecutor which is almost exactly what you want but you need one minor tweak, you simply call executor.with...() and get a fresh copy. You don’t have to worry about other places where the same executor was used (see: Spring integration for further examples) functional programming and immutable data structures are sexy these days ;-). N.B.: AsyncRetryExecutor is not marked final, does you can break immutability by subclassing it and adding mutable state. Please don’t do this, subclassing is only permitted to alter behaviour. Dependencies This library requires Java 8 and SLF4J for logging. Java 7 port additionally depends on Guava. Spring Integration If you are just about to use RetryExecutor in Spring - feel free, but the configuration API might not work for you. Spring promotes (or used to promote) the convention of mutable services with plenty of setters. In XML you define bean and invoke setters (via ) on it. This convention assumes the existence of mutating setters. But I found this approach error-prone and counter-intuitive under some circumstances. Let’s say we globally defined org.springframework.transaction.support.TransactionTemplate bean and injected it in multiple places. Great. Now there is this one single request that requires slightly different timeout: @Autowired private TransactionTemplate template; and later in the same class: final int oldTimeout = template.getTimeout(); template.setTimeout(10_000); //do the work template.setTimeout(oldTimeout); This code is wrong on so many levels! First of all if something fails we never restore oldTimeout. OK, finally to the rescue. But also notice how we changed global, shared TransactionTemplate instance. Who knows how many other beans and threads are just about to use it, unaware of changed configuration? And even if you do want to globally change the transaction timeout, fair enough, but it’s still wrong way to do this. private timeout field is not volatile and thus changes made to it may or may not be visible to other threads. What a mess! The same problem appears with many other classes like JmsTemplate. You see where I’m going? Just create one, immutable service class and safely adjust it by creating copies whenever you need it. And using such services is equally simple these days: @Configuration class Beans { @Bean public RetryExecutor retryExecutor() { return new AsyncRetryExecutor(scheduler()). retryOn(SocketException.class). withExponentialBackoff(500, 2); } @Bean(destroyMethod = "shutdownNow") public ScheduledExecutorService scheduler() { return Executors.newSingleThreadScheduledExecutor(); } } Hey! It’s 21st century, we don’t need XML in Spring any more. Bootstrap is simple as well: final ApplicationContext context = new AnnotationConfigApplicationContext(Beans.class); final RetryExecutor executor = context.getBean(RetryExecutor.class); //... context.close(); As you can see integrating modern, immutable services with Spring is just as simple. BTW if you are not prepared for such a big change when designing your own services, at least consider constructor injection. Maturity This library is covered with a strong battery of unit tests. However it wasn’t yet used in any production code and the API is subject to change. Of course you are encouraged to submit bugs, feature requests and pull requests. It was developed with Java 8 in mind but Java 7 backport exists with slightly more verbose API and mandatory Guava dependency (ListenableFuture instead of CompletableFuture from Java 8). Full source code on GitHub.

A question every customer asks me is: How can you manage and monitor integration routes implemented with Apache Camel (http://camel.apache.org/) and / or Talend ESB (http://en.talend.com/products/esb) (which is based on Apache Camel and also available as open source version)? This blog post will show different alternatives to answer this question. The good news first: As Apache Camel and Talend ESB are based on open standards, you can use your own frameworks and tools if tooling of the product is not sufficient. So, I will not talk just about features of Apache Camel or Talend ESB, but also about additional options. Except for Talend Administration Center (user interface on top of Talend ESB’s service monitoring and clustering features), all tools are open source. Talend Administration Center is only available in Talend’s enterprise editions. Management of Apache Camel and Talend ESB Let’s begin with management of Apache Camel and Talend ESB routes. I define management as “controlling server instances, services and integration routes”. Besides Talend Administration Center, a Talend-specific web application, several other options exist. I will show jconsole and hawtio. However, as Apache Camel and Talend ESB are standards-based, you can use any other management tool, which supports the Java Management Extensions (JMX) standard, e.g. Hyperic, Nagios or IBM Tivoli. Talend Administration Center Talend Administration Center (http://www.talendforge.org/tutorials/tutorial.php?idTuto=54) is a web application to configure and monitor different server instances. Besides, you can install, start and stop Camel routes and SOAP / REST web services. Provisioning (i.e. deploying and managing routes and services on different server instances) can be automated easily because Talend ESB container supports Apache Karaf Cellar, an OSGi addon which is built exactly for this reason. Therefore, you can create so-called “virtual servers” within Talend Administration Center. Service locator is another feature, which is based on Apache Zookeeper to implement load balancing and high availability under the hood. Administrators can monitor the state of server instances, Camel routes and REST / SOAP web services within Talend Administration Center. Since Talend ESB 5.3, Talend Administration Center contains other important enterprise features, especially management of Talend’s new service registry (based on Apache Jackrabbit) for service discovery and policy enforcement, and its new identity management (based on Apache Syncope) for authorization. Let’s now talk about two other alternatives for managing Camel and Talend ESB via JMX standard interface. jconsole jconsole is a graphical monitoring tool to monitor Java Virtual Machine (JVM) and java applications both on a local or remote machine. jconsole uses underlying features of JVM to provide information on performance and resource consumption of applications running on the Java platform using JMX technology. jconsole comes as part of Java Development Kit (JDK) and the graphical console can be started using “jconsole” command. Apache Camel and Talend ESB offer several JMX interfaces to monitor and manage integration routes, endpoints, error handlers, etc. jconsole has a very simple user interface, but is sufficient for some projects. hawtio hawtio (http://hawt.io) is a highly modular single-page JavaScript application that fetches JMX metric data using Jolokia, a JMX-HTTP bridge. hawtio is a relatively new tool “on the market”. Nevertheless, it already contains plugins for several different technologies and frameworks such as Apache Camel (integration framework), Apache Karaf (OSGi container), Apache ActiveMQ (JMS messaging) and Apache Tomcat (Java EE web container). As Talend ESB uses respectively supports all of these technologies and frameworks, hawtio is a perfect match, too! hawtio is not “just another jconsole”, but offers some very cool additional features, for example a graphical presentation of Camel routes including real-time monitoring. hawtio was created by James Strachan (whom else?!) who is the creator of Apache Camel and some other great frameworks. Therefore, at the moment, most contributions come from JBoss (where James works after JBoss’ acquisition of FuseSource last year). In the future, I hope (and expect) to see other companies supporting this great open source project, too. Monitoring of Apache Camel and Talend ESB Let’s continue with some monitoring alternatives for Apache Camel and Talend ESB. I define monitoring as “searching and (visually) analyzing state of integration routes and services at runtime”. I will show you Talend’s Service Activity Monitoring for SOAP and REST web services. Besides, you will learn about logstash and Kibana, two awesome open source tools (based on ElasticSearch) for analysis and visualization of any (distributed) log files. Service Activity Monitoring (Part of Talend Administration Center) Service Activity Monitoring (SAM, http://www.liquid-reality.de/display/liquid/Talend+ESB+Service+Activity+Monitoring) is part of Talend Administration Center and allows for logging and monitoring SOAP and REST web service calls. For example, SAM could be used for collecting usage statistics and fault monitoring. SAM is a part of Talend’s OSGi container and can be installed via a single command. It can be activated for SOAP and REST web service providers and consumers within Talend Studio via a single click. So, without any additional efforts, you can monitor your SOAP and REST web services. You can sort by several different parameters and step into details within Talend Administration Center, i.e. you can look at every incoming request message and outgoing response message. logstash logstash (http://logstash.net) is a tool for managing events and logs. You can use it to collect different (distributed) logs, parse them, and store them for later use. Speaking of searching, logstash comes with a simple, but fine web interface for searching and drilling into all of your logs. It uses ElasticSearch under the hood. So, you can easily query through all your logs for specific errors or business analytics (e.g. searching for all lines matching an unique order id). I never used logstash before. I really love it now! However, documentation is not perfect for getting started. So, two hints for newbies: 1) logstash uses ElasticSearch under the hood. However, you do not have to install it separately. It is included and can be used as embedded server. 2) I thought logstash can process every log file by default. However, your log files have to contain a specific structure or you have to configure logstash to know your custom log structure. Especially, if logstash does not know the correct structure of your timestamp, most features will not work. This was not obvious to me. It took me some time to find out that nothing is happening due to wrong log structures (no errors or exceptions came up, just nothing happened at all). Apache Camel and Talend ESB do not offer support for logstash implicitly. One solution is to use Enterprise Integration Patterns (http://www.eaipatterns.com) such as Wire Tap or Message Store to implement logging. Afterwards, logstash can access and analyse these logs. logstash is really awesome, but its real power can be reached by combining it with Kibana. So let’s go to the next section. Kibana Kibana (http://three.kibana.org ) is a browser based analytics and search interface to logstash and other timestamped data sets stored in ElasticSearch. Kibana strives to be easy to get started with, while also being flexible and powerful, just like logstash and ElasticSearch. Main difference to logstash is a much more powerful HTML5 based web interface. You can use multiple concurrent search inputs highlight to drill down bar charts create line charts, stacked, unstacked, filled or unfilled, with or without points create Pie and donut charts that compare top terms or the results of multiple queries create custom dashboards with multiple charts and much more… Again, some hints for newbies (as documentation for installation and usage is very, very short without many important details, unfortunately): 1) Kibana uses logstash and ElasticSearch under the hood. Both are not included, but must be installed separately. As logstash contains an embedded ElasticSearch server, you can get started quickly by just installing logstash in advance. 2) Kibana installation is tricky! If you google, go to Kibana website and navigate to its installation guide (http://kibana.org/intro.html), you will have to install it via Ruby. As always on my Mac, I had some trouble installing Ruby software (version and compiler problems, etc.). But finally I made it… Just to find out that this is an old version. Accidentally, I came to www.three.kibana.org/intro.html, which is the real up-to-date version. Kibana 3 !!! Here, you do NOT need to install Kibana via Ruby. Just download the web application and deploy it to your web container (e.g. Jetty, Tomcat) or Java EE container (e.g. Glassfish, WebSphere). This is so simple and just works with no further configuration. You can use Kibana to analyse and monitor your data as you do with logstash, i.e. you still have to use Enterprise Integration Patterns to log your data first. Afterwards, you can use Kibana’s web interface, which is much more powerful and comfortable than logstash. Finally, great news for Talend users: Kibana (including ElasticSearch) will be integrated into Talend 5.4 for event logging, complex event processing and visual monitoring. I am really looking forward to this awesome feature! Summary Your mission-critical projects need management and monitoring. Today, this is not just possible with complex and expensive tools of large vendors, but also with lightweight frameworks and products such as Apache Camel or Talend ESB. Management and monitoring of Apache Camel and Talend ESB integration routes and SOAP / REST web services is very easy. Several great alternatives are available. Once again, vendor-independent standards such as JXM show their benefits. You can use tooling which is available implicitly in the framework respectively product, or you can integrate your existing tooling very easily. Content from my blog (www.kai-waehner.de/blog). Feedback and other experiences or alternatives appreciated via comments or directly via @KaiWaehner / [email protected] / LinkedIn / Xing. Best regards, Kai Wähner

As you know, there are many frameworks for mobile app development and a growing number of them are based on HTML5. These next-generation tools help developers create mobile apps for phones and tablets without the steep learning curve associated with native SDKs and other programming languages like Objective-C or Java. For countless developers throughout the world, HTML5 represents the future for cross-platform mobile app development. But the question is why? Why has HTML5 become so popular? The widespread adoption of HTML5 involves the emergence of the bring your own device (BYOD) movement. BYOD means that developers can no longer limit application usage to a single platform because consumers want their apps to run on the devices they use every day. HTML5 allows developers to target multiple devices with a single codebase and deliver experiences that closely emulate native solutions, without writing the same application multiple times, using multiple languages or SDKs. The evolution of modern web browsers means that HTML5 can deliver cross-platform, multi-device solutions that mirror behaviors and experiences of “native” apps to the point that it’s often difficult to distinguish between an app written using native development tools and those using HTML. Multiple platform support, time to market and lower maintenance costs are just a few of the advantages inherent in HTML/JavaScript. It’s advantages don’t stop there. HTML’s ability to mitigate long-term risks associated with emerging technologies such as WinRT, ChromeOS, FirefoxOS, and Tizen is unmatched. Simply said, the only code that will work on all these platforms is HTML/JavaScript. Is there a price? Yes, certainly native app consumes less memory and will have faster or more responsive user experience. But in all cases where a web app would work, you can make a step further and create a mobile web app or even packaged application for the store, for multiple platforms, from single codebase. PhoneJS lets you get started fast. PhoneJS for Your Next Mobile Project PhoneJS is a cross-platform HTML5 mobile app development framework that was built to be versatile, flexible and efficient. PhoneJS is a single page application (SPA) framework, with view management and URL routing. Its layout engine allows you to abstract navigation away from views, so the same app can be rendered differently across different platforms or form factors. PhoneJS includes a rich collection of touch-optimized UI widgets with built-in styles for today’s most popular mobile platforms including iOS, Android, and Windows Phone 8. To better understand the principles of PhoneJS development and how you can create and publish applications in platform stores, let’s take a look at a simple demo app called TipCalculator. This application calculates tip amounts due based on a restaurant bill. The complete source code for this app is available here. The app can be found in the AppStore, Google Play, and Windows Store. PhoneJS Application Layout and Routing TipCalculator is a Single-Page Application (SPA) built with HTML5. The start page is index.html, with standard meta tags and links to CSS and JavaScript resources. It includes a script reference to the JavaScript file index.js, where you’ll find the code that configures PhoneJS app framework logic: TipCalculator.app = new DevExpress.framework.html.HtmlApplication({ namespace: TipCalculator, defaultLayout: "empty" }); Within this section, we must specify the default layout for the app. In this example, we’ll use the simplest option, an empty layout. More advanced layouts are available with full support for interactive navigation styles described in the following images: PhoneJS uses well-established layout methodologies supported by many server-side frameworks, including Ruby on Rails and ASP.NET MVC. Detailed information about Views and Layouts can be found in our online documentation. To configure view routing in our SPA, we must add an additional line of code in index.js: TipCalculator.app.router.register(":view", { view: "home" }); This registers a simple route that retrieves the view name from the URL (from the hash segment of the URL). The home view is used by default. Each view is defined in its own HTML file and is linked into the main application page index.html like this: PhoneJS ViewModel A viewmodel is a representation of data and operations used by the view. Each view has a function with the same base name as the view itself and returns the viewmodel for the view. For the home view, the views/home.js script defines the function home which creates the corresponding viewmodel. TipCalculator.home = function(params) { ... }; Three input parameters are used for the tip calculation algorithm: bill total, the number of people sharing the bill, and a tip percentage. These variables are defined as observables, which will be bound to corresponding UI widgets. Note: Observables functionality is supplied by Knockout.js, an important foundation for viewmodels used in PhoneJS. You can learn more about Knockout.js here. This is the code used in the home function to initialize the variables: var billTotal = ko.observable(), tipPercent = ko.observable(DEFAULT_TIP_PERCENT), splitNum = ko.observable(1); The result of the tip calculation is represented by four values: totalToPay, totalPerPerson, totalTip, tipPerPerson. Each value is a dependent observable (a computed value), which is automatically recalculated when any of the observables used in its definition change. Again, this is standard Knockout.js functionality. var totalTip = ko.computed(...); var tipPerPerson = ko.computed(...); var totalPerPerson = ko.computed(...); var totalToPay = ko.computed(...); For an example of business logic implementation in a viewmodel, let’s take a closer look at the observable totalToPay. The total sum to pay is usually rounded. For this purpose, we have two functions roundUp and roundDown that change the value of roundMode (another observable). These changes cause recalculation of totalToPay, because roundMode is used in the code associated with the totalToPay observable. var totalToPay = ko.computed(function() { var value = totalTip() + billTotalAsNumber(); switch(roundMode()) { case ROUND_DOWN: if(Math.floor(value) >= billTotalAsNumber()) return Math.floor(value); return value; case ROUND_UP: return Math.ceil(value); default: return value; } }); When any input parameter in the view changes, rounding should be disabled to allow the user to view precise values. We subscribe to the changes of the UI-bound observables to achieve this: billTotal.subscribe(function() { roundMode(ROUND_NONE); }); tipPercent.subscribe(function() { roundMode(ROUND_NONE); }); splitNum.subscribe(function() { roundMode(ROUND_NONE); }); The complete viewmodel can be found in home.js. It represents a simple example of a typical viewmodel. Note: In a more complex app, it may be useful to implement a structure that modularizes your viewmodels separate from view implementation files. In other words, a file like home.js need not contain the code to implement the viewmodel and instead call a helper function elsewhere for this purpose. In this walkthrough we’re trying to keep things structurally simple. PhoneJS Views Let’s now turn to the markup of the view located in the view/home.html file. The root div element represents a view with the name ‘home’. Within it is a div containing markup for a placeholder called ‘content’. ... A toolbar is located at the top of the view: dxToolbar is a PhoneJS UI widget. It’s defined in the markup using Knockout.js binding. A fieldset appears below the toolbar. To display a fieldset, we use two special CSS classes understood by PhoneJS: dx-fieldset and dx-field. The fieldset contains a text field for the bill total and two sliders for the tip percentage and the number of diners. Two buttons (dxButton) are displayed below the editors, allowing the user to round the total sum to pay. The remaining view displays fieldsets used for calculated results. Total to pay Total per person Total tip Tip per person This completes the description of the files required to create a simple app using PhoneJS. As you’ve seen, the process is simple, straightforward and intuitive. Start, Debug and Build for Stores Starting and debugging a PhoneJS app is just like any other HTML5 based app. You must deploy the folder containing HTML and JavaScript sources, along with any other required file to your web server. Because there is no server-side component to the architectural model, it doesn’t matter which web server you use as long as it can provide file access through HTTP. Once deployed, you can open the app on a device, in an emulator or a desktop browser by simply navigating app’s start page URL. If you want to view the app as it will appear in a phone or tablet within a desktop browser, you will have to override the UserAgent in the browser. Fortunately, this is easy to do with the developer tools that ship as part of today’s modern browsers: If you prefer not to modify UserAgent settings, you can use the Ripple Emulator to emulate multiple device types. At this point you have a web application that will work in the browser on the mobile device and look like native app. Modern mobile browsers provide access to local storage, location api, camera, so good chances are that your app already has anything it needs. Creating Store Ready Applications Using PhoneJS and PhoneGap But what if you need access to device features that browser does not provide? What if you want an app in the app store, not just a webpage. Then you’ll have to create a hybrid application and de-facto standard for such an app is Apache Cordova aka PhoneGap. PhoneGap project for each platform is a native app project that contains WebView (browser control) and a “bridge” that lets your JavaScript code inside WebView access native functions provided by PhoneGap libraries and plugins. To use it, you need to have SDK for each platform you are targeting, but you don’t need to know details of native development, you just need to put your HTML, CSS, JS files into right places and specify your app’s properties like name, version, icons, splashcreens and so on. To be able to publish your app, you will need to register as developer in the respective development portal. This process is well documented for each store and beyond the scope of this article. After that you’ll be able to receive certificates to sign your app package. The need to have SDK for each platform installed sounds challenging - especially after “write one, run everywhere” promise of HTML5/JS approach. This is a small price to pay for building hybrid application and have everything under control. But still there are several services and products that solves this problem for you. One is Adobe’s online service - PhoneGap Build which allows you to build one app for free (to build more, you’ll need a paid account). If you have all the required platform certificate files, the service can build your app for all supported platforms with a few mouse clicks. You only need to prepare app descriptions, promotional and informational content and icons in order to submit your app to an individual store. For Visual Studio developers, DevExpress offers a product called DevExtreme (it includes PhoneJS), which can build applications for iOS, Android and Windows Phone 8 directly within the Microsoft Visual Studio IDE. To summarize, if you need a web application that looks and feels like native on a mobile device, you need PhoneJS - it contains everything required to build touch-enabled, native-looking web application. If you want to go further and access device features, like the contact list or camera, from JavaScript code, you will need Cordova aka PhoneGap. PhoneGap also lets you compile your web app into a native app package. If you don’t want to install an SDK for each platform you are targeting, you can use the PhoneGapBuild service to build your package. Finally, if you have DevExtreme, you can build packages right inside Visual Studio.

I have a number of Spring MVC based RESTful services that return JSON. In 90% of the cases, the state of objects these services return will not change within a 24 hour period. This makes them (the JSON objects) perfect candidates for simple caching enabled by memcached. The idea was to have every request to Spring controllers intercepted, cache key generated and checked against the cache. If the key and corresponding value (JSON string) is available (a cache hit), it is returned to the caller as-is without making a full round trip to the database. However, if the cache has no entry for the key and hence no corresponding value (a cache miss), the call is forwarded to the controller, which in turn calls the logic to fetch desired object from the database and not only return it to the caller but also update the cache with the returned content. Keys are generated using the URL of the service in case of GET requests and the URL concatenated with POSTed input (as JSON) in case of POST requests. The resultant strings are encoded with MD5 to come up with a 32 character cache key which is well within the 250 character key length limit of memcached. Performance impact of using MD5 is yet to be evaluated during our load testing cycle. I started off trying to get hold of JSON response in the postHandle method of a Spring HandlerInterceptor. However since we are using @ResponseBody annotation in our controller, the JSON would be written directly to the stream. The ModelAndView was of course null because of this reason. If we removed the annotation and returned ModelAndView from the controller, the intended JSON object got enclosed in a map wrapper. A quick question on stack overflow didn’t help as the only suggestion I got was to extract my original object from the map wrapper. I wanted to keep this option (as discussed here as well ) as my last resort. The solution I eventually came up with involved Replacing the HandlerInterceptor with Servlet Filters Using DelegatingFilterProxy to make my filters spring application context aware Using HttpServletRequestWrapper to get control of the POST request body in the filter on the way in Using HttpServletResponseWrapper to get control of the response content in the filter on the way out True, its probably a more complex solution than just overriding MappingJacksonJsonView and extracting my JSON object, but it is more generic as it does not assume that all my content will always be JSON. Lets first start with the filter definition in the web.xml cacheFilter org.springframework.web.filter.DelegatingFilterProxy ... cacheFilter /* A standard filter configuration except for the fact that the filter class is always going to be org.springframework.web.filter.DelegatingFilterProxy. Where do you specify your own class ? As a bean in your spring context xml. The name of the filter and the name of the bean must be the same for the delegation to happen. Using the DelegatingFilterProxy allowed me to use my Filters with Spring. I can inject my dependencies as I would normally. Next, lets look at my MemcacheFilter filter Memcache Filter Class public class MemcacheFilter implements Filter { private static Logger logger = Logger.getLogger(MemcacheFilter.class); private CacheConfig cacheConfig; /** * Memcached lookup is being performed in this method. Firstly, keys are * generated depending on the request method (GET/POST). Then a cache lookup * is performed. If a value is obtained, the value is written to the * response otherwise, the actual target (in this case, Spring's Dispatcher * Servlet) is called by calling doFilter on the filteChain. The dispatcher * servlet calls the controller to produce the desire response which is * intercepted when the doFilter method returns. The Response is added to * the cache if the reponse code was 200(OK). * * @param request * @param response * @param filterChain * @throws IOException * @throws ServletException */ public void doFilter(ServletRequest request, ServletResponse response, FilterChain filterChain) throws IOException, ServletException { try { if ((request instanceof HttpServletRequest) && (response instanceof HttpServletResponse)) { // Wrapping the response in HTTPServletResponseWrapper MemcacheResponseWrapper responseWrap = new MemcacheResponseWrapper((HttpServletResponse) response); // Wrapping the request in HTTPServletResponseWrapper MemcacheRequestWrapper requestWrap = new MemcacheRequestWrapper((HttpServletRequest) request); // Get Memcached Client Instance MemcachedClient client = cacheConfig.getMemcachedClient(); Key keyGenerator = getKeyGenerator(requestWrap); if (keyGenerator != null) { String key = keyGenerator.getKey(requestWrap, cacheConfig); String value = (String) client.get(key); if (value == null) { // cache miss logger.info("Cache miss for key " + key); // call next filter/actual target for value filterChain.doFilter(requestWrap, responseWrap); if (responseWrap.getStatus() == HttpServletResponse.SC_OK) { // obtaining response content from // HttpServletResponseWrapper value = responseWrap.getOutputStream().toString(); // adding response to cache client.add(key, 0, value); logger.info("Adding response to cache: "+ (value.length() > 50 ? value.substring(0,50) + "..." : value)); } else { logger.warn("Did not add content to cache as response status is not 200"); } } else { // This case is a cache hit logger.info("Cache hit for key " + key); response.getWriter().println(value); } } else { logger.warn("Request skipped because no key generator could be found for the request's method"); // attempting call to actual target filterChain.doFilter(request, response); } } } catch (Exception ex) { logger.info("Cache functionality skipped due to exception", ex); // attempting call to actual target filterChain.doFilter(request, response); } } /** * Factory method that returns KeyGenerator based on the request method. * * @param httpRequest * @return */ private Key getKeyGenerator(HttpServletRequest httpRequest) { Key keyGenerator = null; if (httpRequest.getMethod().equalsIgnoreCase("GET")) { keyGenerator = new GetRequestKey(); } else if (httpRequest.getMethod().equalsIgnoreCase("POST")) { keyGenerator = new PostRequestKey(); } return keyGenerator; } public void init(FilterConfig arg0) throws ServletException { logger.debug("init"); } public CacheConfig getCacheConfig() { return cacheConfig; } public void setCacheConfig(CacheConfig cacheConfig) { this.cacheConfig = cacheConfig; } public void destroy() { logger.debug("destroy"); } } 1. I first wrap my request and response objects in the following statements. I have had to create the wrappers as well. Will get to those later. // Wrapping the response in HTTPServletResponseWrapper MemcacheResponseWrapper responseWrap = new MemcacheResponseWrapper((HttpServletResponse) response); // Wrapping the request in HTTPServletResponseWrapper MemcacheRequestWrapper requestWrap = new MemcacheRequestWrapper((HttpServletRequest) request); 2. Next, I have one of my injected classes, CacheConfig, provide me with a memcache client which I will use later to look up the cache. // Get Memcached Client Instance MemcachedClient client = cacheConfig.getMemcachedClient(); 3. I make a call to a function that tells me which key generator I should use, a GET one or a POST one depending on the request method. Key keyGenerator = getKeyGenerator(requestWrap); /** * Factory method that returns KeyGenerator based on the request method. * * @param httpRequest * @return */ private Key getKeyGenerator(HttpServletRequest httpRequest) { Key keyGenerator = null; if (httpRequest.getMethod().equalsIgnoreCase("GET")) { keyGenerator = new GetRequestKey(); } else if (httpRequest.getMethod().equalsIgnoreCase("POST")) { keyGenerator = new PostRequestKey(); } return keyGenerator; } 4. Check for a cache hit using the Key returned by the Key Generator. If its a miss, call next filter or target to compute actual value, get value from the response wrapper, and add it to the cache. if (keyGenerator != null) { String key = keyGenerator.getKey(requestWrap, cacheConfig); String value = (String) client.get(key); if (value == null) { // cache miss logger.info("Cache miss for key " + key); // call next filter/actual target for value filterChain.doFilter(requestWrap, responseWrap); if (responseWrap.getStatus() == HttpServletResponse.SC_OK) { // obtaining response content from // HttpServletResponseWrapper value = responseWrap.getOutputStream().toString(); // adding response to cache client.add(key, 0, value); logger.info("Adding response to cache: "+ (value.length() > 50 ? value.substring(0,50) + "..." : value)); } 5. If its a cache hit, just get return cached value else { // This case is a cache hit logger.info("Cache hit for key " + key); response.getWriter().println(value); } Lets take a look at each of the Wrappers. I am not going into a a lot of detail into how each of these work. Request Wrapper Class On the way in, the original POST content is extracted from the request and put in a String Buffer. To the filter, this content is returned via the toString() method of the WrappedInputStream class whereas the subsequently called controller calls the read method. public class MemcacheRequestWrapper extends HttpServletRequestWrapper { protected ServletInputStream stream; protected HttpServletRequest origRequest = null; protected BufferedReader reader = null; public MemcacheRequestWrapper(HttpServletRequest request) throws IOException { super(request); origRequest = request; } public ServletInputStream createInputStream() throws IOException { return (new WrappedInputStream(origRequest)); } @Override public ServletInputStream getInputStream() throws IOException { if (reader != null) { throw new IllegalStateException("getReader() has already been called for this request"); } if (stream == null) { stream = createInputStream(); } return stream; } @Override public BufferedReader getReader() throws IOException { if (reader != null) { return reader; } if (stream != null) { throw new IllegalStateException("getReader() has already been called for this request"); } stream = createInputStream(); reader = new BufferedReader(new InputStreamReader(stream)); return reader; } private class WrappedInputStream extends ServletInputStream { private StringBuffer originalInput = new StringBuffer(); private HttpServletRequest originalRequest; private ByteArrayInputStream byteArrayInputStream; public WrappedInputStream(HttpServletRequest request) throws IOException { this.originalRequest = request; BufferedReader bufferedReader = null; try { InputStream inputStream = request.getInputStream(); if (inputStream != null) { bufferedReader = new BufferedReader(new InputStreamReader(inputStream)); char[] charBuffer = new char[128]; int bytesRead = -1; while ((bytesRead = bufferedReader.read(charBuffer)) > 0) { originalInput.append(charBuffer, 0, bytesRead); } } byteArrayInputStream = new ByteArrayInputStream(originalInput.toString().getBytes()); } catch (IOException ex) { throw ex; } finally { if (bufferedReader != null) { try { bufferedReader.close(); } catch (IOException ex) { throw ex; } } } } @Override public String toString() { return this.originalInput.toString(); } @Override public int read() throws IOException { return byteArrayInputStream.read(); } } } Response Wrapper Class The response wrapper is similar to the request wrapper. Instead of the read method, there is a write method, called by the controller when its writing JSON content. This is stored in the wrapper and called in the filter. public class MemcacheResponseWrapper extends HttpServletResponseWrapper { protected ServletOutputStream stream; protected PrintWriter writer = null; protected HttpServletResponse origResponse = null; private int httpStatus = 200; public MemcacheResponseWrapper(HttpServletResponse response) { super(response); response.setContentType("application/json"); origResponse = response; } public ServletOutputStream createOutputStream() throws IOException { return (new WrappedOutputStream(origResponse)); } public ServletOutputStream getOutputStream() throws IOException { if (writer != null) { throw new IllegalStateException("getWriter() has already been called for this response"); } if (stream == null) { stream = createOutputStream(); } return stream; } public PrintWriter getWriter() throws IOException { if (writer != null) { return writer; } if (stream != null) { throw new IllegalStateException("getOutputStream() has already been called for this response"); } stream = createOutputStream(); writer = new PrintWriter(stream); return writer; } @Override public void sendError(int sc) throws IOException { httpStatus = sc; super.sendError(sc); } @Override public void sendError(int sc, String msg) throws IOException { httpStatus = sc; super.sendError(sc, msg); } @Override public void setStatus(int sc) { httpStatus = sc; super.setStatus(sc); } public int getStatus() { return httpStatus; } private class WrappedOutputStream extends ServletOutputStream { private StringBuffer originalOutput = new StringBuffer(); private HttpServletResponse originalResponse; public WrappedOutputStream(HttpServletResponse response) { this.originalResponse = response; } @Override public String toString() { return this.originalOutput.toString(); } @Override public void write(int arg0) throws IOException { originalOutput.append((char) arg0); originalResponse.getOutputStream().write(arg0); } } }

Elasticsearch is a flexible and powerful open source, distributed real-time search and analytics engine for the cloud based on Apache Lucene which provides full text search capabilities. It is document oriented and schema free. Asciidoctor is a pure Ruby processor for converting AsciiDoc source files and strings into HTML 5, DocBook 4.5 and other formats. Apart of Asciidoctor Ruby part, there is an Asciidoctor-java-integration project which let us call Asciidoctor functions from Java without noticing that Ruby code is being executed. In this post we are going to see how we can use Elasticsearch over AsciiDocdocuments to make them searchable by their header information or by their content. Let's add required dependencies: junit junit 4.11 test com.googlecode.lambdaj lambdaj 2.3.3 org.elasticsearch elasticsearch 0.90.1 org.asciidoctor asciidoctor-java-integration 0.1.3 Lambdaj library is used to convert AsciiDoc files to a json documents. Now we can start an Elasticsearch instance which in our case it is going to be an embedded instance. node = nodeBuilder().local(true).node(); Next step is parse AsciiDoc document header, read its content and convert them into a json document. An example of json document stored in Elasticsearch can be: { "title":"Asciidoctor Maven plugin 0.1.2 released!", "authors":[ { "author":"Jason Porter", "email":"[email protected]" } ], "version":null, "content":"= Asciidoctor Maven plugin 0.1.2 released!.....", "tags":[ "release", "plugin" ] } And for converting an AsciiDoc File to a json document we are going to useXContentBuilder class which is provided by ElasticsearchJava API to create jsondocuments programmatically. package com.lordofthejars.asciidoctor; import static org.elasticsearch.common.xcontent.XContentFactory.*; import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.IOException; import java.util.List; import org.asciidoctor.Asciidoctor; import org.asciidoctor.Author; import org.asciidoctor.DocumentHeader; import org.asciidoctor.internal.IOUtils; import org.elasticsearch.common.xcontent.XContentBuilder; import ch.lambdaj.function.convert.Converter; public class AsciidoctorFileJsonConverter implements Converter { private Asciidoctor asciidoctor; public AsciidoctorFileJsonConverter() { this.asciidoctor = Asciidoctor.Factory.create(); } public XContentBuilder convert(File asciidoctor) { DocumentHeader documentHeader = this.asciidoctor.readDocumentHeader(asciidoctor); XContentBuilder jsonContent = null; try { jsonContent = jsonBuilder() .startObject() .field("title", documentHeader.getDocumentTitle()) .startArray("authors"); Author mainAuthor = documentHeader.getAuthor(); jsonContent.startObject() .field("author", mainAuthor.getFullName()) .field("email", mainAuthor.getEmail()) .endObject(); List authors = documentHeader.getAuthors(); for (Author author : authors) { jsonContent.startObject() .field("author", author.getFullName()) .field("email", author.getEmail()) .endObject(); } jsonContent.endArray() .field("version", documentHeader.getRevisionInfo().getNumber()) .field("content", readContent(asciidoctor)) .array("tags", parseTags((String)documentHeader.getAttributes().get("tags"))) .endObject(); } catch (IOException e) { throw new IllegalArgumentException(e); } return jsonContent; } private String[] parseTags(String tags) { tags = tags.substring(1, tags.length()-1); return tags.split(", "); } private String readContent(File content) throws FileNotFoundException { return IOUtils.readFull(new FileInputStream(content)); } } Basically we are building the json document by calling startObject methods to start a new object, field method to add new fields, and startArray to start an array. Then this builder will be used to render the equivalent object in json format. Notice that we are using readDocumentHeader method from Asciidoctor class which returns header attributes from AsciiDoc file without reading and rendering the whole document. And finally content field is set with all document content. And now we are ready to start indexing documents. Note that populateData method receives as parameter a Client object. This object is from Elasticsearch Java APIand represents a connection to Elasticsearch database. import static ch.lambdaj.Lambda.convert; //.... private void populateData(Client client) throws IOException { List asciidoctorFiles = new ArrayList() {{ add(new File("target/test-classes/java_release.adoc")); add(new File("target/test-classes/maven_release.adoc")); }; List jsonDocuments = convertAsciidoctorFilesToJson(asciidoctorFiles); for (int i=0; i < jsonDocuments.size(); i++) { client.prepareIndex("docs", "asciidoctor", Integer.toString(i)).setSource(jsonDocuments.get(i)).execute().actionGet(); } client.admin().indices().refresh(new RefreshRequest("docs")).actionGet(); } private List convertAsciidoctorFilesToJson(List asciidoctorFiles) { return convert(asciidoctorFiles, new AsciidoctorFileJsonConverter()); } It is important to note that the first part of the algorithm is converting all our AsciiDocfiles (in our case two) to XContentBuilder instances by using previous converter class and the method convert of Lambdaj project. If you want you can take a look to both documents used in this example in https://github.com/asciidoctor/asciidoctor.github.com/blob/develop/news/asciidoctor-java-integration-0-1-3-released.adoc and https://github.com/asciidoctor/asciidoctor.github.com/blob/develop/news/asciidoctor-maven-plugin-0-1-2-released.adoc. Next part is inserting documents inside one index. This is done by using prepareIndexmethod, which requires an index name (docs), an index type (asciidoctor), and the idof the document being inserted. Then we call setSource method which transforms theXContentBuilder object to json, and finally by calling execute().actionGet(), data is sent to database. The final step is only required because we are using an embedded instance ofElasticsearch (in production this part should not be required), which refresh the indexes by calling refresh method. After that point we can start querying Elasticsearch for retrieving information from our AsciiDoc documents. Let's start with very simple example, which returns all documents inserted: SearchResponse response = client.prepareSearch().execute().actionGet(); Next we are going to search for all documents that has been written by Alex Sotowhich in our case is one. import static org.elasticsearch.index.query.QueryBuilders.matchQuery; //.... QueryBuilder matchQuery = matchQuery("author", "Alex Soto"); QueryBuilder matchQuery = matchQuery("author", "Alexander Soto"); Note that I am searching for field author the string Alex Soto, which returns only one. The other document is written by Jason. But it is interesting to say that if you search for Alexander Soto, the same document will be returned; Elasticsearch is smart enough to know that Alex and Alexander are very similar names so it returns the document too. More queries, how about finding documents written by someone who is called Alex, but not Soto. import static org.elasticsearch.index.query.QueryBuilders.fieldQuery; //.... QueryBuilder matchQuery = fieldQuery("author", "+Alex -Soto"); And of course no results are returned in this case. See that in this case we are using afield query instead of a term query, and we use +, and - symbols to exclude and include words. Also you can find all documents which contains the word released on title. import static org.elasticsearch.index.query.QueryBuilders.matchQuery; //.... QueryBuilder matchQuery = matchQuery("title", "released"); And finally let's find all documents that talks about 0.1.2 release, in this case only one document talks about it, the other one talks about 0.1.3. QueryBuilder matchQuery = matchQuery("content", "0.1.2"); Now we only have to send the query to Elasticsearch database, which is done by using prepareSearch method. SearchResponse response = client.prepareSearch("docs") .setTypes("asciidoctor") .setQuery(matchQuery) .execute() .actionGet(); SearchHits hits = response.getHits(); for (SearchHit searchHit : hits) { System.out.println(searchHit.getSource().get("content")); } Note that in this case we are printing the AsciiDoc content through console, but you could use asciidoctor.render(String content, Options options) method to render the content into required format. So in this post we have seen how to index documents using Elasticsearch, how to get some important information from AsciiDoc files using Asciidoctor-java-integration project, and finally how to execute some queries to inserted documents. Of course there are more kind of queries in Elasticsearch, but the intend of this post wasn't to explore all possibilities of Elasticsearch. Also as corollary, note how important it is using AsciiDoc format for writing your documents. Without much effort you can build a search engine for your documentation. On the other side, imagine all code that would be required to implement the same using any proprietary binary format like Microsoft Word. So we have shown another reason to use AsciiDoc instead of other formats.

I’ve recently had the need to automate configuration of Nginx on an Ubuntu server. Of course, in UNIX land we like to use SSH (Secure Shell) to log into our servers and manage them remotely. Wouldn’t it be nice, I thought, if there was a managed SSH library somewhere so that I could automate logging onto my Ubuntu server, run various commands and transfer files. A short Google turned up SSH.NET by the somewhat mysterious Olegkap (at least I couldn’t find out anything else about them) which turned out to be just what I wanted. Here’s the blurb on the CodePlex site: “This project was inspired by Sharp.SSH library which was ported from java and it seems like was not supported for quite some time. This library is complete rewrite using .NET 4.0, without any third party dependencies and to utilize the parallelism as much as possible to allow best performance I can get.” It does exactly what it says on the tin. It’s on NuGet, so you can grab it with: PM> Install-Package SSH.NET Here’s how you run a remote command. First you need to build a ConnectionInfo object: public ConnectionInfo CreateConnectionInfo() { const string privateKeyFilePath = @"C:\some\private\key.pem"; ConnectionInfo connectionInfo; using (var stream = new FileStream(privateKeyFilePath, FileMode.Open, FileAccess.Read)) { var privateKeyFile = new PrivateKeyFile(stream); AuthenticationMethod authenticationMethod = new PrivateKeyAuthenticationMethod("ubuntu", privateKeyFile); connectionInfo = new ConnectionInfo( "my.server.com", "ubuntu", authenticationMethod); } return connectionInfo; } Then you simply create an SshClient instance and run commands: public void Connect() { using (var ssh = new SshClient(CreateConnectionInfo())) { ssh.Connect(); var command = ssh.CreateCommand("uptime"); var result = command.Execute(); Console.Out.WriteLine(result); ssh.Disconnect(); } } Here I’m running the ‘uptime’ command which output this when I ran it just now: 14:37:46 up 22 days, 3:59, 0 users, load average: 0.08, 0.03, 0.05 To transfer a file, just use the ScpClient: public void GetConfigurationFiles() { using (var scp = new ScpClient(CreateNginxServerConnectionInfo())) { scp.Connect(); scp.Download("/etc/nginx/", new DirectoryInfo(@"D:\Temp\ScpDownloadTest")); scp.Disconnect(); } } Which grabs all my Nginx configuration and transfers it to a directory tree on my windows machine. All in all a very nice little library that’s been working well for me so far. Give it a try if you need to interact with a UNIX-like machine from .NET code.

My team and I are creating a services platform based on a set of RESTful JSON services where each service contributes to the platform by providing distinct feature(s) and/or data. With logs being generated all over the place, we thought it was a good idea to centralize logging and perhaps also provide a rudimentary log viewer that allowed us to view, filter, sort and search our logs. We also wanted our logging to be asynchronous as we didn’t want our services to be held up while trying to write logs say maybe directly to a database. The strategy for achieving this was straight forward. Setup ActiveMQ Create a log4j appender that writes logs to the queue (log4j ships with one such appender but lets write our own. Write a message listener that reads logs from a JMS queue setup on an MQ server and persists them Let’s take a look one by one. Setup ActiveMQ Setting up an external ActiveMQ server is simple enough. A great tutorial is available at http://servicebus.blogspot.com/2011/02/installing-apache-active-mq-on-ubuntu.html to set it up on Ubuntu. You can also choose to embed a message broker within your application. Spring makes this easy. We will see how later. Creating a Lo4j JMS appender First, we create a log4j JMS appender. log4j ships with one such appender (that writes to a JMS topic instead of a queue) import javax.jms.DeliveryMode; import javax.jms.Destination; import javax.jms.MessageProducer; import javax.jms.ObjectMessage; import javax.jms.Session; import org.apache.activemq.ActiveMQConnectionFactory; import org.apache.log4j.Appender; import org.apache.log4j.AppenderSkeleton; import org.apache.log4j.Logger; import org.apache.log4j.PatternLayout; import org.apache.log4j.spi.LoggingEvent; /** * JMSQueue appender is a log4j appender that writes LoggingEvent to a queue. * @author faheem * */ public class JMSQueueAppender extends AppenderSkeleton implements Appender{ private static Logger logger = Logger.getLogger("JMSQueueAppender"); private String brokerUri; private String queueName; @Override public void close() { } @Override public boolean requiresLayout() { return false; } @Override protected synchronized void append(LoggingEvent event) { try { ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory( this.brokerUri); // Create a Connection javax.jms.Connection connection = connectionFactory.createConnection(); connection.start();np // Create a Session Session session = connection.createSession(false,Session.AUTO_ACKNOWLEDGE); // Create the destination (Topic or Queue) Destination destination = session.createQueue(this.queueName); // Create a MessageProducer from the Session to the Topic or Queue MessageProducer producer = session.createProducer(destination); producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT); ObjectMessage message = session.createObjectMessage(new LoggingEventWrapper(event)); // Tell the producer to send the message producer.send(message); // Clean up session.close(); connection.close(); } catch (Exception e) { e.printStackTrace(); } } public void setBrokerUri(String brokerUri) { this.brokerUri = brokerUri; } public String getBrokerUri() { return brokerUri; } public void setQueueName(String queueName) { this.queueName = queueName; } public String getQueueName() { return queueName; } } Lets see whats happening here. Line 19: We implement the Log4J appender interface that asks us to implement three methods. requiresLayout, close and append. We will keep things simple for the moment and implement the append method which gets called whenever a method call to the logger is made. Line 37: log4j calls the append method and passes a LoggingEvent object as a parameter which represents a call to a logger. A LoggingEvent object encapsulates all information about every log item. Line 41 & 42: Create a new connection factory by providing it with a uri of a JMS, in our case activemq, server Line 45, 46 and 49: We establish a connection and a session to the JMS server. A Session can be opened in several modes. An Auto_Acknowledge session is one in which the acknowledgment of message happens automatically. Other modes include Client_Acknowledge in which a client has to explicitly acknowledge receipt and/or processing of a message and two other modes. For details, refer to the docs at http://download.oracle.com/javaee/1.4/api/javax/jms/Session.html Line 52: Create a queue. Send the queue name to connect to as a parameter. Line 56: We set the delivery mode to Non_Persistent. The other option is Persistent where the message is persisted to a persistent store. Persistent mode slows down but adds reliability to the message transfer. Line 58: We are doing multiple things. First of all I am wrapping the LoggingEvent object into a LoggingEventWrapper. This is because there are some properties within the LoggingEvent object that are not serializeable and also because I want to capture some additional information such as IP address and host name. Next, using the JMS session object, I prepare an object (the wrapper) for transport. Line 61: I send the object to the queue. Below is the code for the wrapper. import java.io.Serializable; import java.net.InetAddress; import java.net.UnknownHostException; import org.apache.log4j.EnhancedPatternLayout; import org.apache.log4j.spi.LoggingEvent; /** * Logging Event Wraps a log4j LoggingEvent object. Wrapping is required by some information is lost * when the LoggingEvent is serialized. The idea is to extract all information required from the LoggingEvent * object, place it in the wrapper and then serialize the LoggingEventWrapper. This way all required data remains * available to us. * @author faheem * */ public class LoggingEventWrapper implements Serializable{ private static final String ENHANCED_PATTERN_LAYOUT = "%throwable"; private static final long serialVersionUID = 3281981073249085474L; private LoggingEvent loggingEvent; private Long timeStamp; private String level; private String logger; private String message; private String detail; private String ipAddress; private String hostName; public LoggingEventWrapper(LoggingEvent loggingEvent){ this.loggingEvent = loggingEvent; //Format event and set detail field EnhancedPatternLayout layout = new EnhancedPatternLayout(); layout.setConversionPattern(ENHANCED_PATTERN_LAYOUT); this.detail = layout.format(this.loggingEvent); } public Long getTimeStamp() { return this.loggingEvent.timeStamp; } public String getLevel() { return this.loggingEvent.getLevel().toString(); } public String getLogger() { return this.loggingEvent.getLoggerName(); } public String getMessage() { return this.loggingEvent.getRenderedMessage(); } public String getDetail() { return this.detail; } public LoggingEvent getLoggingEvent() { return loggingEvent; } public String getIpAddress() { try { return InetAddress.getLocalHost().getHostAddress(); } catch (UnknownHostException e) { return "Could not determine IP"; } } public String getHostName() { try { return InetAddress.getLocalHost().getHostName(); } catch (UnknownHostException e) { return "Could not determine Host Name"; } } } The Message Listener The message listener “listens” to the queue (or topic). Whenever a new message is added to the queue, the onMessage method is called. import javax.jms.JMSException; import javax.jms.Message; import javax.jms.MessageListener; import javax.jms.ObjectMessage; import org.apache.log4j.Logger; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; @Component public class LogQueueListener implements MessageListener { public static Logger logger = Logger.getLogger(LogQueueListener.class); @Autowired private ILoggingService loggingService; public void onMessage( final Message message ) { if ( message instanceof ObjectMessage ) { try{ final LoggingEventWrapper loggingEventWrapper = (LoggingEventWrapper)((ObjectMessage) message).getObject(); loggingService.saveLog(loggingEventWrapper); } catch (final JMSException e) { logger.error(e.getMessage(), e); } catch (Exception e) { logger.error(e.getMessage(),e); } } } } Line 23: Checking if the object being picked off the queue is an instance of ObjectMessage Line 26: Extracting LoggingEventWrapper from the Message Line 27: Call a service method to persist the log Wiring up in Spring Lines 5-9: Use the broker tag to setup an embedded message broker. Since I am using an external one, I don’t need it. Line 12: Mention the name of the queue you want to connect to. Line 14: URI of the Broker Server. Line 15-19: Connection Factory setup Line 26-28: Message Listener Setup where we specify the number of concurrent threads that can consume messages off the queue. Of course, the above example will not work out of the box. You still have to include all JMS dependencies and implement the service that persists logs. But I hope it gives you a decent idea.

[This blog was syndicated from http://blog.grallandco.com] Introduction When I was looking for a more effective way to create my cluster I asked some sysadmins which tools I should use to do it. The answer I got during OSDC was not Puppet, nor Chef, but wasAnsible. This article shows you how you can easily configure and create a Couchbase cluster deployed and many linux boxes...and the only thing you need on these boxes is an SSH Server! Thanks to Jan-Piet Mens that was one of the person that convinced me to use Ansible and answered questions I had about Ansible. You can watch the demonstration below, and/or look at all the details in the next paragraph. Ansible Ansible is an open-source software that allows administrator to configure and manage many computers over SSH. I won't go in all the details about the installation, just follow the steps documented in the Getting Started Guide. As you can see from this guide, you just need Python and few other libraries and clone Ansible project from Github. So I am expecting that you have Ansible working with your various servers on which you want to deploy Couchbase. Also for this first scripts I am using root on my server to do all the operations. So be sure you have register the root ssh keys to your administration server, from where you are running the Ansible scripts. Create a Couchbase Cluster So before going into the details of the Ansible script it is interesting to explain how you create a Couchbase Cluster. So here are the 5 steps to create and configure a cluster: Install Couchbase on each nodes of the cluster, as documented here. Take one of the node and "initialize" the cluster, using cluster-init command. Add the other nodes to the cluster, using server-add command. Rebalance, using rebalance command. Create a Bucket, using bucket-create command. So the goal now is to create an Ansible Playbook that executes these steps for you. Ansible Playbook for Couchbase The first think you need is to have the list of hosts you want to target, so I have create a hosts file that contains all my server organized in 2 groups: [couchbase-main] vm1.grallandco.com [couchbase-nodes] vm2.grallandco.com vm3.grallandco.com The group [couchbase-main] group is just one of the node that will drive the installation and configuration, as you probably already know, Couchbase does not have any master... All nodes in the cluster are identical. To ease the configuration of the cluster, I have create another file that contains all parameters that must be sent to all the various commands. This file is located in the group_vars/all see the section Splitting Out Host and Group Specific Data in the documentation. # Adminisrator user and password admin_user: Administrator admin_password: password # ram quota for the cluster cluster_ram_quota: 1024 # bucket and replicas bucket_name: ansible bucket_ram_quota: 512 num_replicas: 2 Use this file to configure your cluster. Let's describe the playbook file : - name: Couchbase Installation hosts: all user: root tasks: - name: download Couchbase package get_url: url=http://packages.couchbase.com/releases/2.0.1/couchbase-server-enterprise_x86_64_2.0.1.deb dest=~/. - name: Install dependencies apt: pkg=libssl0.9.8 state=present - name: Install Couchbase .deb file on all machines shell: dpkg -i ~/couchbase-server-enterprise_x86_64_2.0.1.deb As expected, the installation has to be done on all servers as root then we need to execute 3 tasks: Download the product, the get_url command will only download the file if not already present Install the dependencies with the apt command, the state=present allows the system to only install this package if not already present Install Couchbase with a simple shell command. (here I am not checking if Couchbase is already installed) So we have now installed Couchbase on all the nodes. Let's now configure the first node and add the others: - name: Initialize the cluster and add the nodes to the cluster hosts: couchbase-main user: root tasks: - name: Configure main node shell: /opt/couchbase/bin/couchbase-cli cluster-init -c 127.0.0.1:8091 --cluster-init-username=${admin_user} --cluster-init-password=${admin_password} --cluster-init-port=8091 --cluster-init-ramsize=${cluster_ram_quota} - name: Create shell script for configuring main node action: template src=couchbase-add-node.j2 dest=/tmp/addnodes.sh mode=750 - name: Launch config script action: shell /tmp/addnodes.sh - name: Rebalance the cluster shell: /opt/couchbase/bin/couchbase-cli rebalance -c 127.0.0.1:8091 -u ${admin_user} -p ${admin_password} - name: create bucket ${bucket_name} with ${num_replicas} replicas shell: /opt/couchbase/bin/couchbase-cli bucket-create -c 127.0.0.1:8091 --bucket=${bucket_name} --bucket-type=couchbase --bucket-port=11211 --bucket-ramsize=${bucket_ram_quota} --bucket-replica=${num_replicas} -u ${admin_user} -p ${admin_password} Now we need to execute specific taks on the "main" server: Initialization of the cluster using the Couchbase CLI, on line 06 and 07 Then the system needs to ask all other server to join the cluster. For this the system needs to get the various IP and for each IP address execute the add-server command with the IP address. As far as I know it is not possible to get the IP address from the main playbook YAML file, so I ask the system to generate a shell script to add each node and execute the script. This is done from the line 09 to 13. To generate the shell script, I use Ansible Template, the template is available in the couchbase-add-node.j2 file. {% for host in groups['couchbase-nodes'] %} /opt/couchbase/bin/couchbase-cli server-add -c 127.0.0.1:8091 -u ${admin_user} -p ${admin_password} --server-add={{ hostvars[host]['ansible_eth0']['ipv4']['address'] }:8091 --server-add-username=${admin_user} --server-add-password=${admin_password} {% endfor %} As you can see this script loop on each server in the [couchbase-nodes] group and use its IP address to add the node to the cluster. Finally the script rebalance the cluster (line 16) and add a new bucket (line 19). You are now ready to execute the playbook using the following command : ./bin/ansible-playbook -i ./couchbase/hosts ./couchbase/couchbase.yml -vv I am adding the -vv parameter to allow you to see more information about what's happening during the execution of the script. This will execute all the commands described in the playbook, and after few seconds you will have a new cluster ready to be used! You can for example open a browser and go to the Couchase Administration Console and check that your cluster is configured as expected. As you can see it is really easy and fast to create a new cluster using Ansible. I have also create a script to uninstall properly the cluster.. just launch ./bin/ansible-playbook -i ./couchbase/hosts ./couchbase/couchbase-uninstall.yml

I'd like to show how to instantiate Spring's component with non-default constructor (constructor with parameters) from other Spring driven component because it seems to me that a lot of Java developers doesn't know that this is possible in Spring. First of all, simple annotation based Spring configuration. It's it is scanning target package for components. @Configuration @ComponentScan("sk.lkrnac.blog.componentwithparams") public class SpringContext { } Here is component with non-default constructor. This component needs to be specified as prototype, because we will be creating new instances of it. It also needs to be named (will clarify why below). @Component(SpringConstants.COMPONENT_WITH_PARAMS) @Scope("prototype") public class ComponentWithParams { private String param1; private int param2; public ComponentWithParams(String param1, int param2) { this.param1 = param1; this.param2 = param2; } @Override public String toString() { return "ComponentWithParams [param1=" + param1 + ", param2=" + param2 + "]"; } } Next component instantiates ComponentWithParams. It needs to be aware of Spring's context instance to be able to pass arguments into non-default constructor. getBean(String name, Object... args) method is used for that purpose. First parameter of this method is name of the instantiating component (that is why component needed to be named). Constructor parameters follow. @Component public class CreatorComponent implements ApplicationContextAware{ ApplicationContext context; public void createComponentWithParam(String param1, int param2){ ComponentWithParams component = (ComponentWithParams) context.getBean(SpringConstants.COMPONENT_WITH_PARAMS, param1, param2); System.out.println(component.toString() + " instanciated..."); } @Override public void setApplicationContext(ApplicationContext context) throws BeansException { this.context = context; } } Last code snippet is main class which loads Spring context and runs test. public class Main { public static void main(String[] args) { AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(SpringContext.class); CreatorComponent invoker = context.getBean(CreatorComponent.class); invoker.createComponentWithParam("Fisrt", 1); invoker.createComponentWithParam("Second", 2); invoker.createComponentWithParam("Third", 3); context.close(); } } Console output: ComponentWithParams instanciated... ComponentWithParams instanciated... ComponentWithParams instanciated... And that's it. You can download souce code on Github

In this tutorial we are going to investigate ways you can tell what events a certain element has bound to it. This is really useful when you need to debug the on method to see how it is working. There are a number of ways to bind events to an element, there is bind(), live() and delegate(). Bind - Used to attach events to current elements on the page. If new elements are added with the same selector the event will not be added to the element. Live - This is used to attach an events to all elements on the page and future elements with the same selector. Delegate - This is used to attach events to all elements on the page and future elements based on a specific root element. As of jQuery 1.7 the on method will give you all the functionality that you need to bound events to elements. It will allow you to add an event to all elements on the page and all future elements. $('.element').on('click', function(){ // stuff }); $('.element').on('hover', function(){ // stuff }); $('.element').on('mouseenter', function(){ // stuff }); $('form').on('submit', function(){ // stuff }); When you are assigning events to future elements you might want to find out what events are currently assigned to these elements. Using developer tools in your browser you can see what events are currently bound to the element by using these 2 methods. First you can display the events by using the console, the second method is to view the events in the event listener window in the developer tools. Display Events In Console In your browser if you press F12 a new window called developer tools will open, this allows you to view the entire HTML DOM in more detail, this also has a console feature which will allow you to type in any Javascript to run at this current time on the page. You can use the console to display a list of events currently assigned to an element by using the following code. $._data( $('.element')[0], 'events' ); This will display all the events that are currently assigned to the element. Event Listener Window The other option to use to view what events are currently assigned to an element is to use the event listener window in your browser's developer tools. All you have to do is press F12 to open the developer tools, select the element in the HTML DOM that you want to investigate, on the right side of the window you will see an option called Event Listeners. When you expand this menu you will see all the current events assigned to the element including all click events bound from jQuery.

Have you ever tried to update specific fields of an entity using Entity Framework? If yes, then I can assume that System.Data.Entity.Validation.DbEntityValidationResult – Validation failed for one or more entities. See ‘EntityValidationErrors’ property for more details. Exception is not something new to you. While working recently on a project for which I was using Entity Framework 5.x CF (the part with code first is not relevant) I bumped into an interesting piece of code. The main idea behind it is to perform updates for a single property and ignore the rest even if they don't respect the constraints defined in the model. Please note that I have found PartialEntityValidation class somewhere on the internet and unfortunately I do not remember where, I was not able to find it again. The only changes made by me are related to code formatting. I will present the code first and explain the logic behind it afterwards. using System.Collections.Generic; using System.Data.Entity.Infrastructure; using System.Data.Entity.Validation; namespace MyProject.Data.Contexts.Concrete { public class PartialEntityValidation { private readonly IDictionary _store; public PartialEntityValidation() { _store = new Dictionary(); } public void Register(DbEntityEntry entry, string[] properties) { if (_store.ContainsKey(entry)) _store[entry] = properties; else _store.Add(entry, properties); } public void Unregister(DbEntityEntry entry) { _store.Remove(entry); } public bool IsResponsibleFor(DbEntityEntry entry) { return _store.ContainsKey(entry); } public void Validate(DbEntityValidationResult validationResult) { var entry = validationResult.Entry; foreach (var property in _store[entry]) { var validationErrors = entry.Property(property).GetValidationErrors(); foreach (var validationError in validationErrors) validationResult.ValidationErrors.Add(validationError); } } } } PartialEntityValidation class has a simple constructor which initializes a new dictionary store where the key of it is an DbEntityEntry and the value (currently) an empty string array. The string array will hold later on the properties which should be validated. The class has four methods: Register, Unregister, IsResponsibleFor and Validate. Now let me explain a bit what each method does. Register adds a new DbEntityEntry with the properties that have to be validated to the existing store. It basically registers a new entity entry for validation. It is quite obvious what Unregister method does; it removes an entity entry from the existing store. IsResponsibleFor method determines if an entity entry is present in the existing store and if it will be part of the validation process. The last method is Validate which accepts a single parameter of DbEntityValidationResult type, from which it takes the entity entry associated with the validation result and validates each property of the entity entry present in the store. If there are any validation errors those are added on ValidationErrors collection of the validation result. Now it is time to see how to use PartialEntityValidation class. In My case I have exposed it as a public read only property of the context and had ValidateEntity method overridden at context level. using System; using System.Collections.Generic; using System.Configuration; using System.Data; using System.Data.Entity; using System.Data.Entity.Infrastructure; using System.Data.Entity.Validation; namespace MyProject.Data.Contexts.Concrete { public class MyContext : DbContext { private readonly PartialEntityValidation _partialEntityValidation; static MyContext() { Database.SetInitializer(null); } protected MyContext() : base("MyDb") { _partialEntityValidation = new PartialEntityValidation(); Configuration.LazyLoadingEnabled = false; } public PartialEntityValidation PartialValidation { get { return _partialEntityValidation; } } protected override DbEntityValidationResult ValidateEntity(DbEntityEntry entityEntry, IDictionary items) { if (PartialValidation.IsResponsibleFor(entityEntry)) { var validationResult = new DbEntityValidationResult(entityEntry, new List()); PartialValidation.Validate(validationResult); return validationResult; } return base.ValidateEntity(entityEntry, items); } } } I am sure that I do not have to explain the logic behind ValidateEntity method from the db context. The final piece is the repository where the partial update is invoked. using System; using System.Linq; using System.Linq.Expressions; namespace MyProject.Data.Repositories.Concrete { public class MyRepository { private readonly MyContext _context; public MyRepository(MyContext context) { _context = context; } public void DeleteProduct(Guid productId) { var product = new Product { ProductId = productId, IsDeleted = true }; _context.PartialValidation.Register(_context.Entry(product), new[] { "IsDeleted" }); Update(workspace, p => p.IsDeleted); _context.SaveChanges(); ((IObjectContextAdapter)_context).ObjectContext.Detach(product); } public void Update(T entity, Expression> property) where T : class { var entry = _context.Entry(entity); _context.Set().Attach(entity); entry.Property(property).IsModified = true; } } } Please take into consideration that this is a demo repository which doesn't do much at all except to present how a partial update is performed. The repository has two important methods. First one is DeleteProduct which performs a soft delete by updating "IsDeleted" property of product entity to true. This is achieved by creating a new product object of which primary key is "ProductId" with "IsDeleted" property set to true. The next step is to register the entity entry along with the "IsDeleted" property (this is the only property we want to have it updated, all other properties of Product entity will be ignored) into PartialValidation store. Update method is called which attaches the entity entry to the context and marks it as modified in order to have it updated by entity framework. That is all, I really hope that you got the idea behind this even if I did not provided an actual sample application for download.