This article represents top 4 security vulnerabilities related coding practice to avoid while you are programming with Java language. Recently, I came across few Java projects where these instances were found. Please feel free to comment/suggest if I missed to mention one or more important points. Also, sorry for the typos. Following are the key points described later in this article: Executing a dynamically generated SQL statement Directly writing an Http Parameter to Servlet output Creating an SQL PreparedStatement from dynamic string Array is stored directly Executing a Dynamically Generated SQL Statement This is most common of all. One can find mention of this vulenrability at several places. As a matter of fact, many developers are also aware of this vulnerability, although this is a different thing they end up making mistakes once in a while. In several DAO classes, the instances such as following code were found which could lead to SQL injection attacks. StringBuilder query = new StringBuilder(); query.append( "select * from user u where u.name in (" + namesString + ")" ); try { Connection connection = getConnection(); Statement statement = connection.createStatement(); resultSet = statement.executeQuery(query.toString()); } Instead of above query, one could as well make use of prepared statement such as that demonstrated in the code below. It not only makes code less vulnerable to SQL injection attacks but also makes it more efficient. StringBuilder query = new StringBuilder(); query.append( "select * from user u where u.name in (?)" ); try { Connection connection = getConnection(); PreparedStatement statement = connection.prepareCall(query.toString()); statement.setString( 1, namesString ); resultSet = statement.execute(); } Directly writing an Http Parameter to Servlet Output In Servlet classes, I found instances where the Http request parameter was written as it is, to the output stream, without any validation checks. Following code demonstrate the same: public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { String content = request.getParameter("some_param"); // // .... some code goes here // response.getWriter().print(content); } Note that above code does not persist anything. Code like above may lead to what is called reflected (or non-persistent) cross site scripting (XSS) vulnerability. Reflected XSS occur when an attacker injects browser executable code within a single HTTP response. As it goes by definition (being non-persistent), the injected attack does not get stored within the application; it manifests only users who open a maliciously crafted link or third-party web page. The attack string is included as part of the crafted URI or HTTP parameters, improperly processed by the application, and returned to the victim. You could read greater details on following OWASP page on reflect XSS Creating an SQL PreparedStatement from Dynamic Query String What it essentially means is the fact that although PreparedStatement was used, but the query was generated as a string buffer and not in the way recommended for prepared statement (parametrized). If unchecked, tainted data from a user would create a String where SQL injection could make it behave in unexpected and undesirable manner. One should rather make the query statement parametrized and, use the PreparedStatement appropriately. Take a look at following code to identify the vulnerable code. StringBuilder query = new StringBuilder(); query.append( "select * from user u where u.name in (" + namesString + ")" ); try { Connection connection = getConnection(); PreparedStatement statement = connection.prepareStatement(query.toString()); resultSet = statement.executeQuery(); } Array is Stored Directly Instances of this vulnerability, Array is stored directly, could help the attacker change the objects stored in array outside of program, and the program behave in inconsistent manner as the reference to the array passed to method is held by the caller/invoker. The solution is to make a copy within the object when it gets passed. In this manner, a subsequent modification of the collection won’t affect the array stored within the object. You could read the details on following stackoverflow page. Following code represents the vulnerability: // Note that values is a String array in the code below. // public void setValues(String[] somevalues) { this.values = somevalues; }

If you use Java for programming, read on to learn about tools like Eclipse IDE, the Java Development Kit, and other must-know tools.

in this article, we will try to look at routes in backbone.js. we will try to understand how routes can be useful in a large scale single page applications and how we can use routes to perform action based on requested url. background we have been using web application for more than 2 decades now. this has made us tuned to some of the functionalities that the websites provide. one such functionality is to be able to copy the url and use it for the viewing the exact application area that we were viewing before. another example is the use of browser navigation buttons to navigate back and forth the pages. when we create single page applications, there is only one page being rendered on the screen. there is no separate url for each page. the browser is not loading the separate pages for separate screens. so how can we still perform the above mentioned operations even with a single page application. the answer is backbone routes. link to complete series: backbone tutorial – part 1: introduction to backbone.js [ ^ ] backbone tutorial – part 2: understanding the basics of backbone models [ ^ ] backbone tutorial – part 3: more about backbone models [ ^ ] backbone tutorial – part 4: crud operations on backbonejs models using http rest service [ ^ ] backbone tutorial – part 5: understanding backbone.js collections [ ^ ] backbone tutorial – part 6: understanding backbone.js views [ ^ ] backbone tutorial – part 7: understanding backbone.js routes and history [ ^ ] using the code backbone routes and history provides us the mechanism by which we can copy the urls and use them to reach the exact view. it also enables us to use browser navigation with single page applications. actually routes facilitate the possibility of having deep copied urls and history provides the possibility of using the browser navigation. life without router let us try to create a simple application that is not using the router. lets create three simple views and these views will be rendered in the same area on our application based on user selection. let create 3 very simple views. var view1 = backbone.view.extend({ initialize: function() { this.render(); }, render: function() { this.$el.html(this.model.get('message') + " from the view 1"); return this; } }); var view2 = backbone.view.extend({ initialize: function() { this.render(); }, render: function() { this.$el.html(this.model.get('message') + " from the view 2"); return this; } }); var view3 = backbone.view.extend({ initialize: function() { this.render(); }, render: function() { this.$el.html(this.model.get('message') + " from the view 3"); return this; } }); now we need a view that will contain the view and render it whenever the user makes a choice on the screen. var containerview = backbone.view.extend({ mychildview: null, render: function() { this.$el.html("greeting area"); this.$el.append(this.mychildview.$el); return this; } }); var containerview = backbone.view.extend({ mychildview: null, render: function() { this.$el.html("greeting area"); this.$el.append(this.mychildview.$el); return this; } }); now we need a view that will contain the view and render it whenever the user makes a choice on the screen. now lets create a simple div on the ui which will be used as elto this containerview. we will then position three buttons on the ui which will let the user to change the view. below code shows the application setup that is creating the container view and the functions that will get invoked when the user selects the view from screen. var greeting = new greetmodel({ message: "hello world" }); var container = new containerview({ el: $("#appcontainer"), model: greeting }); var view1 = null; var view2 = null; var view3 = null; function showview1() { if (view1 == null) { view1 = new view1({ model: greeting }); } container.mychildview = view1; container.render(); } function showview2() { if (view2 == null) { view2 = new view2({ model: greeting }); } container.mychildview = view2; container.render(); } function showview3() { if (view3 == null) { view3 = new view3({ model: greeting }); } container.mychildview = view3; container.render(); } now lets run the application and see the results. when we click on the buttons we can see that the actual view is getting changes but the url is not getting changes. that would mean that there is no way, i can copy a url and directly go to any view. also, the second thing to note here is that if we press the browser back button, the application will go away(since its still on the same single page from the browser’s perspective). note: please download and run the sample code to see this in action. hello backbone routes now the above problem can very easily be solved using backbone routesand history. so lets try to first look at what are backbone routes. backbone routes are simple objects that are handle the incoming route value from the url and the invoke any function. lets create a very simple route class for our application. var myrouter = backbone.router.extend({ }); in our route class we will have to define the routes that our application will support and how we want to handle them. so first lets create a simple route where only the url is present. this usually is the starting page of our application. for our application lets just open view1 whenever nothing is present in the route. then if the request is for any specific view we will simply invoke the function which will take care of rendering the appropriate view. var myrouter = backbone.router.extend({ greeting: null, container: null, view1: null, view2: null, view3: null, initialize: function() { this.greeting = new greetmodel({ message: "hello world" }); this.container = new containerview({ el: $("#rappcontainer"), model: this.greeting }); }, routes: { "": "handleroute1", "view1": "handleroute1", "view2": "handleroute2", "view3": "handleroute3" }, handleroute1: function () { if (this.view1 == null) { this.view1 = new view1({ model: this.greeting }); } this.container.mychildview = this.view1; this.container.render(); }, handleroute2: function () { if (this.view2 == null) { this.view2 = new view2({ model: this.greeting }); } this.container.mychildview = this.view2; this.container.render(); }, handleroute3: function () { if (this.view3 == null) { this.view3 = new view3({ model: this.greeting }); } this.container.mychildview = this.view3; this.container.render(); } }); now this route class contains the complete logic of handling the url requests and rendering the view accordingly. not only this, we can see that the code which was written in a global scope earlier i.e. the controller and view creation all that is put inside the route now. this would also mean that routes not only provide us deep copyable urls but also could provide more options to have better structured code(since we can have multiple route classes and each route class can handle all the respective views for the defined routes). backbone history and instantiating routes backbone history is a global router that will keep track of the history and let us enable the routing in the application. to instantiate a route and start tracking the navigation history, we need to simply create the router class and call backbone.history.start for let the backbone start listening to routes and manage history. $(document).ready(function () { router = new myrouter(); backbone.history.start(); }) invoking and requesting routes a route can either be invoked from the other parts of the application or it can simply be requested by the user. invoking route: application wants to navigate to a specific route (this can be done by navigating to a route by calling the navigate function: router.navigate('view1'); route request: user enters the fully qualified url (this will work seamlessly) let us run the application and see the result. passing parameters in the routes we can also pass parameters in the route. lets us try to create a new route where the user will request for a view in a parameterized manner. parameters can be defined as “ route/:param” var myrouter = backbone.router.extend({ greeting: null, container: null, view1: null, view2: null, view3: null, initialize: function () { this.greeting = new greetmodel({ message: "hello world" }); this.container = new containerview({ el: $("#rappcontainer"), model: this.greeting }); }, routes: { "": "handleroute1", "view/:viewid": "handlerouteall" }, handlerouteall: function (viewid) { if (viewid == 1) { this.handleroute1(); } else if (viewid == 2) { this.handleroute2(); } else if (viewid == 3) { this.handleroute3(); } }, handleroute1: function () { if (this.view1 == null) { this.view1 = new view1({ model: this.greeting }); } this.container.mychildview = this.view1; this.container.render(); }, handleroute2: function () { if (this.view2 == null) { this.view2 = new view2({ model: this.greeting }); } this.container.mychildview = this.view2; this.container.render(); }, handleroute3: function () { if (this.view3 == null) { this.view3 = new view3({ model: this.greeting }); } this.container.mychildview = this.view3; this.container.render(); } }); the above route can be invoked by passing view/2 as url. the viewid passed to the router will be 2. having optional parameters in routes we can also pass optional parameters in the routes, lets try to pass a simple parameter in the above defined route and see how it works. optional parameters can be defined as “ route(/:param)“. var myrouter = backbone.router.extend({ routes: { "": "handleroute1", "view1": "handleroute1", "view2": "handleroute2", "view3": "handleroute3", "view/:viewid(/:msg)": "handlerouteall" }, handlerouteall: function (viewid, msg) { if (msg) { alert(msg); } } }); in the above code, if we pass the second parameter i.e. view/2/test, the alert will be shown else not. note: the route definition can also contain complex regex based patterns if we need one route to handle multiple urls based on some regular expression. point of interest in this article we saw backbone.js routes. we saw how routes enable us to create bookmarkable urls and will let the user request a view based on url. we also looked at how we can use browser navigation by using backbone history. this has been written from a beginner’s perspective. i hope this has been informative. download sample code for this article: backboneroutessample

Raygun, Stack Hunter, Sentry, Takipi and Airbrake: Modern developer tools to help you crush bugs before bugs crush your app With the Java ecosystem going forward, web applications serving growing numbers of requests and users’ demand for high performance - comes a new breed of modern development tools. A fast paced environment with rapid new deployments requires tracking errors and gaining insight to an application's behavior on a level traditional methods can’t sustain. In this post we’ve decided to gather 5 of those tools, see how they integrate with Java and find out what kind of tricks they have up their sleeves. It’s time to smash some bugs. Raygun Mindscape’s Raygun is a web based error management system that keeps track of exceptions coming from your apps. It supports various desktop, mobile and web programming languages, including Java, Scala, .NET, Python, PHP, and JavaScript. Besides that, sending errors to Raygun is possible through a REST API and a few more Providers (that’s how they call language and framework integrations) came to life thanks to developer community involvement. Key Features: Error grouping - Every occurrence of a bug is presented within one group with access to single instances of it, including its stack trace. Full text search - Error groups and all collected data is searchable. View app activity - Every action on an error group is displayed for all your team to see: status updates, comments and more. Affected users - Counts of affected users appear by each error. External integrations - Github, Bitbucket, Asana, JIRA, HipChat and many more. The Java angle: To use Raygun with Java, you’ll need to add some dependencies to your pom.xml file if you’re using Maven or add the jars manually. The second step would be to add an UncaughtExceptionHandler that would create an instance of RaygunClient and send your exceptions to it. In addition, you can also add custom data fields to your exceptions and send them together to Raygun. The full walkthrough is available here. Behind the curtain: Meet Robie Robot, the certified operator of Raygun. As in, the actual ray gun. Check it out on: https://raygun.io Sentry Started as a side-project, Sentry is an open-source web based solution that serves as a real time event logging and aggregation platform. It monitors errors and displays when, where and to whom they happen, promising to do so without relying solely on user feedback. Supported languages and frameworks include Ruby, Python, JS, Java, Django, iOS, .NET and more. Key Features: See the impact of new deployments in real time Provide support to specific users interrupted by an error Detect and thwart fraud as its attempted - notifications of unusual amounts of failures on purchases, authentication, and other sensitive areas External Integrations - GitHub, HipChat, Heroku, and many more The Java angle: Sentry’s Java client is called Raven and supports major existing logging frameworks like java.util.logging, Log4j, Log4j2 and Logback with Slf4j. An independent method to send events directly to Sentry is also available. To set up Sentry for Java with Logback for example, you’ll need to add the dependencies manually or through Maven, then add a new Sentry appender configuration and you’re good to do. Instructions are available here. Behind the curtain: Sentry was an internal project at Disqus back in 2010 to solve exception logging on a Django application by Chris Jennings and David Cramer Check it out on: https://www.getsentry.com/ Takipi Unlike most of the other tools, Takipi is far more than a stack trace prettifier. It was built with a simple objective in mind: Telling developers exactly when and why production code breaks. Whenever a new exception is thrown or a log error occurs – Takipi captures it and shows you the variable state which caused it, across methods and machines. Takipi will overlay this over the actual code which executed at the moment of error – so you can analyze the exception as if you were there when it happened. Key features: Detect – Caught/uncaught exceptions, Http and logged errors. Prioritize – How often errors happen across your cluster, if they involve new or modified code, and whether that rate is increasing. Analyze – See the actual code and variable state, even across different machines and applications. Easy to install - No code or configuration changes needed. Less than 2% overhead. The Java angle: Takipi was built for production environments in Java and Scala. The installation takes less than 1min, and includes attaching a Java agent to your JVM. Behind the curtain: Each exception type and error has a unique monster that represents it. You can find these monster here. Check it out on: http://www.takipi.com/ Airbrake Another tool that has put exception tracking on its eyesights is Rackspace’s Airbrake, taking on the mission of “No More Searching Log Files”. It provides users with a web based interface that includes a dashboard with error details and an application specific view. Supported languages include Ruby, PHP, Java, .NET, Python and even… Swift. Key Features: Detailed stack traces, grouping by error type, users and environment variables Team productivity - Filter importance errors from the noise Team collaboration - See who’s causing bugs and whose fixing them External Integrations - HipChat, GitHub, JIRA, Pivotal and over 30 more The Java angle: Airbrake officially supports only Log4j, although a Logback library is also available. Log4j2 support is currently lacking. The installation procedure is similar to Sentry, adding a few dependencies manually or through Maven, adding an appender, and you’re ready to start. Similarly, a direct way to send messages to Airbrake is also available with AirbrakeNotice and AirbrakeNotifier. More details are available here. Behind the curtain: Airbrake was acquired by Exceptional, which then got acquired by Rackspace. Check it out on: https://airbrake.io/ StackHunter Currently in beta, Stack Hunter provides a self hosted tool to track your Java exceptions. A change of scenery from the past hosted tools. Other than that, it aims to provide a similar feature set to inform developers of their exceptions and help solve them faster. Key Features: A single self hosted web interface to view all exceptions Collections of stack trace data and context including key metrics such as total exceptions, unique exceptions, users affected, & sessions affected Instant email alerts when exceptions occur Exceptions grouping by root cause The Java angle: Built specifically for Java, StackHunter runs on any servlet container running Java 6 or above. Installation includes running StackHunter on a local servlet, configuring an outgoing mail server for alerts, and configuring the application you’re wishing to log. Full instructions are available here. Behind the curtain: StackHunter is developed by Dele Taylor, who also works on Data Pipeline - a tool for transforming and migrating data in Java. Check it out on: http://stackhunter.com/ Bonus: ABRT Another approach to error tracking worth mentioning is used by ABRT, an automatic bug detection and reporting tool from the Fedora ecosystem, which is a Red Hat sponsored community project. Unlike the 5 tools we covered here, this one is intended to be used not only by app developers - but their users as well. Reporting bugs back to Red Hat with richer context that otherwise would have been harder to understand and debug. The Java angle: Support for Java exceptions is still in its proof of concept stage. A Java connector developed by Jakub Filák is available here. Behind the curtain: ABRT is an open-source project developed by Red Hat. Check it out on: https://github.com/abrt/abrt Did we miss any other tools? How do you keep track of your exceptions? Please let me know in the comments section below.

Overview Java 8 has added features such as lambdas and type inference. This makes the language less verbose and cleaner, however it comes with more side effects as you don't have to be as explicit in what you are doing. The return type of a lambda matters Java 8 infers the type of a closure. One way it does this is to look at the return type (or whether anything is returned) This can have a surprising side effect. Consider this code. ExecutorService es = Executors.newSingleThreadExecutor(); es.submit(() -> { try(Scanner scanner = new Scanner(new FileReader("file.txt"))) { String line = scanner.nextLine(); process(line); } return null; }); This code compiles fine. However, the line return null; appears redundant and you might be tempted to remove it. However if you remove the line, you get an error. Error:(12, 39) java: unreported exception java.io.FileNotFoundException; must be caught or declared to be thrown This is complaining about the use of FileReader. What has the return null got to do with catching an uncaught exception !? Type inference. ExecutorService.submit() is an overloaded method. It has two methods which take one argument. ExecutorService.submit(Runnable runnable); ExecutorService.submit(Callable callable); Both these methods take no arguments, so how does the javac compiler infer the type of the lambda? It looks at the return type. If you return null; it is aCallable however if nothing is returned, not even null, it is a Runnable. Callable and Runnable have another important difference. Callable throws checked exceptions, however Runnable doesn't allow checked exceptions to be thrown. The side effect of returning null is that you don't have to handle checked exceptions, these will be stored in the Future submit() returns. If you don't return anything, you have to handle checked exceptions. Conclusion While lambdas and type inference remove significant amounts of boiler plate code, you can find more edge cases, where the hidden details of what the compiler infers can be slightly confusing. Footnote You can be explicit about type inference with a cast. Consider this Callable calls = (Callable & Serializable) () -> { return null; } if (calls instanceof Serializable) // is true This cast has a number of side effects. Not only does the call() method return anInteger and a marker interface added, the code generated for the lambda changes i.e. it adds a writeObject() and readObject() method to support serialization of the lambda. Note: Each call site creates a new class meaning the details of this cast is visible at runtime via reflection.

This technical tip shows how to java developers can load and modify an existing email messages inside their java application using Aspose.Email Java API. Aspose.Email API allows developer to load any existing email message and modify its contents before saving back to the disk. One notable point is to specify the MessageFormat while loading the email message from the disk. In addition, it is important to specify the correct MailMessageSaveType while saving the message back to disk. The following sequence of steps lets you modify an existing email message: Create an instance of the MailMessage class. Load an existing message using the MailMessage class' load(), specifying the email' MessageFormat. Get the subject using the getSubject() method, modify it and set it using the MailMessage class' setSubject() method. Get the body using the getHtmlBody() method, modify it and set it using the MailMessage class' setHtmlBody() method. Create an instance of the MailAddressCollection class. Get recipients from the TO field into the MailAddressCollection object using the MailMessage class' getTo() method. Add or remove recipients using the MailAddressCollection collection's add() and remove() methods. Get recipients from the CC field into the MailAddressCollection object using the MailMessage class' getCC() method. Add or remove recipients using the MailAddressCollection collection's add() and remove() methods. Call the MailMessage class' save() method to save the file to disk in MSG format by specifying the correct MailMessageSaveType. //Adding Attachments to a New Email Message public static void main(String[] args) { // Base folder for reading and writing files String strBaseFolder = "D:\\Data\\Aspose\\resources\\"; //Initialize and Load an existing MSG file by specifying the MessageFormat MailMessage email = MailMessage.load(strBaseFolder + "anEmail.msg", MessageFormat.getMsg()); //Initialize a String variable to get the Email Subject String subject = email.getSubject(); //Append some more information to Subject subject = subject + " This text is added to the existing subject"; //Set the Email Subject email.setSubject(subject); //Initialize a String variable to get the Email's HTML Body String body = email.getHtmlBody(); //Apppend some more information to the Body variable body = body + " This text is added to the existing body"; //Set the Email Body email.setHtmlBody(body); //Initialize MailAddressCollection object MailAddressCollection contacts = new MailAddressCollection(); //Retrieve Email's TO list contacts = email.getTo(); //Check if TO list has some values if (contacts.size() > 0) { //Remove the first email address contacts.remove(0); //Add another email address to collection contacts.add("[email protected]"); } //Set the collection as Email's TO list email.setTo(contacts); //Initialize MailAddressCollection contacts = new MailAddressCollection(); //Retrieve Email's CC list contacts = email.getCC(); //Add another email address to collection contacts.add("[email protected]"); //Set the collection as Email's CC list email.setCC(contacts); //Save the Email message to disk by specifying the MessageFormat email.save(strBaseFolder + "message.msg", MailMessageSaveType.getOutlookMessageFormat()); } //Loading a Message with Load Options //To load a message with specific load options, Aspose.Email provides the MessageLoadOptions class that can be used as follow: MesageLoadOptions options = new MesageLoadOptions(); options.PrefferedTextEncoding = Encoding.getEncoding(1252); options.setMessageFormat(MessageFormat.getMsg()); MailMessage eml = MailMessage.Load("EMAIL_497563\\test3.msg", options);

Now that you know the basics of Hibernate dirty checking, we can dig into enhanced dirty checking mechanisms.

The 4 Java Garbage Collectors - How the Wrong Choice Dramatically Impacts Performance The year is 2014 and there are two things that still remain a mystery to most developers - Garbage collection and understanding the opposite sex. Since I don’t know much about the latter, I thought I’d take a whack at the former, especially as this is an area that has seen some major changes and improvements with Java 8, especially with the removal of the PermGen and some new and exciting optimizations (more on this towards the end). When we speak about garbage collection, the vast majority of us know the concept and employ it in our everyday programming. Even so, there’s much about it we don’t understand, and that’s when things get painful. One of the biggest misconceptions about the JVM is that it has one garbage collector, where in fact it provides four different ones, each with its own unique advantages and disadvantages. The choice of which one to use isn’t automatic and lies on your shoulders and the differences in throughput and application pauses can be dramatic. What’s common about these four garbage collection algorithms is that they are generational, which means they split the managed heap into different segments, using the age-old assumptions that most objects in the heap are short lived and should be recycled quickly. As this too is a well-covered area, I’m going to jump directly into the different algorithms, along with their pros and their cons. 1. The Serial Collector The serial collector is the simplest one, and the one you probably won’t be using, as it’s mainly designed for single-threaded environments (e.g. 32 bit or Windows) and for small heaps. This collector freezes all application threads whenever it’s working, which disqualifies it for all intents and purposes from being used in a server environment. How to use it: You can use it by turning on the -XX:+UseSerialGC JVM argument, 2. The Parallel / Throughput collector Next off is the Parallel collector. This is the JVM’s default collector. Much like its name, its biggest advantage is that is uses multiple threads to scan through and compact the heap. The downside to the parallel collector is that it will stop application threads when performing either a minor or full GC collection. The parallel collector is best suited for apps that can tolerate application pauses and are trying to optimize for lower CPU overhead caused by the collector. 3. The CMS Collector Following up on the parallel collector is the CMS collector (“concurrent-mark-sweep”). This algorithm uses multiple threads (“concurrent”) to scan through the heap (“mark”) for unused objects that can be recycled (“sweep”). This algorithm will enter “stop the world” (STW) mode in two cases: when initializing the initial marking of roots (objects in the old generation that are reachable from thread entry points or static variables) and when the application has changed the state of the heap while the algorithm was running concurrently, forcing it to go back and do some final touches to make sure it has the right objects marked. The biggest concern when using this collector is encountering promotion failures which are instances where a race condition occurs between collecting the young and old generations. If the collector needs to promote young objects to the old generation, but hasn’t had enough time to make space clear it, it will have to do so first which will result in a full STW collection - the very thing this CMS collector was meant to prevent. To make sure this doesn’t happen you would either increase the size of the old generation (or the entire heap for that matter) or allocate more background threads to the collector for him to compete with the rate of object allocation. Another downside to this algorithm in comparison to the parallel collector is that it uses more CPU in order to provide the application with higher levels of continuous throughput, by using multiple threads to perform scanning and collection. For most long-running server applications which are adverse to application freezes, that’s usually a good trade off to make. Even so, this algorithm is not on by default. You have to specify XX:+USeParNewGC to actually enable it. If you’re willing to allocate more CPU resources to avoid application pauses this is the collector you’ll probably want to use, assuming that your heap is less than 4Gb in size. However, if it’s greater than 4GB, you’ll probably want to use the last algorithm - the G1 Collector. 4. The G1 Collector The Garbage first collector (G1) introduced in JDK 7 update 4 was designed to better support heaps larger than 4GB. The G1 collector utilizes multiple background threads to scan through the heap that it divides into regions, spanning from 1MB to 32MB (depending on the size of your heap). G1 collector is geared towards scanning those regions that contain the most garbage objects first, giving it its name (Garbage first). This collector is turned on using the –XX:+UseG1GC flag. This strategy the chance of the heap being depleted before background threads have finished scanning for unused objects, in which case the collector will have to stop the application which will result in a STW collection. The G1 also has another advantage that is that it compacts the heap on-the-go, something the CMS collector only does during full STW collections. Large heaps have been a fairly contentious area over the past few years with many developers moving away from the single JVM per machine model to more micro-service, componentized architectures with multiple JVMs per machine. This has been driven by many factors including the desire to isolate different application parts, simplifying deployment and avoiding the cost which would usually come with reloading application classes into memory (something which has actually been improved in Java 8). Even so, one of the biggest drivers to do this when it comes to the JVM stems from the desire to avoid those long “stop the world” pauses (which can take many seconds in a large collection) that occur with large heaps. This has also been accelerated by container technologies like Docker that enable you to deploy multiple apps on the same physical machine with relative ease. Java 8 and the G1 Collector Another beautiful optimization which was just out with Java 8 update 20 for is the G1 Collector String deduplication. Since strings (and their internal char[] arrays) takes much of our heap, a new optimization has been made that enables the G1 collector to identify strings which are duplicated more than once across your heap and correct them to point into the same internal char[] array, to avoid multiple copies of the same string from residing inefficiently within the heap. You can use the -XX:+UseStringDeduplicationJVM argument to try this out. Java 8 and PermGen One of the biggest changes made in Java 8 was removing the permgen part of the heap that was traditionally allocated for class meta-data, interned strings and static variables. This would traditionally require developers with applications that would load significant amount of classes (something common with apps using enterprise containers) to optimize and tune for this portion of the heap specifically. This has over the years become the source of many OutOfMemory exceptions, so having the JVM (mostly) take care if it is a very nice addition. Even so, that in itself will probably not reduce the tide of developers decoupling their apps into multiple JVMs. Each of these collectors is configured and tuned differently with a slew of toggles and switches, each with the potential to increase or decrease throughput, all based on the specific behavior of your app. We’ll delve into the key strategies of configuring each of these in our next posts.

JSF is not what we often think it is. It's also a framework that can be somewhat tricky to debug, especially when first encountered. In this post let's go over on why that is and provide some JSF debugging techniques. We will go through the following topics: JSF is not what we often think The difficulties of JSF debugging How to debug JSF systematically How JSF Works - The JSF lifecycle Debugging an Ajax request from browser to server and back Debugging the JSF frontend Javascript code Final thoughts - alternatives? (questions to the reader) JSF is not what we often think JSF looks on first look like an enterprise Java/XML frontend framework, but under the hood it really isn't. It's really a polyglot Java/Javascript framework, where the client Javascript part is non-neglectable and also important to understand it. It also has good support for direct HTML/CSS use. JSF developers are on ocasion already polyglot developers, whose primary language is Java but still need to use ocasionally Javascript. The difficulties of JSF debugging When comparing JSF to GWT and AngularJS in a previous post, I found that the (most often used) approach that the framework takes of abstracting HTML and CSS from the developer behind XML adds to the difficulty of debugging, because it creates an extra level of indirection. A more direct approach of using HTML/CSS directly is also possible, but it seems enterprise Java developers tend to stick to XML in most cases, because it's a more familiar technology. Also another problem is that the client side Javascript part of the framework/libraries is not very well documented, and it's often important to understand what is going on. The only way to debug JSF systematically When first encountering JSF, I first tried to approach it from a Java, XML and documentation only. While I could do a part of the work that way, there where frequent situations where that approach was really not sufficient. The conclusion that I got to is that in order to be able to debug JSF applications effectively, an understanding of the following is needed: HTML CSS Javascript HTTP Chrome Dev Tools, Firebug or equivalent The JSF Lifecycle This might sound surprising to developers that work mostly in Java/XML, but this web-centric approach to debugging JSF is the only way that I managed to tackle many requirements that needed some significant component customization, or to be able to fix certain bugs. Let’s start by understanding the inner workings of JSF, so that we can debug it better. The JSF take on MVC The way JSF approaches MVC is that the whole 3 components reside on the server side: The Model is a tree of plain Java objects The View is a server side template defined in XML that is read to build an in-memory view definition The Controller is a Java servlet, that receives each request and processes them through a series of steps The browser is assumed to be simply a rendering engine for the HTML generated at server side. Ajax is achieved by submitting parts of the page for server processing, and requesting a server to ‘repaint’ only portions of the screen, without navigating away from the page. The JSF Lifecycle Once an HTTP request reaches the backend, it gets caught by the JSF Controller that will then process it. The request goes through a series of phases known as the JSF lifecycle, which is essential to understand how JSF works: Design Goals of the JSF Lifecycle The whole point of the lifecycle is to manage MVC 100% on the server side, using the browser as a rendering platform only. The initial idea was to decouple the rendering platform from the server-side UI component model, in order to allow to replace HTML with alternative markup languages by swapping the Render Response phase. This was in the early 2000's when HTML could be soon replaced by XML-based alternatives (that never came to be), and then HTML5 came along. Also browsers where much more qwirkier than what they are today, and the idea of cross-browser Javascript libraries was not widespread. So let’s go through each phase and see how to debug it if needed, starting in the browser. Let's base ourselves in a simple example that uses an Ajax request. A JSF 2 Hello World Example The following is a minimal JSF 2 page, that receives an input text from the user, sends the text via an Ajax request to the backend and refreshes only an output label: JSF 2.2 Hello World Example The page looks like this: Following one Ajax request - to the server and back Let’s click submit in order to trigger the Ajax request, and use the Chrome Dev Tools Network tab (right click and inspect any element on the page).What goes over the wire? This is what we see in the Form Data section of the request: j_idt8:input: Hello World javax.faces.ViewState: -2798727343674530263:954565149304692491 javax.faces.source: j_idt8:j_idt9 javax.faces.partial.event: click javax.faces.partial.execute: j_idt8:j_idt9 j_idt8:input javax.faces.partial.render: j_idt8:output javax.faces.behavior.event: action javax.faces.partial.ajax:true This request says: The new value of the input field is "Hello World", send me a new value for the output field only, and don't navigate away from this page. Let's see how this can be read from the request. As we can see, the new values of the form are submitted to the server, namely the “Hello World” value. This is the meaning of the several entries: javax.faces.ViewState identifies the view from which the request was made. The request is an Ajax request, as indicated by the flag javax.faces.partial.ajax, The request was triggered by a click as defined in javax.faces.partial.event. But what are those j_ strings ? Those are space separated generated identifiers of HTML elements. For example this is how we can see what is the page element corresponding to j_idt8:input, using the Chrome Dev Tools: There are also 3 extra form parameters that use these identifiers, that are linked to UI components: javax.faces.source: The identifier of the HTML element that originated this request, in this case the Id of the submit button. javax.faces.execute: The list of identifiers of the elements whose values are sent to the server for processing, in this case the input text field. javax.faces.render: The list of identifiers of the sections of the page that are to be ‘repainted', in this case the output field only. But what happens when the request hits the server ? JSF lifecycle - Restore View Phase Once the request reaches the server, the JSF controller will inspect the javax.faces.ViewState and identify to which view it refers. It will then build or restore a Java representation of the view, that is somehow similar to the document definition in the browser side. The view will be attached to the request and used throughout. There is usually little need to debug this phase during application development. JSF Lifecycle - Apply Request Values The JSF Controller will then apply to the view widgets the new values received via the request. The values might be invalid at this point. Each JSF component gets a call to it’s decode method in this phase. This method will retrieve the submitted value for the widget in question from the HTTP request and store it on the widget itself. To debug this, let’s put a breakpoint in the decode method of the HtmlInputText class, to see the value “Hello World”: Notice the conditional breakpoint using the HTML clientId of the field we want. This would allow to quickly debug only the decoding of the component we want, even in a large page with many other similar widgets. Next after decoding is the validation phase. JSF Lifecycle - Process Validations In this phase, validations are applied and if the value is found to be in error (for example a date is invalid), then the request bypasses Invoke Application and goes directly to Render Response phase. To debug this phase, a similar breakpoint can be put on method processValidators, or in the validators themselves if you happen to know which ones or if they are custom. JSF Lifecycle - Update Model In this phase, we know all the submitted values where correct. JSF can now update the view model by applying the new values received in the requests to the plain Java objects in the view model. This phase can be debugged by putting a breakpoint in the processUpdates method of the component in question, eventually using a similar conditional breakpoint to break only on the component needed. JSF Lifecycle - Invoke Application This is the simplest phase to debug. The application now has an updated view model, and some logic can be applied on it. This is where the action listeners defined in the XML view definition (the 'action' properties and the listener tags) are executed. JSF Lifecycle - Render Response This is the phase that I end up debugging the most: why is the value not being displayed as we expect it, etc, it all can be found here. In this phase the view and the new model values will be transformed from Java objects into HTML, CSS and eventually Javascript and sent back over the wire to the browser. This phase can be debugged using breakpoints in the encodeBegin, encodeChildren and encodeEnd methods of the component in question. The components will either render themselves or delegate rendering to aRenderer class. Back in the browser It was a long trip, but we are back where we started! This is how the response generated by JSF looks once received in the browser: -8188482707773604502:6956126859616189525> What the Javascript part of the framework will do is to take the contents of the partial response, update by update. Using the Id of the update, the client side JSF callback will search for a component with that Id, delete it from the document and replace it with the new updated version. In this case, "Hello World" will show up on the label next to the Input text field! And so thats how JSF works under the hood. But what about if we need to debug the Javascript part of the framework? Debugging the JSF Javascript Code The Chrome Dev Tools can help debug the client part. For example let’s say that we want to halt the client when an Ajax request is triggered. We need to go to the sources tab, add an XHR (Ajax) breakpoint and trigger the browser action. The debugger will stop and the call stack can be examined: For some frameworks like Primefaces, the Javascript sources might be minified (non human-readable) because they are optimized for size. To solve this, download the source code of the library and do a non minified build of the jar. There are usually instructions for this, otherwise check the project poms. This will install in your Maven repository a jar with non minified sources for debugging. The UI Debug tag: The ui:debug tag allows to view a lot of debugging information using a keyboard shortcut, see here for further details. Final Thoughts JSF is very popular in the enterprise Java world, and it handles a lot of problems well, specially if the UI designers take into account the possibilities of the widget library being used. The problem is that there are usually feature requests that force us to dig deeper into the widgets internal implementation in order to customize them, and this requires HTML, CSS, Javascript and HTTP plus JSF lifecycle knowledge. Is polyglot an alternative? We can wonder that if developers have to know a fair amount about web technologies in order to be able to debug JSF effectively, then it would be simpler to build enterprise front ends (just the client part) using those technologies directly instead. It's possible that a polyglot approach of a Java backend plus a Javascript-only frontend could be proved effective in a nearby future, specially using some sort of a client side MVC framework like Angular. This would require learning more Javascript, (have a look at Javascript for Java developers post if curious), but this is already often necessary to do custom widget development in JSF anyway. Conclusions and some questions to the reader Thanks for reading, please take a moment to share your thoughts on these matters on the comments bellow: do you believe polyglot development (Java/Javascript) is a viable alternative in general, and in your workplace in particular? Did you find one of the GWT-based frameworks (plain GWT, Vaadin, Errai), or the Play Framework to be easier to use and of better productivity?

I’ve been working on migrating some batch jobs for Podcastpedia.org to Spring Batch. Before, these jobs were developed in my own kind of way, and I thought it was high time to use a more “standardized” approach. Because I had never used Spring with java configuration before, I thought this were a good opportunity to learn about it, by configuring the Spring Batch jobs in java. And since I am all into trying new things with Spring, why not also throw Spring Boot into the boat… Before you begin with this tutorial I recommend you read first Spring’s Getting started – Creating a Batch Service, because the structure and the code presented here builds on that original. 1. What I’ll build So, as mentioned, in this post I will present Spring Batch in the context of configuring it and developing with it some batch jobs for Podcastpedia.org. Here’s a short description of the two jobs that are currently part of the Podcastpedia-batch project: addNewPodcastJob reads podcast metadata (feed url, identifier, categories etc.) from a flat file transforms (parses and prepares episodes to be inserted with Http Apache Client) the data and in the last step, insert it to the Podcastpedia database and inform the submitter via emailabout it notifyEmailSubscribersJob – people can subscribe to their favorite podcasts on Podcastpedia.orgvia email. For those who did it is checked on a regular basis (DAILY, WEEKLY, MONTHLY) if new episodes are available, and if they are the subscribers are informed via email about those; read from database, expand read data via JPA, re-group it and notify subscriber via email Source code: The source code for this tutorial is available on GitHub – Podcastpedia-batch. Note: Before you start I also highly recommend you read the Domain Language of Batch, so that terms like “Jobs”, “Steps” or “ItemReaders” don’t sound strange to you. 2. What you’ll need A favorite text editor or IDE JDK 1.7 or later Maven 3.0+ 3. Set up the project The project is built with Maven. It uses Spring Boot, which makes it easy to create stand-alone Spring based Applications that you can “just run”. You can learn more about the Spring Boot by visiting theproject’s website. 3.1. Maven build file Because it uses Spring Boot it will have the spring-boot-starter-parent as its parent, and a couple of other spring-boot-starters that will get for us some libraries required in the project: pom.xml of the podcastpedia-batch project 4.0.0 org.podcastpedia.batch podcastpedia-batch 0.1.0 1.1.6.RELEASE 1.7 org.springframework.boot spring-boot-starter-parent 1.1.6.RELEASE org.springframework.boot spring-boot-starter-batch org.springframework.boot spring-boot-starter-data-jpa org.apache.httpcomponents httpclient 4.3.5 org.apache.httpcomponents httpcore 4.3.2 org.apache.velocity velocity 1.7 org.apache.velocity velocity-tools 2.0 org.apache.struts struts-core rome rome 1.0 rome rome-fetcher 1.0 org.jdom jdom 1.1 xerces xercesImpl 2.9.1 mysql mysql-connector-java 5.1.31 org.springframework.boot spring-boot-starter-freemarker org.springframework.boot spring-boot-starter-remote-shell javax.mail mail javax.mail mail 1.4.7 javax.inject javax.inject 1 org.twitter4j twitter4j-core [4.0,) org.springframework.boot spring-boot-starter-test maven-compiler-plugin org.springframework.boot spring-boot-maven-plugin Note: One big advantage of using the spring-boot-starter-parent as the project’s parent is that you only have to upgrade the version of the parent and it will get the “latest” libraries for you. When I started the project spring boot was in version 1.1.3.RELEASE and by the time of finishing to write this post is already at 1.1.6.RELEASE. 3.2. Project directory structure I structured the project in the following way: └── src └── main └── java └── org └── podcastpedia └── batch └── common └── jobs └── addpodcast └── notifysubscribers Note: the org.podcastpedia.batch.jobs package contains sub-packages having specific classes to particular jobs. the org.podcastpedia.batch.jobs.common package contains classes used by all the jobs, like for example the JPA entities that both the current jobs require. 4. Create a batch Job configuration I will start by presenting the Java configuration class for the first batch job: package org.podcastpedia.batch.jobs.addpodcast; import org.podcastpedia.batch.common.configuration.DatabaseAccessConfiguration; import org.podcastpedia.batch.common.listeners.LogProcessListener; import org.podcastpedia.batch.common.listeners.ProtocolListener; import org.podcastpedia.batch.jobs.addpodcast.model.SuggestedPodcast; import org.springframework.batch.core.Job; import org.springframework.batch.core.Step; import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; import org.springframework.batch.item.ItemProcessor; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemWriter; import org.springframework.batch.item.file.FlatFileItemReader; import org.springframework.batch.item.file.LineMapper; import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper; import org.springframework.batch.item.file.mapping.DefaultLineMapper; import org.springframework.batch.item.file.transform.DelimitedLineTokenizer; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Import; import org.springframework.core.io.ClassPathResource; import com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException; @Configuration @EnableBatchProcessing @Import({DatabaseAccessConfiguration.class, ServicesConfiguration.class}) public class AddPodcastJobConfiguration { @Autowired private JobBuilderFactory jobs; @Autowired private StepBuilderFactory stepBuilderFactory; // tag::jobstep[] @Bean public Job addNewPodcastJob(){ return jobs.get("addNewPodcastJob") .listener(protocolListener()) .start(step()) .build(); } @Bean public Step step(){ return stepBuilderFactory.get("step") .chunk(1) //important to be one in this case to commit after every line read .reader(reader()) .processor(processor()) .writer(writer()) .listener(logProcessListener()) .faultTolerant() .skipLimit(10) //default is set to 0 .skip(MySQLIntegrityConstraintViolationException.class) .build(); } // end::jobstep[] // tag::readerwriterprocessor[] @Bean public ItemReader reader(){ FlatFileItemReader reader = new FlatFileItemReader(); reader.setLinesToSkip(1);//first line is title definition reader.setResource(new ClassPathResource("suggested-podcasts.txt")); reader.setLineMapper(lineMapper()); return reader; } @Bean public LineMapper lineMapper() { DefaultLineMapper lineMapper = new DefaultLineMapper(); DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer(); lineTokenizer.setDelimiter(";"); lineTokenizer.setStrict(false); lineTokenizer.setNames(new String[]{"FEED_URL", "IDENTIFIER_ON_PODCASTPEDIA", "CATEGORIES", "LANGUAGE", "MEDIA_TYPE", "UPDATE_FREQUENCY", "KEYWORDS", "FB_PAGE", "TWITTER_PAGE", "GPLUS_PAGE", "NAME_SUBMITTER", "EMAIL_SUBMITTER"}); BeanWrapperFieldSetMapper fieldSetMapper = new BeanWrapperFieldSetMapper(); fieldSetMapper.setTargetType(SuggestedPodcast.class); lineMapper.setLineTokenizer(lineTokenizer); lineMapper.setFieldSetMapper(suggestedPodcastFieldSetMapper()); return lineMapper; } @Bean public SuggestedPodcastFieldSetMapper suggestedPodcastFieldSetMapper() { return new SuggestedPodcastFieldSetMapper(); } /** configure the processor related stuff */ @Bean public ItemProcessor processor() { return new SuggestedPodcastItemProcessor(); } @Bean public ItemWriter writer() { return new Writer(); } // end::readerwriterprocessor[] @Bean public ProtocolListener protocolListener(){ return new ProtocolListener(); } @Bean public LogProcessListener logProcessListener(){ return new LogProcessListener(); } } The @EnableBatchProcessing annotation adds many critical beans that support jobs and saves us configuration work. For example you will also be able to @Autowired some useful stuff into your context: a JobRepository (bean name “jobRepository”) a JobLauncher (bean name “jobLauncher”) a JobRegistry (bean name “jobRegistry”) a PlatformTransactionManager (bean name “transactionManager”) a JobBuilderFactory (bean name “jobBuilders”) as a convenience to prevent you from having to inject the job repository into every job, as in the examples above a StepBuilderFactory (bean name “stepBuilders”) as a convenience to prevent you from having to inject the job repository and transaction manager into every step The first part focuses on the actual job configuration: @Bean public Job addNewPodcastJob(){ return jobs.get("addNewPodcastJob") .listener(protocolListener()) .start(step()) .build(); } @Bean public Step step(){ return stepBuilderFactory.get("step") .chunk(1) //important to be one in this case to commit after every line read .reader(reader()) .processor(processor()) .writer(writer()) .listener(logProcessListener()) .faultTolerant() .skipLimit(10) //default is set to 0 .skip(MySQLIntegrityConstraintViolationException.class) .build(); } The first method defines a job and the second one defines a single step. As you’ve read in The Domain Language of Batch, jobs are built from steps, where each step can involve a reader, a processor, and a writer. In the step definition, you define how much data to write at a time (in our case 1 record at a time). Next you specify the reader, processor and writer. 5. Spring Batch processing units Most of the batch processing can be described as reading data, doing some transformation on it and then writing the result out. This mirrors somehow the Extract, Transform, Load (ETL) process, in case you know more about that. Spring Batch provides three key interfaces to help perform bulk reading and writing: ItemReader, ItemProcessor and ItemWriter. 5.1. Readers ItemReader is an abstraction providing the mean to retrieve data from many different types of input: flat files, xml files, database, jms etc., one item at a time. See the Appendix A. List of ItemReaders and ItemWriters for a complete list of available item readers. In the Podcastpedia batch jobs I use the following specialized ItemReaders: 5.1.1. FlatFileItemReader which, as the name implies, reads lines of data from a flat file that typically describe records with fields of data defined by fixed positions in the file or delimited by some special character (e.g. Comma). This type of ItemReader is being used in the first batch job, addNewPodcastJob. The input file used is named suggested-podcasts.in, resides in the classpath (src/main/resources) and looks something like the following: FEED_URL; IDENTIFIER_ON_PODCASTPEDIA; CATEGORIES; LANGUAGE; MEDIA_TYPE; UPDATE_FREQUENCY; KEYWORDS; FB_PAGE; TWITTER_PAGE; GPLUS_PAGE; NAME_SUBMITTER; EMAIL_SUBMITTER http://www.5minutebiographies.com/feed/; 5minutebiographies; people_society, history; en; Audio; WEEKLY; biography, biographies, short biography, short biographies, 5 minute biographies, five minute biographies, 5 minute biography, five minute biography; https://www.facebook.com/5minutebiographies;https://twitter.com/5MinuteBios; ; Adrian Matei; [email protected] http://notanotherpodcast.libsyn.com/rss; NotAnotherPodcast; entertainment; en; Audio; WEEKLY; Comedy, Sports, Cinema, Movies, Pop Culture, Food, Games; https://www.facebook.com/notanotherpodcastusa;https://twitter.com/NAPodcastUSA;https://plus.google.com/u/0/103089891373760354121/posts; Adrian Matei; [email protected] As you can see the first line defines the names of the “columns”, and the following lines contain the actual data (delimited by “;”), that needs translating to domain objects relevant in the context. Let’s see now how to configure the FlatFileItemReader: @Bean public ItemReader reader(){ FlatFileItemReader reader = new FlatFileItemReader(); reader.setLinesToSkip(1);//first line is title definition reader.setResource(new ClassPathResource("suggested-podcasts.in")); reader.setLineMapper(lineMapper()); return reader; } You can specify, among other things, the input resource, the number of lines to skip, and a line mapper. 5.1.1.1. LineMapper The LineMapper is an interface for mapping lines (strings) to domain objects, typically used to map lines read from a file to domain objects on a per line basis. For the Podcastpedia job I used the DefaultLineMapper, which is two-phase implementation consisting of tokenization of the line into a FieldSet followed by mapping to item: @Bean public LineMapper lineMapper() { DefaultLineMapper lineMapper = new DefaultLineMapper(); DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer(); lineTokenizer.setDelimiter(";"); lineTokenizer.setStrict(false); lineTokenizer.setNames(new String[]{"FEED_URL", "IDENTIFIER_ON_PODCASTPEDIA", "CATEGORIES", "LANGUAGE", "MEDIA_TYPE", "UPDATE_FREQUENCY", "KEYWORDS", "FB_PAGE", "TWITTER_PAGE", "GPLUS_PAGE", "NAME_SUBMITTER", "EMAIL_SUBMITTER"}); BeanWrapperFieldSetMapper fieldSetMapper = new BeanWrapperFieldSetMapper(); fieldSetMapper.setTargetType(SuggestedPodcast.class); lineMapper.setLineTokenizer(lineTokenizer); lineMapper.setFieldSetMapper(suggestedPodcastFieldSetMapper()); return lineMapper; } the DelimitedLineTokenizer splits the input String via the “;” delimiter. if you set the strict flag to false then lines with less tokens will be tolerated and padded with empty columns, and lines with more tokens will simply be truncated. the columns names from the first line are set lineTokenizer.setNames(...); and the fieldMapper is set (line 14) Note: The FieldSet is an “interface used by flat file input sources to encapsulate concerns of converting an array of Strings to Java native types. A bit like the role played by ResultSet in JDBC, clients will know the name or position of strongly typed fields that they want to extract.“ 5.1.1.2. FieldSetMapper The FieldSetMapper is an interface that is used to map data obtained from a FieldSet into an object. Here’s my implementation which maps the fieldSet to the SuggestedPodcast domain object that will be further passed to the processor: public class SuggestedPodcastFieldSetMapper implements FieldSetMapper { @Override public SuggestedPodcast mapFieldSet(FieldSet fieldSet) throws BindException { SuggestedPodcast suggestedPodcast = new SuggestedPodcast(); suggestedPodcast.setCategories(fieldSet.readString("CATEGORIES")); suggestedPodcast.setEmail(fieldSet.readString("EMAIL_SUBMITTER")); suggestedPodcast.setName(fieldSet.readString("NAME_SUBMITTER")); suggestedPodcast.setTags(fieldSet.readString("KEYWORDS")); //some of the attributes we can map directly into the Podcast entity that we'll insert later into the database Podcast podcast = new Podcast(); podcast.setUrl(fieldSet.readString("FEED_URL")); podcast.setIdentifier(fieldSet.readString("IDENTIFIER_ON_PODCASTPEDIA")); podcast.setLanguageCode(LanguageCode.valueOf(fieldSet.readString("LANGUAGE"))); podcast.setMediaType(MediaType.valueOf(fieldSet.readString("MEDIA_TYPE"))); podcast.setUpdateFrequency(UpdateFrequency.valueOf(fieldSet.readString("UPDATE_FREQUENCY"))); podcast.setFbPage(fieldSet.readString("FB_PAGE")); podcast.setTwitterPage(fieldSet.readString("TWITTER_PAGE")); podcast.setGplusPage(fieldSet.readString("GPLUS_PAGE")); suggestedPodcast.setPodcast(podcast); return suggestedPodcast; } } 5.2. JdbcCursorItemReader In the second job, notifyEmailSubscribersJob, in the reader, I only read email subscribers from a single database table, but further in the processor a more detailed read(via JPA) is executed to retrieve all the new episodes of the podcasts the user subscribed to. This is a common pattern employed in the batch world. Follow this link for more Common Batch Patterns. For the initial read, I chose the JdbcCursorItemReader, which is a simple reader implementation that opens a JDBC cursor and continually retrieves the next row in the ResultSet: @Bean public ItemReader notifySubscribersReader(){ JdbcCursorItemReader reader = new JdbcCursorItemReader(); String sql = "select * from users where is_email_subscriber is not null"; reader.setSql(sql); reader.setDataSource(dataSource); reader.setRowMapper(rowMapper()); return reader; } Note I had to set the sql, the datasource to read from and a RowMapper. 5.2.1. RowMapper The RowMapper is an interface used by JdbcTemplate for mapping rows of a Result’set on a per-row basis. My implementation of this interface, , performs the actual work of mapping each row to a result object, but I don’t need to worry about exception handling: public class UserRowMapper implements RowMapper { @Override public User mapRow(ResultSet rs, int rowNum) throws SQLException { User user = new User(); user.setEmail(rs.getString("email")); return user; } } 5.2. Writers ItemWriter is an abstraction that represents the output of a Step, one batch or chunk of items at a time. Generally, an item writer has no knowledge of the input it will receive next, only the item that was passed in its current invocation. The writers for the two jobs presented are quite simple. They just use external services to send email notifications and post tweets on Podcastpedia’s account. Here is the implementation of the ItemWriterfor the first job – addNewPodcast: package org.podcastpedia.batch.jobs.addpodcast; import java.util.Date; import java.util.List; import javax.inject.Inject; import javax.persistence.EntityManager; import org.podcastpedia.batch.common.entities.Podcast; import org.podcastpedia.batch.jobs.addpodcast.model.SuggestedPodcast; import org.podcastpedia.batch.jobs.addpodcast.service.EmailNotificationService; import org.podcastpedia.batch.jobs.addpodcast.service.SocialMediaService; import org.springframework.batch.item.ItemWriter; import org.springframework.beans.factory.annotation.Autowired; public class Writer implements ItemWriter{ @Autowired private EntityManager entityManager; @Inject private EmailNotificationService emailNotificationService; @Inject private SocialMediaService socialMediaService; @Override public void write(List items) throws Exception { if(items.get(0) != null){ SuggestedPodcast suggestedPodcast = items.get(0); //first insert the data in the database Podcast podcast = suggestedPodcast.getPodcast(); podcast.setInsertionDate(new Date()); entityManager.persist(podcast); entityManager.flush(); //notify submitter about the insertion and post a twitt about it String url = buildUrlOnPodcastpedia(podcast); emailNotificationService.sendPodcastAdditionConfirmation( suggestedPodcast.getName(), suggestedPodcast.getEmail(), url); if(podcast.getTwitterPage() != null){ socialMediaService.postOnTwitterAboutNewPodcast(podcast, url); } } } private String buildUrlOnPodcastpedia(Podcast podcast) { StringBuffer urlOnPodcastpedia = new StringBuffer( "http://www.podcastpedia.org"); if (podcast.getIdentifier() != null) { urlOnPodcastpedia.append("/" + podcast.getIdentifier()); } else { urlOnPodcastpedia.append("/podcasts/"); urlOnPodcastpedia.append(String.valueOf(podcast.getPodcastId())); urlOnPodcastpedia.append("/" + podcast.getTitleInUrl()); } String url = urlOnPodcastpedia.toString(); return url; } } As you can see there’s nothing special here, except that the write method has to be overriden and this is where the injected external services EmailNotificationService and SocialMediaService are used to inform via email the podcast submitter about the addition to the podcast directory, and if a Twitter page was submitted a tweet will be posted on the Podcastpedia’s wall. You can find detailed explanation on how to send email via Velocity and how to post on Twitter from Java in the following posts: How to compose html emails in Java with Spring and Velocity How to post to Twittter from Java with Twitter4J in 10 minutes 5.3. Processors ItemProcessor is an abstraction that represents the business processing of an item. While theItemReader reads one item, and the ItemWriter writes them, the ItemProcessor provides access to transform or apply other business processing. When using your own Processors you have to implement the ItemProcessor interface, with its only method O process(I item) throws Exception, returning a potentially modified or a new item for continued processing. If the returned result is null, it is assumed that processing of the item should not continue. While the processor of the first job requires a little bit of more logic, because I have to set the etag andlast-modified header attributes, the feed attributes, episodes, categories and keywords of the podcast: public class SuggestedPodcastItemProcessor implements ItemProcessor { private static final int TIMEOUT = 10; @Autowired ReadDao readDao; @Autowired PodcastAndEpisodeAttributesService podcastAndEpisodeAttributesService; @Autowired private PoolingHttpClientConnectionManager poolingHttpClientConnectionManager; @Autowired private SyndFeedService syndFeedService; /** * Method used to build the categories, tags and episodes of the podcast */ @Override public SuggestedPodcast process(SuggestedPodcast item) throws Exception { if(isPodcastAlreadyInTheDirectory(item.getPodcast().getUrl())) { return null; } String[] categories = item.getCategories().trim().split("\\s*,\\s*"); item.getPodcast().setAvailability(org.apache.http.HttpStatus.SC_OK); //set etag and last modified attributes for the podcast setHeaderFieldAttributes(item.getPodcast()); //set the other attributes of the podcast from the feed podcastAndEpisodeAttributesService.setPodcastFeedAttributes(item.getPodcast()); //set the categories List categoriesByNames = readDao.findCategoriesByNames(categories); item.getPodcast().setCategories(categoriesByNames); //set the tags setTagsForPodcast(item); //build the episodes setEpisodesForPodcast(item.getPodcast()); return item; } ...... } the processor from the second job uses the ‘Driving Query’ approach, where I expand the data retrieved from the Reader with another “JPA-read” and I group the items on podcasts with episodes so that it looks nice in the emails that I am sending out to subscribers: @Scope("step") public class NotifySubscribersItemProcessor implements ItemProcessor { @Autowired EntityManager em; @Value("#{jobParameters[updateFrequency]}") String updateFrequency; @Override public User process(User item) throws Exception { String sqlInnerJoinEpisodes = "select e from User u JOIN u.podcasts p JOIN p.episodes e WHERE u.email=?1 AND p.updateFrequency=?2 AND" + " e.isNew IS NOT NULL AND e.availability=200 ORDER BY e.podcast.podcastId ASC, e.publicationDate ASC"; TypedQuery queryInnerJoinepisodes = em.createQuery(sqlInnerJoinEpisodes, Episode.class); queryInnerJoinepisodes.setParameter(1, item.getEmail()); queryInnerJoinepisodes.setParameter(2, UpdateFrequency.valueOf(updateFrequency)); List newEpisodes = queryInnerJoinepisodes.getResultList(); return regroupPodcastsWithEpisodes(item, newEpisodes); } ....... } Note: If you’d like to find out more how to use the Apache Http Client, to get the etag and last-modifiedheaders, you can have a look at my post – How to use the new Apache Http Client to make a HEAD request 6. Execute the batch application Batch processing can be embedded in web applications and WAR files, but I chose in the beginning the simpler approach that creates a standalone application, that can be started by the Java main() method: package org.podcastpedia.batch; //imports ...; @ComponentScan @EnableAutoConfiguration public class Application { private static final String NEW_EPISODES_NOTIFICATION_JOB = "newEpisodesNotificationJob"; private static final String ADD_NEW_PODCAST_JOB = "addNewPodcastJob"; public static void main(String[] args) throws BeansException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobParametersInvalidException, InterruptedException { Log log = LogFactory.getLog(Application.class); SpringApplication app = new SpringApplication(Application.class); app.setWebEnvironment(false); ConfigurableApplicationContext ctx= app.run(args); JobLauncher jobLauncher = ctx.getBean(JobLauncher.class); if(ADD_NEW_PODCAST_JOB.equals(args[0])){ //addNewPodcastJob Job addNewPodcastJob = ctx.getBean(ADD_NEW_PODCAST_JOB, Job.class); JobParameters jobParameters = new JobParametersBuilder() .addDate("date", new Date()) .toJobParameters(); JobExecution jobExecution = jobLauncher.run(addNewPodcastJob, jobParameters); BatchStatus batchStatus = jobExecution.getStatus(); while(batchStatus.isRunning()){ log.info("*********** Still running.... **************"); Thread.sleep(1000); } log.info(String.format("*********** Exit status: %s", jobExecution.getExitStatus().getExitCode())); JobInstance jobInstance = jobExecution.getJobInstance(); log.info(String.format("********* Name of the job %s", jobInstance.getJobName())); log.info(String.format("*********** job instance Id: %d", jobInstance.getId())); System.exit(0); } else if(NEW_EPISODES_NOTIFICATION_JOB.equals(args[0])){ JobParameters jobParameters = new JobParametersBuilder() .addDate("date", new Date()) .addString("updateFrequency", args[1]) .toJobParameters(); jobLauncher.run(ctx.getBean(NEW_EPISODES_NOTIFICATION_JOB, Job.class), jobParameters); } else { throw new IllegalArgumentException("Please provide a valid Job name as first application parameter"); } System.exit(0); } } The best explanation for SpringApplication-, @ComponentScan- and @EnableAutoConfiguration-magic you get from the source – Getting Started – Creating a Batch Service: “The main() method defers to the SpringApplication helper class, providing Application.class as an argument to its run() method. This tells Spring to read the annotation metadata from Application and to manage it as a component in the Spring application context. The @ComponentScan annotation tells Spring to search recursively through theorg.podcastpedia.batchpackage and its children for classes marked directly or indirectly with Spring’s @Component annotation. This directive ensures that Spring finds and registers BatchConfiguration, because it is marked with @Configuration, which in turn is a kind of @Component annotation. The @EnableAutoConfiguration annotation switches on reasonable default behaviors based on the content of your classpath. For example, it looks for any class that implements the CommandLineRunner interface and invokes its run() method.” Execution construction steps: the JobLauncher, which is a simple interface for controlling jobs, is retrieved from the ApplicationContext. Remember this is automatically made available via the@EnableBatchProcessing annotation. now based on the first parameter of the application (args[0]), I will retrieve the correspondingJob from the ApplicationContext then the JobParameters are prepared, where I use the current date - .addDate("date", new Date()), so that the job executions are always unique. once everything is in place, the job can be executed: JobExecution jobExecution = jobLauncher.run(addNewPodcastJob, jobParameters); you can use the returned jobExecution to gain access to BatchStatus, exit code, or job name and id. Note: I highly recommend you read and understand the Meta-Data Schema for Spring Batch. It will also help you better understand the Spring Batch Domain objects. 6.1. Running the application on dev and prod environments To be able to run the Spring Batch / Spring Boot application on different environments I make use of the Spring Profiles capability. By default the application runs with development data (database). But if I want the job to use the production database I have to do the following: provide the following environment argument -Dspring.profiles.active=prod have the production database properties configured in the application-prod.properties file in the classpath, right besides the default application.properties file Summary In this tutorial we’ve learned how to configure a Spring Batch project with Spring Boot and Java configuration, how to use some of the most common readers in batch processing, how to configure some simple jobs, and how to start Spring Batch jobs from a main method. Note: As I mentioned, I am fairly new to Spring Batch, and especially to Spring Boot and Spring Configuration with Java, so if you see any potential for improvement (code, job design etc.) please make a pull request or leave a comment below. Thanks a lot.

In Java every class has a name. Classes are in packages and this lets us programmers work together avoiding name collision. I can name my class A and you can also name your class A so long as long they are in different packages, they work together fine. If you looked at the API of the class Class you certainly noticed that there are three different methods that give you the name of a class: getSimpleName() gives you the name of the class without the package. getName() gives you the name of the class with the full package name in front. getCanonicalName() gives you the canonical name of the class. Simple is it? Well, the first is simple and the second is also meaningful unless there is that disturbing canonical name. That is not evident what that is. And if you do not know what canonical name is, you may feel some disturbance in the force of your Java skills for the second also. What is the difference between the two? If you want a precise explanation, visit the chapter 6.7 of Java Language Specification. Here we go with something simpler, aimed simpler to understand though not so thorough. Let’s see some examples: package pakage.subpackage.evensubberpackage; import org.junit.Assert; import org.junit.Test; public class WhatIsMyName { @Test public void classHasName() { final Class klass = WhatIsMyName.class; final String simpleNameExpected = "WhatIsMyName"; Assert.assertEquals(simpleNameExpected, klass.getSimpleName()); final String nameExpected = "pakage.subpackage.evensubberpackage.WhatIsMyName"; Assert.assertEquals(nameExpected, klass.getName()); Assert.assertEquals(nameExpected, klass.getCanonicalName()); } ... This “unit test” just runs fine. But as you can see there is no difference between name and canonical name in this case. (Note that the name of the package is pakage and not package. To test your java lexical skills answer the question why?) Let’s have a look at the next example from the same junit test file: @Test public void arrayHasName() { final Class klass = WhatIsMyName[].class; final String simpleNameExpected = "WhatIsMyName[]"; Assert.assertEquals(simpleNameExpected, klass.getSimpleName()); final String nameExpected = "[Lpakage.subpackage.evensubberpackage.WhatIsMyName;"; Assert.assertEquals(nameExpected, klass.getName()); final String canonicalNameExpected = "pakage.subpackage.evensubberpackage.WhatIsMyName[]"; Assert.assertEquals(canonicalNameExpected, klass.getCanonicalName()); } Now there are differences. When we talk about arrays the simple name signals it appending the opening and closing brackets, just like we would do in Java source code. The “normal” name looks a bit weird. It starts with an L and semicolon is appended. This reflects the internal representation of the class names in the JVM. The canonical name changed similar to the simple name: it is the same as before for the class having all the package names as prefix with the brackets appended. Seems that getName() is more the JVM name of the class and getCanonicalName() is more like the fully qualified name on Java source level. Let’s go on with still some other example (we are still in the same file): class NestedClass{} @Test public void nestedClassHasName() { final Class klass = NestedClass.class; final String simpleNameExpected = "NestedClass"; Assert.assertEquals(simpleNameExpected, klass.getSimpleName()); final String nameExpected = "pakage.subpackage.evensubberpackage.WhatIsMyName$NestedClass"; Assert.assertEquals(nameExpected, klass.getName()); final String canonicalNameExpected = "pakage.subpackage.evensubberpackage.WhatIsMyName.NestedClass"; Assert.assertEquals(canonicalNameExpected, klass.getCanonicalName()); } The difference is the dollar sign in the name of the class. Again the “name” is more what is used by the JVM and canonical name is what is Java source code like. If you compile this code, the Java compiler will generate the files: WhatIsMyName.class and WhatIsMyName$NestedClass.class Even though the class is named nested class it actually is an inner class. However in the naming there is no difference: a static or non-static class inside another class is just named the same. Now let’s see something even more interesting: @Test public void methodClassHasName() { class MethodClass{}; final Class klass = MethodClass.class; final String simpleNameExpected = "MethodClass"; Assert.assertEquals(simpleNameExpected, klass.getSimpleName()); final String nameExpected = "pakage.subpackage.evensubberpackage.WhatIsMyName$1MethodClass"; Assert.assertEquals(nameExpected, klass.getName()); final String canonicalNameExpected = null; Assert.assertEquals(canonicalNameExpected, klass.getCanonicalName()); } This time we have a class inside a method. Not a usual scenario, but valid from the Java language point of view. The simple name of the class is just that: the simple name of the class. No much surprise. The “normal” name however is interesting. The Java compiler generates a JVM name for the class and this name contains a number in it. Why? Because nothing would stop me having a class with the same name in another method in our test class and inserting a number is the way to prevent name collisions for the JVM. The JVM does not know or care anything about inner and nested classes or classes defined inside a method. A class is just a class. If you compile the code you will probably see the file WhatIsMyName$1MethodClass.class generated by javac. I had to add “probably” not because I count the possibility of you being blind, but rather because this name is actually the internal matter of the Java compiler. It may choose different name collision avoiding strategy, though I know no compiler that differs from the above. The canonical name is the most interesting. It does not exist! It is null. Why? Because you can not access this class from outside the method defining it. It does not have a canonical name. Let’s go on. What about anonymous classes. They should not have name. After all, that is why they are called anonymous. @Test public void anonymousClassHasName() { final Class klass = new Object(){}.getClass(); final String simpleNameExpected = ""; Assert.assertEquals(simpleNameExpected, klass.getSimpleName()); final String nameExpected = "pakage.subpackage.evensubberpackage.WhatIsMyName$1"; Assert.assertEquals(nameExpected, klass.getName()); final String canonicalNameExpected = null; Assert.assertEquals(canonicalNameExpected, klass.getCanonicalName()); } Actually they do not have simple name. The simple name is empty string. They do, however have name, made up by the compiler. Poor javac does not have other choice. It has to make up some name even for the unnamed classes. It has to generate the code for the JVM and it has to save it to some file. Canonical name is again null. Are we ready with the examples? No. We have something simple (a.k.a. primitive) at the end. Java primitives. @Test public void intClassHasName() { final Class klass = int.class; final String intNameExpected = "int"; Assert.assertEquals(intNameExpected, klass.getSimpleName()); Assert.assertEquals(intNameExpected, klass.getName()); Assert.assertEquals(intNameExpected, klass.getCanonicalName()); } If the class represents a primitive, like int (what can be simpler than an int?) then the simple name, “the” name and the canonical names are all int the name of the primitive. Just as well an array of a primitive is very simple is it? @Test public void intArrayClassHasName() { final Class klass = int[].class; final String simpleNameExpected = "int[]"; Assert.assertEquals(simpleNameExpected, klass.getSimpleName()); final String nameExpected = "[I"; Assert.assertEquals(nameExpected, klass.getName()); final String canonicalNameExpected = "int[]"; Assert.assertEquals(canonicalNameExpected, klass.getCanonicalName()); } Well, it is not simple. The name is [I, which is a bit mysterious unless you read the respective chapter of the JVM specification. Perhaps I talk about that another time. Conclusion The simple name of the class is simple. The “name” returned by getName() is the one interesting for JVM level things. The getCanonicalName() is the one that looks most like Java source. You can get the full source code of the example above from the gist e789d700d3c9abc6afa0 from GitHub.

Some of the hardest problems a Java Developer will ever have to face are classpath errors: ClassNotFoundException, NoClassDefFoundError, Jar Hell, Xerces Hell and company. In this post we will go through the root causes of these problems, and see how a minimal tool (JHades) can help solving them quickly. We will see why Maven cannot (always) prevent classpath duplicates, and also: The only way to deal with Jar Hell Class loaders The Class loader chain Class loader priority: Parent First vs Parent Last Debugging server startup problems Making sense of Jar Hell with jHades Simple strategy for avoiding classpath problems The classpath gets fixed in Java 9? The only way to deal with Jar Hell Classpath problems can be time-consuming to debug, and tend to happen at the worst possible times and places: before releases, and often in environments where there is little to no access by the development team. They can also happen at the IDE level, and become a source of reduced productivity. We developers tend to find these problems early and often, and this is the usual response: Let's try to save us some hair and get to the bottom of this. These type of problems are hard to approach via trial and error. The only real way to solve them is to really understand what is going on, but where to start? It turns out that Jar Hell problems are simpler than what they look, and only a few concepts are needed to solve them. In the end, the common root causes for Jar Hell problems are: a Jar is missing there is one Jar too many a class is not visible where it should be But if it's that simple, then why are classpath problems so hard to debug? Jar Hell stack traces are incomplete One reason is that the stack traces for classpath problems have a lot of information missing that is needed to troubleshoot the problem. Take for example this stack trace: java.lang.IncompatibleClassChangeError: Class org.jhades.SomeServiceImpl does not implement the requested interfaceorg.jhades.SomeService org.jhades.TestServlet.doGet(TestServlet.java:19) It says that a class does not implement a certain interface. But if we look at the class source: publicclassSomeServiceImpl implementsSomeService { @Override publicvoiddoSomething() { System.out.println( "Call successful!"); } Well, the class clearly implements the missing interface! So what is going on then? The problem is that the stack trace is missing a lot of information that is critical to understanding the problem. The stack trace should have probably contained an error message such as this (we will learn what this means): The Class SomeServiceImpl of class loader /path/to/tomcat/lib does not implement the interface SomeService loaded from class loader Tomcat - WebApp - /path/to/tomcat/webapps/test This would be at least an indication of where to start: Someone new learning Java would at least know that there is this notion of class loader that is essential to understand what is going on It would make clear that one class involved was not being loaded from a WAR, but somehow from some directory on the server (SomeServiceImpl). What is a Class Loader? To start, a Class Loader is just a Java class, more exactly an instance of a class at runtime. It is NOT an inaccessible internal component of the JVM like for example the garbage collector. Take for example the WebAppClassLoader of Tomcat, here is it's javadoc. As you can see it's just a plain Java class, we can even write our own class loader if needed. Any subclass of ClassLoader will qualify as a class loader. The main responsibilities of a class loader is to known where class files are located, and then load classes on JVM demand. Everything is linked to a class loader Each object in the JVM is linked to it's Class via getClass(), and each class is linked to a class loader via getClassLoader(). This means that: Every object in the JVM is linked to a class loader! Let's see how this fact can be used to troubleshoot a classpath error scenario. How-To find where a class file really is Let's take an object and see where it's class file is located in the file system: System.out.println(service.getClass() .getClassLoader() .getResource("org/jhades/SomeServiceImpl.class")); This is the full path to the class file: jar:file:/Users/user1/.m2/repository/org/jhades/jar-2/1.0-SNAPSHOT/jar-2-1.0-SNAPSHOT.jar!/org/jhades/SomeServiceImpl.class As we can see the class loader is just a runtime component that knowns where in the file system to look for class files and how to load them. But what happens if the class loader cannot find a given class? The Class loader Chain By default in the JVM, if a class loader does not find a class, it will then ask it's parent class loader for that same class and so forth. This continues all the way up until the JVM bootstrap class loader (more on this later). This chain of class loaders is the class loader delegation chain. Class loader priority: Parent First vs Parent Last Some class loaders delegate requests immediately to the parent class loader, without searching first in their own known set of directories for the class file. A class loader operating on this mode is said to be in Parent First mode. If a class loader first looks for a class locally and only after queries the parent if the class is not found, then that class loader is said to be working in Parent Last mode. Do all applications have a class loader chain ? Even the most simple Hello World main method has 3 class loaders: The Application class loader, responsible for loading the application classes (parent first) The Extensions class loader, that loads jars from $JAVA_HOME/jre/lib/ext (parent first) The Bootstrap class loader, that loads any class shipped with the JDK such as java.lang.String (no parent class loader) What does the class loader chain of a WAR application look like? In the case of application servers like Tomcat or Websphere, the class loader chain is configured differently than a simple Hello World main method program. Take for example the case of the Tomcat class loader chain: Here we wee that each WAR runs in a WebAppClassLoader, that works in parent last mode (it can be set to parent first as well). The Common class loader loads libraries installed at the level of the server. What does the Servlet spec say about class loading? Only a small part of the class loader chain behavior is defined by the Servlet container specification: The WAR application runs on it's own application class loader, that might be shared with other applications or not The files in WEB-INF/classes take precedence over everything else After that, it's anyones guess! The rest is completely open for interpretation by container providers. Why isn't there a common approach for class loading across vendors? Usually open source containers like Tomcat or Jetty are configured by default to look for classes in the WAR first, and only then search in server class loaders. This allows for applications to use their own versions of libraries that override the ones available on the server. What about the big iron servers? Commercial products like Websphere will try to 'sell' you their own server provided libraries, that by default take precedence over the ones installed on the WAR. This is done assuming that if you bought the server you want also to use the JEE libraries and versions it provides, which is often NOT the case. This makes deploying to certain commercial products a huge hassle, as they behave differently then the Tomcat or Jetty that developers use to run applications in their workstation. We will see further on a solution for this. Common Problem: duplicate class versions At this moment you probably have a huge question: What if there are two jars inside a WAR that contain the exact same class? The answer is that the behavior is undetermined and only at runtime one of the two classes will be chosen. Which one gets chosen depends on the internal implementation of the class loader, there is no way to know upfront. But luckily most projects these days use Maven, and Maven solves this problem by ensuring only one version of a given jar is added to the WAR. So a Maven project is immune to this particular type of Jar Hell, right? Why Maven does not prevent classpath duplicates Unfortunately Maven cannot help in all Jar Hell situations. In fact, many Maven projects that don't use certain quality control plugins can have hundreds of duplicate class files on the classpath (I saw trunks with over 500 duplicates). There are several reasons for that: Library publishers occasionally change the artifact name of a jar: This happens due to re-branding or other reasons. Take for example the example of the JAXB jar. There is no way Maven can identify those artifacts as being the same jar! Some jars are published with and without dependencies: Some library providers provide a 'with dependencies' version of a jar, which includes other jars inside. If we have transitive dependencies with the two versions, we will end up with duplicates. Some classes are copied between jars: Some library creators, when faced with the need for a certain class will just grab it from another project and copy it to a new jar without changing the package name. Are all class files duplicates dangerous? If the duplicate class files exist inside the same class loader, and the two duplicate class files are exactly identical then it does not matter which one gets chosen first - this situation is not dangerous. If the two class files are inside the same class loader and they are not identical, then there is no way which one will be chosen at runtime - this is problematic and can manifest itself when deploying to different environments. If the class files are in two different class loaders, then they are never considered identical (see the class identity crisis section further on). How can WAR classpath duplicates be avoided? This problem can be avoided for example by using the Maven Enforcer Plugin, with the extra rule of Ban Duplicate Classes turned on. You can quickly check if your WAR is clean using the JHades WAR duplicate classes report as well. This tool has an option to filter 'harmless' duplicates (same class file size). But even a clean WAR might have deployment problems: Classes missing, classes taken from the server instead of the WAR and thus with the wrong version, class cast exceptions, etc. Debugging the classpath with JHades Classpath problems often show up when the application server is starting up, which is a particularly bad moment specially when deploying to an environment where there is limited access. JHades is a tool to help deal it with Jar Hell (disclaimer: I wrote it). It's a single Jar with no dependencies other than the JDK7 itself. This is an example of how to use it: newJHades() .printClassLoaders() .printClasspath() .overlappingJarsReport() .multipleClassVersionsReport() .findClassByName("org.jhades.SomeServiceImpl") This prints to the screen the class loader chain, jars, duplicate classes, etc. Debugging server startup problems JHades works works well in scenarios where the server does not start properly. A servlet listener is provided that allows to print classpath debugging information even before any other component of the application starts running. ClassCastException and the Class Identity Crisis When troubleshooting Jar Hell, beware of ClassCastExceptions. A class is identified in the JVM not only by it's fully qualified class name, but also by it's class loader. This is counterintuitive but in hindsight makes sense: We can create two different classes with the same package and name, ship them in two jars and put them in two different class loaders. One let's say extends ArrayList and the other is a Map. The classes are therefore completely different (despite the same name) and cannot be cast to each other! The runtime will throw a CCE to prevent this potential error case, because there is no guarantee that the classes are castable. Adding the class loader to the class identifier was the outcome of the Class Identity Crisis that occurred in earlier Java days. A Strategy for Avoiding Classpath Problems This is easier said then done, but the best way to avoid classpath related deployment problems is to run the production server in Parent Last mode. This way the class versions of the WAR take precedence over the ones on the server, and the same classes are used in production and in a developer workstation where it's likely that Tomcat, Jetty or other open source Parent Last server is being used. In certain servers like Websphere, this is not sufficient and you also have to provide special properties on the manifest file to explicitly turn off certain libraries like for example JAX-WS. Fixing the classpath in Java 9 In Java 9 the classpath gets completely revamped with the new Jigsaw modularity system. In Java 9 a jar can be declared as a module and it will run in it's own isolated class loader, that reads class files from other similar module class loaders in an OSGI sort of way. This will allow multiple versions of the same Jar to coexist in the same application if needed. Conclusions In the end, Jar Hell problems are not that low level or unapproachable as they might seem at first. It's all about zip files (jars) being present/ not being present in certain directories, how to find those directories, and how to debug the classpath in environments with limited access. By knowing a limited set of concepts such as Class Loaders, the Class Loader Chain and Parent First / Parent Last modes, these problems can be tackled effectively. External links This presentation Do you really get class loaders from Jevgeni Kabanov of ZeroTurnaround (JRebel company) is a great resource about Jar Hell and the different type of classpath related exceptions.

This article lists some of the best practices that would be useful for developers while they are coding with AngularJS. These are out of my own experiences while working on AngularJS and do not warranty the entire list. I am sure there can be more to this list and thus, request my readers to suggest/comment such that they could be added to the list below. Found some of the following pages which presents a set of good practices you would want to refer. Thanks to the readers for the valuable contribution. AngularJS Style Guide App Structure Best practices Initialization One should try and place the

Today we are going to look at one of the aspects where using streams is a real win – when we need to thread work. As well as parallel streams, we will also look at Spliterators which acts as the machinery which pushes elements into the pipeline. Streams use a technique known as internal iteration. It’s internal because the Iterator (or in our case Spliterator) which supplies work through our stream is hidden from us. To use a stream all we need do [once we have a source] is add the stages of the pipeline and supply the functions that these stages require. We don’t need to know how the data is being passed along the pipeline, just that it is. The benefit is that the workings are hidden from us and we can focus more on the work that must be done rather than how it can be done. The opposite, external iteration is where we are given a loop variable or iterator and we look up the value, and pass it through the code ourselves. This obviously gives us a benefit in that have full control and low overhead. The downside is we have to do all the work looking up the values and passing them through the loop body. This will also mean more test code, and testing loop bodies properly can be tricky. With normal for-loops we also have to be careful of one-off errors. The question we need to ask ourselves when considering the iteration method: Do we really need absolute control for the task? Streams do some things really well but come with a small performance penalty. Perhaps a non-stream (or even non-Java) solution is more appropriate for high performance work. On the other hand, sorting and filtering files to display in say a ‘recently accessed’ menu item doesn’t require high performance. In that case we’d probably settle for an easy and quick way to do it rather than the best performing one. Even if we go with a performant solution some benchmarking will be necessary as surprises often await. Thus we’re trading convenience off against performance, development time and risk of bugs. Streams are easy to parallelise as we’ll see. We just change the type of the stream to a parallel stream using the parallel() operator. To do this with internal iteration is hard because it’s set up that we get one item per iteration. The best we can do in that environment is pass work off to threads. To do things efficiently we’d probably have to ditch looping through all the values in the outer loop and look at dividing the work up another way. We’ll see a way of doing this. With that in mind we’ll look at a prime number generator. First this is not the most efficient prime number generator. For a demonstration it was useful to have an application that was well known, easy to understand, easy to perform with streams and would take a fair bit of computation time to complete. Let’s look at the internal iteration version first: public class ForLoopPrimes { public static Set findPrimes(int maxPrimeTry) { Set s = new HashSet<>(); // The candidates to try (1 is not a prime number by definition!) outer: for (int i = 2; i <= maxPrimeTry; i++) { // Only need to try up to sqrt(i) - see notes int maxJ = (int) Math.sqrt(i); // Our divisor candidates for (int j = 2; j <= maxJ; j++) { // If we can divide exactly by j, i is not prime if (i / j * j == i) { continue outer; } } // If we got here, it's prime s.add(i); } return s; } public static void main(String args[]) { int maxPrimeTry = 9999999; long startTime = System.currentTimeMillis(); Set s = findPrimes(maxPrimeTry); long timeTaken = System.currentTimeMillis() - startTime; s.stream().sorted().forEach(System.out::println); System.out.println("Time taken: " + timeTaken); } } Note: Since we only need to find one divisor, and multiplication is commutative, we only need to exhaust all potential pairs of factors and test one of them [the smaller]. The smaller can’t be any bigger than the square root of the candidate prime and must be at least 2. This is an example of a brute force algorithm. We’re trying every combination rather than using any stealth or optimisation. We’d also in this case expect the internal iteration version to run fast since there is not a lot of work per iteration. So why do we have to demonstrate this? Suppose we want to take advantage of hardware in modern processors and thread this up. How might we do it? Up to Java 7 and certainly before Java 5 this would have been a real pain. We’ve got to divide up the workload, maintain a pool of threads and signal them that there is work available and then collect the work back from them when done. We probably also want to shut the worker threads down at the end if we have any more work to do. While it’s not rocket science, it can be hard to get right quickly and subtle bugs can be hard to spot. Java 7 makes this a lot easier with the ForkJoin framework. It’s still tricky and easy to get wrong. We’ll use a RecursiveAction to break up the outer loop into pieces of work using a divide-and-conqueror strategy. Note that parallel streams do this as well. public class ForkJoinPrimes { private static int workSize; private static Queue resultsQueue; // Use this to collect work private static class Results { public final int minPrimeTry; public final int maxPrimeTry; public final Set resultSet; public Results(int minPrimeTry, int maxPrimeTry, Set resultSet) { this.minPrimeTry = minPrimeTry; this.maxPrimeTry = maxPrimeTry; this.resultSet = resultSet; } } private static class FindPrimes extends RecursiveAction { private final int start; private final int end; public FindPrimes(int start, int end) { this.start = start; this.end = end; } private Set findPrimes(int minPrimeTry, int maxPrimeTry) { Set s = new HashSet<>(); // The candidates to try // (1 is not a prime number by definition!) outer: for (int i = minPrimeTry; i <= maxPrimeTry; i++) { // Only need to try up to sqrt(i) - see notes int maxJ = (int) Math.sqrt(i); // Our divisor candidates for (int j = 2; j <= maxJ; j++) { // If we can divide exactly by j, i is not prime if (i / j * j == i) { continue outer; } } // If we got here, it's prime s.add(i); } return s; } protected void compute() { // Small enough for us? if (end - start < workSize) { resultsQueue.offer(new Results(start, end, findPrimes(start, end))); } else { // Divide into two pieces int mid = (start + end) / 2; invokeAll(new FindPrimes(start, mid), new FindPrimes(mid + 1, end)); } } } public static void main(String args[]) { int maxPrimeTry = 9999999; int maxWorkDivisor = 8; workSize = (maxPrimeTry + 1) / maxWorkDivisor; ForkJoinPool pool = new ForkJoinPool(); resultsQueue = new ConcurrentLinkedQueue<>(); long startTime = System.currentTimeMillis(); pool.invoke(new FindPrimes(2, maxPrimeTry)); long timeTaken = System.currentTimeMillis() - startTime; System.out.println("Number of tasks executed: " + resultsQueue.size()); while (resultsQueue.size() > 0) { Results results = resultsQueue.poll(); Set s = results.resultSet; s.stream().sorted().forEach(System.out::println); } System.out.println("Time taken: " + timeTaken); } } This is quite recognisable since we have reused the sequential code to carry out the work in a subtask. We create two RecursiveActons to break the workload into two pieces. We keep breaking down until the workload is below a certain size when we carry out the action. We finally collect our results on a concurrent queue. Note there is a fair bit of code. Let’s look at a sequential Java 8 streams solution: public class SequentialStreamPrimes { public static Set findPrimes(int maxPrimeTry) { return IntStream.rangeClosed(2, maxPrimeTry) .map(i -> IntStream.rangeClosed(2, (int) (Math.sqrt(i))) .filter(j -> i / j * j == i).map(j -> 0) .findAny().orElse(i)) .filter(i -> i != 0) .mapToObj(i -> Integer.valueOf(i)) .collect(Collectors.toSet()); } public static void main(String args[]) { int maxPrimeTry = 9999999; long startTime = System.currentTimeMillis(); Set s = findPrimes(maxPrimeTry); long timeTaken = System.currentTimeMillis() - startTime; s.stream().sorted().forEach(System.out::println); System.out.println("Time taken: " + timeTaken); } } We can see the streams solution matches up with the external iteration version quite well except for a few tricks needed: Since we only need one factor we use findAny(). This acts like the break statement. findAny() returns an Optional so we need to unwrap it to get our value. If we have no value (i.e. we found a prime) we will store the prime (the outer value, i) by putting it in the orElse clause. If the inner IntStream finds a factor, we can map to 0 for storing, since we’ll never check 0. Unfortunately, this means we attempt to store something for every candidate which adds to the overhead. So let’s make it threaded. We only need to change the findPrimes method slightly: public static Set findPrimes(int maxPrimeTry) { return IntStream.rangeClosed(2, maxPrimeTry) .parallel() .map(i -> IntStream.rangeClosed(2, (int) (Math.sqrt(i))) .filter(j -> i / j * j == i).map(j -> 0) .findAny().orElse(i)) .filter(i -> i != 0) .mapToObj(i -> Integer.valueOf(i)) .collect(Collectors.toSet()); } This time we don’t have to mess around with the algorithm. Simply by adding an intermediate stage parallel() to the stream we make it divide up the work. Parallel(), like filter and map, is an intermediate operation. Intermediate operations can also change the behaviour of a stream as well as affect the passing values. Other intermediate stages we’re not seen yet are: sequential() – make the stream sequential distinct() – only distinct values pass sorted() – a sorted stream is returned, optionally we can pass a Comparator unordered() – return an unordered stream If we fire up jconsole while we’re running and look at the Threads tab, we can compare the sequential and parallel version. In the parallel version we can see several ForkJoin threads doing the work. I did some timings and got the following results [note this is not completely accurate since other tasks might have been running in the background on my machine - values are to the nearest half-second]. External, sequential (for-loop): 8.5 seconds External, parallel (ForkJoin): 2.5 second Internal, sequential (sequential stream): 21 seconds Internal, parallel (parallel stream): 6 seconds This is probably as expected. The amount of work per iteration in the inner loop is low, so any stream actions will have relatively high overhead as seen in the sequential stream version, as well as we had to store a value irrespective of whether it was prime or not. The parallel stream comes in slightly faster than the for-loop, but the ForkJoin version outperforms it by a factor of more than 2. Note how simpler the streams version was [once we get the hang of streams of course] compared to the amount of code in the ForkJoin version. Let’s have a look at the work-horse of this work distribution, the Spliterator. A Spliterator is an interface like an Iterator, but instead of just providing the next value, it can also divide work up into smaller pieces which are executed by ForkJoinTasks. When we create a Spliterator we provide details of the size of the workload and characteristics that the values have. Some types of Spliterators such as RangeIntSpliterator [which IntRange supplies] use the characteristics() method to return characteristics, rather than having them supplied via a constructor like AbstractSpliterator does. We obviously need the size of the workload so we can divide up the work up and know when to stop dividing. The characteristics we can supply are defined in the Spliterator interface as follows: SIZED – we can supply a specific number of values that will be sent prior to processing (versus an InfiniteSupplyingSpliterator) SUBSIZED – implies that any Spliterators that trySplit() creates will be SIZED and SUBSIZED. Not all SIZED Spliterators will split into SUBSIZED spliterators. The API gives an example of a binary tree where we might know how many elements are in the tree, but not in the sub-trees ORDERED – we supply the values in sequence, for example from a list SORTED – the order follows a sort order (rather than sequence); ORDERED must also be set DISTINCT – each value is different from every other, for example if we supply from a set NONNULL – values coming from the source will not be null IMMUTABLE – it’s impossible to change the source (such as add or remove values) – if this is not set and neither is CONCURRENT we’re advised to check the documentation for what happens on modification (such as a ConcurrentModificationException) CONCURRENT – the source may be concurrently modified safely and we’re advised to check the documentation on the policy These characteristics are used by the splitting machinery, for example in the ForEachOps class (which is used to carry out tasks in a pipeline terminated with a forEach). Normally we can just use a pre-built Spliterator [and often don't even need to worry about that because it's supplied by the stream() method]. Remember the streams framework allows us to get work done without having to know all the details of how its being done. It’s only in the rare cases of a special problem or needing maximum performance do we have to worry. Splitting is done by the trySplit() operation. This returns a new Spliterator. For the requirements of this function the API documentation should be referred to. When we consume the contents of [part of] the stream in bulk using the Spliterator, the forEachRemaining(action) operation is called. This takes source data and calls the next action via the action’s accept call. For example if the next operation is filter, the accept call on filter is called. This calls the test method of the contained predicate, and if that is true, the accept method of the next stage is called. At some point a terminal stage will be called [the accept method calls no other stage] and the final value will be consumed, reduced or collected. When we call a stream() method, this pipeline is created and calling intermediate stages chains them to the end of the pipeline. Calling the final consuming stage makes the final link and sets everything off. Alternatively when we need to generate each element from a non-bulk source, the tryAdvance() function is used. This is passed an action which accept is called on as before. However, we return true if we want to continue and false if we don’t. InfiniteSupplyingSpliterator for example always returns true, but we can use an AbstractSpliterator if we want to control this. Remember the AbstractIntSpliterator from our SixGame in the finite generators article? One of our tryAdvance functions was this: @Override public boolean tryAdvance(Consumer action) { if (action == null) throw new NullPointerException(); if (done) return false; action.accept(rollDie()); return true; } In this case if we roll the die we always continue. This would allow the done logic to be set from elsewhere if we didn’t want to roll a die again. It might have been slightly better to have returned !done instead of true to terminate generation immediately as soon as the six was thrown. However in this case going through another cycle was hardly a chore. That’s it for the streams overview. In the next article we’ll look a bit more at lambda expressions.

Learn Fibonacci Series patterns and best practices with easy Java 8 source code examples in this outstanding tutorial by Pierre-Yves Saumont

Overview : Recently, I was working on a project which uses a REST services layer to communicate with the client application (GWT application). So I have spent a lot of to time to figure out how to secure the REST services with Spring Security. This article describes the solution I found, and I have implemented. I hope that this solution will be helpful to someone and will save a much valuable time. The solution : In a normal web application, whenever a secured resource is accessed Spring Security check the security context for the current user and will decide either to forward him to login page (if the user is not authenticated), or to forward him to the resource not authorised page (if he doesn’t have the required permissions). In our scenario this is different, because we don’t have pages to forward to, we need to adapt and override Spring Security to communicate using HTTP protocols status only, below I liste the things to do to make Spring Security works best : The authentication is going to be managed by the normal form login, the only difference is that the response will be on JSON along with an HTTP status which can either code 200 (if the autentication passed) or code 401 (if the authentication failed) ; Override the AuthenticationFailureHandler to return the code 401 UNAUTHORIZED ; Override the AuthenticationSuccessHandler to return the code 20 OK, the body of the HTTP response contain the JSON data of the current authenticated user ; Override the AuthenticationEntryPoint to always return the code 401 UNAUTHORIZED. This will override the default behavior of Spring Security which is forwarding the user to the login page if he don’t meet the security requirements, because on REST we don’t have any login page ; Override the LogoutSuccessHandler to return the code 20 OK ; Like a normal web application secured by Spring Security, before accessing a protected service, it is mandatory to first authenticate by submitting the password and username to the Login URL. Note: The following solution requires Spring Security in version minimum 3.2. Overriding the AuthenticationEntryPoint : Class extends org.springframework.security.web.AuthenticationEntryPoint, and implements only one method, which sends response error (with 401 status code) in cause of unauthorized attempt. @Component public class HttpAuthenticationEntryPoint implements AuthenticationEntryPoint { @Override public void commence(HttpServletRequest request, HttpServletResponse response, AuthenticationException authException) throws IOException { response.sendError(HttpServletResponse.SC_UNAUTHORIZED, authException.getMessage()); } } Overriding the AuthenticationSuccessHandler : The AuthenticationSuccessHandler is responsible of what to do after a successful authentication, by default it will redirect to an URL, but in our case we want it to send an HTTP response with data. @Component public class AuthSuccessHandler extends SavedRequestAwareAuthenticationSuccessHandler { private static final Logger LOGGER = LoggerFactory.getLogger(AuthSuccessHandler.class); private final ObjectMapper mapper; @Autowired AuthSuccessHandler(MappingJackson2HttpMessageConverter messageConverter) { this.mapper = messageConverter.getObjectMapper(); } @Override public void onAuthenticationSuccess(HttpServletRequest request, HttpServletResponse response, Authentication authentication) throws IOException, ServletException { response.setStatus(HttpServletResponse.SC_OK); NuvolaUserDetails userDetails = (NuvolaUserDetails) authentication.getPrincipal(); User user = userDetails.getUser(); userDetails.setUser(user); LOGGER.info(userDetails.getUsername() + " got is connected "); PrintWriter writer = response.getWriter(); mapper.writeValue(writer, user); writer.flush(); } } Overriding the AuthenticationFailureHandler : The AuthenticationFaillureHandler is responsible of what to after a failed authentication, by default it will redirect to the login page URL, but in our case we just want it to send an HTTP response with the 401 UNAUTHORIZED code. @Component public class AuthFailureHandler extends SimpleUrlAuthenticationFailureHandler { @Override public void onAuthenticationFailure(HttpServletRequest request, HttpServletResponse response, AuthenticationException exception) throws IOException, ServletException { response.setStatus(HttpServletResponse.SC_UNAUTHORIZED); PrintWriter writer = response.getWriter(); writer.write(exception.getMessage()); writer.flush(); } } Overriding the LogoutSuccessHandler : The LogoutSuccessHandler decide what to do if the user logged out successfully, by default it will redirect to the login page URL, because we don’t have that I did override it to return an HTTP response with the 20 OK code. @Component public class HttpLogoutSuccessHandler implements LogoutSuccessHandler { @Override public void onLogoutSuccess(HttpServletRequest request, HttpServletResponse response, Authentication authentication) throws IOException { response.setStatus(HttpServletResponse.SC_OK); response.getWriter().flush(); } } Spring security configuration : This is the final step, to put all what we did together, I prefer using the new way to configure Spring Security which is with Java no XML, but you can easily adapt this configuration to XML. @Configuration @EnableWebSecurity public class WebSecurityConfig extends WebSecurityConfigurerAdapter { private static final String LOGIN_PATH = ApiPaths.ROOT + ApiPaths.User.ROOT + ApiPaths.User.LOGIN; @Autowired private NuvolaUserDetailsService userDetailsService; @Autowired private HttpAuthenticationEntryPoint authenticationEntryPoint; @Autowired private AuthSuccessHandler authSuccessHandler; @Autowired private AuthFailureHandler authFailureHandler; @Autowired private HttpLogoutSuccessHandler logoutSuccessHandler; @Bean @Override public AuthenticationManager authenticationManagerBean() throws Exception { return super.authenticationManagerBean(); } @Bean @Override public UserDetailsService userDetailsServiceBean() throws Exception { return super.userDetailsServiceBean(); } @Bean public AuthenticationProvider authenticationProvider() { DaoAuthenticationProvider authenticationProvider = new DaoAuthenticationProvider(); authenticationProvider.setUserDetailsService(userDetailsService); authenticationProvider.setPasswordEncoder(new ShaPasswordEncoder()); return authenticationProvider; } @Override protected void configure(AuthenticationManagerBuilder auth) throws Exception { auth.authenticationProvider(authenticationProvider()); } @Override protected AuthenticationManager authenticationManager() throws Exception { return super.authenticationManager(); } @Override protected void configure(HttpSecurity http) throws Exception { http.csrf().disable() .authenticationProvider(authenticationProvider()) .exceptionHandling() .authenticationEntryPoint(authenticationEntryPoint) .and() .formLogin() .permitAll() .loginProcessingUrl(LOGIN_PATH) .usernameParameter(USERNAME) .passwordParameter(PASSWORD) .successHandler(authSuccessHandler) .failureHandler(authFailureHandler) .and() .logout() .permitAll() .logoutRequestMatcher(new AntPathRequestMatcher(LOGIN_PATH, "DELETE")) .logoutSuccessHandler(logoutSuccessHandler) .and() .sessionManagement() .maximumSessions(1); http.authorizeRequests().anyRequest().authenticated(); } } This was a sneak peak at the overall configuration, I attached in this article a Github repository containing a sample project https://github.com/imrabti/gwtp-spring-security. I hope this will help some of you developers struggling to figure out a solution, please feel free to ask any questions, or post any enhancements that can make this solution better.

In this article we will discuss about Backbone.js collections. We will see how we can use collections to manipulate a group of models and how we can use restul API to easily fetch and save collections. Background Every application needs to create a collection of models which can be ordered, iterated and perhaps sorted and searched when a need arises. Keeping this in mind, Backbone also comes with a collection type which makes dealing with collection of models fairly easy and straight forward. Link to complete series: BackBone Tutorial – Part 1: Introduction to Backbone.Js BackBone Tutorial – Part 2: Understanding the basics of Backbone Models BackBone Tutorial – Part 3: More about Backbone Models BackBone Tutorial – Part 4: CRUD Operations on BackboneJs Models using HTTP REST Service BackBone Tutorial – Part 5: Understanding Backbone.js Collections[^] BackBone Tutorial – Part 6: Understanding Backbone.js Views[^] BackBone Tutorial – Part 7: Understanding Backbone.js Routes and History[^] Using the code Let us start looking at the backbone collections in details. Creating a collection Creating a backbone collection is similar to creating a model. We just need to extend the backbone’s collection class to create our own collection. Let us continue working with the same example where we created a Book model and let's try to create a simple BooksCollection. var BooksCollection = Backbone.Collection.extend({ }); This collection will hold the Book model we have created in our previous articles. var Book = Backbone.Model.extend({ defaults: { ID: "", BookName: "" }, idAttribute: "ID", urlRoot: 'http://localhost:51377/api/Books' }); Specifying the model for a collection To specify which model this collection should hold, we need to specify/override the model property of the collection class. var BooksCollection = Backbone.Collection.extend({ model: Book, }); Once we specify the model property of a collection what will happen internally is that whenever we create this collection, internally it will create an array of the specified models. Then all the operations on this collection object will result in the actual operations on that array. Instantiating a collection A collection can be instantiated by using the new keyword. We can create an empty collection and then add the model objects to it later or we can pass a few model objects in the collection while creating it. // Lets create an empty collection var collection1 = new BooksCollection(); //Lets create a pre-populated collection var book1 = new Book({ ID: 1, BookName: "Book 1" }); var book2 = new Book({ ID: 2, BookName: "Book 2" }); var collection2 = new BooksCollection([book1, book2]); Adding models to collection To add an item to a collection, we can use the add method on the collection. The important thing to notice here is that if the item with the same id exist in the collection, the add will simply be ignored. var book3 = new Book({ ID: 3, BookName: "Book 3" }); collection2.add(book3); Now there might be a scenario where we actually want to update an existing added model in a collection. If that is the case, then we need to pass the {merge:true} option in the add function. var book3 = new Book({ ID: 3, BookName: "Book 3" }); collection2.add(book3); var book3_changed = new Book({ ID: 3, BookName: "Changed Model" }); collection2.add(book3_changed, { merge: true }); Another important point to consider here is that the collection keep a shallow copy of the actual models. So if we change a model attribute after adding it to a collection, the attribute value will also get changed inside the collection. Also, if we want to add multiple models, we can do that by passing the model array in the add method. var book4 = new Book({ ID: 4, BookName: "Book 4" }); var book5 = new Book({ ID: 5, BookName: "Book 5" }); collection2.add([book4, book5]); It is also possible to add the model at a specific index in the collection. To do this we need to pass the {at:location} in the add options. var book0 = new Book({ ID: 0, BookName: "Book 0" }); collection2.add(book0, {at:0}); Note: push and unshift function can also be used to add models to collection. Removing models from collection To remove the model from the collection, we just need to call the remove method on the collection. The remove method simply removes this model from the collection. collection2.remove(book0); Also, if we want to empty the model, we can call the reset method on the collection. collection1.reset(); It is also possible to reset a collection and populate it with new models by passing an array of models in the reset function. collection2.reset([book4, book5]); // this will reset the collection and add book4 and book5 into it Note: pop and shift function can also be used to remove model from collection. Finding the number of items in collection The total number of items in a collection can be found using the length property. var collection2 = new BooksCollection([book1, book2]); console.log(collection2.length); // prints 2 Retrieving models from collection To retrieve a model from a specific location, we can use the at function by passing a 0 based index. var bookRecieved = collection2.at(3); Alternatively, to get the index of a known model in the collection, we can use the indexOf method. var index = collection2.indexOf(bookRecieved); We can also retreive a model from a collection if we know its id or cid. this can be done by using the get function. var bookFetchedbyId = collection2.get(2); // get the book with ID=2 var bookFetchedbyCid = collection2.get("c3"); // get the book with cid=c3 If we want to iterate through all the models in a collection, we can simply use the classic for loop or the each function provided by collections which is very similar to the foreach loop of underscore.js. for (var i = 0; i < collection2.length; ++i) { console.log(collection2.at(i).get("BookName")); } collection2.each(function (item, index, all) { console.log(item.get("BookName")); }); Listening to collection events Backbone collection raises events whenever an item is added removed to updated in the collection. We can subscribe to these events by listening to add, remove and change event respectively. Let us subscribe to these events in our model to see how this can be done. var BooksCollection = Backbone.Collection.extend({ model: Book, initialize: function () { // This will be called when an item is added. pushed or unshifted this.on('add', function(model) { console.log('something got added'); }); // This will be called when an item is removed, popped or shifted this.on('remove', function(model) { console.log('something got removed'); }); // This will be called when an item is updated this.on('change', function(model) { console.log('something got changed'); }); }, }); The set function The set function can be used to update all the items in a model. If we use set function, it will check for all the existing models and the models being passed in set. If any new model is found in the models being passed, it will be added. If some are not present in the new models list, they will be removed. If there are same models, they will be updated. var collection3 = new BooksCollection(); collection3.add(book1); collection3.add(book2); collection3.add(book3); collection3.set([book1, { ID: 3, BookName: "test sort"}, book5]); The above shown set function will call remove for book2, change for book3 and add for book5. Sorting a collection Backbone keeps all the models in the collection in a sorted order. We can call the sort function to forcefully sort it again but the models are always stored in sorted order. By default these items are sorted in the order they are added to the collection. But we can customize this sorting behavior by providing a simple comparator to our collection. var BooksCollection = Backbone.Collection.extend({ model: Book, comparator: function (model) { return model.get("ID"); }, }); What this comparator does is that it overrides the default sorting behavior by specifying the attribute that should be used for sorting. We can even used a custom expression in this comparator too. Fetch collection using HTTP REST service To be able to fetch the collection from the server, we need to specify the url for the api that returns the collection. var BooksCollection = Backbone.Collection.extend({ model: Book, url: "http://localhost:51377/api/Books", }); Now to fetch the collection from the server, lets call the fetch function. var collection4 = new BooksCollection(); collection4.fetch(); Save collection using HTTP REST service Lets see how we can save the items of a collection on the server. var collection4 = new BooksCollection(); collection4.fetch({ success: function (collection4, response) { // fetch successful, lets iterate and update the values here collection4.each(function (item, index, all) { item.set("BookName", item.get("BookName") + "_updated"); // lets update all book names here item.save(); }); } }); In the above code we are calling save on each model object. this can be improved by either overriding the sync function on a collection or perhaps creating a wrapper model for collection and saving the data using that. Note: The web api code for can be downloaded from the previous article of the series. Point of interest In this article we have discusses about the backbone collections. This has been written from a beginner’s perspective. I hope this has been informative. Download sample code for this article: backboneSample

Here's how to create a persistence configuration in the Java SE environment.

Creating a method that has many parameters is a major sin. Whenever there is need to create such a method, sniff in the air: it is code smell. Harden your unit tests and then refactor. No excuse, no buts. Refactor! Use builder pattern or even better use Fluent API. For the latter the annotation processor fluflu may be of great help. Having all that said we may come to a point in our life when we face real life and not the idealistic pattern that we can follow in our hobby projects. There comes the legacy enterprise library monster that has the method of thousands parameters and you do not have the authority, time, courage or interest (bad for you) to modify … ops… refactor it. You could create a builder as a facade that hides the ugly API behind it if you had the time. Creating a builder is still code that you have to unit test even before you write (you know: TDD) and you just may not have the time. The code that calls the monstrous method is also there already, you just maintain it. You can still do some little trick. It may not be perfect, but still something. Assume that there is a method public void monster(String contactName, String contactId, String street, String district, ... Long pT){ ... } The first thing is to select your local variables at the location of the caller wisely. Pity the names are already chosen and you may not want to change it. There can be some reason for that, for example there is an application wide naming convention followed that may make sense even if not your style. So the call monster(nm, "05300" + dI, getStrt(), d, ... , z+g % 3L ); is not exactly what I was talking about. That is what you have and you can live with it, or just insert new variables into the code: String contactName = nm; String contactId = "05300" + dI; String street = getStrt(); Street district = d; ... Long pT = z+g % 3L; monster(contactName, contactId, street, district, ... ,pT ); or you can even write it in a way that is not usual in Java, though perfectly legal: String contactName, contactId, street, district; ... Long pT; monster(contactName = nm, contactId = "05300" + dI, street = getStrt(), district = d, ... ,pT = z+g % 3L ); Tasty is it? Depends. I would not argue on taste. If you do not like that, there is an alternative way. You can define auxiliary and very simple static methods: static T contactName(T t){ return T;} static T contactId(T t){ return T;} static T street(T t){ return T;} static T district(T t){ return T;} ... static T pT(T t){ return T;} monster(contactName(nm), contactId("05300" + dI), street(getStrt()(, district(d), ... ,pT(z+g % 3L) ); The code is still ugly but a bit more readable at the place of the caller. You can even collect static methods into a utility class, or to an interface in case of Java 8 named like with, using, to and so on. You can statically import them to your code and have some method call as nice as doSomething(using(someParameter), with(someOtherParameter), to(resultStore)); When all that is there you can feel honky dory if you answer the final question: what the blessed whatever* is parameter pT. (* “whatever” you can replace with some other words, whichever you like)

Apparently writing a URL shortener service is the new "Hello, world!" in the IoT/microservice/era world. It all started with A URL shortener service in 45 lines of Scala - neat piece of Scala, flavoured with Spray and Redis for storage. This was quickly followed with A url shortener service in 35 lines of Clojure and even URL Shortener in 43 lines of Haskell. So my inner anti-hipster asked: how long would it be in Java? But not plain Java, for goodness' sake. Spring Boot with Spring Data Redis are a good starting point. All we need is a simple controller handling GET and POST: import com.google.common.hash.Hashing; import org.apache.commons.validator.routines.UrlValidator; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.SpringApplication; import org.springframework.data.redis.core.StringRedisTemplate; import org.springframework.http.*; import org.springframework.web.bind.annotation.*; import javax.servlet.http.*; import java.nio.charset.StandardCharsets; @org.springframework.boot.autoconfigure.EnableAutoConfiguration @org.springframework.stereotype.Controller public class UrlShortener { public static void main(String[] args) { SpringApplication.run(UrlShortener.class, args); } @Autowired private StringRedisTemplate redis; @RequestMapping(value = "/{id}", method = RequestMethod.GET) public void redirect(@PathVariable String id, HttpServletResponse resp) throws Exception { final String url = redis.opsForValue().get(id); if (url != null) resp.sendRedirect(url); else resp.sendError(HttpServletResponse.SC_NOT_FOUND); } @RequestMapping(method = RequestMethod.POST) public ResponseEntity save(HttpServletRequest req) { final String queryParams = (req.getQueryString() != null) ? "?" + req.getQueryString() : ""; final String url = (req.getRequestURI() + queryParams).substring(1); final UrlValidator urlValidator = new UrlValidator(new String[]{"http", "https"}); if (urlValidator.isValid(url)) { final String id = Hashing.murmur3_32().hashString(url, StandardCharsets.UTF_8).toString(); redis.opsForValue().set(id, url); return new ResponseEntity<>("http://mydomain.com/" + id, HttpStatus.OK); } else return new ResponseEntity<>(HttpStatus.BAD_REQUEST); } } The code is nicely self-descriptive and is functionally equivalent to a version in Scala. I didn't try to it squeeze too much to keep line count as short as possible, code above is quite typical with few details: I don't normally use wildcard imports I don't use fully qualified class names (I wanted to save one import line, I admit) I surround if/else blocks with braces I almost never use field injection, ugliest brother in inversion of control family. Instead I would go for constructor to allow testing with mocked Redis: private final StringRedisTemplate redis; @Autowired public UrlShortener(StringRedisTemplate redis) { this.redis = redis; } The thing I struggled the most was... obtaining the original, full URL. Basically I needed everything after .com or port. No bloody way (neither servlets, nor Spring MVC), hence the awkward getQueryString() fiddling. You can use the service as follows - creating shorter URL: $ curl -vX POST localhost:8080/https://www.google.pl/search?q=tomasz+nurkiewicz > POST /https://www.google.pl/search?q=tomasz+nurkiewicz HTTP/1.1 > User-Agent: curl/7.30.0 > Host: localhost:8080 > Accept: */* > < HTTP/1.1 200 OK < Server: Apache-Coyote/1.1 < Content-Type: text/plain;charset=ISO-8859-1 < Content-Length: 28 < Date: Sat, 23 Aug 2014 20:47:40 GMT < http://mydomain.com/50784f51 Redirecting through shorter URL: $ curl -v localhost:8080/50784f51 > GET /50784f51 HTTP/1.1 > User-Agent: curl/7.30.0 > Host: localhost:8080 > Accept: */* > < HTTP/1.1 302 Found < Server: Apache-Coyote/1.1 < Location: https://www.google.pl/search?q=tomasz+nurkiewicz < Content-Length: 0 < Date: Sat, 23 Aug 2014 20:48:00 GMT < For completeness, here is a build file in Gradle (maven would work as well), skipped in all previous solutions: buildscript { repositories { mavenLocal() maven { url "http://repo.spring.io/libs-snapshot" } mavenCentral() } dependencies { classpath 'org.springframework.boot:spring-boot-gradle-plugin:1.1.5.RELEASE' } } apply plugin: 'java' apply plugin: 'spring-boot' sourceCompatibility = '1.8' repositories { mavenLocal() maven { url 'http://repository.codehaus.org' } maven { url 'http://repo.spring.io/milestone' } mavenCentral() } dependencies { compile "org.springframework.boot:spring-boot-starter-web:1.1.5.RELEASE" compile "org.springframework.boot:spring-boot-starter-redis:1.1.5.RELEASE" compile 'com.google.guava:guava:17.0' compile 'org.apache.commons:commons-lang3:3.3.2' compile 'commons-validator:commons-validator:1.4.0' compile 'org.apache.tomcat.embed:tomcat-embed-el:8.0.9' compile "org.aspectj:aspectjrt:1.8.1" runtime "cglib:cglib-nodep:3.1" } tasks.withType(GroovyCompile) { groovyOptions.optimizationOptions.indy = true } task wrapper(type: Wrapper) { gradleVersion = '2.0' } Actually also 42 lines... That's the whole application, no XML, no descriptors, not setup. I don't treat this exercise as just a dummy code golf for shortest, most obfuscated working code. URL shortener web service with Redis back-end is an interesting showcase of syntax and capabilities of a given language and ecosystem. Much more entertaining then a bunch of algorithmic problems, e.g. found in Rosetta code. Also it's a good bare minimum template for writing a REST service. One important feature of original Scala implementation, that was somehow silently forgotten in all implementations, including this one, is that it's non-blocking. Both HTTP and Redis access is event-driven (reactive, all right, I said it), thus I suppose it can handle tens of thousands of clients simultaneously. This can't be achieved with blocking controllers backed by Tomcat. But still you have to admit such a service written in Java (not even Java 8!) is surprisingly concise, easy to follow and straightforward - none of the other solutions are that readable (this is of course subjective). Waiting for others!