Sometimes there is too much logic to deal with in a single class. You tried extracting methods, but they are so many that the design is still complex to understand. The next step in the refactoring quest is Extract Class, the creation of a new class whose objects will be referenced from the original class. Fields and methods may be moved in the new class, in order for the original to get smaller and more manageable. Why class inflation happens? This refactoring is always caused by classes growing in responsibilities. My personal hypothesis is that we as developers have a bias for the add field|method operations preferring it to the add class. Usually, creating a new class also mean adding an entire new file (hopefully) and more design considerations like its namespace and name. The mental cost for the developer is heavier, but the results are often better than for smaller extractions, as classes can be reused independently; extracted methods are instead clustered together. Moreover, our libraries (e.g. ORMs such as Doctrine 2) reinforce this bias by making significatively more difficult to extract a Value Object (should you serialize it? write a custom DBAL type?) or even another Entity (should I link to it with a @OneToOne? @OneToMany? Which cascade options will work? Which constraints the relationships has?) By the way, this solution is a manifestation of composition over inheritance in the refactoring realm (while there are also other options where the class is made smaller by introducing superclasses instead of an unrelated type.) What are the signs it's time to extract a class? You may encounter a subset of methods and fields that cluster together: for example, they are identified by a prefix; or they have have a temporal coupling which makes them change together faster or slower than the other fields. They may be of similar types (scalar or superclass). Another option to target higher cohesion is to simply see which fields are used together by each method in the class. Fowler's suggestion is to try removing each field (conceptually), and think about which other fields become useless. Repeat this and you'll find the subsets of fields to extract if they exist. Steps Divide responsibilities in source class and extracted class: fields and methods should be classified to one or the two targets. This is true for public and private members, but the latter may change scope to public to be visible from the source class. Create a new class, and check the names of the source and the new one. In the extracted class, you decide the name on the fly; in the source class, you have to change the old name if it's no longer applicable (and later, also the name of references in the system to its objects). It may be the case that the extracted class steals the name of the source one. Place a field in the source class referencing an object (or more) of the extracted class. The field may be initialized in the constructor, and also injected with a constructor parameter if the change is not too invasive. Move Field iteratively from the source to the extracted class. Move Method iteratively. If you bundle steps 4 and 5, you'll be faster, but the point is you should be able to go in smaller steps when it is necessary. Likewise, TDD is taught with baby steps because it gives you the ability to make them when required: everyone is capable of cutting a giant piece of code and fiddling with it for hours until it works again. But that involves a rewriting part, not only refactoring. Reduce the interfaces that each class exposes. Often the extracted one only needs public methods for what the original class uses, while the original class maintains its old protocol to avoiding ripple effects towards the rest of the object graph. In fact, it's not even said that you should expose the extracted class. In many cases, you don't have to; but you'll be able to use it independently by creating other objects, while there is often no need to expose this particular object, composed by the source class (and the Law of Demeter says you shouldn't.) Tests should be executable after each movement of fields or methods. Example In the example, we pass from an initial state where formatting and HTML logic is crammed into the same class: assertEquals('10,000.00', $moneyAmount->toHtml()); } } class MoneyAmount { /** * @param int $amount */ public function __construct($amount) { $this->amount = $amount; } public function toHtml() { $amount = $this->amount; $formatted = ''; while (strlen($amount) > 3) { $cut = strlen($amount) % 3; $cut = $cut == 0 ? 3 : $cut; $formatted .= substr($amount, 0, $cut) . ','; $amount = substr($amount, $cut); } $formatted .= $amount . '.00'; $html = "$formatted"; return $html; } } To two separated classes, one modelling the logical amount and its formatting, one taking care of printing HTML tags. assertEquals('10,000.00', $moneyAmount->toHtml()); } } class MoneySpan { /** * @param int $amount */ public function __construct(MoneyAmount $amount) { $this->amount = $amount; } public function toHtml() { $html = '' . $this->amount->format() . ''; return $html; } } class MoneyAmount { private $amount; public function __construct($amount) { $this->amount = $amount; } public function format() { $amount = $this->amount; $formatted = ''; while (strlen($amount) > 3) { $cut = strlen($amount) % 3; $cut = $cut == 0 ? 3 : $cut; $formatted .= substr($amount, 0, $cut) . ','; $amount = substr($amount, $cut); } return $formatted . $amount . '.00'; } } You can see the four intermediate steps in the Github history of the file.

Learn more about Java collection performance in this post.

More than a few times recently we’ve wanted to use pattern matching on a collection of pairs/tuples and have run into trouble doing so. It’s easy enough if you don’t try and pattern match: > List(("Mark", 4), ("Charles", 5)).filter(pair => pair._2 == 4) res6: List[(java.lang.String, Int)] = List((Mark,4)) But if we try to use pattern matching: List(("Mark", 4), ("Charles", 5)).filter(case(name, number) => number == 4) We end up with this error: :1: error: illegal start of simple expression List(("Mark", 4), ("Charles", 5)).filter(case(name, number) => number == 4) It turns out that we can only use this if we pass the function to filter using {} instead of (): > List(("Mark", 4), ("Charles", 5)).filter { case(name, number) => number == 4 } res7: List[(java.lang.String, Int)] = List((Mark,4)) It was pointed out to me on the Scala IRC channel that the reason for the compilation failure has nothing to do with trying to do a pattern match inside a higher order function but that it’s not actually possible to use a case token without the {}. [23:16] mneedham: hey – trying to understand how pattern matching works inside higher order functions. Don’t quite get this code -> https://gist.github.com/1079110 any ideas? [23:17] dwins: mneedham: scala requires that “case” statements be inside curly braces. nothing to do with higher-order functions [23:17] mneedham: is there anywhere that’s documented or is that just a known thing? [23:18] mneedham: I expected it to work in normal parentheses [23:21] amacleod: mneedham, it’s documented. Whether it’s documented simply as “case statements need to be in curly braces” is another question The first line of Section 8.5 ‘Pattern Matching Anonymous Functions’ of the Scala language spec proves what I was told: Syntax: BlockExpr ::= ‘{’ CaseClauses ‘} It then goes into further detail about how the anonymous function gets converted into a pattern matching statement which is quite interesting reading. From http://www.markhneedham.com/blog/2011/07/12/scala-pattern-matching-a-pair-inside-mapfilter/

Tomcat 7 has been improved a lot and along with everything else that it brings, a very nice feature is provided - an API for embedding Tomcat into your application. The API was provided in earlier versions of Tomcat but it was quite cumbersome to use. To to start the embedded version of Tomcat one may need to build the required JARs. svn co https://svn.apache.org/repos/asf/tomcat/trunk tomcat cd tomcat ant embed-jars ls -l output/embed total 5092 -rw-r--r-- 1 anton None 56802 2011-03-06 17:09 LICENSE -rw-r--r-- 1 anton None 1194 2011-03-06 17:09 NOTICE -rw-r--r-- 1 anton None 1690519 2011-03-06 17:09 ecj-3.6.jar -rw-r--r-- 1 anton None 234625 2011-03-06 17:09 tomcat-dbcp.jar -rw-r--r-- 1 anton None 2402517 2011-03-06 17:09 tomcat-embed-core.jar -rw-r--r-- 1 anton None 781989 2011-03-06 17:09 tomcat-embed-jasper.jar -rw-r--r-- 1 anton None 34106 2011-03-06 17:09 tomcat-embed-logging-juli.jar The following snippet demonstrates the embedded Tomcat usage with a deployed servlet instance. import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import java.io.File; import java.io.IOException; import java.io.Writer; public class Main { public static void main(String[] args) throws LifecycleException, InterruptedException, ServletException { Tomcat tomcat = new Tomcat(); tomcat.setPort(8080); Context ctx = tomcat.addContext("/", new File(".").getAbsolutePath()); Tomcat.addServlet(ctx, "hello", new HttpServlet() { protected void service(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException { Writer w = resp.getWriter(); w.write("Hello, World!"); w.flush(); } }); ctx.addServletMapping("/*", "hello"); tomcat.start(); tomcat.getServer().await(); } } The only two JARs required are tomcat-embed-core.jar and tomcat-embed-logging-juli.jar. It means that there will be no JSP support and pooling will also be disabled. But that's enough to start a servlet and in most cases that is what you probably need. From http://arhipov.blogspot.com/2011/03/embedded-tomcat-minimal-version.html

In my daily work, I use both an RDBMS and MarkLogic, an XML database. MarkLogic can be considered akin to the newer NoSQL databases, but it has the added structure of XML and standard languages in XQuery and XPath. The NoSQL databases are typically storing documents or key-value pairs, and some other things in between. Given that any datastore will be searched at some point, you will always care how the data is actually stored or whether there is some way to query it easily. Once you start thinking about the problem, you quickly generalize to the “how do I persist any type of data” question. However, my focus is not going to be the comparison of the various data stores, but the comparison of how data is stored. More specifically, I want to show the object serialization, mainly the Java built in method, as a data persistence format is evil. Given what you normally read on this blog, this may seem like an oddly timed post, but I have run into serialization issues lately in some production code and Mark Needham recently wrote an interesting post about this as well. Coincidentally, Mark is also working with MarkLogic, and there is an interesting item in his post: The advantage of doing things this way [using lightweight wrappers] is that it means we have less code to write than we would with the serialisation/deserialisation approach although it does mean that we’re strongly coupled to the data format that our storage mechanism uses. However, since this is one bit of the architecture which is not going to change it seems to makes sense to accept the leakage of that layer. The interesting part of this is that he has accepted using the data format of the storage mechanism, XML in MarkLogic in this case. Why is this interesting? First, it is a move away from the ORM technologies that try to hide the complexities of converting data into objects in the RDBMS world. Also, this is a glimpse into the types of issues that could arise from non-RDBMS storage choices as well as how to persist objects in general. So, an RDBMS is typically used to map object attributes to a table and columns. The mapping is mostly straightforward with some defined relationship for child objects and collections. This is a well-known area, called Object-Relational Mapping (ORM), and several open source and commercial options exist. In this scenario, object attributes are stored in a similar datatype, meaning a String is stored as a varchar and an int is stored as an integer. But, what happens when you move away from an RDBMS for data persistence? If you look at Java and its session objects, pure object serialization is used. Assuming that an application session is fairly short-lived, meaning at most a few hours, object serialization is simple, well supported and built into the Java concept of a session. However, when the data persistence is over a longer period of time, possibly days or weeks, and you have to worry about new releases of the application, serialization quickly becomes evil. As any good Java developer knows, if you plan to serialize an object, even in a session, you need a real serialization ID (serialVersionUID), not just a 1L, and you need to implement the Serializable interface. However, most developers do not know the real rules behind the Java deserialization process. If your object has changed, more than just adding simple fields to the object, it is possible that Java cannot deserialize the object correctly even if the serialization ID has not changed. Suddenly, you cannot retrieve your data any longer, which is inherently bad. Now, may developers reading this may say that they would never write code that would have this problem. That may be true, but what about a library that you use or some other developer no longer employed by your company? Can you guarantee that this problem will never happen? The only way to guarantee that is to use a different serialization method. What options do we have? Obviously, there are the NoSQL datastores but the actual object format is the relevant question not which solution to choose. Besides the obvious serialized object, some NoSQL datastores use JSON to store objects, MarkLogic uses XML and there are others that store just key-value pairs. Key-value pairs are typically a mapping of a text key to a value that is a serialized object, either a binary or textual format. So, that leaves us with XML, JSON and other textual formats. One of the benefits of a structured format like XML or JSON is that they can be made searchable and provide some level of context. I have talked about data formats before, so I won’t go into a comparison again. However, do these types of formats avoid the issues that native Java object serialization has? This is really dependent upon what library you are using for serialization. Some libraries will deserialize an object without any issues regardless of whether the object field list has changed. Other libraries could have problems depending upon whether a serialized field exists in the target object, or there might not be solid support for collections (though that is doubtful at this point). Given that even structured formats could have serialization issues, is the only safe path hand-coded mappings like those used by ORM tools? Some JSON and XML serialization tools use the same mapping methods as the ORM tools in order to avoid these problems. However, once you define these mappings, you are explicitly stating how an object gets translated. This explicit definition will require maintenance, but that is definitely cleaner than trying to trace down a serialization defect in some random stack trace. So is implicit object serialization really worth the potential headaches? Or should we just consider it evil and never speak of it again? From http://regulargeek.com/2011/07/06/is-object-serialization-evil/

Today I'll talk about a famous problem : restarting a Java application. It is especially useful when changing the language of a GUI application, so that we need to restart it to reload the internationalized messages in the new language. Some look and feel also require to relaunch the application to be properly applied. A quick Google search give plenty answers using a simple : Runtime.getRuntime().exec("java -jar myApp.jar"); System.exit(0); This indeed basically works, but this answer that does not convince me for several reasons : 1) What about VM and program arguments ? (this one is secondary in fact, because can be solve it quite easily). 2) What if the main is not a jar (which is usually the case when launching from an IDE) ? 3) Most of all, what about the cleaning of the closing application ? For example if the application save some properties when closing, commit some stuffs etc. 4) We need to change the command line in the code source every time we change a parameter, the name of the jar, etc. Overall, something that works fine for some test, sandbox use, but not a generic and elegant way in my humble opinion. Ok, so my purpose here is to implement a method : public static void restartApplication(Runnable runBeforeRestart) throws IOException { ... } that could be wrapped in some jar library, and could be called, without any code modification, by any Java program, and by solving the 4 points raised previously. Let's start by looking at each point and find a way to answer them in an elegant way (let's say the most elegant way that I found). 1) How to get the program and VM arguments ? Pretty simple, by calling a : ManagementFactory.getRuntimeMXBean().getInputArguments(); Concerning the program arguments, the Java property sun.java.command we'll give us both the main class (or jar) and the program arguments, and both will be useful. String[] mainCommand = System.getProperty("sun.java.command").split(" "); 2) First retrieve the java bin executable given by the java.home property : String java = System.getProperty("java.home") + "/bin/java"; The simple case is when the application is launched from a jar. The jar name is given by a mainCommand[0], and it is in the current path, so we just have to append the application parameters mainCommand[1..n] with a -jar to get the command to execute : String cmd = java + vmArgsOneLine + "-jar " + new File(mainCommand[0]).getPath() + mainCommand[1..n]; We'll suppose here that the Manifest of the jar is well done, and we don't need to specify the main nor the classpath. Second case : when the application is launched from a class. In this case, we'll specify the class path and the main class : String cmd = java + vmArgsOneLine + -cp \"" + System.getProperty("java.class.path") + "\" " + mainCommand[0] + mainCommand[1..n]; 3) Third point, cleaning the old application before launching the new one. To do such a trick, we'll just execute the Runtime.getRuntime().exec(cmd) in a shutdown hook. This way, we'll be sure that everything will be properly clean up before creating the new application instance. Runtime.getRuntime().addShutdownHook(new Thread() { @Override public void run() { ... } }); Run the runBeforeRestart that contains some custom code that we want to be executed before restarting the application : if(runBeforeRestart != null) { runBeforeRestart.run(); } And finally, call the System.exit(0);. And we're done. Here is our generic method : /** * Sun property pointing the main class and its arguments. * Might not be defined on non Hotspot VM implementations. */ public static final String SUN_JAVA_COMMAND = "sun.java.command"; /** * Restart the current Java application * @param runBeforeRestart some custom code to be run before restarting * @throws IOException */ public static void restartApplication(Runnable runBeforeRestart) throws IOException { try { // java binary String java = System.getProperty("java.home") + "/bin/java"; // vm arguments List vmArguments = ManagementFactory.getRuntimeMXBean().getInputArguments(); StringBuffer vmArgsOneLine = new StringBuffer(); for (String arg : vmArguments) { // if it's the agent argument : we ignore it otherwise the // address of the old application and the new one will be in conflict if (!arg.contains("-agentlib")) { vmArgsOneLine.append(arg); vmArgsOneLine.append(" "); } } // init the command to execute, add the vm args final StringBuffer cmd = new StringBuffer("\"" + java + "\" " + vmArgsOneLine); // program main and program arguments String[] mainCommand = System.getProperty(SUN_JAVA_COMMAND).split(" "); // program main is a jar if (mainCommand[0].endsWith(".jar")) { // if it's a jar, add -jar mainJar cmd.append("-jar " + new File(mainCommand[0]).getPath()); } else { // else it's a .class, add the classpath and mainClass cmd.append("-cp \"" + System.getProperty("java.class.path") + "\" " + mainCommand[0]); } // finally add program arguments for (int i = 1; i < mainCommand.length; i++) { cmd.append(" "); cmd.append(mainCommand[i]); } // execute the command in a shutdown hook, to be sure that all the // resources have been disposed before restarting the application Runtime.getRuntime().addShutdownHook(new Thread() { @Override public void run() { try { Runtime.getRuntime().exec(cmd.toString()); } catch (IOException e) { e.printStackTrace(); } } }); // execute some custom code before restarting if (runBeforeRestart!= null) { runBeforeRestart.run(); } // exit System.exit(0); } catch (Exception e) { // something went wrong throw new IOException("Error while trying to restart the application", e); } } From : http://lewisleo.blogspot.jp/2012/08/programmatically-restart-java.html

One thing I’m doing very often and always searching on the Internet is how to obtain a self-signed SSL certificate and install it in both my client browsers and my local Tomcat. Sure enough there are enough resources available online, but since it’s a bore to go looking for the right one (yes, some do not work), I figured let’s do it right once and document it so that it will always be there. Create the keystore Keystores are, guess what, files where your store your keys. In our case, we need to create one that will be used by both Tomcat and for the certificat generation. The command-line is: keytool -genkey -keyalg RSA -alias blog.frankel.ch -keystore keystore.jks -validity 999 -keysize 2048 The parameters are as follow: Parameter Value Description -genkey Requests the keytool to generate a key. For all provided features, type keytool -help -keyalg RSA Wanted algorithm. The specified algorithm must be made available by one of the registered cryptographic service providers -keysize 2048 Key size -validity 999 Validity in days -alias blog.frankel.ch Entry in the keystore -keystore keystore.jks Keystore. If the keystore doesn’t exist yet, it will be created and you’ll be prompted for a new password; otherwise, you’ll prompted for the current store’s password Configure Tomcat Tomcat’s SSL configuration is done in the ${TOMCAT_HOME}/conf/server.xml file. Locate the following snippet: Now, uncomment it and add the following attributes: keystoreFile="/path/to/your/keystore.jks" keystorePass="Your password" Note: if the store only contains a single entry, fine; otherwise, you’ll need to configure the entry’s name with keyAlias="blog.frankel.ch" Starting Tomcat and browsing to https://localhost:8443/ will show you Tomcat’s friendly face. Additionnaly, the logs will display: 28 june 2011 20:25:14 org.apache.coyote.AbstractProtocolHandler init INFO: Initializing ProtocolHandler ["http-bio-8443"] Export the certificate The certificate is created from our previous entry in the keystore. The command-line is: keytool -export -alias blog.frankel.ch -file blog.frankel.ch.crt -keystore keystore.jks Even simpler, we are challenged for the keystore’s password and that’s all. The newly created certificate is now available in the filesystem. We just have to distribute it to all browsers that will connect to Tomcat in order to bypass security warnings (since it’s a self-signed certificate). Spread the word The last step is to put the self-signed certificate in the list of trusted certificates in Firefox. For a quick and dirty way, import it in your own Firefox (Options -> Advanced -> Show certificates -> Import…) and distribute the %USER_HOME%"/Application Data/Mozilla/Firefox/Profiles/xzy.default/cert8.db file. It has to be copied to the %FIREFOX_HOME%/defaults/profile folder so that every single profile on the target machine is updated. Note that this way of doing will lose previously individually accepted certificates (in short, we’re overwriting the whole certificate database). For a more industrial process, look at the next section. To go further: The Most Common Java Keytool Keystore Commands Tomcat 7 SSL Configuration HOW-TO Where can I download certutil.exe for Windows From http://blog.frankel.ch/ssl-your-tomcat-7

This tutorial will walk you through how to configure SSL (https://localhost:8443 access) on Tomcat in 5 minutes. For this tutorial you will need: Java SDK (used version 6 for this tutorial) Tomcat (used version 7 for this tutorial) The set up consists in 3 basic steps: Create a keystore file using Java Configure Tomcat to use the keystore Test it (Bonus ) Configure your app to work with SSL (access through https://localhost:8443/yourApp) 1 – Creating a Keystore file using Java Fisrt, open the terminal on your computer and type: Windows: cd %JAVA_HOME%/bin Linux or Mac OS: cd $JAVA_HOME/bin The $JAVA_HOME on Mac is located on “/System/Library/Frameworks/JavaVM.framework/Versions/{your java version}/Home/” You will change the current directory to the directory Java is installed on your computer. Inside the Java Home directory, cd to the bin folder. Inside the bin folder there is a file named keytool. This guy is responsible for generating the keystore file for us. Next, type on the terminal: keytool -genkey -alias tomcat -keyalg RSA When you type the command above, it will ask you some questions. First, it will ask you to create a password (My password is “password“): loiane:bin loiane$ keytool -genkey -alias tomcat -keyalg RSA Enter keystore password: password Re-enter new password: password What is your first and last name? [Unknown]: Loiane Groner What is the name of your organizational unit? [Unknown]: home What is the name of your organization? [Unknown]: home What is the name of your City or Locality? [Unknown]: Sao Paulo What is the name of your State or Province? [Unknown]: SP What is the two-letter country code for this unit? [Unknown]: BR Is CN=Loiane Groner, OU=home, O=home, L=Sao Paulo, ST=SP, C=BR correct? [no]: yes Enter key password for (RETURN if same as keystore password): password Re-enter new password: password It will create a .keystore file on your user home directory. On Windows, it will be on: C:Documents and Settings[username]; on Mac it will be on /Users/[username] and on Linux will be on /home/[username]. 2 – Configuring Tomcat for using the keystore file – SSL config Open your Tomcat installation directory and open the conf folder. Inside this folder, you will find the server.xml file. Open it. Find the following declaration: Uncomment it and modify it to look like the following: Connector SSLEnabled="true" acceptCount="100" clientAuth="false" disableUploadTimeout="true" enableLookups="false" maxThreads="25" port="8443" keystoreFile="/Users/loiane/.keystore" keystorePass="password" protocol="org.apache.coyote.http11.Http11NioProtocol" scheme="https" secure="true" sslProtocol="TLS" /> Note we add the keystoreFile, keystorePass and changed the protocol declarations. 3 – Let’s test it! Start tomcat service and try to access https://localhost:8443. You will see Tomcat’s local home page. Note if you try to access the default 8080 port it will be working too: http://localhost:8080 4 – BONUS - Configuring your app to work with SSL (access through https://localhost:8443/yourApp) To force your web application to work with SSL, you simply need to add the following code to your web.xml file (before web-app tag ends): securedapp /* CONFIDENTIAL The url pattern is set to /* so any page/resource from your application is secure (it can be only accessed with https). The transport-guarantee tag is set to CONFIDENTIAL to make sure your app will work on SSL. If you want to turn off the SSL, you don’t need to delete the code above from web.xml, simply change CONFIDENTIAL to NONE. Reference: http://tomcat.apache.org/tomcat-7.0-doc/ssl-howto.html (this tutorial is a little confusing, that is why I decided to write another one my own). Happy Coding! From http://loianegroner.com/2011/06/setting-up-ssl-on-tomcat-in-5-minutes-httpslocalhost8443/

After one month of evaluation of frameworks and tools, I choose ExtJS for UI and Spring/SpringMVC for the business layer for my pet project. By using ExtJS we can send data to server by form submits or as request parameters, or in JSON format through Ajax requests. ExtJS uses JSON format in many situations to hold data. So I thought using JSON as data exchange format between EXTJS and Spring would be consistent. The following code snippets explain how we can use ExtJS and SpringMVC3 to exchange data in JSON format. 1. Register MappingJacksonHttpMessageConverter in dispatcher-servlet.xml Add jackson-json jar(s) to WEB-INF/lib 2. Trigger the POST request from ExtJS script as follows: Ext.Ajax.request({ url : 'doSomething.htm', method: 'POST', headers: { 'Content-Type': 'application/json' }, params : { "test" : "testParam" }, jsonData: { "username" : "admin", "emailId" : "[email protected]" }, success: function (response) { var jsonResp = Ext.util.JSON.decode(response.responseText); Ext.Msg.alert("Info","UserName from Server : "+jsonResp.username); }, failure: function (response) { var jsonResp = Ext.util.JSON.decode(response.responseText); Ext.Msg.alert("Error",jsonResp.error); } }); 3. Write a Spring Controller to handle the "/doSomething.htm" reguest. @Controller public class DataController { @RequestMapping(value = "/doSomething", method = RequestMethod.POST) @ResponseBody public User handle(@RequestBody User user) throws IOException { System.out.println("Username From Client : "+user.getUsername()); System.out.println("EmailId from Client : "+user.getEmailId()); user.setUsername("SivaPrasadReddy"); user.setEmailId("[email protected]"); return user; } } Any other better approaches? From http://sivalabs.blogspot.com/2011/06/how-to-post-and-get-json-between-extjs.html

Learn how to survive 16GiB and greater heaps, and control Java GC pauses.

// Convert XML to JSON using Ruby and ActiveSupport #! /usr/bin/env ruby require 'rubygems' require 'active_support/core_ext' require 'json' xml = File.open(ARGV.first).read json = Hash.from_xml(xml).to_json File.open(ARGV.last, 'w+').write json

Sometimes when you are parsing a date string against a pattern(such as MM/dd/yyyy) using java.text.SimpleDateFormat, strange things might happen (for unknown developers) if your date string is dynamic content entered by a user in some input field on the user interface and if it is not entered in the specified format. The parse method in the SimpleDateFormat parses the date string that is in the incorrect format and returns your date object instead of throwing a java.text.ParseException. However, the date returned is not what you expect. The below code-snippet shows you this behaviour. package com.starwood.system.util; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Date; public class DateSample { public static void main(String args[]){ SimpleDateFormat sdf = new SimpleDateFormat () ; sdf.applyPattern("MM/dd/yyyy") ; try { Date d = sdf.parse("2011/02/06") ; System.out.println(d) ; } catch (ParseException e) { e.printStackTrace(); } } } Output: Thu Jul 02 00:00:00 MST 173 See the output, that is a date back in the year 173. To avoid this problem, call the setLenient (false) on SimpleDateFormat instance. That will make the parse method throw ParseException when the given input string is not in the specified format. Here is the modified code-snippet. package com.starwood.system.util; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Date; public class DateSample { public static void main(String args[]){ SimpleDateFormat sdf = new SimpleDateFormat () ; sdf.applyPattern("MM/dd/yyyy") ; sdf.setLenient(false) ; try { Date d = sdf.parse("2011/02/06") ; System.out.println(d) ; } catch (ParseException e) { System.out.println (e.getMessage()) ; } } } Output: Unparseable date: "2011/02/06" http://accordess.com/wpblog/2011/06/02/techtip-use-of-setlenient-method-on-simpledateformat

towards the end of last week i started thinking how to deal with large amounts of xml data in a resource-friendly way.the main problem that i wanted to solve was how to process large xml files in chunks while at the same time providing upstream/downstream systems with some data to process. of course i've been using jaxb technology for few years now; the main advantage of using jaxb is the quick time-to-market; if one possesses an xml schema, there are tools out there to auto-generate the corresponding java domain model classes automatically (eclipse indigo, maven jaxb plugins in various sauces, ant tasks, to name a few). the jaxb api then offers a marshaller and an unmarshaller to write/read xml data, mapping the java domain model. when thinking of jaxb as solution for my problem i suddendlly realised that jaxb keeps the whole objectification of the xml schema in memory, so the obvious question was: "how would our infrastructure cope with large xml files (e.g. in my case with a number of elements > 100,000) if we were to use jaxb?". i could have simply produced a large xml file, then a client for it and find out about memory consumption. as one probably knows there are mainly two approaches to processing xml data in java: dom and sax. with dom, the xml document is represented into memory as a tree; dom is useful if one needs cherry-pick access to the tree nodes or if one needs to write brief xml documents. on the other side of the spectrum there is sax, an event-driven technology, where the whole document is parsed one xml element at the time, and for each xml significative event, callbacks are "pushed" to a java client which then deals with them (such as start_document, start_element, end_element, etc). since sax does not bring the whole document into memory but it applies a cursor like approach to xml processing it does not consume huge amounts of memory. the drawback with sax is that it processes the whole document start to finish; this might not be necessarily what one wants for large xml documents. in my scenario, for instance, i'd like to be able to pass to downstream systems xml elements as they are available, but at the same time maybe i'd like to pass only 100 elements at the time, implementing some sort of pagination solution. dom seems too demanding from a memory-consumption point of view, whereas sax seems to coarse-grained for my needs. i remembered reading something about stax, a java technology which offered a middle ground between the capability to pull xml elements (as opposed to pushing xml elements, e.g. sax) while being ram-friendly. i then looked into the technology and decided that stax was probably the compromise i was looking for; however i wanted to keep the easy programming model offered by jaxb, so i really needed a combination of the two. while investigating stax, i came across woodstox; this open source project promises to be a faster xml parser than many othrers, so i decided to include it in my benchmark as well. i now had all elements to create a benchmark to give me memory consumption and processing speed metrics when processing large xml documents. the benchmark plan in order to create a benchmark i needed to do the following: create an xml schema which defined my domain model. this would be the input for jaxb to create the java domain model create three large xml files representing the model, with 10,000 / 100,000 / 1,000,000 elements respectively have a pure jaxb client which would unmarshall the large xml files completely in memory have a stax/jaxb client which would combine the low-memory consumption of sax technologies with the ease of programming model offered by jaxb have a woodstox/jaxb client with the same characteristics of the stax/jaxb client (in few words i just wanted to change the underlying parser and see if i could obtain any performance boost) record both memory consumption and speed of processing (e.g. how quickly would each solution make xml chunks available in memory as jaxb domain model classes) make the results available graphically, since, as we know, one picture tells one thousands words. the domain model xml schema i decided for a relatively easy domain model, with xml elements representing people, with their names and addresses. i also wanted to record whether a person was active. using jaxb to create the java model i am a fan of maven and use it as my default tool to build systems. this is the pom i defined for this little benchmark: 4.0.0 uk.co.jemos.tests.xml large-xml-parser 1.0.0-snapshot jar large-xml-parser http://www.jemos.co.uk utf-8 org.apache.maven.plugins maven-compiler-plugin 2.3.2 1.6 1.6 org.jvnet.jaxb2.maven2 maven-jaxb2-plugin 0.7.5 generate ${basedir}/src/main/resources **/*.xsd true -enableintrospection -xtostring -xequals -xhashcode true true org.jvnet.jaxb2_commons jaxb2-basics 0.6.1 org.apache.maven.plugins maven-jar-plugin 2.3.1 true uk.co.jemos.tests.xml.xmlpullbenchmarker org.apache.maven.plugins maven-assembly-plugin 2.2 ${project.build.directory}/site/downloads src/main/assembly/project.xml src/main/assembly/bin.xml junit junit 4.5 test uk.co.jemos.podam podam 2.3.11.release commons-io commons-io 2.0.1 com.sun.xml.bind jaxb-impl 2.1.3 org.jvnet.jaxb2_commons jaxb2-basics-runtime 0.6.0 org.codehaus.woodstox stax2-api 3.0.3 just few things to notice about this pom.xml. i use java 6, since starting from version 6, java contains all the xml libraries for jaxb, dom, sax and stax. to auto-generate the domain model classes from the xsd schema, i used the excellent maven-jaxb2-plugin, which allows, amongst other things, to obtain pojos with tostring, equals and hashcode support. i have also declared the jar plugin, to create an executable jar for the benchmark and the assembly plugin to distribute an executable version of the benchmark. the code for the benchmark is attached to this post, so if you want to build it and run it yourself, just unzip the project file, open a command line and run: $ mvn clean install assembly:assembly this command will place *-bin.* files into the folder target/site/downloads. unzip the one of your preference and to run the benchmark use (-dcreate.xml=true will generate the xml files. don't pass it if you have these files already, e.g. after the first run): $ java -jar -dcreate.xml=true large-xml-parser-1.0.0-snapshot.jar creating the test data to create the test data, i used podam , a java tool to auto-fill pojos and javabeans with data. the code is as simple as: jaxbcontext context = jaxbcontext .newinstance("xml.integration.jemos.co.uk.large_file"); marshaller marshaller = context.createmarshaller(); marshaller.setproperty(marshaller.jaxb_formatted_output, boolean.true); marshaller.setproperty(marshaller.jaxb_encoding, "utf-8"); personstype personstype = new objectfactory().createpersonstype(); list persons = personstype.getperson(); podamfactory factory = new podamfactoryimpl(); for (int i = 0; i < nbrelements; i++) { persons.add(factory.manufacturepojo(persontype.class)); } jaxbelement towrite = new objectfactory() .createpersons(personstype); file file = new file(filename); bufferedoutputstream bos = new bufferedoutputstream( new fileoutputstream(file), 4096); try { marshaller.marshal(towrite, bos); bos.flush(); } finally { ioutils.closequietly(bos); } the xmlpullbenchmarker generates three large xml files under ~/xml-benchmark: large-person-10000.xml (approx 3m) large-person-100000.xml (approx 30m) large-person-1000000.xml (approx 300m) each file looks like the following: ult6yn0d7l u8djoutlk2 dxwlpow6x3 o4ggvximo7 io7kuz0xmz lmiy1uqkxs zhtukbtwti gbc7kex9tn kxmwnlprep 9bibs1m5gr hmtqpxjcpw bhpf1rrldm ydjjillyrw xgstdjcfjc [..etc] each file contains 10,000 / 100,000 / 1,000,000 elements. the running environments i tried the benchmarker on three different environments: ubuntu 10, 64-bit running as virtual machine on a windows 7 ultimate, with cpu i5, 750 @2.67ghz and 2.66ghz, 8gb ram of which 4gb dedicated to the vm. jvm: 1.6.0_25, hotspot windows 7 ultimate , hosting the above vm, therefore with same processor. jvm, 1.6.0_24, hotspot ubuntu 10, 32-bit , 3gb ram, dual core. jvm, 1.6.0_24, openjdk the xml unmarshalling to unmarshall the code i used three different strategies: pure jaxb stax + jaxb woodstox + jaxb pure jaxb unmarshalling the code which i used to unmarshall the large xml files using jaxb follows: private void readlargefilewithjaxb(file file, int nbrrecords) throws exception { jaxbcontext ucontext = jaxbcontext .newinstance("xml.integration.jemos.co.uk.large_file"); unmarshaller unmarshaller = ucontext.createunmarshaller(); bufferedinputstream bis = new bufferedinputstream(new fileinputstream( file)); long start = system.currenttimemillis(); long memstart = runtime.getruntime().freememory(); long memend = 0l; try { jaxbelement root = (jaxbelement) unmarshaller .unmarshal(bis); root.getvalue().getperson().size(); memend = runtime.getruntime().freememory(); long end = system.currenttimemillis(); log.info("jaxb (" + nbrrecords + "): - total memory used: " + (memstart - memend)); log.info("jaxb (" + nbrrecords + "): time taken in ms: " + (end - start)); } finally { ioutils.closequietly(bis); } } the code uses a one-liner to unmarshall each xml file: jaxbelement root = (jaxbelement) unmarshaller .unmarshal(bis); i also accessed the size of the underlying persontype collection to "touch" in memory data. btw, debugging the application showed that all 10,000 elements were indeed available in memory after this line of code. jaxb + stax with stax, i just had to use an xmlstreamreader, iterate through all elements, and pass each in turn to jaxb to unmarshall it into a persontype domain model object. the code follows: // set up a stax reader xmlinputfactory xmlif = xmlinputfactory.newinstance(); xmlstreamreader xmlr = xmlif .createxmlstreamreader(new filereader(file)); jaxbcontext ucontext = jaxbcontext.newinstance(persontype.class); unmarshaller unmarshaller = ucontext.createunmarshaller(); long start = system.currenttimemillis(); long memstart = runtime.getruntime().freememory(); long memend = 0l; try { xmlr.nexttag(); xmlr.require(xmlstreamconstants.start_element, null, "persons"); xmlr.nexttag(); while (xmlr.geteventtype() == xmlstreamconstants.start_element) { jaxbelement pt = unmarshaller.unmarshal(xmlr, persontype.class); if (xmlr.geteventtype() == xmlstreamconstants.characters) { xmlr.next(); } } memend = runtime.getruntime().freememory(); long end = system.currenttimemillis(); log.info("stax - (" + nbrrecords + "): - total memory used: " + (memstart - memend)); log.info("stax - (" + nbrrecords + "): time taken in ms: " + (end - start)); } finally { xmlr.close(); } } note that this time when creating the context, i had to specify that it was for the persontype object, and when invoking the jaxb unmarshalling i had to pass also the desired returned class type, with: jaxbelement pt = unmarshaller.unmarshal(xmlr, persontype.class); note that i don't to anything with the object, just create it, to keep the benchmark as truthful and possible by not introducing any unnecessary steps. jaxb + woodstox with woodstox, the approach is very similar to the one used with stax. in fact woodstox provides a stax2 compatible api, so all i had to do was to provide the correct factory and...bang! i had woodstox under the cover working. private void readlargexmlwithfasterstax(file file, int nbrrecords) throws factoryconfigurationerror, xmlstreamexception, filenotfoundexception, jaxbexception { // set up a woodstox reader xmlinputfactory xmlif = xmlinputfactory2.newinstance(); xmlstreamreader xmlr = xmlif .createxmlstreamreader(new filereader(file)); jaxbcontext ucontext = jaxbcontext.newinstance(persontype.class); unmarshaller unmarshaller = ucontext.createunmarshaller(); long start = system.currenttimemillis(); long memstart = runtime.getruntime().freememory(); long memend = 0l; try { xmlr.nexttag(); xmlr.require(xmlstreamconstants.start_element, null, "persons"); xmlr.nexttag(); while (xmlr.geteventtype() == xmlstreamconstants.start_element) { jaxbelement pt = unmarshaller.unmarshal(xmlr, persontype.class); if (xmlr.geteventtype() == xmlstreamconstants.characters) { xmlr.next(); } } memend = runtime.getruntime().freememory(); long end = system.currenttimemillis(); log.info("woodstox - (" + nbrrecords + "): total memory used: " + (memstart - memend)); log.info("woodstox - (" + nbrrecords + "): time taken in ms: " + (end - start)); } finally { xmlr.close(); } } note the following line: xmlinputfactory xmlif = xmlinputfactory2.newinstance(); where i pass in a stax2 xmlinputfactory. this uses the woodstox implementation. the main loop once the files are in place (you obtain this by passing -dcreate.xml=true), the main performs the following: system.gc(); system.gc(); for (int i = 0; i < 10; i++) { main.readlargefilewithjaxb(new file(output_folder + file.separatorchar + "large-person-10000.xml"), 10000); main.readlargefilewithjaxb(new file(output_folder + file.separatorchar + "large-person-100000.xml"), 100000); main.readlargefilewithjaxb(new file(output_folder + file.separatorchar + "large-person-1000000.xml"), 1000000); main.readlargexmlwithstax(new file(output_folder + file.separatorchar + "large-person-10000.xml"), 10000); main.readlargexmlwithstax(new file(output_folder + file.separatorchar + "large-person-100000.xml"), 100000); main.readlargexmlwithstax(new file(output_folder + file.separatorchar + "large-person-1000000.xml"), 1000000); main.readlargexmlwithfasterstax(new file(output_folder + file.separatorchar + "large-person-10000.xml"), 10000); main.readlargexmlwithfasterstax(new file(output_folder + file.separatorchar + "large-person-100000.xml"), 100000); main.readlargexmlwithfasterstax(new file(output_folder + file.separatorchar + "large-person-1000000.xml"), 1000000); } it invites the gc to run, although as we know this is at the gc thread discretion. it then executes each strategy 10 times, to normalise ram and cpu consumption. the final data are then collected by running an average on the ten runs. the benchmark results for memory consumption here follow some diagrams which show memory consumption across the different running environments, when unmarshalling 10,000 / 100,000 / 1,000,000 files. you will probably notice that memory consumption for stax-related strategies often shows a negative value. this means that there was more free memory after unmarshalling all elements than there was at the beginning of the unmarshalling loop; this, in turn, suggests that the gc ran a lot more with stax than with jaxb. this is logical if one thinks about it; since with stax we don't keep all objects into memory there are more objects available for garbage collection. in this particular case i believe the persontype object created in the while loop gets eligible for gc and enters the young generation area and then it gets reclamed by the gc. this, however, should have a minimum impact on performance, since we know that claiming objects from the young generation space is done very efficiently. summary for 10,000 xml elements summary for 100,000 xml elements summary for 1,000,000 xml elements the benchmark results for processing speed results for 10,000 elements results for 100,000 elements results for 1,000,000 elements conclusions the results on all three different environments, although with some differences, all tell us the same story: if you are looking for performance (e.g. xml unmarshalling speed), choose jaxb if you are looking for low-memory usage (and are ready to sacrifice some performance speed), then use stax. my personal opinion is also that i wouldn't go for woodstox, but i'd choose either jaxb (if i needed processing power and could afford the ram) or stax (if i didn't need top speed and was low on infrastructure resources). both these technologies are java standards and part of the jdk starting from java 6. resources benchmarker source code zip version: download large-xml-parser-1.0.0-snapshot-project tar.gz version: download large-xml-parser-1.0.0-snapshot-project.tar tar.bz2 version: download large-xml-parser-1.0.0-snapshot-project.tar benchmarker executables: zip version: download large-xml-parser-1.0.0-snapshot-bin tar.gz version: download large-xml-parser-1.0.0-snapshot-bin.tar tar.bz2 version: download large-xml-parser-1.0.0-snapshot-bin.tar data files: ubuntu 64-bit vm running environment: download stax-vs-jaxb-ubuntu-64-vm ubuntu 32-bit running environment : download stax-vs-jaxb-ubuntu-32-bit windows 7 ultimate running environment : download stax-vs-jaxb-windows7 from http://tedone.typepad.com/blog/2011/06/unmarshalling-benchmark-in-java-jaxb-vs-stax-vs-woodstox.html

One of the biggest promises java EE6 made, was to ease the use of dependency injection. They did, using CDI. CDI, which stands for Contexts and Dependency Injection for Java EE, offers a base set to apply dependency injection in your enterprise application. Before CDI, EJB 3 also introduced dependency injection, but this was a bit basic. You could inject an EJB (statefull or stateless) into another EJB or Servlet (if you container supported this). Offcourse not every application needs EJB’s, that is why CDI is gaining so much popularity. To start, I have made this example. There is a Payment interface, and 2 implementations. A cash payment and a visa payment. I want to be able to choose witch type of payment I inject, still using the same interface. public interface Payment { void pay(BigDecimal amount); } and the 2 implementations public class CashPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(CashPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} cash", amount.toString()); } } public class VisaPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(VisaPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} with visa", amount.toString()); } } To inject the interface we use the @Inject annotation. The annotation does basically what it says. It injects a component, that is available in your application. 1 @Inject private Payment payment; Off course, you saw this coming from a mile away, this won’t work. The container has 2 implementations of our Payment interface, so he does not know which one to inject. Unsatisfied dependencies for type [Payment] with qualifiers [@Default] at injection point [[field] @Inject private be.styledideas.blog.qualifier.web.PaymentBackingAction.payment] So we need some sort of qualifier to point out what implementation we want. CDI offers the @Named Annotation, allowing you to give a name to an implementation. @Named("cash") public class CashPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(CashPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} cash", amount.toString()); } } @Named("visa") public class VisaPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(VisaPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} with visa", amount.toString()); } } When we now change our injection code, we can specify wich implementation we need. @Inject private @Named("visa") Payment payment; This works, but the flexibility is limited. When we want to rename our @Named parameter, we have to change it on everyplace where it is used. There is also no refactoring support. There is a beter alternative using Custom made annotations using the @Qualifier annotation. Let us change the code a little bit. First of all, we create new Annotation types. @java.lang.annotation.Documented @java.lang.annotation.Retention(RetentionPolicy.RUNTIME) @javax.inject.Qualifier public @interface CashPayment {} @java.lang.annotation.Documented @java.lang.annotation.Retention(RetentionPolicy.RUNTIME) @javax.inject.Qualifier public @interface VisaPayment {} The @Qualifier annotation that is added to the annotation, makes this annotation discoverable by the container. We can now simply add these annotations to our implementations. @CashPayment public class CashPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(CashPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} cash", amount.toString()); } } @VisaPayment public class VisaPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(VisaPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} with visa", amount.toString()); } } The only thing we now need to do, is change our injection code to @Inject private @VisaPayment Payment payment; When we now change something to our qualifier, we have nice compiler and refactoring support. This also adds extra flexibilty for API or Domain-specific language design. From http://styledideas.be/blog/2011/06/16/java-ee6-cdi-named-components-and-qualifiers/

Just a quick post to help anyone struggling with this error message, as this issue gets raised from time to time on support forums. The reason for the error is usually that you’re attempting to use empty or isset on a function instead of a variable. While it may be obvious that this doesn’t make sense for isset(), the same cannot be said for empty(). You simply meant to check if the value returned from the function was an empty value; why shouldn’t you be able to do just that? The reason is that empty($foo) is more or less syntactic sugar for isset($foo) && $foo. When written this way you can see that the isset() part of the statement doesn’t make sense for functions. This leaves us with simply the $foo part. The solution is to actually just drop the empty() part: Instead of: if (empty($obj->method())) { } Simply drop the empty construct: if ($obj->method()) { }

We have a variety of val, lazy val and def definitions across our code base but have been led to believe that idiomatic Scala would have us using lazy val as frequently as possible. As far as I understand so far this is what the different things do: val evaluates as soon as you initialise the object and stores the result. lazy val evaluates the first time that it’s accessed and stores the result. def executes the piece of code every time – pretty much like a Java method would. In Java, C# or Ruby I would definitely favour the 3rd option because it reduces the amount of state that an object has to hold. I’m not sure that having that state matters so much in Scala because all the default data structures we use are immutable so you can’t do any harm by having access to them. I recently read an interesting quote from Rich Hickey which seems applicable here: To the extent the data is immutable, there is little harm that can come of providing access, other than that someone could come to depend upon something that might change. Well, okay, people do that all the time in real life, and when things change, they adapt. If the data was mutable then it would be possible to change it from any other place in the class which would make it difficult to reason about the object because the data might be in an unexpected state. If we define something as a val in Scala then it’s not even possible to change the reference to that value so it doesn’t seem problematic. Perhaps I just require a bit of a mind shift to not worry so much about state if it’s immutable. It’s only been a few weeks so I’d be interested to hear the opinions of more seasoned Scala users. — I’ve read that there are various performance gains to be had from making use of lazy val or def depending on the usage of the properties but that would seem to be a premature optimisation so we haven’t been considering it so far. From http://www.markhneedham.com/blog/2011/06/22/scala-val-lazy-val-and-def/

web application security is an important part of developing applications. as developers, i think we often forget this, or simply ignore it. in my career, i've learned a lot about web application security. however, i only recently learned and became familiar with the rapidly growing "appsec" industry. i found a disconnect between what appsec consultants were selling and what i was developing. it seemed like appsec consultants were selling me fear, mostly because i thought my apps were secure. so i set out on a mission to learn more about web application security and penetration testing to see if my apps really were secure. this article is part of that mission, as are the previous articles i've written in this series. java web application security - part i: java ee 6 login demo java web application security - part ii: spring security login demo java web application security - part iii: apache shiro login demo java web application security - part iv: programmatic login apis when i first decided i wanted to do a talk on webapp security, i knew it would be more interesting if i showed the audience how to hack and fix an application. that's why i wrote it into my original proposal : webapp security: develop. penetrate. protect. relax. in this session, you'll learn how to implement authentication in your java web applications using spring security, apache shiro and good ol' java ee container managed authentication. you'll also learn how to secure your rest api with oauth and lock it down with ssl. after learning how to develop authentication, i'll introduce you to owasp, the owasp top 10, its testing guide and its code review guide. from there, i'll discuss using webgoat to verify your app is secure and commercial tools like webapp firewalls and accelerators. at the time, i hadn't done much webapp pentesting . you can tell this from the fact that i mentioned webgoat as the pentesting tool. from webgoat's project page : webgoat is a deliberately insecure j2ee web application maintained by owasp designed to teach web application security lessons. in each lesson, users must demonstrate their understanding of a security issue by exploiting a real vulnerability in the webgoat application. for example, in one of the lessons the user must use sql injection to steal fake credit card numbers. the application is a realistic teaching environment, providing users with hints and code to further explain the lesson. what i really meant to say and use was zed attack proxy , also known as owasp zap. zap is a java desktop application that you setup as a proxy for your browser, then use to find vulnerabilities in your application. this article explains how you can use zap to pentest a web applications and fix its vulnerabilities. the application i'll be using in this article is the ajax login application i've been using throughout this series. i think it's great that projects like damn vulnerable web app and webgoat exist, but i wanted to test one that i think is secure, rather than one i know is not secure. in this particular example, i'll be testing the spring security implementation, since that's the framework i most often use in my open source projects. zed attack proxy tutorial download and run the application install and configure zap perform a scan fix vulnerabilities summary download and run the application to begin, download the application and expand it on your hard drive. this app is the completed version of the ajax login application referenced in java web application security - part ii: spring security login demo . you'll need java 6 and maven installed to run the app. run it using mvn jetty:run and open http://localhost:8080 in your browser. you'll see it's a simple crud application for users and you need to login to do anything. install and configure zap the zed attack proxy (zap) is an easy to use integrated penetration testing tool for finding vulnerabilities in web applications. download the latest version (i used 1.3.0) and install it on your system. after installing, launch the app and change the proxy port to 9000 (tools > options > local proxy). next, configure your browser to proxy requests through port 9000 and allow localhost requests to be proxied. i used firefox 4 (preferences > advanced > network > connection settings). when finished, your proxy settings should look like the following screenshot: another option (instead of removing localhost) is to add an entry to your hosts file with your production domain name. this is what i've done for this demo. 127.0.0.1 demo.raibledesigns.com i've also configured apache to proxy requests to jetty with the following mod_proxy settings in my httpd.conf: proxyrequests off proxypreservehost off proxypass / http://localhost:8080/ sslengine on sslproxyengine on sslcertificatefile "/etc/apache2/ssl.key/server.crt" sslcertificatekeyfile "/etc/apache2/ssl.key/server.key" proxypass / https://localhost:8443/ perform a scan now you need to give zap some data to work with. using firefox, i navigated to http://demo.raibledesigns.com and browsed around a bit, listing users, added a new one and deleted an existing one. after doing this, i noticed a number of flags in the zap ui under sites. i then right-clicked on each site (one for http and one for https) and selected attack > active scan site. you should be able to do this from the "active scan" tab at the bottom of zap, but there's a bug when the urls are the same . after doing this, i received a number of alerts, ranging from high (cross-site scripting) to low (password autocomplete). the screenshot below shows the various issues. now let's take a look at how to fix them. fix vulnerabilities one of the things not mentioned by the scan, but #1 in seven security (mis)configurations in java web.xml files , is custom error pages not configured. custom error pages are configured in this app, but error.jsp contains the following code: please check your log files for further information. stack traces can be really useful to an attacker, so it's important to start by removing the above code from src/main/webapp/error.jsp . the rest of the issues have to do with xss, autocomplete, and cookies. let's start with the easy ones. fixing autocomplete is easy enough; simply changed the html in login.jsp and userform.jsp to have autocomplete="off" as part of the tag. then modify web.xml so http-only and secure cookies are used. while you're at it, add session-timeout and tracking-mode as recommended by the aforementioned web.xml misconfigurations article. 15 true true cookie next, modify spring security's remember me configuration so it uses secure cookies. to do this, add use-secure-cookies="true" to the element in security.xml . unfortunately, spring security doesn't support httponly cookies , but will in a future release. the next issue to solve is disabling directory browsing. you can do this by copying jetty's webdefault.xml (from the org.eclipse.jetty:jetty-webapp jar) into src/test/resources and changing its "dirallowed" to false: default org.mortbay.jetty.servlet.defaultservlet acceptranges true dirallowed false you'll also need to modify the plugin's configuration to point to this file by adding it to the section in pom.xml. / src/test/resources/webdefault.xml of course, if you're running in production you'll want to configure this in your server's settings rather than in your pom.xml file. next, i set out to fix secure page browser cache issues . i had the following settings in my sitemesh decorator: however, according to zap, the first meta tag should have "no-cache" instead of "no-store", so i changed it to "no-cache". after making all these changes, i created a new zap session and ran an active scan on both sites again. below are the results: i believe the first issue (parameter tampering) is because i show the error page when a duplicate user exists. to fix this, i changed userformcontroller so it catches a userexistsexception and sends the user back to the form. try { usermanager.saveuser(user); } catch (userexistsexception uex) { result.adderror(new objecterror("user", uex.getmessage())); return "userform"; } however, this still doesn't seem to cause the alert to go away. this is likely because i'm not filtering/escaping html when it's first submitted. i believe the best solution for this would be to use something like owasp's esapi to filter parameter values. however, i was unable to find integration with spring mvc's data binding, so i decided not to try and fix this vulnerability. finally, i tried to disable jsessionid in urls using suggestions from stack overflow . the previous setting in web.xml (cookie) should do this, but it doesn't seem to work with jetty 8. the other issues (secure page browser cache, httponly cookies and secure cookies), i was unable to solve. the last two are issues caused by spring security as far as i can tell. summary in this article, i've shown you how to pentest a web application using firefox and owasp's zed attack proxy (zap). i found zap to be a nice tool for figuring out vulnerabilities, but it'd be nice if it had a "retest" feature to see if you fixed an issue for a particular url. it does have a "resend" feature, but running it didn't seem to clear alerts after i'd fixed them. the issues i wasn't able to solve seemed to be mostly related to frameworks (e.g. spring security and httponly cookies) or servers (jetty not using cookies for tracking). my suspicion is the jetty issues are because it doesn't support servlet 3 as well as it advertises. i believe this is fair; i am using a milestone release after all. i tried scanning http://demo.raibledesigns.com/ajax-login (which runs on tomcat 7 at contegix ) and confirmed that no jsessionid exists. hopefully this article has helped you understand how to figure out security vulnerabilities in your web applications. i believe zap will continue to get more popular as developers become aware of it. if you feel ambitious and want to try and solve all of the issues in my ajax login application, feel free to fork it on github . if you're interested in talking more about webapp security, please leave a comment, meet me at jazoon later this week or let's talk in july at über conf . from http://raibledesigns.com/rd/entry/java_web_application_security_part4

A few weeks ago I attended a bejug meeting about Java EE 6, Building next generation enterprise applications. Having read much about it, I did not expect to see much shocking hidden features. But there was one part of the demo I really found impressive. Due to its loose coupling, Enterprise possibilities and simplicity. The feature I’m going to talk about today is the event mechanism that is in java EE 6. The general idea is to fire an event and let an eventlistener pick it up. I have created this example that is totally useless, but it simplicity helps me to focus on the important stuff. I’m going to fire a LogEvent from my backing action, that will log to the java.util.Logger. The first thing I need is to create a pojo that contains my log message and my LogLevel. public class LogMessage implements Serializable { private final String message; private final Level level; LogMessage(String message, Level level) { this.message = message; this.level = level; } public String getMessage() { return message; } public Level getLevel() { return level; } } easy peasy. Now that I have my data wrapper, I need something to fire the event and something to pick it up. The first thing I create is my method where I fire the event. Due to CDI I can inject an event. @Inject Event event; So we just need to fire it. event.fire(new LogMessage("Log it baby!", Level.INFO)); Now the event is fired, If no one is registerd to pick it up, it disappears into oblivion, thus we create a listener. The listeners needs a method that has one parameter, the generic type that is given to the previous event. LogMessage. public class LogListener { private static final Logger LOGGER = Logger.getAnonymousLogger(); public void process(@Observes LogMessage message){ LOGGER.log(message.getLevel(), message.getMessage()); } } The @Observes annotation listens to all events with a LogMessage. When the event is fired, this method will be triggered. This is a very nice way to create a loosely coupled application, you can separate heavy operations or encapsulate less essential operations in these event listeners. All of this all happens synchronously. When we want to replace the log statement with a slow database call to a logging table, we could make our operation heavier than it should be. What I’m looking for is to create an asynchronous call. As long as we support EJB, we can transform our Listener to an EJB by adding the @Stateless annotation on top of it. Now it’s a statless enterprise bean. This changes nothing to our sync/async problem, but EJB 3.1 support async operations. So if we also add the @Asynchronous annotation on top of it. It will asynchronously execute our logging statement. @Stateless @Asynchronous public class LogListener { private static final Logger LOGGER = Logger.getAnonymousLogger(); public void process(@Observes LogMessage message){ LOGGER.log(message.getLevel(), message.getMessage()); } } If we would want to combine the database logging and the console logging, we can just create multiple methods that listen to the same event. This is a great way to create a lightweight application with a very flexible components. The alternative solution to this problem is to use JMS, but you don’t want a heavyweight configuration for this kind of loosely coupling. Java EE has worked hard to get rid of the stigma of being heavyweight, I think they are getting there From http://styledideas.be/blog/2011/05/22/java-ee6-events-a-lightweight-alternative-to-jms/

Most people don’t know this, but you cannot have methods of unlimited size in Java. Java has a 64k limit on the size of methods. What happens if I run into this limit? If you run into this limit, the Java compiler will complain with a message which says something like "Code too large to compile". You can also run into this limit at runtime if you had an already large method, just below the 64k limit and some tool or library does bytecode instrumentation on that method, adding to the size of the method and thus making it go beyond the 64k limit. In this case you will get a java.lang.VerifyError at runtime. This is an issue we ran into with the Chronon recorder where most large programs would have atleast a few large methods, and adding instrumentation to them would cause them to blow past the 64k limit, thus causing a runtime error in the program. Before we look into how we went about solving this problem for Chronon, lets look at under what circumstances people write such large methods in the first place. Where do these large methods come from? · Code generators As it turns out, most humans don’t infact write such gigantic methods. We found that most of these large methods were the result of some code generators, eg the ANTLR parser generator generates some very large methods. · Initialization Methods Initialization methods, especially gui initialization methods, where all the layout and attaching listeners, etc to every component in some in one large chunk of code is a common practise and results in a single large method. · Array initializers If you have a large array initialized in your code, eg: static final byte largeArray[] = {10, 20, 30, 40, 50, 60, 70, 80, …}; that is translated by the compiler into a method which uses load/store instructions to initialize the array. Thus an array too large can cause this error too, which may seem very mysterious to those who don’t know about this limit. · Long jsp pages Since most JSP compilers put all the jsp code in one method, large jsp pages can make you run into these errors too. Of course, these are only a few common cases, there can be a lot of other reasons why your method size is too large. How do we get around this issue? If you get this error at compile time, it is usually trivial to split your code into multiple methods. It may be a bit hairy when the method limit is reached due to some automated code generation like ANTLR or JSPs, but usually even these tools have provisions to allow you to split the code into chunks, eg : jsp:include in the case of JSPs. Where things get hairy is the second case I talked about earlier, which is when bytecode instrumentation causes the size of your methods to go beyond the 64k limit, which results in a runtime error. Of course you can still look at the method which is causing the issue, and go back and split it. However, this may not be possible if the method is inside a third party library. Thus, for the Chronon recorder at least, the way we fixed it was to instrument the method, and then check the method's size after instrumentation. If the size is above the 64k limit, we go back and 'deinstrument' the method, thus essentially excluding it from recording. Since both our Recorder and Time Travelling Debugger are already built from the groud up to deal with excluded code, it wasn’t an issue while recording or debugging the rest of the code. That said, the method size limit of 64k is too small and not needed in a world of 64 bit machines. I would urge everyone reading this to go vote on this JVM bug so that this issue can be resolved in some future version of the JVM. From http://eblog.chrononsystems.com/method-size-limit-in-java

I'm starting a new series: Practical PHP Refactoring. Each article will cover one of the refactorings defined by Fowler in its classic book, applied to PHP code. Extract Method means creating a new method to contain part of the existing code: it's one of the most basic refactoring that you should be able to perform, just like every chef is able to chop vegetables or to turn on the gas. It's a building block to more complex refactorings. Many, many issues derive from methods that are (or have become) too long, or that confuse different concepts in the same block of code. Why should I extract a method? Extract Method is one of the simplest tool to help encapsulation. It brings a simplification of the scope, since all variables defined inside the method won't be able to pollute the calling code. The refactoring is called Extract Method since it's about object-oriented programming, but Extract Function would have the same meaning. Extract Method forces you to define a contract with a piece of code, comprehending the inputs (method parameters) and the outputs (a return value). It's not real Design by Contract, but it fits the 80% of the cases. Finally, an extracted method may be reuses independently from the calling code. Eliminating duplication is one of the driving forces that makes most of refactoring techniques interesting. Steps I follow Fowler's recipes, but I will customize it to my style of development (since I write the code samples) and to PHP peculiarities. 1. Create the method, and choose a meaningful name. Thirty seconds spent here can avoid renaming a doSomething() method in thousands of different calls in the future. A good option may be trying to call it from the point where you want to extract it from, to establish a signature handy for the client (like for TDD). But atferwards, comment the call again, since these steps must not break anything. We should go from a green state to another green state. 2. Copy the extracted code into to the new method. Scan it and fix variable references: variables existing prior to the call become the method parameters. Variables that are created in the block of code, and used afterwards, are part of the return value. Typically this is only one variable, but if there is more than one you can wrap them in an object or (temporarily) in an associative array. Local variables may now be hidden inside the method: they are not referenced outside the code and PHP will garbage-collect them when the method finishes (if they are not referenced anymore, of course). 3. Replace original code with a call to the method. 4. Perform renamings or refactorings inside the extracted method. Example The code sample shows you the various steps applied to running PHP code. I also provide a test, since I already use it to check the refactoring has gone well. We start from a method that mixes regular expressions with date formatting: getDayOfTheWeek($logLine); $this->assertEquals('On Saturday we got a visit', $day); } } class LogParser { public function getDayOfTheWeek($logLine) { preg_match('([0-9]{2}/[0-9]{2}/[0-9]{4})', $logLine, $matches); $extractedDate = $matches[0]; $date = new DateTime($extractedDate); return 'On ' . $date->format('l') . ' we got a visit'; } } We extract a method, and in the TDD style we do just as much as it takes to keep a green test. We call this method from the old code immediately: this is the bigger step. getDayOfTheWeek($logLine); $this->assertEquals('On Saturday we got a visit', $day); } } class LogParser { public function getDayOfTheWeek($logLine) { $date = $this->getDate($logLine); return 'On ' . $date->format('l') . ' we got a visit'; } function getDate($logLine) { preg_match('([0-9]{2}/[0-9]{2}/[0-9]{4})', $logLine, $matches); $extractedDate = $matches[0]; $date = new DateTime($extractedDate); return $date; } } Finally, we refine the extracted method, deciding its scope, adding a docblock and eliminating temporary, explanatory variables now rendered useless by the simplicity of this method. getDayOfTheWeek($logLine); $this->assertEquals('On Saturday we got a visit', $day); } } class LogParser { public function getDayOfTheWeek($logLine) { $date = $this->getDate($logLine); return 'On ' . $date->format('l') . ' we got a visit'; } /** * @return DateTime */ private function getDate($logLine) { preg_match('([0-9]{2}/[0-9]{2}/[0-9]{4})', $logLine, $matches); return new DateTime($matches[0]); } }