Even a basic planning problem, such as bin packing, can be notoriously hard to solve and scale. One might consider the brute force algorithm. Let's take a look at how that algorithm works out on the cloud balance example of Drools Planner: Given a set of servers with different hardware (CPU, memory and network bandwidth) and given a set of processes with different hardware requirements, assign each process to 1 server and minimize the total cost of the active servers. The brute force algorithm is simple: try every combination between processes where each process is assigned to each server. For example, if we have 6 processes (P0, P1, P2, P3, P4, P5) and 2 servers (S0, S1), we'd try these solutions: P0->S0, P1->S0, P2->S0, P3->S0, P4->S0, P5->S0 P0->S0, P1->S0, P2->S0, P3->S0, P4->S0, P5->S1 P0->S0, P1->S0, P2->S0, P3->S0, P4->S1, P5->S0 P0->S0, P1->S0, P2->S0, P3->S0, P4->S1, P5->S1 ... P0->S1, P1->S1, P2->S1, P3->S1, P4->S1, P5->S1 On my machine, it takes 15ms to calculate the score of these 2^6 combinations. When I scale out to 9 processes and 3 servers, which are 3^9 combinations, it becomes 1582ms. So it scales like this: Notice that despite that the number of processes has not even doubled, the running time multiplied by 100! For comparison, I 've added the running time of the First Fit algorithm. And it gets worse: for 12 processes and 4 servers, which are 4^12 combinations, it take more than 17 minutes: What if we want to distribute 3000 processes over 1000 servers? With this kind of scalability, it will take too long. In fact, the brute force algorithm is useless. Luckily, Drools Planner implements several other optimization algorithms, which can handle such loads. If you want to know more about them, take a look at the Drools Planner manual or come to my talk at JUDCon London (31 Oct - 1 Nov). This article was originally posted on the Drools & jBPM blog.

f you want to create any of these projects listed below using Eclipse IDE, OSGi Application Project OSGi Bundle Project OSGi Composite Bundle Project OSGi Fragment Project Blueprint File you need to have IBM Rational Development Tools for OSGi Applications installed. Why do we need these tools? Create and edit OSGi bundles, composite bundles, bundle fragments, and applications. Import and export OSGi bundles, composite bundles, bundle fragments, and applications. Convert existing Java Enterprise Edition (Java EE) web, Java Persistence Application (JPA), plug-in, or simple Java projects into OSGi bundles. Edit OSGi bundle, application, and composite bundle manifest files. Create and edit OSGi blueprint configuration files. Edit OSGi blueprint binding configuration files. Diagnose and fix problems in your bundles and applications using validation and quick fixes. Eclipse Plugin Installation Before you install the tools, you must have the Eclipse Helios v3.6.2 package, Eclipse IDE for Java EE Developers installed. 1. Click on Help > Install New Software 2. Point to this update site – http://public.dhe.ibm.com/ibmdl/export/pub/software/rational/OSGiAppTools – and click on Add. 3. You’ll see a list of tools – OSGi Application Development Tools, OSGi Application Development Tools UI, OSGi context-sensitive help, OSGi Help documentation, Rational Development Tools for OSGi Applications help documentation. Select all those are listed (you can ignore help stuff) and go ahead with the installation. As of writing this, the development tools’ version is 1.0.3. Maven Integration If you want to manage any of these OSGi projects using Maven, you can right-click on it and select Maven > Enable Dependency Management. You need to have Maven Integration for Eclipse(m2e) installed for this which you can find in Eclipse Marketplace. If you use Maven, you can try using the plugins provided by Apache Felix project for bundling (building an OSGi bundle). More on this plugin @ http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html, http://felix.apache.org/site/apache-felix-maven-osgi-plugin.html References: http://www.ibm.com/developerworks/rational/downloads/10/rationaldevtoolsforosgiapplications.html#download From http://singztechmusings.wordpress.com/2011/09/12/creating-osgi-projects-using-eclipse-ide-and-maven/

This blog comes as an answer to a question from a reader. Now, as a rule, I don’t do request blogs, but I thought that this sounded like an interesting topic. The question refers to my blog from July 11 on accessing request parameters using Spring 3 MVC and was “Do you know how to encode the uuid so it is not visible to the user for security concerns?”. The answer is “Yes”, but that would make for a very short blog. In this case I’m assuming that the question is talking about using RESTful URLs to maintain session state information between pages. REST is a subject that’s had many words written about it, but putting it in very simplistic terms I’m defining it as not storing session or state information on your web server between browser calls. Even more simply put, it means not using the HttpSession interface implementations. Not maintaining state on the server gives you at least two obvious benefits; the first being improved loading. This is because your server is not using memory for storing state data which may, or may not be needed. The second benefit is scaling. If you’re using a domain with multiple servers, then tying session state to one server is problematic as you can’t usually guarantee that subsequent sequential calls to the domain will be directed to the same physical server. There are various techniques use to solve this problem, including making everything form based and storing state information in hidden fields, using cookies, and manipulating URLs. This blog demonstrates the third technique of encrypting state data and tagging it onto the end of a URL, so that it’s passed back and forth between the browser and the server as a request parameter: http://www.mywebsite.com/thepageIwant?session=ThisIsEncodedData An example scenario for this technique would an e-commerce site. In this case, you’d select a couple of items and add them to your basket. You’d then login and review your basket and then perhaps modify the delivery address before proceeding to pay. In doing all this, you conceptually remain logged in to the server but, in the RESTful world, the server doesn’t remember you. This example uses three screens to demonstrate this idea. The first screen logs you in, the second contains a URL that includes some session data, and the third displays the session data for review. The screen shot above shows a simple login screen. Note that I’ve used a clear text password for demonstration purposes only. The second screen in this scenario is the one that contains the encrypted session information; namely the user name and password, which has both been displayed and glued on to the end of the href of an anchor tag. The last screen shows the decoded session information demonstrating that we’ve still got it and that it’s correct. Moving on to the actual Java code for this demo... The simple bean below I’ve called Session and it holds all the state data we want encoding. public class Session { private static final String SEP = "=/="; private String name; private String password; public Session() { // Blank } public Session(String combined) { String[] split = combined.split(SEP); name = split[0]; password = split[1]; } public String getName() { return name; } public void setName(String username) { this.name = username; } public String getPassword() { return password; } public void setPassword(String password) { this.password = password; } @Override public String toString() { return name + SEP + password; } } If you look at the code above, you’ll see that it contains two features that give it context in the application. The toString(...) returns a string that the application can encode and the single argument constructor can create a full bean from its argument. This works cohesively with the controller code, which is shown below... @Controller public class RewriteController { private static final String ENCODING_KEY = "ThisWillBeTheKey"; /** * Create the initial blank form */ @RequestMapping(value = "/loginform", method = RequestMethod.GET) public String getCreateForm(Model model) { model.addAttribute("Session", new Session()); return "login.page"; } /** * This is where you login... */ @RequestMapping(value = "/login", method = RequestMethod.POST) public String login(Session userDetails, Model model) throws UnsupportedEncodingException { byte[] encodedBytes = encodeURL(userDetails); String encodedURL = new String(Base64.encodeBase64(encodedBytes)); model.addAttribute("encodedURL", encodedURL); return "display.encoded.url"; } private byte[] encodeURL(Session userDetails) throws UnsupportedEncodingException { RC4Lite rc4 = getRC4Lite(); byte[] in = userDetails.toString().getBytes("UTF8"); byte[] out = new byte[in.length]; rc4.encrypt(in, 0, out, 0, in.length); return out; } private RC4Lite getRC4Lite() throws UnsupportedEncodingException { RC4Lite rc4 = new RC4Lite(); rc4.setKey(ENCODING_KEY.getBytes("UTF8")); return rc4; } /** * This is where you're still in the REST session and re-validate the user */ @RequestMapping(value = "/decode", method = RequestMethod.GET) public String someSessionMethod( @RequestParam(value = "session") String sessionParam, Model model) throws UnsupportedEncodingException { byte[] encodedBytes = Base64 .decodeBase64(sessionParam.getBytes("UTF8")); String decodedString = decodeURL(encodedBytes); Session session = new Session(decodedString); model.addAttribute(session); return "display.decoded.url"; } private String decodeURL(byte[] encodedBytes) throws UnsupportedEncodingException { RC4Lite rc4 = getRC4Lite(); byte[] out = new byte[encodedBytes.length]; rc4.decrypt(encodedBytes, 0, out, 0, encodedBytes.length); return new String(out); } } The controller class above contains three handler methods. The first handler method, getCreateForm(...), is fairly straight forward and simply adds a Session bean the model before displaying the login in form. The second handler method, login(...), is where all the action takes place. There are two parts to encoding the session data (the name and password). Part one is to encrypt the data and for that I’ve simply borrowed my RC4Lite class from my RC4 Encryption blog1. Once encrypted, the RC4 bytes will need converting into ASCII and for that I’ve used Apache’s Base64 class as described in my base64 blog. Converting to ASCII ensures that the characters are printable and don’t cause any problems. The string is then added to the model where it’s attached to the anchor tag as shown in the following JSP snippet: The final handler method, someSessionMethod(...) receives the input from the above link, decodes it using a reverse process to encoding It then creates a new Session object and puts that into the model, which is then displayed on the final page. Remember that this is only a sample, it just demonstrates the idea of encrypting session information and adding to a URL as a parameter. There are improvements you could make here, for example: using a interceptor to authenticate with the server upon each request before doing the business logic in the controller, or using a better encryption class, but in essence the principle applies to any webapp. There are also other techniques you can use for achieving secure web communications, so more on those later perhaps... 1 There are better ways of doing encryption than RC4, it's used here only for convenience in this demonstration. From http://www.captaindebug.com/2011/09/using-restful-urls-on-your-spring-3-mvc.html

The ability to inject pieces of code into compiled classes and methods, either statically or at runtime, may be of immense help. This applies especially to troubleshooting problems in third-party libraries without source codes or in an environment where it isn’t possible to use a debugger or a profiler. Code injection is also useful for dealing with concerns that cut across the whole application, such as performance monitoring. Using code injection in this way became popular under the name Aspect-Oriented Programming (AOP). Code injection isn’t something used only rarely as you might think, quite the contrary; every programmer will come into a situation where this ability could prevent a lot of pain and frustration. This post is aimed at giving you the knowledge that you may (or I should rather say “will”) need and at persuading you that learning basics of code injection is really worth the little of your time that it takes. I’ll present three different real-world cases where code injection came to my rescue, solving each one with a different tool, fitting best the constraints at hand. Why You Are Going to Need It A lot has been already said about the advantages of AOP – and thus code injection – so I will only concentrate on a few main points from the troubleshooting point of view. The coolest thing is that it enables you to modify third party, closed-source classes and actually even JVM classes. Most of us work with legacy code and code for which we haven’t the source codes and inevitably we occasionally hit the limitations or bugs of these 3rd-party binaries and need very much to change some small thing in there or to gain more insight into the code’s behavior. Without code injection you have no way to modify the code or to add support for increased observability into it. Also you often need to deal with issues or collect information in the production environment where you can’t use a debugger and similar tools while you usually can at least manage somehow your application’s binaries and dependencies. Consider the following situations: You’re passing a collection of data to a closed-source library for processing and one method in the library fails for one of the elements but the exception provides no information about which element it was. You’d need to modify it to either log the offending argument or to include it in the exception. (And you can’t use a debugger because it only happens on the production application server.) You need to collect performance statistics of important methods in your application including some of its closed-source components under the typical production load. (In the production you of course cannot use a profiler and you want to incur the minimal overhead.) You use JDBC to send a lot of data to a database in batches and one of the batch updates fails. You would need some nice way to find out which batch it was and what data it contained. I’ve in fact encountered these three cases (among others) and you will see possible implementations later. You should keep the following advantages of code injection in your mind while reading this post: Code injection enables you to modify binary classes for which you haven’t the source codes The injected code can be used to collect various runtime information in environments where you cannot use the traditional development tools such as profilers and debuggers Don’t Repeat Yourself: When you need the same piece of logic at multiple places, you can define it once and inject it into all those places. With code injection you do not modify the original source files so it is great for (possibly large-scale) changes that you need only for a limited period of time, especially with tools that make it possible to easily switch the code injection on and off (such as AspectJ with its load-time weaving). A typical case is performance metrics collection and increased logging during troubleshooting You can inject the code either statically, at the build time, or dynamically, when the target classes are being loaded by the JVM Mini Glossary You might encounter the following terms in relation to code injection and AOP: Advice The code to be injected. Typically we talk about before, after, and around advices, which are executed before, after, or instead of a target method. It’s possible to make also other changes than injecting code into methods, e.g. adding fields or interfaces to a class. AOP (Aspect Oriented Programming) A programming paradigm claiming that “cross-cutting concerns” – the logic needed at many places, without a single class where to implement them – should be implemented once and injected into those places. Check Wikipedia for a better description. Aspect A unit of modularity in AOP, corresponds roughly to a class – it can contain different advices and pointcuts. Joint point A particular point in a program that might be the target of code injection, e.g. a method call or method entry. Pointcut Roughly spoken, a pointcut is an expression which tells a code injection tool where to inject a particular piece of code, i.e. to which joint points to apply a particular advice. It could select only a single such point – e.g. execution of a single method – or many similar points – e.g. executions of all methods marked with a custom annotation such as @MyBusinessMethod. Weaving The process of injecting code – advices – into the target places – joint points. The Tools There are many very different tools that can do the job so we will first have a look at the differences between them and then we will get acquainted with three prominent representatives of different evolution branches of code injection tools. Basic Classification of Code Injection Tools I. Level of Abstraction How difficult is it to express the logic to be injected and to express the pointcuts where the logic should be inserted? Regarding the “advice” code: Direct bytecode manipulation (e.g. ASM) – to use these tools you need to understand the bytecode format of a class because they abstract very little from it, you work directly with opcodes, the operand stack and individual instructions. An ASM example: methodVisitor.visitFieldInsn(Opcodes.GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;"); They are difficult to use due to being so low-level but are the most powerful. Usually they are used to implement higher-level tools and only few actually need to use them. Intermediate level – code in strings, some abstraction of the classfile structure (Javassist) Advices in Java (e.g. AspectJ) – the code to be injected is expressed as syntax-checked and statically compiled Java Regarding the specification of where to inject the code: Manual injection – you have to get somehow hold of the place where you want to inject the code (ASM, Javassist) Primitive pointcuts – you have rather limited possibilities for expressing where to inject the code, for example to a particular method, to all public methods of a class or to all public methods of classes in a group (Java EE interceptors) Pattern matching pointcut expressions – powerful expressions matching joint points based on a number of criteria with wildcards, awareness of the context (e.g. “called from a class in the package XY”) etc. (AspectJ) II. When the Magic Happens The code can be injected at different points in time: Manually at run-time – your code has to explicitly ask for the enhanced code, e.g. by manually instantiating a custom proxy wrapping the target object (this is arguably not true code injection) At load-time – the modification are performed when the target classes are being loaded by the JVM At build-time – you add an extra step to your build process to modify the compiled classes before packaging and deploying your application Each of these modes of injection can be more suitable at different situations. III. What It Can Do The code injection tools vary pretty much in what they can or cannot do, some of the possibilities are: Add code before/after/instead of a method – only member-level methods or also the static ones? Add fields to a class Add a new method Make a class to implement an interface Modify an instruction within the body of a method (e.g. a method call) Modify generics, annotations, access modifiers, change constant values, … Remove method, field, etc. Selected Code Injection Tools The best-known code injection tools are: Dynamic Java Proxy The bytecode manipulation library ASM JBoss Javassist AspectJ Spring AOP/proxies Java EE interceptors Practical Introduction to Java Proxy, Javassist and AspectJ I’ve selected three rather different mature and popular code injection tools and will present them on real-world examples I’ve personally experienced. The Omnipresent Dynamic Java Proxy Java.lang.reflect.Proxy makes it possible to create dynamically a proxy for an interface, forwarding all calls to a target object. It is not a code injection tool for you cannot inject it anywhere, you must manually instantiate and use the proxy instead of the original object, and you can do this only for interfaces, but it can still be very useful as we will see. Advantages: It’s a part of JVM and thus is available everywhere You can use the same proxy – more exactly an InvocationHandler – for incompatible objects and thus reuse the code more than you could normally You save effort because you can easily forward all calls to a target object and only modify the ones interesting for you. If you were to implement a proxy manually, you would need to implement all the methods of the interface in question Disadvantages You can create a dynamic proxy only for an interface, you can’t use it if your code expects a concrete class You have to instantiate and apply it manually, there is no magical auto-injection It’s little too verbose Its power is very limited, it can only execute some code before/after/around a method There is no code injection step – you have to apply the proxy manually. Example I was using JDBC PreparedStatement’s batch updates to modify a lot of data in a database and the processing was failing for one of the batch updates because of integrity constraint violation. The exception didn’t contain enough information to find out which data caused the failure and so I’ve created a dynamic proxy for the PreparedStatement that remembered values passed into each of the batch updates and in the case of a failure it automatically printed the batch number and the data. With this information I was able to fix the data and I kept the solution in place so that if a similar problems ever occurs again, I’ll be able to find its cause and resolve it quickly. The crucial part of the code: LoggingStatementDecorator.java – snippet 1 class LoggingStatementDecorator implements InvocationHandler { private PreparedStatement target; ... private LoggingStatementDecorator(PreparedStatement target) { this.target = target; } @Override public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { try { Object result = method.invoke(target, args); updateLog(method, args); // remember data, reset upon successful execution return result; } catch (InvocationTargetException e) { Throwable cause = e.getTargetException(); tryLogFailure(cause); throw cause; } } private void tryLogFailure(Throwable cause) { if (cause instanceof BatchUpdateException) { int failedBatchNr = successfulBatchCounter + 1; Logger.getLogger("JavaProxy").warning( "THE INJECTED CODE SAYS: " + "Batch update failed for batch# " + failedBatchNr + " (counting from 1) with values: [" + getValuesAsCsv() + "]. Cause: " + cause.getMessage()); } } ... Notes: To create a proxy, you first need to implement an InvocationHandler and its invoke method, which is called whenever any of the interface’s methods is invoked on the proxy You can access the information about the call via the java.lang.reflect.* objects and for example delegate the call to the proxied object via method.invoke We’ve also an utility method for creating a proxy instance for a Prepared statement: LoggingStatementDecorator.java – snippet 2 public static PreparedStatement createProxy(PreparedStatement target) { return (PreparedStatement) Proxy.newProxyInstance( PreparedStatement.class.getClassLoader(), new Class[] { PreparedStatement.class }, new LoggingStatementDecorator(target)); }; Notes: You can see that the newProxyInstance call takes a classloader, an array of interfaces that the proxy should implement, and the invocation handler that calls should be delegated to (the handler itself has to manage a reference to the proxied object, if it needs it) It is then used like this: Main.java ... PreparedStatement rawPrepStmt = connection.prepareStatement("..."); PreparedStatement loggingPrepStmt = LoggingStatementDecorator.createProxy(rawPrepStmt); ... loggingPrepStmt.executeBatch(); ... Notes: You see that we have to manually wrap a raw object with the proxy and use the proxy further on Alternative Solutions This problem could be solved in different ways, for example by creating a non-dynamic proxy implementing PreparedStatement and forwarding all calls to the real statement while remembering batch data but it would be lot of boring typing for the interface has many methods. The caller could also manually keep track of the data it has send to the prepared statement but that would obscure its logic with an unrelated concern. Using the dynamic Java proxy we get rather clean and easy to implement solution. The Independent Javassist JBoss Javassist is an intermediate code injection tool providing a higher-level abstraction than bytecode manipulation libraries and offering little limited but still very useful manipulation capabilities. The code to be injected is represented as strings and you have to manually get to the class-method where to inject it. Its main advantage is that the modified code has no new run-time dependencies, on Javassist or anything else. This may be the decisive factor if you are working for a large corporation where the deployment of additional open-source libraries (or just about any additional libraries) such as AspectJ is difficult for legal and other reasons. Advantages Code modified by Javassist doesn’t require any new run-time dependencies, the injection happens at the build time and the injected advice code itself doesn’t depend on any Javassist API Higher-level than bytecode manipulation libraries, the injected code is written in Java syntax, though enclosed in strings Can do most things that you may need such as “advising” method calls and method executions Disadvantages Still little too low-level and thus harder to use – you have to deal a little with structure of methods and the injected code is not syntax-checked The injection is done manually, there isn’t support for injecting the code automatically based on a pattern (though I’ve once implemented a custom Ant task to do execution/call advising for Javassist) Only build-time injection (See GluonJ below for a solution without most of the disadvantages of Javassist.) With Javassist you create a class, which uses the Javassist API to inject code int targets and run it as a part of your build process after the compilation, for example as I once did via a custom Ant task. Example We needed to add some simple performance monitoring to our Java EE application and we were not allowed to deploy any non-approved open-source library (at least not without going through a time-consuming approval process). We’ve therefore used Javassist to inject the performance monitoring code to our important methods and to the places were important external methods were called. The code injector: JavassistInstrumenter.java public class JavassistInstrumenter { public void insertTimingIntoMethod(String targetClass, String targetMethod) throws NotFoundException, CannotCompileException, IOException { Logger logger = Logger.getLogger("Javassist"); final String targetFolder = "./target/javassist"; try { final ClassPool pool = ClassPool.getDefault(); // Tell Javassist where to look for classes - into our ClassLoader pool.appendClassPath(new LoaderClassPath(getClass().getClassLoader())); final CtClass compiledClass = pool.get(targetClass); final CtMethod method = compiledClass.getDeclaredMethod(targetMethod); // Add something to the beginning of the method: method.addLocalVariable("startMs", CtClass.longType); method.insertBefore("startMs = System.currentTimeMillis();"); // And also to its very end: method.insertAfter("{final long endMs = System.currentTimeMillis();" + "iterate.jz2011.codeinjection.javassist.PerformanceMonitor.logPerformance(\"" + targetMethod + "\",(endMs-startMs));}"); compiledClass.writeFile(targetFolder); // Enjoy the new $targetFolder/iterate/jz2011/codeinjection/javassist/TargetClass.class logger.info(targetClass + "." + targetMethod + " has been modified and saved under " + targetFolder); } catch (NotFoundException e) { logger.warning("Failed to find the target class to modify, " + targetClass + ", verify that it ClassPool has been configured to look " + "into the right location"); } } public static void main(String[] args) throws Exception { final String defaultTargetClass = "iterate.jz2011.codeinjection.javassist.TargetClass"; final String defaultTargetMethod = "myMethod"; final boolean targetProvided = args.length == 2; new JavassistInstrumenter().insertTimingIntoMethod( targetProvided? args[0] : defaultTargetClass , targetProvided? args[1] : defaultTargetMethod ); } } Notes: You can see the “low-levelness” – you have to explicitly deal with objects like CtClass, CtMethod, explicitly add a local variable etc. Javassist is rather flexible in where it can look for the classes to modify – it can search the classpath, a particular folder, a JAR file, or a folder with JAR files You would compile this class and run its main during your build process Javassist on Steroids: GluonJ GluonJ is an AOP tool building on top of Javassist. It can use either a custom syntax or Java 5 annotations and it’s build around the concept of “revisers”. Reviser is a class – an aspect – that revises, i.e. modifies, a particular target class and overrides one or more of its methods (contrary to inheritance, the reviser’s code is physically imposed over the original code inside the target class). Advantages No run-time dependencies if build-time weaving used (load-time weaving requires the GluonJ agent library or gluonj.jar) Simple Java syntax using GlutonJ’s annotation – though the custom syntax is also trivial to understand and easy to use Easy, automatic weaving into the target classes with GlutonJ’s JAR tool, an Ant task or dynamically at the load-time Support for both build-time and load-time weaving Disadvantages An aspect can modify only a single class, you cannot inject the same piece of code to multiple classes/methods Limited power – only provides for field/method addition and execution of a code instead of/around a target method, either upon any of its executions or only if the execution happens in a particular context, i.e. when called from a particular class/method If you don’t need to inject the same piece of code into multiple methods then GluonJ is easier and better choice than Javassist and if its simplicity isn’t a problem for you then it also might be a better choice than AspectJ just thanks to this simplicity. The Almighty AspectJ AspectJ is a full-blown AOP tool, it can do nearly anything you might want, including the modification of static methods, addition of new fields, addition of an interface to a class’ list of implemented interfaces etc. The syntax of AspectJ advices comes in two flavours, one is a superset of Java syntax with additional keywords like aspect and pointcut, the other one – called @AspectJ – is standard Java 5 with annotations such as @Aspect, @Pointcut, @Around. The latter is perhaps easier to learn and use but also little less powerful as it isn’t as expressive as the custom AspectJ syntax. With AspectJ you can define which joint points to advise with very powerful expressions but it may be little difficult to learn them and to get them right. There is a useful Eclipse plugin for AspectJ development – the AspectJ Development Tools (AJDT) – but the last time I’ve tried it it wasn’t as helpful as I’d have liked. Advantages Very powerful, can do nearly anything you might need Powerful pointcut expressions for defining where to inject an advice and when to activate it (including some run-time checks) – fully enables DRY, i.e. write once & inject many times Both build-time and load-time code injection (weaving) Disadvantages The modified code depends on the AspectJ runtime library The pointcut expressions are very powerful but it might be difficult to get them right and there isn’t much support for “debugging” them though the AJDT plugin is partially able to visualize their effects It will likely take some time to get started though the basic usage is pretty simple (using @Aspect, @Around, and a simple pointcut expression, as we will see in the example) Example Once upon time I was writing a plugin for a closed-source LMS J2EE application having such dependencies that it wasn’t feasible to run it locally. During an API call, a method deep inside the application was failing but the exception didn’t contain enough information to track the cause of the problem. I therefore needed to change the method to log the value of its argument when it fails. The AspectJ code is quite simple: LoggingAspect.java @Aspect public class LoggingAspect { @Around("execution(private void TooQuiet3rdPartyClass.failingMethod(..))") public Object interceptAndLog(ProceedingJoinPoint invocation) throws Throwable { try { return invocation.proceed(); } catch (Exception e) { Logger.getLogger("AspectJ").warning( "THE INJECTED CODE SAYS: the method " + invocation.getSignature().getName() + " failed for the input '" + invocation.getArgs()[0] + "'. Original exception: " + e); throw e; } } } Notes: The aspect is a normal Java class with the @Aspect annotation, which is just a marker for AspectJ The @Around annotation instructs AspectJ to execute the method instead of the one matched by the expression, i.e. instead of the failingMethod of the TooQuiet3rdPartyClass The around advice method needs to be public, return an Object, and take a special AspectJ object carrying information about the invocation – ProceedingJoinPoint – as its argument and it may have an arbitrary name (Actually this is the minimal form of the signature, it could be more complex.) We use the ProceedingJoinPoint to delegate the call to the original target (an instance of the TooQuiet3rdPartyClass) and, in the case of an exception, to get the argument’s value I’ve used an @Around advice though @AfterThrowing would be simpler and more appropriate but this shows better the capabilities of AspectJ and can be nicely compared to the dynamic java proxy example above Since I hadn’t control over the application’s environment, I couldn’t enable the load-time weaving and thus had to use AspectJ’s Ant task to weave the code at the build time, re-package the affected JAR and re-deploy it to the server. Alternative Solutions Well, if you can’t use a debugger then your options are quite limited. The only alternative solution I could think of is to decompile the class (illegal!), add the logging into the method (provided that the decompilation succeeds), re-compile it and replace the original .class with the modified one. The Dark Side Code injection and Aspect Oriented Programming are very powerful and sometimes indispensable both for troubleshooting and as a regular part of application architecture, as we can see e.g. in the case of Java EE’s Enterprise Java Beans where the business concerns such as transaction management and security checks are injected into POJOs (though implementations actually more likely use proxies) or in Spring. However there is a price to be paid in terms of possibly decreased understandability as the runtime behavior and structure are different from what you’d expect based on the source codes (unless you know to check also the aspects’ sources or unless the injection is made explicit by annotations on the target classes such as Java EE’s @Interceptors). Therefore you must carefully weight the benefits and drawbacks of code injection/AOP – though when used reasonably, they do not obscure the program flow more than interfaces, factories etc. The argument about obscuring code is perhaps often over-estimated. If you want to see an example of AOP gone wild, check the source codes of Glassbox, a JavaEE performance monitoring tool (for that you might need a map not to get too lost). Fancy Uses of Code Injection and AOP The main field of application of code injection in the process of troubleshooting is logging, more exactly gaining visibility into what an application is doing by extracting and somehow communicating interesting runtime information about it. However AOP has many interesting uses beyond – simple or complex – logging, for example: Typical examples: Caching & et al (ex.: on AOP in JBoss Cache), transaction management, logging, enforcement of security, persistence, thread safety, error recovery, automatic implementation of methods (e.g. toString, equals, hashCode), remoting Implementation of role-based programming (e.g. OT/J, using BCEL) or the Data, Context, and Interaction architecture Testing Test coverage – inject code to record whether a line has been executed during test run or not Mutation testing (µJava, Jumble) – inject “random” mutation to the application and verify that the tests failed Pattern Testing – automatic verification that Architecture/Design/Best practices recommendations are implemented correctly in the code via AOP Simulate hardware/external failures by injecting the throwing of an exception Help to achieve zero turnaround for Java applications – JRebel uses an AOP-like approach for framework and server integration plugins – namely its plugins use Javassist for “binary patching” Solving though problems and avoiding monkey-coding with AOP patterns such as Worker Object Creation (turn direct calls into asynchronous with a Runnable and a ThreadPool/task queue) and Wormhole (make context information from a caller available to the callee without having to pass them through all the layers as parameters and without a ThreadLocal) – described in the book AspectJ in Action Dealing with legacy code – overriding the class instantiated on a call to a constructor (this and similar may be used to break tight-coupling with feasible amount of work), ensuring backwards-compatibility o , teaching components to react properly on environment changes Preserving backwards-compatibility of an API while not blocking its ability to evolve e.g. by adding backwards-compatible methods when return types have been narrowed/widened (Bridge Method Injector – uses ASM) or by re-adding old methods and implementing them in terms of the new API Turning POJOs into JMX beans Summary We’ve learned that code injection can be indispensable for troubleshooting, especially when dealing with closed-source libraries and complex deployment environments. We’ve seen three rather different code injection tools – dynamic Java proxies, Javassist, AspectJ – applied to real-world problems and discussed their advantages and disadvantages because different tools may be suitable for different cases. We’ve also mentioned that code injection/AOP shouldn’t be overused and looked at some examples of advanced applications of code injection/AOP. I hope that you now understand how code injection can help you and know how to use these three tools. Source Codes You can get the fully-documented source codes of the examples from GitHub including not only the code to be injected but also the target code and support for easy building. The easiest may be: git clone git://github.com/jakubholynet/JavaZone-Code-Injection.git cd JavaZone-Code-Injection/ cat README mvn -P javaproxy test mvn -P javassist test mvn -P aspectj test (It may take few minutes for Maven do download its dependencies, plugins, and the actual project’s dependencies.) Additional Resources Spring’s introduction into AOP dW: AOP@Work: AOP myths and realities Chapter 1 of AspectJ in Action, 2nd. ed. Acknowledgements I would like to thank all the people who helped me with this post and the presentation including my colleges, the JRebel folk, and GluonJ’s co-author prof. Shigeru Chiba. From http://theholyjava.wordpress.com/2011/09/07/practical-introduction-into-code-injection-with-aspectj-javassist-and-java-proxy/

Here’s how to do it in CXF proprietary way: // Create a list of SOAP headers List headersList = new ArrayList(); Header testHeader = new Header(new QName("uri:singz.ws.sample", "test"), "A custom header", new JAXBDataBinding(String.class)); headersList.add(testHeader); ((BindingProvider)proxy).getRequestContext().put(Header.HEADER_LIST, headersList); The headers in the list are streamed at the appropriate time to the wire according to the databinding object found in the Header object. This doesn’t require changes to WSDL or method signatures. It’s much faster as it doesn’t break streaming and the memory overhead is less. More on this @ http://cxf.apache.org/faq.html#FAQ-HowcanIaddsoapheaderstotherequest%2Fresponse%3F From http://singztechmusings.wordpress.com/2011/09/08/apache-cxf-how-to-add-custom-soap-headers-to-the-web-service-request/

If you're looking for a cloud expert, specifically someone who knows Amazon Web Services and EC2, you'll want to have a battery of questions to assess their knowledge.

Are you building a Java web service based on EJB3? Do you need to return a more significant message to your web service clients other that just the exception message or even worst the recurring javax.transaction.TransactionRolledbackException? Well if the answer is YES to the above questions then keep reading... The code in this article has been tested with JBoss 5.1.0 but it should (!?) work on other EJB containers as well Create a base application exception that will be extended by all the other exception, I will refer to it as MyApplicationBaseException . This exception contains a list of UserMessage, again a class I created with some messages and locale information You need to create a javax.xml.ws.handler.soap.SOAPHandler < SOAPMessageContext > implementation. Mine looks like this import java.util.Set; import javax.xml.bind.JAXBContext; import javax.xml.bind.JAXBException; import javax.xml.bind.Marshaller; import javax.xml.namespace.QName; import javax.xml.soap.SOAPException; import javax.xml.soap.SOAPFault; import javax.xml.soap.SOAPMessage; import javax.xml.ws.handler.MessageContext; import javax.xml.ws.handler.soap.SOAPHandler; import javax.xml.ws.handler.soap.SOAPMessageContext; import org.apache.commons.lang.exception.ExceptionUtils; public class SoapExceptionHandler implements SOAPHandler { private transient Logger logger = ServiceLogFactory.getLogger(SoapExceptionHandler.class); @Override public void close(MessageContext context) { } @Override public boolean handleFault(SOAPMessageContext context) { try { boolean outbound = (Boolean) context.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY); if (outbound) { logger.info("Processing " + context + " for exceptions"); SOAPMessage msg = ((SOAPMessageContext) context).getMessage(); SOAPFault fault = msg.getSOAPBody().getFault(); // Retrives the exception from the context Exception ex = (Exception) context.get("exception"); if (ex != null) { // Add a fault to the body if not there already if (fault == null) { fault = msg.getSOAPBody().addFault(); } // Get my exception int indexOfType = ExceptionUtils.indexOfType(ex, MyApplicationBaseException.class); if (indexOfType != -1) { ex = (MyApplicationBaseException)ExceptionUtils.getThrowableList(ex).get(indexOfType); MyApplicationBaseException myEx = (AmsException) ex; fault.setFaultString(myEx.getMessage()); try { JAXBContext jaxContext = JAXBContext.newInstance(UserMessages.class); Marshaller marshaller = jaxContext.createMarshaller(); //Add the UserMessage xml as a fault detail. Detail interface extends Node marshaller.marshal(amsEx.getUserMessages(), fault.addDetail()); } catch (JAXBException e) { throw new RuntimeException("Can't marshall the user message ", e); } }else { logger.info("This is not an AmsException"); } }else { logger.warn("No exception found in the webServiceContext"); } } } catch (SOAPException e) { logger.warn("Error when trying to access the soap message", e); } return true; } @Override public boolean handleMessage(SOAPMessageContext context) { return true; } @Override public Set getHeaders() { return null; } } Now that you have the exception handler you need to register this SoapHandler with the EJB. To do that you'll need to create an Xml file in your class path and add an annotation to the EJB implementation class. The xml file : ExceptionHandler com.mycompany.utilities.ExceptionHandler and the EJB with annotation will be import javax.jws.HandlerChain; @Local(MyService.class) @Stateless @HandlerChain(file = "soapHandler.xml") @Interceptors( { MyApplicationInterceptor.class }) @SOAPBinding(style = SOAPBinding.Style.RPC) @WebService(endpointInterface = "com.mycompany.services.myservice", targetNamespace = "http://myservice.services.mycompany.com") public final class MyServiceImpl implements MyService { // service implementation } To make sure all my exceptions have proper messages and that the exception is set in the SOAPMessageContext I use an Interceptor to wrap all the service methods and transform any exception to an instance of MyApplicationException The interceptor has a single method @AroundInvoke private Object setException(InvocationContext ic) throws Exception { Object toReturn = null; try { toReturn = ic.proceed(); } catch (Exception e) { logger.error("Exception during the request processing.", e); //converts any exception to MyApplicationException e = MyApplicationExceptionHandler.getMyApplicationException(e); if (context != null && context.getMessageContext() != null) { context.getMessageContext().put("exception", e); } throw e; } return toReturn; } That's it! You're done. From http://www.devinprogress.info/2011/02/how-to-add-information-to-soap-fault.html

Memory is stored within the cache system in units know as cache lines. Cache lines are a power of 2 of contiguous bytes which are typically 32-256 in size. The most common cache line size is 64 bytes. False sharing is a term which applies when threads unwittingly impact the performance of each other while modifying independent variables sharing the same cache line. Write contention on cache lines is the single most limiting factor on achieving scalability for parallel threads of execution in an SMP system. I’ve heard false sharing described as the silent performance killer because it is far from obvious when looking at code. To achieve linear scalability with number of threads, we must ensure no two threads write to the same variable or cache line. Two threads writing to the same variable can be tracked down at a code level. To be able to know if independent variables share the same cache line we need to know the memory layout, or we can get a tool to tell us. Intel VTune is such a profiling tool. In this article I’ll explain how memory is laid out for Java objects and how we can pad out our cache lines to avoid false sharing. Figure 1. Figure 1. above illustrates the issue of false sharing. A thread running on core 1 wants to update variable X while a thread on core 2 wants to update variable Y. Unfortunately these two hot variables reside in the same cache line. Each thread will race for ownership of the cache line so they can update it. If core 1 gets ownership then the cache sub-system will need to invalidate the corresponding cache line for core 2. When Core 2 gets ownership and performs its update, then core 1 will be told to invalidate its copy of the cache line. This will ping pong back and forth via the L3 cache greatly impacting performance. The issue would be further exacerbated if competing cores are on different sockets and additionally have to cross the socket interconnect. Java Memory Layout For the Hotspot JVM, all objects have a 2-word header. First is the “mark” word which is made up of 24-bits for the hash code and 8-bits for flags such as the lock state, or it can be swapped for lock objects. The second is a reference to the class of the object. Arrays have an additional word for the size of the array. Every object is aligned to an 8-byte granularity boundary for performance. Therefore to be efficient when packing, the object fields are re-ordered from declaration order to the following order based on size in bytes: doubles (8) and longs (8) ints (4) and floats (4) shorts (2) and chars (2) booleans (1) and bytes (1) references (4/8) With this knowledge we can pad a cache line between any fields with 7 longs. Within the Disruptor we pad cache lines around the RingBuffer cursor and BatchEventProcessor sequences. To show the performance impact let’s take a few threads each updating their own independent counters. These counters will be volatile longs so the world can see their progress. public final class FalseSharing implements Runnable { public final static int NUM_THREADS = 4; // change public final static long ITERATIONS = 500L * 1000L * 1000L; private final int arrayIndex; private static VolatileLong[] longs = new VolatileLong[NUM_THREADS]; static { for (int i = 0; i < longs.length; i++) { longs[i] = new VolatileLong(); } } public FalseSharing(final int arrayIndex) { this.arrayIndex = arrayIndex; } public static void main(final String[] args) throws Exception { final long start = System.nanoTime(); runTest(); System.out.println("duration = " + (System.nanoTime() - start)); } private static void runTest() throws InterruptedException { Thread[] threads = new Thread[NUM_THREADS]; for (int i = 0; i < threads.length; i++) { threads[i] = new Thread(new FalseSharing(i)); } for (Thread t : threads) { t.start(); } for (Thread t : threads) { t.join(); } } public void run() { long i = ITERATIONS + 1; while (0 != --i) { longs[arrayIndex].value = i; } } public final static class VolatileLong { public volatile long value = 0L; public long p1, p2, p3, p4, p5, p6; // comment out } } Results Running the above code while ramping the number of threads and adding/removing the cache line padding, I get the results depicted in Figure 2. below. This is measuring the duration of test runs on my 4-core Nehalem at home. Figure 2. The impact of false sharing can clearly be seen by the increased execution time required to complete the test. Without the cache line contention we achieve near linear scale up with threads. This is not a perfect test because we cannot be sure where the VolatileLongs will be laid out in memory. They are independent objects. However experience shows that objects allocated at the same time tend to be co-located. So there you have it. False sharing can be a silent performance killer. From http://mechanical-sympathy.blogspot.com/2011/07/false-sharing.html

The integration framework Apache Camel already supports several important cloud services (see my overview article at http://www.kai-waehner.de/blog/2011/07/09/cloud-computing-heterogeneity-will-require-cloud-integration-apache-camel-is-already-prepared for more details). This article describes the combination of Apache Camel and the Amazon Web Services (AWS) interfaces of Simple Storage Service (S3), Simple Queue Service (SQS) and Simple Notification Service (SNS). Thus, The concept of Infrastructure as a Service (IaaS) is used to access messaging systems and data storage without any need for configuration. Registration to AWS and Setup of Camel First, you have to register to the Amazon Web Services (for free). Most AWS services include a free monthly quota, which is absolutely sufficient to play around and develop some simple applications. As its name states, AWS uses technology-independent web services. Besides, APIs for several different programming languages are available to ease development. By the way, Camel uses the AWS SDK for Java (http://aws.amazon.com/sdkforjava), of course. The documentation is detailed and easy to understand, including tutorials, screenshots and code examples . Hint 1: You should read the introductions to S3, SQS and SNS (go to http://aws.amazon.com and click on „products“) and play around with the AWS Management Console (http://aws.amazon.com/console) before you continue. This step is very easy and takes less than one hour. Then, you will have a much better understanding about AWS and where Camel can help you! Hint 2: It really helps to look at the source code of the camel-aws component, It helps you to understand how Camel uses the AWS Java API internally. If you want to write tests, you can do it the same way. In the past, I was afraid of looking at „complex“ source code of open source frameworks. But there is no need to be scared! The camel-aws component (and most other camel components) contain only of a few classes. Everything is easy to understand. It helps you to understand Camel internals, the AWS API, and to spot and solve errors due to exceptions in your code. In the meanwhile, the current Camel version 2.8 supports three AWS services: S3, SQS and SNS. All of them use similar concepts. Therefore, they are included in one single camel component: „camel-aws“. You have to add the libraries to your existing Camel project. As always, the simplest way is to use Maven and add the following dependency to the pom.xml: org.apache.camel camel-aws ${camel-version} Configuration of the Camel Endpoint The implementation and configuration of all three services is very similar. The URI looks like this (the code shows the SQS service): aws-sqs://queue-name[?options] There are two alternatives to configure your endpoint. Using Parameters The easy way is to use two paramters in the URI of your endpoint: „accessKey“ and „secretKey“ (you receive both after your AWS registration). “aws-sqs://unique-queue-name?accessKey=“INSERT_ME“&secretKey=INSERT_ME” Be aware of the following problem, which can result in a strange, non-speaking exception (thanks to Brendan Long): You’ll need to URL encode any +’s in your secret key (otherwise, they’ll be treated as spaces). + = %2B, so if your secretkey was “my+secret\key”, your Camel URL should have “secretKey=my%2Bsecret\key”. “Within the query string, the plus sign is reserved as shorthand notation for a space. Therefore, real plus signs must be encoded. This method was used to make query URIs easier to pass in systems which did not allow spaces.” Source: WC3 URI Recommendations Adding a configured AmazonClient to the Registry If you need to do more configuration (e.g. because your system is behind a firewall), you have to add an AmazonClient object to your registry. The following code shows an example using SQS, but SNS and S3 use exactly the same concept. @Override protected JndiRegistry createRegistry() throws Exception { JndiRegistry registry = super.createRegistry(); AWSCredentials awsCredentials = new BasicAWSCredentials(“INSERT_ME”, “INSERT_ME”); ClientConfiguration clientConfiguration = new ClientConfiguration(); clientConfiguration.setProxyHost(“http://myProxyHost”); clientConfiguration.setProxyPort(8080); AmazonSQSClient client = new AmazonSQSClient(awsCredentials, clientConfiguration); registry.bind(“amazonSQSClient”, client); return registry; } This example overwrites the createRegistry() method of a JUnit test (extending CamelTestSupport). You can also add this information to your runtime Camel application, of course. Apache Camel and the Simple Storage Service (S3) Simple Storage Service (S3) is a key-value-store. You can store small to very large data. The usage is very easy. You create buckets and put key-value data into these buckets. You can also create folders within buckets to organize your data. That’s it. You can monitor your buckets using the AWS Management Console – an intuitive GUI supporting most AWS services. The following example shows both alternatives for accessing the Amazon services (as described above): Paramenters and the AmazonClient. // Transfer data from your file inbox to the AWS S3 service from(“file:files/inbox”) // This is the key of your key-value data .setHeader(S3Constants.KEY, simple(“This is a static key”)) // Using parameters for accessing the AWS service .to(“aws-s3://camel-integration-bucket-mwea-kw?accessKey=INSERT_ME&secretKey=INSERT_ME&region=eu-west-1″); // Transfer data from the AWS S3 service to your file outbox from(“aws-s3://camel-integration-bucket-mwea-kw?amazonS3Client=#amazonS3Client&region=eu-wes”) .to(“file:files/outbox”); There are some additional parameters, for instance you can submit the desired AWS region or delete data after receiving it (see http://camel.apache.org/aws-s3.html and the corresponding SQS and SNS sites for more details about parameters and message headers). As you see in the code, you can use the AWS-S3 endpoint for producing and for consuming messages. Each bucket must be unique, thus you have to add some specific information such as your company to its name. Hint: If a bucket does not exist, Camel is creating it automatically (as the AWS API does). This concept is also used for SQS queues and SNS topics. Apache Camel and the Simple Queue Service (SQS) The Simple Queue Service (SQS) is similar to a JMS provider such as WebSphere MQ or ActiveMQ (but with some differences). You create queues and send messages to them. Consumers receive the messages. Contrary to most other AWS services, you cannot monitor queues by using the AWS management console directly. You have to use the service „Cloudwatch“ (http://aws.amazon.com/cloudwatch) and start an EC2 instance to monitor queues and its content. As you can see in the following code example, the syntax and concepts are almost the same as for the S3 service: from(“file:inbox”) .to(“aws-sqs://camel-integration-queue-mwea-kw?accessKey=INSERT_ME&secretKey=INSERT_ME”); from(“aws-sqs://camel-integration-queue-mwea-kw?amazonSQSClient=#amazonSQSClient”) .to(“file:outbox?fileName=sqs-${date:now:yyyy.MM.dd-hh:mm:ss:SS}”); Again, you can use the AWS-SQS endpoint for producing and for consuming messages. Each queue name must be unique. There exist two important differences to JMS (copy & paste from the AWS documentation): Q: How many times will I receive each message? Amazon SQS is engineered to provide “at least once” delivery of all messages in its queues. Although most of the time each message will be delivered to your application exactly once, you should design your system so that processing a message more than once does not create any errors or inconsistencies. Q: Why are there separate ReceiveMessage and DeleteMessage operations? When Amazon SQS returns a message to you, that message stays in the queue, whether or not you actually received the message. You are responsible for deleting the message; the delete request acknowledges that you’re done processing the message. If you don’t delete the message, Amazon SQS will deliver it again on another receive request. Apache Camel and the Simple Notification Service (SNS) The Simple Notification Service (SNS) acts like JMS topics. You create a topic, consumers subscribe to the topic and then receive notifications. Several transport protocols are supported: HTTP(S), Email and SQS. Further interfaces will be added in the future, e.g. the Short Message Service (SMS) for mobile phones. Contrary to S3 and SQS, Camel only offers a producer endpoint for this AWS service. You can only create topics and send messages via Camel. The reason is simple: Camel already offers endpoints for consuming these messages: HTTP, Email and SQS are already available. There is one tradeoff: A consumer cannot subscribe to topics using Camel – at the moment. The AWS Management Console has to be used. A very interesting discussion can be read on the Camel JIRA issue regarding the following questions: Should Camel be able to subscribe to topics? Should the producer contain this feature or should there be a consumer? In my opinion, there should be a consumer which is able to subscribe to topics, otherwise Camel is missing a key part of the AWS SNS service! Please read the discussion and contribute your opinion: https://issues.apache.org/jira/browse/CAMEL-3476. Apache Camel is already ready for the Cloud Computing Era AWS offers many more services for the cloud. Probably, it does not make sense to integrate everyone into Camel, but more AWS services will be supported in the future. For instance, SimpleDB and the Relational Database Service (RDS) are already planned and make sende, too: http://camel.apache.org/aws.html. The conclusion is easy: Apache Camel is already ready for the cloud computing era. Several important cloud services are already supported. Cloud integration will become very important in the future. Thus, Camel is on a very good way. Hopefully, we will see more cloud components, soon. I will continue to write articles about other Camel cloud components (and new AWS addons, ouf course). For instance, a component for the Platform as a Service (PaaS) product Google App Engine (GAE) is already available. If you have any additional important information, questions or other feedback, please write a comment. Thank you in advance… Best regards, Kai Wähner (Twitter: @KaiWaehner) [Content from my Blog: Cloud Integration with Apache Camel and Amazon Web Services (AWS): S3, SQS and SNS]

when studying both the java nio and io api's, a question quickly pops into mind: when should i use io and when should i use nio? in this text i will try to shed some light on the differences between java nio and io, their use cases, and how they affect the design of your code. main differences of java nio and io the table below summarizes the main differences between java nio and io. i will get into more detail about each difference in the sections following the table. io nio stream oriented buffer oriented blocking io non blocking io selectors stream oriented vs. buffer oriented the first big difference between java nio and io is that io is stream oriented, where nio is buffer oriented. so, what does that mean? java io being stream oriented means that you read one or more bytes at a time, from a stream. what you do with the read bytes is up to you. they are not cached anywhere. furthermore, you cannot move forth and back in the data in a stream. if you need to move forth and back in the data read from a stream, you will need to cache it in a buffer first. java nio's buffer oriented approach is slightly different. data is read into a buffer from which it is later processed. you can move forth and back in the buffer as you need to. this gives you a bit more flexibility during processing. however, you also need to check if the buffer contains all the data you need in order to fully process it. and, you need to make sure that when reading more data into the buffer, you do not overwrite data in the buffer you have not yet processed. blocking vs. non-blocking io java io's various streams are blocking. that means, that when a thread invokes a read() or write(), that thread is blocked until there is some data to read, or the data is fully written. the thread can do nothing else in the meantime. java nio's non-blocking mode enables a thread to request reading data from a channel, and only get what is currently available, or nothing at all, if no data is currently available. rather than remain blocked until data becomes available for reading, the thread can go on with something else. the same is true for non-blocking writing. a thread can request that some data be written to a channel, but not wait for it to be fully written. the thread can then go on and do something else in the mean time. what threads spend their idle time on when not blocked in io calls, is usually performing io on other channels in the meantime. that is, a single thread can now manage multiple channels of input and output. selectors java nio's selectors allow a single thread to monitor multiple channels of input. you can register multiple channels with a selector, then use a single thread to "select" the channels that have input available for processing, or select the channels that are ready for writing. this selector mechanism makes it easy for a single thread to manage multiple channels. how nio and io influences application design whether you choose nio or io as your io toolkit may impact the following aspects of your application design: the api calls to the nio or io classes. the processing of data. the number of thread used to process the data. the api calls of course the api calls when using nio look different than when using io. this is no surprise. rather than just read the data byte for byte from e.g. an inputstream, the data must first be read into a buffer, and then be processed from there. the processing of data the processing of the data is also affected when using a pure nio design, vs. an io design. in an io design you read the data byte for byte from an inputstream or a reader. imagine you were processing a stream of line based textual data. for instance: name: anna age: 25 email: [email protected] phone: 1234567890 this stream of text lines could be processed like this: inputstream input = ... ; // get the inputstream from the client socket bufferedreader reader = new bufferedreader(new inputstreamreader(input)); string nameline = reader.readline(); string ageline = reader.readline(); string emailline = reader.readline(); string phoneline = reader.readline(); notice how the processing state is determined by how far the program has executed. in other words, once the first reader.readline() method returns, you know for sure that a full line of text has been read. the readline() blocks until a full line is read, that's why. you also know that this line contains the name. similarly, when the second readline() call returns, you know that this line contains the age etc. as you can see, the program progresses only when there is new data to read, and for each step you know what that data is. once the executing thread have progressed past reading a certain piece of data in the code, the thread is not going backwards in the data (mostly not). this principle is also illustrated in this diagram: java io: reading data from a blocking stream. a nio implementation would look different. here is a simplified example: bytebuffer buffer = bytebuffer.allocate(48); int bytesread = inchannel.read(buffer); notice the second line which reads bytes from the channel into the bytebuffer. when that method call returns you don't know if all the data you need is inside the buffer. all you know is that the buffer contains some bytes. this makes processing somewhat harder. imagine if, after the first read(buffer) call, that all what was read into the buffer was half a line. for instance, "name: an". can you process that data? not really. you need to wait until at leas a full line of data has been into the buffer, before it makes sense to process any of the data at all. so how do you know if the buffer contains enough data for it to make sense to be processed? well, you don't. the only way to find out, is to look at the data in the buffer. the result is, that you may have to inspect the data in the buffer several times before you know if all the data is inthere. this is both inefficient, and can become messy in terms of program design. for instance: bytebuffer buffer = bytebuffer.allocate(48); int bytesread = inchannel.read(buffer); while(! bufferfull(bytesread) ) { bytesread = inchannel.read(buffer); } the bufferfull() method has to keep track of how much data is read into the buffer, and return either true or false, depending on whether the buffer is full. in other words, if the buffer is ready for processing, it is considered full. the bufferfull() method scans through the buffer, but must leave the buffer in the same state as before the bufferfull() method was called. if not, the next data read into the buffer might not be read in at the correct location. this is not impossible, but it is yet another issue to watch out for. if the buffer is full, it can be processed. if it is not full, you might be able to partially process whatever data is there, if that makes sense in your particular case. in many cases it doesn't. the is-data-in-buffer-ready loop is illustrated in this diagram: java nio: reading data from a channel until all needed data is in buffer. summary nio allows you to manage multiple channels (network connections or files) using only a single (or few) threads, but the cost is that parsing the data might be somewhat more complicated than when reading data from a blocking stream. if you need to manage thousands of open connections simultanously, which each only send a little data, for instance a chat server, implementing the server in nio is probably an advantage. similarly, if you need to keep a lot of open connections to other computers, e.g. in a p2p network, using a single thread to manage all of your outbound connections might be an advantage. this one thread, multiple connections design is illustrated in this diagram: java nio: a single thread managing multiple connections. if you have fewer connections with very high bandwidth, sending a lot of data at a time, perhaps a classic io server implementation might be the best fit. this diagram illustrates a classic io server design: java io: a classic io server design - one connection handled by one thread. from http://tutorials.jenkov.com/java-nio/nio-vs-io.html

In Eclipse, when you press Ctrl button and click on any Class names, the IDE will take you to the source file for that class. This is the normal behavior for the classes you have in your project. But, in case you want the same behavior for Java’s core classes too, you can have it by attaching the Java source with the Eclipse IDE. Once you attach the source, thereafter when you Ctrl+Click any Java class names (String for example), Eclipse will open the source code of that class. To attach the Java source code with Eclipse, When you install the JDK, you must have selected the option to install the Java source files too. This will copy the src.zip file in the installation directory. In Eclipse, go to Window -> Preferences -> Java -> Installed JREs -> Add and choose the JDK you have in your system. Eclipse will now list the JARs found in the dialog box. There, select the rt.jar and choose Source Attachment. By default, this will be pointing to the correct src.zip. If not, choose the src.zip file which you have in your java installation directory. Similarly, if you have the javadoc downloaded in your machine, you can configure that too in this dialog box. Done! Here after, for all the projects for which you are using the above JDK, you’ll be able to browse the Java’s source code just like how you browse your own code. From http://veerasundar.com/blog/2011/08/attaching-java-source-with-eclipse-ide

I'm forever forgetting what the rules are for dependency declarations in MANIFEST.MF and feature.xml for osgi plugins and features. And Googling often results in frustration rather than an answer. So, because today I actually found a concise list of the rules, I thought I'd repost them here, with some minor edits to help clarify. OSGi Plugin Version Ranges Dependencies on bundles and packages have an associated version range which is specified using an interval notation: a square bracket “[” or “]” denotes an inclusive end of the range and a round bracket “(” or “)” denotes an exclusive end of the range. Where one end of the range is to be included and the other excluded, it is permitted to pair a round bracket with a square bracket. The examples below make this clear. If a single version number is used where a version range is required this does not indicate a single version, but the range starting from that version and including all higher versions. There are four common cases: A “strict” version range, such as [1.2.3,1.2.3], which denotes that version and only that version. A “half-open” range, such as [1.2.3,2.0.0), which has an inclusive lower limit and an exclusive upper limit, denoting version 1.2.3 and any version after this, up to, but not including, version 2.0.0. An “unbounded” version range, such as 1.2.3, which denotes version 1.2.3 and all later versions. No version range, which denotes any version will be acceptable. NOT RECOMMENDED. The complete text of the above snippet can be seen here (or here as PDF). Example: Require-Bundle: org.eclipse.core.runtime;bundle-version="[3.4.0,4.0.0)", org.eclipse.core.resources;bundle-version="[3.4.0,4.0.0)", org.eclipse.ui.ide;bundle-version="[3.4.0,4.0.0)", org.eclipse.ui.navigator;bundle-version="3.5.100", com.ibm.icu See also: plugin manifest (plugin.xml) osgi bundle manifest (MANIFEST.MF) In terms of feature manifest (feature.xml) rules, help.eclipse.org has pretty good documentation, but the most important thing to remember - and what I often have to look up - is how to state the matching rules for required upstream features & plugins. Experience says it's always better to state things explicitly so there's no downstream guesswork needed and anyone reading your manifest knows EXACTLY what version(s) are required for or compatible with your feature. Plus, while YOU might be using PDE UI to build, someone else might be using Tycho and Maven, and every tool can interpret missing metadata their own way. When in doubt, spell it out. Valid values and processing are as follows: if version attribute is not specified, the match attribute (if specified) is ignored. perfect - dependent plug-in version must match exactly the specified version. If "patch" is "true", "perfect" is assumed and other values cannot be set. [1.2.3,1.2.3] equivalent - dependent plug-in version must be at least at the version specified, or at a higher service level (major and minor version levels must equal the specified version). [1.2.3,1.3) compatible - dependent plug-in version must be at least at the version specified, or at a higher service level or minor level (major version level must equal the specified version). [1.2.3,2.0) greaterOrEqual - dependent plug-in version must be at least at the version specified, or at a higher service, minor or major level. 1.2.3 The complete text of the above snippet can be seen here. Example: From http://divby0.blogspot.com/2011/07/manifestmf-and-featurexml-versioning.html

This article shows *one* way to mock the JMS infrastructure in a Spring JMS application. This allows us to test our JMS infrastructure without actually having to depend on a physical connection being available. If you are reading this article, chances are that you are also frustrated with failing tests in your continuous integration environment due to a JMS server being (temporarily) unavailable. By mocking the JMS provider, developers are left free to test not only the functionality of their API (unit tests) but also the plumbing of the different components, e.g. in a Spring container. In this article I show how a Spring JMS Hello World application can be fully tested without the need of a physical JMS connection. I would like to stress the fact that the code in this article is by no means meant for production and that the approach shown is just one of many. The infrastructure For this article I use the following infrastructure: Apache ActiveMQ, an open source JMS provider, running on an Ubuntu installation Spring 3 Java 6 MockRunner Eclipse as development environment, running on Windows 7 The Spring configuration It's my belief that using what I define as Spring Configuration Strategy Pattern (SCSP) is the right solution in almost all cases when there is the need for a sound testing infrastructure. I will dedicate an entire article to SCSP, for now this is how it looks: The Spring application context Here follows the content of jemosJms-appContext.xml The only important thing to note here is that there are some services which rely on an existing bean named jmsConnectionFactory but that such bean is not defined in this file. This is key to the SCSP and I will illustrate this in one of my future articles. The Spring application context implementation Here follows the content of jemosJms-appContextImpl.xml which could be seen as an implementation of the Spring application context defined above This Spring context file imports the Spring application context defined above and it is this application context which declared the connection factory. This decoupling of the bean requirement (in the super context) from its actual declaration (Spring application context implementation) represents the cornerstore of SCSP. Mocking the JMS provider - The Spring Test application context and MockRunner Following the same approach I used above, I can now declare a fake connection factory which does not require a physical connection to a JMS provider. Here follows the content of jemosJmsTest-appContext.xml. Please note that this file should reside in the test resources of your project, i.e. it should never make it to production. Here the Spring test application context file imports the Spring application context (not its implementation) and it declares a fake connection factory, thanks to the MockRunner MockQueueConnectionFactory class. A POJO listener The job of handling the message is delegated to a simple POJO, which happens to be declared also as a bean: package uk.co.jemos.experiments; public class HelloWorldHandler { /** The application logger */ private static final org.apache.log4j.Logger LOG = org.apache.log4j.Logger .getLogger(HelloWorldHandler.class); public void handleHelloWorld(String msg) { LOG.info("Received message: " + msg); } } There is nothing glamorous about this class. In real life this should have probably be the implementation of an interface, but here I wanted to keep things simple. A simple JMS message producer Here follows an example of a JMS message producer, which would use the real JMS infrastructure to send messages: package uk.co.jemos.experiments; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import org.springframework.jms.core.JmsTemplate; public class JmsTest { /** The application logger */ private static final org.apache.log4j.Logger LOG = org.apache.log4j.Logger .getLogger(JmsTest.class); /** * @param args */ public static void main(String[] args) { ApplicationContext ctx = new ClassPathXmlApplicationContext( "classpath:jemosJms-appContextImpl.xml"); JmsTemplate jmsTemplate = ctx.getBean(JmsTemplate.class); jmsTemplate.send("jemos.tests", new HelloWorldMessageCreator()); LOG.info("Message sent successfully"); } } The only thing of interest here is that this class retrieves the real JmsTemplate to send a message to the queue. Now if I was to run this class as is, I would obtain the following: 2011-07-31 17:09:46 ClassPathXmlApplicationContext [INFO] Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@19e0ff2f: startup date [Sun Jul 31 17:09:46 BST 2011]; root of context hierarchy 2011-07-31 17:09:46 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJms-appContextImpl.xml] 2011-07-31 17:09:46 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJms-appContext.xml] 2011-07-31 17:09:46 DefaultListableBeanFactory [INFO] Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@3479e304: defining beans [helloWorldConsumer,jmsTemplate,org.springframework.jms.listener.DefaultMessageListenerContainer#0,jmsConnectionFactory]; root of factory hierarchy 2011-07-31 17:09:46 DefaultLifecycleProcessor [INFO] Starting beans in phase 2147483647 2011-07-31 17:09:47 HelloWorldHandler [INFO] Received message: Hello World 2011-07-31 17:09:47 JmsTest [INFO] Message sent successfully Writing the integration test There are various interpretations as to what different types of tests mean and I don't pretend to have the only answer; my interpreation is that an integration test is a functional test which also wires up different components together but which does not interact with real external infrastructure (e.g. a Dao integration test fakes data, a JMS integration test fakes the JMS physical connection, an HTTP integration test fakes the remote Web host, etc). Whereas in my opinion, the main purpose of a unit (aka functional) test is to let the API emerge from the tests, the main goal of an integration test is to test that the plumbing amongst components works as expected so as to avoid surprises in a production environment. Both unit (functional) and integration tests should run very fast (e.g. under 10 minutes) as they constitute what can be considered the "development token". If unit and integration tests are green one should feel pretty confident that 90% of the functionality works as expected; in my projects when both unit and integration tests are green I let developers free to release the token. This does not mean that the other 10% (e.g. the interaction with the real infrastructure) should not be tested, but this can be delegated to system tests which run nightly and don't require the development token. Because unit and integration tests need to run fast, interaction with external infrastructure should be mocked whenever possible. Here follows an integration test for the Hello World handler: package uk.co.jemos.experiments.test.integration; import javax.annotation.Resource; import javax.jms.TextMessage; import junit.framework.Assert; import org.junit.Before; import org.junit.Test; import org.springframework.jms.core.JmsTemplate; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests; import uk.co.jemos.experiments.HelloWorldHandler; import uk.co.jemos.experiments.HelloWorldMessageCreator; import com.mockrunner.jms.DestinationManager; import com.mockrunner.mock.jms.MockQueue; /** * @author mtedone * */ @ContextConfiguration(locations = { "classpath:jemosJmsTest-appContextImpl.xml" }) public class HelloWorldHandlerIntegrationTest extends AbstractJUnit4SpringContextTests { @Resource private JmsTemplate jmsTemplate; @Resource private DestinationManager mockDestinationManager; @Resource private HelloWorldHandler helloWorldHandler; @Before public void init() { Assert.assertNotNull(jmsTemplate); Assert.assertNotNull(mockDestinationManager); Assert.assertNotNull(helloWorldHandler); } @Test public void helloWorld() throws Exception { MockQueue mockQueue = mockDestinationManager.createQueue("jemos.tests"); jmsTemplate.send(mockQueue, new HelloWorldMessageCreator()); TextMessage message = (TextMessage) jmsTemplate.receive(mockQueue); Assert.assertNotNull("The text message cannot be null!", message.getText()); helloWorldHandler.handleHelloWorld(message.getText()); } } And here follows the output: 2011-07-31 17:17:26 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJmsTest-appContextImpl.xml] 2011-07-31 17:17:26 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJms-appContext.xml] 2011-07-31 17:17:26 GenericApplicationContext [INFO] Refreshing org.springframework.context.support.GenericApplicationContext@f01a1e: startup date [Sun Jul 31 17:17:26 BST 2011]; root of context hierarchy 2011-07-31 17:17:27 DefaultListableBeanFactory [INFO] Pre-instantiating singletons in org.springframework.beans.factory.support. DefaultListableBeanFactory@39478a43: defining beans [helloWorldConsumer,jmsTemplate,org.springframework.jms.listener.DefaultMessageListener Container#0,destinationManager,configurationManager,jmsConnectionFactory,org.springframework.context.annotation.internalConfigurationAnnotation Processor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequired AnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor]; root of factory hierarchy 2011-07-31 17:17:27 DefaultLifecycleProcessor [INFO] Starting beans in phase 2147483647 2011-07-31 17:17:27 HelloWorldHandler [INFO] Received message: Hello World 2011-07-31 17:17:27 GenericApplicationContext [INFO] Closing org.springframework.context.support.GenericApplicationContext@f01a1e: startup date [Sun Jul 31 17:17:26 BST 2011]; root of context hierarchy 2011-07-31 17:17:27 DefaultLifecycleProcessor [INFO] Stopping beans in phase 2147483647 2011-07-31 17:17:32 DefaultMessageListenerContainer [WARN] Setup of JMS message listener invoker failed for destination 'jemos.tests' - trying to recover. Cause: Queue with name jemos.tests not found 2011-07-31 17:17:32 DefaultListableBeanFactory [INFO] Destroying singletons in org.springframework.beans.factory.support. DefaultListableBeanFactory@39478a43: defining beans [helloWorldConsumer,jmsTemplate,org.springframework.jms.listener.DefaultMessageListener Container#0,destinationManager,configurationManager,jmsConnectionFactory,org.springframework.context.annotation.internalConfigurationAnnotationProcessor ,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context. annotation.internalCommonAnnotationProcessor]; root of factory hierarchy In this test, although we simulated a message roundtrip to a JMS queue, the message never left the current JVM and it the whole execution did not depend on a JMS infrastructure being up. This gives us the power to simulate the JMS infrastructure, to test the integration of our business components without having to fear a red from time to time due to JMS infrastructure being down or inaccessible. Please note that in the output there are some warnings because the JMS listener container declared in the jemosJms-appContext.xml does not find a queue named "jemos.test" in the fake connection factory, but this is fine; it's a warning and does not impede the test from running successfully. The Maven configuration Here follows the Maven pom.xml to compile the example: 4.0.0 uk.co.jemos.experiments jmx-experiments 0.0.1-SNAPSHOT Jemos JMS experiments junit junit 4.8.2 test com.mockrunner mockrunner 0.3.1 test log4j log4j 1.2.16 compile org.slf4j slf4j-api 1.6.1 compile org.slf4j slf4j-simple 1.6.1 compile org.apache.activemq activemq-all 5.5.0 compile org.springframework spring-beans 3.0.5.RELEASE org.springframework spring-context 3.0.5.RELEASE org.springframework spring-core 3.0.5.RELEASE org.springframework spring-jms 3.0.5.RELEASE org.springframework spring-test 3.0.5.RELEASE test From http://tedone.typepad.com/blog/2011/07/mocking-spring-jms-with-mockrunner.html

i often get asked about ‘rest to soap’ transformation use cases these days. using an soa gateway like securespan to perform this type of transformation at runtime is trivial to setup. with securespan in front of any existing web service (in the dmz for example), you can virtualize a rest version of this same service. using an example, here is a description of the steps to perform this conversion. imagine the geoloc web service for recording geographical locations. it has two methods, one for setting a location and one for getting a location. see below what this would look like in soap. request: 34802398402 response: 52.37706 4.889721 request: 34802398402 52.37706 4.889721 response: ok here is the equivalent rest target that i want to support at the edge. payloads could be xml, but let’s use json to make it more interesting. get /position/34802398402 http 200 ok content-type: text/json { 'latitude' : 52.37706 'longitude' : 4.889721 } post /position/34802398402 content-type: text/json { 'latitude' : 52.37706 'longitude' : 4.889721 } http 200 ok ok now let’s implement this rest version of the service using securespan. i’m assuming that you already have a securespan gateway deployed between the potential rest requesters and the existing soap web service. first, i will create a new service endpoint on the gateway for this service and assign anything that comes at the uri pattern /position/* to this service. i will also allow the http verbs get and post for this service. rest geoloc service properties next, let’s isolate the resource id from the uri and save this as a context variable named ‘trackerid’. we can use a simple regex assertion to accomplish this. also, i will branch on the incoming http verb using an or statement. i am just focusing on get and post for this example but you could add additional logic for other http verbs that you want to support for this rest service. regex for rest service resource identification policy branching for get vs post for get requests, the transformation is very simple, we just declare a message variable using a soap skeleton into which we refer to the trackerid variable. soap request template this soap message is routed to the existing web service and the essential elements are isolated using xpath assertions. processing soap response the rest response is then constructed back using a template response. template json response a similar logic is performed for the post message. see below for the full policy logic. complete policy you’re done for virtualizing the rest service. setting this up with securespan took less than an hour, did not require any change on the existing soap web service and did not require the deployment of an additional component. from there, you would probably enrich the policy to perform some json schema validation , some url and query parameter validation, perhaps some authentication, authorization , etc.

Below is a list of some tools that can help you examine your Java source code for potential problems: 1. PMD from http://pmd.sourceforge.net/ License: PMD is licensed under a “BSD-style” license PMD scans Java source code and looks for potential problems like: * Possible bugs – empty try/catch/finally/switch statements * Dead code – unused local variables, parameters and private methods * Suboptimal code – wasteful String/StringBuffer usage * Overcomplicated expressions – unnecessary if statements, for loops that could be while loops * Duplicate code – copied/pasted code means copied/pasted bugs You can download everything from here, and you can get an overview of all the rules at the rulesets index page. PMD is integrated with JDeveloper, Eclipse, JEdit, JBuilder, BlueJ, CodeGuide, NetBeans/Sun Java Studio Enterprise/Creator, IntelliJ IDEA, TextPad, Maven, Ant, Gel, JCreator, and Emacs. 2. FindBug from http://findbugs.sourceforge.net License: L-GPL FindBugs, a program which uses static analysis to look for bugs in Java code. And since this is a project from my alumni university (IEEE – University of Maryland, College Park – Bill Pugh) , I have to definitely add this contribution to this list. 3. Clover from http://www.cenqua.com/clover/ License: Free for Open Source (more like a GPL) Measures statement, method, and branch coverage and has XML, HTML, and GUI reporting. and comprehensive plug-ins for major IDEs. * Improve Test Quality * Increase Testing Productivity * Keep Team on Track Fully integrated plugins for NetBeans, Eclipse , IntelliJ IDEA, JBuilder and JDeveloper. These plugins allow you to measure and inspect coverage results without leaving the IDE. Seamless Integration with projects using Apache Ant and Maven. * Easy integration into legacy build systems with command line interface and API. Fast, accurate, configurable, detailed coverage reporting of Method, Statement, and Branch coverage. Rich reporting in HTML, PDF, XML or a Swing GUI Precise control over the coverage gathering with source-level filtering. Historical charting of code coverage and other metrics. Fully compatible with JUnit 3.x & 4.x, TestNG, JTiger and other testing frameworks. Can also be used with manual, functional or integration testing. 4. Macker from http://innig.net/macker/ License: GPL Macker is a build-time architectural rule checking utility for Java developers. It’s meant to model the architectural ideals programmers always dream up for their projects, and then break — it helps keep code clean and consistent. You can tailor a rules file to suit a specific project’s structure, or write some general “good practice” rules for your code. Macker doesn’t try to shove anybody else’s rules down your throat; it’s flexible, and writing a rules file is part of the development process for each unique project. 5 EMMA from http://emma.sourceforge.net/ License: EMMA is distributed under the terms of Common Public License v1.0 and is thus free for both open-source and commercial development. Reports on class, method, basic block, and line coverage (text, HTML, and XML). EMMA can instrument classes for coverage either offline (before they are loaded) or on the fly (using an instrumenting application classloader). Supported coverage types: class, method, line, basic block. EMMA can detect when a single source code line is covered only partially. Coverage stats are aggregated at method, class, package, and “all classes” levels. Output report types: plain text, HTML, XML. All report types support drill-down, to a user-controlled detail depth. The HTML report supports source code linking. Output reports can highlight items with coverage levels below user-provided thresholds. Coverage data obtained in different instrumentation or test runs can be merged together. EMMA does not require access to the source code and degrades gracefully with decreasing amount of debug information available in the input classes. EMMA can instrument individial .class files or entire .jars (in place, if desired). Efficient coverage subset filtering is possible, too. Makefile and ANT build integration are supported on equal footing. EMMA is quite fast: the runtime overhead of added instrumentation is small (5-20%) and the bytecode instrumentor itself is very fast (mostly limited by file I/O speed). Memory overhead is a few hundred bytes per Java class. EMMA is 100% pure Java, has no external library dependencies, and works in any Java 2 JVM (even 1.2.x). 6. XRadar from http://xradar.sourceforge.net/ License: BSD (me thinks) The XRadar is an open extensible code report tool currently supporting all Java based systems. The batch-processing framework produces HTML/SVG reports of the systems current state and the development over time – all presented in sexy tables and graphs. The XRadar gives measurements on standard software metrics such as package metrics and dependencies, code size and complexity, code duplications, coding violations and code-style violations. 7. Hammurapi from Hammurapi Group License: (if anyone knows the license for this email me Venkatt.Guhesan at Y! dot com) Hammurapi is a tool for execution of automated inspection of Java program code. Following the example of 282 rules of Hammurabi’s code, we are offered over 120 Java classes, the so-called inspectors, which can, at three levels (source code, packages, repository of Java files), state whether the analysed source code contains violations of commonly accepted standards of coding. Relevant Links: http://en.sdjournal.org/products/articleInfo/93 http://wiki.hammurapi.biz/index.php?title=Hammurapi_4_Quick_Start 8. Relief from http://www.workingfrog.org/ License: GPL Relief is a design tool providing a new look on Java projects. Relying on our ability to deal with real objects by examining their shape, size or relative place in space it gives a “physical” view on java packages, types and fields and their relationships, making them easier to handle. 9. Hudson from http://hudson-ci.org/ License: MIT Hudson is a continuous integration (CI) tool written in Java, which runs in a servlet container, such as Apache Tomcat or the GlassFish application server. It supports SCM tools including CVS, Subversion, Git and Clearcase and can execute Apache Ant and Apache Maven based projects, as well as arbitrary shell scripts and Windows batch commands. 10. Cobertura from http://cobertura.sourceforge.net/ License: GNU GPL Cobertura is a free Java tool that calculates the percentage of code accessed by tests. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage. 11. SonarSource from http://www.sonarsource.org/ (recommended by Vishwanath Krishnamurthi – thanks) License: LGPL Sonar is an open platform to manage code quality. As such, it covers the 7 axes of code quality: Architecture & Design, Duplications, Unit Tests, Complexity, Potential bugs, Coding rules, Comments. From http://mythinkpond.wordpress.com/2011/07/14/java-tools-for-source-code-optimization-and-analysis/

martin fowler has written a really good article describing not only the disruptor , but also how it fits into the architecture at lmax . this gives some of the context that has been missing so far, but the most frequently asked question is still "what is the disruptor?". i'm working up to answering that. i'm currently on question number two: "why is it so fast?". these questions do go hand in hand, however, because i can't talk about why it's fast without saying what it does, and i can't talk about what it is without saying why it is that way. so i'm trapped in a circular dependency. a circular dependency of blogging. to break the dependency, i'm going to answer question one with the simplest answer, and with any luck i'll come back to it in a later post if it still needs explanation: the disruptor is a way to pass information between threads. as a developer, already my alarm bells are going off because the word "thread" was just mentioned, which means this is about concurrency, and concurrency is hard. concurrency 101 imagine two threads are trying to change the same value. case one: thread 1 gets there first: the value changes to "blah" then the value changes to "blahy" when thread 2 gets there. case two: thread 2 gets there first: the value changes to "fluffy" then the value changes to "blah" when thread 1 gets there. case three: thread 1 interrupts thread 2: thread 2 gets the value "fluff" and stores it as myvalue thread 1 goes in and updates value to "blah" then thread 2 wakes up and sets the value to "fluffy". case three is probably the only one which is definitely wrong, unless you think the naive approach to wiki editing is ok ( google code wiki, i'm looking at you...). in the other two cases it's all about intentions and predictability. thread 2 might not care what's in value, the intention might be to append "y" to whatever is in there regardless. in this circumstance, cases one and two are both correct. but if thread 2 only wanted to change "fluff" to "fluffy", then both cases two and three are incorrect. assuming that thread 2 wants to set the value to "fluffy", there are some different approaches to solving the problem. approach one: pessimistic locking (does the "no entry" sign make sense to people who don't drive in britain?) the terms pessimistic and optimistic locking seem to be more commonly used when talking about database reads and writes, but the principal applies to getting a lock on an object. thread 2 grabs a lock on entry as soon as it knows it needs it and stops anything from setting it. then it does its thing, sets the value, and lets everything else carry on. you can imagine this gets quite expensive, with threads hanging around all over the place trying to get hold of objects and being blocked. the more threads you have, the more chance that things are going to grind to a halt. approach two: optimistic locking in this case thread 2 will only lock entry when it needs to write to it. in order to make this work, it needs to check if entry has changed since it first looked at it. if thread 1 came in and changed the value to "blah" after thread 2 had read the value, thread 2 couldn't write "fluffy" to the entry and trample all over the change from thread 1. thread 2 could either re-try (go back, read the value, and append "y" onto the end of the new value), which you would do if thread 2 didn't care what the value it was changing was; or it could throw an exception or return some sort of failed update flag if it was expecting to change "fluff" to "fluffy". an example of this latter case might be if you have two users trying to update a wiki page, and you tell the user on the other end of thread 2 they'll need to load the new changes from thread 1 and then reapply their changes. potential problem: deadlock locking can lead to all sorts of issues, for example deadlock. imagine two threads that need access to two resources to do whatever they need to do: if you've used an over-zealous locking technique, both threads are going to sit there forever waiting for the other one to release its lock on the resource. that's when you reboot windows your computer. definite problem: locks are sloooow... the thing about locks is that they need the operating system to arbitrate the argument. the threads are like siblings squabbling over a toy, and the os kernel is the parent that decides which one gets it. it's like when you run to your dad to tell him your sister has nicked the transformer when you wanted to play with it - he's got bigger things to worry about than you two fighting, and he might finish off loading the dishwasher and putting on the laundry before settling the argument. if you draw attention to yourself with a lock, not only does it take time to get the operating system to arbitrate, the os might decide the cpu has better things to do than servicing your thread. the disruptor paper talks about an experiment we did. the test calls a function incrementing a 64-bit counter in a loop 500 million times. for a single thread with no locking, the test takes 300ms. if you add a lock (and this is for a single thread, no contention, and no additional complexity other than the lock) the test takes 10,000ms. that's, like, two orders of magnitude slower. even more astounding, if you add a second thread (which logic suggests should take maybe half the time of the single thread with a lock) it takes 224,000ms. incrementing a counter 500 million times takes nearly a thousand times longer when you split it over two threads instead of running it on one with no lock. concurrency is hard and locks are bad i'm just touching the surface of the problem, and obviously i'm using very simple examples. but the point is, if your code is meant to work in a multi-threaded environment, your job as a developer just got a lot more difficult: naive code can have unintended consequences. case three above is an example of how things can go horribly wrong if you don't realise you have multiple threads accessing and writing to the same data. selfish code is going to slow your system down. using locks to protect your code from the problem in case three can lead to things like deadlock or simply poor performance. this is why many organisations have some sort of concurrency problems in their interview process (certainly for java interviews). unfortunately it's very easy to learn how to answer the questions without really understanding the problem, or possible solutions to it. how does the disruptor address these issues? for a start, it doesn't use locks. at all. instead, where we need to make sure that operations are thread-safe (specifically, updating the next available sequence number in the case of multiple producers ), we use a cas (compare and swap/set) operation. this is a cpu-level instruction, and in my mind it works a bit like optimistic locking - the cpu goes to update a value, but if the value it's changing it from is not the one it expects, the operation fails because clearly something else got in there first. note this could be two different cores rather than two separate cpus. cas operations are much cheaper than locks because they don't involve the operating system, they go straight to the cpu. but they're not cost-free - in the experiment i mentioned above, where a lock-free thread takes 300ms and a thread with a lock takes 10,000ms, a single thread using cas takes 5,700ms. so it takes less time than using a lock, but more time than a single thread that doesn't worry about contention at all. back to the disruptor - i talked about the claimstrategy when i went over the producers . in the code you'll see two strategies, a singlethreadedstrategy and a multithreadedstrategy. you could argue, why not just use the multi-threaded one with only a single producer? surely it can handle that case? and it can. but the multi-threaded one uses an atomiclong (java's way of providing cas operations), and the single-threaded one uses a simple long with no locks and no cas. this means the single-threaded claim strategy is as fast as possible, given that it knows there is only one producer and therefore no contention on the sequence number. i know what you're thinking: turning one single number into an atomiclong can't possibly have been the only thing that is the secret to the disruptor's speed. and of course, it's not - otherwise this wouldn't be called "why it's so fast (part one )". but this is an important point - there's only one place in the code where multiple threads might be trying to update the same value. only one place in the whole of this complicated data-structure-slash-framework. and that's the secret. remember everything has its own sequence number? if you only have one producer then every sequence number in the system is only ever written to by one thread. that means there is no contention. no need for locks. no need even for cas. the only sequence number that is ever written to by more than one thread is the one on the claimstrategy if there is more than one producer. this is also why each variable in the entry can only be written to by one consumer . it ensures there's no write contention, therefore no need for locks or cas. back to why queues aren't up to the job so you start to see why queues, which may implemented as a ring buffer under the covers, still can't match the performance of the disruptor. the queue, and the basic ring buffer , only has two pointers - one to the front of the queue and one to the end: if more than one producer wants to place something on the queue, the tail pointer will be a point of contention as more than one thing wants to write to it. if there's more than one consumer, then the head pointer is contended, because this is not just a read operation but a write, as the pointer is updated when the item is consumed from the queue. but wait, i hear you cry foul! because we already knew this, so queues are usually single producer and single consumer (or at least they are in all the queue comparisons in our performance tests). there's another thing to bear in mind with queues/buffers. the whole point is to provide a place for things to hang out between producers and consumers, to help buffer bursts of messages from one to the other. this means the buffer is usually full (the producer is out-pacing the consumer) or empty (the consumer is out-pacing the producer). it's rare that the producer and consumer will be so evenly-matched that the buffer has items in it but the producers and consumers are keeping pace with each other. so this is how things really look. an empty queue: ...and a full queue: the queue needs a size so that it can tell the difference between empty and full. or, if it doesn't, it might determine that based on the contents of that entry, in which case reading an entry will require a write to erase it or mark it as consumed. whichever implementation is chosen, there's quite a bit of contention around the tail, head and size variables, or the entry itself if a consume operation also includes a write to remove it. on top of this, these three variables are often in the same cache line , leading to false sharing . so, not only do you have to worry about the producer and the consumer both causing a write to the size variable (or the entry), updating the tail pointer could lead to a cache-miss when the head pointer is updated because they're sat in the same place. i'm going to duck out of going into that in detail because this post is quite long enough as it is. so this is what we mean when we talk about "teasing apart the concerns" or a queue's "conflated concerns". by giving everything its own sequence number and by allowing only one consumer to write to each variable in the entry, the only case the disruptor needs to manage contention is where more than one producer is writing to the ring buffer. in summary the disruptor a number of advantages over traditional approaches: no contention = no locks = it's very fast. having everything track its own sequence number allows multiple producers and multiple consumers to use the same data structure. tracking sequence numbers at each individual place (ring buffer, claim strategy, producers and consumers), plus the magic cache line padding , means no false sharing and no unexpected contention. from http://mechanitis.blogspot.com/2011/07/dissecting-disruptor-why-its-so-fast.html

I am Angelo D'Agnano and currently I work at the NATO Programming Centre as Software Architect.

I’m currently working on a project that is converted from being a Mainframe application, to a Java web/batch application. We don’t ‘big bang’ into production, so the Mainframe and the Java code will work next to each other for a fairly amount of time. Since we have multiple batch processes and many simultaneous users, we start seeing deadlock errors in certain parts of the application. Some specific parts have to take a pessimistic lock, this is where it goes wrong. Since a deadlock is an error that can be solved by repeating the action, we decide to build in a retry mechanism to restart the transaction if it got rolled back. I started of with creating an Annotation. This annotation will mark the entry point that we want to retry in case of a deadlock. @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.METHOD) public @interface DeadLockRetry { /** * Retry count. default value 3 */ int retryCount() default 3; } The retry count is a value you can supply together with your annotation, so you can specify the number of times we want to retry our operation. Using AOP we can pick up this annotation an let us surround the method call with a retry mechanism. @Around(value = "@annotation(deadLockRetry)", argNames = "deadLockRetry") So lets view the aspect, we start with adding an @Aspect annotation on top of our class, this way it is configured to be an Aspect. We also want to implement the Ordered interface. This interface lets us order our aspect. We need this to surround our Transactional aspect. If we don’t surround our Transaction, we will never be able to retry in a new transaction, we would be working in the same (marked as rollback only) transaction. The rest of the code is pretty straight forward. We create a loop where we loop until we have more retries than we should have. Inside that loop we proceed our ProceedingJoinPoint and catch the PersistenceException that JPA would throw when a deadlock would occur. Inside the catch block we check if the error code is a deadlock error code. Off course we could not directly configure the database specific error codes inside our aspect, so I’ve created an interface. /** * Interface that marks a dialect aware of certain error codes. When you have to * do a low level check of the exception you are trying to handle, you can * implement this in this interface, so you can encapsulate the specific error * codes for the specific dialects. * * @author Jelle Victoor * @version 05-jul-2011 */ public interface ErrorCodeAware { Set getDeadlockErrorCodes(); } We already have custom hibernate dialects for our database and database to be, so this let me configure the error codes in the Dialect implementations. It was a bit tricky to get the current dialect. I injected the persistence unit, since we are outside a transaction, and made some casts to get my dialect. The alternative was to use a custom implementation of the ErrorCodeAware interface, not using the dialects. We could inject the needed ErrorCodeAware implementation based on our application context. This added another database specific injection, which added another point of configuration. This is why I chose to store it in our custom dialect. private Dialect getDialect() { final SessionFactory sessionFactory = ((HibernateEntityManagerFactory) emf).getSessionFactory(); return ((SessionFactoryImplementor) sessionFactory).getDialect(); } The only thing left is to configure the aspect, mind the order of the transaction manager and the retry aspect Now when I have a deadlock exception, and I’ve added this annotation, the transaction will rollback and will be reexecuted. /** * This Aspect will cause methods to retry if there is a notion of a deadlock. * * Note that the aspect implements the Ordered interface so we can set the * precedence of the aspect higher than the transaction advice (we want a fresh * transaction each time we retry). * * @author Jelle Victoor * @version 04-jul-2011 handles deadlocks */ @Aspect public class DeadLockRetryAspect implements Ordered { private static final Logger LOGGER = LoggerFactory.getLogger(DeadLockRetryAspect.class); private int order = -1; @PersistenceUnit private EntityManagerFactory emf; /** * Deadlock retry. The aspect applies to every service method with the * annotation {@link DeadLockRetry} * * @param pjp * the joinpoint * @param deadLockRetry * the concurrency retry * @return * * @throws Throwable * the throwable */ @Around(value = "@annotation(deadLockRetry)", argNames = "deadLockRetry") public Object concurrencyRetry(final ProceedingJoinPoint pjp, final DeadLockRetry deadLockRetry) throws Throwable { final Integer retryCount = deadLockRetry.retryCount(); Integer deadlockCounter = 0; Object result = null; while (deadlockCounter < retryCount) { try { result = pjp.proceed(); break; } catch (final PersistenceException exception) { deadlockCounter = handleException(exception, deadlockCounter, retryCount); } } return result; } /** * handles the persistence exception. Performs checks to see if the * exception is a deadlock and check the retry count. * * @param exception * the persistence exception that could be a deadlock * @param deadlockCounter * the counter of occured deadlocks * @param retryCount * the max retry count * @return the deadlockCounter that is incremented */ private Integer handleException(final PersistenceException exception, Integer deadlockCounter, final Integer retryCount) { if (isDeadlock(exception)) { deadlockCounter++; LOGGER.error("Deadlocked ", exception.getMessage()); if (deadlockCounter == (retryCount - 1)) { throw exception; } } else { throw exception; } return deadlockCounter; } /** * check if the exception is a deadlock error. * * @param exception * the persitence error * @return is a deadlock error */ private Boolean isDeadlock(final PersistenceException exception) { Boolean isDeadlock = Boolean.FALSE; final Dialect dialect = getDialect(); if (dialect instanceof ErrorCodeAware && exception.getCause() instanceof GenericJDBCException) { if (((ErrorCodeAware) dialect).getDeadlockErrorCodes().contains(getSQLErrorCode(exception))) { isDeadlock = Boolean.TRUE; } } return isDeadlock; } /** * Returns the currently used dialect * * @return the dialect */ private Dialect getDialect() { final SessionFactory sessionFactory = ((HibernateEntityManagerFactory) emf).getSessionFactory(); return ((SessionFactoryImplementor) sessionFactory).getDialect(); } /** * extracts the low level sql error code from the * {@link PersistenceException} * * @param exception * the persistence exception * @return the low level sql error code */ private int getSQLErrorCode(final PersistenceException exception) { return ((GenericJDBCException) exception.getCause()).getSQLException().getErrorCode(); } /** {@inheritDoc} */ public int getOrder() { return order; } /** * Sets the order. * * @param order * the order to set */ public void setOrder(final int order) { this.order = order; } } From http://styledideas.be/blog/2011/07/05/automatic-deadlock-retry-aspect-with-spring-and-jpahibernate/

This article describes how to create a simple HTML5 chat application using WebSockets to connect to a Java back-end.

One of the biggest promises java EE6 made, was to ease the use of dependency injection. They did, using CDI. CDI, which stands for Contexts and Dependency Injection for Java EE, offers a base set to apply dependency injection in your enterprise application. Before CDI, EJB 3 also introduced dependency injection, but this was a bit basic. You could inject an EJB (statefull or stateless) into another EJB or Servlet (if you container supported this). Offcourse not every application needs EJB’s, that is why CDI is gaining so much popularity. To start, I have made this example. There is a Payment interface, and 2 implementations. A cash payment and a visa payment. I want to be able to choose witch type of payment I inject, still using the same interface. public interface Payment { void pay(BigDecimal amount); } and the 2 implementations public class CashPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(CashPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} cash", amount.toString()); } } public class VisaPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(VisaPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} with visa", amount.toString()); } } To inject the interface we use the @Inject annotation. The annotation does basically what it says. It injects a component, that is available in your application. 1 @Inject private Payment payment; Off course, you saw this coming from a mile away, this won’t work. The container has 2 implementations of our Payment interface, so he does not know which one to inject. Unsatisfied dependencies for type [Payment] with qualifiers [@Default] at injection point [[field] @Inject private be.styledideas.blog.qualifier.web.PaymentBackingAction.payment] So we need some sort of qualifier to point out what implementation we want. CDI offers the @Named Annotation, allowing you to give a name to an implementation. @Named("cash") public class CashPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(CashPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} cash", amount.toString()); } } @Named("visa") public class VisaPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(VisaPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} with visa", amount.toString()); } } When we now change our injection code, we can specify wich implementation we need. @Inject private @Named("visa") Payment payment; This works, but the flexibility is limited. When we want to rename our @Named parameter, we have to change it on everyplace where it is used. There is also no refactoring support. There is a beter alternative using Custom made annotations using the @Qualifier annotation. Let us change the code a little bit. First of all, we create new Annotation types. @java.lang.annotation.Documented @java.lang.annotation.Retention(RetentionPolicy.RUNTIME) @javax.inject.Qualifier public @interface CashPayment {} @java.lang.annotation.Documented @java.lang.annotation.Retention(RetentionPolicy.RUNTIME) @javax.inject.Qualifier public @interface VisaPayment {} The @Qualifier annotation that is added to the annotation, makes this annotation discoverable by the container. We can now simply add these annotations to our implementations. @CashPayment public class CashPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(CashPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} cash", amount.toString()); } } @VisaPayment public class VisaPaymentImpl implements Payment { private static final Logger LOGGER = Logger.getLogger(VisaPaymentImpl.class.toString()); @Override public void pay(BigDecimal amount) { LOGGER.log(Level.INFO, "payed {0} with visa", amount.toString()); } } The only thing we now need to do, is change our injection code to @Inject private @VisaPayment Payment payment; When we now change something to our qualifier, we have nice compiler and refactoring support. This also adds extra flexibilty for API or Domain-specific language design. From http://styledideas.be/blog/2011/06/16/java-ee6-cdi-named-components-and-qualifiers/