In my endless search for the best way to develop applications, I’ve recently been interested in web services in general and contract-first in particular. Web services are coined contract-first when the WSDL is designed in the first place and classes are generated from it. They are acknowledged to be the most interoperable of web services, since the WSDL is agnostic from the underlying technology. In the past, I’ve been using Axis2 and then CXF but now, JavaEE provides us with the power of JAX-WS (which is aimed at SOAP, JAX-RS being aimed at REST). There’s also a Spring Web Services sub-project. My first goal was to check how easy it was to inject through Spring in both technologies but during the course of my development, I came across other comparison areas. Overview With Spring Web Services, publishing a new service is a 3-step process: Add the Spring MessageDispatcherServlet in the web deployment descriptor. contextConfigLocation classpath:/applicationContext.xml spring-ws org.springframework.ws.transport.http.MessageDispatcherServlet transformWsdlLocations true contextConfigLocation classpath:/spring-ws-servlet.xml spring-ws /spring/* org.springframework.web.context.ContextLoaderListener Create the web service class and annotate it with @Endpoint. Annotate the relevant service methods with @PayloadRoot. Bind the method parameters with @RequestPayload and the return type with @ResponsePayload. @Endpoint public class FindPersonServiceEndpoint { private final PersonService personService; public FindPersonServiceEndpoint(PersonService personService) { this.personService = personService; } @PayloadRoot(localPart = "findPersonRequestType", namespace = "http://blog.frankel.ch/ws-inject") @ResponsePayload public FindPersonResponseType findPerson(@RequestPayload FindPersonRequestType parameters) { return new FindPersonDelegate().findPerson(personService, parameters); } } Configure the web service class as a Spring bean in the relevant Spring beans definition file. For JAX-WS (Metro implementation), the process is very similar: Add the WSServlet in the web deployment descriptor: jaxws-servlet com.sun.xml.ws.transport.http.servlet.WSServlet jaxws-servlet /jax/* com.sun.xml.ws.transport.http.servlet.WSServletContextListener Create the web service class: @WebService(endpointInterface = "ch.frankel.blog.wsinject.jaxws.FindPersonPortType") public class FindPersonSoapImpl extends SpringBeanAutowiringSupport implements FindPersonPortType { @Autowired private PersonService personService; @Override public FindPersonResponseType findPerson(FindPersonRequestType parameters) { return new FindPersonDelegate().findPerson(personService, parameters); } } Finally, we also have to configure the service, this time in the standard Metro configuration file (sun-jaxws.xml): Creating a web service in Spring or JAX-WS requires the same number of steps of equal complexity. Code generation In both cases, we need to generate Java classes from the Web Service Definitions File (WSDL). This is completely independent from the chosen technology. However, whereas JAX-WS uses all generated Java classes, Spring WS uses only the ones that maps to WSD types and elements: wiring the WS call to the correct endpoint is achieved by mapping the request and response types. The web service itself Creating the web service in JAX-RS is just a matter of implementing the port type interface, which contains the service method. In Spring, the service class has to be annotated with @Endpoint to be rcognized as a service class. URL configuration In JAX-RS, the sun-jaxws.xml file syntax let us configure very finely how each URL is mapped to a particular web service. In Spring WS, no such configuration is available. Since I’d rather have an overview of the different URLs, my preferences go toward JAX-WS. Spring dependency injection Injecting a bean in a Spring WS is very easy since the service is already a Spring bean thanks to the part. On the contrary, injecting a Spring bean requires our JAX-WS implementation to inherit from SpringBeanAutowiringSupport which prevent us from having our own hierarchy. Also, we are forbidden from using explicit XML wiring then. It’s easier to get dependency injection with Spring WS (but that was expected). Exposing the WSDL Both JAX-WS and Spring WS are able to expose a WSDL. In order to do so, JAX-WS uses the generated classes and as such, the exposed WSDL is the same as the one we designed in the first place. On the contrary, Spring provides us with 2 options: either use the static WSDL in which it replaces the domain and port in the section or generate a dynamic WSDL from XSD files, in which case the designed one and the generated one aren’t the same Integration testing Spring WS has one feature that JAX-WS lacks: integration testing. A test class that will be configured with Spring Beans definitions file(s) can be created to assert sent output messages agains known inputs. Here’s an example of such a test, based on both Spring test and TestNG: @ContextConfiguration(locations = { "classpath:/spring-ws-servlet.xml", "classpath:/applicationContext.xml" }) public class FindPersonServiceEndpointIT extends AbstractTestNGSpringContextTests { @Autowired private ApplicationContext applicationContext; private MockWebServiceClient client; @BeforeMethod protected void setUp() { client = MockWebServiceClient.createClient(applicationContext); } @Test public void findRequestPayloadShouldBeSameAsExpected(String request, String expectedResponse) throws DatatypeConfigurationException { int id = 5; Source requestPayload = new StringSource(request); Source expectedResponsePayload = new StringSource(expectedResponse); String request = "" + id + ""; GregorianCalendar calendar = new GregorianCalendar(); XMLGregorianCalendar xmlCalendar = DatatypeFactory.newInstance().newXMLGregorianCalendar(calendar); xmlCalendar.setHour(FIELD_UNDEFINED); xmlCalendar.setMinute(FIELD_UNDEFINED); xmlCalendar.setSecond(FIELD_UNDEFINED); xmlCalendar.setMillisecond(FIELD_UNDEFINED); String expectedResponse = "" + id + "JohnDoe" + xmlCalendar.toXMLFormat() + ""; client.sendRequest(withPayload(requestPayload)).andExpect(payload(expectedResponsePayload)); } } Note that I encountered problems regarding XML suffixes since not only is the namespace checked, but also the prefix name itself (which is a terrible idea). Included and imported schema resolution On one side, JAX-WS inherently resolves included/imported schemas without a glitch. On the other side, we need to add specific beans in the Spring context in order to be able to do it with Spring WS: Miscellaneous JAX-WS provides an overview of all published services at the root of the JAX-WS servlet mapping: Adress Informations Service name : {http://impl.wsinject.blog.frankel.ch/}FindPersonSoapImplService Port name : {http://impl.wsinject.blog.frankel.ch/}FindPersonSoapImplPort Adress : http://localhost:8080/wsinject/jax/findPerson WSDL: http://localhost:8080/wsinject/jax/findPerson?wsdl Implementation class : ch.frankel.blog.wsinject.impl.FindPersonSoapImpl Conclusion Considering contract-first web services, my little experience has driven me to choose JAX-WS in favor of Spring WS: testing benefits do not balance the ease-of-use of the standard. I must admit I was a little surprised by these results since most Spring components are easier to use and configure than their standard counterparts, but results are here. You can find the sources for this article here. Note: the JAX-WS version used is 2.2 so the Maven POM is rather convoluted to override Java 6 native JAX-WS 2.1 classes.

Why create a mechanism to expect a test failure? There comes a time when one would want and expect a JUnit @Test case fail. Though this is pretty rare, it happens. I had the need to detect when a JUnit Test fails and then, if expected, to pass instead of fail. The specific case was that I was testing a piece of code that could throw an Assert error inside of a call of the object. The code was written to be an enhancement to the popular new Fest Assertions framework, so in order to test the functionality, one would expect test cases to fail on purpose. A Solution One possible solution is to utilize the functionality provided by a JUnit @Rule in conjunction with a custom marker in the form of an annotation. . Why use a @Rule? @Rule objects provide an AOP-like interface to a test class and each test cases. Rules are reset prior to each test case being run and they expose the workings of the test case in the style of an @Around AspectJ advice would. Required code elements @Rule object to check the status of each @Test case @ExpectedFailure custom marker annotation Test cases proving code works! Optional specific exception to be thrown if annotated test case does not fail NOTE: working code is available on my github page and will soon be in Maven Central. Feel free to Fork the project and submit a pull request Example Usage In this example, the "exception" object is a Fest assertion enhanced ExpectedException (look for my next post to expose this functionality). The expected exception will make assertions and in order to test those, the test case must be marked as @ExpectedFailure public class ExceptionAssertTest { @Rule public ExpectedException exception = ExpectedException.none(); @Rule public ExpectedTestFailureWatcher watcher = ExpectedTestFailureWatcher.instance(); @Test @ExpectedFailure("The matcher should fail becasue exception is not a SimpleException") public void assertSimpleExceptionAssert_exceptionIsOfType() { // expected exception will be of type "SimpleException" exception.instanceOf(SimpleException.class); // throw something other than SimpleException...expect failure throw new RuntimeException("this is an exception"); } } Implementation of Solution Reminder, the latest code is available on my github page. @Rule code (ExpectedTestFailureWatcher.java) import org.junit.rules.TestRule; import org.junit.runner.Description; import org.junit.runners.model.Statement; // YEAH Guava!! import static com.google.common.base.Strings.isNullOrEmpty; public class ExpectedTestFailureWatcher implements TestRule { /** * Static factory to an instance of this watcher * * @return New instance of this watcher */ public static ExpectedTestFailureWatcher instance() { return new ExpectedTestFailureWatcher(); } @Override public Statement apply(final Statement base, final Description description) { return new Statement() { @Override public void evaluate() throws Throwable { boolean expectedToFail = description.getAnnotation(ExpectedFailure.class) != null; boolean failed = false; try { // allow test case to execute base.evaluate(); } catch (Throwable exception) { failed = true; if (!expectedToFail) { throw exception; // did not expect to fail and failed...fail } } // placed outside of catch if (expectedToFail && !failed) { throw new ExpectedTestFailureException(getUnFulfilledFailedMessage(description)); } } /** * Extracts detailed message about why test failed * @param description * @return */ private String getUnFulfilledFailedMessage(Description description) { String reason = null; if (description.getAnnotation(ExpectedFailure.class) != null) { reason = description.getAnnotation(ExpectedFailure.class).reason(); } if (isNullOrEmpty(reason)) { reason = "Should have failed but didn't"; } return reason; } }; } } @ExpectedFailure custom annotation (ExpectedFailure.java) import java.lang.annotation.*; /** * Initially this is just a marker annotation to be used by a JUnit4 Test case in conjunction * with ExpectedTestFailure @Rule to indicate that a test is supposed to be failing */ @Documented @Retention(RetentionPolicy.RUNTIME) @Target(value = ElementType.METHOD) public @interface ExpectedFailure { // TODO: enhance by adding specific information about what type of failure expected //Class assertType() default Throwable.class; /** * Text based reason for marking test as ExpectedFailure * @return String */ String reason() default ""; } Custom Exception (Optional, you can easily just throw RuntimeException or existing custom exception) public class ExpectedTestFailureException extends Throwable { public ExpectedTestFailureException(String message) { super(message); } } Can't one exploit the ability to mark a failure as expected? With great power comes great responsibility, it is advised that you do not mark a test as being @ExpectedFailure if you do not understand exactly why the test if failing. It is recommended that this testing method be implemented with care. DO NOT use the @ExpectedFailure annotation as an alternative to @Ignore Possible future enhancements could include ways to specify the specific assertion or the specific message asserted during the test case execution. Known issues In this current state, the @ExpectedFailure annotation can cover up additional assertions and until the future enhancements have been put into place, it is advised to use this methodology wisely.

At work, recently I did a code cleanup of an existing Java project. After that exercise, I could see a common set of code violations that occur again and again in the code. So, I came up with a list of such common violations and shared it with my peers so that an awareness would help to improve the code quality and maintainability. I’m sharing the list here to a bigger audience. The list is not in any particular order and all derived from the rules enforced by code quality tools such as CheckStyle, FindBugs and PMD. Here we go! Format source code and Organize imports in Eclipse: Eclipse provides the option to auto-format the source code and organize the imports (thereby removing unused ones). You can use the following shortcut keys to invoke these functions. Ctrl + Shift + F – Formats the source code. Ctrl + Shift + O – Organizes the imports and removes the unused ones. Instead of you manually invoking these two functions, you can tell Eclipse to auto-format and auto-organize whenever you save a file. To do this, in Eclipse, go to Window -> Preferences -> Java -> Editor -> Save Actions and then enable Perform the selected actions on save and check Format source code + Organize imports. Avoid multiple returns (exit points) in methods: In your methods, make sure that you have only one exit point. Do not use returns in more than one places in a method body. For example, the below code is NOT RECOMMENDED because it has more then one exit points (return statements). private boolean isEligible(int age){ if(age > 18){ return true; }else{ return false; } } The above code can be rewritten like this (of course, the below code can be still improved, but that’ll be later). private boolean isEligible(int age){ boolean result; if(age > 18){ result = true; }else{ result = false; } return result; } Simplify if-else methods: We write several utility methods that takes a parameter, checks for some conditions and returns a value based on the condition. For example, consider the isEligible method that you just saw in the previous point. private boolean isEligible(int age){ boolean result; if(age > 18){ result = true; }else{ result = false; } return result; } The entire method can be re-written as a single return statement as below. private boolean isEligible(int age){ return age > 18; } Do not create new instances of Boolean, Integer or String: Avoid creating new instances of Boolean, Integer, String etc. For example, instead of using new Boolean(true), use Boolean.valueOf(true). The later statement has the same effect of the former one but it has improved performance. Use curly braces around block statements. Never forget to use curly braces around block level statements such as if, for, while. This reduces the ambiguity of your code and avoids the chances of introducing a new bug when you modify the block level statement. NOT RECOMMENDED if(age > 18) return true; else return false; RECOMMENDED if(age > 18){ return true; }else{ return false; } Mark method parameters as final, wherever applicable: Always mark the method parameters as final wherever applicable. If you do so, when you accidentally modify the value of the parameter, you’ll get a compiler warning. Also, it makes the compiler to optimize the byte code in a better way. RECOMMENDED private boolean isEligible(final int age){ ... } Name public static final fields in UPPERCASE: Always name the public static final fields (also known as Constants) in UPPERCASE. This lets you to easily differentiate constant fields from the local variables. NOT RECOMMENDED public static final String testAccountNo = "12345678"; RECOMMENDED public static final String TEST_ACCOUNT_NO = "12345678";, Combine multiple if statements into one: Wherever possible, try to combine multiple if statements into single one. For example, the below code; if(age > 18){ if( voted == false){ // eligible to vote. } } can be combined into single if statements, as: if(age > 18 && !voted){ // eligible to vote } switch should have default: Always add a default case for the switch statements. Avoid duplicate string literals, instead create a constant: If you have to use a string in several places, avoid using it as a literal. Instead create a String constant and use it. For example, from the below code, private void someMethod(){ logger.log("My Application" + e); .... .... logger.log("My Application" + f); } The string literal “My Application” can be made as an Constant and used in the code. public static final String MY_APP = "My Application"; private void someMethod(){ logger.log(MY_APP + e); .... .... logger.log(MY_APP + f); } Additional Resources: A collection of Java best practices. List of available Checkstyle checks. List of PMD Rule sets

While exploring HDFS, I came across these two syntaxes for querying HDFS: > hadoop dfs > hadoop fs Initally I couldn't differentiate between the two, and kept wondering why we have two different syntaxes for a common purpose. I found a number of people online with the same question -- their thoughts are below: Per Chris's explanation: it seems like there's no difference between the two syntaxes. If we look at the definitions of the two commands (hadoop fs and hadoop dfs) in $HADOOP_HOME/bin/hadoop ... elif [ "$COMMAND" = "datanode" ] ; then CLASS='org.apache.hadoop.hdfs.server.datanode.DataNode' HADOOP_OPTS="$HADOOP_OPTS $HADOOP_DATANODE_OPTS" elif [ "$COMMAND" = "fs" ] ; then CLASS=org.apache.hadoop.fs.FsShell HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS" elif [ "$COMMAND" = "dfs" ] ; then CLASS=org.apache.hadoop.fs.FsShell HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS" elif [ "$COMMAND" = "dfsadmin" ] ; then CLASS=org.apache.hadoop.hdfs.tools.DFSAdmin HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS" ... That was his reasoning. Unconvinced, I kept looking for a more persuasive answer, and these excerpts made more sense to me: FS relates to a generic file system which can point to any file systems like local, HDFS etc. But dfs is very specific to HDFS. So when we use FS it can perform operation with from/to local or hadoop distributed file system to destination. But specifying DFS operation relates to HDFS. Below are two excerpts from the Hadoop documentation that describe these two as different shells. FS Shell The FileSystem (FS) shell is invoked by bin/hadoop fs. All the FS shell commands take path URIs as arguments. The URI format is scheme://autority/path. For HDFS the scheme is hdfs, and for the local filesystem the scheme is file. The scheme and authority are optional. If not specified, the default scheme specified in the configuration is used. An HDFS file or directory such as /parent/child can be specified as hdfs://namenodehost/parent/child or simply as /parent/child (given that your configuration is set to point to hdfs://namenodehost). Most of the commands in FS shell behave like corresponding Unix commands. DFShell The HDFS shell is invoked by bin/hadoop dfs. All the HDFS shell commands take path URIs as arguments. The URI format is scheme://autority/path. For HDFS the scheme is hdfs, and for the local filesystem the scheme is file. The scheme and authority are optional. If not specified, the default scheme specified in the configuration is used. An HDFS file or directory such as /parent/child can be specified as hdfs://namenode:namenodeport/parent/child or simply as /parent/child (given that your configuration is set to point to namenode:namenodeport). Most of the commands in HDFS shell behave like corresponding Unix commands. So, based on the above, we can conclude that it all depends on the scheme configuration. When using these two commands with absolute URI (i.e. scheme://a/b) the behavior shall be identical. Only it's the default configured scheme value for file and hdfs for fs and dfs respectively, which is the cause for difference in behavior.

Caches have been around in the software world for long time. They’re one of those really useful things that once you start using them, you wonder how on earth you got along without them so, it seems a little strange that the Guys at Spring only got around to adding a caching implementation to Spring core in version 3.1. I’m guessing that previously it wasn’t seen as a priority and besides, before the introduction of Java annotations one of the difficulties of caching was the coupling of caching code with your business code, which could often become pretty messy. However, the Guys at Spring have now devised a simple to use caching system based around a couple of annotations: @Cacheable and @CacheEvict. The idea of the @Cacheable annotation is that you use it to mark the method return values that will be stored in the cache. The @Cacheable annotation can be applied either at method or type level. When applied at method level, then the annotated method’s return value is cached. When applied at type level, then the return value of every method is cached. @Cacheable(value = "employee") public class EmployeeDAO { public Person findEmployee(String firstName, String surname, int age) { return new Person(firstName, surname, age); } public Person findAnotherEmployee(String firstName, String surname, int age) { return new Person(firstName, surname, age); } } The Cacheable annotation takes three arguments: value, which is mandatory, together with key and condition. The first of these, value, is used to specify the name of the cache (or caches) in which the a method’s return value is stored. @Cacheable(value = "employee") public Person findEmployee(String firstName, String surname, int age) { return new Person(firstName, surname, age); } The code above ensures that the new Person object is stored in the “employee” cache. Any data stored in a cache requires a key for its speedy retrieval. Spring, by default, creates caching keys using the annotated method’s signature as demonstrated by the code above. You can override this using @Cacheable’s second parameter: key. To define a custom key you use a SpEL expression. @Cacheable(value = "employee", key = "#surname") public Person findEmployeeBySurname(String firstName, String surname, int age) { return new Person(firstName, surname, age); } In the findEmployeeBySurname(...) code, the ‘#surname’ string is a SpEL expression that means ‘go and create a key using the surname argument of the findEmployeeBySurname(...) method’. The final @Cacheable argument is the optional condition argument. Again, this references a SpEL expression, but this time it’s specifies a condition that’s used to determine whether or not your method’s return value is added to the cache. @Cacheable(value = "employee", condition = "#age < 25") public Person findEmployeeByAge(String firstName, String surname, int age) { return new Person(firstName, surname, age); } In the code above, I’ve applied the ludicrous business rule of only caching Person objects if the employee is less than 25 years old. Having quickly demonstrated how to apply some caching, the next thing to do is to take a look at what it all means. @Test public void testCache() { Person employee1 = instance.findEmployee("John", "Smith", 22); Person employee2 = instance.findEmployee("John", "Smith", 22); assertEquals(employee1, employee2); } The above test demonstrates caching at its simplest. The first call to findEmployee(...), the result isn’t yet cached so my code will be called and Spring will store its return value in the cache. In the second call to findEmployee(...) my code isn’t called and Spring returns the cached value; hence the local variable employee1 refers to the same object reference a @Test public void testCacheWithAgeAsCondition() { Person employee1 = instance.findEmployeeByAge("John", "Smith", 22); Person employee2 = instance.findEmployeeByAge("John", "Smith", 22); assertEquals(employee1, employee2); } s employee2, which means that the following is true: assertEquals(employee1, employee2); But, things aren’t always so clear cut. Remember that in findEmployeeBySurname I’ve modified the caching key so that the surname argument is used to create the key and the thing to watch out for when creating your own keying algorithm is to ensure that any key refers to a unique object. @Test public void testCacheOnSurnameAsKey() { Person employee1 = instance.findEmployeeBySurname("John", "Smith", 22); Person employee2 = instance.findEmployeeBySurname("Jack", "Smith", 55); assertEquals(employee1, employee2); } The code above finds two Person instances which are clearly refer to different employees; however, because I’m caching on surname only, Spring will return a reference to the object that’s created during my first call to findEmployeeBySurname(...). This isn’t a problem with Spring, but with my poor cache key definition. Similar care has to be taken when referring to objects created by methods that have a condition applied to the @Cachable annotation. In my sample code I’ve applied the arbitrary condition of only caching Person instances where the employee is under 25 years old. @Test public void testCacheWithAgeAsCondition() { Person employee1 = instance.findEmployeeByAge("John", "Smith", 22); Person employee2 = instance.findEmployeeByAge("John", "Smith", 22); assertEquals(employee1, employee2); } In the above code, the references to employee1 and employee2 are equal because in the second call to findEmployeeByAge(...) Spring returns its cached instance. @Test public void testCacheWithAgeAsCondition2() { Person employee1 = instance.findEmployeeByAge("John", "Smith", 30); Person employee2 = instance.findEmployeeByAge("John", "Smith", 30); assertFalse(employee1 == employee2); } Similarly, in the unit test code above, the references to employee1 and employee2 refer to different objects as, in this case, John Smith is over 25. That just about covers @Cacheable, but what about @CacheEvict and clearing items form the cache? Also, there’s the question adding caching to your Spring config and choosing a suitable caching implementation. However, more on that later....

Yes, Perl5 can be embedded in node.js! First of all, do a npm install perl. (P.S. node-perl requires a perl5 binary built with -fPIC and -Duseshrplib.) This is synchronous but useful embedded Perl5 for node.js. If you want to try any version of perl, you must check out perl-node. #>git clone git://github.com/hideo55/node-perl.git #>cd node-perl #>node-waf configure #>node-waf build #>node-waf install And then: var Perl = require('perl').Perl(); var perl = new Perl(); perl.Run({ opts : ["-Mfeature=say","-e","say 'Hello world'"] }, function(out,err){ console.log(out); }); perl.Run({ script : 'example.pl', args : ['foo', 'bar'] }); If you opted for Perl5: var Perl = require('perl-simple').Perl; var perl = new Perl(); var ret = perl.evaluate("reverse 'yoeman'"); console.log(ret); // => nameoy var Perl = require('../index.js').Perl; var perl = new Perl(); perl.use('LWP::UserAgent'); var ua = perl.getClass('LWP::UserAgent').new(); var res = ua.get('http://utf-8.jp/'); console.log(res.as_string()); Happy hacking!

You can't seriously program in a language just with scalar types like numbers, strings and atoms. For this reason, now that we have a basic knowledge of Erlang's syntax and variables, we have to delve into two basic vector types: tuples and lists. Both tuples and lists represent a collection of values; however, some rules of thumb (imho) to choosing between them are: tuples deal with heterogeneous values, while lists are homegeneous. A tuple is then usually built as a sequence of values of different types, while all of the values of a list are of the same type. This struct versus array differentiation is true also in Python. Tuples and lists are pattern-matched differently (we'll see more of this when writing pattern matching code, of course). Tuples have O(1) random access, while lists have O(N) random access, being built of cons cells. In general, fixed-size structures are modelled as tuples while sequences of N values (where N varies at runtime) are modelled as lists. Tuples Erlang tuples are similar in syntax to Python's ones: 1> MyTuple = {number, 42}. {number,42} 2> tuple_size(MyTuple). 2 3> element(1, MyTuple). number 4> element(2, MyTuple). 42 And of course they are immutable, like every other value: 5> setelement(2, MyTuple, 43). {number,43} 6> MyTuple. {number,42} They can have any number of values: 7> {true, 23, "Hello"}. {true,23,"Hello"} And the empty tuple is: 8> {}. {} Lists Lists are built as a sequence of cons cells (one of LISP's basic data structures; cons means construct*). Each cons cell is composed by a value and a pointer to another cons cell, which may be empty. Thus the list [1, 2, 3] is composed of three cons cells: p1: [1, p2] p2: [2, p3] p3: [3, p_to_empty_list] Lists can be either built as sequences or even by specifying the cons cells directly. In the first case, values are separated by `,`, while in the second they are separated by `|`: 1> [] 1> . [] 2> [1]. [1] 3> [1, 2]. [1,2] 4> [1 | [2]]. [1,2] 5> [1, 2, 3]. [1,2,3] 6> [1 | [2 | [3]]]. [1,2,3] Every function operating on lists is defined in terms of two primitives, head and tail, which return respectively the first element of the list and the rest of the list with that element removed. While in other languages these functions are provided as head/1 and tail/1 (car and cdr for friends), in Erlang they are implemented via pattern matching; this means they are built into the language syntax. Our little exercise for today is to write these constructs as ordinary functions, to introduce how pattern matching works on lists. head/1 Let's start with a simple case: an empty list. If you ask for the first element of such a list, our implementation should raise an error as there is no such element. #!/usr/bin/escript head([]) -> throw(error). main(_) -> head([]). This indeed shows: escript: exception throw: error in function erl_eval:local_func/5 in call from escript:interpret/4 in call from escript:start/1 in call from init:start_it/1 in call from init:start_em/1 when executed. What has happened? We can put literal values in the formal arguments of a function, and the body of the function will only be executed if the values match these literals. Of course this also means that when we execute: main(_) -> head([1]). we get: escript: exception error: {function_clause,[{local,head,[[1]]}]} since the function is only defined for [] as an argument. Let's add another clause to make it work also for 1-element lists: #!/usr/bin/escript head([]) -> throw( error); head([Element]) -> Element. main(_) -> E = head([1]), io:format("Head: ~p~n", [E]). Note that cases are separated by ; instead of `.` and are evaluated in sequence, so you should put the corner cases (or the base case of recursion) first. Here we didn't use a literal pattern, since we don't know what is in the list in general. We used a variable name, so that Element is filled with the value of the only element of the list. Now we can extend the code further, so that it deals with multiple-value lists: #!/usr/bin/escript head([]) -> throw( error); head([Element]) -> Element; head([Element | _Tail]) -> Element. main(_) -> io:format("Head of [1]: ~p~n", [head([1])]), io:format("Head of [1, 2]: ~p~n", [head([1, 2])]). We use _Tail instead of Tail to tell Erlang that we don't need the value of this argument, but that it must exist. Actually, we know that [1] is actually [1 | []], so we can simplify this code a bit as the third clause would match the single-element list case: #!/usr/bin/escript head([]) -> throw( error); head([Element | _Tail]) -> Element. main(_) -> io:format("Head of [1]: ~p~n", [head([1])]), io:format("Head of [1, 2]: ~p~n", [head([1, 2])]). You're not limited to pattern matching to act on lists: explore the lists* module to see how member(), nth() or length() can be use to test an element's presence, read a single value or calculate the length of the list. I'm out of space for today, so I leave the similar tail/1 implementation as an exercise for the reader. Conclusions Tuples and lists are base Erlang data structures. Exercise with them and with pattern matching in the shell to make sure that you know how to manipulate variables before we move from defining functions to make them collaborate.

Hadoop MapReduce is a YARN-based system for parallel processing of large data sets. In this article, learn to quickly start writing the simplest MapReduce job.

Caches have been around in the software world for long time. They’re one of those really useful things that once you start using them you wonder how on earth you got along without them so, it seems a little strange that the guys at Spring only got around to adding a caching implementation to Spring core in version 3.1. I’m guessing that previously it wasn’t seen as a priority and besides, before the introduction of Java annotations, one of the difficulties of caching was the coupling of caching code with your business code, which could often become pretty messy. However, the guys at Spring have now devised a simple to use caching system based around a couple of annotations: @Cacheable and @CacheEvict. The idea of the @Cacheable annotation is that you use it to mark the method return values that will be stored in the cache. The @Cacheable annotation can be applied either at method or type level. When applied at method level, then the annotated method’s return value is cached. When applied at type level, then the return value of every method is cached. The code below demonstrates how to apply @Cacheable at type level: @Cacheable(value = "employee") public class EmployeeDAO { public Person findEmployee(String firstName, String surname, int age) { return new Person(firstName, surname, age); } public Person findAnotherEmployee(String firstName, String surname, int age) { return new Person(firstName, surname, age); } } The Cacheable annotation takes three arguments: value, which is mandatory, together with key and condition. The first of these, value, is used to specify the name of the cache (or caches) in which the a method’s return value is stored. @Cacheable(value = "employee") public Person findEmployee(String firstName, String surname, int age) { return new Person(firstName, surname, age); } The code above ensures that the new Person object is stored in the “employee” cache. Any data stored in a cache requires a key for its speedy retrieval. Spring, by default, creates caching keys using the annotated method’s signature as demonstrated by the code above. You can override this using @Cacheable’s second parameter: key. To define a custom key you use a SpEL expression. @Cacheable(value = "employee", key = "#surname") public Person findEmployeeBySurname(String firstName, String surname, int age) { return new Person(firstName, surname, age); } In the findEmployeeBySurname(...) code, the ‘#surname’ string is a SpEL expression that means ‘go and create a key using the surname argument of the findEmployeeBySurname(...) method’. The final @Cacheable argument is the optional condition argument. Again, this references a SpEL expression, but this time it’s specifies a condition that’s used to determine whether or not your method’s return value is added to the cache. @Cacheable(value = "employee", condition = "#age < 25") public Person findEmployeeByAge(String firstName, String surname, int age) { return new Person(firstName, surname, age); } In the code above, I’ve applied the ludicrous business rule of only caching Person objects if the employee is less than 25 years old. Having quickly demonstrated how to apply some caching, the next thing to do is to take a look at what it all means. @Test public void testCache() { Person employee1 = instance.findEmployee("John", "Smith", 22); Person employee2 = instance.findEmployee("John", "Smith", 22); assertEquals(employee1, employee2); } The above test demonstrates caching at its simplest. The first call to findEmployee(...), the result isn’t yet cached so my code will be called and Spring will store its return value in the cache. In the second call to findEmployee(...) my code isn’t called and Spring returns the cached value; hence the local variable employee1 refers to the same object reference as employee2, which means that the following is true: assertEquals(employee1, employee2); But, things aren’t always so clear cut. Remember that in findEmployeeBySurname I’ve modified the caching key so that the surname argument is used to create the key and the thing to watch out for when creating your own keying algorithm is to ensure that any key refers to a unique object. @Test public void testCacheOnSurnameAsKey() { Person employee1 = instance.findEmployeeBySurname("John", "Smith", 22); Person employee2 = instance.findEmployeeBySurname("Jack", "Smith", 55); assertEquals(employee1, employee2); } The code above finds two Person instances which are clearly refer to different employees; however, because I’m caching on surname only, Spring will return a reference to the object that’s created during my first call to findEmployeeBySurname(...). This isn’t a problem with Spring, but with my poor cache key definition. Similar care has to be taken when referring to objects created by methods that have a condition applied to the @Cachable annotation. In my sample code I’ve applied the arbitrary condition of only caching Person instances where the employee is under 25 years old. @Test public void testCacheWithAgeAsCondition() { Person employee1 = instance.findEmployeeByAge("John", "Smith", 22); Person employee2 = instance.findEmployeeByAge("John", "Smith", 22); assertEquals(employee1, employee2); } In the above code, the references to employee1 and employee2 are equal because in the second call to findEmployeeByAge(...) Spring returns its cached instance. @Test public void testCacheWithAgeAsCondition2() { Person employee1 = instance.findEmployeeByAge("John", "Smith", 30); Person employee2 = instance.findEmployeeByAge("John", "Smith", 30); assertFalse(employee1 == employee2); } Similarly, in the unit test code above, the references to employee1 and employee2 refer to different objects as, in this case, John Smith is over 25. That just about covers @Cacheable, but what about @CacheEvict and clearing items form the cache? Also, there’s the question adding caching to your Spring config and choosing a suitable caching implementation. However, more on that later...

In many Java projects, you often see wrapper shell script to invoke the java command with its custom application parameters. For example, $ANT_HOME/bin/ant, $GROOVY_HOME/bin/groovy, or even in our TimeMachine Scheduler you will see $TIMEMACHINE_HOME/bin/scheduler.sh. Writing these wrapper script is boring and error prone. Most of the problems come from setting the correct classpath for the application. If you're working on an in-house project for a company, then you can get away with hardcoding paths and your environment vars. But for open source projects, folks have to make the wrapper more flexible and generic. Most of them even provide a .bat version of it. Windows DOS is really a brutal and limited terminal to script away your project need. For this reason, I often encourage others to use Cygwin as much as they can. It at least has a real bash shell to work with. Another common problem with these wrappers is it can quickly get out of hand and have too many duplication of similar scripts liter every where in your project. In this post, I will show you a Java wrapper script that I've written. It's simple to use and very flexible for running just about any Java program. Let's see how it's used first, and then I will print its content at the bottom of the post. Introducing the run-java wrapper script If you take a look at $TIMEMACHINE_HOME/bin/scheduler.sh, you will see that it in turns calls a run-java script that comes in the same directory. DIR=$(dirname $0) SCHEDULER_HOME=$DIR/.. $DIR/run-java -Dscheduler.home="$SCHEDULER_HOME" timemachine.scheduler.tool.SchedulerServer "$@" As you can see, our run-java can take -D options. Not only this, it can also take -cp option as well! What's more is that you can specify these options even after the main class! This makes the run-java re-wrappable by other script, and still be able to add additional system properties and classpath. For examples, the TimeMachine comes with Groovy library, so instead of downloading it's full distribution again, you can simply invoke the groovy like this $TIMEMACHINE_HOME/bin/run-java groovy.ui.GroovyMain test.groovy You can use run-java in any directory you're in, so it's convenient. It will resolve it's own directory and load any jars in the lib directory automatically. Now if you want Groovy to run with more additional jars, you can use the -cp option like this: $TIMEMACHINE_HOME/bin/run-java -cp "$HOME/apps/my-app/lib/*" groovy.ui.GroovyMain test.groovy Often times things will go wrong if you are not careful with Java classpath, but with run-java script you can perform a dry run first: RUN_JAVA_DRY=1 $TIMEMACHINE_HOME/bin/run-java -cp "$HOME/apps/my-app/lib/*" groovy.ui.GroovyMain test.groovy You would run the above all in single line on a command prompt. It should print out your full java command with all options and arguments for you to inspect. There are many more options to the script, which you can find out more by reading the comments in it. The current script will work on any Linux bash or on a Windows Cygwin terminal. Using run-java during development with Maven Above examples are assuming you are in a released project structure such as this $TIMEMACHINE_HOME +- bin/run-java +- lib/*.jar But what about during development? A frequent use case is that you want to be able to run your latest compiled classes under target/classes without have to package up or release the entire project. You can use our run-java in these scenario as well. First, simply add bin/run-java in your project, then you run mvn compile dependency:copy-dependencies that will generate all the jar files into target/dependency. That's all. The run-java will automatically detect these directories and create the correct classpath to run your main class. If you use Eclipse IDE for development, then your target/classes will be always up-to-date, and the run-java can be a great gem to have in your project even for development. Get the run-java wrapper script now #!/usr/bin/env bash # # Copyright 2012 Zemian Deng # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # A wrapper script that run any Java6 application in unix/cygwin env. # # This script is assumed to be located in an application's "bin" directory. It will # auto resolve any symbolic link and always run in relative to this application # directory (which is one parent up from the script.) Therefore, this script can be # run any where in the file system and it will still reference this application # directory. # # This script will by default auto setup a Java classpath that picks up any "config" # and "lib" directories under the application directory. It also will also add a # any typical Maven project output directories such as "target/test-classes", # "target/classes", and "target/dependency" into classpath. This can be disable by # setting RUN_JAVA_NO_PARSE=1. # # If the "Default parameters" section bellow doesn't match to user's env, then user # may override these variables in their terminal session or preset them in shell's # profile startup script. The values of all path should be in cygwin/unix path, # and this script will auto convert them into Windows path where is needed. # # User may customize the Java classpath by setting RUN_JAVA_CP, which will prefix to existing # classpath, or use the "-cp" option, which will postfix to existing classpath. # # Usage: # run-java [java_opts] [-cp /more/classpath] [-Dsysprop=value] # # Example: # run-java example.Hello # run-java example.Hello -Dname=World # run-java org.junit.runner.JUnitCore example.HelloTest -cp "C:\apps\lib\junit4.8.2\*" # # Created by: Zemian Deng 03/09/2012 # This run script dir (resolve to absolute path) SCRIPT_DIR=$(cd $(dirname $0) && pwd) # This dir is where this script live. APP_DIR=$(cd $SCRIPT_DIR/.. && pwd) # Assume the application dir is one level up from script dir. # Default parameters JAVA_HOME=${JAVA_HOME:=/apps/jdk} # This is the home directory of Java development kit. RUN_JAVA_CP=${RUN_JAVA_CP:=$CLASSPATH} # A classpath prefix before -classpath option, default to $CLASSPATH RUN_JAVA_OPTS=${RUN_JAVA_OPTS:=} # Java options (-Xmx512m -XX:MaxPermSize=128m etc) RUN_JAVA_DEBUG=${RUN_JAVA_DEBUG:=} # If not empty, print the full java command line before executing it. RUN_JAVA_NO_PARSE=${RUN_JAVA_NO_PARSE:=} # If not empty, skip the auto parsing of -D and -cp options from script arguments. RUN_JAVA_NO_AUTOCP=${RUN_JAVA_NO_AUTOCP:=} # If not empty, do not auto setup Java classpath RUN_JAVA_DRY=${RUN_JAVA_DRY:=} # If not empty, do not exec Java command, but just print # OS specific support. $var _must_ be set to either true or false. CYGWIN=false; case "`uname`" in CYGWIN*) CYGWIN=true ;; esac # Define where is the java executable is JAVA_CMD=java if [ -d "$JAVA_HOME" ]; then JAVA_CMD="$JAVA_HOME/bin/java" fi # Auto setup applciation's Java Classpath (only if they exists) if [ -z "$RUN_JAVA_NO_AUTOCP" ]; then if $CYGWIN; then # Provide Windows directory conversion JAVA_HOME_WIN=$(cygpath -aw "$JAVA_HOME") APP_DIR_WIN=$(cygpath -aw "$APP_DIR") if [ -d "$APP_DIR_WIN\config" ]; then RUN_JAVA_CP="$RUN_JAVA_CP;$APP_DIR_WIN\config" ; fi if [ -d "$APP_DIR_WIN\target\test-classes" ]; then RUN_JAVA_CP="$RUN_JAVA_CP;$APP_DIR_WIN\target\test-classes" ; fi if [ -d "$APP_DIR_WIN\target\classes" ]; then RUN_JAVA_CP="$RUN_JAVA_CP;$APP_DIR_WIN\target\classes" ; fi if [ -d "$APP_DIR_WIN\target\dependency" ]; then RUN_JAVA_CP="$RUN_JAVA_CP;$APP_DIR_WIN\target\dependency\*" ; fi if [ -d "$APP_DIR_WIN\lib" ]; then RUN_JAVA_CP="$RUN_JAVA_CP;$APP_DIR_WIN\lib\*" ; fi else if [ -d "$APP_DIR/config" ]; then RUN_JAVA_CP="$RUN_JAVA_CP:$APP_DIR/config" ; fi if [ -d "$APP_DIR/target/test-classes" ]; then RUN_JAVA_CP="$RUN_JAVA_CP:$APP_DIR/target/test-classes" ; fi if [ -d "$APP_DIR/target/classes" ]; then RUN_JAVA_CP="$RUN_JAVA_CP:$APP_DIR/target/classes" ; fi if [ -d "$APP_DIR/target/dependency" ]; then RUN_JAVA_CP="$RUN_JAVA_CP:$APP_DIR/target/dependency/*" ; fi if [ -d "$APP_DIR/lib" ]; then RUN_JAVA_CP="$RUN_JAVA_CP:$APP_DIR/lib/*" ; fi fi fi # Parse addition "-cp" and "-D" after the Java main class from script arguments # This is done for convenient sake so users do not have to export RUN_JAVA_CP and RUN_JAVA_OPTS # saparately, but now they can pass into end of this run-java script instead. # This can be disable by setting RUN_JAVA_NO_PARSE=1. if [ -z "$RUN_JAVA_NO_PARSE" ]; then # Prepare variables for parsing FOUND_CP= declare -a NEW_ARGS IDX=0 # Parse all arguments and look for "-cp" and "-D" for ARG in "$@"; do if [[ -n $FOUND_CP ]]; then if [ "$OS" = "Windows_NT" ]; then # Can't use cygpath here, because cygpath will auto expand "*", which we do not # want. User will just have to use OS path when specifying "-cp" option. #ARG=$(cygpath -w -a $ARG) RUN_JAVA_CP="$RUN_JAVA_CP;$ARG" else RUN_JAVA_CP="$RUN_JAVA_CP:$ARG" fi FOUND_CP= else case $ARG in '-cp') FOUND_CP=1 ;; '-D'*) RUN_JAVA_OPTS="$RUN_JAVA_OPTS $ARG" ;; *) NEW_ARGS[$IDX]="$ARG" let IDX=$IDX+1 ;; esac fi done # Display full Java command. if [ -n "$RUN_JAVA_DEBUG" ] || [ -n "$RUN_JAVA_DRY" ]; then echo "$JAVA_CMD" $RUN_JAVA_OPTS -cp "$RUN_JAVA_CP" "${NEW_ARGS[@]}" fi # Run Java Main class using parsed variables if [ -z "$RUN_JAVA_DRY" ]; then "$JAVA_CMD" $RUN_JAVA_OPTS -cp "$RUN_JAVA_CP" "${NEW_ARGS[@]}" fi else # Display full Java command. if [ -n "$RUN_JAVA_DEBUG" ] || [ -n "$RUN_JAVA_DRY" ]; then echo "$JAVA_CMD" $RUN_JAVA_OPTS -cp "$RUN_JAVA_CP" "$@" fi # Run Java Main class if [ -z "$RUN_JAVA_DRY" ]; then "$JAVA_CMD" $RUN_JAVA_OPTS -cp "$RUN_JAVA_CP" "$@" fi fi

From their site, it says the Apache Camel is a versatile open-source integration framework based on known Enterprise Integration Patterns. It might seem like a vague definition, but I want to tell you that this is a very productive Java library that can solve many of typical IT problems! You can think of it as a very light weight ESB framework with "batteries" included. In every jobs I've been to so far, folks are writing their own solutions in one way or another to solve many common problems (or they would buy some very expensive enterprisy ESB servers that takes months and months to learn, config, and maintain). Things that we commonly solve are integration (glue) code of existing business services together, process data in a certain workflow manner, or move and transform data from one place to another etc. These are very typical need in many IT environments. The Apache Camel can be used in cases like these; not only that, but also in a very productive and effective way! In this article, I will show you how to get started with Apache Camel along with just few lines of Groovy script. You can certainly also start off with a full Java project to try out Camel, but I find Groovy will give you the shortest working example and learning curve. Getting started with Apache Camel using Groovy So let's begin. First let's see a hello world demo with Camel + Groovy. @Grab('org.apache.camel:camel-core:2.10.0') @Grab('org.slf4j:slf4j-simple:1.6.6') import org.apache.camel.* import org.apache.camel.impl.* import org.apache.camel.builder.* def camelContext = new DefaultCamelContext() camelContext.addRoutes(new RouteBuilder() { def void configure() { from("timer://jdkTimer?period=3000") .to("log://camelLogger?level=INFO") } }) camelContext.start() addShutdownHook{ camelContext.stop() } synchronized(this){ this.wait() } Save above into a file named helloCamel.groovy and then run it like this: $ groovy helloCamel.groovy 388 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.10.0 (CamelContext: camel-1) is starting 445 [main] INFO org.apache.camel.management.ManagementStrategyFactory - JMX enabled. 447 [main] INFO org.apache.camel.management.DefaultManagementLifecycleStrategy - StatisticsLevel at All so enabling load performance statistics 678 [main] INFO org.apache.camel.impl.converter.DefaultTypeConverter - Loaded 170 type converters 882 [main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route1 started and consuming from: Endpoint[timer://jdkTimer?period=3000] 883 [main] INFO org.apache.camel.impl.DefaultCamelContext - Total 1 routes, of which 1 is started. 887 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.10.0 (CamelContext: camel-1) started in 0.496 seconds 898 [Camel (camel-1) thread #1 - timer://jdkTimer] INFO camelLogger - Exchange[ExchangePattern:InOnly, BodyType:null, Body:[Body is null]] 3884 [Camel (camel-1) thread #1 - timer://jdkTimer] INFO camelLogger - Exchange[ExchangePattern:InOnly, BodyType:null, Body:[Body is null]] 6884 [Camel (camel-1) thread #1 - timer://jdkTimer] INFO camelLogger - Exchange[ExchangePattern:InOnly, BodyType:null, Body:[Body is null]] ... The little script above is simple but it presented few key features of Camel Groovyness. The first and last section of the helloCamel.groovy script are just Groovy featuers. The @Grab annotation will automatically download the dependency jars you specify. We import Java packages to use its classes later. At the end we ensure to shutdown Camel before exiting JVM through the Java Shutdown Hook mechanism. The program will sit and wait until user press CTRL+C, just as a typical server process behavior. The middle section is where the Camel action is. You would always create a Camel context to begin (think of it as the server or manager for the process.) And then you would add a Camel route (think of it as a workflow or pipeflow) that you like to process data (Camel likes to call these data "messages"). The route consists of a "from" starting point (where data generated), and one or more "to" points (where data going to be processed). Camel calls these destination 'points' as 'Endpoints'. These endpoints can be expressed in simple URI string format such as "timer://jdkTimer?period=3000". Here we are generating timer message in every 3 secs into the pipeflow, and then process by a logger URI, which will simply print to console output. After Camel context started, it will start processing data through the workflow, as you can observe from the output example above. Now try pressing CTRL+C to end its process. Notice how the Camel will shutdown everything very gracefully. 7312 [Thread-2] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.10.0 (CamelContext: camel-1) is shutting down 7312 [Thread-2] INFO org.apache.camel.impl.DefaultShutdownStrategy - Starting to graceful shutdown 1 routes (timeout 300 seconds) 7317 [Camel (camel-1) thread #2 - ShutdownTask] INFO org.apache.camel.impl.DefaultShutdownStrategy - Route: route1 shutdown complete, was consuming from: Endpoint[timer://jdkTimer?period=3000] 7317 [Thread-2] INFO org.apache.camel.impl.DefaultShutdownStrategy - Graceful shutdown of 1 routes completed in 0 seconds 7321 [Thread-2] INFO org.apache.camel.impl.converter.DefaultTypeConverter - TypeConverterRegistry utilization[attempts=2, hits=2, misses=0, failures=0] mappings[total=170, misses=0] 7322 [Thread-2] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.10.0 (CamelContext: camel-1) is shutdown in 0.010 seconds. Uptime 7.053 seconds. So that's our first taste of Camel ride! However, we titled this section as "Hello World!" demo, and yet we haven't seen any. But you might have also noticed that above script are mostly boiler plate code that we setup. No user logic has been added yet. Not even the logging the message part! We simply configuring the route. Now let's modify the script little bit so we will actually add our user logic to process the timer message. @Grab('org.apache.camel:camel-core:2.10.0') @Grab('org.slf4j:slf4j-simple:1.6.6') import org.apache.camel.* import org.apache.camel.impl.* import org.apache.camel.builder.* def camelContext = new DefaultCamelContext() camelContext.addRoutes(new RouteBuilder() { def void configure() { from("timer://jdkTimer?period=3000") .to("log://camelLogger?level=INFO") .process(new Processor() { def void process(Exchange exchange) { println("Hello World!") } }) } }) camelContext.start() addShutdownHook{ camelContext.stop() } synchronized(this){ this.wait() } Notice how I can simply append the process code part right after the to("log...") line. I have added a "processor" code block to process the timer message. The logic is simple: we greet the world on each tick. Making Camel route more concise and practical Now, do I have you at Hello yet? If not, then I hope you will be patient and continue to follow along for few more practical features of Camel. First, if you were to put Camel in real use, I would recommend you setup your business logic separately from the workflow route definition. This is so that you can clearly express and see your entire pipeflow of route at a glance. To do this, you want to move the "processor", into a service bean. @Grab('org.apache.camel:camel-core:2.10.0') @Grab('org.slf4j:slf4j-simple:1.6.6') import org.apache.camel.* import org.apache.camel.impl.* import org.apache.camel.builder.* import org.apache.camel.util.jndi.* class SystemInfoService { def void run() { println("Hello World!") } } def jndiContext = new JndiContext(); jndiContext.bind("systemInfoPoller", new SystemInfoService()) def camelContext = new DefaultCamelContext(jndiContext) camelContext.addRoutes(new RouteBuilder() { def void configure() { from("timer://jdkTimer?period=3000") .to("log://camelLogger?level=INFO") .to("bean://systemInfoPoller?method=run") } }) camelContext.start() addShutdownHook{ camelContext.stop() } synchronized(this){ this.wait() } Now, see how compact this workflow route has become? The Camel's Java DSL such as "from().to().to()" for defining route are so clean and simple to use. You can even show this code snip to your Business Analysts, and they would likely be able to verify your business flow easily! Wouldn't that alone worth a million dollars? How about another demo: FilePoller Processing File polling processing is a very common and effective way to solve many business problems. If you work for commercial companies long enough, you might have written one before. A typical file poller would process incoming files from a directory and then process the content, and then move the file into a output directory. Let's make a Camel route to do just that. @Grab('org.apache.camel:camel-core:2.10.0') @Grab('org.slf4j:slf4j-simple:1.6.6') import org.apache.camel.* import org.apache.camel.impl.* import org.apache.camel.builder.* import org.apache.camel.util.jndi.* class UpperCaseTextService { def String transform(String text) { return text.toUpperCase() } } def jndiContext = new JndiContext(); jndiContext.bind("upperCaseTextService", new UpperCaseTextService()) def dataDir = "/${System.properties['user.home']}/test/file-poller-demo" def camelContext = new DefaultCamelContext(jndiContext) camelContext.addRoutes(new RouteBuilder() { def void configure() { from("file://${dataDir}/in") .to("log://camelLogger") .to("bean://upperCaseTextService?method=transform") .to("file://${dataDir}/out") } }) camelContext.start() addShutdownHook{ camelContext.stop() } synchronized(this){ this.wait() } Here you see I defined a route to poll a $HOME/test/file-poller-demo/in directory for text files. Once it's found it will log it to console, and then process by a service that transform the content text into upper case. After this, it will send the file into $HOME/test/file-poller-demo/out directory. My goodness, reading the Camel route above probably express what I wrote down just as effective. Do you see the benefits here? What's the "batteries" included part. If you've used Python programming before, you might have heard the pharase that they claim often: Python has "batteries" included. This means their interpreter comes with a rich of libaries for most of the common programming need. You can often write python program without have to download separated external libraries. I am making similar analogies here with Apache Camel. The Camel project comes with so many ready to use components that you can find just about any transport protocals that can carry data. These Camel "components" are ones that support different 'Endpoint URI' that we have seen in our demos above. We have simply shown you timer, log, bean, and file components, but there are over 120 more. You will find jms, http, ftp, cfx, or tcp just to name a few. The Camel project also has an option for you to define route in declarative xml format. The xml is just an extension of a Spring xml config with Camel's namespace handler added on top. Spring is optional in Camel, but you can use it together in a very powerful way.

schema noun ( pl. schemata or schemas ) 1 technical a representation of a plan or theory in the form of an outline or model: a schema of scientific reasoning. 2 Logic a syllogistic figure. 3 (in Kantian philosophy) a conception of what is common to all members of a class; a general or essential type or form. CouchDB is a schema-less document store, but there are times when a schema is a good thing to have around, one way or another. So can you have your cake and eat it too? Below I'll take a high level look at adding a kind of schema to an application and the benefits and draw backs associated with this way of working. What I describe below isn't for everyone. It goes against some of the core principles of CouchDB and makes your data much less human readable, but there are cases where that trade off is worth making. Schemas: WTF?! It might seem a bit weird to add a schema to a schema-less database but sometimes it is a very useful thing indeed. When you're dealing with large datasets verbose object key names can be a problem (e.g. cost you money) so you end up stuck between a rock and a hard place; either make your data terse and hard to use or be explicit and spend more on storage and network. { "shape": "triangle", "colour_label": "red", "opposite_length_in_mm": 767.12254256805875, "angle_in_radians": 1.5514293603308698, "adjacent_length_in_mm": 73.59881843627835 } What usually happens is some middle ground where a nice descriptive name like "angle_in_radians" gets reduced to "angle" or "rads". That's fine in that it reduces the storage and network required to deal with all that data. { "adj": 73.59881843627835, "shape": "triangle", "angle": 1.5514293603308698, "opp": 767.12254256805875, "colour": "red" } However, by making this small change you move the description of the data out of your database and into some undefined place; higher level code, documentation, shared knowledge, a whiteboard, a notebook, someones head. As your data becomes more terse you might rely on duck typing (deriving from the data itself what the data describes) to get data that quacks right in your application. That's fine so long as you have data that is sufficiently distinguishable from the other ducks on the pond; if I rely on pulling a triangle object from the database because it has an angle member I might accidentally pull out a rhombus or an icosahedron. To make sure you get the data you expect you might add an explicit type field to each data (e.g. "type=goose" or "shape=triangle") something which I've always felt was rather odd. This starts to add up on storage (remember you have a large dataset/flock of ducks) and, more importantly, it doesn't help with where the description of the data is held - you know that you have a goose but don't know what a goose is. This last point is important, especially if you're working in a team of developers. Knowing what describing a shape as a triangle means is vital in producing consistent code that many people can work on. The straight jacket of a SQL schema looks pretty comfy sometimes. Okay, I'll buy that a schema might be useful... So how do you add a schema into a CouchDB database, something that is inherently schema-less? Can I get the best of both worlds? Here's a little trick that might help. First you define a document that is the schema for a particular type of data: { "_id": "datatype/triangle/v1", "fields": [ "opposite_length_in_mm", "adjacent_length_in_mm", "angle_in_radians", "colour_label" ] } Then you change your document structure to reference that "schema": { "datatype": "triangle/v1", "data": [ 879.07395066446952, 84.607510245708468, 1.4444230241122715, "red" ] } Note that the schema is versioned and that ordering in the data list is important here! I now know precisely what the data represents without having to store that description in the data itself. This way of working has benefits beyond disk storage; you reduce wire traffic, and there is less for a client to parse before rendering it. This is especially useful if you're rendering into a browser based visualisation - you don't need a complex set of objects to make a bar chart, just a list of x and y values. I can also share the data structure with colleagues and be reasonably confident that when I'm talking about a "v1 triangle" they'll know that lengths are in millimeters, are the opposite and adjacent sides and that the angle is in radians, hopefully reducing the chance of costly mistakes. Isn't that error prone? Yes and no. If you make a mistake in the ordering of your fields then, yes you are going to have issues. This is reasonably easy to manage with some form of client verification (e.g. validation on a web form) and generating the interface from the data (e.g. use the schema definition to build the GUI). If you're adding these data into the database by hand (e.g. via a curl or futon) then you aren't going to be in the regime where this trick is useful; your dataset needs to be large for this to make sense. Things still quack What's particularly nice about this way of working is that I can still duck type the data, add additional fields to annotate it etc. since the schema isn't strictly enforced. Nothing stops me from having a triangle document like: { "datatype": "triangle/v1", "data": [ 879.07395066446952, 84.607510245708468, 1.4444230241122715, "red" ], "owner": "Simon", "location" "space" } My views that deal with the data with a schema will still work (by ignoring these additional fields), my MVC framework will still render my pages, and I'll still have all the data I want in my database. Nesting You could have a nested object structure like: { "datatype": "pattern/v1", "data": [ { "datatype": "triangle/v1", "data": [ 879.07395066446952, 84.607510245708468, 1.4444230241122715, "red" ], "owner": "Simon", "location" "space" }, { "datatype": "triangle/v1", "data": [ 879.07395066446952, 84.607510245708468, 1.4444230241122715, "blue" ], "owner": "Fred", "location" "space" }, { "datatype": "square/v1", data: [ 10, "green" ] } ] } But if you're going to have a schema you may as well reflect the nesting inside it, e.g say that you have a list of triangles and a list of squares: { "_id": "datatype/pattern/v1", "fields": [ ["triangle/v1"], ["square/v1"] ] } { "datatype": "pattern/v1", "data": [ [ { "data": [ 879.07395066446952, 84.607510245708468, 1.4444230241122715, "red" ], "owner": "Simon", "location" "space" }, { "data": [ 879.07395066446952, 84.607510245708468, 1.4444230241122715, "blue" ], "owner": "Fred", "location" "space" } ], [ { data: [ 10, "green" ] } ] } Schema evolution A nice feature of this way of working is that you can deal with schema evolutions; changing the format of your data. { "_id": "datatype/triangle/v2", "fields": [ "opposite_length_in_cm", "hypotenuse_length_in_cm", "angle_in_degrees", "colour_label" ] } There are only so many ways you can represent the data. While sometimes you may have a major schema evolution, one where old data is completely unusable, often changes are just tweaks for consistency (say changing the units of a quantity) or extending the schema by adding in optional data. In either case you should be able to use data from multiple schema versions together by using appropriate manipulations on the data. For example you could instantiate shape objects via a factory which knows how to create the right object for different schema versions. Validation The above does no validation of the data; the color field in the input data could be set to a number instead of a string, the angle to something non- physical etc. If you really needed validation you could do it with CouchDB's validation functions. If you go the fully validated route you'd want to define the schema in the design document (instead of as a normal doc) and use a CommonJS include to make sure that the validator in the app was doing the same thing as the schema. This ties you to a version of the design document (which is where the validators live), which may or may not be an issue. It will also considerably slow down insertion rate as CouchDB has to do more work to add your data. Personally I prefer to put validation logic in the client making writes. Views If I were using this way of working I would want to have a view which returned all the schema's defined on the database. This then allows me to build objects appropriately. A view to return schema's documents would look like: function(doc) { if (doc._id.slice(0, 'datatype'.length) == 'datatype') { emit (doc._id.slice('datatype/'.length, doc._id.length), doc.fields) } } You can pull out documents that have a schema with a simple view like: function(doc) { if (doc.datatype){ emit(doc.datatype, doc.data); } } This can be queried to find objects of a given shape using CouchDB's view slicing (e.g. ?startkey="square/v1"&endkey="square/v2") which returns data like: {"id":"datatype/square/v1","key":["square/v1",0],"value":["side_length_in_mm","colour_label"]}, {"id":"f98ffe7e4cd91cbb0d904f9098499ca8","key":["square/v1",1],"value":[872.4342711412228,"green"]}, {"id":"f98ffe7e4cd91cbb0d904f909849a218","key":["square/v1",1],"value":[370.29971491443905,"yellow"]}, {"id":"f98ffe7e4cd91cbb0d904f909849acd0","key":["square/v1",1],"value":[8.799279300193753,"yellow"]} You'll notice the name of the "schema" is the key and the values are held in value. This means I can parse the data into a set of appropriate objects with something like: var objects = []; function build(schema, data){ // Build the appropriate object for the schema... } for (row in data){ // build up the objects in a factory var obj = build(row.key, row.value); objects.push(obj); } If I wanted all versions of a shape the query would be, and used a vNUMERIC_COUNTER notation for versioning, ?startkey="square/v1"&endkey="square/vXXX" as numbers sort lower than strings. Taking it to the extreme If you are really worried about data size you can take this technique to the extreme by encoding the data arrays as a byte string and using the schema documents to describe that byte array. This effectively turns your JSON structure into something not dissimilar to a protocol buffer, at the expense of human readability and view complexity. If you are particularly concerned with data size over the wire (for example are writing an MMORPG) then this may be an acceptable trade off. Reminder This trick isn't suitable for every dataset. If you modify the data by hand it is prone to error. If you have a small dataset, or only ever send a small subset of the data to the client it's massive overkill. But if you have a large dataset of machine generated data, that needs to be frequently accessed over the WAN (think a monitoring app or game) then this is a nice way to reduce storage, network IO and browser render time. It's also worth reiterating that the schema is not enforced, you could have a square with 3 sides, and that adding strict schema enforcement with a validation function will considerably slow down insert rate.

As you may have seen from my past performance related articles and HashMap case studies, Java thread safety problems can bring down your Java EE application and the Java EE container fairly easily. One of most common problems I have observed when troubleshooting Java EE performance problems is infinite looping triggered from the non-thread safe HashMap get() and put() operations. This problem is known since several years but recent production problems have forced me to revisit this issue one more time. This article will revisit this classic thread safety problem and demonstrate, using a simple Java program, the risk associated with a wrong usage of the plain old java.util.HashMap data structure involved in a concurrent threads context. This proof of concept exercise will attempt to achieve the following 3 goals: Revisit and compare the Java program performance level between the non-thread safe and thread safe Map data structure implementations (HashMap, Hashtable, synchronized HashMap, ConcurrentHashMap) Replicate and demonstrate the HashMap infinite looping problem using a simple Java program that everybody can compile, run and understand Review the usage of the above Map data structures in a real-life and modern Java EE container implementation such as JBoss AS7 For more detail on the ConcurrentHashMap implementation strategy, I highly recommend the great article from Brian Goetz on this subject. Tools and server specifications As a starting point, find below the different tools and software’s used for the exercise: Sun/Oracle JDK & JRE 1.7 64-bit Eclipse Java EE IDE Windows Process Explorer (CPU per Java Thread correlation) JVM Thread Dump (stuck thread analysis and CPU per Thread correlation) The following local computer was used for the problem replication process and performance measurements: Intel(R) Core(TM) i5-2520M CPU @ 2.50Ghz (2 CPU cores, 4 logical cores) 8 GB RAM Windows 7 64-bit * Results and performance of the Java program may vary depending of your workstation or server specifications. Java program In order to help us achieve the above goals, a simple Java program was created as per below: The main Java program is HashMapInfiniteLoopSimulator.java A worker Thread class WorkerThread.java was also created The program is performing the following: Initialize different static Map data structures with initial size of 2 Assign the chosen Map to the worker threads (you can chose between 4 Map implementations) Create a certain number of worker threads (as per the header configuration). 3 worker threads were created for this proof of concept NB_THREADS = 3; Each of these worker threads has the same task: lookup and insert a new element in the assigned Map data structure using a random Integer element between 1 – 1 000 000. Each worker thread perform this task for a total of 500K iterations The overall program performs 50 iterations in order to allow enough ramp up time for the HotSpot JVM The concurrent threads context is achieved using the JDK ExecutorService As you can see, the Java program task is fairly simple but complex enough to generate the following critical criteria’s: Generate concurrency against a shared / static Map data structure Use a mix of get() and put() operations in order to attempt to trigger internal locks and / or internal corruption (for the non-thread safe implementation) Use a small Map initial size of 2, forcing the internal HashMap to trigger an internal rehash/resize Finally, the following parameters can be modified at your convenience: ## Number of worker threads private static final int NB_THREADS = 3; ## Number of Java program iterations private static final int NB_TEST_ITERATIONS = 50; ## Map data structure assignment. You can choose between 4 structures // Plain old HashMap (since JDK 1.2) nonThreadSafeMap = new HashMap(2); // Plain old Hashtable (since JDK 1.0) threadSafeMap1 = new Hashtable(2); // Fully synchronized HashMap threadSafeMap2 = new HashMap(2); threadSafeMap2 = Collections.synchronizedMap(threadSafeMap2); // ConcurrentHashMap (since JDK 1.5) threadSafeMap3 = new ConcurrentHashMap(2); /*** Assign map at your convenience ****/ assignedMapForTest = threadSafeMap3; Now find below the source code of our sample program. #### HashMapInfiniteLoopSimulator.java package org.ph.javaee.training4; import java.util.Collections; import java.util.Map; import java.util.HashMap; import java.util.Hashtable; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; /** * HashMapInfiniteLoopSimulator * @author Pierre-Hugues Charbonneau * */ public class HashMapInfiniteLoopSimulator { private static final int NB_THREADS = 3; private static final int NB_TEST_ITERATIONS = 50; private static Map assignedMapForTest = null; private static Map nonThreadSafeMap = null; private static Map threadSafeMap1 = null; private static Map threadSafeMap2 = null; private static Map threadSafeMap3 = null; /** * Main program * @param args */ public static void main(String[] args) { System.out.println("Infinite Looping HashMap Simulator"); System.out.println("Author: Pierre-Hugues Charbonneau"); System.out.println("http://javaeesupportpatterns.blogspot.com"); for (int i=0; i(2); // Plain old Hashtable (since JDK 1.0) threadSafeMap1 = new Hashtable(2); // Fully synchronized HashMap threadSafeMap2 = new HashMap(2); threadSafeMap2 = Collections.synchronizedMap(threadSafeMap2); // ConcurrentHashMap (since JDK 1.5) threadSafeMap3 = new ConcurrentHashMap(2); // ConcurrentHashMap /*** Assign map at your convenience ****/ assignedMapForTest = threadSafeMap3; long timeBefore = System.currentTimeMillis(); long timeAfter = 0; Float totalProcessingTime = null; ExecutorService executor = Executors.newFixedThreadPool(NB_THREADS); for (int j = 0; j < NB_THREADS; j++) { /** Assign the Map at your convenience **/ Runnable worker = new WorkerThread(assignedMapForTest); executor.execute(worker); } // This will make the executor accept no new threads // and finish all existing threads in the queue executor.shutdown(); // Wait until all threads are finish while (!executor.isTerminated()) { } timeAfter = System.currentTimeMillis(); totalProcessingTime = new Float( (float) (timeAfter - timeBefore) / (float) 1000); System.out.println("All threads completed in "+totalProcessingTime+" seconds"); } } } #### WorkerThread.java package org.ph.javaee.training4; import java.util.Map; /** * WorkerThread * * @author Pierre-Hugues Charbonneau * */ public class WorkerThread implements Runnable { private Map map = null; public WorkerThread(Map assignedMap) { this.map = assignedMap; } @Override public void run() { for (int i=0; i<500000; i++) { // Return 2 integers between 1-1000000 inclusive Integer newInteger1 = (int) Math.ceil(Math.random() * 1000000); Integer newInteger2 = (int) Math.ceil(Math.random() * 1000000); // 1. Attempt to retrieve a random Integer element Integer retrievedInteger = map.get(String.valueOf(newInteger1)); // 2. Attempt to insert a random Integer element map.put(String.valueOf(newInteger2), newInteger2); } } } Performance comparison between thread safe Map implementations The first goal is to compare the performance level of our program when using different thread safe Map implementations: Plain old Hashtable (since JDK 1.0) Fully synchronized HashMap (via Collections.synchronizedMap()) ConcurrentHashMap (since JDK 1.5) Find below the graphical results of the execution of the Java program for each iteration along with a sample of the program console output. # Output when using ConcurrentHashMap Infinite Looping HashMap Simulator Author: Pierre-Hugues Charbonneau http://javaeesupportpatterns.blogspot.com All threads completed in 0.984 seconds All threads completed in 0.908 seconds All threads completed in 0.706 seconds All threads completed in 1.068 seconds All threads completed in 0.621 seconds All threads completed in 0.594 seconds All threads completed in 0.569 seconds All threads completed in 0.599 seconds ……………… As you can see, the ConcurrentHashMap is the clear winner here, taking in average only half a second (after an initial ramp-up) for all 3 worker threads to concurrently read and insert data within a 500K looping statement against the assigned shared Map. Please note that no problem was found with the program execution e.g. no hang situation. The performance boost is definitely due to the improved ConcurrentHashMap performance such as the non-blocking get() operation. The 2 other Map implementations performance level was fairly similar with a small advantage for the synchronized HashMap. HashMap infinite looping problem replication The next objective is to replicate the HashMap infinite looping problem observed so often from Java EE production environments. In order to do that, you simply need to assign the non-thread safe HashMap implementation as per code snippet below: /*** Assign map at your convenience ****/ assignedMapForTest = nonThreadSafeMap; Running the program as is using the non-thread safe HashMap should lead to: No output other than the program header Significant CPU increase observed from the system At some point the Java program will hang and you will be forced to kill the Java process What happened? In order to understand this situation and confirm the problem, we will perform a CPU per Thread analysis from the Windows OS using Process Explorer and JVM Thread Dump. 1 - Run the program again then quickly capture the thread per CPU data from Process Explorer as per below. Under explore.exe you will need to right click over the javaw.exe and select properties. The threads tab will be displayed. We can see overall 4 threads using almost all the CPU of our system. 2 – Now you have to quickly capture a JVM Thread Dump using the JDK 1.7 jstack utility. For our example, we can see our 3 worker threads which seems busy/stuck performing get() and put() operations. ..\jdk1.7.0\bin>jstack 272 2012-08-29 14:07:26 Full thread dump Java HotSpot(TM) 64-Bit Server VM (21.0-b17 mixed mode): "pool-1-thread-3" prio=6 tid=0x0000000006a3c000 nid=0x18a0 runnable [0x0000000007ebe000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.put(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) "pool-1-thread-2" prio=6 tid=0x0000000006a3b800 nid=0x6d4 runnable [0x000000000805f000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.get(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:29) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) "pool-1-thread-1" prio=6 tid=0x0000000006a3a800 nid=0x2bc runnable [0x0000000007d9e000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.put(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) .............. 3 – CPU per thread correlation It is now time to convert the Process Explorer thread ID DECIMAL format to HEXA format as per below. The HEXA value allows us to map and identify each thread as per below: ## TID: 1748 (nid=0X6D4) Thread name: pool-1-thread-2 CPU @25.71% Task: Worker thread executing a HashMap.get() operation at java.util.HashMap.get(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:29) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) ## TID: 700 (nid=0X2BC) Thread name: pool-1-thread-1 CPU @23.55% Task: Worker thread executing a HashMap.put() operation at java.util.HashMap.put(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) ## TID: 6304 (nid=0X18A0) Thread name: pool-1-thread-3 CPU @12.02% Task: Worker thread executing a HashMap.put() operation at java.util.HashMap.put(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) ## TID: 5944 (nid=0X1738) Thread name: pool-1-thread-1 CPU @20.88% Task: Main Java program execution "main" prio=6 tid=0x0000000001e2b000 nid=0x1738 runnable [0x00000000029df000] java.lang.Thread.State: RUNNABLE at org.ph.javaee.training4.HashMapInfiniteLoopSimulator.main(HashMapInfiniteLoopSimulator.java:75) As you can see, the above correlation and analysis is quite revealing. Our main Java program is in a hang state because our 3 worker threads are using lot of CPU and not going anywhere. They may appear "stuck" performing HashMap get() & put() but in fact they are all involved in an infinite loop condition. This is exactly what we wanted to replicate. HashMap infinite looping deep dive Now let’s push the analysis one step further to better understand this looping condition. For this purpose, we added tracing code within the JDK 1.7 HashMap Java class itself in order to understand what is happening. Similar logging was added for the put() operation and also a trace indicating that the internal & automatic rehash/resize got triggered. The tracing added in get() and put() operations allows us to determine if the for() loop is dealing with circular dependency which would explain the infinite looping condition. #### HashMap.java get() operation public V get(Object key) { if (key == null) return getForNullKey(); int hash = hash(key.hashCode()); /*** P-H add-on- iteration counter ***/ int iterations = 1; for (Entry e = table[indexFor(hash, table.length)]; e != null; e = e.next) { /*** Circular dependency check ***/ Entry currentEntry = e; Entry nextEntry = e.next; Entry nextNextEntry = e.next != null?e.next.next:null; K currentKey = currentEntry.key; K nextNextKey = nextNextEntry != null?(nextNextEntry.key != null?nextNextEntry.key:null):null; System.out.println("HashMap.get() #Iterations : "+iterations++); if (currentKey != null && nextNextKey != null ) { if (currentKey == nextNextKey || currentKey.equals(nextNextKey)) System.out.println(" ** Circular Dependency detected! ["+currentEntry+"]["+nextEntry+"]"+"]["+nextNextEntry+"]"); } /***** END ***/ Object k; if (e.hash == hash && ((k = e.key) == key || key.equals(k))) return e.value; } return null; } HashMap.get() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.resize() in progress... HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 2 HashMap.resize() in progress... HashMap.resize() in progress... HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 2 HashMap.put() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.put() #Iterations : 1 ** Circular Dependency detected! [362565=362565][333326=333326]][362565=362565] HashMap.put() #Iterations : 2 ** Circular Dependency detected! [333326=333326][362565=362565]][333326=333326] HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.put() #Iterations : 1 ............................. HashMap.put() #Iterations : 56823 Again, the added logging was quite revealing. We can see that following a few internal HashMap.resize() the internal structure became affected, creating circular dependency conditions and triggering this infinite looping condition (#iterations increasing and increasing...) with no exit condition. It is also showing that the resize() / rehash operation is the most at risk of internal corruption, especially when using the default HashMap size of 16. This means that the initial size of the HashMap appears to be a big factor in the risk & problem replication. Finally, it is interesting to note that we were able to successfully run the test case with the non-thread safe HashMap by assigning an initial size setting at 1000000, preventing any resize at all. Find below the merged graph results: The HashMap was our top performer but only when preventing an internal resize. Again, this is definitely not a solution to the thread safe risk but just a way to demonstrate that the resize operation is the most at risk given the entire manipulation of the HashMap performed at that time. The ConcurrentHashMap, by far, is our overall winner by providing both fast performance and thread safety against that test case. JBoss AS7 Map data structures usage We will now conclude this article by looking at the different Map implementations within a modern Java EE container implementation such as JBoss AS 7.1.2. You can obtain the latest source code from the github master branch. Find below the report: Total JBoss AS7.1.2 Java files (August 28, 2012 snapshot): 7302 Total Java classes using java.util.Hashtable: 72 Total Java classes using java.util.HashMap: 512 Total Java classes using synchronized HashMap: 18 Total Java classes using ConcurrentHashMap: 46 Hashtable references were found mainly within the test suite components and from naming and JNDI related implementations. This low usage is not a surprise here. References to the java.util.HashMap were found from 512 Java classes. Again not a surprise given how common this implementation is since the last several years. However, it is important to mention that a good ratio was found either from local variables (not shared across threads), synchronized HashMap or manual synchronization safeguard so “technically” thread safe and not exposed to the above infinite looping condition (pending/hidden bugs is still a reality given the complexity with Java concurrency programming…this case study involving Oracle Service Bus 11g is a perfect example). A low usage of synchronized HashMap was found with only 18 Java classes from packages such as JMS, EJB3, RMI and clustering. Finally, find below a breakdown of the ConcurrentHashMap usage which was our main interest here. As you will see below, this Map implementation is used by critical JBoss components layers such as the Web container, EJB3 implementation etc. ## JBoss Single Sign On Used to manage internal SSO ID's involving concurrent Thread access Total: 1 ## JBoss Java EE & Web Container Not surprising here since lot of internal Map data structures are used to manage the http sessions objects, deployment registry, clustering & replication, statistics etc. with heavy concurrent Thread access. Total: 11 ## JBoss JNDI & Security Layer Used by highly concurrent structures such as internal JNDI security management. Total: 4 ## JBoss domain & managed server management, rollout plans... Total: 7 ## JBoss EJB3 Used by data structures such as File Timer persistence store, application Exception, Entity Bean cache, serialization, passivation... Total: 8 ## JBoss kernel, Thread Pools & protocol management Used by high concurrent Threads Map data structures involved in handling and dispatching/processing incoming requests such as HTTP. Total: 3 ## JBoss connectors such as JDBC/XA DataSources... Total: 2 ## Weld (reference implementation of JSR-299: Contexts and Dependency Injection for the JavaTM EE platform) Used in the context of ClassLoader and concurrent static Map data structures involving concurrent Threads access. Total: 3 ## JBoss Test Suite Used in some integration testing test cases such as an internal Data Store, ClassLoader testing etc. Total: 3 Final words I hope this article has helped you revisit this classic problem and understand one of the common problems and risks associated with a wrong usage of the non-thread safe HashMap implementation. My main recommendation to you is to be careful when using an HashMap in a concurrent threads context. Unless you are a Java concurrency expert, I recommend that you use ConcurrentHashMap instead which offers a very good balance between performance and thread safety. As usual, extra due diligence is always recommended such as performing cycles of load & performance testing. This will allow you to detect thread safety and / or performance problems before you promote the solution to your client production environment. Please provide any comments and share your experience with ConcurrentHashMap or HashMap implementations and troubleshooting.

many times i end up using source files from somebody else. and such files might have a lot of empty lines in it i want to get removed. the question is: how to get rid of empty or blank lines in the eclipse editor view? or even better: how to merge multiple empty lines into one? shortcut: ctrl+d deleting a line is easy: i place the cursor on a line and press ctrl+d (for line d elete). that works as well for multiple line selection. using the ‘ mother of all shortcuts ‘ reveals even more shortcut options: delete commands this approach is fine, but manual. there must be something better. search-and-replace: regular expression and there is: to automatically remove empty lines, the search-and-replace functionality helps. for this i press ctrl-f (for find) and configure it like this: regular expression to remove empty lines the magic is using a regular expression checkbox. it uses the regular expression ^\s*\n to find one or multiple empty lines and replaces it with ‘nothing’. that way i can clean up a full file and get rid of all empty lines. if i do not remember the syntax of regular expressions any more, then there is help too: the dialog has content assist available: content assist so that way pressing ctrl+space helps a lot: content assist help merging empty lines with this in mind, it is easy to do something more advanced : to merge multiple empty lines into a single one. again, a regular expression does the magic work: merging multiple empty lines into a single one with this, the regular expression ^\s*\n finds one or multiple empty lines (as before), and it replaces it with a single empty line: \r now the sources need much less screen real estate . happy removing:-)

Editor's Note: This post is a free chapter from the book from Manning Publications "In the OCA Java SE 7 programmer certification guide" by Mala Gupta In this article, I'll cover if and if-else constructs. We'll examine what happens when these constructs are used with and without curly braces {}. We'll also cover nested if and if-else constructs. The if construct and its flavors An if construct enables you to execute a set of statements in your code based on the result of a condition. This condition must always evaluate to a boolean or a Boolean value. You can specify a set of statements to execute when this condition evaluates to true or false. (In many Java books, you'll notice that the terms constructs and statements are used interchangeably.) Figure 1 shows multiple flavors of the if statement with their corresponding representations. if if-else if-else-if-else Figure 1 Multiple flavors of if statement: if, if-else, and if-else-if In figure 1, condition1 and condition2 refer to a variable or an expression that must evaluate to boolean or Boolean value. statement1, statement2, and statement3 refer to a single line of code or a code block. Because the Boolean wrapper class isn't covered in the OCA Java SE 7 Programmer I exam, we won't cover it here. We'll work with only the boolean data type. Exam Tip: then isn't a keyword in Java and isn't supposed to be used with the if statement. Let's look at the use of some flavors by first defining a set of variables: score, result, name, and file, as follows: int score = 100; String result = ""; String name = "Lion"; java.io.File file = new java.io.File("F"); Figure 2 shows the use of if, if-else, and if-else-if-else constructs and compares them by showing the code side by side. Figure 2 Multiple flavors of if statements implemented using code Let's quickly go through the code used in above if, if-else, and if-else-if-else statements. In the following example code, if condition name.equals("Lion") evaluates to true, a value of 200 is assigned to the variable score: if (name.equals("Lion")) #A score = 200; #A #A Example of if construct In the following example, if condition name.equals("Lion") evaluates to true, a value of 200 is assigned to the variable score. If this condition were to evaluate to false, a value of 300 is assigned to the variable score: if (name.equals("Lion")) #A score = 200; #A else #A score = 300; #A #A Example of if else construct In the following example, if score is equal to 100, the variable result is assigned a value of A. If score is equal to 50, the variable result is assigned a value of B. If the score is equal to 10, the variable result is assigned a value of C. If score doesn't match either of 100, 50, or 10, a value of F is assigned to the variable result. An if-else-if-else construct may use different conditions for all its if constructs: if (score == 100) #A result = "A"; else if (score == 50) #B result = "B"; else if (score == 10) #C result = "C"; else #D result = "F"; #A Condition 1 -> score == 100 #B Condition 2 -> score == 50 #C Condition 3 -> score == 10 #D If none of previous conditions evaluate to true, execute this else Figure 3 shows the previous code. Figure 3 The execution of the if-else-if-else code Figure 3 makes clear multiple points: The last else statement is part of the last if construct and not any of the if constructs before it. The if-else-if-else is an if-else construct, where its else part defines another if construct. A few other programming languages, such as VB and C#, use if-elsif and if-elseif (without space) constructs to define if-else-if constructs. If you've programmed with any of these languages, note the difference is with respect to Java. The following code is equal to the previous code: if (score == 100) result = "A"; else if (score == 50) result = "B"; else if (score == 10) result = "C"; else result="F"; Again, note that none of the previous if constructs use then to define the code to execute if a condition evaluates to true. As mentioned previously, unlike other programming languages, then isn't a keyword in Java and isn't used with the if construct. Exam Tip The if-else-if-else is an if-else construct, where else part defines another if construct. The boolean expression used as a condition for if construct can also include assignment operation. Missing else blocks What happens if you don't define the else statements for an if construct? It's acceptable to define one course of action for an if construct, as follows (omitting the else part): boolean testValue = false; if (testValue == true) System.out.println("value is true"); But you can't define the else part for an if construct, skipping the if code block. The following code won't compile: boolean testValue = false; if (testValue == true) else #A System.out.println("value is false"); #A This won't compile What follows is another interesting and bizarre piece of code: int score = 100; if((score=score+10) > 110); #1 #1 Missing then or else part Line #1 is a valid line of code, even if it doesn't define both the then and else part of the if statement. In this case, if condition evaluates and that's it. The if construct doesn't define any code that should execute based on the result of this condition. Note if(testValue==true) is same as using if(testValue). Similarly, if(testValue==false) is same as using if(!testValue). Implications of presence and absence of {} in if-else constructs You can execute a single statement or a block of statements, when if condition evaluates to true or false values. A block of statement is marked by enclosing single or multiple statements within a pair of curly braces ({}). Examine the following code: String name = "Lion"; int score = 100; if (name.equals("Lion")) score = 200; What happens if you want to execute another line of code, if value of variable name is equal to Lion? Is the following code correct? String name = "Lion"; int score = 100; if (name.equals("Lion")) score = 200; name = "Larry"; #1 #1 Set name to Larry Exam Tip In the exam, watch out for code similar to the above mentioned if construct that uses misleading indentation. In the absence of a code block definition (marked with a pair of {}), only the statement following the if construct forms its part. What happens to the same code if you define an else part for your if construct as follows: String name = "Lion"; int score = 100; if (name.equals("Lion")) score = 200; name = "Larry"; #A else score = 129; #A This statement isn't part of the if construct In this case, the previous code won't compile. The compiler will report that the else part is defined without an if statement. If this leaves you confused, examine the following code, which is indented in order to emphasize the fact that line name = "Larry" isn't part of the else construct: String name = "Lion"; int score = 100; if (name.equals ("Lion")) score = 200; name = "Larry"; #A else #B score = 129; #A Right indentation to emphasize that this statement isn't part of the if construct #B else seems to be defined without a preceding if construct If you want to execute multiple statements for if construct, you should define them within a block of code. You can do so by defining all this code within curly braces ({}). To follow is an example: String name = "Lion"; int score = 100; if (name.equals("Lion")) { #A score = 200; #B name = "Larry"; #B } #C else score = 129; #A Start of code block #B Statements to execute if (name.equals("Lion")) evaluates to true #C End of code block Similarly, you may define multiple lines of code for the else part (incorrectly) as follows: String name = "Lion"; if (name.equals("Lion")) System.out.println("Lion"); else System.out.println("Not a Lion"); System.out.println("Again, not a Lion"); #1 #1 Not part of else construct. Will execute irrespective of the value of variable name The output of the above code is as follows: Lion Again, not a Lion Though code on line #1 seems to execute only if value of variable name matches with value Lion, this is not the case. It is indented incorrectly to trick you into believing that it is a part of the else block. The above code is same as the following code (with correct indentation): String name = "Lion"; if (name.equals("Lion")) System.out.println("Lion"); else System.out.println("Not a Lion"); System.out.println("Again, not a Lion"); #1 #1 Not part of else construct. Will execute irrespective of the value of variable name If you wish to execute the last two statements in the previous code, only if the if condition evaluates to false, you can do so by using {}: String name = "Lion"; if (name.equals("Lion")) System.out.println("Lion"); else { System.out.println("Not a Lion"); System.out.println("Again, not a Lion"); #1 } #1 Now part of else construct. Will execute only when if condition evaluates to false You can define another statement, construct or loop, to execute for an if condition, without using {}, as follows: String name = "Lion"; if (name.equals("Lion")) #A for (int i = 0; i < 3; ++i) #B System.out.println(i); #C #A if condition #B for loop is a single construct that will execute if name.equals("Lion") evaluates to true #C This code is part of the for loop defined at previous line System.out.println(i) is part of the for loop, and not an unrelated statement that follows the for loop. So this code is correct and gives the following output: 0 1 2 Appropriate vs. inappropriate expressions passed as arguments to an if statement The result of an expression used in an if construct must evaluate to a boolean or Boolean value. Given the following definition of variables: int score = 100; boolean allow = false; String name = "Lion"; Up next are examples of some of the valid expressions that can be passed on to an if construct. Note that using == is not a good practice to compare two String objects for equality. The correct way to compare two String objects is to use equals method from the String class. However, comparing two String values using == is a valid expression that returns a boolean value and may also be used in the exam: (score == 100) #A (name == "Lio") #B (score <= 100 || allow) #C (allow) #D #A Evaluates to true #B Evaluates to false #C Evaluates to true #D Evaluates to false Now comes the tricky part of passing an assignment operation to an if construct. What do you think is the output of the following code? boolean allow = false; if (allow = true) #A System.out.println("value is true"); else System.out.println("value is false"); #A This is assignment, not comparison You may think that because the value of the boolean variable allow is set to false, the previous code output's value is false. Revisit the code and notice that assignment operation allow = true assigns the value true to the boolean variable allow. Further, its result is also a boolean value, which makes it eligible to be passed on as an argument to the if construct, Although the previous code has no syntactical errors, it's a logical error-an error in the program logic. The correct code to compare a boolean variable with a boolean literal value should be defined as follows: boolean allow = false; if (allow == true) #A System.out.println("value is true"); else System.out.println("value is false"); #A This is comparison Exam Tip Watch out for the code in the exam that uses the assignment operator (=) to compare a boolean value in the if condition. It won't compare the boolean value; it'll assign a value to it. The correct operator to compare a boolean value is equality operator (==). Nested if constructs A nested if construct is an if construct defined within another if construct. Theoretically, you don't have a limit on the levels of nested if and if-else constructs. Whenever you come across nested if and if-else constructs, you need to be careful about determining the else part of an if statement. If this statement doesn't make a lot of sense, take a look at the following code and determine its output: int score = 110; if (score > 200) #1 if (score <400) #2 if (score > 300) System.out.println(1); else System.out.println(2); else #3 System.out.println(3); #3 #1 if (score>200) #2 if (score<400) #3 To which if does this else belongs? Based on the way the code is indented, you may believe that else at #3 belongs to the if defined at #1. But it belongs to the if defined at #2. To follow is the code with the correct indentation: int score = 110; if (score > 200) if (score <400) if (score > 300) System.out.println(1); else System.out.println(2); else #A System.out.println(3); #A #A This else belongs to the if with condition (score<400) Next, you need to understand how to do the following: How to define an else for an outer if, other than the one that it'll be assigned to by default How to determine to which if does an else belong in nested if constructs Both of these tasks are simple. Let's start with the first one. How to define an else for an outer if other than the one that it'll be assigned to by default The key point is to use curly braces, as follows: int score = 110; if (score > 200) { #1 if (score <400) if (score > 300) System.out.println(1); else System.out.println(2); } #2 else #3 System.out.println(3); #3 #1 Start if construct for score > 200 #2 End if construct for score > 200 #3 else for score > 200 The curly braces at #1 and #2 mark the start and the end of the if condition (score>200) defined at #1. Hence, the else at #3 that follows #2 belongs to the if defined at #1. How to determine to which if an else belongs in nested if constructs If code uses curly braces to mark the start and end of the territory of an if or else construct, it can be simple, as mentioned in the previous section, "How to define an else for an outer if than the one that it'll be assigned to by default." When the if constructs don't use curly braces, don't get confused by the code indentation. Try to match all if with their corresponding else in the following poorly indented code: if (score > 200) if (score <400) if (score > 300) System.out.println(1); else System.out.println(2); else System.out.println(3); Start working inside out, with the innermost if-else statement, matching else with its nearest unmatched if statement. Figure 4 shows how to match the if-else pairs for the previous code, marked with 1, 2, and 3. Figure 4 Matching if-else pairs for poorly indented code Summary We covered the different flavors of the if construct. You saw what happens when these constructs are used with and without curly braces {}. We also covered nested if and if-else constructs. The humble if-else construct can virtually define any set of simple or complicated conditions. OCA Java SE 7 Programmer I Certification Guide By Mala Gupta In the OCA Java SE 7 programmer exam, you'll be asked you'll be asked how to define and control the flow in your code. In this article, based on chapter 4 of OCA Java SE 7 Programmer I Certification Guide, author Mala Gupta How show you to use if, if-else, if-else-if-else and nested if constructs and the difference when these if constructs are used with and without curly braces {}. Here are some other Manning titles you might be interested in: Unit Testing in Java Lasse Koskela Making Java Groovy Kenneth Kousen Play for Java Nicolas Leroux and Sietse de Kaper

Original Article: http://borislam.blogspot.hk/2012/09/accessing-weblogic-embedded-ldap-with.html Oracle Weblogic Application Server includes an embedded LDAP server which acts as the default security provider data store. There are few methods to access the embedded LDAP server. Oracle Weblogic provides an administration console for user to access it. We can create user, create group or edit detail through this administration console. We could also access the embedded LDAP programmatically. Using WLST (Weblogic Scripting tool) is a standard way to do it but you need have knowledge of Jython programming. Alternatively, if you want to access it through Java, you can programmatically call the Weblogic MBeans directly. However, this is not an easy task. You can see this link for more detail. In this article, I will show you an alternative way to access the Weblogic Embedded LDAP programmatically with Spring-LDAP. If you are acquainted with Spring, you may find this method quite easy and useful. In the following example, we do simple operation like search user, create user and add user to group through spring "ldaptemplate". Step 1: Create WlsLdapUser class which represent a LDAP user. import java.util.HashSet; import java.util.Set; import javax.naming.Name; public class WlsLdapUser { private String dnFull; private String id; private String password; private Set group; private Name dn; public String getId() { return this.id; } public void setId(String id) { this.id = id; } public String getPassword() { return this.password; } public void setPassword(String password) { this.password = password; } public Set getGroup() { return this.group; } public void setGroup(Set group) { this.group = group; } public void addToGroup(String group) { if (getGroup() == null) setGroup(new HashSet()); getGroup().add(group); } public void setDnFull(String dn) { this.dnFull = dn; } public String getDnFull() { return this.dnFull; } public void setDn(Name dn) { this.dn = dn; } public Name getDn() { return this.dn; } @Override public String toString() { return "WlsLdapUser [dnFull=" + dnFull + ", id=" + id + ", password="+ password + ", group=" + group + ", dn=" + dn + "]"; } Step 2: Create a base class "BaseLdapRepository" for accessing the Weblogic Embedded LDAP This class makes use of Spring's "ldapTemplate" for accessing the Weblogic Embedded LDAP. It encapsulates basic operations for accessing LDAP (e.g. delete LDAP attribute, replacing LDAP attribute, add LDAP attribute). The "ldapTemplate" is injected to the repository class by define in the XML in step 4. import javax.naming.Name; import javax.naming.NamingEnumeration; import javax.naming.NamingException; import javax.naming.directory.Attribute; import javax.naming.directory.Attributes; import javax.naming.directory.BasicAttribute; import javax.naming.directory.ModificationItem; import org.springframework.ldap.core.ContextMapper; import org.springframework.ldap.core.DirContextAdapter; import org.springframework.ldap.core.LdapTemplate; public class BaseLdapRepository { private LdapTemplate ldapTemplate ; public void setLdapTemplate(LdapTemplate ldapTemplate) { this.ldapTemplate = ldapTemplate; } public LdapTemplate getLdapTemplate() { return ldapTemplate; } protected void deleteAttr(Name dn, String attrName) { List attributes = new ArrayList(); Attribute attr = new BasicAttribute(attrName); ModificationItem item = new ModificationItem(3, attr); attributes.add(item); modify(dn, attributes); } protected void replaceAttr(Name dn, String attrName, String attrVal) { List attributes = new ArrayList(); Attribute attr = new BasicAttribute(attrName); attr.add(attrVal); ModificationItem item = new ModificationItem(2, attr); attributes.add(item); modify(dn, attributes); } protected void addAttr(Name dn, String attrName, String attrVal) { List attributes = new ArrayList(); Attribute attr = new BasicAttribute(attrName); attr.add(attrVal); ModificationItem item = new ModificationItem(1, attr); attributes.add(item); modify(dn, attributes); } private void modify(Name dn, List attributes) { getLdapTemplate().modifyAttributes(dn, (ModificationItem[])attributes.toArray(new ModificationItem[0])); attributes.clear(); } public Map getAttrsStartsWith(Name dn, String opAttr) { return (Map)getLdapTemplate().lookup(dn, new String[] { opAttr }, new ContextMapper() { public Object mapFromContext(Object ctx) { DirContextAdapter context = (DirContextAdapter)ctx; Attributes attrs = context.getAttributes(); NamingEnumeration ids = attrs.getIDs(); Map m = new HashMap(); try { while (ids.hasMore()) { String id = (String)ids.next(); System.out.println("id: " + id); m.put(id, context.getStringAttributes(id)); } } catch (NamingException e) { e.printStackTrace(); } return m; } }); } } Step 3: Create a repository class to access the Weblogic Embedded LDAP In order to implement the create(), find(), addUserGroups() and getUserGroups() methods in this LDAP repository class, we must know some Weblogic specific attributes of the Embedded LDAP. Each user in Embedded LDAP belongs the "top", "person", "origanizationalPerson", "inetOrgPerson", "wlsUser" objectclass. Therefore, when we implement the create() method, we must make sure the user entry belongs to these objectClasses. Also, each user in Embedded LDAP is under the "People" organization unit (OU). Thus, we need to search under "ou=people" when we find a specific user. When we add user to groups, we make use of the "uniquemember" attribute in the group entry. On the contrary, when want to know the groups in which a specific user belongs to, we make use if the "wlsMemberOf" attribute in user entry. import java.util.Map; import java.util.ArrayList; import javax.naming.Name; import org.apache.commons.lang.StringUtils; import org.springframework.ldap.core.DirContextAdapter; import org.springframework.ldap.core.DirContextOperations; import org.springframework.ldap.core.DistinguishedName; import org.springframework.ldap.core.support.AbstractContextMapper; import org.springframework.ldap.filter.EqualsFilter; public class WlsLdapUserRepository extends BaseLdapRepository { protected DistinguishedName buildDn(WlsLdapUser user) { return buildDn(user.getId()); } protected DistinguishedName buildDn(String uid) { DistinguishedName dn = new DistinguishedName(); dn.add("ou", "people"); dn.add("cn", uid); return dn; } protected DistinguishedName buildDnGroup(String cn) { DistinguishedName dn = new DistinguishedName(); dn.add("ou", "groups"); dn.add("cn", cn); return dn; } public void addToGroup(String dn, String gid) { DistinguishedName gdn = buildDnGroup(gid); super.addAttr(gdn, "uniquemember", dn); } public WlsLdapUser create(WlsLdapUser user) { Name dn = buildDn(user); DirContextAdapter context = new DirContextAdapter(dn); context.setAttributeValues("objectclass", new String[] { "top", "person", "inetOrgPerson", "organizationalPerson", "wlsuser" }); context.setAttributeValue("cn", user.getId()); context.setAttributeValue("sn", user.getId()); context.setAttributeValue("uid", user.getId()); context.setAttributeValue("userPassword", user.getPassword()); getLdapTemplate().bind(dn, context, null); user = find(user.getId()); return user; } public WlsLdapUser find(String uid) { WlsLdapUser user; try { user = (WlsLdapUser)getLdapTemplate().searchForObject("ou=people", new EqualsFilter("uid", uid).toString(), new WlsUserContextMapper()); } catch (Exception e) { e.printStackTrace(); return null; } return user; } public List getUserGroups(WlsLdapUser user) { Map groupMmap = null; String[] groupMapArray = null ; ArrayList results = new ArrayList(); groupMmap = this.getAttrsStartsWith(user.getDn(), "wlsMemberOf"); groupMapArray = groupMmap.get("wlsMemberOf"); if (groupMapArray.length >0) { String[] allGroups= StringUtils.split(groupMapArray[0], ","); for (String s:allGroups){ if (StringUtils.contains(s, "cn=")) { String aGroup = StringUtils.remove(s,"cn="); results.add(aGroup); } } } return results; } private static class WlsUserContextMapper extends AbstractContextMapper { protected Object doMapFromContext(DirContextOperations context) { WlsLdapUser user = new WlsLdapUser(); user.setId(context.getStringAttribute("uid")); user.setDnFull(context.getNameInNamespace()); user.setDn(context.getDn()); return user; } } } Step 4: Add your spring application context XML file The connection details to the Embedded LDAP is put inside this file. Step 5: Test your application with the following code @Named public class TestLdapService { @Inject private WlsLdapUserRepository wlsLdapUserRepository; public void createWlsUser(){ WlsLdapUser u = new WlsLdapUser(); u.setId("leonardmessi"); u.setPassword("welcome1"); u = this.wlsLdapUserRepository.create(u); this.wlsLdapUserRepository.addToGroup(u.getDnFull(), "testGroup"); this.wlsLdapUserRepository.addToGroup(u.getDnFull(), "Administrators"); System.out.println("create user :" + u.toString()); } public void findWlsUser(String userId){ WlsLdapUser u = this.wlsLdapUserRepository.find(userId); System.out.println("create user :" + u.toString()); } After adding the user, you can see the user created by the above program through the Weblogic administration console. Besides, you can also verify by directly login the newly created user "leonardmessi" with password "welcome1".

In Java, you can easily implement some business logic in Plain Old Java Object (POJO) classes, and then able to run them in a fancy server or framework without much hassle. There many server/frameworks, such as JBossAS, Spring or Camel etc, that would allow you to deploy POJO without even hardcoding to their API. Obviously you would get advance features if you willing to couple to their API specifics, but even if you do, you can keep these to minimal by encapsulating your own POJO and their API in a wrapper. By writing and designing your own application as simple POJO as possible, you will have the most flexible ways in choose a framework or server to deploy and run your application. One effective way to write your business logic in these environments is to use Service component. In this article I will share few things I learned in writing Services. What is a Service? The word Service is overly used today, and it could mean many things to different people. When I say Service, my definition is a software component that has minimal of life-cycles such as init, start, stop, and destroy. You may not need all these stages of life-cycles in every service you write, but you can simply ignore ones that don't apply. When writing large application that intended for long running such as a server component, definining these life-cycles and ensure they are excuted in proper order is crucial! I will be walking you through a Java demo project that I have prepared. It's very basic and it should run as stand-alone. The only dependency it has is the SLF4J logger. If you don't know how to use logger, then simply replace them with System.out.println. However I would strongly encourage you to learn how to use logger effectively during application development though. Also if you want to try out the Spring related demos, then obviously you would need their jars as well. Writing basic POJO service You can quickly define a contract of a Service with life-cycles as below in an interface. package servicedemo; public interface Service { void init(); void start(); void stop(); void destroy(); boolean isInited(); boolean isStarted(); } Developers are free to do what they want in their Service implementation, but you might want to give them an adapter class so that they don't have to re-write same basic logic on each Service. I would provide an abstract service like this: package servicedemo; import java.util.concurrent.atomic.*; import org.slf4j.*; public abstract class AbstractService implements Service { protected Logger logger = LoggerFactory.getLogger(getClass()); protected AtomicBoolean started = new AtomicBoolean(false); protected AtomicBoolean inited = new AtomicBoolean(false); public void init() { if (!inited.get()) { initService(); inited.set(true); logger.debug("{} initialized.", this); } } public void start() { // Init service if it has not done so. if (!inited.get()) { init(); } // Start service now. if (!started.get()) { startService(); started.set(true); logger.debug("{} started.", this); } } public void stop() { if (started.get()) { stopService(); started.set(false); logger.debug("{} stopped.", this); } } public void destroy() { // Stop service if it is still running. if (started.get()) { stop(); } // Destroy service now. if (inited.get()) { destroyService(); inited.set(false); logger.debug("{} destroyed.", this); } } public boolean isStarted() { return started.get(); } public boolean isInited() { return inited.get(); } @Override public String toString() { return getClass().getSimpleName() + "[id=" + System.identityHashCode(this) + "]"; } protected void initService() { } protected void startService() { } protected void stopService() { } protected void destroyService() { } } This abstract class provide the basic of most services needs. It has a logger and states to keep track of the life-cycles. It then delegate new sets of life-cycle methods so subclass can choose to override. Notice that the start() method is checking auto calling init() if it hasn't already done so. Same is done in destroy() method to the stop() method. This is important if we're to use it in a container that only have two stages life-cycles invocation. In this case, we can simply invoke start() and destroy() to match to our service's life-cycles. Some frameworks might go even further and create separate interfaces for each stage of the life-cycles, such as InitableService or StartableService etc. But I think that would be too much in a typical app. In most of the cases, you want something simple, so I like it just one interface. User may choose to ignore methods they don't want, or simply use an adaptor class. Before we end this section, I would throw in a silly Hello world service that can be used in our demo later. package servicedemo; public class HelloService extends AbstractService { public void initService() { logger.info(this + " inited."); } public void startService() { logger.info(this + " started."); } public void stopService() { logger.info(this + " stopped."); } public void destroyService() { logger.info(this + " destroyed."); } } Managing multiple POJO Services with a container Now we have the basic of Service definition defined, your development team may start writing business logic code! Before long, you will have a library of your own services to re-use. To be able group and control these services into an effetive way, we want also provide a container to manage them. The idea is that we typically want to control and manage multiple services with a container as a group in a higher level. Here is a simple implementation for you to get started: package servicedemo; import java.util.*; public class ServiceContainer extends AbstractService { private List services = new ArrayList(); public void setServices(List services) { this.services = services; } public void addService(Service service) { this.services.add(service); } public void initService() { logger.debug("Initializing " + this + " with " + services.size() + " services."); for (Service service : services) { logger.debug("Initializing " + service); service.init(); } logger.info(this + " inited."); } public void startService() { logger.debug("Starting " + this + " with " + services.size() + " services."); for (Service service : services) { logger.debug("Starting " + service); service.start(); } logger.info(this + " started."); } public void stopService() { int size = services.size(); logger.debug("Stopping " + this + " with " + size + " services in reverse order."); for (int i = size - 1; i >= 0; i--) { Service service = services.get(i); logger.debug("Stopping " + service); service.stop(); } logger.info(this + " stopped."); } public void destroyService() { int size = services.size(); logger.debug("Destroying " + this + " with " + size + " services in reverse order."); for (int i = size - 1; i >= 0; i--) { Service service = services.get(i); logger.debug("Destroying " + service); service.destroy(); } logger.info(this + " destroyed."); } } From above code, you will notice few important things: We extends the AbstractService, so a container is a service itself. We would invoke all service's life-cycles before moving to next. No services will start unless all others are inited. We should stop and destroy services in reverse order for most general use cases. The above container implementation is simple and run in synchronized fashion. This mean, you start container, then all services will start in order you added them. Stop should be same but in reverse order. I also hope you would able to see that there is plenty of room for you to improve this container as well. For example, you may add thread pool to control the execution of the services in asynchronized fashion. Running POJO Services Running services with a simple runner program. In the simplest form, we can run our POJO services on our own without any fancy server or frameworks. Java programs start its life from a static main method, so we surely can invoke init and start of our services in there. But we also need to address the stop and destroy life-cycles when user shuts down the program (usually by hitting CTRL+C.) For this, the Java has the java.lang.Runtime#addShutdownHook() facility. You can create a simple stand-alone server to bootstrap Service like this: package servicedemo; import org.slf4j.*; public class ServiceRunner { private static Logger logger = LoggerFactory.getLogger(ServiceRunner.class); public static void main(String[] args) { ServiceRunner main = new ServiceRunner(); main.run(args); } public void run(String[] args) { if (args.length < 1) throw new RuntimeException("Missing service class name as argument."); String serviceClassName = args[0]; try { logger.debug("Creating " + serviceClassName); Class serviceClass = Class.forName(serviceClassName); if (!Service.class.isAssignableFrom(serviceClass)) { throw new RuntimeException("Service class " + serviceClassName + " did not implements " + Service.class.getName()); } Object serviceObject = serviceClass.newInstance(); Service service = (Service)serviceObject; registerShutdownHook(service); logger.debug("Starting service " + service); service.init(); service.start(); logger.info(service + " started."); synchronized(this) { this.wait(); } } catch (Exception e) { throw new RuntimeException("Failed to create and run " + serviceClassName, e); } } private void registerShutdownHook(final Service service) { Runtime.getRuntime().addShutdownHook(new Thread() { public void run() { logger.debug("Stopping service " + service); service.stop(); service.destroy(); logger.info(service + " stopped."); } }); } } With abover runner, you should able to run it with this command: $ java demo.ServiceRunner servicedemo.HelloService Look carefully, and you'll see that you have many options to run multiple services with above runner. Let me highlight couple: Improve above runner directly and make all args for each new service class name, instead of just first element. Or write a MultiLoaderService that will load multiple services you want. You may control argument passing using System Properties. Can you think of other ways to improve this runner? Running services with Spring The Spring framework is an IoC container, and it's well known to be easy to work POJO, and Spring lets you wire your application together. This would be a perfect fit to use in our POJO services. However, with all the features Spring brings, it missed a easy to use, out of box main program to bootstrap spring config xml context files. But with what we built so far, this is actually an easy thing to do. Let's write one of our POJO Service to bootstrap a spring context file. package servicedemo; import org.springframework.context.ConfigurableApplicationContext; import org.springframework.context.support.FileSystemXmlApplicationContext; public class SpringService extends AbstractService { private ConfigurableApplicationContext springContext; public void startService() { String springConfig = System.getProperty("springContext", "spring.xml); springContext = new FileSystemXmlApplicationContext(springConfig); logger.info(this + " started."); } public void stopService() { springContext.close(); logger.info(this + " stopped."); } } With that simple SpringService you can run and load any spring xml file. For example try this: $ java -DspringContext=config/service-demo-spring.xml demo.ServiceRunner servicedemo.SpringService Inside the config/service-demo-spring.xml file, you can easily create our container that hosts one or more service in Spring beans. Notice that I only need to setup init-method and destroy-method once on the serviceContainer bean. You can then add one or more other service such as the helloService as much as you want. They will all be started, managed, and then shutdown when you close the Spring context. Note that Spring context container did not explicitly have the same life-cycles as our services. The Spring context will automatically instanciate all your dependency beans, and then invoke all beans who's init-method is set. All that is done inside the constructor of FileSystemXmlApplicationContext. No explicit init method is called from user. However at the end, during stop of the service, Spring provide the springContext#close() to clean things up. Again, they do not differentiate stop from destroy. Because of this, we must merge our init and start into Spring's init state, and then merge stop and destroy into Spring's close state. Recall our AbstractService#destory will auto invoke stop if it hasn't already done so. So this is trick that we need to understand in order to use Spring effectively. Running services with JEE app server In a corporate env, we usually do not have the freedom to run what we want as a stand-alone program. Instead they usually have some infrustructure and stricter standard technology stack in place already, such as using a JEE application server. In these situation, the most portable way to run POJO services is in a war web application. In a Servlet web application, you can write a class that implements javax.servlet.ServletContextListener and this will provide you the life-cycles hook via contextInitialized and contextDestroyed. In there, you can instanciate your ServiceContainer object and call start and destroy methods accordingly. Here is an example that you can explore: package servicedemo; import java.util.*; import javax.servlet.*; public class ServiceContainerListener implements ServletContextListener { private static Logger logger = LoggerFactory.getLogger(ServiceContainerListener.class); private ServiceContainer serviceContainer; public void contextInitialized(ServletContextEvent sce) { serviceContainer = new ServiceContainer(); List services = createServices(); serviceContainer.setServices(services); serviceContainer.start(); logger.info(serviceContainer + " started in web application."); } public void contextDestroyed(ServletContextEvent sce) { serviceContainer.destroy(); logger.info(serviceContainer + " destroyed in web application."); } private List createServices() { List result = new ArrayList(); // populate services here. return result; } } You may configure above in the WEB-INF/web.xml like this: servicedemo.ServiceContainerListener The demo provided a placeholder that you must add your services in code. But you can easily make that configurable using the web.xml for context parameters. If you were to use Spring inside a Servlet container, you may directly use their org.springframework.web.context.ContextLoaderListener class that does pretty much same as above, except they allow you to specify their xml configuration file using the contextConfigLocation context parameter. That's how a typical Spring MVC based application is configure. Once you have this setup, you can experiment our POJO service just as the Spring xml sample given above to test things out. You should see our service in action by your logger output. PS: Actually what we described here are simply related to Servlet web application, and not JEE specific. So you can use Tomcat server just fine as well. The importance of Service's life-cycles and it's real world usage All the information I presented here are not novelty, nor a killer design pattern. In fact they have been used in many popular open source projects. However, in my past experience at work, folks always manage to make these extremely complicated, and worse case is that they completely disregard the importance of life-cycles when writing services. It's true that not everything you going to write needs to be fitted into a service, but if you find the need, please do pay attention to them, and take good care that they do invoked properly. The last thing you want is to exit JVM without clean up in services that you allocated precious resources for. These would become more disastrous if you allow your application to be dynamically reloaded during deployment without exiting JVM, in which will lead to system resources leakage. The above Service practice has been put into use in the TimeMachine project. In fact, if you look at the timemachine.scheduler.service.SchedulerEngine, it would just be a container of many services running together. And that's how user can extend the scheduler functionalities as well, by writing a Service. You can load these services dynamically by a simple properties file.

these days i’m playing with backbone and using public api as a source. the web browser has one horrible feature: it don’t allow you to fetch any external resource to our host due to the cross-origin restriction. for example if we have a server at localhost we cannot perform one ajax request to another host different than localhost. nowadays there is a header to allow it: access-control-allow-origin . the problem is that the remote server must set up this header. for example i was playing with github’s api and github doesn’t have this header. if the server is my server, is pretty straightforward to put this header but obviously i’m not the sysadmin of github, so i cannot do it. what the solution? one possible solution is, for example, create a proxy server at localhost with php. with php we can use any remote api with curl (i wrote about it here and here for example). it’s not difficult, but i asked myself: can we create a dummy proxy server with php to handle any request to localhost and redirects to the real server, instead of create one proxy for each request?. let’s start. problably there is one open source solution (tell me if you know it) but i’m on holidays and i want to code a little bit (i now, it looks insane but that’s me ). the idea is: ... $proxy->register('github', 'https://api.github.com'); ... and when i type: http://localhost/github/users/gonzalo123 and create a proxy to : https://api.github.com/users/gonzalo123 the request method is also important. if we create a post request to localhost we want a post request to github too. this time we’re not going to reinvent the wheel, so we will use symfony componets so we will use composer to start our project: we create a conposer.json file with the dependencies: { "require": { "symfony/class-loader":"dev-master", "symfony/http-foundation":"dev-master" } } now php composer.phar install and we can start coding. the script will look like this: register('github', 'https://api.github.com'); $proxy->run(); foreach($proxy->getheaders() as $header) { header($header); } echo $proxy->getcontent(); as we can see we can register as many servers as we want. in this example we only register github. the application only has two classes: restproxy , who extracts the information from the request object and calls to the real server through curlwrapper . request = $request; $this->curl = $curl; } public function register($name, $url) { $this->map[$name] = $url; } public function run() { foreach ($this->map as $name => $mapurl) { return $this->dispatch($name, $mapurl); } } private function dispatch($name, $mapurl) { $url = $this->request->getpathinfo(); if (strpos($url, $name) == 1) { $url = $mapurl . str_replace("/{$name}", null, $url); $querystring = $this->request->getquerystring(); switch ($this->request->getmethod()) { case 'get': $this->content = $this->curl->doget($url, $querystring); break; case 'post': $this->content = $this->curl->dopost($url, $querystring); break; case 'delete': $this->content = $this->curl->dodelete($url, $querystring); break; case 'put': $this->content = $this->curl->doput($url, $querystring); break; } $this->headers = $this->curl->getheaders(); } } public function getheaders() { return $this->headers; } public function getcontent() { return $this->content; } } the restproxy receive two instances in the constructor via dependency injection (curlwrapper and request). this architecture helps a lot in the tests , because we can mock both instances. very helpfully when building restproxy. the restproxy is registerd within packaist so we can install it using composer installer: first install componser curl -s https://getcomposer.org/installer | php and create a new project: php composer.phar create-project gonzalo123/rest-proxy proxy if we are using php5.4 (if not, what are you waiting for?) we can run the build-in server cd proxy php -s localhost:8888 -t www/ now we only need to open a web browser and type: http://localhost:8888/github/users/gonzalo123 the library is very minimal (it’s enough for my experiment) and it does’t allow authorization. of course full code is available in github .

My last blog introduced Spring 3.1’s profiles and explained both the business case for using them and demonstrated their use with Spring XML configuration files. It seems, however, that a good number of developers prefer using Spring’s Java based application configuration, so Spring have designed a way of using profiles with their existing @Configuration annotation. I’m going to demonstrate profiles and the @Configuration annotation using the Person class from my previous blog. This is a simple bean class whose properties vary depending upon which profile is active. public class Person { private final String firstName; private final String lastName; private final int age; public Person(String firstName, String lastName, int age) { this.firstName = firstName; this.lastName = lastName; this.age = age; } public String getFirstName() { return firstName; } public String getLastName() { return lastName; } public int getAge() { return age; } } Remember that the Guys at Spring recommend that Spring profiles should only be used when you need to load different types or sets of classes and that for setting properties you should continue using the PropertyPlaceholderConfigurer. The reason I’m breaking the rules is that I want to try to write the simplest code possible to demonstrate profiles and Java configuration. At the heart of using Spring profiles with Java configuration is Spring’s new @Profile annotation. The @Profile annotation is used attach a profile name to an @Configuration annotation. It takes a single parameter that can be used in two ways. Firstly to attach a single profile to an @Configuration annotation: @Profile("test1") and secondly, to attach multiple profiles: @Profile({ "test1", "test2" }) Again, I’m going to define two profiles “test1” and “test2” and associate each with a configuration file. Firstly “test1”: @Configuration @Profile("test1") public class Test1ProfileConfig { @Bean public Person employee() { return new Person("John", "Smith", 55); } } ...and then “test2”: @Configuration @Profile("test2") public class Test2ProfileConfig { @Bean public Person employee() { return new Person("Fred", "Williams", 22); } } In the code above, you can see that I'm creating a Person bean with an effective id of employee (this is from the method name) that returns differing property values in each profile. Also note that the @Profile is marked as: @Target(value=TYPE) ...which means that is can only be placed next to the @Configuration annotation. Having attached an @Profile to an @Configuration, the next thing to do is to activate your selected @Profile. This uses exactly the same principles and techniques that I described in my last blog and again, to my mind, the most useful activation technique is to use the "spring.profiles.active" system property. @Test public void testProfileActiveUsingSystemProperties() { System.setProperty("spring.profiles.active", "test1"); ApplicationContext ctx = new ClassPathXmlApplicationContext("profiles-config.xml"); Person person = ctx.getBean("employee", Person.class); String firstName = person.getFirstName(); assertEquals("John", firstName); } Obviously, you wouldn’t want to hard code things as I’ve done above and best practice usually means keeping the system properties configuration separate from your application. This gives you the option of using either a simple command line argument such as: -Dspring.profiles.active="test1" ...or by adding # Setting a property value spring.profiles.active=test1 to Tomcat’s catalina.properties So, that’s all there is to it: you create your Spring profiles by annotating an @Configuration with an @Profile annotation and then switching on the profile you want to use by setting the spring.profiles.active system property to your profile’s name. As usual, the Guys at Spring don’t just confine you to using system properties to activate profiles, you can do things programatically. For example, the following code creates an AnnotationConfigApplicationContext and then uses an Environment object to activate the “test1” profile, before registering our @Configuration classes. @Test public void testAnnotationConfigApplicationContextThatWorks() { // Can register a list of config classes AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext(); ctx.getEnvironment().setActiveProfiles("test1"); ctx.register(Test1ProfileConfig.class, Test2ProfileConfig.class); ctx.refresh(); Person person = ctx.getBean("employee", Person.class); String firstName = person.getFirstName(); assertEquals("John", firstName); } This is all fine and good, but beware, you need to call AnnotationConfigApplicationContext’s methods in the right order. For example, if you register your @Configuration classes before you specify your profile, then you’ll get an IllegalStateException. @Test(expected = IllegalStateException.class) public void testAnnotationConfigApplicationContextThatFails() { // Can register a list of config classes AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext( Test1ProfileConfig.class, Test2ProfileConfig.class); ctx.getEnvironment().setActiveProfiles("test1"); ctx.refresh(); Person person = ctx.getBean("employee", Person.class); String firstName = person.getFirstName(); assertEquals("John", firstName); } Before closing today’s blog, the code below demonstrates the ability to attach multiple @Profiles to an @Configuration annotation. @Configuration @Profile({ "test1", "test2" }) public class MulitpleProfileConfig { @Bean public Person tourDeFranceWinner() { return new Person("Bradley", "Wiggins", 32); } } @Test public void testMulipleAssignedProfilesUsingSystemProperties() { System.setProperty("spring.profiles.active", "test1"); ApplicationContext ctx = new ClassPathXmlApplicationContext("profiles-config.xml"); Person person = ctx.getBean("tourDeFranceWinner", Person.class); String firstName = person.getFirstName(); assertEquals("Bradley", firstName); System.setProperty("spring.profiles.active", "test2"); ctx = new ClassPathXmlApplicationContext("profiles-config.xml"); person = ctx.getBean("tourDeFranceWinner", Person.class); firstName = person.getFirstName(); assertEquals("Bradley", firstName); } In the code above, 2012 Tour De France winner Bradley Wiggins appears in both the “test1” and “test2” profiles.

At the end of July 2012, Groovy 2.0 was released with support for static type checking and some performance improvements through the use of JDK7 invokedynamic and type inference as a result of type information now available through static typing. I was interested in seeing some estimate as to how significant the performance improvements in Groovy 2.0 have turned out and how Groovy 2.0 would now compare to Java in terms of performance. In case the performance gap had become minor, or at least acceptable, in the meantime, it would certainly be time to take a serious look at Groovy. Groovy has been ready for production for a long time. So, let's see whether it can compare with Java in terms of performance. The only performance measurement I could find on the Internet was this little benchmark measurment on jlabgroovy. The measurement only consists of calculating Fibonacci numbers with and without the @CompileStatic annotation. That's it; i.e., it's certainly not very meaningful in striving to get an overall impression. I was only interested in obtaining some rough estimate of how Groovy compares to Java as far as performance is concerned. Java performance measurement included Alas, no measurement was included in this little benchmark as to how much time Java takes to calculate Fibonacci numbers. So I "ported" the Groovy code to Java (here it is) and repeated the measurements. All measurements were done on an Intel Core2 Duo CPU E8400 3.00 GHz using JDK7u6 running on Windows 7 with Service Pack 1. I used Eclipse Juno with the Groovy plugin using the Groovy compiler version 2.0.0.xx-20120703-1400-e42-RELEASE. These are the figures I obtained without having a warm-up phase: Groovy 2.0 without @CompileStatic Groovy/Java performance factor Groovy 2.0 with @CompileStatic Groovy/Java performance factor Kotlin 0.1.2580 Java static ternary 4352ms 4.7 926ms 1.0 1005ms 924ms static if 4267ms 4.7 911ms 0.9 1828ms 917ms instance ternary 4577ms 2.7 1681ms 1.8 994ms 917ms instance if 4592ms 2.9 1604ms 1.7 1611ms 969ms I also did measurements with a warm-up phase of various length with the conclusion that there is no benefit for either language with or without the @CompileStatic. Since the Fibonacci algorithm is that recursive the warm-up phase seems to be "included" for any Fibonacci number that is not very small. We can see that the performance improvements due to static typing have made quite a difference. This little comparison does little justice, though. To me, the impression that static typing in Groovy has had in conjunction with type inference has led to significant performance improvements—and in the same way it has led to Groovy++ becoming very strong. With the @CompileStatic, the performance of Groovy is about 1-2 times slower than Java, and without Groovy, it's about 3-5 times slower. Unhappily, the measurements of "instance ternary" and "instance if" are the slowest. Unless we want to create masterpieces in programming with static functions, the measurements for "static ternary" and "static if" are not that relevant for most of the code with the ambition to be object-oriented (based on instances). Conclusion When Groovy was about 10-20 times slower than Java (see benchmark table almost at the end of this article) it is questionable whether the @CompileStatic was used or not. This means to me that Groovy is ready for applications where performance has to be somewhat comparable to Java. Earlier, Groovy (or Ruby, Closure, etc.) could only serve as a plus on your CV because of the performance impediment (at least here in Europe). New JVM kid on the block: Kotlin I added the figures for Kotlin as well (here is the code). Kotlin is a relatively new statically typed JVM-based Java-compatible programming language. Kotlin is more concise than Java by supporting variable type inferences, higher-order functions (closures), extension functions, mixins and first-class delegation, etc. Contrary to Groovy, it is more geared towards Scala, but also integrates well with Java. Kotlin is still under development and has yet to be officially released. So the figures have to be taken with caution as the guys at JetBrains are still working on the code optimization. Ideally, Kotlin should be as fast as Java. The measurements were done with the current "official" release 0.1.2580. And what about future performance improvements? At the time when JDK1.3 was the most recent JDK, I still earned my pay with Smalltalk development. At that time the performance of VisualWorks Smalltalk (now Cincom Smalltalk) and IBM VA for Smalltalk (now owned by Instantiations) was very good comparable to Java. And Smalltalk is a dynamically typed language, like pre-Goovy 2.0 and Ruby, where the compiler cannot make use of type inference to do optimizations. Because of this, it always appeared strange to me that Groovy, Ruby and other JVM-based dynamic languages had such a big performance penalty compared to Java when Smalltalk had not. From that point of view I think there's still room for Groovy performance improvements beyond @CompileStatic.