Imagine you love to cook; it's an intense hobby of yours. Over time, you've accumulated many fun spices, but your pantry is too small, so, you rent an off-site storage facility, and move the less frequently used spice racks there. Problem solved! Suddenly you decide to cook this great new recipe. You head to the pantry to retrieve your Saffron, but it's not there! It was moved out to the storage facility and must now be retrieved (this is a hard page fault). No problem -- your neighbor volunteers to go fetch it for you. Unfortunately, the facility is ~2,900 miles away, all the way across the US, so it takes your friend 6 days to retrieve it! This assumes you normally take 7 seconds to retrieve a spice from the pantry; that your data was in main memory (~100 nanoseconds access time), not in the CPU's caches (which'd be maybe 10 nanoseconds); that your swap file is on a fast (say, WD Raptor) spinning-magnets hard drive with 5 millisecond average access time; and that your neighbor drives non-stop at 60 mph to the facility and back. Even worse, your neighbor drives a motorcycle, and so he can only retrieve one spice rack at a time. So, after waiting 6 days for the Saffron to come back, when you next go to the pantry to get some Paprika, it's also "swapped out" and you must wait another 6 days! It's possible that first spice rack also happened to have the Paprika but it's also likely it did not; that depends on your spice locality. Also, with each trip, your neighbor must pick a spice rack to move out to the facility, so that the returned spice rack has a place to go (it is a "swap", after all), so the Paprika could have just been swapped out! Sadly, it might easily be many weeks until you succeed in cooking your dish. Maybe in the olden days, when memory itself was a core of little magnets, swapping cost wasn't so extreme, but these days, as memory access time has improved drastically while hard drive access time hasn't budged, the disparity is now unacceptable. Swapping has become a badly leaking abstraction. When a typical process (say, your e-mail reader) has to "swap back in" after not being used for a while, it can hit 100s of such page faults, before finishing redrawing its window. It's an awful experience, though it has the fun side effect of letting you see, in slow motion, just what precise steps your email reader goes through when redrawing its window. Swapping is especially disastrous with JVM processes. See, the JVM generally won't do a full GC cycle until it has run out of its allowed heap, so most of your heap is likely occupied by not-yet-collected garbage. Since these pages aren't being touched (because they are garbage and thus unreferenced), the OS happily swaps them out. When GC finally runs, you have a ridiculous swap storm, pulling in all these pages only to then discover that they are in fact filled with garbage and should be discarded; this can easily make your GC cycle take many minutes! It'd be better if the JVM could work more closely with the OS so that GC would somehow run on-demand whenever the OS wants to start swapping so that, at least, we never swap out garbage. Until then, make sure you don't set your JVM's heap size too large! Just use an SSD... These days, many machines ship with solid state disks, which are an astounding (though still costly) improvement over spinning magnets; once you've used an SSD you can never go back; it's just one of life's many one-way doors. You might be tempted to declare that this problem is solved, since SSDs are so blazingly fast, right? Indeed, they are orders of magnitudes faster than spinning magnets, but they are still 2-3 orders of magnitude slower than main memory or CPU cache. The typical SSD might have 50 microsends access time, which equates to ~58 total miles of driving at 60 mph. Certainly a huge improvement, but still unacceptable if you want to cook your dish on time! Just add RAM... Another common workaround is to put lots of RAM in your machine, but this can easily back-fire: operating systems will happily swap out memory pages in favor of caching IO pages, so if you have any processes accessing lots of bytes (say, mencoder encoding a 50 GB bluray movie, maybe a virus checker or backup program, or even Lucene searching against a large index or doing a large merge), the OS will swap your pages out. This then means that the more RAM you have, the more swapping you get, and the problem only gets worse! Fortunately, some OS's let you control this behavior: on Linux, you can tune swappiness down to 0 (most Linux distros default this to a highish number); Windows also has a checkbox, under My Computer -> Properties -> Advanced -> Performance Settings -> Advanced -> Memory Usage, that lets you favor Programs or System Cache, that's likely doing something similar. There are low-level IO flags that these programs are supposed to use so that the OS knows not to cache the pages they access, but sometimes the processes fail to use them or cannot use them (for example, they are not yet exposed to Java), and even if they do, sometimes the OS ignores them! When swapping is OK If your computer never runs any interactive processes, ie, a process where a human is blocked (waiting) on the other end for something to happen, and only runs batch processes which tend to be active at different times, then swapping can be an overall win since it allows that process which is active to make nearly-full use of the available RAM. Net/net, over time, this will give greater overall throughput for the batch processes on the machine. But, remember that the server running your web-site is an interactive process; if your server processes (web/app server, database, search server, etc.) are stuck swapping, your site has for all intents and purposes become unusable to your users. This is a fixable problem Most processes have known data structures that consume substantial RAM, and in many cases these processes could easily discard and later regenerate their data structures in much less time than even a single page fault. Caches can simply be pruned or discarded since they will self-regenerate over time. These data structures should never be swapped out, since regeneration is far cheaper. Somehow the OS should ask each RAM-intensive and least-recently-accessed process to discard its data structures to free up RAM, instead of swapping out the pages occupied by the data structure. Of course, this would require a tighter interaction between the OS and processes than exists today; Java's SoftReference is close, except this only works within a single JVM, and does not interact with the OS. What can you do? Until this problem is solved for real, the simplest workaround is to disable swapping entirely, and stuff as much RAM as you can into the machine. RAM is cheap, memory modules are dense, and modern motherboards accept many modules. This is what I do. Of course, with this approach, when you run out of RAM stuff will start failing. If the software is well written, it'll fail gracefully: your browser will tell you it cannot open a new window or visit a new page. If it's poorly written it will simply crash, thus quickly freeing up RAM and hopefully not losing any data or corrupting any files in the process. Linux takes the simple draconian approach of picking a memory hogging process and SIGKILL'ing it. If you don't want to disable swapping you should at least tell the OS not to swap pages out for IO caching. Just say no to swapping!

Lately, I have been playing around with PHP configuration. And I realized that the runtime configurations do not change using ini_set function, if you are running PHP in the safe mode. A simple check in PHP can tell you this. Notice the code block below and which allows to check this. Next, time when you are going to change any configuration at runtime, I am sure you want to check this, before you could see that change working. Hope that helps. Stay digified !!

Session fixation protection is a new feature that was introduced as part of the Apache Tomcat 7 release process, and has been backported and turned on by default in all versions from version 6.0.21 on. Session fixation happens when an attacker sends a link to the victim with a session ID included. The minute the victim authenticates the session in the link by logging in, the attacker is able to just click the session link and have full access to the intended victims account. According to Mark Thomas, release manager for Tomcat 7 and member of the Apache Security Committee, in an article on TomcatExpert.com today": Essentially, when a user authenticates their session, Tomcat will change the session ID. It does not destroy the previous session, rather it renames it so it is no longer found by that ID. So in our example above, Bob would try and log on with that session, and he would not be able to find it. Turning Off Session Fixation Protection Since it is turned on by default, the configuration is implicit inside of Tomcat. To turn it off, you will need to explicitly add the configuration and turn it off. First you will need to find the details of the specific type of authentication valve you are using in the Apache Tomcat Valve Configuration Documentation. Then you would need to navigate to the context.xml file for your application (not $CATALINA_BASE/conf/context.xml - that is the global context.xml that provides defaults for all web applications). Add the valve with the appropriate class for your authentication type (e.g., for basic authentication add a valve using class="org.apache.catalina.authenticator.BasicAuthenticator". Finally, set the parameter changeSessionIdOnAuthentication="false". When to Turn Off Session Fixation Protection Generally, this is not recommended. The Apache Security Team deemed this feature a basic enough protection that it should be afforded to all users implicitly. However, in some cases application specific behavior could break and the feature would need to be turned off until the functionality could be restored using the protection. For more information on the feature and when to use it, see the full article from Mark Thomas on TomcatExpert.com.

A code kata is an exercise where you focus on your technique instead of on the final product of your mind and fingers. But a kata can also be used as a constant parameter, while other variables change, like in scientific experiments. For example, when learning a new programming language or framework, you can execute an old kata in order to explore it. I decided to perform a small and famous Kata that we used also during interviews to separate programmers from not programmers: the FizzBuzz kata. My goal was to learn how to setup a platform for Test-Driven Development in JavaScript, following the advice of the Test-Driven JavaScript Development book. The parameters that change from my habits are the tools for running tests and the programming language, but my IDE (Unix&Vim) remained fixed along with the Kata: Write a function that returns its numerical argument. But for multiples of three return Fizz instead of the number and for the multiples of five return Buzz. For numbers which are multiples of both three and five return FizzBuzz. Additional requirement: when passed a multiple of 7, return Bang; when passed a multiple of 5 and 7, return BuzzBang; and so on for all the combinations. As my tools for running the tests, I used JsTestDriver and Firefox, as suggested by the book Test-Driven JavaScript Development which I'm currently reading. JsTestDriver JsTestDriver will make you feel the joy of a green bar again. Download its jar, put it somewhere and add an alias in your .bashrc: export JSTESTDRIVER_HOME=~/bin alias jstestdriver="java -jar $JSTESTDRIVER_HOME/JsTestDriver-1.3.2.jar" Start the server: jstestdriver --port 4224 Point an open browser (I used Firefox) to localhost:4224. The browser will ping it via Ajax requests undefinitely to gather tests to run. Now we can use the command line to run tests, like you'll do with PHPUnit if you are a PHPer: jstestdriver --tests all The Kata I started with a simple function, fizzbuzz(), and a single test case. I never wrote a test with JsTestDriver before so I needed to gain some confidence and be sure the configuration file was correct. server: http://localhost:4224 load: - src/*.js - test/*.js In JsTestDriver, a Test case is created by passing to TestCase (global function provided by JsTestDriver) a map containing anonymous functions. TestCase("FizzBuzzTest", { "test should return Fizz when passed 3" : function () { assertEquals("Fizz", fizzbuzz(3)); } }); The functions whose names start with test will be executed; there are some reserved keywords like setUp which are used as hooks for fixture creation. Running the test with the alias command is really simple: jstestdriver --tests all I made the first test pass with fizzbuzz.js, a file containing a first version of the function (with a fake implementation): function fizzbuzz() { return 'Fizz'; } The result? A green bar (metaphorically green; all tests pass.) . Total 1 tests (Passed: 1; Fails: 0; Errors: 0) (0,00 ms) Firefox 4.0 Linux: Run 1 tests (Passed: 1; Fails: 0; Errors 0) (0,00 ms) You can capture more than one browser if you want to run test simultaneously in all of them, but it will probably slow down the TDD basic cycle. You can leave cross-browser testing for later. Going on After this first test, I went on adding new ones and making them pass, until I even converted the function to an object, for the sake of easy configuration (a function returning a function would be the same). Since I also needed to create the object in just one place, I started using setUp for the fixture creation: TestCase("FizzBuzzTest", { setUp : function () { this.fizzbuzz = new FizzBuzz({ 3 : 'Fizz', 5 : 'Buzz', 7 : 'Bang' }); }, "test should return the number when passed 1 or 2" : function () { assertEquals(1, this.fizzbuzz.accept(1)); assertEquals(2, this.fizzbuzz.accept(2)); }, "test should return Fizz when passed 3 or a multiple" : function () { assertEquals("Fizz", this.fizzbuzz.accept(3)); assertEquals("Fizz", this.fizzbuzz.accept(6)); }, "test should return Buzz when passed 5 or a multiple" : function () { assertEquals("Buzz", this.fizzbuzz.accept(5)); assertEquals("Buzz", this.fizzbuzz.accept(10)); }, "test should return FizzBuzz when passed a multiple of both 3 and 5" : function () { assertEquals("FizzBuzz", this.fizzbuzz.accept(15)); assertEquals("FizzBuzz", this.fizzbuzz.accept(30)); }, "test should return Bang when passed a multiple of 7" : function () { assertEquals("Bang", this.fizzbuzz.accept(7)); assertEquals("Bang", this.fizzbuzz.accept(14)); }, "test should return FizzBuzzBang when it is the case" : function () { assertEquals("FizzBuzzBang", this.fizzbuzz.accept(3*5*7)); } }); You can use this to share fixtures between the setUp and the different test methods: the test does not look different from JUnit and PHPUnit ones. Like in all xUnit testing frameworks, the setUp is executed on a brand new object for each test, to preserve isolation. I like a bit the way in which in JavaScript you can tear and put together objects: after all, it's called object-oriented programming, not class-oriented programming. I decided to use a small function constructor as you may infer from the test: function FizzBuzz(correspondences) { this.correspondences = correspondences; this.accept = function (number) { var result = ''; for (var divisor in this.correspondences) { if (number % divisor == 0) { result = result + this.correspondences[divisor]; } } if (result) { return result; } else { return number; } } } All the code is on Github, to see the intermediate steps of the Kata if you need them. You can also use the repository to try out your installation of JsTestDriver: a git pull followed by running the tests will confirm that it's working. Sometimes we don't test code in alien environments like JavaScript console or database queries because we don't know how; but a Kata which takes just two Pomodoros can solve the issue and let you enjoy a green bar even when working with a browser's interpreter.

• @Default - Whenever a bean or injection point does not explicitly declare a qualifier, the container assumes the qualifier @Default. • @Any – When you need to declare an injection point without specifying a qualifier, is useful to know that all beans have the @Any qualifier. This qualifier “belongs” to all beans and injection points (not applicable when @New is present) and when is explicitly specified you suppress the @Default qualifier. This is useful if you want to iterate over all beans with a certain bean type. The example presented in this post will prove how @Any qualifier works in a useful example. You will try to iterate over all beans with a certain bean type. For start, you define a new Java type, like below – a simple interface representing a Wilson tennis racquet type: package com.racquets; public interface WilsonType { public String getWilsonRacquetType(); } Now, three Wilson racquets will implement this interface – notice that no qualifier was specified: package com.racquets; public class Wilson339Racquet implements WilsonType{ @Override public String getWilsonRacquetType() { return ("Wilson BLX Six.One Tour 90 (339 gr)"); } } package com.racquets; public class Wilson319Racquet implements WilsonType { @Override public String getWilsonRacquetType() { return ("Wilson BLX Six.One Tour 90 (319 gr)"); } } package com.racquets; public class Wilson96Racquet implements WilsonType { @Override public String getWilsonRacquetType() { return ("Wilson BLX Pro Tour 96"); } } Next, you can use the @Any qualifier at injection point to iterate over all beans of type WilsonRacquet. An instance of each implementation is injected and the result of getWilsonRacquetType method is stored in an ArrayList: package com.racquets; import java.util.ArrayList; import javax.enterprise.context.RequestScoped; import javax.enterprise.inject.Any; import javax.enterprise.inject.Instance; import javax.enterprise.inject.Produces; import javax.inject.Inject; import javax.inject.Named; @RequestScoped public class WilsonRacquetBean { private @Named @Produces ArrayList wilsons = new ArrayList(); @Inject void initWilsonRacquets(@Any Instance racquets) { for (WilsonType racquet : racquets) { wilsons.add(racquet.getWilsonRacquetType()); } } } From a JSF page, you can easily iterate over the ArrayList (notice here that the wilsons collection was annotated with @Named (allows JSF to have access to this field) and @Produces (as a simple explanation, a producer field suppress the getter method)): The built-in qualifiers @Default and @Any The output is in figure below: From http://e-blog-java.blogspot.com/2011/04/built-in-qualifiers-default-and-any.html

JmsTemplate is easy for simple message sending. What if we want to add headers, intercept or transform the message? Then we have to write more code. So, how do we solve this common task with more configurability in lieu of more code? First, lets review JMS in Spring. Spring JMS Options JmsTemplate – either to send and receive messages inline Use send()/convertAndSend() methods to send messages Use receive()/receiveAndConvert() methods to receive messages. BEWARE: these are blocking methods! If there is no message on the Destination, it will wait until a message is received or times out. MessageListenerContainer – Async JMS message receipt by polling JMS Destinations and directing messages to service methods or MDBs Both JmsTemplate and MessageListenerContainer have been successfully implemented in Spring applications, if we have to do something a little different, we introduce new code. What could possibly go wrong? Future Extensibility? On many projects new use-cases arise, such as: Route messages to different destinations, based on header values or contents? Log the message contents? Add header values? Buffer the messages? Improved response and error handling? Make configuration changes without having to recompile? and more… Now we have to refactor code and introduce new code and test cases, run it through QA, etc. etc. A More Configurable Solution! It is time to graduate Spring JmsTemplate and play with the big kids. We can easily do this with a Spring Integration flow. How it is done with Spring Integration Here we have a diagram illustrating the 3 simple components to Spring Integration replacing the JmsTemplate send. Create a Gateway interface – an interface defining method(s) that accept the type of data you wish to send and any optional header values. Define a Channel – the pipe connecting our endpoints Define an Outbound JMS Adapter – sends the message to your JMS provider (ActiveMQ, RabbitMQ, etc.) Simply inject this into our service classes and invoke the methods. Immediate Gains Add header & header values via the methods defined in the interface Simple invokation of Gateway methods from our service classes Multiple Gateway methods Configure method level or class level destinations Future Gains Change the JMS Adapter (one-way) to a JMS Gateway (two-way) to processes responses from JMS We can change the channel to a queue (buffered) channel We can wire in a transformer for message transformation We can wire in additional destinations, and wire in a “header (key), header value, or content based” router and add another adapter We can wire in other inbound adapters receiving data from another source, such as SMTP, FTP, File, etc. Wiretap the channel to send a copy of the message elsewhere Change the channel to a logging adapter channel which would provide us with logging of the messages coming through Add the “message-history” option to our SI configuration to track the message along its route and more… Optimal JMS Send Solution The Spring Integration Gateway Interface Gateway provides a one or two way communication with Spring Integration. If the method returns void, it is inherently one-way. The interface MyJmsGateway, has one Gateway method declared sendMyMessage(). When this method is invoked by your service class, the first argument will go into a message header field named “myHeaderKey”, the second argument goes into the payload. package com.gordondickens.sijms; import org.springframework.integration.annotation.Gateway;import org.springframework.integration.annotation.Header; public interface MyJmsGateway { @Gateway public void sendMyMessage(@Header("myHeaderKey") String s, Object o);} Spring Integration Configuration Because the interface is proxied at runtime, we need to configure in the Gateway via XML. Sending the Message package com.gordondickens.sijms; import org.junit.Test;import org.junit.runner.RunWith;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.test.context.ContextConfiguration;import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @ContextConfiguration("classpath:/com/gordondickens/sijms/JmsSenderTests-context.xml")@RunWith(SpringJUnit4ClassRunner.class)public class JmsSenderTests { @Autowired MyJmsGateway myJmsGateway; @Test public void testJmsSend() { myJmsGateway.sendMyMessage("myHeaderValue", "MY PayLoad"); } Summary Simple implementation Invoke a method to send a message to JMS – Very SOA eh? Flexible configuration Reconfigure & restart WITHOUT recompiling – SWEET!

In my previous post, I showed how to convert a Spring web controller class to use the @Controller annotation. In this post, I aim to show how forms in a Spring MVC application can also be converted to using annotations. Forms in Spring are typically modelled by extending the org.springframework.web.servlet.mvc.SimpleFormController class, but using Spring annotations, they can also be simplified and defined by the @Controller annotation.Without annotations, a SimpleFormController would be defined as below as in both a Java class and as a bean in XML. import org.springframework.web.servlet.mvc.SimpleFormController public class PriceIncreaseFormController extends SimpleFormController { public ModelAndView onSubmit(Object command) { // Submit the form } protected Object formBackingObject(HttpServletRequest request) throws ServletException { // Initialize the form } } To convert the class to use annotations, we need to add an @Controller and @RequestMapping annotation. We can then define a simple POJO controller that does not need to extend Spring's classes. @Controller @RequestMapping("/priceincrease.htm") public class PriceIncreaseFormController { @Autowired private PriceIncreaseValidator priceIncreaseValidator; @RequestMapping(method=RequestMethod.POST) public String onSubmit(@ModelAttribute("priceIncrease")PriceIncrease priceIncrease, BindingResult result) { int increase = priceIncrease.getPercentage(); priceIncreaseValidator.validate(increase, result); if (result.hasErrors()) return "priceIncrease"; // Validator has succeeded. // Perform necessary actions and return to success page. return "redirect:home.htm"; } @RequestMapping(method=RequestMethod.GET) public String initializeForm(ModelMap model) { // Perform and Model / Form initialization Map priority = new LinkedHashMap(); priority.put(1, "Low"); priority.put(2, "Normal"); priority.put(3, "High"); model.addAttribute("priorityList", priority); return "priceincrease"; } // Other getters and setters an needed. } After defining the class like this, there is no need for the class to be defined within the Spring Context XML file. The two methods outlined above in this simple class show how the initializeForm() method is called when an HTTP GET request is made to the form and how the form is submitted in the onSubmit() method via a HTTP POST request. The method called on a GET request is used to initialize the form, whereas the method called on a POST request is called when the form is submitted. The final thing to notice from this class is that the validation has to be explicitly called within the onSubmit() method. In this example, the PriceValidator is injected into the class via the @Autowired annotation. From http://www.davidsalter.co.uk/1/post/2011/04/converting-a-spring-simpleformcontroller-into-an-controller.html

Last week my colleague T.C. and I had to work out how to HTML encode the values entered by the user when redisplaying those onto the page to prevent a cross site scripting attack on the website. I wrote a blog post a couple of years ago describing how to do this in ASP.NET MVC and the general idea is that we need to have a custom renderer which HTML encodes any strings that pass through it. In our case this means that we needed to write a custom renderer for String Template and hook that into Spring MVC. We already had a view class StringTemplateView so we needed to add to that class and add our custom renderer. The viewResolver was defined like so: @Bean public ViewResolver viewResolver() { InternalResourceViewResolver viewResolver = new InternalResourceViewResolver(); viewResolver.setPrefix("/WEB-INF/templates/"); viewResolver.setViewClass(StringTemplateView.class); viewResolver.setSuffix(".st"); return viewResolver; } And after some guidance from Jim we changed StringTemplateView to look like this: public class StringTemplateView extends InternalResourceView { @Override protected void renderMergedOutputModel(Map model, HttpServletRequest request, HttpServletResponse response) throws Exception { String templateRootDir = format("%s/WEB-INF/templates", getServletContext().getRealPath("/")); StringTemplateGroup group = new StringTemplateGroup("view", templateRootDir); StringTemplate template = group.getInstanceOf(getBeanName()); AttributeRenderer htmlEncodedRenderer = new HtmlEncodedRenderer(); template.registerRenderer(String.class, htmlEncodedRenderer); ... } private class HtmlEncodedRenderer implements AttributeRenderer { @Override public String toString(Object o) { return HtmlUtils.htmlEscape(o.toString()); } @Override public String toString(Object o, String formatName) { return HtmlUtils.htmlEscape(o.toString()); } } } At the moment we want to HTML encode everything that we render through StringTemplate but if that changes then we could make use of the formatName parameter which we’re currently ignoring. In retrospect this looks pretty simple to do but my Googling skills were pretty much failing me at the time so I thought it’d be good to document. From http://www.markhneedham.com/blog/2011/04/09/html-encodingescaping-with-stringtemplate-and-spring-mvc

One of the most common reasons why software projects fail, or suffer unbearable delays, is the misunderstandings between the analysts who define the business rules of the domain for which the software is going to be written and the developers who have to code these rules. The latter write those rules in a language that is completely obscure for the first ones. In this way the business analysts don't have a chance to read, understand and validate what the programmers developed and then they can only empirically test the final software behavior, hardly covering all the possible corner cases and often recognizing mistakes only when it is too late. What Hammurabi is Hammurabi is a rule engine written in Scala that tries to leverage the features of this language making it particularly suitable to implement extremely readable internal Domain Specific Languages. Indeed, what actually makes Hammurabi different from all other rule engines is that it is possible to write and compile its rules directly in the host language. Anyway the Hammurabi's rules also have the important property of being readable even by non technical person. As usual a practical example worth more than a thousand words. The golfers problem This logical puzzle has been taken from the first chapter of the Jess in Action book written by Ernest Friedman-Hill and published by Manning. It is described there as it follows: A foursome of golfers is standing at a tee, in a line from left to right. Each golfer wears different colored pants; one is wearing red pants. The golfer to Fred’s immediate right is wearing blue pants. Joe is second in line. Bob is wearing plaid pants. Tom isn’t in position one or four, and he isn’t wearing the hideous orange pants. In what order will the four golfers tee off, and what color are each golfer’s pants?” The Jess solution Jess is written in Java and is one of the most popular rule engine on the market. The solution to the golfers problems presented in the book mentioned above is the following: first it is necessary to define the data structures representing the problem (deftemplate pants-color (slot of) (slot is)) (deftemplate position (slot of) (slot is)) A deftemplate is a bit like a class declaration in Java and in this case is used to write a first rule that in turns creates the facts representing each of the possible combinations of golfers, pants-color and positions: (defrule generate-possibilities => (foreach ?name (create$ Fred Joe Bob Tom) (foreach ?color (create$ red blue plaid orange) (assert (pants-color (of ?name)(is ?color))) ) (foreach ?position (create$ 1 2 3 4) (assert (position (of ?name)(is ?position))) ) ) ) After that it is possible to translate the sentences of the problem in the corresponding Jess rule: (defrule find-solution ;; There is a golfer named Fred, whose position is ?p1 ;; and pants color is ?c1 (position (of Fred) (is ?p1)) (pants-color (of Fred) (is ?c1)) ;; The golfer to immediate right of Fred ;; is wearing blue pants. (position (of ?n&~Fred)(is ?p&:(eq ?p (+ ?p1 1)))) (pants-color (of ?n&~Fred)(is blue&~?c1)) ;; Joe is in position #2 (position (of Joe) (is ?p2&2&~?p1)) (pants-color (of Joe) (is ?c2&~?c1)) ;; Bob is wearing the plaid pants (position (of Bob)(is ?p3&~?p1&~?p&~?p2)) (pants-color (of Bob&~?n)(is plaid&?c3&~?c1&~?c2)) ;; Tom is not in position 1 or 4 ;; and is not wearing orange (position (of Tom&~?n)(is ?p4&~1&~4&~?p1&~?p2&~?p3)) (pants-color (of Tom)(is ?c4&~orange&~blue&~?c1&~?c2&~?c3)) => (printout t Fred " " ?p1 " " ?c1 crlf) (printout t Joe " " ?p2 " " ?c2 crlf) (printout t Bob " " ?p3 " " ?c3 crlf) (printout t Tom " " ?p4 " " ?c4 crlf) ) where the rows starting with ;; are just comments. In this way if you enter the code for the problem into Jess and then run it, you get the answer directly: Fred 1 orange Joe 2 blue Bob 4 plaid Tom 3 red Note that the facts that the golfers are in different positions and wear pants of different colors is not expressed in an explicit rule but need to be spread and repeated in many statements. This solution is clearly difficult to be maintained and doesn't scale well as underlined by the last condition statement stating that the position ?p4 of Tom is ?p4&~1&~4&~?p1&~?p2&~?p3 where ~ means not in the Jess language. In other words it says that the Tom's position is not only different from the position 1 and 4 but it is also different from the positions of all the other golfers (named one by one) formerly defined. Actually the needs to describe a golfer's position also as the negation of the positions of all the other golfers implies something even worse: it is not possible to translate each sentence of the problem in a different rule, but they have to be combined together in a single big rule. After this huge if part, its then section (the one after the => symbol) prints out a table containing the set of variables ?p1…?p4 and ?c1…?c4 that solves the problem. The Hammurabi solution As done while presenting the Jess solution, also with Hammurabi the first thing to do is to define the domain of the problem. In order to do that, since the Hammurabi rules are valid Scala statements, it is sufficient to create a plain Scala Person class having as attributes the name, the position and the color of the pants of the golfer that it represents: class Person(n: String) { val name = n var pos: Int = _ var color: String = _ } Then we can model the fact that all the golfers must have different position and pants color by putting them in 2 different Set: var availablePos = (1 to 4).toSet var availableColors = Set("blue", "plaid", "red", "orange") and write two small methods that pull off them from the corresponding set once they have been assigned to a specific golfer: val assign = new { def color(color: String) = new { def to(person: Person) = { person.color = color availableColors = availableColors - color } } def position(pos: Int) = new { def to(person: Person) = { person.pos = pos availablePos = availablePos - pos } } } Those methods are written in a quite weird way just to make even more readable the DSL that will be used to define the rules and to stress the idea that it is possible to write the rules in a valid Scala that could be easily understood by a non-technical person. Of course it is easy to go even further by encapsulating other concepts in some convenient methods as it has been done above. Now everything is ready to write the set of rules describing the golfers problem: val ruleSet = Set( rule ("Unique positions") let { val p = any(kindOf[Person]) when { (availablePos.size equals 1) and (p.pos equals 0) } then { assign position availablePos.head to p } }, rule ("Unique colors") let { val p = any(kindOf[Person]) when { (availableColors.size equals 1) and (p.color == null) } then { assign color availableColors.head to p } }, rule ("Joe is in position 2") let { val p = any(kindOf[Person]) when { p.name equals "Joe" } then { assign position 2 to p } }, rule ("Person to Fred’s immediate right is wearing blue pants") let { val p1 = any(kindOf[Person]) val p2 = any(kindOf[Person]) when { (p1.name equals "Fred") and (p2.pos equals p1.pos + 1) } then { assign color "blue" to p2 } }, rule ("Fred isn't in position 4") let { val possibleFredPos = availablePos - 4 val p = any(kindOf[Person]) when { (p.name equals "Fred") and (possibleFredPos.size == 1) } then { assign position possibleFredPos.head to p } }, rule ("Tom isn't in position 1 or 4") let { val possibleTomPos = availablePos - 1 - 4 val p = any(kindOf[Person]) when { (p.name equals "Tom") and (possibleTomPos.size equals 1) } then { assign position possibleTomPos.head to p } }, rule ("Bob is wearing plaid pants") let { val p = any(kindOf[Person]) when { p.name equals "Bob" } then { assign color "plaid" to p } }, rule ("Tom isn't wearing orange pants") let { val possibleTomColors = availableColors - "orange" val p = any(kindOf[Person]) when { (p.name equals "Tom") and (possibleTomColors.size equals 1) } then { assign color possibleTomColors.head to p } } ) Here the first 2 rules explicitly leverage the uniqueness of positions and pants colors by assigning the last available of them to the only person who still doesn't have one. The other rules just match one by one the sentences of the problem as it has been defined. Now it is possible to make Hammurabi solve the problem by creating the four golfers: val tom = new Person("Tom") val joe = new Person("Joe") val fred = new Person("Fred") val bob = new Person("Bob") add them to a working memory (the set of objects against which the rule engine will evaluate and fire the rules): val workingMemory = WorkingMemory(tom, joe, fred, bob) and letting the rule engine, initialized with the formerly defined set of rules, to work on it: RuleEngine(ruleSet) execOn workingMemory Working with immutable data structures Immutability is probably not something that should be enforced at all costs in Hammurabi, for a very simple reason: the largest part of the execution time is spent by Hammurabi, like all other rule engines, looking for rules that can actually be executed (fired), i.e. the ones for which the when condition is true. During this phase data are only read and never written, so immutability doesn't matter at all, and all the rules that can be fired are put in an agenda. During the subsequent phase the rules in the agenda MUST be fired one by one, since the execution of one of them could make false the when condition of another one. It means that lack of immutability shouldn't prevent the rule engine to safely run in parallel during the discovery phase. That said, it is also possible to obtain the same result working with immutable data structures. For example having an immutable person: case class Person(name: String, pos: Int = 0, color: String = null) it's enough to rewrite the methods that assign the position and pants color to the golfers as it follows: val assign = new { def color(color: String) = new { def to(person: Person) = { remove(person) produce(person.copy(color = color)) availableColors = availableColors - color } } def position(pos: Int) = new { def to(person: Person) = { remove(person) produce(person.copy(pos = pos)) availablePos = availablePos - pos } } } leaving all the rules unchanged. In this way the old version of the Person is removed from the working memory and a brand new one, with its position or color set accordingly to the fired rule, is produced and then added to the working memory itself. The methods remove and produce can be indeed used respectively to remove objects from the working memory and produce new objects, that could be then used to evaluate and fire other rules. Hammurabi internals In Hammurabi all the rules are evaluated (but not fired) in parallel basically by assigning each rule to a different Scala actor. The replacement of this actor implementation with one based on the upcoming Scala parallel collections is currently under evaluation but I decided to wait until this technology will be stable. As anticipated, the evaluation of the rule in parallel is safe because is a read only process. While the actors discover set of variables that can fire the rules they are evaluating, they add them to the rule engine's agenda. At the end of this evaluation process the rules are fired sequentially one by one after a re-evaluation of their application condition, because the execution of one of them could change the result of that condition for a subsequent one. All the rules are fired in no particular order unless a different priority has been specified for some of them. Indeed since sometimes there could be the need to treat some rules as special cases they have an optional property called salience that acts as a priority setting for that rule in order to allow activated rules with the highest salience to always fire first, followed by rules of lower salience. By default all rules have salience 0 but you can alter it in the rule definition as it follows: rule ("Important rule") withSalience 10 let { ... } rule ("Negligible rule") withSalience -5 let { ... } Each actor also records the set of values on which the rule it is responsible for has been already executed, in order to don't fire again the same rule on the same values. The evaluation phase and the firing one are then executed again and again until either there is no rule that can be fired or one of the rule during its firing phase invokes one of the methods exitWith, making the rule engine gracefully finishing returning a value representing the result of the whole evaluation process or failWith that cause the rule engine to terminate by throwing an exception. Of course if the rule engine stops just because there are no longer rules that can be fired, the result of the evaluation is represented by the whole working set (as in the former example) and you can read from there the values you are interested in. Further implementations and improvements At the moment it is only possible to take from the working memory the object(s) against which a given rule will be evaluated only selecting them by type, as shown in the statement: val p = any(kindOf[Person]) Further mechanisms to categorize and select those objects in a more precise way are under evaluation, even if it is already possible to limit them with a Boolean function as in the following example: val p = kindOf[Person] having (_.name == "Joe") This is useful also under a performance point of view because it dramatically lowers the number of combination that the rule engine needs to check before to find a rule that can be fired. For example the "Person to Fred’s immediate right is wearing blue pants" rule could be rewritten as it follows bringing the number of combination that has to be tried from 16 to 4: rule ("Person to Fred’s immediate right is wearing blue pants") let { val p1 = kindOf[Person] having (_.name == "Fred") val p2 = any(kindOf[Person]) when { p2.pos equals p1.pos + 1 } then { assign color "blue" to p2 } } I am also evaluating of directly feeding the working memory with a NoSQL database. In other words, with this solution, the data present in the db could represent the working memory itself. At the moment I am experimenting with MongoDB since is the one I know best, but if somebody has some other good idea or even better wants to collaborate with this project I'd be very glad of it. The version 0.1 of the Hammurabi rule engine has been just released and it is available here.

In the Spring Web Framework, its typical to implement a Controller as a class that implements org.springframework.web.servlet.mvc.Controller, for example: public class InventoryController implements Controller { public ModelAndView handleRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { // Handle the request here } } This class would then be defined within an application's Spring context XML file (typically appname-servlet.xml) ... Using Spring annotations however, its possible to remove the need to implement the org.springframework.web.servlet.mvc.controller and remove the bean definition within the XML file. To change the Controller class to use annotations, the class needs to be annotated with the @Controller and @RequestMapping annotations as shown below. The method that will handle the request also needs to be annotated with the @RequestMapping. Note that this method no longer needs to confirm to the same signature as defined in org.springframework.web.servlet.mvc.Controller and now simply returns a ModelAndView instance. @Controller @RequestMapping("/home.htm") public class InventoryController { @RequestMapping(method=RequestMethod.GET) public ModelAndView handleRequest() { // Handle the request here } } Now that we've redefined the Controller class, we can remove the bean definition from the application's context file. The final stage then to allow Spring to use the annotated Controller is to specify in the application's context file that we want to use annotations. This is achieved by adding the annotation into the application context file. ... From http://www.davidsalter.co.uk/1/post/2011/04/converting-a-spring-controller-into-a-controller.html

Those unfamiliar with Clojure are often interested in how you manage changing state within your applications. If you've heard a few things about Clojure but haven't really looked at it, I wouldn't be surprised if you thought it was impossible to write a "real" application with Clojure since "everything is immutable". I've even heard a developer that I respect make the mistake of saying: we're not going to use Clojure because it doesn't handle state well. Clearly, state management in Clojure is greatly misunderstood. I actually had a hard time not calling this blog entry "Clojure, it's about state". I think state shapes Clojure more than any other influence; it's the core of the language (as far as I can tell). Rich Hickey has clearly spent a lot of time thinking about state - there's an essay at http://clojure.org/state which describes common problems with a traditional approach to state management and Clojure's solutions. Rich's essay does a good job of succinctly discussing his views on state; you should read it before you continue with this entry. The remainder of this entry will give examples of how you can manage state using Clojure's functions. At the end of Rich's essay he says: In the local case, since Clojure does not have mutable local variables, instead of building up values in a mutating loop, you can instead do it functionally with recur or reduce. Before we get to reduce, let's start with the simplest example. You have an array of ints and you want to double each integer. In a language with mutable state you can loop through the array and build a new array with each integer doubled. for (int i=0; i < nums.length; i++) { result.add(nums[i] * 2); } In Clojure you would build the new array by calling the map function with a function that doubles each value. (I'm using Clojure 1.2) user=> (map (fn [i] (* i 2)) [1 2 3]) (2 4 6) If you're new to Clojure there's a few things worth mentioning. "user=>" is a REPL prompt. You enter some text and hit enter and the text is evaluated. If you've completed the list (closed the parenthesis), the results of evaluating that list will be printed to the following line. I remember what I thought the first time I looked at a lisp, and I know the code might not look like readable code, so here's a version that breaks up a few of the concepts and might make it easier to digest the example. user=> (defn double-int [i] (* i 2)) #'user/double-int user=> (def the-array [1 2 3]) #'user/the-array user=> (map double-int the-array) (2 4 6) In the first Clojure example you call the fn function to create an anonymous function, that was then passed to the map function (to be applied to each element of the array). The map function is a high order function that can take an anonymous function (example 1) or a named function (double-int, example 2). In Clojure (def ...) is a special form that allows you to define a var and defn is a function that allows you to easily define a function and assign it to a var. The syntax for defn is pretty straightforward, the first argument is the name, the second argument is the argument list of the new function, and any additional forms are the body of the function you are defining. Once you get used to Clojure's syntax you can even have a bit of fun with your function naming that might result in concise and maintainable code. user=> (defn *2 [i] (* 2 i)) #'user/*2 user=> (map *2 [1 2 3]) (2 4 6) but, I digress. Similarly, you may want to sum the numbers from an array. for (int i = 0; i < nums.length; i++) { result += nums[i]; } You can achieve goal of reducing an array to a single value in Clojure using the reduce function. user=> (reduce + [1 2 3]) 6 Clojure has several functions that allow you to create new values from existing values, which should be enough to solve any problem where you would traditionally use local mutable variables. For non-local mutable state you generally have 3 options: atoms, refs, and agents. When I started programming in Clojure, atoms were my primary choice for mutable state. Atoms are very easy to use and only require that you know a few functions to interact with them. Let's assume we're building a trading application that needs to keep around the current price of Apple. Our application will call our apple-price-update function when a new price is received and we'll need to keep that price around for (possible) later usage. The example below shows how you can use an atom to track the current price of Apple. user=> (def apple-price (atom nil)) #'user/apple-price user=> (defn update-apple-price [new-price] (reset! apple-price new-price)) #'user/update-apple-price user=> @apple-price nil user=> (update-apple-price 300.00) 300.0 user=> @apple-price 300.0 user=> (update-apple-price 301.00) 301.0 user=> (update-apple-price 302.00) 302.0 user=> @apple-price 302.0 The above example demonstrates how you can create a new atom and reset its value with each price update. The reset! function sets the value of the atom synchronously and returns its new value. You can also query the price of apple at any time using @ (or deref). If you're coming from a Java background the example above should be the easiest to relate to. Each time we call the update-apple-price function our state is set to a new value. However, atoms provide much more value than simply being a variable that you can reset. You may remember the following example from Java Concurrency in Practice. @NotThreadSafe public class UnsafeSequence { private int value; /** * Returns a unique value. */ public int getNext() { return value++; } } The book explains why this could cause potential problems. The problem with UnsafeSequence is that with some unlucky timing, two threads could call getNext and receive the same value. The increment notation, nextValue++, may appear to be a single operation, but is in fact three separate operations: read the value, add one to it, and write out the new value. Since operations in multiple threads may be arbitrarily interleaved by the runtime, it is possible for two threads to read the value at the same time, both see the same value, and then both add one to it. The result is that the same sequence number is returned from multiple calls in different threads. We could write a get-next function using a Clojure atom and the same race condition would not be a concern. user=> (def uniq-id (atom 0)) #'user/uniq-id user=> (defn get-next [] (swap! uniq-id inc)) #'user/get-next user=> (get-next) 1 user=> (get-next) 2 The above code demonstrates the result of calling get-next multiple times (the inc function just adds one to the value passed in). Since we aren't in a multithreaded environment the example isn't exactly breathtaking; however, what's actually happening under the covers is described very well on clojure.org/atoms - [Y]ou change the value by applying a function to the old value. This is done in an atomic manner by swap! Internally, swap! reads the current value, applies the function to it, and attempts to compare-and-set it in. Since another thread may have changed the value in the intervening time, it may have to retry, and does so in a spin loop. The net effect is that the value will always be the result of the application of the supplied function to a current value, atomically. Also, remember that changes to atoms are synchronous, so our get-next function will never return the same value twice. (note: while Java already provides an AtomicInteger class for handling this issue - that's not the point. The point of the example is to show that an Atom is safe to use across threads.) If you're truly interested in verifying that an atom is safe across threads, The Joy of Clojure provides the following snippet of code (as well as a wonderful explanation of all things Clojure, including mutability). (import '(java.util.concurrent Executors)) (def *pool* (Executors/newFixedThreadPool (+ 2 (.availableProcessors (Runtime/getRuntime))))) (defn dothreads [f & {thread-count :threads exec-count :times :or {thread-count 1 exec-count 1}] (dotimes [t thread-count] (.submit *pool* #(dotimes [_ exec-count] (f))))) (def ticks (atom 0)) (defn tick [] (swap! ticks inc)) (dothreads tick :threads 1000 :times 100) @ticks ;=> 100000 There you have it, 1000 threads updated ticks 100 times without issue. Atoms work wonderfully when you want to insure atomic updates to an individual piece of state; however, it probably wont be long before you find yourself wanting to coordinate some type of state update. For example, if you're running an online store, when a customer cancels an order the order is either active or cancelled; however, the order should never be active and cancelled. If you were to keep a set of active orders and a set of cancelled orders, you would never want to have an order be in both sets at the same time. Clojure addresses this issue by using refs. Refs are similar to atoms, but they also participate in coordinated updates. The following example shows the cancel-order function moving an order-id from the active orders set into the cancelled orders set. user=> (def active-orders (ref #{2 3 4})) #'user/active-orders user=> (def cancelled-orders (ref #{1})) #'user/cancelled-orders user=> (defn cancel-order [id] (dosync (commute active-orders disj id) (commute cancelled-orders conj id))) #'user/cancel-order user=> (cancel-order 2) #{1 2} user=> @active-orders #{3 4} user=> @cancelled-orders #{1 2} As you can see from the example, we're moving an order id from active to cancelled. Again, our REPL session doesn't show the power of what's going on with a ref, but clojure.org/refs contains a good explanation - All changes made to Refs during a transaction (via ref-set, alter or commute) will appear to occur at a single point in the 'Ref world' timeline (its 'write point'). The above quote is actually only 1 item in a 10 point list that discusses what's actually going on. It's worth reviewing the list a few times until you feel comfortable with everything that's going on. But, you don't need to completely understand everything to get started. You can begin to experiment with refs anytime you know you need coordinated changes to more than one piece of state. When you first begin to look at refs you may wonder if you should use commute or alter. For most cases commute will provide more concurrency and is preferred; however, you may need to guarantee that the ref has not been updated during the life of the current transaction. This is generally the case where alter comes into play. The following example shows using commute to update two values. The example demonstrates that the pairs are always updated only once; however, it also shows that the function is simply applied to the current value, so the incrementing is not sequential and @uid can be dereferenced to the same value multiple times. user=> (def uid (ref 0)) #'user/uid user=> (def used-id (ref [])) #'user/used-id user=> (defn use-id [] (dosync (commute uid inc) (commute used-id conj @uid))) #'user/use-id user=> (dothreads use-id :threads 10 :times 10) nil user=> @used-id [1 2 3 4 5 6 7 8 9 10 ... 89 92 92 94 93 94 97 97 99 100] The above example shows that commute simply applies regardless of the underlying value. As a result, you may see duplicate values and gaps in your sequence (shown in the 90s in our output). If you wanted to ensure that the value didn't change during your transaction you could switch to alter. The following example shows the behavior of changing from commute to alter. user=> (def uid (ref 0)) #'user/uid user=> (def used-id (ref [])) #'user/used-id user=> (defn use-id [] (dosync (alter uid inc) (alter used-id conj @uid))) #'user/use-id user=> (dothreads use-id :threads 10 :times 10) nil user=> @used-id [1 2 3 4 5 6 7 8 9 10 ... 91 92 93 94 95 96 97 98 99 100] There are more advanced examples using refs in The Joy of Clojure for those of you looking to discuss corner case conditions. Last, but not least, agents are also available. From clojure.org/agents - Like Refs, Agents provide shared access to mutable state. Where Refs support coordinated, synchronous change of multiple locations, Agents provide independent, asynchronous change of individual locations. While I understand agents conceptually, I haven't used them much in practice. Some people love them, and the last team I was on switched to using agents heavily in one of our applications shortly after I left. But, I personally don't have enough experience to say exactly where I think they fit in. I'm sure that will be a topic for a future blog post. Between Rich's essay and the examples above I hope a few things have become clear: Clojure has plenty of support for managing state Rich's distinction between identity and value allows Clojure to benefit from immutable structures while also allowing identity reassignment. Clojure, it's about state. From http://blog.jayfields.com/2011/04/clojure-state-management.html

Linear algebra is a commonly used area of mathematics with a wide range of applications in various engineering and scientific fields. Examples of applications include line fitting, Kalman filters, face recognition, financial software, and numerical optimization to name a few. Many computer libraries have been developed for linear algebra. One of the better known would be LAPACK, which was originally programmed in Fortran. Introducing Efficient Java Matrix Library The following article provides a brief introduction to one of the newer linear algebra libraries for Java, Efficient Java Matrix Library (EJML), a free open source library. EJML is a linear algebra library for dense real matrices. EJML's design goals are; 1) to be as computationally and memory efficient as possible for both small and large matrices, and 2) to be accessible to both novices and experts. These goals are accomplished by dynamically selecting the best algorithms to use at runtime and through a thoughtfully designed clean API. Providing good documentation, code examples, and constant benchmarking for speed, memory, and stability are also priorities. The following functionality is provided: Basic Operators (addition, multiplication, ... ) Matrix Manipulation (extract, insert, combine, ... ) Linear Solvers (linear, least squares, incremental, ... ) Decompositions (LU, QR, Cholesky, SVD, Eigenvalue, ...) Matrix Features (rank, symmetric, definitiveness, ... ) Random Matrices (covariance, orthogonal, symmetric, ... ) Different Internal Formats (row-major, block) Unit Testing EJML can be downloaded from its website at: http://ejml.org/ Application Programming Interface There are three primary ways to interact with EJML, 1) a simple to use object oriented interface, 2) procedural interface that provides greater control over memory and algorithms, 3) an expert interface that directly access specialized algorithms that is abstracted away using the first two. To see a practical comparison of these three interfaces take a look at the Kalman filter example provided at EJML's website. There each interface is used to code up a functionally identical Kalman filter: Here are three code sniplets showing the Kalman gain being computed: Simple: SimpleMatrix K = P.mult(H.transpose().mult(S.invert())); Procedural: if( !invert(S,S_inv) ) throw new RuntimeException("Invert failed"); multTransA(H,S_inv,d); mult(P,d,K); Specialized: if( !solver.setA(S) ) throw new RuntimeException("Invert failed"); solver.invert(S_inv); MatrixMatrixMult.multTransA_small(H,S_inv,d); MatrixMatrixMult.mult_small(P,d,K); The full examples are available at: http://ejml.org/wiki/index.php?title=Example_Kalman_Filter Going beyond the basics, there are easy to use yet powerful Java interfaces for solving linear systems and matrix decompositions. These provide much more control over the type of algorithm used, what it computes, how much memory is used, and remove as much of the drudgery as possible. DecompositionFactory and LinearSolverFactory are provided for creating these solvers and decompositions. By using the factory the best most update algorithm will be automatically selected. EJML offer many different decomposition algorithms, even within the same family. For instance, there are four QR decompositions provided, each of which is catered towards a different sized matrix. Different factories hide much of these detail from the end user. Who is using EJML? Even though EJML is a fairly new library it has already been picked up by several opensource projects: goGPS: http://code.google.com/p/gogps/ Set Visualiser: http://www-edc.eng.cam.ac.uk/tools/set_visualiser/ Universal Java Matrix Library (UJML): http://www.ujmp.org/ Scalalab: http://code.google.com/p/scalalab/ Java Content Based Image Retrieval (JCBIR): http://code.google.com/p/jcbir/ JquantLib (Will be added): http://www.jquantlib.org/ Performance and Benchmarks What about speed, stability, and correctness? Constant benchmarking for speed and stability is a core part of EJML's development. It has many internal benchmarks and also uses Java Matrix Benchmark (JMatBench)[1] (http://code.google.com/p/java-matrix-benchmark/) to compare its performance against other libraries. At the time of this writing there are a total of 551 unit tests that test basic correctness of almost all the functions in EJML. For runtime performance EJML is one of the fastest single threaded libraries. When compared against multi-threaded libraries on systems with several cores/CPU's it still competitive for many operations, despite its disadvantage of being single threaded. It has one of the lowest memory footprints [2]. A brief summary of EJML's performance relative to other libraries is shown below. Runtime performance is measured on a Core i7M620 system. (2 cores with 4 threads) Performance varies significantly by system. This processor was choosen because it is the most modern system benchmarked. Click here to get a explaination of the plots shown below. [1] Both EJML and JMatBench are developed by the same author. [2] Memory usage is highly dependent on the operation being used and the parameters passed to it. JMatBench only tests a few operations, but at least it shows attention is being paid to memory usage.

Sometimes the mechanics of different tests are really the same, and the only thing that changes between them are input and expected output data. As an example, consider testing mathematical calculations functions, or any kind of stateless method: you invoke it always in the same way, and then check the returned value. Another case is that of triangulation in Test-Driven Development. When an implementation is not obvious, triangulating it leads to the creation of multiple tests with different data, but usually the exact same procedure. To eliminate the duplication between these tests, we can code a method which models the whole test, and that accepts input (and possible initial state) and expected output as parameters. This method will perform the arrange, act, assert, and even teardown phases if needed. It will eliminate the need for maintaining the same test code in different places, of course. An advantage of these refactorings that we already cited in Test Utility Method is that adding a new test becomes equivalent to adding one or two lines of code. This pattern is really a specialized, standalone Test Utility Method. Implementation A simple form of Parameterized Test is a private method which is delegated from in the various test*() ones, which are called by the testing framework. The advantage of this first approach is to present more clear names to the code reader, and to support possible additional work to perform after the method is executed (it may return something which we can make further assertions on.) A second way of implementing Parameterized Test is with PHPUnit's @dataProvider annotation. Is it completely automatic, and displays data used in the test instance in case of failure in order to recognize which iteration we are in. You can also prepare input and output for @dataProvider programmatically (read: with code), so with a process as flexible and complex as you want. Variations Some older versions of Parameterized Test exist. A Loop-Driven Test uses nested loops that verify every combination of inputs/outputs. For example, it may check pairs of x and y coordinates in an image. Often exhaustive testing is an overkill and contains logic for the generation of data which may contain bugs. In a Tabular Test, you have a table, represented as code or in some configuration file, where each row corresponds to one test run and contains the necessary input/output relation. One Test Method works on everything here. This mechanism is obsolete as PHPUnit with @dataProvider executes each instance of the method into a different Testcase Object. You won't observe tests influencing each other, no problem with localization of which row is causing a failure. Example The code sample shows you how to go from a strong duplication between tests to a single copy of the code that uses @dataProvider to be executed N times. It also shows, in the third Testcase Class, how easy is to produce data for the tests with PHP code. assertEquals(0, sqrt(0)); } public function testSquareRootIsCalculatedCorrectlyByPHPFor1() { $this->assertEquals(1, sqrt(1)); } public function testSquareRootIsCalculatedCorrectlyByPHPFor4() { $this->assertEquals(2, sqrt(4)); } public function testSquareRootIsCalculatedCorrectlyByPHPFor9() { $this->assertEquals(3, sqrt(9)); } public function testSquareRootIsCalculatedCorrectlyByPHPFor16() { $this->assertEquals(4, sqrt(16)); } public function testSquareRootIsCalculatedCorrectlyByPHPForMinusOne() { $this->assertEquals('i', sqrt(-1)); } } class ParameterizedTest extends PHPUnit_Framework_TestCase { /** * This should be a static method, returning an array of arrays. * Each row of this table-like array is a set of arguments * for the Test Method. */ public static function squaresAndRoots() { return array( array(0, 0), array(1, 1), array(4, 2), array(9, 3), array(16, 4), array(-1, 'i') ); } /** * The annotation requires as unique argument the name of a public method * in this Testcase Class. * @dataProvider squaresAndRoots */ public function testSquareRootIsCalculatedCorrectlyByPHP($square, $root) { $this->assertEquals($root, sqrt($square)); } } class ProgrammaticDataProviderTest extends PHPUnit_Framework_TestCase { private static $width = 5; private static $height = 10; /** * I don't know why these methods are required to be static. It makes * no difference however as we are never going to call it directly. */ public static function everyCoupleOfCoordinates() { $testParameters = array(); for ($x = 1; $x <= self::$width; $x++) { for ($y = 1; $y <= self::$height; $y++) { $testParameters[] = array($x, $y); } } return $testParameters; } /** * I don't advise you to test an image at every pixel, but in some cases * you may need an exhaustive search. The programmatic generation of data * keeps the test code really short, but don't forget that the generation * logic may hide bugs if it becomes too complex. * @dataProvider everyCoupleOfCoordinates */ public function testImageHasCorrectTransparencyValue($x, $y) { $this->fail("This test will be executed in isolation with the x=$x and y=$y values."); } }

Bixo Labs shows how to use Solr as a NoSQL solution for big data Many people use the Hadoop open source project to process large data sets because it’s a great solution for scalable, reliable data processing workflows. Hadoop is by far the most popular system for handling big data, with companies using massive clusters to store and process petabytes of data on thousands of servers. Since it emerged from the Nutch open source web crawler project in 2006, Hadoop has grown in every way imaginable – users, developers, associated projects (aka the “Hadoop ecosystem”). Starting at roughly the same time, the Solr open source project has become the most widely used search solution on planet Earth. Solr wraps the API-level indexing and search functionality of Lucene with a RESTful API, GUI, and lots of useful administrative and data integration functionality. The interesting thing about combining these two open source projects is that you can use Hadoop to crunch the data, and then serve it up in Solr. And we’re not talking about just free-text search; Solr can be used as a key-value store (i.e. a NoSQL database) via its support for range queries. Even on a single server, Solr can easily handle many millions of records (“documents” in Lucene lingo). Even better, Solr now supports sharding and replication via the new, cutting-edge SolrCloud functionality. Background I started using Hadoop & Solr about five years ago, as key pieces of the Krugle code search startup I co-founded in 2005. Back then, Hadoop was still part of the Nutch web crawler we used to extract information about open source projects. And Solr was fresh out of the oven, having just been released as open source by CNET. At Bixo Labs we use Hadoop, Solr, Cascading, Mahout, and many other open source technologies to create custom data processing workflows. The web is a common source of our input data, which we crawl using the Bixo open source project. The Problem During a web crawl, the state of the crawl is contained in something commonly called a “crawl DB”. For broad crawls, this has to be something that works with billions of records, since you need one entry for each known URL. Each “record” has the URL as the key, and contains important state information such as the time and result of the last request. For Hadoop-based crawlers such as Nutch and Bixo, the crawl DB is commonly kept in a set of flat files, where each file is a Hadoop “SequenceFile”. These are just a packed array of serialized key/value objects. Sometimes we need to poke at this data, and here’s where the simple flat-file structure creates a problem. There’s no easy way run queries against the data, but we can’t store it in a traditional database since billions of records + RDBMS == pain and suffering. Here is where scalable NoSQL solutions shine. For example, the Nutch project is currently re-factoring this crawl DB layer to allow plugging in HBase. Other options include Cassandra, MongoDB, CouchDB, etc. But for simple analytics and exploration on smaller datasets, a Solr-based solution works and is easier to configure. Plus you get useful and surprising fun functionality like facets, geospatial queries, range queries, free-form text search, and lots of other goodies for free. Architecture So what exactly would such a Hadoop + Solr system look like? As mentioned previously, in this example our input data comes from a Bixo web crawler’s CrawlDB, with one entry for each known URL. But the input data could just as easily be log files, or records from a traditional RDBMS, or the output of another data processing workflow. The key point is that we’re going to take a bunch of input data, (optionally) munge it into a more useful format, and then generate a Lucene index that we access via Solr. Hadoop For the uninitiated, Hadoop implements both a distributed file system (aka “HDFS”) and an execution layer that supports the map-reduce programming model. Typically data is loaded and transformed during the map phase, and then combined/saved during the reduce phase. In our example, the map phase reads in Hadoop compressed SequenceFiles that contain the state of our web crawl, and our reduce phase write out Lucene indexes. The focus of this article isn’t on how to write Hadoop map-reduce jobs, but I did want to show you the code that implements the guts of the job. Note that it’s not typical Hadoop key/value manipulation code, which is painful to write, debug, and maintain. Instead we use Cascading, which is an open source workflow planning and data processing API that creates Hadoop jobs from shorter, more representative code. The snippet below reads SequenceFiles from HDFS, and pipes those records into a sink (output) that stores them using a LuceneScheme, which in turn saves records as Lucene documents in an index. Tap source = new Hfs(new SequenceFile(CRAWLDB_FIELDS), inputDir); Pipe urlPipe = new Pipe("crawldb urls"); urlPipe = new Each(urlPipe, new ExtractDomain()); Tap sink = new Hfs(new LuceneScheme(SOLR_FIELDS, STORE_SETTINGS, INDEX_SETTINGS, StandardAnalyzer.class, MAX_FIELD_LENGTH), outputDir, true); FlowConnector fc = new FlowConnector(); fc.connect(source, sink, urlPipe).complete(); We defined CRAWLDB_FIELDS and SOLR_FIELDS to be the set of input and output data elements, using names like “url” and “status”. We take advantage of the Lucene Scheme that we’ve created for Cascading, which lets us easily map from Cascading’s view of the world (records with fields) to Lucene’s index (documents with fields). We don’t have a Cascading Scheme that directly supports Solr (wouldn’t that be handy?), but we can make-do for now since we can do simple analysis for this example. We indexed all of the fields so that we can perform queries against them. Only the status message contains normal English text, so that’s the only one we have to analyze (i.e., break the text up into terms using spaces and other token delimiters). In addition, the ExtractDomain operation pulls the domain from the URL field and builds a new Solr field containing just the domain. This will allow us to do queries against the domain of the URL as well as the complete URL. We could also have chosen to apply a custom analyzer to the URL to break it into several pieces (i.e., protocol, domain, port, path, query parameters) that could have been queried individually. Running the Hadoop Job For simplicity and pay-as-you-go, it’s hard to beat Amazon’s EC2 and Elastic Mapreduce offerings for running Hadoop jobs. You can easily spin up a cluster of 50 servers, run your job, save the results, and shut it down – all without needing to buy hardware or pay for IT support. There are many ways to create and configure a Hadoop cluster; for us, we’re very familiar with the (modified) EC2 Hadoop scripts that you can find in the Bixo distribution. Step-by-step instructions are available at http://openbixo.org/documentation/running-bixo-in-ec2/ The code for this article is available via GitHub, at http://github.com/bixolabs/hadoop2solr. The README displayed on that page contains step-by-step instructions for building and running the job. After the job is done, we’ll copy the resulting index out of the Hadoop distributed file system (HDFS) and onto the Hadoop cluster’s master server, then kill off the one slave we used. The Hadoop master is now ready to be configured as our Solr server. Solr On the Solr side of things, we need to create a schema that matches the index we’re generating. The key section of our schema.xml file is where we define the fields. These fields have a one-to-one correspondence with the SOLR_FIELDS we defined in our Hadoop workflow. They also need to use the same Lucene settings as what we defined in the static IndexWorkflow.java STORE_SETTINGS and INDEX_SETTINGS. Once we have this defined, all that’s left is to set up a server that we can use. To keep it simple, we’ll use the single EC2 instance in Amazon’s cloud (m1.large) that we used as our master for the Hadoop job, and run the simple Solr search server that relies on embedded Jetty to provide the webapp container. Similar to the Hadoop job, step-by-step instructions are in the README for the hadoop2solr project on GitHub. But in a nutshell, we’ll copy and unzip a Solr 1.4.1 setup on the EC2 server, do the same for our custom Solr configuration, create a symlink to the index, and then start it running with: Giving it a Try Now comes the interesting part. Since we opened up the default Jetty port used by Solr (8983) on this EC2 instance, we can directly access Solr’s handy admin console by pointing our browser at http://:8983/solr/admin % cd solr % java -Dsolr.solr.home=../solr-conf -Dsolr.data.dir=../solr-data -jar start.jar From here we can run queries against Solr: We can also use curl to talk to the server via HTTP requests: curl http://:8983/solr/select/?q=-status%3AFETCHED+and+-status%3AUNFETCHED The response is XML by default. Below is an example of the response from the above request, where we found 2,546 matches in 94ms. Now here’s what I find amazing. For an index of 82 million documents, running on a fairly wimpy box (EC2 m1.large = 2 virtual cores), the typical response time for a simple query like “status:FETCHED” is only 400 milliseconds, to find 9M documents. Even a complex query such as (status not FETCHED and not UNFETCHED) only takes six seconds. Scaling Obviously we could use beefier boxes. If we switched to something like m1.xlarge (15GB of memory, 4 virtual cores) then it’s likely we could handle upwards of 200M “records” in our Solr index and still get reasonable response times. If we wanted to scale beyond a single box, there are a number of solutions. Even out of the box Solr supports sharding, where your HTTP request can specify multiple servers to use in parallel. More recently, the Solr trunk has support for SolrCloud. This uses the ZooKeeper open source project to simplify coordination of multiple Solr servers. Finally, the Katta open source project supports Lucene-level distributed search, with many of the features needed for production quality distributed search that have not yet been added to SolrCloud. Summary The combination of Hadoop and Solr makes it easy to crunch lots of data and then quickly serve up the results via a fast, flexible search & query API. Because Solr supports query-style requests, it’s suitable as a NoSQL replacement for traditional databases in many situations, especially when the size of the data exceeds what is reasonable with a typical RDBMS. Solr has some limitations that you should be aware of, specifically: · Updating the index works best as a batch operation. Individual records can be updated, but each commit (index update) generates a new Lucene segment, which will impact performance. · Current support for replication, fail-over, and other attributes that you’d want in a production-grade solution aren’t yet there in SolrCloud. If this matters to you, consider Katta instead. · Many SQL queries can’t be easily mapped to Solr queries. The code for this article is available via GitHub, at http://github.com/bixolabs/hadoop2solr. The README displayed on that page contains additional technical details.

I recently found the Windows PowerShell and can't stop using it. I've setup a couple of helper functions to let me start CF from a simple command in the console. Before you can do anything you need to create a profile script. From within PowerShell you can run $profile to get the expected location for your profile. If you don't already have the file create it now. You will also need to tell PowerShell that it is ok to execute the new script by running Set-ExecutionPolicy Unrestricted You can, and should, read more about script execution. If you have the standard ColdFusion install you can try this script (note: you should change line 1 to point at your correct ColdFusion install, my example shows CF9): new-alias jrun "C:\ColdFusion9\runtime\bin\jrun.exe" function cf($server="coldfusion"){ jrun -start $server } Using MultiServer? Try adding this to your profile script: new-alias jrun "C:\JRun4\bin\jrun.exe" function cf($server="cfusion"){ jrun -start $server } Now, in the console, you can start your default CF servers by simply typing cf If you want to use something other than the default server, like when using multiserver, you can type cf myserver or cf -server myserver To stop the server just type Ctrl+C Happy PowerShell scripting!

I’ve written a couple of posts (here and here) about Java and the JDBC Driver for SQL Server with the promise of eventually writing about how to get a Java application running on the Windows Azure platform. In this post, I’ll deliver on that promise. Specifically, I’ll show you two things: 1) how to connect to a SQL Azure Database from a Java application running locally, and 2) how to connect to a SQL Azure database from an application running in Windows Azure. You should consider these as two ordered steps in moving an application from running locally against SQL Server to running in Windows Azure against SQL Azure. In both steps, connection to SQL Azure relies on the JDBC Driver for SQL Server and SQL Azure. The instructions below assume that you already have a Windows Azure subscription. If you don’t already have one, you can create one here: http://www.microsoft.com/windowsazure/offers/. (You’ll need a Windows Live ID to sign up.) I chose the Free Trial Introductory Special, which allows me to get started for free as long as keep my usage limited. (This is a limited offer. For complete pricing details, see http://www.microsoft.com/windowsazure/pricing/.) After you purchase your subscription, you will have to activate it before you can begin using it (activation instructions will be provided in an email after signing up). Connecting to SQL Azure from an application running locally I’m going to assume you already have an application running locally and that it uses the JDBC Driver for SQL Server. If that isn’t the case, then you can start from scratch by following the steps in this post: Getting Started with the SQL Server JDBC Driver. Once you have an application running locally, then the process for running that application with a SQL Azure back-end requires two steps: 1. Migrate your database to SQL Azure. This only takes a couple of minutes (depending on the size of your database) with the SQL Azure Migration Wizard - follow the steps in the Creating a SQL Azure Server and Creating a SQL Azure Database sections of this post. 2. Change the database connection string in your application. Once you have moved your local database to SQL Azure, you only have to change the connection string in your application to use SQL Azure as your data store. In my case (using the Northwind database), this meant changing this… String connectionUrl = "jdbc:sqlserver://serverName\\sqlexpress;" + "database=Northwind;" + "user=UserName;" + "password=Password"; …to this… String connectionUrl = "jdbc:sqlserver://xxxxxxxxxx.database.windows.net;" + "database=Northwind;" + "user=UserName@xxxxxxxxxx;" + "password=Password"; (where xxxxxxxxxx is your SQL Azure server ID). Connecting to SQL Azure from an application running in Windows Azure The heading for this section might be a bit misleading. Once you have a locally running application that is using SQL Azure, then all you have to do is move your application to Windows Azure. The connecting part is easy (see above), but moving your Java application to Windows Azure takes a bit more work. Fortunately, Ben Lobaugh has written a great post that that shows how to use the Windows Azure Starter Kit for Java to get a Java application (a JSP application, actually) running in Windows Azure: Deploying a Java application to Windows Azure with Command-Line Ant. (If you are using Eclipse, see Ben’s related post: Deploying a Java application to Windows Azure with Eclipse.) I won’t repeat his work here, but I will call out the steps I took in modifying his instructions to deploy a simple JSP page that connects to SQL Azure. 1. Add the JDBC Driver for SQL Server to the Java archive. One step in Ben’s tutorial (see the Select the Java Runtime Environment section) requires that you create a .zip file from your local Java installation and add it to your Java/Azure application. Most likely, your local Java installation references the JDBC driver by setting the classpath environment variable. When you create a .zip file from your java installation, the JDBC driver will not be included and the classpath variable will not be set in the Azure environment. I found the easiest way around this was to simply add the sqljdbc4.jar file (probably located in C:\Program Files\Microsoft SQL Server JDBC Driver\sqljdbc_3.0\enu) to the \lib\ext directory of my local Java installation before creating the .zip file. Note: You can put the JDBC driver in a separate directory, include it when you create the .zip folder, and set the classpath environment variable in the startup.bat script. But, I found the above approach to be easier. 2. Modify the JSP page. Instead of the code Ben suggests for the HelloWorld.jsp file (see the Prepare your Java Application section), use code from your locally running application. In my case, I just used the code from this post after changing the connection string and making a couple minor JSP-specific changes: Northwind Customers That’s it!. To summarize the steps… Migrate your database to SQL Azure with the SQL Azure Migration Wizard. Change the database connection in your locally running application. Use the Windows Azure Starter Kit for Java to move your application to Windows Azure. (You’ll need to follow instructions in this post and instructions above.) Thanks. -Brian

This article discusses dependency injection in a tutorial format. It covers some of the features of CDI such as type safe annotations configuration, alternatives and more.

JSF and JavaScript can combine forces to develop powerful applications. For example, let's see how we can use JavaScript code with h:commandLink and h:commandButton to obtain a confirmation before getting into action. Getting ready We have developed this recipe with NetBeans 6.8, JSF 2.0, and GlassFish v3. The JSF 2.0 classes were obtained from the NetBeans JSF 2.0 bundled library. How to do it... As you know the h:commandLink takes an action after a link is clicked (on the mouse click event), while h:commandButton does the same thing, but renders a button, instead of a text link. In this case, we place a JavaScript confirmation box before the action starts its effect. This is useful in user tasks that can't be reversed, such as deleting accounts, database records, and so on. Therefore, the onclick event was implemented as shown next: How it works... Notice that we embed the JavaScript code inside the onclick event (you also may put it separately in a JS function, per example). When the user clicks the link or the button, a JS confi rmation box appear with two buttons. If you confirm the choice, the JSF action takes place, while if you deny it then nothing happens. There's more... You can use this recipe to display another JS box, such as prompt box or alert box. You can find this recipe in JSF 2.0 Cookbook from Packt From http://e-blog-java.blogspot.com/2011/03/how-to-integrate-javascript-and-jsf.html

Try with resources is a new feature in Java 7 which lets us write more elegant code by automatically closing resources like FileInputStream at the end of the try-block. Old Try Catch Finally Dealing with resources like InputStreams is painful when it comes to the try-catch-finally blocks. You need to declare the resources outside the try so that they are is accessible from finally, then you must initialize the variable to null and check for non-null when closing the resource in finally. File file = new File("input.txt"); InputStream is = null; try { is = new FileInputStream(file); // do something with this input stream // ... } catch (FileNotFoundException ex) { System.err.println("Missing file " + file.getAbsolutePath()); } finally { if (is != null) { is.close(); } } Java 7: Try with resources With Java 7, you can create one or more “resources” in the try statement. A “resources” is something that implements the java.lang.AutoCloseable interface. This resource would be automatically closed and the end of the try block. File file = new File("input.txt"); try (InputStream is = new FileInputStream(file)) { // do something with this input stream // ... } catch (FileNotFoundException ex) { System.err.println("Missing file " + file.getAbsolutePath()); } Exception handling If both the (explicit) try block and the (implicit) resource handling code throw an exception, then the try block exception is the one which will be thrown. The resource handling exception will be made available via the Throwable.getSupressed() method of the thrown exception. Throwable.getSupressed() is a new method added to the Throwable class since 1.7 specifically for this purpose. If there were no suppressed exceptions then this will return an empty array. Reference http://download.java.net/jdk7/docs/technotes/guides/language/try-with-resources.html From http://www.vineetmanohar.com/2011/03/java-7-try-with-auto-closable-resources/

this tutorial will walk you through how to setup ibatis ( mybatis ) in a simple java project and will present how to work with dynamic queries (sql). pre-requisites for this tutorial i am using: ide: eclipse (you can use your favorite one) database: mysql libs/jars: mybatis , mysql conector and junit (for testing) this is how your project should look like: sample database please run the script into your database before getting started with the project implementation. you will find the script (with dummy data) inside the sql folder. 1 – article pojo i represented the pojo we are going to use in this tutorial with a uml diagram, but you can download the complete source code in the end of this article. the goal of this tutorial is to demonstrate how to retrieve the article information from database using dynamic sql to filter the data. 2 – article mapper – xml one of the most powerful features of mybatis has always been its dynamic sql capabilities. if you have any experience with jdbc or any similar framework, you understand how painful it is to conditionally concatenate strings of sql together, making sure not to forget spaces or to omit a comma at the end of a list of columns. dynamic sql can be downright painful to deal with. while working with dynamic sql will never be a party, mybatis certainly improves the situation with a powerful dynamic sql language that can be used within any mapped sql statement. the dynamic sql elements should be familiar to anyone who has used jstl or any similar xml based text processors. in previous versions of mybatis, there were a lot of elements to know and understand. mybatis 3 greatly improves upon this, and now there are less than half of those elements to work with. mybatis employs powerful ognl based expressions to eliminate most of the other elements. if choose (when, otherwise) trim (where, set) foreach let’s explain each one with examples. 1 – first scenario : we want to retrieve all the articles from database with an optional filter: title. in other words, if user specify an article title, we are going to retrieve the articles that match with the title, otherwise we are going to retrieve all the articles from database. so we are going to implement a condition (if) : select id, title, author from article where id_status = 1 and title like #{title} 2 – second scenario : now we have two optional filters: article title and author. the user can specify both, none or only one filter. so we are going to implement two conditions: select id, title, author from article where id_status = 1 and title like #{title} and author like #{author} 3 – third scenario : now we want to give the user only one option: the user will have to specify only one of the following filters: title, author or retrieve all the articles from ibatis category. so we are going to use a choose element: select id, title, author from article where id_status = 1 and title like #{title} and author like #{author} and id_category = 3 4 – fourth scenario : take a look at all three statements above. they all have a condition in common: where id_status = 1. it means we are already filtering the active articles let’s remove this condition to make it more interesting. select id, title, author from article where title like #{title} and author like #{author} what if both title and author are null? we are going to have the following statement: select id, title, authorfrom articlewhere and what if only the author is not null? we are going to have the fololwing statement: select id, title, authorfrom articlewhereand author like #{author} and both fails! how to fix it? 5 – fifth scenario : we want to retrieve all the articles with two optional filters: title and author. to avoid the 4th scenatio, we are going to use a where element: select id, title, author from article title like #{title} and author like #{author} mybatis has a simple answer that will likely work in 90% of the cases. and in cases where it doesn’t, you can customize it so that it does. the where element knows to only insert “where” if there is any content returned by the containing tags. furthermore, if that content begins with “and” or “or”, it knows to strip it off. if the where element does not behave exactly as you like, you can customize it by defining your own trim element. for example,the trim equivalent to the where element is: select id, title, author from article title like #{title} and author like #{author} the overrides attribute takes a pipe delimited list of text to override, where whitespace is relevant. the result is the removal of anything specified in the overrides attribute, and the insertion of anything in the with attribute. you can also use the trim element with set. 6 – sixth scenario : the user will choose all the categories an article can belong to. so in this case, we have a list (a collection), and we have to interate this collection and we are going to use a foreach element: select id, title, author from article title like #{title} and author like #{author} the foreach element is very powerful, and allows you to specify a collection, declare item and index variables that can be used inside the body of the element. it also allows you to specify opening and closing strings, and add a separator to place in between iterations. the element is smart in that it won’t accidentally append extra separators. note : you can pass a list instance or an array to mybatis as a parameter object. when you do, mybatis will automatically wrap it in a map, and key it by name. list instances will be keyed to the name “list” and array instances will be keyed to the name “array”. the complete article.xml file looks like this: select id, title, author from article where id_status = 1 and title like #{title} select id, title, author from article where id_status = 1 and title like #{title} and author like #{author} select id, title, author from article where id_status = 1 and title like #{title} and author like #{author} and id_category = 3 select id, title, author from article title like #{title} and author like #{author} select id, title, author from article title like #{title} and author like #{author} select id, title, author from article where id_category in #{category} download if you want to download the complete sample project, you can get it from my github account: https://github.com/loiane/ibatis-dynamic-sql if you want to download the zip file of the project, just click on download: there are more articles about ibatis to come. stay tooned! happy coding! from http://loianegroner.com/2011/03/ibatis-mybatis-working-with-dynamic-queries-sql/