In web applications searching for information based on the selected criteria and displaying the results is a very common requirement. Suppose we need to search users based on their name. The end user will enter the username in the textbox and hit the search button and the user results will be fetched from database and display in a grid. At first this looks simple and we can start to implement it as follows: public class UserSearchAction extends Action { public ActionForward execute(...) { SearchForm sf = (SearchForm)form; String searchName = sf.getSearchName(); UserService userService = new UserService(); List searchResults = userService.search(searchName); //put search results in request and dsplay in JSP } } public class UserService { public List search(String username) { // query the DB and get the results by applying filter on USERNAME column List users = UserDAO.search(username); } } The above implementation works fine for the current requirement. Later client wants to display only 10 rows per page and display a message like "Displaying 1-10 of 35 Users". Now the code need to be changed for the change request. public class UserSearchAction extends Action { public ActionForward execute(...) { SearchForm sf = (SearchForm)form; String searchName = sf.getSearchName(); UserService userService = new UserService(); Map searchResultsMap = userService.search(searchName, start, pageSize); List users = (List)searchResultsMap.get("DATA"); Integer count = (Integer)searchResultsMap.get("COUNT"); //put search results in request and dsplay in JSP } } public class UserService { public Map search(String username, int start, int pageSize) { //Get the total number of results for this criteria int count = UserDAO.searchResultsCount(username); List users = UserDAO.search(username, start, pageSize); // query the DB and get the start to start+pageSize results by applying filter on USERNAME column Map RESULTS_MAP = new HashMap(); RESULTS_MAP.put("DATA",users); RESULTS_MAP.put("COUNT",count); return RESULTS_MAP; } } Later the client wants to give an option to the end user to choose the search type either by UserID or by Username and show the paginated results. Now again the code needs to be changed for the change request. public class UserSearchAction extends Action { public ActionForward execute(...) { SearchForm sf = (SearchForm)form; String searchName = sf.getSearchName(); String searchId = sf.getSearchId(); UserService userService = new UserService(); Map searchCriteriaMap = new HashMap(); //searchCriteriaMap.put("SEARCH_BY","NAME"); searchCriteriaMap.put("SEARCH_BY","ID"); searchCriteriaMap.put("ID",searchId); searchCriteriaMap.put("START",start); searchCriteriaMap.put("PAGESIZE",pageSize); Map searchResultsMap = userService.search(searchCriteriaMap); List users = (List)searchResultsMap.get("DATA"); Integer count = (Integer)searchResultsMap.get("COUNT"); //put search results in request and dsplay in JSP } } public class UserService { public Map search(Map searchCriteriaMap) { return UserDAO.search(searchCriteriaMap); } } public class UserDAO { public Map search(Map searchCriteriaMap) { String SEARCH_BY = (String)searchCriteriaMap.get("SEARCH_BY"); int start = (Integer)searchCriteriaMap.get("START"); int pageSize = (Integer)searchCriteriaMap.get("PAGESIZE"); if("ID".equals(SEARCH_BY)) { int id = (Integer)searchCriteriaMap.get("ID"); //Get the total number of results for this criteria int count = UserDAO.searchResultsCount(id); // query the DB and get the start to start+pageSize results by applying filter on USER_ID column List users = search(id, start, pageSize); } else { String username = (String)searchCriteriaMap.get("USERNAME"); //Get the total number of results for this criteria int count = UserDAO.searchResultsCount(username); // query the DB and get the start to start+pageSize results by applying filter on USERNAME column List users = search(username, start, pageSize); } Map RESULTS_MAP = new HashMap(); RESULTS_MAP.put("DATA",users); RESULTS_MAP.put("COUNT",count); return RESULTS_MAP; } } Finally the code becomes a big mess and completely violates object oriented principles. There are lot of problems with the above code. 1. For each change request the method signatures are changing 2. Code needs to be changed for each enhancement such as adding more search criteria We can design a better object model for this kind of search functionality which is Object Oriented and scalable as follows. A generic SearchCriteria which holds common search criteria like pagination, sorting details. package com.sivalabs.javabp; public abstract class SearchCriteria { private boolean pagination = false; private int pageSize = 25; private String sortOrder = "ASC"; public boolean isPagination() { return pagination; } public void setPagination(boolean pagination) { this.pagination = pagination; } public String getSortOrder() { return sortOrder; } public void setSortOrder(String sortOrder) { this.sortOrder = sortOrder; } public int getPageSize() { return pageSize; } public void setPageSize(int pageSize) { this.pageSize = pageSize; } } A generic SearchResults object which holds the actual results and other detials like total available results count, page wise results provider etc. package com.sivalabs.javabp; import java.util.ArrayList; import java.util.List; public abstract class SearchResults { private int totalResults = 0; private int pageSize = 25; private List results = null; public int getPageSize() { return pageSize; } public void setPageSize(int pageSize) { this.pageSize = pageSize; } public int getTotalResults() { return totalResults; } private void setTotalResults(int totalResults) { this.totalResults = totalResults; } public List getResults() { return results; } public List getResults(int page) { if(page <= 0 || page > this.getNumberOfPages()) { throw new RuntimeException("Page number is zero or there are no that many page results."); } List subList = new ArrayList(); int start = (page -1)*this.getPageSize(); int end = start + this.getPageSize(); if(end > this.results.size()) { end = this.results.size(); } for (int i = start; i < end; i++) { subList.add(this.results.get(i)); } return subList; } public int getNumberOfPages() { if(this.results == null || this.results.size() == 0) { return 0; } return (this.totalResults/this.pageSize)+(this.totalResults%this.pageSize > 0 ? 1: 0); } public void setResults(List aRresults) { if(aRresults == null) { aRresults = new ArrayList(); } this.results = aRresults; this.setTotalResults(this.results.size()); } } A SearchCriteria class specific to User Search. package com.sivalabs.javabp; public class UserSearchCriteria extends SearchCriteria { public enum UserSearchType { BY_ID, BY_NAME }; private UserSearchType searchType = UserSearchType.BY_NAME; private int id; private String username; public UserSearchType getSearchType() { return searchType; } public void setSearchType(UserSearchType searchType) { this.searchType = searchType; } public int getId() { return id; } public void setId(int id) { this.id = id; } public String getUsername() { return username; } public void setUsername(String username) { this.username = username; } } A SearchResults class specific to User Search. package com.sivalabs.javabp; import java.text.MessageFormat; public class UserSearchResults extends SearchResults { public static String getDataGridMessage(int start, int end, int total) { return MessageFormat.format("Displaying {0} to {1} Users of {2}", start, end, total); } } UserService takes the SearchCriteria, invokes the DAO and get the results, prepares the UserSearchResults and return it back. package com.sivalabs.javabp; import java.util.ArrayList; import java.util.List; import com.sivalabs.javabp.UserSearchCriteria.UserSearchType; public class UserService { public SearchResults search(UserSearchCriteria searchCriteria) { UserSearchType searchType = searchCriteria.getSearchType(); String sortOrder = searchCriteria.getSortOrder(); System.out.println(searchType+":"+sortOrder); List results = null; if(searchType == UserSearchType.BY_NAME) { //Use hibernate Criteria API to get and sort results based on USERNAME field in sortOrder results = userDAO.searchUsers(...); } else if(searchType == UserSearchType.BY_ID) { //Use hibernate Criteria API to get and sort results based on USER_ID field in sortOrder results = userDAO.searchUsers(...); } UserSearchResults searchResults = new UserSearchResults(); searchResults.setPageSize(searchCriteria.getPageSize()); searchResults.setResults(results); return searchResults; } } package com.sivalabs.javabp; import com.sivalabs.javabp.UserSearchCriteria.UserSearchType; public class TestClient { public static void main(String[] args) { UserSearchCriteria criteria = new UserSearchCriteria(); criteria.setPageSize(3); //criteria.setSearchType(UserSearchType.BY_ID); //criteria.setId(2); criteria.setSearchType(UserSearchType.BY_NAME); criteria.setUsername("s"); UserService userService = new UserService(); SearchResults searchResults = userService.search(criteria); System.out.println(searchResults.getTotalResults()); System.out.println(searchResults.getResults().size()+":"+searchResults.getResults()); System.out.println(searchResults.getResults(1).size()+":"+searchResults.getResults(1)); } } With this approach if we want to add a new criteria like search by EMAIL we can do it as follows: 1. Add BY_EMAIL criteria type to UserSearchType enum 2. Add new property "email" to UserSearchCriteria 3. criteria.setSearchType(UserSearchType.BY_EMAIL); criteria.setEmail("gmail"); 4. In UserService prepare the HibernateCriteria with email filter. Thats it :-) From : http://sivalabs.blogspot.com/2011/02/java-coding-best-practices-better.html

i started to write a new article series about ibatis / mybatis . this is the first article and it will walk you through what is ibatis / mybatis and why you should use it. for those who does not know ibatis / mybatis yet, it is a persistence framework – an alternative to jdbc and hibernate , available for java and .net platforms. i’ve been working with it for almost two years, and i am enjoying it! the first thing you may notice in this and following articles about ibatis/mybatis is that i am using both ibatis and mybatis terms. why? until june 2010, ibatis was under apache license and since then, the framework founders decided to move it to google code and they renamed it to mybatis. the framework is still the same though, it just has a different name now. i gathered some resources, so i am just going to quote them: what is mybatis/ibatis? the mybatis data mapper framework makes it easier to use a relational database with object-oriented applications. mybatis couples objects with stored procedures or sql statements using a xml descriptor. simplicity is the biggest advantage of the mybatis data mapper over object relational mapping tools.to use the mybatis data mapper, you rely on your own objects, xml, and sql. there is little to learn that you don’t already know. with the mybatis data mapper, you have the full power of both sql and stored procedures at your fingertips. ( www.mybatis.org ) ibatis is based on the idea that there is value in relational databases and sql, and that it is a good idea to embrace the industrywide investment in sql. we have experiences whereby the database and even the sql itself have outlived the application source code, and even multiple versions of the source code. in some cases we have seen that an application was rewritten in a different language, but the sql and database remained largely unchanged. it is for such reasons that ibatis does not attempt to hide sql or avoid sql. it is a persistence layer framework that instead embraces sql by making it easier to work with and easier to integrate into modern object-oriented software. these days, there are rumors that databases and sql threaten our object models, but that does not have to be the case. ibatis can help to ensure that it is not. ( ibatis in action book) so… what is ibatis ? a jdbc framework developers write sql, ibatis executes it using jdbc. no more try/catch/finally/try/catch. an sql mapper automatically maps object properties to prepared statement parameters. automatically maps result sets to objects. support for getting rid of n+1 queries. a transaction manager ibatis will provide transaction management for database operations if no other transaction manager is available. ibatis will use external transaction management (spring, ejb cmt, etc.) if available. great integration with spring, but can also be used without spring (the spring folks were early supporters of ibatis). what isn’t ibatis ? an orm does not generate sql does not have a proprietary query language does not know about object identity does not transparently persist objects does not build an object cache essentially, ibatis is a very lightweight persistence solution that gives you most of the semantics of an o/r mapping toolkit, without all the drama. in other words ,ibatis strives to ease the development of data-driven applications by abstracting the low-level details involved in database communication (loading a database driver, obtaining and managing connections, managing transaction semantics, etc.), as well as providing higher-level orm capabilities (automated and configurable mapping of objects to sql calls, data type conversion management, support for static queries as well as dynamic queries based upon an object’s state, mapping of complex joins to complex object graphs, etc.). ibatis simply maps javabeans to sql statements using a very simple xml descriptor. simplicity is the key advantage of ibatis over other frameworks and object relational mapping tools.( http://www.developersbook.com ) who is using ibatis/mybatis? see the list in this link: http://www.apachebookstore.com/confluence/oss/pages/viewpage.action?pageid=25 i think the biggest case is myspace , with millions of users. very nice! this was just an introduction, so in next articles i will show how to create an application using ibatis/mybatis – step-by-step. enjoy! from http://loianegroner.com/2011/02/introduction-to-ibatis-mybatis-an-alternative-to-hibernate-and-jdbc/

We often want to send XML via web services. We may already have the schema or annotated JAXB2 classes configured in our application. What if we want to send the same format via JMS? By default Spring JMS is configured to send & receive objects serialized. How can we switch to using JAXB2 (or any other OXM marshaling strategy)? The following example assumes we are going from annotations first instead of from XML Schema. Quick Overview Annotate Bean with JAXB2 Configure OXM Converter Integration Test Visualize Results Logging Configuration Maven Configuration 1. Annotate Bean with JAXB2 Use JAXB2 annotations for our bean package com.gordondickens.jmswithoxm; import java.math.BigDecimal; import javax.xml.bind.annotation.XmlAccessType; import javax.xml.bind.annotation.XmlAccessorType; import javax.xml.bind.annotation.XmlElement; import javax.xml.bind.annotation.XmlRootElement; @XmlRootElement(name = "account") @XmlAccessorType(XmlAccessType.FIELD) public class Account { @XmlElement(required = true) private String name; @XmlElement private String description; @XmlElement private BigDecimal balance; public String getName() { return name; } public void setName(String name) { this.name = name; } public String getDescription() { return description; } public void setDescription(String description) { this.description = description; } public BigDecimal getBalance() { return balance; } public void setBalance(BigDecimal balance) { this.balance = balance; } @Override public String toString() { return "Account [name=" + name + ", description=" + description + ", balance=" + balance + "]"; } } 2. Configure OXM Converter Define our Marshalers – We see defines JAXB2 as our marshaller for the Account class Register our MarshallingMessageConverter – We register the MarshallingMessageConverter to use the JAXB2 marshaller for both inbound and outbound data Register our Converter – In the JmsTemplate, we register our oxmMessageConverter as the messageConverter. This replaces the default SimpleMessageConverter which will relies on Serialization Notice the ActiveMQ namespace? 3. Integration Test Populate the Account bean Calls convertAndSend on the JmsTemplate to marshal & send the Account Includes a postProcessor callback to log the XML data Calls receiveAndConvert on the JmsTemplate to get & unmarshal the Account Asserts end state ? package com.gordondickens.jmswithoxm; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNotNull; import java.math.BigDecimal; import javax.jms.BytesMessage; import javax.jms.JMSException; import javax.jms.Message; import org.junit.Test; import org.junit.runner.RunWith; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jms.core.JmsTemplate; import org.springframework.jms.core.MessagePostProcessor; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration public class JmsWithOxmTest { private static final Logger logger = LoggerFactory .getLogger(JunitWithOxmTest.class); private static final String TEST_DEST = "oxmTestQueue"; @Autowired JmsTemplate jmsTemplate; @Test public void testSendingMessage() { Account account = generateTestMessage(); jmsTemplate.convertAndSend(TEST_DEST, account, new MessagePostProcessor() { @Override public Message postProcessMessage(Message message) throws JMSException { if (message instanceof BytesMessage) { BytesMessage messageBody = (BytesMessage) message; // message is in write mode, close & reset to start // of byte stream messageBody.reset(); Long length = messageBody.getBodyLength(); logger.debug("***** MESSAGE LENGTH is {} bytes", length); byte[] byteMyMessage = new byte[length.intValue()]; int red = messageBody.readBytes(byteMyMessage); logger.debug( "***** SENDING MESSAGE - \n\n{}\n", new String(byteMyMessage)); } return message; } }); Account account2 = (Account) jmsTemplate.receiveAndConvert(TEST_DEST); assertNotNull("Account MUST return from JMS", account2); assertEquals("Name MUST match", account.getName(), account2.getName()); assertEquals("Description MUST match", account.getDescription(), account2.getDescription()); assertEquals("Balance MUST match", account.getBalance(), account2.getBalance()); } private Account generateTestMessage() { Account account = new Account(); account.setBalance(new BigDecimal(12345.67)); account.setDescription("A no account varmint"); account.setName("Waskally Wabbit"); logger.debug("Generated Test Message: " + account.toString()); return account; } } 4. Visualizing Results Run: mvn clean test See Account XML between the block & Output has been Formatted for clarity ------------------------------------------------------- T E S T S ------------------------------------------------------- Running com.gordondickens.jmswithoxm.JmsWithOxmTest INFO o.s.o.j.Jaxb2Marshaller - Creating JAXBContext with classes to be bound [class com.gordondickens.jmswithoxm.Account] DEBUG c.g.j.JmsWithOxmTest - Generated Test Message: Account [name=Waskally Wabbit, description=A no account varmint, balance=12345.670000000000072759576141834259033203125] DEBUG o.s.j.c.JmsTemplate - Executing callback on JMS Session: ActiveMQSession {id=ID:Technophiliac-61135-1296856347600-2:1:1,started=false} DEBUG c.g.j.JmsWithOxmTest - ***** MESSAGE LENGTH is 213 bytes DEBUG c.g.j.JmsWithOxmTest - ***** SENDING MESSAGE - Waskally Wabbit A no account varmint 12345.670000000000072759576141834259033203125 DEBUG o.s.j.c.JmsTemplate - Sending created message: ActiveMQBytesMessage {commandId = 0, responseRequired = false, messageId = null, originalDestination = null, originalTransactionId = null, producerId = null, destination = null, transactionId = null, expiration = 0, timestamp = 0, arrival = 0, brokerInTime = 0, brokerOutTime = 0, correlationId = null, replyTo = null, persistent = false, type = null, priority = 0, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@b364dcb, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = null, readOnlyProperties = false, readOnlyBody = true, droppable = false} ActiveMQBytesMessage{ bytesOut = null, dataOut = null, dataIn = java.io.DataInputStream@1a2d502d } DEBUG o.s.j.c.JmsTemplate - Executing callback on JMS Session: ActiveMQSession {id=ID:Technophiliac-61135-1296856347600-2:2:1,started=true} Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.276 sec 5. Logging Configuration Logback is very similar to Log4J Logback patterns use Log4J characters also See: Logback Pattern Layout Documentation %-5level %logger{5} - %msg%n 6. Maven Configuration Using Logback to support Log4J, SLF4J, Apache (JCL) & Java Util Logging Included IDE builders for STS/Eclipse & IntelliJ IDEA 4.0.0 com.gordondickens.jmswithoxm spring-jms-oxm 1.0.0.CI-SNAPSHOT jar JMS to OXM Spring http://gordondickens.com Sample JMS with OXM Message Conversion gordon.dickens Gordon Dickens [email protected] Author http://www.gordondickens.com 3.0.5.RELEASE 4.8.1 1.1.1 1.6.1 5.4.2 0.9.27 1.2.16 UTF-8 false quick true junit junit ${junit.version} test log4j log4j ${log4j.version} org.slf4j slf4j-api ${slf4j.version} org.slf4j jcl-over-slf4j ${slf4j.version} org.slf4j jul-to-slf4j ${slf4j.version} ch.qos.logback logback-classic ${logback.version} ch.qos.logback logback-core ${logback.version} ch.qos.logback logback-access ${logback.version} org.springframework spring-core ${spring.version} commons-logging commons-logging org.springframework spring-test ${spring.version} test commons-logging commons-logging org.apache.activemq activemq-core ${activemq.version} org.springframework spring-context commons-logging commons-logging commons-logging commons-logging-api org.apache.xbean xbean-spring 3.7 commons-logging commons-logging org.springframework spring-context ${spring.version} org.springframework spring-jms ${spring.version} org.springframework spring-oxm ${spring.version} javax.xml.bind jaxb-api 2.2.2 org.apache.geronimo.specs geronimo-jms_1.1_spec ${jms.version} org.apache.maven.plugins maven-surefire-plugin 2.7.1 org.apache.maven.plugins maven-eclipse-plugin 2.8 true true 2.0 org.springframework.ide.eclipse.core.springbuilder org.springframework.ide.eclipse.core.springnature org.apache.maven.plugins maven-compiler-plugin 2.3.2 1.6 1.6 org.apache.maven.plugins maven-idea-plugin 2.2 true true true 6. Getting the code Code is in my Git Repo: https://github.com/gordonad/core-spring-demos/tree/master/demos/JmsOxmDemo Further Reading Spring Reference for JMS – Section 21.3.1 Using Message Converters Core Spring Training – Where I teach JMS with Spring Enterprise Integration with Spring Training – Where I teach JMS with Spring From http://gordondickens.com/wordpress/2011/02/07/sending-beans-as-xml-with-jmstemplate/

Here’s a quick guide to creating a RESTful web service with Glassfish using JAX-RS. First create a new maven project called restwebdemo using the jee6-sandbox-archetype so we have a model and some data to work with. To get this working with Glassfish, open the persistence.xml file and change the jta-data-source name to jdbc/__default. Also, make sure that the javaDB is up and running by going to $glassfish_dir/bin and typing asadmin start-database. Verify that the application is working correctly by going to http://localhost:8080/restwebdemo/ and you should get a list of courses. Before we start getting to the interesting stuff, we have one more boring piece of configuration to perform specific to web services. We need to add the jersey servlet container to our web.xml file: Jersey Web Application com.sun.jersey.spi.container.servlet.ServletContainer 1 Jersey Web Application /rest/* This also tells Jersey to handle urls starting with /rest and pass it along to our web service methods. Now we can dive right in an create a new server bean that will respond to requests for web services. For now we’ll just return a simple message from a POJO. @Path("sample") public class SimpleService { @Path("greet") @GET public String doGreet() { return "Hello Stranger, the time is "+ new Date(); } } The path annotation on the class indicates that this is a root resource class and the path value given specifies the base URI for all the web service methods contained in the class. On the doGreet method we have @Path which is used to specify the path template this method should match. The @GET annotation is used to differentiate between a sub-resource method that handles the actual web service request and a sub-resource locator method that returns an object that will instead be used to handle the request. In this case, the method has the @GET annotation which means this method handles the request and returns the result. If you navigate to http://localhost:8080/restwebdemo/rest/sample/greet/ you should see a welcome message with the current date and time. Now we’ll look at adding parameterized web services that extracts parameters from the request URL and uses them to form the output. Add the following method to the web service class: @Path("sayHello/{name}") @GET public String doSayHello(@PathParam("name") String name) { return "Hello there "+name; } Again we have the path annotation to indicate what URLs this method will match, and this time we have have {name} added to the URL. This lets us extract a part of the url and give it a name. This name is used in the @PathParam annotation in the method signature to assign the URL fragment to the name parameter . To test our new method, redeploy the application and go to the URL http://localhost:8080/restwebdemo/rest/sample/sayHello/Andy to get the response Hello there Andy We can also use request parameters to provide values to the method by using the @QueryParam annotation. We’ll create another method that is similar but uses a query parameter instead. @Path("sayHello") @GET public String doSayHelloWithRequestParam(@QueryParam("name") String name) { return "Hi there "+name; } This time, the URL to use is http://localhost:8080/restwebdemo/rest/sample/sayHello?name=Andy to get the same message. To make things more interesting, lets add a new page that lets us enter a name in a form and submit it to the web service. Add a new page called form.html with the following content : Name Go to this page at http://localhost:8080/restwebdemo/form.html, enter your name and click submit and you should be greeted by name in the next page. Notice that we had to set the form method to GET because our web service is only set up to respond to GET requests. If we change the form method to POST we can get the following error message : HTTP Status 405 - Method Not Allowed type Status report message Method Not Allowed description The specified HTTP method is not allowed for the requested resource (Method Not Allowed). Remember, with REST, those actual verbs have meaning and adds meaning to the request so it is strict on how it matches the method to be called. To solve this problem, we can add a new method to handle form POSTs like so : @Path("sayHello") @POST public String doSayHelloWithFormParam(@FormParam("name") String name) { return "Hi there " + name; } Here we changed the @GET to a @POST to allow the different verb and changed the annotation on the name method parameter to @FormParam. The path remains the same because we can have service methods that match the same path, but for different request verbs. We can even have the same verb and path as long as the content type returned is different. The content type is used to specify the type of output the is returned from the method. It is set by adding a @javax.ws.rs.Produces (not to be confused with the CDI Produces annotation). The annotation takes a string parameter that indicates the type of media returned from the method. Common media types are defined as constants in the MediaType class so you can use : @Path("sayHello") @POST @Produces(MediaType.APPLICATION_XML) public String doSayHelloWithFormParam(@FormParam("name") String name) { return "Hi there " + name+""; } If you run your form again, and post it, you will get an xml response as follows : Hi there Andy Depending on your browser, if you return just the text, you will get an error because the plain text isn’t valid XML and the browser expects XML because that is the response type set on the response from the web service. To finish up, we are going to do something a little more interesting, we will create a web service to return the name of a course from the database using the sandbox data built into the archetype. For various reasons, we will take the most direct route to getting data access which is to make the web service bean a stateless bean and inject a persistence context using the @PersistenceContext annotation. Add the @Stateless annotation to the SimpleService class and an entity manager field annotated with @PersistenceContext along with the getters and setters. Add a new method to return the course name for the given course id parameter. We will return it as text for the time being : @Path("courseName/{id}") @GET public String getCourseNameFromId(@PathParam("id") Long id) { Course c = entityManager.find(Course.class, id); if (c == null) { return "Not Found, try the index page and come back"; } else { return c.getTitle(); } } Note that the automatic type conversion takes place and the value is converted to a Long automatically. If the course is not found, we suggest the user goes to the main page of the demo. We aren’t just being overly helpful, the test data is generated when you request one of the application pages for the first time. In the current persistence context, when you redeploy, the database is dropped and rebuilt so it will be empty. You need to go to the front page to automatically create the data and then go back to your page to view the course. An example URL is http://localhost:8080/restwebdemo/rest/sample/courseName/124. Of course you could grab the course object and build your own XML or JSON response to send back to the client, or use a third party library like Jackson to build the JSON response. However, as we’ll see next time, Java EE 6 has all these goodies built in for us, and with a few annotations, we’ll be slinging objects back and forth in no time at all. You can download the source code for the project from here. Simply unzip, build with maven (mvn clean package) and deploy to Glassfish.

I am a Spring framework user for last three years and I have really enjoyed working with Spring. One thing that I am seeing these days is the heavy use of AspectJ in most of SpringSource products. Spring Roo is a RAD tool for Java developers which makes use of AspectJ ITD's for separate compilation units. Spring transaction management and exception translation is also done using Aspectj. There are also numerous other usages of Aspectj in Spring products. In this article, I am going to talk about another cool usage of AspectJ by Spring - Mocking Static Methods. These days most of the developers write unit test cases and it is very common to use any of the mocking libraries like EasyMock, Mockito etc. to mock the external dependencies of a class. Using any of these mocking libraries it is very easy to mock calls to other class instance method. But most of these mocking framework does not provide the facility to mock the calls to static methods. Spring provides you the capability to mock static methods by using Spring Aspects library. In order to use this feature you need to add spring-aspects.jar dependency in your pom.xml. org.springframework spring-aspects 3.0.4.RELEASE Next thing you need to do is to convert your project in to a AspectJ project. If you are using Eclipse or STS(SpringSource Tool Suite) you can do that by right-click your project -> configure -> convert to AspectJ project. STS by default has AspectJ plugin, for eclipse users you need to install AspectJ plugin for above to work. I would recommend using STS for developing Spring based applications. The two aspects and one annotation that is of interest in spring-aspects.jar are : AbstractMethodMockingControl : This is an abstract aspect to enable mocking of methods picked out by a pointcut. All the child aspects need to define mockStaticsTestMethod() and methodToMock() pointcuts. The mockStaticTestMethod() pointcut is used to indicate when mocking should be triggered and methodToMock() pointcut is used to define which method invocations to mock. AnnotationDrivenStaticEntityMockingControl : This is the single implementation of AbstractMethodMockingControl aspect which exists in spring-aspects.jar. This is an annotation-based aspect to use in a test build to enable mocking static methods on Entity classes. In this aspect mockStaticTestMethod() pointcut defines that for classes marked with @MockStaticEntityMethods annotation mocking should be triggered and methodToMock() pointcut defines that all the public static methods in the classes marked with @Entity annotation should be mocked. MockStaticEntityMethods : Annotation to indicate a test class for whose @Test methods static methods on Entity classes should be mocked. The AnnotationDrivenStaticEntityMockingControl provide the facility to mock static methods of any class which is marked with @Entity annotation. But usually we would need to mock static method of classes other than marked with @Entity annotation. The only thing we need to do to make it work is to extend AbstractMethodMockingControl aspect and provide definitions for mockStaticsTestMethod() and methodToMock() pointcuts. For example, lets write an aspect which should mock all the public static methods of classes marked with @Component annotation. package com.shekhar.javalobby; import org.springframework.mock.staticmock.AbstractMethodMockingControl; import org.springframework.stereotype.Component; import org.springframework.mock.staticmock.MockStaticEntityMethods;; public aspect AnnotationDrivenStaticComponentMockingControl extends AbstractMethodMockingControl { public static void playback() { AnnotationDrivenStaticComponentMockingControl.aspectOf().playbackInternal(); } public static void expectReturn(Object retVal) { AnnotationDrivenStaticComponentMockingControl.aspectOf().expectReturnInternal(retVal); } public static void expectThrow(Throwable throwable) { AnnotationDrivenStaticComponentMockingControl.aspectOf().expectThrowInternal(throwable); } protected pointcut mockStaticsTestMethod() : execution(public * (@MockStaticEntityMethods *).*(..)); protected pointcut methodToMock() : execution(public static * (@Component *).*(..)); } The only difference between AnnotationDrivenStaticEntityMockingControl(comes with spring-aspects.jar) and AnnotationDrivenStaticComponentMockingControl(custom that we have written above) is in methodToMock() pointcut. In methodToMock() pointcut we have specified that it should mock all the static methods in any class marked with @Component annotation. Now that we have written the custom aspect lets test it. I have created a simple ExampleService with one static method. This is the method which we want to mock. @Component public class ExampleService implements Service { /** * Reads next record from input */ public String getMessage() { return myName(); } public static String myName() { return "shekhar"; } } This class will return "shekhar" when getMessage() method will be called. Lets test this without mocking package com.shekhar.javalobby; import org.junit.Assert; import org.junit.Test; public class ExampleConfigurationTests { private ExampleService service = new ExampleService(); @Test public void testSimpleProperties() throws Exception { String myName = service.getMessage(); Assert.assertEquals("shekhar", myName); } } This test will work fine. Now let's add mocking to this test class. There are two things that we need to do in our test We need to annotate our test with @MockStaticEntityMethods to indicate that static methods of @Component classes will be mocked. Please note that it is not required to use @MockStaticEntityMethods annotation you can create your own annotation and use that in mockStaticsTestMethod() pointcut. So, I could have created an annotation called @MockStaticComponentMethods and used that in mockStaticsTestMethod() pointcut. But I just reused the @MockStaticEntityMethods annotation. In our test methods we need to first invoke the static method which we want to mock so that it gets recorded. Next we need to set our expectation i.e. what should be returned from the mock and finally we need to call the playback method to stop recording mock calls and enter playback state. To make it more concrete lets apply mocking to the above test import org.junit.Assert; import org.junit.Test; import org.springframework.mock.staticmock.MockStaticEntityMethods; @MockStaticEntityMethods public class ExampleConfigurationTests { private ExampleService service = new ExampleService(); @Test public void testSimpleProperties() throws Exception { ExampleService.myName(); AnnotationDrivenStaticComponentMockingControl.expectReturn("shekhargulati"); AnnotationDrivenStaticComponentMockingControl.playback(); String myName = service.getMessage(); Assert.assertEquals("shekhargulati", myName); } } As you can see we annotated the test class with @MockStaticEntityMethods annotation and in the test method we first recorded the call (ExampleService.myName()), then we set the expectations, then we did the playback and finally called the actual code. In this way you can mock the static method of class.

Almost all .NET developers who have been working in several applications up to date are probably familiar with writing SQL queries for specific needs within the application. Before LINQ as a technology came on scene, my daily programming life was about 60-70% of the day writing code either in the front-end (ASPX, JavaScript, jQuery, HTML/CSS etc…) or in the back-end (C#, VB.NET etc…), and about 30-40% writing SQL queries for specific needs used within the application. Now, when LINQ is there, I feel that the percentage for writing SQL queries got down to about 10% per day. I don’t say it won’t change with time depending what technology I use within the projects or what way would be better, but since I’m writing a lot LINQ code in the latest projects, I thought to see if there is a tool that can automatically translate SQL to LINQ so that I can transfer many queries as a LINQ statements within the code. Linqer is a tool that I have tested in the previous two weeks and I see it works pretty good. Even I’m not using it yet to convert SQL to LINQ code because I did it manually before I discovered that Linqer could have really helped me, I would recommend it for those who are just starting with LINQ and have knowledge of writing SQL queries. Let’s pass through several steps so that I will help you get started faster… 1. Go to http://www.sqltolinq.com/ website and download the version you want. There is a Linqer Version 4.0.1 for .NET 4.0 or Linqer Version 3.5.1 for .NET 3.5. 2. Once you download the zip file, extract it and launch the Linqer4Inst.exe then add install location. In the location you will add, the Linqer.exe will be created. 3. Launch the Linqer.exe. Once you run it for first time, the Linqer Connection Pool will be displayed so that you can create connection to your existing Model Click the Add button Right after this, the following window will appear #1 – The name of the connection string you are creating #2 – Click “…” to construct your connection string using Wizard window #3 – Chose your language, either C# or VB #4 – Model LINQ to SQL or LINQ to Entities Right after you select LINQ to SQL, the options to select the files for the Model will be displayed. In our case I will select LINQ to SQL, and here is the current progress So, you can select existing model from your application or you can Generate LINQ to SQL Files so that the *.dbml and *.designer.cs will be automatically filled #5 – At the end, you can chose your context name of the model which will be used when generating the LINQ code Once you are done, click OK. You will get back to the parent window filled with all needed info and click Close. Note: You can later add additional connections in your Linqer Connections Pool from Tools –> Linqer Connections In the root folder where your Linqer.exe is placed, now you have Linqer.ini file containing the Connection string settings. Ok, now lets go to the interesting part. Lets create one (first) simple SQL query and try to translate it to LINQ statement. SQL Query select * from authors a where a.city = 'Oakland' If we add this query to Linqer, here is the result: So, the LINQ code is similar to the SQL code and is easy to read since it’s simple. Also, if you notice, the tool generates class (you can add class name) with prepared code for using in your project. Perfect! Now, lets try to translate a query with two joined tables (little bit more complex): SQL Query select * from employee left join publishers on employee.pub_id = publishers.pub_id where employee.fname like '%a' The LINQ generated code is: from employee in db.Employee join publishers in db.Publishers on employee.Pub_id equals publishers.Pub_id into publishers_join from publishers in publishers_join.DefaultIfEmpty() where employee.Fname.EndsWith("a") select new { employee.Emp_id, employee.Fname, employee.Minit, employee.Lname, employee.Job_id, employee.Job_lvl, employee.Pub_id, employee.Hire_date, Column1 = publishers.Pub_id, Pub_name = publishers.Pub_name, City = publishers.City, State = publishers.State, Country = publishers.Country } So, if you can notice the where clause, we said in the SQL query: ... like "%a" and the corresponding LINQ code in C# is ... EndsWith("a"); - Excellent! And the Class automatically generated by the tool is public class EmployeePubClass { private String _Emp_id; private String _Fname; private String _Minit; private String _Lname; private Int16? _Job_id; private Byte? _Job_lvl; private String _Pub_id; private DateTime? _Hire_date; private String _Column1; private String _Pub_name; private String _City; private String _State; private String _Country; public EmployeePubClass( String AEmp_id, String AFname, String AMinit, String ALname, Int16? AJob_id, Byte? AJob_lvl, String APub_id, DateTime? AHire_date, String AColumn1, String APub_name, String ACity, String AState, String ACountry) { _Emp_id = AEmp_id; _Fname = AFname; _Minit = AMinit; _Lname = ALname; _Job_id = AJob_id; _Job_lvl = AJob_lvl; _Pub_id = APub_id; _Hire_date = AHire_date; _Column1 = AColumn1; _Pub_name = APub_name; _City = ACity; _State = AState; _Country = ACountry; } public String Emp_id { get { return _Emp_id; } } public String Fname { get { return _Fname; } } public String Minit { get { return _Minit; } } public String Lname { get { return _Lname; } } public Int16? Job_id { get { return _Job_id; } } public Byte? Job_lvl { get { return _Job_lvl; } } public String Pub_id { get { return _Pub_id; } } public DateTime? Hire_date { get { return _Hire_date; } } public String Column1 { get { return _Column1; } } public String Pub_name { get { return _Pub_name; } } public String City { get { return _City; } } public String State { get { return _State; } } public String Country { get { return _Country; } } } public class List: List { public List(Pubs db) { var query = from employee in db.Employee join publishers in db.Publishers on employee.Pub_id equals publishers.Pub_id into publishers_join from publishers in publishers_join.DefaultIfEmpty() where employee.Fname.EndsWith("a") select new { employee.Emp_id, employee.Fname, employee.Minit, employee.Lname, employee.Job_id, employee.Job_lvl, employee.Pub_id, employee.Hire_date, Column1 = publishers.Pub_id, Pub_name = publishers.Pub_name, City = publishers.City, State = publishers.State, Country = publishers.Country }; foreach (var r in query) Add(new EmployeePubClass( r.Emp_id, r.Fname, r.Minit, r.Lname, r.Job_id, r.Job_lvl, r.Pub_id, r.Hire_date, r.Column1, r.Pub_name, r.City, r.State, r.Country)); } } Great! We have ready-to-use class for our application and we don't need to type all this code. Besides this way to generate code, you can in same time use this tool to see the db results I like this tool because mainly it’s very easy to use, lightweight and does the job pretty straight forward. You can try the tool and send me feedback using the comments in this blog post.

Sometimes you have tests where the fixtures can be recyled, instead of being recreated; but that's not all: sometimes, even the state reset we preach for Shared Fixture gets in the way instead of being mandatory. For example, think of testing addition and removal of values over a collection object: to test the removal, you need a collection containing something. But this collection has to be populated somewhere, involving other methods which would have their own independent tests. If you look closer, however, you notice that the collection you need for testing values removal is just like the one that testAddsElements() produces when it succeeds, and simply throws away when the Test Methods returns... The pattern In these cases, the Chained Tests pattern can help you design isolated tests, by expressing the dependencies they implicitly have. The application of this pattern results in a form of Api encapsulation: a test that tests Collection::remove() does not need to use Collection::add(), and won't be influenced by a change in the name or the signature of add(), which it does not focus on. The test just needs just a collection which has been already filled with values. Before "spolverare" unserialization and other strange techniques, just consider using the leftovers from a previous test. Let's suppose we have two tests A and B, where B uses a variable produced in A. Of course, if A fails, the dependent Test Method B cannot (and shouldn't) be run; it will only add noise with an obvious failure which derives not from the behavior really under test there, but just from the fixture setup, which happens to be the previous test. If A instead runs correctly, B can be run. Implementation In PHPUnit, Chained Tests are supported when they are in the same Testcase Class. In this case, the Testcase Objects are not totally isolated anymore: you can return variables and objects from one test, in order for PHPUnit to pass them to other tests, that simply asks for them as dependencies. You can do so by adding @depends annotation to dependent tests. Note that this annotation can also be used without passing fixtures around, just for the sake of not executing complex tests where the base ones have already failed. @depends working by stopping the execution of tests down in the chain when the dependencies fail: they will be marked as skipped and signaled by an S in PHPUnit's ordinary output. The tests which produce a fixture as a leftover must return that fixture at the end of the test. The tests accepting fixtures, tagged with @depends, must accept parameters in the Test Method signatures. In case of multiple parameters to return, you can return an array() and use list() in the dependent method to extract variables from it. There is a known issue in running dependent tests with --filter: dependent tests would not run alone, so if you plan to use filter you may want to pick similar test names for couples of dependent/dependency in order to execute both of them when needed. Example Here we have a sample Testcase Class where the second and third tests are dependent on the first. Long chains of tests are not encouraged as it becomes increasingly difficult to tell what is the state of the fixture in the tests at the end of chain. assertEquals(15, $array[2], 'The third element was not 15.'); return $array; } /** * @depends testArrayAcceptsNewValues */ public function testArrayCanDeleteValues($array) { unset($array[2]); $this->assertEquals(2, count($array), 'There are more than 2 elements in the array.'); } /** * @depends testArrayAcceptsNewValues */ public function testArrayAcceptsDuplicateValues($array) { $array[] = 15; $this->assertEquals(4, count($array), 'There are less than 4 elements in the array.'); } } If we execute the whole test case, we obtain: [10:40:55][giorgio@Desmond:~]$ phpunit txt/articles/chainedtest.php PHPUnit 3.5.5 by Sebastian Bergmann. ... Time: 0 seconds, Memory: 5.00Mb OK (3 tests, 3 assertions) If we put a $this->fail() in the first test, instead, we get: [10:41:40][giorgio@Desmond:~]$ phpunit txt/articles/chainedtest.php PHPUnit 3.5.5 by Sebastian Bergmann. FSS Time: 0 seconds, Memory: 5.00Mb There was 1 failure: 1) ArrayTest::testArrayAcceptsNewValues /home/giorgio/Dropbox/txt/articles/chainedtest.php:13 FAILURES! Tests: 1, Assertions: 1, Failures: 1, Skipped: 2. If we filter the second test, we cannot run it due to the missing dependency: [10:49:25][giorgio@Desmond:~]$ phpunit --filter testArrayCan txt/articles/chainedtest.php PHPUnit 3.5.5 by Sebastian Bergmann. S Time: 0 seconds, Memory: 4.75Mb OK, but incomplete or skipped tests! Tests: 0, Assertions: 0, Skipped: 1. But if we filter the third test, thanks to the well-chosen name, we get: [10:49:16][giorgio@Desmond:~]$ phpunit --filter testArrayAccepts txt/articles/chainedtest.php PHPUnit 3.5.5 by Sebastian Bergmann. .. Time: 0 seconds, Memory: 5.00Mb OK (2 tests, 2 assertions)

Recently, I was writing a piece of code where I need to write a copy factory for a class. A copy factory is a static factory that will construct a copy of the same type as that of the argument passed to the factory. The copy factory will copy the state of the argument object into the new object. This will make sure that the newly constructed object is equal to the old one. A copy constructor is similar to a copy factory with just one difference - a copy constructor can only exist in the class containing the constructor whereas copy factory can exist in any other class as well. For example, // Copy Factory public static Field getNewInstance(Field field) // Copy Constructor public Field Field(Field field) I choose static copy factory because I didn't have the source code for the class With copy factory I can code to an interface instead of a class. The class for which I wanted to write copy factory was similar to the one shown below : public class Field { private String name; private T value; public String getName() { return name; } public void setName(String name) { this.name = name; } public T getValue() { return value; } public void setValue(T value) { this.value = value; } } The first implementation of FieldUtils class that came to my mind was as shown below public static Field copy(Field field) { Field objField = new Field();//1 objField.setName(field.getName());//2 objField.setValue(field.getValue());//3 return objField;//4 } The above code will not compile because of two compilation errors. The first compilation error is at line 1 because you can't create an instance of Field. Field means Field and when you have ? extends Something you can only get values out of it, you can't set values in it except null. For an object to be useful you should be able to do both, so the compiler does not allow you to create an object. The second compilation error will be at line number 3 and you will get a cryptic error message like this The method setValue(capture#3-of ?) in the type Field is not applicable for the arguments (capture#4-of ?) In simple terms this error message is saying that you are trying to set a wrong value in objField. But what if I had written the following method? public static Field copy(Field field) { field.setName(field.getName()); field.setValue(field.getValue()); return field; } Will the above code compile? No. You will again get a similar error message as mentioned above. To fix this error we will write a private helper method which will capture the wildcard and assign it to a Type variable. This technique is called Wildcard Capture. I have read this in the Java Generics and Collection book which is a must-read for understanding Java Generics. Wildcard Capture works by type inference. public static Field copy(Field field) { return copyHelper(field); } private static Field copyHelper(Field field) { Field objField = new Field(); objField.setName(field.getName()); objField.setValue(field.getValue()); return objField ; } Wildcard capture is very useful when you work with wildcards and knowing it will save a lot of your time.

Two tips for the Eclipse Modeling Framework (EMF) 2.2.1: Accessing model’s meta model – accessing EClass/attribute by name – so that you can set an attribute when you only know its name and haven’t its EAttribute How to force EMF to serialize an object as an XML element instead of an XML attribute Excuse my rather liberal and slightly confusing use of the terms model, model instance and meta model. Tip 1: Accessing model’s meta model – accessing EClass/attribute by name Normally you cannot do any operation on a dynamic EMF model instance – such as instantiating an EObject or settings its property via eSet – without the corresponding meta model objects such as EClass and EAttribute. But there is a solution – you can build an ExtendedMetaData instance and use its methods to find the meta model objects based on search criteria such as element name and namespace. Examples Create ExtendedMetaData from a model One way to build a meta data instance is to instantiate BasicExtendedMetaData based on a Registry, containing all registered packages. This is usually available either via ResourceSet.getPackageRegistry() or globally, via EPackage.Registry.INSTANCE. import org.eclipse.emf.ecore.util.*; ... ExtendedMetaData modelMetaData = new BasicExtendedMetaData(myResourceSet.getPackageRegistry()); Get EClass for a namespace and name Provided that your model contains an element with the name Book and namespace http://example.com/book: EClass bookEClass = (EClass) modelMetaData.getType("http://example.com/book", "Book"); Get EClass’ attribute by name Beware: Some properties (such as those described by named complex types) are not represented by an EAttribute but an EReference (both extend EStructuralFeature) and are accessible as the EClass’ elements, not attributes, even though from developer’s points of view they’re attributes of the owning class. Let’s suppose that the book has the attribute name: EStructuralFeature nameAttr = modelMetaData.getElement(bookEClass, null, "name"); The namespace is null because normally attributes/nested elements are not classified with a schema. Here is how you would print the name and namespace URI of an attribute/element: System.out.println("attr: " + modelMetaData.getNamespace(nameAttr) + ":" + nameAttr.getName()); //prints "null:name" Tip 2: How to force EMF to serialize an object as an XML element instead of an XML attribute Normally EMF stores simple Java properties as attributes of the XML element representing the owning class: but you might prefer to have it rather as a nested element: The Book of Songs To achieve that: Enable the save option OPTION_EXTENDED_META_DATA (so that extended meta data such as annotations and an XML map aren’t ignored) Tell EMF that you want this property to be stored as an element By attaching an eAnnotation to it (not shown) By supplying a XML Map with this information upon save To enable the extended metadata: Map saveOptions = new HashMap(); saveOptions.put(XMLResource.OPTION_EXTENDED_META_DATA, Boolean.TRUE); According to some documentation the value should be an implementation of ExtendedMetaData, according to others Boolean.TRUE is the correct choice – I use the latter for it’s easier and works for me. To tell EMF to write a property as an element when serailizing to XML: import org.eclipse.emf.ecore.xmi.impl.*; ... EAttribute bookNameEAttribute = ...; // retrieved e.g. from meta data, see Tip 1 XMLMapImpl map = new XMLMapImpl(); XMLInfoImpl x = new XMLInfoImpl(); x.setXMLRepresentation(XMLInfoImpl.ELEMENT); map.add(bookNameEAttribute, x); saveOptions.put(XMLResource.OPTION_XML_MAP, map); The XMLInfoImpl enables you to customize the namespace, name and representation of the element. When saving you then just supply the save options: EObject target = ...; org.eclipse.emf.ecore.resource.Resource outResource = ...; outResource.getContents().add(target); outResource.save(saveOptions); Reference: the Redbook Eclipse Development using the Graphical Editing Framework and the Eclipse Modeling Framework, page 74, section 2.3.4 From http://theholyjava.wordpress.com/2011/01/11/emf-tips-accessing-model-meta-data-serializing-into-elementattribute/

Troy Giunipero (pictured, right) is a student at the University of Edinburgh studying toward an MSc in Computer Science. Formerly, he was one of the NetBeans Docs writers based in Prague, Czech Republic, where he spent most of his time writing Java web tutorials. In this interview, Troy introduces you to The NetBeans E-commerce Tutorial. This is a very detailed tutorial describing just about everything you need to know when creating an e-commerce web application in Java. It has received a lot of very positive feedback. Let's find out about the background of this tutorial and what Troy learnt in writing it. Hi Troy! During your time on the NetBeans team, you wrote a very large tutorial describing how to create an e-commerce site. How and why did you start writing it? Well, there’s a short answer and a long answer to this. The short answer is that I was lucky to take part in Sun’s SEED (Sun Engineering Enrichment and Development) program. I wanted to focus on technical aspects, so I based my curriculum on developing an e-commerce application using Java technologies. I documented my efforts and applied them toward deliverables for the IDE’s 6.8 and 6.9 releases, resulting in the 13-part NetBeans E-commerce Tutorial. The long answer is that I had previously been tasked with creating an e-commerce application for my degree project (I was studying toward a BSc in IT and Computing), and ran into loads of trouble trying to integrate the various technologies into a cohesive, functioning application. I was coming from a non-technical background and found there was a steep learning curve involved in web development. My work was fraught with problems which I can now attribute to poor time-management, and a lack of good, practical, hands-on learning resources. So in a way, working on the AffableBean project (this is the project used in the NetBeans E-commerce Tutorial) was a way for me to go back and attempt to do the whole thing right. With the tutorial, I had two goals in mind: one, I wanted to consolidate my understanding of everything by writing about it, and two, I wanted to help others avoid the problems and pitfalls that I’d earlier ran into by designing a piece of documentation that puts everything together. Can you run us through the basic parts and what they provide? Certainly. First I want to point out that there’s a live demo application (http://dot.netbeans.org:8080/AffableBean/) which I managed to get up and running with help from Honza Pirek from the NetBeans Web Team (thanks Honza!): The application is modeled on the well-known MVC architecture: The tutorial refers to the above diagram at various points, and covers a bunch of different concepts and technologies along the way, including: Project design and planning (unit 2) Designing a data model (using MySQL WorkBench) (unit 4) Forward-engineering the data model into a database schema (unit 4) Database connectivity (units 3, 4, 6) Servlet, JSP/EL and JSTL technologies (units 3, 5, 6) EJB 3 and JPA 2 technologies (unit 7), and transactional support (unit 9) Session management (i.e., for the shopping cart mechanism) (unit 8) Form validation and data conversion (unit 9) Multilingual support (unit 10) Security (i.e., using form-based authentication and encrypted communication) (unit 11) Load testing with JMeter (unit 12) Monitoring the application with the IDE’s Profiler (unit 12) Tips for deployment to a production server (units 12, 13) Also, the tutorial aims to provide ample background information on the whole “Java specifications” concept, with an introduction to the Java Community Process, how final releases include reference implementations, and how these relate to the tutorial application using the IDE’s bundled GlassFish server (units 1, 7). Finally, the tutorial is as much about the above concepts and technologies as it is about learning to make best use of the IDE. I really tried to squeeze as much IDE-centric information in there as possible. So for example you’ll find: An introduction to the IDE’s main windows and their functions (unit 3) A section dedicated to editor tips and tricks (unit 5), and abundant usage of keyboard shortcuts in steps throughout the tutorial Use of the debugger (unit 8) Special “tip boxes” that discuss IDE functionality that is sometimes difficult to fit into conventional documentation. For example, there are tips on using the IDE’s Template Manager (unit 5), GUI support for database tables (unit 6), Javadoc support (unit 8), and support for code templates (unit 9). Did you learn any new things yourself while writing it? Yes! Three things immediately come to mind: EJB 3 technology. Initially this was a big hurdle for me. Using EJB 3 effectively seems to be something of an art form. If you know what you’re doing and understand exactly how to use the EntityManager to handle persistence operations on a database, EJB lets you do some amazingly smart things with just a few lines of code. But there seems to be a lack of good free documentation online—especially since EJB 3 is a significant departure from EJB 2. Therefore, almost all of the tutorial’s information on EJB comes from the very excellent book, EJB in Action by Debu Panda and Reza Rahman. Interpreting the NetBeans Profiler. The final hands-on unit, Testing an Profiling, was the most difficult for me to write, primarily because I just wasn’t familiar with the Profiler at all. I spent an unhealthy amount of time just watching the Telemetry graph run against my JMeter test plan, which is only slightly more stimulating than watching water come to boil. That being said, I feel that by just examining the graphs and other windows over time, critical logical associations start to jump out at you after a while. Likewise with JMeter. Hopefully unit 12 was able to capture and relay some of these. How to search online for decent articles and learning materials. The old Sun Technical Articles site was a great resource. Many of the links in the See Also sections at the bottom of tutorial units were found by adding site:java.sun.com/developer/technicalArticles/ to a Google search. Also the official forums (found at forums.sun.com) became a good place for questions I couldn’t find ready answers to. I had both the Java EE 5 and 6 Tutorials bookmarked. And Marty Hall’s Core Servlets and JavaServer Pages became an invaluable resource for the first half of the tutorial. What are your personal favorite features of the technologies discussed in the tutorial? I particularly liked learning about session management—using the HttpSession object to carry user-specific data between requests, and working with JSP’s corresponding implicit objects in the front-end pages. Session management is a defining aspect for e-commerce applications, as they need to provide some sort of shopping cart mechanism... ...and so the Managing Sessions unit (unit 8) was a key chapter in the tutorial. It’s extremely useful to be able to suspend the debugger on a portion of code that includes session-scoped variables, then hover the mouse over a given variable in the editor to determine its current value. I used the debugger continuously during this phase, and so I went so far as to incorporate use of the debugger throughout the Managing Sessions unit. What kind of background does someone starting the tutorial need to have? Someone can come to the tutorial with little or absolutely no experience using NetBeans. I’ve tried to be particularly careful in providing clear and easy-to-follow instructions in this respect. But one would be best off having some background or knowledge in basic web technologies, and at least some exposure to relational databases. With this foundation, I think that the topics covered in the second half of the tutorial, like applying entity classes and session beans, language support and security, won’t seem too daunting. I’ve noticed that the vast majority of feedback that comes in relates to the first half of the tutorial, and I sometimes get the impression that people feel they need to follow the tutorial units consecutively. Not so. The units are 90% modular. In other words, if somebody just wants to run through the security unit (unit 11), they can do so by downloading the associated project snapshot, follow the setup instructions, and then just follow along without needing to even look at other parts of the tutorial. What will they be able to do at the end of it? Naturally, anybody who completes individual tutorial units will be able to apply the concepts and technologies to their own work. But anyone who completes the tutorial in its entirety will gain an insight into the development process as a whole, and I think will also get a certain confidence that comes with knowing how “all the pieces fit together”—from gathering customer requirements all the way to deployment of the completed app to a production server. They’ll also have gained a solid familiarity with the NetBeans IDE, and be in a good position to explore popular Java web frameworks that work on top of servlet technology or impose an MVC architecture on their own, such as JSF, Spring, Struts, or Wicket. Do you see any problems in the technologies discussed and what would be your suggestions for enhancements? Well there’s one thing that comes to mind. When I started working on this project, I was studying the Duke’s BookStore example from the Java EE 5 Tutorial. A wonderful example that demonstrates how to progressively implement the same application using various technologies and combinations thereof. So for example you start out with an all-servlet implementation, then move on to a JSP/servlet version. Then there’s a JSTL implementation and ultimately, a version using JavaServer Faces. It’s great learning material, but also terrifically outdated. Right around this time, Sun was gearing up for the big Java EE 6 release (Dec. 2009), and I was also trying to learn about the new upcoming technologies, namely CDI, JSF 2, and EJB 3, for my regular NetBeans documentation work. I was getting the definite sense that JSP and JSTL were slowly being pushed aside—in the case of JavaServer Faces, Facelets templating was the new page authoring technology. So really, the E-commerce Tutorial application has become a sort of EE 5/EE 6 hybrid by combining JSP/JSTL with EJB 3 and JPA 2. Now the problem I see from the perspective of a student trying to learn this stuff from scratch, is that the leap from basic servlet technology to a full-blown JSF/EJB/JPA solution is tremendous, and cannot readily be taught through a single tutorial. Naturally, others may disagree with me here. I’m not sure if there’s a solution other than to compensate by producing a lot of quality learning material that covers lots of different use-cases. I’d suggest that the E-commerce Tutorial puts one in a very advantageous position to begin learning about Java-based frameworks, such as GWT, Spring, and JSF, which is a natural course of action for people looking to get a job with this knowledge. Planning any more parts to the tutorial or a new one? No more parts. The E-commerce Tutorial is done. Upon committing the final installments and changes last November, I rejoiced. However, I’m still actively responding to feedback [the ‘Send Us Your Feedback’ links at the bottom of tutorials] and plan to maintain it indefinitely, so if anyone spots any typos, has questions or comments, recommendations for improvement, etc., please write in! :-)

Clojure provides the get and get-in functions for returning values from a map and the select-keys function for returning a new map of only the specified keys. Clojure doesn't provide a function that returns a list of values; however, it's very easy to create such a function (which I call select-values). Once you have the ability to select-values it becomes very easy to create a function that applies a function to the selected values (which I call apply-values). The select-keys function returns a map containing only the entries of the specified keys. The following (pasted) REPL session shows a few different select-keys behaviors. user=> (select-keys {:a 1 :b 2} [:a]) {:a 1} user=> (select-keys {:a 1 :b 2} [:a :b]) {:b 2, :a 1} user=> (select-keys {:a 1 :b 2} [:a :b :c]) {:b 2, :a 1} user=> (select-keys {:a 1 :b 2} []) {} user=> (select-keys {:a 1 :b 2} nil) {} The select-keys function is helpful in many occassions; however, sometimes you only care about selecting the values of certain keys in a map. A simple solution is to call select-keys and then vals. Below you can find the results of applying this idea. user=> (def select-values (comp vals select-keys)) #'user/select-values user=> (select-values {:a 1 :b 2} [:a]) (1) user=> (select-values {:a 1 :b 2} [:a :b]) (2 1) user=> (select-values {:a 1 :b 2} [:a :b :c]) (2 1) user=> (select-values {:a 1 :b 2} []) nil user=> (select-values {:a 1 :b 2} nil) nil The select-values implementation from above may be sufficient for what you are doing, but there are two things worth noticing: in cases where you might be expecting an empty list you are seeing nil; and, the values are not in the same order that the keys were specified in. Given that (standard) maps are unsorted, you can't be sure of the ordering the values. (side-note: If you are concerned with microseconds, it's also been reported that select-keys is a bit slow/garbage heavy.) An alternative definition of select-values uses the reduce function and pulls the values by key and incrementally builds the (vector) result. user=> (defn select-values [map ks] (reduce #(conj %1 (map %2)) [] ks)) #'user/select-values user=> (select-values {:a 1 :b 2} [:a]) [1] user=> (select-values {:a 1 :b 2} [:a :b]) [1 2] user=> (select-values {:a 1 :b 2} [:a :b :c]) [1 2 nil] user=> (select-values {:a 1 :b 2} []) [] user=> (select-values {:a 1 :b 2} nil) [] The new select-values function returns the values in order and returns an empty vector in the cases where previous examples returned nil, but we have a new problem: Keys specified that don't exist in the map are now included in the vector as nil. This issue is easily addressed by adding a call to the remove function. The implementation that includes removing nils can be found below. user=> (defn select-values [map ks] (remove nil? (reduce #(conj %1 (map %2)) [] ks))) #'user/select-values user=> (select-values {:a 1 :b 2} [:a]) (1) user=> (select-values {:a 1 :b 2} [:a :b]) (1 2) user=> (select-values {:a 1 :b 2} [:a :b :c]) (1 2) user=> (select-values {:a 1 :b 2} []) () user=> (select-values {:a 1 :b 2} nil) () There is no "correct" implementation for select-values. If you don't care about ordering and nil is a reasonable return value: the first implementation is the correct choice due to it's concise definition. If you do care about ordering and performance: the second implementation might be the right choice. If you want something that follows the principle of least surprise: the third implementation is probably the right choice. You'll have to decide what's best for your context. In fact, here's a few more implementations that might be better based on your context. user=> (defn select-values [m ks] (map #({:a 1 :b 2} %) ks)) #'user/select-values user=> (select-values {:a 1 :b 2} [:a]) (1) user=> (select-values {:a 1 :b 2} [:a :b :c]) (1 2 nil) user=> (defn select-values [m ks] (reduce #(if-let [v ({:a 1 :b 2} %2)] (conj %1 v) %1) [] ks)) #'user/select-values user=> (select-values {:a 1 :b 2} [:a]) [1] user=> (select-values {:a 1 :b 2} [:a :b :c]) [1 2] Pulling values from a map is helpful, but it's generally not the end goal. If you find yourself pulling values from a map, it's likely that you're going to want to apply a function to the extracted values. With that in mind, I generally define an apply-values function that returns the result of applying a function to the values returned from specified keys. A good example of this is returning the total for a line item represented as a map. Given a map that specifies a line item costing $5 and having a quantity of 4, you can use (* price quantity) to determine the total price for the line item. Using our previously defined select-values function we can do the work ourselves, as the example below shows. user=> (let [[price quantity] (select-values {:price 5 :quantity 4 :upc 1123} [:price :quantity])] (* price quantity)) 20 The example above works perfectly well; however, applying a function to the values of a map seems like a fairly generic operation that can easily be extracted to it's own function (the apply-values function). The example below shows the definition and usage of my definition of apply-values. user=> (defn apply-values [map f & ks] (apply f (select-values map ks))) #'user/apply-values user=> (apply-values {:price 5 :quantity 4 :upc 1123} * :price :quantity) 20 I find select-keys, select-values, & apply-values to be helpful when writing Clojure applications. If you find you need these functions, feel free to use them in your own code. However, you'll probably want to check the comments - I'm sure someone with more Clojure experience than I have will provide superior implementations. From http://blog.jayfields.com/2011/01/clojure-select-keys-select-values-and.html

I read "Domain Object Dependency Injection with Spring" the other day. Actually, what interested me were those comments about the concepts and patterns around "Domain Object". I had been always confused about those buzzwords related to Domain Object: Domain Object, Business Object, DTO, POJO, Anemic Domain Object, Active Record Pattern. After some research, I found several points critical to help understand those concepts (at least for myself): Among those concepts, POJO is not that relevant. It's a general principal applied to all enterprise beans (service beans, dao beans, etc.) not to be framework-specifc, though framework-specific annotations are commonly used in 'POJO' these days. Domain Object == Business Object. They are entity representitives in business layer, which shoud be understood by non-programming people, such as business analysts. Business Object should have both data and behaviour. Anemic Domain Object are those business objects only have data without behaviour. Data Transfer Object (DTO) is a concept within data access layer. DTOs are often used in conjunction with DAOs to represent data retrieved from a DB. DTOs and Domain Objects can be designed to use the same class, even represented by the same object for the same entity. Personally, I think Domain Objects refer to the combination of DTO and Business Objects for plenty of cases. Active Record Pattern: Domain objects have rich behaviour, for example, inject DAO into domain objects to make it has CRUD ability. Understanding the above concepts is just half way, I would like to going further for things behind these domain object variants: On the one hand, people like Martin Fowler thinks domain object should have data and behaviour, which is consistent with OOD. Anemic Domain Object is anti-pattern, which in effect causes 'procedure script' style thick service layer. On the other hand, anemic domain objects are still popular. For myself, the following two facts explain the phenomenon: Active Record Pattern is really painful. I don't like the idea of putting DAO into Domain Object at all. And the clear multi-layer design, makes coding more intuitive and very easy to maintain. Encapsulating behaviour into business object doesn't mean removing service layer at all. Just don't put all behaviour into service layer, this layer should be as thin as possible. What I learned: I will put behaviour only relevant to the individual entity into domain object, for example: validation, calculation, maybe CRUD (I still prefer having separate DAO to fufill the tasks, don't like any huge object) Service layer should serve as a gateway to all above layer, which means any business atomic operation should find its place in service layer. Then, transaction can be managed in one layer. Behaviour involves different entities regardless same type of entity or different types may go into service layer. This is my personal opinion based on what I understand so far. Any comments or critisim are welcome. PS. This is my first post:) Wish it good luck, and Merry Christmas to everyone. More to say: I've read the "Domain Driven Design" refcardz after writing this article, found out some similar points were already mentioned there, though DDD covers broad aspects. I think my article exposes some facts or problems of this topic, which would help people obtain deep insides via further reading. Actually, I am planning to read some DDD book as the second session after current looking up.

Introduction In recent years, I have experienced the same developer's need again and again. The need for improved persistence support. After lots of years of experience with Java, I have grown tired with all the solutions that are "standard", "J2EE compliant", but in the end, just ever so complicated. I don't deny, there are many good ideas around, that have eventually brought up excellent tools, such as Hibernate, JPA/EJB3, iBatis, etc. But all of those tools seem to go to a single direction without giving up any of that thought: Object-relational Mapping. So you end up using a performant database that cost's 100k+$ of license every year just to abstract it with a "standard" persistence layer. I wanted to go a different direction. And take the best of OR-Mapping (code generation, type safety, object oriented query construction, SQL dialect abstraction, etc) without denying the fact, that beneath, I'm running an RDBMS. That's right. R like Relational. Read on about how jOOQ (Java Object Oriented Querying) succeeds in bringing the "relational to the object" Abstract Many companies and software projects seem to implement one of the following two approaches to interfacing Java with SQL The very basic approach: Using JDBC directly or adding a home-grown abstraction on top of it. There is a lot of manual work associated with the creation, maintenance, and extension of the data layer code base. Developers can easily use the full functionality of the underlying database, but will always operate on a very low level, concatenating Strings all over the place. The very sophisticated approach: There is a lot of configuration and a steep learning curve associated with the introduction of sophisticated database abstraction layers, such as the ones created by Hibernate, JPA, iBatis, or even plain old EJB entity beans. While the generated objects and API's may allow for easy manipulation of data, the setup and maintenance of the abstraction layer may become very complex. Besides, these abstraction layers provide so much abstraction on top of SQL, that SQL-experienced developers have to rethink. A different paradigm I tried to find a new solution addressing many issues that I think most developers face every day. With jOOQ - Java Object Oriented Querying, I want to embrace the following paradigm: SQL is a good thing. Many things can be expressed quite nicely in SQL. The relational data model is a good thing. It should not be abstracted by OR-Mapping SQL has a structure and syntax. It should not be expressed using "low-level" String concatenation. Variable binding tends to be very complex when dealing with major queries. POJO's (or data transfer objects) are great when writing Java code manipulating database data. POJO's are a pain to write and maintain manually. Source code generation is the way to go The database comes first. Then the code on top of it. Yes, you do have stored procedures and user defined types (UDT's) in your legacy database. Your database-tool should support that. I think that these key ideas are useful for a very specific type of developer. That specific developer interfaces Java with huge legacy databases. knows SQL well and wants to use it extensively. doesn't want to learn any new language (HQL, JPQL, etc) doesn't want to spend one minute fine-tuning some sophisticated XML-configuration. wants little abstraction over SQL, because his software is tightly coupled with his database. Something that I think the guys at Hibernate or JPA seem to have ignored. needs a strong but light-weight library for database access. For instance to develop for mobile devices. How does jOOQ fit in this paradigm? Not only does jOOQ completely address the above paradigm, it does so quite elegantly. Let's say you have this database that models your bookstore. And you need to run a query selecting all books by authors born after 1920. You know how to do this in SQL: -- Select all books by authors born after 1920, named "Paulo" from a catalogue: SELECT * FROM t_author a JOIN t_book b ON a.id = b.author_id WHERE a.year_of_birth > 1920 AND a.first_name = 'Paulo' ORDER BY b.title The same query expressed with jOOQ-Objects // Instanciate your factory using a JDBC connection // and specify the SQL dialect you're using. Of course you can // have several factories in your application. Factory create = new Factory(connection, SQLDialect.MYSQL); // Create the query using generated, type-safe objects. You could // write even less code than that with static imports! SelectQuery q = create.selectQuery(); q.addFrom(TAuthor.T_AUTHOR); q.addJoin(TBook.T_BOOK, TAuthor.ID, TBook.AUTHOR_ID); // Note how you do not need to worry about variable binding. // jOOQ does that for you, dynamically q.addCompareCondition(TAuthor.YEAR_OF_BIRTH, 1920, Comparator.GREATER); // The AND operator and EQUALS comparator are implicit here q.addCompareCondition(TAuthor.FIRST_NAME, "Paulo"); q.addOrderBy(TBook.TITLE); The jOOQ query object model uses generated classes, such as TAuthor or TBook. Like many other code generation tools do, jOOQ will generate static final objects for the fields contained in each table. In this case, TAuthor holds a member called TAuthor.T_AUTHOR to represent the table itself, and members such as TAuthor.ID, TAuthor.YEAR_OF_BIRTH, etc to hold the table's fields. But you could also use the jOOQ DSL API to stay closer to SQL // Do it all "on one line". SelectQuery q = create.select() .from(T_AUTHOR) .join(T_BOOK).on(TAuthor.ID.equal(TBook.AUTHOR_ID)) .where(TAuthor.YEAR_OF_BIRTH.greaterThan(1920) .and(TAuthor.FIRST_NAME.equal("Paulo"))) .orderBy(TBook.TITLE).getQuery(); jOOQ ships with a DSL (Domain Specific Language) somewhat similar to Linq that facilitates query creation. The strength of DSL becomes obvious when you are using jOOQ constructs such as the decode function: // Create a case statement. Unfortunately "case" is a reserved word in Java // Hence the method is called DECODE after its related Oracle function Field nationality = create.decode() .when(TAuthor.FIRST_NAME.equal("Paulo"), "brazilian") .when(TAuthor.FIRST_NAME.equal("George"), "english") .otherwise("unknown"); // "else" is also a reserved word ;-) The above will render this SQL code: CASE WHEN T_AUTHOR.FIRST_NAME = 'Paulo' THEN 'brazilian' WHEN T_AUTHOR.FIRST_NAME = 'George' THEN 'english' ELSE 'unknown' END Use the DSL API when: You want your Java code to look like SQL You want your IDE to help you with auto-completion (you will not be able to write select .. order by .. where .. join or any of that stuff) Use the regular API when: You want to create your query step-by-step, creating query parts one-by-one You need to assemble your query from various places, passing the query around, adding new conditions and joins on the way In any case, all API's will construct the same underlying implementation object, and in many cases, you can combine the two approaches Once you have established the query, execute it and fetch results // Execute the query and fetch the results q.execute(); Result result = q.getResult(); // Result is Iterable, so you can loop over the resulting records like this: for (Record record : result) { // Type safety assured with generics String firstName = record.getValue(TAuthor.FIRST_NAME); String lastName = record.getValue(TAuthor.LAST_NAME); String title = record.getValue(TBook.TITLE); Integer publishedIn = record.getValue(TBook.PUBLISHED_IN); System.out.println(title + " (published in " + publishedIn + ") by " + firstName + " " + lastName); } Or simply write for (Record record : q.fetch()) { // [...] } Fetch data from a single table and use jOOQ as a simple OR-Mapper // Similar query, but don't join books to authors. // Note the generic record type that is added to your query: SimpleSelectQuery q = create.select(T_AUTHOR) .where(TAuthor.YEAR_OF_BIRTH.greaterThan(1920) .and(TAuthor.FIRST_NAME.equal("Paulo"))) .orderBy(TAuthor.LAST_NAME).getQuery(); // When executing this query, also Result holds a generic type: q.execute(); Result result = q.getResult(); for (TAuthorRecord record : result) { // With generate record classes, you can use generated getters and setters: String firstName = record.getFirstName(); String lastName = record.getLastName(); System.out.println("Author : " + firstName + " " + lastName + " wrote : "); // Use generated foreign key navigation methods for (TBookRecord book : record.getTBooks()) { System.out.println(" Book : " + book.getTitle()); } } jOOQ not only generates code to model your schema, but it also generates domain model classes to represent tuples in your schema. In the above example, you can see how selecting from the TAuthor.T_AUTHOR table will produce results containing well-defined TAuthorRecord types. These types hold getters and setters like any POJO, but also some more advanced OR-code, such as foreign key navigator methods like // Return all books for an author that are obtained through the // T_AUTHOR.ID = T_BOOK.AUTHOR_ID foreign key relationship public List getTBooks() Now, for true OR-mapping, you would probably prefer mature and established frameworks such as Hibernate or iBATIS. Don't panic. Better integration with Hibernate and JPA is on the feature roadmap. The goals of jOOQ should not be to reimplement things that are already well-done, but to bring true SQL to Java Execute CRUD operations with jOOQ as an OR-mapper // Create a new record and insert it into the database TBookRecord book = create.newRecord(T_BOOK); book.setTitle("My first book"); book.store(); // Update it with new values book.setPublishedIn(2010); book.store(); // Delete it book.delete(); Nothing new in the OR-mapping world. These ideas have been around since EJB entity beans or even before. It's still quite useful for simple purposes. Execute CRUD operations the way you're used to You don't need to go into that OR-mapping business. You can create your own INSERT, "INSERT SELECT", UPDATE, DELETE queries. Some examples: InsertQuery i = create.insertQuery(T_AUTHOR); i.addValue(TAuthor.FIRST_NAME, "Hermann"); i.addValue(TAuthor.LAST_NAME, "Hesse"); i.execute(); UpdateQuery u = create.updateQuery(T_AUTHOR); u.addValue(TAuthor.FIRST_NAME, "Hermie"); u.addCompareCondition(TAuthor.LAST_NAME.equal("Hesse")); u.execute(); // etc... Now for the advanced stuff Many tools can do similar stuff as what we have seen before. Especially Hibernate and JPA have a feature called criteria query, that provides all of the type-safety and query object building using DSL's while being based on a solid (but blown-up) underlying architecture. An important goal for jOOQ is to provide you with all (or at least: most) SQL features that you are missing in other frameworks but that you would like to use because you think SQL is a great thing but JDBC is too primitive for the year 2010, 2011, or whatever year we're in, when you're reading this. So, jOOQ comes along with aliasing, nested selects, unions and many other SQL features. Check out the following sections: Aliasing That's a very important feature. How could you have self-joins or in/exists clauses without aliasing? Let's say we have a "T_TREE" table with fields "ID", "PARENT_ID", and "NAME". If we want to find all parent/child NAME couples, we will need to execute a self-join on T_TREE. In SQL, this reads: SELECT parent.NAME parent_name, child.NAME child_name FROM T_TREE parent JOIN T_TREE child ON (parent.ID = child.PARENT_ID) No problem for jOOQ. We'll write: // Create table aliases Table parent = TTree.T_TREE.as("parent"); Table child = TTree.T_TREE.as("child"); // Create field aliases from aliased table Field parentName = parent.getField(TTree.NAME).as("parent_name"); Field childName = child.getField(TTree.NAME).as("child_name"); // Execute the above select Record record = create.select(parentName, childName) .from(parent) .join(child).on(parent.getField(TTree.ID).equal(child.getField(TTree.PARENT_ID))) .fetchAny(); // The aliased fields can be read from the record as in the simpler examples: record.getValue(parentName); Functionally, it is easy to see how this works. Look out for future releases of jOOQ for improvements in the DSL support of field and table aliasing IN clause The org.jooq.Field class provides many methods to construct conditions. In previous examples, we have seen how to create regular compare conditions with = < <= >= > != operators. Now Field also has a couple of methods to create IN conditions: // Create IN conditions with constant values that are bound to the // query via JDBC's '?' bind variable placeholders Condition in(T... values); Condition in(Collection values); Condition notIn(T... values); Condition notIn(Collection values); // Create IN conditions with a sub-select Condition in(QueryProvider query) Condition notIn(QueryProvider query) The constant set of values for IN conditions is an obvious feature. But the sub-select is quite nice: -- Select authors with books that are sold out SELECT * FROM T_AUTHOR WHERE T_AUTHOR.ID IN (SELECT DISTINCT T_BOOK.AUTHOR_ID FROM T_BOOK WHERE T_BOOK.STATUS = 'SOLD OUT'); In jOOQ, this translates to create.select(T_AUTHOR) .where (TAuthor.ID.in(create.selectDistinct(TBook.AUTHOR_ID) .from(T_BOOK) .where(TBook.STATUS.equal(TBookStatus.SOLD_OUT)))); EXISTS clause Very similar statements can be expressed with the EXISTS clause. The above set of authors could also be obtained with this statement: -- Select authors with books that are sold out SELECT * FROM T_AUTHOR a WHERE EXISTS (SELECT 1 FROM T_BOOK WHERE T_BOOK.STATUS = 'SOLD OUT' AND T_BOOK.AUTHOR_ID = a.ID); In jOOQ (as of version 1.5.0), this translates to // Alias the author table Table a = T_AUTHOR.as("a"); // Use the aliased table in the select statement create.selectFrom(a) .where(create.exists(create.select(create.constant(1)) .from(T_BOOK) .where(TBook.STATUS.equal(TBookStatus.SOLD_OUT) .and(TBook.AUTHOR_ID.equal(a.getField(TAuthor.ID)))))); UNION clauses SQL knows of four types of "UNION operators": UNION UNION ALL EXCEPT INTERSECT All of these operators are supported by all types of select queries. So in order to write things like: SELECT TITLE FROM T_BOOK WHERE PUBLISHED_IN > 1945 UNION SELECT TITLE FROM T_BOOK WHERE AUTHOR_ID = 1 You can write the following jOOQ logic: create.select(TBook.TITLE).from(T_BOOK).where(TBook.PUBLISHED_IN.greaterThan(1945)).union( create.select(TBook.TITLE).from(T_BOOK).where(TBook.AUTHOR_ID.equal(1))); Of course, you can then again nest the union query in another one (but be careful to correctly use aliases): -- alias_38173 is an example of a generated alias, -- generated by jOOQ for union queries SELECT alias_38173.TITLE FROM ( SELECT T_BOOK.TITLE, T_BOOK.AUTHOR_ID FROM T_BOOK WHERE T_BOOK.PUBLISHED_IN > 1945 UNION SELECT T_BOOK.TITLE, T_BOOK.AUTHOR_ID FROM T_BOOK WHERE T_BOOK.AUTHOR_ID = 1 ) alias_38173 ORDER BY alias_38173.AUTHOR_ID DESC In jOOQ: Select union = create.select(TBook.TITLE, TBook.AUTHOR_ID).from(T_BOOK).where(TBook.PUBLISHED_IN.greaterThan(1945)).union( create.select(TBook.TITLE, TBook.AUTHOR_ID).from(T_BOOK).where(TBook.AUTHOR_ID.equal(1))); create.select(union.getField(TBook.TITLE)) .from(union) .orderBy(union.getField(TBook.AUTHOR_ID).descending()); Note that a UNION query will automatically generate an alias if you use it as a nested table. In order to nest this query correctly, you need to get the aliased field from the query as seen in the example abov. Other, non-standard SQL features See more examples about stored procedures, UDT's, enums, etc on https://sourceforge.net/apps/trac/jooq/wiki/Examples Summary jOOQ brings the relational world to Java without trying to cover up its origins. jOOQ is relational. And object oriented. Just in a different way. Try it for yourself and I would be very glad for any feedback you may have. Find jOOQ on http://jooq.sourceforge.net Cheers Lukas Eder

Does MySQL work with Entity Framework 4.0? The answer is: yes, it works! I just put up one experimental project to play with MySQL and Entity Framework 4.0 and in this posting I will show you how to get MySQL data to EF. Also I will give some suggestions how to deploy your applications to hosting and cloud environments. MySQL stuff As you may guess you need MySQL running somewhere. I have MySQL installed to my development machine so I can also develop stuff when I’m offline. The other thing you need is MySQL Connector for .NET Framework. Currently there is available development version of MySQL Connector/NET 6.3.5 that supports Visual Studio 2010. Before you start download MySQL and Connector/NET: MySQL Community Server Connector/NET 6.3.5 If you are not big fan of phpMyAdmin then you can try out free desktop client for MySQL – HeidiSQL. I am using it and I am really happy with this program. NB! If you just put up MySQL then create also database with couple of table there. To use all features of Entity Framework 4.0 I suggest you to use InnoDB or other engine that has support for foreign keys. Connecting MySQL to Entity Framework 4.0 Now create simple console project using Visual Studio 2010 and go through the following steps. 1. Add new ADO.NET Entity Data Model to your project. For model insert the name that is informative and that you are able later recognize. Now you can choose how you want to create your model. Select “Generate from database” and click OK. 2. Set up database connection Change data connection and select MySQL Database as data source. You may also need to set provider – there is only one choice. Select it if data provider combo shows empty value. Click OK and insert connection information you are asked about. Don’t forget to click test connection button to see if your connection data is okay. If everything works then click OK. 3. Insert context name Now you should see the following dialog. Insert your data model name for application configuration file and click OK. Click next button. 4. Select tables for model Now you can select tables and views your classes are based on. I have small database with events data. Uncheck the checkbox “Include foreign key columns in the model” – it is damn annoying to get them away from model later. Also insert informative and easy to remember name for your model. Click finish button. 5. Define your classes Now it’s time to define your classes. Here you can see what Entity Framework generated for you. Relations were detected automatically – that’s why we needed foreign keys. The names of classes and their members are not nice yet. After some modifications my class model looks like on the following diagram. Note that I removed attendees navigation property from person class. Now my classes look nice and they follow conventions I am using when naming classes and their members. NB! Don’t forget to see properties of classes (properties windows) and modify their set names if set names contain numbers (I changed set name for Entity from Entity1 to Entities). 6. Let’s test! Now let’s write simple testing program to see if MySQL data runs through Entity Framework 4.0 as expected. My program looks for events where I attended. using(var context = new MySqlEntities()) { var myEvents = from e in context.Events from a in e.Attendees where a.Person.FirstName == "Gunnar" && a.Person.LastName == "Peipman" select e; Console.WriteLine("My events: "); foreach(var e in myEvents) { Console.WriteLine(e.Title); } } Console.ReadKey(); And when I run it I get the result shown on screenshot on right. I checked out from database and these results are correct. At first run connector seems to work slow but this is only the effect of first run. As connector is loaded to memory by Entity Framework it works fast from this point on. Now let’s see what we have to do to get our program work in hosting and cloud environments where MySQL connector is not installed. Deploying application to hosting and cloud environments If your hosting or cloud environment has no MySQL connector installed you have to provide MySQL connector assemblies with your project. Add the following assemblies to your project’s bin folder and include them to your project (otherwise they are not packaged by WebDeploy and Azure tools): MySQL.Data MySQL.Data.Entity MySQL.Web You can also add references to these assemblies and mark references as local so these assemblies are copied to binary folder of your application. If you have references to these assemblies then you don’t have to include them to your project from bin folder. Also add the following block to your application configuration file. ... ... Conclusion It was not hard to get MySQL connector installed and MySQL connected to Entity Framework 4.0. To use full power of Entity Framework we used InnoDB engine because it supports foreign keys. It was also easy to query our model. To get our project online we needed some easy modifications to our project and configuration files.

As promised I am going to provide an article on how we can use Sphinx with Java to perform a full text search. I will begin the article with an introduction to Sphinx. Introduction to Sphinx Databases are continually growing and sometimes tend to hold about 100M records and need an external solution for full text search to be performed. I have picked Sphinx, an open source full-text search engine, distributed under GPL version 2 to perform a full text search on such a huge amount of data. Generally, it's a standalone search engine meant to provide fast, size-efficient and relevant full-text search functions to other applications very much compatible with an SQL Database. So my example will be based on the MySQL database, as we cannot produce millions of data to evaluate the real power of Sphinx, we will have a small amount of data and I think that should not be a problem. Here are few Sphinx Unique Features: high indexing speed (up to 10 MB/sec on modern CPUs) high search speed (avg query is under 0.1 sec on 2-4 GB text collections) high scalability (up to 100 GB of text, upto 100 M documents on a single CPU) provides distributed searching capabilities provides searching from within MySQL through pluggable storage engine supports boolean, phrase, and word proximity queries supports multiple full-text fields per document (upto 32 by default) supports multiple additional attributes per document (ie. groups, timestamps, etc) supports MySQL natively (MyISAM and InnoDB tables are both supported) The important features which have been adopted to perform a full text search are the provision of the Java API to integrate easily with the web application and considerably high indexing and searching speed with an average of 4-10 MB/sec & 20-30 ms/q @5GB,3.5M docs(wikipedia) Sphinx Terms & How It Works The fist principle part of sphinx is indexer. It is solely responsible for gathering the data that will be searchable. From the Sphinx point of view, the data it indexes is a set of structured documents, each of which has the same set of fields. This is biased towards SQL, where each row corresponds to a document, and each column to a field. Sphinx builds a special data structure optimized for our queries from the data provided. This structure is called index; and the process of building index from data is called indexing and the element of sphinx which carries out these tasks is called indexer. Indexer can be executed either from a regular script or command-line interface. Sphinx documents are equal to records in DB. Document is set of text fields and number attributes + unique ID – similar to row in DB Set of fields and attributes is constant for index – similar to table in DB Fields are searchable for FullText queries Attributes may be used for filtering, sorting, grouping searchd is the second principle tools as part of Sphinx. It is the part of the system which actually handles searches; it functions as a server and is responsible for receiving queries, processing them and returning a dataset back to the different APIs for client applications. Unlike indexer, searchd is not designed to be run either from a regular script or command-line calling, but instead either as a daemon to be called from init.d (on Unix/Linux type systems) or to be called as a service (on Windows-type systems). I am going to focus on Windows environment so later I will show you how we can install sphinx on windows as a service. Finally search is one of the helper tools within the Sphinx package. Whereas searchd is responsible for searches in a server-type environment, search is aimed at testing the index quickly without building a framework to make the connection to the server and process its response. This will only be used for testing sphinx from command – line and with respect to application’s requirement; searchd service will be used to query the MySql Server with a pre created index. Installation on Windows So now we come to the part of installing Sphinx on Windows: Download Sphinx from the official Sphinx download site i.e http://sphinxsearch.com (I downloaded Win32 release binaries with MySQL support: sphinx-0.9.9-win32.zip) Unzip the file to some folder, I unzipped to C:\devel\sphinx-0.9.9-win32 and added the bin directory to the windows path variable Well Sphinx is installed. Nice, simple, easy. Later I will tell how to set up indexes and search. Sample Application Till now I guess the whole motto of this article is clear to you, let's move ahead to define our sample application. We all use the Address Book to search for people by using their name or e-mail address when we want to immediately address an e-mail message to a specific person, people, or distribution list. We also search for people by using other basic information, such as e-mail alias, office location, and telephone number etc. I think most of the people on this planet are quire familiar with this kind of search, so let's make outlook address book as our sample database schema. Most of the fields are mapped from microsoft outlook, the only additional column is date of joining so that we can filter our queries based on joining dates of the employees. The example that I am going to put forth will use Sphinx to search for a particluar address entry using free text search, meaning the user is free to type in anything, here is our search screen, the DOJ (date of joining) search parameter is optional. The screen is self explanatory, let's move ahead and define our database. As Sphinx works well with MySQL and MySQL being free also, lets create our db scripts around mysql database (Those who wish to install MySQL can dowload it from http://www.mysql.com) Let's create our sample database 'addressbook' mysql> create database addressbook; Query OK, 1 row affected (0.03 sec) mysql> use addressbook; Database changed Note: The fields defined in the following tables are for the purpose of learning only and may not contain a complete set of fields that microsoft address book or any similar software may provide. mysql> CREATE TABLE addressbook ( Id int(11) NOT NULL, FirstName varchar(30) NOT NULL, LastName varchar(30) NOT NULL, OfficeId int(11) DEFAULT NULL, Title varchar(20) DEFAULT NULL, Alias varchar(20) NOT NULL, Email varchar(50) NOT NULL, DOJ date NOT NULL, PhoneNo varchar(20) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8; mysql> CREATE TABLE CompanyLocations ( Id int(11) NOT NULL, Location varchar(60) NOT NULL, Country varchar(20) NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8; It's time to put some dummy data into the table, so let's fill our tables. Our virtual company 'gogs.it' has six offices across India and Singapore as defined in the following insert script. mysql> insert into CompanyLocations (Id, Location, Country) VALUES (1, 'Tower One, Harbour Front, Singapore', 'SG'); insert into CompanyLocations (Id, Location, Country) VALUES (2, 'DLF Phase 3, Gurgaon, India', 'IN'); insert into CompanyLocations (Id, Location, Country) VALUES (3, 'Hiranandani Gardens, Powai, Mumbai, India', 'IN'); insert into CompanyLocations (Id, Location, Country) VALUES (4, 'Hinjwadi, Pune, India', 'IN'); insert into CompanyLocations (Id, Location, Country) VALUES (5, 'Toll Post, Nagrota, Jammu, India', 'IN'); insert into CompanyLocations (Id, Location, Country) VALUES (6, 'Bani (Kathua), India', 'IN'); Now comes the real stuff... The data sphinx is going to index, let's populate that as well...wooooo mysql> INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (1,'Aabheer','Kumar',1,'Mr','u534','[email protected]','2008-9-3', '+911234599990'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (2,'Aadarsh','Gupta',6,'Mr','u668','[email protected]','2007-2-23','+911234599991'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (3,'Aachman','Singh',5,'Mr','u2766','[email protected]','2006-12-18','+911234599992'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (4,'Aadesh','Shrivastav',5,'Mr','u3198','[email protected]','2007-11-23','+911234599993'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (5,'Aadi','manav',1,'Mr','u2686','[email protected]','2010-7-20','+911234599994'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (6,'Aadidev','singh',4,'Mr','u572','[email protected]','2010-8-18','+911234599995'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (7,'Aafreen','sheikh',4,'Smt','u1092','[email protected]','2007-7-11','+911234599996'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (8,'Aakar','Sherpa',5,'Mr','u1420','[email protected]','2009-10-3','+911234599997'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (9,'Aakash','Singh',4,'Mrs','u2884','[email protected]','2008-6-11','+911234599998'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (10,'Aalap','Singhania',4,'Mrs','u609','[email protected]','2010-10-8','+911234599999'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (11,'Aandaleeb','mahajan',1,'Smt','u131','[email protected]','2010-10-21','+911234580001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (12,'Mamata','kumari',5,'Sh','u2519','[email protected]','2009-6-12','+911234580002'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (13,'Mamta','sharma',6,'Smt','u4123','[email protected]','2009-2-8','+911234580003'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (14,'Manali','singh',6,'Mr','u1078','[email protected]','2008-6-14','+911234580004'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (15,'Manda','saxena',1,'Mrs','u196','[email protected]','2010-9-4','+911234580005'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (16,'Salila','shetty',3,'Miss','u157','[email protected]','2009-11-15','+911234580006'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (17,'Salima','happy',3,'Mrs','u3445','[email protected]','2006-7-14','+911234580007'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (18,'Salma','haik',5,'Sh','u4621','[email protected]','2008-6-23','+911234580008'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (19,'Samita','patil',3,'Smt','u3156','[email protected]','2006-6-7','+911234580009'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (20,'Sameena','sheikh',5,'Mrs','u952','[email protected]','2008-8-13','+911234580010'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (21,'Ranita','gupta',5,'Mrs','u2664','[email protected]','2008-10-20','+911234580011'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (22,'Ranjana','sharma',1,'Sh','u3085','[email protected]','2010-6-21','+911234580012'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (23,'Ranjini','singh',6,'Mrs','u4200','[email protected]','2007-4-13','+911234580013'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (24,'Ranjita','vyapari',2,'Smt','u1109','[email protected]','2008-1-22','+911234580014'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (25,'Rashi','gupta',6,'Mrs','u3492','[email protected]','2006-2-2','+911234580015'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (26,'Rashmi','sehgal',3,'Mr','u3248','[email protected]','2008-9-9','+911234580016'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (27,'Rashmika','sexy',1,'Mrs','u4599','[email protected]','2009-3-12','+911234580017'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (28,'Rasika','dulari',3,'Smt','u2089','[email protected]','2009-1-24','+911234580018'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (29,'Dilber','lover',6,'Mr','u4241','[email protected]','2007-10-11','+911234580019'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (30,'Dilshad','happy',1,'Mr','u1564','[email protected]','2007-4-8','+911234580020'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (31,'Dipali','lights',5,'Sh','u1127','[email protected]','2006-11-1','+911234580021'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (32,'Dipika','lamp',1,'Sh','u2271','[email protected]','2010-12-17','+911234580022'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (33,'Dipti','brightness',5,'Smt','u422','[email protected]','2010-9-25','+911234580023'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (34,'Disha','singh',3,'Sh','u4604','[email protected]','2006-5-2','+911234580024'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (35,'Maadhav','Krishna',1,'Miss','u2561','[email protected]','2007-11-6','+911234580025'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (36,'Maagh','month',5,'Miss','u874','[email protected]','2008-5-8','+911234580026'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (37,'Maahir','Skilled',4,'Mr','u3372','[email protected]','2007-8-4','+911234580027'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (38,'Maalolan','Ahobilam',5,'Mrs','u3498','[email protected]','2007-7-9','+911234580028'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (39,'Maandhata','King',1,'Smt','u2089','[email protected]','2009-9-3','+911234580029'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (40,'Maaran','Brave',2,'Miss','u4020','[email protected]','2008-4-5','+9112345606001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (41,'Maari','Rain',2,'Sh','u3593','[email protected]','2007-12-5','+9112345606002'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (42,'Madan','Cupid',4,'Mrs','u795','[email protected]','2007-11-11','+9112345606003'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (43,'Madangopal','Krishna',3,'Sh','u438','[email protected]','2007-2-19','+9112345606004'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (44,'sahil','gogna',1,'Sh','u2273','[email protected]','2007-10-7','+9112345606005'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (45,'nikhil','gogna',2,'Mr','u1240','[email protected]','2009-9-14','+9112345606006'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (46,'amit','gogna',5,'Sh','u3879','[email protected]','2006-2-8','+9112345606007'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (47,'krishan','gogna',4,'Miss','u3632','[email protected]','2010-9-20','+9112345606008'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (48,'anil','kashyap',4,'Smt','u3939','[email protected]','2010-3-15','+9112345606009'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (49,'sunil','kashyap',5,'Mrs','u3493','[email protected]','2008-3-16','+9112345606010'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (50,'sandy','singh',6,'Mrs','u4691','[email protected]','2009-6-2','+9112345606011'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (51,'vishal','kapoor',3,'Mr','u1087','[email protected]','2010-5-13','+9112345606012'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (52,'bala','ji',5,'Mrs','u4762','[email protected]','2007-8-9','+9112345606013'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (53,'karan','sarin',4,'Miss','u3030','[email protected]','2008-4-8','+9112345606014'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (54,'abhishek','kumar',4,'Miss','u1093','[email protected]','2008-12-21','+9112345605001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (55,'babu','the',1,'Miss','u1055','[email protected]','2008-7-2','+9112345506001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (56,'sandeep','gainda',3,'Miss','u1320','[email protected]','2010-5-14','+9112345606301'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (57,'dheeraj','kumar',3,'Miss','u3685','[email protected]','2007-10-14','+9112345606091'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (58,'dharmendra','chauhan',1,'Smt','u3235','[email protected]','2008-8-1','+9112345806001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (59,'max','alan',3,'Smt','u3465','[email protected]','2009-5-5','+9112345608011'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (60,'hidayat','khan',3,'Smt','u958','[email protected]','2007-11-18','+911234599101'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (61,'himnashu','singh',4,'Miss','u2027','[email protected]','2008-3-2','+911234599102'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (62,'dinesh','kumar',6,'Sh','u3233','[email protected]','2008-5-9','+911234599103'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (63,'toshi','prakash',1,'Mr','u3766','[email protected]','2010-9-17','+911234599104'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (64,'niti','puri',3,'Mr','u3575','[email protected]','2009-11-15','+911234599105'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (65,'pawan','tikki',3,'Sh','u3919','[email protected]','2006-3-19','+911234599106'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (66,'gaurav','sharma',2,'Sh','u413','[email protected]','2010-4-2','+911234599107'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (67,'himanshu','verma',2,'Mrs','u4732','[email protected]','2009-3-20','+911234599108'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (68,'priyanshu','verma',3,'Sh','u183','[email protected]','2010-8-12','+911234599109'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (69,'nitika','luthra',2,'Mrs','u4259','[email protected]','2010-7-12','+911234599110'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (70,'neeru','gogna',2,'Sh','u1633','[email protected]','2010-6-23','+91532110000'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (71,'bindu','gupta',1,'Sh','u1859','[email protected]','2006-11-10','+91532110001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (72,'gurleen','bakshi',5,'Miss','u1423','[email protected]','2007-7-1','+91532110003'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (73,'rahul','gupta',3,'Sh','u1223','[email protected]','2009-8-11','+91532110004'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (74,'jagdish','salgotra',3,'Mr','u12','[email protected]','2008-5-19','+91532110005'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (75,'vikas','sharma',3,'Smt','u465','[email protected]','2006-6-2','+91532110006'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (76,'poonam','mahendra',2,'Sh','u1744','[email protected]','2009-12-2','+91532110007'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (77,'pooja','kulkarni',3,'Mrs','u1903','[email protected]','2008-10-6','+91532110008'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (78,'priya','mahajan',6,'Sh','u4205','[email protected]','2010-8-5','+91532110009'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (79,'manoj','zerger',1,'Mrs','u3369','[email protected]','2009-12-4','+91532110010'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (80,'mohan','master',5,'Mr','u2841','[email protected]','2010-10-7','+91532110011'); Please note that above employee data is just a data *only data* I created using a small java programme using random number generators and reading some names file, so you may find titles getting messed up :( We next create a procedure that we will use from java to fetch records that we just inserted. DROP PROCEDURE IF EXISTS search_address_book; CREATE PROCEDURE search_address_book(IN address_ids VARCHAR(1000) ) BEGIN DECLARE search_address_query VARCHAR(2000) DEFAULT ''; SET address_ids = CONCAT('\'', REPLACE(address_ids, ',', '\',\''), '\''); SET search_address_query = CONCAT(search_address_query, ' select ab.Id as Id , ab.FirstName as FName, ab.LastName as LName, cl.Location as Location, ab.Title as Title, ab.Alias as Alias, ab.Email as Email, ab.DOJ as DOJ, ab.PhoneNo as PhoneNo ' ); SET search_address_query = CONCAT(search_address_query, ' from AddressBook ab left join CompanyLocations cl on ab.OfficeId=cl.Id '); SET search_address_query = CONCAT(search_address_query, ' where ab.id IN (', address_ids ,') '); SET @statement = search_address_query; PREPARE dynquery FROM @statement; EXECUTE dynquery; DEALLOCATE PREPARE dynquery; END; # To get records for ids 1, 6 and 7, we run following commands: call search_address_book('1,6,7'); Configuring Sphinx It turns out that it was not terribly difficult to setup sphinx, but I had a hard time finding instructions on the web, so I'll post my steps here. By default Sphinx looks for 'sphinx.co.in' configuration file to come with indexes and other stuff, lets create and define source and index for our sample application addressbook.conf (read between the lines) ############################################################################# ## data source definition ############################################################################# source addressBookSource { ## SQL settings for 'mysql' ## type = mysql # some straightforward parameters for SQL source types sql_host = localhost sql_user = root sql_pass = root sql_db = addressbook sql_port = 3306 # optional, default is 3306 # pre-query, executed before the main fetch query sql_query_pre = SET NAMES utf8 # main document fetch query, integer document ID field MUST be the first selected column sql_query = \ select ab.Id as Id , ab.FirstName as FName, ab.LastName as LName, cl.Location as Location, \ ab.Title as Title, ab.Alias as Alias, ab.Email as Email, UNIX_TIMESTAMP(ab.DOJ) as DOJ, ab.PhoneNo as PhoneNo \ from AddressBook ab left join CompanyLocations cl on ab.OfficeId=cl.Id sql_attr_timestamp = DOJ # document info query, ONLY for CLI search (ie. testing and debugging) , optional, default is empty must contain $id macro and must fetch the document by that id sql_query_info = SELECT * FROM AddressBook WHERE id=$id } ############################################################################# ## index definition ############################################################################# # local index example, this is an index which is stored locally in the filesystem index addressBookIndex { # document source(s) to index source = addressBookSource # index files path and file name, without extension, make sure you have this folder path = C:\devel\sphinx-0.9.9-win32\data\addressBookIndex # document attribute values (docinfo) storage mode docinfo = extern # memory locking for cached data (.spa and .spi), to prevent swapping mlock = 0 morphology = none # make sure this file exists exceptions =C:\devel\sphinx-0.9.9-win32\data\exceptions.txt enable_star = 1 } ############################################################################# ## indexer settings ############################################################################# indexer { # memory limit, in bytes, kiloytes (16384K) or megabytes (256M) # optional, default is 32M, max is 2047M, recommended is 256M to 1024M mem_limit = 32M # maximum IO calls per second (for I/O throttling) # optional, default is 0 (unlimited) # # max_iops = 40 # maximum IO call size, bytes (for I/O throttling) # optional, default is 0 (unlimited) # # max_iosize = 1048576 # maximum xmlpipe2 field length, bytes # optional, default is 2M # # max_xmlpipe2_field = 4M # write buffer size, bytes # several (currently up to 4) buffers will be allocated # write buffers are allocated in addition to mem_limit # optional, default is 1M # # write_buffer = 1M } ############################################################################# ## searchd settings ############################################################################# searchd { # hostname, port, or hostname:port, or /unix/socket/path to listen on listen = 9312 # log file, searchd run info is logged here # optional, default is 'searchd.log' log = C:\devel\sphinx-0.9.9-win32\data\log\searchd.log # query log file, all search queries are logged here # optional, default is empty (do not log queries) query_log = C:\devel\sphinx-0.9.9-win32\data\log\query.log # client read timeout, seconds # optional, default is 5 read_timeout = 5 # request timeout, seconds # optional, default is 5 minutes client_timeout = 300 # maximum amount of children to fork (concurrent searches to run) # optional, default is 0 (unlimited) max_children = 30 # PID file, searchd process ID file name # mandatory pid_file = C:\devel\sphinx-0.9.9-win32\data\log\searchd.pid # max amount of matches the daemon ever keeps in RAM, per-index # WARNING, THERE'S ALSO PER-QUERY LIMIT, SEE SetLimits() API CALL # default is 1000 (just like Google) max_matches = 1000 # seamless rotate, prevents rotate stalls if precaching huge datasets # optional, default is 1 seamless_rotate = 1 # whether to forcibly preopen all indexes on startup # optional, default is 0 (do not preopen) preopen_indexes = 0 } # --eof-- Once the configuration is done, its time to index our sql data, the command to use is 'indexer' as shown below. C:\devel\sphinx-0.9.9-win32\bin>indexer.exe --all --config C:\devel\sphinx-0.9.9-win32\addressbook.conf CONSOLE: Sphinx 0.9.9-release (r2117) Copyright (c) 2001-2009, Andrew Aksyonoff using config file 'C:\devel\sphinx-0.9.9-win32\addressbook.conf'... indexing index 'addressBookIndex'... collected 80 docs, 0.0 MB sorted 0.0 Mhits, 100.0% done total 80 docs, 5514 bytes total 0.057 sec, 96386 bytes/sec, 1398.43 docs/sec total 2 reads, 0.000 sec, 3.5 kb/call avg, 0.0 msec/call avg total 7 writes, 0.000 sec, 2.5 kb/call avg, 0.0 msec/call avg Note: As I told earlier that Sphinx creates 1 document for each row, as we had 80 rows in the database so a total of 80 docs are created. Time taken is also very very small, believe me I tried with half million rows and it took around 3-4 seconds :) cool isn't it? Once the index is up let's try to search few records, the utility command to perform search is 'search'. Ok Sphinx maharaj* please search for employee whose alias is u4732 C:\devel\sphinx-0.9.9-win32\bin>search.exe --config C:\devel\sphinx-0.9.9-win32\addressbook.conf u4732 CONSOLE: Sphinx 0.9.9-release (r2117) Copyright (c) 2001-2009, Andrew Aksyonoff using config file 'C:\devel\sphinx-0.9.9-win32\addressbook.conf'... index 'addressBookIndex': query 'u4732 ': returned 1 matches of 1 total in 0.001 sec displaying matches: 1. document=67, weight=1, doj=Fri Mar 20 00:00:00 2009 Id=67 FirstName=himanshu LastName=verma OfficeId=2 Title=Mrs Alias=u4732 [email protected] DOJ=2009-03-20 PhoneNo=+911234599108 words: 1. 'u4732': 1 documents, 1 hits words: 1. 'u4732': 1 documents, 1 hits As you can see above this is a unique record for Himanshu. Note: You see a lot of information for the result, this is because of following line in our configuration file sql_query_info = SELECT * FROM AddressBook WHERE id=$id If you want to see less columns you need to change the sql_query_info in configuration file. Let's try another search, sphinx maharaj* please tell me which all rows have gurleen or toshi in them. C:\devel\sphinx-0.9.9-win32\bin>search.exe --config C:\devel\sphinx-0.9.9-win32\addressbook.conf --any toshi gurleen CONSOLE: displaying matches: 1. document=63, weight=2, doj=Fri Sep 17 00:00:00 2010 Id=63 FirstName=toshi LastName=prakash OfficeId=1 Title=Mr Alias=u3766 [email protected] DOJ=2010-09-17 PhoneNo=+911234599104 2. document=72, weight=2, doj=Sun Jul 01 00:00:00 2007 Id=72 FirstName=gurleen LastName=bakshi OfficeId=5 Title=Miss Alias=u1423 [email protected] DOJ=2007-07-01 PhoneNo=+91532110003 Exactly two records were returned and this is what we were expecting. The following special operators and modifiers can be used when using the extended matching mode: operator OR: nikhil | sahil operator NOT: hello -sandy hello !sandy field search operator: @Email [email protected] For a complete set of search features , I advise you to go through http://sphinxsearch.com/docs/manual-0.9.9.html#searching link. Sphinx as Windows Service Now our main aim is to use sphinx with JAVA API, so let's move towards that now, before java can utilize the true power of Sphinx, we need to start 'searchd' as a windows service so that our java programme can connect to sphinx search engine. Let's install Sphinx as a windows service so that our java program can use this daemon service to query the index that we just created, the command is : C:\devel\sphinx-0.9.9-win32\bin>searchd.exe --install --config C:\devel\sphinx-0.9.9-win32\addressbook.conf --servicename --port 9312 SphinxSearch CONSOLE: Sphinx 0.9.9-release (r2117) Copyright (c) 2001-2009, Andrew Aksyonoff Installing service... Service 'SphinxSearch' installed succesfully. Well now the sphinx is ready to serve us on port 9312 Note: If you try to install Sphinx without admin rights, you may get following error messages. C:\devel\sphinx-0.9.9-win32\bin>searchd.exe --install --config C:\devel\sphinx-0.9.9-win32\addressbook.conf --servicename --port 9312 SphinxSearch CONSOLE: Installing service... FATAL: OpenSCManager() failed: code=5, error=Access is denied. Once done you can start the service as: c:\>sc start SphinxSearch (or alternatively from the services screen, start 'services.msc' in windows Run) If some how you want to delete the service , use c:\>sc delete SphinxSearch Let's create an adapter to fetch data from the database. package it.gogs.sphinx.util; import it.gogs.sphinx.AddressBoook; import it.gogs.sphinx.exception.AddressBookBizException; import it.gogs.sphinx.exception.AddressBookTechnicalException; import java.sql.CallableStatement; import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.SQLException; import java.util.ArrayList; import java.util.List; import org.apache.log4j.Logger; /** * Adapter to fetch data from the database. * * @author Munish Gogna * */ public class AddressBookAdapter { private static Logger logger = Logger.getLogger(AddressBookAdapter.class); private AddressBookAdapter() { // use in static way.. } private static Connection getConnection() throws AddressBookTechnicalException { String userName = "root"; String password = "root"; String url = "jdbc:mysql://localhost/addressbook"; try { Class.forName("com.mysql.jdbc.Driver").newInstance(); return DriverManager.getConnection(url, userName, password); } catch (Exception e) { throw new AddressBookTechnicalException("could not get connection"); } } public static List getAddressBookList(List addressIds) throws AddressBookTechnicalException, AddressBookBizException { List addressBoookList = new ArrayList(); if (addressIds == null || addressIds.size() == 0){ logger.error("AddressIds was null or empty, returning empty list"); return addressBoookList; } Connection connection = null; CallableStatement callableStatement = null; try { connection = getConnection(); callableStatement = connection.prepareCall("{ call search_address_book(?)}"); callableStatement.setString(1, Utils.toCommaString(addressIds)); callableStatement.execute(); ResultSet resultSet = callableStatement.getResultSet(); prepareResults(resultSet, addressBoookList); connection.close(); } catch (SQLException e) { logger.error("Problem connecting MYSQL - " + e.getMessage()); throw new AddressBookTechnicalException(e.getMessage()); } catch (AddressBookTechnicalException e) { logger.error("Problem connecting MYSQL - " + e.getMessage()); throw e; } finally{ if(connection != null){ try { connection.close(); } catch (SQLException e) { logger.error("Problem closing conection - " + e.getMessage()); e.printStackTrace(); } } } return addressBoookList; } private static void prepareResults(ResultSet resultSet, List addressBoookList) throws SQLException { AddressBoook addressBoook; while (resultSet.next()) { addressBoook = new AddressBoook(); addressBoook.setAlias(resultSet.getString("Alias")); addressBoook.setEmail(resultSet.getString("Email")); addressBoook.setfName(resultSet.getString("FName")); addressBoook.setlName(resultSet.getString("LName")); addressBoook.setOfficeLocation(resultSet.getString("Location")); addressBoook.setPhoneNo(resultSet.getString("PhoneNo")); addressBoook.setTitle(resultSet.getString("Title")); addressBoook.setDateOfJoining(resultSet.getDate("DOJ")); addressBoook.setId(resultSet.getLong("Id")); addressBoookList.add(addressBoook); } } } Next we create the SphinxInstance that will parse the keywords and date range and provide us a list of Ids that matches the search. package it.gogs.sphinx.util; import it.gogs.sphinx.DateRange; import it.gogs.sphinx.SearchCriteria; import it.gogs.sphinx.api.SphinxClient; import it.gogs.sphinx.api.SphinxException; import it.gogs.sphinx.api.SphinxMatch; import it.gogs.sphinx.api.SphinxResult; import it.gogs.sphinx.exception.AddressBookBizException; import java.util.ArrayList; import java.util.Date; import java.util.List; import org.apache.log4j.Logger; /** * Instance that will parse our free text and provide the results. * * Note: Make sure that 'searchd' is up and running before you use this class * @author Munish Gogna * */ public class SphinxInstance { private static String SPHINX_HOST = "localhost"; private static String SPHINX_INDEX = "addressBookIndex"; private static int SPHINX_PORT = 9312; private static SphinxClient sphinxClient; private static Logger logger = Logger.getLogger(SphinxInstance.class); static { sphinxClient = new SphinxClient(SPHINX_HOST, SPHINX_PORT); } public static List getAddressBookIds(SearchCriteria criteria) throws AddressBookBizException, SphinxException { List addressIdsList = new ArrayList(); try { if (Utils.isNull(criteria)) { logger.error("criteria is null"); throw new AddressBookBizException("criteria is null"); } if (Utils.isNull(criteria.getKeywords())) { logger.error("keyword is a required field"); throw new AddressBookBizException("keyword is a required field"); } DateRange dateRange = criteria.getDateRage(); if (!Utils.isNull(dateRange)) { if (Utils.isDateRangeValid(dateRange)) { // this is to filter results based on joining dates if they are provided sphinxClient.SetFilterRange("DOJ", getTimeInSeconds(dateRange.getFromDate()), getTimeInSeconds(dateRange.getToDate()), false); } else { logger.error(" fromDate/toDate should not be empty and 'fromDate' should be less than equal to 'toDate'"); throw new AddressBookBizException("fromDate/toDate should not be empty and 'fromDate' should be less than equal to 'toDate'"); } } sphinxClient.SetMatchMode(SphinxClient.SPH_MATCH_EXTENDED2); sphinxClient.SetSortMode(SphinxClient.SPH_SORT_RELEVANCE, ""); SphinxResult result = sphinxClient.Query(buildSearchQuery(criteria), SPHINX_INDEX, "buidling query for address book search"); SphinxMatch[] matches = result.matches; for (SphinxMatch match : matches) { addressIdsList.add(String.valueOf(match.docId)); } } catch (SphinxException e) { throw e; } catch (AddressBookBizException e) { throw e; } logger.info("Total record(s):" + addressIdsList.size()); return addressIdsList; } private static long getTimeInSeconds(Date time) { return time.getTime()/1000; } private static String buildSearchQuery(SearchCriteria criteria) throws AddressBookBizException { String keywords[] = criteria.getKeywords().split(" "); StringBuilder searchFor = new StringBuilder(); for (String key : keywords) { if (!Utils.isEmpty(key)) { searchFor.append(key); if (searchFor.length() > 1) { searchFor.append("*|*"); } } } searchFor.delete(searchFor.lastIndexOf("|*"), searchFor.length()); StringBuilder queryBuilder = new StringBuilder(); String query = searchFor.toString(); queryBuilder.append("@FName *" + query + " | "); queryBuilder.append("@LName *" + query + " | "); queryBuilder.append("@Title *" + query + " | "); queryBuilder.append("@Location *"+ query + " | "); queryBuilder.append("@Alias *" + query + " | "); queryBuilder.append("@Email *" + query + " | "); queryBuilder.append("@PhoneNo *" + query); logger.info("Sphinx Query: " + queryBuilder.toString()); return queryBuilder.toString(); } } Here is the interface that I will expose to the outside world (in my future article I will expose this interface as Web Service) import it.gogs.sphinx.AddressBoook; import it.gogs.sphinx.SearchCriteria; import it.gogs.sphinx.api.SphinxException; import it.gogs.sphinx.exception.AddressBookBizException; import it.gogs.sphinx.exception.AddressBookTechnicalException; import java.util.List; /** * * @author Munish Gogna * */ public interface AddressBook { /** * Returns the list of AddressBook objects based on search criteria. * * @param criteria * @throws AddressBookTechnicalException * @throws AddressBookBizException * @throws SphinxException */ public List getAddressBookList(SearchCriteria criteria) throws AddressBookTechnicalException, AddressBookBizException, SphinxException; } and here is the implementation class for the same. package it.gogs.sphinx.addressbook.impl; import java.util.List; import it.gogs.sphinx.AddressBoook; import it.gogs.sphinx.SearchCriteria; import it.gogs.sphinx.addressbook.AddressBook; import it.gogs.sphinx.api.SphinxException; import it.gogs.sphinx.exception.AddressBookBizException; import it.gogs.sphinx.exception.AddressBookTechnicalException; import it.gogs.sphinx.util.AddressBookAdapter; import it.gogs.sphinx.util.SphinxInstance; /** * Implementation for our Address Book example * * @author Munish Gogna * */ public class AddressBookImpl implements AddressBook{ public List getAddressBookList(SearchCriteria criteria) throws AddressBookTechnicalException, AddressBookBizException, SphinxException { List addressIds= SphinxInstance.getAddressBookIds(criteria); return AddressBookAdapter.getAddressBookList(addressIds); } } ok so far so good, let's run some tests now ............ package it.gogs.sphinx.test; import java.util.Calendar; import java.util.GregorianCalendar; import java.util.List; import it.gogs.sphinx.AddressBoook; import it.gogs.sphinx.DateRange; import it.gogs.sphinx.SearchCriteria; import it.gogs.sphinx.addressbook.AddressBook; import it.gogs.sphinx.addressbook.impl.AddressBookImpl; import it.gogs.sphinx.api.SphinxException; import it.gogs.sphinx.exception.AddressBookBizException; import it.gogs.sphinx.exception.AddressBookTechnicalException; import junit.framework.TestCase; /** * * @author Munish Gogna * */ public class AddressBookTest extends TestCase { private AddressBook addressBook; @Override protected void setUp() throws Exception { super.setUp(); addressBook = new AddressBookImpl(); } @Override protected void tearDown() throws Exception { super.tearDown(); } /** this should be a unique record for Himanshu */ public void test_search_for_himanshu() throws Exception { SearchCriteria criteria = new SearchCriteria(); // remember the first 'search' example?? criteria.setKeywords("u4732"); List addressList = addressBook.getAddressBookList(criteria); assertTrue(addressList.size() == 1); assertTrue("expecting himanshu here", "himanshu".equals(addressList.get(0).getfName())); } /** only two employees have name gurleen or toshi */ public void test_search_for_gurleen_or_toshi() throws Exception { SearchCriteria criteria = new SearchCriteria(); // remember the second 'search' example?? criteria.setKeywords("gurleen toshi"); List addressList = addressBook.getAddressBookList(criteria); assertTrue(addressList.size() == 2); assertTrue("expecting toshi here", "toshi".equals(addressList.get(0).getfName())); assertTrue("expecting gurleen here", "gurleen".equals(addressList.get(1).getfName())); } /** there are 16 people from jammu location */ public void test_search_for_people_from_jammu_location() throws Exception { SearchCriteria criteria = new SearchCriteria(); criteria.setKeywords("jammu"); List addressList = addressBook.getAddressBookList(criteria); assertTrue(addressList.size() == 16); } /** only Aalap, Manda and nitika are having title as Mrs and joined in 2010 */ public void test_joined_in_2010_with_title_Mrs() throws Exception { DateRange dateRange = new DateRange(); GregorianCalendar calendar1 = new GregorianCalendar(); calendar1.set(Calendar.YEAR, 2010); calendar1.set(Calendar.MONTH, Calendar.JANUARY); calendar1.set(Calendar.DAY_OF_MONTH, 1); dateRange.setFromDate(calendar1.getTime()); GregorianCalendar calendar2 = new GregorianCalendar(); calendar2.set(Calendar.YEAR, 2010); calendar2.set(Calendar.MONTH, Calendar.DECEMBER); calendar2.set(Calendar.DAY_OF_MONTH, 31); dateRange.setToDate(calendar2.getTime()); SearchCriteria criteria = new SearchCriteria(); criteria.setKeywords("Mrs"); criteria.setDateRage(dateRange); List addressList = addressBook.getAddressBookList(criteria); assertTrue("expecting 3 records here", addressList.size() == 3); } /** should get a business exception here */ public void test_without_specifying_keywords(){ SearchCriteria criteria = new SearchCriteria(); //criteria.setKeywords("Mrs"); try { addressBook.getAddressBookList(criteria); } catch (Exception e) { assertTrue(e instanceof AddressBookBizException); assertTrue(e.getMessage().indexOf("keyword is a required field") >-1); } } } How we update the Index once database changes? For these kinds of requirements, we can set up two sources and two indexes, with one "main" index for the data which only changes rarely (if ever), and one "delta" for the new documents. First Time data will go in the "main" index and the newly inserted address book entries will go into "delta". Delta index could then be reindexed very frequently, and the documents can be made available to search in a matter of minutes. Also one thing to take from this article is once 'searchd' daemon is running we can't index the data in normal way,we have to use --rotate option in such cases. For some applications where there is a timely batch update for the data, we can configure some cron job to reindex our documents in Sphinx as shown below. C:\devel\sphinx-0.9.9-win32\bin>indexer.exe --all --config C:\devel\sphinx-0.9.9-win32\addressbook.conf --rotate Capsule We asked Sphinx to provide us the Document Ids corresponding to our search parameters and then we used those Ids to fire database query. In case the data we want to return is included in Index (DOJ attribute for example in our case) we can skip the database portion, so choose wisely how much information (attributes) you want to include while you index your sql data. Well that's all ... it's time to say good bye. Take good care of your health and don't forget to vote, its a must :) - Munish Gogna

In the previous article of this series, we saw how assertions are the key to self-validating tests. Even if an assertion's result is only a pass/fail value, programmers and testers find very handy to associate to the fail value some kind of additional information about the failure, to allow easier debugging without insertion of var_dump() and similar statements into the code. As I showed you in the previous article's code sample, assertions are useful the most when they fail, and in case they do, they should explain what happened so that a correction can be made quickly. An assertion that passes does not do anything; an assertion that fails shows an error. Whata happens when tests are failing The typical scenario is that of a large test suite, with hundreds or thousands of tests. At the time of a commit to the source code repository (or of a push if we're talking about git), the whole test suite must be green. Even if only one tests is red, the test suite is regarded as red. Sometimes, a regression is introduced by a programming error or an unforeseen scenario, and the output of the test suite becomes something like this: [12:30:02][giorgio@Desmond:~/txt/articles]$ phpunit assertionmessage.php PHPUnit 3.5.5 by Sebastian Bergmann. FFFF Time: 0 seconds, Memory: 3.75Mb There were 4 failures: 1) AssertionMessagesTest::testAsserTrue The object is not an Iterator. Failed asserting that is true. /home/giorgio/Dropbox/txt/articles/assertionmessage.php:9 2) AssertionMessagesTest::testAssertEquals The square root of 17 is not 4 but 4.1231056256177. Failed asserting that matches expected . /home/giorgio/Dropbox/txt/articles/assertionmessage.php:17 3) AssertionMessagesTest::testAssertContainsFailsWithCustomMessage The array does not contain 4. Failed asserting that an array contains . /home/giorgio/Dropbox/txt/articles/assertionmessage.php:24 4) AssertionMessagesTest::testAssertContains Failed asserting that an array contains . /home/giorgio/Dropbox/txt/articles/assertionmessage.php:36 FAILURES! Tests: 4, Assertions: 4, Failures: 4. The green tests are not noisy (which would be a smell), since they are not interesting at the moment. The red tests, which our focus is on, are shown with all the associated information. Along with the test name, the line of interest, and the kind of problem (failure/error), the assertion message is shown. How to write a meaningful message An assertion message can be passed as an argument of the assertion method, to be used in case of failure. Writing good assertion messages is an art, but before diving into some code to show you some messages I wrote, we can see how assertion messages are categorized in literature: Assertion-Identifying Message: includes information on which of the assertions in the same method caused the failure. This information can be for example the name of the checked variables. It's good to understand at a glance which assertion failed, but PHPUnit shows you the line of the test method that caused the failure, so it's not strictly necessary to include this kind of messages. Expectation-Describing Message: tells us what should have happened, but did not according to the data passed to the assertion method. Argument-Describing Message: when using assertions which are not very smart, like assertFalse() or assertTrue(), a failure would tell us by default only something like False was passed to assertTrue(). Adding an Argument-Describing Message will tell us instead The predicate on the set containing all red cars was false instead of true. Watch the test fail I can never say it enoug: when you write your tests, watch them fail (this mean also that each test should be written in advance with regard to the code it exercises, following the discipline of Test-Driven Development). This practice is not only useful for avoiding false positives, which are rare anyway, but to show at least for once the assertion message. You can check that when the test fails, the error message is meaningful and the test does not cause a Fatal Error. Watch your test fail to ensure that when they'll fail due to a regression in the future, you will be pleased by how fast you'll learn what's wrong. Example The code sample shows how to use PHPUnit's assertion methods with the $message optional argument. Any time you use basic methods like assertTrue(), a message is mandatory. The need for $message is inversely proportional to the specificity of the assertion: for example when assertContains() fails, PHPUnit has already valuable information in the the method itself to produce a readable error message. Note that the usage of "" (double quotes) enables fast composition of error messages via variable substitution. assertTrue($object instanceof Iterator, 'The object is not an Iterator.'); } public function testAssertEquals() { $expected = 4; $square = 17; $result = sqrt($square); $this->assertEquals($expected, $result, "The square root of $square is not $expected but $result."); } public function testAssertContainsFailsWithCustomMessage() { $array = array(1, 2, 3); $testElement = 4; $this->assertContains($testElement, $array, "The array does not contain $testElement."); } /** * This Assertion Message would be enough. It will even be better in some cases, * since PHPUnit would display nicely $testElement even if it was not "castable" * to a string (an object or array). */ public function testAssertContains() { $array = array(1, 2, 3); $testElement = 4; $this->assertContains($testElement, $array); } }

Some years back I had to provide a wrapper around an EJB 3.0 remote service to come up with a simple web service project that would be deployed over Tomcat and accessed in a simple http way due to some accessibility issues. Now as I cannot reveal the actual requirement I implemented that time so here I am presenting a simple demo kind of service with following signature. public AccountDetails getAccountDetails(String accountNo, SecurityToken token); The service will return the account details of a particular account number, provided the token is valid (generated using some Security module of the application). In nutshell, the client will ask for a token from Security module and then invoke this method. The service will validate the token to see if the caller can invoke the method or not? In general you should use handlers (message interceptors that can be easily plugged in to the JAX-WS runtime to do additional processing of the inbound and outbound messages) to validate the stuff, freeing implementation class from that overhead, it is just an example exercise so our implementation class will check the security token also.Sounds good.. lets move ahead. Libraries we are going to use include JAXB and JAX-WS, as both of them have sensible defaults, the number of annotations can be kept to the minimum. Also in my opinion, it is always best to develop WSDL and schemas by hand to ensure that the service contract is appropriately defined and also that the schema can be re-used (by other services) and extended if necessary. I do not prefer using annotations for automatically producing WSDL and schema at runtime so let's start with the definition, i am using inline schema as we have a very simple requirement here, though we should have schema definitions in separate xsd files. accounts.wsdl: I am not going to explain the In and outs of this wsdl , in a nutshell it defines one operation 'getAccountDetails' which takes accountNo and returns back the account details like account type, balance etc. Please note that I have also added a security header token that will validate the caller (left to the implementation of the service). Now as we are done with our wsdl and schema, let's generate the portable artifacts from our service definition. JAX-WS includes a tool that can do this for us, we will use this tool to generate portable artifacts like Service Endpoint Interface (SEI), Service and Exception classes.These artifacts can be packaged in a WAR file with the WSDL and schema documents along with the endpoint implementation to be deployed. To generate the artifacts , we run following command: wsimport C:\devel\workspace\webservice\WebContent\WEB-INF\wsdl\accounts.wsdl -p com.mg.ws -keep -Xnocompile This will create the following artifacts in com.mg.ws package. AccountDetails.java AccountDetailsFault.java AccountDetailsFault_Exception.java AccountDetailsPortType.java AccountDetailsTO.java GeAccountDetailsINTO.java ObjectFactory.java package-info.java SecurityToken.java Now as we have got all the artifacts we are ready to implement our service, the interface we need to implement is AccountDetailsPortType , so let's do it. Here is our dummy implementation class: package com.mg.ws.impl;import java.math.BigDecimal;import javax.jws.WebService;import com.mg.ws.AccountDetailsFault;import com.mg.ws.AccountDetailsFault_Exception;import com.mg.ws.AccountDetailsPortType;import com.mg.ws.AccountDetailsTO;import com.mg.ws.GeAccountDetailsINTO;import com.mg.ws.SecurityToken;@WebService(name = "AccountDetailsService", portName = "AccountDetailsPort", endpointInterface = "com.mg.ws.AccountDetailsPortType", wsdlLocation = "WEB-INF/wsdl/accounts.wsdl", targetNamespace="http://gognamunish.com/accounts") public class AccountDetailsServiceImpl implements AccountDetailsPortType {public AccountDetailsTO getAccountDetails(GeAccountDetailsINTO parameters, SecurityToken requestHeader) throws AccountDetailsFault_Exception { AccountDetailsTO detailsTO = new AccountDetailsTO(); // validate token validateToken(requestHeader); // populate response detailsTO = getDetailsFromSomewhere (parameters.getAccountNo()); return detailsTO; } private AccountDetailsTO getDetailsFromSomewhere(String accountNo) throws AccountDetailsFault_Exception { if(accountNo == null || accountNo.trim().length()==0){ AccountDetailsFault faultInfo = new AccountDetailsFault(); faultInfo.setFaultInfo("missing account number"); faultInfo.setMessage("account number is required field"); throw new AccountDetailsFault_Exception("account no missing", faultInfo); } AccountDetailsTO detailsTO = new AccountDetailsTO(); detailsTO.setAccNo(accountNo); detailsTO.setAccType("SAVING"); detailsTO.setBalance(new BigDecimal(10000)); return detailsTO; } private void validateToken(SecurityToken requestHeader) throws AccountDetailsFault_Exception { if ("83711070".equals(requestHeader.getToken()) && requestHeader.getValidTill() != null){ System.out.println("token processed successfully..."); } else { AccountDetailsFault faultInfo = new AccountDetailsFault(); faultInfo.setFaultInfo("Header token Invalid"); faultInfo.setMessage("can't help"); throw new AccountDetailsFault_Exception("invalid token", faultInfo); } } This is just a dummy implementation for illustration purpose only. Now as our service implementation is done, we proceed to package this in a war and deploy it on tomcat. Now we create a standard web.xml, which defines WSServletContextListener, WSServlet and structure of a web project. com.sun.xml.ws.transport.http.servlet.WSServletContextListener AccountDetailsServicecom.sun.xml.ws.transport.http.servlet.WSServlet 1AccountDetailsService/details Next we create a sun-jaxws.xml, defines the web service implementation class. This file is required regardless of whether we publish our web service on tomcat, glassfish or any other server. OK so far so good , let's build our application and deploy it on tomcat, here is the ant script. NOTE: jars in lib should be carefully chosen, otherwise it will make your life hell. Now let's test the application, point your browser to http://localhost:8080/account/details , if you see something like this, it means you have successfully deployed the service. Now let's test the service, we can use the client generated by the wsimport tool as : public static void main(String[] args) throws Exception { AccountDetails accountDetails = new AccountDetails(); AccountDetailsPortType port = accountDetails.getAccountDetailsPort(); AccountDetailsTO details = port.getAccountDetails(new GeAccountDetailsINTO(), new SecurityToken()); } For those who want to invoke the service using soap stuff, they can use tool like soapUI (can be downloaded from www.soapui.org), lets make some soap calls now: case: invalid security header case: valid security header By default, Tomcat does not comes with any JAX-WS dependencies, So, you have to include it manually. Go here Download JAX-WS RI distribution, you will find the wsimport tool in lib directory. Please note for running this project you can just skip this step as I have already done this step for you and included the required libraries in the lib folder of the project. I have included eclipse project for this demo, please get it from resource section. To deploy just change build.properties to point to TOMCAT_HOME and run ant build. Thats all for now ... Please provide your valuable comments or any suggestion and DON'T forget to vote :) It's Munish Gogna signing off now :)

Setting the mouse cursor position on a Windows machine with the help of .NET Framework shouldn't be that big of a problem. After all, there is the built-in Cursor class that lets you do that by executing a simple line of code: Cursor.Position = new System.Drawing.Point(0, 0); Of course, here 0 and 0 are the absolute coordinates for the mouse cursor on the screen. One thing to mention about this type of position setting is that Cursor requires a reference to System.Windows.Forms. And in some cases you don't want this extra reference. If that's the case, WinAPI is your solution. It requires some more work compared to the regular .NET way (class instance -> method call) but at the end you get more control than you would expect. When using WinAPI to set the cursor position, there are two ways you can go: mouse_event SendInput mouse_event is the very basic function that is only able to set the mouse coordinates. It was superseded and Microsoft recomends using SendInput instead. Nonetheless, it still works (although I cannot say for sure whether it will be working in future releases of Windows). So to start, I have a very basic class: class WINAPI_SUPERSEDED { [DllImport("user32.dll",SetLastError=true)] public static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint dwData, int dwExtraInfo); public enum MouseFlags { MOUSEEVENTF_ABSOLUTE = 0x8000, MOUSEEVENTF_LEFTDOWN = 0x0002, MOUSEEVENTF_LEFTUP = 0x0004, MOUSEEVENTF_MIDDLEDOWN = 0x0020, MOUSEEVENTF_MIDDLEUP = 0x0040, MOUSEEVENTF_MOVE = 0x0001, MOUSEEVENTF_RIGHTDOWN = 0x0008, MOUSEEVENTF_RIGHTUP = 0x0010, MOUSEEVENTF_WHEEL = 0x0800, MOUSEEVENTF_XDOWN = 0x0080, MOUSEEVENTF_XUP = 0x0100 } public enum DataFlags { XBUTTON1 = 0x0001, XBUTTON2 = 0x0002 } } So to set the cursor position to 0,0 I would use this: WINAPI_SUPERSEDED.mouse_event((int)WINAPI_SUPERSEDED.MouseFlags.MOUSEEVENTF_MOVE | (int)WINAPI_SUPERSEDED.MouseFlags.MOUSEEVENTF_ABSOLUTE, 0, 0, 0, 0); If you go through the documentation, you will notice that in fact, dx and dy are in no way direct coordinates but rather normalized values ranged between 0 and 65,535 - that is only when the MOUSEEVENTF_ABSOLUTE flag is present. Otherwise, the position will be adjusted according to the current mouse cursor position. This method doesn't return any value so I cannot be informed whether it was successful or not. GetLastError won't give much information either. In this case, SendInput comes to the rescue. Here is what I have defined as the main class: class WINAPI { [DllImport("kernel32.dll")] public static extern uint GetLastError(); public enum MouseData { XBUTTON1 = 0x0001, XBUTTON2 = 0x0002 } public enum MouseFlags { MOUSEEVENTF_ABSOLUTE = 0x8000, MOUSEEVENTF_HWHEEL = 0x01000, MOUSEEVENTF_MOVE = 0x0001, MOUSEEVENTF_MOVE_NOCOALESCE = 0x2000, MOUSEEVENTF_LEFTDOWN = 0x0002, MOUSEEVENTF_LEFTUP = 0x0004, MOUSEEVENTF_RIGHTDOWN = 0x0008, MOUSEEVENTF_RIGHTUP = 0x0010, MOUSEEVENTF_MIDDLEDOWN = 0x0020, MOUSEEVENTF_MIDDLEUP = 0x0040, MOUSEEVENTF_VIRTUALDESK = 0x4000, MOUSEEVENTF_WHEEL = 0x0800, MOUSEEVENTF_XDOWN = 0x0080, MOUSEEVENTF_XUP = 0x0100 } [DllImport("user32.dll", SetLastError=true)] public static extern uint SendInput(uint nInputs, ref INPUT pInputs, int cbSize); [StructLayout (LayoutKind.Explicit)] public struct INPUT { [FieldOffset(0)] public int type; [FieldOffset(4)] public MOUSEINPUT mi; } public struct MOUSEINPUT { public int dx; public int dy; public int mouseData; public int dwFlags; public int time; public int extraInfo; } } This class is a bit more complicated, but at the same time you have to understand that in some cases, SendInput is used for hardware and keyboard input as well. Of course, for experimentation purposes I removed those parts in the sample class. Here you have the same MouseFlags enum that will let you pass custom flags defining the mouse behavior. Notice, that SendInput has SetLastError set to true, therefore if something wrong happens via this method, the error will be easily obtained via GetLastError, that is implemented as a helper method in the same class. VERY IMPORTANT: When you define the INPUT struct, make sure you use LayoutKind.Explicit since when passed to an unamanaged call, a specific field layout is required - as you can see, every field is decorated with a FieldOffset attribute. Also taking about StructLayout, you don't have to set StructLayout.Sequential to MOUSEINPUT since it is setautomatically by CLR. When I want to call the method above, I can simply use this snippet: WINAPI.MOUSEINPUT mouseInput = new WINAPI.MOUSEINPUT(); mouseInput.dx = 100; mouseInput.dy = 10; mouseInput.dwFlags = (int)WINAPI.MouseFlags.MOUSEEVENTF_ABSOLUTE | (int)WINAPI.MouseFlags.MOUSEEVENTF_MOVE; WINAPI.INPUT input = new WINAPI.INPUT(); input.type = 0; input.mi = mouseInput; uint x = WINAPI.SendInput(1, ref input, Marshal.SizeOf(input)); Console.WriteLine(x); Console.WriteLine(WINAPI.GetLastError()); Console.ReadLine(); Notice that I have to call the unmanaged version of sizeof and pass the INPUT struct to it in order for the method to correctly execute. The regular C# sizeof won't cut it here. When I am defining the type of input, 0 is repesenting the INPUT_MOUSE flag, since I am only handling the mouse here. Of course, I can re-organize my method to accept a set of INPUT instances - the native call itself allows this by requesting an array of INPUT and the correct indication of the number of INPUT instances passed, but that is not required for testing purposes.

Read about what a Thread Local is, and learn how to use it in this awesome tutorial.

Devoxx, and all similar conferences, is a place where you make new discoveries, continually. One of these, in my case, at last week's Devoxx, started from a discussion with Jaroslav Bachorik from the VisualVM team. He had presented VisualVM's extensibility in a session at Devoxx. I had heard that, when creating extensions for VisualVM, one can also create new charts using VisualVM's own charting API. Jaroslav confirmed this and we created a small demo together to prove it, i.e., there's a charting API in VisualVM. Since VisualVM is based on the NetBeans Platform, I went further and included the VisualVM charts in a generic NetBeans Platform application. Then I wondered what the differences are between JFreeChart and VisualVM charts, so asked the VisualVM chart architect, Jiri Sedlacek. He sent me a very interesting answer: JFreeCharts are great for creating any kind of static graphs (typically for reports). They provide support for all types of existing chart types. The benefit of using JFreeChart is fully customizable appearance and export to various formats. The only problem of this library is that it's not primarily designed for displaying live data. You can hack it to display data in real time, but the performance is poor. That's why I've created the VisualVM charts. The primary (and so far only) goal is to provide charts optimized for displaying live data with minimal performance and memory overhead. You can easily display a fullscreen graph and it will still scroll smoothly while running and adding new values (when running on physical hardware, virtualized environment may give slightly worse results). There's a real rendering engine behind the charts which ensures that only the changed areas of the chart are repainted (no full-repaints because of a 1px change). Scrolling the chart means moving the already rendered image and only painting the newly displayed area. Last but not least, the charts are optimized for displaying over a remote X session - rendering is automatically switched to low-quality ensuring good response times and interactivity. The Tracer engine introduced in VisualVM 1.3 further improves performance of the charts. I've intensively profiled and optimized the charts to minimize the cpu cycles/memory allocations for each repaint. As of now, I believe that the VisualVM charts are the fastest real time Java charts with the lowest cpu/memory footprint. Best of all is that everything described above is in the JDK. That's because VisualVM is in the JDK. Here's a small NetBeans Platform application (though you could also use the VisualVM chart API without using the NetBeans Platform, just include these JARs on your classpath: org-netbeans-lib-profiler-charts.jar, com-sun-tools-visualvm-charts.jar, com-sun-tools-visualvm-uisupport.jar and org-netbeans-lib-profiler-ui.jar) that makes use of the VisualVM chart API outlined above: The chart that you see above is updated in real time and you can change to full screen and you can scroll through it and, at the same time, there is no lag and it is very performant. Below is all the code (from the unit test package in the VisualVM sources) that you see in the JPanel above: public class Demo extends JPanel { private static final long SLEEP_TIME = 500; private static final int VALUES_LIMIT = 150; private static final int ITEMS_COUNT = 8; private SimpleXYChartSupport support; public Demo() { createModels(); setLayout(new BorderLayout()); add(support.getChart(), BorderLayout.CENTER); } private void createModels() { SimpleXYChartDescriptor descriptor = SimpleXYChartDescriptor.decimal(0, 1000, 1000, 1d, true, VALUES_LIMIT); for (int i = 0; i < ITEMS_COUNT; i++) { descriptor.addLineFillItems("Item " + i); } descriptor.setDetailsItems(new String[]{"Detail 1", "Detail 2", "Detail 3"}); descriptor.setChartTitle("Demo Chart"); descriptor.setXAxisDescription("X Axis [time]"); descriptor.setYAxisDescription("Y Axis [units]"); support = ChartFactory.createSimpleXYChart(descriptor); new Generator(support).start(); } private static class Generator extends Thread { private SimpleXYChartSupport support; public void run() { while (true) { try { long[] values = new long[ITEMS_COUNT]; for (int i = 0; i < values.length; i++) { values[i] = (long) (1000 * Math.random()); } support.addValues(System.currentTimeMillis(), values); support.updateDetails(new String[]{1000 * Math.random() + "", 1000 * Math.random() + "", 1000 * Math.random() + ""}); Thread.sleep(SLEEP_TIME); } catch (Exception e) { e.printStackTrace(System.err); } } } private Generator(SimpleXYChartSupport support) { this.support = support; } } } Here is the related Javadoc. To get started using the VisualVM charts in your own application, read this blog, and then look in the "lib" folder of the JDK to find the JARs you will need. And then have fun with real-time data in your Java desktop applications.