DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Java Topics

article thumbnail
A Look Inside JBoss Microcontainer, Part 3 - the Virtual File System
We're finally back with our next article in the Microcontainer series. In the first two articles we demonstrated how Microcontainer supports , and showed its powerful . In this article, we'll explain Classloading and Deployers, but first we must familiarize ourselves with VFS. VFS stands, as expected, for Virtual File System. What does VFS solve for us, or why is it useful? Here, at JBoss, we saw that a lot of similar resource handling code was scattered/duplicated all over the place. In most cases it was code that was trying to determine what type of resource a particular resource was, e.g. is it a file, a directory, or a jar loading resources through URLs. Processing of nested archives was also reimplemented again, and again in different libraries. Read the other parts in DZone's exclusive JBoss Microcontainer Series: Part 4 -- ClassLoading Layer Example: public static URL[] search(ClassLoader cl, String prefix, String suffix) throws IOException { Enumeration[] e = new Enumeration[]{ cl.getResources(prefix), cl.getResources(prefix + "MANIFEST.MF") }; Set all = new LinkedHashSet(); URL url; URLConnection conn; JarFile jarFile; for (int i = 0, s = e.length; i < s; ++i) { while (e[i].hasMoreElements()) { url = (URL)e[i].nextElement(); conn = url.openConnection(); conn.setUseCaches(false); conn.setDefaultUseCaches(false); if (conn instanceof JarURLConnection) { jarFile = ((JarURLConnection)conn).getJarFile(); } else { jarFile = getAlternativeJarFile(url); } if (jarFile != null) { searchJar(cl, all, jarFile, prefix, suffix); } else { boolean searchDone = searchDir(all, new File(URLDecoder.decode(url.getFile(), "UTF-8")), suffix); if (searchDone == false) { searchFromURL(all, prefix, suffix, url); } } } } return (URL[])all.toArray(new URL[all.size()]); } private static boolean searchDir(Set result, File file, String suffix) throws IOException { if (file.exists() && file.isDirectory()) { File[] fc = file.listFiles(); String path; for (int i = 0; i < fc.length; i++) { path = fc[i].getAbsolutePath(); if (fc[i].isDirectory()) { searchDir(result, fc[i], suffix); } else if (path.endsWith(suffix)) { result.add(fc[i].toURL()); } } return true; } return false; } There were also many problems with file locking on Windows systems, which forced us to copy all hot-deployable archives to another location to prevent locking those in deploy folders (which would prevent their deletion and filesystem based undeploy). File locking was a major problem that could only be addressed by centralizing all the resource loading code in one place. Recognizing a need to deal with all of these issues in one place, wrapping it all into a simple and useful API, we created the VFS project. VFS public API Basic usage in VFS can be split in two pieces: simple resource navigation visitor pattern API As mentioned, in plain JDK resource handling navigation over resources is far from trivial. You must always check what kind of resource you're currently handling, and this is very cumbersome. With VFS we wanted to limit this to a single resource type - VirtualFile. public class VirtualFile implements Serializable { /** * Get certificates. * * @return the certificates associated with this virtual file */ Certificate[] getCertificates() /** * Get the simple VF name (X.java) * * @return the simple file name * @throws IllegalStateException if the file is closed */ String getName() /** * Get the VFS relative path name (org/jboss/X.java) * * @return the VFS relative path name * @throws IllegalStateException if the file is closed */ String getPathName() /** * Get the VF URL (file://root/org/jboss/X.java) * * @return the full URL to the VF in the VFS. * @throws MalformedURLException if a url cannot be parsed * @throws URISyntaxException if a uri cannot be parsed * @throws IllegalStateException if the file is closed */ URL toURL() throws MalformedURLException, URISyntaxException /** * Get the VF URI (file://root/org/jboss/X.java) * * @return the full URI to the VF in the VFS. * @throws URISyntaxException if a uri cannot be parsed * @throws IllegalStateException if the file is closed * @throws MalformedURLException for a bad url */ URI toURI() throws MalformedURLException, URISyntaxException /** * When the file was last modified * * @return the last modified time * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ long getLastModified() throws IOException /** * Returns true if the file has been modified since this method was last called * Last modified time is initialized at handler instantiation. * * @return true if modifed, false otherwise * @throws IOException for any error */ boolean hasBeenModified() throws IOException /** * Get the size * * @return the size * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ long getSize() throws IOException /** * Tests whether the underlying implementation file still exists. * @return true if the file exists, false otherwise. * @throws IOException - thrown on failure to detect existence. */ boolean exists() throws IOException /** * Whether it is a simple leaf of the VFS, * i.e. whether it can contain other files * * @return true if a simple file. * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ boolean isLeaf() throws IOException /** * Is the file archive. * * @return true if archive, false otherwise * @throws IOException for any error */ boolean isArchive() throws IOException /** * Whether it is hidden * * @return true when hidden * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ boolean isHidden() throws IOException /** * Access the file contents. * * @return an InputStream for the file contents. * @throws IOException for any error accessing the file system * @throws IllegalStateException if the file is closed */ InputStream openStream() throws IOException /** * Do file cleanup. * * e.g. delete temp files */ void cleanup() /** * Close the file resources (stream, etc.) */ void close() /** * Delete this virtual file * * @return true if file was deleted * @throws IOException if an error occurs */ boolean delete() throws IOException /** * Delete this virtual file * * @param gracePeriod max time to wait for any locks (in milliseconds) * @return true if file was deleted * @throws IOException if an error occurs */ boolean delete(int gracePeriod) throws IOException /** * Get the VFS instance for this virtual file * * @return the VFS * @throws IllegalStateException if the file is closed */ VFS getVFS() /** * Get the parent * * @return the parent or null if there is no parent * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ VirtualFile getParent() throws IOException /** * Get a child * * @param path the path * @return the child or null if not found * @throws IOException for any problem accessing the VFS * @throws IllegalArgumentException if the path is null * @throws IllegalStateException if the file is closed or it is a leaf node */ VirtualFile getChild(String path) throws IOException /** * Get the children * * @return the children * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ List getChildren() throws IOException /** * Get the children * * @param filter to filter the children * @return the children * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed or it is a leaf node */ List getChildren(VirtualFileFilter filter) throws IOException /** * Get all the children recursively * * This always uses {@link VisitorAttributes#RECURSE} * * @return the children * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ List getChildrenRecursively() throws IOException /** * Get all the children recursively * * This always uses {@link VisitorAttributes#RECURSE} * * @param filter to filter the children * @return the children * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed or it is a leaf node */ List getChildrenRecursively(VirtualFileFilter filter) throws IOException /** * Visit the virtual file system * * @param visitor the visitor * @throws IOException for any problem accessing the virtual file system * @throws IllegalArgumentException if the visitor is null * @throws IllegalStateException if the file is closed */ void visit(VirtualFileVisitor visitor) throws IOException } As you can see you have all of the usual read-only File System operations, plus a few options to cleanup or delete the resource. Cleanup or deletion handling is needed when we're dealing with some internal temporary files; e.g. from nested jars handling. To switch from JDK's File or URL resource handling to new VirtualFile we need a root. It is the VFS class that knows how to create one with the help of URL or URI parameter. public class VFS { /** * Get the virtual file system for a root uri * * @param rootURI the root URI * @return the virtual file system * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL is null */ static VFS getVFS(URI rootURI) throws IOException /** * Create new root * * @param rootURI the root url * @return the virtual file * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL */ static VirtualFile createNewRoot(URI rootURI) throws IOException /** * Get the root virtual file * * @param rootURI the root uri * @return the virtual file * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL is null */ static VirtualFile getRoot(URI rootURI) throws IOException /** * Get the virtual file system for a root url * * @param rootURL the root url * @return the virtual file system * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL is null */ static VFS getVFS(URL rootURL) throws IOException /** * Create new root * * @param rootURL the root url * @return the virtual file * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL */ static VirtualFile createNewRoot(URL rootURL) throws IOException /** * Get the root virtual file * * @param rootURL the root url * @return the virtual file * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL */ static VirtualFile getRoot(URL rootURL) throws IOException /** * Get the root file of this VFS * * @return the root * @throws IOException for any problem accessing the VFS */ VirtualFile getRoot() throws IOException } You can see three different methods that look a lot alike - getVFS, createNewRoot and getRoot. Method getVFS returns a VFS instance, and what's important, it doesn't yet create a VirtualFile instance. Why is this important? Because there are methods which help us configure a VFS instance (see VFS class API javadocs), before telling it to create a VirtualFile root. The other two methods, on the other hand, use default settings for root creation. The difference between createNewRoot and getRoot is in caching details, which we'll delve in later on. URL rootURL = ...; // get root url VFS vfs = VFS.getVFS(rootURL); // configure vfs instance VirtualFile root1 = vfs.getRoot(); // or you can get root directly VirtualFile root2 = VFS.crateNewRoot(rootURL); VirtualFile root3 = VFS.getRoot(rootURL); The other useful thing about VFS API is its implementation of a proper visitor pattern. This way it's very simple to recursively gather different resources, something quite impossible to do with plain JDK resource loading. public interface VirtualFileVisitor { /** * Get the search attribues for this visitor * * @return the attributes */ VisitorAttributes getAttributes(); /** * Visit a virtual file * * @param virtualFile the virtual file being visited */ void visit(VirtualFile virtualFile); } VirtualFile root = ...; // get root VirtualFileVisitor visitor = new SuffixVisitor(".class"); // get all classes root.visit(visitor); VFS Architecture While public API is quite intuitive, real implementation details are a bit more complex. We'll try to explain the concepts in a quick pass. Each time you create a VFS instance, its matching VFSContext instance is created. This creation is done via VFSContextFactory. Different protocols map to different VFSContextFactory instances - e.g. file/vfsfile map to FileSystemContextFactory, zip/vfszip map to ZipEntryContextFactory. Also, each time a VirtualFile instance is created, its matching VirtualFileHandler is created. It's this VirtualFileHandler instance that knows how to handle different resource types properly - VirtualFile API just delegates invocations to its VirtualFileHandler reference. As one could expect, VFSContext instance is the one that knows how to create VirtualFileHandler instances accordingly to a resource type - e.g. ZipEntryContextFactory creates ZipEntryContext, which then creates ZipEntryHandler. Existing implementations Apart from files, directories (FileHandler) and zip archives (ZipEntryHandler) we also support other more exotic usages. The first one is Assembled, which is similar to what Eclipse calls Linked Resources. Its idea is to take existing resources from different trees, and "mock" them into single resource tree. AssembledDirectory sar = AssembledContextFactory.getInstance().create("assembled.sar"); URL url = getResource("/vfs/test/jar1.jar"); VirtualFile jar1 = VFS.getRoot(url); sar.addChild(jar1); url = getResource("/tmp/app/ext.jar"); VirtualFile ext1 = VFS.getRoot(url); sar.addChild(ext); AssembledDirectory metainf = sar.mkdir("META-INF"); url = getResource("/config/jboss-service.xml"); VirtualFile serviceVF = VFS.getRoot(url); metainf.addChild(serviceVF); AssembledDirectory app = sar.mkdir("app.jar"); url = getResource("/app/someapp/classes"); VirtualFile appVF = VFS.getRoot(url); app.addPath(appVF, new SuffixFilter(".class")); Another implementation is in-memory files. In our case this came out of a need to easily handle AOP generated bytes. Instead of mucking around with temporary files, we simply drop bytes into in-memory VirtualFileHandlers. URL url = new URL("vfsmemory://aopdomain/org/acme/test/Test.class"); byte[] bytes = ...; // some AOP generated class bytes MemoryFileFactory.putFile(url, bytes); VirtualFile classFile = VFS.getVirtualFile(new URL("vfsmemory://aopdomain"), "org/acme/test/Test.class"); InputStream bis = classFile.openStream(); // e.g. load class from input stream Extension hooks It's quite easy to extend VFS with a new protocol, similar to what we've done with Assembled and Memory. All you need is a combination of VFSContexFactory, VFSContext, VirtualFileHandler, FileHandlerPlugin and URLStreamHandler implementations. The first one is trivial, while the others depend on the complexity of your task - e.g. you could implement rar, tar, gzip or even remote access. In the end you simply register this new VFSContextFactory with VFSContextFactoryLocator. See this article's demo for a simple gzip example Features One of the first major problems we stumbled upon was proper usage of nested resources, more exactly nested jar files. e.g. normal ear deployments: gema.ear/ui.war/WEB-INF/lib/struts.jar In order to read contents of struts.jar we have two options: handle resources in memory create top level temporary copies of nested jars, recursively The first option is easier to implement, but it's very memory-consuming--just imagine huge apps in memory. The other approach leaves a bunch of temporary files, which should be invisible to plain user. Hence expecting them to disappear once the deployment is undeployed. Now imagine the following scenario: A user gets a hold of VFS's URL instance, which points to some nested resource. The way plain VFS would handle this is to re-create the whole path from scratch, meaning it would unpack nested resources over and over again. This would (and it did) lead to a huge pile of temporary files. How to avoid this? The way we approached this is by using VFSRegistry, VFSCache and TempInfo. When you ask for VirtualFile over VFS (getRoot, not createNewRoot), VFS asks VFSRegistry implementation to provide the file. Existing DefaultVFSRegistry first checks if matching root VFSContext for provided URI exists. If it does, it first tries to navigate to existing TempInfo (link to temporary files), falling back to regular navigation if no such temporary file exists. This way we completely re-use any already unpacked temporary files, saving time and disk space. If no matching VFSContext is found in cache, we create a new VFSCache entry, and continue with default navigation. It's then up to VFSCache implementation used, how it handles cached VFSContext entries. VFSCache is configurable via VFSCacheFactory - by default we don't cache anything, but there are a few useful existing VFSCache implementations, ranging from LRU to timed cache. API Use case There is a class called VFSUtils which is part of a public API, and it is sort of a dumping ground of useful functionality. It contains a bunch of helpful methods and configuration settings (system property keys, actually). Check the API javadocs for more details. Existing issues / workarounds Another issue that came up - expectedly - was inability of some frameworks to properly work on top of VFS. The problem lied in custom VFS urls like: vfsfile, vfszip, vfsmemory. In most cases you could still work around it with plain URL or URLConnection usage, but a lot of frameworks do a strict match on file or jar protocol, which of course fails. We were able to patch some frameworks (e.g. Facelets) and provide extensions to others (e.g. Spring). If you are a library developer, and your library has a simple pluggable resource loading mechanism, then we suggest you simply extend it with VFS based implementation. If there are no hooks, try to limit your assumptions to more general usage based on URL or URLConnection. Conclusion While VFS is very nice to use, it comes at a price. It adds additional layer on top of JDK's resource handling, meaning extra invocations are always present when you're dealing with resources. We also keep some of the jar handling info in memory to make it easy to get hold of a specific resource, but at the expense of some extra memory consumption. Overall VFS proved to be a very useful library as it hides away many use cases that are painful with plain JDK, and provides a comprehensive API for working with resources - i.e. visitor pattern implementation. We're constantly following user feedback to VFS issues they encounter, making each version a bit better. Now, that we got to know VFS, it's time we move on to MC's new Classloading layer! About the Author Ales Justin was born in Ljubljana, Slovenia and graduated with a degree in mathematics from the University of Ljubljana. He fell in love with Java seven years ago and has spent most of his time developing information systems, ranging from customer service to energy management. He joined JBoss in 2006 to work full time on the Microcontainer project, currently serving as its lead. He also contributes to JBoss AS and is Seam and Spring integration specialist. He represent JBoss on 'JSR-291 Dynamic Component Support for Java SE' and 'OSGi' expert groups.
September 24, 2009
by Ales Justin
· 43,381 Views
article thumbnail
Sorting Collections in Hibernate Using SQL in @OrderBy
When you have collections of associated objects in domain objects, you generally want to specify some kind of default sort order. For example, suppose I have domain objects Timeline and Event: @Entity class Timeline { @Required String description @OneToMany(mappedBy = "timeline") @javax.persistence.OrderBy("startYear, endYear") Set events } @Entity class Event { @Required Integer startYear Integer endYear @Required String description @ManyToOne Timeline timeline } In the above example I've used the standard JPA (Java Persistence API) @OrderBy annotation which allows you to specify the order of a collection of objects via object properties, in this example a @OneToMany association . I'm ordering first by startYear in ascending order and then by endYear, also in ascending order. This is all well and good, but note that I've specified that only the start year is required. (The @Required annotation is a custom Hibernate Validator annotation which does exactly what you would expect.) How are the events ordered when you have several events that start in the same year but some of them have no end year? The answer is that it depends on how your database sorts null values by default. Under Oracle 10g nulls will come last. For example if two events both start in 2001 and one of them has no end year, here is how they are ordered: 2001 2002 Some event 2001 2003 Other event 2001 Event with no end year What if you want to control how null values are ordered so they come first rather than last? In Hibernate there are several ways you could do this. First, you could use the Hibernate-specific @Sort annotation to perform in-memory (i.e. not in the database) sorting, using natural sorting or sorting using a Comparator you supply. For example, assume I have an EventComparator helper class that implements Comparator. I could change Timeline's collection of events to look like this: @OneToMany(mappedBy = "timeline") @org.hibernate.annotations.Sort(type = SortType.COMPARATOR, comparator = EventCompator) Set events Using @Sort will perform sorting in-memory once the collection has been retrieved from the database. While you can certainly do this and implement arbitrarily complex sorting logic, it's probably better to sort in the database when you can. So we now need to turn to Hibernate's @OrderBy annotation, which lets you specify a SQL fragment describing how to perform the sort. For example, you can change the events mapping to : @OneToMany(mappedBy = "timeline") @org.hibernate.annotations.OrderBy("start_year, end_year") Set events This sort order is the same as using the JPA @OrderBy with "startYear, endYear" sort order. But since you write actual SQL in Hibernate's @OrderBy you can take advantage of whatever features your database has, at the possible expense of portability across databases. As an example, Oracle 10g supports using a syntax like "order by start_year, end_year nulls first" to order null end years before non-null end years. You could also say "order by start_year, end year nulls last" which sorts null end years last as you would expect. This syntax is probably not portable, so another trick you can use is the NVL function, which is supported in a bunch of databases. You can rewrite Timeline's collection of events like so: @OneToMany(mappedBy = "timeline") @org.hibernate.annotations.OrderBy("start_year, nvl(end_year , start_year)") Set events The expression "nvl(end_year , start_year)" simply says to use end_year as the sort value if it is not null, and start_year if it is null. So for sorting purposes you end up treating end_year as the same as the start_year if end_year is null. In the contrived example earlier, applying the nvl-based sort using Hibernate's @OrderBy to specify SQL sorting criteria, you now end with the events sorted like this: 2001 Event with no end year 2001 2002 Some event 2001 2003 Other event Which is what you wanted in the first place. So if you need more complex sorting logic than what you can get out of the standard JPA @javax.persistence.OrderBy, try one of the Hibernate sorting options, either @org.hibernate.annotations.Sort or @org.hibernate.annotations.OrderBy. Adding a SQL fragment into your domain class isn't necessarily the most elegant thing in the world, but it might be the most pragmatic thing.
September 16, 2009
by Scott Leberknight
· 102,293 Views
article thumbnail
Java Performance Tuning, Profiling, and Memory Management
Get a perspective on the aspects of JVM internals, controls, and switches that can be used to optimize your Java application.
September 1, 2009
by Vikash Ranjan
· 257,808 Views · 17 Likes
article thumbnail
JPA Implementation Patterns: Lazy Loading
Model your complete database with all its relations with this JPA pattern for lazy loading.
August 19, 2009
by Vincent Partington
· 120,524 Views · 6 Likes
article thumbnail
Spring Integration: A Hands-On Tutorial, Part 1
This tutorial is the first in a two-part series on Spring Integration. In this series we're going to build out a lead management system based on a message bus that we implement using Spring Integration. Our first tutorial will begin with a brief overview of Spring Integration and also just a bit about the lead management domain. After that we'll build our message bus. The second tutorial continues where the first leaves off and builds the rest of the bus. I’ve written the sample code for this tutorial as a Maven 2 project. I’m using Java 5, Spring Integration 1.0.3 and Spring 2.5.6. The code also works for Java 6. I've used Maven profiles to isolate the dependencies you’ll need if you’re running Java 5. The tutorials assume that you're comfortable with JEE, the core Spring framework and Maven 2. Also, Eclipse users may find the m2eclipse plug-in helpful. To complete the tutorial you'll need an IMAP account, and you'll also need access to an SMTP server. Let's begin with an overview of Spring Integration. A bird's eye view of Spring Integration Spring Integration is a framework for implementing a dynamically configurable service integration tier. The point of this tier is to orchestrate independent services into meaningful business solutions in a loosely-coupled fashion, which makes it easy to rearrange things in the face of changing business needs. The service integration tier sits just above the service tier as shown in figure 1. Following the book Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf (Addison-Wesley), Spring Integration adopts the well-known pipes and filters architectural style as its approach to building the service integration layer. Abstractly, filters are information-processing units (any type of processing—doesn’t have to be information filtering per se), and pipes are the conduits between filters. In the context of integration, the network we’re building is a messaging infrastructure—a so-called message bus—and the pipes and filters and called message channels and message endpoints, respectively. The network carries messages from one endpoint to another via channels, and the message is validated, routed, split, aggregated, resequenced, reformatted, transformed and so forth as the different endpoints process it. Figure 1. The service integration tier orchestrates the services below it. That should give you enough technical context to work through the tutorial. Let’s talk about the problem domain for our sample integration, which is enrollment lead management in an online university setting. Lead management overview In many industries, such as the mortgage industry and for-profit education, one important component of customer relationship management (CRM) is managing sales leads. This is a fertile area for enterprise integration because there are typically multiple systems that need to play nicely together in order to pull the whole thing off. Examples include front-end marketing/lead generation websites, external lead vendor systems, intake channels for submitted leads, lead databases, e-mail systems (e.g., to accept leads, to send confirmation e-mails), lead qualification systems, sales systems and potentially others. This tutorial and the next use Spring Integration to integrate several of systems of the kind just mentioned into an overall lead management capability for a hypothetical online university. Specifically we’ll integrate the following: • a CRM system that allows campus and call center staff to create leads directly, as they might do for walk-in or phone-in leads • a Request For Information (RFI) form on a lead generation ("lead gen") marketing website • a legacy e-mail based RFI channel • an external CRM that the international enrollment staff uses to process international leads • confirmation e-mails Figure 2 shows what it will look like when we’re done with both tutorials. For now focus on the big picture rather than the details. Figure 2. This is the lead management system we'll build. For this first tutorial we're simply going to establish the base staff interface, the (dummy) backend service that saves leads to a database, and confirmation e-mails. The second tutorial will deal with lead routing, web-based RFIs and e-mail-based RFIs. Let's dive in. We’ll begin with the basic lead creation page in the CRM and expand out from there. Building the core components [You can download the source code for this section of the tutorial here] We’re going to start by creating a lead creation HTML form for campus and call center staff. That way, if walk-in or phone-in leads express an interest, we can get them into the system. This is something that might appear as a part of a lead management module in a CRM system, as shown in figure 3. Figure 3. We'll build our lead management module with integration in mind from the beginning. Because we’re interested in the integration rather than the actual app features, we’re not really going to save the lead to the database. Instead we’ll just call a createLead() method against a local LeadService bean and leave it at that. But we will use Spring Integration to move the lead from the form to the service bean. Our first stop will be the domain model. DZone readers get 30% off Spring in Practice by Willie Wheeler and John Wheeler. Use code dzone30 when checking out with any version of the book at www.manning.com. Create the domain model We’ll need a domain object for leads, so listing 1 shows the one we’ll use. It’s not an industrial-strength representation, but it will do for the purposes of the tutorial. Listing 1. Lead.java, a basic domain object for leads. package crm.model;... other imports ...public class Lead { private static DateFormat dateFormat = new SimpleDateFormat(); private String firstName; private String middleInitial; private String lastName; private String address1; private String address2; ... other fields ... public Lead() { } public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } ... other getters and setters, and a toString() method ...} There is nothing special happening here at all. So far the Lead class is just a bunch of getters and setters. You can see the full code listing in the download. If you thought that was underwhelming, just wait until you see the LeadServiceImpl service bean in listing 2. Listing 2. LeadServiceImpl.java, a dummy service bean. package crm.service;import java.util.logging.Logger;import org.springframework.stereotype.Service;import crm.model.Lead;@Service("leadService")public class LeadServiceImpl implements LeadService { private static Logger log = Logger.getLogger("global"); public void createLead(Lead lead) { log.info("Creating lead: " + lead); } This is just a dummy bean. In real life we’d save the lead to a database. The bean implements a basic LeadService interface that we've suppressed here, but it's available in the code download. Now that we have our domain model, let’s use Spring Integration to create a service integration tier above it. Create the service integration tier If you look back at figure 3, you’ll see that the CRM app pushes lead data to the service bean by way of a channel called newLeadChannel. While it’s possible for the CRM app to push messages onto the channel directly, it’s generally more desirable to keep the systems you’re integrating decoupled from the underlying messaging infrastructure, such as channels. That allows you to configure service orchestrations dynamically instead of having to go into the code. Spring Integration supports the Gateway pattern (described in the aforementioned Enterprise Integration Patterns book), which allows an application to push messages onto the message bus without knowing anything about the messaging infrastructure. Listing 3 shows how we do this. Listing 3. LeadGateway.java, a gateway offering access to the messaging system. package crm.integration.gateways;import org.springframework.integration.annotation.Gateway;import crm.model.Lead;public interface LeadGateway { @Gateway(requestChannel = "newLeadChannel") void createLead(Lead lead);} We are of course using the Spring Integration @Gateway annotation to map the method call to the newLeadChannel, but gateway clients don’t know that. Spring Integration will use this interface to create a dynamic proxy that accepts a Lead instance, wraps it with an org.springframework.integration.core.Message, and then pushes the Message onto the newLeadChannel. The Lead instance is the Message body, or payload, and Spring Integration wraps the Lead because only Messages are allowed on the bus. We need to wire up our message bus. Figure 4 shows how to do that with an application context configuration file. Listing 4. /WEB-INF/applicationContext-integration.xml message bus definition. The first thing to notice here is that we've made the Spring Integration namespace our default namespace instead of the standard beans namespace. The reason is that we're using this configuration file strictly for Spring Integration configuration, so we can save some keystrokes by selecting the appropriate namespace. This works pretty nicely for some of the other Spring projects as well, such as Spring Batch and Spring Security. In this configuration we've created the three messaging components that we saw in figure 3. First, we have an incoming lead gateway to allow applications to push leads onto the bus. We simply reference the interface from listing 3; Spring Integration takes care of the dynamic proxy. Next we create a publish/subscribe ("pub-sub") channel called newLeadChannel. This is the channel that the @Gateway annotation referenced in listing 3. A pub-sub channel can publish a message to multiple endpoints simultaneously. For now we have only one subscriber—a service activator—but we already know we're going to have others, so we may as well make this a pub-sub channel. The service activator is an endpoint that allows us to bring our LeadServiceImpl service bean onto the bus. We're injecting the newLeadChannel into the input end of the service activator. When a message appears on the newLeadChannel, the service activator will pass its Lead payload to the leadService bean's createLead() method. Stepping back, we've almost implemented the design described by figure 3. The only part that remains is the lead creation frontend, which we'll address right now. Create the web tier Our user interface for creating new leads will be a web-based form that we implement using Spring Web MVC. The idea is that enrollment staff at campuses or call centers might use such an interface to handle walk-in or phone-in traffic. Listing 5 shows our simple @Controller. Listing 5. LeadController.java, a @Controller to allow staff to create leads package crm.web;import java.util.Date;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Controller;import org.springframework.ui.Model;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RequestMethod;import crm.integration.gateways.LeadGateway;import crm.model.Country;import crm.model.Lead;@Controllerpublic class LeadController { @Autowired private LeadGateway leadGateway; @RequestMapping(value = "/lead/form.html", method = RequestMethod.GET) public void getForm(Model model) { model.addAttribute(Country.getCountries()); model.addAttribute(new Lead()); } @RequestMapping(value = "/lead/form.html", method = RequestMethod.POST) public String postForm(Lead lead) { lead.setDateCreated(new Date()); leadGateway.createLead(lead); return "redirect:form.html?created=true"; } This isn't an industrial-strength controller as it doesn't do HTTP parameter whitelisting (for example, via an @InitBinder method) and form validation, both of which you would expect from a real implementation. But the main pieces from a Spring Integration perspective are here. We're autowiring the gateway into the @Controller, and we have methods for serving up the empty form and for processing the submitted form. The getForm() method references a Countries class that we've suppressed (it's in the code download); it just puts a list of countries on the model so the form can present a Country field to the staff member. The postForm() method invokes the createLead() method on the gateway. This will pass the Lead to the dynamic proxy LeadGateway implementation, which in turn will wrap the Lead with a Message and then place the Message on the newLeadChannel. There are a few other configuration files you will need to put in place, including web.xml, main-servlet.xml and applicationContext.xml. There's also a JSP for the web form. As none of these relates directly to Spring Integration, we won't treat them here. Please see the code download for details. With that, we've established a baseline system. To try it out, run mvn jetty:run against crm/pom.xml and point your browser at http://localhost:8080/crm/main/lead/form.html You should see a very basic-looking web form for entering lead information. Enter some user information (it doesn't matter what you enter—recall that we don't have any form validation) and press Submit. The console should report that LeadServiceImpl.createLead() created a lead. Congratulations! Even though we now have a working system, it isn't very interesting. From here on out (this tutorial and the next) we'll be adding some common features to make the lead management system more capable. Our first addition will be confirmation e-mails; the next tutorial will present further additions. Adding confirmation e-mails [The source for this section is available here] After an enrollment advisor (or some other staff member) creates a lead in the system, we want to send the lead an e-mail letting him know that that's happened. Actually—and this is a critical point—we really don't care how the lead was created. Anytime a lead appears on the newLeadChannel, we want to fire off a confirmation e-mail. I'm making the distinction because it points to an important aspect of the message bus: it allows us to control lead processing code centrally instead of having to chase it down in a bunch of different places. Right now there's only one way to create leads, but figure 2 revealed that we'll be adding others. No matter how many we add, they'll all result in sending a confirmation e-mail out to the lead. Figure 4 shows the new bit of plumbing we're going to add to our message bus. Figure 4. Send a confirmation e-mail when creating a lead. To do this, we're going to need to make a few changes to the configuration and code. POM changes First we need to update the POM. Here's a summary of the changes; see the code download for details: • Add a JavaMail dependency to the Jetty plug-in. • Add an org.springframework.context.support dependency. • Add a spring-integration-mail dependency. • Set the mail.version property. These changes will allow us to use JavaMail. Expose JavaMail sessions through JNDI We'll also need to add a /WEB-INF/jetty-env.xml configuration to make our JavaMail sessions available via JNDI. Once again, see the code download for details. I've included a /WEB-INF/jetty-env.xml.sample configuration for your convenience. As mentioned previously, you'll need access to an SMTP server. Besides creating jetty-env.xml, we'll need to update applicationContext.xml. Listing 6 shows the changes we need so we can use JavaMail and SMTP. Listing 6. /WEB-INF/applicationContext.xml changes supporting JavaMail and SMTP The changes expose JavaMail sessions as a JNDI resource. We've declared the jee namespace and its schema location, configured the JNDI lookup, and created a JavaMailSenderImpl bean that we'll use for sending mail. We won't need any domain model changes to generate confirmation e-mails. We will however need to create a bean to back our new transformer endpoint. Service integration tier changes First, recall from figure 4 that the newLeadChannel feeds into a LeadToEmailTransformer endpoint. This endpoint takes a lead as an input and generates a confirmation e-mail as an output, and the e-mail gets pipes out to an SMTP transport. In general, transformers transform given inputs into desired outputs. No surprises there. Figure 4 is slightly misleading since it's actually the POJO itself that we're going to call LeadToEmailTransformer; the endpoint is really just a bean adapter that the messaging infrastructure provides so we can place the POJO on the message bus. Listing 7 presents the LeadToEmailTransformer POJO. Listing 7. LeadToEmailTransformer.java, a POJO to generate confirmation e-mails package crm.integration.transformers;import java.util.Date;import java.util.logging.Logger;import org.springframework.integration.annotation.Transformer;import org.springframework.mail.MailMessage;import org.springframework.mail.SimpleMailMessage;import crm.model.Lead;public class LeadToEmailTransformer { private static Logger log = Logger.getLogger("global"); private String confFrom; private String confSubj; private String confText; ... getters and setters for the fields ... @Transformer public MailMessage transform(Lead lead) { log.info("Transforming lead to confirmation e-mail: " + lead); String leadFullName = lead.getFullName(); String leadEmail = lead.getEmail(); MailMessage msg = new SimpleMailMessage(); msg.setTo(leadFullName == null ? leadEmail : leadFullName + " <" + leadEmail + ">"); msg.setFrom(confFrom); msg.setSubject(confSubj); msg.setSentDate(new Date()); msg.setText(confText); log.info("Transformed lead to confirmation e-mail: " + msg); return msg; } Again, LeadToEmailTransformer is a POJO, so we use the @Transformer annotation to select the method that's performing the transformation. We use a Lead for the input and a MailMessage for the output, and perform a simple transformation in between. When defining backing beans for the various Spring Integration filters, it's possible to specify a Message as an input or an output. That is, if we want to deal with the messages themselves rather than their payloads, we can do that. (Don't confuse the MailMessage in listing 7 with a Spring Integration message; MailMessage represents an e-mail message, not a message bus message.) We might do that in cases where we want to read or manipulate message headers. In this tutorial we don't need to do that, so our backing beans just deal with payloads. Now we'll need to build out our message bus so that it looks like figure 4. We do this by updating applicationContext-integration.xml as shown in listing 8. Listing 8. /WEB-INF/applicationContext-integration.xml updates to support confirmation e-mails The property-placeholder configuration loads the various ${...} properties from a properties file; see /crm/src/main/resources/applicationContext.properties in the code download. You don't have to change anything in the properties file. The transformer configuration brings the LeadToEmailTransformer bean into the picture so it can transform Leads that appear on the newLeadChannel into MailMessages that it puts on the confEmailChannel. As a side note, the p namespace way of specifying bean properties doesn't seem to work here (I assume it's a bug: http://jira.springframework.org/browse/SPR-5990), so I just did it the more verbose way. The channel definition defines a point-to-point channel rather than a pub-sub channel. That means that only one endpoint can pull messages from the channel. Finally we have an outbound-channel-adapter that grabs MailMessages from the confEmailChannel and then sends them using the referenced mailSender, which we defined in listing 6. That's it for this section. We should have working confirmation e-mails. Restart your Jetty instance and go again to http://localhost:8080/crm/main/lead/form.html Fill it out and provide your real e-mail address in the e-mail field. A few moments after submitting the form you should receive a confirmation e-mail. If you don't see it, you might check your SMTP configuration in jetty-env.xml, or else check your spam folder. Summary In this tutorial we've taken our first steps toward developing an integrated lead management system. Though the current bus configuration is simple, we've already seen some key Spring Integration features, including • support for the Gateway pattern, allowing us to connect apps to the message bus without knowing about messages • point-to-point and pub-sub channels • service activators to allow us to place service beans on the bus • message transformers • outbound SMTP channel adapters to allow us to send e-mail The second tutorial will continue elaborating what we've developed here, demonstrating the use of several additional Spring Integration features, including • message routers (including content-based message routers) • outbound web service gateways for sending SOAP messages • inbound HTTP adapters for collecting HTML form data from external systems • inbound e-mail channel adapters (we'll use IMAP IDLE, though POP and IMAP are also possible) for processing incoming e-mails Enjoy, and stay tuned. Willie is a solutions architect with 12 years of Java development experience. He and his brother John are coauthors of the upcoming book Spring in Practice by Manning Publications (www.manning.com/wheeler/). Willie also publishes technical articles (including many on Spring) to wheelersoftware.com/articles/.
August 18, 2009
by Willie Wheeler
· 249,506 Views · 3 Likes
article thumbnail
JPA 2.0 Concurrency and Locking
Optimistic locking lets concurrent transactions process simultaneously, but detects and prevent collisions, this works best for applications where most concurrent transactions do not conflict. JPA Optimistic locking allows anyone to read and update an entity, however a version check is made upon commit and an exception is thrown if the version was updated in the database since the entity was read. In JPA for Optimistic locking you annotate an attribute with @Version as shown below: public class Employee { @ID int id; @Version int version; The Version attribute will be incremented with a successful commit. The Version attribute can be an int, short, long, or timestamp. This results in SQL like the following: “UPDATE Employee SET ..., version = version + 1 WHERE id = ? AND version = readVersion” The advantages of optimistic locking are that no database locks are held which can give better scalability. The disadvantages are that the user or application must refresh and retry failed updates. Optimistic Locking Example In the optimistic locking example below, 2 concurrent transactions are updating employee e1. The transaction on the left commits first causing the e1 version attribute to be incremented with the update. The transaction on the right throws an OptimisticLockException because the e1 version attribute is higher than when e1 was read, causing the transaction to roll back. Additional Locking with JPA Entity Locking APIs With JPA it is possible to lock an entity, this allows you to control when, where and which kind of locking to use. JPA 1.0 only supported Optimistic read or Optimistic write locking. JPA 2.0 supports Optimistic and Pessimistic locking, this is layered on top of @Version checking described above. JPA 2.0 LockMode values : OPTIMISTIC (JPA 1.0 READ): perform a version check on locked Entity before commit, throw an OptimisticLockException if Entity version mismatch. OPTIMISTIC_FORCE_INCREMENT (JPA 1.0 WRITE) perform a version check on locked Entity before commit, throw an OptimisticLockException if Entity version mismatch, force an increment to the version at the end of the transaction, even if the entity is not modified. PESSIMISTIC: lock the database row when reading PESSIMISTIC_FORCE_INCREMENT lock the database row when reading, force an increment to the version at the end of the transaction, even if the entity is not modified. There are multiple APIs to specify locking an Entity: EntityManager methods: lock, find, refresh Query methods: setLockMode NamedQuery annotation: lockMode element OPTIMISTIC (READ) LockMode Example In the optimistic locking example below, transaction1 on the left updates the department name for dep , which causes dep's version attribute to be incremented. Transaction2 on the right gives an employee a raise if he's in the "Eng" department. Version checking on the employee attribute would not throw an exception in this example since it was the dep Version attribute that was updated in transaction1. In this example the employee change should not commit if the department was changed after reading, so an OPTIMISTIC lock is used : em.lock(dep, OPTIMISTIC). This will cause a version check on the dep Entity before committing transaction2 which will throw an OptimisticLockException because the dep version attribute is higher than when dep was read, causing the transaction to roll back. OPTIMISTIC_FORCE_INCREMENT (write) LockMode Example In the OPTIMISTIC_FORCE_INCREMENT locking example below, transaction2 on the right wants to be sure that the dep name does not change during the transaction, so transaction2 locks the dep Entity em.lock(dep, OPTIMISTIC_FORCE_INCREMENT) and then calls em.flush() which causes dep's version attribute to be incremented in the database. This will cause any parallel updates to dep to throw an OptimisticLockException and roll back. In transaction1 on the left at commit time when the dep version attribute is checked and found to be stale, an OptimisticLockException is thrown Pessimistic Concurrency Pessimistic concurrency locks the database row when data is read, this is the equivalent of a (SELECT . . . FOR UPDATE [NOWAIT]) . Pessimistic locking ensures that transactions do not update the same entity at the same time, which can simplify application code, but it limits concurrent access to the data which can cause bad scalability and may cause deadlocks. Pessimistic locking is better for applications with a higher risk of contention among concurrent transactions. The examples below show: reading an entity and then locking it later reading an entity with a lock reading an entity, then later refreshing it with a lock The Trade-offs are the longer you hold the lock the greater the risks of bad scalability and deadlocks. The later you lock the greater the risk of stale data, which can then cause an optimistic lock exception, if the entity was updated after reading but before locking. The right locking approach depends on your application: what is the risk of risk of contention among concurrent transactions? What are the requirements for scalability? What are the requirements for user re-trying on failure? For More Information: Preventing Non-Repeatable Reads in JPA Using EclipseLink Java Persistence API 2.0: What's New ? What's New and Exciting in JPA 2.0 Beginning Java™ EE 6 Platform with GlassFish™ 3 Pro EJB 3: Java Persistence API (JPA 1.0)
August 3, 2009
by Carol McDonald
· 51,497 Views · 1 Like
article thumbnail
Autocomplete Combobox In Java With Filtering And Inserting New Text
import java.awt.EventQueue; import java.awt.GridLayout; import java.awt.event.ActionEvent; import java.awt.event.ActionListener; import java.awt.event.KeyAdapter; import java.awt.event.KeyEvent; import java.util.ArrayList; import java.util.List; import javax.swing.AbstractListModel; import javax.swing.ComboBoxModel; import javax.swing.JComboBox; import javax.swing.JFrame; import javax.swing.JLabel; import javax.swing.Timer; import javax.swing.text.AttributeSet; import javax.swing.text.BadLocationException; import javax.swing.text.JTextComponent; import javax.swing.text.PlainDocument; import org.apache.log4j.ConsoleAppender; import org.apache.log4j.Logger; import org.apache.log4j.PatternLayout; /** * Autocomplete combobox with filtering and * text inserting of new text * @author Exterminator13 */ public class AutoCompleteCombo extends JComboBox{ private static final Logger logger = Logger.getLogger(AutoCompleteCombo.class); private Model model = new Model(); private final JTextComponent textComponent = (JTextComponent) getEditor().getEditorComponent(); private boolean modelFilling = false; private boolean updatePopup; public AutoCompleteCombo() { setEditable(true); logger.debug("setPattern() called from constructor"); setPattern(null); updatePopup = false; textComponent.setDocument(new AutoCompleteDocument()); setModel(model); setSelectedItem(null); new Timer(20, new ActionListener() { @Override public void actionPerformed(ActionEvent e) { if (updatePopup && isDisplayable()) { setPopupVisible(false); if (model.getSize() > 0) { setPopupVisible(true); } updatePopup = false; } } }).start(); } private class AutoCompleteDocument extends PlainDocument { boolean arrowKeyPressed = false; public AutoCompleteDocument() { textComponent.addKeyListener(new KeyAdapter() { @Override public void keyPressed(KeyEvent e) { int key = e.getKeyCode(); if (key == KeyEvent.VK_ENTER) { logger.debug("[key listener] enter key pressed"); //there is no such element in the model for now String text = textComponent.getText(); if (!model.data.contains(text)) { logger.debug("addToTop() called from keyPressed()"); addToTop(text); } } else if (key == KeyEvent.VK_UP || key == KeyEvent.VK_DOWN) { arrowKeyPressed = true; logger.debug("arrow key pressed"); } } }); } void updateModel() throws BadLocationException { String textToMatch = getText(0, getLength()); logger.debug("setPattern() called from updateModel()"); setPattern(textToMatch); } @Override public void remove(int offs, int len) throws BadLocationException { if (modelFilling) { logger.debug("[remove] model is being filled now"); return; } super.remove(offs, len); if (arrowKeyPressed) { arrowKeyPressed = false; logger.debug("[remove] arrow key was pressed, updateModel() was NOT called"); } else { logger.debug("[remove] calling updateModel()"); updateModel(); } clearSelection(); } @Override public void insertString(int offs, String str, AttributeSet a) throws BadLocationException { if (modelFilling) { logger.debug("[insert] model is being filled now"); return; } // insert the string into the document super.insertString(offs, str, a); // if (enterKeyPressed) { // logger.debug("[insertString] enter key was pressed"); // enterKeyPressed = false; // return; // } String text = getText(0, getLength()); if (arrowKeyPressed) { logger.debug("[insert] arrow key was pressed, updateModel() was NOT called"); model.setSelectedItem(text); logger.debug( String.format("[insert] model.setSelectedItem(%s)", text) ); arrowKeyPressed = false; } else if(!text.equals(getSelectedItem())){ logger.debug("[insert] calling updateModel()"); updateModel(); } clearSelection(); } } public void setText(String text) { if (model.data.contains(text)) { setSelectedItem(text); } else { addToTop(text); setSelectedIndex(0); } } public String getText() { return getEditor().getItem().toString(); } private String previousPattern = null; private void setPattern(String pattern) { if(pattern!=null && pattern.trim().isEmpty()) pattern = null; if(previousPattern==null && pattern ==null || pattern!=null && pattern.equals(previousPattern)) { logger.debug("[setPatter] pattern is the same as previous: "+previousPattern); return; } previousPattern = pattern; modelFilling = true; // logger.debug("setPattern(): start"); model.setPattern(pattern); if(logger.isDebugEnabled()) { StringBuilder b = new StringBuilder(100); b.append("pattern filter '").append(pattern==null ? "null" : pattern).append("' set:\n"); for(int i=0; i list = new ArrayList(limit); private List lowercase = new ArrayList(limit); private List filtered; void add(String s) { list.add(s); lowercase.add(s.toLowerCase()); } void addToTop(String s) { list.add(0, s); lowercase.add(0, s.toLowerCase()); } void remove(int index) { list.remove(index); lowercase.remove(index); } List getList() { return list; } List getFiltered() { if(filtered==null) filtered = list; return filtered; } int size() { return list.size(); } void setPattern(String pattern) { if (pattern == null || pattern.isEmpty()) { filtered = list; AutoCompleteCombo.this.setSelectedItem(model.getElementAt(0)); logger.debug( String.format("[setPattern] combo.setSelectedItem(null)") ); } else { filtered = new ArrayList(limit); pattern = pattern.toLowerCase(); for(int i=0; isize2) { fireIntervalRemoved(this, size2, size1-1); fireContentsChanged(this, 0, size2-1); } } public void addToTop(String aString) { if(aString==null || data.contains(aString)) return; if(data.size()==0) data.add(aString); else data.addToTop(aString); while(data.size()>limit) { int index = data.size()-1; data.remove(index); } setPattern(null); model.setSelectedItem(aString); logger.debug( String.format("[addToTop] model.setSelectedItem(%s)", aString) ); //saving into options if (data.size() > 0) { writeData(); } } @Override public Object getSelectedItem() { return selected; } @Override public void setSelectedItem(Object anObject) { if ((selected != null && !selected.equals(anObject)) || selected == null && anObject != null) { selected = (String) anObject; fireContentsChanged(this, -1, -1); } } @Override public int getSize() { return data.getFiltered().size(); } @Override public Object getElementAt(int index) { return data.getFiltered().get(index); } } public static void main(String[] args) { EventQueue.invokeLater(new Runnable() { @Override public void run() { // Logger root = Logger.getRootLogger(); // root.addAppender(new ConsoleAppender(new PatternLayout("%d{ISO8601} [%5p] %m at %l%n"))); Logger root = Logger.getRootLogger(); root.addAppender(new ConsoleAppender(new PatternLayout("%d{ISO8601} %m at %L%n"))); // BasicConfigurator.configure(); JFrame frame = new JFrame(); frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); frame.setLayout(new GridLayout(3, 1)); final JLabel label = new JLabel("label "); frame.add(label); final AutoCompleteCombo combo = new AutoCompleteCombo(); // combo.getEditor().getEditorComponent().addKeyListener(new KeyAdapter() { // // @Override // public void keyReleased(KeyEvent e) { // if (e.getKeyCode() == KeyEvent.VK_ENTER) { // String text = combo.getEditor().getItem().toString(); // if(text.isEmpty()) // return; // combo.addToTop(text); // } // } // }); frame.add(combo); JComboBox combo2 = new JComboBox(new String[] {"Item 1", "Item 2", "Item 3", "Item 4"}); combo2.setEditable(true); frame.add(combo2); frame.pack(); frame.setSize( 500, frame.getHeight() ); frame.setLocationRelativeTo(null); frame.setVisible(true); } }); } }
July 25, 2009
by Snippets Manager
· 19,423 Views
article thumbnail
Android vs iPhone Development: A Comparison
A few months ago I ventured into the world of Mobile development and created an application (Hudson Helper) for both iPhone and Android. This article is about my experiences, comparing Android and iPhone development with a focus on tools, platform and the developer experience. Before going much further I should note that my comparison is with considerable bias. I’ve spent the past 12+ years in Java development, having spent much of my career building developer tools. Since January of 2004 I’ve been building plug-ins for Eclipse, and before that plug-ins for NetBeans. This bias is somewhat tempered with several years of C and C++ development. With this background I find that I’m very critical of developer tools. Developer productivity is key — anything that takes away from the flow of a developer in the zone is a real problem. Language, Programming Model and Platform Language The language of choice for iPhone development is Objective-C. Objective-C is a language based on C with extensions for object-oriented concepts, such as classes, inheritance, interfaces, messages, dynamic typing, etc. The Java language is used when developing for Android (though it doesn’t actually get compiled to bytecode). Java is a no-brainer. I have to say that it’s nice not to have to learn a new language to target mobile. Existing skillsets don’t come easy — so reuse of expertise is worth a lot. It took a little while to wrap my head around some of the language features available with Objective-C. I soon discovered that I really loved certain language features, such as message passing (instead of calling methods), categories and named arguments. I did find however that the syntax of Objective-C is cumbersome. I’m still not used to ‘+’ and ‘-’ for static and member methods, too many parentheses are required, and in general I just felt like I had to type way to much to express a simple concept. The IDE didn’t help much with this either (more on that later). One thing that really became clear to me is that Objective-C, though it may have been visionary for its time, is really a language of the '80s. Certain issues such as split header and implementation files and violation of DRY are really time-wasters, and not small ones at that. I found myself constantly switching back and forth between files, which not only has a cost in navigation (which file to open?) but with every file opened your sense of context must be recreated (where’s the caret, what’s selected, where am I in the file, how is this file organized). As far as DRY, must I really do 5 things to declare a property?? (declare in the class definition, again to declare getter/settter, initialize in the init method, @synthesize in the implementation, release in dealloc). Here’s what I mean: Server.h @interface Server : Updatable {NSString *name; <-- declare the property }@property (nonatomic,retain) NSString *name; <--- declare the property again Server.m @synthesize name; <-- implement getter/setter-(void) dealloc {[name release]; <-- release memory} If you ask me, everything in Server.m should go away. Another gotcha here is the positional relevance of @synthesize. Java has a similar problem with properties, though not quite so bad — and the IDE helps you write your getter/setter. Pointers in Objective-C, though powerful, are also another time-waster. This is where Java really shines with its garbage collection. I found that I was constantly considering whether allocated objects were freed appropriately. Code flow is poor since application logic is littered with memory management. I only have so many brain cycles available — why do I have to think about this other cruft that’s not really a core concern of the application domain? Of course this gets even worse when trying to figure out where things went wrong if you make a mistake. Zombies help, but still don’t make it obvious if you’ve accessed something that was deallocated. Other issues include deallocating something twice, autoreleasing something twice. I also found it non-intuitive when to retain return values from methods. Another annoyance of Objective-C is the patterns that must be followed: implementing correct init and dealloc methods is non-trivial. @synthesized getters and setters for properties with retain should not be called in these methods. So many conventions and rules to remember! Though I understand why there’s a separation of alloc and init, it’s still overly wordy to specify [[aloc Foo] initWithArg: arg]. Why not just [new Foo arg]? Or how about new Foo(arg) -- oh, wait, that’s just like Java! Objective-C’s imports and forward-declarations (@class) are a pain. Though these issues exist with Java development, Eclipse’s JDT is so good that I’ve almost forgotten what it’s like to write an import. All you have to do is Ctrl+Space to auto-complete a class name or Ctrl+Shift+O to organize imports and voila! Of course Java is not perfect either, however this fact is hidden from me due to the fact that I’ve been living in Java for a very long time. Sometimes I wish that Java were more Groovy-like, however I’m used to it and the tooling is so good. Platform On Android I found that I could readily use the Java runtime classes. Some, but not all, of the standard Java RT classes are available on Android. I didn’t find this a problem, since most of the standard Java IO, network and regex libraries are available. Android RT classes appear to be based on Harmony, which has been around long enough to be stable. With iPhone on the other hand, finding the functionality that I needed was painful. Classes and methods are poorly organized. When to look for a static method versus a class with members was not clear to me. Also depending on the framework used, naming conventions and code organization would differ. I suppose this is the legacy of an older platform. Areas where functionality was lacking that I found painful were regular expressions, string handling and XML parsing. I ended up using the excellent Regex Kit Lite for regular expressions. For XML parsing I implemented a parser abstraction over libxml, only to discover later that I may have had an easier time with NSXMLParser which is a lot more like SAX. On the iPhone when things didn’t work as expected I had to resort to Google and hope that others had encountered the same problem. This technique was hampered by Apple’s earlier NDA policy, which meant that iPhone content is pretty thin on the net. In some cases I would resort to guesswork and experimentation to find a solution. Android has the benefit of being open source. Within minutes I had the full Android platform source code on my system, and had re-built the SDK from sources to ensure that the source I had matched the runtime classes in the emulator. So not only could I see how things were implemented in the Android platform and learn by example, I could step through the platform code in the emulator and discover why my code wasn’t producing the desired results. In general I found the layout, organization, and naming conventions of Android platform classes was consistent and predictable. This made it much easier to learn. Programming Model The iPhone platform does a great job of encouraging an MVC design pattern. With this design pattern built in to the platform, building the UI was simple and I didn’t have to figure out how to organize the UI component design myself. It also means that when looking at sample code, it’s all organized in the same way. Android also does a good job with design patterns, though their concepts varied significantly from the iPhone. With Android’s support for multiple processes and component reuse, the platform itself provides support for Intents and Activities (an Intent is just a variant of a command). The design results in a better user experience, however it does introduce some complexity for the developer: when starting one Activity from another, an Intent is used to communicate any parameters. These parameters cannot be passed by reference — only by value. Where on the iPhone it’s simple to have screens sharing the same data structures, on Android this requires some forethought. Apparently Android applications can manage the back button and have everything occur inside a single Activity, however this is not the norm. Both Android and iPhone provide a way of declaring user preferences in XML. Both platforms provide a default UI for editing those preferences, which is great. Android’s XML format is extensible allowing custom UI components to be integrated, which makes user preferences a breeze. iPhone developers that wish to customize preferences will have to implement a UI from scratch, which is a lot more work. Testing and Continuous Integration I’m of the opinion that every development effort should include unit tests. Teams of size greater than one should also include Continuous Integration. Android developers will be happy to know that they can write JUnit tests. I could even launch these from the Eclipse UI after some classpath fiddling. Though I didn’t try it, I assume that it’s trivial to run these from Ant and your favorite CI server such as Hudson. I did see some iPhone unit test documentation with the iPhone SDK but didn’t take the time to explore it — so I can’t comment there. Resources Apple does an excellent job of providing lots of resources for developers. Important concepts are explained in videos, which makes grasping concepts easy — however I did find that videos progressed slowly and I was watching for what seemed like hours to find information that should have taken minutes. Luckily Apple also provides lots of sample applications and code to demonstrate API usage. Android developers also have access to loads of resources. The guide and API reference are installed with the SDK, so everything is available when offline (which for me is important since I do a lot of my work in transit). I found the Android development resources better organized and spent less time looking and more time finding. In particular the ApiDemos sample app provides a great starting point. I also downloaded many open source Android projects for ideas on architecture and API usage. This is an area where Android has the advantage, with Apple’s previous NDA policy there isn’t much out there in terms of open source for iPhone. Tooling For me tooling was a real shocker. These are the categories of tooling that I’ll cover: IDE, UI builder, debugger, profiler. Almost everything else is related to provisioning, and in that area I didn’t notice much in the way of differences between Android and iPhone. IDE Android development leverages the excellent JDT tools, which are pretty much stock and standard with every Eclipse installation. I’ve used these tools now for many years and they’re excellent. Everything Java is indexed, the IDE has a rich model of the source code, and refactoring is so seamless that it has changed the way that I work. Perhaps the best feature of JDT is its incremental compiler, which provides immediate feedback with errors and warnings as you type. This eliminates the code-compile-wait-for-feedback cycle that was so common in the '80s and '90s. Errors and warnings are updated in the Java editor as I type, giving me instant feedback. I didn’t realize just how valuable this feature is until I was coding Objective-C in XCode — when I became acutely aware at how waiting for compiler feedback can break the flow of programming. Other key features that make Eclipse so amazing to work with are: content assist quick-fixes organize imports open type (CTRL+Shift+T) refactorings Integrated javadoc and content assist is quite possibly the best way to learn an unfamiliar API. In Ecipse not only are all classes and methods immediately available in the context in which you’re writing code, their documentation is presented alongside. Content Assist with Integrated Javadoc XCode is so shockingly bad that I almost don’t know where to start. Here’s a minimum list of things that I think need fixing in order for XCode to become a viable IDE: Content assist that actually works. Content assist provided by XCode is often wrong, and almost always suggests a small subset of what’s actually available. A decent window/editor management system. XCode and it’s associated tools (debugger) like to open lots of windows. Want to open a file? How about a new window for you! Very quickly I found myself in open-window-hell. The operating system’s window management is designed for managing multiple applications, not multiple editors within an IDE. It’s simply not capable of providing management of editors in an environment as sophisticated as an IDE. A project tree view that sorts files alphabetically. Really! Integrated API documentation. I found that I was constantly switching out of the IDE and searching for API documentation using Appkido. This may seem trivial, but it really breaks the flow. One area of Eclipse that simply can’t be matched is Mylyn. Integrated task management and a focused interface introduce huge efficiencies into any project, small or large. If you haven’t yet tried out Mylyn, it’s definitely worth your time to take a look. A good place to start is Mylyn’s Getting Started page. UI Builder iPhone app developers are given a pretty good UI builder. It does a great job of showing the UI as it will actually appear. It’s flexible and can model some pretty sophisticated UIs, so I was impressed. I found that using it was a little tricky — I had to read the documentation two or three times before I could really figure out how to use it properly. The Android UI builder I found pretty useless: it can’t display UIs how they’ll actually appear, and it’s UI is way too inefficient. I found that I coded all of the UIs directly in the XML source view of the UI builder. There the content assist and validation were pretty good, making it the easiest way for me to build a UI. Debugger Having used to the Java debugger in Eclipse I was shocked at the state of the debugger in XCode. With Eclipse I can see and modify variable values. Not so in XCode. Maybe this is simply the state of affairs when debugging native code, but it sure affects the usefulness of the debugger. XCode often seemed confused as to the type of an object and presented me with a pointer value and no detail. This is a sharp contrast to Eclipse, where I can drill down through an object graph with ease. I found the XCode debugger UI extremely difficult to use. Clicking on the stack to show code in an editor caused new windows to open, eventually resulting in dozens of windows open. In addition I found that watch expressions rarely worked for me. Profiler and Heap Analysis An area where Apple development tools excel is in profiling and heap analysis. These tools seemed mature and easy to use. With no prior experience with these specific tools I was able to gain a better understanding of my app within minutes, find and fix several memory leaks and improve performance. XCode Memory Leak Detection Android developers must use Android’s traceview application, which I found worked well but required significantly more effort to configure and operate. I was surprised to find that the source code must be changed in order to get the trace files required for analysis. I’m not sure if Android can provide heap dumps in hprof format. If it can then the awesome MAT tool could be used to analyze heap usage. According to this article Android can produce hprof heap data, though I haven’t tried it. App Store It goes without saying that the iPhone app store is excellent in that you can sell into many countries worldwide with a single setup. I was able to provide my Canadian bank account number, sign a few legal agreements and I was up and running. Getting an app into the store however is frustrating to say the least. Apple must approve every app before it is accepted into the store. Mine got rejected multiple times. Each time it was rejected I was given almost no information about why. When I emailed them to clarify the problem, I received what looked like a canned response indicating that I should refer to previous correspondence. If it weren’t so frustrating I would have found it funny. I highly recommend reading Brian Stormont’s Avoiding iPhone App Rejection from Apple and Dan Grigsby’s Part 2 follow-up. Of course once I started selling Hudson Helper I realized that Apple won’t send me any money unless the payout is greater than $250. This is true not only of the first payout, but every payout. Google market on the other hand requires a minimum of $1 for each payout. Both the iPhone app store and Google market take about %30 of your app selling price. $0.99 applications have to have high volume, or they’re simply not worth your time. The Google market by comparison to the Apple app store is terrible in that you can only sell into a handful of countries. You also can’t see or install apps that cost money on a developer phone. Actually you can, but not if the app has copy protection — which is almost every non-free app. On the other hand when you upload your app to the app store it’s available within minutes, so you don’t have to worry about an approval process. To set up a merchant account with Google market, I had to provide a US address and bank account number, since Google doesn’t support Canada. For me this was a pain, but not too bad since I live within a few kilometers of the US border. I rode my bike down to the US and opened an account with Horizon bank. The bank required a passport and driver’s license, so no problem there. Why Google doesn’t support more countries I don’t know. At the very least Google market should accept alternate payment methods for countries that are not supported by Google checkout. Summary Android’s platform and developer tools are excellent. Leveraging Java and the Eclipse IDE are major winning factors for Android. Apple’s developer tools are shockingly bad by comparison. The Objective-C language and platform APIs are cumbersome and poorly organized. Overall when developing for the iPhone I felt like I was back in 1993. These factors combined in my estimation make application development about three times more expensive when developing for iPhone. The only area where Apple’s developer tools excelled was in profiling and heap analysis. Apple’s app store from a user’s standpoint and from a worldwide coverage standpoint are excellent. In this area Google market for Android is weak. Development for iPhone may improve as tools such as iphonical (MDD for iPhone) and objectiveclipse (Eclipse plug-in for Objective-C) emerge. We may see a shake-up in the mobile market, with at least 18 new Android handsets being released this year. Until that happens, iPhone will remain a market leader and developers will have to put up with XCode and Objective-C. For me, my love is with Android. Sure, the iPhone is great — but can you install a new Linux kernel? From http://greensopinion.blogspot.com/
July 6, 2009
by David Green
· 93,204 Views
article thumbnail
JMS Over HTTP Using OpenMQ (Sun Java System Message Queue and HTTP Tunneling)
You may have already faced a situation where you need to have messaging capabilities in your application while your client application runs in an environment, which you have no control over its network configuration and restrictions. Such situation lead to the fact that you can not use JMS communication over designated ports like 7676 and so. You may simply put JMS away and follow another protocol or even plain HTTP communication to address your architecture and design requirement, but we can not easily put pure JMS API capabilities away as we may need durable subscription over a proven and already well tested and established design and architecture. Different JMS implementation providers provide different type capabilities to address such a requirement. ActiveMQ provide HTTP and HTTPS transport for JMS API and described it here. OpenMQ provide different transport protocol and access channels to access OpenMQ functionalities from different prgramming . One of the access channles which involve HTTP is named Universal Message Service (UMS) which provide a simple REST interaction template to place messages and comsume them from any programming language and device which can interact with a network server. UMS has some limitations which is simply understandable based on the RESTful nature of UMS. For current version of OpenMQ, it only supports Message Queues as destinations so there is no publish-subscribe functionality available. Another capability which involve HTTP is JMS over HTTP or simply JMS HTTP tunneling. OpenMQ JMS implementation supports HTTP/s as transport protocol if we configure the message broker properly. Well I said the JMS implementation support HTTP/S as a transport protocol which means we as user of the JMS API see almost no difference in the way that we use the JMS API. We can use publish-subscribe, point to point, durable subscription, transaction and whatever provided in the JMS API. First, lets overview the OpenMQ installation and configuration then we will take a look at an example to see what changes between using JMS API and JMS API over HTTP transport. Installation process is as follow: OpenMQ project provides a Java EE web application which interact with OpenMQ broker from one side and Sun JMS implementation on the other side. Interaction of this application with the client and MQ broker is highly customizable in different aspects like Broker port number, client poll inrval, Broker address and so on. Download OpenMQ from https://mq.dev.java.net/ it should be a zip which is different for each platform. Install the MQ by unzipping the above file and running ./installer or installer.bat or so. After the installation completed, go to install_folder/var/mq/instances/imqbroker/props and open the config.properties in a text editor like notepad or emeditor or gedit. Add the following line to the end of the above file: imq.service.activelist=jms,admin,httpjms Now goto install_folder/mq/lib/ and pick the imqhttp.war file. deploy the file into your Servlet container or application server (I went with GlassFish). After you deployed the file start the application server or Servlet container Now it is time to start the MQ broker: launch a terminal or cmd console and goto install_folder/mq/bin now execute ./imqbrokerd -port 7979 (it maybe like imqbrokerd.bat -port 7979 for Windows ) This command will start the MQ broker and keep it listening on port 7979 for incoming connection To test the overall operations: Open a browser and tray to surf http://127.0.0.1:8080/imqhttp/tunnel or whatever URL which points to the newly deployed application . If you saw "HTTP tunneling Servlet ready." as the first line in the response page then we are ready for last step. Now let's see how we can publish some messages, this sample code assume that we have configured the message broker and assumes that we have the following two JAR files in the classpath. These JAR files are available in install_folder/mq/lib/ imq.jar jms.jar Now the Publisher code: public class Publisher { public void publish(String messageContent) { try { String addressList = "http://127.0.0.1:8080/imqhttp/tunnel"; com.sun.messaging.TopicConnectionFactory topicConnectionFactory = new com.sun.messaging.TopicConnectionFactory(); topicConnectionFactory.setProperty(com.sun.messaging.ConnectionConfiguration.imqAddressList, addressList); javax.jms.Topic top; javax.jms.Connection con = topicConnectionFactory.createTopicConnection("admin", "admin"); javax.jms.Session session = con.createSession(false, javax.jms.Session.AUTO_ACKNOWLEDGE); top = session.createTopic("DIRECT_TOPIC"); MessageProducer prod = session.createProducer(top); Message textMessage = session.createTextMessage(messageContent); prod.send(textMessage); prod.close(); session.close(); con.close(); } catch (JMSException ex) { ex.printStackTrace(); } } public static void main(String args[]) { Publisher p = new Publisher(); for (int i = 1; i < 10; i++) { p.publish("Sample Text Message Content: " + i); } } } As you can see the only difference is the connection URL which uses http instead of mq and point to a Servlet container address instead of pointing to the Broker listening address. The subscriber sample code follow a similar pattern. Here I write a sample durable subscriber so you can see that we can use durable subscribtion over HTTP. But you should note that HTTP transport uses polling and continuesly open communication channel which can introduce some overload on the server. class SimpleListener implements MessageListener { public void onMessage(Message msg) { System.out.println("On Message Called"); if (msg instanceof TextMessage) { try { System.out.print(((TextMessage) msg).getText()); } catch (JMSException ex) { ex.printStackTrace(); } } } } public class Subscriber { /** * @param args the command line arguments */ public void subscribe(String clientID, String susbscritpionID) { try { // TODO code application logic here String addressList = "http://127.0.0.1:8080/imqhttp/tunnel"; com.sun.messaging.TopicConnectionFactory topicConnectionFactory = new com.sun.messaging.TopicConnectionFactory(); topicConnectionFactory.setProperty(com.sun.messaging.ConnectionConfiguration.imqAddressList, addressList); javax.jms.Topic top; javax.jms.Connection con = topicConnectionFactory.createTopicConnection("admin", "admin"); con.setClientID(clientID); javax.jms.Session session = con.createSession(false, javax.jms.Session.AUTO_ACKNOWLEDGE); top = session.createTopic("DIRECT_TOPIC"); TopicSubscriber topicSubscriber = session.createDurableSubscriber(top, susbscritpionID); topicSubscriber.setMessageListener(new SimpleListener()); con.start(); } catch (JMSException ex) { ex.printStackTrace(); } } public static void main(String args[]) { Subscriber sub = new Subscriber(); sub.subscribe("C19", "C1_011"); } } Now how you can test the entire example and monitor the MQ? it is very simple by utilizing the provided tools. Do the following steps to test the overall durable subscription system: Run a subscriber Run another subscriber with a different client ID Run a publisher once or twice Kill the second subscriber Run a publisher once Run the subscriber and you can see that the subscriber will fetch all messages which arrived after it shut down-ed. Note that we can not have two separate client with same client ID running because the broker will not be able to distinguish which client it should send the messages. You can monitor the queue and the broker using: ./imqadmin which can be found in install_folder/mq/bin this software shows how many durable subscribers are around and how many messages are pending for each subscriber and so on. You can monitor the Queue in real-time mode using the following command which can be executed in install_folder/mq/bin ./imqcmd -b 127.0.0.1:7979 metrics dst -t t -n DIRECT_TOPIC The command will ask for username and password, give admin/admin for it The sample code for this entry can be found Here. The sample code is a NetBeans project with the publisher and the subscriber source code. A complete documentation of OpenMQ is available at its documentation centre. You can see how you can change different port numbers or configure different properties of the Broker and HTTP tunneling web application communication.
June 22, 2009
by Masoud Kalali
· 34,792 Views
article thumbnail
Hibernate Performance Tuning
Hibernate is a powerful, high performance object/relational persistence and query service. Hibernate lets you develop persistent classes following object-oriented idiom - including association, inheritance, polymorphism, composition, and collections. Hibernate allows you to express queries in its own portable SQL extension (HQL), as well as in native SQL, or with an object-oriented Criteria and Example API. Quintessential to using any ORM framework like hibernate is to know how to leverage the various performance tuning methods supported by the framework. In this volume Wings Jiang discusses three performance tuning strategies for hibernate: SQL Optimization Session Management Data Caching SQL Optimization When using Hibernate in your application, you already have been coding HQL (Hibernate Query Language) somewhere. For example, “from User user where user.name = ‘John’”. If issuing your SQL statement like this, Hibernate cannot use the SQL cache implemented by database because name of the user, in most scenarios, is extremely distinct. On the contrary, while using placeholder to achieve this, like “from User user where user.name =?” will be cached by the Database to fulfill the performance improvement. You can also set some Hibernate properties to improve performance, such as setting the number of records retrieved while fetching records via configuring property hibernate.jdbc.fetch_size, setting the batch size when committing the batch processing via configuring property hibernate.jdbc.batch_size and switching off the SQL output via setting property hibernate.show_sql to false in product environments. In addition, the performance tuning of your target Database is also significant, like SQL clauses tuning, reasonable indexes, delicate table structures, data partitions etc. Session Management Undoubtedly, Session is the pith of Hibernate. It manages the Database related attributes, such as JDBC connections, data entities’ states. Managing the Session efficiently is the key to getting high performance in enterprise applications. One of the many commonly used and equally elegant approaches to session management in hibernate is to use ThreadLocal. Threadlocal will create a local copy of session for every thread. Thus synchronization problems are averted, when objects are put in the Threadlocal, . To understand how ThreadLocal variables are used in Java, refer to Sun Java Documentation at http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ThreadLocal.html Data Caching Before accomplishing any data caching, it is essential to set the property hibernate.cache.user_query_cache = true. There are three kinds of commonly used Caching Strategies in Hibernate: Using cache based on Session level (aka Transaction layer level cache). This is also called first-level cache. Using cache based on SessionFactory level (Application layer level cache). This is also called second-level cache. Using cluster cache which is employed in distributed application (in different JVMs). In fact, some techniques, like loading data by id, lazy initialization which betokens loading appropriate data in proper time rather than obtaining a titanic number of useless records, which are fairly useless in the subsequent operations are consummated via data caching. First Level Cache (aka Transaction layer level cache) Fetching an object from database always has a cost associated with it. This can be offset by storing the entities in hibernate session. Next time the entities are required, they are fetched from the session, rather than fetching from the database. To clear an object from the session use: session.evict(object). To clear all the objects from the session use session.clear(). Second Level Cache (aka Application layer level cache) In this approach, if an object is not found in session, it is searched for in the session factory before querying the database for the object. If an object is indeed fetched from database, the selected data should be put in session cache. This would improve the performance when the object is required next time. To remove an entity from session factory use the various overloaded implementations of evict() method of SessionFactory. In fact, Hibernate lets you tailor your own caching implementation by specifying the name of a class that implements org.hibernate.cache.CacheProvider using the property hibernate.cache.provider_class. But it is recommended to employ a few built-in integrations with open source cache providers (listed below). Cache Type Cluster Safe Query Cache Supported Hashtable Memory NO YES EHCache Memory, Disk NO YES OSCache Memory, Disk NO YES SwarmCache Clustered YES (clustered invalidation) NO JBoss TreeCache Clustered YES (replication) YES Terracota Clustered YES YES In order to use second level caching, developers have to append some configurations in hibernate.cfg.xml (for example, using EHCache here). net.sf.ehcache.hibernate.Provider In addition, developers also need to create a cache specific configuration file (Example: ehcache.xml for EHCache). (1) diskStore : Sets the path to the directory where cache .data files are created. The following properties are translated: a.user.home - User's home directory b.user.dir - User's current working directory c.java.io.tmpdir (Default temp file path) maxElementsInMemory : Sets the maximum number of objects that will be created in memory. eternal : Sets whether elements are eternal. If eternal, timeouts are ignored and the element is never expired. timeToIdleSeconds : Sets the time to idle for an element before it expires. Is only used if the element is not eternal. Idle time is now - last accessed time. timeToLiveSeconds : Sets the time to live for an element before it expires. Is only used if the element is not eternal. TTL is now - creation time overflowToDisk : Sets whether elements can overflow to disk when the in-memory cache has reached the maxInMemory limit. Finally the cache concurrency strategy has to be specified in mapping files. For example, the following code fragment shows how to configure your cache strategy. … … Cache Concurrency Strategies There are four kinds of built-in cache concurrency strategies provided by Hibernate. Chosing a right concurrency strategy for your hibernate implementation is the key to cache performance optimization. Besides to ensure data consistency and transaction integrity it is indispensable to master these strategies. read-only If your application needs to read but never modify instances of a persistent class, a read-only cache may be used. This is the simplest and best performing strategy. It's even perfectly safe for use in a cluster. nonstrict-read-write If the application only occasionally needs to update data (For example, if it is extremely unlikely that two transactions would try to update the same item simultaneously) and strict transaction isolation is not required, a nonstrict-read-write cache might be appropriate. read-write If the application needs to update data, a read-write cache might be appropriate. This cache strategy should never be used if serializable transaction isolation level is required. transactional If the application seldom needs to update data and at the same time, application also needs to avoid “dirty read” and “repeatable read”, this kind of concurrency strategy can be employed. The transactional cache strategy provides support for fully transactional cache providers such as JBoss TreeCache. The following table lists cache concurrency strategy supported by various cache providers. Cache Read-only Nonstrict-read-write Read-write Transactional Hashtable YES YES YES N/A EHCache YES YES YES N/A OSCache YES YES YES N/A SwarmCache YES YES N/A N/A JBoss TreeCache YES N/A N/A YES Cluster Cache (in different JVMs) Hibernate also supports cluster caching in disparate JVMs. At present, both SwarmCache and JBoss TreeCache support cluster caching across multiple JVMs. In some situations, especially at the level of enterprise, certain application has to support the concurrency accessing of thousands of users, at that time, cluster cache can help you because the cluster can provide failover and load balancing which improve the performance of application. Points to Note When employing one of the four cache strategies above, pay close attention to the following situation: Data cached almost immutable If data you want to cache is almost constant, you can use data caching which can improve the performance of the application. On the contrary, if the caching data are quiet volatile, Hibernate have to maintain and update the caching over time which extremely leads to performance hit. Data sizes in reasonable range If the size of data you is caching is massive, Hibernate will occupy the most memories of system, which causes the long waiting time of the whole application. Low frequency of data updating If data you are caching needs to be modified frequently, Hibernate have to take an array of time to update and modify the data in caching, which impacts the performance of the application as well. High frequency of data querying If data you are caching is steady, which means that most of the operations are querying, searching, no updating and modifying, making the most use of caching will be affording huge performance improvement. None crucial data Because of existing some incongruities when keeping the data in caching, so if the data you are caching is fairly crucial, do not use caching. By contrast, if the data in caching is insignificant, just use it without any vacillation. Summary Actually, after employing SQL Optimization, Session Management, Data Caching, we will obtain great battalions of performance gains, which make applications achieve acceptable waiting time for the final customers. External Links for Further Study http://www.hibernate.org/hib_docs/reference/en/html/performance.html http://blogs.jboss.com/blog/acoliver/2006/01/23/Hibernate_EJB3_Tuning.txt About Author I am Wings Jiang from BCM China. I have mainly focused on J2EE technologies in recent years and worked in several projects involving Struts/Tapestry, Spring, Hibernate, WebLogic, Websphere, Oracle, DB2 etc. I have experience in design and code of several Java applications. Hibernate performance is one of the areas I pay close heed to in my current working.
June 10, 2009
by Ming Jiang
· 141,899 Views · 4 Likes
article thumbnail
Java Properties Without Getters and Setters
During the last Devoxx conference, Mark Reinhold, Sun's chief engineer for Java SE, gave a presentation on the latest directions for Java 7. (Hamlet D'Arcy's summary of Mark's presentation is available here.) One of the features that was discussed for possible inclusion in Java 7, but won't find its way into the final release, is first class properties for Java. First-class support for properties would have gone beyond the simple setter and getter methods of the JavaBeans specification, and provided a succinct and elegant way for defining properties for Java objects. Properties are already first-class elements of many modern languages, so this lack in Java 7 will be felt by many developers accustomed to other languages' property support. At the same time, Java developers can resort to a handful of other techniques in working with property-like Java attributes, and some of the possible techniques work even in Java 1.4. In the rest of this article, I will demonstrate one such technique use a simple aspect, with AspectJ, and some following some conventions. Motivation The idea for this solution arose while working with the OpenXava 3 framework. OpenXava defines a mechanism to develop Java Enterprise applications using Business Components. It's an alternative way to MVC: instead of organizing the code into Model, a View and a Controller, the code with OpenXava is organized around Invoice, Customer, Order, and similar business-centric objects. OpenXava initially used XML for defining components, but since version 3 the use of POJOs with Java 5 annotations also became available as the preferred way to define business components. When using the XML-based object definition, you may specify a component in the following manner: Using the annotation-based OpenXava 3, the same component would be defined as follows: @Entity public class Teacher { @Id @Column(length=5) @Required private String id; @Column(length=40) @Required private String name; @OneToMany(mappedBy="teacher") private Collection pupils; public String getId() { return id; } public void setId(String id) { this.id = id; } public String getName() { return name; } public void setName(String name) { this.name = name; } public Collection getPupils() { return pupils; } public void setPupils(Collection pupils) { this.pupils = pupils; } } This illustrates some of the verbosity with the Java definition: 37 lines against the 13 of XML version. The problem occurs because of the getters and setters. The boilerplate code adds noise and increases the size of the file without increasing the information to the programmer. Use Java and annotations as a definition language in OpenXava would welcome a more succinct and elegant syntax. A More Succinct Solution A better solution can be obtained by defining the business component in this way: @Entity public class Teacher { @Id @Column(length=5) @Required public String id; @Column(length=40) @Required public String name; @OneToMany(mappedBy="teacher") public Collection pupils; } This makes Java more beautiful, succinct and elegant... just as XML. With this, you have to use the following properties without getters and setters. That is, instead of: teacher.setName("M. Carmen"); String result = teacher.getName(); you write: teacher.name = "M. Carmen"; String result = teacher.name; In this case name is not a field, but a property. You can refine the access to this property, and even create pure calculated properties. For example, if you wanted a calculated property, you could define it in this way: @Entity public class Person { ... public String name; public String surname; transient public String fullName; // (1) protected String getFullName() { // (2) return name + " " + surname; } } In order to define a calculated property, you define a public field (1) and then a protected (or private, but not public) getter (2). This getter, getFullName() in this case, is for implementing the logic for the property fullName. Using this property is simple: Person p = new Person(); p.name = "M. Carmen"; p.surname = "Gimeno Alabau"; assertEquals("M. Carmen Gimeno Alabau", p.fullName); When p.fullName is used, the method getFullName() is executed for returning the value. You can see that fullName is a property, not a simple field. You can also refine access to writing and reading the properties. For example, you can define a property as following: public String address; protected String getAddress() { return address.toUpperCase(); } When address is used from outside the class the method getAddress() is used to obtain the result, but this method only returns the address with some refinement. As we use address field from inside getAddress() field address from inside of its class is used, and the getter is not called. Now you can use this property as follows: Person p = new Person(); p.address = "Av. Baron de Carcer"; assertEquals("AV. BARON DE CARCER", p.address); As well, you can define setters, even for vetoing the data to be set, as in the next example: public int age; protected void setAge(int age) { if (age > 200) { throw new IllegalArgumentException("Too old"); } this.age = age; } As in the case of the getter, if a setter method is present it will be executed to set the data, you can use this logic to set the data to the field or for any other logic you want. Now you can use this property in this way: Person p = new Person(); p.age = 33; assertEquals(p.age, 33); p.age = 250; // Here an IllegalArgumentException is thrown In summary: Properties behave from outside of the class as public fields. If no getter or setter for the field exists, a direct access for the field is performed. If a getter exists for the field, it will be executed when trying to read the field from outside. If a setter exists for the field, it will be executed when trying to write the field from outside. From inside the class, all references to the field are direct access, without calling getter or setter. This provides a simple and natural way to have properties in Java without unnecessary getters and setters. Implementation This simple mechanism is easy to implement using AspectJ and a simple aspect, and it will work even in Java 1.4. For this to work, you only need to have an aspect in your project: aspect PropertyAspect { pointcut getter() : get(public !final !static * *..model*.*.*) && !get(public !final !static * *..model*.*Key.*) && !within(PropertyAspect); pointcut setter() : set(public !final !static * *..model*.*.*) && !set(public !final !static * *..model*.*Key.*) && !within(PropertyAspect); Object around(Object target, Object source) : target(target) && this(source) && getter() { if (target == source) { return proceed(target, source); } try { String methodName = "get" + Strings.firstUpper( thisJoinPointStaticPart.getSignature().getName()); Method method = target.getClass(). getDeclaredMethod(methodName, null); method.setAccessible(true); return method.invoke(target, null); } catch (NoSuchMethodException ex) { return proceed(target, source); } catch (InvocationTargetException ex) { Throwable tex = ex.getTargetException(); if (tex instanceof RuntimeException) { throw (RuntimeException) tex; } else throw new RuntimeException( XavaResources.getString("getter_failed", thisJoinPointStaticPart. getSignature().getName(), target.getClass()), ex); } catch (Exception ex) { throw new RuntimeException( XavaResources.getString("getter_failed", thisJoinPointStaticPart.getSignature().getName(), target.getClass()), ex); } } void around(Object target, Object source, Object newValue) : target(target) && this(source) && args(newValue) && setter() { if (target == source) { proceed(target, source, newValue); return; } try { String methodName = "set" + Strings.firstUpper(thisJoinPointStaticPart.getSignature().getName()); Class fieldType = ((FieldSignature) thisJoinPointStaticPart.getSignature()).getFieldType( Method method = target.getClass(). getDeclaredMethod(methodName, new Class[] {fieldType}); method.setAccessible(true); method.invoke(target, new Object[] { newValue } ); } catch (NoSuchMethodException ex) { proceed(target, source, newValue); } catch (InvocationTargetException ex) { Throwable tex = ex.getTargetException(); if (tex instanceof RuntimeException) { throw (RuntimeException) tex; } else throw new RuntimeException( XavaResources.getString("setter_failed", thisJoinPointStaticPart.getSignature().getName(), target.getClass()), ex); } catch (Exception ex) { throw new RuntimeException( XavaResources.getString("setter_failed", thisJoinPointStaticPart.getSignature().getName(), target.getClass()), ex); } } } Having defined this aspect, you will need to compile your project using AspectJ. The aspect intercepts all access to public fields in the model package, then use introspection to call to the getter or setter method if they exists. Drawbacks Unfortunately, this approach has at least two important drawbacks: It does not work when you access to the properties using introspection. With JPA, at least using the Hibernate implementation, lazy initialization does not work. You may think that the introspection problem can be overcome with your own custom introspection code. But the man third party libraries, frameworks, and toolkits will not obey those rules. For example, if you are using a report generator that can generate a report from a collection of Java objects, the report generator may expect real public getters and setters in the objects. The JPA the specification for Java Persistence API states that: 2.0: "Instance variables must not be accessed by clients of the entity. The state of the entity is available to clients only through the entity methods i.e., accessor methods (getter/setter methods) or other business methods." JSR 317 : JavaTM Persistence API, Version 2.0 - Public Review Draft - Section 2.1 That is, portable JPA code should not rely on direct access to properties. Conclusion This article presents a very simple way to work with properties in Java, even though the language doesn't support properties. If the AspectJ implementation is not practical, you can find other implementations of this idea using asm or cglib. References OpenXava - http://www.openxava.org/ AspectJ - http://www.eclipse.org/aspectj/
May 30, 2009
by Javier Paniza
· 79,725 Views · 1 Like
article thumbnail
The Best Kept Secret in the JDK: VisualVM
It's amazing the things that are right in front of you that you don't realise. VisualVM is probably the best example of this in the Java community.
May 28, 2009
by James Sugrue
· 226,839 Views · 7 Likes
article thumbnail
JavaFX: Using Patterns & Clean Code
We are seeing quite a number of exciting JavaFX demos around, demonstrating the pretty features of the language and the capability of easily integrating cool graphics. But, as a software designer, I can't prevent myself from seeing that in most examples we see bloated code - no good separation of concerns and poor applications of the MVC pattern. This is reasonable, as JavaFX was a new language and people needed first to be taught about the syntax and features; but now that - I presume - many of us have been introduced with the new language, it's high time we started worrying about good practices, as well as writing self-commenting code. Let's remember that good practices and good design are always more important than the language! So, let's introduce a very simple project - a Contact List whose specifications are: We have a list of "contacts", where each item is name, last name, email, phone, etc... The UI shows a list of contacts, that can be filtered by a text field; if you enter a value in it, only the contacts whose name starts with the entered text are shown. Selections are applied as you type. Selecting an item in the list of contacts, the details are shown in a form, where you can edit them. At any selection change, the form is animated (a rotation and a blur effect). You can get the code from: svn co -r 37 https://kenai.com/svn/javafxstuff~svn/trunk/ContactList/src/ContactList The model Let's first have a quick look at the model classes. First a simple value object representing a piece of data: package it.tidalwave.javafxstuff.contactlist.model; public class Contact { public var id: String; public var firstName: String; public var lastName: String; public var phone: String; public var email: String; public var photo: String; override function toString() { return "\{id: {id}, value: {firstName} {lastName}, phone: {phone}, email: {email}" } } Then a small service which provides a bunch of data: package it.tidalwave.javafxstuff.contactlist.model; public abstract class ContactRegistry { public abstract function items() : Contact[]; } In the demo code, you'll find a mock implementation with some wired values; in a real case this could be a Business Delegate encapsulating code for retrieving data remotely. So far, so good - it's pretty normal to keep these things separated from the UI. The Controllers We're not going to see a classic Controller here; actually, we're slightly departing from the "pure" MVC. The code I'm showing you is more a "Presentation Model", a pattern described by Martin Fowler as: The essence of a Presentation Model is of a fully self-contained class that represents all the data and behavior of the UI window, but without any of the controls used to render that UI on the screen. A view then simply projects the state of the presentation model onto the glass. This class is basically an hybrid between a classic model and a controller. It is also a façade between the view and the domain model. package it.tidalwave.javafxstuff.contactlist.model; import it.tidalwave.javafxstuff.contactlist.model.ContactRegistry; import it.tidalwave.javafxstuff.contactlist.model.ContactRegistryMock; public class PresentationModel { public var searchText : String; public var selectedIndex : Integer; // The Business Delegate def contactRegistry = ContactRegistryMock{} as ContactRegistry; def allContacts = bind contactRegistry.items(); // The contacts filtered according the contents of the search field public-read def contacts = bind allContacts[contact | "{contact.firstName} {contact.lastName}".startsWith(searchText)]; // The selected contact; the code also triggers a notification at each change public-read def selectedContact = bind contacts[selectedIndex] on replace previousContact { onSelectedContactChange(); }; // Notifies a change in the current Contact selection public-init var onSelectedContactChange = function() { } } Note the extreme compactness brought by functional programming. There's almost no imperative programming as everything is achieved by properly using the binding feature. The only imperative part is the 'onSelectedContactChange' function, which is just a listener to notify selection changes to some external code - it will be used only for triggering the animation. BTW, I'd like to remove it from here, but I wasn't able to. Maybe it's a JavaFX thing that I've not understood yet, but I'm keeping it for another post. Now, everything about the animation goes encapsulated in a specific class, which only exposes two properties controlling the animation: the effect and the rotation angle. A single play() function is provided to start the animation. package it.tidalwave.javafxstuff.contactlist.view; import javafx.animation.Interpolator; import javafx.animation.Timeline; import javafx.scene.effect.GaussianBlur; public class AnimationController { public-read var rotation = 0; public-read def effect = GaussianBlur{} def timeline = Timeline { repeatCount: 1 keyFrames: [ at(0s) { effect.radius => 20; rotation => 45 } at(300ms) { effect.radius => 0 tween Interpolator.EASEBOTH; rotation => 0 tween Interpolator.EASEBOTH } ] } public function play() { timeline.playFromStart(); } } The Views Now, a UI component. The way we design it largely depends on the process, as a graphic designer could be involved. In any case, I think that the whole UI should not be implemented in a single, bloated class; rather relevant pieces should be split apart. For instance, a CustomNode can model the "form" that renders the contact details (in the code below I've omitted all the attributes related to rendering): package it.tidalwave.javafxstuff.contactlist.view; import javafx.scene.CustomNode; import javafx.scene.Group; import javafx.scene.layout.HBox; import javafx.scene.layout.VBox; import javafx.scene.Node; import javafx.ext.swing.SwingLabel; import javafx.ext.swing.SwingTextField; import it.tidalwave.javafxstuff.contactlist.model.Contact; public class ContactView extends CustomNode { public var contact : Contact; public override function create() : Node { return Group { content: [ VBox { content: [ SwingLabel { text: bind "{contact.firstName} {contact.lastName}" } HBox { content: [ SwingLabel { text: "First name: " } SwingTextField { text: bind contact.firstName } ] } HBox { content: [ SwingLabel { text: "Last name: " } SwingTextField { text: bind contact.lastName } ] } HBox { content: [ SwingLabel { text: "Email: " } SwingTextField { text: bind contact.email } ] } HBox { content: [ SwingLabel { text: "Phone: " } SwingTextField { text: bind contact.phone } ] } ] } ] }; } } As you can see, we have only layout and data binding here, the only things a view should do. Putting all together Now the last piece of code, the Main, which builds up the application (again, I've omitted all attributes only related to rendering): package it.tidalwave.javafxstuff.contactlist.view; import javafx.scene.layout.HBox; import javafx.scene.layout.VBox; import javafx.scene.Scene; import javafx.stage.Stage; import javafx.ext.swing.SwingLabel; import javafx.ext.swing.SwingList; import javafx.ext.swing.SwingListItem; import javafx.ext.swing.SwingTextField; import it.tidalwave.javafxstuff.contactlist.controller.PresentationModel; Stage { def animationController = AnimationController{}; def presentationModel = PresentationModel { onSelectedContactChange : function() { animationController.play(); } }; scene: Scene { content: VBox { content: [ HBox { content: SwingLabel { text: "Contact List" } } HBox { content: [ SwingLabel { text: "Search: " } SwingTextField { text: bind presentationModel.searchText with inverse } ] } HBox { content: [ SwingList { items: bind for (contact in presentationModel.contacts) { SwingListItem { text:"{contact.firstName} {contact.lastName}" } } selectedIndex: bind presentationModel.selectedIndex with inverse } ContactView { contact: bind presentationModel.selectedContact effect: bind animationController.effect rotate: bind animationController.rotation } ] } ] } } } As in the previous code snippets, I think the listing can be easily read and understood. Basically, we are glueing all the pieces together and binding the relevant models. In the end, each class in this small project does a simple, cohese thing: representing data, encapsulating the presentation logic, controlling the animation, rendering the views. Dependencies are reduced to the minimum and they have the correct direction: views depend on the models (and not the opposite) and the AnimationController; the AnimationController is independent. You could replace the view components without affecting the rest of the classes, as well as removing or adding other animations by properly using different AnimationControllers. This good separation of roles and responsibilities is the good way to apply OO. There is a detail which is worth discussing. Note the two bind ... with inverse. They implement the so-called "bidirectional binding" where not only a change in the model (e.g. PresentationModel.selectedIndex) is reflected to attributes in the UI (e.g. SwingList.selectedIndex), but also the opposite happens. Indeed, the reverse binding is more important in our example, because it implements the controller responsibility (it captures user's gestures from the view and changes the model); the direct binding from the PresentationModel to SwingList, instead, is useless, as in our case the PresentationModel is never the originator of a change. So, why not using a simple, direct binding in PresentationModel towards SwingList? Such as: public class PresentationModel { var list : SwingList; public def selectedIndex = bind list.selectedIndex; ... } Because this would introduce a dependency from the model/controller to the view, which is plain wrong. Here, bind ... with inverse not only works as a shortcut for writing less code (that is, explicitly declaring a binding and its inverse), but it's also an essential feature for a better design. As far as I know - but I could be wrong - for example in ActionScript (Adobe Flex language) there's no bidirectional binding (you need to put binding keywords at both ends of the association), thus introducing unneeded or circular dependencies. I believe this is true at least at code level (as far as I understand, there are different ways to do binding in ActionScript).
May 12, 2009
by Fabrizio Giudici
· 46,076 Views
article thumbnail
Why Java (and Other Languages) Don't Need Operator Overloading
If you knew that the language that you are reading supports it, you are just going to extend this mental path to operations that involve overridable operators.
April 13, 2009
by Cedric Beust
· 86,120 Views · 1 Like
article thumbnail
Apache Camel: Integration Nirvana
Take any integration project and you have multiple applications talking over multiple transports on multiple platforms. As you can imagine, in large enterprise applications this can get complex very fast. Much of the complexity stems from two issues: 1. dealing with the specifics of applications and transports, and 2. coming up with good solutions to integration problems. Making your applications speak transports and APIs is relatively easy on its own. I'm sure everyone knows how to send JMS messages to their broker of choice; though it still requires in depth knowledge of the JMS specification, which many developers may not have. On top of that, what happens when you want to route that JMS message to another application? You then have to take care of mapping the JMS message to the application plus handle any new concepts related to the application. Add a dozen other applications into the mix and you've got quite a headache on your hands. Ignoring the mechanics of how to connect with multiple transports and APIs, we can focus on the high level design of how applications interact. Fortunately, most solutions to enterprise integration problems have been formalized already. Gregor Hohpe and Bobby Woolfe's book, Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions, boils down years of experience from enterprise architects into a set of sixty five Enterprise Integration Patterns (EIPs). This is great but we still have to hand code all parts of these patterns; these are not packaged solutions, only recommendations. Apache Camel was created with the intention of addressing these two issues. In this article I'll show you how it actually does this. What is Camel? Apache Camel is an open source Java framework that focuses on making integration easier and more accessible to developers. It does this by providing: • concrete implementations of all the widely used EIPs • connectivity to a great variety of transports and APIs • easy to use Domain Specific Language (DSL) to wire EIPs and transports together Figure 1 shows how these three items actually map to Camel concepts. To give you a good understanding of how Camel is organized, we will discuss Components, Endpoints, Processors, and the Domain Specific Language (DSL). There is of course a lot more going on here under the hood but we'll leave that for another discussion. Figure 1: High level view of Camel's architecture. Components are the extension point in Camel to add connectivity to other systems. The core of Camel is very small to keep dependencies low, promote embeddability, etc. and as a result contains only 12 essential components. There are over 60 components outside the core. To expose these systems to the rest of Camel, Components provide an Endpoint interface. By using URIs, you can send or receive messages on Endpoints in a uniform way. For instance, to receive messages from a JMS queue aQueue and send them to a file system directory "c:/tmp", you could use URIs like "jms:aQueue" and "file:c:\tmp". Processors are used to manipulate and mediate messages in between Endpoints. All of the EIPs are defined as Processors or sets of Processors. As of writing, Camel supports 41 patterns from the EIP book, 6 other integration patterns, and many other useful Processors. To wire Processors and Endpoints together, Camel defines a Java DSL. The term DSL is used a bit loosely here as it usually implies the involvement of a compiler or interpreter that can process keywords specific to a particular domain. In Camel, DSL means a fluent Java API that contains methods named like terms from the EIP book. Its best explained with an example from("jms:aQueue") .filter().xpath("/person[@name='Jon']") .to("file:c:\tmp"); Here we define a routing rule in a single Java statement that will consume messages from the "jms:aQueue" Endpoint, send them through a Message Filter Processor, which will then send on messages passing the XPath condition to the "file:c:\tmp" endpoint. Messages failing the condition will be dropped. You can also configure your routes in a XML-based Spring configuration file. This configuration file is a lot more verbose and less auto complete friendly than the Java DSL; many prefer it though because of its direct access to Spring concepts and no requirement for compilation after changes. Here is what the earlier example would look like in Spring: /person[@name='Jon'] These are the concepts that Camel was built upon. Since then many other interesting features have been added. Details of these are left up to the reader to investigate. To get you started, some of these include: • Pluggable data formats and type converters for easy message transformation between Artix Data Services, CSV, EDI, Flatpack, HL7, JAXB, JSON, XmlBeans, XStream, Zip, Camel-bindy, etc. • Pluggable languages to create expressions or predicates for use in the DSL. Some of these languages include: EL, JXPath, Mvel, OGNL, BeanShell, JavaScript, Groovy, Python, PHP, Ruby, SQL, XPath, XQuery, etc. • Support for the integration of beans and POJOs in various places in Camel. • Excellent support for testing distributed and asynchronous systems using a messaging approach • and much more... Example A motorcycle parts business, Rider Auto Parts, supplies parts to motorcycle manufacturers. Over the years they've changed the way they receive orders several times. Initially, orders were placed by uploading CSV files to an FTP server. The message format was later changed to XML. Currently they provide a web site to submit orders as XML messages over HTTP. All of these messages are converted to an internal POJO format before processing. Rider Auto Parts states to any new customers to use the web interface to place orders. However, because of existing agreements with customers, they must keep all the old message formats and interfaces up and running. Solution using EIPs Rider Auto Parts faces a pretty common problem; over years of operation businesses acquire software baggage in the form of transports/data formats that are popular at the time. Using patterns from the EIP book we can envision the solution as something like Figure 2. Figure 2: This shows the solution to Rider Auto Parts integration problem using notation from the Enterprise Integration Patterns book. So we have several patterns in use here. 1. There are two Message Endpoints; one for FTP connectivity and another for HTTP. 2. Messages from these endpoints are fed into the incomingOrderQueue Message Channel 3. The messages are consumed from the incomingOrderQueue and routed by a Content-Based Router to one of two Message Translators. As the EIP name implies, the routing destination depends on the content of the message. In this case we need to route based on whether the content is a CSV or XML file. 4. Both Message Translators convert the message content into a POJO, which is fed into the orderQueue Message Channel. The whole section that uses a Content-Based Router and several Message Translators is referred to as a Normalizer. This composite pattern has a unique graphic to depict it but was left out here in favor of its sub-patterns to make things clearer. Implementation using Camel As mentioned before, Camel has a small core set of components included by default. The rest of the components exist as separate modules. In applications that require many types of connectivity it is useful to figure out what Camel modules to include. Listing 1 shows the dependencies using Apache Maven for the Camel implementation of the Rider Auto Parts example. Of course, you don't need to use Apache Maven for dependencies - it is just the easiest way to rapidly add new dependencies to your applications. The list of dependencies includes support for core Camel, ActiveMQ, JAXB marshaling, CSV marshaling, and HTTP. To make the example easier to try out, I've opted to use the File endpoint instead of the FTP. If we were using the FTP endpoint we would need to add a dependency on the camel-ftp module as well. Listing 1: Maven dependencies for the Camel implementation org.apache.camel camel-core ${camel-version} org.apache.camel camel-spring ${camel-version} org.apache.activemq activemq-camel ${activemq-version} org.apache.camel camel-jaxb ${camel-version} org.apache.camel camel-csv ${camel-version} org.apache.camel camel-jetty ${camel-version} org.apache.activemq activemq-core ${activemq-version} org.apache.xbean xbean-spring ${xbean-spring-version} While it is perfectly legitimate to use Camel as a standalone Java application, it is often useful to embed it in a container. In this case, we will be loading Camel from Spring. The Spring beans XML file is shown in Listing 2. First we start an embedded Apache ActiveMQ broker and connect Camel to it. We also load up some helper beans that we will reference from the DSL. Finally, the camelContext element tells Camel to look for routes in the org.fusesource.camel package. Routes are Java classes that extend the RouteBuilder class in Camel. Listing 2: Spring XML file that configures an embedded ActiveMQ broker, several beans used in the Camel route, and initializes the Camel Context to search for routes in the org.fusesource.camel package. org.fusesource.camel The real meat of the Camel implementation lies in the OrderRouter class (shown in Listing 3). This class extends RouteBuilder, so it will be automatically picked up and loaded by Camel's runtime. Looking back at Figure 2, we need to receive orders from an FTP (substituted with File) and HTTP endpoint, formatted as shown in Listing 4. In the DSL we can specify these incoming endpoints with two from elements. Both from elements are connected to a to("jms:incomingOrderQueue") element, which will send the messages to a queue on the ActiveMQ broker. Listing 3: Route definitions for the example. The routing rules are specified using a fluent API, referred to as Camel's DSL. public class OrderRouter extends RouteBuilder { @Override public void configure() throws Exception { JaxbDataFormat jaxb = new JaxbDataFormat("org.fusesource.camel"); // Receive orders from two endpoints from("file:src/data?noop=true").to("jms:incomingOrderQueue"); from("jetty:http://localhost:8888/placeorder") .inOnly().to("jms:incomingOrderQueue") .transform().constant("OK"); // Do the normalization from("jms:incomingOrderQueue") .convertBodyTo(String.class) .choice() .when().method("orderHelper", "isXml") .unmarshal(jaxb) .to("jms:orderQueue") .when().method("orderHelper", "isCsv") .unmarshal().csv() .to("bean:normalizer") .to("jms:orderQueue"); } } In the case of the HTTP endpoint, there are a couple of extra things to mention. First off the HTTP client will be expecting a response from the application so we have to handle that. In Camel, we have full control over what the client gets back from the HTTP endpoint. Each response is determined by the last method in our current route definition (each Java statement is a route definition). In our case we use the transform method to set the response to the constant string "OK". Since we handle the response ourselves, we don’t want any response to come from the JMS incomingOrderQueue. To send to this queue in a fire-and-forget fashion we add the inOnly modifier. It is important to note at this point that when writing Camel DSL in a modern Java IDE, selection of the next processing step is easy because of auto complete. The auto complete feature basically gives you a list of processors (i.e. EIPs) to choose from at any point in your route. Since fluent APIs chain methods together, the only method you need to remember is the from; all other methods are shown via auto complete. Listing 4: Incoming message formats; XML on top, CSV below. "name", "amount" "brake pad", "2" The next section of DSL in Listing 3 specifies the Normalizer, complete with Content-Based Router and two Message Translators. First we specify that we want to consume messages from the incomingOrderQueue on the ActiveMQ broker. The content based routing of the messages is done with the choice and when methods. In our case, we want to send CSV messages to one Message Translator and XML messages to another. To check what type of message we have we will be using a simple Java bean shown in Listing 5. Of course, this is demonstration code only; for production cases you would want to add more thorough checking of content types. Listing 5: Java bean that contains helper methods to be used in the DSL. public class OrderHelper { public boolean isCsv(String body) { return !body.contains("> body) { List orderHeaders = body.get(0); List orderValues = body.get(1); return new Order(orderValues.get(0), Integer.parseInt(orderValues.get(1))); } } At this point, successfully normalized messages are sent to the orderQueue for processing by some other application at the Rider Auto Parts business. Conclusion In this article I've shown two common problems that an integration developer may face: dealing with the specifics of applications and transports, and coming up with good solutions to integration problems. The Apache Camel project provides a nice answer to both of these problems. As the example has shown, solving integration problems with Camel is straight forward and results in relatively concise code. In my opinion it is the closest thing to integration nirvana that we have today. Links Apache Camel – http://camel.apache.org FUSE Mediation Router (based on Apache Camel) – http://fusesource.com/products/enterprise-camel Enterprise Integration Patterns – http://www.enterpriseintegrationpatterns.com Jon’s Blog – http://janstey.blogspot.com Camel in Action book - http://www.manning.com/ibsen Article source code - http://repo.fusesource.com/maven2/org/fusesource/examples/rider-auto-example/1.0/rider-auto-example-1.0.zip Author Jonathan Anstey is a senior engineer working for Progress Software Corporation specializing in the enterprise integration space. Jon focuses mostly on Apache Camel and its Progress endorsed likeness, FUSE Mediation Router. He also works on the Apache ActiveMQ and Apache ServiceMix projects
March 23, 2009
by Jonathan Anstey
· 223,920 Views · 8 Likes
article thumbnail
How to Fix Memory Leaks in Java
Your pager hasn’t been sleeping well. It periodically wakes you up in the middle of the night to tell you that your server is firing off “OutOfMemoryError” messages. Worse still, your significant other forcibly relocated you to the couch and told you not to return until your pager stops buzzing. Sound familiar? If so, you may have a case of memory leak induced insomnia, but fortunately we’ve got a cure for what ails you. This tutorial will teach you everything you need to know to ease your suffering, including what memory leaks are, why they happen, and how to diagnose and fix ‘em. In this article we’ll focus on techniques that will enable you to address memory leaks with any commercial or free/open source memory profiler; we’re not here to recommend one tool over another. After all, the most important thing is that you fix the problem and get some rest, not the tool you use to get it done. Before You Start Ever heard the story about how Java has “automatic memory management”—you know, the one that someone in marketing upgraded to an epic tale about how you’ll never have to worry about memory leaks ever again? As is often the case, the truth is more complex than the marketing department made it out to be. While it’s true that Java’s garbage collector (GC) helps to eliminate the most common memory leak issues from applications, it is unfortunately still possible to experience memory leaks in Java. However, they happen a lot less often than they used to in the C or C++ days. Many people believe that black magic and complex tools are required to fix memory leaks. This undeserved reputation is caused by the lack of good explanations of what they are and what to do when you encounter them. But with the proper tools and knowledge to fix memory leaks, they aren’t nearly as intimidating. Methodology In this article we’ll cover everything from memory leak basics to analyzing heap dumps, so—whether you’re an experienced Java developer or encountering Java memory leaks for the first time—you’ll be better prepared to deal with memory leaks by the time you reach the conclusion. We won’t outline a series of steps, like “do ABC with commercial tool XYZ and don’t ask why,” as that approach doesn’t work and implies that remedies are more complex then they really are. Instead, we’ll give you the background information necessary to address memory leaks, with emphasis placed on particular steps you’ll need to execute. Similarly, we’ll assume that you can learn on your own how to use the memory profiler of your choice; what’s missing is an understanding of what the tool is trying to do and why, so that will be the focus of this article. Java will be used for all examples, so all information in this article directly applies to Java applications running standalone or as a part of J2EE/JEE/Tomcat-based application server. But remember, although our primary focus is on Java, most of the process of diagnosing and fixing memory leaks described herein applies to other languages with garbage collectors. So even if you’re using Ruby, C#, or Python, there should be something for you in this article. What Are Memory Leaks? Let’s start by describing how memory leaks happen in Java. Java implements automatic garbage collection (GC), and once you stop using an object you can depend on the garbage collector to collect it. While additional details of the collection process are important when tuning GC performance, for the sole purpose of fixing a memory leak we can safely ignore them. When is memory eligible for GC? Let’s take a look at an example: We don't have to do anything special to make an object eligible for GC—we just eliminate any references to it, and it "magically" disappears and stops using memory. That's why we say that Java performs "automatic" GC. Why "eligible" for GC? Because objects are not collected immediately. GC is not instantaneous and comes with some performance impacts. Consequently, Java doesn't immediately collect every object that is eligible for collection; it typically postpones collection until a more convenient time later on. The way to think about GC in Java is that it's a "lazy bachelor" that hates taking out the trash and typically postpones the process for some period of time. However, if the trash can begins to overflow, Java immediately takes it out. In other words, if memory becomes scarce, Java immediately runs GC to free memory. Since we don't need to do anything special in order to dispose of objects in Java, how do memory leaks happen in Java? Memory leaks occur when a program never stops using an object, thus keeping a permanent reference to it. Let's take a look at an example that helps illustrate this point. The following code will cause all available memory in the JVM to be exhausted: When no more memory is remaining, an OutOfMemoryError alert will be thrown and generate an exception like this: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at MemoryLeakDemo.main(MemoryLeakDemo.java:14) In the example above, we continue adding new elements to the list memoryLeakArea without ever removing them. In addition, we keep references to the memoryLeakArea, thereby preventing GC from collecting the list itself. So although there is GC available, it cannot help because we are still using memory. The more time passes the more memory we use, which in effect requires an infinite amount memory for this program to continue running. This is an example of unbounded memory leak—the longer the program runs, the more memory it takes. So even if the memory size is increased, the application will still run out of memory at a later date. Meat & Potatoes Fixing Memory Leaks As illustrated by the flowchart below, the process of fixing memory leaks is fairly simple: Memory leaks are misunderstood creatures. Just getting an OutOfMemoryError alert doesn’t necessary mean that you're suffering from a memory leak. So, before you dive into "fixing" the problem, you must first find out whether or not a memory leak actually exists. If a memory leak does in fact exist, the next step is to determine which objects are leaking and uncover the source of the memory leak. Then, you fix it. We'll skip past the initial steps and dive right into diagnosing whether or not the problem is a memory leak. Is My Program Leaking Memory? Not every OutOfMemoryError alert indicates that a program is suffering from a memory leak. Some programs simply need more memory to run. In other words, some OutOfMemoryError alerts are caused by the load, not by the passage of time, and as a result they indicate the need for more memory in a program rather than a memory leak. To distinguish between a memory leak and an application that simply needs more memory, we need to look at the "peak load" concept. When program has just started no users have yet used it, and as a result it typically needs much less memory then when thousands of users are interacting with it. Thus, measuring memory usage immediately after a program starts is not the best way to gauge how much memory it needs! To measure how much memory an application needs, memory size measurements should be taken at the time of peak load—when it is most heavily used. The graph below shows the memory usage in a healthy Java application that does not suffer from memory leaks, with the peak load occurring around 10 AM and application usage drastically decreasing at 5 PM. Naturally, the peak load on business applications often correlates with normal business hours. The application illustrated by the chart above reaches its peak load around 10 AM and needs around 900MB of memory to run. This is normal behavior for an application suffering from no memory leaks; the difference in memory requirements throughout the day is caused solely by the user load. Now, let's suppose that we have a memory leak in the application. The primary characteristic of memory leaks is that memory requirements increase as a function of time, not as a function of the load. Let's see how the application would look after running for a few days with a memory leak and the same peak user loads reached around 10 AM every day: Because peak loads on the system are similar every morning but memory usage is growing over a period of a few days, this picture indicates a strong possibility of memory leaks. If the program eventually started suffering from OutOfMemory exceptions, it would be a very strong indication that there’s a problem with memory leaks. The picture above shows a memory leak of about 100MB per day. Note that the key to this example is that the only thing changing is the amount of time the system is up—the system peak load doesn’t change over time. This is not the case for all businesses. For example, the peak load for a tax preparation service is seasonal, as there are likely more users on the system in April than July. There is one special case that should be noted here: a program that needs to be restarted periodically in order to prevent it from crashing with an OutOfMemoryError alert. Imagine that on the previous graph the max memory size was 1100MB. If the program started with about 900MB of memory used, it would take about 48 hours to crash because it leaks about 100MB of memory per day. Similarly, if the max memory size was set to 1000MB, the program would crash every 24 hours. However, if the program was regularly restarted more often than this interval, it would appear that all is fine. Regularly scheduled restarts may appear to help, but also might make “upward sloping memory use” (as shown in the previous graph) more difficult to notice because the graph is cut short before the pattern emerges. In a case like this, you’ll need to look more carefully at the memory usage, or try to increase the available memory so that it’s easier to see the pattern. Monitoring Memory in Java As you are already aware, you need to measure memory that is free and used inside of the JVM, not memory that the JVM process is using. In other words, the top/Task Manager/Activity Monitor will measure how much memory your Java process is using, but that’s not what you need. You need a tool that can look inside the JVM process and tell you how much memory is available for your program running inside the JVM. In addition, keep in mind that Java GC is not a constant process—it runs in intervals. The memory usage that you see in the JVM is usually higher then what your program needs at the moment, as GC hasn’t yet run. Remember, lazy bachelors usually have some trash in their apartments waiting to be taken out. So, what you need to investigate is not current memory usage, but rather the average usage over a long period of time. For example, if your program is currently using 100MB of memory and five seconds later it’s using 101MB, that’s not an indication of a memory leak because GC might free up memory when it eventually runs. But if your program’s memory usage increases over a long period of time under constant usage, you might have trouble on your hands. There are a couple of options for measuring the amount of memory a program uses. The simplest one, which does not require any tools and works even with production systems, is the Verbose GC log. Verbose GC Log The Verbose GC log is defined when the JVM process is started. There are a couple of switches that can be used: -verbose:gc — prints basic information about GC to the standard output -XX:+PrintGCTimeStamps — prints the times that GC executes -XX:+PrintGCDetails — prints statistics about different regions of memory in the JVM -Xloggc: — logs the results of GC in the given file The following is an example of the output generated for Tomcat running in the default configuration with all of the previous switches enabled: 1.854: [GC 1.854: [DefNew: 570K->62K(576K), 0.0012355 secs] 2623K->2175K(3980K), 0.0012922 secs] 1.871: [GC 1.871: [DefNew: 574K->55K(576K), 0.0009810 secs] 2687K->2229K(3980K), 0.0010752 secs] 1.881: [GC 1.881: [DefNew: 567K->30K(576K), 0.0007417 secs] 2741K->2257K(3980K), 0.0007947 secs] 1.890: [GC 1.890: [DefNew: 542K->64K(576K), 0.0012155 secs] 2769K->2295K(3980K), 0.0012808 secs] The most important set of numbers is located in the second column after the second -> (e.g., in the top line shown it is 2623K->2175K(3980K). These numbers indicate that as a result of GC, we are using around 2200K of memory at the end of each GC cycle. This trace is not an indication of a memory leak—it shows a short-term trend with less then a second between samples, and that’s why we must observe long-term trends. However, if the Verbose GC log showed that the program was using around 2200K of memory after running for two days, and after running for 10 days it was using 2GB of memory (even after GC had just run), we could then conclude that there’s a memory leak. All the information that needs to be collected in order to determine if a memory leak exists can be found in the results of the Verbose GC logs. The other memory monitoring tools we’ll cover in this article simply provide more information in a form that’s easier to interpret. Monitoring the Java Process The following approach works for any Java process, including standalone clients as well as application servers like JBoss and servlet containers like Tomcat. It is based on starting the Java process with JMX monitoring enabled and attaching with the JMX monitoring tools. We’ll use Tomcat in the following example. To start Tomcat or the Java process with JMX monitoring enabled, use the following options when starting JVM: -Dcom.sun.management.jmxremote — enables JMX monitoring -Dcom.sun.management.jmxremote.port= — controls the port for JMX monitoring Note that if you’re on a production system, you’ll most likely want to secure your JVM before running it with these parameters. For that, you can specify these additional options: com.sun.management.jmxremote.ssl com.sun.management.jmxremote.authenticate Once started, you can use JConsole or VisualVM to attach to the process. Note that later JDK 6 versions include VisualVM. This is an example of the JConsole monitoring Tomcat. As shown in the example below, click on the Memory tab to get memory information: Again, we're interested in the long-term trends of heap memory usage, not trends from just a few minutes of running. Finally, note that monitoring tools like Hyperic can typically show historical trends over longer periods of time than VisualVM or JConsole. In addition, tools like Hyperic allow for more fine-grained control over operations that are permitted by users. As a result, we recommend using monitoring tools for production systems use, while all the other tools discussed in this section are more appropriate for developers or "first aid" in the absence of the real monitoring tools. Speed Kills We now know how to find out whether or not a memory leak exists, and we can even determine the speed at which we're leaking memory (e.g., 100MB per day). But are we better off with a "slow" leak (e.g., 1MB per day) or a "fast" leak (e.g., 500MB/hour)? It depends. With a "slow" leak, it takes longer for the system to run out of memory. For example, if we have 256MB of free memory remaining at the peak load time, it can take quite a long time to run out of memory with a 1MB/day memory leak. On the other hand, a faster memory leak makes it easier to reproduce and fix the problem. One of the best ways to quickly determine whether there's a fast memory leak or you simply need more memory to run at the peak time is to allot more memory to the program and see what happens. If you increase the available heap in a Java program and the time between crashes increases, you are likely suffering from memory leak. Let's assume that we're lucky enough to have a fast memory leak—on the order of 100MB per hour—on our hands, and that the initial memory picture looks like this: In this example, we can expect to run out of memory after about five hours of running time. How To Find Leaked Objects in a Fast Memory Leak Somewhat surprisingly, it is much easier to debug large memory leaks than small memory leaks. The reason is that memory leaks present a sort of "needle in the haystack" type problem—you need to find the leaked objects amongst all the other objects in the program. Suppose that the program we're debugging just ran out of memory. If the program initially had 512MB of free memory and now has none, it is obvious that the leaked objects used about 512MB of memory. The figure below illustrates this example: In this situation, half of the objects in memory have been leaked! If we randomly select an object, there's a 50% chance that it has leaked. And as memory leaks usually consist of objects of a few classes, you can get a really good start on determining which objects are leaking memory by sorting memory usage based on aggregate memory use of all objects of the same class. What if the ratio is less favorable (e.g., 512MB heap size, initially used memory 480MB, and 32MB of leaked objects at the end)? If you have an easy to reproduce memory leak, you can increase the heap size because an unbounded memory leak will eventually fill any amount of memory allotted to the program. So, you can increase the heap to 1GB, reproduce the memory leak, and get 544MB of leaked objects in a 1GB of heap. If we had a way to look at the "complete picture" of memory, it would be fairly easy to pinpoint leaked objects. Fortunately, there's a way to do exactly this: heap dump the process. Dump me Gently A heap dump is a list of objects in the memory of JVM as well as the content of the memory occupied by those objects. It preserves the value of any attributes of the objects, including references to other objects. In other words, a heap dump gives you a complete picture of the memory. There are multiple tools that allow you to dump heap in a Java process: If you're using JDK 6, you can use tool called jmap on any platform. If you're using JDK 5, the situation is slightly more complex: If you're running UNIX (Linux, Solaris, OS X) with JDK 5 you can use jmap. If you're using JDK 5 update 14 or later, you can use the -XX:+HeapDumpOnCtrlBreak option when starting JVM, then use the CTRL+BREAK key combination on Windows (or CTRL + \ on UNIX) to dump the heap. If you're running Windows and using JDK 5 pre-update 14, you'll soon wish you weren't. Trying to reproduce the problem with a more recent JDK is probably the best bet here. Some tools like VisualVM and memory profilers allow you to initiate a heap dump from the GUI, but you don’t need any fancy tools here—jmap will do just fine. As it provides the most general case, we'll use jmap in the next example. Before you dump heap, be sure to keep the following issues in mind: Programs in the JVM should be paused for the duration of the heap dump, which might take anywhere from ten seconds to several minutes. Many enterprise applications—and users of those applications—don’t take kindly to pauses that long, which may cause various timeouts to expire. So don’t try this at home or in production (unless the application is already a goner)! Heap dumps are saved on disk, and the files might be fairly large. A good rule is to make sure that you have at least twice the size of the physical memory free on the disk before you initiate a memory dump. With those final words of caution out of the way, you should now be ready to run the following command: jmap -heap:live,format=b,file=FILENAME PID Note that the -F option, which will dump non-responsive programs, might be useful on UNIX systems, but is not available on Windows. Note also that JDK 6 includes the option +XX:+HeapDumpOnOutOfMemoryError that will dump heap whenever the OutOfMemoryError alert is encountered. This can be a useful option, but keep in mind that it has the potential to consume significant amounts of disk space. You now have a heap dump in the file FILENAME and are ready to analyze it. What's In "Leaked" Memory? With the heap dump complete, we can now take a look at the memory and find out what's really causing the memory leak. Suppose that objects are holding references to each other as illustrated by the picture below. For the sake of easy calculation, let's assume that each object is 100 bytes, so that all of them together occupy 600 bytes of memory. Now, suppose that the program holds reference to object A for a prolonged period of time. As a result, objects B, C, D, E, and F are all ineligible for garbage collection, and we have the following amount of memory leaking: 100 bytes for object A 500 bytes for objects B, C, D, E and F that are retained due to the retention of object A So, holding reference to object A causes a memory leak of 600 bytes. The shallow heap of object A is 100 bytes (object A itself), and the retained heap of object A is 600 bytes. Although objects A through F are all leaked, the real cause of the memory leak is the program holding reference to object A. So how can we fix the root cause of this leak? If we first identify that object F is leaked, we can follow the reference chain back through objects D, C and A to find the cause of the memory leak. However, there are some complications to this "follow the reference chain" process: Reference chains can be really long, so manually following them can be time consuming An object is sometimes retained by more then one object, and there can even be circles involved as shown in the picture below: If we start following inbound references from object F in this example, we have to choose between following object C or object D. In addition, there's the possibility of getting caught in a circle by repeatedly following the path between objects D, E and B. On this small diagram it's easy to see that the root cause is holding object A, but when you're dealing with a situation that involves hundreds of thousands of objects (as any self-respecting memory leak does) you quickly realize that manually following the reference chain be very complex and time consuming. This is where some shortcuts can come in handy: If we had a tool that allowed us to play a "what would happen if I remove this reference" type of guessing game, we could run experiments that help locate the cause. For example, we could see that if we removed reference from "Cause of Leak" to A in the diagram above, objects A through F would all be freed. Some tools (like Quest's JProbe) have this capability. If the memory leak is large and we have a tool that allows us to sort objects by retained heap, we'll get an even greater head start because the objects with the largest retained heap are usually the cause of large memory leaks. Now that we understand what memory leaks are and how they can be corrected, let's find out how to fix them by analyzing heap dumps. Tools for Dealing with Heap Dumps Strictly speaking, you don’t need any tools that are not already part of the JDK. JDK ships with a tool called jhat, which you can use to inspect the heap dump. The process of fixing memory leaks is the same with all tools, and it's our opinion that no single tool is "light years" ahead of the others when it comes to fixing memory leaks. Although jhat will get the job done, better tools often provide a few extra helpful features: Present a better summary of heap statistics. Sort objects by retained heap. In other words, some tools can tell you the memory usage of an object and all other objects that are referenced by it, as well as list the objects referenced by other objects. This makes it much faster to diagnose the cause of a memory leak. Function on machines that have less memory then the size of the heap dump. For example, they'll allow you to analyze a 16GB heap dump from your server on a machine with only 1GB of physical memory. There are several free tools that are useful for analyzing heap dumps in Java. One that's widely used and is even included with the later versions of the JVM 6 is VisualVM. VisualVM is nice tool that gives you just enough to resolve memory leaks, and it shows heap dumps and relations between objects in graphical form. Feature-wise, one step above VisualVM is the Eclipse Memory Analyzer Tool (MAT), a free tool that includes a lot of additional options. Although it's still in incubation phase as of publication of this article, MAT is free and we've found it to be extremely useful. Commercial products like JProfiler, YourKit, and JProbe are also excellent tools for debugging memory leaks. These applications include a few options that go above and beyond VisualVM and MAT, but they're certainly not necessary to successfully debug memory leaks. Unless you already have a license for one of these commercial tools, we recommend trying MAT first. Analyzing the Heap Dump for Fast Memory Leaks It's usually easy to find a fast memory leak. A lot of memory is leaked, and you simply need to find the big hog that leaked all that memory. Analyzing the heap dump is done in order to: Find objects that are "leaking" Find the root cause of the memory leak If you have a fast memory leak and are able to reproduce it in such a way as to make the leaked objects a significant portion of the final memory picture, then determining which objects are leaked is simple because they occupy a significant portion of the memory. To determine which objects are leaked, sort the classes by total memory usage of all instances of each class. The objects near the top of the list are usually the leaked objects. Then, you can follow the reference chain to them until you find the cause of the memory leak. A useful heuristic here is that if you sort all the objects by retained heap, you can find the objects that are likely the root cause of the memory leak. Again, an example: If we assume that each instance of object C is 100 bytes, then holding object A resulted in a memory leak of almost 1GB! In other words, the retained heap of object A in this example is about 1GB. So, if we were investigating this memory leak, it would be fairly obvious that we should start there. Remember, the Eclipse Memory Analyzer Tool (MAT) allows you to sort the heap dump by the retained heap usage of the objects, so it would have been easy to use MAT to determine the retained heap of object A. Slow and Small Memory Leaks It always helps if you can easily reproduce the problem, but what if the memory leak is really small and slow? What should you do if after a few days you can reproduce only a 10MB leak in a heap that contains 2GB of objects? In situations like this, looking for the cause of the problem can feel like finding a needle in the haystack. One solution in this case is the brute force approach—simply devote enough time looking at the objects to find the problem. While that will ultimately work, there's another way to address the problem if you have an idea of which operations are leaking memory. If you know or suspect which operations leak memory, you can find even relatively slow memory leaks. The secret here is to compare heap dumps, as this highlights objects that are present in one heap dump but not in another. For example, suppose that objects A, B, and C are present in the first heap dump, while objects A, B, C, D, and E are present in the second heap dump. In this case, objects D and E are the difference. Comparing heap dumps makes it easier to find memory leaks because it effectively reduces the size of the "haystack" containing your needle. Comparing heap dumps can easily reduce the size of the objects for which you need to examine heaps from 2GBs to a few KB. Many commercial profilers as well as Eclipse MAT can be used to compare heaps. Once you've selected a tool that can compare heaps, simply follow the process explained in the following diagram: In effect, what you’re doing is creating two snapshots, one before and one after executing the use case that you know is leaking memory. That way, you significantly reduce the number of objects that you need to investigate to find leaked objects. After you find the objects that leaked, you can find the cause of the memory leak as described in the previous section. Why perform garbage collection before taking a snapshot? Because some tools won’t automatically perform GC for you, and consequently the snapshots might include objects that are no longer reachable. Many modern tools will perform GC before taking the snapshot, but if you are unsure whether or not your tool performs full GC before taking a snapshot it is recommended that you do so yourself. That way you don’t have to worry about objects that are eligible for GC but have not yet been collected. Among the objects that are created during the use case, some are supposed to be there because they are the result of the use case execution (e.g., we created a new customer, and that customer object should be retained), while other objects are the memory leak. However, since the only objects present in the difference between the snapshots are the ones that were created during the use case, the size of the haystack you need to look through is significantly reduced. Practical Problems Why do memory leaks sometimes take a long time to fix if the process is this simple? In our opinion, the main reasons are: There isn’t a sufficient amount of easily available information about fixing Java memory leaks. We hope this article will help improve the situation. It can be difficult to reliably reproduce an issue, and a lot of time is typically required to reproduce an issue before you can really start addressing it. People lack the right tools for the job—in particular, memory profilers. With many free profilers available and reasonably low prices for commercial profilers, there’s no reason you shouldn’t have good tools in your toolbox. There are a few practical issues with the use of tools and techniques that might be perceived as road blocks the first time someone tries them. We’ll address some of these issues below. Fortunately, the practical issues most commonly encountered aren’t very difficult to solve. The most common problems are: You can’t load the snapshot because you don’t have enough memory in your development box. For example, this can easily happen if you have 64-bit servers with 8GB of memory allocated to the JVM. You might not have physical memory in your development box, or you might be using the 32-bit JVM on your development box, but the tool you’re using insists on loading most of the snapshot into memory. If you like the idea of having a really powerful development box, use this situation to your advantage and ask your boss for a new machine. Alternatively, try using a tool with less of an appetite for memory, like Eclipse MAT. You can’t increase your memory on the server to get a bigger set of leaked objects, yet you need a bigger set of leaked objects to speed up the process of finding the memory leak. This often happens with 32-bit JVMs. One option in this situation is to apply the techniques described above for finding slow memory leaks, although this approach requires a significant amount of time. Alternatively, you can try to reproduce problem on a 64-bit JVM, which will allow you to increase the memory. If the problem causing the memory leak is in your application, changing the JVM is extremely unlikely to “hide” the leak. The memory leak is not in objects of just one class—you’re leaking objects from thousands of classes, so it’s difficult to find leaked objects. Or, your graph of object relations is so complex that you can find the initially-leaked objects but can’t locate the cause. In either case, you’ve got quite a pickle on your hands. The only words of encouragement we can offer are that the situation will eventually improve, as the longer you track object references the better you’ll get at it. One other piece of advice: you should probably call your significant other and let him or her know that you won’t be home for dinner. You’re getting an OutOfMemoryError alert with a new, different format. For example, you might be looking at something like this: Exception in thread “pool-2-thread-1″ java.lang.OutOfMemoryError: GC overhead limit exceeded. This message can happen with some GC settings that limit the overhead of the GC. It often indicates a memory leak, although there are some corner cases in which problems with GC performance can cause it. The best way to distinguish the cause of this message (is it a GC misconfiguration or a memory leak?) is to examine the pattern of errors. If they happen with regularity (in other words, they’re a function of time the program is running), then it’s a memory leak. If they happen only occasionally under heavy loads and clear later (in other words, they’re not a function of running time), they might indicate a need to tweak the GC or increase available memory. 3rd party caching systems can be setup so that caches are allowed to expand and fill up the available memory the program doesn’t need. However, if the program no longer needs more memory, the cache automatically releases it. The point here is that in the absence of any OutOfMemoryError alerts, continually increasing memory usage does not necessary indicate a memory leak. Both “used memory grows as function of time” and “free memory eventually runs out” conditions are necessary in order to determine the presence of a memory leak. Finishing Up This article described the methodology and techniques necessary to fix memory leaks. These techniques are universal—they apply to any profiler, and some of them even apply to languages other than Java that use garbage collection. The information presented above should give you everything you need in order to use the tools of your choice to investigate and fix memory leaks. The proper usage of specific tools will be discussed in future articles, which will focus primarily on free tools and problems in Java. However, if there’s enough interest we might tackle other languages as well. If you’re interested in follow-up articles related to memory leak resolution in a different environment or language, please leave a comment or contact us at docs-at-openlogic.com. Additional Resources The following links provide additional information on the behavior of GC in Java as well as different tools that you may find useful. http://java.sun.com/j2se/1.5.0/docs/guide/management/agent.html http://java.sun.com/j2se/1.5.0/docs/guide/management/jconsole.html http://java.sun.com/j2se/reference/whitepapers/memorymanagement_whitepaper.pdf http://java.sun.com/developer/technicalArticles/Programming/GCPortal/ http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html http://www.eclipse.org/mat/ https://visualvm.dev.java.net/ http://www.ej-technologies.com/products/jprofiler/overview.html http://www.yourkit.com/ http://www.quest.com/jprobe/ http://java.sun.com/j2se/reference/whitepapers/memorymanagement_whitepaper.pdf http://java.sun.com/javase/6/docs/technotes/tools/share/jhat.html http://java.sun.com/j2se/1.5.0/docs/tooldocs/share/jmap.html http://java.sun.com/j2se/1.5.0/docs/tooldocs/share/jstat.html The above article originally appeared on Wazi, a clearinghouse for the timeliest thinking on open source software."
March 18, 2009
by Veljko Krunic
· 295,285 Views · 2 Likes
article thumbnail
Dynamic Java Programming With Rule Engine
Rules are statements that define business procedures and policies. Normally, every business application contains many embedded business rules that determine business process flow and execution. When we program these rules with static languages like Java, there is no problem at all. However, the problem is that some business rules change over time very frequently. It is very expensive to fix these changing parts since it requires new versions and software development cycles; develop, deploy on a test server, test, package, deploy on a production server etc. Here “Dynamic Programming” comes to our rescue. Our program doesn’t have to necessarily be written with a dynamic language. We can use this advantage in Java with Rule Engine mechanism. Rule Engines don’t serve only the dynamic programming. They evaluate rules and extract some inferences by using some algorithms like RETE. This inference mechanism is hardly possible with many “if-else” statements in normal program code. Rule Engines have generally 4 types of rules; Constraint Rule: We may need to define restriction or limits for some entities. “A customer sales order amount must be lower than 100 if the plant is over capacity ” Validation Rule: Some validation rules may change over time. “Production order can’t be accepted if current day is Sunday” Action Rule: Some actions may be triggered. “Send a notification e-mail if a purchase order is over $1000 to boss”. Computation Rule: Some formula may be executed. “Discount is 20% if customer is Mr. Anderson” We can use Rule Engines on the market but even we can write our own Rule Engine to benefit the dynamic programming in Java. By using Java compilers, classloaders, it is not so much hard. My favorite rule definition format is Java source code instead of some natural languages or XML. This can also reduce the overall development cost of your custom rule engine. By writing Java rule definitions, we can also use the IDE debugger features without learning a new syntax. If we develop our own Rule Engine, we may skip the inference engine part since we only want dynamic programming feature for now. Best place for plugging rule execution is database access layer. Rules should work invisibly behind the database objects. In some cases, some rules might be invoked from static program codes. The benefits that we have when using Rule Engine are: Dynamic programming with hotswapping feature that provides fix&run easiness. Minimal cost of business rule changes that occurs many times. Accumulation of business rules in one place that may result re-usability and code repetition prevention. Dynamic validations that may change over time. Constraints can be defines for default values, assertion checks etc. In any time rule may be turned off/on. Some of the features of Rule Engine may remind us database constraints and triggers. Using database may hinder portability and require DBA involvement. I think Rule Engine is better for business rule definitions. These database features may be used for other purposes. The critical question we should ask ourselves is which part of business rules should be taken into Rule Engine from the programs. The answer is “frequently changing” rules. Some changing parts can be defined within parameter tables (processing options) but we can’t define computation or many if-else statements in these tables. But we should prefer parameter table if it is enough for the problem. Links: http://java.sun.com/developer/technicalArticles/J2SE/JavaRule.html http://www.javaworld.com/javaworld/jw-04-2005/jw-0425-ruleengine.html http://www.javaworld.com/javaworld/jw-06-2006/jw-0612-dynamic.html http://www.amazon.com/Principles-Business-Addison-Wesley-Information-Technology/dp/0201788934/ref=sr_1_3?ie=UTF8&s=books&qid=1236927606&sr=1-3 http://www.amazon.com/Business-Rules-Applied-Building-Approach/dp/0471412937/ref=sr_1_1?ie=UTF8&s=books&qid=1236927606&sr=1-1
March 13, 2009
by Adam Brown
· 33,229 Views
article thumbnail
Wicket Tutorial Series: Setting Up the Project
Each day this week will feature a new article with an in-depth look at the creation process behind setting up a Java project and implementing the frontend with Apache Wicket. Wicket is a Java web application framework which allows “Designers” (people good with Dreamweaver) and “Developers” (people good with Java and Databases) to collaborate on a project with minimal chances of stepping on each other’s toes or wearing each other’s hats. The beauty of Wicket is that it uses plain xhtml pages as it’s templating markup. This means that html pages can be loaded into Dreamweaver (or whatever tool the Designer is comfortable with) and they will look very close to the same as they would when rendered on the deployment web server. Workflow The basic workflow involved in creating and maintaining html rendered by Wicket is as follows: The Designer creates the html for the website and fleshes it out with “mock” sections. For instance in the application we intend to create during our Five Days of Wicket will be a pastebin application called “Mystic Paste”. In our application we’ll have an “Add Code to Mystic Paste” page, mock data might include some user created content in the textarea of the page. All css/images, etc… are setup such that if they were to be put on a webserver, everything would work. The Developer needs to flesh out the dynamic areas of the webpage, that is, he needs to instruct Wicket where it will need to show information from the server. The developer does this by decorating the designer’s html page with special Wicket tags and attributes. Because these tags and attributes are just considered part of another namespace separate from xhtml’s, editors like Dreamweaver and browsers will simply ignore them - It is important to note: The developer will still keep the “mocked” sections of the page intact, this is so the page renders and looks fleshed out on its own. The mocked sections will be replaced by real data when rendered by Wicket. The Developer hands the file back to the Designer. The Designer is free to make further edits, so long as he/she does not remove or manipulate the Wicket tags and attributes present in the file. If the Designer does need to remove any Wicket tags or attributes, they need to consult the Developer as such an action will “break” the webpage when Wicket renders it. Example Wicket Page Here is an example of a Wicket page. This example was taken from Manning Publishing’s book “Wicket in Action”: ... Gouda Gouda is a Dutch... $1.99 add to cart Emmental Emmental is a Swiss che... $2.99 add to cart ... This looks almost 100% like a normal webpage would look, the only difference is the addition of the “wicket:XXX” attributes and tags sprinkled through the document. The parts of the document using the special Wicket namespace modifiers will be replaced/removed in the final markup when Wicket renders the page to the user’s browser. Notice the “” element? This is where your designer can put a “mocked” version of what that area of the page should look like. You as a developer can take that mocked html and divide it out into a template that is dynamically driven from the backend. Here is how the final page looks if you were to simply load the page into a web browser (or Dreamweaver) from your hard drive: Preparing for Setup Deviating a bit from the Standard Wicket Convention One of the first things a developer notices when starting out with Wicket is the convention where Wicket likes having its html template files live at the same level and in the same packages as it’s Java source files. Sure you can jump through hoops to get Wicket to load the html template files from elsewhere but a nice compromise is to simply keep your html template files within the same package directory structure but in a source folder separate from the Java classes. Why? Well quite simply to keep your designers (Dreamweaver folks!) from having to grab Java source files along with the html files they are working on. It will just confuse them and clutter their directories. You can of course stick with the typical “Java source files along side html” convention if you wish, but I find it much cleaner to separate them during design time, and have Maven combine them only at build time into the target war (which it will gladly do automagically). Project Folder Structure . |-- pom.xml |-- src | `-- main | |-- filters | |-- java | | `-- com | | `-- mysticcoders | | `-- mysticpaste | | |-- model | | |-- persistence | | | |-- exception | | | |-- hibernate | | |-- services | | `-- web | | |-- ajax | | |-- pages | | |-- panels | | `-- servlet | |-- resources | | |-- com | | | `-- mysticcoders | | | `-- mysticpaste | | | |-- spring | | | `-- web | | | |-- ajax | | | |-- pages | | | `-- panels | | `-- log4j.properties | |-- sql | `-- webapp | |-- WEB-INF | |-- images | |-- js | `-- css src/main: maven builds source and resources from this directory to the main deployable target (i.e. our war file) filters: we keep a set of “filters” files that maven can use to interpolate variables at build time. What does this mean? It means that inside your configuration files, the files you use to setup database connections or file paths, you can insert variable place holders like ${db.host}. When maven does a build, it looks up the correct filter file to use and looks for the key=value part corresponding to “db.host” and inserts it into the configuration file for you. This ensures that you are able to configure your application per environment you deploy to (i.e. DEV, QA, PROD, etc…) by having different filter files with the same keys but different values. For more information see Maven’s documentation on filtering resource. java/*: this folder will contain all of the application’s source code. Everything from the database access code, wicket code and services code for the mysticpaste application (see below). model: all “domain” classes, that is, classes that represent the objects in the application. For mysticpaste you’ll see classes like “PasteItem” which represents an item pasted to the mysticpaste. persistence: at this level of the persistence package a list of interfaces will be kept. The interfaces comprise the basic access layer the services layer will use to save, retrieve and update items to/from the paste bin. exception: the peristence layer needs to tell the services layer when things have gone wrong. It does this via delcaring and throwing exceptions. hibernate: such is our case, our persistence interfaces will be implemented via the ORM known as Hibernate. This package will store all of the custom hibernate implementations and hibernate specific classes services: The services layer will be stored here. Both the generic interfaces and their implementation classes. The persistence layer will be injected via spring. web: this folder is where our Wicket classes will reside and it’s split into several category packages which are as follows: ajax: mysticpaste uses Ajax to render portions of its UI. The wicket classes which render the xml/html to be injected dynamically into the page are stored here. pages: standard Wicket page classes which are used throughout the application are stored here panels: reusable panel classes are stored here. Panels may be included within Wicket pages for sake of templating servlet: any run of the mill servlet code we need is stored here. A good example might be an ImageUploadServlet resources/*: the resources folder will hold our non-java based files. Noteably html files and spring confguration files spring: Holds any spring configuration files needed to wire the services and persistence layer web: this folder and all subfolders mirror the packages under src/main/java/…/web and hold the .html files that the Wicket page/panel classes use as their templates. As described above, a “standard” wicket application simply stores the .html files along side their Wicket classes under src/main/java/…/web, however we want to keep these files separate from the Java source so as to keep the directory our designers checkout from version control contianing only the files they need to work on. sql: any sql scripts we need to keep handy for building the mysticpaste database. webapp: this folder will keep the files which live at the base directory of our war file WEB-INF: where you keep your web.xml file images: any image resource, .gif/.png/.jpg files your webapp will reference js: javascript files your webapp will reference css: style sheets your webapp uses src/test/*: All files which reside under this folder are test classes and resources needed to support the tests. Maven will build everything under src/main/java and add it to the class path of the JUnit or TestNG classes you create. java: JUnit or TestNG test classes which will be run during a build resources: resource files which are needed to support the tests Getting Started Since we are using Maven as our build tool we can take advantage of the fact that the fine folks at the Wicket project have created a specialized “archetype“ which creates a skeleton web application complete with a folder structure which mimics roughly what we have outlined above and Maven pom.xml file used to build a war. The Wicket contributors have even gone one step further and have created a little web page which will, based off a few drop down options, generate the maven command you need to execute in order to create the boiler plate Wicket project. You can find this web page over on the Apache Wicket site under the “Quick Start” link. Copying the above Maven command creates a Skeleton Wicket Project To be precise, the command I used was: mvn archetype:create -DarchetypeGroupId=org.apache.wicket -DarchetypeArtifactId=wicket-archetype-quickstart -DarchetypeVersion=1.3.5 -DgroupId=com.mysticcoders -DartifactId=mysticpaste And I ended up with the following folder structure: . `-- mysticpaste |-- pom.xml `-- src |-- main | |-- java | | `-- com | | `-- mysticcoders | | |-- HomePage.html | | |-- HomePage.java | | `-- WicketApplication.java | |-- resources | | `-- log4j.properties | `-- webapp | `-- WEB-INF | `-- web.xml `-- test `-- java `-- com `-- mysticcoders |-- Start.java `-- TestHomePage.java Now obviously we’ll have to rearrange a few things, for instance I want my base package to be com.mysticcoders.mysticpaste, but that’s easy enough to do once we are in an IDE. For now, let’s test this example webapp out and see if it works. To do that switch into the mysticpaste directory (the directory that has pom.xml in it) and type the following: mvn jetty:run This will start up a Jetty webapp container running on port 8080 (if you have something running there already, use the -Djetty.port= option). Startup a webbrowser and navigate to http://localhost:8080/mysticpaste/ You should see: Your IDE Sooner or later you’re going to want to crack open your IDE and start hacking away. Maven makes this extremely easy by allowing you to create IDE specific project files based off of the Maven pom.xml file. Eclipse mvn eclipse:eclipse For eclipse you’ll also have to set the M2_REPO classpath variable for the workspace your project resides under. Do this by entering the following command: mvn -Declipse.workspace= eclipse:add-maven-repo IntelliJ IDEA mvn idea:idea -OR- in IDEA 7+ simply open the pom.xml file Netbeans Netbeans supports maven out of the box, just “Open Project” and choose the mysticpaste directory that contains the pom.xml file When generating the project files through Maven, the project is setup such that classpath entries point to your local Maven repository (i.e. ~/.m2/repository, or C:Documents and Settingsyourusername.M2repository on Windows). It also sets up src/main/java, src/main/resources as “source folders”. You may add other folders to the source folder list as per your IDE if needed, the only thing you have to remember is if you ever use mvn eclipse:clean followed by mvn eclipse:eclipse again, those other source folders will have to be readded through your IDE. Instead, you should add the source/resource folders directly to your pom.xml, this way they will be maintained. Spring The Mystic Paste application will use Spring, and really you should too. Unless you have been hiding under a rock or work in a corporate environment so lame as to which technologies newer than 2002 are forbidden you should learn to accept Spring as a defacto standard. Dependency injection for the win! We add the following to our pom.xml: org.apache.wicket wicket-spring-annot org.springframework spring org.springframework spring-test org.springframework spring-tx wicket-spring-annot: allows us to wire our Wicket application via handy dependency injection annotations (i.e. @SpringBean, see Wicket documentation for more detail) spring: is just the core spring libraries spring-test: is a set of Spring integration classes for Unit testing spring-tx: is the Spring Transaction Management api for declarative transactions web.xml additions for Spring In order for Spring to manage our Wicket application we need to setup the Wicket filter with a Spring-aware application factory. This allows us to wire our Wicket Application class in our applicationContext.xml file, which is really handy if you have a services and configuration settings you want to inject into the Wicket Application object so the rest of your application can access them. To do this, we change the original Wicket filter like so: wicket.mysticpaste org.apache.wicket.protocol.http.WicketFilter applicationFactoryClassName org.apache.wicket.spring.SpringWebApplicationFactory As well, we want our Spring context to be available to our webapp if ever there is a need for one of our pages to access the Spring managed beans directly: contextConfigLocation classpath:com/mysticcoders/mysticpaste/spring/applicationContext.xml org.springframework.web.context.ContextLoaderListener Hibernate Hibernate is our ORM of choice, it will allow us to persist and retrieve our model objects to and from the underlying database, whatever that database may be. We add the following to our pom.xml: org.hibernate hibernate-annotations c3p0 c3p0 commons-dbcp commons-dbcp javax.transaction jta 1.0.1B hibernate-annotations: used so we can annotate our model classes with mapping information, instead of having to create a separate mysticpaste.hbm.xml file. c3p0: provides a connection pooling library Hibernate can use commons-dbcp: another connection pooling library, we’ll add it as well and decide whether to use it or c3p0 later jta: this is the Java Transaction API which is needed by Hibernate (Hibernate provides an implementation of the API) web.xml additions for Hibernate To have a Hibernate Session open and ready for our webapplication during a Request Cycle we need to setup a Hibernate filter like so (otherwise, good luck getting lazy loading working!): open.hibernate.session.in.view org.springframework.orm.hibernate3.support.OpenSessionInViewFilter open.hibernate.session.in.view /* As the comment states above, make sure this filter-mapping exists *before* your wicket.mysticpaste filter or else it just plain won’t work. Database For the Mystic Paste we decided to use the freely available PostgreSQL. Adding support for PostgreSQL is very easy, unlike with some of the commercial DBMSes where you have to download and install their JDBC driver into your repository. To add support for Postgres, we simply add the following to our pom.xml: postgresql postgresql Servlets Regardless of which webapplication framework you choose there are just some times when a plain jane Servlet comes in really handy. If you have a need for Servlets and the Servlet must have access to the Wicket session add the following to your web.xml: wicket.session org.apache.wicket.protocol.http.servlet.WicketSessionFilter filterName wicket.mysticpaste And then, after your other filter-mappings add the following (assuming you mount your servlet-mappings under /servlet/): wicket.session /servlet/* Maven Filters and Profiles In order to build our Mystic Paste project for various environments (DEV/QA/PROD) we need to implement both Maven profiles and filters. Filters Filters allow you to place variables inside your configuration files and have those variables filled in durring build time. This is very handy for setting environment specific things such as database connection information. Enabling filters is quite easy, we open up the pom.xml file and find the section for and set the value for the element to true as follows: true src/main/resources . . . But for filtering to work, we need to specify a filters file. It’s not enough to specify only one filter file because we need to specify different filters per environment and we’ll do that by using Maven Profiles. Profiles To setup a profile, create a new set of elements following the section in your pom.xml file. Like so: DEV DEV DEV QA QA PROD PROD and just above your tag underneith your tag you would add the following elements: mysticpaste src/main/filters/filters-${env}.properties true src/main/resources src/main/filters will contain the following files. |-- pom.xml |-- src | `-- main | |-- filters | | `-- filters-DEV.properties | | `-- filters-QA.properties | | `-- filters-PROD.properties filters-DEV.properties jdbc.url=jdbc:postgresql://localhost:5432/mysticpaste jdbc.user=mysticpaste jdbc.password=password image.upload.path=/tmp/pasetbin/userimages image.upload.size.limit=4096K filters-PROD.properties jdbc.url=jdbc:postgresql://192.168.2.10:5432/mysticpaste jdbc.user=mysticpaste jdbc.password=CrYp71c image.upload.path=/mysticpaste/userimages image.upload.size.limit=4096K Now within any file under src/main/resources that has variables of the form ${variable.name} will have those variables replaced with the values specified in the proper filters file located under src/main/filters. For instance here is an example of a Spring applicationContext.xml file which will be interpolated with proper variables values at build time: applicationContext.xml To determine which filters file will be used depends on the profile chosen when building. For example, to build to production using the filters-PROD.properties we would execute the following: mvn clean deploy -P PROD The profile you use with the -P switch must match one of the values of the element for a profile. Conclusion Although it’s quite easy to get started with the Maven QuickStart project it is sometimes a bit frustrating putting some of the additonal pieces together. Building to several environments, setting up depenencies not included in the QuickStart project and strucuturing your project in an effort to make life easy for yourself as a developer and for your designer. I hope our Day 1 tutorial leaves you with a good sense of how a Wicket project is setup, now we can move onto coding the app! In the next four days we will be covering: Writing the Tests Designing the backend Designing the Wicket components Putting it all together Mystic Coders, LLC has been coding web magic since 2000. Mystic is a full-service Development Agency specializing in Enterprise development with Java. They are usually involved in developing enterprise-grade software for companies large and small, and have experience working in diverse industries, including b2b, b2c, and government-based projects. Mystic has done work with large companies such as LeapFrog, Nestlé, Harrah's Entertainment and the Los Angeles Conventions & Visitor's Bureau, among others. Andrew Lombardi, CTO of Mystic, is available for speaking engagements. For more about Mystic, check us out at http://www.mysticcoders.com
March 10, 2009
by Craig Tataryn
· 30,211 Views
article thumbnail
JAX-RPC client with Maven2
Recently I needed to make my Maven2 web project communicate with an old style RPC encoded web service. We run on GlassFish which comes with JAX-RPC RI built-in, so I was hoping to find a way to use it without having to bundle another runtime such as Axis into the project. Specifically I was looking for a way to make my build-script generate client stubs from a WSDL file in the project tree. I want the client stubs to use standardized JAX-RPC APIs which are serviced by the implementation provided by GlassFish. I found that the JAX-WS RI has a Maven2 plugin but not the JAX-RPC RI. I did a lot of googling, and posted messages in a few mailing lists. Surprisingly there is almost nobody using JAX-RPC on Maven2 projects because there are barely any examples online and nobody could answer my question. I'm surprised because there are a lot of legacy systems out there that developers need to integrate with, so I can't be the only one still needing JAX-RPC support. Some people suggested that I update the web service to modern RPC literal to make client-side development easier with JAX-WS, but that is not an option. Others suggested that I use Maven's antrun plugin to call out to an ant build script that does JAX-RPC work. Finally I found an all Maven solution that works for me: org.apache.axis axis 1.4 compile org.apache.axis axis-jaxrpc 1.4 compile commons-discovery commons-discovery 0.4 compile wsdl4j wsdl4j 1.6.2 compile org.codehaus.mojo axistools-maven-plugin 1.3 wsdl2java com.mycompany.service.client src/main/resources/META-INF/wsdl target/generated-sources/wsdl2java private MyServiceSEI getMyServicePort() throws ServiceException { MyServiceLocator locator = new MyServiceLocator(); MyServiceSEI port; Stub stub; port = locator.getMyServiceSEIPort(); // OPTIONALLY configure URL and HTTP BASIC authentication stub = (Stub) port; stub._setProperty(Stub.ENDPOINT_ADDRESS_PROPERTY, "http://hostname/ContextRoot/ServicePort"); stub._setProperty(Stub.USERNAME_PROPERTY, "user"); stub._setProperty(Stub.PASSWORD_PROPERTY, "secret"); return port; } It works, but I think the client stubs are dependent on the Axis 1 runtime instead of using the GlassFish provided JAX-RPC runtime through the standardized JAX-RPC APIs. I've wasted enough time on this so I moved on. If you know a better way to do this, please let me know. No RESTful services rants please! I think SOAP and REST both have their uses depending on what you are doing. From my perspective, JAX-WS SOAP development is effortless and requires significantly less code than REST because I don't have to write the building/parsing code. After examining raw SOAP messages without extra WS-* headers added in, I decided that overhead is not a valid argument against SOAP. When you need more advanced features they are available to you without having to re-invent the wheel. I think most Java developers' negative impressions on SOAP comes from the JAX-RPC and Axis days, and because they haven't tried JAX-WS yet. From http://www.ryandelaplante.com/
March 9, 2009
by Ryan Developer
· 36,984 Views
article thumbnail
Using Java, POST A Block Of XML To A Web Server
// This is a working example of POSTing a string (representing a block of XML) to a web server try { URL url = new URL( argUrl ); URLConnection con = url.openConnection(); // specify that we will send output and accept input con.setDoInput(true); con.setDoOutput(true); con.setConnectTimeout( 20000 ); // long timeout, but not infinite con.setReadTimeout( 20000 ); con.setUseCaches (false); con.setDefaultUseCaches (false); // tell the web server what we are sending con.setRequestProperty ( "Content-Type", "text/xml" ); OutputStreamWriter writer = new OutputStreamWriter( con.getOutputStream() ); writer.write( requestXml ); writer.flush(); writer.close(); // reading the response InputStreamReader reader = new InputStreamReader( con.getInputStream() ); StringBuilder buf = new StringBuilder(); char[] cbuf = new char[ 2048 ]; int num; while ( -1 != (num=reader.read( cbuf ))) { buf.append( cbuf, 0, num ); } String result = buf.toString(); System.err.println( "\nResponse from server after POST:\n" + result ); } catch( Throwable t ) { t.printStackTrace( System.out ); }
March 4, 2009
by Douglas Wyatt
· 22,649 Views
  • Previous
  • ...
  • 243
  • 244
  • 245
  • 246
  • 247
  • 248
  • 249
  • 250
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×