DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Deployment Topics

article thumbnail
Apache Solr: Get Started, Get Excited!
we've all seen them on various websites. crappy search utilities. they are a constant reminder that search is not something you should take lightly when building a website or application. search is not just google's game anymore. when a java library called lucene was introduced into the apache ecosystem, and then solr was built on top of that, open source developers began to wield some serious power when it came to customizing search features. in this article you'll be introduced to apache solr and a wealth of applications that have been built with it. the content is divided as follows: introduction setup solr applications summary 1. introduction apache solr is an open source search server. it is based on the full text search engine called apache lucene . so basically solr is an http wrapper around an inverted index provided by lucene. an inverted index could be seen as a list of words where each word-entry links to the documents it is contained in. that way getting all documents for the search query "dzone" is a simple 'get' operation. one advantage of solr in enterprise projects is that you don't need any java code, although java itself has to be installed. if you are unsure when to use solr and when lucene, these answers could help. if you need to build your solr index from websites, you should take a look into the open source crawler called apache nutch before creating your own solution. to be convinced that solr is actually used in a lot of enterprise projects, take a look at this amazing list of public projects powered by solr . if you encounter problems then the mailing list or stackoverflow will help you. to make the introduction complete i would like to mention my personal link list and the resources page which lists books, articles and more interesting material. 2. setup solr 2.1. installation as the very first step, you should follow the official tutorial which covers the basic aspects of any search use case: indexing - get the data of any form into solr. examples: json, xml, csv and sql-database. this step creates the inverted index - i.e. it links every term to its documents. querying - ask solr to return the most relevant documents for the users' query to follow the official tutorial you'll have to download java and the latest version of solr here . more information about installation is available at the official description . next you'll have to decide which web server you choose for solr. in the official tutorial, jetty is used, but you can also use tomcat. when you choose tomcat be sure you are setting the utf-8 encoding in the server.xml . i would also research the different versions of solr, which can be quite confusing for beginners: the current stable version is 1.4.1. use this if you need a stable search and don't need one of the latest features. the next stable version of solr will be 3.x the versions 1.5 and 2.x will be skipped in order to reach the same versioning as lucene. version 4.x is the latest development branch. solr 4.x handles advanced features like language detection via tika, spatial search , results grouping (group by field / collapsing), a new "user-facing" query parser ( edismax handler ), near real time indexing, huge fuzzy search performance improvements, sql join-a like feature and more. 2.2. indexing if you've followed the official tutorial you have pushed some xml files into the solr index. this process is called indexing or feeding. there are a lot more possibilities to get data into solr: using the data import handler (dih) is a really powerful language neutral option. it allows you to read from a sql database, from csv, xml files, rss feeds, emails, etc. without any java knowledge. dih handles full-imports and delta-imports. this is necessary when only a small amount of documents were added, updated or deleted. the http interface is used from the post tool, which you have already used in the official tutorial to index xml files. client libraries in different languages also exist. (e.g. for java (solrj) or python ). before indexing you'll have to decide which data fields should be searchable and how the fields should get indexed. for example, when you have a field with html in it, then you can strip irrelevant characters , tokenize the text into 'searchable terms', lower case the terms and finally stem the terms . in contrast, if you would have a field with text in it that should not be interpreted (e.g. urls) you shouldn't tokenize it and use the default field type string. please refer to the official documentation about field and field type definitions in the schema.xml file. when designing an index keep in mind the advice from mauricio : "the document is what you will search for. " for example, if you have tweets and you want to search for similar users, you'll need to setup a user index - created from the tweets. then every document is a user. if you want to search for tweets, then setup a tweet index; then every document is a tweet. of course, you can setup both indices with the multi index options of solr. please also note that there is a project called solr cell which lets you extract the relevant information out of several different document types with the help of tika. 2.3. querying for debugging it is very convenient to use the http interface with a browser to query solr and get back xml. use firefox and the xml will be displayed nicely: you can also use the velocity contribution , a cross-browser tool, which will be covered in more detail in the section about 'search application prototyping' . to query the index you can use the dismax handler or standard query handler . you can filter and sort the results: q=superman&fq=type:book&sort=price asc you can also do a lot more ; one other concept is boosting. in solr you can boost while indexing and while querying. to prefer the terms in the title write: q=title:superman^2 subject:superman when using the dismax request handler write: q=superman&qf=title^2 subject check out all the various query options like fuzzy search , spellcheck query input , facets , collapsing and suffix query support . 3. applications now i will list some interesting use cases for solr - in no particular order. to see how powerful and flexible this open source search server is. 3.1. drupal integration the drupal integration can be seen as generic use case to integrate solr into php projects. for the php integration you have the choice to either use the http interface for querying and retrieving xml or json. or to use the php solr client library . here is a screenshot of a typical faceted search in drupal : for more information about faceted search look into the wiki of solr . more php projects which integrates solr: open source typo3- solr module magento enterprise - solr module . the open source integration is out dated. oxid - solr module . no open source integration available. 3.2. hathi trust the hathi trust project is a nice example that proves solr's ability to search big digital libraries. to quote directly from the article : "... the index for our one million book index is over 200 gigabytes ... so we expect to end up with a two terabyte index for 10 million books" other examples for libraries: vufind - aims to replace opac internet archive national library of australia 3.3. auto suggestions mainly, there are two approaches to implement auto-suggestions (also called auto-completion) with solr: via facets or via ngramfilterfactory . to push it to the extreme you can use a lucene index entirely in ram. this approach is used in a large music shop in germany. live examples for auto suggestions: kaufda.de 3.4. spatial search applications when mentioning spatial search, people have geographical based applications in mind. with solr, this ordinary use case is attainable . some examples for this are : city search - city guides yellow pages kaufda.de spatial search can be useful in many different ways : for bioinformatics, fingerprints search, facial search, etc. (getting the fingerprint of a document is important for duplicate detection). the simplest approach is implemented in jetwick to reduce duplicate tweets, but this yields a performance of o(n) where n is the number of queried terms. this is okay for 10 or less terms, but it can get even better at o(1)! the idea is to use a special hash set to get all similar documents. this technique is called local sensitive hashing . read this nice paper about 'near similarity search and plagiarism analysis' for more information. 3.5. duckduckgo duckduckgo is made with open source and its "zero click" information is done with the help of solr using the dismax query handler: the index for that feature contains 18m documents and has a size of ~12gb. for this case had to tune solr: " i have two requirements that differ a bit from most sites with respect to solr: i generally only show one result, with sometimes a couple below if you click on them. therefore, it was really important that the first result is what people expected. false positives are really bad in 0-click, so i needed a way to not show anything if a match wasn't too relevant. i got around these by a) tweaking dismax and schema and b) adding my own relevancy filter on top that would re-order and not show anything in various situations. " all the rest is done with tuned open source products. to quote gabriel again: "the main results are a hybrid of a lot of things, including external apis, e.g. bing, wolframalpha, yahoo, my own indexes and negative indexes (spam removal), etc. there are a bunch of different types of data i'm working with. " check out the other cool features such as privacy or bang searches . 3.6. clustering support with carrot2 carrot2 is one of the "contributed plugins" of solr. with carrot2 you can support clustering : " clustering is the assignment of a set of observations into subsets (called clusters) so that observations in the same cluster are similar in some sense. " see some research papers regarding clustering here . here is one visual example when applying clustering on the search "pannous" - our company : 3.7. near real time search solr isn't real time yet, but you can tune solr to the point where it becomes near real time, which means that the time ('real time latency') that a document takes to be searchable after it gets indexed is less than 60 seconds even if you need to update frequently. to make this work, you can setup two indices. one write-only index "w" for the indexer and one read-only index "r" for your application. index r refers to the same data directory of w, which has to be defined in the solrconfig.xml of r via: /pathto/indexw/data/ to make sure your users and the r index see the indexed documents of w, you have to trigger an empty commit every 60 seconds: wget -q http://localhost:port/solr/update?stream.body=%3ccommit/%3e -o /dev/null everytime such a commit is triggered a new searcher without any cache entries is created. this can harm performance for visitors hitting the empty cache directly after this commit, but you can fill the cache with static searches with the help of the newsearcher entry in your solrconfig.xml. additionally, the autowarmcount property needs to be tuned, which fills the cache with a newsearcher from old entries. also, take a look at the article 'scaling lucene and solr' , where experts explain in detail what to do with large indices (=> 'sharding') and what to do for high query volume (=> 'replicating'). 3.8. loggly = full text search in logs feeding log files into solr and searching them at near real-time shows that solr can handle massive amounts of data and queries the data quickly. i've setup a simple project where i'm doing similar things , but loggly has done a lot more to make the same task real-time and distributed. you'll need to keep the write index as small as possible otherwise commit time will increase too great. loggly creates a new solr index every 5 minutes and includes this when searching using the distributed capabilities of solr ! they are merging the cores to keep the number of indices small, but this is not as simple as it sounds. watch this video to get some details about their work. 3.9. solandra = solr + cassandra solandra combines solr and the distributed database cassandra , which was created by facebook for its inbox search and then open sourced. at the moment solandra is not intended for production use. there are still some bugs and the distributed limitations of solr apply to solandra too. tthe developers are working very hard to make solandra better. jetwick can now run via solandra just by changing the solrconfig.xml. solandra also has the advantages of being real-time (no optimize, no commit!) and distributed without any major setup involved. the same is true for solr cloud. 3.10. category browsing via facets solr provides facets , which make it easy to show the user some useful filter options like those shown in the "drupal integration" example. like i described earlier , it is even possible to browse through a deep category tree. the main advantage here is that the categories depend on the query. this way the user can further filter the search results with this category tree provided by you. here is an example where this feature is implemented for one of the biggest second hand stores in germany. a click on 'schauspieler' shows its sub-items: other shops: game-change 3.11. jetwick - open twitter search you may have noticed that twitter is using lucene under the hood . twitter has a very extreme use case: over 1,000 tweets per second, over 12,000 queries per second, but the real-time latency is under 10 seconds! however, the relevancy at that volume is often not that good in my opinion. twitter search often contains a lot of duplicates and noise. reducing this was one reason i created jetwick in my spare time. i'm mentioning jetwick here because it makes extreme use of facets which provides all the filters to the user. facets are used for the rss-alike feature (saved searches), the various filters like language and retweet-count on the left, and to get trending terms and links on the right: to make jetwick more scalable i'll need to decide which of the following distribution options to choose: use solr cloud with zookeeper use solandra move from solr to elasticsearch which is also based on apache lucene other examples with a lot of facets: cnet reviews - product reviews. electronics reviews, computer reviews & more. shopper.com - compare prices and shop for computers, cell phones, digital cameras & more. zappos - shoes and clothing. manta.com - find companies. connect with customers. 3.12. plaxo - online address management plaxo.com , which is now owned by comcast, hosts web addresses for more than 40 million people and offers smart search through the addresses - with the help of solr. plaxo is trying to get the latest 'social' information of your contacts through blog posts, tweets, etc. plaxo also tries to reduce duplicates . 3.13. replace fast or google search several users report that they have migrated from a commercial search solution like fast or google search appliance (gsa) to solr (or lucene). the reasons for that migration are different: fast drops linux support and google can make integration problems. the main reason for me is that solr isn't a black box —you can tweak the source code, maintain old versions and fix your bugs more quickly! 3.14. search application prototyping with the help of the already integrated velocity plugin and the data import handler it is possible to create an application prototype for your search within a few hours. the next version of solr makes the use of velocity easier. the gui is available via http://localhost:port/solr/browse if you are a ruby on rails user, you can take a look into flare. to learn more about search application prototyping, check out this video introduction and take a look at these slides. 3.15. solr as a whitelist imagine you are the new google and you have a lot of different types of data to display e.g. 'news', 'video', 'music', 'maps', 'shopping' and much more. some of those types can only be retrieved from some legacy systems and you only want to show the most appropriated types based on your business logic . e.g. a query which contains 'new york' should result in the selection of results from 'maps', but 'new yorker' should prefer results from the 'shopping' type. with solr you can set up such a whitelist-index that will help to decide which type is more important for the search query. for example if you get more or more relevant results for the 'shopping' type then you should prefer results from this type. without the whitelist-index - i.e. having all data in separate indices or systems, would make it nearly impossible to compare the relevancy. the whitelist-index can be used as illustrated in the next steps. 1. query the whitelist-index, 2. decide which data types to display, 3. query the sub-systems and 4. display results from the selected types only. 3.16. future solr is also useful for scientific applications, such as a dna search systems. i believe solr can also be used for completely different alphabets so that you can query nucleotide sequences - instead of words - to get the matching genes and determine which organism the sequence occurs in, something similar to blast . another idea you could harness would be to build a very personalized search. every user can drag and drop their websites of choice and query them afterwards. for example, often i only need stackoverflow, some wikis and some mailing lists with the expected results, but normal web search engines (google, bing, etc.) give me results that are too cluttered. my final idea for a future solr-based app could be a lucene/solr implementation of desktop search. solr's facets would be especially handy to quickly filter different sources (files, folders, bookmarks, man pages, ...). it would be a great way to wade through those extra messy desktops. 4. summary the next time you think about a problem, think about solr! even if you don't know java and even if you know nothing about search: solr should be in your toolbox. solr doesn't only offer professional full text search, it could also add valuable features to your application. some of them i covered in this article, but i'm sure there are still some exciting possibilities waiting for you!
January 25, 2011
by Peter Karussell
· 147,293 Views
article thumbnail
Maven Profile Best Practices
Maven profiles, like chainsaws, are a valuable tool, with whose power you can easily get carried away, wielding them upon problems to which they are unsuited. Whilst you're unlikely to sever a leg misusing Maven profiles, I thought it worthwhile to share some suggestions about when and when not to use them. These three best practices are all born from real-world mishaps: The build must pass when no profile has been activated Never use Use profiles to manage build-time variables, not run-time variables and not (with rare exceptions) alternative versions of your artifact I'll expand upon these recommendations in a moment. First, though, let's have a brief round-up of what Maven profiles are and do. Maven Profiles 101 A Maven profile is a sub-set of POM declarations that you can activate or disactivate according to some condition. When activated, they override the definitions in the corresponding standard tags of the POM. One way to activate a profile is to simply launch Maven with a -P flag followed by the desired profile name(s), but they can also be activated automatically according to a range of contextual conditions: JDK version, OS name and version, presence or absence of a specific file or property. The standard example is when you want certain declarations to take effect automatically under Windows and others under Linux. Almost all the tags that can be placed directly in a POM can also be enclosed within a tag. The easiest place to read up further about the basics is the Build Profiles chapter of Sonatype's Maven book. It's freely available, readable, and explains the motivation behind profiles: making the build portable across different environments. The build must pass when no profile has been activated (Thanks to for this observation.) Why? Good practice is to minimise the effort required to make a successful build. This isn't hard to achieve with Maven, and there's no excuse for a simple mvn clean package not to work. A maintainer coming to the project will not immediately know that profile wibblewibble has to be activated for the build to succeed. Don't make her waste time finding it out. How to achieve it It can be achieved simply by providing sensible defaults in the main POM sections, which will be overridden if a profile is activated. Never use Why not? This flag activates the profile if no other profile is activated. Consequently, it will fail to activate the profile if any other profile is activated. This seems like a simple rule which would be hard to misunderstand, but in fact it's surprisingly easy to be fooled by its behaviour. When you run a multimodule build, the activeByDefault flag will fail to operate when any profile is activated, even if the profile is not defined in the module where the activeByDefault flag occurs. (So if you've got a default profile in your persistence module, and a skinny war profile in your web module... when you build the whole project, activating the skinny war profile because you don't want JARs duplicated between WAR and EAR, you'll find your persistence layer is missing something.) activeByDefault automates profile activation, which is a good thing; activates implicitly, which is less good; and has unexpected behaviour, which is thoroughly bad. By all means activate your profiles automatically, but do it explicitly and automatically, with a clearly defined rule. How to avoid it There's another, less documented way to achieve what aims to achieve. You can activate a profile in the absence of some property: !foo.bar This will activate the profile "nofoobar" whenever the property foo.bar is not defined. Define that same property in some other profile: nofoobar will automatically become active whenever the other is not. This is admittedly more verbose than , but it's more powerful and, most importantly, surprise-free. Use profiles to adapt to build-time context, not run-time context, and not (with rare exceptions) to produce alternative versions of your artifact Profiles, in a nutshell, allow you to have multiple builds with a single POM. You can use this ability in two ways: Adapt the build to variable circumstances (developer's machine or CI server; with or without integration tests) whilst still producing the same final artifact, or Produce variant artifacts. We can further divide the second option into: structural variants, where the executable code in the variants is different, and variants which vary only in the value taken by some variable (such as a database connection parameter). If you need to vary the value of some variable at run-time, profiles are typically not the best way to achieve this. Producing structural variants is a rarer requirement -- it can happen if you need to target multiple platforms, such as JDK 1.4 and JDK 1.5 -- but it, too, is not recommended by the Maven people, and profiles are not the best way of achieving it. The most common case where profiles seem like a good solution is when you need different database connection parameters for development, test and production environments. It is tempting to meet this requirement by combining profiles with Maven's resource filtering capability to set variables in the deliverable artifact's configuration files (e.g. Spring context). This is a bad idea. Why? It's indirect: the point at which a variable's value is determined is far upstream from the point at which it takes effect. It makes work for the software's maintainers, who will need to retrace the chain of events in reverse It's error prone: when there are multiple variants of the same artifact floating around, it's easy to generate or use the wrong one by accident. You can only generate one of the variants per build, since the profiles are mutually exclusive. Therefore you will not be able to use the Maven release plugin if you need release versions of each variant (which you typically will). It's against Maven convention, which is to produce a single artifact per project (plus secondary artifacts such as documentation). It slows down feedback: changing the variable's value requires a rebuild. If you configured at run-time you would only need to restart the application (and perhaps not even that). One should always aim for rapid feedback. Profiles are there to help you ensure your project will build in a variety of environments: a Windows developer's machine and a CI server, for instance. They weren't intended to help you build variant artifacts from the same project, nor to inject run-time configuration into your project. How to achieve it If you need to get variable runtime configuration into your project, there are alternatives: Use JNDI for your database connections. Your project only contains the resource name of the datasource, which never changes. You configure the appropriate database parameters in the JNDI resource on the server. Use system properties: Spring, for example, will pick these up when attempting to resolve variables in its configuration. Define a standard mechanism for reading values from a configuration file that resides outside the project. For example, you could specify the path to a properties file in a system property. Structural variants are harder to achieve, and I confess I have no first-hand experience with them. I recommend you read this explanation of how to do them and why they're a bad idea, and if you still want to do them, take the option of multiple JAR plugin or assembly plugin executions, rather than profiles. At least that way, you'll be able to use the release plugin to generate all your artifacts in one build, rather than a single one at a time. Further reading Profiles chapter from the Sonatype Maven book. Deploying to multiple environments (prod, test, dev): Stackoverflow.com discussion; see the first and top-rated answer. Short of creating a specific project for the run-time configuration, you could simply use run-time parameters such as system properties. Creating multiple artifacts from one project: How to Create Two JARs from One Project (…and why you shouldn’t) by Tim O'Brien of Sonatype (the Maven people) Blog post explaining the same technique Maven best practices (not specifically about profiles): http://mindthegab.com/2010/10/21/boost-your-maven-build-with-best-practices/ http://blog.tallan.com/2010/09/16/maven-best-practices/ This article is a completely reworked version of a post from my blog.
November 27, 2010
by Andrew Spencer
· 141,008 Views · 4 Likes
article thumbnail
Naming Conventions for Parameterized Types
Parameterized types - the <> expressions that can be used in Java as of JDK 5 are not just for collections. I find myself frequently using them in APIs I design. They really do let you write things which are more generic in the non-Java sense of the word - and the result is more reusable code, which means less code overall, which means fewer bugs and things to test. The verbosity, and some of the weirdness of type-erasure are less than ideal, but used right, the benefits are worth the complexity. The standard (and somewhere recommended) naming convention for parameterized types is to use a single-letter name. That works fine in signatures that have only one such type. But in practice, single-letter names make code less self-describing, and if you're defining a class with more than one parameterized type, it can be confusing and hard to read. People other than me will have to call, understand and maintain my code - the more self-describing I can make it, the better. So I am looking for a naming convention that makes it obvious that something is a parameterized type, but allows for descriptive names. I am wondering if anybody else has run into this problem, and if there is any emerging consensus on naming generics. Do you work on a project that uses generics a lot? If so, what do you do? Here's an example. At the moment, I'm writing a generic (in both senses) class which simply limits the number of threads which can access some resource. It's basically a wrapper around a Semaphore which uses a Runnable-like object to ensure that the Semaphore is accessed correctly, and does some non-blocking statistic gathering about thread contention. So to access the scarce resource, you pass in a ResourceAccessor: public interface ResourceAccessor { public Result run (ProtectedResource resource, Argument argument); } The problem is that, when somebody looks at this interface, they will instantly get the idea that there are really classes they need to go find, which are called ProtectedResource, Argument and Result - and of course, no such classes exist - these are just names for generic types. The standard-naming-convention is worse: public interface ResourceAccessor { public S run (T resource, R argument); } Here, nobody could possibly figure out what on earth this class is for without extensive documentation - this is a really horrible idea. So I've concluded that the standard recommendations for generic type names are simply wrong for any non-trivial usage (I.e. Collection is fine, since there is one type and Collections are well-understood). You simply can't do this on a non-collection code structure you have invented, or people will just be confused and not use it. The best suggestion I've heard thus far is using $ as a prefix: public interface ResourceAccessor <$ProtectedResource, $Argument, $Result> { public $Result run ($ProtectedResource resource, $Argument argument); } I don't find this pretty, but I don't have any better ideas, and at least it makes it crystal-clear that there is something different about these names. Any thoughts? What do you do in this situation?
September 20, 2010
by Tim Boudreau
· 17,830 Views
article thumbnail
Throwing Undeclared Checked Exceptions
Sometimes checked exceptions can be a problem. For instance, recently I tried to implement some common logic to retry failing network operations and it resulted in a kind of command pattern on which, as usual, the execute() method throws java.lang.Exception. That complicated the caller code which has to catch and handle java.lang.Exception instead of the more specific exceptions... I knew that checked exceptions are enforced by the compiler, while in the virtual machine there is nothing preventing a checked exception to be thrown by a method not declaring it, so I started to check on internet how to implement this. I found two posts on Anders Noras's blog (#1 #2) on how to perform this magic. Method #1: the sun.misc.Unsafe class import java.lang.reflect.Field; import sun.misc.Unsafe; public class UnsafeSample { public void methodWithNoDeclaredExceptions( ) { Unsafe unsafe = getUnsafe(); unsafe.throwException( new Exception( "this should be checked" ) ); } private Unsafe getUnsafe() { try { Field field = Unsafe.class.getDeclaredField("theUnsafe"); field.setAccessible(true); return (Unsafe) field.get(null); } catch(Exception e) { throw new RuntimeException(e); } } public static void main( String[] args ) { new UnsafeSample().methodWithNoDeclaredExceptions(); } } This makes use of internal Sun JRE libraries implementation classes. It could not work if you use a non Sun VM. And in fact it doesn't if you use GCJ (The GNU compiler for Java). The getUnsafe() method exposed above does some tricks to access a private field in the Unsafe class, because Unsafe.getUnsafe() can only be called by classes loaded by the bootstrap ClassLoader. See also the article Avoiding Checked Exceptions by Don Schwarz. Method #2: the Thread.stop(Exception) public class ThreadStopExample { @SuppressWarnings("deprecation") public void methodWithNoDeclaredExceptions( ) { Thread.currentThread().stop(new Exception( "this should be checked" )); } public static void main( String[] args ) { new ThreadStopExample().methodWithNoDeclaredExceptions(); } } This uses a deprecated method, but works. No portability issue, until the Java specification guys decide to remove the method. It could have some side effects on the current thread as we are calling stop(). I'm not sure. Method #3: using Class.newInstance() Look at the signature of java.lang.Class.newInstance() and compare it to Constructor.newInstance() public final class Class ... { public T newInstance() throws InstantiationException, IllegalAccessException } public final class Constructor ... { public T newInstance(Object ... initargs) throws InstantiationException, IllegalAccessException, IllegalArgumentException, InvocationTargetException } You see it? no InvocationTargetException! If you call SomeObject.class.newInstance() and the constructor throws an exception, the exception doesn't get wrapped into the InvocationTargetException (that is a checked exception). So you can write an utility class like this, to throw checked exceptions without needing to declare them on the method signature. public class Exceptions { private static Throwable throwable; private Exceptions() throws Throwable { throw throwable; } public static synchronized void spit(Throwable throwable) { Exceptions.throwable = throwable; try { Exceptions.class.newInstance(); } catch(InstantiationException e) { } catch(IllegalAccessException e) { } finally { Exceptions.throwable = null; } } } public class TestExceptionSpit { public static void main(String[] args) { Exceptions.spit(new Exception( "this should be checked" )); } } Internally the Class.newInstance() uses the sun.misc.Unsafe class, but in this case this technique is fully portable because you are not using any deprecated or internal method. In fact it works also with GCJ JVM. I tried to remove the synchronization stuff and the static field using an inner class, but it seems that the compiler does some strange trick translating the empty constructor in something else preventing Class.newInstace() to be used on that inner class. The behavior of the Class.newInstance() is also documented: "Note that this method propagates any exception thrown by the nullary constructor, including a checked exception. Use of this method effectively bypasses the compile-time exception checking that would otherwise be performed by the compiler." So your code is fully safe and compliant to the rules :) Method #4: the sun.corba.Bridge import java.rmi.RemoteException; public class Bridge { public void methodWithNoDeclaredExceptions( ) { sun.corba.Bridge.get().throwException(new RemoteException("bang!")); } public static void main( String[] args ) { new Bridge().methodWithNoDeclaredExceptions(); } } This is more or less the same as using the Unsafe.class. The difference is that in this case you don't need to do the reflection stuff to access the private field "theUnsafe", because the Bridge class is doing that for you. Still using an internal JRE class with same portability issues. Method #5: Generics The following example takes advantage of the fact that the compiler does not type check generics... import java.rmi.RemoteException; class Thrower { public static void spit(final Throwable exception) { class EvilThrower { @SuppressWarnings("unchecked") private void sneakyThrow(Throwable exception) throws T { throw (T) exception; } } new EvilThrower().sneakyThrow(exception); } } public class ThrowerSample { public static void main( String[] args ) { Thrower.spit(new RemoteException("go unchecked!")); } } Credits to "Harald" that posted a comment on Johannes Brodwall's blog. I personally think this last one is the best solution: it uses a feature of the compiler against itself. Conclusions I think that having checked exception in Java is better than not having it. I already expressed why I am in favor of checked exceptions here. It's a design decision, you can choose to make your exceptions checked or unchecked, if you want to force your client to handle them or not; you can't do that on .NET, where checked exceptions simply do not exist. Sometimes you have (or you have to write) methods throwing java.lang.Exception, and you get into the trap. So you may like to know that there is a dirty escape, and you can decide to use it or not... we saw that Sun is throwing undeclared checked exceptions in Class.newInstace(), ask yourself: if this is good for the JRE code, could it be good also for yours? Usually you can wrap checked exception into RuntimeExceptions but this doesn't simplify the client code, because the caller in case of needing has to catch the RuntimeException, unwrap the cause and deal with it. Maybe a new Java keyword to throw checked exception without requiring the caller to handle them could help: I recommend reading post on Ricky Clarkson's about checked exceptions. Finally I come to the decision to not use those tricks in my object doing the retry logic, and keep the messy catch logic on the caller code. In case of needing I will evaluate to use a Dynamic Proxy doing the retry logic and keeping its behavior transparent to the client. To those who wants unchecked exceptions in Java... well, there is the way to have it: the example with Generics is a clean way to have it. Use it if you want, at your own risk. Personally I would choose to use libraries with checked exceptions... Other related articles Friday Free Stuff by Chris Nokleberg, uses bytecode manipulation. Don't Try This at Home by Bob Lee, exposes some methods also covered above. From: http://en.newinstance.it/2008/11/17/throwing-undeclared-checked-exceptions/
September 15, 2010
by Luigi Viggiano
· 26,232 Views
article thumbnail
How to resize an ExtJS Panel, Grid, Component on Window Resize without using Ext.Viewport
This post will walk through how to resize an ExtJS Panel, Grid, Component on Window Resize without using Ext.Viewport. Problem: You have a legacy page and you want to change an html grid for an ExtJS DataGrid, because it has so many cool features. Or you have a page with some design and you are going to use only one ExtJS Component. In both cases, you also want to render your ExtJS Component to a specific DIV. Also, you want you component to be resized in case you resize the browser window. How can you do that if resize a single component in an HTML page it is not the default behavior of an ExtJS Component (except if you use Ext.Viewport)? Solution: Condor (from ExtJS Community Support Team) developed a plugin that can do that for you. I had to spend some time to understand how the plugin works, and I finally got it working as I wanted. Well, I recommend you to spend some time reading this thread: http://www.sencha.com/forum/showthread.php?28318 (if you have any issues or questions, please publish it on the thread, so other members can give you the support you need). Requirements to make the plugin work: Your have to apply the following style to the DIV (the width is up to you, the other styles are mandatory, otherwise it will not work): If you have any border around your ExtJS component, you have to set a HEIGHT. And you will also have to set a height to your ExtJS component. In this case, autoHeight will not work. If you DO NOT have any border or other design on the ExtJS component side, you do not need to set height and you can use autoHeight. In my case, I put a border on the external DIV, so I have to set Height: HTML code (all DIVs): And you need to add the plugin to the component (In this case, I’m using an ExtJS DataGrid): var grid = new Ext.grid.GridPanel({ store: store, columns: [ {header: 'Company', width: 160, sortable: true, dataIndex: 'company'}, {header: 'Price', width: 75, sortable: true, renderer: 'usMoney', dataIndex: 'price'}, {header: 'Change', width: 75, sortable: true, renderer: change, dataIndex: 'change'}, {header: '% Change', width: 75, sortable: true, renderer: pctChange, dataIndex: 'pctChange'}, {header: 'Last Updated', width: 85, sortable: true, renderer: Ext.util.Format.dateRenderer('m/d/Y'), dataIndex: 'lastChange'} ], stripeRows: true, autoExpandColumn: 'company', height: 490, autoWidth:true, title: 'Array Grid', // config options for stateful behavior stateful: true, stateId: 'grid' ,viewConfig:{forceFit:true} ,renderTo: 'reportTabContent' // render the grid to the specified div in the page ,plugins: [new Ext.ux.FitToParent("reportTabContent")] }); And done! Now you can resize the browser and the component will resize itself! I tested it on Firefox, Chrome and IE6. You can download my sample project from my GitHub: http://github.com/loiane/extjs-fit-to-parent PS.: If you want to use the full browser window, use a Viewport. Happy coding!
August 24, 2010
by Loiane Groner
· 48,810 Views
article thumbnail
Getting started with Nexus Maven Repo Manager
This tutorial outlines steps required to install Nexus (Maven Repository Manager) under Tomcat, or another webapp container. It shows you practical configuration and includes code snippets that go in your pom.xml and settings.xml in order to read and publish artifacts to your Nexus server. Step 1: Download Download Nexus from here (at the time of writing, latest is 1.6.0) Step 2: Install Copy the war to TOMCAT_HOME/webapps/nexus.war Though not required, it is a generally good idea to restart tomcat after installing a new war /etc/init.d/tomcat restart /etc/init.d/tomcat restart Step 3: Configure security a) Change default admin password: The default admin username/password is admin/admin123. Login as admin and change the password to a secure password. Login -> [admin, admin123] -> Left Menu -> Security -> Change Password -> click “Change Password” b) Anonymous Access: By default Nexus is open to the public. If you want to secure access to nexus, disable ‘Nexus anonymous user’ Admin -> Left Menu -> Users -> ‘Nexus anonymous user’ -> Status=Disabled c) Deployment user: Change password for deployment user Admin -> Left menu -> Users -> Deployment user -> Change email address Admin -> Left menu -> Users -> Right click on ‘Deployment user’ in the user list -> Set Password -> click ‘Set password’ to finish Step 4: Set SMTP server It is a good idea to configure SMTP server, so that you can receive emails from Nexus. Admin login -> Left menu -> Administration -> Server ->SMTP Settings -> (host localhost, port 25, no login, no password mostly works on a linux machine) Step 5: Change Base Url If you are running Nexus behind Apache using mod_jk or mod_proxy, change your base url here. Admin login -> Left menu -> Administration -> Server -> Application Server Settings -> Base url Step 6: Add a task to periodically remove old snapshots If you or your CI server publishes snapshots to Nexus several times a day, then you should consider adding a task to delete duplicate/old snapshots for the same GAV (group, artifact, version). If you don’t do this, you will notice that the Nexus disk usage will increase with time. Admin login -> Left menu -> Administration -> Scheduled tasks -> Add… -> name=”Remove old snapshots”, Repository/Group=Snapshots (Repo), Minimum Snapshot Count=1, Snapshot Retention(days)=3, Recurrence=Daily, Recurring time=2:00 -> click ‘Save’ Step 7: Using Nexus: reading and publishing artifacts If you want to deploy your artifacts to your Nexus, you need to configure 2 files: pom.xml and settings.xml a) pom.xml – for each project which wishes to publish to Nexus, add your repo to the pom.xml vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/releases vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/snapshots vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/releases vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/snapshots vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true b) settings.xml – If you have disabled anonymous access to Nexus, add the deployment password to your ~/.m2/repository/settings.xml file vineetmanohar-nexus deployment password_goes_here From http://www.vineetmanohar.com/2010/06/getting-started-with-nexus-maven-repo-manager
June 7, 2010
by Vineet Manohar
· 105,113 Views · 3 Likes
article thumbnail
FlexMonkey 4 and FlexMonkium for Selenium
FlexMonkey is a free and open source Adobe AIR application used for testing Flex and AIR based applications. It can record, playback, and verify Flex UI interactions. FlexMonkey also generates ActionScript-based testing scripts that you can easily include within a continuous integration environment. Gorilla Logic is the company that builds FlexMonkey, and its CEO, Stuart Stern, recently spoke with DZone about their launch of FlexMonkey 4, which supports all of the new Spark components in Flex 4. For more info on FlexMonkey, see our interview with Stuart Stern at Adobe Max 2009. DZone: First thing's first. What's new in FlexMonkey 4? Stuart Stern: Before we talk about the updates to FlexMonkey, let me give you a bit of background for those who have not used any of the previous versions. We (Gorilla Logic) built and open sourced the first version of FlexMonkey in late 2008 because we needed a serious Flex testing solution for our enterprise customers. Basically, FlexMonkey allows developers and QA people to create comprehensive tests for their Flex applications by easily recording real interactions with the user interface, and by letting the test creator add verification checks on both data and visual snapshots of the UI. Once the interactions have been recorded the test can be played back through the FlexMonkey console or through generated test code in Fluint / FlexUnit. The generated code can be extended to create complex, data-driven test scenarios, and can be easily run within build and continuous integration environments. In our software consulting engagements, we have found that FlexMonkey reduces the overall numbers of tests that developers need to create, since driving testing from the user interface can exercise the entire application stack, top-to-bottom and even front-to-back. Let’s be clear though, api-level testing and tools like FlexUnit are still an essential part of Flex development, especially in testing non ui components. Where FlexMonkey is a better fit for testing is around visual components, which are difficult, if not impossible, to test as a ‘unit.’ On our typical applications, we tend to end up with about 80% of our developer created tests constructed through FlexMonkey, with the other 20% being created as more traditional unit tests. As far as FlexMonkey 4, the goals were pretty simple; the community has been beating down our door for Spark Component (Flex 4) support. So, we’ve added full support for the new component library recently released by Adobe. This is key for enterprise Flex development projects that have come to depend on FlexMonkey for regression and QA testing, and that are ready to move to Flex 4. We've also simplified the setup for FlexMonkey 4, so it's easier for new users to get up and running quickly. DZone: What were some of the difficulties in implementing support for all of Flex 4's Spark components? Stuart: From a FlexMonkey perspective, there is no difference between Spark and Halo components. However, one of the things that makes FlexMonkey so powerful is that it records "semantic" events such as "open combobox" rather than "click at this screen coordinate". So FlexMonkey needs to "understand" every Flex component, and we had to tell it some new things about the new Spark components and their events. . DZone: Are there any trends your seeing in how developers are using FlexMonkey in their UI design workflow? Stuart: FlexMonkey was initially envisioned as a tool for developers. Because developers test code that is still under development, it is important for a test automation tool to be able to express tests in a largely logical fashion. Tests that are too tied to the precise look of a screen at a particular point in time are two brittle for use by developers. FlexMonkey tests are typically robust across application skinning, since tests can be written independent of the exact positions or styling of the components on the screen, and can pinpoint specific functionality. In this way developers can automate testing of portions of an application even before the UI design is fully finalized. Although we designed it for developer testing, it's ability to record tests automatically, add verification logic by pointing and clicking, and do fuzzy bitmap comparisons on select portions of the screen, make FlexMonkey highly effective for QA testing purposes as well. Additionally, when developers and testers use the same tools, they can share some of the same tests, with QA using developer tests as a starting point, and developers incorporating some QA tests into continuous integration builds. DZone: Tell me about the next tool you'll be focusing on: FlexMonkium. Stuart: FlexMonkium is a plugin for Selenium IDE and Selenium RC. It adds FlexMonkey recording and playback capability to Selenium so you can create tests for applications that mix HTML and Flex. We recently completed development and are now doing final testing and documentation. We expect it make it publicly available any day now. FlexMonkium makes all of FlexMonkey's functionality available within the Selenium IDE, and generates JUnit-based tests that can be run with Selenium RC. DZone: Are there any interesting or exciting things you see down the road for the Flash platform ecosystem? How do you think the platform will fare against emerging UI design technologies like HTML5, CSS3, etc.? Stuart: The recent attacks on the Flash platform by Apple have certainly put the ‘HTML 5 vs. Flash’ battle on everyone’s radar. At Gorilla, we build both native browser applications (HTML 5, etc.) and Flex applications -- and even native iPhone applications -- for our customers. There are pros and cons to each and situations that definitively call for one versus another. Having said that, we are a consulting company that builds serious enterprise software. We embrace Flex because it enables us to do things we cannot do otherwise, and do them quickly. On any given project, we don't ask if we should use Flex, we ask if there is any reason why we can't.
June 4, 2010
by Mitch Pronschinske
· 14,718 Views
article thumbnail
JMS Clustering by Example
It's amazing how the JBoss Team put together an easy way to do JMS Clustering, out of the box!!. I'll start with an easy example, creating a Queue named "MyClusteredQueue". In this example I'm using JBoss AS 5.1. and two computers connected on the same network, with these IP's: - Computer A: 192.168.0.143 - Computer B: 192.168.0.210 So, here are the steps: 1) Install the JBoss on both computers. We are going to use the "all" configuration for both computers. 2) We create our Queue on both servers. Go to $JBOSS_HOME/server/all/deploy/messaging/ and edit the destinations-service.xml file. Add the MyClusteredQueue before the last server tag. It looks like this: jboss.messaging:service=ServerPeer jboss.messaging:service=PostOffice true This is how you add a Queue to the JBoss, and the people how are familiar with this, the only new thing is to add the attribute "Clustered". This step must be set on both computers. At the end of the article you can find the files. 3) Write the MDB to consume the messages, and deploy it on the two computers. (I'm using an EJB 3 - MDB style). import java.net.InetAddress; import javax.ejb.ActivationConfigProperty; import javax.ejb.MessageDriven; import javax.jms.Message; import javax.jms.MessageListener; import javax.jms.ObjectMessage; import org.apache.log4j.Logger; /** * @author felipeg * */ @MessageDriven(activationConfig = { @ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue"), @ActivationConfigProperty(propertyName="destination", propertyValue="queue/MyClusteredQueue") }) public class JMSClusterClientHandler implements MessageListener { Logger log = Logger.getLogger(JMSClusterClientHandler.class); @Override public void onMessage(Message message) { try{ if (message instanceof ObjectMessage) { InetAddress addr = InetAddress.getLocalHost(); log.info("########## Processing Host: " + addr.getHostName() + " ##########" ); ObjectMessage objMessage = (ObjectMessage) message; Object obj = objMessage.getObject(); log.info("Object received:" + obj.toString()); } } catch (Exception e) { e.printStackTrace(); } } } 4) Start the jboss with the following options: Computer A: $ cd $JBOSS_HOME/bin $ ./run.sh -c all -b 192.168.0.143 -Djboss.messaging.ServerPeerID=1 Computer B: $ cd $JBOSS_HOME/bin $ ./run.sh -c all -b 192.168.0.210 -Djboss.messaging.ServerPeerID=2 It is necesary to give an ID to each server and this is accomplished with this directive: -Djboss.messaging.ServerPeerID When you start the jboss on computer A, you should see the logs (server.log) telling you that there is one node ready and listening, and once you start the jboss on computer B, on the log will appear the two nodes, the two IP's ready to consume messages. 5) Now it's time to send a Message to the Queue. To accomplish this it's necessary to change the connection factory to "ClusteredConnectionFactory" (JMSDispatcher.java - See the code below). Also on the jndi.properties (if you are using the default InitialContext) file it's necessary to add the two computers ip's separated by comma to the java.naming.provider.url property. (In my case a create a Properties variable and I set all the necessary properties, JMSDispatcher.java - see the code below). java.naming.provider.url=192.168.0.143:1099,192.168.0.210:1099 The client that I wrote is a web application, that consist in one index.jsp page, which contains a form that prompts you for the name of the queue, the type of messaging (Queue or Topic), the server ip and port, how many times it will send the message and the actual message to be sent; also the web application has a Servlet (JMSClusteredClient.java - see code below) that receives the postback and helper class (JMSDispatcher.java - see code below) that sends the message to the jboss servers. You can to deploy it in any computer. In my case I deployed it on the Computer A. And you can access it through this URL: http://192.168.0.143:8080/JMSWeb/ (just modify the IP where the client war was deployed). If you notice (on the index.jsp - code below) I've already put some default values that reflects the name of the Queue, and the IP's of my two computers. Now, If you increment the number of times that the message will be sent (maybe a 10) and fill out the message box, and click "Send" you should see on the two servers some of the messages being consumed by the MDB. Here are the Files to create the client: index.jsp JMS Clustered - Test Client Server: QueueTopic Times:Message: Servlet: JMSClusteredClient.java public class JMSClusteredClient extends HttpServlet { private static final long serialVersionUID = 1L; /** * @see HttpServlet#service(HttpServletRequest request, HttpServletResponse response) */ protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { PrintWriter out = response.getWriter(); String topicqueue = request.getParameter("topicqueue"); String message = request.getParameter("message"); String server = request.getParameter("server"); String messageType = request.getParameter("messageType"); String times = request.getParameter("times"); int intTimes = Integer.parseInt(times); JMSDispatcher dispatcher = new JMSDispatcher(); dispatcher.setTopicQueueName(topicqueue); dispatcher.setServer(server); dispatcher.setMessageType(messageType); try { for(int count =1; count <= intTimes;count++){ dispatcher.sendMessage( count + " of " + times + " " + message); } out.println("Message [" + message + "] sent successfully to [" + topic + "] to the [" + server + "] server " + times + " times."); } catch (JMSException e) { e.printStackTrace(); out.println("Error:" + e.getMessage()); } catch (NamingException e) { out.println("Error:" + e.getMessage()); e.printStackTrace(); } finally{ out.close(); } } } A utility to send the messages: JMSDispatcher.java public class JMSDispatcher { /** * */ private static final long serialVersionUID = 7105145023422143880L; private static Logger log = Logger.getLogger(JMSDispatcher.class); private final String CONNECTION_FACTORY_CLUSTERED = "ClusteredConnectionFactory"; private final String CONNECTION_FACTORY = "ConnectionFactory"; private final String TOPIC = "TOPIC"; private final String QUEUE = "QUEUE"; private String topicQueueName; private String server; private String messageType; public void setTopicQueueName(String value){ this.topicQueueName = value; } public void setServer(String value){ this.server = value; } public void setMessageType(String value){ this.messageType = value; } public void sendMessage(Object objectMessage) throws JMSException, NamingException{ log.debug("##### Setting up a Queue/Topic Message: #####"); if (TOPIC.equals(messageType)){ sendTopicMessage(objectMessage); } else if (QUEUE.equals(messageType)){ sendQueueMessage(objectMessage); } log.debug("##### Publishing Message: Done #####"); } private void sendQueueMessage(Object objectMessage) throws JMSException, NamingException{ try{ InitialContext initialContext = getInitialContext(); QueueConnectionFactory qcf = (QueueConnectionFactory) initialContext.lookup(CONNECTION_FACTORY_CLUSTERED); QueueConnection queueConn = qcf.createQueueConnection(); Queue queue = (Queue) initialContext.lookup(topicQueueName); QueueSession queueSession = queueConn.createQueueSession(false, Session.AUTO_ACKNOWLEDGE); queueConn.start(); QueueSender send = queueSession.createSender(queue); ObjectMessage om = queueSession.createObjectMessage((Serializable)objectMessage); setMessageProperties(om); log.debug("##### Publishing Message to a Queue: " + queueName + "#####"); send.send(om); send.close(); queueConn.stop(); queueSession.close(); queueConn.close(); }catch(MessageFormatException ex){ log.error("##### The MESSAGE is not Serializable ####"); throw ex; }catch(MessageNotWriteableException ex){ log.error("##### The MESSAGE is not Readable ####"); throw ex; }catch(JMSException ex){ log.error("##### JMS provider fails to set the object due to some internal error. ####"); throw ex; } } private void sendTopicMessage(Object objectMessage) throws JMSException, NamingException{ try{ InitialContext initialContext = getInitialContext(); TopicConnectionFactory tcf = (TopicConnectionFactory)initialContext.lookup(CONNECTION_FACTORY_CLUSTERED); TopicConnection topicConn = tcf.createTopicConnection(); Topic topic = (Topic) initialContext.lookup(topicQueueName); TopicSession topicSession = topicConn.createTopicSession(false,TopicSession.AUTO_ACKNOWLEDGE); topicConn.start(); TopicPublisher send = topicSession.createPublisher(topic); ObjectMessage om = topicSession.createObjectMessage(); om.setObject((Serializable)objectMessage); setMessageProperties(om); log.debug("##### Publishing Message to a Topic: " + topicName + "#####"); send.publish(om); send.close(); topicConn.stop(); topicSession.close(); topicConn.close(); }catch(MessageFormatException ex){ log.error("##### The MESSAGE is not Serializable ####"); throw ex; }catch(MessageNotWriteableException ex){ log.error("##### The MESSAGE is not Readable ####"); throw ex; }catch(JMSException ex){ log.error("##### JMS provider fails to set the object due to some internal error. ####"); throw ex; } } private InitialContext getInitialContext() throws NamingException{ Properties jboss = new Properties(); jboss.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory"); jboss.put("java.naming.factory.url.pkgs", "org.jboss.naming:org.jnp.interfaces"); jboss.put("java.naming.provider.url", server); return new InitialContext(jboss); } } And the web.xml JMSWeb index.jsp JMSClusteredClient JMSClusteredClient com.blogspot.felipeg48.jms.web.JMSClusteredClient JMSClusteredClient /JMSClusteredClient Happy Clustering!!
May 26, 2010
by Felipe Gutierrez
· 16,731 Views
article thumbnail
Running Hazelcast on a 100 Node Amazon EC2 Cluster
The purpose of this article is to give you the details of our 100 node cluster demo. This demo is recorded and you can watch the 5 minute screencast Hazelcast is an open source clustering and highly scalable data distribution platform for Java. JVMs that are running Hazelcast will dynamically cluster and allow you to easily share and partition your application data across the cluster. Hazelcast is a peer-to-peer solution (there is no master node, every node is a peer) so there is no single point of failure. Communication among cluster members is always TCP/IP with Java NIO beauty. The default configuration comes with 1 backup so if a node fails, no data will be lost (you can specify the backup count). It is as simple as using java.util.{Map, Queue, Set, List}. Just add the hazelcast.jar into your classpath and start coding. When you download the Hazelcast, you will find a test.sh under bin directory. The test.sh runs an application which randomly makes 40% get, 40% put and 20% remove on a distributed map. In this demo the same test application will be used to see how it performs on 100 node cluster. Amazon EC2 and S3 An easy to use and scalable cloud environment was needed for demo so we decided to use Amazon EC2 for server instances (nodes) and S3 service to store demo application zip and configuration files. With its newly announced Java SDK, it is very simple to start/stop server instances and upload files to S3 programatically. Hazelcast AMI & Launcher The challenge here is that we are running an application on 100 nodes and dealing with each and every server in the cluster is a huge task. We don't want to ssh into every server and manually start the application. This part is automated by creating a special server image (AMI). The AMI contains Java Runtime and a launcher application we developed, which will download the demo application from Amazon S3, unzip it, and run the hazelcast/bin/test.sh in it. The Launcher is actually so generic that it can run any application; it doesn't care/know what test.sh contains. Deployer Deployment of the demo application is also automated so that we don't need to login into AWS Management Console and manually start instances. Deployer instantiates any number of Amazon EC2 servers with any AMI and also uploads the demo application zip file to S3. So the idea here is that, the Deployer will store the application into S3 and launch 100 EC2 instances with our image. The Launcher on each instance will download the application from S3 and run it. Demo Details. The smallest EC2 instances (m1.small) are used to run the demo. These are the virtual instances with CPU about 1.0 GHz. Also keep in mind that EC2 platform suffers from considerable amount of network latency. That's why we increased the thread count to 250 in our application. The following steps performed during the demo Download hazelcast-1.8.3.zip from www.hazelcast.com. Unzip the file and move the monitoring war file into tomcat6/webapps directory. Edit the test.sh under the bin directory: Add -Xmx1G -Xms1G Add -Dhazelcast.initial.wait.seconds=100 to make the cluster evenly partition on start so that migration can be avoided for better performance. Add t250 as an argument to the application to set thread count to 250. Remember the latency issue. Run the Deployer from IDE. Check from EC2 Management Console if 100 servers started. Start tomcat. Copy the public DNS name of one of the servers to connect to from monitoring tool. Go to http://localhost:8080/hazelcast-monitor-1.8.3/ (Hazelcast Monitoring Tool). Paste the address and connect to the cluster. Enjoy! Results You should always look for programatic ways of launching applications on the cloud. With these tools we were able to deploy and run the demo application on 100 servers in minutes. The entire Hazelcast cluster was making over 400,000 operations per second on the smallest EC2 instances. In our next demo we will experiment Hazelcast on large data set and even bigger cluster. Watch the screencast
April 16, 2010
by Fuad Malikov
· 62,676 Views · 1 Like
article thumbnail
Free Online SVN Repositories
This week, I searched for free online SVN repositories for closed-source projects.
February 23, 2010
by Nicolas Fränkel
· 52,858 Views
article thumbnail
Four Methods to Automate Development Environment Setup
There are at least four methods that can be used in different combinations to make the process of setting up a complete development environment a lot less painful.
February 16, 2010
by Mitch Pronschinske
· 31,725 Views
article thumbnail
Checkout Multiple Projects Automatically Into Your Eclipse Workspace With Team Project Sets
When working in Eclipse, you’ll often end up with a number of projects in your workspace that constitute an application. You could have a multi-tiered system with a web, server and database project and other miscellaneous ones. Or if you’re an Eclipse RCP developer, you could end up with dozens of plugins each represented by a project. Although multiple projects give you modularity (which is good), they can make it difficult to manage the workspace (which is bad). Developers have to check out each project individually from different locations in the repository. Sometimes they even have to get projects from multiple repositories. This is a painstakingly long and error-prone task. But an easier way to manage multiple projects is with Eclipse’s Team Project Sets (TPS). Creating a workspace becomes as easy as importing an XML file and waiting for Eclipse to do its job. Yes, there are other more sophisticated tools out there that do this and more (eg. Maven and Buckminster) but team project sets are a good enough start if you haven’t got anything set up and may be good enough for the longer term as well, depending on how your team works. Create a Team Project Set to share with other developers It’s easy to create a team project set (TPS). The first thing is to start with a workspace that already has all the projects checked out. Then it’s as easy as choosing File > Export > Team > Team Project Set, selecting the projects you want to export and then entering a file name. Done. But it’s always better to see it in action. In the video, I export 3 projects that I’ve already checked out from Subversion into a TPS file. Notes: You can select which projects should go into the TPS. This way you can exclude irrelevant or personal projects you’ve got in your workspace. Eclipse adds the extension .psf if you don’t provide one. The exported file is an XML file, with the default extension of psf, so in the video the file would be music.psf. There is a project entry for each project you exported that includes the project’s name and its repository location, separated by commas. Once created, the file is easy to edit so go ahead and make your own changes if you want to. Here is an example of what it looks like: svn/repo/music-application/trunk,music-application"/> svn/repo/music-db/trunk,music-db"/> svn/repo/music-web/trunk,music-web"/> Import the Team Project Set to checkout multiple projects into your workspace Now for the fun part. To import a team project set (TPS), start with any workspace (normally an empty one) and choose File > Import > Team > Team Project Set. Choose the TPS file that someone else kindly exported for you and then wait for Eclipse to do its magic. Notes: If you have an existing project in your workspace whose name matches a project in the TPS, Eclipse will prompt you whether you want to overwrite the project. I always choose No To All, since overwriting the project will mean you lose any changes you made to it. But if you have the urge to start from scratch then you can choose Yes. The import also creates a link to the repository in SVN Repositories, so you don’t have to do that. If one already exists, it will not duplicate it but reuse the existing connection. The process may take a while depending on the number of projects in the TPS and the speed of your repo checkouts. You can choose to run the import in the background (as I did in the video), giving you the opportunity to use Eclipse while the import happens. Otherwise, grab some coffee and wait for it to finish the checkouts. Gotcha: You may find that Eclipse 3.4 and lower may actually create a repository connection per project if the repository didn’t exist beforehand, which is not ideal. To solve this, create an initial repository root that’s shared by the projects and then do the import of the TPS. This problem has been fixed in 3.5 Managing the team project set and working with branches I’d recommend checking in the team project set into your repository and versioning/tagging it along with the rest of your code base. With each release you may be adding/removing projects and consequently updating the TPS, so it’s important that the TPS matches what the repo looks like at that point. As projects are added/removed with each release, you have 3 possibilities: Recreate the TPS from an existing workspace: Same as the steps above, but it means that whoever does the export needs to maintain an up to date workspace to reflect the current project structure. Modify an existing TPS with the new/deleted project: This entails adding/removing an entry from the PSF file. Not a lot of maintenance, but someone needs to remember to do this. Automatically create/update the TPS: You could write a script that somehow updates the TPS to reflect the new repo structure. For example, if you’re developing an Eclipse RCP application, the PDE Build provides a map file that could be used as input to create the PSF file. If you want to checkout a branch other than trunk, just open the PSF file and do a Find/Replace of trunk with your branch name. You could also introduce an automated process as part of your build/release scripts to update the TPS with the correct branch and check it back in automatically, but that’s really optional. From http://eclipseone.wordpress.com
February 13, 2010
by Byron M
· 22,851 Views
article thumbnail
Promiscuous Integration vs. Continuous Integration
The emergence of version control systems makes both promiscuous and continuous integration merging techniques more attractive. Which is better?
February 10, 2010
by Martin Fowler
· 50,092 Views · 2 Likes
article thumbnail
Java Content Repository: The Best Of Both Worlds
Learn the basics of Java Content Repositories, including how they work, and how they're used.
January 4, 2010
by Bertrand Delacretaz
· 144,409 Views · 5 Likes
article thumbnail
Spring Integration and Apache Camel
Spring Integration and Apache Camel are open source frameworks providing a simpler solution for the Integration problems in the enterprise, to quote from their respective websites: Apache Camel - Apache Camel is a powerful open source integration framework based on known Enterprise Integration Patterns with powerful Bean Integration. Spring Integration - It provides an extension of the Spring programming model to support the well-known Enterprise Integration Patterns while building on the Spring Framework's existing support for enterprise integration. Essentially Spring Integration and Apache Camel enable applications to integrate with other systems. This article seeks to provide an implementation for an integration problem using both Spring Integration and Apache Camel. The objective is to show how easy it is to use these frameworks for a fairly complicated integration problem and to recommend either of these great products for your next Integration challenge. Problem: To illustrate the use of these frameworks consider a simple integration scenario, described using EIP terminology: The application needs to get a "Report" by aggregating "Sections" from a Section XML over http service. Each request for Report consists of a set of request for sections – in this specific example there are requests for three sections, the header, body and footer. The XML over http service returns a Section for the Section Request. The responses need to be aggregated into a single report. A sample test for this scenario is of the following type: ReportGenerator reportGenerator = reportGeneratorFactory.createReportGenerator(); List sectionRequests = new ArrayList(); String entityId="A Company"; sectionRequests.add(new SectionRequest(entityId,"header")); sectionRequests.add(new SectionRequest(entityId,"body")); sectionRequests.add(new SectionRequest(entityId,"footer")); ReportRequest reportRequest = new ReportRequest(sectionRequests); Report report = reportGenerator.generateReport(reportRequest); List sectionOfReport = report.getSections(); System.out.println(report); assertEquals(3, sectionOfReport.size()); The “ReportGenerator” is the messaging gateway, hiding the details of the underlying messaging infrastructure and in this specific case also the integration API – Apache Camel or Spring Integration. To start with, let us implement a solution to this integration problem using Spring Integration as the Framework, followed by Apache Camel. The complete working code using Spring Integration and Apache Camel is also available with the article. Solution Using Spring Integration: The Gateway component is easily configured using the following entry in the Spring Configuration. Internally Spring Integration uses AOP to hook up a component which routes the requests from an internal input channel and waits for the response in the response channel. The component to Split the Input Report Request to Section Request is fairly straightforward: public class SectionRequestSplitter { public List split(ReportRequest reportRequest){ return reportRequest.getSectionRequests(); } } and to hook this splitter with Spring Integration: Next, to transform the Section Request to an XML format - The component is the following: public class SectionRequestToXMLTransformer { public String transform(SectionRequest sectionRequest){ //this needs to be optimized...purely for demonstration of the concept String sectionRequestAsString = "" + sectionRequest.getEntityId() + "" + sectionRequest.getSectionId() + ""; return sectionRequestAsString; } } and is hooked up in the Spring Integration configuration file in the following way: To send an XML over http request using the Section Request XML to a section Service: To transform the Section Response XML to a Section Object - The component is the following: public class SectionResponseXMLToSectionTransformer { public Section transform(String sectionXML) { SAXReader saxReader = new SAXReader(); Document document; String sectionName = ""; String entityId = ""; try { document = saxReader.read(new StringReader(sectionXML)); sectionName = document .selectSingleNode("/section/meta/sectionName").getText(); entityId = document.selectSingleNode("/section/meta/entityId") .getText(); } catch (DocumentException e) { e.printStackTrace(); } return new Section(entityId, sectionName, sectionXML); } } and is hooked up in the Spring Integration configuration file in the following way: To aggregate the Sections together into a report, the component is the following:: public class SectionResponseAggregator { public Report aggregate(List sections) { return new Report(sections); } } and is hooked up in the Spring Integration configuration file in the following way: This completes the Spring Integration implementation for this Integration Problem. The following is the complete Spring Integration configuration file: A working sample is provided with the article(Download, extract and run "mvn test") Solution using Apache Camel: Apache Camel allows the route to be defined using multiple DSL implementations – Java DSL, Scala DSL and an XML based DSL. The recommended approach is to use Spring CamelContext as a runtime and the Java DSL for route development. The following is to build the Spring Camel Context: The route is configured by the Java based DSL: public class CamelRouteBuilder extends RouteBuilder { private String serviceURL; @Override public void configure() throws Exception { from("direct:start") .split().method("sectionRequestSplitterBean", "split") .aggregationStrategy(new ReportAggregationStrategy()) .transform().method("sectionRequestToXMLBean", "transform") .to(serviceURL) .transform().method("sectionResponseXMLToSectionBean", "transform"); } public void setServiceURL(String serviceURL) { this.serviceURL = serviceURL; } } Apache Camel does not provide an out of the box Message Gateway feature, however it is fairly easy to create a wrapper component that can hide the underlying details in the following way: Reader davsclaus has provided references to two mechanisms with Apache Camel to provide an out of the box Messaging Gateway - Messaging Gateway EIP and Camel Proxy which allows a POJO to be used as a Mesaging Gateway. Camel Proxy will be used with the article, and can be configured in the Camel Configuration files in the following way: Per davsclaus, there is a bug in Apache Camel(2.1 or older) when invoking a bean later in the route(the splitter bean), which is to be fixed in Apache Camel 2.2. To work around this bug, a convertBody step will be introduced in the route: from("direct:start") .convertBodyTo(ReportRequest.class) .split(bean("sectionRequestSplitterBean", "split"), new ReportAggregationStrategy()) .transform().method("sectionRequestToXMLBean", "transform") .to(serviceURL) .transform().method("sectionResponseXMLToSectionBean", "transform"); The component to Split the Input Report Request to Section Request is exactly same as Spring Integration component: public class SectionRequestSplitter { public List split(ReportRequest reportRequest){ return reportRequest.getSectionRequests(); } } To hook the component with Apache Camel: from("direct:start") .split().method("sectionRequestSplitterBean", "split") .... Next to transform the Section Request to an XML format, again this is exactly same as the implementation for Spring Integration, with hook being provided in the following manner: ...... .transform().method("sectionRequestToXMLBean", "transform") ...... To send an XML over http request using the Section Request XML to a section Service: ...... .transform().method("sectionRequestToXMLBean", "transform") .to(serviceURL) ......... To transform the Section Response XML to a Section object, the component is exactly same as the one used with Spring Integration, with the following highlighted hook in the Camel route: ...... .transform().method("sectionResponseXMLToSectionBean", "transform"); To aggregate the Section responses together into a report, the component is a bit more complicated than Spring Integration. Apache Camel supports a Scatter/Gather pattern using a route of the following type: ...... .split().method("sectionRequestSplitterBean", "split") .aggregationStrategy(new ReportAggregationStrategy()) with an aggregation strategy being passed on to the Splitter, the aggregation strategy implementation is the following: public class ReportAggregationStrategy implements AggregationStrategy { @Override public Exchange aggregate(Exchange oldExchange, Exchange newExchange) { if (oldExchange == null) { Section section = newExchange.getIn().getBody(Section.class); Report report = new Report(); report.addSection(section); newExchange.getIn().setBody(report); return newExchange; } Report report = oldExchange.getIn().getBody(Report.class); Section section = newExchange.getIn().getBody(Section.class); report.addSection(section); oldExchange.getIn().setBody(report); return oldExchange; } } This completes the Apache Camel based implementation. A working sample for Camel is provided with the article - just download, extract and run "mvn test". Conclusion: Spring Integration and Apache Camel provide a simple and clean approach for the Integration problems in a typical enterprise. They are lightweight frameworks – Spring Integration builds on top of Spring portfolio and extends the familiar programming model for the Integration domain and is easy to pick up, Apache camel provides a good Java based DSL and integrates well with Spring Core, with a fairly gentle learning curve. The article does not recommend one product over the other but encourages the reader to evaluate and learn from both these frameworks. References: Spring Integration Website: http://www.springsource.org/spring-integration Apache Camel Website: http://camel.apache.org/ Spring Integration Reference: http://static.springsource.org/spring-integration/reference/htmlsingle/spring-integration-reference.html Apache Camel User Guide: http://camel.apache.org/user-guide.html Plug for my blog: http://biju-allandsundry.blogspot.com/
December 31, 2009
by Biju Kunjummen
· 101,911 Views · 3 Likes
article thumbnail
Maven Repository Manager: Nexus Vs. Artifactory
My goal is to compare Sonatype Nexus and JFrog Artifactory,the two leading open source Maven repository managers.
December 14, 2009
by Ori Dar
· 136,080 Views · 4 Likes
article thumbnail
An Introduction to Feature-Driven Development – Part 2
This is the second part of a two-part article introducing Jeff De Luca’s Feature Driven Development (FDD) process. In particular, we are looking at how FDD differs from Scrum and eXtreme Programming-inspired approaches when it comes to working with larger teams and projects. In the first part we briefly introduced the ‘just enough’ upfront activities that FDD uses to support the additional communication that inevitably is needed in a larger project/team. In the second part of the article we cover how FDD leverages the results of those upfront activities within the highly iterative, self-managing, organized-chaos that is the delivery engine room of an FDD project. The Engine Room: Delivering Frequent, Tangible Working Results Once there is an initial overall model (FDD Process #1), an initial overall features list (FDD Process #2), and an initial overall plan (FDD Process #3) in place, an FDD project is ready to start delivering the required software feature by feature. Peter Coad, the Chief Architect on the original FDD project used the phrase ‘Deliver frequent, tangible, working results’ as a mantra to impress upon people the idea of delivering real, completed, client-valued function as often as possible. Scrum and eXtreme Programming do this using fixed length iterations of a calendar month or 2-4 weeks. FDD is different. Each Chief Programmer (lead developer) runs a series of iterations, each of which is normally a matter of a few days, and never longer than two weeks. At the start of each of these iterations, each Chief Programmer selects the next few features that make sense to implement from the backlog of feature sets (activities) that were assigned to him or her in FDD Process #2. The Chief Programmer leads the development of these features through FDD processes #4 and #5, Design by Feature (DBF) and Build By Feature (BBF). Note that iterations through the DBF/BBF processes are not fixed length, and Chief Programmers do not synchronize the start and end of their iterations with each other. In addition, the DBF/BBF processes are always executed as a pair (FDD describes them as two separate processes rather than one combined process for psychological reasons). FDD Process #4: Design By Feature After selecting the features for the iteration, a Chief Programmer needs to form their feature team. Yes, feature teams are formed and disbanded for each iteration through the DBF/BBF process pair. Using the knowledge gained from the modeling process (FDD Process #1), the Chief Programmer identifies the domain classes that are likely to be involved in this iteration, and forms his or her feature team from the owners of those classes. In practice, this means: a feature team is small, typically 3 to 5 people, because features are small. By definition, a feature team comprises of all the class owners who need to modify their classes in the development of the features during that iteration. There is no need to wait for members of other teams to change code. Therefore, there are all the benefits of code ownership and a sense of collective ownership too. Class owners may find themselves a member of multiple feature teams at the same time. This does not happen as frequently as might be supposed because iterations are so short – days not weeks. When it does, it is not a big problem in practice. Chief Programmers work together to resolve any problematic conflicts and, with care, most developers can manage the demands of occasionally belonging to more than one feature team for a short time. Once formed, the Chief Programmer facilitates the collaborative analysis and design of the features for that iteration. Depending on the complexity, this may involve the team walking through the requirements in detail with a domain expert, and studying any existing relevant documents. It also involves agreeing on the interactions and other details that need to be added to the model to support the new features. The final step in the DBF part of the iteration is to review the design. For simple features, this may be a brief sanity check of the design held within the feature team. For more significant features, the Chief Programmer will typically involve other Chief Programmers or class owners so that they are aware and can comment on the impact of the proposed design. For small team projects, the object models are frequently small enough for individual or pairs of developers to create good designs while writing tests for a particular feature or user story. For larger projects, this is not necessarily the case and designs created purely by considering the tests a feature or user story must pass are more likely to be brittle and require significant refactoring. The DBF process in FDD ensures that the overall model also guides the design, helping to maintain its ‘conceptual integrity’ [Brooks]. FDD Process #5: Build By Feature The Build by Feature (BBF) part of the iteration involves the team members coding up the features, testing them at both unit level and feature level, and holding a code inspection before promoting the completed features into the project's regular build process. Testing FDD expects developers to unit test their code. It expects feature teams to test their features. FDD is not overly concerned with how this is achieved. Projects and feature teams are free to adopt the testing tools, frameworks, and level of formality and completeness that are most appropriate. FDD does not mind if tests are written before or after code. What FDD mandates, is that the feature team deliver code that has been appropriately tested and inspected. Only once the new features have passed testing and inspection is the source code allowed into the build process. Code Inspections Most people want to know why FDD mandates code inspections, especially those that have endured sitting through hours of boring, unproductive, ego-polishing/demolishing, point-scoring sessions that formed so-called code reviews, inspections or walkthroughs. The reason FDD mandates code inspections is that research has shown time and again that when done well, inspections find more defects and different kinds of defects than testing [McConnell]. Not only that but by examining the code of the more experienced, knowledgeable developers on the team and having them explain the idioms they use, less experienced developers learn better coding techniques. In addition, knowing that their code will be inspected and not be allowed in the build unless it conforms to the agreed standards encourages developers to pay more attention to conforming to those standards. One of the benefits of working in feature teams is that the whole feature team is on the hot seat during an inspection, not just one individual. This removes much of the intensity and anxiety inherent in inspecting one individuals work. The Chief Programmer decides on the level of formality of each inspection depending on the complexity and impact of the features developed in that iteration. Where the code has little or no impact outside the feature team, an inspection will usually only involve the feature team inspecting each other’s work. Where there is significant impact the Chief Programmer pulls in other Chief Programmers and developers to both verify the code and communicate the impact of the new features. eXtreme Programming acknowledges inspections as a ‘best practice' but promotes pair programming as the logical conclusion of applying this practice. Pair programming is obviously better than individual developers delivering code without any form of inspection. However, while FDD neither mandates nor forbids pair programming, a more-traditional inspection is: fresh eyes looking at the code, catching bad assumptions made by the coder/s a Chief Programmer present to ensure the techniques passed on are good. After all, developers can just as easily teach each other bad habits as well as good habits. a change of pace for developers, a chance to step away from the keyboard and mouse for a short while. With the wide availability of automated source code formatting and static analysis tools, code inspections can now be shorter, concentrating on the logic and coding idioms involved and not getting bogged down in nit-picking such as alignment of braces, etc. The Build FDD assumes some sort of regular build process. Some teams build weekly, others daily and others continuously. FDD avoids mandating any particular build regime. This enables the project team to apply the most applicable. If a continuous integration environment makes sense, then the team is free to employ the best there is. Progress Reports Agile projects like highly visible progress information. FDD projects are no exception. In fact, because larger projects frequently have higher profiles within an organization, presenting meaningful, accurate, timely project information appropriately at the different levels of leadership/management is even more important. Conventionally, FDD projects track the development of each feature through its DBF/BBF iteration against six milestones: domain walkthrough, design, design inspection, coding, testing and inspection, and promoted to build. For each feature, Chief Programmers record the actual date a milestone is reached. Tracking each feature through these six milestones enables the project to keep an eye on how much work is 'in progress'. Too many features at a particular milestone indicate a process problem. Those promoting Kanban and other Limited Work In Progress methods have formalized this idea to strictly define what is meant by 'too many' for each of their development iteration milestones/statuses. They then refuse to move an item to a new milestone/status if the limit on the number of items at that status has been reached. This forces a team to keep items moving forward through the process [Kanban]. FDD is not so formal, leaving the Chief Programmers and Development Manager to keep an eye informally on the amount of work in progress. The Big Wallchart, Burn-Down/Up Charts, Etc For general visibility of progress within a project, the team typically lists all the features in the project complete with their owning Chief Programmer, feature team members, and the dates of each milestone achieved on a suitable wall. In addition, features can be colored to show if they are started, in-progress, completed or blocked. This allows people to stand back from the wall and get a good visual feel for the overall status of the project. They can then walk up to the wall to zoom in on particular areas and activities in more detail. Recording the date each milestone is achieved enables a team to produce burn-down or burn-up charts analogous to those produced in Scrum and XP. Chief Programmers and Project Managers can determine from these if the underlying rate of feature completion is increasing, decreasing, or stable, etc. One of the best ways to achieve this is to have the Chief Programmers regularly (typically once a week) communicate progress to either the project manager or someone dedicated to the task. That person then produces whatever roll-up and burn-down charts desired. Having an administrative person, the equivalent of the Tracker role in eXtreme Programming, perform these report formatting duties frees the Chief Programmers to spend more time on making progress rather than formatting reports about it. Parking Lot Charts For reporting to senior management, the level of individual features is often too granular. Here, FDD projects typically use a graphical report format that known as the Parking Lot chart. In a Parking Lot chart, each group of ‘parking lots’ represents one of the subject areas from the features list. Each parking lot represents one of the activities within that subject area, and displays the name of that set of features, the number of features within it, and the percentage of those features that have been completed (typically both in text and using a progress bar). The parking lots are also colored to indicate whether the features in that activity have been started, completed, or have significant blockages. The FDD parking lot format has become so popular that Mike Cohn included it in his book, Agile Planning and Estimating [Cohn]. (click for larger image) Figure 1: Example Parking Lot Chart Conclusion Feature-Driven Development combines the key advantages of other popular agile approaches with model-centric techniques and other best practices that scale to much larger teams and projects. It defines three upfront activities that provide a conceptual and management framework within which a larger-than-usual agile team can add functionality to the software, feature by feature. It is also just as applicable for smaller teams tackling non-trivial problem domains where it is worth spending just a little time to sketch a map of the journey before dashing off down the agile coding highway. Even if you and your team decide not to adopt FDD as a whole, understanding why FDD is the way it is, can provide insight into scaling traditional agile approaches beyond small, largely independent teams. Finally, I would like to say thank you to Serguei Khramtchenko and Mark Lesk at Nebulon for their corrections and suggestions incorporated in this article. References [Brooks] Frederick P. Brooks, Jr., The Mythical Man-Month, Addison Wesley [Cohn] Cohn, Agile Planning and Estimating, Prentice-Hall PTR [FDD] FDD Community Site, www.featuredrivendevelopment.com/ [Kanban] The home of Kanban software development, www.limitedwipsociety.org/ [McConnell] McConnell, Code Complete, Microsoft [Nebulon] The Latest FDD Processes available from www.nebulon.com/articles/fdd/latestprocesses.html [Palmer-1] Palmer, Felsing, A Practical Guide to Feature-Driven Development, Prentice Hall PTR
December 4, 2009
by Stephen Palmer
· 25,508 Views · 1 Like
article thumbnail
A Groovy ride on Camel
Apache Camel is a routing and mediation engine which implements the Enterprise Integration Patterns. But don't let the words Enterprise Integration scare you off. Camel is designed to be really light weight and has a small footprint. It can be reused anywhere, whether in a servlet, in a Web services stack, inside a full ESB or a standalone messaging application. Camel makes it really simple to implement messaging application. So there are not many reasons why you could not use it in non-enterprise application. In fact, it is possible to use Camel as a tool, similar to the way you use scripting languages. For example, you could fire up the Camel Web Console and define a messaging application without write a single line of code. This article is a getting-started type of tutorial. As you might have guessed, I'm going to use Groovy as the programming language. And the programs in this article are intend to be ran with Groovysh, the Groovy Shell. Reasons: Groovy is concise, expressive and has less noice than Java. All programs in this article are just a couple dozen lines long and should be real easy to follow along. Using Groovysh allows the reader to interact with the application. I'm a Linux guy and am comfortable with VIM and working in command line. So bare with me. Putting the pieces together First, I'm going to write a simple program to make sure I'm able to talk to Camel in Groovy. I'm not using an IDE like Eclipse, nor creating a project, nor going to use any build tools like Maven. Any text editor will be sufficient. Save the following code to a file named CamelDemo.groovy (source download). import groovy.grape.Grape Grape.grab(group:"org.apache.camel", module:"camel-core", version:"2.0.0") class MyRouteBuilder extends org.apache.camel.builder.RouteBuilder { void configure() { from("direct://foo").to("mock://result") } } mrb = new MyRouteBuilder() ctx = new org.apache.camel.impl.DefaultCamelContext() ctx.addRoutes mrb ctx.start() p = ctx.createProducerTemplate() p.sendBody "direct:foo", "Camel Ride for beginner" e = ctx.getEndpoint("mock://result") ex = e.exchanges.first() println "INFO> ${ex}" This little program does a couple things: Imports the camel-core jar using Grape.grab() Defines a our custom RouteBuilder, which defines a simple route between a direct:foo and a mock:result enpoints. Instantiates the CamelContext, adds our custom RouteBuilder to it and starts Camel by ctx.start(). Tests the route by sending a message exchange using the producerTemplate obtained from the CamelContext Lookups the mock:result endpoint (ctx.getEndpoint("mock:result")) and dislays the first Exchange, which should contain the message we just sent. Now start the groovysh in a command window and load the program: $ groovysh groovy:000> load CamelDemo.groovy you should see a bunch of output and then the output from the script: INFO> Exchange[Message: Camel Ride for beginner] ===> null groovy:000> At this point, you can interact with the program via groovysh. For example the following shows a few things you can do. groovy:000> ctx.routes ===> [EventDrivenConsumerRoute[Endpoint[seda://foo] -> UnitOfWork(Channel[sendTo(Endpoint[mock://result])])]] groovy:000> ctx.components ===> {mock=org.apache.camel.component.mock.MockComponent@14f2bd7, seda=org.apache.camel.component.seda.SedaComponent@c759f5} groovy:000> ctx.endpoints ===> [Endpoint[seda://foo], Endpoint[mock://result]] groovy:000> ctx.endpoints[1].exchanges ===> [Exchange[Message: Camel Ride for beginner]] groovy:000> ctx.endpoints[1].exchanges[0].in.body ===> Camel Ride for beginner groovy:000> p.sendBody("seda:foo", "Camel Kicking") ===> null groovy:000> e.exchanges ===> [Exchange[Message: Camel Ride for beginner], Exchange[Message: Camel Kicking]] This is it for our first Groovy/Camel program. For the curious, you can actually modify the program and reload it without terminating and restarting groovysh. Camel Stock Quote This is a simple stock quote application. Initially, I planed to walk you thru the development steps, from adding a simple bean as a Processor to transforming it to a Multi-Channel, Multi-Data-Format service application. But after I've finished developing the program, it turns out that it is too simple to justify for such elaboration. To save your time and mine, I'm just going to show you the final version right here. Take a look at it, and if you can understand what it does, then may be you should skip the rest of this article :) Save the following code to a file named StockQuote.groovy (source download). import groovy.grape.Grape Grape.grab(group:"org.apache.camel", module:"camel-core", version:"2.0.0") Grape.grab(group:"org.apache.camel", module:"camel-jetty", version:"2.0.0") Grape.grab(group:"org.apache.camel", module:"camel-freemarker", version:"2.0.0") class QuoteServiceBean { public String usStock(String symbol) { "${symbol}: 123.50 US\$" } public String hkStock(String symbol) { "${symbol}: 90.55 HK\$" } } class MyRouteBuilder extends org.apache.camel.builder.RouteBuilder { void configure() { from("direct://quote").choice() .when(body().contains(".HK")).bean(QuoteServiceBean.class, "hkStock") .otherwise().bean(QuoteServiceBean.class, "usStock") .end().to("mock://result") from("direct://xmlquote").transform().xpath("//quote/@symbol", String.class).to("direct://quote") //curl -H "Content-Type: text/xml" http://localhost:8080/quote?symbol=IBM from('jetty:http://localhost:8080/quote').transform() .simple('').to("direct://xmlquote").choice() .when(header("Content-Type").isEqualTo("text/xml")).to("freemarker:xmlquote.ftl") .otherwise().to("freemarker:htmlquote.ftl") .end() } } ctx = new org.apache.camel.impl.DefaultCamelContext() mrb = new MyRouteBuilder() ctx.addRoutes mrb ctx.start() p = ctx.createProducerTemplate() //p.sendBody("direct:quote", "00005.HK") //p.sendBody("direct:xmlquote", "") //p.sendBody("direct:xmlquote", "") e = ctx.getEndpoint("mock://result") //e.exchanges.each { ex -> // println "INFO> in.body='${ex.in.body}'" //} OK, you are still here. It is assumed that: We have two market data providers, one for U.S. market and the other for Hong Kong market. An existing QuoteServiceBean class has been implemented as a POJO. It has two methods, usStock() and hkStock(). It is part of a legacy system, it works great, it hides the underlying details of interacting with the data providers. No one understands it and no one dares to modify it. We would like to use the existing QuoteServiceBean to provide a stock quote service that can be consume easily. i.e. Multi-Channel and Multi-Data-Format. Content Based Router and Message Translator from("direct://quote").choice() .when(body().contains(".HK")).bean(QuoteServiceBean.class, "hkStock") .otherwise().bean(QuoteServiceBean.class, "usStock") .end().to("mock://result") The first route (start at line 17) represented by the direct:quote endpoint. It routes the message according to the content of the body of the exchange, which it's assumed to contain the stock symbol. When the body of the exchange contains the string ".HK" the hkStock(String symbol) of QuoteServiceBean is called, otherwise the usStock(String symbol) of QuoteServiceBean is called. Notice that the route DSL almost reads like plain English! Let us try it out. First start groovysh, load the program and send two messages to the direct:quote endpoint: jack@localhost tmp]$ groovysh Groovy Shell (1.6.6, JVM: 1.6.0_11) groovy:000> load StockQuote.groovy .............. groovy:000> p.sendBody("direct:quote", "00001.HK") ===> null groovy:000> e.exchanges.last() ===> Exchange[Message: 00001.HK: 90.55 HK$] groovy:000> p.sendBody("direct:quote", "SUNW") ===> null groovy:000> e.exchanges.last() ===> Exchange[Message: SUNW: 123.50 US$] groovy:000> That is it, our simple content-based router successfully routes request to the corresponding processor methods. XML Quote Request, message Transform from("direct://xmlquote").transform().xpath("//quote/@symbol", String.class).to("direct://quote") This next route simply accepts requests in XML, transforms the request and chains it to direct://quote. With this, we've added the capability to accept requests in XML format! We are using XPath here to expression our transform. Check out the hosts of Expression Langauges supported by Camel. Let us try it out: groovy:000> p.sendBody("direct:xmlquote", "") ===> null groovy:000> e.exchanges.last() ===> Exchange[Message: GOOG: 123.50 US$] groovy:000> Multi-Channel, Multi-Data-Format Provisioning Grape.grab(group:"org.apache.camel", module:"camel-jetty", version:"2.0.0") Grape.grab(group:"org.apache.camel", module:"camel-freemarker", version:"2.0.0") // ........... lines removed for brevity ............. from('jetty:http://localhost:8080/quote').transform() .simple('').to("direct://xmlquote").choice() .when(header("Content-Type").isEqualTo("text/xml")).to("freemarker:xmlquote.ftl") .otherwise().to("freemarker:htmlquote.ftl") .end() Here we use the camel-jetty to expose an endpoint jetty:http://localhost:8080/quote to our quote service. Note that camel-jetty is not part of camel-core. That is why we have to grab it into our program. The HTTP request is translate into XML using simple expression. Note that the camel-jetty has kindly extracted the request parameters as well as the HTTP header and placed them on the Message header. So the request parameter symbol is access as ${header.symbol} in the expression. Next we simply chain the message exchange to the direct:xmlquote endpoint. The result from direct:xmlquote went thru another translation, which depends on the content-type of the orginating HTTP request. Here, I make use of the camel-freemarker to generate the desire output. So we need to create the two Freemarker templates: htmlquote.ftl ${body} xmlquote.ftl ${body} So let us see it in action, I'm going to use curl to make HTTP requests. Do this on another command window: [jack@localhost tmp]$ curl http://localhost:8080/quote?symbol=IBM IBM: 123.50 US$ [jack@localhost tmp]$ curl http://localhost:8080/quote?symbol=00001.HK 00001.HK: 90.55 HK$ [jack@localhost tmp]$ And to request XML content: [jack@localhost tmp]$ curl -H "Content-Type: text/xml" http://localhost:8080/quote?symbol=IBM IBM: 123.50 US$ [jack@localhost tmp]$ That's about it for our multi-channel/multi-data-format ser vice provision. Summary In this tutorial, I hoped to illustrate how Camel supports message passing paradigm style of application development. Camel provides all sort of components to help you build processing pipelines. All you need is to implement your business logic as simple POJOs and let Camel handle all the translating, routing, filtering, spliting and forwarding for you. Not shown in this tutorial is how to consume external resources and services from within a route. No sweat, it is just as easy. Camel integrates nicely with Spring as well as Guice, but works nicely on its own. It won't be in your way if you don't need DI support in your application. As they say: Keep the simple easy. Camel works nicely in a JBI environment like ServiceMix and OpenESB. Camel is OSGI-ready and tracks newly deployed bundles for Route definitions at runtime. So you can gear it all to way up to be part of an enterprise SOA infrastructure. Disclaimer, I'm not an experienced Camel user and still learning. Thank you for staying up with me.
December 3, 2009
by Jack Hung
· 25,947 Views
article thumbnail
Spring Integration: A Hands-On Tutorial, Part 1
This tutorial is the first in a two-part series on Spring Integration. In this series we're going to build out a lead management system based on a message bus that we implement using Spring Integration. Our first tutorial will begin with a brief overview of Spring Integration and also just a bit about the lead management domain. After that we'll build our message bus. The second tutorial continues where the first leaves off and builds the rest of the bus. I’ve written the sample code for this tutorial as a Maven 2 project. I’m using Java 5, Spring Integration 1.0.3 and Spring 2.5.6. The code also works for Java 6. I've used Maven profiles to isolate the dependencies you’ll need if you’re running Java 5. The tutorials assume that you're comfortable with JEE, the core Spring framework and Maven 2. Also, Eclipse users may find the m2eclipse plug-in helpful. To complete the tutorial you'll need an IMAP account, and you'll also need access to an SMTP server. Let's begin with an overview of Spring Integration. A bird's eye view of Spring Integration Spring Integration is a framework for implementing a dynamically configurable service integration tier. The point of this tier is to orchestrate independent services into meaningful business solutions in a loosely-coupled fashion, which makes it easy to rearrange things in the face of changing business needs. The service integration tier sits just above the service tier as shown in figure 1. Following the book Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf (Addison-Wesley), Spring Integration adopts the well-known pipes and filters architectural style as its approach to building the service integration layer. Abstractly, filters are information-processing units (any type of processing—doesn’t have to be information filtering per se), and pipes are the conduits between filters. In the context of integration, the network we’re building is a messaging infrastructure—a so-called message bus—and the pipes and filters and called message channels and message endpoints, respectively. The network carries messages from one endpoint to another via channels, and the message is validated, routed, split, aggregated, resequenced, reformatted, transformed and so forth as the different endpoints process it. Figure 1. The service integration tier orchestrates the services below it. That should give you enough technical context to work through the tutorial. Let’s talk about the problem domain for our sample integration, which is enrollment lead management in an online university setting. Lead management overview In many industries, such as the mortgage industry and for-profit education, one important component of customer relationship management (CRM) is managing sales leads. This is a fertile area for enterprise integration because there are typically multiple systems that need to play nicely together in order to pull the whole thing off. Examples include front-end marketing/lead generation websites, external lead vendor systems, intake channels for submitted leads, lead databases, e-mail systems (e.g., to accept leads, to send confirmation e-mails), lead qualification systems, sales systems and potentially others. This tutorial and the next use Spring Integration to integrate several of systems of the kind just mentioned into an overall lead management capability for a hypothetical online university. Specifically we’ll integrate the following: • a CRM system that allows campus and call center staff to create leads directly, as they might do for walk-in or phone-in leads • a Request For Information (RFI) form on a lead generation ("lead gen") marketing website • a legacy e-mail based RFI channel • an external CRM that the international enrollment staff uses to process international leads • confirmation e-mails Figure 2 shows what it will look like when we’re done with both tutorials. For now focus on the big picture rather than the details. Figure 2. This is the lead management system we'll build. For this first tutorial we're simply going to establish the base staff interface, the (dummy) backend service that saves leads to a database, and confirmation e-mails. The second tutorial will deal with lead routing, web-based RFIs and e-mail-based RFIs. Let's dive in. We’ll begin with the basic lead creation page in the CRM and expand out from there. Building the core components [You can download the source code for this section of the tutorial here] We’re going to start by creating a lead creation HTML form for campus and call center staff. That way, if walk-in or phone-in leads express an interest, we can get them into the system. This is something that might appear as a part of a lead management module in a CRM system, as shown in figure 3. Figure 3. We'll build our lead management module with integration in mind from the beginning. Because we’re interested in the integration rather than the actual app features, we’re not really going to save the lead to the database. Instead we’ll just call a createLead() method against a local LeadService bean and leave it at that. But we will use Spring Integration to move the lead from the form to the service bean. Our first stop will be the domain model. DZone readers get 30% off Spring in Practice by Willie Wheeler and John Wheeler. Use code dzone30 when checking out with any version of the book at www.manning.com. Create the domain model We’ll need a domain object for leads, so listing 1 shows the one we’ll use. It’s not an industrial-strength representation, but it will do for the purposes of the tutorial. Listing 1. Lead.java, a basic domain object for leads. package crm.model;... other imports ...public class Lead { private static DateFormat dateFormat = new SimpleDateFormat(); private String firstName; private String middleInitial; private String lastName; private String address1; private String address2; ... other fields ... public Lead() { } public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } ... other getters and setters, and a toString() method ...} There is nothing special happening here at all. So far the Lead class is just a bunch of getters and setters. You can see the full code listing in the download. If you thought that was underwhelming, just wait until you see the LeadServiceImpl service bean in listing 2. Listing 2. LeadServiceImpl.java, a dummy service bean. package crm.service;import java.util.logging.Logger;import org.springframework.stereotype.Service;import crm.model.Lead;@Service("leadService")public class LeadServiceImpl implements LeadService { private static Logger log = Logger.getLogger("global"); public void createLead(Lead lead) { log.info("Creating lead: " + lead); } This is just a dummy bean. In real life we’d save the lead to a database. The bean implements a basic LeadService interface that we've suppressed here, but it's available in the code download. Now that we have our domain model, let’s use Spring Integration to create a service integration tier above it. Create the service integration tier If you look back at figure 3, you’ll see that the CRM app pushes lead data to the service bean by way of a channel called newLeadChannel. While it’s possible for the CRM app to push messages onto the channel directly, it’s generally more desirable to keep the systems you’re integrating decoupled from the underlying messaging infrastructure, such as channels. That allows you to configure service orchestrations dynamically instead of having to go into the code. Spring Integration supports the Gateway pattern (described in the aforementioned Enterprise Integration Patterns book), which allows an application to push messages onto the message bus without knowing anything about the messaging infrastructure. Listing 3 shows how we do this. Listing 3. LeadGateway.java, a gateway offering access to the messaging system. package crm.integration.gateways;import org.springframework.integration.annotation.Gateway;import crm.model.Lead;public interface LeadGateway { @Gateway(requestChannel = "newLeadChannel") void createLead(Lead lead);} We are of course using the Spring Integration @Gateway annotation to map the method call to the newLeadChannel, but gateway clients don’t know that. Spring Integration will use this interface to create a dynamic proxy that accepts a Lead instance, wraps it with an org.springframework.integration.core.Message, and then pushes the Message onto the newLeadChannel. The Lead instance is the Message body, or payload, and Spring Integration wraps the Lead because only Messages are allowed on the bus. We need to wire up our message bus. Figure 4 shows how to do that with an application context configuration file. Listing 4. /WEB-INF/applicationContext-integration.xml message bus definition. The first thing to notice here is that we've made the Spring Integration namespace our default namespace instead of the standard beans namespace. The reason is that we're using this configuration file strictly for Spring Integration configuration, so we can save some keystrokes by selecting the appropriate namespace. This works pretty nicely for some of the other Spring projects as well, such as Spring Batch and Spring Security. In this configuration we've created the three messaging components that we saw in figure 3. First, we have an incoming lead gateway to allow applications to push leads onto the bus. We simply reference the interface from listing 3; Spring Integration takes care of the dynamic proxy. Next we create a publish/subscribe ("pub-sub") channel called newLeadChannel. This is the channel that the @Gateway annotation referenced in listing 3. A pub-sub channel can publish a message to multiple endpoints simultaneously. For now we have only one subscriber—a service activator—but we already know we're going to have others, so we may as well make this a pub-sub channel. The service activator is an endpoint that allows us to bring our LeadServiceImpl service bean onto the bus. We're injecting the newLeadChannel into the input end of the service activator. When a message appears on the newLeadChannel, the service activator will pass its Lead payload to the leadService bean's createLead() method. Stepping back, we've almost implemented the design described by figure 3. The only part that remains is the lead creation frontend, which we'll address right now. Create the web tier Our user interface for creating new leads will be a web-based form that we implement using Spring Web MVC. The idea is that enrollment staff at campuses or call centers might use such an interface to handle walk-in or phone-in traffic. Listing 5 shows our simple @Controller. Listing 5. LeadController.java, a @Controller to allow staff to create leads package crm.web;import java.util.Date;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Controller;import org.springframework.ui.Model;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RequestMethod;import crm.integration.gateways.LeadGateway;import crm.model.Country;import crm.model.Lead;@Controllerpublic class LeadController { @Autowired private LeadGateway leadGateway; @RequestMapping(value = "/lead/form.html", method = RequestMethod.GET) public void getForm(Model model) { model.addAttribute(Country.getCountries()); model.addAttribute(new Lead()); } @RequestMapping(value = "/lead/form.html", method = RequestMethod.POST) public String postForm(Lead lead) { lead.setDateCreated(new Date()); leadGateway.createLead(lead); return "redirect:form.html?created=true"; } This isn't an industrial-strength controller as it doesn't do HTTP parameter whitelisting (for example, via an @InitBinder method) and form validation, both of which you would expect from a real implementation. But the main pieces from a Spring Integration perspective are here. We're autowiring the gateway into the @Controller, and we have methods for serving up the empty form and for processing the submitted form. The getForm() method references a Countries class that we've suppressed (it's in the code download); it just puts a list of countries on the model so the form can present a Country field to the staff member. The postForm() method invokes the createLead() method on the gateway. This will pass the Lead to the dynamic proxy LeadGateway implementation, which in turn will wrap the Lead with a Message and then place the Message on the newLeadChannel. There are a few other configuration files you will need to put in place, including web.xml, main-servlet.xml and applicationContext.xml. There's also a JSP for the web form. As none of these relates directly to Spring Integration, we won't treat them here. Please see the code download for details. With that, we've established a baseline system. To try it out, run mvn jetty:run against crm/pom.xml and point your browser at http://localhost:8080/crm/main/lead/form.html You should see a very basic-looking web form for entering lead information. Enter some user information (it doesn't matter what you enter—recall that we don't have any form validation) and press Submit. The console should report that LeadServiceImpl.createLead() created a lead. Congratulations! Even though we now have a working system, it isn't very interesting. From here on out (this tutorial and the next) we'll be adding some common features to make the lead management system more capable. Our first addition will be confirmation e-mails; the next tutorial will present further additions. Adding confirmation e-mails [The source for this section is available here] After an enrollment advisor (or some other staff member) creates a lead in the system, we want to send the lead an e-mail letting him know that that's happened. Actually—and this is a critical point—we really don't care how the lead was created. Anytime a lead appears on the newLeadChannel, we want to fire off a confirmation e-mail. I'm making the distinction because it points to an important aspect of the message bus: it allows us to control lead processing code centrally instead of having to chase it down in a bunch of different places. Right now there's only one way to create leads, but figure 2 revealed that we'll be adding others. No matter how many we add, they'll all result in sending a confirmation e-mail out to the lead. Figure 4 shows the new bit of plumbing we're going to add to our message bus. Figure 4. Send a confirmation e-mail when creating a lead. To do this, we're going to need to make a few changes to the configuration and code. POM changes First we need to update the POM. Here's a summary of the changes; see the code download for details: • Add a JavaMail dependency to the Jetty plug-in. • Add an org.springframework.context.support dependency. • Add a spring-integration-mail dependency. • Set the mail.version property. These changes will allow us to use JavaMail. Expose JavaMail sessions through JNDI We'll also need to add a /WEB-INF/jetty-env.xml configuration to make our JavaMail sessions available via JNDI. Once again, see the code download for details. I've included a /WEB-INF/jetty-env.xml.sample configuration for your convenience. As mentioned previously, you'll need access to an SMTP server. Besides creating jetty-env.xml, we'll need to update applicationContext.xml. Listing 6 shows the changes we need so we can use JavaMail and SMTP. Listing 6. /WEB-INF/applicationContext.xml changes supporting JavaMail and SMTP The changes expose JavaMail sessions as a JNDI resource. We've declared the jee namespace and its schema location, configured the JNDI lookup, and created a JavaMailSenderImpl bean that we'll use for sending mail. We won't need any domain model changes to generate confirmation e-mails. We will however need to create a bean to back our new transformer endpoint. Service integration tier changes First, recall from figure 4 that the newLeadChannel feeds into a LeadToEmailTransformer endpoint. This endpoint takes a lead as an input and generates a confirmation e-mail as an output, and the e-mail gets pipes out to an SMTP transport. In general, transformers transform given inputs into desired outputs. No surprises there. Figure 4 is slightly misleading since it's actually the POJO itself that we're going to call LeadToEmailTransformer; the endpoint is really just a bean adapter that the messaging infrastructure provides so we can place the POJO on the message bus. Listing 7 presents the LeadToEmailTransformer POJO. Listing 7. LeadToEmailTransformer.java, a POJO to generate confirmation e-mails package crm.integration.transformers;import java.util.Date;import java.util.logging.Logger;import org.springframework.integration.annotation.Transformer;import org.springframework.mail.MailMessage;import org.springframework.mail.SimpleMailMessage;import crm.model.Lead;public class LeadToEmailTransformer { private static Logger log = Logger.getLogger("global"); private String confFrom; private String confSubj; private String confText; ... getters and setters for the fields ... @Transformer public MailMessage transform(Lead lead) { log.info("Transforming lead to confirmation e-mail: " + lead); String leadFullName = lead.getFullName(); String leadEmail = lead.getEmail(); MailMessage msg = new SimpleMailMessage(); msg.setTo(leadFullName == null ? leadEmail : leadFullName + " <" + leadEmail + ">"); msg.setFrom(confFrom); msg.setSubject(confSubj); msg.setSentDate(new Date()); msg.setText(confText); log.info("Transformed lead to confirmation e-mail: " + msg); return msg; } Again, LeadToEmailTransformer is a POJO, so we use the @Transformer annotation to select the method that's performing the transformation. We use a Lead for the input and a MailMessage for the output, and perform a simple transformation in between. When defining backing beans for the various Spring Integration filters, it's possible to specify a Message as an input or an output. That is, if we want to deal with the messages themselves rather than their payloads, we can do that. (Don't confuse the MailMessage in listing 7 with a Spring Integration message; MailMessage represents an e-mail message, not a message bus message.) We might do that in cases where we want to read or manipulate message headers. In this tutorial we don't need to do that, so our backing beans just deal with payloads. Now we'll need to build out our message bus so that it looks like figure 4. We do this by updating applicationContext-integration.xml as shown in listing 8. Listing 8. /WEB-INF/applicationContext-integration.xml updates to support confirmation e-mails The property-placeholder configuration loads the various ${...} properties from a properties file; see /crm/src/main/resources/applicationContext.properties in the code download. You don't have to change anything in the properties file. The transformer configuration brings the LeadToEmailTransformer bean into the picture so it can transform Leads that appear on the newLeadChannel into MailMessages that it puts on the confEmailChannel. As a side note, the p namespace way of specifying bean properties doesn't seem to work here (I assume it's a bug: http://jira.springframework.org/browse/SPR-5990), so I just did it the more verbose way. The channel definition defines a point-to-point channel rather than a pub-sub channel. That means that only one endpoint can pull messages from the channel. Finally we have an outbound-channel-adapter that grabs MailMessages from the confEmailChannel and then sends them using the referenced mailSender, which we defined in listing 6. That's it for this section. We should have working confirmation e-mails. Restart your Jetty instance and go again to http://localhost:8080/crm/main/lead/form.html Fill it out and provide your real e-mail address in the e-mail field. A few moments after submitting the form you should receive a confirmation e-mail. If you don't see it, you might check your SMTP configuration in jetty-env.xml, or else check your spam folder. Summary In this tutorial we've taken our first steps toward developing an integrated lead management system. Though the current bus configuration is simple, we've already seen some key Spring Integration features, including • support for the Gateway pattern, allowing us to connect apps to the message bus without knowing about messages • point-to-point and pub-sub channels • service activators to allow us to place service beans on the bus • message transformers • outbound SMTP channel adapters to allow us to send e-mail The second tutorial will continue elaborating what we've developed here, demonstrating the use of several additional Spring Integration features, including • message routers (including content-based message routers) • outbound web service gateways for sending SOAP messages • inbound HTTP adapters for collecting HTML form data from external systems • inbound e-mail channel adapters (we'll use IMAP IDLE, though POP and IMAP are also possible) for processing incoming e-mails Enjoy, and stay tuned. Willie is a solutions architect with 12 years of Java development experience. He and his brother John are coauthors of the upcoming book Spring in Practice by Manning Publications (www.manning.com/wheeler/). Willie also publishes technical articles (including many on Spring) to wheelersoftware.com/articles/.
August 18, 2009
by Willie Wheeler
· 249,419 Views · 3 Likes
article thumbnail
JPA 2.0 Concurrency and Locking
Optimistic locking lets concurrent transactions process simultaneously, but detects and prevent collisions, this works best for applications where most concurrent transactions do not conflict. JPA Optimistic locking allows anyone to read and update an entity, however a version check is made upon commit and an exception is thrown if the version was updated in the database since the entity was read. In JPA for Optimistic locking you annotate an attribute with @Version as shown below: public class Employee { @ID int id; @Version int version; The Version attribute will be incremented with a successful commit. The Version attribute can be an int, short, long, or timestamp. This results in SQL like the following: “UPDATE Employee SET ..., version = version + 1 WHERE id = ? AND version = readVersion” The advantages of optimistic locking are that no database locks are held which can give better scalability. The disadvantages are that the user or application must refresh and retry failed updates. Optimistic Locking Example In the optimistic locking example below, 2 concurrent transactions are updating employee e1. The transaction on the left commits first causing the e1 version attribute to be incremented with the update. The transaction on the right throws an OptimisticLockException because the e1 version attribute is higher than when e1 was read, causing the transaction to roll back. Additional Locking with JPA Entity Locking APIs With JPA it is possible to lock an entity, this allows you to control when, where and which kind of locking to use. JPA 1.0 only supported Optimistic read or Optimistic write locking. JPA 2.0 supports Optimistic and Pessimistic locking, this is layered on top of @Version checking described above. JPA 2.0 LockMode values : OPTIMISTIC (JPA 1.0 READ): perform a version check on locked Entity before commit, throw an OptimisticLockException if Entity version mismatch. OPTIMISTIC_FORCE_INCREMENT (JPA 1.0 WRITE) perform a version check on locked Entity before commit, throw an OptimisticLockException if Entity version mismatch, force an increment to the version at the end of the transaction, even if the entity is not modified. PESSIMISTIC: lock the database row when reading PESSIMISTIC_FORCE_INCREMENT lock the database row when reading, force an increment to the version at the end of the transaction, even if the entity is not modified. There are multiple APIs to specify locking an Entity: EntityManager methods: lock, find, refresh Query methods: setLockMode NamedQuery annotation: lockMode element OPTIMISTIC (READ) LockMode Example In the optimistic locking example below, transaction1 on the left updates the department name for dep , which causes dep's version attribute to be incremented. Transaction2 on the right gives an employee a raise if he's in the "Eng" department. Version checking on the employee attribute would not throw an exception in this example since it was the dep Version attribute that was updated in transaction1. In this example the employee change should not commit if the department was changed after reading, so an OPTIMISTIC lock is used : em.lock(dep, OPTIMISTIC). This will cause a version check on the dep Entity before committing transaction2 which will throw an OptimisticLockException because the dep version attribute is higher than when dep was read, causing the transaction to roll back. OPTIMISTIC_FORCE_INCREMENT (write) LockMode Example In the OPTIMISTIC_FORCE_INCREMENT locking example below, transaction2 on the right wants to be sure that the dep name does not change during the transaction, so transaction2 locks the dep Entity em.lock(dep, OPTIMISTIC_FORCE_INCREMENT) and then calls em.flush() which causes dep's version attribute to be incremented in the database. This will cause any parallel updates to dep to throw an OptimisticLockException and roll back. In transaction1 on the left at commit time when the dep version attribute is checked and found to be stale, an OptimisticLockException is thrown Pessimistic Concurrency Pessimistic concurrency locks the database row when data is read, this is the equivalent of a (SELECT . . . FOR UPDATE [NOWAIT]) . Pessimistic locking ensures that transactions do not update the same entity at the same time, which can simplify application code, but it limits concurrent access to the data which can cause bad scalability and may cause deadlocks. Pessimistic locking is better for applications with a higher risk of contention among concurrent transactions. The examples below show: reading an entity and then locking it later reading an entity with a lock reading an entity, then later refreshing it with a lock The Trade-offs are the longer you hold the lock the greater the risks of bad scalability and deadlocks. The later you lock the greater the risk of stale data, which can then cause an optimistic lock exception, if the entity was updated after reading but before locking. The right locking approach depends on your application: what is the risk of risk of contention among concurrent transactions? What are the requirements for scalability? What are the requirements for user re-trying on failure? For More Information: Preventing Non-Repeatable Reads in JPA Using EclipseLink Java Persistence API 2.0: What's New ? What's New and Exciting in JPA 2.0 Beginning Java™ EE 6 Platform with GlassFish™ 3 Pro EJB 3: Java Persistence API (JPA 1.0)
August 3, 2009
by Carol McDonald
· 51,418 Views · 1 Like
  • Previous
  • ...
  • 389
  • 390
  • 391
  • 392
  • 393
  • 394
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×