I've blogged a few times now about Git (which I pronounce with a hard 'g' a la "get", as it's supposed to be named for Linus Torvalds, a self-described git, but which I've also heard called pronounced with a soft 'g' like "jet"). Either way, I'm finding it way more efficient and less painful than either CVS or SVN combined. So, to continue this series ([1], [2], [3]), here is how (and why) to pull an SVN repo down as a Git repo, but with the omission of old (irrelevant) revisions and branches. Using SVN for SVN repos In days of yore when working with the JBoss Tools and JBoss Developer Studio SVN repos, I would keep a copy of everything in trunk on disk, plus the current active branch (most recent milestone or stable branch maintenance). With all the SVN metadata, this would eat up substantial amounts of disk space but still require network access to pull any old history of files. The two repos were about 2G of space on disk, for each branch. Sure, there's tooling to be able to diff and merge between branches w/o having both branches physically checked out, but nothing beats the ability to place two folders side by side OFFLINE for deep comparisons. So, at times, I would burn as much as 6-8G of disk simply to have a few branches of source for comparison and merging. With my painfullly slow IDE drive, this would grind my machine to a halt, especially when doing any SVN operation or counting files / disk usage. Using Git for SVN repos naively Recently, I started using git-svn to pull the whole JBDS repo into a local Git repo, but it was slow to create and still unwieldy. And the JBoss Tools repo was too large to even create as a Git repo - the operation would run out of memory while processing old revisions of code to play forward. At this point, I was stuck having individual Git repos for each JBoss Tools component (major source folder) in SVN: archives, as, birt, bpel, build, etc. It worked, but replicating it when I needed to create a matching repo-collection for a branch was painful and time-consuming. As well, all the old revision information was eating even more disk than before: jbosstools' trunk as multiple git-svn clones: 6.1G devstudio's trunk as single git-svn clone: 1.3G So, now, instead of a couple Gb per branch, I was at nearly 4x as much disk usage. But at least I could work offline and not deal w/ network-intense activity just to check history or commit a change. Still, far from ideal. Cloning SVN with standard layout & partial history This past week, I discovered two ways to make the git-svn experience at least an order of magnitude better: Standard layout (-s) - this allows your generated Git repo to contain the usual trunk, branches/* and tags/* layout that's present in the source SVN repo. This is a win because it means your repo will contain the branch information so you can easily switch between branches within the same repo on disk. No more remote network access needed! Revision filter (-r) - this allows your generated Git repo to start from a known revision number instead of starting at its birth. Now instead of taking hours to generate, you can get a repo in minutes by excluding irrelevant (ancient) revisions. So, why is this cool? Because now, instead of having 2G of source+metadata to copy when I want to do a local comparison between branches, the size on disk is merely: jbosstools' trunk as single git-svn clone w/ trunk and single branch: 1.3G devstudio's trunk as single git-svn clone w/ trunk and single branch: 0.13G So, not only is the footprint smaller, but the performance is better and I need never do a full clone (or svn checkout) again - instead, I can just copy the existing Git repo, and rebase it to a different branch. Instead of hours, this operation takes seconds (or minutes) and happens without the need for a network connection. Okay, enough blather. Show me the code! Check out the repo, including only the trunk & most recent branch # Figure out the revision number based on when a branch was created, then # from r28571, returns -r28571:HEAD rev=$(svn log --stop-on-copy \ http://svn.jboss.org/repos/jbosstools/branches/jbosstools-3.2.x \ | egrep "r[0-9]+" | tail -1 | sed -e "s#\(r[0-9]\+\).\+#-\1:HEAD#") # now, fetch repo starting from the branch's initial commit git svn clone -s $rev http://svn.jboss.org/repos/jbosstools jbosstools_GIT Now you have a repo which contains trunk & a single branch git branch -a # list local (Git) and remote (SVN) branches * master remotes/jbosstools-3.2.x remotes/trunk Switch to the branch git checkout -b local/jbosstools-3.2.x jbosstools-3.2.x # connect a new local branch to remote one Checking out files: 100% (609/609), done. Switched to a new branch 'local/jbosstools-3.2.x' git svn info # verify now working in branch URL: http://svn.jboss.org/repos/jbosstools/branches/jbosstools-3.2.x Repository Root: http://svn.jboss.org/repos/jbosstools Switch back to trunk git checkout -b local/trunk trunk # connect a new local branch to remote trunk Switched to a new branch 'local/trunk' git svn info # verify now working in branch URL: http://svn.jboss.org/repos/jbosstools/trunk Repository Root: http://svn.jboss.org/repos/jbosstools Rewind your changes, pull updates from SVN repo, apply your changes; won't work if you have local uncommitted changes git svn rebase Fetch updates from SVN repo (ignoring local changes?) git svn fetch Create a new branch (remotely with SVN) svn copy \ http://svn.jboss.org/repos/jbosstools/branches/jbosstools-3.2.x \ http://svn.jboss.org/repos/jbosstools/branches/some-new-branch From http://divby0.blogspot.com/2011/01/howto-partially-clone-svn-repo-to-git.html

we've all seen them on various websites. crappy search utilities. they are a constant reminder that search is not something you should take lightly when building a website or application. search is not just google's game anymore. when a java library called lucene was introduced into the apache ecosystem, and then solr was built on top of that, open source developers began to wield some serious power when it came to customizing search features. in this article you'll be introduced to apache solr and a wealth of applications that have been built with it. the content is divided as follows: introduction setup solr applications summary 1. introduction apache solr is an open source search server. it is based on the full text search engine called apache lucene . so basically solr is an http wrapper around an inverted index provided by lucene. an inverted index could be seen as a list of words where each word-entry links to the documents it is contained in. that way getting all documents for the search query "dzone" is a simple 'get' operation. one advantage of solr in enterprise projects is that you don't need any java code, although java itself has to be installed. if you are unsure when to use solr and when lucene, these answers could help. if you need to build your solr index from websites, you should take a look into the open source crawler called apache nutch before creating your own solution. to be convinced that solr is actually used in a lot of enterprise projects, take a look at this amazing list of public projects powered by solr . if you encounter problems then the mailing list or stackoverflow will help you. to make the introduction complete i would like to mention my personal link list and the resources page which lists books, articles and more interesting material. 2. setup solr 2.1. installation as the very first step, you should follow the official tutorial which covers the basic aspects of any search use case: indexing - get the data of any form into solr. examples: json, xml, csv and sql-database. this step creates the inverted index - i.e. it links every term to its documents. querying - ask solr to return the most relevant documents for the users' query to follow the official tutorial you'll have to download java and the latest version of solr here . more information about installation is available at the official description . next you'll have to decide which web server you choose for solr. in the official tutorial, jetty is used, but you can also use tomcat. when you choose tomcat be sure you are setting the utf-8 encoding in the server.xml . i would also research the different versions of solr, which can be quite confusing for beginners: the current stable version is 1.4.1. use this if you need a stable search and don't need one of the latest features. the next stable version of solr will be 3.x the versions 1.5 and 2.x will be skipped in order to reach the same versioning as lucene. version 4.x is the latest development branch. solr 4.x handles advanced features like language detection via tika, spatial search , results grouping (group by field / collapsing), a new "user-facing" query parser ( edismax handler ), near real time indexing, huge fuzzy search performance improvements, sql join-a like feature and more. 2.2. indexing if you've followed the official tutorial you have pushed some xml files into the solr index. this process is called indexing or feeding. there are a lot more possibilities to get data into solr: using the data import handler (dih) is a really powerful language neutral option. it allows you to read from a sql database, from csv, xml files, rss feeds, emails, etc. without any java knowledge. dih handles full-imports and delta-imports. this is necessary when only a small amount of documents were added, updated or deleted. the http interface is used from the post tool, which you have already used in the official tutorial to index xml files. client libraries in different languages also exist. (e.g. for java (solrj) or python ). before indexing you'll have to decide which data fields should be searchable and how the fields should get indexed. for example, when you have a field with html in it, then you can strip irrelevant characters , tokenize the text into 'searchable terms', lower case the terms and finally stem the terms . in contrast, if you would have a field with text in it that should not be interpreted (e.g. urls) you shouldn't tokenize it and use the default field type string. please refer to the official documentation about field and field type definitions in the schema.xml file. when designing an index keep in mind the advice from mauricio : "the document is what you will search for. " for example, if you have tweets and you want to search for similar users, you'll need to setup a user index - created from the tweets. then every document is a user. if you want to search for tweets, then setup a tweet index; then every document is a tweet. of course, you can setup both indices with the multi index options of solr. please also note that there is a project called solr cell which lets you extract the relevant information out of several different document types with the help of tika. 2.3. querying for debugging it is very convenient to use the http interface with a browser to query solr and get back xml. use firefox and the xml will be displayed nicely: you can also use the velocity contribution , a cross-browser tool, which will be covered in more detail in the section about 'search application prototyping' . to query the index you can use the dismax handler or standard query handler . you can filter and sort the results: q=superman&fq=type:book&sort=price asc you can also do a lot more ; one other concept is boosting. in solr you can boost while indexing and while querying. to prefer the terms in the title write: q=title:superman^2 subject:superman when using the dismax request handler write: q=superman&qf=title^2 subject check out all the various query options like fuzzy search , spellcheck query input , facets , collapsing and suffix query support . 3. applications now i will list some interesting use cases for solr - in no particular order. to see how powerful and flexible this open source search server is. 3.1. drupal integration the drupal integration can be seen as generic use case to integrate solr into php projects. for the php integration you have the choice to either use the http interface for querying and retrieving xml or json. or to use the php solr client library . here is a screenshot of a typical faceted search in drupal : for more information about faceted search look into the wiki of solr . more php projects which integrates solr: open source typo3- solr module magento enterprise - solr module . the open source integration is out dated. oxid - solr module . no open source integration available. 3.2. hathi trust the hathi trust project is a nice example that proves solr's ability to search big digital libraries. to quote directly from the article : "... the index for our one million book index is over 200 gigabytes ... so we expect to end up with a two terabyte index for 10 million books" other examples for libraries: vufind - aims to replace opac internet archive national library of australia 3.3. auto suggestions mainly, there are two approaches to implement auto-suggestions (also called auto-completion) with solr: via facets or via ngramfilterfactory . to push it to the extreme you can use a lucene index entirely in ram. this approach is used in a large music shop in germany. live examples for auto suggestions: kaufda.de 3.4. spatial search applications when mentioning spatial search, people have geographical based applications in mind. with solr, this ordinary use case is attainable . some examples for this are : city search - city guides yellow pages kaufda.de spatial search can be useful in many different ways : for bioinformatics, fingerprints search, facial search, etc. (getting the fingerprint of a document is important for duplicate detection). the simplest approach is implemented in jetwick to reduce duplicate tweets, but this yields a performance of o(n) where n is the number of queried terms. this is okay for 10 or less terms, but it can get even better at o(1)! the idea is to use a special hash set to get all similar documents. this technique is called local sensitive hashing . read this nice paper about 'near similarity search and plagiarism analysis' for more information. 3.5. duckduckgo duckduckgo is made with open source and its "zero click" information is done with the help of solr using the dismax query handler: the index for that feature contains 18m documents and has a size of ~12gb. for this case had to tune solr: " i have two requirements that differ a bit from most sites with respect to solr: i generally only show one result, with sometimes a couple below if you click on them. therefore, it was really important that the first result is what people expected. false positives are really bad in 0-click, so i needed a way to not show anything if a match wasn't too relevant. i got around these by a) tweaking dismax and schema and b) adding my own relevancy filter on top that would re-order and not show anything in various situations. " all the rest is done with tuned open source products. to quote gabriel again: "the main results are a hybrid of a lot of things, including external apis, e.g. bing, wolframalpha, yahoo, my own indexes and negative indexes (spam removal), etc. there are a bunch of different types of data i'm working with. " check out the other cool features such as privacy or bang searches . 3.6. clustering support with carrot2 carrot2 is one of the "contributed plugins" of solr. with carrot2 you can support clustering : " clustering is the assignment of a set of observations into subsets (called clusters) so that observations in the same cluster are similar in some sense. " see some research papers regarding clustering here . here is one visual example when applying clustering on the search "pannous" - our company : 3.7. near real time search solr isn't real time yet, but you can tune solr to the point where it becomes near real time, which means that the time ('real time latency') that a document takes to be searchable after it gets indexed is less than 60 seconds even if you need to update frequently. to make this work, you can setup two indices. one write-only index "w" for the indexer and one read-only index "r" for your application. index r refers to the same data directory of w, which has to be defined in the solrconfig.xml of r via: /pathto/indexw/data/ to make sure your users and the r index see the indexed documents of w, you have to trigger an empty commit every 60 seconds: wget -q http://localhost:port/solr/update?stream.body=%3ccommit/%3e -o /dev/null everytime such a commit is triggered a new searcher without any cache entries is created. this can harm performance for visitors hitting the empty cache directly after this commit, but you can fill the cache with static searches with the help of the newsearcher entry in your solrconfig.xml. additionally, the autowarmcount property needs to be tuned, which fills the cache with a newsearcher from old entries. also, take a look at the article 'scaling lucene and solr' , where experts explain in detail what to do with large indices (=> 'sharding') and what to do for high query volume (=> 'replicating'). 3.8. loggly = full text search in logs feeding log files into solr and searching them at near real-time shows that solr can handle massive amounts of data and queries the data quickly. i've setup a simple project where i'm doing similar things , but loggly has done a lot more to make the same task real-time and distributed. you'll need to keep the write index as small as possible otherwise commit time will increase too great. loggly creates a new solr index every 5 minutes and includes this when searching using the distributed capabilities of solr ! they are merging the cores to keep the number of indices small, but this is not as simple as it sounds. watch this video to get some details about their work. 3.9. solandra = solr + cassandra solandra combines solr and the distributed database cassandra , which was created by facebook for its inbox search and then open sourced. at the moment solandra is not intended for production use. there are still some bugs and the distributed limitations of solr apply to solandra too. tthe developers are working very hard to make solandra better. jetwick can now run via solandra just by changing the solrconfig.xml. solandra also has the advantages of being real-time (no optimize, no commit!) and distributed without any major setup involved. the same is true for solr cloud. 3.10. category browsing via facets solr provides facets , which make it easy to show the user some useful filter options like those shown in the "drupal integration" example. like i described earlier , it is even possible to browse through a deep category tree. the main advantage here is that the categories depend on the query. this way the user can further filter the search results with this category tree provided by you. here is an example where this feature is implemented for one of the biggest second hand stores in germany. a click on 'schauspieler' shows its sub-items: other shops: game-change 3.11. jetwick - open twitter search you may have noticed that twitter is using lucene under the hood . twitter has a very extreme use case: over 1,000 tweets per second, over 12,000 queries per second, but the real-time latency is under 10 seconds! however, the relevancy at that volume is often not that good in my opinion. twitter search often contains a lot of duplicates and noise. reducing this was one reason i created jetwick in my spare time. i'm mentioning jetwick here because it makes extreme use of facets which provides all the filters to the user. facets are used for the rss-alike feature (saved searches), the various filters like language and retweet-count on the left, and to get trending terms and links on the right: to make jetwick more scalable i'll need to decide which of the following distribution options to choose: use solr cloud with zookeeper use solandra move from solr to elasticsearch which is also based on apache lucene other examples with a lot of facets: cnet reviews - product reviews. electronics reviews, computer reviews & more. shopper.com - compare prices and shop for computers, cell phones, digital cameras & more. zappos - shoes and clothing. manta.com - find companies. connect with customers. 3.12. plaxo - online address management plaxo.com , which is now owned by comcast, hosts web addresses for more than 40 million people and offers smart search through the addresses - with the help of solr. plaxo is trying to get the latest 'social' information of your contacts through blog posts, tweets, etc. plaxo also tries to reduce duplicates . 3.13. replace fast or google search several users report that they have migrated from a commercial search solution like fast or google search appliance (gsa) to solr (or lucene). the reasons for that migration are different: fast drops linux support and google can make integration problems. the main reason for me is that solr isn't a black box —you can tweak the source code, maintain old versions and fix your bugs more quickly! 3.14. search application prototyping with the help of the already integrated velocity plugin and the data import handler it is possible to create an application prototype for your search within a few hours. the next version of solr makes the use of velocity easier. the gui is available via http://localhost:port/solr/browse if you are a ruby on rails user, you can take a look into flare. to learn more about search application prototyping, check out this video introduction and take a look at these slides. 3.15. solr as a whitelist imagine you are the new google and you have a lot of different types of data to display e.g. 'news', 'video', 'music', 'maps', 'shopping' and much more. some of those types can only be retrieved from some legacy systems and you only want to show the most appropriated types based on your business logic . e.g. a query which contains 'new york' should result in the selection of results from 'maps', but 'new yorker' should prefer results from the 'shopping' type. with solr you can set up such a whitelist-index that will help to decide which type is more important for the search query. for example if you get more or more relevant results for the 'shopping' type then you should prefer results from this type. without the whitelist-index - i.e. having all data in separate indices or systems, would make it nearly impossible to compare the relevancy. the whitelist-index can be used as illustrated in the next steps. 1. query the whitelist-index, 2. decide which data types to display, 3. query the sub-systems and 4. display results from the selected types only. 3.16. future solr is also useful for scientific applications, such as a dna search systems. i believe solr can also be used for completely different alphabets so that you can query nucleotide sequences - instead of words - to get the matching genes and determine which organism the sequence occurs in, something similar to blast . another idea you could harness would be to build a very personalized search. every user can drag and drop their websites of choice and query them afterwards. for example, often i only need stackoverflow, some wikis and some mailing lists with the expected results, but normal web search engines (google, bing, etc.) give me results that are too cluttered. my final idea for a future solr-based app could be a lucene/solr implementation of desktop search. solr's facets would be especially handy to quickly filter different sources (files, folders, bookmarks, man pages, ...). it would be a great way to wade through those extra messy desktops. 4. summary the next time you think about a problem, think about solr! even if you don't know java and even if you know nothing about search: solr should be in your toolbox. solr doesn't only offer professional full text search, it could also add valuable features to your application. some of them i covered in this article, but i'm sure there are still some exciting possibilities waiting for you!

Maven profiles, like chainsaws, are a valuable tool, with whose power you can easily get carried away, wielding them upon problems to which they are unsuited. Whilst you're unlikely to sever a leg misusing Maven profiles, I thought it worthwhile to share some suggestions about when and when not to use them. These three best practices are all born from real-world mishaps: The build must pass when no profile has been activated Never use Use profiles to manage build-time variables, not run-time variables and not (with rare exceptions) alternative versions of your artifact I'll expand upon these recommendations in a moment. First, though, let's have a brief round-up of what Maven profiles are and do. Maven Profiles 101 A Maven profile is a sub-set of POM declarations that you can activate or disactivate according to some condition. When activated, they override the definitions in the corresponding standard tags of the POM. One way to activate a profile is to simply launch Maven with a -P flag followed by the desired profile name(s), but they can also be activated automatically according to a range of contextual conditions: JDK version, OS name and version, presence or absence of a specific file or property. The standard example is when you want certain declarations to take effect automatically under Windows and others under Linux. Almost all the tags that can be placed directly in a POM can also be enclosed within a tag. The easiest place to read up further about the basics is the Build Profiles chapter of Sonatype's Maven book. It's freely available, readable, and explains the motivation behind profiles: making the build portable across different environments. The build must pass when no profile has been activated (Thanks to for this observation.) Why? Good practice is to minimise the effort required to make a successful build. This isn't hard to achieve with Maven, and there's no excuse for a simple mvn clean package not to work. A maintainer coming to the project will not immediately know that profile wibblewibble has to be activated for the build to succeed. Don't make her waste time finding it out. How to achieve it It can be achieved simply by providing sensible defaults in the main POM sections, which will be overridden if a profile is activated. Never use Why not? This flag activates the profile if no other profile is activated. Consequently, it will fail to activate the profile if any other profile is activated. This seems like a simple rule which would be hard to misunderstand, but in fact it's surprisingly easy to be fooled by its behaviour. When you run a multimodule build, the activeByDefault flag will fail to operate when any profile is activated, even if the profile is not defined in the module where the activeByDefault flag occurs. (So if you've got a default profile in your persistence module, and a skinny war profile in your web module... when you build the whole project, activating the skinny war profile because you don't want JARs duplicated between WAR and EAR, you'll find your persistence layer is missing something.) activeByDefault automates profile activation, which is a good thing; activates implicitly, which is less good; and has unexpected behaviour, which is thoroughly bad. By all means activate your profiles automatically, but do it explicitly and automatically, with a clearly defined rule. How to avoid it There's another, less documented way to achieve what aims to achieve. You can activate a profile in the absence of some property: !foo.bar This will activate the profile "nofoobar" whenever the property foo.bar is not defined. Define that same property in some other profile: nofoobar will automatically become active whenever the other is not. This is admittedly more verbose than , but it's more powerful and, most importantly, surprise-free. Use profiles to adapt to build-time context, not run-time context, and not (with rare exceptions) to produce alternative versions of your artifact Profiles, in a nutshell, allow you to have multiple builds with a single POM. You can use this ability in two ways: Adapt the build to variable circumstances (developer's machine or CI server; with or without integration tests) whilst still producing the same final artifact, or Produce variant artifacts. We can further divide the second option into: structural variants, where the executable code in the variants is different, and variants which vary only in the value taken by some variable (such as a database connection parameter). If you need to vary the value of some variable at run-time, profiles are typically not the best way to achieve this. Producing structural variants is a rarer requirement -- it can happen if you need to target multiple platforms, such as JDK 1.4 and JDK 1.5 -- but it, too, is not recommended by the Maven people, and profiles are not the best way of achieving it. The most common case where profiles seem like a good solution is when you need different database connection parameters for development, test and production environments. It is tempting to meet this requirement by combining profiles with Maven's resource filtering capability to set variables in the deliverable artifact's configuration files (e.g. Spring context). This is a bad idea. Why? It's indirect: the point at which a variable's value is determined is far upstream from the point at which it takes effect. It makes work for the software's maintainers, who will need to retrace the chain of events in reverse It's error prone: when there are multiple variants of the same artifact floating around, it's easy to generate or use the wrong one by accident. You can only generate one of the variants per build, since the profiles are mutually exclusive. Therefore you will not be able to use the Maven release plugin if you need release versions of each variant (which you typically will). It's against Maven convention, which is to produce a single artifact per project (plus secondary artifacts such as documentation). It slows down feedback: changing the variable's value requires a rebuild. If you configured at run-time you would only need to restart the application (and perhaps not even that). One should always aim for rapid feedback. Profiles are there to help you ensure your project will build in a variety of environments: a Windows developer's machine and a CI server, for instance. They weren't intended to help you build variant artifacts from the same project, nor to inject run-time configuration into your project. How to achieve it If you need to get variable runtime configuration into your project, there are alternatives: Use JNDI for your database connections. Your project only contains the resource name of the datasource, which never changes. You configure the appropriate database parameters in the JNDI resource on the server. Use system properties: Spring, for example, will pick these up when attempting to resolve variables in its configuration. Define a standard mechanism for reading values from a configuration file that resides outside the project. For example, you could specify the path to a properties file in a system property. Structural variants are harder to achieve, and I confess I have no first-hand experience with them. I recommend you read this explanation of how to do them and why they're a bad idea, and if you still want to do them, take the option of multiple JAR plugin or assembly plugin executions, rather than profiles. At least that way, you'll be able to use the release plugin to generate all your artifacts in one build, rather than a single one at a time. Further reading Profiles chapter from the Sonatype Maven book. Deploying to multiple environments (prod, test, dev): Stackoverflow.com discussion; see the first and top-rated answer. Short of creating a specific project for the run-time configuration, you could simply use run-time parameters such as system properties. Creating multiple artifacts from one project: How to Create Two JARs from One Project (…and why you shouldn’t) by Tim O'Brien of Sonatype (the Maven people) Blog post explaining the same technique Maven best practices (not specifically about profiles): http://mindthegab.com/2010/10/21/boost-your-maven-build-with-best-practices/ http://blog.tallan.com/2010/09/16/maven-best-practices/ This article is a completely reworked version of a post from my blog.

Both approaches have some advantages and disadvantages and so it’s a question of the likelihood of particular problems and business requirements and priorities.

Parameterized types - the <> expressions that can be used in Java as of JDK 5 are not just for collections. I find myself frequently using them in APIs I design. They really do let you write things which are more generic in the non-Java sense of the word - and the result is more reusable code, which means less code overall, which means fewer bugs and things to test. The verbosity, and some of the weirdness of type-erasure are less than ideal, but used right, the benefits are worth the complexity. The standard (and somewhere recommended) naming convention for parameterized types is to use a single-letter name. That works fine in signatures that have only one such type. But in practice, single-letter names make code less self-describing, and if you're defining a class with more than one parameterized type, it can be confusing and hard to read. People other than me will have to call, understand and maintain my code - the more self-describing I can make it, the better. So I am looking for a naming convention that makes it obvious that something is a parameterized type, but allows for descriptive names. I am wondering if anybody else has run into this problem, and if there is any emerging consensus on naming generics. Do you work on a project that uses generics a lot? If so, what do you do? Here's an example. At the moment, I'm writing a generic (in both senses) class which simply limits the number of threads which can access some resource. It's basically a wrapper around a Semaphore which uses a Runnable-like object to ensure that the Semaphore is accessed correctly, and does some non-blocking statistic gathering about thread contention. So to access the scarce resource, you pass in a ResourceAccessor: public interface ResourceAccessor { public Result run (ProtectedResource resource, Argument argument); } The problem is that, when somebody looks at this interface, they will instantly get the idea that there are really classes they need to go find, which are called ProtectedResource, Argument and Result - and of course, no such classes exist - these are just names for generic types. The standard-naming-convention is worse: public interface ResourceAccessor { public S run (T resource, R argument); } Here, nobody could possibly figure out what on earth this class is for without extensive documentation - this is a really horrible idea. So I've concluded that the standard recommendations for generic type names are simply wrong for any non-trivial usage (I.e. Collection is fine, since there is one type and Collections are well-understood). You simply can't do this on a non-collection code structure you have invented, or people will just be confused and not use it. The best suggestion I've heard thus far is using $ as a prefix: public interface ResourceAccessor <$ProtectedResource, $Argument, $Result> { public $Result run ($ProtectedResource resource, $Argument argument); } I don't find this pretty, but I don't have any better ideas, and at least it makes it crystal-clear that there is something different about these names. Any thoughts? What do you do in this situation?

This post will walk through how to resize an ExtJS Panel, Grid, Component on Window Resize without using Ext.Viewport. Problem: You have a legacy page and you want to change an html grid for an ExtJS DataGrid, because it has so many cool features. Or you have a page with some design and you are going to use only one ExtJS Component. In both cases, you also want to render your ExtJS Component to a specific DIV. Also, you want you component to be resized in case you resize the browser window. How can you do that if resize a single component in an HTML page it is not the default behavior of an ExtJS Component (except if you use Ext.Viewport)? Solution: Condor (from ExtJS Community Support Team) developed a plugin that can do that for you. I had to spend some time to understand how the plugin works, and I finally got it working as I wanted. Well, I recommend you to spend some time reading this thread: http://www.sencha.com/forum/showthread.php?28318 (if you have any issues or questions, please publish it on the thread, so other members can give you the support you need). Requirements to make the plugin work: Your have to apply the following style to the DIV (the width is up to you, the other styles are mandatory, otherwise it will not work): If you have any border around your ExtJS component, you have to set a HEIGHT. And you will also have to set a height to your ExtJS component. In this case, autoHeight will not work. If you DO NOT have any border or other design on the ExtJS component side, you do not need to set height and you can use autoHeight. In my case, I put a border on the external DIV, so I have to set Height: HTML code (all DIVs): And you need to add the plugin to the component (In this case, I’m using an ExtJS DataGrid): var grid = new Ext.grid.GridPanel({ store: store, columns: [ {header: 'Company', width: 160, sortable: true, dataIndex: 'company'}, {header: 'Price', width: 75, sortable: true, renderer: 'usMoney', dataIndex: 'price'}, {header: 'Change', width: 75, sortable: true, renderer: change, dataIndex: 'change'}, {header: '% Change', width: 75, sortable: true, renderer: pctChange, dataIndex: 'pctChange'}, {header: 'Last Updated', width: 85, sortable: true, renderer: Ext.util.Format.dateRenderer('m/d/Y'), dataIndex: 'lastChange'} ], stripeRows: true, autoExpandColumn: 'company', height: 490, autoWidth:true, title: 'Array Grid', // config options for stateful behavior stateful: true, stateId: 'grid' ,viewConfig:{forceFit:true} ,renderTo: 'reportTabContent' // render the grid to the specified div in the page ,plugins: [new Ext.ux.FitToParent("reportTabContent")] }); And done! Now you can resize the browser and the component will resize itself! I tested it on Firefox, Chrome and IE6. You can download my sample project from my GitHub: http://github.com/loiane/extjs-fit-to-parent PS.: If you want to use the full browser window, use a Viewport. Happy coding!

This tutorial outlines steps required to install Nexus (Maven Repository Manager) under Tomcat, or another webapp container. It shows you practical configuration and includes code snippets that go in your pom.xml and settings.xml in order to read and publish artifacts to your Nexus server. Step 1: Download Download Nexus from here (at the time of writing, latest is 1.6.0) Step 2: Install Copy the war to TOMCAT_HOME/webapps/nexus.war Though not required, it is a generally good idea to restart tomcat after installing a new war /etc/init.d/tomcat restart /etc/init.d/tomcat restart Step 3: Configure security a) Change default admin password: The default admin username/password is admin/admin123. Login as admin and change the password to a secure password. Login -> [admin, admin123] -> Left Menu -> Security -> Change Password -> click “Change Password” b) Anonymous Access: By default Nexus is open to the public. If you want to secure access to nexus, disable ‘Nexus anonymous user’ Admin -> Left Menu -> Users -> ‘Nexus anonymous user’ -> Status=Disabled c) Deployment user: Change password for deployment user Admin -> Left menu -> Users -> Deployment user -> Change email address Admin -> Left menu -> Users -> Right click on ‘Deployment user’ in the user list -> Set Password -> click ‘Set password’ to finish Step 4: Set SMTP server It is a good idea to configure SMTP server, so that you can receive emails from Nexus. Admin login -> Left menu -> Administration -> Server ->SMTP Settings -> (host localhost, port 25, no login, no password mostly works on a linux machine) Step 5: Change Base Url If you are running Nexus behind Apache using mod_jk or mod_proxy, change your base url here. Admin login -> Left menu -> Administration -> Server -> Application Server Settings -> Base url Step 6: Add a task to periodically remove old snapshots If you or your CI server publishes snapshots to Nexus several times a day, then you should consider adding a task to delete duplicate/old snapshots for the same GAV (group, artifact, version). If you don’t do this, you will notice that the Nexus disk usage will increase with time. Admin login -> Left menu -> Administration -> Scheduled tasks -> Add… -> name=”Remove old snapshots”, Repository/Group=Snapshots (Repo), Minimum Snapshot Count=1, Snapshot Retention(days)=3, Recurrence=Daily, Recurring time=2:00 -> click ‘Save’ Step 7: Using Nexus: reading and publishing artifacts If you want to deploy your artifacts to your Nexus, you need to configure 2 files: pom.xml and settings.xml a) pom.xml – for each project which wishes to publish to Nexus, add your repo to the pom.xml vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/releases vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/snapshots vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/releases vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/snapshots vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true b) settings.xml – If you have disabled anonymous access to Nexus, add the deployment password to your ~/.m2/repository/settings.xml file vineetmanohar-nexus deployment password_goes_here From http://www.vineetmanohar.com/2010/06/getting-started-with-nexus-maven-repo-manager

FlexMonkey is a free and open source Adobe AIR application used for testing Flex and AIR based applications. It can record, playback, and verify Flex UI interactions. FlexMonkey also generates ActionScript-based testing scripts that you can easily include within a continuous integration environment. Gorilla Logic is the company that builds FlexMonkey, and its CEO, Stuart Stern, recently spoke with DZone about their launch of FlexMonkey 4, which supports all of the new Spark components in Flex 4. For more info on FlexMonkey, see our interview with Stuart Stern at Adobe Max 2009. DZone: First thing's first. What's new in FlexMonkey 4? Stuart Stern: Before we talk about the updates to FlexMonkey, let me give you a bit of background for those who have not used any of the previous versions. We (Gorilla Logic) built and open sourced the first version of FlexMonkey in late 2008 because we needed a serious Flex testing solution for our enterprise customers. Basically, FlexMonkey allows developers and QA people to create comprehensive tests for their Flex applications by easily recording real interactions with the user interface, and by letting the test creator add verification checks on both data and visual snapshots of the UI. Once the interactions have been recorded the test can be played back through the FlexMonkey console or through generated test code in Fluint / FlexUnit. The generated code can be extended to create complex, data-driven test scenarios, and can be easily run within build and continuous integration environments. In our software consulting engagements, we have found that FlexMonkey reduces the overall numbers of tests that developers need to create, since driving testing from the user interface can exercise the entire application stack, top-to-bottom and even front-to-back. Let’s be clear though, api-level testing and tools like FlexUnit are still an essential part of Flex development, especially in testing non ui components. Where FlexMonkey is a better fit for testing is around visual components, which are difficult, if not impossible, to test as a ‘unit.’ On our typical applications, we tend to end up with about 80% of our developer created tests constructed through FlexMonkey, with the other 20% being created as more traditional unit tests. As far as FlexMonkey 4, the goals were pretty simple; the community has been beating down our door for Spark Component (Flex 4) support. So, we’ve added full support for the new component library recently released by Adobe. This is key for enterprise Flex development projects that have come to depend on FlexMonkey for regression and QA testing, and that are ready to move to Flex 4. We've also simplified the setup for FlexMonkey 4, so it's easier for new users to get up and running quickly. DZone: What were some of the difficulties in implementing support for all of Flex 4's Spark components? Stuart: From a FlexMonkey perspective, there is no difference between Spark and Halo components. However, one of the things that makes FlexMonkey so powerful is that it records "semantic" events such as "open combobox" rather than "click at this screen coordinate". So FlexMonkey needs to "understand" every Flex component, and we had to tell it some new things about the new Spark components and their events. . DZone: Are there any trends your seeing in how developers are using FlexMonkey in their UI design workflow? Stuart: FlexMonkey was initially envisioned as a tool for developers. Because developers test code that is still under development, it is important for a test automation tool to be able to express tests in a largely logical fashion. Tests that are too tied to the precise look of a screen at a particular point in time are two brittle for use by developers. FlexMonkey tests are typically robust across application skinning, since tests can be written independent of the exact positions or styling of the components on the screen, and can pinpoint specific functionality. In this way developers can automate testing of portions of an application even before the UI design is fully finalized. Although we designed it for developer testing, it's ability to record tests automatically, add verification logic by pointing and clicking, and do fuzzy bitmap comparisons on select portions of the screen, make FlexMonkey highly effective for QA testing purposes as well. Additionally, when developers and testers use the same tools, they can share some of the same tests, with QA using developer tests as a starting point, and developers incorporating some QA tests into continuous integration builds. DZone: Tell me about the next tool you'll be focusing on: FlexMonkium. Stuart: FlexMonkium is a plugin for Selenium IDE and Selenium RC. It adds FlexMonkey recording and playback capability to Selenium so you can create tests for applications that mix HTML and Flex. We recently completed development and are now doing final testing and documentation. We expect it make it publicly available any day now. FlexMonkium makes all of FlexMonkey's functionality available within the Selenium IDE, and generates JUnit-based tests that can be run with Selenium RC. DZone: Are there any interesting or exciting things you see down the road for the Flash platform ecosystem? How do you think the platform will fare against emerging UI design technologies like HTML5, CSS3, etc.? Stuart: The recent attacks on the Flash platform by Apple have certainly put the ‘HTML 5 vs. Flash’ battle on everyone’s radar. At Gorilla, we build both native browser applications (HTML 5, etc.) and Flex applications -- and even native iPhone applications -- for our customers. There are pros and cons to each and situations that definitively call for one versus another. Having said that, we are a consulting company that builds serious enterprise software. We embrace Flex because it enables us to do things we cannot do otherwise, and do them quickly. On any given project, we don't ask if we should use Flex, we ask if there is any reason why we can't.

It's amazing how the JBoss Team put together an easy way to do JMS Clustering, out of the box!!. I'll start with an easy example, creating a Queue named "MyClusteredQueue". In this example I'm using JBoss AS 5.1. and two computers connected on the same network, with these IP's: - Computer A: 192.168.0.143 - Computer B: 192.168.0.210 So, here are the steps: 1) Install the JBoss on both computers. We are going to use the "all" configuration for both computers. 2) We create our Queue on both servers. Go to $JBOSS_HOME/server/all/deploy/messaging/ and edit the destinations-service.xml file. Add the MyClusteredQueue before the last server tag. It looks like this: jboss.messaging:service=ServerPeer jboss.messaging:service=PostOffice true This is how you add a Queue to the JBoss, and the people how are familiar with this, the only new thing is to add the attribute "Clustered". This step must be set on both computers. At the end of the article you can find the files. 3) Write the MDB to consume the messages, and deploy it on the two computers. (I'm using an EJB 3 - MDB style). import java.net.InetAddress; import javax.ejb.ActivationConfigProperty; import javax.ejb.MessageDriven; import javax.jms.Message; import javax.jms.MessageListener; import javax.jms.ObjectMessage; import org.apache.log4j.Logger; /** * @author felipeg * */ @MessageDriven(activationConfig = { @ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue"), @ActivationConfigProperty(propertyName="destination", propertyValue="queue/MyClusteredQueue") }) public class JMSClusterClientHandler implements MessageListener { Logger log = Logger.getLogger(JMSClusterClientHandler.class); @Override public void onMessage(Message message) { try{ if (message instanceof ObjectMessage) { InetAddress addr = InetAddress.getLocalHost(); log.info("########## Processing Host: " + addr.getHostName() + " ##########" ); ObjectMessage objMessage = (ObjectMessage) message; Object obj = objMessage.getObject(); log.info("Object received:" + obj.toString()); } } catch (Exception e) { e.printStackTrace(); } } } 4) Start the jboss with the following options: Computer A: $ cd $JBOSS_HOME/bin $ ./run.sh -c all -b 192.168.0.143 -Djboss.messaging.ServerPeerID=1 Computer B: $ cd $JBOSS_HOME/bin $ ./run.sh -c all -b 192.168.0.210 -Djboss.messaging.ServerPeerID=2 It is necesary to give an ID to each server and this is accomplished with this directive: -Djboss.messaging.ServerPeerID When you start the jboss on computer A, you should see the logs (server.log) telling you that there is one node ready and listening, and once you start the jboss on computer B, on the log will appear the two nodes, the two IP's ready to consume messages. 5) Now it's time to send a Message to the Queue. To accomplish this it's necessary to change the connection factory to "ClusteredConnectionFactory" (JMSDispatcher.java - See the code below). Also on the jndi.properties (if you are using the default InitialContext) file it's necessary to add the two computers ip's separated by comma to the java.naming.provider.url property. (In my case a create a Properties variable and I set all the necessary properties, JMSDispatcher.java - see the code below). java.naming.provider.url=192.168.0.143:1099,192.168.0.210:1099 The client that I wrote is a web application, that consist in one index.jsp page, which contains a form that prompts you for the name of the queue, the type of messaging (Queue or Topic), the server ip and port, how many times it will send the message and the actual message to be sent; also the web application has a Servlet (JMSClusteredClient.java - see code below) that receives the postback and helper class (JMSDispatcher.java - see code below) that sends the message to the jboss servers. You can to deploy it in any computer. In my case I deployed it on the Computer A. And you can access it through this URL: http://192.168.0.143:8080/JMSWeb/ (just modify the IP where the client war was deployed). If you notice (on the index.jsp - code below) I've already put some default values that reflects the name of the Queue, and the IP's of my two computers. Now, If you increment the number of times that the message will be sent (maybe a 10) and fill out the message box, and click "Send" you should see on the two servers some of the messages being consumed by the MDB. Here are the Files to create the client: index.jsp JMS Clustered - Test Client Server: QueueTopic Times:Message: Servlet: JMSClusteredClient.java public class JMSClusteredClient extends HttpServlet { private static final long serialVersionUID = 1L; /** * @see HttpServlet#service(HttpServletRequest request, HttpServletResponse response) */ protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { PrintWriter out = response.getWriter(); String topicqueue = request.getParameter("topicqueue"); String message = request.getParameter("message"); String server = request.getParameter("server"); String messageType = request.getParameter("messageType"); String times = request.getParameter("times"); int intTimes = Integer.parseInt(times); JMSDispatcher dispatcher = new JMSDispatcher(); dispatcher.setTopicQueueName(topicqueue); dispatcher.setServer(server); dispatcher.setMessageType(messageType); try { for(int count =1; count <= intTimes;count++){ dispatcher.sendMessage( count + " of " + times + " " + message); } out.println("Message [" + message + "] sent successfully to [" + topic + "] to the [" + server + "] server " + times + " times."); } catch (JMSException e) { e.printStackTrace(); out.println("Error:" + e.getMessage()); } catch (NamingException e) { out.println("Error:" + e.getMessage()); e.printStackTrace(); } finally{ out.close(); } } } A utility to send the messages: JMSDispatcher.java public class JMSDispatcher { /** * */ private static final long serialVersionUID = 7105145023422143880L; private static Logger log = Logger.getLogger(JMSDispatcher.class); private final String CONNECTION_FACTORY_CLUSTERED = "ClusteredConnectionFactory"; private final String CONNECTION_FACTORY = "ConnectionFactory"; private final String TOPIC = "TOPIC"; private final String QUEUE = "QUEUE"; private String topicQueueName; private String server; private String messageType; public void setTopicQueueName(String value){ this.topicQueueName = value; } public void setServer(String value){ this.server = value; } public void setMessageType(String value){ this.messageType = value; } public void sendMessage(Object objectMessage) throws JMSException, NamingException{ log.debug("##### Setting up a Queue/Topic Message: #####"); if (TOPIC.equals(messageType)){ sendTopicMessage(objectMessage); } else if (QUEUE.equals(messageType)){ sendQueueMessage(objectMessage); } log.debug("##### Publishing Message: Done #####"); } private void sendQueueMessage(Object objectMessage) throws JMSException, NamingException{ try{ InitialContext initialContext = getInitialContext(); QueueConnectionFactory qcf = (QueueConnectionFactory) initialContext.lookup(CONNECTION_FACTORY_CLUSTERED); QueueConnection queueConn = qcf.createQueueConnection(); Queue queue = (Queue) initialContext.lookup(topicQueueName); QueueSession queueSession = queueConn.createQueueSession(false, Session.AUTO_ACKNOWLEDGE); queueConn.start(); QueueSender send = queueSession.createSender(queue); ObjectMessage om = queueSession.createObjectMessage((Serializable)objectMessage); setMessageProperties(om); log.debug("##### Publishing Message to a Queue: " + queueName + "#####"); send.send(om); send.close(); queueConn.stop(); queueSession.close(); queueConn.close(); }catch(MessageFormatException ex){ log.error("##### The MESSAGE is not Serializable ####"); throw ex; }catch(MessageNotWriteableException ex){ log.error("##### The MESSAGE is not Readable ####"); throw ex; }catch(JMSException ex){ log.error("##### JMS provider fails to set the object due to some internal error. ####"); throw ex; } } private void sendTopicMessage(Object objectMessage) throws JMSException, NamingException{ try{ InitialContext initialContext = getInitialContext(); TopicConnectionFactory tcf = (TopicConnectionFactory)initialContext.lookup(CONNECTION_FACTORY_CLUSTERED); TopicConnection topicConn = tcf.createTopicConnection(); Topic topic = (Topic) initialContext.lookup(topicQueueName); TopicSession topicSession = topicConn.createTopicSession(false,TopicSession.AUTO_ACKNOWLEDGE); topicConn.start(); TopicPublisher send = topicSession.createPublisher(topic); ObjectMessage om = topicSession.createObjectMessage(); om.setObject((Serializable)objectMessage); setMessageProperties(om); log.debug("##### Publishing Message to a Topic: " + topicName + "#####"); send.publish(om); send.close(); topicConn.stop(); topicSession.close(); topicConn.close(); }catch(MessageFormatException ex){ log.error("##### The MESSAGE is not Serializable ####"); throw ex; }catch(MessageNotWriteableException ex){ log.error("##### The MESSAGE is not Readable ####"); throw ex; }catch(JMSException ex){ log.error("##### JMS provider fails to set the object due to some internal error. ####"); throw ex; } } private InitialContext getInitialContext() throws NamingException{ Properties jboss = new Properties(); jboss.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory"); jboss.put("java.naming.factory.url.pkgs", "org.jboss.naming:org.jnp.interfaces"); jboss.put("java.naming.provider.url", server); return new InitialContext(jboss); } } And the web.xml JMSWeb index.jsp JMSClusteredClient JMSClusteredClient com.blogspot.felipeg48.jms.web.JMSClusteredClient JMSClusteredClient /JMSClusteredClient Happy Clustering!!

The purpose of this article is to give you the details of our 100 node cluster demo. This demo is recorded and you can watch the 5 minute screencast Hazelcast is an open source clustering and highly scalable data distribution platform for Java. JVMs that are running Hazelcast will dynamically cluster and allow you to easily share and partition your application data across the cluster. Hazelcast is a peer-to-peer solution (there is no master node, every node is a peer) so there is no single point of failure. Communication among cluster members is always TCP/IP with Java NIO beauty. The default configuration comes with 1 backup so if a node fails, no data will be lost (you can specify the backup count). It is as simple as using java.util.{Map, Queue, Set, List}. Just add the hazelcast.jar into your classpath and start coding. When you download the Hazelcast, you will find a test.sh under bin directory. The test.sh runs an application which randomly makes 40% get, 40% put and 20% remove on a distributed map. In this demo the same test application will be used to see how it performs on 100 node cluster. Amazon EC2 and S3 An easy to use and scalable cloud environment was needed for demo so we decided to use Amazon EC2 for server instances (nodes) and S3 service to store demo application zip and configuration files. With its newly announced Java SDK, it is very simple to start/stop server instances and upload files to S3 programatically. Hazelcast AMI & Launcher The challenge here is that we are running an application on 100 nodes and dealing with each and every server in the cluster is a huge task. We don't want to ssh into every server and manually start the application. This part is automated by creating a special server image (AMI). The AMI contains Java Runtime and a launcher application we developed, which will download the demo application from Amazon S3, unzip it, and run the hazelcast/bin/test.sh in it. The Launcher is actually so generic that it can run any application; it doesn't care/know what test.sh contains. Deployer Deployment of the demo application is also automated so that we don't need to login into AWS Management Console and manually start instances. Deployer instantiates any number of Amazon EC2 servers with any AMI and also uploads the demo application zip file to S3. So the idea here is that, the Deployer will store the application into S3 and launch 100 EC2 instances with our image. The Launcher on each instance will download the application from S3 and run it. Demo Details. The smallest EC2 instances (m1.small) are used to run the demo. These are the virtual instances with CPU about 1.0 GHz. Also keep in mind that EC2 platform suffers from considerable amount of network latency. That's why we increased the thread count to 250 in our application. The following steps performed during the demo Download hazelcast-1.8.3.zip from www.hazelcast.com. Unzip the file and move the monitoring war file into tomcat6/webapps directory. Edit the test.sh under the bin directory: Add -Xmx1G -Xms1G Add -Dhazelcast.initial.wait.seconds=100 to make the cluster evenly partition on start so that migration can be avoided for better performance. Add t250 as an argument to the application to set thread count to 250. Remember the latency issue. Run the Deployer from IDE. Check from EC2 Management Console if 100 servers started. Start tomcat. Copy the public DNS name of one of the servers to connect to from monitoring tool. Go to http://localhost:8080/hazelcast-monitor-1.8.3/ (Hazelcast Monitoring Tool). Paste the address and connect to the cluster. Enjoy! Results You should always look for programatic ways of launching applications on the cloud. With these tools we were able to deploy and run the demo application on 100 servers in minutes. The entire Hazelcast cluster was making over 400,000 operations per second on the smallest EC2 instances. In our next demo we will experiment Hazelcast on large data set and even bigger cluster. Watch the screencast

A pipeline in software context is a very well-known architectural style in which a process consists of a series of steps to be followed in order to proceed the data, and the output of one step is the input of another step. This is also called the Pipes and Filters design pattern. The naming comes from the physical pipeline as this architectural style is very similar to a pipeline in which a stream of data comes in and leaves after being processed. The original idea of pipeline in software is implemented in Unix. This pattern is used in many places. Compiler pipeline, ASP.NET HTTP Pipeline, and workflows are three of many examples that I can mention. The pipes and filters style is implemented in various platforms with different techniques and technologies. Recently I was in a situation to implement this pattern and did some research to find more about the possible options to implement this pattern in the .NET Framework. Doing my research, I found many approaches introduced by community members but the most mature technique is the one that Oren Eini has described in his blog post using Generics. There is also an interesting technique described by Jeremy Likness using the yield keyword in C#. In this post I’m going to apply Oren’s approach and expand it to write a simple implementation of the classic KWIC example in Software Engineering. I liked Oren’s code because as he said, it’s comparatively simpler than other solutions introduced for this problem in the .NET Framework. An Overview of KWIC KWIC stands for Key Word in Context and is a classic problem in Software Engineering papers in which you try to create an index of words by sorting and aligning each word in a piece of text. David Parnas has a famous paper on modularity that uses KWIC as an example. There are some basic and advanced implementations of KWIC in different platforms but the main steps are: Reading the input Shifting the words in each line to get a new permutation Sorting the results Writing the output Interestingly, in this case the output of each step is the input of the next step which makes this a good candidate for the Pipes and Filters pattern. Implement the Pipes and Filters Pattern with Generics Oren’s technique for implementing the Pipes and Filters in the .NET Framework is based on a Generic interface and a Generic class. The Generic interface simulates the filter and the Generic class simulates the pipeline. The IOperation interface has a single method called Execute that is the implementation of the filter logic. Each filter should implement this interface. using System.Collections.Generic;namespace KwicPipesFilters{ public interface IOperation { IEnumerable Execute(IEnumerable input); } The use of a generic IEnumerable is a good choice because it leaves a lot of space for the developers to plug in any type that they want and use various types for their filters. The Pipeline class has an Execute and a Register method. Using the Register method, you add different filters to the pipeline and using the Execute method, you start processing the item in all the registered filters. using System.Collections.Generic;namespace KwicPipesFilters{ public class Pipeline { private readonly List> operations = new List>(); public Pipeline Register(IOperation operation) { operations.Add(operation); return this; } public void Execute() { IEnumerable current = new List(); foreach (IOperation operation in operations) { current = operation.Execute(current); } IEnumerator enumerator = current.GetEnumerator(); while (enumerator.MoveNext()); } } The implementation of the Pipeline class is straightforward: it keeps a list of filters and provides a Register function that lets you add new filters to your pipeline, and then use the Execute method to execute all the filters in the list to process an input. Reader The Reader filter reads the input text from a file and returns an IEnumerable list of lines. Of course, for the first filter in the pipe we don’t care about the input as the input is read inside the filter itself. using System;using System.Collections.Generic;using System.IO;namespace KwicPipesFilters{ public class Reader : IOperation { public IEnumerable Execute(IEnumerable input) { Console.Title = "Pipes and Filters Pattern in .NET"; Console.WriteLine("Enter the path of the file:"); return File.ReadLines(Console.ReadLine()); } } Shifter The Shifter filter is where the main logic of the KWIC application is implemented. It shifts the words in each line to find all the possible permutations suitable for the index. using System.Collections.Generic;namespace KwicPipesFilters{ public class Shifter : IOperation { public IEnumerable Execute(IEnumerable input) { List shifts = new List(); foreach (string line in input) { string[] words = line.Split(new char[] { ' ' }); for (int i = 0; i <= words.Length - 1; i++) { shifts.Add(string.Join(" ", words)); string firstWord = words[0]; for (int j = 1; j <= words.Length - 1; j++) { words.SetValue(words[j], j - 1); } words.SetValue(firstWord, words.Length - 1); } } return shifts; } } Here we have a basic implementation of the Shifter filter where we split the line into separate words based on the space between them, then shift all the words to find various permutations. Sorter Before returning the final results in the Writer filter, we need to sort the index alphabetically. This is done in the Sorter filter. using System.Collections.Generic;using System.Linq;namespace KwicPipesFilters{ public class Sorter : IOperation { public IEnumerable Execute(IEnumerable input) { LineComparer lineComparer = new LineComparer(); input.ToList().Sort(lineComparer); return input; } } Here I used a LineComparer class to implement the ICcomparer interface for the string type. using System.Collections.Generic;namespace KwicPipesFilters{ public class LineComparer : IComparer { public int Compare(string x, string y) { return string.Compare(x, y); } } Writer Obviously, the last filter should write the index to the output for the user and that’s the purpose of the Writer filter. using System;using System.Collections.Generic;namespace KwicPipesFilters{ public class Writer : IOperation { public IEnumerable Execute(IEnumerable input) { foreach (string line in input) { Console.WriteLine(); Console.WriteLine(line); } Console.ReadLine(); yield break; } } As you see, this filter uses a yield break to avoid returning any result. Pipeline Having all the filter implemented, I also need to implement the pipeline itself in order to register the filters and make the whole thing work. I do this in my KwicPipeline class with a simple code that it has. namespace KwicPipesFilters{ public class KwicPipeline : Pipeline { public KwicPipeline() { Register(new Reader()); Register(new Shifter()); Register(new Sorter()); Register(new Writer()); } } I inherit from the Pipeline class and register my filters in the public constructor. Putting It Together There is only one step remained and that is putting all these things together to start the pipeline. All I need to do is to create an instance of the KwicPipeline class, call its Execute method, and leave the rest to my pipes and filters. namespace KwicPipesFilters{ class Program { static void Main(string[] args) { KwicPipeline pipeline = new KwicPipeline(); pipeline.Execute(); } } Conclusion In this post I implemented the Pipes and Filters pattern in the .NET Framework using a simple and generalized technique that relies on Generics to implement the KWIC application. In my opinion this is one of the best ways to implement this pattern in the .NET Framework. I have uploaded the sample source code package here. Note that the solution is created using Visual Studio 2010 RC1. There are other techniques to implement this pattern in .NET and one specific technique that I have in mind is using the Windows Workflow Foundation. I may work more on this idea and write about it later.

There are at least four methods that can be used in different combinations to make the process of setting up a complete development environment a lot less painful.

The emergence of version control systems makes both promiscuous and continuous integration merging techniques more attractive. Which is better?

My goal is to compare Sonatype Nexus and JFrog Artifactory,the two leading open source Maven repository managers.

This is the second part of a two-part article introducing Jeff De Luca’s Feature Driven Development (FDD) process. In particular, we are looking at how FDD differs from Scrum and eXtreme Programming-inspired approaches when it comes to working with larger teams and projects. In the first part we briefly introduced the ‘just enough’ upfront activities that FDD uses to support the additional communication that inevitably is needed in a larger project/team. In the second part of the article we cover how FDD leverages the results of those upfront activities within the highly iterative, self-managing, organized-chaos that is the delivery engine room of an FDD project. The Engine Room: Delivering Frequent, Tangible Working Results Once there is an initial overall model (FDD Process #1), an initial overall features list (FDD Process #2), and an initial overall plan (FDD Process #3) in place, an FDD project is ready to start delivering the required software feature by feature. Peter Coad, the Chief Architect on the original FDD project used the phrase ‘Deliver frequent, tangible, working results’ as a mantra to impress upon people the idea of delivering real, completed, client-valued function as often as possible. Scrum and eXtreme Programming do this using fixed length iterations of a calendar month or 2-4 weeks. FDD is different. Each Chief Programmer (lead developer) runs a series of iterations, each of which is normally a matter of a few days, and never longer than two weeks. At the start of each of these iterations, each Chief Programmer selects the next few features that make sense to implement from the backlog of feature sets (activities) that were assigned to him or her in FDD Process #2. The Chief Programmer leads the development of these features through FDD processes #4 and #5, Design by Feature (DBF) and Build By Feature (BBF). Note that iterations through the DBF/BBF processes are not fixed length, and Chief Programmers do not synchronize the start and end of their iterations with each other. In addition, the DBF/BBF processes are always executed as a pair (FDD describes them as two separate processes rather than one combined process for psychological reasons). FDD Process #4: Design By Feature After selecting the features for the iteration, a Chief Programmer needs to form their feature team. Yes, feature teams are formed and disbanded for each iteration through the DBF/BBF process pair. Using the knowledge gained from the modeling process (FDD Process #1), the Chief Programmer identifies the domain classes that are likely to be involved in this iteration, and forms his or her feature team from the owners of those classes. In practice, this means: a feature team is small, typically 3 to 5 people, because features are small. By definition, a feature team comprises of all the class owners who need to modify their classes in the development of the features during that iteration. There is no need to wait for members of other teams to change code. Therefore, there are all the benefits of code ownership and a sense of collective ownership too. Class owners may find themselves a member of multiple feature teams at the same time. This does not happen as frequently as might be supposed because iterations are so short – days not weeks. When it does, it is not a big problem in practice. Chief Programmers work together to resolve any problematic conflicts and, with care, most developers can manage the demands of occasionally belonging to more than one feature team for a short time. Once formed, the Chief Programmer facilitates the collaborative analysis and design of the features for that iteration. Depending on the complexity, this may involve the team walking through the requirements in detail with a domain expert, and studying any existing relevant documents. It also involves agreeing on the interactions and other details that need to be added to the model to support the new features. The final step in the DBF part of the iteration is to review the design. For simple features, this may be a brief sanity check of the design held within the feature team. For more significant features, the Chief Programmer will typically involve other Chief Programmers or class owners so that they are aware and can comment on the impact of the proposed design. For small team projects, the object models are frequently small enough for individual or pairs of developers to create good designs while writing tests for a particular feature or user story. For larger projects, this is not necessarily the case and designs created purely by considering the tests a feature or user story must pass are more likely to be brittle and require significant refactoring. The DBF process in FDD ensures that the overall model also guides the design, helping to maintain its ‘conceptual integrity’ [Brooks]. FDD Process #5: Build By Feature The Build by Feature (BBF) part of the iteration involves the team members coding up the features, testing them at both unit level and feature level, and holding a code inspection before promoting the completed features into the project's regular build process. Testing FDD expects developers to unit test their code. It expects feature teams to test their features. FDD is not overly concerned with how this is achieved. Projects and feature teams are free to adopt the testing tools, frameworks, and level of formality and completeness that are most appropriate. FDD does not mind if tests are written before or after code. What FDD mandates, is that the feature team deliver code that has been appropriately tested and inspected. Only once the new features have passed testing and inspection is the source code allowed into the build process. Code Inspections Most people want to know why FDD mandates code inspections, especially those that have endured sitting through hours of boring, unproductive, ego-polishing/demolishing, point-scoring sessions that formed so-called code reviews, inspections or walkthroughs. The reason FDD mandates code inspections is that research has shown time and again that when done well, inspections find more defects and different kinds of defects than testing [McConnell]. Not only that but by examining the code of the more experienced, knowledgeable developers on the team and having them explain the idioms they use, less experienced developers learn better coding techniques. In addition, knowing that their code will be inspected and not be allowed in the build unless it conforms to the agreed standards encourages developers to pay more attention to conforming to those standards. One of the benefits of working in feature teams is that the whole feature team is on the hot seat during an inspection, not just one individual. This removes much of the intensity and anxiety inherent in inspecting one individuals work. The Chief Programmer decides on the level of formality of each inspection depending on the complexity and impact of the features developed in that iteration. Where the code has little or no impact outside the feature team, an inspection will usually only involve the feature team inspecting each other’s work. Where there is significant impact the Chief Programmer pulls in other Chief Programmers and developers to both verify the code and communicate the impact of the new features. eXtreme Programming acknowledges inspections as a ‘best practice' but promotes pair programming as the logical conclusion of applying this practice. Pair programming is obviously better than individual developers delivering code without any form of inspection. However, while FDD neither mandates nor forbids pair programming, a more-traditional inspection is: fresh eyes looking at the code, catching bad assumptions made by the coder/s a Chief Programmer present to ensure the techniques passed on are good. After all, developers can just as easily teach each other bad habits as well as good habits. a change of pace for developers, a chance to step away from the keyboard and mouse for a short while. With the wide availability of automated source code formatting and static analysis tools, code inspections can now be shorter, concentrating on the logic and coding idioms involved and not getting bogged down in nit-picking such as alignment of braces, etc. The Build FDD assumes some sort of regular build process. Some teams build weekly, others daily and others continuously. FDD avoids mandating any particular build regime. This enables the project team to apply the most applicable. If a continuous integration environment makes sense, then the team is free to employ the best there is. Progress Reports Agile projects like highly visible progress information. FDD projects are no exception. In fact, because larger projects frequently have higher profiles within an organization, presenting meaningful, accurate, timely project information appropriately at the different levels of leadership/management is even more important. Conventionally, FDD projects track the development of each feature through its DBF/BBF iteration against six milestones: domain walkthrough, design, design inspection, coding, testing and inspection, and promoted to build. For each feature, Chief Programmers record the actual date a milestone is reached. Tracking each feature through these six milestones enables the project to keep an eye on how much work is 'in progress'. Too many features at a particular milestone indicate a process problem. Those promoting Kanban and other Limited Work In Progress methods have formalized this idea to strictly define what is meant by 'too many' for each of their development iteration milestones/statuses. They then refuse to move an item to a new milestone/status if the limit on the number of items at that status has been reached. This forces a team to keep items moving forward through the process [Kanban]. FDD is not so formal, leaving the Chief Programmers and Development Manager to keep an eye informally on the amount of work in progress. The Big Wallchart, Burn-Down/Up Charts, Etc For general visibility of progress within a project, the team typically lists all the features in the project complete with their owning Chief Programmer, feature team members, and the dates of each milestone achieved on a suitable wall. In addition, features can be colored to show if they are started, in-progress, completed or blocked. This allows people to stand back from the wall and get a good visual feel for the overall status of the project. They can then walk up to the wall to zoom in on particular areas and activities in more detail. Recording the date each milestone is achieved enables a team to produce burn-down or burn-up charts analogous to those produced in Scrum and XP. Chief Programmers and Project Managers can determine from these if the underlying rate of feature completion is increasing, decreasing, or stable, etc. One of the best ways to achieve this is to have the Chief Programmers regularly (typically once a week) communicate progress to either the project manager or someone dedicated to the task. That person then produces whatever roll-up and burn-down charts desired. Having an administrative person, the equivalent of the Tracker role in eXtreme Programming, perform these report formatting duties frees the Chief Programmers to spend more time on making progress rather than formatting reports about it. Parking Lot Charts For reporting to senior management, the level of individual features is often too granular. Here, FDD projects typically use a graphical report format that known as the Parking Lot chart. In a Parking Lot chart, each group of ‘parking lots’ represents one of the subject areas from the features list. Each parking lot represents one of the activities within that subject area, and displays the name of that set of features, the number of features within it, and the percentage of those features that have been completed (typically both in text and using a progress bar). The parking lots are also colored to indicate whether the features in that activity have been started, completed, or have significant blockages. The FDD parking lot format has become so popular that Mike Cohn included it in his book, Agile Planning and Estimating [Cohn]. (click for larger image) Figure 1: Example Parking Lot Chart Conclusion Feature-Driven Development combines the key advantages of other popular agile approaches with model-centric techniques and other best practices that scale to much larger teams and projects. It defines three upfront activities that provide a conceptual and management framework within which a larger-than-usual agile team can add functionality to the software, feature by feature. It is also just as applicable for smaller teams tackling non-trivial problem domains where it is worth spending just a little time to sketch a map of the journey before dashing off down the agile coding highway. Even if you and your team decide not to adopt FDD as a whole, understanding why FDD is the way it is, can provide insight into scaling traditional agile approaches beyond small, largely independent teams. Finally, I would like to say thank you to Serguei Khramtchenko and Mark Lesk at Nebulon for their corrections and suggestions incorporated in this article. References [Brooks] Frederick P. Brooks, Jr., The Mythical Man-Month, Addison Wesley [Cohn] Cohn, Agile Planning and Estimating, Prentice-Hall PTR [FDD] FDD Community Site, www.featuredrivendevelopment.com/ [Kanban] The home of Kanban software development, www.limitedwipsociety.org/ [McConnell] McConnell, Code Complete, Microsoft [Nebulon] The Latest FDD Processes available from www.nebulon.com/articles/fdd/latestprocesses.html [Palmer-1] Palmer, Felsing, A Practical Guide to Feature-Driven Development, Prentice Hall PTR

We’re using TFS on my current project. Yes, yes, I know. It’s generally good practice to collect all the code under your team’s control in a single uber-solution as described in this Patterns and Practices PDF, Team Development with TFS Guide. If you then configure the TFS build server to build this solution, it’s default behaviour is to place the build output into a single folder, ‘Release’. Any web application projects in your solution will also be output to a folder called _PublishedWebsites\. This is very nice because it means that you can simply robocopy deploy the web application. Unfortunately there’s no similar default behaviour for other project types such as WinForms, console or library. It would be very nice if we could have a _PublishedApplications\ sub folder with the output of any selected project(s). Fortunately it’s not that hard to do. The way _PublishedWebsites works is pretty simple. If you look at the project file of your web application you’ll notice an import near the bottom: On my machine the MSBuildExtensionsPath property evaluates to C:\Program Files\MSBuild, if we open the Microsoft.WebApplication.targets file we can see that it’s a pretty simple MSBuild file that recognises when the build is not a desktop build, i.e. it’s a TFS build, and copies the output to: $(OutDir)_PublishedWebsites\$(MSBuildProjectName) I simply copied the Micrsoft.WebApplication.targets file, put it under source control with a relative path from my project files and changed _PublishedWebsites to _PublishedApplications and renamed the file CI.exe.targets. For each project that I want to output to _PublishedApplications, I simply added this import at the bottom of the project file: You can edit CI.exe.targets (or whatever you want to call it) to do your bidding. In my case, the only change so far is to add a couple of lines to copy the App.config file: There’s a lot of stuff in Microsoft.WebApplication.targets that’s only relevant to web applications and can be stripped out for other project types, but I’ll leave that as an exercise for the reader. There was also a discussion on StackOverflow, with some nice alternative suggestions of how you might want to do this. It’s worth checking out.

Hibernate is a powerful, high performance object/relational persistence and query service. Hibernate lets you develop persistent classes following object-oriented idiom - including association, inheritance, polymorphism, composition, and collections. Hibernate allows you to express queries in its own portable SQL extension (HQL), as well as in native SQL, or with an object-oriented Criteria and Example API. Quintessential to using any ORM framework like hibernate is to know how to leverage the various performance tuning methods supported by the framework. In this volume Wings Jiang discusses three performance tuning strategies for hibernate: SQL Optimization Session Management Data Caching SQL Optimization When using Hibernate in your application, you already have been coding HQL (Hibernate Query Language) somewhere. For example, “from User user where user.name = ‘John’”. If issuing your SQL statement like this, Hibernate cannot use the SQL cache implemented by database because name of the user, in most scenarios, is extremely distinct. On the contrary, while using placeholder to achieve this, like “from User user where user.name =?” will be cached by the Database to fulfill the performance improvement. You can also set some Hibernate properties to improve performance, such as setting the number of records retrieved while fetching records via configuring property hibernate.jdbc.fetch_size, setting the batch size when committing the batch processing via configuring property hibernate.jdbc.batch_size and switching off the SQL output via setting property hibernate.show_sql to false in product environments. In addition, the performance tuning of your target Database is also significant, like SQL clauses tuning, reasonable indexes, delicate table structures, data partitions etc. Session Management Undoubtedly, Session is the pith of Hibernate. It manages the Database related attributes, such as JDBC connections, data entities’ states. Managing the Session efficiently is the key to getting high performance in enterprise applications. One of the many commonly used and equally elegant approaches to session management in hibernate is to use ThreadLocal. Threadlocal will create a local copy of session for every thread. Thus synchronization problems are averted, when objects are put in the Threadlocal, . To understand how ThreadLocal variables are used in Java, refer to Sun Java Documentation at http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ThreadLocal.html Data Caching Before accomplishing any data caching, it is essential to set the property hibernate.cache.user_query_cache = true. There are three kinds of commonly used Caching Strategies in Hibernate: Using cache based on Session level (aka Transaction layer level cache). This is also called first-level cache. Using cache based on SessionFactory level (Application layer level cache). This is also called second-level cache. Using cluster cache which is employed in distributed application (in different JVMs). In fact, some techniques, like loading data by id, lazy initialization which betokens loading appropriate data in proper time rather than obtaining a titanic number of useless records, which are fairly useless in the subsequent operations are consummated via data caching. First Level Cache (aka Transaction layer level cache) Fetching an object from database always has a cost associated with it. This can be offset by storing the entities in hibernate session. Next time the entities are required, they are fetched from the session, rather than fetching from the database. To clear an object from the session use: session.evict(object). To clear all the objects from the session use session.clear(). Second Level Cache (aka Application layer level cache) In this approach, if an object is not found in session, it is searched for in the session factory before querying the database for the object. If an object is indeed fetched from database, the selected data should be put in session cache. This would improve the performance when the object is required next time. To remove an entity from session factory use the various overloaded implementations of evict() method of SessionFactory. In fact, Hibernate lets you tailor your own caching implementation by specifying the name of a class that implements org.hibernate.cache.CacheProvider using the property hibernate.cache.provider_class. But it is recommended to employ a few built-in integrations with open source cache providers (listed below). Cache Type Cluster Safe Query Cache Supported Hashtable Memory NO YES EHCache Memory, Disk NO YES OSCache Memory, Disk NO YES SwarmCache Clustered YES (clustered invalidation) NO JBoss TreeCache Clustered YES (replication) YES Terracota Clustered YES YES In order to use second level caching, developers have to append some configurations in hibernate.cfg.xml (for example, using EHCache here). net.sf.ehcache.hibernate.Provider In addition, developers also need to create a cache specific configuration file (Example: ehcache.xml for EHCache). (1) diskStore : Sets the path to the directory where cache .data files are created. The following properties are translated: a.user.home - User's home directory b.user.dir - User's current working directory c.java.io.tmpdir (Default temp file path) maxElementsInMemory : Sets the maximum number of objects that will be created in memory. eternal : Sets whether elements are eternal. If eternal, timeouts are ignored and the element is never expired. timeToIdleSeconds : Sets the time to idle for an element before it expires. Is only used if the element is not eternal. Idle time is now - last accessed time. timeToLiveSeconds : Sets the time to live for an element before it expires. Is only used if the element is not eternal. TTL is now - creation time overflowToDisk : Sets whether elements can overflow to disk when the in-memory cache has reached the maxInMemory limit. Finally the cache concurrency strategy has to be specified in mapping files. For example, the following code fragment shows how to configure your cache strategy. … … Cache Concurrency Strategies There are four kinds of built-in cache concurrency strategies provided by Hibernate. Chosing a right concurrency strategy for your hibernate implementation is the key to cache performance optimization. Besides to ensure data consistency and transaction integrity it is indispensable to master these strategies. read-only If your application needs to read but never modify instances of a persistent class, a read-only cache may be used. This is the simplest and best performing strategy. It's even perfectly safe for use in a cluster. nonstrict-read-write If the application only occasionally needs to update data (For example, if it is extremely unlikely that two transactions would try to update the same item simultaneously) and strict transaction isolation is not required, a nonstrict-read-write cache might be appropriate. read-write If the application needs to update data, a read-write cache might be appropriate. This cache strategy should never be used if serializable transaction isolation level is required. transactional If the application seldom needs to update data and at the same time, application also needs to avoid “dirty read” and “repeatable read”, this kind of concurrency strategy can be employed. The transactional cache strategy provides support for fully transactional cache providers such as JBoss TreeCache. The following table lists cache concurrency strategy supported by various cache providers. Cache Read-only Nonstrict-read-write Read-write Transactional Hashtable YES YES YES N/A EHCache YES YES YES N/A OSCache YES YES YES N/A SwarmCache YES YES N/A N/A JBoss TreeCache YES N/A N/A YES Cluster Cache (in different JVMs) Hibernate also supports cluster caching in disparate JVMs. At present, both SwarmCache and JBoss TreeCache support cluster caching across multiple JVMs. In some situations, especially at the level of enterprise, certain application has to support the concurrency accessing of thousands of users, at that time, cluster cache can help you because the cluster can provide failover and load balancing which improve the performance of application. Points to Note When employing one of the four cache strategies above, pay close attention to the following situation: Data cached almost immutable If data you want to cache is almost constant, you can use data caching which can improve the performance of the application. On the contrary, if the caching data are quiet volatile, Hibernate have to maintain and update the caching over time which extremely leads to performance hit. Data sizes in reasonable range If the size of data you is caching is massive, Hibernate will occupy the most memories of system, which causes the long waiting time of the whole application. Low frequency of data updating If data you are caching needs to be modified frequently, Hibernate have to take an array of time to update and modify the data in caching, which impacts the performance of the application as well. High frequency of data querying If data you are caching is steady, which means that most of the operations are querying, searching, no updating and modifying, making the most use of caching will be affording huge performance improvement. None crucial data Because of existing some incongruities when keeping the data in caching, so if the data you are caching is fairly crucial, do not use caching. By contrast, if the data in caching is insignificant, just use it without any vacillation. Summary Actually, after employing SQL Optimization, Session Management, Data Caching, we will obtain great battalions of performance gains, which make applications achieve acceptable waiting time for the final customers. External Links for Further Study http://www.hibernate.org/hib_docs/reference/en/html/performance.html http://blogs.jboss.com/blog/acoliver/2006/01/23/Hibernate_EJB3_Tuning.txt About Author I am Wings Jiang from BCM China. I have mainly focused on J2EE technologies in recent years and worked in several projects involving Struts/Tapestry, Spring, Hibernate, WebLogic, Websphere, Oracle, DB2 etc. I have experience in design and code of several Java applications. Hibernate performance is one of the areas I pay close heed to in my current working.

Until a few days ago, I've always needed to work with the rather cumbersome Office Bean and UNO Runtime when integrating OpenOffice into a Java application. I also had to configure a whole bunch of things to force OpenOffice to play nicely with the Java integration. Two days ago, however, I found out about ODF Toolkit. It seems to be a relatively new project, independent since last year some time, though I could be wrong. What's especially interesting is the ODFDOM: ''ODFDOM is an OpenDocument (ODF) framework. It's purpose is to provide an easy common way to create, access and manipulate ODF files, without requiring detailed knowledge of the ODF specification. It is designed to provide the ODF developer community an easy lightwork programming API, portable to any object-oriented language.'' Here's a snippet of it in action: public static void main(String[] args) { try { OdfDocument odfDoc = OdfDocument.loadDocument(new File("/home/geertjan/test.ods")); OdfFileDom odfContent = odfDoc.getContentDom(); XPath xpath = odfDoc.getXPath(); DTMNodeList nodeList = (DTMNodeList) xpath.evaluate("//table:table-row/table:table-cell[1]", odfContent, XPathConstants.NODESET); for (int i = 0; i < nodeList.getLength(); i++) { Node cell = nodeList.item(i); if (!cell.getTextContent().isEmpty()) { System.out.println(cell.getTextContent()); } } } catch (Exception ex) { //Handle... } } Let's assume that the 'test.ods' file above has this content: From the above, the code listing would print the following: Cuthbert Algernon Wilbert And, as a second example, here's me reading the first paragraph of an OpenOffice Text document: public static void main(String[] args) { try { OdfDocument odfDoc = OdfDocument.loadDocument(new File("/home/geertjan/chapter2.odt")); OdfFileDom odfContent = odfDoc.getContentDom(); XPath xpath = odfDoc.getXPath(); OdfParagraphElement para = (OdfParagraphElement) xpath.evaluate("//text:p[1]", odfContent, XPathConstants.NODE); System.out.println(para.getFirstChild().getNodeValue()); } catch (Exception ex) { //Handle... } } On my classpath I have "odfdom.jar" and "xerces-2.8.0.jar". I don't necessarily have OpenOffice installed, which means I can very easily process a whole bunch of spreadsheets (or other OpenOffice output) without (a) installing OpenOffice and (b) faster than I would otherwise do, since OpenOffice doesn't need to be started up, via the Office Bean or otherwise. In fact, Aljoscha Rittner from Sepix, who told me about this project and who is using it in his commercial applications, reports that his processing has sped up to a fraction of the original, also because he doesn't need to handle the situation where OpenOffice would crash randomly in the middle of long running processes, such as during the night when there's no human interaction for restarting it.

Continuous Integration commonly known as CI is a process that consists of continuously compiling, testing, inspecting, and deploying source code. In any typical CI environment, this means running a new build every time code changes within a version control repository. Martin Fowler describes CI as: A software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. While CI is actually a process, the term Continuous Integration often is associated with three important tools in particular. As shown in the image the three pillars of CI are: 1. A version control repository like Subversion, or CVS. 2. A CI Server such as Hudson, or Cruise Control 3. An automated build process like Ant or Nant So, let’s look at each of these in detail: Version Control Repository: Version control repositories also known as SCM (source code management) play a crucial role in any software development environment. They also play a very important role for a successful CI process. The SCM is a central place for the team to store every needed artifact for the project. It is mandatory for the teams to put everything needed for a successful build into this repository. This includes the build scripts, property files, database scripts, all the libraries required to build the software and so on. The CI Server: For CI to function properly, we also need to have an automated process that monitors a version control repository and runs a build when any changes are detected. There are several CI servers available, both open source and commercial. Most of them are similar in their basic configuration and monitor a particular version control repository and run builds when any changes are detected. Some of the most commonly used open source CI servers are; Cruise Control, Continuum, and Hudson. Hudson is particularly interesting because of its ease of configuration and compelling plug-ins, which makes integration with test and static analysis tools much easier. Automated Build: The process of CI is about building software often, which is accomplished through the use of a build. A sturdy build strategy is by far the most important aspect of a successful CI process. In the absence of a solid build that does more than compile your code, CI withers. With automated builds, teams can reliably perform (in an automated fashion) otherwise manual tasks like compilation, testing, and even more interesting things like software inspection and deployment. Now that we have seen the important tools in our CI process, let’s see how a typical CI scenario looks like for a developer: CI server is configured to poll the version control repository continuously for changes. Developer commits code to the repository. CI server detects this change, and retrieves the latest code from the repository. This causes the CI server to invoke the build script with the given targets and options. If configured, CI Server will send out an e-mail to the specified recipients when a certain important event occurs. The CI server continues to poll for changes. Why is CI Important? This is one of the most frequently asked questions, and here are a few points to note about this powerful technique: Building software often greatly increases the likelihood that you will spot defects early, when they still are relatively manageable. Extends defect visibility. CI ensures that you have production ready software at every change. CI also ensures that you have reduced the risk of integration issues by building software at every change. CI server can also be configured to run continuous inspection which can assist the development team in finding potential bugs, bad programming practice, automatically check coding standards, and also provide valuable feedback on the quality of code being written. Over the past several months, I have assisted several companies in implementing CI. There was a little bit of resistance from the developers in the early stages when we implemented continuous feedback. But, never heard a single negative comment about this approach. If you already have a version control repository and automated builds, you are very close to the CI process. Download one of the open source CI servers, configure and setup a simple project. It should take less than an hour if you have automated build scripts. Start adding additional features like code inspections, generating reports, metrics, documentation and so on. Most important, send continuous feedback to your team. Give this process a try, you sure will be surprised to see how effective it is. And, as always share your thoughts, concerns or questions.

Looking at the current state of Java, we can see that POJOs (Plain Old Java Objects) rule the land yet again. Their dominance stretches from enterprise applications to middleware services. At JBoss we were known for our modular JMX-based kernel. The application server was nothing more than a bunch of flexible MBeans and a powerful MicroKernel in the middle. But, as you could feel the change is coming, we still wanted to be ahead of the pack. We could see different POJO based component models popping up all over the place (EJB3, JPA, Spring, Guice, … to name a few), yet nothing was out there that would bind them all, flattening out the differences on a single component model. Hence the Microcontainer project was born. The JBoss Microcontainer project is about many things. The Microcontainer's features range from reflection abstraction, a virtual file system, a simple state machine to transparent AOP integration, a new classloading layer, a deployment framework and an OSGi framework implementation. I'll try to address them all over a short series of articles that will be published here on DZone. This article will examine the Microcontainer's component models. Read the other parts in DZone's exclusive JBoss Microcontainer Series: Part 1 -- Component Models Part 2 –- Advanced Dependency Injection and IoC Part 3 -- the Virtual File System Part 4 -- ClassLoading Layer Part 5 - the Virtual Deployment Framework Part 6 - The Scanning Library What is a component model? What do we consider a component model? First of all, what do we consider being a component? One abstract way to express this would be that “components are reusable software programs that you can develop and assemble easily to create sophisticated applications.” To consider a bunch of components as an actual model, we also need to declare what kind of interactions we allow. JMX MBeans are one example of a component model. Their interactions include executing MBean operations, referencing attributes, setting attributes and declaring explicit dependencies between named MBeans. As mentioned, we had advanced JMX handling already with the MicroKernel. And as expected, the Microcontainer brought extensive POJO support. The default behavior and interactions in the Microcontainer are what you also normally get from any other IoC container and are similar to the functionality that MBeans provided: plain method invocations for operations, setters/getters for attributes and explicit dependencies. However, having only this functionality would mean we didn't get much further than just relieving the pain of declaring MBeans, hence it was only logical for us to do something more. I'll leave the discussion of these new IoC features for the next article in the series. There are many existing POJO component models out there, Guice and Spring being amongst the most popular. Effective integration with these frameworks was an important goal for us. Demo environment setup Before we begin, I'd like to first describe the various parts that constitute the demo. All the source code can be found at this location of our Subversion repository: • http://anonsvn.jboss.org/repos/jbossas/projects/demos/microcontainer/branches/DZone_1_0/ The project is fully mavenized, so it should be easy to adjust it to your IDE. I will first go over the sub-projects within the demo and describe their usage. At the end of my article series, I will provide a more detailed look at what certain sub-projects do. Below are the JBoss Microcontainer demos and sub-projects that are relevant for this article: • bootstrap (as the name suggests, it bootstraps the Microcontainer with demo code) • jmx (adds the JMX notion to demo's bootstrap) • models (source code of our components / services) The demo has only one variable you need to set - demos home - but even this one can be optional if you checked-out your project into the \projects\demos directory. Otherwise, you need to set the system property demos.home (e.g. -Ddemos.home=). You should now be able to run JMXMain as a main class. Make sure you include the models sub-project in the classpath, since some of the services require additional classes in the classpath, several more then what the jmx sub-project expects. Once the Microcontainer is booted it starts to scan the ${demos.home}/sandbox directory for any changes. Now all we need to do is provide a deployable unit and drop it in there. Models Let’s now turn our attention to the models sub-project. This is where the previously mentioned deployable unit should come from. You can see if everything is in place by building the models sub-project (mvn package) and dropping it into the sandbox. You should get a bunch of debug and info messages on the console log, showing how the Microcontainer got booted. Any error message indicates some problems. Let’s go over exactly what the models sub-project does, its integration code and try to redeploy it. If we look at the models src/main/resources/META-INF directory, we'll see plenty of -beans.xml resource files and one -service.xml. You’ll notice that each has a meaningful name that matches the source code package from models's src/main/java/org/jboss/demos/models directory. Let's dissect them one by one, starting with the ones that have no dependencies. org.jboss.demos.models.plain.PojoFactory This is a simple Micrcocontainer beans descriptor file. Anyone who crossed paths with some other IoC’s configuration file should be familiar with it. And, as I already mentioned, I'll follow up on more advanced usage in the next article. The next file shows what we have done to support Spring integration: Note that this file's namespace is different from the previous Microcontainer bean’s plain-beans.xml file. The urn:jboss:spring-beans:2.0 namespace points to our version of the Spring schema port, meaning you can describe your beans Spring style, but it's the Microcontainer that's going deploy then, not Spring's bean factory notion. public class Pojo extends AbstractPojo implements BeanNameAware { private String beanName; public void setBeanName(String name) { beanName = name; } public String getBeanName() { return beanName; } public void start() { if ("SpringPojo".equals(getBeanName()) == false) throw new IllegalArgumentException("Name doesn't match: " + getBeanName()); } } Although the SpringPojo bean has a dependency on Spring lib via implementing BeanNameAware interface, this dependency is only there to expose and mock some of the Spring's callback behavior (see SpringBeanAnnotationPlugin for more details). Since we introduced Spring integration, let's have a look at Guice integration. As Guice users know, Guice is all about type matching. Configuration of Guice beans is done via Modules which means that in order to generate beans, one must implement a Module. Two important parts to watch from this file are PojoModule and GuiceKernelRegistryEntryPlugin. The first one is where we configure our beans: public class PojoModule extends AbstractModule { private Controller controller; @Constructor public PojoModule(@Inject(bean = KernelConstants.KERNEL_CONTROLLER_NAME) Controller controller) { this.controller = controller; } protected void configure() { bind(Controller.class).toInstance(controller); bind(IPojo.class).to(Pojo.class).in(Scopes.SINGLETON); bind(IPojo.class).annotatedWith(FromMC.class).toProvider(GuiceIntegration.fromMicrocontainer(IPojo.class, "PlainPojo")); } } The second class provides integration with the Microcontainer: public class GuiceKernelRegistryEntryPlugin implements KernelRegistryPlugin { private Injector injector; public GuiceKernelRegistryEntryPlugin(Module... modules) { injector = Guice.createInjector(modules); } public void destroy() { injector = null; } public KernelRegistryEntry getEntry(Object name) { KernelRegistryEntry entry = null; try { if (name instanceof Class) { Class clazz = (Class)name; entry = new AbstractKernelRegistryEntry(name, injector.getInstance(clazz)); } else if (name instanceof Key) { Key key = (Key)name; entry = new AbstractKernelRegistryEntry(name, injector.getInstance(key)); } } catch (Exception ignored) { } return entry; } } Notice how we created an Injector from the Modules and then did a lookup on it for matching beans. In mbeans-service.xml we declare our legacy usage of MBeans: What’s interesting to note here is how we injected a POJO into an MBean. The preceding demonstrated our first interactions between different component models. In order to allow for MBean deployment via the Microcontainer, we had to write entirely new component model handling code. See the system-jmx-beans.xml for more details. The code from this file lives in the JBossAS source code: system-jmx sub-project. One note here, this is currently only possible with JBoss's JMX implementation, since the system-jmx code uses some implementation details. Now what if we wanted to expose our existing POJOs as MBeans, registering them into an Mbean server? @org.jboss.aop.microcontainer.aspects.jmx.JMX(exposedInterface=org.jboss.demos.models.mbeans.PojoMBean.class, registerDirectly=true) As you can see from looking at any of the beans in this file, in order to expose your POJOs as MBeans, it’s simply a matter of annotating them with an @JMX annotation. You can either expose the bean directly or its property: Here we see how you can use any of the injection lookup types by either looking up a plain POJO or getting a handle to an MBean from the MBean server. One of the injection options is to use type injection, which is also sometimes called autowiring: The FromGuice bean injects the Guice bean via type matching, where PlainPojo is injected with a common name injection. We then test if Guice binding works as expected: public class FromGuice { private IPojo plainPojo; private org.jboss.demos.models.guice.Pojo guicePojo; public FromGuice(IPojo plainPojo) { this.plainPojo = plainPojo; } public void setGuicePojo(org.jboss.demos.models.guice.Pojo guicePojo) { this.guicePojo = guicePojo; } public void start() { f (plainPojo != guicePojo.getMcPojo()) throw new IllegalArgumentException("Pojos are not the same: " + plainPojo + "!=" + guicePojo.getMcPojo()); } } This only leaves us with an alias component model. Even though the alias is quite a trivial feature, in order to implement it as a true dependency, it has to be introduced as a new component model inside the Microcontainer. The implementation details for this is part of the AbstractController source code: springPojo Here we’ve mapped the SpringPojo name to the springPojo alias. The beauty of having aliases as true component models is that it doesn't matter when the real bean gets deployed. This means that the alias will wait in a non-installed state until the real bean triggers it. Conclusion We've seen how we can deploy simple Microcontainer beans, legacy MBeans, Guice POJOs, Spring beans and aliases. And since all of this is controlled by the Microcontainer, we saw how easily we can mix and match the different component models. I can easily say, with the level of abstraction we put in our component model design, the sky is the limit on what we can handle. An example of this is the upcoming OSGi services, but that's another story, another article. Stayed tuned for my next article which will provide a detailed look at the Microcontainer's IoC functionality. About the Author Ales Justin was born in Ljubljana, Slovenia and graduated with a degree in mathematics from the University of Ljubljana. He fell in love with Java seven years ago and has spent most of his time developing information systems, ranging from customer service to energy management. He joined JBoss in 2006 to work full time on the Microcontainer project, currently serving as its lead. He also contributes to JBoss AS and is Seam and Spring integration specialist. He represent JBoss on 'JSR-291 Dynamic Component Support for Java SE' and 'OSGi' expert groups.