we've all seen them on various websites. crappy search utilities. they are a constant reminder that search is not something you should take lightly when building a website or application. search is not just google's game anymore. when a java library called lucene was introduced into the apache ecosystem, and then solr was built on top of that, open source developers began to wield some serious power when it came to customizing search features. in this article you'll be introduced to apache solr and a wealth of applications that have been built with it. the content is divided as follows: introduction setup solr applications summary 1. introduction apache solr is an open source search server. it is based on the full text search engine called apache lucene . so basically solr is an http wrapper around an inverted index provided by lucene. an inverted index could be seen as a list of words where each word-entry links to the documents it is contained in. that way getting all documents for the search query "dzone" is a simple 'get' operation. one advantage of solr in enterprise projects is that you don't need any java code, although java itself has to be installed. if you are unsure when to use solr and when lucene, these answers could help. if you need to build your solr index from websites, you should take a look into the open source crawler called apache nutch before creating your own solution. to be convinced that solr is actually used in a lot of enterprise projects, take a look at this amazing list of public projects powered by solr . if you encounter problems then the mailing list or stackoverflow will help you. to make the introduction complete i would like to mention my personal link list and the resources page which lists books, articles and more interesting material. 2. setup solr 2.1. installation as the very first step, you should follow the official tutorial which covers the basic aspects of any search use case: indexing - get the data of any form into solr. examples: json, xml, csv and sql-database. this step creates the inverted index - i.e. it links every term to its documents. querying - ask solr to return the most relevant documents for the users' query to follow the official tutorial you'll have to download java and the latest version of solr here . more information about installation is available at the official description . next you'll have to decide which web server you choose for solr. in the official tutorial, jetty is used, but you can also use tomcat. when you choose tomcat be sure you are setting the utf-8 encoding in the server.xml . i would also research the different versions of solr, which can be quite confusing for beginners: the current stable version is 1.4.1. use this if you need a stable search and don't need one of the latest features. the next stable version of solr will be 3.x the versions 1.5 and 2.x will be skipped in order to reach the same versioning as lucene. version 4.x is the latest development branch. solr 4.x handles advanced features like language detection via tika, spatial search , results grouping (group by field / collapsing), a new "user-facing" query parser ( edismax handler ), near real time indexing, huge fuzzy search performance improvements, sql join-a like feature and more. 2.2. indexing if you've followed the official tutorial you have pushed some xml files into the solr index. this process is called indexing or feeding. there are a lot more possibilities to get data into solr: using the data import handler (dih) is a really powerful language neutral option. it allows you to read from a sql database, from csv, xml files, rss feeds, emails, etc. without any java knowledge. dih handles full-imports and delta-imports. this is necessary when only a small amount of documents were added, updated or deleted. the http interface is used from the post tool, which you have already used in the official tutorial to index xml files. client libraries in different languages also exist. (e.g. for java (solrj) or python ). before indexing you'll have to decide which data fields should be searchable and how the fields should get indexed. for example, when you have a field with html in it, then you can strip irrelevant characters , tokenize the text into 'searchable terms', lower case the terms and finally stem the terms . in contrast, if you would have a field with text in it that should not be interpreted (e.g. urls) you shouldn't tokenize it and use the default field type string. please refer to the official documentation about field and field type definitions in the schema.xml file. when designing an index keep in mind the advice from mauricio : "the document is what you will search for. " for example, if you have tweets and you want to search for similar users, you'll need to setup a user index - created from the tweets. then every document is a user. if you want to search for tweets, then setup a tweet index; then every document is a tweet. of course, you can setup both indices with the multi index options of solr. please also note that there is a project called solr cell which lets you extract the relevant information out of several different document types with the help of tika. 2.3. querying for debugging it is very convenient to use the http interface with a browser to query solr and get back xml. use firefox and the xml will be displayed nicely: you can also use the velocity contribution , a cross-browser tool, which will be covered in more detail in the section about 'search application prototyping' . to query the index you can use the dismax handler or standard query handler . you can filter and sort the results: q=superman&fq=type:book&sort=price asc you can also do a lot more ; one other concept is boosting. in solr you can boost while indexing and while querying. to prefer the terms in the title write: q=title:superman^2 subject:superman when using the dismax request handler write: q=superman&qf=title^2 subject check out all the various query options like fuzzy search , spellcheck query input , facets , collapsing and suffix query support . 3. applications now i will list some interesting use cases for solr - in no particular order. to see how powerful and flexible this open source search server is. 3.1. drupal integration the drupal integration can be seen as generic use case to integrate solr into php projects. for the php integration you have the choice to either use the http interface for querying and retrieving xml or json. or to use the php solr client library . here is a screenshot of a typical faceted search in drupal : for more information about faceted search look into the wiki of solr . more php projects which integrates solr: open source typo3- solr module magento enterprise - solr module . the open source integration is out dated. oxid - solr module . no open source integration available. 3.2. hathi trust the hathi trust project is a nice example that proves solr's ability to search big digital libraries. to quote directly from the article : "... the index for our one million book index is over 200 gigabytes ... so we expect to end up with a two terabyte index for 10 million books" other examples for libraries: vufind - aims to replace opac internet archive national library of australia 3.3. auto suggestions mainly, there are two approaches to implement auto-suggestions (also called auto-completion) with solr: via facets or via ngramfilterfactory . to push it to the extreme you can use a lucene index entirely in ram. this approach is used in a large music shop in germany. live examples for auto suggestions: kaufda.de 3.4. spatial search applications when mentioning spatial search, people have geographical based applications in mind. with solr, this ordinary use case is attainable . some examples for this are : city search - city guides yellow pages kaufda.de spatial search can be useful in many different ways : for bioinformatics, fingerprints search, facial search, etc. (getting the fingerprint of a document is important for duplicate detection). the simplest approach is implemented in jetwick to reduce duplicate tweets, but this yields a performance of o(n) where n is the number of queried terms. this is okay for 10 or less terms, but it can get even better at o(1)! the idea is to use a special hash set to get all similar documents. this technique is called local sensitive hashing . read this nice paper about 'near similarity search and plagiarism analysis' for more information. 3.5. duckduckgo duckduckgo is made with open source and its "zero click" information is done with the help of solr using the dismax query handler: the index for that feature contains 18m documents and has a size of ~12gb. for this case had to tune solr: " i have two requirements that differ a bit from most sites with respect to solr: i generally only show one result, with sometimes a couple below if you click on them. therefore, it was really important that the first result is what people expected. false positives are really bad in 0-click, so i needed a way to not show anything if a match wasn't too relevant. i got around these by a) tweaking dismax and schema and b) adding my own relevancy filter on top that would re-order and not show anything in various situations. " all the rest is done with tuned open source products. to quote gabriel again: "the main results are a hybrid of a lot of things, including external apis, e.g. bing, wolframalpha, yahoo, my own indexes and negative indexes (spam removal), etc. there are a bunch of different types of data i'm working with. " check out the other cool features such as privacy or bang searches . 3.6. clustering support with carrot2 carrot2 is one of the "contributed plugins" of solr. with carrot2 you can support clustering : " clustering is the assignment of a set of observations into subsets (called clusters) so that observations in the same cluster are similar in some sense. " see some research papers regarding clustering here . here is one visual example when applying clustering on the search "pannous" - our company : 3.7. near real time search solr isn't real time yet, but you can tune solr to the point where it becomes near real time, which means that the time ('real time latency') that a document takes to be searchable after it gets indexed is less than 60 seconds even if you need to update frequently. to make this work, you can setup two indices. one write-only index "w" for the indexer and one read-only index "r" for your application. index r refers to the same data directory of w, which has to be defined in the solrconfig.xml of r via: /pathto/indexw/data/ to make sure your users and the r index see the indexed documents of w, you have to trigger an empty commit every 60 seconds: wget -q http://localhost:port/solr/update?stream.body=%3ccommit/%3e -o /dev/null everytime such a commit is triggered a new searcher without any cache entries is created. this can harm performance for visitors hitting the empty cache directly after this commit, but you can fill the cache with static searches with the help of the newsearcher entry in your solrconfig.xml. additionally, the autowarmcount property needs to be tuned, which fills the cache with a newsearcher from old entries. also, take a look at the article 'scaling lucene and solr' , where experts explain in detail what to do with large indices (=> 'sharding') and what to do for high query volume (=> 'replicating'). 3.8. loggly = full text search in logs feeding log files into solr and searching them at near real-time shows that solr can handle massive amounts of data and queries the data quickly. i've setup a simple project where i'm doing similar things , but loggly has done a lot more to make the same task real-time and distributed. you'll need to keep the write index as small as possible otherwise commit time will increase too great. loggly creates a new solr index every 5 minutes and includes this when searching using the distributed capabilities of solr ! they are merging the cores to keep the number of indices small, but this is not as simple as it sounds. watch this video to get some details about their work. 3.9. solandra = solr + cassandra solandra combines solr and the distributed database cassandra , which was created by facebook for its inbox search and then open sourced. at the moment solandra is not intended for production use. there are still some bugs and the distributed limitations of solr apply to solandra too. tthe developers are working very hard to make solandra better. jetwick can now run via solandra just by changing the solrconfig.xml. solandra also has the advantages of being real-time (no optimize, no commit!) and distributed without any major setup involved. the same is true for solr cloud. 3.10. category browsing via facets solr provides facets , which make it easy to show the user some useful filter options like those shown in the "drupal integration" example. like i described earlier , it is even possible to browse through a deep category tree. the main advantage here is that the categories depend on the query. this way the user can further filter the search results with this category tree provided by you. here is an example where this feature is implemented for one of the biggest second hand stores in germany. a click on 'schauspieler' shows its sub-items: other shops: game-change 3.11. jetwick - open twitter search you may have noticed that twitter is using lucene under the hood . twitter has a very extreme use case: over 1,000 tweets per second, over 12,000 queries per second, but the real-time latency is under 10 seconds! however, the relevancy at that volume is often not that good in my opinion. twitter search often contains a lot of duplicates and noise. reducing this was one reason i created jetwick in my spare time. i'm mentioning jetwick here because it makes extreme use of facets which provides all the filters to the user. facets are used for the rss-alike feature (saved searches), the various filters like language and retweet-count on the left, and to get trending terms and links on the right: to make jetwick more scalable i'll need to decide which of the following distribution options to choose: use solr cloud with zookeeper use solandra move from solr to elasticsearch which is also based on apache lucene other examples with a lot of facets: cnet reviews - product reviews. electronics reviews, computer reviews & more. shopper.com - compare prices and shop for computers, cell phones, digital cameras & more. zappos - shoes and clothing. manta.com - find companies. connect with customers. 3.12. plaxo - online address management plaxo.com , which is now owned by comcast, hosts web addresses for more than 40 million people and offers smart search through the addresses - with the help of solr. plaxo is trying to get the latest 'social' information of your contacts through blog posts, tweets, etc. plaxo also tries to reduce duplicates . 3.13. replace fast or google search several users report that they have migrated from a commercial search solution like fast or google search appliance (gsa) to solr (or lucene). the reasons for that migration are different: fast drops linux support and google can make integration problems. the main reason for me is that solr isn't a black box —you can tweak the source code, maintain old versions and fix your bugs more quickly! 3.14. search application prototyping with the help of the already integrated velocity plugin and the data import handler it is possible to create an application prototype for your search within a few hours. the next version of solr makes the use of velocity easier. the gui is available via http://localhost:port/solr/browse if you are a ruby on rails user, you can take a look into flare. to learn more about search application prototyping, check out this video introduction and take a look at these slides. 3.15. solr as a whitelist imagine you are the new google and you have a lot of different types of data to display e.g. 'news', 'video', 'music', 'maps', 'shopping' and much more. some of those types can only be retrieved from some legacy systems and you only want to show the most appropriated types based on your business logic . e.g. a query which contains 'new york' should result in the selection of results from 'maps', but 'new yorker' should prefer results from the 'shopping' type. with solr you can set up such a whitelist-index that will help to decide which type is more important for the search query. for example if you get more or more relevant results for the 'shopping' type then you should prefer results from this type. without the whitelist-index - i.e. having all data in separate indices or systems, would make it nearly impossible to compare the relevancy. the whitelist-index can be used as illustrated in the next steps. 1. query the whitelist-index, 2. decide which data types to display, 3. query the sub-systems and 4. display results from the selected types only. 3.16. future solr is also useful for scientific applications, such as a dna search systems. i believe solr can also be used for completely different alphabets so that you can query nucleotide sequences - instead of words - to get the matching genes and determine which organism the sequence occurs in, something similar to blast . another idea you could harness would be to build a very personalized search. every user can drag and drop their websites of choice and query them afterwards. for example, often i only need stackoverflow, some wikis and some mailing lists with the expected results, but normal web search engines (google, bing, etc.) give me results that are too cluttered. my final idea for a future solr-based app could be a lucene/solr implementation of desktop search. solr's facets would be especially handy to quickly filter different sources (files, folders, bookmarks, man pages, ...). it would be a great way to wade through those extra messy desktops. 4. summary the next time you think about a problem, think about solr! even if you don't know java and even if you know nothing about search: solr should be in your toolbox. solr doesn't only offer professional full text search, it could also add valuable features to your application. some of them i covered in this article, but i'm sure there are still some exciting possibilities waiting for you!

I have been using JBoss 4.2.3 for over a year right now and really like it (although the clustering / JMS stuff seems WAY overcomplicated - and needing 80 config files - not fun!). But anyway it is time to upgrade to JBoss 5 and the path has been marred by many stops and starts and has been surprisingly difficult. In any event I found the following links to be a life saver and thought I would share them http://community.jboss.org/wiki/MigrationfromJBoss4.pdf http://venugopaal.wordpress.com/2009/02/02/jboss405-to-jboss-5ga/ http://www.tikalk.com/java/migrating-your-application-jboss-4x-jboss-5x The real kickers have to be 1) Increased adherence to the Java spec that cause WARs / JARs to no longer deploy 2) Changed location of XML files 3) Changed XML filenames 4) Changed XML file contents Man they don't make it easy do they? From http://softarc.blogspot.com/2011/01/migrating-from-jboss-4-to-jboss-5.html

Try creating a new JSP file in your Eclipse and you’ll notice that the JSP page directive will have encoding something like: But for a better I18N and L10N support, it is recommended to follow UTF-8 encoding where ever possible. So, how do we change the default JSP file encoding to UTF-8 in eclipse? Simple. Just do these things: In Eclipse, go to Windows -> Preferences -> Web -> JSP Files Select UTF-8 encoding from the Encoding dropdown box there. That’s it! And if you wonder how this change works, you can very well see that. In the same Preferences window, go to this location: Preferences -> Web -> JSP Files -> Editor -> Templates. Then in the right hand side, you’ll see a list of templates defined for JSP files. And in a new JSP file template, you can see this code: Here as you might have guessed, the ${encoding} will be replaced by whatever we set in the above step. The same method can be used to change the default encoding type for other file types too (css, html). From http://veerasundar.com/blog/2010/12/changing-eclipse-default-encoding-to-utf-8-for-jsp-files/

Today JavaLobby published yet anotherdomain-model-as-RESTarticle, using Spring and Jersey. As already pointed out, this really is an anti-pattern, and is not RESTful, and cannot be so either. The point that all of these types of articles miss is theHATEOASpart, the hyperlinking of resources. If you expose your domain model, basically saying "here's all I got, use as you see fit", there is no sensible way to create links between resources that expose the application state. There is no sensible way to tell the client "here's what you can do next", because the "REST" API allows anything at any time. Here's an example, from my own app, which showed me the problem with this approach. I have users in my system. I need two ways to work with their passwords: the user must be allowed to change his own password, and the administrator shall be allowed to reset any users' password. In the beginning, when I was exposing my domain model, the URL's for this were as follows: /user//changepassword/user//resetpassword I'm just doing what the article suggests, which is to expose my domain model, and all that I can do with it. What's the problem with this? The first and most obvious problem is that it's very hard to determine who is allowed to do what here. I need to have authorization checks on both "changepassword" and "resetpassword". The first needs to check if it is the same user that accesses it, and the second needs to check if the accessing user has the administrator role. Also, since the client MUST get to these resources by following links in hypermedia (that's a constraint, remember?), the most obvious thing to do is to list them when accessing /user//. But then my link lists need to do these authorization checks as well, because if the user is not an administrator, then the "resetpassword" link should not be there. I should not allow clients to see links they cannot reasonably follow! My UI will also be quite complicated, because in one screen I might do "resetpassword" and a number of other administrative tasks, each of which uses different parts of the domain model, and so its exposure to the API, and consequently, its brittleness if the API changes, is immense. It's just a very bad situation, and one which you'll get into by following the guidelines in all of these expose-your-domain-model articles. So what to do instead? The trick is to expose usecases instead. It's that simple. Now, the main problem with doing this is that you have to actually know what your usecases are! And this is probably why all of these articles do the domain-model anti-pattern: because of their simplified nature they only thought to the point of "we gotta have users in our system" and not take the next step "what can we do with them?", because then you need to decide a whole lot more about what your system does. Since articles need to be reasonably focused on one thing, they just don't go there. But for You, if you do that, you end up with the mess outlined above. What did we do in our REST API to fix the above? We simply looked at our usecases, and rearranged the REST API accordingly. For the above, they relate to two different usecases which are account handling and user administration. And so we changed the API to something like this: /account/changepassword/administration/users//resetpassword If a client goes to /account/, which it can do by first going to "/" and finding that link, it will receive a list of links with what you can do on your own account, such as "changepassword". Do a GET on that link, and the client gets a form with two fields: the old password and the new password. The client might show this as three fields though, with a duplicate new password field to ensure that the user typed it correctly. The client can then POST to "changepassword" to make the actual change. I don't have to make any authorization checks, since the client is implicitly accessing its own account, so there's no way to screw it up, even deliberately. For the admin side, the client browses to /administration/users/ (and again, that link was retrieved from "/" if the user is an administrator), lists/searches the users, gets the link to a particular user, do a GET on "resetpassword" to get the form for it, and then fills it in and POST it to make the change. The REST API has at all times told the client what it is allowed to do, by using hypertext to drive the application state. This is what HATEOAS means in practice, and it is VERY helpful if you expose your usecases, and VERY annoying if you expose your domain model (simply because you can't). This approach also removes the issue outlined in the article with circular references, simply because in usecases, there are no circular references. Another effect of this is that links in the REST API will almost always be relative, since all they do is guide the client further into a usecase, or sub-usecase. If you expose domain models you have to have absolute links, and they will be going here there and everywhere, exposing the internal associations between entities. It's stupid beyond belief, but this is what pretty much all of the current articles on "RESTful" frameworks tell you to do. And I say: DON'T! It'll only bring you misery. From a security point of view the above is also easier to work with. Now all I have to do is add a check on "/administration/" for the administrator role, and after that the user can do anything below that point. No need to duplicate that check everywhere (and no need to use aspects to get around this annoyance)! Now that you know what to do, the next question might be: how to do it? The problem here is that since all the current "REST" frameworks haven't thought HATEOAS through, they don't really allow you to make REST API's for your applications, and so, they will not be very easy to use. You have to work against them. This goes for Jersey and friends as well, as they don't allow this usecase approach to URL parsing. What we have done in my project, Streamflow, is to create our own wrapper on top ofRestlet. Restlet provides an excellent base for REST applications, but is too low-level to have to deal with in the application code. By putting a thin wrapper on top of it, which understand the notion of usecases and links, we have been able to really reduce the amount of code needed to do all of this. Here's sample code for "change password". The code uses a routing technique, so in order to get to /account/changepassword, the framework first takes "/" and finds a resource for that. This resource then knows how to get to "/account/", which in turn knows how to get to "changepassword". The code looks like this: public class RootResource extends CommandQueryResource{ @SubResource public void account() { subResource( AccountResource.class ); }...} When the client hits "/" the framework will automatically look at this class, and present hypermedia (JSON or HTML at this point, but Atom Services is also an option) for it. The client clicks "/account/", which leads to this: public class AccountResource extends CommandQueryResource{ public AccountResource( ) { super( AccountContext.class ); } @SubResource public void profile() { subResourceContexts( ProfileContext.class, ContactableContext.class ); } To find out what the client can do on this level you look in AccountContext. If you want to continue further down into the profile handling of an account (setting email, phone, etc.) the client would follow "/account/profile/" link that is presented. In our case, let's look at AccountContext: public class AccountContext{ public void changepassword( ChangePasswordCommand newPassword ) throws WrongPasswordException { UserAuthentication user = RoleMap.role( UserAuthentication.class ); user.changePassword( newPassword.oldPassword().get(), newPassword .newPassword().get() ); } It exposes one interaction on this level of the usecase, which is /account/changepassword. The parameter tells the framwork what it requires as input, so if the client does GET it can look at the ChangePasswordCommand value object and present it as a form. If the client does a POST the framework parses the input into the value object and invokes the method, allowing it to "do it's thing". In this case it looks up the UserAuthentication role, which is a Qi4j mixin that our user entity implements. This entity was registered automatically by the authentication filter, so I don't have to look it up. For the administration usecase the code would get access to the "" part of the URL so that it can locate the user from the repository. And that's pretty much it. Not only is this easier to work with for clients (who simply follow links in hypermedia), but the code in both client and application is also absolutely trivial. I can also find out what my REST API looks like in total by starting with RootResource and explore the API by clicking on the classes it references. Since the API is so deterministic given these classes I can also generate documentation automatically that basically says, "given your user authorization level, here are all the resources you can access in the API". In particular, when your API becomes hypermedia driven like this, what you want to expose as documentation is the set of "rel" attributes that the client must know, in this case "account" and "changepassword". What the URL's look like is none of the clients business! It should just go to "/" and follow links based on the "rel" attribute of those links. Then you can say that your API is "RESTful". Until you do that, no, you may have a "web API" or a "HTTP API", but it aint REST. There are more details in how this works, and how to turn links on/off depending on internal state, but the gist of how to think is as above. From http://www.jroller.com/rickard/entry/the_domain_model_as_rest

Vince Sheard (pictured right) is the Manager of the MPLAB® Integrated Development Environment (IDE) team at Microchip Technology Inc. He has been working at Microchip for more than 10 years, and was the lead architect for the MPLAB X IDE version 1.0, prior to stepping into the management roll. This, the port of MPLAB to the NetBeans Platform, is the sixth major architecture change of the MPLAB IDE since its inception in 1992. Microchip Technology Inc., headquartered in Chandler, Arizona, was spun off from General Instrument in 1989. The Company went public in 1993, and is a leading provider of microcontroller, analog, and Flash-IP solutions, providing low-risk product development, lower total system cost and faster time to market for thousands of diverse customer applications worldwide. Headquartered in Chandler, Arizona, Microchip offers outstanding technical support along with dependable delivery and quality. For more information, visit the Microchip website at http://www.microchip.com. (The MPLAB X landing page is http://www.microchip.com/mplabx, which will be available in the next few days.) Hi, Vince. What's MPLAB, in a nutshell? MPLAB® is an integrated development environment or IDE. It is similar to other IDEs, but there are two important differences. The first difference is that the MPLAB IDE is our customers’ window into Microchip’s PIC® microcontrollers embedded in their designs. Many people believe they understand their PC, because it is “right here.” An embedded device is more difficult to get a handle on. It’s the brain of a product “over there.” It’s not a computer, it’s a thing. The MPLAB IDE gives embedded developers an opportunity to dig into the brain of that thing. The second important difference is that the MPLAB IDE seamlessly covers Microchip’s entire portfolio of more than 700 8-bit, 16-bit and 32-bit PIC microcontrollers. The differences between these devices are massive, from a tiny 6-pin, 8-bit microcontroller that could fit under your fingernail, to a huge 32-bit microcontroller that is much more powerful than the iconic IBM mainframe of last century. The MPLAB IDE provides a consistent and supportive environment in which to debug our customers’ original, creative works of software that differentiate their products. What are the MPLAB IDE’s main features and how does it distinguish itself from its competitors? Integrated development environments share many features: project creation and management, programmers’ editor, language tool integration and build tools, image preparation and programming, and debug facilities. These are the MPLAB X IDE’s main features, too. A large difference comes in the presentation of an embedded target, rather than a PC target, which presents a developer with a less coupled and less easily controlled object for their development. That tightens the focus, but there are other IDEs that support embedded development. The MPLAB X IDE is distinguished by its seamless and timely support of Microchip devices, the vast ecosystem of tightly integrated compilers and hardware tools that also support those devices, and the evolutionary grace of a tool that has grown with our customers and technologies until the three form a smooth and supportive system for developing innovative embedded products. What are the typical technical challenges of an application of this kind? The principal challenge we face is how to provide the facilities developers need in the most intuitive and useful way possible. It’s easy to provide tons and tons of features, but with the GUI technologies of today, one can only present a small fraction of what is provided and available for use. One of the gauges for how well we’ve done our job is when our customers present a fine point of usage that we’ve already discussed and disagreed on how to implement. Those "complaints" are really wonderful because they come from expert tool users who understand the tools they are using. Of course, we are always hustling to present new Microchip devices in the same light that we present established devices. We’re also challenged to exploit the advanced debug facilities of new devices in a way that is most beneficial for our customers. What's the architecture of the application and why did you make the choices that you did? We’re moving away from a Windows OS-only, COM-based architecture. As our customers’ development sophistication has grown, so have their needs. Our customers now require Linux, Mac, and Windows support. Microchip is a worldwide company with customers who may only be fluent in a single language, so that impacts our choices. We also have a number of educational institutions who place interesting demands on the IDE, along with a number of advanced customers who are very forward-looking and are really pushing the envelope of what was previously thought possible. The most telling choice we had to make, though, was the choice of NetBeans as a fully capable, modern, lightweight and flexible platform for our next-generation MPLAB X IDE: Where does the NetBeans Platform fit into all of this? NetBeans is unique among current open-source IDE platform offerings, in that it is the most advanced for addressing our primary challenge, which I described above. The NetBeans IDE presents standard operations in a way that really minimizes hunting around and wasting time, to find out how to accomplish what you need to do. Take, for instance, the classic edit, compile, debug “cycle.” In NetBeans, like in other IDEs, the edit part is pretty easy: open the source file in the programmers’ editor and make the changes you need. It’s really not much different than editing a document with a word processor. The next step is where the difference really shows. In some IDEs, you have to compile or build an image and then figure out how to load it and start a debugging session. In some IDEs, that’s just brutal to figure out, especially in embedded systems where, as we said, the target is not “right here,” it’s “over there.” In NetBeans, it’s a single button press (DebugRun), even in an embedded context like ours. After that, the rest of the steps are taken care of. If any errors occur during the sequence, we take it home by capturing the errors and placing the user in context for an easy solution; and the ability to move on. That’s just one example. NetBeans is way ahead of the curve compared to other menu/toolbar/property sheet IDEs. What are the 3 main benefits of the NetBeans Platform, in this context? First and foremost is the ability to really optimize developers’ time. That’s a major factor in our customers’ focus. Second is the fact that NetBeans is a modern, lightweight and fully capable IDE platform. It doesn’t suffer from the bloat and outdated aspects of some other IDE platforms. Included with these benefits are the abilities to localize the IDE and to utilize it on multiple operating systems. Last, but not least, is the huge benefit of a professionally executed development, maintained by a focused organizing committee and maintained by a tight-knit development community—all headed by the Oracle contributions. That’s a significant departure from some other high-profile, open-source platforms that are available. How did you end up choosing the NetBeans Platform over its alternatives? When you consider all of the benefits I just mentioned, I think it’s pretty much a no-brainer. How did you get started with it? We first discussed the concepts with Sun Microsystems, as we wanted to provide a benefit to the overall NetBeans IDE from an embedded side. We created a proof of concept by plugging in our existing debuggers to the NetBeans IDE on a Windows operating system. We were fortunate to find someone who previously worked within Sun Microsystems on the NetBeans Platform, who came in on contract and gave us some one-on-one training. This was mainly because we had an aggressive schedule and wanted to be able to get up to speed faster. Any tips and tricks for others going down the same road? The mail lists are very helpful. We found many things we wanted to do being asked by others on the mailing lists. Many of the developers monitor these lists frequently, and respond more often than not in a timely fashion. The documentation suite, videos, written documents and interaction with the community are all extremely helpful in getting to the root of any matter quickly. The IDE is well organized and uses common techniques within the code base, which quickly become familiar. Other platforms treat information as job security. Some people may know, but they want you to pay for the information, in one way or another. Not so with NetBeans; everyone involved is completely open and collaborative. Any specific things that surprised you about the NetBeans Platform (in a good or bad way)? NetBeans prevents development with any form of interdependency between modules. For instance, you cannot have A->B and B->A. This really is a good thing and ensures that you break your modules up more to create a common module. The way JUnit is integrated into the IDE makes it so seamless to create your unit tests while developing the code. Source-code revision control worked quite smoothly from within the IDE. A hindrance for us is that there is no way to view a block of memory in hex form, when running under Java (using the JVM debugger). This is something often required when doing embedded development. Since we are developing code for our debug tools, we wanted to see the code blocks being transported from the IDE to the tool in hex, but could not. Anything else you'd like to share? It is great how fully featured the editor is. We are often so deep in developing what is required for the embedded side that we still discover things the editor does that we hadn’t come across. We are using the NetBeans IDE to create our own IDE, which is a completely new, yet compatible, incarnation of our IDE. The NetBeans developers at Oracle/Sun are extremely open to assist in getting value added to the current IDE, which makes our job even easier.

create a soap ui project using your wsdl. set the preferences in soap ui for axis2 home directory. right click on the wsdl in soap ui and click generate code. select adb binding and the following settings and click generate following is the directory structure and code files generated. that’s it, you can now use this code from you ide by importing it. ps: you will need to add the axis2 jars to your project class path. for more details visit my blog @ http://nitinaggarwal.wordpress.com/

MyBatis is a lightweight persistence framework for Java and .NET. This blog entry addresses the Java side. MyBatis is an alternative positioned somewhere between plain JDBC and ORM frameworks (e.g. EclipseLink or Hibernate). MyBatis usually uses XML, but it also supports annotations since version 3. The documentation is very detailed for XML, but lacks annotation examples. Just the Annotations itself are described, but no examples how to use them. I could not find any good and easy examples anywhere, so I will describe some very basic examples for SELECT, INSERT and UPDATE statements by implementing a Data Access Object (DAO) using MyBatis. These examples are a good starting point to create more complex MyBatis queries using a DAO. You can find the full source code at the end of this blog. A simple SQL-Table I use a very simple table with two attributes. The name of the table is “simple_information”. The primary key is a Integer and may not be null (info_id). The only real data is a character and may also not be null (info_content). That is enough “complexity” to learn the usage of MyBatis annotations. The Java class “SimpleInformationEntity” is a POJO which contains these two attributes. @SELECT-Statement The @Select annotation is very easy to use, if you want to use exactly one paramter. If you need more than one paramter, use the @Param annotation (which is described below at the update example). You do not have to map the found information to a SimpleInformationEntity object, as you would have to do with a JDBC ResultSet. The magic of the framework does this for you. final String GET_INFO = “SELECT * FROM simple_information WHERE info_id = #{info_id}”; @Select(GET_INFO) public SimpleInformationEntity getSimpleInformationById(int info_id) throws Exception; @INSERT-Statement You can use the object (which you want to be persist) as parameter. You do not have to use several parameters for each attribute of the object. The magic of the framework does this for you. final String PERSIST_INFO = “INSERT INTO simple_information(info_id, info_content) VALUES (#{infoId}, #{infoContent})”; @Insert(PERSIST_INFO) public int persistInformation(SimpleInformationEntity simpleInfo) throws Exception; @UPDATE-Statement You cannot use more than one parameter within a method. If you want to understand why, look at some MyBatis XML examples in the documenation: There you use the attribute “parameterType”, which must be exactly one “parameter”! So you will get a (strange) exception, if you use two or more parameters. Instead you have to use the @Param annotation, if you need more than one parameter. final String UPDATE_INFO = “UPDATE simple_information SET info_content = #{newInfo} WHERE info_id = #{infoId}”; @Update(UPDATE_INFO) public int updateInformation(@Param(“infoId”) int info_id, @Param(“newInfo”) String new_content) throws Exception; Configuration You have to add your MyBatis-Interface for the Mapper to the SQLSessionFactory: sqlSessionFactory.getConfiguration().addMapper(InfoMapper.class); Some Hints for developing with MyBatis The following means helped me a lot to use MyBatis annotations despite the lack of documentation about using annotations: - MyBatis is open source! So add the sources to your build path and use the debugging function of your IDE to enter the MyBatis source code while executing some queries. You will see what MyBatis expects as input and how it is processed. - Read the documentation about using MyBatis XML. This does not really make any sense you think? It does! The processing deep inside MyBatis does not change if you use annotations instead of XML. It is just another way to develop persistence queries. E.g. if you know that a XML select query may just use exactly one parameterType-attribute, then you know that you may just use one parameter in an annotation-based method too! If you need more parameters, you have to use the @Param annotation. - If you get any strange exception that does not make any sense, then clean and re-compile your project. This often helps, because as with other persistence frameworks such as Hibernate, the bytecode enhancement sometimes confuses your IDE. Conclusion: @MyBatis-Team: Improve and extend the documentation instead of improving the framework itself! MyBatis is a nice lightweigt persistence framework. But the documentation is not enough detailed. Some important information is completely missing. Especially, if you are a newbie to MyBatis / iBatis, it is very tough to develop with MyBatis using annotations instead of XML. Besides the usage of annotations, another good example for missing documentation is how to configure transactions in MyBatis by using JNDI and a J2EE / JEE Application server. You have to use google to find out, and if you are lucky you will find a mailing list or blog entry describing your problem. If not, you have to try it out. The missing documentation makes MyBatis much more tough than it actually is. So in the next months, the MyBatis team should improve and extend the documentation instead of improving the framework itself… Best regards, Kai Wähner (Twitter: @Kai Waehner) [Content from my Blog: MyBatis (formerly iBatis) - Examples and Hints using Select, Insert and Update Annotations - Kai Wähner's IT-Blog] Appendix: Source Code Here you see all the necessary source code, also including a MyBatis Connection Factory, which reads the configuration data from a XML file. ############################################################################ Connection Factory (using a static initializer) ############################################################################ import java.io.FileNotFoundException; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.Reader; import org.apache.ibatis.session.SqlSessionFactory; import org.apache.ibatis.session.SqlSessionFactoryBuilder; import de.waehner.kai.persistence.InfoDAO.InfoMapper; public class MyBatisConnectionFactory { private static SqlSessionFactory sqlSessionFactory; static { Reader reader = null; try { InputStream in = MyBatisConnectionFactory.class.getResourceAsStream(“myBatisConfiguration.xml”); reader = new InputStreamReader(in); if (sqlSessionFactory == null) { sqlSessionFactory = new SqlSessionFactoryBuilder().build(reader); sqlSessionFactory.getConfiguration().addMapper(InfoMapper.class); } in.close(); } catch (FileNotFoundException fileNotFoundException) { fileNotFoundException.printStackTrace(); } catch (IOException iOException) { iOException.printStackTrace(); } } public static SqlSessionFactory getSqlSessionFactory() { return sqlSessionFactory; } } ############################################################################ Data Acces Object (including the MyBatis-Mapper as inner Class) ############################################################################ import org.apache.ibatis.annotations.Insert; import org.apache.ibatis.annotations.Param; import org.apache.ibatis.annotations.Select; import org.apache.ibatis.annotations.Update; import org.apache.ibatis.session.SqlSession; import org.apache.ibatis.session.SqlSessionFactory; public class InfoDAO { public interface InfoMapper { final String GET_INFO = “SELECT * FROM simple_information WHERE info_id = #{info_id}”; @Select(GET_INFO) public SimpleInformationEntity getSimpleInformationById(int info_id) throws Exception; final String PERSIST_INFO = “INSERT INTO simple_information(info_id, info_content) VALUES (#{infoId}, #{infoContent})”; @Insert(PERSIST_INFO) public int persistInformation(SimpleInformationEntity simpleInfo) throws Exception; final String UPDATE_INFO = “UPDATE simple_information SET info_content = #{newInfo} WHERE info_id = #{infoId}”; @Update(UPDATE_INFO) public int updateInformation(@Param(“infoId”) int info_id, @Param(“newInfo”) String new_content) throws Exception; } public SimpleInformationEntity getSingleAlarm(int info_id) throws Exception { SqlSessionFactory sqlSessionFactory = MyBatisConnectionFactory.getSqlSessionFactory(); SqlSession session = sqlSessionFactory.openSession(); try { InfoMapper mapper = session.getMapper(InfoMapper.class); SimpleInformationEntity simpleInfo = mapper.getSimpleInformationById(info_id); return simpleInfo; } finally { session.close(); } } public int persistInformation(SimpleInformationEntity simpleInfo) throws Exception { SqlSessionFactory sqlSessionFactory = MyBatisConnectionFactory.getSqlSessionFactory(); SqlSession session = sqlSessionFactory.openSession(); try { InfoMapper mapper = session.getMapper(InfoMapper.class); int answer = mapper.persistInformation(simpleInfo); return answer; } finally { session.close(); } } public int updateInformation(int info_id, String new_content) throws Exception { SqlSessionFactory sqlSessionFactory = MyBatisConnectionFactory.getSqlSessionFactory(); SqlSession session = sqlSessionFactory.openSession(); try { InfoMapper mapper = session.getMapper(InfoMapper.class); int answer = mapper.updateInformation(info_id, new_content); return answer; } finally { session.close(); } } } ############################################################################ SimpleInformation Entity (a simple POJO) ############################################################################ import java.io.Serializable; public class SimpleInformationEntity implements Serializable{ private static final long serialVersionUID = -821826330941829539L; private int infoId; private String infoContent; public int getInfoId() { return infoId; } public void setInfoId(int infoId) { this.infoId = infoId; } public String getInfoContent() { return infoContent; } public void setInfoContent(String infoContent) { this.infoContent = infoContent; } }

despite the arguments and debates about getters and setters in java, the fact is that they’re a reality and you have to work with them. but managing getters and setters is a time-consuming effort. creating a getter/setter for 5 fields in a class can take minutes, renaming one is error-prone and deleting one is just plain inconvenient. there are options like project lombok (that implicitly creates getters/setters without the need to code them) and you could avoid getters/setters altogether by redesigning your classes. but these options aren’t always available, so it’s a good thing eclipse has some handy features for managing getters and setters. combined with the ability to generate constructors based on fields , you can get the boilerplate code out of the way in seconds and get on with the real coding. generate getters and setters to generate getters and setters, do the following: create the fields you want in the class then press alt+shift+s, r . a dialog will pop up allowing you to choose the fields you want to generate getters and setters for. click select all to create getters/setters for all fields. of course you can choose individual fields as required. change insertion point to last member . this tells eclipse that you want to put the methods at the bottom of the class. this is normally the best option for me as i want them out of the way. click ok . eclipse will create the getters and setters for you. here’s an example of what the dialog should look like. note: by default, eclipse doesn’t allow you to create a setter for a final field – the setter just doesn’t appear in the dialog. this can be a nuisance, especially if you’ve enabled autoformatting to make fields final where possible. to bypass this restriction, enable the checkbox allow setters for final fields on the dialog. the setter for the field will now appear in the dialog. once you click ok, eclipse will remove the final keyword from the field and generate the setter. eclipse also remembers this setting. another way to add just a single getter/setter is to position your cursor anywhere in the class (outside any method), start typing either “get” or “set” and press ctrl+space . the options on the autocomplete menu will include any getters/setters of fields that don’t have any defined yet. this is a quick way to create a single getter/setter but isn’t geared for bulk creation. here’s an example of how the autocomplete looks: rename getters and setters the easiest way to rename getters/setters is to use the rename refactoring. place your cursor on the field name (anywhere in the class, not just the declaration) and press alt+shift+r . if you’re using in-place rename (the default), then just rename the field, press enter and eclipse will rename the corresponding getters and setters as well. if you’ve chosen to use the classic refactor dialog (see note below) then make sure you enable rename getter and rename setter on the rename dialog. note: you can choose to do renaming using the traditional rename dialog by going to window > preferences > java and unchecking rename in editor without dialog . i prefer using the rename dialog as it highlights the whole name by default making it easier to overwrite and i have the option of not renaming the getters and setters if i don’t want to. the eclipse default these days is to use the new in-place renaming. although eclipse will rename the getter/setter, it won’t rename the argument passed to the setter method. if you want consistency, you can navigate to that method (eg. using ctrl+o ) and rename the argument yourself. delete getters and setters deleting getters and setters isn’t as straightforward as just deleting the field in the editor. however, you can delete a field and its getters/setters from the outline view. open the outline view ( alt+shift+q, o ), select the field you want to delete and press delete (or right-click, delete). eclipse will ask you whether you want to delete the getters/setters as well. just choose yes to all and they will be removed. you need to have fields visible in the outline view to use this feature (ie. untoggle the hide fields button ). you can select multiple fields simultaneously. and you can delete individual getters/setters (excluding the field) by just selecting the getter/setter and pressing delete . from http://eclipseone.wordpress.com/2010/10/26/generate-rename-and-delete-getterssetters-in-eclipse/

Shutdown Hooks are a special construct that allow developers to plug in a piece of code to be executed when the JVM is shutting down.

It seems that lots of projects/products/services want to expose a REST API these days. But I have found very few that actually follow the REST constraints, and in a lot of the cases it doesn't even make sense for them to follow REST constraints in the first place. One of the main constraints that is commonly violated is the hypertext constraint. Basically, all state changes have to be done by following links, starting from a bookmarked URL. But almost noone does that. However, should they? This article will outline various layers that REST API's can be implemented in, and when it makes sense, and when not. To begin with, in a typical enterprise app there are three options for layers that you might want to expose using a REST API. These are the infrastructure layer, the domain layer, and the application layer. Infrastructure layer If we start with the infrastructure layer, we are typically talking about a database vendor that wants to allow developers to access it using "REST". The API would allow you to create/remove databases, and then insert/update/delete data. Typically it's pretty normal stuff, and the API doesn't change all that much between versions. Accessing this over HTTP maybe makes sense, but is it RESTful? I'll give you an example. I installed CouchDB, and given the hypermedia constraint I should then be able to go to "http://localhost:5984/", and it will tell me what I can do next (like create a database). But when I do a GET on that URL I get this: {"couchdb":"Welcome","version":"1.0.1"} So now what? The hypermedia doesn't tell me what I can do, so therefore as a REST client I will assume there's nothing I can do. This very simple test shows that the HTTP API for CouchDB isn't really RESTful at all. The question is: should it be? That is obviously up to the developers to decide. But if I were the architect I would maybe say, no, it shouldn't be RESTful. Why? Because I want to allow URL templates to be used, so that the client, given the server URL and a document id, is allowed to construct a URL on its own and GET the document. If this was truly RESTful the client would have to do a query in a form first, with the id, in order to get the URL of the document to be retrieved. That might be inefficient for a database, so I might opt not to do this. Which is, in effect, what they already have done. The only problem is that they call it RESTful, when it isn't, so it gives me as a developer the wrong impression of what I can expect from it. This line of reasoning could be done for pretty much most infrastructure layer API's. They're not RESTful, though many say they are, and most likely they shouldn't try to be! IT'S OK! Just say "Accessible over HTTP, see docs for URL templates and whatnot", and be done with it. Domain layer The next potential layer to be exposed over REST is the domain layer. This typically means that you take your domain entities and expose their data straight on the web, through CRUD operations. Very straightforward. There are tons of articles and blogs that show how to do this. But is it RESTful? Or is it even a good idea in the first place? The first test, again, would be to see if the app follows the hypermedia constraint. In this option it is technically possible to allow queries that will list the various URL's to entities in your domain, which you then can update/delete. So on the surface it might seem like you are following the hypermedia constraint. The problem usually comes with the fact that you are exposing domain state rather than application state. Let me explain through a simple example. Let's say you are building an issue tracker. You can access individual issues through links like: /issue/123 which on GET gives you documents such as: {"status":"OPEN","description":"Some issue"} Awesome. Now a client can change the status to "CLOSED" and PUT that. Tada! Case closed. Or is it? What if a client then decides to reopen it, by simply posting a new status of "OPEN" to it. Ok, that worked. But should it? Maybe your domain model really would have wanted it to only go to "REOPENED" from the "CLOSED" state. But how do you express that? How is the client to know that this is the only valid transition? And what happens when we have many versions of clients, each of which has a slightly different set of rules for what you are allowed to do when? Basically, chaos is ensured. And this is the problem with exposing your domain model using a REST API. The client has to own the application logic, and there's no way the server can be sure that it has the "right" logic. And the client, even if it *wants* to play nice (if code ever wants anything is debatable), will have a hard time knowing whether it is playing by the rules or not. It might even get a bit neurotic, trying to do the right thing, whatever that means. In summary, exposing your domain model does not help the client know what the valid state transitions are, and makes it very hard to do other things like role-based security authorization (maybe only an admin is allowed to REOPEN a CLOSED case?). I would therefore recommend that noone exposes their domain models using a REST API. Application layer Finally we come to the application layer. The application layer is designed to implement usecases of the domain model, and has all the context and logic needed to ensure that only valid state transitions are made. In short, it seems like it is especially appropriate to being exposed through a REST API, as it can at all points tell the client what it can do (either based on state or authorization rules or any other type of rules it might have). If we go back to the issue tracker, what would this mean in practice? It could mean that when you do a GET on /issue/123 you get something like this back: {"data":{"status":"OPEN","description":"Some issue"},"links":[{"close":"/issue/123/close.json"}]} This now instead of referring to viewing the domain state of an issue refers to the usecase of viewing an issue with the intent of working on it. There might be other URL's and other queries that only return the data, or maybe a table of the data, or somesuch. But this one, specifically, refers to the usecase of working with the issue. So, as a REST client I can now inspect the data, and then look at what links are available. If the client has a UI it can enable a button that says "Close issue" based on the available link, since it detected a link relation "close" that it understands. The client can then do GET on that link, find out whether the server expects any form to be filled in, and then submit it using POST, thereby letting the server application layer logic transition the issue to the "CLOSED" state. We are no longer relying on the client to contain the logic of knowing when to allow what, and the client also does not have to know how to construct the URL. As long as it can parse the hypertext (and we might use a custom JSON mediatype to indicate what "data" and "links" mean) and do something with it, we're fine. If we in the future change the domain model to also allow the "resolve" link relation for "OPEN" issues, old clients can ignore it, and new clients can enable new actions in the UI that uses it. In summary, the application layer is a very good candidate to be exposed through a REST API. It encapsulates the application rules for when the various state transitions are allowed, and can make use of user authorization to further enable/disable actions. This takes away a lot of responsibilities from the client, which now also can be "dynamic" in the sense that it can easily react to what state changes are available when by simply checking link availability in the hypermedia returned from the server. The main issue with exposing the application layer through a REST API is that there are pretty much no available frameworks that help you do all this in an easy way. But this is not REST's "fault", obviously, but rather that the "REST" community hasn't yet matured to understand what it should and what it should not do. In the Streamflow project we rolled our own simple framework for doing the above, and I'm very happy with that, but unfortunately most other frameworks seems to be in the "expose your domain model" camp, which means that a lot of this link management is non-trivial to do. This is a fixable situation though. I hope that this post has somewhat clarified what the issues are with exposing infrastructure and domain models through REST API's, and why it's not really a good idea in the first place, and why exposing the application layer really is the logical and simpler option. From http://www.jroller.com/rickard/entry/rest_api_for_infrastructure_domain

Most web applications will have DAO's for accessing the database layer. A DAO provides an interface for some type of database or persistence mechanism, providing CRUD and finders operations without exposing any database details. So, in your application you will have different DAO's for different entities. Most of the time, code that you have written in one DAO will get duplicated in other DAO's because much of the functionality in DAO's is same (like CRUD and finder methods). One of way of avoiding this problem is to have generic DAO and have your domain classes inherit this generic DAO implementation. You can also add finders using Spring AOP; this approach is explained Per Mellqvist in this article. There is a problem with the approach: this boiler plate code becomes part of your application source code and you will have to maintain it. The more code you write, there are more chances of new bugs getting introduced in your application. So, to avoid writing this code in an application, we can use an open source framework called Hades. Hades is a utility library to work with Data Access Objects implemented with Spring and JPA. The main goal is to ease the development and operation of a data access layer in applications. In this article, I will show you how easy it is write DAO's using Hades without writing any boiler plate code. In order to introduce you to Hades, I will show you how we can manage an entity like Book. Before we write any code we need to add the following dependencies to pom.xml. org.synyx.hades org.synyx.hades 2.0.0.RC3 org.hibernate hibernate-entitymanager 3.5.5-Final So, lets start by creating a Book Entity import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; @Entity public class Book { @Id @GeneratedValue(strategy = GenerationType.AUTO) private Long id; @Column(unique = true) private String title; private String author; private String isbn; private double price; // setters and getters } This is a very simple JPA entity without any relationships. Now that we have modeled our entity we need to add a DAO interface for handling persistence operations. You need to create a BookDao interface which will extend GenericDao interface provided by Hades. GenericDao is an interface for generic CRUD operations on a DAO for a specific type. So, we passed the type parameters Book for entity and Long for id. import org.synyx.hades.dao.GenericDao; public interface BookDao extends GenericDao { } GenericDao has an default implementation called GenericJpaDao which provides implementation of all its operations. Now that we have created a BookDao interface, we will configure it in the Spring application context xml. Hades provides a factory bean which will provide the DAO instance for the given interface (in our case BookDao). In the xml shown above I have used a new feature introduced in Spring 3 Embedded Databases to give me the instance of HSQL database datasource. You can refer to my earlier post on Embedded databases in case you are not aware of it. I have used Hibernate as my JPA provider so you need to configure it in persistence.xml as shown below org.hibernate.ejb.HibernatePersistence Next we will write a JUnit test for testing this code. @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration @Transactional public class BookDaoTest { @Autowired private BookDao bookDao; private Book book; @Before public void setUp() throws Exception { book = new Book(); book.setAuthor("shekhar"); book.setTitle("Effective Java"); book.setPrice(500); book.setIsbn("1234567890123"); } @Test public void shouldCreateABook() throws Exception { Book persistedBook = bookDao.save(book); assertBook(persistedBook); } @Test public void shouldReadAPersistedBook() throws Exception { Book persistedBook = bookDao.save(book); Book bookReadByPrimaryKey = bookDao.readByPrimaryKey(persistedBook.getId()); assertBook(bookReadByPrimaryKey); } @Test public void shouldDeleteBook() throws Exception { Book persistedBook = bookDao.save(book); bookDao.delete(persistedBook); Book bookReadByPrimaryKey = bookDao.readByPrimaryKey(persistedBook.getId()); assertNull(bookReadByPrimaryKey); } private void assertBook(Book persistedBook) { assertThat(persistedBook, is(notNullValue())); assertThat(persistedBook.getId(), is(not(equalTo(null)))); assertThat(persistedBook.getAuthor(), equalTo(book.getAuthor())); } } Auto Configuration Using Spring Namespaces The way that we have configured the DAO can become quite cumbersome if the number of DAO's increases. To overcome this we can make use of namespaces to configure daos. This configuration will trigger the auto detection mechanism of DAOs that extend GenericDAO or Extended-GenericDAO. It will create DAO instances for all the DAO interfaces found in this package. You can use or for including or excluding interfaces from getting their beans created. Adding Finders and Query Methods So far we have used the inbuilt operations provided by GenericDao but most of the time we need to add our own finders methods like findByAuthorAndTitle, findWithPriceLessThan . Hades makes it very easy for you to add such methods in your domain dao interface like BookDao. Hades provides 3 strategies for creating JPA query at runtime. These are :- CREATE : This will create a JPA query from method name. This strategy ties you with the method name so you have to think twice before changing the method name. public interface BookDao extends GenericDao { public Book findByAuthorAndTitle(String author, String title); } // test code @Test public void shouldFindByAuthorAndTitle() throws Exception { Book persistedBook = bookDao.save(book); Book bookByAuthorAndTitle = bookDao.findByAuthorAndTitle("shekhar", "Effective Java"); assertBook(bookByAuthorAndTitle); } USE DECLARED QUERY : This lets you define query using JPA @NamedQuery or Hades @Query annotation. If no query is found exception will be thrown. @Query("FROM Book b WHERE b.author = ?1") public Book findBookByAuthorName(String author); // test code public void shouldFindBookByAuthorName() { Book persistedBook = bookDao.save(book); Book bookByAuthor = bookDao.findBookByAuthorName("shekhar"); assertBook(bookByAuthor); } CREATE IF NOT FOUND : It is the combination of both the strategies mentioned above. It will first lookup for the declared query and if it is not found will lookup for method name. This is by default option and you can change it by changing query-lookup-strategy attribute in hades:dao-config element. Hades queries also has the support for pagination and sorting. You can pass the instance of Pageable and Sort to the finder methods create above. public Page findByAuthor(String author, Pageable pageable); public List findByAuthor(String author, Sort sort); Hades is not limited to just limited to CRUD and adding custom finder methods. There are some other features like auditing ,Specifications,etc that I will discuss in second part of this article.

It's been almost 4 years since I wrote a blog post called "EasyMock - Pros, Cons, and Best Practices, and a lot has happened since. You don't hear about EasyMock much any more, and Mockito seems to have replaced it in mindshare. And for good reason: it is better. A Good Humane Interface for Stubbing Just like EasyMock, Mockito allows you to chain method calls together to produce less imperative looking code. Here's how you can make a Stub for the canonical Warehouse object: Warehouse mock = Mockito.mock(Warehouse.class); Mockito.when(mock.hasInventory(TALISKER, 50)). thenReturn(true); I know, I like a crazy formatting. Regardless, giving your System Under Test (SUT) indirect input couldn't be easier. There is no big advantage over EasyMock for stubbing behavior and passing a stub off to the SUT. Giving indirect input with mocks and then using standard JUnit asserts afterwards is simple with both tools, and both support the standard Hamcrest matchers. Class (not just Interface) Mocks Mockito allows you to mock out classes as well as interfaces. I know the EasyMock ClassExtensions allowed you to do this as well, but it is a little nicer to have it all in one package with Mockito. Supports Test Spies, not just Mocks There is a difference between spies and mocks. Stubs allow you to give indirect input to a test (the values are read but never written), Spies allow you to gather indirect output from a test (the mock is written to and verified, but does not give the test input), and Mocks are both (your object gives indirect input to your test through Stubbing and gathers indirect output through spying). The difference is illustrated between two code examples. In EasyMock, you only have mocks. You must set all input and output expectations before running the test, then verify afterwards. // arrange Warehouse mock = EasyMock.createMock(Warehouse.class); EasyMock.expect( mock.hasInventory(TALISKER, 50)). andReturn(true).once(); EasyMock.expect( mock.remove(TALISKER, 50)). andReturn(true).once(); EasyMock.replay(mock); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert EasyMock.verify(mock); That's a lot of code, and not all of it is needed. The arrange section is setting up a stub (the warehouse has inventory) and setting up a mock expectation (the remove method will be called later). The assertion in all this is actually the little verify() method at the end. The main point of this test is that remove() was called, but that information is buried in a nest of expectations. Mockito improves on this by throwing out both the record/playback mode and a generic verify() method. It is shorter and clearer this way: // arrange Warehouse mock = Mockito.mock(Warehouse.class); Mockito.when(mock.hasInventory(TALISKER, 50)). thenReturn(true); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert Mockito.verify(warehouse).remove(TALISKER, 50); The verify step with Mockito is spying on the results of the test, not recording and verifying. Less code and a clearer picture of what really is expected. Update: There is a separate Spy API you can use in Mockito as well: http://mockito.googlecode.com/svn/branches/1.8.3/javadoc/org/mockito/Mockito.html#13 Better Void Method Handling Mockito handles void methods better than EasyMock. The fluent API works fine with a void method, but in EasyMock there were some special methods you had to write. First, the Mockito code is fairly simple to read: // arrange Warehouse mock = Mockito.mock(Warehouse.class); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert Mockito.verify(warehouse).remove(TALISKER, 50); Here is the same in EasyMock. Not as good: // arrange Warehouse mock = EasyMock.createMock(Warehouse.class); mock.remove(TALISKER, 50); EasyMock.expectLastMethodCall().once(); EasyMock.replay(mock); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert EasyMock.verify(mock); Mock Object Organization Patterns Both Mockito and EasyMock suffer from difficult maintenance. What I said in my original EasyMock post holds true for Mockito: The method chaining style interface is easy to write, but I find it difficult to read. When a test other than the one I'm working on fails, it's often very difficult to determine what exactly is going on. I end up having to examine the production code and the test expectation code to diagnose the issue. Hand-rolled mock objects are much easier to diagnose when something breaks... This problem is especially nasty after refactoring expectation code to reduce duplication. For the life of me, I cannot follow expectation code that has been refactored into shared methods. Now, four years later, I have a solution that works well for me. With a little care you can make your mocks reusable, maintainable, and readable. This approach was battle tested over many months in an Enterprise Environment(tm). Create a private static method the first time you need a mock. Any important data needs to be passed in as a parameter. Using constants or "magic" fields hides important information and obfuscates tests. For example: User user = createMockUser("userID", "name"); ... assertEquals("userID", result.id()); assertEquals("name", result.name(); Everything important is visible and in the test, nothing important is hidden. You need to completely hide the replay state behind this factory method if you're still on EasyMock. The Mock framework in use is an implementation detail and try not to let it leak. Next, as your dependencies grow, be sure to always pass them in as factory method parameters. If you need a User and a Role object, then don't create one method that creates both mocks. One method instantiates one object, otherwise it is a parameter and compose your mock objects in the test method: User user = createMockUser( "userID", "name", createMockRole("role1"), createMockRole("role2") ); When each object type has a factory method, then it makes it much easier to compose the different types of objects together. Reuse. But you can only reuse the methods when they are simple and with few dependencies, otherwise they become too specific and difficult to understand. The first time you need to reuse one of these methods, then move the method to a utility class called "*Mocker", like UserMocker or RoleMocker. Follow a naming convention so that they are always easy to find. If you remembered to make the private factory methods static then moving them should be very simple. Your client code ends up looking like this, but you can use static imports to fix that: User user = UserMocker.createMockUser( "userID", "name", RoleMocker.createMockRole("role1"), RoleMocker.createMockRole("role2") ); User overloaded methods liberally. Don't create one giant method with every possible parameter in the parameter list. There are good reasons to avoid overloading in production, but this is test. Use overloading so that the test methods only display data relevant to that test and nothing more. Using Varargs can also help keep a clean test. Lastly, don't use constants. Constants hide the important information out of sight, at the top of the file where you can't see it or in a Mocker class. It's OK to use constants within the test case, but don't define constants in the Mockers, it just hides relevant information and makes the test harder to read later. Avoid Abstract Test Cases Managing mock objects within abstract test cases has been very difficult for me, especially when managing replay and record states. I've given up mixing mock objects and abstract TestCase objects. When something breaks it simply takes too long to diagnose. An alternative is to create custom assertion methods that can be reused. Beyond that, I've given up on Abstract TestCase objects anyway, on the grounds of preferring composition of inheritance. Don't Replace Asserts with Verify My original comments about EasyMock are still relevant for Mockito: The easiest methods to understand and test are methods that perform some sort of work. You run the method and then use asserts to make sure everything worked. In contrast, mock objects make it easy to test delegation, which is when some object other than the SUT is doing work. Delegation means the method's purpose is to produce a side-effect, not actually perform work. Side-effect code is sometimes needed, but often more difficult to understand and debug. In fact, some languages don't even allow it! If you're test code contains assert methods then you have a good test. If you're code doesn't contain asserts, and instead contains a long list of verify() calls, then you're relying on side effects. This is a unit-test bad smell, especially if there are several objects than need to be verified. Verifying several objects at the end of a unit test is like saying, "My test method needs to do several things: x, y, and z." The charter and responsibility of the method is no longer clear. This is a candidate for refactoring. No More All or Nothing Testing Mockito's verify() methods are much more flexible than EasyMock's. You can verify that only one or two methods on the mock were called, while EasyMock had just one coarse verify() method. With EasyMock I ended up littering the code with meaningless expectations, but not so in Mockito. This alone is reason enough to switch. Failure: Expected X received X For the most part, Mockito error messages are better than EasyMock's. However, you still sometimes see a failure that reads "Failure. Got X Expected X." Basically, this means that your toString() methods produce the same results but equals() does not. Every user who starts out gets confused by this message at some point. Be Warned. Don't Stop Handrolling Mocks Don't throw out hand-rolled mock objects. They have their place. Subclass and Override is a very useful technique for creating a testing seam, use it. Learn to Write an ArgumentMatcher Learn to write an ArgumentMatcher. There is a learning curve but it's over quickly. This post is long enough, so I won't give an example. That's it. See you again in 4 years when the next framework comes out! From http://hamletdarcy.blogspot.com/2010/09/mockito-pros-cons-and-best-practices.html

The Separated Interface pattern can often be used to provide hook points to client code, in the form of interfaces to implement or classes to extend with client code. The right implementation to use in a part of the system can then be chosen via configuration: the Factory or Dependency Injection container with the largest scope would process the configuration and execute conditionals only one time, and inject the right Plugin as a collaborator of a standard object. This pattern is a evolution of the Separated Interface one, where the implementor package is not even under your maintenance, but it is provided by some external developer that links his code to your work. Implementation In PHP the concept of compile time does not exist, apart from the just-in-time cached compilation of the scripts to operation codes, a phrase which you can peacefully ignore if you are not into caching. By the way, even if some checks are performed while loading and parsing the PHP code, PHP is by design a dynamic language where you can write nearly everything and it will not explode until executed. This design leaves open many possibilities for inserting plugins, but due to the lack of compile there is often a lack of a clean separation between code and configuration. For example, database credentials are embedded in PHP code more often than in other languages. Think now of a framework or a library: you cannot change the code but you must adapt or create a configuration to make it work. To implement a Plugin pattern, your application should strive towards the flexibility of a library: think of your production code as external and untouchable, and try to deploy a particular configuration to make it work and to modify a functionality. For example, extract it in a temporary working copy with svn checkout or git clone and hook in the necessary extensions. When you succeed, and your svn diff or git diff is clean, you'll have implemented a Plugin system. Modification of vendor code (and you are the vendor here) is out of the question. Future changes Kent Beck says in Implementation Patterns that providing hooks via implementation and inheritance is one of the most effective ways to tie a framework down from future evolution. For example, once you have published an interface, you cannot add methods to it without breaking all the implementors. You can publish versioned interfaces, but this adds complexity to your application. With a published abstract class instead, you can include a default implementation for new methods, but you can't remove methods or refactor protected members without breaking Plugin implementators. This is the specular situation of providing an interface. Zend Framework includes both an interface and an abstract class for most of its components, but it does not get right the management of extension points (at least in the 1.x branch). When including the possibility of Plugins in your application, default as much as possible to private visibility and hide the internals of your Plugin hook point. What is left to protected is a seam that screams "extend me", and the interfaces not marked as internal will be implemented by someone else. There is no built-in language mechanism to protect interfaces,m so you'll have to rely on some kind of convention (like a particular prefix or namespace), but for private methods left to protected scope we can only blame ourselves. Configuration The configuration of your Plugin system can be managed with solution of different levels of complexity, each more powerful than the previous ones. Of course, you shouldn't provide a needlessly complex system when all you need is a class name. The first solution is indeed to insert class names into configuration files. This is a totally declarative approach, which uses simple INI files. This is commonly done in Zend Framework, for example with bootstrap resources, and in some cases can even manage dependencies of the Plugins. Bootstrap resources can request other object of the same kind, but cannot pull in arbitrary collaborators (unless they create them by themselves... ugly if you know what DI is). A second, widely applicable solution is to request Factory objects. this solution still involves writing PHP code, but it is one step towards textual configuration. However, a Factory object can fetch and inject all the dependencies into a Plugin without cluttering it with this kind glue code (only a constructor or some setters). The problem with Factories is that they tend to contain all the same boilerplate code. A third solution can be used to provide quick construction of objects: Dependency Injection containers, which have recently been introduced even in PHP. A DI container is configured textually, via an XML or INI file containing parameters like the collaborators each object requires, its lifetime, and so on. DI containers are probably the future of flexible PHP applications, but beware of growing too dependent on them: they are a library like every other open source component, and should be isolated from your code as much as possible like you would do with your models and Doctrine 2, or your services and Zend Framework. Example The code sample shows hot to predispose a class for receiving an injected simple Factory that manages user-defined plugins. // plugin_view.php formatDate(time()), ".\n"; factory = $factory; } public function render($script) { include $script; } /** * Forwards the call to the View Helper invoked. */ public function __call($name, $args) { $callback = $this->factory->getHelper($name); return call_user_func_array($callback, $args); } } /** * Extension code. */ class UserDefinedFactory implements ViewHelperFactory { private $helpers; /** * In this example, we only define a simple Plugin for * formatting dates using PHP's internal function. */ public function __construct() { $this->helpers = array( 'formatDate' => function($time) { return date('Y-m-d', $time); } ); } public function getHelper($name) { return $this->helpers[$name]; } } // client code $view = new View(new UserDefinedFactory); $view->render('plugin_view.php');

Originally Barcodes were 1D representation of data using width and spacing of bars. Common bar code types are UPC barcodes which are seen on product packages. There are 2D barcodes as well (they are still called Barcodes even though they don’t use bars). A common example of 2D bar code is QR code (shown on right) which is commonly used by mobile phone apps. You can read history and more info about Barcodes on Wikipedia. There is an open source Java library called ‘zxing’ (Zebra Crossing) which can read and write many differently types of bar codes formats. I tested zxing and it was able to read a barcode embedded in the middle of a 100 dpi grayscale busy text document! This article demonstrates how to use zxing to read and write bar codes from a Java program. Getting the library It would be nice if the jars where hosted in a maven repo somewhere, but there is no plan to do that (see Issue 88). Since I could not find the binaries available for download, I decided to download the source code and build the binaries, which was actually quite easy. The source code of the library is available on Google Code. At the time of writing, 1.6 is the latest version of zxing. 1. Download the release file ZXing-1.6.zip (which contains of mostly source files) from here. 2. Unzip the file in a local directory 3. You will need to build 2 jar files from the downloaded source: core.jar, javase.jar Building core.jar cd zxing-1.6/core mvn install cd zxing-1.6/core mvn install This will install the jar in your local maven repo. Though not required, you can also deploy it to your company’s private repo by using mvn:deploy or by manually uploading it to your maven repository. There is an ant script to build the jar as well. Building javase.jar Repeat the same procedure to get javase.jar cd zxing-1.6/javase mvn install cd zxing-1.6/javase mvn install Including the libraries in your project If you are using ant, add the core.jar and javase.jar to your project’s classpath. If you are using maven, add the following to your pom.xml. com.google.zxing core 1.6-SNAPSHOT com.google.zxing javase 1.6-SNAPSHOT com.google.zxing core 1.6-SNAPSHOT com.google.zxing javase 1.6-SNAPSHOT Once you have the jars included in your project’s classpath, you are now ready to read and write barcodes from java! Reading a Bar Code from Java You can read the bar code by first loading the image as an input stream and then calling this utility method. InputStream barCodeInputStream = new FileInputStream("file.jpg"); BufferedImage barCodeBufferedImage = ImageIO.read(barCodeInputStream); LuminanceSource source = new BufferedImageLuminanceSource(barCodeBufferedImage); BinaryBitmap bitmap = new BinaryBitmap(new HybridBinarizer(source)); Reader reader = new MultiFormatReader(); Result result = reader.decode(bitmap); System.out.println("Barcode text is " + result.getText()); InputStream barCodeInputStream = new FileInputStream("file.jpg"); BufferedImage barCodeBufferedImage = ImageIO.read(barCodeInputStream); LuminanceSource source = new BufferedImageLuminanceSource(barCodeBufferedImage); BinaryBitmap bitmap = new BinaryBitmap(new HybridBinarizer(source)); Reader reader = new MultiFormatReader(); Result result = reader.decode(bitmap); System.out.println("Barcode text is " + result.getText()); Writing a Bar Code from Java You can encode a small text string as follows: String text = "98376373783"; // this is the text that we want to encode int width = 400; int height = 300; // change the height and width as per your requirement // (ImageIO.getWriterFormatNames() returns a list of supported formats) String imageFormat = "png"; // could be "gif", "tiff", "jpeg" BitMatrix bitMatrix = new QRCodeWriter().encode(text, BarcodeFormat.QR_CODE, width, height); MatrixToImageWriter.writeToStream(bitMatrix, imageFormat, new FileOutputStream(new File("qrcode_97802017507991.png"))); String text = "98376373783"; // this is the text that we want to encode int width = 400; int height = 300; // change the height and width as per your requirement // (ImageIO.getWriterFormatNames() returns a list of supported formats) String imageFormat = "png"; // could be "gif", "tiff", "jpeg" BitMatrix bitMatrix = new QRCodeWriter().encode(text, BarcodeFormat.QR_CODE, width, height); MatrixToImageWriter.writeToStream(bitMatrix, imageFormat, new FileOutputStream(new File("qrcode_97802017507991.png"))); In the above example, the bar code for “97802017507991″ is written to the file “qrcode_97802017507991.png” (click to see the output). JavaDocs and Documentation The Javadocs are part of the downloaded zip file. You can find a list of supported bar code formats in the Javadocs. Open the following file to see the javadocs. zxing-1.6/docs/javadoc/index.html zxing-1.6/docs/javadoc/index.html From http://www.vineetmanohar.com/2010/09/java-barcode-api/

In the first and second parts of this series I talked about some of the new features like enum and concurrency support that have been added in commons-lang 3. In this article, I am going to talk about a new package 'org.apache.commons.lang3.text.translate' which has been added in commons-lang 3. This package is added to fix problems in the design and implementation of the StringEscapeUtils class which exists in versions prior to 3.0. To make it clearer, let's first talk about the purpose of StringEscapeUtils class and the problems it had prior to version 3. Purpose of StringEscapeUtils StringEscapeUtils is a utility class which escapes and unescapes String for Java, JavaScript, HTML, XML, and SQL. For example, @Test public void test_StringEscapeUtils() { assertEquals("\\\\\\n\\t\\r", StringEscapeUtils.escapeJava("\\\n\t\r")); // escapes the Java String assertEquals("\\\n\t\r",StringEscapeUtils.unescapeJava("\\\\\\n\\t\\r")); //unescapes the Java String assertEquals("I didn\\'t say \\\"you to run!\\\"",StringEscapeUtils.escapeJavaScript("I didn't say \"you to run!\""));//escapes the Javascript assertEquals("<xml>", StringEscapeUtils.escapeXml(""));//escapes the xml } Problems with StringEscapeUtils There were a lot of problems in the StringEscapeUtils implementation prior to version3. Some of these were: The implementation was not extensible. Let's take an example of escapeJava, suppose we want to add support in the escapeJava method that it should start escaping single quotes. To add such support we would have to change the existing class code and another if condition which if satisfied will escape single quotes. So, the API was breaking the open-closed principle i.e. a class should be open for extension and closed for modification. It was not symmetric i.e. original should be equal to unescape(escape(original)) but it was not the case. StringEscapeUtils.escapeHtml() escapes multibyte characters like Chinese, Japanese etc. Issue 339 @Test public void testEscapeHiragana() { // Some random Japanese unicode characters String original = "\u304B\u304C\u3068"; String escaped = StringEscapeUtils.escapeHtml(original); assertEquals(original, escaped); } StringEscapeUtils.escapeHtml incorrectly converts unicode characters above U+00FFFF into 2 characters. Issue 480 @Test public void testEscapeHtmlHighUnicode() throws java.io.UnsupportedEncodingException { byte[] data = new byte[] { (byte) 0xF0, (byte) 0x9D, (byte) 0x8D,(byte) 0xA2 }; String original = new String(data, "UTF8"); String escaped = StringEscapeUtils.escapeHtml(original); assertEquals(original, escaped); } StringEscaper.escapeXml() escapes characters > 0x7f . Issue 66 @Test public void shouldNotEscapeValuesGreaterThan0x7f() { assertEquals("XML should not escape >0x7f values", "\u00A1",StringEscapeUtils.escapeXml("\u00A1")); } Solution -- Rewritten StringEscapeUtils In version 3.0, StringEscapeUtils is completely rewritten to fix all the bugs associated with this class and to provide a way for the user to customize the behavior of its methods. They have moved all the logic present in the StringEscapeUtils to the classes in the package 'org.apache.commons.lang3.text.translate'. Let's take an example of escapeJava function in StringEscapeUtils, escapeJava function does not contain any business logic, it just calls the translate method on CharSequenceTranslator reference. What they did can be best understood by looking at the code below public static final CharSequenceTranslator ESCAPE_JAVA = new AggregateTranslator(new LookupTranslator( new String[][] { {"\"", "\\\""}, {"\\", "\\\\"}, }),new LookupTranslator(EntityArrays.JAVA_CTRL_CHARS_ESCAPE()),UnicodeEscaper.outsideOf(32, 0x7f)); and in the escapeJava method public static final String escapeJava(String input) { return ESCAPE_JAVA.translate(input); } A constant of type CharSequenceTranslator was assigned an AggregateTranslator object. AggregateTranslator can take an array of translators, and it iterates over each of them. The LookupTranslator replaces the string at zeroth index with the string at the first index. UnicodeEscaper translates values outside the given range to unicode values. As you can see, you can very easily write your own escape methods. For example, if you want to add the support of escaping &, you can do it like this public static final CharSequenceTranslator ESCAPE_JAVA = new LookupTranslator( new String[][] { {"\"", "\\\""}, {"\\", "\\\\"}, }).with(new LookupTranslator( new String[][]{ {"&", "&"}, {"<", "<"} )).with( new LookupTranslator(EntityArrays.JAVA_CTRL_CHARS_ESCAPE()) ).with( UnicodeEscaper.outsideOf(32, 0x7f) ); StringEscapeUtils.escapeSql has been removed from the API as it was misleading developers to not use PreparedStatement.This method was not of much use as it was only escaping single quotes.

this post describes how to configure the jetty-maven-plugin and the jetty servlet container to run a web application that uses a data source and requires users to log in, which are the basic requirements of most web applications. i use jetty in development because it’s fast and easy to work with. why jetty? well, because it’s much faster then the websphere as i normally use and it really well supports fast (or shall i say agile? ) development thanks to its fast turnaround. and because it’s simply cool to type bash$ svn checkout http://example.com/repo/trunk/mywebapp bash$ cd mywebapp bash$ mvn jetty:run bash$ firefox http://localhost:8080/mywebapp and to be able to immediatelly log into and interact with the application. however it should be noted that jetty isn’t a full-featured javaee server and thus may not be always usable. project setup general configuration you need to add the jetty plugin to your pom.xml : 4.0.0 com.example mywebapp war ... ... org.mortbay.jetty maven-jetty-plugin 6.1.0 3 ... ... ... as you can see, i’m using jetty 6.1.0. defining a datasource let’s assume that the application uses a datasource configured at the server and accesses it normally via jndi. then we must define a reference to the data source in src/main/webapp/ web-inf/web.xml : ... ... ... jdbc/lmsdb javax.sql.datasource container shareable next we need to describe the datasource to jetty. there are multiple ways to do that, i’ve chosen to do so in src/main/webapp/ web-inf/jetty-env.xml : jdbc/lmsdb lmsdb myuser secret db.toronto.ca.ibm.com 3711 notice that the class used is db2simpledatasource and not a jdbc driver. that is, of course, because we need a datasource, not a driver. the jetty wiki pages also contain examples of datasource configuration for other dbs . finally we must make the corresponding jdbc implementation available to jetty by adding it to the plugin’s dependencies in the pom.xml : org.mortbay.jetty maven-jetty-plugin 6.1.0 <... com.ibm.db2 db2jcc 9.7 jar system ${basedir}/../lms.sharedlibraries/db2/db2jcc.jar com.ibm.db2 db2jcc_license_cisuz 9.7 jar system ${basedir}/../lms.sharedlibraries/db2/db2jcc_license_cisuz.jar please do not scorn me for using system-scoped dependencies , sometimes that is unfortunatelly the most feasible way. enabling security and configuring an authentication mechanism we would like to limit access to the application only to the authenticated users in the admin role with the exception of pages under public/. therefore we declare the appropriate security constraints in web.xml: ... authorizedusers all urls /* admin publicaccess public pages /public/* basic learning@ibm mini person feed management administrator access admin ... beware that jetty doesn’t support https out of the box and thus if you will add the data constraint confidential to any resource, you will automatically get http 403 forbidden no matter what you do. that’s why i’ve commented it out above. it is possible to enable ssl in jetty but i didn’t want to bother with certificate generation etc. next we need to tell jetty how to authenticate users. this is done via realms and we will use the simplest, file-based one. again there are multiple ways to configure it, for example in the pom.xml : org.mortbay.jetty maven-jetty-plugin 6.1.0 3 learning@ibm mini person feed management src/test/resources/jetty-users.properties ... the name must match exactly the realm-name in web.xml. you then define the users and their passwords and roles in the declared file, in this case in src/test/resources/ jetty-users.properties : user=psw,admin the format of the file is username=password[,role1,role2,...]. when you download jetty, you will find a fine example of using jaas with a file-based back-end for authentication and authorization under examples/test-jaas-webapp (invoke mvn jetty:run from the folder and go to http://localhost:8080/jetty-test-jaas/). however it seems that jaas causes an additional overhead visible as a few-seconds delay when starting the server so it might be preferrable not to use it. conclusion with jetty it’s easy to enable security and create a data source, which are the basic requirements of most web applications. anybody can then very easily run the application to test and develop it. development is where jetty really shines provided that you don’t need any feature it doesn’t have. when troubleshooting, you may want to tell jetty to log at the debug level with mvn -ddebug .. or to log requests , which can be also configured in the jetty-env.xml. beware that this post describes configuration for jetty 6.1.0. it can be different in other versions and it certainly is different in jetty 7. from http://theholyjava.wordpress.com/2010/09/10/jetty-maven-plugin-running-a-webapp-with-a-datasource-and-security/

The new Apple Magic Mouse is a controversial piece of hardware. Most people either really hate it or adore it. Personally, I think it is probably the best mouse I've ever used. There's a lot of criticism regarding the low profile of the mouse, suggesting it is not ergonomic. From my experience, I don't have any more wrist pains since I started using it. Ergonomics aside, the highlight of the mouse is the upper multi-touch surface with the enabled gestures. I use the back and forward gestures a lot. Especially when browsing the web. Swipe two fingers to the left and go back. Swipe right to go forward. It is very easy to get used to it. It works in web browsers, it works in Finder windows and native applications are adding support as well. However, it doesn't work in Eclipse. I want to swipe back and forward when browsing code. Back, go to previous location. Forward, return to the next location. I incidentally found a solution for that. There are many programs on the market that augment the Magic Mouse behavior. The reason for their existence is because Apple provides very limited gesture functionality. Other than back/forward and scroll, there's simply no support for other functions, not even Exposé or Spaces which were supported in the previous Mighty Mouse and are supported on the multi-touch trackpads. The most popular tools are MagicPrefs and BetterTouchTool (both free) but there are many others, free and commercial. Personally, I use MagicDriver, which is commercial (free while in beta). The reason I prefer it is because it has much lower CPU utilization, which was an issue for me in MagicPrefs. MagicDriver replaces the back/forward gestures with their keyboard equivalent: ⌘+[ and ⌘+]. These shortcuts are commonly used in OS X. Eclipse, by default, also uses these keyboard shortcuts to navigate back and forward. It just works. MagicPrefs and BetterTouchTool will require some customization: you can define the two finger swipe left and right to fire these keyboard shortcuts rather than use the default back/forward functionality. If you use a newer MacBook with a multi-touch trackpad or a Magic Trackpad, you can achieve the same functionality by using BetterTouchTool. AFAIK MagicPrefs does not support it and the current version of MagicDriver doesn't support it either. BetterTouchTool also has the ability to define gestures per application, so you can customize the behavior specifically for Eclipse and leave it as is for the rest of the applications. If you are new to these tools, I should warn you: defining too many gestures doesn't work very well. There are tons of options and it is very easy to get carried away and use as much as you can. However, there's probably a reason why Apple did not include support for all those gestures in the first place. It is very easy to "miss-fire" and perform gestures by accident. You don't always pay close attention to the number of fingers you have on the surface, so mistakes are very common. I just use a 3-finger click for expose. Don't be greedy and it will work just fine. Finally, if you want proper native support for back/forward gestures in Eclipse, you can vote for this bug.

whenever you hover over any piece of code in eclipse, it pops up a tooltip that displays more information about the item, such as its declaration, variable values or javadoc information, as in the example below. although useful at times , this becomes extremely annoying after a while, especially when you’re using your mouse to browse some code. popup after popup of unwanted information keeps obscuring your view of the code, leading to some lengthy expletives and big productivity loss. it’s useful information, but not every time all the time , almost like your car’s gps giving you directions to 10 different places at once while you’re still parked in the driveway. luckily there is a way to alleviate the problem and all it takes is changing some preferences in eclipse. we don’t want to completely disable tooltips (they can be useful), so i’ll show you how to tell eclipse to bring up the tooltips only when you request them. show tooltips only on request to tell eclipse to only show tooltips on request, do the following: go to window > preferences > java > editor > hovers . select the source item in the list. make sure the entire row is highlighted and that the checkbox is selected. move focus to the pressed key modifier while hovering field, positioning the cursor at the end of the default value shift . press and release ctrl . the value should now read shift+ctrl . if you made a mistake, just clear the field by pressing backspace or delete and try again. select the combined hover item in the list. move focus to the pressed key modifier while hovering field and press shift . the value should now read shift . click ok to accept the changes. the preferences should look something like this: you should now notice the following: if you want to show the javadoc or the declaration of some element hold down shift while hovering over the element. variable values won’t popup automatically anymore. to show a variable’s value in debug mode, press shift and hover over the variable. this is probably the only time when it’s a bit of a hassle to press shift, but the pros totally outweigh the cons here as i view variable values a lot less than the number of times annoying tooltips appear. if you want to show a quick preview of a method’s code, press ctrl+shift and hover over the method. this is a nice feature combined with the tooltip enrichment eclipse provides, so you can view a method’s code in a scrollable tooltip without having to navigate to the method and back again. other options – configure, disable or use the keyboard you can, of course, choose your own modifier keys in preferences, but these are the ones i found that work the best, especially given that ctrl and a click is a way to go to an element’s declaration or implementation in eclipse. i’ve found shift to conflict the least with existing features and easy enough to use on a frequent basis. keep in mind that 2 hovers can’t share the same modifier key (which is why we reassigned the source hover to use shift+ctrl ). you can also disable tooltips completely by deselecting the combined hover checkbox in the preference above, but i’ve found the shift key to be a nice middle ground so keep it enabled. to get similar information using the keyboard, press f2 while the cursor’s positioned on the code. to show variable values in debug mode, press ctrl+shift+d . a nice tip if you’ve disabled tooltips completely. from http://eclipseone.wordpress.com/2010/08/24/configure-tooltips-in-eclipse-to-only-popup-on-request/

In this series we are now entering the realm of concurrency, an option which adds complexity to an application as many different threads of execution are accessing the state storage at the same time. There is no native multithreading support in PHP (every script gets its own isolated process), but still concurrency can easily become an issue: multiple clients from all over the world continuosly make requests to PHP applications, and they can easily mutually overwrite their changesets. A classic example of race condition in the PHP world is two different clients filling an editing form referred to the same entity. They both will submit the form once it is complete, and the first one will get his changes overwritten by the second request. There are many other common situations were concurrent user can provoke errors in the system. Just think of different people choosing the same username and being told via Ajax that it is available; the slower one of them will be surprised when he submits his registration form. Background Before entering the explanation of patterns emerged to solve the concurrency problems, we need some definitions. First of all the notion of transaction is necessary: we define a transaction as a change of state of the application. Editing an entity is a transaction; adding or deleting another is still a transaction. The first kind of transaction we are interested in is the database transactions, which is totally accomplished in one PHP script. This is usually automatically enforced by mechanisms supported at the database level. The other kind of transaction is the business transaction, which spans over multiple HTTP requests and makes uses of from one to N database transactions. It comprehends checking out data, populating a form or other kind of rich user interface, modifying or adding data (a human-based action), and sending it back to the server. There is no automatic enforcement for business transaction, since they are defined by the business rules of the domain. This is not a problem that originates because of PHP nature, but because of the separation between client and server which the web is based on. Optimistic lock The Optimistic Offline Lock pattern is a way of ensuring integrity of data, avoiding the option that different clients submit conflicting changes. As the name suggests, it assumes that the chances of conflict are low. Indeed, when this is the case the optimistic lock does not slow down the user interaction a bit. The goal of this pattern is detecting a conflicting change and instead of applying it, rollback the business transaction and present an error to the user. It accomplishes this goal by validating that no one else has tampered with a record in the data source prior to allowing the modification to be committed. All open source version control systems such as Subversion and Git implement optimistic lock: anyone may check out a source file and work on it, to end his little fork later with a commit or push. The pain comes while merging, so you are supposed to integrate often. We also borrowed terminology from the source control systems, so in this article you'll encounter terms like commit, checkout, and merge. Implementation The most common implementation of Optimistic Offline Lock is a numerical version field on the record to protect from condurrency issues; to aid rollback notifications, other additional fields are useful for signaling the conflict, like the id of the last user that modified the record. The pattern inner working is not complex: when the data is submitted along with the version field value kept on the client, the version field in it must be the same currently present in the database. Only then it is incremented and the changeset committed. Encountering a different version field value in the database record means someone else has modified the data in between our checkout and commit, and so it must be preserved. For example we can show a diff to the user, like VCS does; in any case, we should interrupt the transaction. RDBMS and ORMs can simply use an additional column on the table where the root object is stored to support this pattern. Alternatives An alternative implementation consists in using all fields in the WHERE clause of the UPDATE (or only the sensible ones, or only the modified ones to let transactions that affect different field succeed when the business logic allows it. See below). This solution is handy when you can't add a version field, but it may have performance impact. Another alternative is to check conditions instead of version fields, which is practical in different use cases. For example, we can check the existence of a record before deleting it. This is already indirectly done to a cartain extent by ORMs and other abstraction layers when they provide you with an object abstraction you can calla delete() method on. An extension to the functionality of this pattern is checking that a current editing will (probably) commit, as a feature available at any time during editing of the data. This feature should check that the checkout data is still current. The domain An important is that it is part of the job of business domain logic to decide when a conflict occurs: some concurent changesets may be acceptable, while others may not be allowed even if they modify different fields. In his book, Fowler makes the example of adding elements to a collection concurrently. We can't know if this is right by seeing that the object is a collection, because it is the abstraction that it represent that must be maintained valid: sometimes it is right to add elements, sometimes the transaction should be stopped. Also merging strategies, which solve the conflicts, are subsceptible to domain considerations. Some are valuable and should be pursued, while some are costly and a rollback with manual user editing of the data is fine. Advantages The greatest advantage of this pattern is that it can support real time concurrency, like the check out of multiple items by multiple users simultaneously, as long as there is a merging strategy in place. It can also easily prevent race conditions. This pattern is also easy to implement, and thus it is the default choice to solve concurrency issues. Disadvantages When the conflict probability is high, since there are many concurrent transactions, this pattern produces too many rollbacks. It is not adequate for use cases where a pessimistic pattern should be adopted. Examples Doctrine 2 uses natively database transactions: it only commits the changes made in a PHP script when the EntityManager::flush() method is called. It automatically rolls back if an error is detected. Besides that, Doctrine 2 has also automatic Optimistic Offline Locking support, via the addition of a version field to the entity to lock.

Yesterday I wondered if the copyFile method in JTheque Utils was the best method or if I need to change. So I decided to do a benchmark. So I searched all the methods to copy a File in Java, even the bad ones and found the following methods : Naive Streams Copy : Open two streams, one to read, one to write and transfer the content byte by byte. Naive Readers Copy : Open two readers, one to read, one to write and transfer the content character by character. Buffered Streams Copy : Same as the first but using buffered streams instead of simple streams. Buffered Readers Copy : Same as the second but using buffered readers instead of simple readers. Custom Buffer Stream Copy : Same as the first but reading the file not byte by byte but using a simple byte array as buffer. Custom Buffer Reader Copy : Same as the fifth but using a Reader instead of a stream. Custom Buffer Buffered Stream Copy : Same as the fifth but using buffered streams. Custom Buffer Buffered Reader Copy : Same as the sixth but using buffered readers. NIO Buffer Copy : Using NIO Channel and using a ByteBuffer to make the transfer. NIO Transfer Copy : Using NIO Channel and direct transfer from one channel to other. I think, this is the ten principal methods to copy a file to another file. The different methods are available at the end of the post. Pay attention that the methods with Readers only works with text files because Readers are using character by character reading so it doesn’t work on a binary file like an image. Here I used a buffer size of 4096 bytes. Of course, use a higher value improve the performances of custom buffer strategies. For the benchmark, I made the tests using different files. Little file (5 KB) Medium file (50 KB) Big file (5 MB) Fat file (50 MB) And I made the tests first using text files and then using binary files. The source file is not on the same hard disk as the target file. I used a benchmark framework, described here, to make the tests of all the methods. The tests have been made on my personal computer (Ubuntu 10.04 64 bits, Intel Core 2 Duo 3.16 GHz, 6 Go DDR2, SATA Hard Disks). And after a long time of bench, here are the results : Little Text File - All results We see that the method with a simple stream (Naive Streams) is from far the slowest followed by the simple readers methods (Naive Readers). The readers method is a lot faster than the simple stream because FileReader use a buffer internally. To see what happens to the other, here are the same graph but without the first two methods : Little Text File - Best results The best two versions are the Buffered Streams and Buffered Readers. Here this is because the buffered streams and readers can write the file in only one operation. Here the times are in microseconds, so there is really little differences between the methods. So the results are not really relevant. Now, let’s test with a bigger file. Medium Text File We can see that the versions with the Readers are a little slower than the version with the streams. This is because Readers works on character and for every read() operation, a char conversion must be made, and the same conversion must be made on the other side. Another observation is that the custom buffer strategy is faster than the buffering of the streams and than using custom buffer with a buffered stream or a single stream doesn’t change anything. The same observation can be made using the custom buffer using readers, it’s the same with buffered readers or not. This is logical, because with custom buffer we made 4096 (size of the buffer) times less invocations to the read method and because we ask for a complete buffer we have not a lot of I/O operations. So the buffer of the streams (or the readers) is not useful here. The NIO buffer strategy is almost equivalent to custom buffer. And the direct transfer using NIO is here slower than the custom buffer methods. I think this is because here the cost of invoking native methods in the operating system level is higher than simply the cost of making the file copy. Big Text File - All results Here we see that the Naive Readers shows its limit when the file size if growing. So let’s concentrate us on the best methods only, namely, remove the Naive Readers : Big Text File - Best results Here, it’s now clear that the custom buffer strategy is a better than the simple buffered streams or readers and that using custom buffer and buffered streams is really useful for bigger files. The Custom Buffer Readers method is better than Custom Buffer Streams because FileReader use a buffer internally. And now, continue with a bigger file : Fat Text File Results You can see that it doesn’t take 500 ms to copy a 50 MB file using the custom buffer strategy and that it even doesn’t take 400 ms with the NIO Transfer method. Really quick isn’t it ? We can see that for a big file, the NIO Transfer start to show an advantage, we’ll better see that in the binary file benchmarks. We will directly start with a big file (5 MB) for this benchmark : Big Binary File Results So we can make the same conclusion as for the text files, of course, the buffered streams methods is not fast. The other methods are really close. Fat Binary File Results We see here again that the NIO Transfer is gaining advantages more the files is bigger. And just for the pleasure, a great file (1.3 GB) : Enormous Binary File Results We see that all the methods are really close, but the NIO Transfer method has an advantage of 500 ms. It’s not negligible. Conclusion In conclusion, the NIO Transfer method is the best one for big files but it’s not the fastest for little files (< 5 MB). But the custom buffer strategy (and the NIO Buffer too) are also really fast methods to take files. So perhaps, the best method is a method that make a custom buffer strategy on the little files and a NIO Transfer on the big ones. But it will be interesting to also make the tests on an other computer and operating system. We can take several rules from this benchmark : Never made a copy of file byte by byte (or char by char) Prefer a buffer in your side more than in the stream to make less invocations of the read method, but don’t forget the buffer in the side of the streams Pay attention to the size of the buffers Don’t use char conversion if you only need to tranfer the content of a file Don’t hesitate to use channels to make file transfer, it’s the fastest way to make a file transfer I’ve also made some tests, but not complete, for files in the same hard disk and here the NIO Transfer method is a lot faster than the other. I think this is because on the same disk this method can make better use of the filesystem cache. I hope this benchmark (and its results) interested you. Here are the sources of the benchmark : Java Benchmark of File Copy methods From http://www.baptiste-wicht.com/2010/08/file-copy-in-java-benchmark