DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Performance Topics

article thumbnail
Apache Solr: Get Started, Get Excited!
we've all seen them on various websites. crappy search utilities. they are a constant reminder that search is not something you should take lightly when building a website or application. search is not just google's game anymore. when a java library called lucene was introduced into the apache ecosystem, and then solr was built on top of that, open source developers began to wield some serious power when it came to customizing search features. in this article you'll be introduced to apache solr and a wealth of applications that have been built with it. the content is divided as follows: introduction setup solr applications summary 1. introduction apache solr is an open source search server. it is based on the full text search engine called apache lucene . so basically solr is an http wrapper around an inverted index provided by lucene. an inverted index could be seen as a list of words where each word-entry links to the documents it is contained in. that way getting all documents for the search query "dzone" is a simple 'get' operation. one advantage of solr in enterprise projects is that you don't need any java code, although java itself has to be installed. if you are unsure when to use solr and when lucene, these answers could help. if you need to build your solr index from websites, you should take a look into the open source crawler called apache nutch before creating your own solution. to be convinced that solr is actually used in a lot of enterprise projects, take a look at this amazing list of public projects powered by solr . if you encounter problems then the mailing list or stackoverflow will help you. to make the introduction complete i would like to mention my personal link list and the resources page which lists books, articles and more interesting material. 2. setup solr 2.1. installation as the very first step, you should follow the official tutorial which covers the basic aspects of any search use case: indexing - get the data of any form into solr. examples: json, xml, csv and sql-database. this step creates the inverted index - i.e. it links every term to its documents. querying - ask solr to return the most relevant documents for the users' query to follow the official tutorial you'll have to download java and the latest version of solr here . more information about installation is available at the official description . next you'll have to decide which web server you choose for solr. in the official tutorial, jetty is used, but you can also use tomcat. when you choose tomcat be sure you are setting the utf-8 encoding in the server.xml . i would also research the different versions of solr, which can be quite confusing for beginners: the current stable version is 1.4.1. use this if you need a stable search and don't need one of the latest features. the next stable version of solr will be 3.x the versions 1.5 and 2.x will be skipped in order to reach the same versioning as lucene. version 4.x is the latest development branch. solr 4.x handles advanced features like language detection via tika, spatial search , results grouping (group by field / collapsing), a new "user-facing" query parser ( edismax handler ), near real time indexing, huge fuzzy search performance improvements, sql join-a like feature and more. 2.2. indexing if you've followed the official tutorial you have pushed some xml files into the solr index. this process is called indexing or feeding. there are a lot more possibilities to get data into solr: using the data import handler (dih) is a really powerful language neutral option. it allows you to read from a sql database, from csv, xml files, rss feeds, emails, etc. without any java knowledge. dih handles full-imports and delta-imports. this is necessary when only a small amount of documents were added, updated or deleted. the http interface is used from the post tool, which you have already used in the official tutorial to index xml files. client libraries in different languages also exist. (e.g. for java (solrj) or python ). before indexing you'll have to decide which data fields should be searchable and how the fields should get indexed. for example, when you have a field with html in it, then you can strip irrelevant characters , tokenize the text into 'searchable terms', lower case the terms and finally stem the terms . in contrast, if you would have a field with text in it that should not be interpreted (e.g. urls) you shouldn't tokenize it and use the default field type string. please refer to the official documentation about field and field type definitions in the schema.xml file. when designing an index keep in mind the advice from mauricio : "the document is what you will search for. " for example, if you have tweets and you want to search for similar users, you'll need to setup a user index - created from the tweets. then every document is a user. if you want to search for tweets, then setup a tweet index; then every document is a tweet. of course, you can setup both indices with the multi index options of solr. please also note that there is a project called solr cell which lets you extract the relevant information out of several different document types with the help of tika. 2.3. querying for debugging it is very convenient to use the http interface with a browser to query solr and get back xml. use firefox and the xml will be displayed nicely: you can also use the velocity contribution , a cross-browser tool, which will be covered in more detail in the section about 'search application prototyping' . to query the index you can use the dismax handler or standard query handler . you can filter and sort the results: q=superman&fq=type:book&sort=price asc you can also do a lot more ; one other concept is boosting. in solr you can boost while indexing and while querying. to prefer the terms in the title write: q=title:superman^2 subject:superman when using the dismax request handler write: q=superman&qf=title^2 subject check out all the various query options like fuzzy search , spellcheck query input , facets , collapsing and suffix query support . 3. applications now i will list some interesting use cases for solr - in no particular order. to see how powerful and flexible this open source search server is. 3.1. drupal integration the drupal integration can be seen as generic use case to integrate solr into php projects. for the php integration you have the choice to either use the http interface for querying and retrieving xml or json. or to use the php solr client library . here is a screenshot of a typical faceted search in drupal : for more information about faceted search look into the wiki of solr . more php projects which integrates solr: open source typo3- solr module magento enterprise - solr module . the open source integration is out dated. oxid - solr module . no open source integration available. 3.2. hathi trust the hathi trust project is a nice example that proves solr's ability to search big digital libraries. to quote directly from the article : "... the index for our one million book index is over 200 gigabytes ... so we expect to end up with a two terabyte index for 10 million books" other examples for libraries: vufind - aims to replace opac internet archive national library of australia 3.3. auto suggestions mainly, there are two approaches to implement auto-suggestions (also called auto-completion) with solr: via facets or via ngramfilterfactory . to push it to the extreme you can use a lucene index entirely in ram. this approach is used in a large music shop in germany. live examples for auto suggestions: kaufda.de 3.4. spatial search applications when mentioning spatial search, people have geographical based applications in mind. with solr, this ordinary use case is attainable . some examples for this are : city search - city guides yellow pages kaufda.de spatial search can be useful in many different ways : for bioinformatics, fingerprints search, facial search, etc. (getting the fingerprint of a document is important for duplicate detection). the simplest approach is implemented in jetwick to reduce duplicate tweets, but this yields a performance of o(n) where n is the number of queried terms. this is okay for 10 or less terms, but it can get even better at o(1)! the idea is to use a special hash set to get all similar documents. this technique is called local sensitive hashing . read this nice paper about 'near similarity search and plagiarism analysis' for more information. 3.5. duckduckgo duckduckgo is made with open source and its "zero click" information is done with the help of solr using the dismax query handler: the index for that feature contains 18m documents and has a size of ~12gb. for this case had to tune solr: " i have two requirements that differ a bit from most sites with respect to solr: i generally only show one result, with sometimes a couple below if you click on them. therefore, it was really important that the first result is what people expected. false positives are really bad in 0-click, so i needed a way to not show anything if a match wasn't too relevant. i got around these by a) tweaking dismax and schema and b) adding my own relevancy filter on top that would re-order and not show anything in various situations. " all the rest is done with tuned open source products. to quote gabriel again: "the main results are a hybrid of a lot of things, including external apis, e.g. bing, wolframalpha, yahoo, my own indexes and negative indexes (spam removal), etc. there are a bunch of different types of data i'm working with. " check out the other cool features such as privacy or bang searches . 3.6. clustering support with carrot2 carrot2 is one of the "contributed plugins" of solr. with carrot2 you can support clustering : " clustering is the assignment of a set of observations into subsets (called clusters) so that observations in the same cluster are similar in some sense. " see some research papers regarding clustering here . here is one visual example when applying clustering on the search "pannous" - our company : 3.7. near real time search solr isn't real time yet, but you can tune solr to the point where it becomes near real time, which means that the time ('real time latency') that a document takes to be searchable after it gets indexed is less than 60 seconds even if you need to update frequently. to make this work, you can setup two indices. one write-only index "w" for the indexer and one read-only index "r" for your application. index r refers to the same data directory of w, which has to be defined in the solrconfig.xml of r via: /pathto/indexw/data/ to make sure your users and the r index see the indexed documents of w, you have to trigger an empty commit every 60 seconds: wget -q http://localhost:port/solr/update?stream.body=%3ccommit/%3e -o /dev/null everytime such a commit is triggered a new searcher without any cache entries is created. this can harm performance for visitors hitting the empty cache directly after this commit, but you can fill the cache with static searches with the help of the newsearcher entry in your solrconfig.xml. additionally, the autowarmcount property needs to be tuned, which fills the cache with a newsearcher from old entries. also, take a look at the article 'scaling lucene and solr' , where experts explain in detail what to do with large indices (=> 'sharding') and what to do for high query volume (=> 'replicating'). 3.8. loggly = full text search in logs feeding log files into solr and searching them at near real-time shows that solr can handle massive amounts of data and queries the data quickly. i've setup a simple project where i'm doing similar things , but loggly has done a lot more to make the same task real-time and distributed. you'll need to keep the write index as small as possible otherwise commit time will increase too great. loggly creates a new solr index every 5 minutes and includes this when searching using the distributed capabilities of solr ! they are merging the cores to keep the number of indices small, but this is not as simple as it sounds. watch this video to get some details about their work. 3.9. solandra = solr + cassandra solandra combines solr and the distributed database cassandra , which was created by facebook for its inbox search and then open sourced. at the moment solandra is not intended for production use. there are still some bugs and the distributed limitations of solr apply to solandra too. tthe developers are working very hard to make solandra better. jetwick can now run via solandra just by changing the solrconfig.xml. solandra also has the advantages of being real-time (no optimize, no commit!) and distributed without any major setup involved. the same is true for solr cloud. 3.10. category browsing via facets solr provides facets , which make it easy to show the user some useful filter options like those shown in the "drupal integration" example. like i described earlier , it is even possible to browse through a deep category tree. the main advantage here is that the categories depend on the query. this way the user can further filter the search results with this category tree provided by you. here is an example where this feature is implemented for one of the biggest second hand stores in germany. a click on 'schauspieler' shows its sub-items: other shops: game-change 3.11. jetwick - open twitter search you may have noticed that twitter is using lucene under the hood . twitter has a very extreme use case: over 1,000 tweets per second, over 12,000 queries per second, but the real-time latency is under 10 seconds! however, the relevancy at that volume is often not that good in my opinion. twitter search often contains a lot of duplicates and noise. reducing this was one reason i created jetwick in my spare time. i'm mentioning jetwick here because it makes extreme use of facets which provides all the filters to the user. facets are used for the rss-alike feature (saved searches), the various filters like language and retweet-count on the left, and to get trending terms and links on the right: to make jetwick more scalable i'll need to decide which of the following distribution options to choose: use solr cloud with zookeeper use solandra move from solr to elasticsearch which is also based on apache lucene other examples with a lot of facets: cnet reviews - product reviews. electronics reviews, computer reviews & more. shopper.com - compare prices and shop for computers, cell phones, digital cameras & more. zappos - shoes and clothing. manta.com - find companies. connect with customers. 3.12. plaxo - online address management plaxo.com , which is now owned by comcast, hosts web addresses for more than 40 million people and offers smart search through the addresses - with the help of solr. plaxo is trying to get the latest 'social' information of your contacts through blog posts, tweets, etc. plaxo also tries to reduce duplicates . 3.13. replace fast or google search several users report that they have migrated from a commercial search solution like fast or google search appliance (gsa) to solr (or lucene). the reasons for that migration are different: fast drops linux support and google can make integration problems. the main reason for me is that solr isn't a black box —you can tweak the source code, maintain old versions and fix your bugs more quickly! 3.14. search application prototyping with the help of the already integrated velocity plugin and the data import handler it is possible to create an application prototype for your search within a few hours. the next version of solr makes the use of velocity easier. the gui is available via http://localhost:port/solr/browse if you are a ruby on rails user, you can take a look into flare. to learn more about search application prototyping, check out this video introduction and take a look at these slides. 3.15. solr as a whitelist imagine you are the new google and you have a lot of different types of data to display e.g. 'news', 'video', 'music', 'maps', 'shopping' and much more. some of those types can only be retrieved from some legacy systems and you only want to show the most appropriated types based on your business logic . e.g. a query which contains 'new york' should result in the selection of results from 'maps', but 'new yorker' should prefer results from the 'shopping' type. with solr you can set up such a whitelist-index that will help to decide which type is more important for the search query. for example if you get more or more relevant results for the 'shopping' type then you should prefer results from this type. without the whitelist-index - i.e. having all data in separate indices or systems, would make it nearly impossible to compare the relevancy. the whitelist-index can be used as illustrated in the next steps. 1. query the whitelist-index, 2. decide which data types to display, 3. query the sub-systems and 4. display results from the selected types only. 3.16. future solr is also useful for scientific applications, such as a dna search systems. i believe solr can also be used for completely different alphabets so that you can query nucleotide sequences - instead of words - to get the matching genes and determine which organism the sequence occurs in, something similar to blast . another idea you could harness would be to build a very personalized search. every user can drag and drop their websites of choice and query them afterwards. for example, often i only need stackoverflow, some wikis and some mailing lists with the expected results, but normal web search engines (google, bing, etc.) give me results that are too cluttered. my final idea for a future solr-based app could be a lucene/solr implementation of desktop search. solr's facets would be especially handy to quickly filter different sources (files, folders, bookmarks, man pages, ...). it would be a great way to wade through those extra messy desktops. 4. summary the next time you think about a problem, think about solr! even if you don't know java and even if you know nothing about search: solr should be in your toolbox. solr doesn't only offer professional full text search, it could also add valuable features to your application. some of them i covered in this article, but i'm sure there are still some exciting possibilities waiting for you!
January 25, 2011
by Peter Karussell
· 147,268 Views
article thumbnail
Migrating from JBoss 4 to JBoss 5
I have been using JBoss 4.2.3 for over a year right now and really like it (although the clustering / JMS stuff seems WAY overcomplicated - and needing 80 config files - not fun!). But anyway it is time to upgrade to JBoss 5 and the path has been marred by many stops and starts and has been surprisingly difficult. In any event I found the following links to be a life saver and thought I would share them http://community.jboss.org/wiki/MigrationfromJBoss4.pdf http://venugopaal.wordpress.com/2009/02/02/jboss405-to-jboss-5ga/ http://www.tikalk.com/java/migrating-your-application-jboss-4x-jboss-5x The real kickers have to be 1) Increased adherence to the Java spec that cause WARs / JARs to no longer deploy 2) Changed location of XML files 3) Changed XML filenames 4) Changed XML file contents Man they don't make it easy do they? From http://softarc.blogspot.com/2011/01/migrating-from-jboss-4-to-jboss-5.html
January 14, 2011
by Frank Kelly
· 20,426 Views
article thumbnail
Mockito - Pros, Cons, and Best Practices
It's been almost 4 years since I wrote a blog post called "EasyMock - Pros, Cons, and Best Practices, and a lot has happened since. You don't hear about EasyMock much any more, and Mockito seems to have replaced it in mindshare. And for good reason: it is better. A Good Humane Interface for Stubbing Just like EasyMock, Mockito allows you to chain method calls together to produce less imperative looking code. Here's how you can make a Stub for the canonical Warehouse object: Warehouse mock = Mockito.mock(Warehouse.class); Mockito.when(mock.hasInventory(TALISKER, 50)). thenReturn(true); I know, I like a crazy formatting. Regardless, giving your System Under Test (SUT) indirect input couldn't be easier. There is no big advantage over EasyMock for stubbing behavior and passing a stub off to the SUT. Giving indirect input with mocks and then using standard JUnit asserts afterwards is simple with both tools, and both support the standard Hamcrest matchers. Class (not just Interface) Mocks Mockito allows you to mock out classes as well as interfaces. I know the EasyMock ClassExtensions allowed you to do this as well, but it is a little nicer to have it all in one package with Mockito. Supports Test Spies, not just Mocks There is a difference between spies and mocks. Stubs allow you to give indirect input to a test (the values are read but never written), Spies allow you to gather indirect output from a test (the mock is written to and verified, but does not give the test input), and Mocks are both (your object gives indirect input to your test through Stubbing and gathers indirect output through spying). The difference is illustrated between two code examples. In EasyMock, you only have mocks. You must set all input and output expectations before running the test, then verify afterwards. // arrange Warehouse mock = EasyMock.createMock(Warehouse.class); EasyMock.expect( mock.hasInventory(TALISKER, 50)). andReturn(true).once(); EasyMock.expect( mock.remove(TALISKER, 50)). andReturn(true).once(); EasyMock.replay(mock); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert EasyMock.verify(mock); That's a lot of code, and not all of it is needed. The arrange section is setting up a stub (the warehouse has inventory) and setting up a mock expectation (the remove method will be called later). The assertion in all this is actually the little verify() method at the end. The main point of this test is that remove() was called, but that information is buried in a nest of expectations. Mockito improves on this by throwing out both the record/playback mode and a generic verify() method. It is shorter and clearer this way: // arrange Warehouse mock = Mockito.mock(Warehouse.class); Mockito.when(mock.hasInventory(TALISKER, 50)). thenReturn(true); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert Mockito.verify(warehouse).remove(TALISKER, 50); The verify step with Mockito is spying on the results of the test, not recording and verifying. Less code and a clearer picture of what really is expected. Update: There is a separate Spy API you can use in Mockito as well: http://mockito.googlecode.com/svn/branches/1.8.3/javadoc/org/mockito/Mockito.html#13 Better Void Method Handling Mockito handles void methods better than EasyMock. The fluent API works fine with a void method, but in EasyMock there were some special methods you had to write. First, the Mockito code is fairly simple to read: // arrange Warehouse mock = Mockito.mock(Warehouse.class); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert Mockito.verify(warehouse).remove(TALISKER, 50); Here is the same in EasyMock. Not as good: // arrange Warehouse mock = EasyMock.createMock(Warehouse.class); mock.remove(TALISKER, 50); EasyMock.expectLastMethodCall().once(); EasyMock.replay(mock); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert EasyMock.verify(mock); Mock Object Organization Patterns Both Mockito and EasyMock suffer from difficult maintenance. What I said in my original EasyMock post holds true for Mockito: The method chaining style interface is easy to write, but I find it difficult to read. When a test other than the one I'm working on fails, it's often very difficult to determine what exactly is going on. I end up having to examine the production code and the test expectation code to diagnose the issue. Hand-rolled mock objects are much easier to diagnose when something breaks... This problem is especially nasty after refactoring expectation code to reduce duplication. For the life of me, I cannot follow expectation code that has been refactored into shared methods. Now, four years later, I have a solution that works well for me. With a little care you can make your mocks reusable, maintainable, and readable. This approach was battle tested over many months in an Enterprise Environment(tm). Create a private static method the first time you need a mock. Any important data needs to be passed in as a parameter. Using constants or "magic" fields hides important information and obfuscates tests. For example: User user = createMockUser("userID", "name"); ... assertEquals("userID", result.id()); assertEquals("name", result.name(); Everything important is visible and in the test, nothing important is hidden. You need to completely hide the replay state behind this factory method if you're still on EasyMock. The Mock framework in use is an implementation detail and try not to let it leak. Next, as your dependencies grow, be sure to always pass them in as factory method parameters. If you need a User and a Role object, then don't create one method that creates both mocks. One method instantiates one object, otherwise it is a parameter and compose your mock objects in the test method: User user = createMockUser( "userID", "name", createMockRole("role1"), createMockRole("role2") ); When each object type has a factory method, then it makes it much easier to compose the different types of objects together. Reuse. But you can only reuse the methods when they are simple and with few dependencies, otherwise they become too specific and difficult to understand. The first time you need to reuse one of these methods, then move the method to a utility class called "*Mocker", like UserMocker or RoleMocker. Follow a naming convention so that they are always easy to find. If you remembered to make the private factory methods static then moving them should be very simple. Your client code ends up looking like this, but you can use static imports to fix that: User user = UserMocker.createMockUser( "userID", "name", RoleMocker.createMockRole("role1"), RoleMocker.createMockRole("role2") ); User overloaded methods liberally. Don't create one giant method with every possible parameter in the parameter list. There are good reasons to avoid overloading in production, but this is test. Use overloading so that the test methods only display data relevant to that test and nothing more. Using Varargs can also help keep a clean test. Lastly, don't use constants. Constants hide the important information out of sight, at the top of the file where you can't see it or in a Mocker class. It's OK to use constants within the test case, but don't define constants in the Mockers, it just hides relevant information and makes the test harder to read later. Avoid Abstract Test Cases Managing mock objects within abstract test cases has been very difficult for me, especially when managing replay and record states. I've given up mixing mock objects and abstract TestCase objects. When something breaks it simply takes too long to diagnose. An alternative is to create custom assertion methods that can be reused. Beyond that, I've given up on Abstract TestCase objects anyway, on the grounds of preferring composition of inheritance. Don't Replace Asserts with Verify My original comments about EasyMock are still relevant for Mockito: The easiest methods to understand and test are methods that perform some sort of work. You run the method and then use asserts to make sure everything worked. In contrast, mock objects make it easy to test delegation, which is when some object other than the SUT is doing work. Delegation means the method's purpose is to produce a side-effect, not actually perform work. Side-effect code is sometimes needed, but often more difficult to understand and debug. In fact, some languages don't even allow it! If you're test code contains assert methods then you have a good test. If you're code doesn't contain asserts, and instead contains a long list of verify() calls, then you're relying on side effects. This is a unit-test bad smell, especially if there are several objects than need to be verified. Verifying several objects at the end of a unit test is like saying, "My test method needs to do several things: x, y, and z." The charter and responsibility of the method is no longer clear. This is a candidate for refactoring. No More All or Nothing Testing Mockito's verify() methods are much more flexible than EasyMock's. You can verify that only one or two methods on the mock were called, while EasyMock had just one coarse verify() method. With EasyMock I ended up littering the code with meaningless expectations, but not so in Mockito. This alone is reason enough to switch. Failure: Expected X received X For the most part, Mockito error messages are better than EasyMock's. However, you still sometimes see a failure that reads "Failure. Got X Expected X." Basically, this means that your toString() methods produce the same results but equals() does not. Every user who starts out gets confused by this message at some point. Be Warned. Don't Stop Handrolling Mocks Don't throw out hand-rolled mock objects. They have their place. Subclass and Override is a very useful technique for creating a testing seam, use it. Learn to Write an ArgumentMatcher Learn to write an ArgumentMatcher. There is a learning curve but it's over quickly. This post is long enough, so I won't give an example. That's it. See you again in 4 years when the next framework comes out! From http://hamletdarcy.blogspot.com/2010/09/mockito-pros-cons-and-best-practices.html
October 14, 2010
by Hamlet D'Arcy
· 57,065 Views
article thumbnail
JMS Clustering by Example
It's amazing how the JBoss Team put together an easy way to do JMS Clustering, out of the box!!. I'll start with an easy example, creating a Queue named "MyClusteredQueue". In this example I'm using JBoss AS 5.1. and two computers connected on the same network, with these IP's: - Computer A: 192.168.0.143 - Computer B: 192.168.0.210 So, here are the steps: 1) Install the JBoss on both computers. We are going to use the "all" configuration for both computers. 2) We create our Queue on both servers. Go to $JBOSS_HOME/server/all/deploy/messaging/ and edit the destinations-service.xml file. Add the MyClusteredQueue before the last server tag. It looks like this: jboss.messaging:service=ServerPeer jboss.messaging:service=PostOffice true This is how you add a Queue to the JBoss, and the people how are familiar with this, the only new thing is to add the attribute "Clustered". This step must be set on both computers. At the end of the article you can find the files. 3) Write the MDB to consume the messages, and deploy it on the two computers. (I'm using an EJB 3 - MDB style). import java.net.InetAddress; import javax.ejb.ActivationConfigProperty; import javax.ejb.MessageDriven; import javax.jms.Message; import javax.jms.MessageListener; import javax.jms.ObjectMessage; import org.apache.log4j.Logger; /** * @author felipeg * */ @MessageDriven(activationConfig = { @ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue"), @ActivationConfigProperty(propertyName="destination", propertyValue="queue/MyClusteredQueue") }) public class JMSClusterClientHandler implements MessageListener { Logger log = Logger.getLogger(JMSClusterClientHandler.class); @Override public void onMessage(Message message) { try{ if (message instanceof ObjectMessage) { InetAddress addr = InetAddress.getLocalHost(); log.info("########## Processing Host: " + addr.getHostName() + " ##########" ); ObjectMessage objMessage = (ObjectMessage) message; Object obj = objMessage.getObject(); log.info("Object received:" + obj.toString()); } } catch (Exception e) { e.printStackTrace(); } } } 4) Start the jboss with the following options: Computer A: $ cd $JBOSS_HOME/bin $ ./run.sh -c all -b 192.168.0.143 -Djboss.messaging.ServerPeerID=1 Computer B: $ cd $JBOSS_HOME/bin $ ./run.sh -c all -b 192.168.0.210 -Djboss.messaging.ServerPeerID=2 It is necesary to give an ID to each server and this is accomplished with this directive: -Djboss.messaging.ServerPeerID When you start the jboss on computer A, you should see the logs (server.log) telling you that there is one node ready and listening, and once you start the jboss on computer B, on the log will appear the two nodes, the two IP's ready to consume messages. 5) Now it's time to send a Message to the Queue. To accomplish this it's necessary to change the connection factory to "ClusteredConnectionFactory" (JMSDispatcher.java - See the code below). Also on the jndi.properties (if you are using the default InitialContext) file it's necessary to add the two computers ip's separated by comma to the java.naming.provider.url property. (In my case a create a Properties variable and I set all the necessary properties, JMSDispatcher.java - see the code below). java.naming.provider.url=192.168.0.143:1099,192.168.0.210:1099 The client that I wrote is a web application, that consist in one index.jsp page, which contains a form that prompts you for the name of the queue, the type of messaging (Queue or Topic), the server ip and port, how many times it will send the message and the actual message to be sent; also the web application has a Servlet (JMSClusteredClient.java - see code below) that receives the postback and helper class (JMSDispatcher.java - see code below) that sends the message to the jboss servers. You can to deploy it in any computer. In my case I deployed it on the Computer A. And you can access it through this URL: http://192.168.0.143:8080/JMSWeb/ (just modify the IP where the client war was deployed). If you notice (on the index.jsp - code below) I've already put some default values that reflects the name of the Queue, and the IP's of my two computers. Now, If you increment the number of times that the message will be sent (maybe a 10) and fill out the message box, and click "Send" you should see on the two servers some of the messages being consumed by the MDB. Here are the Files to create the client: index.jsp JMS Clustered - Test Client Server: QueueTopic Times:Message: Servlet: JMSClusteredClient.java public class JMSClusteredClient extends HttpServlet { private static final long serialVersionUID = 1L; /** * @see HttpServlet#service(HttpServletRequest request, HttpServletResponse response) */ protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { PrintWriter out = response.getWriter(); String topicqueue = request.getParameter("topicqueue"); String message = request.getParameter("message"); String server = request.getParameter("server"); String messageType = request.getParameter("messageType"); String times = request.getParameter("times"); int intTimes = Integer.parseInt(times); JMSDispatcher dispatcher = new JMSDispatcher(); dispatcher.setTopicQueueName(topicqueue); dispatcher.setServer(server); dispatcher.setMessageType(messageType); try { for(int count =1; count <= intTimes;count++){ dispatcher.sendMessage( count + " of " + times + " " + message); } out.println("Message [" + message + "] sent successfully to [" + topic + "] to the [" + server + "] server " + times + " times."); } catch (JMSException e) { e.printStackTrace(); out.println("Error:" + e.getMessage()); } catch (NamingException e) { out.println("Error:" + e.getMessage()); e.printStackTrace(); } finally{ out.close(); } } } A utility to send the messages: JMSDispatcher.java public class JMSDispatcher { /** * */ private static final long serialVersionUID = 7105145023422143880L; private static Logger log = Logger.getLogger(JMSDispatcher.class); private final String CONNECTION_FACTORY_CLUSTERED = "ClusteredConnectionFactory"; private final String CONNECTION_FACTORY = "ConnectionFactory"; private final String TOPIC = "TOPIC"; private final String QUEUE = "QUEUE"; private String topicQueueName; private String server; private String messageType; public void setTopicQueueName(String value){ this.topicQueueName = value; } public void setServer(String value){ this.server = value; } public void setMessageType(String value){ this.messageType = value; } public void sendMessage(Object objectMessage) throws JMSException, NamingException{ log.debug("##### Setting up a Queue/Topic Message: #####"); if (TOPIC.equals(messageType)){ sendTopicMessage(objectMessage); } else if (QUEUE.equals(messageType)){ sendQueueMessage(objectMessage); } log.debug("##### Publishing Message: Done #####"); } private void sendQueueMessage(Object objectMessage) throws JMSException, NamingException{ try{ InitialContext initialContext = getInitialContext(); QueueConnectionFactory qcf = (QueueConnectionFactory) initialContext.lookup(CONNECTION_FACTORY_CLUSTERED); QueueConnection queueConn = qcf.createQueueConnection(); Queue queue = (Queue) initialContext.lookup(topicQueueName); QueueSession queueSession = queueConn.createQueueSession(false, Session.AUTO_ACKNOWLEDGE); queueConn.start(); QueueSender send = queueSession.createSender(queue); ObjectMessage om = queueSession.createObjectMessage((Serializable)objectMessage); setMessageProperties(om); log.debug("##### Publishing Message to a Queue: " + queueName + "#####"); send.send(om); send.close(); queueConn.stop(); queueSession.close(); queueConn.close(); }catch(MessageFormatException ex){ log.error("##### The MESSAGE is not Serializable ####"); throw ex; }catch(MessageNotWriteableException ex){ log.error("##### The MESSAGE is not Readable ####"); throw ex; }catch(JMSException ex){ log.error("##### JMS provider fails to set the object due to some internal error. ####"); throw ex; } } private void sendTopicMessage(Object objectMessage) throws JMSException, NamingException{ try{ InitialContext initialContext = getInitialContext(); TopicConnectionFactory tcf = (TopicConnectionFactory)initialContext.lookup(CONNECTION_FACTORY_CLUSTERED); TopicConnection topicConn = tcf.createTopicConnection(); Topic topic = (Topic) initialContext.lookup(topicQueueName); TopicSession topicSession = topicConn.createTopicSession(false,TopicSession.AUTO_ACKNOWLEDGE); topicConn.start(); TopicPublisher send = topicSession.createPublisher(topic); ObjectMessage om = topicSession.createObjectMessage(); om.setObject((Serializable)objectMessage); setMessageProperties(om); log.debug("##### Publishing Message to a Topic: " + topicName + "#####"); send.publish(om); send.close(); topicConn.stop(); topicSession.close(); topicConn.close(); }catch(MessageFormatException ex){ log.error("##### The MESSAGE is not Serializable ####"); throw ex; }catch(MessageNotWriteableException ex){ log.error("##### The MESSAGE is not Readable ####"); throw ex; }catch(JMSException ex){ log.error("##### JMS provider fails to set the object due to some internal error. ####"); throw ex; } } private InitialContext getInitialContext() throws NamingException{ Properties jboss = new Properties(); jboss.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory"); jboss.put("java.naming.factory.url.pkgs", "org.jboss.naming:org.jnp.interfaces"); jboss.put("java.naming.provider.url", server); return new InitialContext(jboss); } } And the web.xml JMSWeb index.jsp JMSClusteredClient JMSClusteredClient com.blogspot.felipeg48.jms.web.JMSClusteredClient JMSClusteredClient /JMSClusteredClient Happy Clustering!!
May 26, 2010
by Felipe Gutierrez
· 16,718 Views
article thumbnail
A Look Inside JBoss Microcontainer - The Scanning Library
Today's JEE doesn't require configuration files any more. Most of the configuration, if not all, is done over properly annotated classes. As such, it's the responsibility of the underlying containers to find these annotations and act accordingly. At a first glance it appears that there is no other way for a container to implement that than to fully scan a given deployment. And we all know this can be very time consuming, especially if there are multiple container components that require this information, and have no integration hooks available to get to a container's already gathered information. From requirements collected here, I've introduced a new MC Scanning sub-project. The main goal or idea behind this lib is very simple: unify all of JBossAS component scanning into a single-pass scan. Instead of doing the resource scanning for every component, we just do it once, properly delegating the work to various container components. Another goal was to also enable usage of pre-indexed information, so that there would actually be no need for the scanning itself - e.g. one could pre-index all of jar's annotations during the build. Read the other parts in DZone's exclusive JBoss Microcontainer Series: Part 1 -- Component Models Part 2 –- Advanced Dependency Injection and IoC Part 3 -- the Virtual File System Project structure scanning-spi - Contains a simple scanner, metadata SPI, and initial helpers to help you extend / use a simple version of this lib. scanning-impl - Provides component agnostic scanning API. It also includes generic metadata implementation and its usage. plugins - This module holds custom component-scanning implementations. Current implementations are: Annotations Hibernate Hierarchy JSF Web Weld deployers - Integration with VDF; new custom deployers. indexer - This module contains utils for creating pre-indexed handles, and merging them into existing jars. It includes Ant task and Maven plugin. testsuite - Tests for all other modules. Basic building blocks The org.jboss.scanning.spi.Scanner class is the most abstract - most basic interface to interact with any scanner implementation. It only has scan() method. For any really useful operation one will have to use some concrete implementation's constructors, properties ... and then use scan() to trigger the scan operation. The main interface of interest for us is org.jboss.scanning.spi.ScanningPlugin: package org.jboss.scanning.spi; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import org.jboss.classloading.spi.visitor.ResourceFilter; import org.jboss.classloading.spi.visitor.ResourceVisitor; /** * Scanning plugin. * Defines what to do with a resource. * * @param exact handle type * @param exact handle interface * @author Ales Justin */ public interface ScanningPlugin extends ResourceFilter, ResourceVisitor { /** * Create plugins handle/utility. * e.g. AnnotationRepository for annotations scanning * * @return new handle instance */ T createHandle(); /** * Read serialized handle. * * @param is the serialized handle's input stream. * @return de-serialized handle * @throws Exception for any error */ ScanningHandle readHandle(InputStream is) throws Exception; /** * Write / serialize handle. * * @param os the output stream to serialize handle. * @param handle the handle * @throws IOException for any IO error */ void writeHandle(OutputStream os, T handle) throws IOException; /** * Cleanup handle. * * @param handle the handle to cleanup */ void cleanupHandle(U handle); /** * Get handle interface. * * @return the handle interface */ Class getHandleInterface(); /** * Get handle's key. * Used to attach handle to map/attachments. * * @return the handle's key */ String getAttachmentKey(); /** * Get handle's file name. * Used to attach handle to jar and/or get pre-indexed. * * @return the handle's file name */ String getFileName(); /*** * Get recurse filter. * * @return the recurse filter */ ResourceFilter getRecurseFilter(); } Most of the functionality is already implemented in its abstract form (AbstractScanningPlugin) so you only need to provide the custom logic. As you can already see from the plugin's signature, the plugin introduces a handle. A handle is what will hold the scanning information for particular component; e.g. an annotation repository. We can see handle's implementation defined as parameter T, where handle's interface is parameter U. /** * Scanning handle. * * Represents a simple interface resource scanning results must implement * in order to be able to merge pre-existing results. * * @param exact handle type * @author Ales Justin */ public interface ScanningHandle { /** * Merge existing handle with sub-handle / pre-existing handle. * * @param subHandle the sub handle */ void merge(T subHandle); } The main purpose of handle introduction is to allow for pre-existing handle's merging in type safe manner. How to make usage of plugins as easy as possible in MC? As we can see Scanner (or its actual implementations) takes a set of plugins to handle artifacts. But since plugins are mostly mutable, we need some sort of factory to help use create these plugins. For VDF based usage this is how our factory looks like: import org.jboss.deployers.structure.spi.DeploymentUnit; import org.jboss.scanning.spi.ScanningHandle; import org.jboss.scanning.spi.ScanningPlugin; /** * Deployment based scanning plugin factory. * Used for incallback automatching. * * @param exact handle type * @author Ales Justin */ public interface DeploymentScanningPluginFactory { /** * Is this plugin relevant to unit. * * @param unit the unit to check against * @return true if it's relevant, false otherwise */ boolean isRelevant(DeploymentUnit unit); /** * Create scanning plugin from deployment unit. * * @param unit the deployment unit * @return new scanning plugin */ ScanningPlugin create(DeploymentUnit unit); } Also, as the javadoc says, this interface is nicely used for MC's incallback usage (incallback is a kind of dependency injection where one component can insert itself into another via a method call, as explained in one of the previous articles on JBoss Microcontainer). Usage example Let's see what we need to implement in order to get annotation scanning into the repository. public class AnnotationsScanningPluginFactory implements DeploymentScanningPluginFactory { public boolean isRelevant(DeploymentUnit unit) { // any better check? -- metadata complete is already done elsewhere // see JBossMetaDataDeploymentUnitFilter in JBossAS return true; } public ScanningPlugin create(DeploymentUnit unit) { ReflectProvider provider = DeploymentUtilsFactory.getProvider(unit); ResourceOwnerFinder finder = DeploymentUtilsFactory.getFinder(unit); return new AnnotationsScanningPlugin(provider, finder, unit.getClassLoader()); } } public class AnnotationsScanningPlugin extends AbstractClassLoadingScanningPlugin { /** The repository */ private final DefaultAnnotationRepository repository; /** The visitor */ private final ResourceVisitor visitor; public AnnotationsScanningPlugin(ClassLoader cl) { this(IntrospectionReflectProvider.INSTANCE, ClassResourceOwnerFinder.INSTANCE, cl); } public AnnotationsScanningPlugin(ReflectProvider provider, ResourceOwnerFinder finder, ClassLoader cl) { repository = new DefaultAnnotationRepository(cl); visitor = new GenericAnnotationVisitor(provider, finder, repository); } protected DefaultAnnotationRepository doCreateHandle() { return repository; } protected ClassLoader getClassLoader() { return repository.getClassLoader(); } @Override public void cleanupHandle(AnnotationIndex handle) { if (handle instanceof DefaultAnnotationRepository) DefaultAnnotationRepository.class.cast(handle).cleanup(); } public Class getHandleInterface() { return AnnotationIndex.class; } public ResourceFilter getFilter() { return visitor.getFilter(); } public void visit(ResourceContext resource) { visitor.visit(resource); } } public class GenericAnnotationVisitor extends ClassHierarchyResourceVisitor { /** The mutable repository */ private MutableAnnotationRepository repository; public GenericAnnotationVisitor(ReflectProvider provider, ResourceOwnerFinder finder, MutableAnnotationRepository repository) { super(provider, finder); if (repository == null) throw new IllegalArgumentException("Null repository"); this.repository = repository; } protected boolean isRelevant(ClassInfo classInfo) { return repository.isAlreadyChecked(classInfo.getName()) == false; } public ResourceFilter getFilter() { return ClassFilter.INSTANCE; } @Override protected void handleAnnotations(ElementType type, Signature signature, Annotation[] annotations, String className, URL ownerURL) { if (annotations != null && annotations.length > 0) { for (Annotation annotation : annotations) { repository.putAnnotation(annotation, type, className, signature, ownerURL); } } } } While the repository is just a-bit-smarter Map. Integration with VDF Using the Indexer public class Main { private static final Logger log = Logger.getLogger(Main.class.getName()); /** * Usage */ private static void usage() { System.out.println("Usage: Indexer "); } /** * Main. * The output is file named .jar.mcs. * * @param args the program arguments */ public static void main(String[] args) { try { int offset = 2; if (args.length < offset) { File input = new File(args[0]); String[] providers = args[1].split(","); URL[] urls = new URL[args.length - offset]; // add the rest of classpath for (int i = 0; i < urls.length; i++) urls[i] = new File(args[i + offset]).toURI().toURL(); ScanUtils.scan(input, Constants.applyAliases(providers), urls); } else { usage(); } } catch (Throwable t) { log.log(Level.SEVERE, t.getMessage(), t); } } } Pre-existing or pre-indexed information For each scanning plugin we look for artifact's META-INF/ entry. String fileName = plugin.getFileName(); for (URL root : roots) { InputStream is = getInputStream(root, Scanner.META_INF + fileName); if (is != null) { ScanningHandle pre = plugin.readHandle(is); handle.merge(pre); It's plugin's responsibility to know how to read pre-existing handle. By default we use plain Java serialization together with gzip. public ScanningHandle readHandle(InputStream is) throws Exception { try { GZIPInputStream gis = new GZIPInputStream(is); ObjectInputStream ois = createObjectInputStream(gis); return (ScanningHandle) ois.readObject(); } finally { is.close(); } } public void writeHandle(OutputStream os, T handle) throws IOException { GZIPOutputStream gos = new GZIPOutputStream(os); ObjectOutputStream oos = new ObjectOutputStream(gos); try { oos.writeObject(handle); oos.flush(); } finally { oos.close(); } } How to limit scanning? There already was a jboss-scanning.xml, I've just enhanced it a bit. The recurse filter is now a bit smarter, and consequently faster, than it used to be in previous version. package org.jboss.scanning.plugins.filter; import java.net.URL; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.Set; import org.jboss.classloading.spi.visitor.ResourceContext; import org.jboss.classloading.spi.visitor.ResourceFilter; import org.jboss.scanning.spi.metadata.PathEntryMetaData; import org.jboss.scanning.spi.metadata.PathMetaData; import org.jboss.scanning.spi.metadata.ScanningMetaData; import org.jboss.vfs.util.PathTokenizer; /** * Simple recurse filter. * * It searches for path substring in url string, * and tries to match the tree structure as far as it goes. */ public class ScanningMetaDataRecurseFilter implements ResourceFilter { /** Path tree roots */ private Map roots; public ScanningMetaDataRecurseFilter(ScanningMetaData smd) { if (smd == null) throw new IllegalArgumentException("Null metadata"); List paths = smd.getPaths(); if (paths != null && paths.isEmpty() == false) { roots = new HashMap(); for (PathMetaData pmd : paths) { RootNode pathNode = new RootNode(); roots.put(pmd.getPathName(), pathNode); Set includes = pmd.getIncludes(); if (includes != null && includes.isEmpty() == false) { pathNode.explicitInclude = true; for (PathEntryMetaData pemd : includes) { String name = pemd.getName(); String[] tokens = name.split("\\."); Node current = pathNode; for (String token : tokens) current = current.addChild(token); if (pemd.isRecurse()) current.recurse = true; // mark last one as recurse } } } } } public boolean accepts(ResourceContext resource) { if (roots == null) return false; URL url = resource.getUrl(); String urlString = url.toExternalForm(); for (Map.Entry root : roots.entrySet()) { if (urlString.contains(root.getKey())) { RootNode rootNode = root.getValue(); if (rootNode.explicitInclude) // we have explicit includes in path, try tree path { String resourceName = resource.getResourceName(); List tokens = PathTokenizer.getTokens(resourceName); Node current = rootNode; // let's try to walk some tree path for (String token : tokens) { // if we're here, the rest is recursively matched if (current.recurse) break; current = current.getChild(token); // no fwd path if (current == null) return false; } } return true; } } return false; } private static class Node { private Map children; private boolean recurse; public Node addChild(String value) { if (children == null) children = new HashMap(); Node child = children.get(value); if (child == null) { child = new Node(); children.put(value, child); } return child; } public Node getChild(String child) { return children != null ? children.get(child) : null; } } private static class RootNode extends Node { private boolean explicitInclude; } } Javassist based JBoss Reflect In order to avoid loading the actual resource's underlying class, we use Javassist under the hood - via JBoss Refect project. DeploymentUnit unit = assertDeploy(jar); try { TIFScanningPlugin plugin = unit.getAttachment(TIFScanningPlugin.class); assertNotNull(plugin); Kernel kernel = assertBean("Kernel", Kernel.class); KernelConfigurator configurator = kernel.getConfigurator(); ClassLoader cl = unit.getClassLoader(); String name = JarMarkOnClass.class.getName(); TypeInfo ti = configurator.getTypeInfo(name, cl); TypeInfo visited = plugin.getResources().get(name); assertSame(ti, visited); // let's check if the cache is working Method findLoadedClass = ClassLoader.class.getDeclaredMethod("findLoadedClass", String.class); findLoadedClass.setAccessible(true); Object clazz = findLoadedClass.invoke(cl, name); assertNull(clazz); // should not be loaded } finally { undeploy(unit); } But the overall usage of helper utils is pluggable: /** * Find the util for deployment. * Newly created utils are grouped per module. * * @author Ales Justin */ public class DeploymentUtilsFactory { /** The default impls */ private static Map, UtilFactory> defaults = new WeakHashMap, UtilFactory>(); static { addImplementation(ReflectProvider.class, new ReflectProviderUtilFactory()); addImplementation(ResourceOwnerFinder.class, new ResourceOwnerFinderUtilFactory()); } /** * Add the util impl. * * @param iface the interface * @param factory the util factory */ public static void addImplementation(Class iface, UtilFactory factory) { defaults.put(iface, factory); } /** * Remove the util impl. * * @param iface the interface */ public static void removeImplementation(Class iface) { defaults.remove(iface); } /** * Get util. * * @param unit the deployment unit * @param utilType the util type * @return util instance */ public static T getUtil(DeploymentUnit unit, Class utilType) { if (utilType == null) throw new IllegalArgumentException("Null util type"); DeploymentUnit moduleUnit = getModuleUnit(unit); T util = moduleUnit.getAttachment(utilType); if (util == null) { UtilFactory factory = defaults.get(utilType); if (factory == null) throw new IllegalArgumentException("No util factory defined for " + utilType); Object instance = factory.create(moduleUnit); util = utilType.cast(instance); moduleUnit.addAttachment(utilType, util); } return util; } /** * Get module unit. * * @param unit the current unit * @return unit containing Module, or exception if no such unit exists */ public static DeploymentUnit getModuleUnit(DeploymentUnit unit) { if (unit == null) throw new IllegalArgumentException("Null unit"); // group util per module DeploymentUnit moduleUnit = unit; while(moduleUnit != null && moduleUnit.isAttachmentPresent(Module.class) == false) moduleUnit = moduleUnit.getParent(); if (moduleUnit == null) throw new IllegalArgumentException("No module in unit: " + unit); return moduleUnit; } /** * Wrap util lookup in lazy lookup. * * @param unit the deployment unit * @param utilType the util type * @return lazy util proxy */ public static T getLazyUtilProxy(DeploymentUnit unit, Class utilType) { // null check is in handler LazyUtilsProxyHandler handler = new LazyUtilsProxyHandler(unit, utilType); Object proxy = Proxy.newProxyInstance(unit.getClassLoader(), new Class[]{utilType}, handler); return utilType.cast(proxy); } /** * Get reflect provider. * * @param unit the depoyment unit * @return the provider */ public static ReflectProvider getProvider(DeploymentUnit unit) { return getUtil(unit, ReflectProvider.class); } /** * Get finder. * * @param unit the depoyment unit * @return the finder */ public static ResourceOwnerFinder getFinder(DeploymentUnit unit) { return getUtil(unit, ResourceOwnerFinder.class); } /** * Cleanup the util. * * @param util the util to cleanup */ public static void cleanup(Object util) { if (util instanceof CachingResourceOwnerFinder) CachingResourceOwnerFinder.class.cast(util).cleanup(); } } Meaning it's easy to swap utils behavior for particular deployment unit. e.g. different ResourceOwnerFinder or ReflectProvider Reporting issues As usual, use the forums: MC user forum MC dev forum In my previous article I've actually promised an article on our current native OSGi framework work, but since I was heavily into writing this new scanning lib I though I could share my thoughts / ideas on it while they were still hot. Specially with scanning being a constantly present topic in today's enterprise usage. One thing to note here - all of this is still at prototype stage, with no real release yet, although the main concepts have been in the making for a while, from initial support in VDF, to custom Papaki library, Scannotations, ... hence they've grown from experience. But this doesn't mean feedback is not welcome. :-) I'll be trying to fulfill my promise next time with an OSGi article, unless our Reflect gets the best of me. ;-) P.S.: As usual, again thanks to Marko for doing the editing of this article. About the Author Ales Justin was born in Ljubljana, Slovenia and graduated with a degree in mathematics from the University of Ljubljana. He fell in love with Java eight years ago and has spent most of his time developing information systems, ranging from customer service to energy management. He joined JBoss in 2006 to work full time on the Microcontainer project, currently serving as its lead. He also contributes to JBoss AS and is Seam, Weld and Spring integration specialist. He represent JBoss on 'OSGi' expert groups.
May 11, 2010
by Ales Justin
· 24,075 Views
article thumbnail
Practical PHP Patterns: Service Layer
The last domain logic pattern we will treat in this series is the Service Layer one. In its simplest form, a Service Layer is a set of service classes that deal with application logic, and that are characterized from being used from different front-ends. Source code, at every level of abstraction, is the representation of data entities and their related behavior, particularly in an object-oriented paradigm. There are different types of logic which this behavior can be partitioned into: business logic is encapsulated in a Domain Model, and it is specific to the particular domain the application works in. The added value of an enterprise application ensues primarily from its business logic. application logic is in the scope of a Service Layer, and it is not strictly domain-specific, although its implementation may be. For example, translating objects into an XML or Json representation is part of application logic, even if it is executed with application-specific classes which depends on an underlying Domain Model. The task of representing data in a particular format is oblivious to the domain, as it does not belong to forums or social networks platforms only, or to an electronic or chemistry domain. presentation logic finds in an user interface its quintessence, and it can be considered as the subset of application logic which governs the end user view of the system. I would not consider a difference of format in the whole representation of an object as presentation logic, though, as it falls into the realm of reusability I would want to keep in a Service Layer. Commonly this different concerns of an application are organized in different layers, where each layer resides on the top of the previous one. In classic approaches, there is an infrastructure layer which the business logic layer depends on, and which deals for example with persistence issues. In more evolved approaches, however, keeping the Domain Model as the very core of the application is the most sound choice, moving infrastructure in a Service Layer which can plug in the Domain Model via implementing certain adapter interfaces (like a Repository or a Factory). Thus a Service Layer is particularly useful for example when there are different front-ends to a common Domain Model. These front-ends, such as user interfaces or REST Apis, delegate the application logic to a Service Layer, which encapsulates it. Pushing this logic into the Domain Model would clutter the core of the application, since is is not strictly necessary to work with it. Responsibilities of a Service Layer include, for starters, CRUD functions over objects of the respective Domain Model. Some of these responsibilities may be already included in the Domain Model (Factories for Entities and Value Objects), while other ones are usually only specified as interfaces with the implementations left to an infrastructure Service Layer (Repositories). Example of the latter components are the bread and butter of a Service Layer. Persistence-related actions such as saving objects in a database, data translation (to and from Json, XML, yaml), integration of mail and every service which do not reside in the same PHP execution environment of the Domain Model is a candidate for a class or component in a Service Layer, whose implementations can be stubbed out in acceptance testing. Note that services are also implemented in a Domain Model when they do not contain knowledge which spans outside of the domain and of basic language structures. Generally speaking, services are a point of connection between different Entities and objects, which accomodate operations that would couple the other objects together if implemented directly on them. Only when there are more concerns involved than the execution of logic on domain objects - persistence or orthogonal operations - these services shall be moved to an upper level component like a Service Layer. In some cases, the Service Layer becomes overly generic and orthogonal to the underlying Domain Model, to the point that it is recycled in different applications. Object-relational mappers are part of the infrastructure, and a common example of reusable services. Though, the composition of libraries objects (has-a relationship) is preferable to the direct usage of them, or to their subclassing. Generic frameworks and libraries have a catch-all Api, while a specific Service Layer defines only the use cases actually employed by the front-ends - for example removing the unnecessary update of crucial entities that should be immutable by modelling. Finally, a Service Layer can work without an underlying Domain Model, by interfacing to external web (and non-web) services. If there is local state involved, however, it is difficult to avoid having a basic Domain Model just to define data structures that the Services pass around. The client code, which resides in front-end, has no knowledge of the difference between methods that act over a local Domain Model or an external entity: the Service Layer effectively isolates the upper layer from changes in the lower one. The code sample builds on the sample presented in the Domain Model article, continuing with the idea of an upper level layer. The sample features two service classes, one as a stubbed implementation of an interface of the Domain Model and one that it is employed only at an higher level. setSender('[email protected]'); $mail->setRecipient('[email protected]'); $mail->setSubject('Important stuff'); $mail->setText('I wanted to talk you about...'); return array( $mail ); } } /** * Service that belongs only to the very Service Layer. * It may have an interface, but it will be kept in this layer. */ class EmailTransferService { /** * Transforming an object to XML is a common task for a Service Layer. * However, dependencies towards the Domain Model are well-accepted * but they don't mean that this layer's classes should be * included in the lower layer. * Keeping the XML transformation in one place aids different front-ends * in sharing code. */ public function toXml(Email $mail) { return "\n" . " " . $mail->getSender() . "\n" . " " . $mail->getRecipient() . "\n" . " " . $this->_encode($mail->getSubject()) . "\n" . " " . $this->_encode($mail->getText()) . "\n" . "\n"; } protected function _encode($text) { return htmlentities($text); } } // client code $repository = new DumbEmailRepository(); $transferService = new EmailTransferService(); foreach ($repository->getEmailsFor('[email protected]') as $mail) { echo $transferService->toXml($mail); }
May 2, 2010
by Giorgio Sironi
· 24,382 Views
article thumbnail
Four Methods to Automate Development Environment Setup
There are at least four methods that can be used in different combinations to make the process of setting up a complete development environment a lot less painful.
February 16, 2010
by Mitch Pronschinske
· 31,705 Views
article thumbnail
Fault Injection Testing - First Steps with JBoss Byteman
Fault injection testing[1] is a very useful element of a comprehensive test strategy in that it enables you to concentrate on an area that can be difficult to test; the manner in which the application under test is able to handle exceptions. It's always possible to perform exception testing in a black box mode, where you set up external conditions that will cause the application to fail, and then observe those application failures. Setting and automating (and reproducing) these such as these can, however, be time consuming. (And a pain in the neck, too!) JBoss Byteman I recently found a bytecode injection tool that makes it possible to automate fault injection tests. JBoss Byteman[2] is an open-source project that lets you write scripts in a Java-like syntax to insert events, exceptions, etc. into application code. Byteman version 1.1.0 is available for download from: http://www.jboss.org/byteman - the download includes a programmer's guide. There's also a user forum for asking questions here: http://www.jboss.org/index.html?module=bb&op=viewforum&f=310, and a jboss.org JIRA project for submitted issues and feature requests here: https://jira.jboss.org/jira/browse/BYTEMAN A Simple Example The remainder of this post describes a simple example, on the scale of the classic "hello world" example, of using Byteman to insert an exception into a running application. Let's start by defining the exception that we will inject into our application: package sample.byteman.test; /** * Simple exception class to demonstrate fault injection with byteman */ public class ApplicationException extends Exception { private static final long serialVersionUID = 1L; private int intError; private String theMessage = "hello exception - default string"; public ApplicationException(int intErrNo, String exString) { intError = intErrNo; theMessage = exString; } public String toString() { return "**********ApplicationException[" + intError + " " + theMessage + "]**********"; } } /* class */ There's nothing complicated here, but note the string that is passed to the exception constructor at line 13. Now, let's define our application class: package sample.byteman.test; /** * Simple class to demonstrate fault injection with byteman */ public class ExceptionTest { public void doSomething(int counter) throws ApplicationException { System.out.println("called doSomething(" + counter + ")"); if (counter > 10) { throw new ApplicationException(counter, "bye!"); } System.out.println("Exiting method normally..."); } /* doSomething() */ public static void main(String[] args) { ExceptionTest theTest = new ExceptionTest(); try { for (int i = 0; i < 12; i ++) { theTest.doSomething (i); } } catch (ApplicationException e) { System.out.println("caught ApplicationException: " + e); } } } /* class*/ The application instantiates an instance of ExceptionTest at line 18, then runs the doSomething method in a loop until a counter is greater then 10. Then it raises the exception that we defined earlier. When we run the application, we see this output: java -classpath bytemanTest.jar sample.byteman.test.ExceptionTest called doSomething(0) Exiting method normally... called doSomething(1) Exiting method normally... called doSomething(2) Exiting method normally... called doSomething(3) Exiting method normally... called doSomething(4) Exiting method normally... called doSomething(5) Exiting method normally... called doSomething(6) Exiting method normally... called doSomething(7) Exiting method normally... called doSomething(8) Exiting method normally... called doSomething(9) Exiting method normally... called doSomething(10) Exiting method normally... called doSomething(11) caught ApplicationException: **********ApplicationException[11 bye!]********** OK. Nothing too exciting so far. Let's make things more interesting by scripting a Byteman rule to inject an exception before the doSomething method has a chance to print any output. Our Byteman script looks like this: # # A simple script to demonstrate fault injection with byteman # RULE Simple byteman example - throw an exception CLASS sample.byteman.test.ExceptionTest METHOD doSomething(int) AT INVOKE PrintStream.println BIND buffer = 0 IF TRUE DO throw sample.byteman.test.ApplicationException(1,"ha! byteman was here!") ENDRULE Line 4 - RULE defines the start of the RULE. The following text on this line is not executed Line 5 - Reference to the class of the application to receive the injection Line 6 - And the method in that class. Note that since if we had written this line as "METHOD doSomething", the rule would have matched any signature of the soSomething method Line 7 - Our rule will fire when the PrintStream.println method is invoked Line 8 - BIND determince values for variables which can be referenced in the rule body - in our example, the recipient of the doSomething method call that triggered the rule, is identified by the parameter reference $0 Line 9 - A rule has to include an IF clause - in our example, it's always true Line 10 - When the rule is triggered, we throw an exception - note that we supply a string to the exception constructor Now, before we try to run this run, we should check the its syntax. To do this, we build our application into a .jar (bytemanTest.jar in our case) and use bytemancheck.sh sh bytemancheck.sh -cp bytemanTest.jar byteman.txt checking rules in sample_byteman.txt TestScript: parsed rule Simple byteman example - throw an exception RULE Simple byteman example - throw an exception CLASS sample.byteman.test.ExceptionTest METHOD doSomething(int) AT INVOKE PrintStream.println BIND buffer : int = 0 IF TRUE DO throw (1"ha! byteman was here!") TestScript: checking rule Simple byteman example - throw an exception TestScript: type checked rule Simple byteman example - throw an exception TestScript: no errors Once we get a clean result, we can run the application with Byteman. To do this, we run the application and specify an extra argument to the java command. Note that Byteman requires JDK 1.6 or newer. java -javaagent:/opt/Byteman_1_1_0/build/lib/byteman.jar=script:sample_byteman.txt -classpath bytemanTest.jar sample.byteman.test.ExceptionTest And the result is: caught ApplicationException: **********ApplicationException[1 ha! byteman was here!]********** Now that the Script Works, Let's Improve it! Let's take a closer look and how we BIND to a method parameter. If we change the script to read as follows: # # A simple script to demonstrate fault injection with byteman # RULE Simple byteman example - throw an exception CLASS sample.byteman.test.ExceptionTest METHOD doSomething(int) AT INVOKE PrintStream.println BIND counter = $1 IF TRUE DO throw sample.byteman.test.ApplicationException(counter,"ha! byteman was here!") ENDRULE In line 8, the BIND clause now refers to the int method parameter by index using the syntax $1. This change makes the value available inside the rule body by enabling us to use the name "counter." The value of counter is then supplied as the argument to the constructor for the ApplicationException class. This new version of the rule demonstrates shows how we can use local state as derived from the trigger method to construct our exception object. But wait there's more! Let's use the "counter" value as a counter. It's useful to be able to force an exception the first time a method is called. But, it's even more useful to be able to force an exception at a selected invocation of a method. Let's add a test for that counter value to the script: # # A simple script to demonstrate fault injection with byteman # RULE Simple byteman example 2 - throw an exception at 3rd call CLASS sample.byteman.test.ExceptionTest METHOD doSomething(int) AT INVOKE PrintStream.println BIND counter = $1 IF counter == 3 DO throw sample.byteman.test.ApplicationException(counter,"ha! byteman was here!") ENDRULE In line 9, we've changed the IF clause to make use of the counter value. When we run the test with this script, the first 2 calls to doSomething succeed, but the third one fails. One Last Thing - Changing the Script for a Running Process So far, so good. We've been able to inject a fault/exception into our running application, and even specify which iteration of a loop in which it happens. Suppose, however, we want to change a value in a byteman script, while the application is running? No problem! Here's how. First, we need to alter our application so that it can run for a long enough time for us to alter the byteman script. Here's a modified version of the doSomething method that waits for user input: public void doSomething(int counter) throws ApplicationException { BufferedReader lineOfText = new BufferedReader(new InputStreamReader(System.in)); try { System.out.println("Press "); String textLine = lineOfText.readLine(); } catch (IOException e) { e.printStackTrace(); } System.out.println("called doSomething(" + counter + ")"); if (counter > 10) { throw new ApplicationException(counter, "bye!"); } System.out.println("Exiting method normally..."); } If we run this version of the application, we'll see output like this: Press called doSomething(0) Exiting method normally... Press called doSomething(1) Exiting method normally... Press called doSomething(2) Exiting method normally... caught ApplicationException: **********ApplicationException[3 ha! byteman was here!]********** Let's run the application again, but this time, don't press . While the application is waiting for input, create a copy of the byteman script. In this copy, change the IF clause to have a loop counter set to a different value, say '5.' Then, open up a second command shell window and enter this command: Byteman_1_1_0/bin/submit.sh sample_byteman_changed.txt Then, return to the first command shell window and start pressing return, and you'll see this output: Press redefining rule Simple byteman example - throw an exception called doSomething(0) Exiting method normally... Press called doSomething(1) Exiting method normally... Press called doSomething(2) Exiting method normally... Press called doSomething(3) Exiting method normally... Press called doSomething(4) Exiting method normally... caught ApplicationException: **********ApplicationException[5 ha! byteman was here!]********** So, we were able to alter the value in the original byteman script, without stopping the application under test! Pitfalls Along the Way Some of the newbee mistakes that I made along the way were: Each RULE needs an IF clause - even if you want the rule to always fire The methods referenced in a RULE cannot be static - if they are static, then there is no $0 (aka this) to reference Yes, I had several errors and some typos the first few times I tried this. A syntax checker is always my best friend. ;-) Closing Thoughts With this simple example, we're able to inject injections into a running application in an easily automated/scripted manner. But, We've only scratched the surface with Byteman. In subsequent posts, I'm hoping to explore using Byteman to cause more widespread havoc in software testing. References [1] http://en.wikipedia.org/wiki/Fault_injection [2] http://www.jboss.org/byteman (Special thanks to Andrew Dinn for his help! ;-)
October 16, 2009
by Len DiMaggio
· 17,474 Views
article thumbnail
A Look Inside JBoss Microcontainer, Part 3 - the Virtual File System
We're finally back with our next article in the Microcontainer series. In the first two articles we demonstrated how Microcontainer supports , and showed its powerful . In this article, we'll explain Classloading and Deployers, but first we must familiarize ourselves with VFS. VFS stands, as expected, for Virtual File System. What does VFS solve for us, or why is it useful? Here, at JBoss, we saw that a lot of similar resource handling code was scattered/duplicated all over the place. In most cases it was code that was trying to determine what type of resource a particular resource was, e.g. is it a file, a directory, or a jar loading resources through URLs. Processing of nested archives was also reimplemented again, and again in different libraries. Read the other parts in DZone's exclusive JBoss Microcontainer Series: Part 4 -- ClassLoading Layer Example: public static URL[] search(ClassLoader cl, String prefix, String suffix) throws IOException { Enumeration[] e = new Enumeration[]{ cl.getResources(prefix), cl.getResources(prefix + "MANIFEST.MF") }; Set all = new LinkedHashSet(); URL url; URLConnection conn; JarFile jarFile; for (int i = 0, s = e.length; i < s; ++i) { while (e[i].hasMoreElements()) { url = (URL)e[i].nextElement(); conn = url.openConnection(); conn.setUseCaches(false); conn.setDefaultUseCaches(false); if (conn instanceof JarURLConnection) { jarFile = ((JarURLConnection)conn).getJarFile(); } else { jarFile = getAlternativeJarFile(url); } if (jarFile != null) { searchJar(cl, all, jarFile, prefix, suffix); } else { boolean searchDone = searchDir(all, new File(URLDecoder.decode(url.getFile(), "UTF-8")), suffix); if (searchDone == false) { searchFromURL(all, prefix, suffix, url); } } } } return (URL[])all.toArray(new URL[all.size()]); } private static boolean searchDir(Set result, File file, String suffix) throws IOException { if (file.exists() && file.isDirectory()) { File[] fc = file.listFiles(); String path; for (int i = 0; i < fc.length; i++) { path = fc[i].getAbsolutePath(); if (fc[i].isDirectory()) { searchDir(result, fc[i], suffix); } else if (path.endsWith(suffix)) { result.add(fc[i].toURL()); } } return true; } return false; } There were also many problems with file locking on Windows systems, which forced us to copy all hot-deployable archives to another location to prevent locking those in deploy folders (which would prevent their deletion and filesystem based undeploy). File locking was a major problem that could only be addressed by centralizing all the resource loading code in one place. Recognizing a need to deal with all of these issues in one place, wrapping it all into a simple and useful API, we created the VFS project. VFS public API Basic usage in VFS can be split in two pieces: simple resource navigation visitor pattern API As mentioned, in plain JDK resource handling navigation over resources is far from trivial. You must always check what kind of resource you're currently handling, and this is very cumbersome. With VFS we wanted to limit this to a single resource type - VirtualFile. public class VirtualFile implements Serializable { /** * Get certificates. * * @return the certificates associated with this virtual file */ Certificate[] getCertificates() /** * Get the simple VF name (X.java) * * @return the simple file name * @throws IllegalStateException if the file is closed */ String getName() /** * Get the VFS relative path name (org/jboss/X.java) * * @return the VFS relative path name * @throws IllegalStateException if the file is closed */ String getPathName() /** * Get the VF URL (file://root/org/jboss/X.java) * * @return the full URL to the VF in the VFS. * @throws MalformedURLException if a url cannot be parsed * @throws URISyntaxException if a uri cannot be parsed * @throws IllegalStateException if the file is closed */ URL toURL() throws MalformedURLException, URISyntaxException /** * Get the VF URI (file://root/org/jboss/X.java) * * @return the full URI to the VF in the VFS. * @throws URISyntaxException if a uri cannot be parsed * @throws IllegalStateException if the file is closed * @throws MalformedURLException for a bad url */ URI toURI() throws MalformedURLException, URISyntaxException /** * When the file was last modified * * @return the last modified time * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ long getLastModified() throws IOException /** * Returns true if the file has been modified since this method was last called * Last modified time is initialized at handler instantiation. * * @return true if modifed, false otherwise * @throws IOException for any error */ boolean hasBeenModified() throws IOException /** * Get the size * * @return the size * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ long getSize() throws IOException /** * Tests whether the underlying implementation file still exists. * @return true if the file exists, false otherwise. * @throws IOException - thrown on failure to detect existence. */ boolean exists() throws IOException /** * Whether it is a simple leaf of the VFS, * i.e. whether it can contain other files * * @return true if a simple file. * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ boolean isLeaf() throws IOException /** * Is the file archive. * * @return true if archive, false otherwise * @throws IOException for any error */ boolean isArchive() throws IOException /** * Whether it is hidden * * @return true when hidden * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ boolean isHidden() throws IOException /** * Access the file contents. * * @return an InputStream for the file contents. * @throws IOException for any error accessing the file system * @throws IllegalStateException if the file is closed */ InputStream openStream() throws IOException /** * Do file cleanup. * * e.g. delete temp files */ void cleanup() /** * Close the file resources (stream, etc.) */ void close() /** * Delete this virtual file * * @return true if file was deleted * @throws IOException if an error occurs */ boolean delete() throws IOException /** * Delete this virtual file * * @param gracePeriod max time to wait for any locks (in milliseconds) * @return true if file was deleted * @throws IOException if an error occurs */ boolean delete(int gracePeriod) throws IOException /** * Get the VFS instance for this virtual file * * @return the VFS * @throws IllegalStateException if the file is closed */ VFS getVFS() /** * Get the parent * * @return the parent or null if there is no parent * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ VirtualFile getParent() throws IOException /** * Get a child * * @param path the path * @return the child or null if not found * @throws IOException for any problem accessing the VFS * @throws IllegalArgumentException if the path is null * @throws IllegalStateException if the file is closed or it is a leaf node */ VirtualFile getChild(String path) throws IOException /** * Get the children * * @return the children * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ List getChildren() throws IOException /** * Get the children * * @param filter to filter the children * @return the children * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed or it is a leaf node */ List getChildren(VirtualFileFilter filter) throws IOException /** * Get all the children recursively * * This always uses {@link VisitorAttributes#RECURSE} * * @return the children * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed */ List getChildrenRecursively() throws IOException /** * Get all the children recursively * * This always uses {@link VisitorAttributes#RECURSE} * * @param filter to filter the children * @return the children * @throws IOException for any problem accessing the virtual file system * @throws IllegalStateException if the file is closed or it is a leaf node */ List getChildrenRecursively(VirtualFileFilter filter) throws IOException /** * Visit the virtual file system * * @param visitor the visitor * @throws IOException for any problem accessing the virtual file system * @throws IllegalArgumentException if the visitor is null * @throws IllegalStateException if the file is closed */ void visit(VirtualFileVisitor visitor) throws IOException } As you can see you have all of the usual read-only File System operations, plus a few options to cleanup or delete the resource. Cleanup or deletion handling is needed when we're dealing with some internal temporary files; e.g. from nested jars handling. To switch from JDK's File or URL resource handling to new VirtualFile we need a root. It is the VFS class that knows how to create one with the help of URL or URI parameter. public class VFS { /** * Get the virtual file system for a root uri * * @param rootURI the root URI * @return the virtual file system * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL is null */ static VFS getVFS(URI rootURI) throws IOException /** * Create new root * * @param rootURI the root url * @return the virtual file * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL */ static VirtualFile createNewRoot(URI rootURI) throws IOException /** * Get the root virtual file * * @param rootURI the root uri * @return the virtual file * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL is null */ static VirtualFile getRoot(URI rootURI) throws IOException /** * Get the virtual file system for a root url * * @param rootURL the root url * @return the virtual file system * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL is null */ static VFS getVFS(URL rootURL) throws IOException /** * Create new root * * @param rootURL the root url * @return the virtual file * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL */ static VirtualFile createNewRoot(URL rootURL) throws IOException /** * Get the root virtual file * * @param rootURL the root url * @return the virtual file * @throws IOException if there is a problem accessing the VFS * @throws IllegalArgumentException if the rootURL */ static VirtualFile getRoot(URL rootURL) throws IOException /** * Get the root file of this VFS * * @return the root * @throws IOException for any problem accessing the VFS */ VirtualFile getRoot() throws IOException } You can see three different methods that look a lot alike - getVFS, createNewRoot and getRoot. Method getVFS returns a VFS instance, and what's important, it doesn't yet create a VirtualFile instance. Why is this important? Because there are methods which help us configure a VFS instance (see VFS class API javadocs), before telling it to create a VirtualFile root. The other two methods, on the other hand, use default settings for root creation. The difference between createNewRoot and getRoot is in caching details, which we'll delve in later on. URL rootURL = ...; // get root url VFS vfs = VFS.getVFS(rootURL); // configure vfs instance VirtualFile root1 = vfs.getRoot(); // or you can get root directly VirtualFile root2 = VFS.crateNewRoot(rootURL); VirtualFile root3 = VFS.getRoot(rootURL); The other useful thing about VFS API is its implementation of a proper visitor pattern. This way it's very simple to recursively gather different resources, something quite impossible to do with plain JDK resource loading. public interface VirtualFileVisitor { /** * Get the search attribues for this visitor * * @return the attributes */ VisitorAttributes getAttributes(); /** * Visit a virtual file * * @param virtualFile the virtual file being visited */ void visit(VirtualFile virtualFile); } VirtualFile root = ...; // get root VirtualFileVisitor visitor = new SuffixVisitor(".class"); // get all classes root.visit(visitor); VFS Architecture While public API is quite intuitive, real implementation details are a bit more complex. We'll try to explain the concepts in a quick pass. Each time you create a VFS instance, its matching VFSContext instance is created. This creation is done via VFSContextFactory. Different protocols map to different VFSContextFactory instances - e.g. file/vfsfile map to FileSystemContextFactory, zip/vfszip map to ZipEntryContextFactory. Also, each time a VirtualFile instance is created, its matching VirtualFileHandler is created. It's this VirtualFileHandler instance that knows how to handle different resource types properly - VirtualFile API just delegates invocations to its VirtualFileHandler reference. As one could expect, VFSContext instance is the one that knows how to create VirtualFileHandler instances accordingly to a resource type - e.g. ZipEntryContextFactory creates ZipEntryContext, which then creates ZipEntryHandler. Existing implementations Apart from files, directories (FileHandler) and zip archives (ZipEntryHandler) we also support other more exotic usages. The first one is Assembled, which is similar to what Eclipse calls Linked Resources. Its idea is to take existing resources from different trees, and "mock" them into single resource tree. AssembledDirectory sar = AssembledContextFactory.getInstance().create("assembled.sar"); URL url = getResource("/vfs/test/jar1.jar"); VirtualFile jar1 = VFS.getRoot(url); sar.addChild(jar1); url = getResource("/tmp/app/ext.jar"); VirtualFile ext1 = VFS.getRoot(url); sar.addChild(ext); AssembledDirectory metainf = sar.mkdir("META-INF"); url = getResource("/config/jboss-service.xml"); VirtualFile serviceVF = VFS.getRoot(url); metainf.addChild(serviceVF); AssembledDirectory app = sar.mkdir("app.jar"); url = getResource("/app/someapp/classes"); VirtualFile appVF = VFS.getRoot(url); app.addPath(appVF, new SuffixFilter(".class")); Another implementation is in-memory files. In our case this came out of a need to easily handle AOP generated bytes. Instead of mucking around with temporary files, we simply drop bytes into in-memory VirtualFileHandlers. URL url = new URL("vfsmemory://aopdomain/org/acme/test/Test.class"); byte[] bytes = ...; // some AOP generated class bytes MemoryFileFactory.putFile(url, bytes); VirtualFile classFile = VFS.getVirtualFile(new URL("vfsmemory://aopdomain"), "org/acme/test/Test.class"); InputStream bis = classFile.openStream(); // e.g. load class from input stream Extension hooks It's quite easy to extend VFS with a new protocol, similar to what we've done with Assembled and Memory. All you need is a combination of VFSContexFactory, VFSContext, VirtualFileHandler, FileHandlerPlugin and URLStreamHandler implementations. The first one is trivial, while the others depend on the complexity of your task - e.g. you could implement rar, tar, gzip or even remote access. In the end you simply register this new VFSContextFactory with VFSContextFactoryLocator. See this article's demo for a simple gzip example Features One of the first major problems we stumbled upon was proper usage of nested resources, more exactly nested jar files. e.g. normal ear deployments: gema.ear/ui.war/WEB-INF/lib/struts.jar In order to read contents of struts.jar we have two options: handle resources in memory create top level temporary copies of nested jars, recursively The first option is easier to implement, but it's very memory-consuming--just imagine huge apps in memory. The other approach leaves a bunch of temporary files, which should be invisible to plain user. Hence expecting them to disappear once the deployment is undeployed. Now imagine the following scenario: A user gets a hold of VFS's URL instance, which points to some nested resource. The way plain VFS would handle this is to re-create the whole path from scratch, meaning it would unpack nested resources over and over again. This would (and it did) lead to a huge pile of temporary files. How to avoid this? The way we approached this is by using VFSRegistry, VFSCache and TempInfo. When you ask for VirtualFile over VFS (getRoot, not createNewRoot), VFS asks VFSRegistry implementation to provide the file. Existing DefaultVFSRegistry first checks if matching root VFSContext for provided URI exists. If it does, it first tries to navigate to existing TempInfo (link to temporary files), falling back to regular navigation if no such temporary file exists. This way we completely re-use any already unpacked temporary files, saving time and disk space. If no matching VFSContext is found in cache, we create a new VFSCache entry, and continue with default navigation. It's then up to VFSCache implementation used, how it handles cached VFSContext entries. VFSCache is configurable via VFSCacheFactory - by default we don't cache anything, but there are a few useful existing VFSCache implementations, ranging from LRU to timed cache. API Use case There is a class called VFSUtils which is part of a public API, and it is sort of a dumping ground of useful functionality. It contains a bunch of helpful methods and configuration settings (system property keys, actually). Check the API javadocs for more details. Existing issues / workarounds Another issue that came up - expectedly - was inability of some frameworks to properly work on top of VFS. The problem lied in custom VFS urls like: vfsfile, vfszip, vfsmemory. In most cases you could still work around it with plain URL or URLConnection usage, but a lot of frameworks do a strict match on file or jar protocol, which of course fails. We were able to patch some frameworks (e.g. Facelets) and provide extensions to others (e.g. Spring). If you are a library developer, and your library has a simple pluggable resource loading mechanism, then we suggest you simply extend it with VFS based implementation. If there are no hooks, try to limit your assumptions to more general usage based on URL or URLConnection. Conclusion While VFS is very nice to use, it comes at a price. It adds additional layer on top of JDK's resource handling, meaning extra invocations are always present when you're dealing with resources. We also keep some of the jar handling info in memory to make it easy to get hold of a specific resource, but at the expense of some extra memory consumption. Overall VFS proved to be a very useful library as it hides away many use cases that are painful with plain JDK, and provides a comprehensive API for working with resources - i.e. visitor pattern implementation. We're constantly following user feedback to VFS issues they encounter, making each version a bit better. Now, that we got to know VFS, it's time we move on to MC's new Classloading layer! About the Author Ales Justin was born in Ljubljana, Slovenia and graduated with a degree in mathematics from the University of Ljubljana. He fell in love with Java seven years ago and has spent most of his time developing information systems, ranging from customer service to energy management. He joined JBoss in 2006 to work full time on the Microcontainer project, currently serving as its lead. He also contributes to JBoss AS and is Seam and Spring integration specialist. He represent JBoss on 'JSR-291 Dynamic Component Support for Java SE' and 'OSGi' expert groups.
September 24, 2009
by Ales Justin
· 43,257 Views
article thumbnail
Java Performance Tuning, Profiling, and Memory Management
Get a perspective on the aspects of JVM internals, controls, and switches that can be used to optimize your Java application.
September 1, 2009
by Vikash Ranjan
· 257,670 Views · 17 Likes
article thumbnail
JPA 2.0 Concurrency and Locking
Optimistic locking lets concurrent transactions process simultaneously, but detects and prevent collisions, this works best for applications where most concurrent transactions do not conflict. JPA Optimistic locking allows anyone to read and update an entity, however a version check is made upon commit and an exception is thrown if the version was updated in the database since the entity was read. In JPA for Optimistic locking you annotate an attribute with @Version as shown below: public class Employee { @ID int id; @Version int version; The Version attribute will be incremented with a successful commit. The Version attribute can be an int, short, long, or timestamp. This results in SQL like the following: “UPDATE Employee SET ..., version = version + 1 WHERE id = ? AND version = readVersion” The advantages of optimistic locking are that no database locks are held which can give better scalability. The disadvantages are that the user or application must refresh and retry failed updates. Optimistic Locking Example In the optimistic locking example below, 2 concurrent transactions are updating employee e1. The transaction on the left commits first causing the e1 version attribute to be incremented with the update. The transaction on the right throws an OptimisticLockException because the e1 version attribute is higher than when e1 was read, causing the transaction to roll back. Additional Locking with JPA Entity Locking APIs With JPA it is possible to lock an entity, this allows you to control when, where and which kind of locking to use. JPA 1.0 only supported Optimistic read or Optimistic write locking. JPA 2.0 supports Optimistic and Pessimistic locking, this is layered on top of @Version checking described above. JPA 2.0 LockMode values : OPTIMISTIC (JPA 1.0 READ): perform a version check on locked Entity before commit, throw an OptimisticLockException if Entity version mismatch. OPTIMISTIC_FORCE_INCREMENT (JPA 1.0 WRITE) perform a version check on locked Entity before commit, throw an OptimisticLockException if Entity version mismatch, force an increment to the version at the end of the transaction, even if the entity is not modified. PESSIMISTIC: lock the database row when reading PESSIMISTIC_FORCE_INCREMENT lock the database row when reading, force an increment to the version at the end of the transaction, even if the entity is not modified. There are multiple APIs to specify locking an Entity: EntityManager methods: lock, find, refresh Query methods: setLockMode NamedQuery annotation: lockMode element OPTIMISTIC (READ) LockMode Example In the optimistic locking example below, transaction1 on the left updates the department name for dep , which causes dep's version attribute to be incremented. Transaction2 on the right gives an employee a raise if he's in the "Eng" department. Version checking on the employee attribute would not throw an exception in this example since it was the dep Version attribute that was updated in transaction1. In this example the employee change should not commit if the department was changed after reading, so an OPTIMISTIC lock is used : em.lock(dep, OPTIMISTIC). This will cause a version check on the dep Entity before committing transaction2 which will throw an OptimisticLockException because the dep version attribute is higher than when dep was read, causing the transaction to roll back. OPTIMISTIC_FORCE_INCREMENT (write) LockMode Example In the OPTIMISTIC_FORCE_INCREMENT locking example below, transaction2 on the right wants to be sure that the dep name does not change during the transaction, so transaction2 locks the dep Entity em.lock(dep, OPTIMISTIC_FORCE_INCREMENT) and then calls em.flush() which causes dep's version attribute to be incremented in the database. This will cause any parallel updates to dep to throw an OptimisticLockException and roll back. In transaction1 on the left at commit time when the dep version attribute is checked and found to be stale, an OptimisticLockException is thrown Pessimistic Concurrency Pessimistic concurrency locks the database row when data is read, this is the equivalent of a (SELECT . . . FOR UPDATE [NOWAIT]) . Pessimistic locking ensures that transactions do not update the same entity at the same time, which can simplify application code, but it limits concurrent access to the data which can cause bad scalability and may cause deadlocks. Pessimistic locking is better for applications with a higher risk of contention among concurrent transactions. The examples below show: reading an entity and then locking it later reading an entity with a lock reading an entity, then later refreshing it with a lock The Trade-offs are the longer you hold the lock the greater the risks of bad scalability and deadlocks. The later you lock the greater the risk of stale data, which can then cause an optimistic lock exception, if the entity was updated after reading but before locking. The right locking approach depends on your application: what is the risk of risk of contention among concurrent transactions? What are the requirements for scalability? What are the requirements for user re-trying on failure? For More Information: Preventing Non-Repeatable Reads in JPA Using EclipseLink Java Persistence API 2.0: What's New ? What's New and Exciting in JPA 2.0 Beginning Java™ EE 6 Platform with GlassFish™ 3 Pro EJB 3: Java Persistence API (JPA 1.0)
August 3, 2009
by Carol McDonald
· 51,387 Views · 1 Like
article thumbnail
Performance Monitoring Using Glassbox
The industry is recognizing the fact that performance testing & engineering should be part of the project execution road map starting from the requirements gathering phase. At many times during project executions, performance engineering related activities are executed based on customer need or slow response time of application after development phase gets completed. Glassbox can be leveraged (by developers/testers/business users) during and after the development cycle to monitor the response times of requests with-out being aware of underlying application structure and code details. Analysis generated by Glassbox gives direct pointers on where is the bottleneck which causes slow response time for that particular request/page/URL. About Glassbox Glassbox is an open source web application which aid in performance monitoring and troubleshooting of multiple web applications deployed in container. Troubleshooting It contains the built-in knowledge repository of common problems which are used to pinpoint the issues and suggestions on causes as Java code executes. Performance Monitoring It monitors the requests as Java code executes and provides details about response times. Glassbox web client (AJAX GUI) provides nice summary dashboard view which contains various attributes like (server-name, application name, operation/request-URL, average time, no. of executions, status (slow / OK) and analysis details). By default, an operation that takes more than 1 sec execution time is marked as SLOW status. Such SLA can be modified using Glassbox properties file. Analysis part describes the problem precisely and very clearly in plain English words, rather than displaying large code/exception trace. This definitely increases developer productivity by reducing developer’s time spent in log files and using IDE debuggers. Internals The two main components of Glassbox are Monitor and Agent. Monitor uses Aspect-Oriented Programming (AOP) to monitor the JVM activity. Agent diagnoses and presents the monitoring results and uses knowledge repository to cross reference the problem with suggestions/solutions. Glassbox agent supports viewing of the analysis results using JMX (eg. Java 5 JConsole) Consoles. Glassbox extensively uses the AOP approach internally to monitor the Java code. This gives the benefit of not making any changes to source code or build-process and hence can work with any legacy web application/jar file as well. Technologies Glassbox should work on any application server that supports Servlet 2.3 or later. The servers where Glassbox is tested and installation process is automated are Apache Tomcat, weblogic, websphere, Resin, Oracle OC4J, websphere, Resin, Jetty & GlassFish. Overhead Having Glassbox application running on same container would generate a performance overhead. Typically this would affect the response time and memory overhead. Hence it is recommended to start the Glassbox application only when it’s required for performance monitoring. Licensing Glassbox is an open source project, it is free to download and run. Glassbox uses the GNU Lesser General Public License to distribute software and documentation. Demo Application Development & Deployment to Tomcat To test the capabilities of Glassbox, a sample application is developed which has a TestServlet class. This servlet calls DelayGenerator class’s generateDelay() method. This method calls Thread class’s sleep() method which suspends the execution of servlet. A counter is being initialized in DelayGenerator class which determines the time interval till which servlet is needed to be suspended. TestServlet.java /** * File: TestServlet.java * @author Viral Thakkar */ package com.infosys.star.glassbox; import java.io.IOException; import java.io.PrintWriter; import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; public class TestServlet extends HttpServlet { protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { DelayGenerator delayObj = new DelayGenerator(); int delay = delayObj.generateDelay(); response.setContentType("text/html"); PrintWriter out = response.getWriter(); out.println(""); out.println(" Hello World from Test Servlet : "+delay+" milliseconds "); out.println(""); out.flush(); } } DelayGenerator.java /** * File: DelayGenerator.java * @author Viral Thakkar */ package com.infosys.star.glassbox; public class DelayGenerator { private static int counter = 1; public int generateDelay() { try { Thread.sleep(counter * 100); counter++; } catch (InterruptedException e) { e.printStackTrace(); } return counter*100; } } Glassbox Installation & Integration to Apache Tomcat 6.0 Glassbox installation is very straightforward for non-clustered environment for the server where it’s automated. Simply drop the glassbox.war file at the appropriate folder inside server folder or perform the server specific steps/configuration to deploy the war file. Browse to server url with context root as glassbox – http://<>:/glassbox. Follow the instructions available on this page. According to specific server, this page would suggest the configuration changes for a server. Please refer to Glassbox User Guide document for details on how to install Glassbox for clustered application server environment. For Apache Tomcat 6.0- Add following command line arguments to Tomcat’s Java options: -Dglassbox.install.dir=C:\Tomcat6.0\lib\glassbox -Djava.rmi.server.useCodebaseOnly=true -javaagent:C:\Tomcat6.0\lib\aspectjweaver.jar Monitoring & Technical Analysis Glassbox web client (URL- http://<>:<>/glassbox ) shows the summary and detailed view of all the requests/operations that container/JVM has executed. Summary Section View Different attributes (columns) which gets displayed in this table are as below - Attribute / Column Name Comments Status This indicates whether operation/request is performing OK, SLOW or FAILING Analysis For SLOW/FAILING status, this value provides the small summary of the cause of the problem. Operation This is name of the operation/request of an application Server Name of the server where monitoring is being done. In a clustered environment, this allows to distinguish operations on different servers. Executions This value indicates how many times this operation has run since the application server was started or Glassbox’s statistics were last reset. Click the request in above summary table to view its detailed analysis in below detailed section. Detailed Section View The details area provides information relating to operations selected in the summary table. Different sub-sections which gets displayed in this view are as below - Sub-section Name Comments Executive Summary High level summary view of the selected operation gets displayed in a table format. This is neat view to senior stake holders who are not interested in technical details. Technical Summary This section contains more technical details in paragraph and table representation formats to provide insight into root cause of the problem if any, like which operation, query is slow and statistics of same. Details like stack trace, thread lock name are provided to find and fix the problem. “Common solutions” sub section shows pointers to resolve the identified problem/s. “Glassbox has ruled out other potential problems” sub section saves time to know what problems have already been ruled out. Executive Summary View Technical Summary -> Technical Details Views Above two snapshots are parts of the Technical Details section and provide minute details at code level with line number so as to pinpoint where the problem is. Here cause is identified at Class com.infosys.star.glassbox.DelayGenerator inside Method generateDelay at line number 12 where Thread.sleep is invoked. Perform Load Testing Using JMeter and Monitor Using Glassbox Apache JMeter is used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. It can be used to make a graphical analysis of performance or to test server/script/object behavior under heavy concurrent load. Using JMeter, create a test plan that simulates 10 users requesting for 1 page 5 times. i.e. 10 x 1 x 5 = 50 HTTP requests. First step is to add a Thread Group element. The Thread Group tells JMeter the number of users to simulate, how often the users should send requests, and the how many requests they should send. Next step is to add HTTP Request element to added Thread Group. In parallel, have the Glassbox up and running to monitor response time statistics of the load generated by JMeter application. Below is the Executive summary view of above test in Glassbox web UI interface. Section “Monitoring & Technical Analysis” contains the details to understand the Glassbox generated analysis. Conclusion Glassbox is not the replacement for performance testing tool like load runner. Glassbox aids in the project to various stakeholders in finding, conveying and fixing the performance problems at all phases starting build (development) to post deployment. Glassbox application to be started/installed only during monitoring time so as to avoid the performance overhead for other applications due to CPU & memory footprint occupied by Glassbox application on the container. During load testing of the application, Glassbox turns out to be good option to figure out the root causes inside an application code. References Glassbox web site - http://www.glassbox.com/glassbox/Home.html Glassbox User Guide - http://nchc.dl.sourceforge.net/sourceforge/glassbox/Glassboxv2.0UserGuide.pdf Apache JMeter - http://jakarta.apache.org/jmeter/ Download & Support Glassbox Download Link - http://www.glassbox.com/glassbox/Downloads.html Glassbox forum Link - http://sourceforge.net/forum/forum.php?forum_id=575670 About Author Viral Thakkar is a Technical Architect with the Banking and Capital Markets vertical at Infosys. He has 9.5 years of technology consulting experience mainly on Java/JEE technologies and frameworks with large banks and financial institutions across the globe. He has been part of many small and large-scale initiatives related to application development, architecture creation and strategy definition. From http://viralpatel.net/blogs
March 5, 2009
by Viral Thakkar
· 20,616 Views
article thumbnail
JBoss RichFaces with Spring
This article is going to show you how to build a RichFaces application with Spring.
February 16, 2009
by Max Katz
· 203,767 Views
article thumbnail
Check Directory/dir/file Size In Linux
#check partition sizes df -h #check directory size du -s -h /var/log/ #check every directory and file sizes under a dir. du -s -h /var/log/* #check individual size size du -s -h /var/log/lastlog
December 9, 2008
by Snippets Manager
· 17,033 Views
article thumbnail
Configuring Logging in JBoss
Learn how to properly configure logs in JBoss.
November 19, 2008
by Meera Subbarao
· 223,075 Views
article thumbnail
Seven Forms of Business Process Management With JBoss jBPM
This article will explain Business Process Management (BPM) in terms of 7 distinct use cases for JBoss jBPM. By giving more insight in those use cases, you'll get a better understanding of the different forms of BPM and workflow and when a BPM engine like jBPM makes sense in your project. We'll also highlight the specific jPDL process language features related to those use cases. The term BPM is highly overloaded and used for many different things resulting in a lot of confusion. These use cases give concrete descriptions for the different interpretations of the term BPM. The individual nature of these use cases is important. BPM software vendors often take a mix of different aspects and concerns into account when developing their products. That often results into BPM products that are suitable only for a specific purpose in a specific environment. This is in my opinion the reason why there are so many different BPM products which only serve a small niche market. It also explains why new products and standards in this space keep appearing, don't get enough momentum and then get pushed aside by yet another new product or standard. When evaluating a BPM products, it should be done with specific use cases in mind. What is JBoss jBPM JBoss jBPM is a flexible and powerful BPM engine. In essence, BPM systems allow for execution flows to be specified graphically. As an example, here is a process diagram for a business trip: [img_assist|nid=5583|title=|desc=|link=none|align=undefined|width=623|height=487] A key capability of BPM systems is that processes steps can be wait states. For example in the business trip process above, nodes 'manager evaluation' and 'ticket purchase' are human tasks. When the execution of the process arrives in those nodes, the system executing the process should wait till the assigned user completes the task. From a software technical point of view that capability is a big deal. As the alternative is a bunch of methods that are linked by HTTP requests, Message Driven Beans (MDB), database triggers, task forms, etc. Even when using the most applicable architectural components available in Java today, it is still very easy to end up in a bunch of unmaintainable hooks and eyes. Using an overall business process makes it a lot easier to see and maintain the overall execution flow, even from a software technical perspective. JBoss jBPM does exactly that and it differentiates itself from other BPM projects in the following topics: Easily embeddable into a Java project. Traditional BPM Systems typically require a separate server to be installed which makes it hard to integrate into the Java software development cycle. One of the deployments that JBoss jBPM supports is just adding the jBPM library to the classpath. The jBPM tables can be hosted in any database next to the application's tables. Using JBoss jBPM really fits with the normal way of developing Java software. Support for multiple process languages. The view on what BPM actually is has not yet been stabilized. There are currently many different interpretations of what BPM is, resulting in big fragmentation in the market. In fact, the body of this article tries to identify 6 distinct concepts that all are associated to BPM. Very flexible transaction management. If your application just uses a JDBC connection in a standard Java environment, then jBPM can use that very JDBC connection to perform its work. If your application uses hibernate in a standard environment, then jBPM can use the same hibernate session factory. If your application runs in an enterprise environment, then jBPM can bind to the surrounding JTA transaction. Readable jPDL process language. For developers it is very convenient if the process language is compact and readable. Not all developers want to keep using the graphical editor. Most hard core programmers start to hand code the processes after a while. Then jBPM's jPDL process language is the most readable, compact and complete. Standards BPEL and BPMN Todays most known standards in the space of BPM are BPEL and BPMN. Those two standards have a completely different background. That is a clear indication of the diverse type of problems that is currently associated to BPM. BPEL is defined with an Enterprise Service Bus (ESB) in mind. It's all based on WSDL, which binds naturally to web services. The result of deploying a BPEL process is that a new service is being published. A BPEL process can hence be seen as a scripting language for services on the bus. A more elaborate conceptual explanation of BPEL can be found in Process Component Models: The Next Generation In Workflow ? So BPEL is an executable process language, which implies that a it is human to computer communication, just like a programming language. On the other hand, the first target of BPMN is modeling process diagrams by non-technical business analysts. It defines the shapes, types and meaning of the boxes and arrows. This type of modeling should be seen in the context of 'BPM as a discipline' (see below). By definition, non technical business analyst do not think in terms of web services or other technology aspects. BPMN processes are primarily intended for communication between humans. So far, so good. But now comes the hard and confusing reality: BPEL has been presented as a solution for 'BPM as a discipline' and BPMN 2.0 plans to add concrete executable semantics. These days, most IT people already realized that BPEL is not a convenient solution for 'BPM as a discipline'. On the other hand adding concrete executable semantics to a business analyst modeling notation means that non-technical analysts will be implicitly writing a piece of executable software. Would you put software into production written by a non-technical person ? Use Case 1 : BPM as a discipline BPM as a discipline refers to the analysis, documentation and improvement of the procedures that describe how people and systems work together in an organization. Realize that most BPM is done in this way, not even resulting into any form of software or IT support. MS Visio is one the most used tools to document business processes. Of course, once the business process is documented, it might make sense to develop software support for it. Suppose that a business analyst modeled and documented a 'business trip' process, then one usage might be to find optimizations in that process. And another usage might be the development of software support for business trips. Please be aware that an automatic translation between pure analysis models and executable process models are not feasible in general. Initially BPM products tried to make this translation automagically transparent. In the context of BPM as a discipline, we believe that a process model from a non technical business analyst can never be translated into an executable process model by just adding technical details to it. The next attempt was to acknowledge existence of an analysis model and an executable process model and then try to keep them in sync. Especially because of the big differences between BPMN and BPEL this approach got a lot of attention. But then the problem of maintaining the links between analysis and executable process model appears. Who is going to spend the time to link the analysis blocks to the executable blocks and then keep that mapping up to date. This is illustrated in the following picture: [img_assist|nid=5584|title=|desc=|link=none|align=undefined|width=539|height=295] For mainstream software development resulting from 'BPM as a discipline', we believe that the original analysis diagram should serve as input with the rest of the software requirements. Developers should then construct an executable process that looks as close as possible to the original analysis diagram. After that, the analysis diagram is discarded and replaced by the executable process diagram. That way, the developer is in full control of the software technical aspects, while the analyst will still recognize the diagram. jBPM's jPDL language is designed to facilitate this approach. First of all, it's based on free graph modeling. Secondly, it has so called actions, which can be seen as event listeners. These event listeners are pieces of Java code that are called when process execution produces the event, but they are hidden from the process diagram. This allows the developer to add programming logic without changing the diagram structure. Also super-states are often used in the context of creating better communication between business analyst and developer. Use Case 2 : Combining template based and ad hoc task management Orchestrating human tasks can be found in most process engines in one form or another. But what is often overlooked is that template based task orchestration only suits for a limited number of scenarios. First the process must be relatively stable. And secondly, enough executions of this process have to happen so that the gain that can be achieved with software support is worth the development effort. A lot of work being done in organizations does not meet those two requirements. For example, organizing a one-time team building event, a restyling of the office space or cleaning up after the sprinkler system went off unexpectedly. For that type of work, people in charge will invent the process on the spot and it will only be executed once. Human Interaction Management (HIM) focuses on this type of work. A story in HIM reflects a task that person creates for which an ad-hoc process will be created. People can get involved in different roles. There are big benefits of tracing such work with a task management system. First, people that get involved with such a task after a while will get instant visibility into the history of the whole story. And secondly, an audit trail is logged automatically. In jBPM 4, the task management component will support this ad hoc human tasks. The combination will be awsome. Human tasks in processes can be specified at a course grained level. When such a process task is created for a person, the assignee can involve other people by creating subtasks and assigning people in different roles to these tasks. Only when the owner decides that the overall task is finished, then the process will continue. Use case 3 : Transactional asynchronous architectures Gregor Hohpe's Entperprise Integration Patterns describes building blocks for asynchronous architectures. These patterns are based on the notion of Message Oriented Middleware (MOM) aka message brokers. MOM's are good at asynchronous transactional communication between two systems and JMS is the Java standard to work with it. Of course point-to-point communication can be set up easily with existing Java infrastructure like Message Driven Beans. But in many cases many of these point-to-point communications are related. For example, to process one order, a message might come from a client into the order processing system. As part of that transaction, a notification might be sent to the warehouse and stock planning team. Eventually after many more steps, a message might be sent back to the client as a confirmation of the order. If all these transactional communications are build linearly, it will be very hard to manage the overall state of order processing. BPEL also focuses on asynchronous architectures, but then in a (web) services environment, rather then a Java environment. In BPEL only (web) services can be invoked and XML based technologies like XPath are integrated. So for more complex calculations, a piece of programming logic needs to be included. To include programming logic in BPEL, it needs t be wrapped and exposed as a service. That can be quite cumbersome in certain situations. In contrast, jPDL is embedded in a Java environment. Such a central dispatching functionality is really suited to be implemented by a process language like jPDL. An incoming message can start a new jPDL process execution or provide a trigger for an existing execution. jPDL allows to include programming logic straight into the process transaction. Cause jPDL is based on a Java architecture, that results into much more flexibility and convenience. Using the central dispatching approach creates immediately more insight into the overall state of a process. In terms of our order processing example, it will be very easy to keep track of the state of each order. Further more, process engines like jPDL collect the history information. This is an extra feature that you get for free when using a process engine in this way. From the history information it is very easy to collect valuable statistical information like the average time in each step of the process. The difference with the use case 'BPM as a discipline' is that in this case, the goal is software technical in nature. The process is looked at from an implementation perspective. There doesn't have to be a relation to a business level analysis process. Use case 4 : Service orchestration Service orchestration is actually a variant of the previous use case 'transactional asynchronous architectures'. Interactions between services on an Enterprise Service Bus (ESB) are usually asynchronous. This time, the communication is not done through sending messages over a MOM, but instead by calling services over an ESB. BPEL is a process language specifically designed for this use case. Apart from a construct to invoke WSDL services, it has a whole infrastructure for supporting conversations. Conversations are so called long running processes. The clue here is that a BPEL process specifies a number of service operations that are being published when a BPEL process is deployed. A BPEL process can then contain receive activities, which mean that the process execution will wait until a service operation invocation is received. In jBPM we have implemented the BPEL process language as on of the languages that we support on top of our Process Virtual Machine. Use Case 5 : Visual programming With visual programming, we will target developers that do not yet have the full skillset to develop in Java. They can graphically specify the activities and draw transitions to indicate the control flow. Instead of typing Java code, just put blocks like 'perform hql query', 'generate pdf', 'send email', 'read file', 'parse xml', 'send msg to esb'. Instead of writing Java, they will be composing software programs by linking activity boxes in the diagram and filling in the configuration details. This will, of course, not have the same kind of flexibility as Java programming itself. Visual programming as we describe it here will not replace programming as we know it today. Today's programmers can be considered the power users and that kind will always be needed. But visual programming can lower the treshold to build applications for developers that have no or limited Java knowledge. The Process Virtual Machine (PVM) is the foundation of jBPM. In the PVM infrastructure it is really easy to add a new activity type. Think of it as a kind of command pattern, where the commands are configurable. So it will be very easy for us to develop new activity types. This way, we can expose the bulk part of the Java programming functionalities as activity types in the jPDL process language. In jPDL, these visual programs will be executable transactionally or without persisting the state of the execution. Use Case 6 : Thread Control Language This is a specific aspect of visual programming that also can be used by experienced developers. Coding multi-threaded Java programs is not easy. In fact, it is at least tedious and mostly pretty hard to code. Starting new threads, passing data into them, joining and making them stop properly can be a challenge. We'll develop a Thread Control Language which lets you specify a multithreaded Java concurrency by drawing forks, joins and method activities. A method activity invokes a methods on your Java object. And the concurrency control is handled by the forks and join in the process. These processes will be executed without any persistence. Use Case 7 : Easy creation of DSLs General purpose process languages are different from domain specific process languages. For example, think of a process language to specify approvals in an Enterprise Content Management (ECM) system. This gives a scoped functionality and a fixed environment. In such cases, a specific process language could be developed for this purpose. One of the advantages of such dedicated languages is that they can be made simple enough so that non technical business users can actually create fully executable processes. Another example of a domain specific process language is SEAM's Pageflow. It allows developers of a JSF based web site to specify the pages and navigations between the pages graphically. There is even an easier way. Instead of creating a full process language for a specific purpose, it is also possible to leverage jPDL's capabilities and just add new node types to it. That is already possible in jBPM 3 and will be peanuts in jBPM 4, even adding graphical support in the process designer for these new node types will be made simple. Conclusion The typical understanding of BPM is that non technical business people create diagrams that then automagically get executed on a BPM system. The first use case 'BPM as a discipline' describes that point of view. The value lies in the fact that non technical business people can communicate with the developers around a diagram. That diagram facilitates the communication between business analysts and developers. But even in that use case, the traditional understanding needs fine tuning. An analysis diagram at some point has to be converted into an executable process. At that moment, the responsibility over the process is transferred from the analyst to the developer. Many vendors have created the tools and illusion that this can be done automagically. But in practice, this black box approach creates more problems then it solves. The description of the 7 individual use cases shows distinct aspects of BPM. Knowing the difference between service orchestration and 'BPM as a discipline' will be key for organizations to select the most appropriate technology. All of the use cases have at least a technical side. That is understandable as all the use cases here target some form of software automation at some point. And other use cases target solely technical aspects. That is why the embeddability of jBPM is so important. Monolithic BPM engines have a high treshold to be incorporated into a typical software development project. jBPM has a big focus on delivering solutions for these distinct use cases to developers in a way that is easy for them to consume and integrate into their own software development project. Tom Baeyens is the founder and lead of JBoss jBPM, the leading open source BPM system. Tom mission is to bring the power of BPM technology into the hands of the developers. He's a frequent speaker at international conferences and maintains a blog at http://processdevelopments.blogspot.com
October 21, 2008
by Tom Baeyens
· 57,137 Views · 1 Like
article thumbnail
A Look Inside the JBoss Microcontainer, Part I -- Component Models
Looking at the current state of Java, we can see that POJOs (Plain Old Java Objects) rule the land yet again. Their dominance stretches from enterprise applications to middleware services. At JBoss we were known for our modular JMX-based kernel. The application server was nothing more than a bunch of flexible MBeans and a powerful MicroKernel in the middle. But, as you could feel the change is coming, we still wanted to be ahead of the pack. We could see different POJO based component models popping up all over the place (EJB3, JPA, Spring, Guice, … to name a few), yet nothing was out there that would bind them all, flattening out the differences on a single component model. Hence the Microcontainer project was born. The JBoss Microcontainer project is about many things. The Microcontainer's features range from reflection abstraction, a virtual file system, a simple state machine to transparent AOP integration, a new classloading layer, a deployment framework and an OSGi framework implementation. I'll try to address them all over a short series of articles that will be published here on DZone. This article will examine the Microcontainer's component models. Read the other parts in DZone's exclusive JBoss Microcontainer Series: Part 1 -- Component Models Part 2 –- Advanced Dependency Injection and IoC Part 3 -- the Virtual File System Part 4 -- ClassLoading Layer Part 5 - the Virtual Deployment Framework Part 6 - The Scanning Library What is a component model? What do we consider a component model? First of all, what do we consider being a component? One abstract way to express this would be that “components are reusable software programs that you can develop and assemble easily to create sophisticated applications.” To consider a bunch of components as an actual model, we also need to declare what kind of interactions we allow. JMX MBeans are one example of a component model. Their interactions include executing MBean operations, referencing attributes, setting attributes and declaring explicit dependencies between named MBeans. As mentioned, we had advanced JMX handling already with the MicroKernel. And as expected, the Microcontainer brought extensive POJO support. The default behavior and interactions in the Microcontainer are what you also normally get from any other IoC container and are similar to the functionality that MBeans provided: plain method invocations for operations, setters/getters for attributes and explicit dependencies. However, having only this functionality would mean we didn't get much further than just relieving the pain of declaring MBeans, hence it was only logical for us to do something more. I'll leave the discussion of these new IoC features for the next article in the series. There are many existing POJO component models out there, Guice and Spring being amongst the most popular. Effective integration with these frameworks was an important goal for us. Demo environment setup Before we begin, I'd like to first describe the various parts that constitute the demo. All the source code can be found at this location of our Subversion repository: • http://anonsvn.jboss.org/repos/jbossas/projects/demos/microcontainer/branches/DZone_1_0/ The project is fully mavenized, so it should be easy to adjust it to your IDE. I will first go over the sub-projects within the demo and describe their usage. At the end of my article series, I will provide a more detailed look at what certain sub-projects do. Below are the JBoss Microcontainer demos and sub-projects that are relevant for this article: • bootstrap (as the name suggests, it bootstraps the Microcontainer with demo code) • jmx (adds the JMX notion to demo's bootstrap) • models (source code of our components / services) The demo has only one variable you need to set - demos home - but even this one can be optional if you checked-out your project into the \projects\demos directory. Otherwise, you need to set the system property demos.home (e.g. -Ddemos.home=). You should now be able to run JMXMain as a main class. Make sure you include the models sub-project in the classpath, since some of the services require additional classes in the classpath, several more then what the jmx sub-project expects. Once the Microcontainer is booted it starts to scan the ${demos.home}/sandbox directory for any changes. Now all we need to do is provide a deployable unit and drop it in there. Models Let’s now turn our attention to the models sub-project. This is where the previously mentioned deployable unit should come from. You can see if everything is in place by building the models sub-project (mvn package) and dropping it into the sandbox. You should get a bunch of debug and info messages on the console log, showing how the Microcontainer got booted. Any error message indicates some problems. Let’s go over exactly what the models sub-project does, its integration code and try to redeploy it. If we look at the models src/main/resources/META-INF directory, we'll see plenty of -beans.xml resource files and one -service.xml. You’ll notice that each has a meaningful name that matches the source code package from models's src/main/java/org/jboss/demos/models directory. Let's dissect them one by one, starting with the ones that have no dependencies. org.jboss.demos.models.plain.PojoFactory This is a simple Micrcocontainer beans descriptor file. Anyone who crossed paths with some other IoC’s configuration file should be familiar with it. And, as I already mentioned, I'll follow up on more advanced usage in the next article. The next file shows what we have done to support Spring integration: Note that this file's namespace is different from the previous Microcontainer bean’s plain-beans.xml file. The urn:jboss:spring-beans:2.0 namespace points to our version of the Spring schema port, meaning you can describe your beans Spring style, but it's the Microcontainer that's going deploy then, not Spring's bean factory notion. public class Pojo extends AbstractPojo implements BeanNameAware { private String beanName; public void setBeanName(String name) { beanName = name; } public String getBeanName() { return beanName; } public void start() { if ("SpringPojo".equals(getBeanName()) == false) throw new IllegalArgumentException("Name doesn't match: " + getBeanName()); } } Although the SpringPojo bean has a dependency on Spring lib via implementing BeanNameAware interface, this dependency is only there to expose and mock some of the Spring's callback behavior (see SpringBeanAnnotationPlugin for more details). Since we introduced Spring integration, let's have a look at Guice integration. As Guice users know, Guice is all about type matching. Configuration of Guice beans is done via Modules which means that in order to generate beans, one must implement a Module. Two important parts to watch from this file are PojoModule and GuiceKernelRegistryEntryPlugin. The first one is where we configure our beans: public class PojoModule extends AbstractModule { private Controller controller; @Constructor public PojoModule(@Inject(bean = KernelConstants.KERNEL_CONTROLLER_NAME) Controller controller) { this.controller = controller; } protected void configure() { bind(Controller.class).toInstance(controller); bind(IPojo.class).to(Pojo.class).in(Scopes.SINGLETON); bind(IPojo.class).annotatedWith(FromMC.class).toProvider(GuiceIntegration.fromMicrocontainer(IPojo.class, "PlainPojo")); } } The second class provides integration with the Microcontainer: public class GuiceKernelRegistryEntryPlugin implements KernelRegistryPlugin { private Injector injector; public GuiceKernelRegistryEntryPlugin(Module... modules) { injector = Guice.createInjector(modules); } public void destroy() { injector = null; } public KernelRegistryEntry getEntry(Object name) { KernelRegistryEntry entry = null; try { if (name instanceof Class) { Class clazz = (Class)name; entry = new AbstractKernelRegistryEntry(name, injector.getInstance(clazz)); } else if (name instanceof Key) { Key key = (Key)name; entry = new AbstractKernelRegistryEntry(name, injector.getInstance(key)); } } catch (Exception ignored) { } return entry; } } Notice how we created an Injector from the Modules and then did a lookup on it for matching beans. In mbeans-service.xml we declare our legacy usage of MBeans: What’s interesting to note here is how we injected a POJO into an MBean. The preceding demonstrated our first interactions between different component models. In order to allow for MBean deployment via the Microcontainer, we had to write entirely new component model handling code. See the system-jmx-beans.xml for more details. The code from this file lives in the JBossAS source code: system-jmx sub-project. One note here, this is currently only possible with JBoss's JMX implementation, since the system-jmx code uses some implementation details. Now what if we wanted to expose our existing POJOs as MBeans, registering them into an Mbean server? @org.jboss.aop.microcontainer.aspects.jmx.JMX(exposedInterface=org.jboss.demos.models.mbeans.PojoMBean.class, registerDirectly=true) As you can see from looking at any of the beans in this file, in order to expose your POJOs as MBeans, it’s simply a matter of annotating them with an @JMX annotation. You can either expose the bean directly or its property: Here we see how you can use any of the injection lookup types by either looking up a plain POJO or getting a handle to an MBean from the MBean server. One of the injection options is to use type injection, which is also sometimes called autowiring: The FromGuice bean injects the Guice bean via type matching, where PlainPojo is injected with a common name injection. We then test if Guice binding works as expected: public class FromGuice { private IPojo plainPojo; private org.jboss.demos.models.guice.Pojo guicePojo; public FromGuice(IPojo plainPojo) { this.plainPojo = plainPojo; } public void setGuicePojo(org.jboss.demos.models.guice.Pojo guicePojo) { this.guicePojo = guicePojo; } public void start() { f (plainPojo != guicePojo.getMcPojo()) throw new IllegalArgumentException("Pojos are not the same: " + plainPojo + "!=" + guicePojo.getMcPojo()); } } This only leaves us with an alias component model. Even though the alias is quite a trivial feature, in order to implement it as a true dependency, it has to be introduced as a new component model inside the Microcontainer. The implementation details for this is part of the AbstractController source code: springPojo Here we’ve mapped the SpringPojo name to the springPojo alias. The beauty of having aliases as true component models is that it doesn't matter when the real bean gets deployed. This means that the alias will wait in a non-installed state until the real bean triggers it. Conclusion We've seen how we can deploy simple Microcontainer beans, legacy MBeans, Guice POJOs, Spring beans and aliases. And since all of this is controlled by the Microcontainer, we saw how easily we can mix and match the different component models. I can easily say, with the level of abstraction we put in our component model design, the sky is the limit on what we can handle. An example of this is the upcoming OSGi services, but that's another story, another article. Stayed tuned for my next article which will provide a detailed look at the Microcontainer's IoC functionality. About the Author Ales Justin was born in Ljubljana, Slovenia and graduated with a degree in mathematics from the University of Ljubljana. He fell in love with Java seven years ago and has spent most of his time developing information systems, ranging from customer service to energy management. He joined JBoss in 2006 to work full time on the Microcontainer project, currently serving as its lead. He also contributes to JBoss AS and is Seam and Spring integration specialist. He represent JBoss on 'JSR-291 Dynamic Component Support for Java SE' and 'OSGi' expert groups.
October 1, 2008
by Ales Justin
· 57,216 Views
article thumbnail
An Introduction to Aspect-Oriented programming with JBoss AOP
JBoss Application Server ships with support for aspect-oriented programming, so you can use AOP in your applications deployed in JBoss AS. JBoss AS 5, which is currently available as a community release, has AOP built into its core. However, JBoss AOP is also available as a standalone framework for use in your other applications. This article will take a simple example, and use JBoss AOP to add to its behaviour, while explaining some of the core functionality of JBoss AOP. We will look at encapsulating cross-cutting concerns into aspects, have a look at around advices vs the new lightweight advices in the upcoming JBoss AOP 2.0 release, and also look at using interface-introductions and mixins to add interfaces to your classes. Table of Contents Our Core Application Our Core Application The application we will look at is is a simple banking application. It has a BankAccount class: package bank; public class BankAccount { int accountNumber; int balance; public BankAccount(int accountNumber) { System.out.println("*** Bank Account constructor"); this.accountNumber = accountNumber; } public int getAccountNumber() { return accountNumber; } public int getBalance() { return balance; } public void debit(int amount) { System.out.println("*** BankAccount.debit()"); balance -= amount; } public void credit(int amount) { System.out.println("*** BankAccount.credit()"); balance += amount; } } In addition it has a main Bank class: package bank; import java.util.HashMap; import java.util.Map; public class Bank { static Map bankAccounts = new HashMap(); public static void transfer(BankAccount from, BankAccount to, int amount) { from.debit(amount); to.credit(amount); } public static void main(String[] args) { System.out.println("*** Creating account 1"); BankAccount acc1 = new BankAccount(1); acc1.credit(150); bankAccounts.put(acc1.getAccountNumber(), acc1); System.out.println("*** Creating account 2"); BankAccount acc2 = new BankAccount(2); acc2.credit(230); bankAccounts.put(acc2.getAccountNumber(), acc2); System.out.println("*** Balance acount 1: " + acc1.getBalance()); System.out.println("*** Balance acount 2: " + acc2.getBalance()); //Transfer some money System.out.println("*** Transfer 50 from account 1 to account 2"); transfer(acc1, acc2, 50); System.out.println("*** Balance acount 1: " + acc1.getBalance()); System.out.println("*** Balance acount 2: " + acc2.getBalance()); } } As you can see, we create two bank accounts with their account numbers, set the initial balances, and then transfer 50 from account1 to account 2. Running this simple example we get the following expected output: *** Creating account 1 *** Bank Account constructor *** BankAccount.credit() *** Creating account 2 *** Bank Account constructor *** BankAccount.credit() *** Balance acount 1: 150 *** Balance acount 2: 230 *** Transfer 50 from account 1 to account 2 *** BankAccount.debit() *** BankAccount.credit() *** Balance acount 1: 100 *** Balance acount 2: 280 The code for this example can be found in the listing1/ folder of . Logging as a Cross-Cutting Concern One of the main uses of AOP is to extract out cross-cutting concerns. Say we wanted to add some logging whenever an object is created, when its fields are set, and when its methods are called. The “obvious” way to do that in our example would be to go in and add logging statements to various points of our application. The problem with this approach is that there might be lots of places to edit, scattered around our system, so we would need to identify those places, make our changes, and recompile our application. To turn this behaviour off, we would need to remove our logging statements again and recompile our code again. So the “obvious” way both leads to bloat, and is difficult to turn on and off. Using AOP we can encapsulate this cross-cutting code in an aspect. The aspect itself is just a normal class, which can hold state, extend other classes, everything you can do in a normal Java class. package bank; import org.jboss.aop.joinpoint.ConstructorInvocation; import org.jboss.aop.joinpoint.FieldWriteInvocation; import org.jboss.aop.joinpoint.MethodInvocation; public class LoggingAspect { public Object log(ConstructorInvocation invocation) throws Throwable { try { System.out.println("C: Creating BankAccount using constructor " + invocation.getConstructor()); System.out.println("C: Account number: " + invocation.getArguments()[0]); return invocation.invokeNext(); } finally { System.out.println("C: Done"); } } public Object log(MethodInvocation invocation) throws Throwable { try { System.out.println("M: Calling method " + invocation.getMethod().getName()); System.out.println("M: Amount " + invocation.getArguments()[0]); return invocation.invokeNext(); } finally { System.out.println("M: Done"); } } public Object log(FieldWriteInvocation invocation) throws Throwable { BankAccount account = (BankAccount)invocation.getTargetObject(); System.out.println("F: setting field " + invocation.getField().getName() + " for BankAccount " + account.getAccountNumber()); System.out.println("F: Field old value " + account.getBalance()); System.out.println("F: New value will be " + invocation.getValue()); try { return invocation.invokeNext(); } finally { System.out.println("F: Field new value " + account.getBalance()); System.out.println("F: Done"); } } } The actual code implementing the cross-cutting concerns is encapsulated in methods called advice methods. For the type of advice methods that we are looking at now, around advice, the signature and format of the advice method is as shown here: public Object (org.jboss.aop.joinpoint.Invocation invocation) throws Throwable { //Do something before try { return invocation.invokeNext(); } finally { //Do something after } } The invocation parameter can be of type org.jboss.aop.joinpoint.Invocation, or one of its subclasses. Some of these are shown in the LoggingAspect, and which of the overloaded log methods gets called depends on what is being called once we apply our aspect. In the LoggingAspect we are able to handle calls to an object's constructor (the first log() method, starting on line 9), calls to an object's methods (the second log() method, starting on line 23) and calls to set a field (the third log() method, starting on line 37). These “events” are called joinpoints in AOP terminology. The invocation contains information about what field/method/constructor is being called. It also allows access to any arguments, and the object on which we are making the call. The around advice methods also contain a call to Invocation.invokeNext(), which propogates the call chain. If there are several advice methods applied to the target joinpoint, this call invokes the next advice in the chain. If this is the last advice in the chain, or as in our example, we are the only advice in the chain, the target joinpoint is called when we call Invocation.invokeNext(). To apply our aspects, we need some configuration to declare our aspects, and to select the joinpoints in our application where the advice methods should be applied. In JBoss AOP the easiest way to do this is with an xml configuration file, typically called jboss-aop.xml. First we declare our aspect: A binding has a pointcut to choose which methods we should apply the advice to. In this case we have a constructor expression that selects the constructor of the bank.BankAccount class that has an int parameter. Note that all class names used in pointcut expressions must be fully qualified. The word new is a special identifier within the pointcut language to pick out a constructor. Next, the around part of the binding says that we should apply the log advice from the bank.LoggingAspect to the joinpoints matched by its pointcut. In other words, when we call BankAccount's constructor, the first LoggingAspect.log() method, which takes a ConstructorInvocation, is invoked. The previous binding's pointcut only captures one joinpoint. The next one uses a wildcard in place of the method name, so we pick out all methods returning void that take an int parameter, regardless of their name. It looks a lot like the constructor expression in the previous binding, but also contains the return type of the methods we are interested in. This pointcut picks out BankAccount.debit() and BankAccount.credit(), and invokes the second LoggingAspect.log() method, which takes a MethodInvocation, when these methods are called. Finally, we have a binding capturing writes to the balance field of the BankAccount class. In this case we are using a wildcard in place of the type's name, and the third LoggingAspect.log method, which takes a FieldWriteInvocation, will be invoked when the field's value is set: In addition to wildcards, you can also capture whole inheritance hierarchies of classes, use typedefs for complex class expressions, all classes belonging to a package, and as we will see use annotations to capture a wide range of jonpoints. To run this application with aop enabled, you need to pass in a few extra parameters into the jvm when starting it up, as we can see in the example's build.xml The jboss.aop.path parameter contains the path to the jboss-aop.xml that declares our aspects and binds advice methods to joinpoints. The -javaagent switch points to the JBoss AOP library, which in turn turns on load-time weaving. When a class is first loaded, JBoss AOP will intercept that event and modify the bytecode of the class to add the hooks required to trigger the aspects for our selected joinpoints. In addition to weaving at load-time, you can weave the classes using our aopc post-processor. An example of this is shown in the example's build.xml. Running the example with load-time weaving yields the following output, now with quite a lot of extra information coming from our logging aspect: *** Creating account 1 C: Creating BankAccount using constructor public bank.BankAccount(int) C: Account number: 1 *** Bank Account constructor C: Done M: Calling method credit M: Amount 150 *** BankAccount.credit() F: setting field balance for BankAccount 1 F: Field old value 0 F: New value will be 150 F: Field new value 150 F: Done M: Done *** Creating account 2 C: Creating BankAccount using constructor public bank.BankAccount(int) C: Account number: 2 *** Bank Account constructor C: Done M: Calling method credit M: Amount 230 *** BankAccount.credit() F: setting field balance for BankAccount 2 F: Field old value 0 F: New value will be 230 F: Field new value 230 F: Done M: Done *** Balance acount 1: 150 *** Balance acount 2: 230 *** Transfer 50 from account 1 to account 2 M: Calling method debit M: Amount 50 *** BankAccount.debit() F: setting field balance for BankAccount 1 F: Field old value 150 F: New value will be 100 F: Field new value 100 F: Done M: Done M: Calling method credit M: Amount 50 *** BankAccount.credit() F: setting field balance for BankAccount 2 F: Field old value 230 F: New value will be 280 F: Field new value 280 F: Done M: Done *** Balance acount 1: 100 *** Balance acount 2: 280 The code for this example can be found in the listing2/ folder of . Externalising Security Checks Logging is the common example used for introductions to AOP, so let's try doing something more interesting. Say we want to make sure that only users with the correct permissions can call a method. We could annotate our methods from BankAccount as follows: package bank; public class BankAccount { int accountNumber; int balance; @Roles(roles= {"admin"}) public BankAccount(int accountNumber) { System.out.println("*** Bank Account constructor"); this.accountNumber = accountNumber; } ... @Roles(roles= {"admin"}) public void debit(int amount) { System.out.println("*** BankAccount.debit()"); balance -= amount; } @Roles(roles= {"admin", "user"}) public void credit(int amount) { System.out.println("*** BankAccount.credit()"); balance += amount; } } So only users with the role “admin” can create BankAccount instances and debit accounts, while users with the role “admin” or “user” can credit accounts. We have created a security.properties file to configure users and their roles: admin=password;admin,user guest=password;user There is a user called 'admin' whose password is 'password' who has the roles 'admin' and 'user', and a user called 'guest' whose password is 'password' who only has the role 'user'. We can then apply a SecurityAspect to the methods annotated with the @Roles annotation. Note that like everything else in the pointcut language the annotations need to be fully qualified. The “..” in place of the parameters in the pointcut expressions means we want this aspect to be applied to all constructors and methods annotated with @Roles regardless of the parameters it takes. The SecurityAspect then checks that the correct user is used: package bank; import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.IOException; import java.net.URISyntaxException; import java.net.URL; import java.util.Arrays; import java.util.Enumeration; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.Properties; import org.jboss.aop.joinpoint.Invocation; public class SecurityAspect { Map usernamePassword = new HashMap(); Map> userRoles = new HashMap>(); public SecurityAspect() throws FileNotFoundException, IOException, URISyntaxException { //Initialise the usernamePassword and userRoles maps //with information from security.properties } public Object checkSecurity(Invocation invocation) throws Throwable { String username = LoginInfo.getUsername(); String password = usernamePassword.get(username); if (password == null) { throw new SecurityException("Unknown user"); } if (!password.equals(LoginInfo.getPassword())) { throw new SecurityException("Wrong password"); } Roles rolesAnnotaton = (Roles)invocation.resolveAnnotation(Roles.class); List hasRoles = userRoles.get(username); boolean hasRole = false; if (hasRoles != null) { for (String role : rolesAnnotaton.roles()) { if (hasRoles.contains(role)) { hasRole = true; break; } } } if (!hasRole) { throw new SecurityException("Wrong roles for user"); } return invocation.invokeNext(); } } The roles needed to invoke the target joinpoint are got from the invocation using this call: Roles rolesAnnotaton = (Roles)invocation.resolveAnnotation(Roles.class); This does the same as calling java.lang.reflect.Method.getAnnotation() or java.lang.reflect.Constructor.getAnnotation() for the called method or constructor, but also allows for annotation overrides as part of the aop configuration, which are beyond the scope of this article. Although the type of invocation used in this example is Invocation, the specific type created by JBoss AOP will be ConstructorInvocation or MethodInvocation depending on what we are calling. The referenced LoginInfo class is just a wrapper around some static fields containing the username and password. package bank; public class LoginInfo { private static String username; private static String password; public static void setUsernameAndPassword(String username, String password) { LoginInfo.username = username; LoginInfo.password = password; } public static String getUsername() { return username; } public static String getPassword() { return password; } } Now let us modify the Bank.main() method to populate the LoginInfo fields: public static void main(String[] args) { System.out.println("*** Log in as 'guest' - it does not have the correct roles to create an account"); LoginInfo.setUsernameAndPassword("guest", "password"); System.out.println("*** Attempting to create account 1"); try { BankAccount acc1 = new BankAccount(1); acc1.credit(150); bankAccounts.put(acc1.getAccountNumber(), acc1); } catch(SecurityException e) { System.out.println("!!! Expected SecurityException " + e.getMessage()); } System.out.println("*** Log in as 'admin' - the roles are fine for the rest now"); LoginInfo.setUsernameAndPassword("admin", "password"); System.out.println("*** Creating account 1"); BankAccount acc1 = new BankAccount(1); acc1.credit(150); bankAccounts.put(acc1.getAccountNumber(), acc1); System.out.println("*** Creating account 2"); BankAccount acc2 = new BankAccount(2); acc2.credit(230); bankAccounts.put(acc2.getAccountNumber(), acc2); System.out.println("*** Balance acount 1: " + acc1.getBalance()); System.out.println("*** Balance acount 2: " + acc2.getBalance()); //Transfer some money System.out.println("*** Transfer 50 from account 1 to account 2"); transfer(acc1, acc2, 50); System.out.println("*** Balance acount 1: " + acc1.getBalance()); System.out.println("*** Balance acount 2: " + acc2.getBalance()); } We first try to create a BankAccount with a user who does not have the required admin role, and then we do the rest, as before, with an user with the required roles. The output of running this application is: *** Log in as 'guest' - it does not have the correct roles to create an account *** Attempting to create account 1 !!! Expected SecurityException Wrong roles for user *** Log in as 'admin' - the roles are fine for the rest now *** Creating account 1 *** Bank Account constructor *** BankAccount.credit() *** Creating account 2 *** Bank Account constructor *** BankAccount.credit() *** Balance acount 1: 150 *** Balance acount 2: 230 *** Transfer 50 from account 1 to account 2 *** BankAccount.debit() *** BankAccount.credit() *** Balance acount 1: 100 *** Balance acount 2: 280 By using AOP to apply security, we have extracted the security checks into one place in our application, and used annotations to configure that. If we wanted to use a different mechanism of configuring the users we could leave the core application the same, write another aspect and easily change how we use security everywhere by modifying the jboss-aop.xml file. The code for this example can be found in the listing3/ folder of . Observer with Introductions and Mixins Say we want to be able to be notified somehow when BankAccount.balance is changed. An option would be to implement the Observer pattern, but we don't want to include all the Observable plumbing code in our BankAccount class. First of all let's add an annotation to the field we want to monitor. package bank; public class BankAccount { int accountNumber; @Observed int balance; ... } Next we can write an implementation of Observable: package bank; import java.util.ArrayList; import java.util.List; public class ObservableMixin implements Observable { List observers = new ArrayList(); public void addObserver(Observer listener) { observers.add(listener); } public void removeObserver(Observer listener) { observers.remove(listener); } public void notifyObservers(Object event) { for (Observer observer : observers) { observer.update(event); } } } This is a mixin class, which implements the Observable interface. It contains all the plumbing code needed for the Observable part of the pattern. It can be introduced into other POJOs using the following xml bank.Observable bank.ObservableMixin ... This does two things. It makes all classes that have a field annotated with @Observed implement the @Observable interface and implement those methods. Second, when anybody attempts to call the methods from the Observable interface, it delegates all the calls to an instance of the ObservableMixin. Next we have an aspect to trap when the annotated fields change their values bound using the following xml. ... The aspect is an around advice that captures the values before and after modifying the field. package bank; import java.lang.reflect.Field; import org.jboss.aop.joinpoint.FieldWriteInvocation; public class ObserverAspect { public Object fieldChanged(FieldWriteInvocation invocation) throws Throwable { Observable tgt = (Observable)invocation.getTargetObject(); Field fld = invocation.getField(); String fieldName = fld.getName(); fld.setAccessible(true); Object oldVal = invocation.getField().get(tgt); Object result = invocation.invokeNext(); Object newVal = invocation.getField().get(tgt); tgt.notifyObservers("Changed " + fieldName + " from " + oldVal + " to " + newVal); return result; } } Since the classes captured by the pointcut to select which classes should have the ObserverAspect applied are in the set of classes captured by the class expression to pick out classes that should have the Observable introduction and mixin, we can safely cast the target of the invocation to Observable. We then get the values before and after writing the field, and then use the Observable target object to notify the observers. As mentioned the call to Observable.notifyObservers() will end up inside BankAccount's ObservableMixin. Finally, let us modify the Bank.main() method to register an Observer with a BankAccount instance public static void main(String[] args) { System.out.println("*** Creating account 1"); BankAccount acc1 = new BankAccount(1); acc1.credit(150); bankAccounts.put(acc1.getAccountNumber(), acc1); System.out.println("*** Creating account 2"); BankAccount acc2 = new BankAccount(2); acc2.credit(230); bankAccounts.put(acc2.getAccountNumber(), acc2); System.out.println("Installing observer"); ((Observable)acc2).addObserver(new Observer(){ public void update(Object evt) { System.out.println("!!! Observer: " + evt); } }); System.out.println("*** Balance acount 1: " + acc1.getBalance()); System.out.println("*** Balance acount 2: " + acc2.getBalance()); //Transfer some money System.out.println("*** Transfer 50 from account 1 to account 2"); transfer(acc1, acc2, 50); System.out.println("*** Balance acount 1: " + acc1.getBalance()); System.out.println("*** Balance acount 2: " + acc2.getBalance()); } Although until woven the BankAccount class does not implement the Observable interface, the Java compiler does not perform any checks when casting to an interface (only to a superclass), so the cast from BankAccount to Observable will compile. (If the example is run without weaving you would get a ClassCastException since then BankAccount would not implement the Observable interface). When we run this example we can see the registered Observer gets triggered: *** Creating account 1 *** Bank Account constructor *** BankAccount.credit() *** Creating account 2 *** Bank Account constructor *** BankAccount.credit() Installing observer *** Balance acount 1: 150 *** Balance acount 2: 230 *** Transfer 50 from account 1 to account 2 *** BankAccount.debit() *** BankAccount.credit() !!! Observer: Changed balance from 230 to 280 *** Balance acount 1: 100 *** Balance acount 2: 280 The code for this example can be found in the listing5/ folder of . Conclusion We have taken a simple application, added extra behaviour to it using JBoss AOP, and explored a few of the features offered. Apart from adding a few annotations to help with our pointcut expressions (and there are even less intrusive, although more incovenient ways to achieve the similar effect), we have left the core application as is, and added extra cross-cutting behaviour such as logging and security by applying aspects, as well as using a combination of aspects, interface introductions and mixins to implement the Observer pattern. JBoss AOP can be downloaded from http://www.jboss.org/jbossaop/ and comes with a tutorial to get you started.
August 28, 2008
by Kabir Khan
· 84,955 Views · 2 Likes
article thumbnail
Compute Grids vs. Data Grids
in a nutshell, grid computing is a way to distribute your computations across multiple computers (nodes). however, even jms does that, but jms is not a grid computing product - it's a messaging protocol. to correctly classify grid computing products we have to split them into 2 categories: compute grids and data grids. compute grid compute grids allow you to take a computation, optionally split it into multiple parts, and execute them on different grid nodes in parallel. the obvious benefit here is that your computation will perform faster as it now can use resources from all grid nodes in parallel. one of the most common design patterns for parallel execution is mapreduce . however, compute grids are useful even if you don't need to split your computation - they help you improve overall scalability and fault-tolerance of your system by offloading your computations onto most available nodes. some of the "must have" compute grid features are: automatic deployment - allows for automatic deployment of classes and resources onto grid without any extra steps from user. this feature alone provides one of the largest productivity boosts in distributed systems. users usually are able to simply execute a task from one grid node and as task execution penetrates the grid, all classes and resources are also automatically deployed. topology resolution - allows to provision nodes based on any node characteristic or user-specific configuration. for example, you can decide to only include linux nodes for execution, or to only include a certain group of nodes within certain time window. you should also be able to choose all nodes with cpu loaded, say, under 50% that have more than 2gb of available heap memory. collision resolution - allows users to control which jobs get executed, which jobs get rejected, how many jobs can be executed in parallel, order of overall execution, etc. load balancing - allows to balance properly balance your system load within grid. usually range of load balancing policies varies within products. some of the most common ones are round robin, random, or adaptive. more advanced vendors also provide affinity load balancing where grid jobs always end up on the same node based on job's affinity key. this policy works well with data grids described below. fail-over - grid jobs should automatically fail-over onto other nodes in case of node crash or some other job failure. checkpoints - long running jobs should be able to periodically store their intermediate state. this is useful for fail-overs, when a failed job should be able to pick up its execution from the latest checkpoint, rather than start from scratch. grid events - a querying mechanism for all grid events is essential. any grid node should be able to query all events that happened on remote grid nodes during grid task execution. node metrics - a good compute grid solution should be able to provide dynamic grid metrics for all grid nodes. metrics should include vital node statistics, from cpu load to average job execution time. this is especially useful for load balancing, when the system or user need to pick the least loaded node for execution. pluggability - in order to blend into any environment a good compute grid should have well thought out pluggability points. for example, if running on top of jboss, a compute grid should totally reuse jboss communication and discovery protocols. data grid integration - it is important that compute grid are able to natively integrate with data grids as quite often businesses will need both, computational and data features working within same application. some compute grid vendors: - gridgain - professional open source - jppf - open source data grid data grids allow you to distribute your data across the grid. most of us are used to the term distributed cache rather than data grid (data grid does sound more savvy though). the main goal of data grid is to provide as much data as possible from memory on every grid node and to ensure data coherency. some of the important data grid features include: data replication - all data is fully replicated to all nodes in the grid. this strategy consumes the most resources, however it is the most effective solution for read-mostly scenarios, as data is available everywhere for immediate access. data invalidation - in this scenario, nodes load data on demand. whenever data changes on one of the nodes, then the same data on all other nodes is purged (invalidated). then this data will be loaded on-demand the next time it is accessed. distributed transactions - transactions are required to ensure data coherency. cache updates must work just like database updates - whenever an update failed, then the whole transaction must be rolled back. most data grid support various transaction policies, such as read committed, write committed, serializable, etc... data backups - useful for fail-over. some data grid products provide ability to assign backup nodes for the data. this way whenever a node crashes, the data is immediately available from another node. data affinity/partitioning - data affinity allows you to split/partition your whole data set into multiple subsets and assign every subset to a grid node. in the purest form, data is not replicated between nodes at all, every node is only responsible for it's own subset of data. however, various data grid products may provide different flavors of data affinity, such as replication only to back up nodes for example. data affinity is one of the more advanced features, and is not provided by every vendor. to my knowledge, according to product websites, out of commercial vendors oracle coherence and gemstone have it (there may be others). in professional open source space you can take a look at combination of gridgain with affinity load balancing and jbosscache . some data grid/cache vendors: - oracle coherence - commercial - gemstone - commercial - gigaspaces - commercial - jbosscache - professional open source - ehcache - open source
July 31, 2008
by Dmitriy Setrakyan
· 28,298 Views · 3 Likes
  • Previous
  • ...
  • 126
  • 127
  • 128
  • 129
  • 130
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×