we've all seen them on various websites. crappy search utilities. they are a constant reminder that search is not something you should take lightly when building a website or application. search is not just google's game anymore. when a java library called lucene was introduced into the apache ecosystem, and then solr was built on top of that, open source developers began to wield some serious power when it came to customizing search features. in this article you'll be introduced to apache solr and a wealth of applications that have been built with it. the content is divided as follows: introduction setup solr applications summary 1. introduction apache solr is an open source search server. it is based on the full text search engine called apache lucene . so basically solr is an http wrapper around an inverted index provided by lucene. an inverted index could be seen as a list of words where each word-entry links to the documents it is contained in. that way getting all documents for the search query "dzone" is a simple 'get' operation. one advantage of solr in enterprise projects is that you don't need any java code, although java itself has to be installed. if you are unsure when to use solr and when lucene, these answers could help. if you need to build your solr index from websites, you should take a look into the open source crawler called apache nutch before creating your own solution. to be convinced that solr is actually used in a lot of enterprise projects, take a look at this amazing list of public projects powered by solr . if you encounter problems then the mailing list or stackoverflow will help you. to make the introduction complete i would like to mention my personal link list and the resources page which lists books, articles and more interesting material. 2. setup solr 2.1. installation as the very first step, you should follow the official tutorial which covers the basic aspects of any search use case: indexing - get the data of any form into solr. examples: json, xml, csv and sql-database. this step creates the inverted index - i.e. it links every term to its documents. querying - ask solr to return the most relevant documents for the users' query to follow the official tutorial you'll have to download java and the latest version of solr here . more information about installation is available at the official description . next you'll have to decide which web server you choose for solr. in the official tutorial, jetty is used, but you can also use tomcat. when you choose tomcat be sure you are setting the utf-8 encoding in the server.xml . i would also research the different versions of solr, which can be quite confusing for beginners: the current stable version is 1.4.1. use this if you need a stable search and don't need one of the latest features. the next stable version of solr will be 3.x the versions 1.5 and 2.x will be skipped in order to reach the same versioning as lucene. version 4.x is the latest development branch. solr 4.x handles advanced features like language detection via tika, spatial search , results grouping (group by field / collapsing), a new "user-facing" query parser ( edismax handler ), near real time indexing, huge fuzzy search performance improvements, sql join-a like feature and more. 2.2. indexing if you've followed the official tutorial you have pushed some xml files into the solr index. this process is called indexing or feeding. there are a lot more possibilities to get data into solr: using the data import handler (dih) is a really powerful language neutral option. it allows you to read from a sql database, from csv, xml files, rss feeds, emails, etc. without any java knowledge. dih handles full-imports and delta-imports. this is necessary when only a small amount of documents were added, updated or deleted. the http interface is used from the post tool, which you have already used in the official tutorial to index xml files. client libraries in different languages also exist. (e.g. for java (solrj) or python ). before indexing you'll have to decide which data fields should be searchable and how the fields should get indexed. for example, when you have a field with html in it, then you can strip irrelevant characters , tokenize the text into 'searchable terms', lower case the terms and finally stem the terms . in contrast, if you would have a field with text in it that should not be interpreted (e.g. urls) you shouldn't tokenize it and use the default field type string. please refer to the official documentation about field and field type definitions in the schema.xml file. when designing an index keep in mind the advice from mauricio : "the document is what you will search for. " for example, if you have tweets and you want to search for similar users, you'll need to setup a user index - created from the tweets. then every document is a user. if you want to search for tweets, then setup a tweet index; then every document is a tweet. of course, you can setup both indices with the multi index options of solr. please also note that there is a project called solr cell which lets you extract the relevant information out of several different document types with the help of tika. 2.3. querying for debugging it is very convenient to use the http interface with a browser to query solr and get back xml. use firefox and the xml will be displayed nicely: you can also use the velocity contribution , a cross-browser tool, which will be covered in more detail in the section about 'search application prototyping' . to query the index you can use the dismax handler or standard query handler . you can filter and sort the results: q=superman&fq=type:book&sort=price asc you can also do a lot more ; one other concept is boosting. in solr you can boost while indexing and while querying. to prefer the terms in the title write: q=title:superman^2 subject:superman when using the dismax request handler write: q=superman&qf=title^2 subject check out all the various query options like fuzzy search , spellcheck query input , facets , collapsing and suffix query support . 3. applications now i will list some interesting use cases for solr - in no particular order. to see how powerful and flexible this open source search server is. 3.1. drupal integration the drupal integration can be seen as generic use case to integrate solr into php projects. for the php integration you have the choice to either use the http interface for querying and retrieving xml or json. or to use the php solr client library . here is a screenshot of a typical faceted search in drupal : for more information about faceted search look into the wiki of solr . more php projects which integrates solr: open source typo3- solr module magento enterprise - solr module . the open source integration is out dated. oxid - solr module . no open source integration available. 3.2. hathi trust the hathi trust project is a nice example that proves solr's ability to search big digital libraries. to quote directly from the article : "... the index for our one million book index is over 200 gigabytes ... so we expect to end up with a two terabyte index for 10 million books" other examples for libraries: vufind - aims to replace opac internet archive national library of australia 3.3. auto suggestions mainly, there are two approaches to implement auto-suggestions (also called auto-completion) with solr: via facets or via ngramfilterfactory . to push it to the extreme you can use a lucene index entirely in ram. this approach is used in a large music shop in germany. live examples for auto suggestions: kaufda.de 3.4. spatial search applications when mentioning spatial search, people have geographical based applications in mind. with solr, this ordinary use case is attainable . some examples for this are : city search - city guides yellow pages kaufda.de spatial search can be useful in many different ways : for bioinformatics, fingerprints search, facial search, etc. (getting the fingerprint of a document is important for duplicate detection). the simplest approach is implemented in jetwick to reduce duplicate tweets, but this yields a performance of o(n) where n is the number of queried terms. this is okay for 10 or less terms, but it can get even better at o(1)! the idea is to use a special hash set to get all similar documents. this technique is called local sensitive hashing . read this nice paper about 'near similarity search and plagiarism analysis' for more information. 3.5. duckduckgo duckduckgo is made with open source and its "zero click" information is done with the help of solr using the dismax query handler: the index for that feature contains 18m documents and has a size of ~12gb. for this case had to tune solr: " i have two requirements that differ a bit from most sites with respect to solr: i generally only show one result, with sometimes a couple below if you click on them. therefore, it was really important that the first result is what people expected. false positives are really bad in 0-click, so i needed a way to not show anything if a match wasn't too relevant. i got around these by a) tweaking dismax and schema and b) adding my own relevancy filter on top that would re-order and not show anything in various situations. " all the rest is done with tuned open source products. to quote gabriel again: "the main results are a hybrid of a lot of things, including external apis, e.g. bing, wolframalpha, yahoo, my own indexes and negative indexes (spam removal), etc. there are a bunch of different types of data i'm working with. " check out the other cool features such as privacy or bang searches . 3.6. clustering support with carrot2 carrot2 is one of the "contributed plugins" of solr. with carrot2 you can support clustering : " clustering is the assignment of a set of observations into subsets (called clusters) so that observations in the same cluster are similar in some sense. " see some research papers regarding clustering here . here is one visual example when applying clustering on the search "pannous" - our company : 3.7. near real time search solr isn't real time yet, but you can tune solr to the point where it becomes near real time, which means that the time ('real time latency') that a document takes to be searchable after it gets indexed is less than 60 seconds even if you need to update frequently. to make this work, you can setup two indices. one write-only index "w" for the indexer and one read-only index "r" for your application. index r refers to the same data directory of w, which has to be defined in the solrconfig.xml of r via: /pathto/indexw/data/ to make sure your users and the r index see the indexed documents of w, you have to trigger an empty commit every 60 seconds: wget -q http://localhost:port/solr/update?stream.body=%3ccommit/%3e -o /dev/null everytime such a commit is triggered a new searcher without any cache entries is created. this can harm performance for visitors hitting the empty cache directly after this commit, but you can fill the cache with static searches with the help of the newsearcher entry in your solrconfig.xml. additionally, the autowarmcount property needs to be tuned, which fills the cache with a newsearcher from old entries. also, take a look at the article 'scaling lucene and solr' , where experts explain in detail what to do with large indices (=> 'sharding') and what to do for high query volume (=> 'replicating'). 3.8. loggly = full text search in logs feeding log files into solr and searching them at near real-time shows that solr can handle massive amounts of data and queries the data quickly. i've setup a simple project where i'm doing similar things , but loggly has done a lot more to make the same task real-time and distributed. you'll need to keep the write index as small as possible otherwise commit time will increase too great. loggly creates a new solr index every 5 minutes and includes this when searching using the distributed capabilities of solr ! they are merging the cores to keep the number of indices small, but this is not as simple as it sounds. watch this video to get some details about their work. 3.9. solandra = solr + cassandra solandra combines solr and the distributed database cassandra , which was created by facebook for its inbox search and then open sourced. at the moment solandra is not intended for production use. there are still some bugs and the distributed limitations of solr apply to solandra too. tthe developers are working very hard to make solandra better. jetwick can now run via solandra just by changing the solrconfig.xml. solandra also has the advantages of being real-time (no optimize, no commit!) and distributed without any major setup involved. the same is true for solr cloud. 3.10. category browsing via facets solr provides facets , which make it easy to show the user some useful filter options like those shown in the "drupal integration" example. like i described earlier , it is even possible to browse through a deep category tree. the main advantage here is that the categories depend on the query. this way the user can further filter the search results with this category tree provided by you. here is an example where this feature is implemented for one of the biggest second hand stores in germany. a click on 'schauspieler' shows its sub-items: other shops: game-change 3.11. jetwick - open twitter search you may have noticed that twitter is using lucene under the hood . twitter has a very extreme use case: over 1,000 tweets per second, over 12,000 queries per second, but the real-time latency is under 10 seconds! however, the relevancy at that volume is often not that good in my opinion. twitter search often contains a lot of duplicates and noise. reducing this was one reason i created jetwick in my spare time. i'm mentioning jetwick here because it makes extreme use of facets which provides all the filters to the user. facets are used for the rss-alike feature (saved searches), the various filters like language and retweet-count on the left, and to get trending terms and links on the right: to make jetwick more scalable i'll need to decide which of the following distribution options to choose: use solr cloud with zookeeper use solandra move from solr to elasticsearch which is also based on apache lucene other examples with a lot of facets: cnet reviews - product reviews. electronics reviews, computer reviews & more. shopper.com - compare prices and shop for computers, cell phones, digital cameras & more. zappos - shoes and clothing. manta.com - find companies. connect with customers. 3.12. plaxo - online address management plaxo.com , which is now owned by comcast, hosts web addresses for more than 40 million people and offers smart search through the addresses - with the help of solr. plaxo is trying to get the latest 'social' information of your contacts through blog posts, tweets, etc. plaxo also tries to reduce duplicates . 3.13. replace fast or google search several users report that they have migrated from a commercial search solution like fast or google search appliance (gsa) to solr (or lucene). the reasons for that migration are different: fast drops linux support and google can make integration problems. the main reason for me is that solr isn't a black box —you can tweak the source code, maintain old versions and fix your bugs more quickly! 3.14. search application prototyping with the help of the already integrated velocity plugin and the data import handler it is possible to create an application prototype for your search within a few hours. the next version of solr makes the use of velocity easier. the gui is available via http://localhost:port/solr/browse if you are a ruby on rails user, you can take a look into flare. to learn more about search application prototyping, check out this video introduction and take a look at these slides. 3.15. solr as a whitelist imagine you are the new google and you have a lot of different types of data to display e.g. 'news', 'video', 'music', 'maps', 'shopping' and much more. some of those types can only be retrieved from some legacy systems and you only want to show the most appropriated types based on your business logic . e.g. a query which contains 'new york' should result in the selection of results from 'maps', but 'new yorker' should prefer results from the 'shopping' type. with solr you can set up such a whitelist-index that will help to decide which type is more important for the search query. for example if you get more or more relevant results for the 'shopping' type then you should prefer results from this type. without the whitelist-index - i.e. having all data in separate indices or systems, would make it nearly impossible to compare the relevancy. the whitelist-index can be used as illustrated in the next steps. 1. query the whitelist-index, 2. decide which data types to display, 3. query the sub-systems and 4. display results from the selected types only. 3.16. future solr is also useful for scientific applications, such as a dna search systems. i believe solr can also be used for completely different alphabets so that you can query nucleotide sequences - instead of words - to get the matching genes and determine which organism the sequence occurs in, something similar to blast . another idea you could harness would be to build a very personalized search. every user can drag and drop their websites of choice and query them afterwards. for example, often i only need stackoverflow, some wikis and some mailing lists with the expected results, but normal web search engines (google, bing, etc.) give me results that are too cluttered. my final idea for a future solr-based app could be a lucene/solr implementation of desktop search. solr's facets would be especially handy to quickly filter different sources (files, folders, bookmarks, man pages, ...). it would be a great way to wade through those extra messy desktops. 4. summary the next time you think about a problem, think about solr! even if you don't know java and even if you know nothing about search: solr should be in your toolbox. solr doesn't only offer professional full text search, it could also add valuable features to your application. some of them i covered in this article, but i'm sure there are still some exciting possibilities waiting for you!

I am a Spring framework user for last three years and I have really enjoyed working with Spring. One thing that I am seeing these days is the heavy use of AspectJ in most of SpringSource products. Spring Roo is a RAD tool for Java developers which makes use of AspectJ ITD's for separate compilation units. Spring transaction management and exception translation is also done using Aspectj. There are also numerous other usages of Aspectj in Spring products. In this article, I am going to talk about another cool usage of AspectJ by Spring - Mocking Static Methods. These days most of the developers write unit test cases and it is very common to use any of the mocking libraries like EasyMock, Mockito etc. to mock the external dependencies of a class. Using any of these mocking libraries it is very easy to mock calls to other class instance method. But most of these mocking framework does not provide the facility to mock the calls to static methods. Spring provides you the capability to mock static methods by using Spring Aspects library. In order to use this feature you need to add spring-aspects.jar dependency in your pom.xml. org.springframework spring-aspects 3.0.4.RELEASE Next thing you need to do is to convert your project in to a AspectJ project. If you are using Eclipse or STS(SpringSource Tool Suite) you can do that by right-click your project -> configure -> convert to AspectJ project. STS by default has AspectJ plugin, for eclipse users you need to install AspectJ plugin for above to work. I would recommend using STS for developing Spring based applications. The two aspects and one annotation that is of interest in spring-aspects.jar are : AbstractMethodMockingControl : This is an abstract aspect to enable mocking of methods picked out by a pointcut. All the child aspects need to define mockStaticsTestMethod() and methodToMock() pointcuts. The mockStaticTestMethod() pointcut is used to indicate when mocking should be triggered and methodToMock() pointcut is used to define which method invocations to mock. AnnotationDrivenStaticEntityMockingControl : This is the single implementation of AbstractMethodMockingControl aspect which exists in spring-aspects.jar. This is an annotation-based aspect to use in a test build to enable mocking static methods on Entity classes. In this aspect mockStaticTestMethod() pointcut defines that for classes marked with @MockStaticEntityMethods annotation mocking should be triggered and methodToMock() pointcut defines that all the public static methods in the classes marked with @Entity annotation should be mocked. MockStaticEntityMethods : Annotation to indicate a test class for whose @Test methods static methods on Entity classes should be mocked. The AnnotationDrivenStaticEntityMockingControl provide the facility to mock static methods of any class which is marked with @Entity annotation. But usually we would need to mock static method of classes other than marked with @Entity annotation. The only thing we need to do to make it work is to extend AbstractMethodMockingControl aspect and provide definitions for mockStaticsTestMethod() and methodToMock() pointcuts. For example, lets write an aspect which should mock all the public static methods of classes marked with @Component annotation. package com.shekhar.javalobby; import org.springframework.mock.staticmock.AbstractMethodMockingControl; import org.springframework.stereotype.Component; import org.springframework.mock.staticmock.MockStaticEntityMethods;; public aspect AnnotationDrivenStaticComponentMockingControl extends AbstractMethodMockingControl { public static void playback() { AnnotationDrivenStaticComponentMockingControl.aspectOf().playbackInternal(); } public static void expectReturn(Object retVal) { AnnotationDrivenStaticComponentMockingControl.aspectOf().expectReturnInternal(retVal); } public static void expectThrow(Throwable throwable) { AnnotationDrivenStaticComponentMockingControl.aspectOf().expectThrowInternal(throwable); } protected pointcut mockStaticsTestMethod() : execution(public * (@MockStaticEntityMethods *).*(..)); protected pointcut methodToMock() : execution(public static * (@Component *).*(..)); } The only difference between AnnotationDrivenStaticEntityMockingControl(comes with spring-aspects.jar) and AnnotationDrivenStaticComponentMockingControl(custom that we have written above) is in methodToMock() pointcut. In methodToMock() pointcut we have specified that it should mock all the static methods in any class marked with @Component annotation. Now that we have written the custom aspect lets test it. I have created a simple ExampleService with one static method. This is the method which we want to mock. @Component public class ExampleService implements Service { /** * Reads next record from input */ public String getMessage() { return myName(); } public static String myName() { return "shekhar"; } } This class will return "shekhar" when getMessage() method will be called. Lets test this without mocking package com.shekhar.javalobby; import org.junit.Assert; import org.junit.Test; public class ExampleConfigurationTests { private ExampleService service = new ExampleService(); @Test public void testSimpleProperties() throws Exception { String myName = service.getMessage(); Assert.assertEquals("shekhar", myName); } } This test will work fine. Now let's add mocking to this test class. There are two things that we need to do in our test We need to annotate our test with @MockStaticEntityMethods to indicate that static methods of @Component classes will be mocked. Please note that it is not required to use @MockStaticEntityMethods annotation you can create your own annotation and use that in mockStaticsTestMethod() pointcut. So, I could have created an annotation called @MockStaticComponentMethods and used that in mockStaticsTestMethod() pointcut. But I just reused the @MockStaticEntityMethods annotation. In our test methods we need to first invoke the static method which we want to mock so that it gets recorded. Next we need to set our expectation i.e. what should be returned from the mock and finally we need to call the playback method to stop recording mock calls and enter playback state. To make it more concrete lets apply mocking to the above test import org.junit.Assert; import org.junit.Test; import org.springframework.mock.staticmock.MockStaticEntityMethods; @MockStaticEntityMethods public class ExampleConfigurationTests { private ExampleService service = new ExampleService(); @Test public void testSimpleProperties() throws Exception { ExampleService.myName(); AnnotationDrivenStaticComponentMockingControl.expectReturn("shekhargulati"); AnnotationDrivenStaticComponentMockingControl.playback(); String myName = service.getMessage(); Assert.assertEquals("shekhargulati", myName); } } As you can see we annotated the test class with @MockStaticEntityMethods annotation and in the test method we first recorded the call (ExampleService.myName()), then we set the expectations, then we did the playback and finally called the actual code. In this way you can mock the static method of class.

Sometimes you have tests where the fixtures can be recyled, instead of being recreated; but that's not all: sometimes, even the state reset we preach for Shared Fixture gets in the way instead of being mandatory. For example, think of testing addition and removal of values over a collection object: to test the removal, you need a collection containing something. But this collection has to be populated somewhere, involving other methods which would have their own independent tests. If you look closer, however, you notice that the collection you need for testing values removal is just like the one that testAddsElements() produces when it succeeds, and simply throws away when the Test Methods returns... The pattern In these cases, the Chained Tests pattern can help you design isolated tests, by expressing the dependencies they implicitly have. The application of this pattern results in a form of Api encapsulation: a test that tests Collection::remove() does not need to use Collection::add(), and won't be influenced by a change in the name or the signature of add(), which it does not focus on. The test just needs just a collection which has been already filled with values. Before "spolverare" unserialization and other strange techniques, just consider using the leftovers from a previous test. Let's suppose we have two tests A and B, where B uses a variable produced in A. Of course, if A fails, the dependent Test Method B cannot (and shouldn't) be run; it will only add noise with an obvious failure which derives not from the behavior really under test there, but just from the fixture setup, which happens to be the previous test. If A instead runs correctly, B can be run. Implementation In PHPUnit, Chained Tests are supported when they are in the same Testcase Class. In this case, the Testcase Objects are not totally isolated anymore: you can return variables and objects from one test, in order for PHPUnit to pass them to other tests, that simply asks for them as dependencies. You can do so by adding @depends annotation to dependent tests. Note that this annotation can also be used without passing fixtures around, just for the sake of not executing complex tests where the base ones have already failed. @depends working by stopping the execution of tests down in the chain when the dependencies fail: they will be marked as skipped and signaled by an S in PHPUnit's ordinary output. The tests which produce a fixture as a leftover must return that fixture at the end of the test. The tests accepting fixtures, tagged with @depends, must accept parameters in the Test Method signatures. In case of multiple parameters to return, you can return an array() and use list() in the dependent method to extract variables from it. There is a known issue in running dependent tests with --filter: dependent tests would not run alone, so if you plan to use filter you may want to pick similar test names for couples of dependent/dependency in order to execute both of them when needed. Example Here we have a sample Testcase Class where the second and third tests are dependent on the first. Long chains of tests are not encouraged as it becomes increasingly difficult to tell what is the state of the fixture in the tests at the end of chain. assertEquals(15, $array[2], 'The third element was not 15.'); return $array; } /** * @depends testArrayAcceptsNewValues */ public function testArrayCanDeleteValues($array) { unset($array[2]); $this->assertEquals(2, count($array), 'There are more than 2 elements in the array.'); } /** * @depends testArrayAcceptsNewValues */ public function testArrayAcceptsDuplicateValues($array) { $array[] = 15; $this->assertEquals(4, count($array), 'There are less than 4 elements in the array.'); } } If we execute the whole test case, we obtain: [10:40:55][giorgio@Desmond:~]$ phpunit txt/articles/chainedtest.php PHPUnit 3.5.5 by Sebastian Bergmann. ... Time: 0 seconds, Memory: 5.00Mb OK (3 tests, 3 assertions) If we put a $this->fail() in the first test, instead, we get: [10:41:40][giorgio@Desmond:~]$ phpunit txt/articles/chainedtest.php PHPUnit 3.5.5 by Sebastian Bergmann. FSS Time: 0 seconds, Memory: 5.00Mb There was 1 failure: 1) ArrayTest::testArrayAcceptsNewValues /home/giorgio/Dropbox/txt/articles/chainedtest.php:13 FAILURES! Tests: 1, Assertions: 1, Failures: 1, Skipped: 2. If we filter the second test, we cannot run it due to the missing dependency: [10:49:25][giorgio@Desmond:~]$ phpunit --filter testArrayCan txt/articles/chainedtest.php PHPUnit 3.5.5 by Sebastian Bergmann. S Time: 0 seconds, Memory: 4.75Mb OK, but incomplete or skipped tests! Tests: 0, Assertions: 0, Skipped: 1. But if we filter the third test, thanks to the well-chosen name, we get: [10:49:16][giorgio@Desmond:~]$ phpunit --filter testArrayAccepts txt/articles/chainedtest.php PHPUnit 3.5.5 by Sebastian Bergmann. .. Time: 0 seconds, Memory: 5.00Mb OK (2 tests, 2 assertions)

In the previous article of this series, we saw how assertions are the key to self-validating tests. Even if an assertion's result is only a pass/fail value, programmers and testers find very handy to associate to the fail value some kind of additional information about the failure, to allow easier debugging without insertion of var_dump() and similar statements into the code. As I showed you in the previous article's code sample, assertions are useful the most when they fail, and in case they do, they should explain what happened so that a correction can be made quickly. An assertion that passes does not do anything; an assertion that fails shows an error. Whata happens when tests are failing The typical scenario is that of a large test suite, with hundreds or thousands of tests. At the time of a commit to the source code repository (or of a push if we're talking about git), the whole test suite must be green. Even if only one tests is red, the test suite is regarded as red. Sometimes, a regression is introduced by a programming error or an unforeseen scenario, and the output of the test suite becomes something like this: [12:30:02][giorgio@Desmond:~/txt/articles]$ phpunit assertionmessage.php PHPUnit 3.5.5 by Sebastian Bergmann. FFFF Time: 0 seconds, Memory: 3.75Mb There were 4 failures: 1) AssertionMessagesTest::testAsserTrue The object is not an Iterator. Failed asserting that is true. /home/giorgio/Dropbox/txt/articles/assertionmessage.php:9 2) AssertionMessagesTest::testAssertEquals The square root of 17 is not 4 but 4.1231056256177. Failed asserting that matches expected . /home/giorgio/Dropbox/txt/articles/assertionmessage.php:17 3) AssertionMessagesTest::testAssertContainsFailsWithCustomMessage The array does not contain 4. Failed asserting that an array contains . /home/giorgio/Dropbox/txt/articles/assertionmessage.php:24 4) AssertionMessagesTest::testAssertContains Failed asserting that an array contains . /home/giorgio/Dropbox/txt/articles/assertionmessage.php:36 FAILURES! Tests: 4, Assertions: 4, Failures: 4. The green tests are not noisy (which would be a smell), since they are not interesting at the moment. The red tests, which our focus is on, are shown with all the associated information. Along with the test name, the line of interest, and the kind of problem (failure/error), the assertion message is shown. How to write a meaningful message An assertion message can be passed as an argument of the assertion method, to be used in case of failure. Writing good assertion messages is an art, but before diving into some code to show you some messages I wrote, we can see how assertion messages are categorized in literature: Assertion-Identifying Message: includes information on which of the assertions in the same method caused the failure. This information can be for example the name of the checked variables. It's good to understand at a glance which assertion failed, but PHPUnit shows you the line of the test method that caused the failure, so it's not strictly necessary to include this kind of messages. Expectation-Describing Message: tells us what should have happened, but did not according to the data passed to the assertion method. Argument-Describing Message: when using assertions which are not very smart, like assertFalse() or assertTrue(), a failure would tell us by default only something like False was passed to assertTrue(). Adding an Argument-Describing Message will tell us instead The predicate on the set containing all red cars was false instead of true. Watch the test fail I can never say it enoug: when you write your tests, watch them fail (this mean also that each test should be written in advance with regard to the code it exercises, following the discipline of Test-Driven Development). This practice is not only useful for avoiding false positives, which are rare anyway, but to show at least for once the assertion message. You can check that when the test fails, the error message is meaningful and the test does not cause a Fatal Error. Watch your test fail to ensure that when they'll fail due to a regression in the future, you will be pleased by how fast you'll learn what's wrong. Example The code sample shows how to use PHPUnit's assertion methods with the $message optional argument. Any time you use basic methods like assertTrue(), a message is mandatory. The need for $message is inversely proportional to the specificity of the assertion: for example when assertContains() fails, PHPUnit has already valuable information in the the method itself to produce a readable error message. Note that the usage of "" (double quotes) enables fast composition of error messages via variable substitution. assertTrue($object instanceof Iterator, 'The object is not an Iterator.'); } public function testAssertEquals() { $expected = 4; $square = 17; $result = sqrt($square); $this->assertEquals($expected, $result, "The square root of $square is not $expected but $result."); } public function testAssertContainsFailsWithCustomMessage() { $array = array(1, 2, 3); $testElement = 4; $this->assertContains($testElement, $array, "The array does not contain $testElement."); } /** * This Assertion Message would be enough. It will even be better in some cases, * since PHPUnit would display nicely $testElement even if it was not "castable" * to a string (an object or array). */ public function testAssertContains() { $array = array(1, 2, 3); $testElement = 4; $this->assertContains($testElement, $array); } }

JUnit 4.8 introduced Categories: a mechanism to label and group tests, giving developers the option to include or exclude groups (or categories.) This post presents a brief overview of JUnit categories and some unexpected behavior I have found while using them. 1. Quick Introduction The following example shows how to use categories: (adapted from JUnit’s release notes) public interface FastTests { /* category marker */ } public interface SlowTests { /* category marker */ } public class A { @Category(SlowTests.class) @Test public void a() {} } @Category(FastTests.class}) public class B { @Test public void b() {} } @RunWith(Categories.class) @IncludeCategory(SlowTests.class) @ExcludeCategory(FastTests.class) @SuiteClasses({ A.class, B.class }) public class SlowTestSuite {} Lines 1, 2: we define two categories, FastTests and SlowTests. JUnit categories can be defined as classes or interfaces. Since a category acts like a label or marker, my intuition tells me to use interfaces. Line 5: we use the annotation @org.junit.experimental.categories.Category to label test classes and test methods with one or more categories. Lines 6, 9: test methods and test classes can be marked as belonging to one or more categories of tests. Labeling a test class with a category automatically includes all its test methods in such category. Lines 14 to 18: currently programmatic test suites (line 17) are the only way to specify which test categories (line 14) should be included (line 15) or excluded (line 16) when the suite is executed. I find this approach (especially the way test classes need to included in the suite) too verbose and not so flexible. Hopefully Ant, Maven and IDEs will provide support for categories (with a simpler configuration) in the very near future. Note: I recently discovered ClasspathSuite, a project that simplifies the creation of programmatic JUnit test suites. For example, we can specify we want to include in a test suite all tests whose names end with “UnitTest.” 2. Category Subtyping Categories also support subtyping. Let’s say we have the category IntegrationTests that extends SlowTests: public interface IntegrationTests extends SlowTests {} Any test class or test method labeled with the category IntegrationTests is also part of the category SlowTests. To be honest, I don’t know how handy category subtyping could be. I’ll need to experiment with it more to have an opinion. 3. Categories and Test Inheritance 3a. Method-level Categories JUnit behaves as expected when test inheritance is combined with method-level categories. For example: public class D { @Category(GuiTest.class) @Test public void d() {} } public class E extends D { @Category(GuiTest.class) @Test public void e() {} } @RunWith(Categories.class) @IncludeCategory(GuiTest.class) @SuiteClasses(E.class) public class TestSuite {} As I expected, when running TestSuite, test methods d and e are executed (both methods belong to the GuiTest category and E inherits method d from superclass D.) Nice! 3b. Class-level Categories On the other hand, unless I’m missing something, I think I found some strange behavior in JUnit in this scenario. Consider the following classes: @Category(GuiTest.class) public class A { @Test public void a() {} } public class B extends A { @Test public void b() {} } @RunWith(Categories.class) @IncludeCategory(GuiTest.class) @SuiteClasses(B.class) public class TestSuite {} As we can see, TestSuite should execute the tests in B that belong to the category GuiTest. I was expecting TestSuite to execute test method a, even though B is not marked as a GuiTest. Here is my reasoning: test method a belongs to the category GuiTest because test class A is labeled with such category test class B is an A and it inherits test method a Therefore, TestSuite should execute test method a. But it doesn’t! Here is a screenshot of the results I get (click to see full size.) There are two ways to fix this issue, depending on what test methods we want to actually run: Label class B with GuiTest. In this case, both methods, a and b, will be executed. Label method a with GuiTest. In this case, only method a will be executed. (I’ll be posting a question regarding this issue in the JUnit mailing list shortly.) 4. Categories vs. TestNG Groups (You saw this one coming, didn’t you?) Categories (or groups) have been part of TestNG for long time. Unlike JUnit’s, TestNG’s groups are defined as simple strings, not as classes or interfaces. As a static typing lover, I was pretty happy with JUnit categories. By using an IDE, we could safely rename a category or look for usages of a category within a project. Even though my observation was correct, I was missing one important point: all this works great as long as your test suite is written in Java. In the real world, I’d like to define a test suite in either Ant or Maven (or Gradle, or Rake.) In this scenario, having categories as Java types does not bring any benefit. In fact, I suspect it would be very verbose and error-prone to specify the fully-qualified name of a category in a build script. Renaming a category now would be limited to a text-based “search and replace.” Ant and Maven really need to provide a way to specify JUnit categories, clever enough to be fool-proof. As you may expect, I prefer the simplicity and pragmatism of TestNG’s groups. Update: my good friend (and creator of the TestNG framework,) Cédric, reminded me that we can use regular expressions to include or exclude groups in a test suite (details here.) This is really powerful! 5. My Usage of Categories I’m not using JUnit categories in my test suites yet. I started to look into JUnit categories because I wasn’t completely happy with the way we recognized GUI tests in FEST. We recognize test methods or test classes as “GUI tests” if they have been annotated with the @GUITest (provided by FEST.) When a “GUI test” fails, FEST automatically takes a screenshot of the desktop and includes it in the JUnit or TestNG HTML report. The problem is, our @GUITest annotation is duplicating the functionality of JUnit categories. To solve this issue, I created a JUnit extension that recognizes test methods or test classes as “GUI tests” if they belong to the GuiTest category. At this moment GuiTest is an interface provided by FEST, but I’m thinking about letting users specify their own GuiTest category as well. I also refactored this functionality out of the Swing testing module, expecting to reuse it once I implement a JavaFX testing module :) You can find the FEST code that deals with JUnit categories at github. 6. Conclusion Having the ability to label and group tests via categories is really a great feature. I still have some reservations about the practicality of defining categories as Java types, the lack of support for this feature from Ant and Maven (not JUnit’s fault,) and the unexpected behavior I noticed when combining class-level categories and test inheritance. On the brighter side, categories are still an experimental, non-final feature. I’m sure will see many improvements in future JUnit releases :) Feedback is always welcome. From http://alexruiz.developerblogs.com/?p=1711

Maven profiles, like chainsaws, are a valuable tool, with whose power you can easily get carried away, wielding them upon problems to which they are unsuited. Whilst you're unlikely to sever a leg misusing Maven profiles, I thought it worthwhile to share some suggestions about when and when not to use them. These three best practices are all born from real-world mishaps: The build must pass when no profile has been activated Never use Use profiles to manage build-time variables, not run-time variables and not (with rare exceptions) alternative versions of your artifact I'll expand upon these recommendations in a moment. First, though, let's have a brief round-up of what Maven profiles are and do. Maven Profiles 101 A Maven profile is a sub-set of POM declarations that you can activate or disactivate according to some condition. When activated, they override the definitions in the corresponding standard tags of the POM. One way to activate a profile is to simply launch Maven with a -P flag followed by the desired profile name(s), but they can also be activated automatically according to a range of contextual conditions: JDK version, OS name and version, presence or absence of a specific file or property. The standard example is when you want certain declarations to take effect automatically under Windows and others under Linux. Almost all the tags that can be placed directly in a POM can also be enclosed within a tag. The easiest place to read up further about the basics is the Build Profiles chapter of Sonatype's Maven book. It's freely available, readable, and explains the motivation behind profiles: making the build portable across different environments. The build must pass when no profile has been activated (Thanks to for this observation.) Why? Good practice is to minimise the effort required to make a successful build. This isn't hard to achieve with Maven, and there's no excuse for a simple mvn clean package not to work. A maintainer coming to the project will not immediately know that profile wibblewibble has to be activated for the build to succeed. Don't make her waste time finding it out. How to achieve it It can be achieved simply by providing sensible defaults in the main POM sections, which will be overridden if a profile is activated. Never use Why not? This flag activates the profile if no other profile is activated. Consequently, it will fail to activate the profile if any other profile is activated. This seems like a simple rule which would be hard to misunderstand, but in fact it's surprisingly easy to be fooled by its behaviour. When you run a multimodule build, the activeByDefault flag will fail to operate when any profile is activated, even if the profile is not defined in the module where the activeByDefault flag occurs. (So if you've got a default profile in your persistence module, and a skinny war profile in your web module... when you build the whole project, activating the skinny war profile because you don't want JARs duplicated between WAR and EAR, you'll find your persistence layer is missing something.) activeByDefault automates profile activation, which is a good thing; activates implicitly, which is less good; and has unexpected behaviour, which is thoroughly bad. By all means activate your profiles automatically, but do it explicitly and automatically, with a clearly defined rule. How to avoid it There's another, less documented way to achieve what aims to achieve. You can activate a profile in the absence of some property: !foo.bar This will activate the profile "nofoobar" whenever the property foo.bar is not defined. Define that same property in some other profile: nofoobar will automatically become active whenever the other is not. This is admittedly more verbose than , but it's more powerful and, most importantly, surprise-free. Use profiles to adapt to build-time context, not run-time context, and not (with rare exceptions) to produce alternative versions of your artifact Profiles, in a nutshell, allow you to have multiple builds with a single POM. You can use this ability in two ways: Adapt the build to variable circumstances (developer's machine or CI server; with or without integration tests) whilst still producing the same final artifact, or Produce variant artifacts. We can further divide the second option into: structural variants, where the executable code in the variants is different, and variants which vary only in the value taken by some variable (such as a database connection parameter). If you need to vary the value of some variable at run-time, profiles are typically not the best way to achieve this. Producing structural variants is a rarer requirement -- it can happen if you need to target multiple platforms, such as JDK 1.4 and JDK 1.5 -- but it, too, is not recommended by the Maven people, and profiles are not the best way of achieving it. The most common case where profiles seem like a good solution is when you need different database connection parameters for development, test and production environments. It is tempting to meet this requirement by combining profiles with Maven's resource filtering capability to set variables in the deliverable artifact's configuration files (e.g. Spring context). This is a bad idea. Why? It's indirect: the point at which a variable's value is determined is far upstream from the point at which it takes effect. It makes work for the software's maintainers, who will need to retrace the chain of events in reverse It's error prone: when there are multiple variants of the same artifact floating around, it's easy to generate or use the wrong one by accident. You can only generate one of the variants per build, since the profiles are mutually exclusive. Therefore you will not be able to use the Maven release plugin if you need release versions of each variant (which you typically will). It's against Maven convention, which is to produce a single artifact per project (plus secondary artifacts such as documentation). It slows down feedback: changing the variable's value requires a rebuild. If you configured at run-time you would only need to restart the application (and perhaps not even that). One should always aim for rapid feedback. Profiles are there to help you ensure your project will build in a variety of environments: a Windows developer's machine and a CI server, for instance. They weren't intended to help you build variant artifacts from the same project, nor to inject run-time configuration into your project. How to achieve it If you need to get variable runtime configuration into your project, there are alternatives: Use JNDI for your database connections. Your project only contains the resource name of the datasource, which never changes. You configure the appropriate database parameters in the JNDI resource on the server. Use system properties: Spring, for example, will pick these up when attempting to resolve variables in its configuration. Define a standard mechanism for reading values from a configuration file that resides outside the project. For example, you could specify the path to a properties file in a system property. Structural variants are harder to achieve, and I confess I have no first-hand experience with them. I recommend you read this explanation of how to do them and why they're a bad idea, and if you still want to do them, take the option of multiple JAR plugin or assembly plugin executions, rather than profiles. At least that way, you'll be able to use the release plugin to generate all your artifacts in one build, rather than a single one at a time. Further reading Profiles chapter from the Sonatype Maven book. Deploying to multiple environments (prod, test, dev): Stackoverflow.com discussion; see the first and top-rated answer. Short of creating a specific project for the run-time configuration, you could simply use run-time parameters such as system properties. Creating multiple artifacts from one project: How to Create Two JARs from One Project (…and why you shouldn’t) by Tim O'Brien of Sonatype (the Maven people) Blog post explaining the same technique Maven best practices (not specifically about profiles): http://mindthegab.com/2010/10/21/boost-your-maven-build-with-best-practices/ http://blog.tallan.com/2010/09/16/maven-best-practices/ This article is a completely reworked version of a post from my blog.

Both approaches have some advantages and disadvantages and so it’s a question of the likelihood of particular problems and business requirements and priorities.

Spock is an open source testing framework for Java and Groovy that has been attracting a growing following, especially in the Groovy community. It lets you write concise, expressive tests, using a quite readable BDD-style notation. It even comes with its own mocking library built in. Oh. I thought he was a sci-fi character. Can I see an example? Sure. Here's a simple one from a coding kata I did recently: import spock.lang.Specification; class RomanCalculatorSpec extends Specification { def "I plus I should equal II"() { given: def calculator = new RomanCalculator() when: def result = calculator.add("I", "I") then: result == "II" } } In Spock, you don't have tests, you have specifications. These are normal Groovy classes that extend the Specifications class, which is actually a JUnit class. Your class contains a set of specifications, represented by methods with funny-method-names-in-quotes™. The funny-method-names-in-quotes™ take advantage of some Groovy magic to let you express your requirements in a very readable form. And since these classes are derived from JUnit, you can run them from within Eclipse like a normal Groovy unit test, and they produce standard JUnit reports, which is nice for CI servers. Another thing: notice the structure of this test? We are using given:, when: and then: to express actions and expected outcomes. This structure is common in Behaviour-Driven Development, or BDD, frameworks like Cucumber and easyb. Though Spock-style tests are generally more concise more technically-focused than tools like Cucumber and easyb, which are often used for automating acceptance tests. But I digress... Actually, the example I gave earlier was a bit terse. We could make our intent clearer by adding text descriptions after the when: and then: labels, as I've done here: def "I plus I should equal II"() { when: "I add two roman numbers together" def result = calculator.add("I", "I") then: "the result should be the roman number equivalent of their sum" result == "II" } This is an excellent of clarifying your ideas and documenting your API. But where are the AssertEquals statements? Aha! I'm glad you asked! Spock uses a feature called Power Asserts. The statement after the then: is your assert. If this test fails, Spock will display a detailed analysis of what went wrong, along the following lines: I plus I should equal II(com.wakaleo.training.spocktutorial.RomanCalculatorSpec) Time elapsed: 0.33 sec <<< FAILURE! Condition not satisfied: result == "II" | | I false 1 difference (50% similarity) I(-) I(I) at com.wakaleo.training.spocktutorial .RomanCalculatorSpec.I plus I should equal II(RomanCalculatorSpec.groovy:17) Nice! But in JUnit, I have @Before and @After for fixtures. Can I do that in Spock? Sure, but you don't use annotations. Instead you implement setup() and cleanup() methods (which are run before and after each specification). I've added one here to show you what they look like: import spock.lang.Specification; class RomanCalculatorSpec extends Specification { def calculator def setup() { calculator = new RomanCalculator() } def "I plus I should equal II"() { when: def result = calculator.add("I", "I") then: result == "II" } } You can also define a setupSpec() and cleanupSpec(), which are run just before the first test and just after the last one. I'm a big fan of parameterized tests in JUnit 4. Can I do that in Spock! You sure can! In fact it's one of Spock's killer features! def "The lowest number should go at the end"() { when: def result = calculator.add(a, b) then: result == sum where: a | b | sum "X" | "I" | "XI" "I" | "X" | "XI" "XX" | "I" | "XXI" "XX" | "II" | "XXII" "II" | "XX" | "XXII" } This code will run the test 5 times. The variables a, b, and sum are initialized from the rows in the table in the where: clause. And if any of the tests fail, you get That's pretty cool too. What about mocking? Can I use Mockito? Sure, if you want. but Spock actually comes with it's own mocking framework, which is pretty neat. You set up a mock or a stub using the Mock() method. I've shown two possible ways to use this method here: given: Subscriber subscriber1 = Mock() def subscriber2 = Mock(Subscriber) ... You can set these mocks up to behave in certain ways. Here are a few examples. You can say a method should return a certain value using the >> operator: subscriber1.isActive() >> true subscriber2.isActive() >> false Or you could get a method to throw an exception when it is called: subscriber.activate() >> { throw new BlacklistedSubscriberException() } Then you can test outcomes in a few different ways. Here is a more complicated example to show you some of your options: def "Messages published by the publisher should only be received by active subscribers"() { given: "a publisher" def publisher = new Publisher() and: "some active subscribers" Subscriber activeSubscriber1 = Mock() Subscriber activeSubscriber2 = Mock() activeSubscriber1.isActive() >> true activeSubscriber2.isActive() >> true publisher.add activeSubscriber1 publisher.add activeSubscriber2 and: "a deactivated subscriber" Subscriber deactivatedSubscriber = Mock() deactivatedSubscriber.isActive() >> false publisher.add deactivatedSubscriber when: "a message is published" publisher.publishMessage("Hi there") then: "the active subscribers should get the message" 1 * activeSubscriber1.receive("Hi there") 1 * activeSubscriber2.receive({ it.contains "Hi" }) and: "the deactivated subscriber didn't receive anything" 0 * deactivatedSubscriber.receive(_) } That does look neat. So what is the best place to use Spock? Spock is great for unit or integration testing of Groovy or Grails projects. On the other hand, tools like easyb amd cucumber are probably better for automated acceptance tests - the format is less technical and the reporting is more appropriate for non-developers. From http://www.wakaleo.com/blog/303-an-introduction-to-spock

Have you ever had to create a mock object in which most methods do nothing and are not called, but in others something useful needs to be done? EasyMock has some newish functionality to let you stub individual methods. But before I had heard about that, I had built a little framework (one base class) for creating mock objects which stubs those methods you want to stub, as well as logging every call made to the classes being mocked. It works like this: you choose a class which you need to mock, for example a service class called FooService, and you create a new class called FooServiceMock. You make it extend from AbstractMock, where T is the class you are mocking. As an example: public class FooServiceMock extends AbstractMock { public FooServiceMock() { super(FooService.class); } It needs to have a constructor to call the super constructor passing the class being mocked too. Perhaps that could be optimised, I don't have too much time right now. Next, you implement only those methods you expect to be called. For example: public class FooServiceMock extends AbstractMock { public FooServiceMock() { super(FooService.class); } /** * this is a method which exists in FooService, * but I want it to do something else. */ public String sayHello(String name){ return "Hello " + name + ", Foo here! This is a stub method!"; } To use the mock, you'll notice that it doesn't extend the class which it mocks, which might be problematic... Well, there are good reasons. To do the mocking, the abstract base class is actually going to create a dynamic proxy which wraps itself behind the interface of the class being mocked. To the caller, it looks like the FooService, but it's not actually anything related to it. Anytime a call to the FooService is made, the first thing which the proxy does is log that call, using XStream to create an XML representation of the parameters being passed into the method. Then, the proxy goes and looks in the instance of the mock class to see if it can find the method being called (well at least a method which takes the same parameters and has the same name and return type). If it finds such a method, it calls it. In our example, the sayHello(String) method would get called. It returns the result if there is one, to the caller. In the case where it cannot find the method, it throws an exception, because it assumes that if it was not implemented, you didn't expect it to be called. You could of course change this to suit your needs, maybe even calling the actual FooService. So, how to you use the FooServiceMock to create a FooService instance which you can use to mock your service? In the test, where you setup the class under test, you do this: FooServiceMock fooService = new FooServiceMock(); //perhaps tell it about objects you would //like it to return... instanceOfClassUnderTest.setFooService( fooService.getMock()); The setFooService(FooService) method on the instance of the class you are testing is in my case present, but you might not have it and may need to use reflection to do it. It's a question of how testable you write your classes, and is a design choice. The getMock() method on the AbstractMock class is the method which creates the dynamic proxy which wraps the instance of the mock. You can now test the class. There is however still something useful you can do after testing, i.e. assert that the right calls were made in the correct order with the right parameters. You do this in the test class to: assertEquals(1, fooService.getCalls().size()); assertEquals("[sayHello: Ant]", fooService.getCalls().toString()); The above tests that the sayHello(String) method was called just once, and passed the name "Ant". There are times when you might want to clear the call log, between parts of the test. For that, call the clearCalls() method on the mock object: fooService.clearCalls(); Have fun! From http://blog.maxant.co.uk/pebble/2010/11/03/1288813500000.html

It seems that lots of projects/products/services want to expose a REST API these days. But I have found very few that actually follow the REST constraints, and in a lot of the cases it doesn't even make sense for them to follow REST constraints in the first place. One of the main constraints that is commonly violated is the hypertext constraint. Basically, all state changes have to be done by following links, starting from a bookmarked URL. But almost noone does that. However, should they? This article will outline various layers that REST API's can be implemented in, and when it makes sense, and when not. To begin with, in a typical enterprise app there are three options for layers that you might want to expose using a REST API. These are the infrastructure layer, the domain layer, and the application layer. Infrastructure layer If we start with the infrastructure layer, we are typically talking about a database vendor that wants to allow developers to access it using "REST". The API would allow you to create/remove databases, and then insert/update/delete data. Typically it's pretty normal stuff, and the API doesn't change all that much between versions. Accessing this over HTTP maybe makes sense, but is it RESTful? I'll give you an example. I installed CouchDB, and given the hypermedia constraint I should then be able to go to "http://localhost:5984/", and it will tell me what I can do next (like create a database). But when I do a GET on that URL I get this: {"couchdb":"Welcome","version":"1.0.1"} So now what? The hypermedia doesn't tell me what I can do, so therefore as a REST client I will assume there's nothing I can do. This very simple test shows that the HTTP API for CouchDB isn't really RESTful at all. The question is: should it be? That is obviously up to the developers to decide. But if I were the architect I would maybe say, no, it shouldn't be RESTful. Why? Because I want to allow URL templates to be used, so that the client, given the server URL and a document id, is allowed to construct a URL on its own and GET the document. If this was truly RESTful the client would have to do a query in a form first, with the id, in order to get the URL of the document to be retrieved. That might be inefficient for a database, so I might opt not to do this. Which is, in effect, what they already have done. The only problem is that they call it RESTful, when it isn't, so it gives me as a developer the wrong impression of what I can expect from it. This line of reasoning could be done for pretty much most infrastructure layer API's. They're not RESTful, though many say they are, and most likely they shouldn't try to be! IT'S OK! Just say "Accessible over HTTP, see docs for URL templates and whatnot", and be done with it. Domain layer The next potential layer to be exposed over REST is the domain layer. This typically means that you take your domain entities and expose their data straight on the web, through CRUD operations. Very straightforward. There are tons of articles and blogs that show how to do this. But is it RESTful? Or is it even a good idea in the first place? The first test, again, would be to see if the app follows the hypermedia constraint. In this option it is technically possible to allow queries that will list the various URL's to entities in your domain, which you then can update/delete. So on the surface it might seem like you are following the hypermedia constraint. The problem usually comes with the fact that you are exposing domain state rather than application state. Let me explain through a simple example. Let's say you are building an issue tracker. You can access individual issues through links like: /issue/123 which on GET gives you documents such as: {"status":"OPEN","description":"Some issue"} Awesome. Now a client can change the status to "CLOSED" and PUT that. Tada! Case closed. Or is it? What if a client then decides to reopen it, by simply posting a new status of "OPEN" to it. Ok, that worked. But should it? Maybe your domain model really would have wanted it to only go to "REOPENED" from the "CLOSED" state. But how do you express that? How is the client to know that this is the only valid transition? And what happens when we have many versions of clients, each of which has a slightly different set of rules for what you are allowed to do when? Basically, chaos is ensured. And this is the problem with exposing your domain model using a REST API. The client has to own the application logic, and there's no way the server can be sure that it has the "right" logic. And the client, even if it *wants* to play nice (if code ever wants anything is debatable), will have a hard time knowing whether it is playing by the rules or not. It might even get a bit neurotic, trying to do the right thing, whatever that means. In summary, exposing your domain model does not help the client know what the valid state transitions are, and makes it very hard to do other things like role-based security authorization (maybe only an admin is allowed to REOPEN a CLOSED case?). I would therefore recommend that noone exposes their domain models using a REST API. Application layer Finally we come to the application layer. The application layer is designed to implement usecases of the domain model, and has all the context and logic needed to ensure that only valid state transitions are made. In short, it seems like it is especially appropriate to being exposed through a REST API, as it can at all points tell the client what it can do (either based on state or authorization rules or any other type of rules it might have). If we go back to the issue tracker, what would this mean in practice? It could mean that when you do a GET on /issue/123 you get something like this back: {"data":{"status":"OPEN","description":"Some issue"},"links":[{"close":"/issue/123/close.json"}]} This now instead of referring to viewing the domain state of an issue refers to the usecase of viewing an issue with the intent of working on it. There might be other URL's and other queries that only return the data, or maybe a table of the data, or somesuch. But this one, specifically, refers to the usecase of working with the issue. So, as a REST client I can now inspect the data, and then look at what links are available. If the client has a UI it can enable a button that says "Close issue" based on the available link, since it detected a link relation "close" that it understands. The client can then do GET on that link, find out whether the server expects any form to be filled in, and then submit it using POST, thereby letting the server application layer logic transition the issue to the "CLOSED" state. We are no longer relying on the client to contain the logic of knowing when to allow what, and the client also does not have to know how to construct the URL. As long as it can parse the hypertext (and we might use a custom JSON mediatype to indicate what "data" and "links" mean) and do something with it, we're fine. If we in the future change the domain model to also allow the "resolve" link relation for "OPEN" issues, old clients can ignore it, and new clients can enable new actions in the UI that uses it. In summary, the application layer is a very good candidate to be exposed through a REST API. It encapsulates the application rules for when the various state transitions are allowed, and can make use of user authorization to further enable/disable actions. This takes away a lot of responsibilities from the client, which now also can be "dynamic" in the sense that it can easily react to what state changes are available when by simply checking link availability in the hypermedia returned from the server. The main issue with exposing the application layer through a REST API is that there are pretty much no available frameworks that help you do all this in an easy way. But this is not REST's "fault", obviously, but rather that the "REST" community hasn't yet matured to understand what it should and what it should not do. In the Streamflow project we rolled our own simple framework for doing the above, and I'm very happy with that, but unfortunately most other frameworks seems to be in the "expose your domain model" camp, which means that a lot of this link management is non-trivial to do. This is a fixable situation though. I hope that this post has somewhat clarified what the issues are with exposing infrastructure and domain models through REST API's, and why it's not really a good idea in the first place, and why exposing the application layer really is the logical and simpler option. From http://www.jroller.com/rickard/entry/rest_api_for_infrastructure_domain

When I am writing enumerations I very often found myself implementing a static method similar to the standard enum’s valueOf() but based on field rather than name: public static TestOne valueOfDescription(String description) { for (TestOne v : values()) { if (v.description.equals(description)) { return v; } } throw new IllegalArgumentException( "No enum const " + TestOne.class + "@description." + description); } Where “description” is yet another String field in my enum. And I am not alone. See this article for example. Obviously this method is very ineffective. Every time it is invoked it iterates over all members of the enum. Here is the improved version that uses a cache: private static Map map = null; public static TestTwo valueOfDescription(String description) { synchronized(TestTwo.class) { if (map == null) { map = new HashMap(); for (TestTwo v : values()) { map.put(v.description, v); } } } TestTwo result = map.get(description); if (result == null) { throw new IllegalArgumentException( "No enum const " + TestTwo.class + "@description." + description); } return result; } It is fine if we have only one enum and only one custom field that we use to find the enum value. But if we have 20 enums, and each has 3 such fields, then the code will be very verbose. As I dislike copy/paste programming I have implemented a utility that helps to create such methods. I called this utility class ValueOf. It has 2 public methods: public static , V> T valueOf(Class enumType, String fieldName, V value); which finds the required field in specified enum. It is implemented utilizing reflection and uses a hash table initialized during the first call for better performance. The other overridden valueOf() looks like: public static > T valueOf(Class enumType, Comparable comparable); This method does not cache results, so it iterates over enum members on each invocation. But it is more universal: you can implement comparable as you want, so this method may find enum members using more complicated criteria. Full code with examples and JUnit test case are available here. Conclusions Java Enums provide the ability to locate enum members by name. This article describes a utility that makes it easy to locate enum members by any other field.

It's been almost 4 years since I wrote a blog post called "EasyMock - Pros, Cons, and Best Practices, and a lot has happened since. You don't hear about EasyMock much any more, and Mockito seems to have replaced it in mindshare. And for good reason: it is better. A Good Humane Interface for Stubbing Just like EasyMock, Mockito allows you to chain method calls together to produce less imperative looking code. Here's how you can make a Stub for the canonical Warehouse object: Warehouse mock = Mockito.mock(Warehouse.class); Mockito.when(mock.hasInventory(TALISKER, 50)). thenReturn(true); I know, I like a crazy formatting. Regardless, giving your System Under Test (SUT) indirect input couldn't be easier. There is no big advantage over EasyMock for stubbing behavior and passing a stub off to the SUT. Giving indirect input with mocks and then using standard JUnit asserts afterwards is simple with both tools, and both support the standard Hamcrest matchers. Class (not just Interface) Mocks Mockito allows you to mock out classes as well as interfaces. I know the EasyMock ClassExtensions allowed you to do this as well, but it is a little nicer to have it all in one package with Mockito. Supports Test Spies, not just Mocks There is a difference between spies and mocks. Stubs allow you to give indirect input to a test (the values are read but never written), Spies allow you to gather indirect output from a test (the mock is written to and verified, but does not give the test input), and Mocks are both (your object gives indirect input to your test through Stubbing and gathers indirect output through spying). The difference is illustrated between two code examples. In EasyMock, you only have mocks. You must set all input and output expectations before running the test, then verify afterwards. // arrange Warehouse mock = EasyMock.createMock(Warehouse.class); EasyMock.expect( mock.hasInventory(TALISKER, 50)). andReturn(true).once(); EasyMock.expect( mock.remove(TALISKER, 50)). andReturn(true).once(); EasyMock.replay(mock); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert EasyMock.verify(mock); That's a lot of code, and not all of it is needed. The arrange section is setting up a stub (the warehouse has inventory) and setting up a mock expectation (the remove method will be called later). The assertion in all this is actually the little verify() method at the end. The main point of this test is that remove() was called, but that information is buried in a nest of expectations. Mockito improves on this by throwing out both the record/playback mode and a generic verify() method. It is shorter and clearer this way: // arrange Warehouse mock = Mockito.mock(Warehouse.class); Mockito.when(mock.hasInventory(TALISKER, 50)). thenReturn(true); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert Mockito.verify(warehouse).remove(TALISKER, 50); The verify step with Mockito is spying on the results of the test, not recording and verifying. Less code and a clearer picture of what really is expected. Update: There is a separate Spy API you can use in Mockito as well: http://mockito.googlecode.com/svn/branches/1.8.3/javadoc/org/mockito/Mockito.html#13 Better Void Method Handling Mockito handles void methods better than EasyMock. The fluent API works fine with a void method, but in EasyMock there were some special methods you had to write. First, the Mockito code is fairly simple to read: // arrange Warehouse mock = Mockito.mock(Warehouse.class); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert Mockito.verify(warehouse).remove(TALISKER, 50); Here is the same in EasyMock. Not as good: // arrange Warehouse mock = EasyMock.createMock(Warehouse.class); mock.remove(TALISKER, 50); EasyMock.expectLastMethodCall().once(); EasyMock.replay(mock); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert EasyMock.verify(mock); Mock Object Organization Patterns Both Mockito and EasyMock suffer from difficult maintenance. What I said in my original EasyMock post holds true for Mockito: The method chaining style interface is easy to write, but I find it difficult to read. When a test other than the one I'm working on fails, it's often very difficult to determine what exactly is going on. I end up having to examine the production code and the test expectation code to diagnose the issue. Hand-rolled mock objects are much easier to diagnose when something breaks... This problem is especially nasty after refactoring expectation code to reduce duplication. For the life of me, I cannot follow expectation code that has been refactored into shared methods. Now, four years later, I have a solution that works well for me. With a little care you can make your mocks reusable, maintainable, and readable. This approach was battle tested over many months in an Enterprise Environment(tm). Create a private static method the first time you need a mock. Any important data needs to be passed in as a parameter. Using constants or "magic" fields hides important information and obfuscates tests. For example: User user = createMockUser("userID", "name"); ... assertEquals("userID", result.id()); assertEquals("name", result.name(); Everything important is visible and in the test, nothing important is hidden. You need to completely hide the replay state behind this factory method if you're still on EasyMock. The Mock framework in use is an implementation detail and try not to let it leak. Next, as your dependencies grow, be sure to always pass them in as factory method parameters. If you need a User and a Role object, then don't create one method that creates both mocks. One method instantiates one object, otherwise it is a parameter and compose your mock objects in the test method: User user = createMockUser( "userID", "name", createMockRole("role1"), createMockRole("role2") ); When each object type has a factory method, then it makes it much easier to compose the different types of objects together. Reuse. But you can only reuse the methods when they are simple and with few dependencies, otherwise they become too specific and difficult to understand. The first time you need to reuse one of these methods, then move the method to a utility class called "*Mocker", like UserMocker or RoleMocker. Follow a naming convention so that they are always easy to find. If you remembered to make the private factory methods static then moving them should be very simple. Your client code ends up looking like this, but you can use static imports to fix that: User user = UserMocker.createMockUser( "userID", "name", RoleMocker.createMockRole("role1"), RoleMocker.createMockRole("role2") ); User overloaded methods liberally. Don't create one giant method with every possible parameter in the parameter list. There are good reasons to avoid overloading in production, but this is test. Use overloading so that the test methods only display data relevant to that test and nothing more. Using Varargs can also help keep a clean test. Lastly, don't use constants. Constants hide the important information out of sight, at the top of the file where you can't see it or in a Mocker class. It's OK to use constants within the test case, but don't define constants in the Mockers, it just hides relevant information and makes the test harder to read later. Avoid Abstract Test Cases Managing mock objects within abstract test cases has been very difficult for me, especially when managing replay and record states. I've given up mixing mock objects and abstract TestCase objects. When something breaks it simply takes too long to diagnose. An alternative is to create custom assertion methods that can be reused. Beyond that, I've given up on Abstract TestCase objects anyway, on the grounds of preferring composition of inheritance. Don't Replace Asserts with Verify My original comments about EasyMock are still relevant for Mockito: The easiest methods to understand and test are methods that perform some sort of work. You run the method and then use asserts to make sure everything worked. In contrast, mock objects make it easy to test delegation, which is when some object other than the SUT is doing work. Delegation means the method's purpose is to produce a side-effect, not actually perform work. Side-effect code is sometimes needed, but often more difficult to understand and debug. In fact, some languages don't even allow it! If you're test code contains assert methods then you have a good test. If you're code doesn't contain asserts, and instead contains a long list of verify() calls, then you're relying on side effects. This is a unit-test bad smell, especially if there are several objects than need to be verified. Verifying several objects at the end of a unit test is like saying, "My test method needs to do several things: x, y, and z." The charter and responsibility of the method is no longer clear. This is a candidate for refactoring. No More All or Nothing Testing Mockito's verify() methods are much more flexible than EasyMock's. You can verify that only one or two methods on the mock were called, while EasyMock had just one coarse verify() method. With EasyMock I ended up littering the code with meaningless expectations, but not so in Mockito. This alone is reason enough to switch. Failure: Expected X received X For the most part, Mockito error messages are better than EasyMock's. However, you still sometimes see a failure that reads "Failure. Got X Expected X." Basically, this means that your toString() methods produce the same results but equals() does not. Every user who starts out gets confused by this message at some point. Be Warned. Don't Stop Handrolling Mocks Don't throw out hand-rolled mock objects. They have their place. Subclass and Override is a very useful technique for creating a testing seam, use it. Learn to Write an ArgumentMatcher Learn to write an ArgumentMatcher. There is a learning curve but it's over quickly. This post is long enough, so I won't give an example. That's it. See you again in 4 years when the next framework comes out! From http://hamletdarcy.blogspot.com/2010/09/mockito-pros-cons-and-best-practices.html

The Separated Interface pattern can often be used to provide hook points to client code, in the form of interfaces to implement or classes to extend with client code. The right implementation to use in a part of the system can then be chosen via configuration: the Factory or Dependency Injection container with the largest scope would process the configuration and execute conditionals only one time, and inject the right Plugin as a collaborator of a standard object. This pattern is a evolution of the Separated Interface one, where the implementor package is not even under your maintenance, but it is provided by some external developer that links his code to your work. Implementation In PHP the concept of compile time does not exist, apart from the just-in-time cached compilation of the scripts to operation codes, a phrase which you can peacefully ignore if you are not into caching. By the way, even if some checks are performed while loading and parsing the PHP code, PHP is by design a dynamic language where you can write nearly everything and it will not explode until executed. This design leaves open many possibilities for inserting plugins, but due to the lack of compile there is often a lack of a clean separation between code and configuration. For example, database credentials are embedded in PHP code more often than in other languages. Think now of a framework or a library: you cannot change the code but you must adapt or create a configuration to make it work. To implement a Plugin pattern, your application should strive towards the flexibility of a library: think of your production code as external and untouchable, and try to deploy a particular configuration to make it work and to modify a functionality. For example, extract it in a temporary working copy with svn checkout or git clone and hook in the necessary extensions. When you succeed, and your svn diff or git diff is clean, you'll have implemented a Plugin system. Modification of vendor code (and you are the vendor here) is out of the question. Future changes Kent Beck says in Implementation Patterns that providing hooks via implementation and inheritance is one of the most effective ways to tie a framework down from future evolution. For example, once you have published an interface, you cannot add methods to it without breaking all the implementors. You can publish versioned interfaces, but this adds complexity to your application. With a published abstract class instead, you can include a default implementation for new methods, but you can't remove methods or refactor protected members without breaking Plugin implementators. This is the specular situation of providing an interface. Zend Framework includes both an interface and an abstract class for most of its components, but it does not get right the management of extension points (at least in the 1.x branch). When including the possibility of Plugins in your application, default as much as possible to private visibility and hide the internals of your Plugin hook point. What is left to protected is a seam that screams "extend me", and the interfaces not marked as internal will be implemented by someone else. There is no built-in language mechanism to protect interfaces,m so you'll have to rely on some kind of convention (like a particular prefix or namespace), but for private methods left to protected scope we can only blame ourselves. Configuration The configuration of your Plugin system can be managed with solution of different levels of complexity, each more powerful than the previous ones. Of course, you shouldn't provide a needlessly complex system when all you need is a class name. The first solution is indeed to insert class names into configuration files. This is a totally declarative approach, which uses simple INI files. This is commonly done in Zend Framework, for example with bootstrap resources, and in some cases can even manage dependencies of the Plugins. Bootstrap resources can request other object of the same kind, but cannot pull in arbitrary collaborators (unless they create them by themselves... ugly if you know what DI is). A second, widely applicable solution is to request Factory objects. this solution still involves writing PHP code, but it is one step towards textual configuration. However, a Factory object can fetch and inject all the dependencies into a Plugin without cluttering it with this kind glue code (only a constructor or some setters). The problem with Factories is that they tend to contain all the same boilerplate code. A third solution can be used to provide quick construction of objects: Dependency Injection containers, which have recently been introduced even in PHP. A DI container is configured textually, via an XML or INI file containing parameters like the collaborators each object requires, its lifetime, and so on. DI containers are probably the future of flexible PHP applications, but beware of growing too dependent on them: they are a library like every other open source component, and should be isolated from your code as much as possible like you would do with your models and Doctrine 2, or your services and Zend Framework. Example The code sample shows hot to predispose a class for receiving an injected simple Factory that manages user-defined plugins. // plugin_view.php formatDate(time()), ".\n"; factory = $factory; } public function render($script) { include $script; } /** * Forwards the call to the View Helper invoked. */ public function __call($name, $args) { $callback = $this->factory->getHelper($name); return call_user_func_array($callback, $args); } } /** * Extension code. */ class UserDefinedFactory implements ViewHelperFactory { private $helpers; /** * In this example, we only define a simple Plugin for * formatting dates using PHP's internal function. */ public function __construct() { $this->helpers = array( 'formatDate' => function($time) { return date('Y-m-d', $time); } ); } public function getHelper($name) { return $this->helpers[$name]; } } // client code $view = new View(new UserDefinedFactory); $view->render('plugin_view.php');

The Special Case pattern is a very simple base pattern that describes a subclass representing, as the name suggests, a special case of the computation made by your program. Don't think that the technical simplicity of the solution means that this pattern is very diffused. If vs. polymorphism The idea of the pattern is to implements two classes with the same interface or base superclass, and rely on polymorphism to target the special case, instead that inserting if and switch statements in the original class. The extracted piece of functionality can be a method to override (specialization in the Template Method pattern) or an independent collaborator injected into the client code (Strategy pattern and many others). Dispatching a method call instead of inserting if statements is simpler to read and understand, as the code of the class has a lower cyclomatic complexity overall (few possible execution paths). If you ever tried to debug Doctrine 1 or a similar piece of software where the methods contain many nested ifs, you have been probably forced to insert echo statements to reveal the actually executed path, even when a single, isolated unit test was exercising the code. The alternative to some ifs is to introduce a Special Case. A rule of thumb for discovering if the substitution is possible is to check if the condition of the if is based on a state that is longer lived with respect to the parameters of the method that it resides in. Ifs that depend only on the state of the object fields or on collaborators are the simplest to replace. Null Object The Null Object pattern is a specialized version of Special Case (what a pun), and probably the most famous one. Instead of returning false or null when a computation fails to provide a result, you return an object that as a matter of fact, does nothing: an User subclass AnonymousUser with authorize() that always return false an empty array (it can be thought of as a Null Object, even if it is a primitive value) an empty ArrayObject. When a Null Object is returned, it effectively removes the checks for the null value or empty result from the client code. The client class object can call methods on the return value without worrying (calling methods on NULL is a fatal error in PHP and would crash a test suite). Or it can execute foreach() over a returned array and skip the cycle altogether if the array is empty. Since null can be dispatched, we may in fact use it as a Test Double in our test code to ensure a collaborator is never called in a particular scenario. If you have a method that shouldn't refer to a collaborator, you can inject null in the constructor or via a setter. PHP is different from Java in the type hinting behavior: in Java you can pass null to this constructor: public MyClass(Collaborator c) { ... while in PHP you have to resort to this: public function __construct(Collaborator $c = null) { ... Implementation The Special Case can be a Flyweight or an object with, since it has usually no internal state: the behavior depending on state is encapsulate in the code itself. You can also have more than one Special Case for each superclass or interface: Fowler makes the example of a MissingCustomer and AnonymousCustomer as special cases for the Customer class. By the way, every method of a Null Object should return a plain scalar value or another Special Case object. Note that with more than one level of Special Case objects, you may be violating the Law of Demeter: your client code access the first Special Case and then the other contained one, navigating the object graph instead of asking for its dependencies or sending a message. Examples In this example we apply the pattern to go from this situation: type = $type; } /** * This if() is based only on the object state * and can probably be modelled differently. * You'll need two tests for this method. */ public function accelerate() { if ($this->type == 'Ferrari') { $this->speed += 2; } else { $this->speed++; } } public function brake() { $this->speed--; } public function __toString() { return $this->type; } } // client code $car = new Car('Fiat'); $ferrari = new Car('Ferrari'); $car->accelerate(); $ferrari->accelerate(); var_dump($car, $ferrari); to this one: speed--; } public function __toString() { return $this->type; } } /** * One Special Case: a car with $type parametrized in the constructor * and ordinary acceleration properties. */ class OrdinaryCar extends Car { public function __construct($type) { $this->type = $type; } public function accelerate() { $this->speed++; } } /** * Another Special Case: a car with fixed $type and greater acceleration. */ class FerrariCar extends Car { /** * This is state encapsulate in code: you don't have to set up * it in tests or with configuration, only to instantiate this class. */ protected $type = 'Ferrari'; public function accelerate() { $this->speed += 2; } } // client code $car = new OrdinaryCar('Fiat'); $ferrari = new FerrariCar(); $car->accelerate(); $ferrari->accelerate(); var_dump($car, $ferrari);

Fowler's definition for the Registry pattern is this one: A well known object that other ones can use to find related objects or service. This vague definition leaves open the possibility of abuse. Implementation The idea of a Registry is simple: providing a dynamic possibility for discovering collaborator objects, so that we not hardcode static calls to global objects like Singletons in each of them. In the testing environment, we can fill the Registry with mocks. However, the problem is that then we hardcode static calls to the Registry, which is an effective sinkhole for dependencies in the case of a general purpose implementation. You can't limit which objects of a Registry are accessed by a client, so depending on the Registry may mean depending on each stored component. In this scenario, the Registry becomes a Service Locator, the poor man's version of Dependency Injection. Fixing it My suggestion before implementing this pattern is thinking about how many of the registered objects would the ordinary client object actually need; maybe you can inject them directly. A constraint I'd like to set for Registry implementations is that all the contained objects should be of the same type (or of a Layer Supertype). With this limitation, incarnations of the Registry become very useful as they can be injected and passed around as a virtual collection of object, even if they do not exist/currently are in persistence/are not all in memory at the same time. If this reminds you of the Repository, you're right; the Repository pattern is a specialization of the Registry one, with this limitation written with blood in its contract in a night of full moon. This is what a Registry is supposed to be: not an hidden blackboard like Zend_Registry where you can practice accumulate and fire, and setup time-ticking bombs that will explode later. Here's an example from our test suite, which used Zend_Registry to set up the locale. One day, uncommenting a test would make another one fail (it used Zend_Registry, and the uncommented one messed with some keys). Fixing it takes some minutes (but it should have taken seconds). Scale that to a test suite with one thousands tests and you can easily destroy your tests isolation. Shortly, one test would pass when executed alone, but explode when the whole test suite (which maybe takes 15 minutes to run) because a previous test, whose identity you do not know, modified some global state. Good luck fixing that and grepping for 'Zend_Registry::'. Instead, a Registry can be a first-class citizen, an object that can be injected, by itself or hidden behind an interface of a composer, into any client object that sincerely needs access to the whole encapsulated collection. Why you now feel a generic Registry could be handy Lifecycle problems can create the need for a generic Registry. How do you access an object A from a client object B if A do not exist when B is created? Use a Registry. Of course, this solution does not go a long way: you're never sure that A would be created in time, and you should revise your design to discover why a long-lived object should hold a reference to a short-lived one. In fact, higher-level or injected Factories ara a similar, but more powerful solution. If B needs A, and A has the same lifecycle, simply inject it via a request-wide Factory. If B and A have a shorter lifecycle than the one of the request (for example they are view helpers that may not be used at all in certain requests, so they are instantiated only when needed in the presentation layer), provide a Factory that craates both and inject the Factory in the right scope. However, never inject a generic registry: it is a false solution. The Registry becomes a Service Locator, which is depended on by every piece of your application, and depends again on every piece of it. This is a benignuos form of what I call "hiding dependencies under the carpet", which can be solved by injecting the direct dependency instead of a provider. Examples Our code sample is the infamous Zend_Registry component from Zend Framework 1. It is a Singleton, and when testing controllers it is a typical resource that the testing harness must remember to reset between each test. My comments are in italic. offsetExists($index)) { require_once 'Zend/Exception.php'; throw new Zend_Exception("No entry is registered for key '$index'"); } return $instance->offsetGet($index); } /** * setter method, basically same as offsetSet(). * * This method can be called from an object of type Zend_Registry, or it * can be called statically. In the latter case, it uses the default * static instance stored in the class. * * @param string $index The location in the ArrayObject in which to store * the value. * @param mixed $value The object to store in the ArrayObject. * @return void */ public static function set($index, $value) { $instance = self::getInstance(); $instance->offsetSet($index, $value); } /** * Returns TRUE if the $index is a named value in the registry, * or FALSE if $index was not found in the registry. * * @param string $index * @return boolean */ public static function isRegistered($index) { if (self::$_registry === null) { return false; } return self::$_registry->offsetExists($index); } /** * Constructs a parent ArrayObject with default * ARRAY_AS_PROPS to allow acces as an object * * @param array $array data array * @param integer $flags ArrayObject flags */ public function __construct($array = array(), $flags = parent::ARRAY_AS_PROPS) { parent::__construct($array, $flags); } }

a few days back, i was browsing the web and found an interesting open source framework called checkthread . it is a static analysis tool for catching java concurrency bugs at compile time. static analysis tools are used to find out programming error in the code by analyzing their byte code. to me a tool aiming to catch concurrency bugs at compile time was worth spending time on. so, i decided to play with checkthread to find out its capabilities. checkthread requires developers to specify the thread policy in either xml or annotations. thread policy defines whether the piece of code (i.e. a method) is thread safe, not thread safe, or thread confined. thread confined means that this method is confined to a specific runtime thread. for example, the swing api must be invoked on the event-dispatch thread. prerequisite before you start: download eclipse plugin . put plugin in jar in eclipse plugins folder and restart eclipse. for more information refer here . download checkthread annotation jar . this jar is not present in any maven repository, so manually install the jar in your maven repository. mvn install:install-file -dfile=checkthread-annotations-1.0.9.jar -dgroupid=org.checkthread -dartifactid=checkthread-annotations -dversion=1.0.9 -dpackaging=jar checkthread capabilities let's look at an example: public class threadsafetyexample { final map helpermap = new hashmap(); public void addelementtomap() { helpermap.put("name", "shekhar"); } } this is a simple class which puts a key value pair in a map. is this code thread safe? no. lets test this class using multiple threads. in the unit test i am using countdownlatch for producing maximum parallelism. i talked about countdownlatch in an earlier article . @test public void regressiontest() throws exception{ for (int i = 1; i <= 100; i++) { system.out.println("runing "+i); addelementtomapwhenaccessedbymultiplethread(); } } public void addelementtomapwhenaccessedbymultiplethread() throws exception { final countdownlatch latch = new countdownlatch(1); final threadsafetyexample helper = new threadsafetyexample(); class mythread extends thread { @override public void run() { try { latch.await(); } catch (interruptedexception e) { } helper.addelementtomap(); } } int threadcount = 2000; mythread[] mythreads = new mythread[threadcount]; for (int i = 0; i < mythreads.length; i++) { mythreads[i] = new mythread(); mythreads[i].start(); } latch.countdown(); for (mythread mythread : mythreads) { mythread.join(); } assertequals(1, helper.helpermap.size()); } i am calling the method addelementtomapwhenaccessedbymultiplethread in a for loop because if you run the test only once it might not fail (thread timing). so, how can these types of errors be detected at compile time if most of the unit-tests and integration tests that we write do not test the code in multi-threaded environments? checkthread can help you out in such situations. checkthread can only help you if you annotate your method with threadsafe annotation. for example, lets apply @threadsafe annotation to the threadsafetyexample class: public class threadsafetyexample { final map helpermap = new hashmap(); @threadsafe public void addelementtomap() { helpermap.put("name", "shekhar"); } } now when you run the checkthread by pressing the button you will see a compile time error as shown below: as you can see in the above image, the tool shows the code where problems exist and descriptions of errors in the problems section. this can come in handy while writing multi threaded code. you just need to think about your contract, whether the method you have written should be thread safe or not. this was just a simple use case but it can also help you detect race conditions . have a look at this tool, maybe it can help you detect concurrency bugs.

Parameterized types - the <> expressions that can be used in Java as of JDK 5 are not just for collections. I find myself frequently using them in APIs I design. They really do let you write things which are more generic in the non-Java sense of the word - and the result is more reusable code, which means less code overall, which means fewer bugs and things to test. The verbosity, and some of the weirdness of type-erasure are less than ideal, but used right, the benefits are worth the complexity. The standard (and somewhere recommended) naming convention for parameterized types is to use a single-letter name. That works fine in signatures that have only one such type. But in practice, single-letter names make code less self-describing, and if you're defining a class with more than one parameterized type, it can be confusing and hard to read. People other than me will have to call, understand and maintain my code - the more self-describing I can make it, the better. So I am looking for a naming convention that makes it obvious that something is a parameterized type, but allows for descriptive names. I am wondering if anybody else has run into this problem, and if there is any emerging consensus on naming generics. Do you work on a project that uses generics a lot? If so, what do you do? Here's an example. At the moment, I'm writing a generic (in both senses) class which simply limits the number of threads which can access some resource. It's basically a wrapper around a Semaphore which uses a Runnable-like object to ensure that the Semaphore is accessed correctly, and does some non-blocking statistic gathering about thread contention. So to access the scarce resource, you pass in a ResourceAccessor: public interface ResourceAccessor { public Result run (ProtectedResource resource, Argument argument); } The problem is that, when somebody looks at this interface, they will instantly get the idea that there are really classes they need to go find, which are called ProtectedResource, Argument and Result - and of course, no such classes exist - these are just names for generic types. The standard-naming-convention is worse: public interface ResourceAccessor { public S run (T resource, R argument); } Here, nobody could possibly figure out what on earth this class is for without extensive documentation - this is a really horrible idea. So I've concluded that the standard recommendations for generic type names are simply wrong for any non-trivial usage (I.e. Collection is fine, since there is one type and Collections are well-understood). You simply can't do this on a non-collection code structure you have invented, or people will just be confused and not use it. The best suggestion I've heard thus far is using $ as a prefix: public interface ResourceAccessor <$ProtectedResource, $Argument, $Result> { public $Result run ($ProtectedResource resource, $Argument argument); } I don't find this pretty, but I don't have any better ideas, and at least it makes it crystal-clear that there is something different about these names. Any thoughts? What do you do in this situation?

Sometimes checked exceptions can be a problem. For instance, recently I tried to implement some common logic to retry failing network operations and it resulted in a kind of command pattern on which, as usual, the execute() method throws java.lang.Exception. That complicated the caller code which has to catch and handle java.lang.Exception instead of the more specific exceptions... I knew that checked exceptions are enforced by the compiler, while in the virtual machine there is nothing preventing a checked exception to be thrown by a method not declaring it, so I started to check on internet how to implement this. I found two posts on Anders Noras's blog (#1 #2) on how to perform this magic. Method #1: the sun.misc.Unsafe class import java.lang.reflect.Field; import sun.misc.Unsafe; public class UnsafeSample { public void methodWithNoDeclaredExceptions( ) { Unsafe unsafe = getUnsafe(); unsafe.throwException( new Exception( "this should be checked" ) ); } private Unsafe getUnsafe() { try { Field field = Unsafe.class.getDeclaredField("theUnsafe"); field.setAccessible(true); return (Unsafe) field.get(null); } catch(Exception e) { throw new RuntimeException(e); } } public static void main( String[] args ) { new UnsafeSample().methodWithNoDeclaredExceptions(); } } This makes use of internal Sun JRE libraries implementation classes. It could not work if you use a non Sun VM. And in fact it doesn't if you use GCJ (The GNU compiler for Java). The getUnsafe() method exposed above does some tricks to access a private field in the Unsafe class, because Unsafe.getUnsafe() can only be called by classes loaded by the bootstrap ClassLoader. See also the article Avoiding Checked Exceptions by Don Schwarz. Method #2: the Thread.stop(Exception) public class ThreadStopExample { @SuppressWarnings("deprecation") public void methodWithNoDeclaredExceptions( ) { Thread.currentThread().stop(new Exception( "this should be checked" )); } public static void main( String[] args ) { new ThreadStopExample().methodWithNoDeclaredExceptions(); } } This uses a deprecated method, but works. No portability issue, until the Java specification guys decide to remove the method. It could have some side effects on the current thread as we are calling stop(). I'm not sure. Method #3: using Class.newInstance() Look at the signature of java.lang.Class.newInstance() and compare it to Constructor.newInstance() public final class Class ... { public T newInstance() throws InstantiationException, IllegalAccessException } public final class Constructor ... { public T newInstance(Object ... initargs) throws InstantiationException, IllegalAccessException, IllegalArgumentException, InvocationTargetException } You see it? no InvocationTargetException! If you call SomeObject.class.newInstance() and the constructor throws an exception, the exception doesn't get wrapped into the InvocationTargetException (that is a checked exception). So you can write an utility class like this, to throw checked exceptions without needing to declare them on the method signature. public class Exceptions { private static Throwable throwable; private Exceptions() throws Throwable { throw throwable; } public static synchronized void spit(Throwable throwable) { Exceptions.throwable = throwable; try { Exceptions.class.newInstance(); } catch(InstantiationException e) { } catch(IllegalAccessException e) { } finally { Exceptions.throwable = null; } } } public class TestExceptionSpit { public static void main(String[] args) { Exceptions.spit(new Exception( "this should be checked" )); } } Internally the Class.newInstance() uses the sun.misc.Unsafe class, but in this case this technique is fully portable because you are not using any deprecated or internal method. In fact it works also with GCJ JVM. I tried to remove the synchronization stuff and the static field using an inner class, but it seems that the compiler does some strange trick translating the empty constructor in something else preventing Class.newInstace() to be used on that inner class. The behavior of the Class.newInstance() is also documented: "Note that this method propagates any exception thrown by the nullary constructor, including a checked exception. Use of this method effectively bypasses the compile-time exception checking that would otherwise be performed by the compiler." So your code is fully safe and compliant to the rules :) Method #4: the sun.corba.Bridge import java.rmi.RemoteException; public class Bridge { public void methodWithNoDeclaredExceptions( ) { sun.corba.Bridge.get().throwException(new RemoteException("bang!")); } public static void main( String[] args ) { new Bridge().methodWithNoDeclaredExceptions(); } } This is more or less the same as using the Unsafe.class. The difference is that in this case you don't need to do the reflection stuff to access the private field "theUnsafe", because the Bridge class is doing that for you. Still using an internal JRE class with same portability issues. Method #5: Generics The following example takes advantage of the fact that the compiler does not type check generics... import java.rmi.RemoteException; class Thrower { public static void spit(final Throwable exception) { class EvilThrower { @SuppressWarnings("unchecked") private void sneakyThrow(Throwable exception) throws T { throw (T) exception; } } new EvilThrower().sneakyThrow(exception); } } public class ThrowerSample { public static void main( String[] args ) { Thrower.spit(new RemoteException("go unchecked!")); } } Credits to "Harald" that posted a comment on Johannes Brodwall's blog. I personally think this last one is the best solution: it uses a feature of the compiler against itself. Conclusions I think that having checked exception in Java is better than not having it. I already expressed why I am in favor of checked exceptions here. It's a design decision, you can choose to make your exceptions checked or unchecked, if you want to force your client to handle them or not; you can't do that on .NET, where checked exceptions simply do not exist. Sometimes you have (or you have to write) methods throwing java.lang.Exception, and you get into the trap. So you may like to know that there is a dirty escape, and you can decide to use it or not... we saw that Sun is throwing undeclared checked exceptions in Class.newInstace(), ask yourself: if this is good for the JRE code, could it be good also for yours? Usually you can wrap checked exception into RuntimeExceptions but this doesn't simplify the client code, because the caller in case of needing has to catch the RuntimeException, unwrap the cause and deal with it. Maybe a new Java keyword to throw checked exception without requiring the caller to handle them could help: I recommend reading post on Ricky Clarkson's about checked exceptions. Finally I come to the decision to not use those tricks in my object doing the retry logic, and keep the messy catch logic on the caller code. In case of needing I will evaluate to use a Dynamic Proxy doing the retry logic and keeping its behavior transparent to the client. To those who wants unchecked exceptions in Java... well, there is the way to have it: the example with Generics is a clean way to have it. Use it if you want, at your own risk. Personally I would choose to use libraries with checked exceptions... Other related articles Friday Free Stuff by Chris Nokleberg, uses bytecode manipulation. Don't Try This at Home by Bob Lee, exposes some methods also covered above. From: http://en.newinstance.it/2008/11/17/throwing-undeclared-checked-exceptions/

A fundamental trait of modern software is that it does not live in isolation, especially in the realm of web applications, which can easily interact with external resources like web services and databases. The majority of PHP applications must access external resources, that by architecture do not run in the same memory segment or programming language of their core Domain Model. There are many examples of these situations: web services like Google's or Yahoo! ones. Relational and NoSQL databases. The filesystem of the server. Other web and non-web applications for data interoperability. I'll call any instance of this external dependency a resource, which is an umbrella term for each item of this list. Motivation When you have to access an external resource, you get an API which you code may call. However accessing an API directly, like a PDO object or a HTTP request stream, presents many issues. First of all, your application ends up becoming very coupled to the particular product or application instance you're using. There is no room for change, since every resource has its specific API, unless it is a commodity like a relational database. More subtly, general purpose APIs are designed as catch-all interfaces for providing any functionality, and capturing any use case from every possible client. The entire set of methods becomes a possible requirement of your application, since you cannot instantly easily distinguish the primitives really called by your application from the one ignored. Moreover, the external resource may use data formats and models different from the ones used by your application. This is the case with relational database used as a storage for object models. Implementation There is an easy solution to these interaction problems, which I feel is never pushed enough. The Gateway pattern is this solution: wrap into a single object all the interaction specifical to the integrated resource, so that your object provides a specialized API of exactly what you want, as you want. This pattern is similar to the Facade classic one, but it is applied on other people's code instead of our own. You can also compare it to an Adapter, when the Adaptee is not even object-oriented or in the same process of your application's code. By the way, this pattern is specialized by many other ones, and it can be thought of as their superclass. Wrapping Wrapping is the mechanism used for this pattern's implementation. Only the functionality needed is really exposed from the Gateway. This minimalism help the Gateway in becoming the target of integration tests or pragmatic unit tests that exercise only the functionalities actually exposed and that may cause a regression. This pattern insulate the application layer or the Domain Model from external changes. The Hexagonal Architecture is really an evolution of this pattern applied systematically to every external resource, until only an in-memory object structure stands as the core domain, and every dependency is injected as an adapter for an application's port. A Gateway can also be implemented with more than one object (back end and front end) when the work to do is both on the protocol side (procedural vs. oo, XML vs. variables) and at the workflow side (different slicing of functionalities, APIs at the wrong level of abstraction fro your use case). Advantages I'll never get done with talking of the advantage of introducing a Gateway over an external dependency. You achieve greater insulation over the dependency: changes do not spread into your system and you can test them separately and efficiently. The system is also easier to read and understand as it does not pull in the whole complexity of the resource, but only the abstraction needed by client code. Disadvantages There's hardly any downside in coding up a Gateway class, unless you introduce a leaky abstraction. Peculiarity According to Fowler, this pattern is somewhat different from the other integration-related ones, and due to these differences it has earned a name and an article here. A Facade simplifies a complex API, and it is written by the developers of the resource used. A Gateway is written by the client code developers to simplify their own job. The Facade also implies a different interface, while Gateway can simply wrap it and transform it or hiding part of it. An Adapter alters an implementation to provide a new API. With a Gateway there may not be an existing interface, or if there is, the Adapter is part of the Gateway implementation, which comprehends a back end side. A Mediator separates different objects, but Gateway is much more specialized in separating two objects and keeping the dependency side (the external resource) not aware of being used. Example Today's example is a Gateway to a web service, in the form of the classic Twitter client. For simplicity and readability we'll deal only with a single operations that does not require authentication, badly implemented with OAuth by Twitter at the time of this writing. status->text; } } // having an object to represent Twitter means we can mock it, // pass it around, injecting it, composing it... $gateway = new TwitterGateway(); // client code echo $gateway->getLastTweet('giorgiosironi'), "\n";

Automated testing supports your constant effort in design and refactoring, and besides that ensures that your application actually works in a reliable and repeatable way. Tests at every level of detail are a form of executable specification and documentation. They give you immediate feedback and confidence that your code works, plus a satisfying green bar many times a day. I've been consulting on a Zend Framework application, with the goal of repairing the test suite and expanding it. In this article I'll describe the different categories of testing, as applied to a Zend Framework 1 application, but this classification pertains to every web application based on object-oriented programming. Since this kind of applications is obviously PHP-based, PHPUnit will be the tool of choice along with some of its standard extensions. For a panoramic of PHPUnit and its features, feel free to download my free ebook on the subject, which condenses much of the technical informations about it to a mere 50 pages. Let's start with the most debated and simple kind of testing - the one at the unit level. Unit testing Each unit tests target a unit of code in isolation - usually a class, and thus one or more objects instantiated from this class. The isolation property is what defines a unit test: its code must not have dependencies on other classes than the one under test, since they should be tested independently, by their own test classes. Since PHPUnit models a test case for a production code class as another class extending PHPUnit_Framework_TestCase, implementing unit testing leads very often to a parallel hierarchy of classes, where every Foo_Bar class has a corresponding Foo_BarTest test case. Given these premises, a unit test that fails tells you immediately where the error is: in the class it exercises. Moreover, it will be very fast to execute, since it works on only a single object at the time. Unit test should target mostly your models, and any code written by you that is not framework-specified: these would also be the classes that contain the majority of the business logic, and the most interesting to test. This code is usually composed of Plain Old PHP Objects and of subclasses of framework or library base classes when when they leave no other choice for integration. For writing unit tests, usually no external library other than PHPUnit is necessary. In a Zend Framework application you can usually reuse the bootstrap files, which set up things like autoloading, in the phpunit --bootstrap option or by defining it in the phpunit.xml configuration file. This way it will be executed only once for each test suite run. I prefer to leave initialization of the single components to test in the test cases itself, to ensure maximum isolation. However, a simpler and standard solution is to just run the whole Bootstrap class, with a custom configuration (application/config/application.ini), which 'testing' environment section is created by default by Zend_Tool. Pragmatic unit testing That's not a standard name. In some cases, you should also be pragmatic: you cannot usually mock all the external resources, nor you should since mocking a contract which you can't change can lead you to madness. You should configure a lightweight version of your dependencies and test with them. For example, if you're using the Doctrine Object-Relational Mapper, you must test the interaction with the database somewhere, and mocking the whole Doctrine infrastructure will be prohibitive and unuseful. The standard practice here is to use the real Doctrine infrastructure to test database-coupled classes, like Repositories and Data Access Objects, but to instantiate a lightweight database like an sqlite in-memory one which is much faster in its operations than a production one. This database can then be discarded or truncated at the end of each test to ensure no global state is shared between test cases. The downside in this approach is that sqlite is not the real database; one time I was testing with it and due to a bug (feature?) in Doctrine 1 the code failed in MySQL while passing with Sqlite. The reason was sqlite does not support foreign key constraints and was simply ignoring them, while MySQL correctly throwed exceptions when they were violated. Moreover, these tests are never fast as the ones totally isolated from external libraries. The upside is that the tests for classes interacting with the database via Doctrine or another ORM still have the benefit of the unit level: when the test fail, it is clear that the related production code class has encountered a regression, because the ORM code is only imported in discrete, distant points of time, when the test suite is green, and so could never change while you're expanding your code. Nevertheless this kind of testing should be applied only to the adapters of your application, which constitute the boundary of the object graph towards external components like databases, web services or the filesystem. Functional testing Functional testing's goal is to exercise a medium-sized object graph, without instantiating the whole application, to a cover a full functionality and make sure the classes adhere to the same contract. For example, these tests can target a service layer built upon your Domain Model, if you want to enhance to cover your factories or DI mechanisms. In other cases, they can target the controllers: this happens when you have supplemental logic on the client side. In case of functional testing on plain old classes, PHPUnit suffices again. In case you target controllers instead, the Zend_Test component gives you a Zend_Test_PHPUnit_ControllerTestCase class which you can extend to gain helper functionalities. Basically, every test method of a Zend_Test test case makes at least a HTTP request. The helper test case sets up a fake HTTP request and response objects in every setUp(), and lets you check the result, being it written HTML (via querying and asserting), XML or JSON. Integration testing Integration tests target an external component such as a library to ensure the expectations of the developers on it are met. Integration tests are usually started as exploratory tests, which are used to learn about the library and to encapsulate this knowledge into a repeatable, executable form. With time, they become regression tests, which allow you to upgrade the library to a new release or version by catching the changes in behavior. Some of these tests target the PHP runtime itself, to check for example that an extension assumed as present is really available. For example, this week we were surprised when a === check inside a Domain Model class was failing. We started writing integration tests for Doctrine_Query, and it turned out that PDO and Doctrine returned strings for numeric fields on their Active Record. By having a specific test to cover our expectations, we understood where our assumption was wrong, and cease to suspect a bug in our own code where the === resided. For this kind of test, again only PHPUnit is necessary; moreover, you'll have to bootstrap the involved library, but it can be simply a matter of adding it to the include_path. Acceptance testing Acceptance tests are end-to-end tests, which see the application as a black box. They exercise the behavior of the whole application, from the user inserting data to the reports created and the actions performed as a consequence. These tests are much slower, but they work on the end result of your work, and define what the user will see and interact with. For old-style applications, which do not involve rich clients, Zend_Test is usually enough for these kinds of tests. A thin layer of CSS expression built over it in order to check the pages without duplicating the same selectors all over the suite may help. However, for Javascript-rich apps, a tool like Selenium is necessary. Selenium drives a real web browser to a fresh instance of your application, and execute your tests, which can be defined manually or via a record-and-replay browser extension. Many PHPUnit extensions offer the means for connecting to a Selenium server, which manages the browsers, and navigate the web application. As a result of its focus on real web browsers such as Firefox and Chrome, Selenium tests are much slower than Zend_Test ones. However, they are the only tool available to execute acceptance tests which involve JavaScript. Conclusion Note that everyone of these kinds of tests (except the integration ones) can be written before the production code it exercises. Unit tests ahead of their referred class; functional tests ahead of the Facade they target; acceptance tests before a whole vertical slice of functionality is implemented. Moreover, if you're doing Test-Driven Development you should in general start at the higher level of abstraction (acceptance) and descending into the lower levels as needed. These different types of testing are always present, maybe as a small part of the suite, in every web application of moderate size. Learning to recognize them when they emerge will help you organizing the test suite better and maintaining it productive and responsive to change.