JUnit has a very cool feature called RunnerScheduler. A custom RunnerScheduler can be set on a ParentRunner to control how child elements are executed. If you are on a Suite, the child elements would be each test class. If you are on a simple class (Junit4 runner) the child elements are all the test methods. Thus, with a RunnerScheduler you are able to control the overall execution of your test flow. As an example, suppose you want to execute your test methods concurrently on a given test. You could have a runner called ConcurrentJunitRunner. @RunWith(ConcurrentJunitRunner.class) @Concurrent(threads = 6) public final class ATest { @Test public void test0() throws Throwable { printAndWait(); } @Test public void test1() throws Throwable { printAndWait(); } @Test public void test2() throws Throwable { printAndWait(); } @Test public void test3() throws Throwable { printAndWait(); } @Test public void test4() throws Throwable { printAndWait(); } @Test public void test5() throws Throwable { printAndWait(); } @Test public void test6() throws Throwable { printAndWait(); } @Test public void test7() throws Throwable { printAndWait(); } @Test public void test8() throws Throwable { printAndWait(); } @Test public void test9() throws Throwable { printAndWait(); } void printAndWait() throws Throwable { int w = new Random().nextInt(1000); System.out.println(String.format("[%s] %s %s %s", Thread.currentThread().getName(), getClass().getName(), new Throwable().getStackTrace()[1].getMethodName(), w)); Thread.sleep(w); } } The @Concurrent annotation controls the thread count. The runner implements a custom RunnerScheduler which delegates to a thread pool and Java Concurrent API each test method. Thus all test are executed concurrently and the RunnerScheduler waits for all tests to finish. But wait ! There's even more ! This runner just makes the test methods of a class runnable concurrently. But if you have a lot of tests in your project, you would probably want to also run all these tests concurrently ! Here come the ConcurrentSuite runner ! @RunWith(ConcurrentSuite.class) @Suite.SuiteClasses({ATest.class, ATest2.class, ATest3.class}) public class MySuite { } This runner will run all the tests in your suite. If a test class uses the ConcurrentJunitRunner or is annotated by @Concurrent then its method will be run concurrently. Otherwise it will be run sequentially. The runners provided on this article demonstrates how to use a custom RunnerScheduler, but can be safely used in any projects and be modified according to your needs. All the code for this article can be found here. You can also checkout the classes: svn co http://mycila.googlecode.com/svn/sandbox/ sandbox Mathieu Carbou http://blog.mycila.com/ http://www.junit.org/node/589

When unit testing a class that queries an LDAP server using Java’s JNDI API I needed to replace the actual remote LDAP server with a mock LDAP access layer so that the unit test (remember, this is not an integration test) doesn’t depend on any external SW/HW. Few hours of googling haven’t yielded any suitable mock implementation and so I had to create my own one. It turned out to be an easy task after all. I hope it could be useful for you too. To create a test implementation of LDAP/JNDI you need to: Hook you mock JNDI implementation into the JVM and make sure that you use it Actually implement the JNDI layer by implementing/mocking few (3) JNDI classes, mostly from the javax.naming.directory package Configure your mock implementation to return the data expected 1. Configuring the JVM to use the test JNDI implementation The best way to “inject” your test LDAP/JNDI implementation depends on the way your code is accessing it. There are basically two options: You specify explicitely the implementation to use via the parameter INITIAL_CONTEXT_FACTORY You use the default implementation for the JVM Let’s see an example: new javax.naming.directory.InitialDirContext(new Hashtable(){{ put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory"); put(Context.PROVIDER_URL, "ldap://ldap.example.com:389");}); The javax.naming.directory.InitialDirContext will delegate most operations to the actual implementation, which is provided either by the requested initial context factory if the line #2 is included or based on the JVM’s configuration – see NamingManager.getInitialContext(..). Therefore: If your code specifies explicitely the initial context factory, configure it to use your test initial context factory implementation, i.e. you modify the code to something like put(Context.INITIAL_CONTEXT_FACTORY, "your.own.MockInitialDirContextFactory") (you have that configurable, right?) If your code relies on the JVM’s configuration to provide the proper implementation, configure it with a custom InitialContextFactoryBuilder, which will return your test initial context factory implementation. I won’t go into the details here, you can see an example in the Spring’s mock jndi SimpleNamingContextBuilder [source] (it mocks unfortunately only javax.naming, not the javax.naming.directory we need for LDAP) 2. Implementing/mocking JNDI classes The test LDAP over JNDI implementation is quite simple. We need: The InitialContextFactory for creating our test contexts, as described above The test DirContext implementation itself, which we will mock using Mockito (the interface has many methods to implement while my code is using only one of them) And a NamingEnumeration implementation for returning search results from the mock DirContext’s search method The test initial context factory is very simple: public class MockInitialDirContextFactory implements InitialContextFactory {private static DirContext mockContext = null;/** Returns the last DirContext (which is a Mockito mock) retrieved from this factory. */public static DirContext getLatestMockContext() {return mockContext;}public Context getInitialContext(Hashtable environment)throws NamingException {synchronized(MockInitialDirContextFactory.class) {mockContext = (DirContext)Mockito.mock(DirContext.class);}return mockContext;} We store the latest DirContext mock (the class under test only creates one so this is enough) so that we can tell it what calls to expect and what to return (i.e. to do some “stubbing”). We also need an implementation of the NamingEnumeration, which is returned by the various search methods. Because we actually do not use it we could also mock it with Mockito (simple Mockito.mock(NamingEnumeration.class) would be enough to replace all the lines below) but I’ve decided to create a real implementation so that in more involved tests it could be extended to actually be able of holding and returning some fake LDAP search data. In this case the NamingEnumeration should hold instances of the conrete class SearchResult with the actual LDAP data in its field of the type Attributes, for which we can use the concrete BasicAttributes implementation provided by the JVM. But for now let’s just return an empty enumeration. public class MockNamingEnumeration/**/ implements NamingEnumeration {public void close() throws NamingException {}public boolean hasMore() throws NamingException {return hasMoreElements();}public Object next() throws NamingException {return nextElement();}public boolean hasMoreElements() {return false;}public Object nextElement() {return null;} As you can see, this implementation will behave as if the search matched no records. 3. Using the test LDAP/JNDI implementation The last piece is the actual JUnit test of a hypothetical TestedLdapReader class, which searches an LDAP server: public class MyMockLdapTest extends TestCase {private TestedLdapReader ldapReader; ...protected void setUp() throws Exception {super.setUp();ldapReader = new TestedLdapReader();ldapReader.setInitialContextFactory(MockInitialDirContextFactory.class.getName());ldapReader.setLdapUrl("ldap://thisIsIgnoredInTests");}public void testLdapSearch() throws Exception {ldapReader.initLdap(); // obtains an InitialDirContext...final DirContext mockContext = MockInitialDirContextFactory.getLatestMockContext(); //Stub the public NamingEnumeration search(String name, String filter, SearchControls cons)Mockito.when( mockContext.search( (String) Mockito.eq("ou=bluepages,o=ibm.com") , Mockito.anyString() , (SearchControls) Mockito.any(SearchControls.class))) // a custom 'answer', which records the queries issued .thenAnswer(new Answer() { public Object answer(InvocationOnMock invocation) throws Throwable { LOG.debug("LDAP query:" + invocation.getArguments()[1] ); return new MockNamingEnumeration(); } }); try { ldapReader.searchLdap(); } catch (Exception e) { LOG.warn("exception during execution", e); } // Uncomment to find out the methods called on the context: // Mockito.verifyNoMoreInteractions(new Object[]{mockContext});} Let’s summarize what we do here: #07,08: We tell the class under test to use our test JNDI implementation #13: It’s assumed that this call instantiates an InitialDirContext supplying it the initial context factory class parameter set on the lines 07-08 #16-26: We use Mockito to configure the mock DirContext to expect a search call for the context “ou=bluepages,o=ibm.com”, any query string and any search controls and tell it to return an empty MockNamingEnumeration while also logging the actual LDAP query (the 2nd argument). #29: The tested method is called #35: If we are not sure what methods the tested method calls on the DirContext, we may uncomment this line to let Mockito check it (adding Mockito.verify(mockContext.(..)) prior to #35 for each method we know about already) Summary We’ve created a minimalist LDAP over JNDI implementation using partly real and partly mock objects. It could be easily extended to make it possible to configure the data returned from individual LDAP searches (currently we always return an empty collection) and thus test the behavior in reaction to different data sets. There is of course some space left for simplification. From http://theholyjava.wordpress.com/2010/05/05/mocking-out-ldapjndi-in-unit-tests/

In a well-organized build process, you want lightning-fast unit tests to run first, and provide whatever feedback they can very quickly. A nice way to do this is to be able to class your tests into different categories. For example, this can make it easier to distinguish between faster running unit tests, and slower tests such as integration, performance, load or acceptance tests. This feature exists in TestNG, but, until recently, not in JUnit. Indeed, this has been missing from the JUnit world for a long time. Using JUnit, I typically use test names (integration tests end in 'IntegrationTest', for example) or packages to identify different types of test. It is easy to configure a build script using Maven or Ant to run different types of test at different points in the build lifecycle. However it would be nice to be able to do this in a more elegant manner. JUnit 4.8 introduced a new feature along these lines, called Categories. However, like most new JUnit features, it is almost entirely undocumented. In this article we'll see how it works and what it can do for you. In JUnit 4.8, you can define your own categories for your tests. Categories are implemented as classes or interfaces. Since they simply act as markers, I prefer to use interfaces. One such category interface might look like this: public interface IntegrationTests {} You can also use inheritance to organize your test categories: public interface SlowTests {} public interface IntegrationTests extends SlowTests {} public interface PerformanceTests extends SlowTests {} So far so good. Now you can use these categories in your tests. In this example we flag a particular test class as containing integration tests: @Category(IntegrationTests.class) public class AccountIntegrationTest { @Test public void thisTestWillTakeSomeTime() { ... } @Test public void thisTestWillTakeEvenLonger() { .... } } You can also flag individual test methods if you prefer: public class AccountTest { @Test @Category(IntegrationTests.class) public void thisTestWillTakeSomeTime() { ... } @Test @Category(IntegrationTests.class) public void thisTestWillTakeEvenLonger() { ... } @Test public void thisOneIsRealFast() { ... } } To run tests in a particular category, you need to set up a test suite. In JUnit 4, a test suite is essentially an empty annotated class. To run only tests in a particular category, you use the @Runwith(Categories.class) annotation, and specify what category you want to run using the @IncludeCategory annotation @RunWith(Categories.class) @IncludeCategory(SlowTests.class) @SuiteClasses( { AccountTest.class, ClientTest.class }) public class LongRunningTestSuite {} You can also ask JUnit not to run tests in a particular category using the @ExcludeCategory annotation @RunWith(Categories.class) @ExcludeCategory(SlowTests.class) @SuiteClasses( { AccountTest.class, ClientTest.class }) public class UnitTestSuite {} Test categories are great if you use JUnit test suites. I haven't used test suites for years: Maven can find all my tests by itself, thank you very much, so I don't have to remember to add my test classes to the right test suite each time a create a new one. However, test suites do give you finer control over what order your tests are executed in, so you might still find them useful in that regard. Once you've done this, it is then easy to run tests in a particular category from within your IDE simply by running the test suite. On the tooling and build automation side of things, JUnit categories are not supported as well as TestNG groups. For example, the Maven surefire plugin lets you specify the TestNG groups you want to run in a particular phase, but no such support exists as yet for JUnit categories. You can of course configure the Surefire plugin to run a particular test suite (or test suites) in a particular phase, but it doesn't dispense you with the need to write and maintain a test suite. So test categories are great, but having to run them via a test suite (and to remember to add new test classes to the test suite) seems a bit clunky in these days of annotations and reflection. From http://weblogs.java.net/blog/johnsmart/archive/2010/04/25/grouping-tests-using-junit-categories-0

The purpose of this article is to give you the details of our 100 node cluster demo. This demo is recorded and you can watch the 5 minute screencast Hazelcast is an open source clustering and highly scalable data distribution platform for Java. JVMs that are running Hazelcast will dynamically cluster and allow you to easily share and partition your application data across the cluster. Hazelcast is a peer-to-peer solution (there is no master node, every node is a peer) so there is no single point of failure. Communication among cluster members is always TCP/IP with Java NIO beauty. The default configuration comes with 1 backup so if a node fails, no data will be lost (you can specify the backup count). It is as simple as using java.util.{Map, Queue, Set, List}. Just add the hazelcast.jar into your classpath and start coding. When you download the Hazelcast, you will find a test.sh under bin directory. The test.sh runs an application which randomly makes 40% get, 40% put and 20% remove on a distributed map. In this demo the same test application will be used to see how it performs on 100 node cluster. Amazon EC2 and S3 An easy to use and scalable cloud environment was needed for demo so we decided to use Amazon EC2 for server instances (nodes) and S3 service to store demo application zip and configuration files. With its newly announced Java SDK, it is very simple to start/stop server instances and upload files to S3 programatically. Hazelcast AMI & Launcher The challenge here is that we are running an application on 100 nodes and dealing with each and every server in the cluster is a huge task. We don't want to ssh into every server and manually start the application. This part is automated by creating a special server image (AMI). The AMI contains Java Runtime and a launcher application we developed, which will download the demo application from Amazon S3, unzip it, and run the hazelcast/bin/test.sh in it. The Launcher is actually so generic that it can run any application; it doesn't care/know what test.sh contains. Deployer Deployment of the demo application is also automated so that we don't need to login into AWS Management Console and manually start instances. Deployer instantiates any number of Amazon EC2 servers with any AMI and also uploads the demo application zip file to S3. So the idea here is that, the Deployer will store the application into S3 and launch 100 EC2 instances with our image. The Launcher on each instance will download the application from S3 and run it. Demo Details. The smallest EC2 instances (m1.small) are used to run the demo. These are the virtual instances with CPU about 1.0 GHz. Also keep in mind that EC2 platform suffers from considerable amount of network latency. That's why we increased the thread count to 250 in our application. The following steps performed during the demo Download hazelcast-1.8.3.zip from www.hazelcast.com. Unzip the file and move the monitoring war file into tomcat6/webapps directory. Edit the test.sh under the bin directory: Add -Xmx1G -Xms1G Add -Dhazelcast.initial.wait.seconds=100 to make the cluster evenly partition on start so that migration can be avoided for better performance. Add t250 as an argument to the application to set thread count to 250. Remember the latency issue. Run the Deployer from IDE. Check from EC2 Management Console if 100 servers started. Start tomcat. Copy the public DNS name of one of the servers to connect to from monitoring tool. Go to http://localhost:8080/hazelcast-monitor-1.8.3/ (Hazelcast Monitoring Tool). Paste the address and connect to the cluster. Enjoy! Results You should always look for programatic ways of launching applications on the cloud. With these tools we were able to deploy and run the demo application on 100 servers in minutes. The entire Hazelcast cluster was making over 400,000 operations per second on the smallest EC2 instances. In our next demo we will experiment Hazelcast on large data set and even bigger cluster. Watch the screencast

I have written up a checklist to use for unit-level Test-Driven Development, to make sure I do not skip steps while writing code, at a very low level of the development process. Ideally I will soon internalize this process to the point that I would recognize smells as soon as they show up the first time. This checklist is also applicable to the outer cycle of Acceptance TDD, but the Green part becomes much longer and it comprehends writing other tests. Ignore this paragraph if this get you confused. TDD is described by a basic red-green-refactor cycle, constantly repeatead to add new features or fix bugs. I do not want to descend too much in object-oriented design in this post as you may prefer different techniques than me, so I will insist on the best practices to apply as soon as possible in the development of tests and production code. The checklist is written in the form of questions we should ask ourselves while going through the different phases, and that are often overlooked for the perceived simplicity of this cycle. Red The development of every new feature should start with a failing test. Have you checked in the code in your remote or local repository? In case the code breaks, a revert is faster than a rewrite. Have you already written some production code? If so, comment it or (best) delete it to not be implicitly tied to an Api while writing the test. Have you chosen the right unit to expand? The modified class should be the one that remains more cohesive after the change, and often in new classes should be introduced instead of accomodating functionalites in existing ones. Does the test fail? If not, rewrite the test to expose the lack of functionality. Does a subset of the test already fail? Is so, you can remove the surplus part of the test, avoiding verbosity; it can come back in different test methods. Does the test prescribe overly specific assertions or expectations? If so, lessen the mock expectations by not checking method calls order or how many times a method is called; improve the assertions by substituting equality matches with matches over properties of the result object. Does the test name describe its intent? Make sure it is not tied to implementation details and works as low-level documentation. How much can you change in an hypothetical implementation without breaking the test (making it brittle)? Is the failure message expressive about what is broken? Make sure it describes where the failing functionality resides, highlighting the right location if it breaks in the future. Are magic numbers and strings expressed as constants? Is there repeated code? Test code refactoring is easy when done early and while a test fails, since in this paradigm it is more important to keep it failing then to keep it passing. Green Enough production code should be written to make the test pass. Does the production code make the test pass? (Plainly obvious) Does a subset of the production code make the test pass? If so, you can comment or (best) remove the unnecessary production code. Any more lines you write are untested lines you'll have to read and maintain in the future. Every other specific action will be taken in the Refactor phase. Refactor Improve the structure of the code to ease future changes and maintenance. Does repeated code exist in the current class? Is the name of the class under test appropriate? Do the public and protected method names describe their intent? Are they readable? Rename refactorings are between the most powerful ones. Does repeated code exist in different classes? Is there a missing domain concept? You can extract abstract classes or refactor towards composition. At this high-level the refactoring should be also applied to the unit tests, and there are many orthogonal techniques you can apply so I won't describe them all here. Feel free to add insights and items on the list in the comments. I value very much feedback from other TDDers. From http://giorgiosironi.blogspot.com/2010/03/tdd-checklist-red-green-refactor-in.html

A pipeline in software context is a very well-known architectural style in which a process consists of a series of steps to be followed in order to proceed the data, and the output of one step is the input of another step. This is also called the Pipes and Filters design pattern. The naming comes from the physical pipeline as this architectural style is very similar to a pipeline in which a stream of data comes in and leaves after being processed. The original idea of pipeline in software is implemented in Unix. This pattern is used in many places. Compiler pipeline, ASP.NET HTTP Pipeline, and workflows are three of many examples that I can mention. The pipes and filters style is implemented in various platforms with different techniques and technologies. Recently I was in a situation to implement this pattern and did some research to find more about the possible options to implement this pattern in the .NET Framework. Doing my research, I found many approaches introduced by community members but the most mature technique is the one that Oren Eini has described in his blog post using Generics. There is also an interesting technique described by Jeremy Likness using the yield keyword in C#. In this post I’m going to apply Oren’s approach and expand it to write a simple implementation of the classic KWIC example in Software Engineering. I liked Oren’s code because as he said, it’s comparatively simpler than other solutions introduced for this problem in the .NET Framework. An Overview of KWIC KWIC stands for Key Word in Context and is a classic problem in Software Engineering papers in which you try to create an index of words by sorting and aligning each word in a piece of text. David Parnas has a famous paper on modularity that uses KWIC as an example. There are some basic and advanced implementations of KWIC in different platforms but the main steps are: Reading the input Shifting the words in each line to get a new permutation Sorting the results Writing the output Interestingly, in this case the output of each step is the input of the next step which makes this a good candidate for the Pipes and Filters pattern. Implement the Pipes and Filters Pattern with Generics Oren’s technique for implementing the Pipes and Filters in the .NET Framework is based on a Generic interface and a Generic class. The Generic interface simulates the filter and the Generic class simulates the pipeline. The IOperation interface has a single method called Execute that is the implementation of the filter logic. Each filter should implement this interface. using System.Collections.Generic;namespace KwicPipesFilters{ public interface IOperation { IEnumerable Execute(IEnumerable input); } The use of a generic IEnumerable is a good choice because it leaves a lot of space for the developers to plug in any type that they want and use various types for their filters. The Pipeline class has an Execute and a Register method. Using the Register method, you add different filters to the pipeline and using the Execute method, you start processing the item in all the registered filters. using System.Collections.Generic;namespace KwicPipesFilters{ public class Pipeline { private readonly List> operations = new List>(); public Pipeline Register(IOperation operation) { operations.Add(operation); return this; } public void Execute() { IEnumerable current = new List(); foreach (IOperation operation in operations) { current = operation.Execute(current); } IEnumerator enumerator = current.GetEnumerator(); while (enumerator.MoveNext()); } } The implementation of the Pipeline class is straightforward: it keeps a list of filters and provides a Register function that lets you add new filters to your pipeline, and then use the Execute method to execute all the filters in the list to process an input. Reader The Reader filter reads the input text from a file and returns an IEnumerable list of lines. Of course, for the first filter in the pipe we don’t care about the input as the input is read inside the filter itself. using System;using System.Collections.Generic;using System.IO;namespace KwicPipesFilters{ public class Reader : IOperation { public IEnumerable Execute(IEnumerable input) { Console.Title = "Pipes and Filters Pattern in .NET"; Console.WriteLine("Enter the path of the file:"); return File.ReadLines(Console.ReadLine()); } } Shifter The Shifter filter is where the main logic of the KWIC application is implemented. It shifts the words in each line to find all the possible permutations suitable for the index. using System.Collections.Generic;namespace KwicPipesFilters{ public class Shifter : IOperation { public IEnumerable Execute(IEnumerable input) { List shifts = new List(); foreach (string line in input) { string[] words = line.Split(new char[] { ' ' }); for (int i = 0; i <= words.Length - 1; i++) { shifts.Add(string.Join(" ", words)); string firstWord = words[0]; for (int j = 1; j <= words.Length - 1; j++) { words.SetValue(words[j], j - 1); } words.SetValue(firstWord, words.Length - 1); } } return shifts; } } Here we have a basic implementation of the Shifter filter where we split the line into separate words based on the space between them, then shift all the words to find various permutations. Sorter Before returning the final results in the Writer filter, we need to sort the index alphabetically. This is done in the Sorter filter. using System.Collections.Generic;using System.Linq;namespace KwicPipesFilters{ public class Sorter : IOperation { public IEnumerable Execute(IEnumerable input) { LineComparer lineComparer = new LineComparer(); input.ToList().Sort(lineComparer); return input; } } Here I used a LineComparer class to implement the ICcomparer interface for the string type. using System.Collections.Generic;namespace KwicPipesFilters{ public class LineComparer : IComparer { public int Compare(string x, string y) { return string.Compare(x, y); } } Writer Obviously, the last filter should write the index to the output for the user and that’s the purpose of the Writer filter. using System;using System.Collections.Generic;namespace KwicPipesFilters{ public class Writer : IOperation { public IEnumerable Execute(IEnumerable input) { foreach (string line in input) { Console.WriteLine(); Console.WriteLine(line); } Console.ReadLine(); yield break; } } As you see, this filter uses a yield break to avoid returning any result. Pipeline Having all the filter implemented, I also need to implement the pipeline itself in order to register the filters and make the whole thing work. I do this in my KwicPipeline class with a simple code that it has. namespace KwicPipesFilters{ public class KwicPipeline : Pipeline { public KwicPipeline() { Register(new Reader()); Register(new Shifter()); Register(new Sorter()); Register(new Writer()); } } I inherit from the Pipeline class and register my filters in the public constructor. Putting It Together There is only one step remained and that is putting all these things together to start the pipeline. All I need to do is to create an instance of the KwicPipeline class, call its Execute method, and leave the rest to my pipes and filters. namespace KwicPipesFilters{ class Program { static void Main(string[] args) { KwicPipeline pipeline = new KwicPipeline(); pipeline.Execute(); } } Conclusion In this post I implemented the Pipes and Filters pattern in the .NET Framework using a simple and generalized technique that relies on Generics to implement the KWIC application. In my opinion this is one of the best ways to implement this pattern in the .NET Framework. I have uploaded the sample source code package here. Note that the solution is created using Visual Studio 2010 RC1. There are other techniques to implement this pattern in .NET and one specific technique that I have in mind is using the Windows Workflow Foundation. I may work more on this idea and write about it later.

Some of the Spock Framework 0.4 features are starting to see the light of day, with the Data Tables being explained last week in a nice blog post from Peter Niederwieser. One of the new features that I had not seen before is the new advanced @Unroll usage. Mixed with Data Tables, it produces some very cool results, and it can still be used with 0.3 style specs as well. Here's the juice: JUnit Integration and @Unroll Spock is built on JUnit, and has always had good IDE support without any effort from you as a user. For the most part, the IDEs just think Spock is another unit test. Here's the a Spock spec for the new Data Tables feature and how it shows up in an IDE. import spock.lang.* class TableTest extends Specification { def "maximum of two numbers"() { expect: Math.max(a, b) == c where: a | b | c 3 | 7 | 7 5 | 4 | 5 9 | 9 | 9 } } The assertion will be run 3 times: once for each row in the data table. And JUnit faithfully reports the method name correctly, even when the method names has a space in it: The problem with data driven tests and xUnit is poor error location. When a test fails you will receive an error stating which method is the culprit... but what if the method runs an assertion across 50 or 60 pieces of data? The cause of a failure is almost never clear with data driven tests. At it's worst you have to step through several iterations of code waiting for an exception. Good tests have a clear point of failure, but good tests also do not repeat themselves with boilerplate. This is exactly why Spock has the @Unroll annotation. As a test author you get to write one concise unit test, and JUnit does the work of reporting results that help you isolate failures. Consider the same test method with the @Unroll annotation and the accompanying IDE output. @Unroll def "maximum of two numbers"() { expect: Math.max(a, b) == c where: a | b | c 3 | 7 | 7 5 | 4 | 5 9 | 9 | 9 } When executed, JUnit sees three test methods instead of one: one for each row in the data table: The end result for you as a test writer is accurate failure resolution. You can pinpoint exactly which row failed. This feature is available in Spock 0.3 and you can use it today. What is new in 0.4 is the ability to change the test name dynamically. Here is a full @Unroll annotation that changes the method name: @Unroll("maximum of #a and #b is #c") def "maximum of two numbers"() { expect: Math.max(a, b) == c where: a | b | c 3 | 7 | 7 5 | 4 | 5 9 | 9 | 9 } Notice the #variable syntax in the annotation parameter. The # produces a sort of GString-like variable substitution that lets you bind columns from your data table into your test name. The annotation parameter references #a, #b, and #c, which aligns with the data table definition of a | b | c. Check out the IDE output: Previously, the test name was just the iteration number within the test. The new @Unroll parameter allows you to make the test name much more meaningful. Your tests will improve because failures become more descriptive. Unrolled failure messages before simply had the iteration name embedded in them, while now they can have meaningful data that you prescribe. My favorite part of playing with the new @Unroll was to see the default value of the parameter within the Spock source code: java.lang.String value() default "#featureName[#iterationCount]"; Talk about eating your own dog food... the default value is a test name template, just like you could have written in your own test. Makes you wonder what other variables are in scope, huh? Spock snapshot builds for 0.4 are available at: http://m2repo.spockframework.org. Get it before the link breaks. From http://hamletdarcy.blogspot.com

Perhaps the most striking thing about about the Play! framework is that its biggest advantage over other Java web application development frameworks does not fit into a neat feature list, and is only apparent after you have used it to build something. That advantage is usability. Note that usability is separate from functionality. In what follows, I am not suggesting that you cannot do this in some other framework: I merely claim that it is easier and more pleasant in Play! I need to emphasise this because geeks often have a total blind spot for usability because they enjoying figuring out difficult things, and under-appreciate the value of things that Just Work. Written by web developers for web developers The first hint that something different is going on here is when you first hear that the Play! framework is 'written by web developers for web developers', an unconventional positioning that puts the web's principles and conventions first and Java's second. Specifically, this means that the Play! framework is more in line with the W3C's Architecture of the World Wide Web than it is with Java Enterprise Edition (Java EE) conventions. URLs for perfectionists For example, the Play! framework, like other modern web frameworks, provides first-class support for arbitrary 'clean' URLs, which has always been lacking from the Servlet API. It is no coincidence that at the time of writing, Struts URLs for perfectionists, a set of work-arounds for the Servlet API-based Struts 1.x web framework, remains the third-most popular out of 160 articles on www.lunatech-research.com despite being a 2005 article about a previous-generation Java web technology. In Servlet-based frameworks, the Servlet API does not provide useful URL-routing support; Servlet-based frameworks configure web.xml to forward all requests to a single controller Servlet, and then implement URL routing in the framework, with additional configuration. At this point, it does not matter whether the Servlet API was ever intended to solve the URL-routing problem and failed by not being powerful enough, or whether it was intended to be a lower-level API that you do not build web applications in directly. Either way, the result is the same: web frameworks add an additional layer on top of the Servlet API, itself a layer on top of HTTP. Play! combines the web framework, HTTP API and the HTTP server, which allows it to implement the same thing more directly with fewer layers and a single URL routing configuration. This configuration, like Groovy's and Cake PHP's, reflects the structure of an HTTP request - HTTP method, URL path, and then the mapping: # Play! 'routes' configuration file… # Method URL path Controller GET / Application.index GET /about Application.about POST /item Item.addItem GET /item/{id} Item.getItem GET /item/{id}.pdf Item.getItemPdf In this example, there is more than one controller. We also see the use of an id URL parameter in the last two URLs. HttpServletRequest Another example is Play!'s Http.Request class, which is a far simpler than the Servlet API's HttpServletRequest interface. In addition, Play! uses a class where Java EE 6 uses the Java EE convention of using an interface. This interface is also split between HttpServletRequest and the more generic ServletRequest interface. This separation may be useful if you want to use Servlets for things other than web applications, or if you want to allow for the unlikely possibility of the web changing protocol, but for most of us it is merely irrelevant complexity. In other words, the Servlet API is always used with a framework on top these days because it is sub-optimised for building web applications, which is what all of us actually use it for. Play! fixes that. Better usability is not just for normal people Another way of looking at the idea that Play! is by and for web developers is to consider how a web developer might approach software design differently to a Java EE developer. When you write software, what is the primary interface? If you are a web developer, the primary interface is a web-based user-interface constructed with HTML, CSS and (increasingly) JavaScript. A Java EE developer, on the other hand, may consider their primary interface to be a Java API, or perhaps a web services API, for use by other layers in the system. This difference is a big deal, because a Java interface is intended for use by other programmers, while a web user-interface interface is intended for use by non-programmers. In both cases, good design includes usability, but usability for normal people is not the same as usability for programmers. In a way, usability for everyone is a higher standard than usability for programmers, when it comes to software, because programmers can cope better with poor usability. This is a bit like the Good Grips kitchen utensils: although they were originally designed to have better usability for elderly people with arthritis, it turns out that making tools easier to hold is better for all users. The Play! framework is different because the usability that you want to achieve in your web application is present in the framework itself. For example, the web interface to things like the framework documentation and error messages shown in the browser is just more usable. Along similar lines, the server's console output avoids the pages full of irrelevant logging and pages of stack traces when there is an error, leaving more focused and more usable information for the web developer. $ play run phase ~ _ _ ~ _ __ | | __ _ _ _| | ~ | '_ \| |/ _' | || |_| ~ | __/|_|\____|\__ (_) ~ |_| |__/ ~ ~ play! 1.0, http://www.playframework.org ~ ~ Ctrl+C to stop ~ Listening for transport dt_socket at address: 8000 10:15:58,629 INFO ~ Starting /Users/peter/Documents/work/workspace/phase 10:16:00,007 WARN ~ You're running Play! in DEV mode 10:16:00,424 INFO ~ Listening for HTTP on port 9000 (Waiting a first request to start) ... 10:16:11,847 INFO ~ Connected to jdbc:hsqldb:mem:playembed 10:16:13,448 INFO ~ Application 'phase' is now started ! 10:16:14,825 INFO ~ starting DispatcherThread 10:16:48,168 ERROR ~ @61lagcl6i Internal Server Error (500) for request GET /application/startprocess?account=x Java exception (In /app/controllers/Application.java around line 41) IllegalArgumentException occured : Person not found for account x play.exceptions.JavaExecutionException: Person not found for account x at play.mvc.ActionInvoker.invoke(ActionInvoker.java:200) at Invocation.HTTP Request(Play!) Caused by: java.lang.IllegalArgumentException: Person not found for account x at controllers.Application.startProcess(Application.java:41) at play.utils.Java.invokeStatic(Java.java:129) at play.mvc.ActionInvoker.invoke(ActionInvoker.java:127) ... 1 more Try to imagine a JSF web application producing a stack trace this short. In fact, Play! goes further: instead of showing the stack trace, the web application shows the last line of code within the application that appears in the stack trace. After all, what you really want to know is where things first went wrong in your own code. This kind of usability does not happen by itself; the Play! framework goes to considerable effort to filter out duplicate and irrelevant information, and focus on what is essential. Quality is in the details In the Play! framework, much of the quality turns out to be in the details: they may be small things individually, rather than big important features, but they add up to result in a more comfortable and more productive development experience. The warm feeling you get when building something with Play! is the absence of the frustration that usually results from fighting the framework. We recommend that you go to http://www.playframework.org/, download the latest binary release, and spend half an hour on the tutorial. Peter Hilton is a senior software developer at Lunatech Research.

This week, I searched for free online SVN repositories for closed-source projects.

There are at least four methods that can be used in different combinations to make the process of setting up a complete development environment a lot less painful.

When working in Eclipse, you’ll often end up with a number of projects in your workspace that constitute an application. You could have a multi-tiered system with a web, server and database project and other miscellaneous ones. Or if you’re an Eclipse RCP developer, you could end up with dozens of plugins each represented by a project. Although multiple projects give you modularity (which is good), they can make it difficult to manage the workspace (which is bad). Developers have to check out each project individually from different locations in the repository. Sometimes they even have to get projects from multiple repositories. This is a painstakingly long and error-prone task. But an easier way to manage multiple projects is with Eclipse’s Team Project Sets (TPS). Creating a workspace becomes as easy as importing an XML file and waiting for Eclipse to do its job. Yes, there are other more sophisticated tools out there that do this and more (eg. Maven and Buckminster) but team project sets are a good enough start if you haven’t got anything set up and may be good enough for the longer term as well, depending on how your team works. Create a Team Project Set to share with other developers It’s easy to create a team project set (TPS). The first thing is to start with a workspace that already has all the projects checked out. Then it’s as easy as choosing File > Export > Team > Team Project Set, selecting the projects you want to export and then entering a file name. Done. But it’s always better to see it in action. In the video, I export 3 projects that I’ve already checked out from Subversion into a TPS file. Notes: You can select which projects should go into the TPS. This way you can exclude irrelevant or personal projects you’ve got in your workspace. Eclipse adds the extension .psf if you don’t provide one. The exported file is an XML file, with the default extension of psf, so in the video the file would be music.psf. There is a project entry for each project you exported that includes the project’s name and its repository location, separated by commas. Once created, the file is easy to edit so go ahead and make your own changes if you want to. Here is an example of what it looks like: svn/repo/music-application/trunk,music-application"/> svn/repo/music-db/trunk,music-db"/> svn/repo/music-web/trunk,music-web"/> Import the Team Project Set to checkout multiple projects into your workspace Now for the fun part. To import a team project set (TPS), start with any workspace (normally an empty one) and choose File > Import > Team > Team Project Set. Choose the TPS file that someone else kindly exported for you and then wait for Eclipse to do its magic. Notes: If you have an existing project in your workspace whose name matches a project in the TPS, Eclipse will prompt you whether you want to overwrite the project. I always choose No To All, since overwriting the project will mean you lose any changes you made to it. But if you have the urge to start from scratch then you can choose Yes. The import also creates a link to the repository in SVN Repositories, so you don’t have to do that. If one already exists, it will not duplicate it but reuse the existing connection. The process may take a while depending on the number of projects in the TPS and the speed of your repo checkouts. You can choose to run the import in the background (as I did in the video), giving you the opportunity to use Eclipse while the import happens. Otherwise, grab some coffee and wait for it to finish the checkouts. Gotcha: You may find that Eclipse 3.4 and lower may actually create a repository connection per project if the repository didn’t exist beforehand, which is not ideal. To solve this, create an initial repository root that’s shared by the projects and then do the import of the TPS. This problem has been fixed in 3.5 Managing the team project set and working with branches I’d recommend checking in the team project set into your repository and versioning/tagging it along with the rest of your code base. With each release you may be adding/removing projects and consequently updating the TPS, so it’s important that the TPS matches what the repo looks like at that point. As projects are added/removed with each release, you have 3 possibilities: Recreate the TPS from an existing workspace: Same as the steps above, but it means that whoever does the export needs to maintain an up to date workspace to reflect the current project structure. Modify an existing TPS with the new/deleted project: This entails adding/removing an entry from the PSF file. Not a lot of maintenance, but someone needs to remember to do this. Automatically create/update the TPS: You could write a script that somehow updates the TPS to reflect the new repo structure. For example, if you’re developing an Eclipse RCP application, the PDE Build provides a map file that could be used as input to create the PSF file. If you want to checkout a branch other than trunk, just open the PSF file and do a Find/Replace of trunk with your branch name. You could also introduce an automated process as part of your build/release scripts to update the TPS with the correct branch and check it back in automatically, but that’s really optional. From http://eclipseone.wordpress.com

The emergence of version control systems makes both promiscuous and continuous integration merging techniques more attractive. Which is better?

As I had a MVVM session at my office, I was re-reading a few articles about MVVM. We have very interesting discussion about MVVM in WPF Disciples User Group as well. You can read that post from here. Someone in Silverlight Forum (link) posted that ~ “There are currently three main areas of criticism regarding the MVVM pattern. The first is that MVVM currently lacks standardization from Microsoft both in implementation and in toolsets. For example, the community has some lack of clarity about where and whether to implement View logic in the View layer or the ViewModel. Given that the MVVM pattern is still relatively new, and that new tool-sets, walkthroughs, or patterns, such as Onyx, Prism, the Microsoft WPF Toolkit, Crack.net, Caliburn and MVVM Light Toolkit are being released, this problem may be solved over time. Microsoft has announced in discussion boards that the MVVM template pattern will be released in Visual Studio 2010. The second comes from MVVM creator John Gossman himself, who points out that the overhead in implementing MVVM is “overkill” for simple UI operations. He also states that for larger applications, generalizing the View layer becomes more difficult. Moreover, he illustrates that data binding, if not managed well, can result in a considerable excess of metadata in an application. Given these limitations, MVVM may have a practical minimum and maximum size for the type of application it can support, suggesting it may not perform well with large enterprise applications. The third is that the exercise in creating large numbers of data bindings to the ViewModel results in duplicate code and maintenance problems. Additionally, because of the nature of the semantics of data bindings, critics suggest that the ViewModel does not directly describe the View.” So, I was thinking it would be great if John and our WPF/Silverlight community can define some simple and obvious rules for MVVM pattern.I understand that there are a lot of way to implement MVVM but at least, there are some obvious rules that everyone can follow so everyone has same understanding about that pattern. Here are some of my thoughts about MVVM. Goals Testabiltiy ( ViewModel is easier to unit test than code-behind or event driven code) Clear seperation between UX designer and developer Increases the “Blendability” of your view Model never needs to be changed to support changes to the view ViewModel rarely needs to be changed to support changes to the view No duplicated code to update views Do and Don’t in View shouldn’t contain any logic that you want to test : As Glenn said that MVVM is not code counting exercise, we can write code in code-behind. But you should never write any logic that you want to test. For example: If user select a country then you want to display the list of states or city in your view. This is the business requirement so you should have unit test to test this logic. So, you shouldn’t write it in code-behind. can be a user control or Data Template Keep the view as simple as possible. : We can still use Data Trigger or Value Converter or Visual State or Blend Behivor in XAML with care. use attached property if something is not bindable : Do and Don’t in ViewModel Connector between View and Model Keep View State, Value Conversion No strong or weak (via Interface) reference of View Make VM as testable as possible (e.g. no call to Singleton class) No Control related Stuff in VM ( Because if you are changing the view then you will have to change VM as well. ) Model can be Data Model, DTO, POCO, auto-generated proxy of domain class and UI Model based on how you want to have the separation between Domain Service and Presentation Layer No reference to ViewModel What do you think about that? Feel free to let me know if you have any comment or suggestion.. Thanks.

What can you do with a Groovy AST Transformation? A difficult question, considering the answer is "almost anything".

Learn the basics of Java Content Repositories, including how they work, and how they're used.

Spring Integration and Apache Camel are open source frameworks providing a simpler solution for the Integration problems in the enterprise, to quote from their respective websites: Apache Camel - Apache Camel is a powerful open source integration framework based on known Enterprise Integration Patterns with powerful Bean Integration. Spring Integration - It provides an extension of the Spring programming model to support the well-known Enterprise Integration Patterns while building on the Spring Framework's existing support for enterprise integration. Essentially Spring Integration and Apache Camel enable applications to integrate with other systems. This article seeks to provide an implementation for an integration problem using both Spring Integration and Apache Camel. The objective is to show how easy it is to use these frameworks for a fairly complicated integration problem and to recommend either of these great products for your next Integration challenge. Problem: To illustrate the use of these frameworks consider a simple integration scenario, described using EIP terminology: The application needs to get a "Report" by aggregating "Sections" from a Section XML over http service. Each request for Report consists of a set of request for sections – in this specific example there are requests for three sections, the header, body and footer. The XML over http service returns a Section for the Section Request. The responses need to be aggregated into a single report. A sample test for this scenario is of the following type: ReportGenerator reportGenerator = reportGeneratorFactory.createReportGenerator(); List sectionRequests = new ArrayList(); String entityId="A Company"; sectionRequests.add(new SectionRequest(entityId,"header")); sectionRequests.add(new SectionRequest(entityId,"body")); sectionRequests.add(new SectionRequest(entityId,"footer")); ReportRequest reportRequest = new ReportRequest(sectionRequests); Report report = reportGenerator.generateReport(reportRequest); List sectionOfReport = report.getSections(); System.out.println(report); assertEquals(3, sectionOfReport.size()); The “ReportGenerator” is the messaging gateway, hiding the details of the underlying messaging infrastructure and in this specific case also the integration API – Apache Camel or Spring Integration. To start with, let us implement a solution to this integration problem using Spring Integration as the Framework, followed by Apache Camel. The complete working code using Spring Integration and Apache Camel is also available with the article. Solution Using Spring Integration: The Gateway component is easily configured using the following entry in the Spring Configuration. Internally Spring Integration uses AOP to hook up a component which routes the requests from an internal input channel and waits for the response in the response channel. The component to Split the Input Report Request to Section Request is fairly straightforward: public class SectionRequestSplitter { public List split(ReportRequest reportRequest){ return reportRequest.getSectionRequests(); } } and to hook this splitter with Spring Integration: Next, to transform the Section Request to an XML format - The component is the following: public class SectionRequestToXMLTransformer { public String transform(SectionRequest sectionRequest){ //this needs to be optimized...purely for demonstration of the concept String sectionRequestAsString = "" + sectionRequest.getEntityId() + "" + sectionRequest.getSectionId() + ""; return sectionRequestAsString; } } and is hooked up in the Spring Integration configuration file in the following way: To send an XML over http request using the Section Request XML to a section Service: To transform the Section Response XML to a Section Object - The component is the following: public class SectionResponseXMLToSectionTransformer { public Section transform(String sectionXML) { SAXReader saxReader = new SAXReader(); Document document; String sectionName = ""; String entityId = ""; try { document = saxReader.read(new StringReader(sectionXML)); sectionName = document .selectSingleNode("/section/meta/sectionName").getText(); entityId = document.selectSingleNode("/section/meta/entityId") .getText(); } catch (DocumentException e) { e.printStackTrace(); } return new Section(entityId, sectionName, sectionXML); } } and is hooked up in the Spring Integration configuration file in the following way: To aggregate the Sections together into a report, the component is the following:: public class SectionResponseAggregator { public Report aggregate(List sections) { return new Report(sections); } } and is hooked up in the Spring Integration configuration file in the following way: This completes the Spring Integration implementation for this Integration Problem. The following is the complete Spring Integration configuration file: A working sample is provided with the article(Download, extract and run "mvn test") Solution using Apache Camel: Apache Camel allows the route to be defined using multiple DSL implementations – Java DSL, Scala DSL and an XML based DSL. The recommended approach is to use Spring CamelContext as a runtime and the Java DSL for route development. The following is to build the Spring Camel Context: The route is configured by the Java based DSL: public class CamelRouteBuilder extends RouteBuilder { private String serviceURL; @Override public void configure() throws Exception { from("direct:start") .split().method("sectionRequestSplitterBean", "split") .aggregationStrategy(new ReportAggregationStrategy()) .transform().method("sectionRequestToXMLBean", "transform") .to(serviceURL) .transform().method("sectionResponseXMLToSectionBean", "transform"); } public void setServiceURL(String serviceURL) { this.serviceURL = serviceURL; } } Apache Camel does not provide an out of the box Message Gateway feature, however it is fairly easy to create a wrapper component that can hide the underlying details in the following way: Reader davsclaus has provided references to two mechanisms with Apache Camel to provide an out of the box Messaging Gateway - Messaging Gateway EIP and Camel Proxy which allows a POJO to be used as a Mesaging Gateway. Camel Proxy will be used with the article, and can be configured in the Camel Configuration files in the following way: Per davsclaus, there is a bug in Apache Camel(2.1 or older) when invoking a bean later in the route(the splitter bean), which is to be fixed in Apache Camel 2.2. To work around this bug, a convertBody step will be introduced in the route: from("direct:start") .convertBodyTo(ReportRequest.class) .split(bean("sectionRequestSplitterBean", "split"), new ReportAggregationStrategy()) .transform().method("sectionRequestToXMLBean", "transform") .to(serviceURL) .transform().method("sectionResponseXMLToSectionBean", "transform"); The component to Split the Input Report Request to Section Request is exactly same as Spring Integration component: public class SectionRequestSplitter { public List split(ReportRequest reportRequest){ return reportRequest.getSectionRequests(); } } To hook the component with Apache Camel: from("direct:start") .split().method("sectionRequestSplitterBean", "split") .... Next to transform the Section Request to an XML format, again this is exactly same as the implementation for Spring Integration, with hook being provided in the following manner: ...... .transform().method("sectionRequestToXMLBean", "transform") ...... To send an XML over http request using the Section Request XML to a section Service: ...... .transform().method("sectionRequestToXMLBean", "transform") .to(serviceURL) ......... To transform the Section Response XML to a Section object, the component is exactly same as the one used with Spring Integration, with the following highlighted hook in the Camel route: ...... .transform().method("sectionResponseXMLToSectionBean", "transform"); To aggregate the Section responses together into a report, the component is a bit more complicated than Spring Integration. Apache Camel supports a Scatter/Gather pattern using a route of the following type: ...... .split().method("sectionRequestSplitterBean", "split") .aggregationStrategy(new ReportAggregationStrategy()) with an aggregation strategy being passed on to the Splitter, the aggregation strategy implementation is the following: public class ReportAggregationStrategy implements AggregationStrategy { @Override public Exchange aggregate(Exchange oldExchange, Exchange newExchange) { if (oldExchange == null) { Section section = newExchange.getIn().getBody(Section.class); Report report = new Report(); report.addSection(section); newExchange.getIn().setBody(report); return newExchange; } Report report = oldExchange.getIn().getBody(Report.class); Section section = newExchange.getIn().getBody(Section.class); report.addSection(section); oldExchange.getIn().setBody(report); return oldExchange; } } This completes the Apache Camel based implementation. A working sample for Camel is provided with the article - just download, extract and run "mvn test". Conclusion: Spring Integration and Apache Camel provide a simple and clean approach for the Integration problems in a typical enterprise. They are lightweight frameworks – Spring Integration builds on top of Spring portfolio and extends the familiar programming model for the Integration domain and is easy to pick up, Apache camel provides a good Java based DSL and integrates well with Spring Core, with a fairly gentle learning curve. The article does not recommend one product over the other but encourages the reader to evaluate and learn from both these frameworks. References: Spring Integration Website: http://www.springsource.org/spring-integration Apache Camel Website: http://camel.apache.org/ Spring Integration Reference: http://static.springsource.org/spring-integration/reference/htmlsingle/spring-integration-reference.html Apache Camel User Guide: http://camel.apache.org/user-guide.html Plug for my blog: http://biju-allandsundry.blogspot.com/

// Lists of common US names for generating test data. There are 3 lists: // surnames_list, male_names_list, female_names_list surnames_list=['Smith','Johnson','Williams','Brown','Jones','Miller','Davis','Garcia','Rodriguez','Wilson','Martinez','Anderson','Taylor','Thomas','Hernandez','Moore','Martin','Jackson','Thompson','White','Lopez','Lee','Gonzalez','Harris','Clark','Lewis','Robinson','Walker','Perez','Hall','Young','Allen','Sanchez','Wright','King','Scott','Green','Baker','Adams','Nelson','Hill','Ramirez','Campbell','Mitchell','Roberts','Carter','Phillips','Evans','Turner','Torres','Parker','Collins','Edwards','Stewart','Flores','Morris','Nguyen','Murphy','Rivera','Cook','Rogers','Morgan','Peterson','Cooper','Reed','Bailey','Bell','Gomez','Kelly','Howard','Ward','Cox','Diaz','Richardson','Wood','Watson','Brooks','Bennett','Gray','James','Reyes','Cruz','Hughes','Price','Myers','Long','Foster','Sanders','Ross','Morales','Powell','Sullivan','Russell','Ortiz','Jenkins','Gutierrez','Perry','Butler','Barnes','Fisher','Henderson','Coleman','Simmons','Patterson','Jordan','Reynolds','Hamilton','Graham','Kim','Gonzales','Alexander','Ramos','Wallace','Griffin','West','Cole','Hayes','Chavez','Gibson','Bryant','Ellis','Stevens','Murray','Ford','Marshall','Owens','Mcdonald','Harrison','Ruiz','Kennedy','Wells','Alvarez','Woods','Mendoza','Castillo','Olson','Webb','Washington','Tucker','Freeman','Burns','Henry','Vasquez','Snyder','Simpson','Crawford','Jimenez','Porter','Mason','Shaw','Gordon','Wagner','Hunter','Romero','Hicks','Dixon','Hunt','Palmer','Robertson','Black','Holmes','Stone','Meyer','Boyd','Mills','Warren','Fox','Rose','Rice','Moreno','Schmidt','Patel','Ferguson','Nichols','Herrera','Medina','Ryan','Fernandez','Weaver','Daniels','Stephens','Gardner','Payne','Kelley','Dunn','Pierce','Arnold','Tran','Spencer','Peters','Hawkins','Grant','Hansen','Castro','Hoffman','Hart','Elliott','Cunningham','Knight','Bradley','Carroll','Hudson','Duncan','Armstrong','Berry','Andrews','Johnston','Ray','Lane','Riley','Carpenter','Perkins','Aguilar','Silva','Richards','Willis','Matthews','Chapman','Lawrence','Garza','Vargas','Watkins','Wheeler','Larson','Carlson','Harper','George','Greene','Burke','Guzman','Morrison','Munoz','Jacobs','Obrien','Lawson','Franklin','Lynch','Bishop','Carr','Salazar','Austin','Mendez','Gilbert','Jensen','Williamson','Montgomery','Harvey','Oliver','Howell','Dean','Hanson','Weber','Garrett','Sims','Burton','Fuller','Soto','Mccoy','Welch','Chen','Schultz','Walters','Reid','Fields','Walsh','Little','Fowler','Bowman','Davidson','May','Day','Schneider','Newman','Brewer','Lucas','Holland','Wong','Banks','Santos','Curtis','Pearson','Delgado','Valdez','Pena','Rios','Douglas','Sandoval','Barrett','Hopkins','Keller','Guerrero','Stanley','Bates','Alvarado','Beck','Ortega','Wade','Estrada','Contreras','Barnett','Caldwell','Santiago','Lambert','Powers','Chambers','Nunez','Craig','Leonard','Lowe','Rhodes','Byrd','Gregory','Shelton','Frazier','Becker','Maldonado','Fleming','Vega','Sutton','Cohen','Jennings','Parks','Mcdaniel','Watts','Barker','Norris','Vaughn','Vazquez','Holt','Schwartz','Steele','Benson','Neal','Dominguez','Horton','Terry','Wolfe','Hale','Lyons','Graves','Haynes','Miles','Park','Warner','Padilla','Bush','Thornton','Mccarthy','Mann','Zimmerman','Erickson','Fletcher','Mckinney','Page','Dawson','Joseph','Marquez','Reeves','Klein','Espinoza','Baldwin','Moran','Love','Robbins','Higgins','Ball','Cortez','Le','Griffith','Bowen','Sharp','Cummings','Ramsey','Hardy','Swanson','Barber','Acosta','Luna','Chandler','Blair','Daniel','Cross','Simon','Dennis','Oconnor','Quinn','Gross','Navarro','Moss','Fitzgerald','Doyle','Mclaughlin','Rojas','Rodgers','Stevenson','Singh','Yang','Figueroa','Harmon','Newton','Paul','Manning','Garner','Mcgee','Reese','Francis','Burgess','Adkins','Goodman','Curry','Brady','Christensen','Potter','Walton','Goodwin','Mullins','Molina','Webster','Fischer','Campos','Avila','Sherman','Todd','Chang','Blake','Malone','Wolf','Hodges','Juarez','Gill','Farmer','Hines','Gallagher','Duran','Hubbard','Cannon','Miranda','Wang','Saunders','Tate','Mack','Hammond','Carrillo','Townsend','Wise','Ingram','Barton','Mejia','Ayala','Schroeder','Hampton','Rowe','Parsons','Frank','Waters','Strickland','Osborne','Maxwell','Chan','Deleon','Norman','Harrington','Casey','Patton','Logan','Bowers','Mueller','Glover','Floyd','Hartman','Buchanan','Cobb','French','Kramer','Mccormick','Clarke','Tyler','Gibbs','Moody','Conner','Sparks','Mcguire','Leon','Bauer','Norton','Pope','Flynn','Hogan','Robles','Salinas','Yates','Lindsey','Lloyd','Marsh','Mcbride','Owen','Solis','Pham','Lang','Pratt','Lara','Brock','Ballard','Trujillo','Shaffer','Drake','Roman','Aguirre','Morton','Stokes','Lamb','Pacheco','Patrick','Cochran','Shepherd','Cain','Burnett','Hess','Li','Cervantes','Olsen','Briggs','Ochoa','Cabrera','Velasquez','Montoya','Roth','Meyers','Cardenas','Fuentes','Weiss','Wilkins','Hoover','Nicholson','Underwood','Short','Carson','Morrow','Colon','Holloway','Summers','Bryan','Petersen','Mckenzie','Serrano','Wilcox','Carey','Clayton','Poole','Calderon','Gallegos','Greer','Rivas','Guerra','Decker','Collier','Wall','Whitaker','Bass','Flowers','Davenport','Conley','Houston','Huff','Copeland','Hood','Monroe','Massey','Roberson','Combs','Franco','Larsen','Pittman','Randall','Skinner','Wilkinson','Kirby','Cameron','Bridges','Anthony','Richard','Kirk','Bruce','Singleton','Mathis','Bradford','Boone','Abbott','Charles','Allison','Sweeney','Atkinson','Horn','Jefferson','Rosales','York','Christian','Phelps','Farrell','Castaneda','Nash','Dickerson','Bond','Wyatt','Foley','Chase','Gates','Vincent','Mathews','Hodge','Garrison','Trevino','Villarreal','Heath','Dalton','Valencia','Callahan','Hensley','Atkins','Huffman','Roy','Boyer','Shields','Lin','Hancock','Grimes','Glenn','Cline','Delacruz','Camacho','Dillon','Parrish','Oneill','Melton','Booth','Kane','Berg','Harrell','Pitts','Savage','Wiggins','Brennan','Salas','Marks','Russo','Sawyer','Baxter','Golden','Hutchinson','Liu','Walter','Mcdowell','Wiley','Rich','Humphrey','Johns','Koch','Suarez','Hobbs','Beard','Gilmore','Ibarra','Keith','Macias','Khan','Andrade','Ware','Stephenson','Henson','Wilkerson','Dyer','Mcclure','Blackwell','Mercado','Tanner','Eaton','Clay','Barron','Beasley','Oneal','Small','Preston','Wu','Zamora','Macdonald','Vance','Snow','Mcclain','Stafford','Orozco','Barry','English','Shannon','Kline','Jacobson','Woodard','Huang','Kemp','Mosley','Prince','Merritt','Hurst','Villanueva','Roach','Nolan','Lam','Yoder','Mccullough','Lester','Santana','Valenzuela','Winters','Barrera','Orr','Leach','Berger','Mckee','Strong','Conway','Stein','Whitehead','Bullock','Escobar','Knox','Meadows','Solomon','Velez','Odonnell','Kerr','Stout','Blankenship','Browning','Kent','Lozano','Bartlett','Pruitt','Buck','Barr','Gaines','Durham','Gentry','Mcintyre','Sloan','Rocha','Melendez','Herman','Sexton','Moon','Hendricks','Rangel','Stark','Lowery','Hardin','Hull','Sellers','Ellison','Calhoun','Gillespie','Mora','Knapp','Mccall','Morse','Dorsey','Weeks','Nielsen','Livingston','Leblanc','Mclean','Bradshaw','Glass','Middleton','Buckley','Schaefer','Frost','Howe','House','Mcintosh','Ho','Pennington','Reilly','Hebert','Mcfarland','Hickman','Noble','Spears','Conrad','Arias','Galvan','Velazquez','Huynh','Frederick','Randolph','Cantu','Fitzpatrick','Mahoney','Peck','Villa','Michael','Donovan','Mcconnell','Walls','Boyle','Mayer','Zuniga','Giles','Pineda','Pace','Hurley','Mays','Mcmillan','Crosby','Ayers','Case','Bentley','Shepard','Everett','Pugh','David','Mcmahon','Dunlap','Bender','Hahn','Harding','Acevedo','Raymond','Blackburn','Duffy','Landry','Dougherty','Bautista','Shah','Potts','Arroyo','Valentine','Meza','Gould','Vaughan','Fry','Rush','Avery','Herring','Dodson','Clements','Sampson','Tapia','Bean','Lynn','Crane','Farley','Cisneros','Benton','Ashley','Mckay','Finley','Best','Blevins','Friedman','Moses','Sosa','Blanchard','Huber','Frye','Krueger','Bernard','Rosario','Rubio','Mullen','Benjamin','Haley','Chung','Moyer','Choi','Horne','Yu','Woodward','Ali','Nixon','Hayden','Rivers','Estes','Mccarty','Richmond','Stuart','Maynard','Brandt','Oconnell','Hanna','Sanford','Sheppard','Church','Burch','Levy','Rasmussen','Coffey','Ponce','Faulkner','Donaldson','Schmitt','Novak','Costa','Montes','Booker','Cordova','Waller','Arellano','Maddox','Mata','Bonilla','Stanton','Compton','Kaufman','Dudley','Mcpherson','Beltran','Dickson','Mccann','Villegas','Proctor','Hester','Cantrell','Daugherty','Cherry','Bray','Davila','Rowland','Madden','Levine','Spence','Good','Irwin','Werner','Krause','Petty','Whitney','Baird','Hooper','Pollard','Zavala','Jarvis','Holden','Haas','Hendrix','Mcgrath','Bird','Lucero','Terrell','Riggs','Joyce','Mercer','Rollins','Galloway','Duke','Odom','Andersen','Downs','Hatfield','Benitez','Archer','Huerta','Travis','Mcneil','Hinton','Zhang','Hays','Mayo','Fritz','Branch','Mooney','Ewing','Ritter','Esparza','Frey','Braun','Gay','Riddle','Haney','Kaiser','Holder','Chaney','Mcknight','Gamble','Vang','Cooley','Carney','Cowan','Forbes','Ferrell','Davies','Barajas','Shea','Osborn','Bright','Cuevas','Bolton','Murillo','Lutz','Duarte','Kidd','Key','Cooke'] male_names_list=['James','John','Robert','Michael','William','David','Richard','Charles','Joseph','Thomas','Christopher','Daniel','Paul','Mark','Donald','George','Kenneth','Steven','Edward','Brian','Ronald','Anthony','Kevin','Jason','Matthew','Gary','Timothy','Jose','Larry','Jeffrey','Frank','Scott','Eric','Stephen','Andrew','Raymond','Gregory','Joshua','Jerry','Dennis','Walter','Patrick','Peter','Harold','Douglas','Henry','Carl','Arthur','Ryan','Roger','Joe','Juan','Jack','Albert','Jonathan','Justin','Terry','Gerald','Keith','Samuel','Willie','Ralph','Lawrence','Nicholas','Roy','Benjamin','Bruce','Brandon','Adam','Harry','Fred','Wayne','Billy','Steve','Louis','Jeremy','Aaron','Randy','Howard','Eugene','Carlos','Russell','Bobby','Victor','Martin','Ernest','Phillip','Todd','Jesse','Craig','Alan','Shawn','Clarence','Sean','Philip','Chris','Johnny','Earl','Jimmy','Antonio','Danny','Bryan','Tony','Luis','Mike','Stanley','Leonard','Nathan','Dale','Manuel','Rodney','Curtis','Norman','Allen','Marvin','Vincent','Glenn','Jeffery','Travis','Jeff','Chad','Jacob','Lee','Melvin','Alfred','Kyle','Francis','Bradley','Jesus','Herbert','Frederick','Ray','Joel','Edwin','Don','Eddie','Ricky','Troy','Randall','Barry','Alexander','Bernard','Mario','Leroy','Francisco','Marcus','Micheal','Theodore','Clifford','Miguel','Oscar','Jay','Jim','Tom','Calvin','Alex','Jon','Ronnie','Bill','Lloyd','Tommy','Leon','Derek','Warren','Darrell','Jerome','Floyd','Leo','Alvin','Tim','Wesley','Gordon','Dean','Greg','Jorge','Dustin','Pedro','Derrick','Dan','Lewis','Zachary','Corey','Herman','Maurice','Vernon','Roberto','Clyde','Glen','Hector','Shane','Ricardo','Sam','Rick','Lester','Brent','Ramon','Charlie','Tyler','Gilbert','Gene','Marc','Reginald','Ruben','Brett','Angel','Nathaniel','Rafael','Leslie','Edgar','Milton','Raul','Ben','Chester','Cecil','Duane','Franklin','Andre','Elmer','Brad','Gabriel','Ron','Mitchell','Roland','Arnold','Harvey','Jared','Adrian','Karl','Cory','Claude','Erik','Darryl','Jamie','Neil','Jessie','Christian','Javier','Fernando','Clinton','Ted','Mathew','Tyrone','Darren','Lonnie','Lance','Cody','Julio','Kelly','Kurt','Allan','Nelson','Guy','Clayton','Hugh','Max','Dwayne','Dwight','Armando','Felix','Jimmie','Everett','Jordan','Ian','Wallace','Ken','Bob','Jaime','Casey','Alfredo','Alberto','Dave','Ivan','Johnnie','Sidney','Byron','Julian','Isaac','Morris','Clifton','Willard','Daryl','Ross','Virgil','Andy','Marshall','Salvador','Perry','Kirk','Sergio','Marion','Tracy','Seth','Kent','Terrance','Rene','Eduardo','Terrence','Enrique','Freddie','Wade'] female_names_list=['Mary','Patricia','Linda','Barbara','Elizabeth','Jennifer','Maria','Susan','Margaret','Dorothy','Lisa','Nancy','Karen','Betty','Helen','Sandra','Donna','Carol','Ruth','Sharon','Michelle','Laura','Sarah','Kimberly','Deborah','Jessica','Shirley','Cynthia','Angela','Melissa','Brenda','Amy','Anna','Rebecca','Virginia','Kathleen','Pamela','Martha','Debra','Amanda','Stephanie','Carolyn','Christine','Marie','Janet','Catherine','Frances','Ann','Joyce','Diane','Alice','Julie','Heather','Teresa','Doris','Gloria','Evelyn','Jean','Cheryl','Mildred','Katherine','Joan','Ashley','Judith','Rose','Janice','Kelly','Nicole','Judy','Christina','Kathy','Theresa','Beverly','Denise','Tammy','Irene','Jane','Lori','Rachel','Marilyn','Andrea','Kathryn','Louise','Sara','Anne','Jacqueline','Wanda','Bonnie','Julia','Ruby','Lois','Tina','Phyllis','Norma','Paula','Diana','Annie','Lillian','Emily','Robin','Peggy','Crystal','Gladys','Rita','Dawn','Connie','Florence','Tracy','Edna','Tiffany','Carmen','Rosa','Cindy','Grace','Wendy','Victoria','Edith','Kim','Sherry','Sylvia','Josephine','Thelma','Shannon','Sheila','Ethel','Ellen','Elaine','Marjorie','Carrie','Charlotte','Monica','Esther','Pauline','Emma','Juanita','Anita','Rhonda','Hazel','Amber','Eva','Debbie','April','Leslie','Clara','Lucille','Jamie','Joanne','Eleanor','Valerie','Danielle','Megan','Alicia','Suzanne','Michele','Gail','Bertha','Darlene','Veronica','Jill','Erin','Geraldine','Lauren','Cathy','Joann','Lorraine','Lynn','Sally','Regina','Erica','Beatrice','Dolores','Bernice','Audrey','Yvonne','Annette','June','Samantha','Marion','Dana','Stacy','Ana','Renee','Ida','Vivian','Roberta','Holly','Brittany','Melanie','Loretta','Yolanda','Jeanette','Laurie','Katie','Kristen','Vanessa','Alma','Sue','Elsie','Beth','Jeanne','Vicki','Carla','Tara','Rosemary','Eileen','Terri','Gertrude','Lucy','Tonya','Ella','Stacey','Wilma','Gina','Kristin','Jessie','Natalie','Agnes','Vera','Willie','Charlene','Bessie','Delores','Melinda','Pearl','Arlene','Maureen','Colleen','Allison','Tamara','Joy','Georgia','Constance','Lillie','Claudia','Jackie','Marcia','Tanya','Nellie','Minnie','Marlene','Heidi','Glenda','Lydia','Viola','Courtney','Marian','Stella','Caroline','Dora','Jo','Vickie','Mattie','Terry','Maxine','Irma','Mabel','Marsha','Myrtle','Lena','Christy','Deanna','Patsy','Hilda','Gwendolyn','Jennie','Nora','Margie','Nina','Cassandra','Leah','Penny','Kay','Priscilla','Naomi','Carole','Brandy','Olga','Billie','Dianne','Tracey','Leona','Jenny','Felicia','Sonia','Miriam','Velma','Becky','Bobbie','Violet','Kristina','Toni','Misty','Mae','Shelly','Daisy','Ramona','Sherri','Erika','Katrina','Claire','Lindsey','Lindsay','Geneva','Guadalupe','Belinda','Margarita','Sheryl','Cora','Faye','Ada','Natasha','Sabrina','Isabel','Marguerite','Hattie','Harriet','Molly','Cecilia','Kristi','Brandi','Blanche','Sandy','Rosie','Joanna','Iris','Eunice','Angie','Inez','Lynda','Madeline','Amelia','Alberta','Genevieve','Monique','Jodi','Janie','Maggie','Kayla','Sonya','Jan','Lee','Kristine','Candace','Fannie','Maryann','Opal','Alison','Yvette','Melody','Luz','Susie','Olivia','Flora','Shelley','Kristy','Mamie','Lula','Lola','Verna','Beulah','Antoinette','Candice','Juana','Jeannette','Pam','Kelli','Hannah','Whitney','Bridget','Karla','Celia','Latoya','Patty','Shelia','Gayle','Della','Vicky','Lynne','Sheri','Marianne','Kara','Jacquelyn','Erma','Blanca','Myra','Leticia','Pat','Krista','Roxanne','Angelica','Johnnie','Robyn','Francis','Adrienne','Rosalie','Alexandra','Brooke','Bethany','Sadie','Bernadette','Traci','Jody','Kendra','Jasmine','Nichole','Rachael','Chelsea','Mable','Ernestine','Muriel','Marcella','Elena','Krystal','Angelina','Nadine','Kari','Estelle','Dianna','Paulette','Lora','Mona','Doreen','Rosemarie','Angel','Desiree','Antonia','Hope','Ginger','Janis','Betsy','Christie','Freda','Mercedes','Meredith','Lynette','Teri','Cristina','Eula','Leigh','Meghan','Sophia','Eloise','Rochelle','Gretchen','Cecelia','Raquel','Henrietta','Alyssa','Jana','Kelley','Gwen','Kerry','Jenna','Tricia','Laverne','Olive','Alexis','Tasha','Silvia','Elvira','Casey','Delia','Sophie','Kate','Patti','Lorena','Kellie','Sonja','Lila','Lana','Darla','May','Mindy','Essie','Mandy','Lorene','Elsa','Josefina','Jeannie','Miranda','Dixie','Lucia','Marta','Faith','Lela','Johanna','Shari','Camille','Tami','Shawna','Elisa','Ebony','Melba','Ora','Nettie','Tabitha','Ollie','Jaime','Winifred','Kristie','Marina','Alisha','Aimee','Rena','Myrna','Marla','Tammie','Latasha','Bonita','Patrice','Ronda','Sherrie','Addie','Francine','Deloris','Stacie','Adriana','Cheri','Shelby','Abigail','Celeste','Jewel','Cara','Adele','Rebekah','Lucinda','Dorthy','Chris','Effie','Trina','Reba','Shawn','Sallie','Aurora','Lenora','Etta','Lottie','Kerri','Trisha','Nikki','Estella','Francisca','Josie','Tracie','Marissa','Karin','Brittney','Janelle','Lourdes','Laurel','Helene','Fern','Elva','Corinne','Kelsey','Ina','Bettie','Elisabeth','Aida','Caitlin','Ingrid','Iva','Eugenia','Christa','Goldie','Cassie','Maude','Jenifer','Therese','Frankie','Dena','Lorna','Janette','Latonya','Candy','Morgan','Consuelo','Tamika','Rosetta','Debora','Cherie','Polly','Dina','Jewell','Fay','Jillian','Dorothea','Nell','Trudy','Esperanza','Patrica','Kimberley','Shanna','Helena','Carolina','Cleo','Stefanie','Rosario','Ola','Janine','Mollie','Lupe','Alisa','Lou','Maribel','Susanne','Bette','Susana','Elise','Cecile','Isabelle','Lesley','Jocelyn','Paige','Joni','Rachelle','Leola','Daphne','Alta','Ester','Petra','Graciela','Imogene','Jolene','Keisha','Lacey','Glenna','Gabriela','Keri','Ursula','Lizzie','Kirsten','Shana','Adeline','Mayra','Jayne','Jaclyn','Gracie','Sondra','Carmela','Marisa','Rosalind','Charity','Tonia','Beatriz','Marisol','Clarice','Jeanine','Sheena','Angeline','Frieda','Lily','Robbie','Shauna','Millie','Claudette','Cathleen','Angelia','Gabrielle','Autumn','Katharine','Summer','Jodie','Staci','Lea','Christi','Jimmie','Justine','Elma','Luella','Margret','Dominique','Socorro','Rene','Martina','Margo','Mavis','Callie','Bobbi','Maritza','Lucile','Leanne','Jeannine','Deana','Aileen','Lorie','Ladonna','Willa','Manuela','Gale','Selma','Dolly','Sybil','Abby','Lara','Dale','Ivy','Dee','Winnie','Marcy','Luisa','Jeri','Magdalena','Ofelia','Meagan','Audra','Matilda','Leila','Cornelia','Bianca','Simone','Bettye','Randi','Virgie','Latisha','Barbra','Georgina','Eliza','Leann','Bridgette','Rhoda','Haley','Adela','Nola','Bernadine','Flossie','Ila','Greta','Ruthie','Nelda','Minerva','Lilly','Terrie','Letha','Hilary','Estela','Valarie','Brianna','Rosalyn','Earline','Catalina','Ava','Mia','Clarissa','Lidia','Corrine','Alexandria','Concepcion','Tia','Sharron','Rae','Dona','Ericka','Jami','Elnora','Chandra','Lenore','Neva','Marylou','Melisa','Tabatha','Serena','Avis','Allie','Sofia','Jeanie','Odessa','Nannie','Harriett','Loraine','Penelope','Milagros','Emilia','Benita','Allyson','Ashlee','Tania','Tommie','Esmeralda','Karina','Eve','Pearlie','Zelma','Malinda','Noreen','Tameka','Saundra','Hillary','Amie','Althea','Rosalinda','Jordan','Lilia','Alana','Gay','Clare','Alejandra','Elinor','Michael','Lorrie','Jerri','Darcy','Earnestine','Carmella','Taylor','Noemi','Marcie','Liza','Annabelle','Louisa','Earlene','Mallory','Carlene','Nita','Selena','Tanisha','Katy','Julianne','John','Lakisha','Edwina','Maricela','Margery','Kenya','Dollie','Roxie','Roslyn','Kathrine','Nanette','Charmaine','Lavonne','Ilene','Kris','Tammi','Suzette','Corine','Kaye','Jerry','Merle','Chrystal','Lina','Deanne','Lilian','Juliana','Aline','Luann','Kasey','Maryanne','Evangeline','Colette','Melva','Lawanda','Yesenia','Nadia','Madge','Kathie','Eddie','Ophelia','Valeria','Nona','Mitzi','Mari','Georgette','Claudine','Fran','Alissa','Roseann','Lakeisha','Susanna','Reva','Deidre','Chasity','Sheree','Carly','James','Elvia','Alyce','Deirdre','Gena','Briana','Araceli','Katelyn','Rosanne','Wendi','Tessa','Berta','Marva','Imelda','Marietta','Marci','Leonor','Arline','Sasha','Madelyn','Janna','Juliette','Deena','Aurelia','Josefa','Augusta','Liliana','Young','Christian','Lessie','Amalia','Savannah','Anastasia','Vilma','Natalia','Rosella','Lynnette','Corina','Alfreda','Leanna','Carey','Amparo','Coleen','Tamra','Aisha','Wilda','Karyn','Cherry','Queen','Maura','Mai','Evangelina','Rosanna','Hallie','Erna','Enid','Mariana','Lacy','Juliet','Jacklyn','Freida','Madeleine','Mara','Hester','Cathryn','Lelia','Casandra','Bridgett','Angelita','Jannie','Dionne','Annmarie','Katina','Beryl','Phoebe','Millicent','Katheryn','Diann','Carissa','Maryellen','Liz','Lauri','Helga','Gilda','Adrian','Rhea','Marquita','Hollie','Tisha','Tamera','Angelique','Francesca','Britney','Kaitlin','Lolita','Florine','Rowena','Reyna','Twila','Fanny','Janell','Ines','Concetta','Bertie','Alba','Brigitte','Alyson','Vonda','Pansy','Elba','Noelle','Letitia','Kitty','Deann','Brandie','Louella','Leta','Felecia','Sharlene','Lesa','Beverley','Robert','Isabella','Herminia','Terra','Celina']

Using "pseudo"-activators, fragment bundles can be more active than you think! Writing test code in an OSGi-setting requires some more work and decisions than in a plain Java-main()-applications-setting. For example, we typically don't want to put test code in the "functional" bundles themselves, as this makes the build process more complex, as we need to be able to extract the test code from the "production" release package. On the other hand, we don't want to export all bundle packages. So if we put our test code in separate bundles, we're not able to access all classes/packages that we might like to test... An often-cited approach can then be to include the test code in fragment bundles. These can be easily included/excluded in test launches or release packaging, and they use the same classloader as the host bundle, so they have access to all of the host's classes. For more info/discussion on this, check: http://rcpquickstart.com/2007/06/20/unit-testing-plug-ins-with-fragments/ http://www.modumind.com/2008/06/12/running-unit-tests-for-rcp-and-osgi-applications/ http://michaelscharf.blogspot.com/2008/04/is-osgi-enemy-of-junit-tests_17.html http://alblue.blogspot.com/2006/03/javaeclipse-running-all-tests-in.html Ok, and now what? Indeed. Once the decision to put test code in fragments is made, the next step is : how can we get these tests executed??? The problems are : fragments do not get activated, so they can not register services, start service trackers etc. It's not possible AFAIK to have the fragment initiate any kind of action on "start-up". fragment classes/resources are only visible in the host bundle, unless the host bundle exports them. But if the fragment is meant to be optional, we can not export its packages from the host. host bundles can not easily discover their registered fragment bundles. The only approach I found was to inspect the MANIFEST of all running bundles, and found out if there are some with a Fragment-Host entry that refers to the host bundle. And even there, we may need features that are only practically available in the new OSGi 4.2. See http://java.dzone.com/articles/osgi-junit-test-extender-using So how can we "discover" the presence of test cases? In the links above, all kinds of tricks with reflection are described, or using eclipse-specific things. Others ensure that their test fragments provide a "magic file" on the root etc., for which the host can search. All these approaches are based on the host bundle being aware of the optional presence of a test fragment, and checking for the presence of a class or a file on some predefined path. In this way, test cases can be found and added to test suites etc. But this implies that the host bundle is aware of the purpose of the fragment(s). Now, to make things even more complicated, we often like to run tests using Equinox Console commands. This implies that we must be able to register CommandProvider implementations as OSGi services. But fragments do not have activators, so how can they register service implementations!? Enter the TestFragmentActivator To increase testing flexibility, we don't want the host bundle to be aware whether the fragment is providing JUnit test cases, CommandProviders, or other test approaches. At the same time, we want to allow the fragment to be able to register OSGi services or perform other OSGi-magic typically done in Activators. So we propose the following approach : each test fragment must provide a BundleActivator implementation with a predefined qualified name. E.g. for a project FOO, to test backend services, the name could be com.foo.backend.service.test.TestFragmentActivator in the host bundle's Activator, we add code like : public class Activator implements BundleActivator {private static Activator defaultInstance;// a reference to the fragment's pseudo-activatorprivate BundleActivator testFragmentActivator;public void start(BundleContext context) throws Exception {defaultInstance = this;try {Class frgActClass = (Class) Class.forName("com.foo.backend.service.test.TestFragmentActivator");testFragmentActivator = frgActClass.newInstance();testFragmentActivator.start(context);} catch (ClassNotFoundException e) {// ignore, means the test fragment is not present...// it's a dirty way to find out, but don't know how to // discover fragment contribution in a better way...}public void stop(BundleContext context) throws Exception {defaultInstance = null;if(testFragmentActivator!=null) testFragmentActivator.stop(context);}public static Activator getDefault() {return defaultInstance;} and in the fragment we add the implementation, that can e.g. register a new CommandProvider : package com.foo.backend.service.test;import org.eclipse.osgi.framework.console.CommandProvider;import org.osgi.framework.BundleActivator;import org.osgi.framework.BundleContext;import org.osgi.framework.ServiceRegistration;import com.foo.backend.service.test.impl.ServiceTestCommandProvider;/** * This is a fake activator, i.e. the OSGI platform will never see or invoke it, * as we're inside a fragment. The goal is that the host bundle tries to * figure out if this kind of activator is present and if so, invoke its start() * & stop() methods from inside its own ones. * * Remark that this activator will not really start the fragment, it will remain * in the RESOLVED state as far as the OSGi platform is concerned! */public class TestFragmentActivator implements BundleActivator {private ServiceRegistration svcReg;public void start(BundleContext context) throws Exception {ServiceTestCommandProvider svcTester = new ServiceTestCommandProvider();svcReg = context.registerService(CommandProvider.class.getName(), svcTester, null);}public void stop(BundleContext context) throws Exception {svcReg.unregister();} In a similar way, we could use the fragment's pseudo activator to register JUnit testcases to some centralized test suite executor etc. Couldn't it even be a useful pattern in general for fragments, i.e. not only for test-oriented fragments?? What do you think? Erwin De Ley iSencia Belgium Voorhavenlaan 31 - 011 B-9000 Gent Belgium http://www.isencia.be/

My goal is to compare Sonatype Nexus and JFrog Artifactory,the two leading open source Maven repository managers.

This is the second part of a two-part article introducing Jeff De Luca’s Feature Driven Development (FDD) process. In particular, we are looking at how FDD differs from Scrum and eXtreme Programming-inspired approaches when it comes to working with larger teams and projects. In the first part we briefly introduced the ‘just enough’ upfront activities that FDD uses to support the additional communication that inevitably is needed in a larger project/team. In the second part of the article we cover how FDD leverages the results of those upfront activities within the highly iterative, self-managing, organized-chaos that is the delivery engine room of an FDD project. The Engine Room: Delivering Frequent, Tangible Working Results Once there is an initial overall model (FDD Process #1), an initial overall features list (FDD Process #2), and an initial overall plan (FDD Process #3) in place, an FDD project is ready to start delivering the required software feature by feature. Peter Coad, the Chief Architect on the original FDD project used the phrase ‘Deliver frequent, tangible, working results’ as a mantra to impress upon people the idea of delivering real, completed, client-valued function as often as possible. Scrum and eXtreme Programming do this using fixed length iterations of a calendar month or 2-4 weeks. FDD is different. Each Chief Programmer (lead developer) runs a series of iterations, each of which is normally a matter of a few days, and never longer than two weeks. At the start of each of these iterations, each Chief Programmer selects the next few features that make sense to implement from the backlog of feature sets (activities) that were assigned to him or her in FDD Process #2. The Chief Programmer leads the development of these features through FDD processes #4 and #5, Design by Feature (DBF) and Build By Feature (BBF). Note that iterations through the DBF/BBF processes are not fixed length, and Chief Programmers do not synchronize the start and end of their iterations with each other. In addition, the DBF/BBF processes are always executed as a pair (FDD describes them as two separate processes rather than one combined process for psychological reasons). FDD Process #4: Design By Feature After selecting the features for the iteration, a Chief Programmer needs to form their feature team. Yes, feature teams are formed and disbanded for each iteration through the DBF/BBF process pair. Using the knowledge gained from the modeling process (FDD Process #1), the Chief Programmer identifies the domain classes that are likely to be involved in this iteration, and forms his or her feature team from the owners of those classes. In practice, this means: a feature team is small, typically 3 to 5 people, because features are small. By definition, a feature team comprises of all the class owners who need to modify their classes in the development of the features during that iteration. There is no need to wait for members of other teams to change code. Therefore, there are all the benefits of code ownership and a sense of collective ownership too. Class owners may find themselves a member of multiple feature teams at the same time. This does not happen as frequently as might be supposed because iterations are so short – days not weeks. When it does, it is not a big problem in practice. Chief Programmers work together to resolve any problematic conflicts and, with care, most developers can manage the demands of occasionally belonging to more than one feature team for a short time. Once formed, the Chief Programmer facilitates the collaborative analysis and design of the features for that iteration. Depending on the complexity, this may involve the team walking through the requirements in detail with a domain expert, and studying any existing relevant documents. It also involves agreeing on the interactions and other details that need to be added to the model to support the new features. The final step in the DBF part of the iteration is to review the design. For simple features, this may be a brief sanity check of the design held within the feature team. For more significant features, the Chief Programmer will typically involve other Chief Programmers or class owners so that they are aware and can comment on the impact of the proposed design. For small team projects, the object models are frequently small enough for individual or pairs of developers to create good designs while writing tests for a particular feature or user story. For larger projects, this is not necessarily the case and designs created purely by considering the tests a feature or user story must pass are more likely to be brittle and require significant refactoring. The DBF process in FDD ensures that the overall model also guides the design, helping to maintain its ‘conceptual integrity’ [Brooks]. FDD Process #5: Build By Feature The Build by Feature (BBF) part of the iteration involves the team members coding up the features, testing them at both unit level and feature level, and holding a code inspection before promoting the completed features into the project's regular build process. Testing FDD expects developers to unit test their code. It expects feature teams to test their features. FDD is not overly concerned with how this is achieved. Projects and feature teams are free to adopt the testing tools, frameworks, and level of formality and completeness that are most appropriate. FDD does not mind if tests are written before or after code. What FDD mandates, is that the feature team deliver code that has been appropriately tested and inspected. Only once the new features have passed testing and inspection is the source code allowed into the build process. Code Inspections Most people want to know why FDD mandates code inspections, especially those that have endured sitting through hours of boring, unproductive, ego-polishing/demolishing, point-scoring sessions that formed so-called code reviews, inspections or walkthroughs. The reason FDD mandates code inspections is that research has shown time and again that when done well, inspections find more defects and different kinds of defects than testing [McConnell]. Not only that but by examining the code of the more experienced, knowledgeable developers on the team and having them explain the idioms they use, less experienced developers learn better coding techniques. In addition, knowing that their code will be inspected and not be allowed in the build unless it conforms to the agreed standards encourages developers to pay more attention to conforming to those standards. One of the benefits of working in feature teams is that the whole feature team is on the hot seat during an inspection, not just one individual. This removes much of the intensity and anxiety inherent in inspecting one individuals work. The Chief Programmer decides on the level of formality of each inspection depending on the complexity and impact of the features developed in that iteration. Where the code has little or no impact outside the feature team, an inspection will usually only involve the feature team inspecting each other’s work. Where there is significant impact the Chief Programmer pulls in other Chief Programmers and developers to both verify the code and communicate the impact of the new features. eXtreme Programming acknowledges inspections as a ‘best practice' but promotes pair programming as the logical conclusion of applying this practice. Pair programming is obviously better than individual developers delivering code without any form of inspection. However, while FDD neither mandates nor forbids pair programming, a more-traditional inspection is: fresh eyes looking at the code, catching bad assumptions made by the coder/s a Chief Programmer present to ensure the techniques passed on are good. After all, developers can just as easily teach each other bad habits as well as good habits. a change of pace for developers, a chance to step away from the keyboard and mouse for a short while. With the wide availability of automated source code formatting and static analysis tools, code inspections can now be shorter, concentrating on the logic and coding idioms involved and not getting bogged down in nit-picking such as alignment of braces, etc. The Build FDD assumes some sort of regular build process. Some teams build weekly, others daily and others continuously. FDD avoids mandating any particular build regime. This enables the project team to apply the most applicable. If a continuous integration environment makes sense, then the team is free to employ the best there is. Progress Reports Agile projects like highly visible progress information. FDD projects are no exception. In fact, because larger projects frequently have higher profiles within an organization, presenting meaningful, accurate, timely project information appropriately at the different levels of leadership/management is even more important. Conventionally, FDD projects track the development of each feature through its DBF/BBF iteration against six milestones: domain walkthrough, design, design inspection, coding, testing and inspection, and promoted to build. For each feature, Chief Programmers record the actual date a milestone is reached. Tracking each feature through these six milestones enables the project to keep an eye on how much work is 'in progress'. Too many features at a particular milestone indicate a process problem. Those promoting Kanban and other Limited Work In Progress methods have formalized this idea to strictly define what is meant by 'too many' for each of their development iteration milestones/statuses. They then refuse to move an item to a new milestone/status if the limit on the number of items at that status has been reached. This forces a team to keep items moving forward through the process [Kanban]. FDD is not so formal, leaving the Chief Programmers and Development Manager to keep an eye informally on the amount of work in progress. The Big Wallchart, Burn-Down/Up Charts, Etc For general visibility of progress within a project, the team typically lists all the features in the project complete with their owning Chief Programmer, feature team members, and the dates of each milestone achieved on a suitable wall. In addition, features can be colored to show if they are started, in-progress, completed or blocked. This allows people to stand back from the wall and get a good visual feel for the overall status of the project. They can then walk up to the wall to zoom in on particular areas and activities in more detail. Recording the date each milestone is achieved enables a team to produce burn-down or burn-up charts analogous to those produced in Scrum and XP. Chief Programmers and Project Managers can determine from these if the underlying rate of feature completion is increasing, decreasing, or stable, etc. One of the best ways to achieve this is to have the Chief Programmers regularly (typically once a week) communicate progress to either the project manager or someone dedicated to the task. That person then produces whatever roll-up and burn-down charts desired. Having an administrative person, the equivalent of the Tracker role in eXtreme Programming, perform these report formatting duties frees the Chief Programmers to spend more time on making progress rather than formatting reports about it. Parking Lot Charts For reporting to senior management, the level of individual features is often too granular. Here, FDD projects typically use a graphical report format that known as the Parking Lot chart. In a Parking Lot chart, each group of ‘parking lots’ represents one of the subject areas from the features list. Each parking lot represents one of the activities within that subject area, and displays the name of that set of features, the number of features within it, and the percentage of those features that have been completed (typically both in text and using a progress bar). The parking lots are also colored to indicate whether the features in that activity have been started, completed, or have significant blockages. The FDD parking lot format has become so popular that Mike Cohn included it in his book, Agile Planning and Estimating [Cohn]. (click for larger image) Figure 1: Example Parking Lot Chart Conclusion Feature-Driven Development combines the key advantages of other popular agile approaches with model-centric techniques and other best practices that scale to much larger teams and projects. It defines three upfront activities that provide a conceptual and management framework within which a larger-than-usual agile team can add functionality to the software, feature by feature. It is also just as applicable for smaller teams tackling non-trivial problem domains where it is worth spending just a little time to sketch a map of the journey before dashing off down the agile coding highway. Even if you and your team decide not to adopt FDD as a whole, understanding why FDD is the way it is, can provide insight into scaling traditional agile approaches beyond small, largely independent teams. Finally, I would like to say thank you to Serguei Khramtchenko and Mark Lesk at Nebulon for their corrections and suggestions incorporated in this article. References [Brooks] Frederick P. Brooks, Jr., The Mythical Man-Month, Addison Wesley [Cohn] Cohn, Agile Planning and Estimating, Prentice-Hall PTR [FDD] FDD Community Site, www.featuredrivendevelopment.com/ [Kanban] The home of Kanban software development, www.limitedwipsociety.org/ [McConnell] McConnell, Code Complete, Microsoft [Nebulon] The Latest FDD Processes available from www.nebulon.com/articles/fdd/latestprocesses.html [Palmer-1] Palmer, Felsing, A Practical Guide to Feature-Driven Development, Prentice Hall PTR