It's awesome that JUnit is recognizing the usefulness of Hamcrest, because I use these two a lot. However, I find JUnit packaging of their dependencies odd, and can cause class loading problem if you are not careful. Let's take a closer look. If you look at junit:junit:4.10 from Maven Central, you will see that it has this dependencies graph: +- junit:junit:jar:4.10:test | - org.hamcrest:hamcrest-core:jar:1.1:test This is great, except that inside the junit-4.10.jar, you will also find the hamcrest-core-1.1.jar content are embedded! But why??? I suppose it's a convenient for folks who use Ant, so that they save one jar to package in their lib folder, but it's not very Maven friendly. And you also expect classloading trouble if you want to upgrade Hamcrest or use extra Hamcrest modules. Now if you use Hamcrest long enough, you know that most of their goodies are in the second module named hamcrest-library, but this JUnit didn't package in. JUnit however chose to include some JUnit+Hamcrest extension of their own. Now including duplicated classes in jar are very trouble maker, so JUnit has a separated module junit-dep that doesn't include Hamcrest core package and help you avoid this issue. So if you are using Maven project, you should use this instead. junit junit-dep 4.10 test org.hamcrest hamcrest-core org.hamcrest hamcrest-library 1.2.1 test See how I have to exclude hamcrest from junit. This is needed if you want hamcrest-library that has higher version than the one JUnit comes with, which is 1.1. Interesting enough, Maven's dependencies in pom is order sensitive when it comes to auto resolving conflicting versions dependencies. Actually it would just pick the first one found and ignore the rest. So you can shorten above without exclusion if, only if, you place the Hamcrest bofore JUnit like this: org.hamcrest hamcrest-library 1.2.1 test junit junit-dep 4.10 test This should make Maven use the following dependencies: +- org.hamcrest:hamcrest-library:jar:1.2.1:test | \- org.hamcrest:hamcrest-core:jar:1.2.1:test +- junit:junit-dep:jar:4.10:test However I think using the exclusion tag would probably give you more stable build and not rely on Maven implicit ordering rule. And it avoid easy mistake for Maven beginer users. However I wish JUnit would do a better job at packaging and remove duplicated classes in jar. I personally think it's more productive for JUnit to also include hamcrest-libray instead of just the hamcrest-core jar. What do you think?

Curator's note: This tutorial originally appeared at the Windows Azure Java Developer Center. With Windows Azure, you can use a virtual machine to provide server capabilities. As an example, a virtual machine running on Windows Azure can be configured to host a Java application server, such as Apache Tomcat. On completing this guide, you will have an understanding of how to create a virtual machine running on Windows Azure and configure it to run a Java application server. You will learn: How to create a virtual machine. How to remotely log in to your virtual machine. How to install a JDK on your virtual machine. How to install a Java application server on your virtual machine. How to create an endpoint for your virtual machine. How to open a port in the firewall for your application server. For purposes of this tutorial, an Apache Tomcat application server will be installed on a virtual machine. The completed installation will result in a Tomcat installation such as the following. Note To complete this tutorial, you need a Windows Azure account that has the Windows Azure Virtual Machines feature enabled. You can create a free trial account and enable preview features in just a couple of minutes. For details, see Create a Windows Azure account and enable preview features. To create a virtual machine Log in to the Windows Azure Preview Management Portal. Click New. Click Virtual machine. Click Quick create. In the Create virtual machine screen, enter a value for DNS name. From the Image dropdown list, select an image, such as Windows Server 2008 R2 SP1. Enter a password in the New password field, and re-enter it in the Confirm field. This is the Administrator account password. Remember this password, you will use it when you remotely log in to the virtual machine. From the Location drop down list, select the data center location for your virtual machine; for example, West US. Your screen will look similar to the following. Click Create virtual machine. Your virtual machine will be created. You can monitor the status in the Virtual machines section of the management portal. To remotely log in to your virtual machine Log in to the Preview Management Portal. Click Virtual Machines, and then select the MyTestVM1 virtual machine that you previously created. On the command bar, click Connect. Click Open to use the remote desktop protocol file that was automatically created for the virtual machine Click Connect to proceed with the connection process. Type the password that you specified as the password of the Administrator account when you created the virtual machine, and then click OK. Click Yes to verify the identity of the virtual machine. To install a JDK on your virtual machine You can copy a Java Developer Kit (JDK) to your virtual machine, or install a JDK through an installer. For purposes of this tutorial, a JDK will be installed from Oracle's site. Log in to your virtual machine. Within your browser, open http://www.oracle.com/technetwork/java/javase/downloads/index.html. Click the Download button for the JDK that you want to download. For purposes of this tutorial, the Download button for the Java SE 6 Update 32 JDK was used. Accept the license agreement. Click the download executable for Windows x64 (64-bit). Follow the prompts and respond as needed to install the JDK to your virtual machine. To install a Java application server on your virtual machine You can copy a Java application server to your virtual machine, or install a Java application server through an installer. For purposes of this tutorial, a Java application server will be installed by copying a zip file from Apache's site. Log in to your virtual machine. Within your browser, open http://tomcat.apache.org/download-70.cgi. Double-click 64-bit Windows zip. (This tutorial used the zip for Tomcat Apache 7.0.27.) When prompted, choose to save the zip. When the zip is saved, open the folder that contains the zip and double-click the zip. Extract the zip. For purposes of this tutorial, the path used was C:\program files\apache-tomcat-7.0.27-windows-x64. To run the Java application server privately on your virtual machine The following steps show you how to run the Java application server and test it within the virtual machine's browser. It won't be usable by external computers until you create an endpoint and open a port (those steps are described later). Log in to your virtual machine. Add the JDK bin folder to the Pathenvironment variable: Click Windows Start. Right-click Computer. Click Properties. Click Advanced system settings. Click Advanced. Click Environment variables. In the System variables section, click the Path variable and then click Edit. Add a trailing ; to the Path variable value (if there is not one already) and then add c:\program files\java\jdk\bin to the end of the Path variable value (adjust the path as needed if you did not use c:\program files\java\jdk as the path for your JDK installation). Press OK on the opened dialogs to save your Path change. Set the JAVA_HOMEenvironment variable: Click Windows Start. Right-click Computer. Click Properties. Click Advanced system settings. Click Advanced. Click Environment variables. In the System variables section, click New. Create a variable named JRE_HOME and set its value to c:\program files\java\jdk\jre (adjust the path as needed if you did not use c:\program files\java\jdk as the path for your JDK installation). Press OK on the open dialogs to save your JRE_HOME environment variable. Start Tomcat: Open a command prompt. Change the current directory to the Apache Tomcat binfolder. For example: cd c:\program files\apache-tomcat-7.0.27-windows-x64\apache-tomcat-7.0.27\bin (Adjust the path as needed if you used a differrent installation path for Tomcat.) Run catalina.bat start. You should now see Tomcat running if you run the virtual machine's browser and open http://localhost:8080. To see Tomcat running from external machines, you'll need to create an endpoint and open a port. To create an endpoint for your virtual machine Log in to the Preview Management Portal. Click Virtual machines. Click the name of the virtual machine that is running your Java application server. Click Endpoints. Click Add endpoint. In the Add endpoint dialog, ensure Add endpoint is checked and click the Next button. In the New endpoint detailsdialog Specify a name for the endpoint; for example, HttpIn. Specify TCP for the protocol. Specify 80 for the public port. Specify 8080for the private port. Your screen should look similar to the following: Click the Check button to close the dialog. Your endpoint will now be created. To open a port in the firewall for your virtual machine Log in to your virtual machine. Click Windows Start. Click Control Panel. Click System and Security, click Windows Firewall, and then click Advanced Settings. Click Inbound Rules and then click New Rule. For the new rule, select Port for the Rule type and click Next. Select TCP for the protocol and specify 8080 for the port, and click Next. Choose Allow the connection and click Next. Ensure Domain, Private, and Public are checked for the profile and click Next. Specify a name for the rule, such as HttpIn (the rule name is not required to match the endpoint name, however), and then click Finish. At this point, your Tomcat web site should now be viewable from an external browser, using a URL of the form http://your_DNS_name.cloudapp.net, where your_DNS_name is the DNS name you specified when you created the virtual machine. Application lifecycle considerations You could create your own application web archive (WAR) and add it to the webapps folder. For example, create a basic Java Service Page (JSP) dynamic web project and export it as a WAR file, copy the WAR to the Apache Tomcat webapps folder on the virtual machine, then run it in a browser. This tutorial runs Tomcat through a command prompt where catalina.bat start was called. You may instead want to run Tomcat as a service, a key benefit being to have it automatically start if the virtual machine is rebooted. To run Tomcat as a service, you can install it as a service via the service.bat file in the Apache Tomcat bin folder, and then you could set it up to run automatically via the Services snap-in. You can start the Services snap-in by clicking Windows Start, Administrative Tools, and then Services. If you run service.bat install MyTomcat in the Apache Tomcat bin folder, then within the Services snap-in, your service name will appear as Apache Tomcat MyTomcat. By default when the service is installed, it will be set to start manually. To set it to start automatically, double-click the service in the Services snap-in and set Startup Type to Automatic, as shown in the following. You'll need to start the service the first time, which you can do through the Services snap-in (alternatively, you can reboot the virtual machine). Close the running occurrence of catalina.bat start if it is still running before starting the service.

EasyNetQ, my super simple .NET API for RabbitMQ, now (from version 0.7.2.34) supports RabbitMQ clusters without any need to deploy a load balancer. Simply list the nodes of the cluster in the connection string ... var bus = RabbitHutch.CreateBus("host=ubuntu:5672,ubuntu:5673"); In this example I have set up a cluster on a single machine, 'ubuntu', with node 1 on port 5672 and node 2 on port 5673. When the CreateBus statement executes, EasyNetQ will attempt to connect to the first host listed (ubuntu:5672). If it fails to connect it will attempt to connect to the second host listed (ubuntu:5673). If neither node is available it will sit in a re-try loop attempting to connect to both servers every five seconds. It logs all this activity to the registered IEasyNetQLogger. You might see something like this if the first node was unavailable: DEBUG: Trying to connect ERROR: Failed to connect to Broker: 'ubuntu', Port: 5672 VHost: '/'. ExceptionMessage: 'None of the specified endpoints were reachable' DEBUG: OnConnected event fired INFO: Connected to RabbitMQ. Broker: 'ubuntu', Port: 5674, VHost: '/' If the node that EasyNetQ is connected to fails, EasyNetQ will attempt to connect to the next listed node. Once connected, it will re-declare all the exchanges and queues and re-start all the consumers. Here's an example log record showing one node failing then EasyNetQ connecting to the other node and recreating the subscribers: INFO: Disconnected from RabbitMQ Broker DEBUG: Trying to connect DEBUG: OnConnected event fired DEBUG: Re-creating subscribers INFO: Connected to RabbitMQ. Broker: 'ubuntu', Port: 5674, VHost: '/' You get automatic fail-over out of the box. That’s pretty cool. If you have multiple services using EasyNetQ to connect to a RabbitMQ cluster, they will all initially connect to the first listed node in their respective connection strings. For this reason the EasyNetQ cluster support is not really suitable for load balancing high throughput systems. I would recommend that you use a dedicated hardware or software load balancer instead, if that’s what you want.

When working with Entity Framework - Code First model approach, a developer creates POCO entities for database tables. The benefit of using Code First model is to have POCO entity for each table that can be used as either WCF Data Contracts or you can apply your own custom attributes to handle Security, Logging, etc. and there is no mapping needed as we used to do in Entity Framework (Model First) approach if the application architecture is n-tier based. Considering the Data Access layer, we will implement a repository pattern that encapsulates the persistence logic in a separate class. This class will be responsible to perform database operations. Let’s suppose the application is based on n-tier architecture and having 3 tiers namely Presentation, Business and Data Access. Common library contains all our POCO entities that will be used by all the layers. Presentation Layer: Contains Views, Forms Business Layer: Managers that handle logic functionality Data Access Layer: Contains Repository class that handles CRUD operations Common Library: Contain POCO entities. We will implement an interface named “IRepository” that defines the signature of all the appropriate generic methods needed to perform CRUD operation and then implement the Repository class that defines the actual implementation of each method. We can also instantiate Repository object using Dependency Injection or apply Factory pattern. Code Snippet: IRepository public interface IRepository : IDisposable { /// /// Gets all objects from database /// /// IQueryable All() where T : class; /// /// Gets objects from database by filter. /// /// Specified a filter /// IQueryable Filter(Expression> predicate) where T : class; /// /// Gets objects from database with filting and paging. /// /// /// Specified a filter /// Returns the total records count of the filter. /// Specified the page index. /// Specified the page size /// IQueryable Filter(Expression> filter, out int total, int index = 0, int size = 50) where T : class; /// /// Gets the object(s) is exists in database by specified filter. /// /// Specified the filter expression /// bool Contains(Expression> predicate) where T : class; /// /// Find object by keys. /// /// Specified the search keys. /// T Find(params object[] keys) where T : class; /// /// Find object by specified expression. /// /// /// T Find(Expression> predicate) where T : class; /// /// Create a new object to database. /// /// Specified a new object to create. /// T Create(T t) where T : class; /// /// Delete the object from database. /// /// Specified a existing object to delete. int Delete(T t) where T : class; /// /// Delete objects from database by specified filter expression. /// /// /// int Delete(Expression> predicate) where T : class; /// /// Update object changes and save to database. /// /// Specified the object to save. /// int Update(T t) where T : class; /// /// Select Single Item by specified expression. /// /// /// /// T Single(Expression> expression) where T : class; void SaveChanges(); void ExecuteProcedure(String procedureCommand, params SqlParameter[] sqlParams); } Code Snippet: Repository public class Repository : IRepository { DbContext Context; public Repository() { Context = new DBContext(); } public Repository(DBContext context) { Context = context; } public void CommitChanges() { Context.SaveChanges(); } public T Single(Expression> expression) where T : class { return All().FirstOrDefault(expression); } public IQueryable All() where T : class { return Context.Set().AsQueryable(); } public virtual IQueryable Filter(Expression> predicate) where T : class { return Context.Set().Where(predicate).AsQueryable(); } public virtual IQueryable Filter(Expression> filter, out int total, int index = 0, int size = 50) where T : class { int skipCount = index * size; var _resetSet = filter != null ? Context.Set().Where(filter).AsQueryable() : Context.Set().AsQueryable(); _resetSet = skipCount == 0 ? _resetSet.Take(size) : _resetSet.Skip(skipCount).Take(size); total = _resetSet.Count(); return _resetSet.AsQueryable(); } public virtual T Create(T TObject) where T : class { var newEntry = Context.Set().Add(TObject); Context.SaveChanges(); return newEntry; } public virtual int Delete(T TObject) where T : class { Context.Set().Remove(TObject); return Context.SaveChanges(); } public virtual int Update(T TObject) where T : class { try { var entry = Context.Entry(TObject); Context.Set().Attach(TObject); entry.State = EntityState.Modified; return Context.SaveChanges(); } catch (OptimisticConcurrencyException ex) { throw ex; } } public virtual int Delete(Expression> predicate) where T : class { var objects = Filter(predicate); foreach (var obj in objects) Context.Set().Remove(obj); return Context.SaveChanges(); } public bool Contains(Expression> predicate) where T : class { return Context.Set().Count(predicate) > 0; } public virtual T Find(params object[] keys) where T : class { return (T)Context.Set().Find(keys); } public virtual T Find(Expression> predicate) where T : class { return Context.Set().FirstOrDefault(predicate); } public virtual void ExecuteProcedure(String procedureCommand, params SqlParameter[] sqlParams){ Context.Database.ExecuteSqlCommand(procedureCommand, sqlParams); } public virtual void SaveChanges() { Context.SaveChanges(); } public void Dispose() { if (Context != null) Context.Dispose(); } } The benefit of using Repository pattern is that all the database operations will be managed centrally and in future if you want to change the underlying database connector you can add another Repository class and defines its own implementation or change the existing one.

How to allow instance methods to run as JUnit BeforeClass behavior.

JUnit 4.4 added a new assertion mechanism with the method assertThat(). Have a look and consider using it in place of assertEquals(). assertThat(result, is(42)); assertThat(output, containsString("foo"));

Curator's note: This article originally appeared at WindowsAzure.com. To use this feature and other new Windows Azure capabilities, sign up for the free preview. Just as you can quickly create and deploy a web application created from the gallery, you can also deploy a website created on a workstation with traditional developer tools from Microsoft or other companies. Table of Contents Deployment Options How to: Create a Website Using the Management Portal How to: Create a Website from the Gallery How to: Delete a Website Next Steps Deployment Options Windows Azure supports deploying websites from remote computers using WebDeploy, FTP, GIT or TFS. Many development tools provide integrated support for publication using one or more of these methods and may only require that you provide the necessary credentials, site URL and hostname or URL for your chosen deployment method. Credentials and deployment URLs for all enabled deployment methods are stored in the website's publish profile, a file which can be downloaded in the Windows Azure (Preview) Management Portal from the Quick Start page or the quick glance section of the Dashboard page. If you prefer to deploy your website with a separate client application, high quality open source GIT and FTP clients are available for download on the Internet for this purpose. How to: Create a Website Using the Management Portal Follow these steps to create a website in Windows Azure. Login to the Windows Azure (Preview) Management Portal. Click the Create New icon on the bottom left of the Management Portal. Click the Web Site icon, click the Quick Create icon, enter a value for URL and then click the check mark next to create web site on the bottom right corner of the page. When the website has been created you will see the text Creation of Web Site '[SITENAME]' Completed. Click the name of the website displayed in the list of websites to open the website's Quick Start management page. On the Quick Start page you are provided with options to set up TFS or GIT publishing if you would like to deploy your finished website to Windows Azure using these methods. FTP publishing is set up by default for websites and the FTP Host name is displayed under FTP Hostname on the Quick Start and Dashboard pages. Before publishing with FTP or GIT choose the option to Reset deployment credentials on the Dashboard page. Then specify the new credentials (username and password) to authenticate against the FTP Host or the Git Repository when deploying content to the website. The Configure management page exposes several configurable application settings in the following sections: Framework: Set the version of .NET framework or PHP required by your web application. Diagnostics: Set logging options for gathering diagnostic information for your website in this section. App Settings: Specify name/value pairs that will be loaded by your web application on start up. For .NET sites, these settings will be injected into your .NET configuration AppSettings at runtime, overriding existing settings. For PHP and Node sites these settings will be available as environment variables at runtime. Connection Strings: View connection strings for linked resources. For .NET sites, these connection strings will be injected into your .NET configuration connectionStrings settings at runtime, overriding existing entries where the key equals the linked database name. For PHP and Node sites these settings will be available as environment variables at runtime. Default Documents: Add your web application's default document to this list if it is not already in the list. If your web application contains more than one of the files in the list then make sure your website's default document appears at the top of the list. How to: Create a Website from the Gallery The gallery makes available a wide range of popular web applications developed by Microsoft, third party companies, and open source software initiatives. Web applications created from the gallery do not require installation of any software other than the browser used to connect to the Windows Azure Management Portal. In this tutorial, you'll learn: How to create a new site through the gallery. How to deploy the site through the Windows Azure Portal. You'll build a Word press blog that uses a default template. The following illustration shows the completed application: Note To complete this tutorial, you need a Windows Azure account that has the Windows Azure Web Sites feature enabled. You can create a free trial account and enable preview features in just a couple of minutes. For details, see Create a Windows Azure account and enable preview features. Create a web site in the portal Login to the Windows Azure Management Portal. Click the New icon on the bottom left of the dashboard. Click the Web Site icon, and click From Gallery. Locate and click the WordPress icon in list, and then click Next. On the Configure Your App page, enter or select values for all fields: Enter a URL name of your choice Leave Create a new MySQL database selected in the Database field Select the region closest to you Then click Next. On the Create New Database page, you can specify a name for your new MySQL database or use the default name. Select the region closest to you as the hosting location. Select the box at the bottom of the screen to agree to ClearDB's usage terms for your hosted MySQL database. Then click the check to complete the site creation. After you click Complete Windows Azure will initiate build and deploy operations. While the web site is being built and deployed the status of these operations is displayed at the bottom of the Web Sites page. After all operations are performed, A final status message when the site has been successfully deployed. Launch and manage your WordPress site Click on your new site from the Web Sites page to open the dashboard for the site. On the Dashboard management page, scroll down and click the link on the left under Site Url to open the site’s welcome page. Enter appropriate configuration information required by WordPress and click Install WordPress to finalize configuration and open the web site’s login page. Login to the new WordPress web site by entering the username and password that you specified on the Welcome page. You'll have a new WordPress site that looks similar to the site below. How to: Delete a Website Websites are deleted using the Delete icon in the Windows Azure Management Portal. The Delete icon is available in the Windows Azure Portal when you click Web Sites to list all of your websites and at the bottom of each of the website management pages. Next Steps For more information about Websites, see the following: Walkthrough: Troubleshooting a Website on Windows Azure

Exception Testing Why test exception flows? Just like with all of your code, test coverage writes a contract between your code and the business functionality that the code is supposed to produce leaving you with a living documentation of the code along with the added ability to stress the functionality early and often. I won't go into the many benefits of testing instead I will focus on just Exception Testing. There are many ways to test an exception flow thrown from a piece of code. Lets say that you have a guarded method that requires an argument to be not null. How would you test that condition? How do you keep JUnit from reporting a failure when the exception is thrown? This blog covers a few different methods culminating with JUnit's ExpectedException implemented with JUnit's @Rule functionality. The "old" way In a not so distant past the process to test an exception required a dense amount of boilerplate code in which you would start a try/catch block, report a failure if your code did not produce the expected behavior and then catch the exception looking for the specific type. Here is an example: public class MyObjTest { @Test public void getNameWithNullValue() { try { MyObj obj = new MyObj(); myObj.setName(null); fail("This should have thrown an exception"); } catch (IllegalArgumentException e) { assertThat(e.getMessage().equals("Name must not be null")); } } } As you can see from this old example, many of the lines in the test case are just to support the lack of functionality present to specifically test exception handling. One good point to make for the try/catch method is the ability to test the specific message and any custom fields on the expected exception. We will explore this a bit further down with JUnit's ExpectedException and @Rule annotation. JUnit adds expected exceptions JUnit responded back to the users need for exception handling by adding a @Test annotation field "expected". The intention is that the entire test case will pass if the type of exception thrown matched the exception class present in the annotation. public class MyObjTest { @Test(expected = IllegalArgumentException.class) public void getNameWithNullValue() { MyObj obj = new MyObj(); myObj.setName(null); } } As you can see from the newer example, there is quite a bit less boiler plate code and the test is very concise, however, there are a few flaws. The main flaw is that the test condition is too broad. Suppose you have two variables in a signature and both cannot be null, then how do you know which variable the IllegalArgumentException was thrown for? What happens when you have extended a Throwable and need to check for the presence of a field? Keep these in mind as you read further, solutions will follow. JUnit @Rule and ExpectedException If you look at the previous example you might see that you are expecting an IllegalArgumentException to be thrown, but what if you have a custom exception? What if you want to make sure that the message contains a specific error code or message? This is where JUnit really excelled by providing a JUnit @Rule object specifically tailored to exception testing. If you are unfamiliar with JUnit @Rule, read the docs here. ExpectedException JUnit provides a JUnit class ExpectedException intended to be used as a @Rule. The ExpectedException allows for your test to declare that an exception is expected and gives you some basic built in functionality to clearly express the expected behavior. Unlike the @Test(expected) annotation feature, ExpectedException class allows you to test for specific error messages and custom fields via the Hamcrest matchers library. An example of JUnit's ExpectedException import org.junit.rules.ExpectedException; public class MyObjTest { @Rule public ExpectedException thrown = ExpectedException.none(); @Test public void getNameWithNullValue() { thrown.expect(IllegalArgumentException.class); thrown.expectMessage("Name must not be null"); MyObj obj = new MyObj(); obj.setName(null); } } As I eluded to above, the framework allows you to test for specific messages ensuring that the exception being thrown is the case that the test is specifically looking for. This is very helpful when the nullability of multiple arguments is in question. Custom Fields Arguably the most useful feature of the ExpectedException framework is the ability to use Hamcrest matchers to test your custom/extended exceptions. For example, you have a custom/extended exception that is to be thrown in a method and inside the exception has an "errorCode". How do you test that functionality without introducing the boiler plate code from the try/catch block listed above? How about a custom Matcher! This code is available at: https://github.com/mike-ensor/custom-exception-testing Solution: First the test case import org.junit.rules.ExpectedException; public class MyObjTest { @Rule public ExpectedException thrown = ExpectedException.none(); @Test public void someMethodThatThrowsCustomException() { thrown.expect(CustomException.class); thrown.expect(CustomMatcher.hasCode("110501")); MyObj obj = new MyObj(); obj.methodThatThrowsCustomException(); } } Solution: Custom matcher import com.thepixlounge.exceptions.CustomException; import org.hamcrest.Description; import org.hamcrest.TypeSafeMatcher; public class CustomMatcher extends TypeSafeMatcher { public static BusinessMatcher hasCode(String item) { return new BusinessMatcher(item); } private String foundErrorCode; private final String expectedErrorCode; private CustomMatcher(String expectedErrorCode) { this.expectedErrorCode = expectedErrorCode; } @Override protected boolean matchesSafely(final CustomException exception) { foundErrorCode = exception.getErrorCode(); return foundErrorCode.equalsIgnoreCase(expectedErrorCode); } @Override public void describeTo(Description description) { description.appendValue(foundErrorCode) .appendText(" was not found instead of ") .appendValue(expectedErrorCode); } } NOTE: Please visit https://github.com/mike-ensor/custom-exception-testing to get a copy of a working Hamcrest Matcher, JUnit @Rule and ExpectedException. And there you have it, a quick overview of different ways to test Exceptions thrown by your code along with the ability to test for specific messages and fields from within custom exception classes. Please be specific with your test cases and try to target the exact case you have setup for your test, remember, tests can save you from introducing side-effect bugs!

This is the first in what will be a series of posts on how to establish a continuous delivery pipeline. The eventual goal is to have an app that we can push out into production anytime we like, safely and with little effort. But we’re going to start small and proceed incrementally, which is the best way to undertake this sort of effort. In this post we’ll start by establishing continuous integration for an arbitrary Java/Maven-based open source library project. It shouldn’t take more than an afternoon to do this more or less from scratch if you’re reasonably familiar with the technologies in question, except for the part where we have to get a Maven repo. We have to wait for Sonatype to approve the repo, but they’re pretty fast about it. I set this up for an open source library project I’m doing called Kite. The details of the project itself aren’t important for this exercise, but the open source library part is: Libraries (as opposed to apps) are easier to deal with, because deployment is just a matter of getting the binary into a Maven repo, as opposed to getting it running on a live server. Open source means that we can get a bunch of infrastructure freebies. Here’s what it will look like at the end of this post: So if you have an open source library you’ve been wanting to develop, now’s the time to get started! Find a place to host your project sources The right host depends on which SCM you prefer. If you like Git, then GitHub and Bitbucket are two obvious (free) choices. Bitbucket supports Mercurial as well, and it supports not just free public repos like GitHub, but free private repos as well. Anyway, I chose GitHub for Kite. Here’s the repo: GitHub Kite repo Set up a continuous integration server The next step is to set up a continuous integration server. Hudson and Jenkins are both pretty popular open source offerings, but I chose Atlassian’s Bamboo because the UI is a lot more polished. Bamboo is free for open source projects, and even if you decide to buy it, it’s only $10 for the starter license (proceeds go to charity), which gives you a local build agent. Anyway, you can’t go wrong with any of those, so choose one that makes sense and set it up. They’re all easy to set up. For Bamboo, you’ll need to install Java, Tomcat (or whichever container you like), Git and Maven 3 on the server as well. Bamboo is a Java web app, which is why you need Java and Tomcat. Bamboo uses Git to pull the code down from GitHub, and Maven to build the code. You’ll need to configure this using the Bamboo admin console, once you’ve installed the packages above, you can do the configuration entirely through the Bamboo UI and it’s very easy. After installing Ubuntu 11.10 server manually, I used Chef to set up everything on my Bamboo server except for deploying the Bamboo WAR itself, since Chef has community cookbooks for Java, Tomcat, Maven and Git. (Note however that currently the URL in the Maven 3 recipe is wrong, so if you go that way then you’ll need to update the URL and the checksum.) Jo Liss has a great tutorial on Chef Solo if you’re interested in giving this a shot. But you don’t have to use Chef. You can just use your native package manager if you prefer to do it that way. Once you have the executables all set up, you’ll need to create a build plan that slurps source code from GitHub and builds it with a Maven 3 task. In the screenshot you’ll notice that I created three separate stages here: a commit stage, an acceptance stage and a deploy stage. The idea, following Continuous Delivery, is to separate the fast-but-not-so-comprehensive feedback part from the slow-but-more-comprehensive feedback, and run those as different stages in the build. That way you know right away in most cases where you break the build (commit stage fails), and you still find out soon enough even in those cases where the breakage involves some kind of integration or business acceptance criterion. So in my Bamboo plan I do the commit stage first, then run integration tests during the acceptance stage, and finally deploy the snapshot build to my Maven repo using mvn deploy if the previous two stages pass. That makes it available to others for continuous integration. Let’s look at the Maven repo part now. Set up a Maven repo Sonatype offers a hosted Nexus-based Maven repo for free to open source projects. Moreover they’ll push your artifacts out to Maven central for you as well. So if that sounds good to you, here are instructions on signing up. JFrog’s Artifactory is an option as well. As you can see from the link, there’s an open source option. We use this at work and it’s pretty nice. For Sonatype, you’ll need to create a JIRA ticket to get the repo, as per the instructions above. Here’s mine. Also, your POM will need to conform to certain requirements; see this example for the POM that I’m using for Kite. Nothing too out-of-the-ordinary, though do take note of the parent POM. Anyway, once you have your Maven repo, practice manually deploying from the command line via mvn deploy just to make sure you’re able to push code. If it works, then you’ll want to try it out from Bamboo. Be sure to update Bamboo’s copy of Maven’s settings.xml so Bamboo can authenticate into the Sonatype repo: sonatype-nexus-snapshots your_username your_password sonatype-nexus-staging your_username your_password Once you have it working, you should be able to push your snapshots over to Sonatype. Here’s the Nexus UI, and here’s the raw directory browser view. Conclusion Though it takes a bit of effort to set this up, at the end of the day you have a good foundation for your continuous delivery efforts. In the next post in this series, we’ll expand our deployment pipeline so it can handle pushing a web application to a live, cloud-based container.

A couple of times recently I’ve needed to set up a .Net application to use Registration-Free COM, and each time I’ve had to hunt around to recall the details. Furthermore, just this week I needed to write some unit tests that involve instantiating these un-registered COM objects, and that wasn’t straightforward. So, as much for the benefit of my future self as for you, my loyal reader, I’m going to summarise my know-how in quick blog post before it becomes used-to-know-how. What is Registration-Free COM? If you’re still reading, I’ll assume you know all about COM, Microsoft’s ancient technology for enabling components written in different languages to talk to each other (I wrote a little about it here, with some links to introductory articles). You are probably also aware of DLL Hell. That isn’t a place where bad executables are sent when they are terminated. Rather, it was a pain inflicted on developers by the necessity of registering COM components (and other DLLs) in a central place in the OS. Since all components were dumped into the same pool, one application could cause all kinds of hell for others by registering different versions of shared DLLs. The OS doesn’t police this pool, and it certainly doesn’t enforce compatibility, so much unexpected weird and wonderful behaviour was the result. Starting with Windows XP, it has been possible to more-or-less escape this hell by not registering components in a central location, and instead using Registration-Free COM. This makes it much easier to deploy applications, because you can just copy a bunch of files – RegSvr32 is not involved, and there are no Registry keys to be written. You can be confident that your application will have no impact on others once installed. It is all done using manifests. Individual Manifest Files For each dll, or ocx file (or ax files in my case – I’m working with DirectShow filters) containing COM components you need to create a manifest. Suppose your dll is called MyCOMComponent.dll. Your manifest file should be called MyCOMComponent.sxs.manifest, and it should contain the following: Obviously you need to make sure that the clsid inside comClass is correct for your component. If you have more than one COM object in your dll you can add multiple comClass elements. For those not wanting to generate these manifests by hand, a StackOverflow answer lists some tools that might help. About Deployment When you deploy your application you should deploy both the dll/ocx/ax file and its manifest into the same directory as your .Net exe/dlls. When developing in Visual Studio, I customise the build process to make sure all these dlls get copied into the correct place for running and debugging the application. I stole the technique for doing this from the way ASP.Net MVC applications manage their dlls. Put all the dlls and manifests into a folder called _bin_deployableAssemblies alongside the rest of your source code. Then modify your csproj file and add the following Target at the end of it: To make sure that target is called when you build, update the AfterBuild target (uncomment it first if you’re not currently using it): The Application Manifest Now you need to make sure your application declares its dependencies. First add an app.manifest file to your project, if you haven’t already got one. To do this in Visual Studio, right click the project, select Add –> New Item … and then choose Application Manifest File. Having added the manifest, you need to ensure it is compiled into your executable. You do this by right-clicking the project, choosing Properties, then going to the Application tab. In the resources section you’ll see a Manifest textbox: make sure your app.manifest file is selected. Now you need to add a section to the app.manifest file for each dependency. By default your app.manifest file will probably already have a dependency for the Windows Common Controls. After that (so, nested directly inside the root element) you should add the following for each of the manifest files you created earlier: Notice that we drop the “.manifest” off the end of the manifest file name when we refer to it here. The other important thing is that the version number here and the one in the manifest file should exactly match, though I don’t think there’s any reason to change it from 1.0.0.0. Disabling the Visual Studio Hosting Process There’s just one more thing to do before you try running your application, and that is to turn off the Visual Studio hosting process. The hosting process apparently helps improve debugging performance, amongst other things (though I’ve not noticed greatly decreased performance with it disabled). The problem is that, when enabled, application executables are not loaded directly- rather, they are loaded by an intermediary executable with a name ending .vshost.exe. The upshot is that the manifest embedded in your exe is ignored, and COM components are not loaded. Disabling the hosting process is simple: go to the Debug tab of your project’s Properties and uncheck “Enable the Visual Studio hosting process” With everything set up, you’ll want to try running your application. If you got everything right first time, everything will go smoothly. If not you might see an error like this: If you do, check Windows’ Application event log for errors coming from SideBySide. These are usually pretty helpful in telling you which part of your configuration has a problem. Summary To re-cap briefly, here are the steps to enabling Registration-Free COM for you application: Create a manifest file for each COM dll Make sure both COM dlls and manifest files are deployed alongside your main executable Add a manifest file to your executable which references each individual manifest file Make sure you turn off the Visual Studio hosting process before debugging Unit Testing and Registration-Free COM And now, as promised, a word about running Unit Tests when Registration-Free COM is involved. If you have a Unit Test which tries to create a Registration-Free COM object you’ll probably get an exception like Retrieving the COM class factory for component with CLSID {1C123B56-3774-4EE4-A482-512B3AB7CABB} failed due to the following error: 80040154 Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG)). If you don’t get this error, it’s probably because the component is still registered centrally on your machine. Running regsvr32 /u [Path_to_your_dll] will unregister it. Why do Unit Tests fail, when the application works? It is for the same reason that the Visual Studio hosting process breaks Registration-Free COM: your unit tests are actually being run in a different process (for example, the Resharper.TaskRunner), and the manifest file which you so carefully crafted for your exe is being ignored. Only the manifest on the entry executable is taken into account, and since that’s a generic unit test runner it says nothing about your COM dependencies. But there’s a workaround. Win32 has some APIs –the Activation Context APIs- which allow you to manually load up a manifest for each thread which needs to create COM components. Spike McLarty has written some code to make these easy to use from .Net, and I’ll show you a technique to incorporate this into your code so that it works correctly whether called from unit tests or not. Here’s Spike’s code, with a few minor modifications of my own: /// /// Code from http://www.atalasoft.com/blogs/spikemclarty/february-2012/dynamically-testing-an-activex-control-from-c-and /// class ActivationContext { static public void UsingManifestDo(string manifest, Action action) { UnsafeNativeMethods.ACTCTX context = new UnsafeNativeMethods.ACTCTX(); context.cbSize = Marshal.SizeOf(typeof(UnsafeNativeMethods.ACTCTX)); if (context.cbSize != 0x20) { throw new Exception("ACTCTX.cbSize is wrong"); } context.lpSource = manifest; IntPtr hActCtx = UnsafeNativeMethods.CreateActCtx(ref context); if (hActCtx == (IntPtr)(-1)) { throw new Win32Exception(Marshal.GetLastWin32Error()); } try // with valid hActCtx { IntPtr cookie = IntPtr.Zero; if (!UnsafeNativeMethods.ActivateActCtx(hActCtx, out cookie)) { throw new Win32Exception(Marshal.GetLastWin32Error()); } try // with activated context { action(); } finally { UnsafeNativeMethods.DeactivateActCtx(0, cookie); } } finally { UnsafeNativeMethods.ReleaseActCtx(hActCtx); } } [SuppressUnmanagedCodeSecurity] internal static class UnsafeNativeMethods { // Activation Context API Functions [DllImport("Kernel32.dll", SetLastError = true, EntryPoint = "CreateActCtxW")] internal extern static IntPtr CreateActCtx(ref ACTCTX actctx); [DllImport("Kernel32.dll", SetLastError = true)] [return: MarshalAs(UnmanagedType.Bool)] internal static extern bool ActivateActCtx(IntPtr hActCtx, out IntPtr lpCookie); [DllImport("kernel32.dll", SetLastError = true)] [return: MarshalAs(UnmanagedType.Bool)] internal static extern bool DeactivateActCtx(int dwFlags, IntPtr lpCookie); [DllImport("Kernel32.dll", SetLastError = true)] internal static extern void ReleaseActCtx(IntPtr hActCtx); // Activation context structure [StructLayout(LayoutKind.Sequential, Pack = 4, CharSet = CharSet.Unicode)] internal struct ACTCTX { public Int32 cbSize; public UInt32 dwFlags; public string lpSource; public UInt16 wProcessorArchitecture; public UInt16 wLangId; public string lpAssemblyDirectory; public string lpResourceName; public string lpApplicationName; public IntPtr hModule; } } } The method UsingManifestDo allows you to run any code of your choosing with an Activation Context loaded from a manifest file. Clearly we only need to invoke this when our code is being called from a Unit Test. But how do we structure code elegantly so that it uses the activation context when necessary, but not otherwise? Here’s my solution: public static class COMFactory { private static Func, object> _creationWrapper = function => function(); public static T CreateComObject() where T:new() { var instance = (T)_creationWrapper(() => new T()); return instance; } public static object CreateComObject(Guid guid) { Type type = Type.GetTypeFromCLSID(guid); var instance = _creationWrapper(() => Activator.CreateInstance(type)); return instance; } public static void UseManifestForCreation(string manifest) { _creationWrapper = function => { object result = null; ActivationContext.UsingManifestDo(manifest, () => result = function()); return result; }; } } Whenever I need to create a COM Object in my production code, I do it by calling COMFactory.CreateCOMObject. By default this will create the COM objects directly, relying on the manifest which is embedded in the executable. But in my Test project, before running any tests I call COMFactory.UseManifestForCreation and pass in the path to the manifest file. This ensures that the manifest gets loaded up before we try to create any COM objects in the tests. To avoid duplicating the manifest file, I share the same file between my Test project and main executable project. You can do this right clicking your test project, choosing Add->Existing Item… then app.manifest in your main project. Finally, click the down arrow on the Add split button, and choose Add as Link. If you’ve got any tips to share on using Registration-Free COM, whether in Unit Tests or just in applications, please do leave a comment.

People tell me A and B, They tell me how I have to see, Things that I have seen already clear, So they push me then from side to side (I Want Out - Helloween) Developing an application that uses RESTful web API may imply developing server and client side. Writing integration tests for the server side can be as easy as using Arquillian to start up server and REST-assured to test that the services works as expected. The problem is how to test the client side. In this post, we are going to see how to test the client side apart from using mocks. As a brief description, to test the client side, what we need is a local server which can return recorded JSON responses. The rest-client-driver is a library which simulates a RESTful service. You can set expectations on the HTTP requests you want to receive during a test. So it is exactly what we need for our java client side. Note that this project is really helpful to write tests when we are developing RESTful web clients for connecting to services developed by third parties like Flickr Rest API, Jira Rest API, Github ... First thing to do is adding rest-client-driver dependency: com.github.rest-driver rest-client-driver 1.1.27 test Next step we are going to create a very simple Jersey application which simply invokes a get method to required URI. public class GithubClient { private static final int HTTP_STATUS_CODE_OK = 200; private String githubBaseUri; public GithubClient(String githubBaseUri) { this.githubBaseUri = githubBaseUri; } public String invokeGetMethod(String resourceName) { Client client = Client.create(); WebResource webResource = client.resource(githubBaseUri+resourceName); ClientResponse response = webResource.type("application/json") .accept("application/json").get(ClientResponse.class); int statusCode = response.getStatus(); if(statusCode != HTTP_STATUS_CODE_OK) { throw new IllegalStateException("Error code "+statusCode); } return response.getEntity(String.class); } } And now we want to test that invokeGetMethod really gets the required resource. Let's suppose that this method in production code will be responsible of getting all issues name from a project registered on github. Now we can start to write the test: @Rule public ClientDriverRule driver = new ClientDriverRule(); @Test public void issues_from_project_should_be_retrieved() { driver.addExpectation( onRequestTo("/repos/lordofthejars/nosqlunit/issues"). withMethod(Method.GET), giveResponse(GET_RESPONSE)); GithubClient githubClient = new GithubClient(driver.getBaseUrl()); String issues = githubClient.invokeGetMethod("/repos/lordofthejars/nosqlunit/issues"); assertThat(issues, is(GET_RESPONSE)); } We use ClientDriverRule @Rule annotation to add the client-driver to a test. And then using methods provided by RestClientDriver class, expectations are recorded. See how we are setting the base URL using driver.getBaseUrl() With rest-client-driver we can also record http status response using giveEmptyResponse method: @Test(expected=IllegalStateException.class) public void http_errors_should_throw_an_exception() { driver.addExpectation( onRequestTo("/repos/lordofthejars/nosqlunit/issues") .withMethod(Method.GET), giveEmptyResponse().withStatus(401)); GithubClient githubClient = new GithubClient(driver.getBaseUrl()); githubClient.invokeGetMethod("/repos/lordofthejars/nosqlunit/issues"); } And obviously we can record a put action: Note that in this example, we are setting that our request should contain given message body to response a 204 status code. This is a very simple example, but keep in mind that also works with libraries like gson or jackson. Also rest-driver project comes with a module that can be used to assert server responses (like REST-assured project) but this topic will be addressed into another post. I wish you have found this post useful. We keep learning, Alex.

when writing testable code, your first port of call is often to abstract any dependencies and make them easy to mock. this is the same for any of your codebase that talks to ftp servers. testing the way your code behaves under real world conditions makes integration tests important regardless of abstraction, though. here’s a simple trick to test ftp code in the wild. a recent project of mine has involved writing code that talks to ftp servers with the goal of adding additional continuous integration automation to a project. although all of my main methods are easily abstracted and injectable, my project still needs to actually talk to ftp servers at the end of the day, and i need to test that these very methods do the right thing when they are met with different conditions; be they bad credentials, lack of read/write permissions etc. the challenge integration tests can be brittle at the best of times, so ensuring that they are repeatable and can be setup and torn down can often be almost as much of a challenge as writing your actual code itself. an ftp server is usually a static service that is installed on a server. you might think that running one and ensuring it stays up and doesn’t get hacked just so that all your integration tests work is a necessary evil, but there is an easier way. run local. run often. i was running an ftp server on my build server just so that it was “always around” for my tests until i stumbled across an interesting project over on github to do just this . the approach i'm about to show you doesn’t need you to go to the effort of running a dedicated server at all. all you need to do is add a single executable to your unit test project and wrap your unit test in a using statement. the ftp server executable is a single file ftp server called ftpdmin which offers a read/write ftp server that can be fired up from the command line with a minimum feature set and only a few command line parameters to make it all tick. by implementing idisposable the helper class that wraps around this command line exe allows you to take advantage of the using() pattern to take care of your executable’s lifetime and have it die when your code is done testing. steps to make it happen download ftpdmin from here . add the exe to the root of your test project (you can put this anywhere, but you’ll have to update the helper class below). now add the exe to your project (i.e “view all items” in your test project’s solution explorer, and add the exe). set the exe to “copy always” in it’s solution properties. add the following code to a helper class in your test project: public class ftptestserver: idisposable { private readonly process ftpprocess; public ftptestserver(string rootdirectory, int port = 21, bool allowuploads = true) { var psinfo = new processstartinfo { filename = appdomain.currentdomain.basedirectory + "\\ftpdmin.exe", arguments = string.format("-p {0} -ha 127.0.0.1 \"{1}\" {2}", port, rootdirectory, allowuploads ? string.empty : "-g"), windowstyle = processwindowstyle.hidden }; ftpprocess = process.start(psinfo); } public void dispose() { if (ftpprocess.hasexited) return; ftpprocess.kill(); ftpprocess.waitforexit(); } } now you can enjoy being able to write really clean integration testing code that starts and ftp server every time you run your tests and then tear it down when your test is done. an example integration test showing connecting to “127.0.0.1”: [testmethod] public void ftpcode_upload_canconnect() { try { // fire up a new ftp server instance using (new ftptestserver(rootdirectory: "./")) { // code that talks to an ftp server on 127.0.0.1 } } catch (webexception e) { assert.fail("failed to connect to our ftp server"); } } how awesome is that? the power of using ftpdmin is that it can be told to deny write permissions to simulate bad user permissions as well: [testmethod] public void ftpcode_upload_throwswebexception() { try { // fire up a new ftp server instance using (new ftptestserver(rootdirectory: "./", allowuploads: false)) { // code that talks to an ftp server on 127.0.0.1 } } catch (webexception e) { assert.fail("our code failed to upload a file because of invalid permissions"); } } all in all, the above has been a complete lifesaver when it comes to making my integration test projects portable – if a new developer joins my project, they instantly get access to my ftp test harness just by pulling down my project’s source code.

Spring Data is a very convenient library. However, as the project as quite new, it is not well featured. By default, Spring Data JPA will provide implementation of the DAO based on SimpleJpaRepository. In recent project, I have developed a customize repository base class so that I could add more features on it. You could add vendor specific features to this repository base class as you like. Configuration You have to add the following configuration to you spring beans configuration file. You have to specified a new repository factory class. We will develop the class later. extends SimpleJpaRepository implements GenericRepository , Serializable{ private static final long serialVersionUID = 1L; static Logger logger = Logger.getLogger(GenericRepositoryImpl.class); private final JpaEntityInformation entityInformation; private final EntityManager em; private final DefaultPersistenceProvider provider; private Class springDataRepositoryInterface; public Class getSpringDataRepositoryInterface() { return springDataRepositoryInterface; } public void setSpringDataRepositoryInterface( Class springDataRepositoryInterface) { this.springDataRepositoryInterface = springDataRepositoryInterface; } /** * Creates a new {@link SimpleJpaRepository} to manage objects of the given * {@link JpaEntityInformation}. * * @param entityInformation * @param entityManager */ public GenericRepositoryImpl (JpaEntityInformation entityInformation, EntityManager entityManager , Class springDataRepositoryInterface) { super(entityInformation, entityManager); this.entityInformation = entityInformation; this.em = entityManager; this.provider = DefaultPersistenceProvider.fromEntityManager(entityManager); this.springDataRepositoryInterface = springDataRepositoryInterface; } /** * Creates a new {@link SimpleJpaRepository} to manage objects of the given * domain type. * * @param domainClass * @param em */ public GenericRepositoryImpl(Class domainClass, EntityManager em) { this(JpaEntityInformationSupport.getMetadata(domainClass, em), em, null); } public S save(S entity) { if (this.entityInformation.isNew(entity)) { this.em.persist(entity); flush(); return entity; } entity = this.em.merge(entity); flush(); return entity; } public T saveWithoutFlush(T entity) { return super.save(entity); } public List saveWithoutFlush(Iterable entities) { List result = new ArrayList(); if (entities == null) { return result; } for (T entity : entities) { result.add(saveWithoutFlush(entity)); } return result; } } As a simple example here, I just override the default save method of the SimpleJPARepository. The default behaviour of the save method will not flush after persist. I modified to make it flush after persist. On the other hand, I add another method called saveWithoutFlush() to allow developer to call save the entity without flush. Define Custom repository factory bean The last step is to create a factory bean class and factory class to produce repository based on your customized base repository class. public class DefaultRepositoryFactoryBean , S, ID extends Serializable> extends JpaRepositoryFactoryBean { /** * Returns a {@link RepositoryFactorySupport}. * * @param entityManager * @return */ protected RepositoryFactorySupport createRepositoryFactory( EntityManager entityManager) { return new DefaultRepositoryFactory(entityManager); } } /** * * The purpose of this class is to override the default behaviour of the spring JpaRepositoryFactory class. * It will produce a GenericRepositoryImpl object instead of SimpleJpaRepository. * */ public class DefaultRepositoryFactory extends JpaRepositoryFactory{ private final EntityManager entityManager; private final QueryExtractor extractor; public DefaultRepositoryFactory(EntityManager entityManager) { super(entityManager); Assert.notNull(entityManager); this.entityManager = entityManager; this.extractor = DefaultPersistenceProvider.fromEntityManager(entityManager); } @SuppressWarnings({ "unchecked", "rawtypes" }) protected JpaRepository getTargetRepository( RepositoryMetadata metadata, EntityManager entityManager) { Class repositoryInterface = metadata.getRepositoryInterface(); JpaEntityInformation entityInformation = getEntityInformation(metadata.getDomainType()); if (isQueryDslExecutor(repositoryInterface)) { return new QueryDslJpaRepository(entityInformation, entityManager); } else { return new GenericRepositoryImpl(entityInformation, entityManager, repositoryInterface); //custom implementation } } @Override protected Class getRepositoryBaseClass(RepositoryMetadata metadata) { if (isQueryDslExecutor(metadata.getRepositoryInterface())) { return QueryDslJpaRepository.class; } else { return GenericRepositoryImpl.class; } } /** * Returns whether the given repository interface requires a QueryDsl * specific implementation to be chosen. * * @param repositoryInterface * @return */ private boolean isQueryDslExecutor(Class repositoryInterface) { return QUERY_DSL_PRESENT && QueryDslPredicateExecutor.class .isAssignableFrom(repositoryInterface); } } Conclusion You could now add more features to base repository class. In your program, you could now create your own repository interface extending GenericRepository instead of JpaRepository. public interface MyRepository extends GenericRepository { void someCustomMethod(ID id); } In next post, I will show you how to add hibernate filter features to this GenericRepository.

Everyone is thinking why in the world would anyone pick static, when you can be dynamic? Usually the thought process is, "what language am I most proficient in, that can do the job." Totally not a bad way to go about it. Now does this choice affect anything else? Testing? Speed of development? Robustness? Dynamic vs. Static Dynamic languages are languages that don’t necessarily need variables to be declared before they are used. Examples of dynamic languages are Python, Ruby, and PHP. So in dynamic languages the following is possible: num = 10 We have successfully assigned a value to variable without declaring it before hand. Simple enough, try doing this in Java (you can’t). This can *increase* development speed, without having to write boilerplate code. This can somewhat be a double edge sword, since dynamic languages types are checked during runtime, there is no way to tell if there is a bug in code until it is run. I know you can test, but you can’t test for everything. You can’t test for everything. Here is an example albeit trivial. def get_first_problem(problems): for problem in problems: problam = problem + 1 return problam Now if you are raging to some serious dubstep, its easy enough to miss that small typo, you go screw it and do it live, and deploy to production. Python will simply create the new variable and not a single thing will be said. Only you can stop bugs in production! Static languages are languages that variables need to be declared before use and type checking is done at compile time. Examples of static languages include Java, C, and C++. So in static languages the following is enforced static int awesomeNumber; awesomeNumber = 10; Many argue this increases robustness as well as decrease chances of Runtime Errors. Since the compiler will catch those horrible horrible mistakes you made throughout your code. Your methods contracts are tighter, downside to this is crap ton of boilerplate code. Weak and Strong Typing can be often be confused with dynamic and static languages. Weak typed languages can lead to philosophical questions like what does the number 2 added to the word ‘two’ give you? Things like this are possible with a weak typed language. a = 2 b = "2" concatenate(a, b) // Returns "22" add(a, b) // Returns 4 Traditionally languages may place restriction on what transaction may occur for example in a strong typed language adding a string and integer will result in a type error as shown below. >>> a = 10 >>> b = 'ten' >>> a + b Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for +: 'int' and 'str' >>> Conclusion Regardless of where you land on this discussion, claiming one is better than the other would lead to flame war, but there are places where each is strong. Dynamic languages are good for fast quick development cycles and prototyping, while static languages are better suited to longer development cycles where trivial bugs could be extremely costly (telecommunication systems, air traffic control). For example if some giant company called Moo Corp. spent millions of dollars on QA and Testing and a bug somehow gets into the field, to fix it would mean another round of testing. When sitting in that chair the choice is clear static languages FTW, its a hard job but someone has to milk the cows. Test, test, and test. Just a little food for thought, for when you are starting your next project. You never know what limitations you maybe placing on yourself and your team. What do you do consider when selecting a programming language for a project?

First of all we should define EIPs and why we should use them. As the name implies, these are tested solutions for specific design problems encountered during many years in the development of IT systems. And what is all the more important is that they are technology-agnostic which means it does not matter what programming language or operating system you use. Patterns are divided into seven sections: 1. Messaging Systems, 2. Messaging Channels, 3. Message Constructions, 4. Message Routing, 5. Message Transformation, 6. Messaging endpoints, 7. System management. The purpose of this article is to encourage you to use the patterns so I will discuss briefly only one or two such patterns from each of above sections. If you want to explore then further, visit http://www.eaipatterns.com/ or read Gregor Hohpe’s book mentioned in the introduction of this series. Message Channel (from Messaging Systems) A message channel is a logical channel which is used to connect the applications. One application writes messages to the channel and the other one (or others) reads that message from the channel. Message queue and message topic are examples of message channels. Message Translator (from Messaging Systems) Message translator transforms messages in one format to another. For example one application sends a message in XML format, but the other accepts only JSON messages so one of the parties (or mediator) has to transform XML data to JSON. This is probably the most widely used integration pattern. Publish-Subscribe Channel (from Messaging Channels) This type of channel broadcasts an event or notification to all subscribed receivers. This is in contrast with a point-to-point channel . Each subscriber receive the message once and next copy of this message is deleted from channel. The most common implementation of this patter is messaging topic. Dead Letter Channel (from Messaging Channels) The Dead Letter Channel describe scenario, what to do if the messaging system determines that it cannot deliver a message to the specified recipient. This may be caused for example by connection problems or other exception like overflowed memory or disc space. Usually, before sending the message to the Dead Letter Channel, multiple attempts to redeliver message are taken. Correlation Identifier (from Message Construction) Correlation Identifier gives the possibility to match request and reply message when asynchronous messaging system is used. This is usually accomplished in the following way: Producer: Generate unique correlation identifier. Producer: Send message with attached generated correlation identifier. Consumer: Process messages and send reply with attached correlation identifier given in request message. Producer: Correlate request and reply message based on correlation identifier. Content-Based Router (from Message Routing) Content-Based Router examines message contents and route messages based on data contained in the message. Content Enricher (from Message Transformation) Content Enricher as the name suggests enrich message with missing information. Usually external data source like database or web service is used. Event-Driven Consumer (from Messaging Endpoints) Event-Driver Consumer enables you to provide a action that is called automatically by the messaging channel or transport layer. It is asynchronous type of pattern because receiver does not have a running thread until a callback thread delivers a message. Polling Consumer (from Messaging Endpoints) Polling Consumer is used when we want receiver to poll for a message, process it and next poll for another. What is very important is that this pattern is synchronous because it blocks thread until a message is received. This is in contrast with a event-driven consumer. An example of using this pattern is file polling. Wire Tap (from System Management) Wire Tap copy a message and route it to a separate channel, while the original message is forwarded to the destination channel. Usually Wire Tap is used to inspect message or for analysis purposes.

the test was run locally (no network involved ) on a lenovo w520 laptop with 8 cores & 8 gb ram with an ssd card. the storage engine we used was esent, safe transactions. default ravendb configuration, running in console, with logging disabled. we took the most obvious approach both in the code we wrote and the test approach. i am pretty sure that i’ll get a lot of helpful suggestions about the load testing. the code is available here, and you are more than welcome to take it for a spin and get your own results. what is important for me to note is that we have done exactly zero performance tuning . that is relevant to both the index we use, to the code that we wrote, everything. i just wrote things down, and didn’t worry about performance, even though this code is going to go through a load test. why don’t i worry about it? because ravendb is setup to do the right thing. it will self optimize itself without you need to take care of that. with that said, here are the test results: you can see that the red line is the number of users we have, and we have this worrying green line that seems to go crazy… except that this is actually the number of page served. the part that we care about is actually the avg. page time, and that is the blue line. this line , however, is basically flat no matter the load. here are the test results in details load test summary test run information load test name loadtest1 description start time 04/09/12 14:16:38 end time 04/09/12 14:21:38 warm-up duration 00:00:20 duration 00:05:00 controller local run number of agents 1 run settings used run settings1 overall results max user load 300 tests/sec 20.0 tests failed 0 avg. test time (sec) 12.5 transactions/sec 0 avg. transaction time (sec) 0 pages/sec 77.1 avg. page time (sec) 0.0062 requests/sec 77.1 requests failed 0 requests cached percentage 0 avg. response time (sec) 0.0062 avg. content length (bytes) 3,042 key statistic: top 5 slowest pages url (link to more details) 95% page time (sec) 0.018 0.018 0.014 0.014 0.014 key statistic: top 5 slowest tests name 95% test time (sec) browsing 19.3 browseandsearch 17.6 10.6 6 test results name scenario total tests failed tests (% of total) avg. test time (sec) browsing load 1,533 0 (0) 16.0 browseandsearch load 1,685 0 (0) 15.0 load 2,770 0 (0) 9.00 6 page results url (link to more details) scenario test avg. page time (sec) count load browsing 0.0072 1,629 load browseandsearch 0.0071 1,783 load browseandsearch 0.0064 3,443 load searching 0.0064 2,798 load browsing 0.0063 1,617 load browsing 0.0063 1,580 load browseandsearch 0.0063 1,760 load searching 0.0055 2,810 load searching 0.0055 2,839 load searching 0.0054 2,866 6 transaction results name scenario test response time (sec) elapsed time (sec) count 6 system under test resources machine name % processor time available memory at test completion (mb) 6 controller and agents resources machine name % processor time available memory at test completion (mb) 13.0 1,356 6 errors type subtype count last message you can dig in and look at the data, it is quite interesting. under the load of 300 users, the average page response time was… 0.0062 seconds. and ravendb was using just 13% of the cpu, and that include running the agents running the tests. after seeing how well ravendb does in perf testing, i decided to take it up a notch. starting from 10 users, with a step duration of 1 sec, add 50 users for each step, all the way to 3,000. start with a warm up period of 20 seconds, then run the test for 10 minutes. let us see what happens, okay? just to be clear, this is a ravendb application running with three thousands concurrent users , on an off the shelve laptop while i was busy doing other stuff. one word of warning before hand, because i run everything on a single machine, just running so many users on the machine significantly slowed down how ravendb is reacting. basically, the code for managing the perf test took so many resources that ravendb had to fight to get some to actually answer the queries. scared yet, because here are the results in graph form. now you can actually see that we have some fluctuations in the graphs, the number of users grows and grows until we get to 3,000 and we have 0.37 seconds response times. again, i remind you, we have done zero optimizations and this is idiomatic ravendb code. and we were able to serve requests at a frankly pretty amazing rate of speed. and here are they in their full details: load test summary test run information load test name loadtest1 description start time 04/09/12 15:28:48 end time 04/09/12 15:38:48 warm-up duration 00:00:20 duration 00:10:00 controller local run number of agents 1 run settings used load overall results max user load 3,000 tests/sec 196 tests failed 0 avg. test time (sec) 14.3 transactions/sec 0 avg. transaction time (sec) 0 pages/sec 741 avg. page time (sec) 0.37 requests/sec 741 requests failed 0 requests cached percentage 0 avg. response time (sec) 0.37 avg. content length (bytes) 3,080 key statistic: top 5 slowest pages url (link to more details) 95% page time (sec) 0.83 0.82 0.82 0.82 0.81 key statistic: top 5 slowest tests name 95% test time (sec) browsing 20.8 browseandsearch 19.8 12.9 6 test results name scenario total tests failed tests (% of total) avg. test time (sec) browsing load 31,843 0 (0) 17.4 browseandsearch load 33,989 0 (0) 16.8 load 51,650 0 (0) 10.8 6 page results url (link to more details) scenario test avg. page time (sec) count load browsing 0.40 32,338 load searching 0.39 52,597 load browsing 0.39 32,627 load browseandsearch 0.39 68,576 load browsing 0.38 32,803 load searching 0.38 52,283 load browseandsearch 0.37 34,766 load browseandsearch 0.36 34,982 load searching 0.35 51,991 load searching 0.33 51,846 6 transaction results name scenario test response time (sec) elapsed time (sec) count 6 system under test resources machine name % processor time available memory at test completion (mb) 6 controller and agents resources machine name % processor time available memory at test completion (mb) 85.4 1,203 6 errors type subtype count last message note that the reason fro the high cpu usage is that the tests and ravendb were running on the same machine. the code for the entire series can be found here: https://github.com/ayende/nuget.perf no, i’ll not do a similar sql version, if you want to, i would be very interested in seeing one, but that isn’t something that i intend to do. yes, it is a simple and trivial implementation, but that was pretty much the whole point. being able to get to that scale without actually doing anything special is what we strive for in ravendb.

Why create a mechanism to expect a test failure? There comes a time when one would want and expect a JUnit @Test case fail. Though this is pretty rare, it happens. I had the need to detect when a JUnit Test fails and then, if expected, to pass instead of fail. The specific case was that I was testing a piece of code that could throw an Assert error inside of a call of the object. The code was written to be an enhancement to the popular new Fest Assertions framework, so in order to test the functionality, one would expect test cases to fail on purpose. A Solution One possible solution is to utilize the functionality provided by a JUnit @Rule in conjunction with a custom marker in the form of an annotation. . Why use a @Rule? @Rule objects provide an AOP-like interface to a test class and each test cases. Rules are reset prior to each test case being run and they expose the workings of the test case in the style of an @Around AspectJ advice would. Required code elements @Rule object to check the status of each @Test case @ExpectedFailure custom marker annotation Test cases proving code works! Optional specific exception to be thrown if annotated test case does not fail NOTE: working code is available on my github page and will soon be in Maven Central. Feel free to Fork the project and submit a pull request Example Usage In this example, the "exception" object is a Fest assertion enhanced ExpectedException (look for my next post to expose this functionality). The expected exception will make assertions and in order to test those, the test case must be marked as @ExpectedFailure public class ExceptionAssertTest { @Rule public ExpectedException exception = ExpectedException.none(); @Rule public ExpectedTestFailureWatcher watcher = ExpectedTestFailureWatcher.instance(); @Test @ExpectedFailure("The matcher should fail becasue exception is not a SimpleException") public void assertSimpleExceptionAssert_exceptionIsOfType() { // expected exception will be of type "SimpleException" exception.instanceOf(SimpleException.class); // throw something other than SimpleException...expect failure throw new RuntimeException("this is an exception"); } } Implementation of Solution Reminder, the latest code is available on my github page. @Rule code (ExpectedTestFailureWatcher.java) import org.junit.rules.TestRule; import org.junit.runner.Description; import org.junit.runners.model.Statement; // YEAH Guava!! import static com.google.common.base.Strings.isNullOrEmpty; public class ExpectedTestFailureWatcher implements TestRule { /** * Static factory to an instance of this watcher * * @return New instance of this watcher */ public static ExpectedTestFailureWatcher instance() { return new ExpectedTestFailureWatcher(); } @Override public Statement apply(final Statement base, final Description description) { return new Statement() { @Override public void evaluate() throws Throwable { boolean expectedToFail = description.getAnnotation(ExpectedFailure.class) != null; boolean failed = false; try { // allow test case to execute base.evaluate(); } catch (Throwable exception) { failed = true; if (!expectedToFail) { throw exception; // did not expect to fail and failed...fail } } // placed outside of catch if (expectedToFail && !failed) { throw new ExpectedTestFailureException(getUnFulfilledFailedMessage(description)); } } /** * Extracts detailed message about why test failed * @param description * @return */ private String getUnFulfilledFailedMessage(Description description) { String reason = null; if (description.getAnnotation(ExpectedFailure.class) != null) { reason = description.getAnnotation(ExpectedFailure.class).reason(); } if (isNullOrEmpty(reason)) { reason = "Should have failed but didn't"; } return reason; } }; } } @ExpectedFailure custom annotation (ExpectedFailure.java) import java.lang.annotation.*; /** * Initially this is just a marker annotation to be used by a JUnit4 Test case in conjunction * with ExpectedTestFailure @Rule to indicate that a test is supposed to be failing */ @Documented @Retention(RetentionPolicy.RUNTIME) @Target(value = ElementType.METHOD) public @interface ExpectedFailure { // TODO: enhance by adding specific information about what type of failure expected //Class assertType() default Throwable.class; /** * Text based reason for marking test as ExpectedFailure * @return String */ String reason() default ""; } Custom Exception (Optional, you can easily just throw RuntimeException or existing custom exception) public class ExpectedTestFailureException extends Throwable { public ExpectedTestFailureException(String message) { super(message); } } Can't one exploit the ability to mark a failure as expected? With great power comes great responsibility, it is advised that you do not mark a test as being @ExpectedFailure if you do not understand exactly why the test if failing. It is recommended that this testing method be implemented with care. DO NOT use the @ExpectedFailure annotation as an alternative to @Ignore Possible future enhancements could include ways to specify the specific assertion or the specific message asserted during the test case execution. Known issues In this current state, the @ExpectedFailure annotation can cover up additional assertions and until the future enhancements have been put into place, it is advised to use this methodology wisely.

Option 1: JMX Many people asked can they manage Quartz via JMX, and that the documentation on this is not clear enough to help them get started. So, let me highlight couple ways you can do this. Yes you can enable JMX in quartz with the following in quartz.properties org.quartz.scheduler.jmx.export = true After this, you use standard JMX client such as $JAVA_HOME/bin/jconsole to connect and manage remotely. Option 2: RMI Another way to manage quartz remotely is to enable RMI in Quartz. If you use this, you basically run one instance of Quartz as RMI server, and then you can create second Quartz instance as RMI client. These two can talk remotely via a TCP port. For server scheduler instance, you want to add these in quartz.properties org.quartz.scheduler.rmi.export = true org.quartz.scheduler.rmi.createRegistry = true org.quartz.scheduler.rmi.registryHost = localhost org.quartz.scheduler.rmi.registryPort = 1099 org.quartz.scheduler.rmi.serverPort = 1100 And for client scheduler instance, you want to add these in quartz.properties org.quartz.scheduler.rmi.proxy = true org.quartz.scheduler.rmi.registryHost = localhost org.quartz.scheduler.rmi.registryPort = 1099 The RMI feature is mentioned in Quartz doc here. Quartz doesn't have a client API, but use the same org.quartz.Scheduler for both server and client. It's just the configuration are different. By different configuration, you get very different behavior. For server, your scheduler is running all the jobs, while for client, it's simply a proxy. Your client scheduler instance will not run any jobs! You must be really careful when shutting down client because it does allow you to bring down the server! These configurations have been highlighted in the MySchedule project. If you run the webapp, you should see a screen like this demo, and you will see it provided many sample of quartz configurations with these remote managment config properties. If configure with RMI option, you can actually still use MySchedule web UI to manage the Quartz as proxy. You can view and drill down jobs, and you can even stop or shutdown remote server! Based on my experience, there is a down side of using Quartz RMI feature though. That is it creates a single point of failure. There is no fail over if your RMI server port is down!

For the last two weeks I’ve been working on some interesting use cases for the good ol’ failover transport. I finally have some time at my hands, so here’s a brief recap of what’s coming in 5.6 release in this area. First there’s a new feature, called Priority Backup. It’s described in details here, but in a nutshell it provides you with the mechanism of prioritizing your failover urls and keep your clients connected to them as soon as they are available. The most obvious use case for this is to keep your clients connected to the broker in local data center whenever you can. By doing this, you can both have better performances and stability of your clients, but also save on your bandwidth bills. Another improvement is coming for automatic broker cluster feature. Although this feature is not new, I spent some time hardening it and thought to share some more insight in how (and when) to use it in your projects. In search of high availability, people often default to master-slave architecture. This makes sense in most use cases, but if your flow is purely non-persistent you can probably come up with more optimal architecture. Instead of having one broker at the time handling all your load, and other one just waiting for it to fail, you’ll get more efficient system with some kind of active-active configuration where (possibly multiple) brokers share the load all the time. Ideally clients would be evenly distributed and would rebalance if anything changes. Brokers don’t need to share any messages as clients are distributed and messages are non-persistent so they will be lost if broker fails. So can you achieve this kind of architecture with ActiveMQ? Sure you do. That’s where automatic rebalance and clustering shines. First of all, brokers should be networked but only so they can exchange information on their availability. They shouldn’t exchange the messages (but of course can if your use case needs it). In 5.6 you do that with pure static networks, using configuration like So now imagine three brokers A,B and C forming a full mesh. In addition every broker uses rebalance options on their transport connectors All that is left for the client to do is connect to one of the brokers it knows like failover:(brokerA) and the broker will fill it with all information on other brokers in the cluster and whether it should reconnect to one of them or not. So having a large number of clients connecting like this, very soon they’ll rebalance over available brokers. You can stop one of the brokers in the cluster for updates and clients will rebalance over remaining ones. You can even add a new broker to the cluster and everything will get rebalanced without any need for you to touch your clients. So, basically in this way you have both load balancing and high availability for your non-persistent messages. Additionally, your clients are automatically updated with all information they need, and no manual intervention is needed. Although the basic support for clustering was there since 5.4, I did some more hardening and better rebalancing, so it’s coming in the Apache ActiveMQ 5.6 (and the next Fuse 5.5.1) release. Also, there are some more great stuff regarding broker clustering coming soon, so stay tuned and happy messaging.

Fixing a problem usually starts with reproducing it – what Steve McConnell calls “stabilizing the error.” Technically speaking, you can’t be sure you are fixing the problem unless you can run through the same steps, see the problem happen yourself, fix it, and then run through the same steps and make sure that the problem went away. If you can’t reproduce it, then you are only guessing at what’s wrong, and that means you are only guessing that your fix is going to work. But let’s face it – it’s not always practical or even possible to reproduce a problem. Lots of bug reports don’t include enough information for you to understand what the hell the problem actually was, never mind what was going on when the problem occurred – especially bug reports from the field. Rahul Premraj and Thomas Zimmermann found in The Art of Collecting Bug Reports (from the book Making Software), that the two most important factors in determining whether a bug report will get fixed or not are: Is the description well-written, can the programmer understand what was wrong or why the customer thought something was wrong? Does it include steps to reproduce the problem, even basic information about what they were doing when the problem happened? It’s not a lot to ask – from a good tester at least. But you can’t reasonably expect this from customers. There are other cases where you have enough information, but don’t have the tools or expertise to reproduce a problem – for example, when a pen tester has found a security bug using specialist tools that you don’t have or don’t understand how to use. Sometimes you can fix a problem without being able to see it happen in front of you, come up with a theory on your own, trusting your gut – especially if this is code that you recently worked on. But reproducing the problem first gives you the confidence that you aren’t wasting your time and that you actually fixed the right issue. Trying to reproduce the problem should almost always be your first step. What’s involved in reproducing a bug? What you want to do is to find, as quickly as possible, a simple test that consistently shows the problem, so that you can then run a set of experiments, trace through the code, isolate what’s wrong, and prove that it went away after you fixed the code. The best explanation that I’ve found of how to reproduce a bug is in Debug It! where Paul Butcher patiently explains the pre-conditions (identifying the differences between your test environment and the customer’s environment, and trying to control as many of them as possible), and then how to walk backwards from the error to recreate the conditions required to make the problem happen again. Butcher is confident that if you take a methodical approach, you will (almost) always be able to reproduce the problem successfully. In Why Programs Fail: A guide to Systematic Debugging, Andreas Zeller, a German Comp Sci professor, explains that it’s not enough just to make the problem happen again. Your goal is to come up with the simplest set of circumstances that will trigger the problem – the smallest set of data and dependencies, the simplest and most efficient test(s) with the fewest variables, the shortest path to making the problem happen. You need to understand what is not relevant to the problem, what’s just noise that adds to the cost and time of debugging and testing – and get rid of it. You do this using binary techniques to slice up the input data set, narrowing in on the data and other variables that you actually need, repeating this until the problem starts to become clear. Code Complete’s chapter on Debugging is another good guide on how to reproduce a problem following a set of iterative steps, and how to narrow in on the simplest and most useful set of test conditions required to make the problem happen; as well as common places to look for bugs: checking for code that has been changed recently, code that has a history of other bugs, code that is difficult to understand (if you find it hard to understand, there’s a good chance that the programmers who worked on it before you did too). Replay Tools One of the most efficient ways to reproduce a problem, especially in server code, is by automatically replaying the events that led up to the problem. To do this you’ll need to capture a time-sequenced record of what happened, usually from an audit log, and a driver to read and play the events against the system. And for this to work properly, the behavior of the system needs to be deterministic – given the same set of inputs in the same sequence, the same results will occur each time. Otherwise you’ll have to replay the logs over and over and hope for the right set of circumstances to occur again. On one system that I worked on, the back-end engine was a deterministic state machine designed specifically to support replay. All of the data and events, including configuration and control data and timer events, were recorded in an inbound event log that we could replay. There were no random factors or unpredictable external events – the behavior of the system could always be recreated exactly by replaying the log, making it easy to reproduce bugs from the field. It was a beautiful thing, but most code isn’t designed to support replay in this way. Recent research in virtual machine technology has led to the development of replay tools to snapshot and replay events in a virtual machine. VMWare Workstation, for example, included a cool replay debugging facility for C/C++ programmers which was “guaranteed to have instruction-by-instruction identical behavior each time.” Unfortunately, this was an expensive thing to make work, and it was dropped in version 8, at the end of last year. Replay Solutions provides replay for Java programs, creating a virtual machine to record the complete stream of events (including database I/O, network I/O, system calls, interrupts) as the application is running, and then later letting you simulate and replay the same events against a copy of the running system, so that you can debug the application and observe its behavior. They also offer similar application record and replay technology for mobile HTML5 and JavaScript applications. This is exciting stuff, especially for complex systems where it is difficult to setup and reproduce problems in different environments. Fuzzing and Randomness If the problem is non-deterministic, or you can't come up with the right set of inputs, one approach to try is to simulate random data inputs and watch to see what happens - hoping to happen on a set of input variables that will trigger the problem. This is called fuzzing. Fuzzing is a brute force testing technique that is used to uncover data validation weaknesses that can cause reliability and security problems. It's effective at finding bugs, but it’s a terribly inefficient way to reproduce a specific problem. First you need to setup something to fuzz the inputs (this is easy if a program is reading from a file, or a web form – there are fuzzing tools to help with this – but a hassle if you need to write your own smart protocol fuzzer to test against internal APIs). Then you need time to run through all of the tests (with mutation fuzzing, you may need to run tens of thousands or hundreds of thousands of tests to get enough interesting combinations) and more time to sift through and review all of the test results and understand any problems that are found. Through fuzzing you will get new information about the system to help you identity problem areas in the code, and maybe find new bugs, but you may not end up any closer to fixing the problem that you started on. Reproducing problems, especially when you are working from a bad bug report (“the system was running fine all day, then it crashed… the error said something about a null pointer I think?”) can be a serious time sink. But what if you can’t reproduce the problem at all? Let’s look at that next…