For most large web applications, uptime is of foremost importants. Any outage can be seen by customers as a frustration, or opportunity to move to a competitor. What's more for a site that also includes e-commerce, it can mean real lost sales. Zero Downtime describes a site without service interruption. To achieve such lofty goals, redundancy becomes a critical requirement at every level of your infrastructure. If you're using cloud hosting, are you redundant to alternate availability zones and regions? Are you using geographically distributed load balancing? Do you have multiple clustered databases on the backend, and multiple webservers load balanced. All of these requirements will increase uptime, but may not bring you close to zero downtime. For that you'll need thorough testing. The solution is to pull the trigger on sections of your infrastructure, and prove that it fails over quickly without noticeable outage. The ultimate test is the outage itself. Sean Hull on Quora: What is zero downtime and why is it important? Source: http://www.iheavy.com/2011/06/23/zero-downtime-what-is-it-and-why-is-it-important/

A brief overview of dependency injection in Java EE, the difference between @Resource/@EJB and @Inject, and how does that all relate to Spring – mostly in the form of links. Context Dependency Injection (CDI, JSR 299) is a part of Java EE 6 Web Profile and itself builds on Dependency Injection for Java (JSR 330), which introduces @Inject, @Named etc. While JSR 330 is for DI only and is implemented e.g. by Guice and Spring, CDI adds various EE stuff such as @RequestScoped, interceptors/decorators, producers, eventing and a base for integration with JSF, EJBs etc. Java EE components such as EJBs have been redefined to build on top of CDI (=> @Stateless is now a CDI managed bean with additional services). A key part of CDI aside of its DI capabilities is its awarness of bean contexts and the management of bean lifecycle and dependencies within those contexts (such as @RequestScoped or @ConversationScoped). CDI is extensible – you can define new context scopes, drop-in interceptors and decorators, make other beans (e.g. from Spring) available for CDI,… . Resources to check: Contexts and Dependency Injection in Java EE 6 by Adam Bien – a very good explanation of the basics of CDI and how it differs from DI in Java EE 5 (hint: context awarness) Slideshow with a good overview of CDI and all it offers About CDI extensibility and SPIs (e.g. Seam 3 is basically a set of portable CDI extensions) Guice and Spring do not implement CDI (3/2011) – and Spring perhaps isn’t motivated to do so (it supports JSR 330, CDI would be too much work) DZone CDI Refcard may be handy CDI 1.0 vs. Spring 3.1 feature comparsion: bean definition & dependency injection: “in the area that I compared in this article [= DI], there is only little critical difference in the two technologies” (though Spring more fine-tunable) Java EE 6 (CDI / EJB 3.1) XOR Spring Core Reloaded: New projects should preferably start with pure Java EE including CDI and add Spring utilities such as JDBC/JMS when needed Oracle: CDI in the Java EE 6 Ecosystem – 62 pages slideshow, the stuff is explained more than in the previously mentioned slideshow Note: CDI 1.1 (JSR 346, Java EE 7) should have a standard way of bootstrapping it in non-EE environment (i.e. SE) From http://theholyjava.wordpress.com/2011/11/09/what-is-cdi-how-does-it-relate-to-ejb-and-spring/

Dependency injection is a very simple concept: if you have an object that interacts with other objects the responsibility of finding a reference to those objects at run time is moved outside of the object itself. What does it mean for an object to "interact" with other objects? Generally it means invoking methods or reading properties from those objects. So if we have a class A that invokes method Calculate on class B, we can say that A interacts with B. In the following example we show class A interacting with class B. We can equally say that A depends on class B to fulfill a responsibility. In this case, it not only invokes its method Calculate but it also creates a new instance of that class. class A { private B _b; public A { _b = new B(); } public int SomeMethod() { return (_b.Calculate() * 2); } } In the following example, on the other side, the responsibility of getting a reference to an implementation of a class of type B is moved outside of A: class A { private _b = B; public A(B b) { _b = b; } public int SomeMethod() { return _(b.Calculate * 2); } } In this case we say that a dependency (B) has been injected into A, via the constructor. Of course, you can also inject dependencies via a property (or even a regular method), like in the following example: class A { private _b = B; public B B { get { return _b; } set { _b = value; } } public int SomeMethod() { if (_b != null) return _b.RetrieveValue() * 2;} else // HANDLE THIS ERROR CASE return -1; } } So this is all there is about dependency injection. Everything else just builds on this core concept. Like for example Inversion Of Control (IoC) tools which helps you wiring together your objects at run time, injecting all dependencies as needed. So what exactly is Inversion of Control and how does it relate to Dependency Injection (DI)? I like to associate IoC to the Hollywood Principle: "Don't call us, we'll call you". IoC is a design principle where reusable generic code controls the execution of problem-specific code: it is a characteristic of many frameworks, where the application is built extending or customizing a common skeleton; you put your own classes at specific points and the framework will call you when needed. You can use an IoC container as a framework to perform Dependency Injection on your behalf: you tell the container which are the concrete implementation classes for your dependencies and the container will make sure that your constructors or setters will be called with the right objects. Therefore, IoC containers are just a convenience to simplify how dependency injection is handled. But even if you don't use one you could still manually perform dependency injection. (If you want to have a look at how an IoC container works you can jump to my mini tutorial on Ninject). Why is the concept of dependency injection important? Because by applying it, you simplify your design (separating the responsibility of using an object from the responsibility of using the object) and your code becomes much easier to test, since you can mock out the dependencies substituting them with fake (stub) objects. But that is the subject for another post.

What is JIT Compiler? The Just In Time Compiler (JIT) concept and more generally adaptive optimization is well known concept in many languages besides Java (.Net, Lua, JRuby). In order to explain what is JIT Compiler I want to start with a definition of compiler concept. According to wikipedia compiler is "a computer program that transforms the source language into another computer language (the target language)". We are all familiar with static java compiler (javac) that compiles human readable .java files to a byte code that can be interpreted by JVM - .class files. Then what does JIT compile? The answer will given a moment later after explanation of what is "Just in Time". According to most researches, 80% of execution time is spent in executing 20% of code. That would be great if there was a way to determine those 20% of code and to optimize them. That's exactly what JIT does - during runtime it gathers statistics, finds the "hot" code compiles it from JVM interpreted bytecode (that is stored in .class files) to a native code that is executed directly by Operating System and heavily optimizes it. Smallest compilation unit is single method. Compilation and statistics gathering is done in parallel to program execution by special threads. During statistics gathering the compiler makes hypotheses about code function and as the time passes tries to prove or to disprove them. If the hypothesis is dis-proven the code is deoptimized and recompiled again. The name "Hotspot" of Sun (Oracle) JVM is chosen because of the ability of this Virtual Machine to find "hot" spots in code. What optimizations does JIT? Let's look closely at more optimizations done by JIT. Inline methods - instead of calling method on an instance of the object it copies the method to caller code. The hot methods should be located as close to the caller as possible to prevent any overhead. Eliminate locks if monitor is not reachable from other threads Replace interface with direct method calls for method implemented only once to eliminate calling of virtual functions overhead Join adjacent synchronized blocks on the same object Eliminate dead code Drop memory write for non-volatile variables Remove prechecking NullPointerException and IndexOutOfBoundsException Et cetera When the Java VM invokes a Java method, it uses an invoker method as specified in the method block of the loaded class object. The Java VM has several invoker methods, for example, a different invoker is used if the method is synchronized or if it is a native method. The JIT compiler uses its own invoker. Sun production releases check the method access bit for value ACC_MACHINE_COMPILED to notify the interpreter that the code for this method has already been compiled and stored in the loaded class. JIT compiler compiles the method block into native code for this method and stores that in the code block for that method. Once the code has been compiled the ACC_MACHINE_COMPILED bit, which is used on the Sun platform, is set. How do we know what JIT is doing in our program and how can it be controlled? First of all to disable JIT Djava.compiler=NONE parameter can be used. There are 2 types of JIT compilers in Hotspot - one is used for client program and one for server (-server option in VM parameters). Program, running on server enjoys usually from more resources than program running on client and to server program top throughput is usually more important. Hence JIT in server is more resource consuming and gathering statistics takes more time to make the statistics more accurate. For client program gathering statics for a method lasts 1500 method calls, for server 15000. These default values can be changed by -XX:CompileThreshold=XXX VM parameter. In order to find out whether default value is good for you try enabling "XX:+PrintCompilation" and "-XX:-CITime" parameters that print JIT statistics and time CPU spent by JIT. Benchmarks Most of the benchmarks show that JITed code runs 10 to 20 times faster than interpreted code. There are many benchmarks done. Below given result graphs of two of them: Its worth to mention that programs that run in JIT mode, but are still in "learning mode" run much slower than non JITed programs. Drawbacks of JIT JIT Increases level of unpredictability and complexity in Java program. It adds another layer that developers don't really understand. Example of possible bugs - 'happens before relations" in concurrency. JIT can easily reorder code if the change is safe for a program running in single thread. To solve this problem developers make hints to JIT using "synchronized" word or explicit locking. Increases non heap memory footprint - JITed code is stored in "Code Cache" generation. Advanced JIT JIT and garbage collection. For GC to occur program must reach safe points. For this purpose JIT injects yieldpoints at regular intervals in native code. In addition to scanning of stack to find root references, registers must be scanned as they may hold objects created by JIT Comments are appresiated. The article can be found also at: artiomg.blogspot.com/2011/10/just-in-time-compiler-jit-in-hotspot.html

There is more and more writing on DevOps lately, which is good and bad. There still remains a small core of thoughtful people that are worth listening to and learning from. There’s more and more marketing from vendors and consultants jumping on the Devops bandwagon. There’s some naïve silliness (“Hire wicked smart people and give them all access to root.”) which can probably be safely ignored. And then there’s stuff that is half-right and half-wrong, too dangerous to be ignored. Like this recent post on Rollbacks and Other Deployment Myths, in which John Vincent lists 5 “myths” about system deployment, which I want to take some time to respond to here. Change Is Change? The author tries to make the point that “Change is neither good or bad. It’s just change.” Therefore we do not need to be afraid of making changes. I don’t agree. This attitude to change ignores the fact that once a system is up and running and customers are relying on it to conduct their business, whatever change you are making to the system is almost never as important as making sure that the system keeps running properly. Unfortunately, changes often lead to problems. We know from Visible Ops that based on studies of hundreds of companies, 80% of operational failures are caused by mistakes made during changes. This is where heavyweight process control frameworks like ITIL and COBIT, and detective change control tools like Tripwire came from. To help companies get control over IT change, because people had to find some way to stop shit from breaking. Yes, I get the point that in IT we tend to over-compensate, and I agree that calling in the Release Police and trying to put up a wall around all changes isn’t sustainable. People don’t have to work this way. But trivializing change, pretending that changes don't lead to problems, is dangerous. Deploys Are Not Risky? You can be smart and careful and break changes down into small steps and try to automate code pushes and configuration changes, and plan ahead and stage and review and test all your changes, and after all of this you can still mess up the deploy. Even if you make frequent small changes and simplify the work and practice it a lot. For systems like Facebook and online games and a small number of other cases, maybe deployment really is a non-issue. I don’t care if Facebook deploys in the middle of the day – I can usually tell when they are doing a “zero downtime” deploy (or maybe they are “transparently” recovering from a failure) because data disappears temporarily or shows up in the wrong order, functions aren’t accessible for a while, forms don’t resolve properly, and other weird shit happens, and then things come back later or they don’t. As a customer, do I care? No. It’s an inconvenience, and it’s occasionally unsettling ("WTF just happened?"), but I get used to it and so do millions of others. That’s because most of us don’t use Facebook or systems like this for anything important. For business-critical systems handling thousands of transactions a second that are tied into hundreds of other company’s systems (the world that I work in) this doesn’t cut it. Maybe I spend too much time at this extreme, where even small problems with compatibility that only affect a small number of customers, or slight and temporary performance slow downs, are a big deal. But most people I work with and talk to in software development and maintenance and system operations agree that deployment is a big deal and needs to be done with care and attention, no matter how simple and small the changes are and no matter how clean and simple and automated the deployment process is. Rollbacks Are a Myth? Vincent wants us to “understand that it’s typically more risky to rollback than rolling forward. Always be rolling forward.” Not even the Continuous Deployment advocates (who are often some of the most radical – and I think some of the most irresponsible – voices in the Devops community) agree with this – they still roll back if they find problems with changes. “Rollbacks are a myth” is an echo of the “real men fail forward” crap I heard at Velocity last year and it is where I draw the line. It's one thing to state an extreme position for argument's sake or put up a straw man – but this is just plain wrong. If you're going to deploy, you have to anticipate roll back and think about it when you make changes and you have to test rolling back to make sure that it works. All of this is hard. But without a working roll back you have no choice other than to fail forward (whatever this means, because nobody who talks about it actually explains how to do it), and this is putting your customers and their business at unnecessary risk. It’s not another valid way of thinking. It’s irresponsible. James Hamilton wrote an excellent paper on Designing and Delivering Internet-Scale Services when he was at Microsoft (now he’s a an executive and Distinguished Engineer at Amazon). Hamilton’s paper remains one of the smartest things that anyone has written about how to deal with deployment and operational problems at scale. Everyone who designs, builds, maintains or operates an online system should read it. His position on roll back is simple and obvious and right: Reverting to the previous version is a rip cord that should always be available on any deployment. Everything Fails. Embrace Failure. I agree that everything can and will fail some day, that we can’t pretend that we can prevent failures in any system. But I don’t agree with embracing failure, at least in business-critical enterprise systems, where recovering from a failure means lost business and requires unraveling chains of transactions between different upstream and downstream systems and different companies, messing up other companies' businesses as well as your own and dealing the follow-on compliance problems. Failures in these kinds of systems, and a lot of other systems, are ugly and serious, and they should be treated seriously. We do whatever we can to make sure that failures are controlled and isolated, and we make sure that we can recover quickly if something goes wrong (which includes being able to roll back!). But we also do everything that we can to prevent failures. Embracing failure is fine for online consumer web site startups – let’s leave it to them. SLAs I wanted to respond to the points about SLAs, but it’s not clear to me what the author was trying to say. SLAs are not about servers. Umm, yes that’s right of course... SLAs are important to set business expectations with your customers (the people who are using the system) and with your partners and suppliers. So that your partners and suppliers know what you need from them and what you are paying them for, and so that you know if you can depend on them when you have to. So that your customers know what they are paying you for. And SLAs (not just high-level uptime SLAs, but SLAs for Recovery Time and Recovery Point goals and incident response and escalation) are important so that your team understands the constraints that they need to work under, what trade-offs to make in design and implementation and in operations. Under-Compensating Is Worse Than Over-Compensating I spent more time than I thought I would responding to this post, because some of what the author says is right – especially in the second part of his post, Deploy All the Things where he provides some good practical advice on how to reduce risk in deployment. He’s right that Operations main purpose isn’t to stop change – it can’t be. We have to be able to keep changing, and developers and operations have to work together to do this in safe and efficient ways. But trivializing the problems and risks of change and over-simplifying how to deal with these risks and how to deal with failures, isn’t the way to do this. There has to be a middle way between the ITIL and COBIT world of controls and paper and process, and cool Web startups failing forward, a way that can really work for the rest of us. Source: http://swreflections.blogspot.com/2011/10/rolling-forward-and-other-deployment.html

this article is intended to give the reader enough information to understand what jdepend is, what it does, and how to use it in a maven build. it’s a kind of cheat sheet, if you like. what is it? jdepend is more of a design metric than a code metric, it gives you information about your classes with regards to how they’re related to each other. using this information you should be able to identify any unwanted or dubious dependencies. how does it do that? it traverses java class files and generates design quality metrics, such as: number of classes and interfaces afferent couplings (ca) – what is this?? someone probably feels very proud of themselves for coming up with this phrase. afferent coupling means the number of other packages which depend on the package being measured, in a nutshell. jdepend define this as a measure of a package’s “responsibility” efferent couplings (ce) – sort of the opposite of ca. it’s a measure of the number of other packages that your package depends on abstractness (a) – the ratio of abstract classes to total classes. instability (i) – the ratio of efferent coupling (ce) to total coupling (ce + ca) distance from the main sequence (d) – this sounds fairly wishy-washy and i’ve never paid any attention to it. it’s defined as: “an indicator of the package’s balance between abstractness and stability”. meh. to use jdepend with maven you’ll need maven 2.0 or higher and jdk 1.4 or higher. you don’t need to install anything, as maven will sort this out for you by downloading it at build time. here’s a snippet from one of my project poms, it comes from in the section: org.codehaus.mojo jdepend-maven-plugin 1.6 build/maven/${pom.artifactid}/target/jdepend-reports what you’ll get is a jdepend entry under the project reports section of your maven site, like this: maven project reports page and this is what the actual report looks like (well, some of it): jdepend report summary: jdepend isn’t something i personally use very heavily, but i can understand how it could be used to good effect as a general measure of how closely related your classes are, which, in certain circumstances could prompt you to redesign or refactor your code. i don’t think this sort of information is required on a per commit basis, so i’d be tempted to only include it in my nightly reports. however, i also use sonar, and that has a built-in measure of afferent coupling, so if you’re only interested in that measurement and you’re already running sonar, then jdepend is probably a bit of an unnecessary overhead. also, sonar itself has some good plugins which can provide architectural and design governance features, at least one of which i know implemented jdepend. source: http://jamesbetteley.wordpress.com/2011/04/06/jdepend-design-metrics-in-ci/

In my opinion, one of the main benefits of dependency injection is that you can inject mock and/or stub objects into your code in order to improve testability, increase test coverage and write better and more meaningful tests. There are those times, however, when you come across some legacy code that doesn’t use dependency injection and held together by composition rather than aggregation. When this happens, you have three options: Ignore the problem and not write any tests. Refactor like mad, changing everything to use dependency injection. Use PowerMock to mock constructors Obviously, option 1 isn’t a serious option, and although I’d recommend refactoring to move everything over to dependency injection, that takes time and you have to be pragmatic. That’s where PowerMock comes in... this blog demonstrates how to use PowerMock to mock a constructor, which means that when your code calls new it doesn’t create a real object, it creates a mock object. To demonstrate this idea, the first thing we need is some classes to test, which are shown below. public class AnyOldClass { public String someMethod() { return "someMethod"; } } public class UsesNewToInstantiateClass { public String createThing() { AnyOldClass myclass = new AnyOldClass(); String returnValue = myclass.someMethod(); return returnValue; } } The first class, AnyOldClass, is the class that the code instantiates by calling new. In this example, as the name suggests, it can be anything. The second class, the aptly named UsesNewToInstantiateClass, has one method, createThing(), which when called does a: AnyOldClass myclass = new AnyOldClass(); This is all pretty straight forward, so we’ll move quickly on to the PowerMock assisted JUnit test: import static org.easymock.EasyMock.expect; import static org.junit.Assert.assertEquals; import static org.powermock.api.easymock.PowerMock.expectNew; import static org.powermock.api.easymock.PowerMock.replay; import static org.powermock.api.easymock.PowerMock.verify; import org.junit.Test; import org.junit.runner.RunWith; import org.powermock.api.easymock.annotation.Mock; import org.powermock.core.classloader.annotations.PrepareForTest; import org.powermock.modules.junit4.PowerMockRunner; @RunWith(PowerMockRunner.class) @PrepareForTest(UsesNewToInstantiateClass.class) public class MockConstructorTest { @Mock private AnyOldClass anyClass; private UsesNewToInstantiateClass instance; @Test public final void testMockConstructor() throws Exception { instance = new UsesNewToInstantiateClass(); expectNew(AnyOldClass.class).andReturn(anyClass); final String expected = "MY_OTHER_RESULT"; expect(anyClass.someMethod()).andReturn(expected); replay(AnyOldClass.class, anyClass); String result = instance.createThing(); verify(AnyOldClass.class, anyClass); assertEquals(expected, result); } } Firstly, this class has the usual PowerMock additions of: @RunWith(PowerMockRunner.class) @PrepareForTest(UsesNewToInstantiateClass.class) at the top of the file plus the creation of the anyOldClass mock object. The important line of code to consider is: expectNew(AnyOldClass.class).andReturn(anyClass); This line of code tells PowerMock to expect a call to new AnyOldClass() and return our anyClass mock object. Also of interest are the calls to replay and verify. In the example above, they both have two arguments. The first, AnyOldClass.class relates to the expectNew(...) call above, whilst the second, anyClass refers to the straight forward mock call expect(anyClass.someMethod()).andReturn(expected);. There are those times when you should really let new do what it does: create a new object of the requested type. There is a body of opinion that says you can over-isolate your code when testing and that mocking everything reduces the meaning and value of a test. To me there’s no right answer to this and it’s a matter of choice. It’s fairly obvious that if your code accesses an external resource such as a database, then you’d either refactor and implement DI or use PowerMock. If your code under test doesn’t access any external resources, then it’s more of a judgement call on how much code isolation is too much? This perhaps needs some thought and may be the subject for another blog on anther day... From http://www.captaindebug.com/2011/10/using-powermock-to-mock-constructors.html

One of the challenges in building a distributed web application is in handling sessions. When you have multiple instances of an application running and session data is written to local files (as is the default behavior for the session handling functions in PHP) a user session can be lost when a session is started on one instance but subsequent requests are directed (via a load balancer) to other instances. To successfully manage sessions across multiple instances, you need a common data store. In this post I’ll show you how the Windows Azure SDK for PHP makes this easy by storing session data in Windows Azure Table storage. In the 4.0 release of the Windows Azure SDK for PHP, session handling via Windows Azure Table and Blob storage was included in the newly added SessionHandler class. Note: The SessionHandler class supports storing session data in Table storage or Blob storage. I will focus on using Table storage in this post largely because I haven’t been able to come up with a scenario in which using Blob storage would be better (or even necessary). If you have ideas about how/why Blob storage would be better, I’d love to hear them. The SessionHandler class makes it possible to write code for handling sessions in the same way you always have, but the session data is stored on a Windows Azure Table instead of local files. To accomplish this, precede your usual session handling code with these lines: require_once 'Microsoft/WindowsAzure/Storage/Table.php'; require_once 'Microsoft/WindowsAzure/SessionHandler.php'; $storageClient = new Microsoft_WindowsAzure_Storage_Table('table.core.windows.net', 'your storage account name', 'your storage account key'); $sessionHandler = new Microsoft_WindowsAzure_SessionHandler($storageClient , 'sessionstable'); $sessionHandler->register(); Now you can call session_start() and other session functions as you normally would. Nicely, it just works. Really, that’s all there is to using the SessionHandler, but I found it interesting to take a look at how it works. The first interesting thing to note is that the register method is simply calling the session_set_save_handler function to essentially map the session handling functionality to custom functions. Here’s what the method looks like from the source code: public function register() { return session_set_save_handler(array($this, 'open'), array($this, 'close'), array($this, 'read'), array($this, 'write'), array($this, 'destroy'), array($this, 'gc') ); } The reading, writing, and deleting of session data is only slightly more complicated. When writing session data, the key-value pairs that make up the data are first serialized and then base64 encoded. The serialization of the data allows for lots of flexibility in the data you want to store (i.e. you don’t have to worry about matching some schema in the data store). When storing data in a table, each entry must have a partition key and row key that uniquely identify it. The partition key is a string (“sessions” by default, but this is changeable in the class constructor) and the the row key is the session ID. (For more information about the structure of Tables, see this post.) Finally, the data is either updated (it it already exists in the Table) or a new entry is inserted. Here’s a portion of the write function: $serializedData = base64_encode(serialize($serializedData)); $sessionRecord = new Microsoft_WindowsAzure_Storage_DynamicTableEntity($this->_sessionContainerPartition, $id); $sessionRecord->sessionExpires = time(); $sessionRecord->serializedData = $serializedData; try { $this->_storage->updateEntity($this->_sessionContainer, $sessionRecord); } catch (Microsoft_WindowsAzure_Exception $unknownRecord) { $this->_storage->insertEntity($this->_sessionContainer, $sessionRecord); } Not surprisingly, when session data is read from the table, it is retrieved by session ID, base64 decoded, and unserialized. Again, here’s a snippet that show’s what is happening: $sessionRecord = $this->_storage->retrieveEntityById( $this->_sessionContainer, $this->_sessionContainerPartition, $id ); return unserialize(base64_decode($sessionRecord->serializedData)); As you can see, the SessionHandler class makes good use of the storage APIs in the SDK. To learn more about the SessionHandler class (and the storage APIs), check out the documentation on Codeplex. You can, of course, get the complete source code here: http://phpazure.codeplex.com/SourceControl/list/changesets. As I investigated the session handling in the Windows Azure SDK for PHP, I noticed that the absence of support for SQL Azure as a session store was conspicuous. I’m curious about how many people would prefer to use SQL Azure over Azure Tables as a session store. If you have an opinion on this, please let me know in the comments.

I’ve been going over a couple of posts by Steve Kilner that question whether Agile methods can be used effectively in software maintenance. It’s a surprising question really. There are a lot of maintenance teams who have had success following Agile methods like Scrum and Extreme Programming (XP) for some time now. We’ve been doing it for almost 5 years, enhancing and maintaining and supporting enterprise systems, and I know that it works. Agile development naturally leads into maintenance – the goal of incremental Agile development is to get working software out to customers as soon as possible, and get customers using it. At some point, when customers are relying on the software to get real business done and need support and help to keep the system running, teams cross from development over to maintenance. But there’s no reason for Agile development teams to fundamentally change the way that they work when this happens. It is harder to introduce Agile practices into a legacy maintenance team – there are a lot of technical requirements and some cultural changes that need to be made. But most maintenance teams have little to lose and lots to gain from borrowing from what Agile development teams are doing. Agile methods are designed to help small teams deal with a lot of change and uncertainty, and to deliver software quickly – all things that are at least as important in maintenance as they are in development. Technical practices in Extreme Programming especially help ensure that the code is always working – which is even more important in maintenance than it is in development, because the code has to work the first time in production. Agile methods have to be adapted to maintenance, but most teams have found it necessary to adapt these methods to fit their situations anyways. Let’s look at what works and what has to be changed to make Agile methods like Scrum and XP work in maintenance. What works well and what doesn’t Planning Game Managing maintenance isn’t the same as managing a development project – even an Agile development project. Although Agile development teams expect to deal with ambiguity and constant change, maintenance teams need to be even more flexible and responsive, to manage conflicts and unpredictable resourcing problems. Work has to be continuously reviewed and prioritized as it comes in – the customer can’t wait for 2 weeks for you to look at a production bug. The team needs a fast path for urgent changes and especially for hot fixes. You have to be prepared for support demands and interruptions. Structure the team so that some people can take care of second-level support, firefighting and emergency bug fixing and the rest of the team can keep moving forward and get something done. Build slack into schedules to allow for last-minute changes and support escalation. You will also have to be more careful in planning out maintenance work, to take into account technical and operational dependencies and constraints and risks. You’re working in the real world now, not the virtual reality of a project. Standups Standups play an important role in Agile projects to help teams come up to speed and bond. But most maintenance teams work fine without standups – since a lot of maintenance work can be done by one person working on their own, team members don’t need to listen to each other each morning talking about what they did yesterday and what they’re going to do – unless the team is working together on major changes. If someone has a question or runs into a problem, they can ask for help without waiting until the next day. Small releases Most changes and fixes that maintenance teams need to make are small, and there is almost always pressure from the business to get the code out as soon as it is ready, so an Agile approach with small and frequent releases makes a lot of sense. If the time boxes are short enough, the customer is less likely to interrupt and re-prioritize work in progress – most businesses can wait a few days or a couple of weeks to get something changed. Time boxing gives teams a way to control and structure their work, an opportunity to batch up related work to reduce development and testing costs, and natural opportunities to add in security controls and reviews and other gates. It also makes maintenance work more like a project, giving the team a chance to set goals and to see something get done. But time boxing comes with overhead – the planning and setup at the start, then deployment and reviews at the end – all of which adds up over time. Maintenance teams need to be ruthless with ceremonies and meetings, pare them down, keep only what’s necessary and what works. It’s even more important in maintenance than in development to remember that the goal is to deliver working code at the end of each time box. If some code is not working, or you’re not sure if it is working, then extend the deadline, back some of the changes out, or pull the plug on this release and start over. Don’t risk a production failure in order to hit an arbitrary deadline. If the team is having problems fitting work into time boxes, then stop and figure out what you’re doing wrong – the team is trying to do too much too fast, or the code is too unstable, or people don’t understand the code enough – and fix it and move on. Reviews and Retrospectives Retrospectives are important in maintenance to keep the team moving forward, to find better ways of working, and to solve problems. But like many practices, regular reviews reach a point of diminishing returns over time – people end up going through the motions. Once the team is setup, reviews don’t need to be done in each iteration unless the team runs into problems. Schedule reviews when you or the team need them. Collect data on how the team is working, on cycle time and bug report/fix ratios, correlate problems in production with changes, and get the team together to review if the numbers move off track. If the team runs into a serious problem like a major production failure, then get to the bottom of it through Root Cause Analysis. Sustainable pace / 40-hour week It’s not always possible to work a 40-hour week in maintenance. There are times when the team will be pushed to make urgent changes, spend late nights firefighting, releasing after hours and testing on weekends. But if this happens too often or goes on too long the team will burn out. It’s critical to establish a sustainable pace over the long term, to treat people fairly and give them a chance to do a good job. Pairing Pairing is hard to do in small teams where people are working on many different things. Pairing does make sense in some cases – people naturally pair-up when trying to debug a nasty problem or walking through a complicated change – but it’s not necessary to force it on people, and there are good reasons not to. Some teams (like mine) rely more on code reviews instead of pairing, or try to get developers to pair when first looking at a problem or change, and at the end again to review the code and tests. The important thing is to ensure that changes get looked at by at least one other person if possible, however this gets done. Collective Code Ownership Because maintenance teams are usually small and have to deal with a lot of different kinds of work, sooner or later different people will end up working on different parts of the code. It’s necessary, and it’s a good thing because people get a chance to learn more about the system and work with different technologies and on different problems. But there’s still a place for specialists in maintenance. You want the people who know the code the best to make emergency fixes or high-risk changes – or at least have them review the changes – because it has to work the first time. And sometimes you have no choice – sometimes there is only one person who understands a framework or language or technical problem well enough to get something done. Coding Guidelines – follow the rules Getting the team to follow coding guidelines is important in maintenance to help ensure the consistency and integrity of the code base over time – and to help ensure software security. Of course teams may have to compromise on coding standards and style conventions, depending on what they have inherited in the code base; and teams that maintain multiple systems will have to follow different guidelines for each system. Metaphor In XP, teams are supposed to share a Metaphor: a simple high-level expression of the system architecture (the system is a production line, or a bill of materials) and common names and patterns that can be used to describe the system. It’s a fuzzy concept at best, a weak substitute for more detailed architecture or design, and it’s not of much practical value in maintenance. Maintenance teams have to work with the architecture and patterns that are already in place in the system. What is important is making sure that the team has a common understanding of these patterns and the basic architecture so that the integrity isn’t lost – if it hasn’t been lost already. Getting the team together and reviewing the architecture, or reverse-engineering it, making sure that they all agree on it and documenting it in a simple way is important especially when taking over maintenance of a new system and when you are planning major changes. Simple Design Agile development teams start with simple designs and try to keep them simple. Maintenance teams have to work with whatever design and architecture that they inherit, which can be overwhelmingly complex, especially in bigger and older systems. But the driving principle should still be to design changes and new features as simple as the existing system lets you – and to simplify the system’s design further whenever you can. Especially when making small changes, simple, just-enough design is good – it means less documentation and less time and less cost. But maintenance teams need to be more risk adverse than development teams – even small mistakes can break compatibility or cause a run-time failure or open a security hole. This means that maintainers can’t be as iterative and free to take chances, and they need to spend more time upfront doing analysis, understanding the existing design and working through dependencies, as well as reviewing and testing their changes for regressions afterwards. Refactoring Refactoring takes on a lot of importance in maintenance. Every time a developer makes a change or fix they should consider how much refactoring work they should do and can do to make the code and design clearer and simpler, and to pay off technical debt. What and how much to refactor depends on what kind of work they are doing (making a well-thought-out isolated change, or doing shotgun surgery, or pushing out an emergency hot fix) and the time and risks involved, how well they understand the code, how good their tools are (development IDEs for Java and .NET at least have good built-in tools that make many refactorings simple and safe) and what kind of safety net they have in place to catch mistakes – automated tests, code reviews, static analysis. Some maintenance teams don’t refactor because they are too afraid of making mistakes. It’s a vicious circle – over time the code will get harder and harder to understand and change, and they will have more reasons to be more afraid. Others claim that a maintenance team is not working correctly if they don’t spend at least 50% of their time refactoring. The real answer is somewhere in between – enough refactoring to make changes and fixes safe. There are cases where extensive refactoring, restructuring or rewriting code is the right thing to do. Some code is too dangerous to change or too full of bugs to leave the way it is – studies show that in most systems, especially big systems, 80% of the bugs can cluster in 20% of the code. Restructuring or rewriting this code can pay off quickly, reducing problems in production, and significantly reducing the time needed to make changes and test them as you go forward. Continuous Testing Testing is even more important and necessary in maintenance than it is in development. And it’s a major part of maintenance costs. Most maintenance teams rely on developers to test their own changes and fixes by hand to make sure that the change worked and that they didn’t break anything as a side effect. Of course this makes testing expensive and inefficient and it limits how much work the team can do. In order to move fast, to make incremental changes and refactoring safe, the team needs a better safety net, by automating unit and functional tests and acceptance tests. It can take a long time to put in test scaffolding and tools and write a good set of automated tests. But even a simple test framework and a small set of core fat tests can pay back quickly in maintenance, because a lot changes (and bugs) tend to be concentrated in the same parts of the code – the same features, framework code and APIs get changed over and over again, and will need to be tested over and over again. You can start small, get these tests running quickly and reliably and get the team to rely on them, fill in the gaps with manual tests and reviews, and then fill out the tests over time. Once you have a basic test framework in place, developers can take advantage of TFD/TDD especially for bug fixes – the fix has to be tested anyways, so why not write the test first and make sure that you fixed what you were supposed to? Continuous Integration To get Continuous Testing to work, you need a Continuous Integration environment. Understanding, automating and streamlining the build and getting the CI server up and running and wiring in tests and static analysis checks and reporting can take a lot of work in an enterprise system, especially if you have to deal with multiple languages and platforms and dependencies between systems. But doing this work is also the foundation for simplifying release and deployment – frequent short releases means that release and deployment has to be made as simple as possible. Onsite Customer / Product Owner Working closely with the customer to make sure that the team is delivering what the customer needs when the customer needs it is as important in maintenance as it is in developing a new system. Getting a talented and committed Customer engaged is hard enough on a high-profile development project – but it’s even harder in maintenance. You may end up with too many customers with conflicting agendas competing for the team’s attention, or nobody who has the time or ability to answer questions and make decisions. Maintenance teams often have to make compromises and help fill in this role on their own. But it doesn’t all fit…. Kilner’s main point of concern isn’t really with Agile methods in maintenance. It’s with incremental design and development in general – that some work doesn’t fit nicely into short time boxes. Short iterations might work ok for bug fixes and small enhancements (they do), but sometimes you need to make bigger changes that have lots of dependencies. He argues that while Agile teams building new systems can stub out incomplete work and keep going in steps, maintenance teams have to get everything working all at once – it’s all or nothing. It’s not easy to see how big changes can be broken down into small steps that can be fit into short time boxes. I agree that this is harder in maintenance because you have to be more careful in understanding and untangling dependencies before you make changes, and you have to be more careful not to break things. The code and design will sometimes fight the kinds of changes that you need to make, because you need to do something that was never anticipated in the original design, or whatever design there was has been lost over time and any kind of change is hard to make. It’s not easy – but teams solve these problems all the time. You can use tools to figure out how much of a dependency mess you have in the code and what kind of changes you need to make to get out of this mess. If you are going to spend “weeks, months, or even years” to make changes to a system, then it makes sense to take time upfront to understand and break down build dependencies and isolate run-time dependencies, and put in test scaffolding and tests to protect the team from making mistakes as they go along. All of this can be done in time boxed steps. Just because you are following time boxes and simple, incremental design doesn’t mean that you start making changes without thinking them through. Read Working With Legacy Code – Michael Feathers walks through how to deal with these problems in detail, in both object oriented and procedural languages. What to do if it takes forever to make a change. How to break dependencies. How to find interception points and pinch points. How to find structure in the design and the code. What tests to write and how to get automated tests to work. Changing data in a production system, especially data shared with other systems, isn’t easy either. You need to plan out API changes and data structure changes as carefully as possible, but you can still make data and database changes in small, structured steps. To make code changes in steps you can use Branching by Abstraction where it makes sense (like making back-end changes) and you can protect customers from changes through Feature Flags and Dark Launching like Facebook and Twitter and Flickr do to continuously roll out changes – although you need to be careful, because if taken too far these practices can make code more fragile and harder to work with. Agile development teams follow incremental design and development to help them discover an optimal solution through trial-and-error. Maintenance teams work this way for a different reason – to manage technical risks by breaking big changes down and making small bets instead of big ones. Working this way means that you have to put in scaffolding (and remember to take it out afterwards) and plan out intermediate steps and review and test everything as you make each change. Sometimes it might feel like you are running in place, that it is taking longer and costing more. But getting there in small steps is much safer, and gives you a lot more control. Teams working on large legacy code bases and old technology platforms will have a harder time taking on these ideas and succeeding with them. But that doesn’t mean that they won’t work. Yes, you can be Agile in maintenance.

We recently wanted to get the Git history of a file which we knew existed but had now been deleted so we could find out what had happened to it. Using a simple git log didn’t work: git log deletedFile.txt fatal: ambiguous argument 'deletedFile.txt': unknown revision or path not in the working tree. We eventually came across Francois Marier’s blog post which points out that you need to use the following command instead: git log -- deletedFile.txt I’ve tried reading through the man page but I’m still not entirely sure what the distinction between using – and not using it is supposed to be. If someone could explain it that’d be cool… From http://www.markhneedham.com/blog/2011/10/04/git-getting-the-history-of-a-deleted-file

The Goal by Eli Goldratt is a business book in the form of a novel, where the protagonist must save his factory from closing due to very low productivity. The Goal is not limited to the management of a large organization (not even to for-profit companies): you simply have to define different units of measurement, like goal units instead of making money, the default goal. In fact, from the applications of the Theory of Constraints in our field I think it applies to software development too. What follows is my translation of the themes of The Goal to our field. The Goal is... Mking money, of course. For the ones of you with knowledge in accounting, the original goal is: raising throghput, the amounts of items sold (not produced) in the unit of time. Lowering investment/inventory, all the money tied up in the system in the form of assets that could be sold or products that stay in a warehouse. Lowering operational expense, all the money that we have spent as support and that cannot be recovered. How does these measurements apply to software development? A team does not always have an impact on contract negotiation, so often talking about money is far from everyday reality (kudos to you if you can apply that point of the Agile Manifesto.) The goal for software development can be translated, in my opinion, to: raising the throughput, the amount of features delivered (deployed, not implemented or tested) in the unit of time. You can measure this amount in story points, since feature vary in size. lowering investment/inventory, all the time tied up in the system in the form of undeployed or untested features that clutter the code base. In a minor part, also investment in the form of hardware, but that's by far less important than the team's time. lower operational expense, the time spent by developers every day in order to support the development. Automation is a kind of time investment that will bring more time (and quality) in the future, lowering operational expense. Like for material products, WIP has storage and opportunity costs which goes into the operational expense. Kanban is a tool that tries to reduce WIP in order to foster the two latter points. Throughput accounting This kind of throughput accounting is emphasized in the novel, over the use of cost accounting, where each developer (ehm, factory worker) has to be occupied all the time, even if the work he is doing isn't moving towards the goal: neverending refactoring. Specification of a feature which cannot be implemented until two months are gone, and will have to be rewritten. Implementation of features which won't be merged with the main branch any time soon. With Test-Driven Development, we are getting good at moving a feature from implemented to tested directly in the same commit. Yet the missing step is getting the feature to the users: maybe that's also what Continuous Deployment is all about... Dependent events Dependent events and statistical fluctuations are production systems topics that make a balanced plant close to bankruptcy: however, we're not at the point in which we can model our team as precisely as a factory. The basic point is that a plant in which everyone is working all the time is inefficient: when an early stage (like defining a specification or implementing a feature) gets delayed, downstream step such as deployment are dalayed too. Converely, when an upstream step finish earlier, the downstream stage is already at maximum efficiency and cannot process the intermediate result faster. I wonder if this applies to software development too. In a factory, workers are specialized and can do just a few jobs across the plant. Since workers and machines have different production rates, there will be just one bottleneck: the slowest one. If products have to pass from the bottleneck, anyone producing faster than the bottleneck will just accumulate WIP in front of him. Continuing with our example, if the analyst or domain expert is churning out specifications for new features every day, most of them are just WIP in front of the development team. Once there is an established buffer, any additional specification won't raise throughput any faster; instead, it will raise the inventory (partial features) and the time spent in managing it. I think this is not always true in the most technical phases of development instead. For example, in a small team a developer may be moved to testing or refactoring, or setting up Continuous Integration or evaluation of a new library. Unless you have a DBA which can just manage databases, your developer is not fixed into a stage of the system. The bottleneck The previous example featured a bottleneck, the most famous concept of the Theory of Constraints introduced in the book. This translation to the software development case is mine and could be incomplete. A feature (or a user story) has to pass in a series of stations where different people will work on it to make it real: specification from a domain expert, implementation from a technical team, extended testing with optional customer validation, deployment which should be fast but at the same time must not kill the current version of the application. Each station has an average velocity. By definition, there is a station which is the slowest and can process fewest story points in the unit of time. This is the bottleneck. (This is not always true, as velocity may vary greatly in time with the addition of new people or hidden lines discovered in a feature. Becoming good at estimation and stabilizing a team are two objectives that help reach the assumption.) You can identify a bottleneck by looking at where is the WIP: it will accumulate in front of it. If you already have a kanban board, this phase is simpler... Once identified, the throughput of the system can only be improved by raising the bottleneck's capacity enough so that it is no more a bottleneck. You can move people to it (keeping an eye on communication costs); ensure it is used at maximum efficiency by freeing the specialized developers from other mundane tasks. Now you can restart and find a new bottleneck... Conclusions This is just a little introduction to the themes of the Theory of Constraints and Goldratt's teachings; I don't pretend to explain the whole book in an article. It is also against the Socratic method: you should reach the answers yourself, and these are just examples from my experience. There is more to Goldratt and The Goal than bottlenecks and throughput, such as continuous improvement. If you're working or managing in a software development team, I suggest you to read this book if you have the opportunity. Even when freelancing, it is an eye-opener in moving towards a Goal instead of busyworking; and it's written with a never boring teaching method.

If you're looking for a cloud expert, specifically someone who knows Amazon Web Services and EC2, you'll want to have a battery of questions to assess their knowledge.

here’s a quick tip for soapui users that i have been stumbling over recently. soapui allows you to refresh an already defined service definition from an updated wsdl file. but, it’s easy to blow away the data that’s in your soap test steps if you chose the wrong options. first, here’s the specific task i’m talking about. you have an existing soapui workspace that has interface definitions already imported and you have at least one test suite using that interface. now you’ve made a change to the wsdl and you need to refresh your interface definition in soapui and update the soap test steps in your test suite. clearly, you want to update the soap requests to match the new wsdl without losing the test data that already exists in the test steps. you reload a wsdl definition by right-clicking on an interface definition in soapui as shown below. here’s a rundown of the effect of some of the “update definition” parameter configurations. i haven’t gone through every permutation of the parameters, just the ones that seem most useful. i’ve portrayed the configurations in terms of actions you’re likely to want to take after changing your wsdl. you added a new operation to the wsdl and you want to add default requests for it. you made a change to an existing operation in your wsdl and you want to regenerate existing requests using the new schema without creating optional elements. this configuration will cause you to lose the data in your existing requests . you made a change to an existing operation in your wsdl and you want to regenerate existing requests using the new schema and create optional elements. this configuration will cause you to lose the data in your existing requests, unless the elements are optional . you made a change to an existing operation in your wsdl and you want to regenerate existing requests using the new schema and create optional elements. this configuration will keep the existing data in tact. you made a change to an existing operation in your wsdl and you want to regenerate existing requests and test steps using the new schema without creating optional elements. this configuration will keep the existing data in tact but will remove optional elements from the existing requests . you made a change to an existing operation in your wsdl and you want to regenerate existing requests and test steps using the new schema and maintain optional elements. this configuration will keep the existing data and optional elements in tact. the last configuration is the safest bet for most refreshes. it combines all of the above. that means that it picks up all wsdl changes and incorporates them into both default requests and your soap test steps. none of your existing data will be overwritten. it also opens up a window that lists exactly which items have been modified. i have not mentioned the “keep soap headers” and “create backups” parameters because i have not used them yet. i suggest playing around with the “update definition” feature a little before using it. soapui is a powerful tool, so it assumes you know what you want. it accomplishes that by exposing panels with lots of parameters on them, like the “update definiton” panel. if you get yourself into a bind, remember that you can just reload the project you’re working on by right-clicking on the project name and choosing “reload project”. this option will reload the project into the workspace without saving the changes you just made. from http://thewonggei.wordpress.com/2010/12/29/soapui-tip-options-for-refreshing-a-wsdl-definition/

There is little support for testing the database of your enterprise application. We will describe some problems and possible solutions based on Hibernate and JUnit.

The integration framework Apache Camel already supports several important cloud services (see my overview article at http://www.kai-waehner.de/blog/2011/07/09/cloud-computing-heterogeneity-will-require-cloud-integration-apache-camel-is-already-prepared for more details). This article describes the combination of Apache Camel and the Amazon Web Services (AWS) interfaces of Simple Storage Service (S3), Simple Queue Service (SQS) and Simple Notification Service (SNS). Thus, The concept of Infrastructure as a Service (IaaS) is used to access messaging systems and data storage without any need for configuration. Registration to AWS and Setup of Camel First, you have to register to the Amazon Web Services (for free). Most AWS services include a free monthly quota, which is absolutely sufficient to play around and develop some simple applications. As its name states, AWS uses technology-independent web services. Besides, APIs for several different programming languages are available to ease development. By the way, Camel uses the AWS SDK for Java (http://aws.amazon.com/sdkforjava), of course. The documentation is detailed and easy to understand, including tutorials, screenshots and code examples . Hint 1: You should read the introductions to S3, SQS and SNS (go to http://aws.amazon.com and click on „products“) and play around with the AWS Management Console (http://aws.amazon.com/console) before you continue. This step is very easy and takes less than one hour. Then, you will have a much better understanding about AWS and where Camel can help you! Hint 2: It really helps to look at the source code of the camel-aws component, It helps you to understand how Camel uses the AWS Java API internally. If you want to write tests, you can do it the same way. In the past, I was afraid of looking at „complex“ source code of open source frameworks. But there is no need to be scared! The camel-aws component (and most other camel components) contain only of a few classes. Everything is easy to understand. It helps you to understand Camel internals, the AWS API, and to spot and solve errors due to exceptions in your code. In the meanwhile, the current Camel version 2.8 supports three AWS services: S3, SQS and SNS. All of them use similar concepts. Therefore, they are included in one single camel component: „camel-aws“. You have to add the libraries to your existing Camel project. As always, the simplest way is to use Maven and add the following dependency to the pom.xml: org.apache.camel camel-aws ${camel-version} Configuration of the Camel Endpoint The implementation and configuration of all three services is very similar. The URI looks like this (the code shows the SQS service): aws-sqs://queue-name[?options] There are two alternatives to configure your endpoint. Using Parameters The easy way is to use two paramters in the URI of your endpoint: „accessKey“ and „secretKey“ (you receive both after your AWS registration). “aws-sqs://unique-queue-name?accessKey=“INSERT_ME“&secretKey=INSERT_ME” Be aware of the following problem, which can result in a strange, non-speaking exception (thanks to Brendan Long): You’ll need to URL encode any +’s in your secret key (otherwise, they’ll be treated as spaces). + = %2B, so if your secretkey was “my+secret\key”, your Camel URL should have “secretKey=my%2Bsecret\key”. “Within the query string, the plus sign is reserved as shorthand notation for a space. Therefore, real plus signs must be encoded. This method was used to make query URIs easier to pass in systems which did not allow spaces.” Source: WC3 URI Recommendations Adding a configured AmazonClient to the Registry If you need to do more configuration (e.g. because your system is behind a firewall), you have to add an AmazonClient object to your registry. The following code shows an example using SQS, but SNS and S3 use exactly the same concept. @Override protected JndiRegistry createRegistry() throws Exception { JndiRegistry registry = super.createRegistry(); AWSCredentials awsCredentials = new BasicAWSCredentials(“INSERT_ME”, “INSERT_ME”); ClientConfiguration clientConfiguration = new ClientConfiguration(); clientConfiguration.setProxyHost(“http://myProxyHost”); clientConfiguration.setProxyPort(8080); AmazonSQSClient client = new AmazonSQSClient(awsCredentials, clientConfiguration); registry.bind(“amazonSQSClient”, client); return registry; } This example overwrites the createRegistry() method of a JUnit test (extending CamelTestSupport). You can also add this information to your runtime Camel application, of course. Apache Camel and the Simple Storage Service (S3) Simple Storage Service (S3) is a key-value-store. You can store small to very large data. The usage is very easy. You create buckets and put key-value data into these buckets. You can also create folders within buckets to organize your data. That’s it. You can monitor your buckets using the AWS Management Console – an intuitive GUI supporting most AWS services. The following example shows both alternatives for accessing the Amazon services (as described above): Paramenters and the AmazonClient. // Transfer data from your file inbox to the AWS S3 service from(“file:files/inbox”) // This is the key of your key-value data .setHeader(S3Constants.KEY, simple(“This is a static key”)) // Using parameters for accessing the AWS service .to(“aws-s3://camel-integration-bucket-mwea-kw?accessKey=INSERT_ME&secretKey=INSERT_ME&region=eu-west-1″); // Transfer data from the AWS S3 service to your file outbox from(“aws-s3://camel-integration-bucket-mwea-kw?amazonS3Client=#amazonS3Client&region=eu-wes”) .to(“file:files/outbox”); There are some additional parameters, for instance you can submit the desired AWS region or delete data after receiving it (see http://camel.apache.org/aws-s3.html and the corresponding SQS and SNS sites for more details about parameters and message headers). As you see in the code, you can use the AWS-S3 endpoint for producing and for consuming messages. Each bucket must be unique, thus you have to add some specific information such as your company to its name. Hint: If a bucket does not exist, Camel is creating it automatically (as the AWS API does). This concept is also used for SQS queues and SNS topics. Apache Camel and the Simple Queue Service (SQS) The Simple Queue Service (SQS) is similar to a JMS provider such as WebSphere MQ or ActiveMQ (but with some differences). You create queues and send messages to them. Consumers receive the messages. Contrary to most other AWS services, you cannot monitor queues by using the AWS management console directly. You have to use the service „Cloudwatch“ (http://aws.amazon.com/cloudwatch) and start an EC2 instance to monitor queues and its content. As you can see in the following code example, the syntax and concepts are almost the same as for the S3 service: from(“file:inbox”) .to(“aws-sqs://camel-integration-queue-mwea-kw?accessKey=INSERT_ME&secretKey=INSERT_ME”); from(“aws-sqs://camel-integration-queue-mwea-kw?amazonSQSClient=#amazonSQSClient”) .to(“file:outbox?fileName=sqs-${date:now:yyyy.MM.dd-hh:mm:ss:SS}”); Again, you can use the AWS-SQS endpoint for producing and for consuming messages. Each queue name must be unique. There exist two important differences to JMS (copy & paste from the AWS documentation): Q: How many times will I receive each message? Amazon SQS is engineered to provide “at least once” delivery of all messages in its queues. Although most of the time each message will be delivered to your application exactly once, you should design your system so that processing a message more than once does not create any errors or inconsistencies. Q: Why are there separate ReceiveMessage and DeleteMessage operations? When Amazon SQS returns a message to you, that message stays in the queue, whether or not you actually received the message. You are responsible for deleting the message; the delete request acknowledges that you’re done processing the message. If you don’t delete the message, Amazon SQS will deliver it again on another receive request. Apache Camel and the Simple Notification Service (SNS) The Simple Notification Service (SNS) acts like JMS topics. You create a topic, consumers subscribe to the topic and then receive notifications. Several transport protocols are supported: HTTP(S), Email and SQS. Further interfaces will be added in the future, e.g. the Short Message Service (SMS) for mobile phones. Contrary to S3 and SQS, Camel only offers a producer endpoint for this AWS service. You can only create topics and send messages via Camel. The reason is simple: Camel already offers endpoints for consuming these messages: HTTP, Email and SQS are already available. There is one tradeoff: A consumer cannot subscribe to topics using Camel – at the moment. The AWS Management Console has to be used. A very interesting discussion can be read on the Camel JIRA issue regarding the following questions: Should Camel be able to subscribe to topics? Should the producer contain this feature or should there be a consumer? In my opinion, there should be a consumer which is able to subscribe to topics, otherwise Camel is missing a key part of the AWS SNS service! Please read the discussion and contribute your opinion: https://issues.apache.org/jira/browse/CAMEL-3476. Apache Camel is already ready for the Cloud Computing Era AWS offers many more services for the cloud. Probably, it does not make sense to integrate everyone into Camel, but more AWS services will be supported in the future. For instance, SimpleDB and the Relational Database Service (RDS) are already planned and make sende, too: http://camel.apache.org/aws.html. The conclusion is easy: Apache Camel is already ready for the cloud computing era. Several important cloud services are already supported. Cloud integration will become very important in the future. Thus, Camel is on a very good way. Hopefully, we will see more cloud components, soon. I will continue to write articles about other Camel cloud components (and new AWS addons, ouf course). For instance, a component for the Platform as a Service (PaaS) product Google App Engine (GAE) is already available. If you have any additional important information, questions or other feedback, please write a comment. Thank you in advance… Best regards, Kai Wähner (Twitter: @KaiWaehner) [Content from my Blog: Cloud Integration with Apache Camel and Amazon Web Services (AWS): S3, SQS and SNS]

This article shows *one* way to mock the JMS infrastructure in a Spring JMS application. This allows us to test our JMS infrastructure without actually having to depend on a physical connection being available. If you are reading this article, chances are that you are also frustrated with failing tests in your continuous integration environment due to a JMS server being (temporarily) unavailable. By mocking the JMS provider, developers are left free to test not only the functionality of their API (unit tests) but also the plumbing of the different components, e.g. in a Spring container. In this article I show how a Spring JMS Hello World application can be fully tested without the need of a physical JMS connection. I would like to stress the fact that the code in this article is by no means meant for production and that the approach shown is just one of many. The infrastructure For this article I use the following infrastructure: Apache ActiveMQ, an open source JMS provider, running on an Ubuntu installation Spring 3 Java 6 MockRunner Eclipse as development environment, running on Windows 7 The Spring configuration It's my belief that using what I define as Spring Configuration Strategy Pattern (SCSP) is the right solution in almost all cases when there is the need for a sound testing infrastructure. I will dedicate an entire article to SCSP, for now this is how it looks: The Spring application context Here follows the content of jemosJms-appContext.xml The only important thing to note here is that there are some services which rely on an existing bean named jmsConnectionFactory but that such bean is not defined in this file. This is key to the SCSP and I will illustrate this in one of my future articles. The Spring application context implementation Here follows the content of jemosJms-appContextImpl.xml which could be seen as an implementation of the Spring application context defined above This Spring context file imports the Spring application context defined above and it is this application context which declared the connection factory. This decoupling of the bean requirement (in the super context) from its actual declaration (Spring application context implementation) represents the cornerstore of SCSP. Mocking the JMS provider - The Spring Test application context and MockRunner Following the same approach I used above, I can now declare a fake connection factory which does not require a physical connection to a JMS provider. Here follows the content of jemosJmsTest-appContext.xml. Please note that this file should reside in the test resources of your project, i.e. it should never make it to production. Here the Spring test application context file imports the Spring application context (not its implementation) and it declares a fake connection factory, thanks to the MockRunner MockQueueConnectionFactory class. A POJO listener The job of handling the message is delegated to a simple POJO, which happens to be declared also as a bean: package uk.co.jemos.experiments; public class HelloWorldHandler { /** The application logger */ private static final org.apache.log4j.Logger LOG = org.apache.log4j.Logger .getLogger(HelloWorldHandler.class); public void handleHelloWorld(String msg) { LOG.info("Received message: " + msg); } } There is nothing glamorous about this class. In real life this should have probably be the implementation of an interface, but here I wanted to keep things simple. A simple JMS message producer Here follows an example of a JMS message producer, which would use the real JMS infrastructure to send messages: package uk.co.jemos.experiments; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import org.springframework.jms.core.JmsTemplate; public class JmsTest { /** The application logger */ private static final org.apache.log4j.Logger LOG = org.apache.log4j.Logger .getLogger(JmsTest.class); /** * @param args */ public static void main(String[] args) { ApplicationContext ctx = new ClassPathXmlApplicationContext( "classpath:jemosJms-appContextImpl.xml"); JmsTemplate jmsTemplate = ctx.getBean(JmsTemplate.class); jmsTemplate.send("jemos.tests", new HelloWorldMessageCreator()); LOG.info("Message sent successfully"); } } The only thing of interest here is that this class retrieves the real JmsTemplate to send a message to the queue. Now if I was to run this class as is, I would obtain the following: 2011-07-31 17:09:46 ClassPathXmlApplicationContext [INFO] Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@19e0ff2f: startup date [Sun Jul 31 17:09:46 BST 2011]; root of context hierarchy 2011-07-31 17:09:46 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJms-appContextImpl.xml] 2011-07-31 17:09:46 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJms-appContext.xml] 2011-07-31 17:09:46 DefaultListableBeanFactory [INFO] Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@3479e304: defining beans [helloWorldConsumer,jmsTemplate,org.springframework.jms.listener.DefaultMessageListenerContainer#0,jmsConnectionFactory]; root of factory hierarchy 2011-07-31 17:09:46 DefaultLifecycleProcessor [INFO] Starting beans in phase 2147483647 2011-07-31 17:09:47 HelloWorldHandler [INFO] Received message: Hello World 2011-07-31 17:09:47 JmsTest [INFO] Message sent successfully Writing the integration test There are various interpretations as to what different types of tests mean and I don't pretend to have the only answer; my interpreation is that an integration test is a functional test which also wires up different components together but which does not interact with real external infrastructure (e.g. a Dao integration test fakes data, a JMS integration test fakes the JMS physical connection, an HTTP integration test fakes the remote Web host, etc). Whereas in my opinion, the main purpose of a unit (aka functional) test is to let the API emerge from the tests, the main goal of an integration test is to test that the plumbing amongst components works as expected so as to avoid surprises in a production environment. Both unit (functional) and integration tests should run very fast (e.g. under 10 minutes) as they constitute what can be considered the "development token". If unit and integration tests are green one should feel pretty confident that 90% of the functionality works as expected; in my projects when both unit and integration tests are green I let developers free to release the token. This does not mean that the other 10% (e.g. the interaction with the real infrastructure) should not be tested, but this can be delegated to system tests which run nightly and don't require the development token. Because unit and integration tests need to run fast, interaction with external infrastructure should be mocked whenever possible. Here follows an integration test for the Hello World handler: package uk.co.jemos.experiments.test.integration; import javax.annotation.Resource; import javax.jms.TextMessage; import junit.framework.Assert; import org.junit.Before; import org.junit.Test; import org.springframework.jms.core.JmsTemplate; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests; import uk.co.jemos.experiments.HelloWorldHandler; import uk.co.jemos.experiments.HelloWorldMessageCreator; import com.mockrunner.jms.DestinationManager; import com.mockrunner.mock.jms.MockQueue; /** * @author mtedone * */ @ContextConfiguration(locations = { "classpath:jemosJmsTest-appContextImpl.xml" }) public class HelloWorldHandlerIntegrationTest extends AbstractJUnit4SpringContextTests { @Resource private JmsTemplate jmsTemplate; @Resource private DestinationManager mockDestinationManager; @Resource private HelloWorldHandler helloWorldHandler; @Before public void init() { Assert.assertNotNull(jmsTemplate); Assert.assertNotNull(mockDestinationManager); Assert.assertNotNull(helloWorldHandler); } @Test public void helloWorld() throws Exception { MockQueue mockQueue = mockDestinationManager.createQueue("jemos.tests"); jmsTemplate.send(mockQueue, new HelloWorldMessageCreator()); TextMessage message = (TextMessage) jmsTemplate.receive(mockQueue); Assert.assertNotNull("The text message cannot be null!", message.getText()); helloWorldHandler.handleHelloWorld(message.getText()); } } And here follows the output: 2011-07-31 17:17:26 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJmsTest-appContextImpl.xml] 2011-07-31 17:17:26 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJms-appContext.xml] 2011-07-31 17:17:26 GenericApplicationContext [INFO] Refreshing org.springframework.context.support.GenericApplicationContext@f01a1e: startup date [Sun Jul 31 17:17:26 BST 2011]; root of context hierarchy 2011-07-31 17:17:27 DefaultListableBeanFactory [INFO] Pre-instantiating singletons in org.springframework.beans.factory.support. DefaultListableBeanFactory@39478a43: defining beans [helloWorldConsumer,jmsTemplate,org.springframework.jms.listener.DefaultMessageListener Container#0,destinationManager,configurationManager,jmsConnectionFactory,org.springframework.context.annotation.internalConfigurationAnnotation Processor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequired AnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor]; root of factory hierarchy 2011-07-31 17:17:27 DefaultLifecycleProcessor [INFO] Starting beans in phase 2147483647 2011-07-31 17:17:27 HelloWorldHandler [INFO] Received message: Hello World 2011-07-31 17:17:27 GenericApplicationContext [INFO] Closing org.springframework.context.support.GenericApplicationContext@f01a1e: startup date [Sun Jul 31 17:17:26 BST 2011]; root of context hierarchy 2011-07-31 17:17:27 DefaultLifecycleProcessor [INFO] Stopping beans in phase 2147483647 2011-07-31 17:17:32 DefaultMessageListenerContainer [WARN] Setup of JMS message listener invoker failed for destination 'jemos.tests' - trying to recover. Cause: Queue with name jemos.tests not found 2011-07-31 17:17:32 DefaultListableBeanFactory [INFO] Destroying singletons in org.springframework.beans.factory.support. DefaultListableBeanFactory@39478a43: defining beans [helloWorldConsumer,jmsTemplate,org.springframework.jms.listener.DefaultMessageListener Container#0,destinationManager,configurationManager,jmsConnectionFactory,org.springframework.context.annotation.internalConfigurationAnnotationProcessor ,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context. annotation.internalCommonAnnotationProcessor]; root of factory hierarchy In this test, although we simulated a message roundtrip to a JMS queue, the message never left the current JVM and it the whole execution did not depend on a JMS infrastructure being up. This gives us the power to simulate the JMS infrastructure, to test the integration of our business components without having to fear a red from time to time due to JMS infrastructure being down or inaccessible. Please note that in the output there are some warnings because the JMS listener container declared in the jemosJms-appContext.xml does not find a queue named "jemos.test" in the fake connection factory, but this is fine; it's a warning and does not impede the test from running successfully. The Maven configuration Here follows the Maven pom.xml to compile the example: 4.0.0 uk.co.jemos.experiments jmx-experiments 0.0.1-SNAPSHOT Jemos JMS experiments junit junit 4.8.2 test com.mockrunner mockrunner 0.3.1 test log4j log4j 1.2.16 compile org.slf4j slf4j-api 1.6.1 compile org.slf4j slf4j-simple 1.6.1 compile org.apache.activemq activemq-all 5.5.0 compile org.springframework spring-beans 3.0.5.RELEASE org.springframework spring-context 3.0.5.RELEASE org.springframework spring-core 3.0.5.RELEASE org.springframework spring-jms 3.0.5.RELEASE org.springframework spring-test 3.0.5.RELEASE test From http://tedone.typepad.com/blog/2011/07/mocking-spring-jms-with-mockrunner.html

This blog is about my new Spring-CDI modules effort. It's pupose is to make useful CDI patterns like decorators or interceptors available to a Spring application. I do believe that the explicit pattern implementation in CDI is very useful. It makes it obvious and simple to use patterns for not so experienced developers. Therefore I decided to investigate how to make those patterns and the corresponding CDI annotations available for Spring managed beans. Here is the current status of my work. If you're interested and you have some time left, take a look or try out my early version of the Spring-CDI decorator module. The set-up is straight forward. You'll find all you need below. Please notice: The intention of my blog is to share and discuss ideas. If you use any of this in your applications you're acting at your own risk. JSR-299 decorator pattern implementation Features The decorator module provides the following features: - Use JSR-299 @Decorator and @Delegate in Spring managed beans - Support chains of multiple decorators for the same target delegate bean - Allow to qualify decorators to decorate multiple implementations of the same interface with different decorators - Support scoped beans, allow scoped decorators - Integrate with Spring AOP, both dynamic JDK proxies and CGLIB proxies - Allow definition of custom decorator and delegate annotations Download Link The Spring-CDI decorator module is an usual Spring IoC-container extension delivered as JAR archive. You can download the module JAR and put that on the classpath of your Spring application. Compiled Spring-CDI decorator module JAR: download here Sources: download here API-Doc: view here Everything is hosted on a git repository on Github.com. Dependencies org.springframework spring-context ${spring.version} org.springframework org.springframework.aop ${spring.version} javax.enterprise cdi-api 1.0 cglib cglib 2.2.2 Configuration If the Spring-CDI decorator module JAR and its dependencies are on your classpath, all you need to do is: (1) register DecoratorAwareBeanFactoryPostProcessor in your application context (2) define an include-filter to include javax.decorator.Decorator as component annotation in your context:component-scan tag Use Case The following code snippets show how you can use the decorator pattern ones you have configured your Spring application as described above. For more complex scenarios see my unit test cases. Let's assume you have a business interface called: MyService package com.mycompany.springapp.springcdi; public interface MyService { String sayHello(); } This is your implementation of the service. package com.mycompany.springapp.springcdi; import org.springframework.stereotype.Component; @Component public class MyServiceImpl implements MyService { public String sayHello() { return "Hello"; } } You want to do some transaction and security stuff, but you do not want to mess up the business code with it. For security you'd write a decorator that points to the MyService business service. package com.mycompany.springapp.springcdi; import javax.decorator.Decorator; import javax.decorator.Delegate; @Decorator public class MyServiceSecurityDecorator implements MyService { @Delegate private MyService delegate; public String sayHello() { // do some security stuff return delegate.sayHello(); } } To seperate the cross-cutting-concerns you write another decorator for transaction handling that points to the MyService business service. package com.mycompany.springapp.springcdi; import javax.decorator.Decorator; import javax.decorator.Delegate; @Decorator public class MyServiceTransactionDecorator implements MyService{ @Delegate private MyService delegate; public String sayHello() { // do some transaction stuff return delegate.sayHello(); } } Then you can just use standard Spring @Autowired annotation to make that work. The injected bean will be decorated with your new security and transaction decorator. package com.mycompany.springapp.springcdi; import org.junit.Assert; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @ContextConfiguration("/test-decorator-context.xml") @RunWith(SpringJUnit4ClassRunner.class) public class DecoratorTestCase { // This injected bean will be a decorated MyServiceImpl @Autowired private MyService service; @Test public void testHelloWorld() { Assert.assertTrue(service.sayHello().equals("Hello")); } } How it works The core is the DecoratorAwareBeanFactoryPostProcessor that scans the registered bean definitions for existing decorators. It gathers meta data and stores that data in the DecoratorMetaDataBean. The DecoratorAwareBeanPostProcessor uses the meta data to wire the decorators into a chain and creates a CGLIB proxy that intercepts method calls to the target delegate bean. It redirects those calls to the decorator chain. The DecoratorAutowireCandidateResolver applies autowiring rules specific to the CDI decorator pattern. It also uses meta data to do that. The two modes The DecoratorAwareBeanFactoryPostProcessor accepts two runtime modes. The 'processor' (default) mode uses DecoratorAwareBeanPostProcessor and the DecoratorChainingStrategy to wire the decorator chain. The 'resolver' mode uses DecoratorAwareAutowireCandidateResolver to implement custom wiring logic based on complex wiring rules implemented in ResolverCDIAutowiringRules. The 'resolver' mode was just another option how one can implement such complex logic. I tried two different options and both work. The 'processor' alternative however implements simpler logic. Therefore it's my prefered mode at the moment. Decorator Meta Data Model The DecoratorAwareBeanFactoryPostProcessor scans bean definitions and stores meta data about the decorators and delegates in the application context. These are the model beans in their hierarchical access order: DecoratorMetaDataBean.java: Top level entry point to the meta-data. Registered and available in the application context. QualifiedDecoratorChain.java: A chain of decorators for the same target delegate bean. DecoratorInfo.java: A decorator bean definition wrapper class. DelegateField.java: Contains the delegate field of the decorator implementation. Strategies The Spring-CDI decorator module is easy to adopt by users through the use of strategy pattern in many places. These are the strategies that allow users to change processing logic if required: DecoratorChainingStrategy.java: Wires the decorators for a specific target delegate bean. DecoratorOrderingStrategy.java: Orders the decorators for a specific target delegate bean. DecoratorResolutionStrategy.java: Scans the bean factory for available decorator beans. DelegateResolutionStrategy.java: Searches the delegate bean for a specific decorator bean. Decorator Autowiring Rules The 'processor' mode and the 'resolver' mode both use a custom AutowireCandidateResolver applied to the current bean factory. The class is called DecoratorAwareAutowireCandidateResolver and it is applied to the bean factory in the DecoratorAwareBeanFactoryPostProcessor. The custom resolver works with different rule sets. In the 'processor' mode it works with a very simple rule set called BeanPostProcessorCDIAutowiringRules. In the 'resolver' mode it uses ResolverCDIAutowiringRules which is far more complex. If these rule sets are not sufficient for your autowiring logic, it's easy to apply additional rule sets by implementing a custom SpringCDIPlugin and adding it to the DecoratorAwareAutowireCandidateResolver. Spring-CDI Plugin System The Spring-CDI decorator module contains two infrastructure interfaces that allow the modularized approach of Spring-CDI project: SpringCDIPlugin and SpringCDIInfrastructure. When I implement additional modules - like the interceptor module - users can decide which modules to use and import into their projects. It's not required to add all Spring-CDI functionality if one only needs decorators. From http://niklasschlimm.blogspot.com/2011/08/jsr-299-cdi-decorators-for-spring-beans.html

So far, in all my examples, I have been using the declarative way of creating an Android UI using XML. However, there could arise certain situations when you may have to create UI programmatically. Sincere advice would be to avoid such a design since android has a wonderful architecture where the UI and the program are well separated. However, for those few exceptional cases where we may need too… here is how we do it. Every single view or viewgroup element has an equivalent java class in the SDK. The structure and naming of the classes and methods is very similar to the XML vocabulary that we are used to so far. Let us start with a LinearLayout. How would we declare it in an XML? This just contains a TextView embedded in a LinearLayout. A very trivial example. But serves the purpose intended. Let me show how almost every single element here corresponds to a class or a method call in the class. So the equivalent code in the onCreate(…) method of an activity would be like this: super.onCreate(savedInstanceState); lLayout = new LinearLayout(this); lLayout.setOrientation(LinearLayout.VERTICAL); //-1(LayoutParams.MATCH_PARENT) is fill_parent or match_parent since API level 8 //-2(LayoutParams.WRAP_CONTENT) is wrap_content lLayout.setLayoutParams(new LayoutParams( LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT)); tView = new TextView(this); tView.setText("Hello, This is a view created programmatically! " + "You CANNOT change me that easily :-)"); tView.setLayoutParams(new LayoutParams( LayoutParams.MATCH_PARENT, LayoutParams.WRAP_CONTENT)); lLayout.addView(tView); setContentView(lLayout); Like this any layout view can be created. But from this small example you can notice two outstanding things – very tedious to code for every attribute of the view. And any simple change in the view, you need to change the code, compile, deploy and only then you see the effect of the change – unlike in a layout editor. You can download the sample code here.

In Spring Roo in Action, Chapter 3, I discuss how Roo automatically executes the Bean Validators when persisting a live entity. However, when running unit tests, we don't have a live entity at all, nor do we have a Spring container - so how can we exercise the validation without actually hitting our Roo application and the database? The following post is ancillary material from the upcoming book Spring Roo in Action, by Ken Rimple and Srini Penchikala, with Gordon Dickens. You can purchase the MEAP edition of the book, and participate in the author forum, at www.manning.com/rimple. The answer is that we have to bootstrap the validation framework within the test ourselves. We can use the CourseDataOnDemand class's getNewTransientEntityName method to generate a valid, transient JPA entity. Then, we can: Mock static entity methods, such as findById, to bring back pre-fabricated class instances of your entity Initialize the validation engine, bootstrapping a JSR-303 bean validation framework engine, and perform validation on your entity Set any appropriate properties to apply to a particular test condition Initialize a test instance of the entity validator and assert the appropriate validation results are returned The concept in action... Given a Student entity with the following definition: @RooEntity @RooJavaBean @RooToString public class Student { @NotNull private String emergencyContactInfo; ... } The listing below shows a unit test method that ensures the NotNull validation fires against missing emergency contact information on the Student entity: @Test public void testStudentMissingEmergencyContactValidation() { // setup our test data StudentDataOnDemand dod = new StudentDataOnDemand(); // tell the mock to expect this call Student.findStudent(1L); // tell the mocking API to expect a return from the prior call in the form of // a new student from the test data generator, dod AnnotationDrivenStaticEntityMockingControl.expectReturn( dod.getNewTransientStudent(0)); // put our mock in playback mode AnnotationDrivenStaticEntityMockingControl.playback(); // Setup the validator API in our unit test LocalValidatorFactoryBean validator = new LocalValidatorFactoryBean(); validator.afterPropertiesSet(); // execute the call from the mock, set the emergency contact field // to an invalid value Student student = Student.findStudent(1L); student.setEmergencyContactInfo(null); // execute validation, check for violations Set> violations = validator.validate(student, Default.class); // do we have one? Assert.assertEquals(1, violations.size()); // now, check the constraint violations to check for our specific error ConstraintViolation violation = violations.iterator().next(); // contains the right message? Assert.assertEquals("{javax.validation.constraints.NotNull.message}", violation.getMessageTemplate()); // from the right field? Assert.assertEquals("emergencyContactInfo", violation.getPropertyPath().toString()); } Analysis The test starts with a declaration of a StudentOnDemand object, which we'll use to generate our test data. We'll get into the more advanced uses of the DataOnDemand Framework later in the chapter. For now, keep in mind that we can use this class to create an instance of an Entity, with randomly assigned, valid data. We then require that the test calls the Student.findStudent method, passing it a key of 1L. Next, we'll tell the entity mocking framework that the call should return a new transient Student instance. At this point, we've defined our static mocking behavior, so we'll put the mocking framework into playback mode. Next, we issue the actual Student.findById(1L) call, this time storing the result as the member variable student. This call will trip the mock, which will return a new transient instance. We then set the emergencyContactInfo field to null, so that it becomes invalid, as it is annotated with a @NotNull annotation. Now we are ready to set up our bean validation framework. We create a LocalValidatorFactoryBean instance, which will boot the Bean Validation Framework in the afterPropertiesSet() method, which is defined for any Spring Bean implementing InitializingBean. We must call this method ourselves, because Spring is not involved in our unit test. Now we're ready to run our validation and assert the proper behavior has occurred. We call our validator's validate method, passing it the student instance and the standard Default validation group, which will trigger validation. We'll then check that we only have one validation failure, and that the message template for the error is the same as the one for the @NotNull validation. We also check to ensure that the field that caused the validation was our emergencyContactInfo field. In our answer callback, we can launch the Bean Validation Framework, and execute the validate method against our entity. In this way, we can exercise our bean instance any way we want, and instead of persisting the entity, can perform the validation phase and exit gracefully. Caveats... There are a few things slightly wrong here. First of all, the Data on Demand classes actually use Spring to inject relationships to each other, which I've logged a bug against as ROO-2497. You can override the setup of the data on demand class and manually create the DoD of the referring one, which is fine. They have slated to work on this bug for Roo 1.2, so it should be fixed sometime in the next few months. Also, realize that this is NOT easy to do, compared to writing an integration test. However, this test runs markedly faster. If you have some sophisticated logic that you've attached to a @AssertTrue annotation, this is the way to test it in isolation. From http://www.rimple.com/tech/2011/7/17/testing-entity-validations-with-a-mock-entity-roo-in-action.html

Recently I was asked by a customer how they could import VMs from an existing vSphere environment into a vCloud Director environment. This particular customer is aiming to pull VMs from an existing managed hosting business so they’ll use the third of three options that I’ve described below. For pulling VMs into vCD you basically have 3 options. 1) You can log into the vCD interface and upload VMs manually through the interface. This is done one by one. This is a good option if you’re an end user that doesn’t have vCC setup (option 2) or doesn’t want to go through the process of using the vCloud API (a more automated version of this option). The downside of this option for my customer is it would require additional storage since the VMs would first upload to a staging area (the vCloud Director transfer pool) and then copy to their destination. 2) You can setup vCloud Connector (vCC) in your existing vSphere environment and use that to move VMs from the vCenter Client interface into vCloud Director. This again is done one at a time. The benefit is you’re using your existing tool set (vCenter Client) to perform this operation. The down side is you’ll need to configure vCC for each user of the vCenter Client that wants to do this operation. This would be a viable option for my customer since they own the source vSphere environment. It still isn’t the most straightforward for them and it can’t be automated with an API call so this option is out. 3) If you own the vSphere environment like this customer does and you want to migrate that vSphere environment into a parallel vCD environment then you can just add the managing VC to the vCD environment and import VMs directly into vCD. This is the method that I’m going to detail below. If you’re ready to go about moving VMs into your vCD environment then the first thing you need to do is prepare to add your managing VC to your vCD environment. To do this you’ll need to deploy a vShield Manager instance. Even if you don’t use the security features of vSM you’ll still need to have one present so you can get through the add VC wizard in vCD. vSM comes as a virtual appliance so we’ll begin by just adding it through the deploy OVF wizard in VC. After vSM is deployed you can see it as a resource VM in our current vSphere environment. After we finish importing everything we can remove the vSM VM again. Now that we have vSM in place we can go and attach our managing VC to our vCD environment. Right now my test vCD environment already has one VC under its control as you can see here. Now we’re going to go through the wizard to add our second VC which is the one we want to import our VMs from. The wizard is pretty simple. First we give it our VC name and credentials. On the next screen we give the name and credentials to the vSM we deployed earlier. After the wizard completes you can see we have our old VC and our new VC under the control of our vCD environment. This does not preclude you from continuing to use your vSphere environment as you always have. It just now allows vCD to see and consume some of those resources such as the existing VMs. The next step is very important. We need to login to vCD as the system admin. This is because org admins don’t get to see the underlying vSphere resources and so they won’t be able to do the import straight from VC. Once we’re logged in as the system admin we can open up an organization to perform the import of VMs into that org. In my case the test organization is called MikeD. You can see the System tab in the screenshot below as well which indicates that we were initially logged in as the system admin and simply opened the organization in another tab. Now that we’re inside the organization of our choice we can click the import VC button on the top toolbar. The import vApp/VM wizard should appear and we can see a listing of all of the VMs in our vSphere environment. To import one it’s as simple as selecting the VM, giving it a name, telling vCD which orgVDC we would like to place it in and then hitting ok. There is one last choice to make and that’s whether or not to copy the VM to vCD which will put a new instance in the orgVDC but leave a copy on VC or move the VM which will delete the VM from the original VC environment and make a new copy in the vCD orgVDC resource destination. This choice is up to the person doing the migration and both choices have merit. A lot of people like copy because they have the original copy in place should a customer decide they need to continue using it or they just want it for backup. After you’re done with the wizard your VM will start to import. After a little while you’ll see the finished VM in the interface. Of course you’ll also see the in progress VM along with a task that’s running. As you can see the overall process for setting this up is not that labor intensive. It is a little labor intensive to go into the interface to do the import one VM at a time. Of course you could always use the vCloud API to do the imports once you have the vSphere environment added to vCD. All of this will allow current managed hosting providers to migrate their VMs to the new vCD based clouds they are creating. Hopefully this has been useful. As always if there are any comments or questions then just put them below and I’ll get back to you.