There are many good ideas and practices in Agile development, ideas and practices that definitely work: breaking projects into Small Releases to manage risk and accelerate feedback; time-boxing to limit WIP and keep everyone focused; relying only on working software as the measure of progress; simple estimating and using velocity to forecast team performance; working closely and constantly with the customer; and Continuous Integration – and Continuous Delivery – to ensure that code is always working and stable. But there are other commonly accepted ideas and best practices that aren’t important: if you don’t follow them, nothing bad will happen to you and your project will still succeed. And there are a couple that you are better off not following at all. Test-Driven Development Teams that need to move quickly need to depend on a fast, efficient testing safety net. With Test First Development or Test-Driven Development (TDD), there’s no excuse for not writing tests – after all, you have to write a failing test before you write the code. So you end up with a good set of working automated tests that ensure a high level of coverage and regression protection. TDD is not only a way of ensuring that developers test their code. It is also advocated as a design technique that leads to better quality code and a simpler, cleaner design. A study of teams at Microsoft and IBM (Realizing Quality Improvement through Test Driven Development, Microsoft Research, 2008) found that while TDD increased upfront development costs between 15-35% (TDD demands developers change the way that they think and work, which slows developers down, at least at first), it reduced defect density by 40% (IBM) or as much as 60-90% (Microsoft) over teams that did not follow disciplined unit testing. But in Making Software Chapter 12 “How Effective is Test-Driven Development” researchers led by Burak Turhan found that while TDD improves external quality (measured by one or more of test cases passed, number of defects, defect density, defects per test, effort required to fix defects, change density, % of preventative changes) and can improve the quality of the tests (fewer mistakes in the tests, tests that are easier to maintain), TDD does not consistently improve the quality of the design. TDD seems to reduce code complexity and improve reuse, however it also negatively impacts coupling and cohesion. And while method and class-level complexity is better in code developed using TDD, project/package level complexity is worse. People who like TDD like it a lot, so if you like it, do it. And even if you are not TDD-infected, there are times when working test first is natural – when you have to solve a specific problem in a specific way, or if you’re fixing a bug where the failing test case is already written up for you. But the important thing is that you write a good set of tests and keep them up to date and run them frequently – it doesn't matter if you write them before, or after, you write the code. Pair Programming According to the VersionOne State of Agile Development Survey 2012, almost 1/3 of teams follow pair programming – a surprisingly high number, given how disciplined pair programming is, and how few teams follow XP (2%) or Scrum/XP Hybrid (11%) methods where pair programming would be prescribed. There are good reasons for pairing: information sharing and improving code quality through continuous, informal code reviews as developers work together. And there are natural times to pair developers, or sometimes developers and testers, together: when you’re working through a hard design problem; or on code that you’ve never seen before and somebody who has worked on it is available to help; or when you’re over your head in troubleshooting a high-pressure problem; or testing a difficult part of the system; or when a new person joins the team and needs to learn about the code and coding practices. Some (extroverted) people enjoy pairing up, the energy it creates and the opportunities it provides to get to know others on the team. But forcing people who prefer working on their own or who don’t like each other to work closely together is definitely not a good idea. There are real social costs in pairing: you have to be careful to pair people up by skill, experience, style, personality type and work ethic. And sustained pair programming can be exhausting, especially over the long term – one study (Vanhanen and Lassenius 2007) found that people only pair between 1.5 and 4 hours a day on average, because it’s too intense to do all day long. In Pair Programming Considered Harmful? Jon Evans says that pairing can have also negative effects on creativity: Research strongly suggests that people are more creative when they enjoy privacy and freedom from interruption … What distinguished programmers at the top-performing companies wasn’t greater experience or better pay. It was how much privacy, personal workspace and freedom from interruption they enjoyed,” says a New York Times article castigating “the new groupthink”. And in “Still Questioning Extreme Programming” Pete McBreen points out some other disadvantages and weaknesses of pair programming: Exploration of ideas is not encouraged, pairing makes a developer focus on writing the code, so unless there is time in the day for solo exploration the team gets a very superficial level of understanding of the code. Developers can come to rely too much on the unit tests, assuming that if the tests pass then the code is OK. (This follows on from the lack of exploration.) Corner cases and edge cases are not investigated in detail, especially if they are hard to write tests for. Code that requires detail thinking about the design is hard to do when pairing unless one partner completely dominates the session. With the usual tradeoff between partners, it is hard to build technically complex designs unless they have been already been worked out in a solo session. Personal styles matter when pairing, and not all pairings are as productive as others. Pairs with different typing skills and proficiencies often result in the better typist doing all of the coding with the other partner being purely passive. And of course pairing in distributed teams doesn't work well if at all (depending on distance, differences in time zones, culture, working styles, language), although some people still try. While pairing does improve code quality over solo programming, you can get the same improvements in code quality, and at least some of the information sharing advantages, through code reviews, at less cost. Code reviews – especially lightweight, offline reviews – are easier to schedule, less expensive and less intrusive than pairing. And as Jason Cohen points out even if developers are pair programming, you may still need to do code reviews, because pair programming is really about joint problem solving, and doesn’t cover all of the issues that a code review would. Back to Jon Evans for the final word on pair programming: The true answer is that there is no one answer; that what works best is a dynamic combination of solitary, pair, and group work, depending on the context, using your best judgement. Paired programming definitely has its place. (Betteridge’s Law strikes again!) In some cases that place may even be “much of most days.” But insisting on 100 percent pairing is mindless dogma, and like all mindless dogma, ultimately counterproductive. Emergent Design and Metaphor Incremental development works, and trying to keep design simple makes good sense, but attempting to define an architecture on the fly is foolish and impractical. There’s a reason that almost nobody actually follows Emergent Design: it doesn't work. Relying on a high-level metaphor (the system is an "assembly line" or a "bill of materials" or a "hive of bees") shared by the team as some kind of substitute for architecture is even more ridiculous. Research from Carnegie Mellon University found that … natural language metaphors are relatively useless for either fostering communication among technical and non-technical project members or in developing architecture. Almost no one understands what a system metaphor is any ways, or how it is to be used, or how to choose a meaningful metaphor or how to change it if you got it wrong (and how you would know if you got it wrong), including one of the people who helped come up with the idea: Okay I might as well say it publicly - I still haven't got the hang of this metaphor thing. I saw it work, and work well on the C3 project, but it doesn't mean I have any idea how to do it, let alone how to explain how to do it. Martin Fowler, Is Design Dead? Agile development methods have improved development success and shown better ways to approach many different software development problems – but not architecture and design. Daily Standups When you have a new team and everyone needs to get to know each other and more time to understand what the project is about; or when the team is working under emergency conditions trying to fix something or finish something under extreme pressure, then getting everyone together in regular meetings, maybe even more than once a day, is necessary and valuable. But whether everyone stands up or sits down and what they end up talking about in a meeting should be up to you. If your team has been working well together for a while and everyone knows each other and knows what they are working on, and if developers update cards on a task board or a Kanban board or the status in an electronic system as they get things done, and if they are grown up enough to ask for help when they need it, then you don’t need to make them all stand up in a room every morning. Collective Code Ownership Letting everyone work on all of the code isn't always practical (because not everyone on the team has the requisite knowledge or experience to work on every problem) and collective code ownership can have negative effects on code quality. Share code where it makes sense to do so, but realize that not everybody can – or should – work on every part of the system. Writing All Requirements as Stories The idea that every requirement specification can be written as User Stories in 1 or 2 lines on cards, that requirements should be too short on purpose (so that the developer has to talk to someone to explain what’s really needed) and insisting that they should all be in the same template form “As a type of user I want some goal so that some reason…” is silly and unnecessary. This is the same kind of simple minded orthodoxy that led everyone to try to capture all requirements in UML Use Case format with stick men and bubbles 15 years ago. There are many different ways to effectively express requirements. Sometimes requirements need to be specified in detail (when you have to meet regulatory compliance or comply with a standard or integrate with an existing system or implement a specific algorithm or…). Sometimes it’s better to work from a test case or a detailed use case scenario or a wire frame or some other kind of model, because somebody who knows what’s going on has already worked out the details for you. So pick the format and level of detail that works best and get to work. Relying on a Product Owner Relying on one person as the Product Owner, as the single solitary voice of the customer and the “one throat to choke” when the project fails, doesn't scale, doesn't last, and puts the team and the project and eventually the business at risk. It’s a naïve, dangerous approach to designing a product and to managing a development project, and it causes more problems than it solves. Many teams have realized this and are trying to work around the Product Owner idea because they have to. To succeed, a team needs real and sustained customer engagement at multiple levels, and they should take responsibility themselves for making sure that they get what they need, rather than relying on one person to do it all.

this is the first blog post in a series which looks at some data organization patterns in mapreduce. we’ll look at how to bucket output across multiple files in a single task, how to multiplex data across multiple files, and also how to coalesce data. these are all common patterns that are useful to have in your mapreduce toolkit. we’ll kick things off with a look at bucketing data outputs in your map or reduce tasks. by default when using a fileoutputformat-derived outputformat (such as textoutputformat), all the outputs for a reduce task (or a map task in a map-only job) are written to a single file in hdfs. imagine a situation where you have user activity logs being streamed into hdfs, and you want to write a mapreduce job to better organize the incoming data. as an example a large organization with multiple products may want to bucket the logs based on the product. to do this you’ll need the ability to write to multiple output files in a single task. let’s take a look at how we can make that happen. multipleoutputformat there are a few ways you can achieve your goal, and the first option we’ll look at is the multipleoutputformat class in hadoop. this is an abstract class that lets you do the following: define the output path for each and every key/value output record being emitted by a task. incorporate the input paths into the output directory for map-only jobs. redefine the key and value that are used to write to the underlying recordwriter . this is useful in situations where you want to remove data from the outputs as it duplicates data in the filename. for each output path, define the recordwriter that should be used to write the outputs. ok enough with the words - let’s look at some data and code. first up is the simple data we’ll use in our example - imagine you work at a fruit market with locations in multiple cities, and you have a purchase transaction stream which contains the store location along with the fruit that was purchased. cupertino apple sunnyvale banana cupertino pear to help bucket your data for future analysis, you want to bin each record into city-specific files. for the simple data set above you don’t want to filter, project or transform your data, just bucket it out, so a simple identity map-only job will do the job. to force more than one mapper, we’ll write the data to two separate files. $ tab="$(printf '\t')" $ hdfs -put - file1.txt << eof cupertino${tab}apple sunnyvale${tab}banana eof $ hdfs -put - file2.txt << eof cupertino${tab}pear eof here’s the code which will let you write city-specific output files. import org.apache.commons.lang.stringutils; import org.apache.hadoop.conf.configuration; import org.apache.hadoop.conf.configured; import org.apache.hadoop.fs.filesystem; import org.apache.hadoop.fs.path; import org.apache.hadoop.io.text; import org.apache.hadoop.mapred.*; import org.apache.hadoop.mapred.lib.identitymapper; import org.apache.hadoop.mapred.lib.multipletextoutputformat; import org.apache.hadoop.util.progressable; import org.apache.hadoop.util.tool; import org.apache.hadoop.util.toolrunner; import java.io.ioexception; import java.util.arrays; /** * an example of how to use {@link org.apache.hadoop.mapred.lib.multipleoutputformat}. */ public class mofexample extends configured implements tool { /** * create output files based on the output record's key name. */ static class keybasedmultipletextoutputformat extends multipletextoutputformat { @override protected string generatefilenameforkeyvalue(text key, text value, string name) { return key.tostring() + "/" + name; } } /** * the main job driver. */ public int run(final string[] args) throws exception { string csvinputs = stringutils.join(arrays.copyofrange(args, 0, args.length - 1), ","); path outputdir = new path(args[args.length - 1]); jobconf jobconf = new jobconf(super.getconf()); jobconf.setjarbyclass(mofexample.class); jobconf.setnumreducetasks(0); jobconf.setmapperclass(identitymapper.class); jobconf.setinputformat(keyvaluetextinputformat.class); jobconf.setoutputformat(keybasedmultipletextoutputformat.class); fileinputformat.setinputpaths(jobconf, csvinputs); fileoutputformat.setoutputpath(jobconf, outputdir); return jobclient.runjob(jobconf).issuccessful() ? 0 : 1; } /** * main entry point for the utility. * * @param args arguments * @throws exception when something goes wrong */ public static void main(final string[] args) throws exception { int res = toolrunner.run(new configuration(), new mofexample(), args); system.exit(res); } } run this code and you’ll see the following files in hdfs, where /output is the job output directory: $ hadoop fs -lsr /output /output/cupertino/part-00000 /output/cupertino/part-00001 /output/sunnyvale/part-00000 if you look at the output files you’ll see that the files contain the correct buckets. $ hadoop fs -lsr /output/cupertino/* cupertino apple cupertino pear $ hadoop fs -lsr /output/sunnyvale/* sunnyvale banana awesome, you have your data bucketed by store. now that we have everything working, let’s look at what we did to get there. we had to do two things to get this working: extend multipletextoutputformat this is where the magic happened - let’s look at that class again. static class keybasedmultipletextoutputformat extends multipletextoutputformat { @override protected string generatefilenameforkeyvalue(text key, text value, string name) { return key.tostring() + "/" + name; } } you are working with text, which is why you extended multipletextoutputformat , a class that in turn extends multipleoutputformat . multipletextoutputformat is a simple class which instructs the multipleoutputformat to use textoutputformat as the underlying output format for writing out the records. if you were to use multipleoutputformat as-is it behaves as if you were using the regular textoutputformat , which is to say that it’ll only write to a single output file. to write data to multiple files you had to extend it, as with the example above. the generatefilenameforkeyvalue method allows you to return the output path for an input record. the third argument, name , is the original fileoutputformat -created filename, which is in the form “part-nnnnn”, where “nnnnn” is the task index, to ensure uniqueness. to avoid file collisions, it’s a good idea to make sure your generated output paths are unique, and leveraging the original output file is certainly a good way of doing this. in our example we’re using the key as the directory name, and then writing to the original fileoutputformat filename within that directory. specify the outputformat the next step was easy - specify that this output format should be used for your job: jobconf.setoutputformat(keybasedmultipletextoutputformat.class); earlier we also mentioned that you can use the input path as part of the output path, which we will look at next. using the input filename as part of the output filename in map-only jobs what if we wanted to keep the input filename as part of the output filename? this only works for map-only jobs, and can be accomplished by overriding the getinputfilebasedoutputfilename method. let’s look at the following code to understand how this method fits into the overall sequence of actions that the multipleoutputformat class performs: public void write(k key, v value) throws ioexception { // get the file name based on the key string keybasedpath = generatefilenameforkeyvalue(key, value, myname); // get the file name based on the input file name string finalpath = getinputfilebasedoutputfilename(myjob, keybasedpath); // get the actual key k actualkey = generateactualkey(key, value); v actualvalue = generateactualvalue(key, value); recordwriter rw = this.recordwriters.get(finalpath); if (rw == null) { // if we don't have the record writer yet for the final path, create // one // and add it to the cache rw = getbaserecordwriter(myfs, myjob, finalpath, myprogressable); this.recordwriters.put(finalpath, rw); } rw.write(actualkey, actualvalue); }; the getinputfilebasedoutputfilename method is called with the output of generatefilenameforkeyvalue , which contains our already-customized output file. our new keybasedmultipletextoutputformat can now be updated to override getinputfilebasedoutputfilename and append the original input filename to the output filename: static class keybasedmultipletextoutputformat extends multipletextoutputformat { @override protected string generatefilenameforkeyvalue(object key, object value, string name) { return key.tostring() + "/" + name; } @override protected string getinputfilebasedoutputfilename(jobconf job, string name) { string infilename = new path(job.get("map.input.file")).getname(); return name + "-" + infilename; } if you run with your modified outputformat class you’ll see the following files in hdfs, confirming that the input filenames are now concatenated to the end of each output file. $ hadoop fs -lsr /output /output/cupertino/part-00000-file1.txt /output/cupertino/part-00001-file2.txt /output/sunnyvale/part-00000-file1.txt the implementation of getinputfilebasedoutputfilename in multipleoutputformat doesn’t do anything interesting by default, but if you set the value of the mapred.outputformat.numoftrailinglegs configurable to an integer greater than 0, then the getinputfilebasedoutputfilename will use part of the input path as the output path. let’s see what happens when we set the value to 1: jobconf.setint("mapred.outputformat.numoftrailinglegs", 1); the output files in hdfs now exactly mirror the input files used for the job: $ hadoop fs -lsr /output /output/file1.txt /output/file2.txt if we set mapred.outputformat.numoftrailinglegs to 2, and our input files exist in the /inputs directory, then our output directory looks like this: $ hadoop fs -lsr /output /output/input/file1.txt /output/input/file2.txt basically as you keep incrementing mapred.outputformat.numoftrailinglegs , then multipleoutputformat will continue to go up the parent directories of the input file and use them in the output path. modifying the output key and value it’s very possible that the actual key and value you want to emit are different from those that were used to determine the output file. in our example, we took the output key and wrote to a directory using the key name. if you do that keeping the key in the output file may be redundant. how would we modify the output record so that the key isn’t written? multipleoutputformat has your back with the generateactualkey method. class keybasedmultipletextoutputformat extends multipletextoutputformat { @override protected string generatefilenameforkeyvalue(text key, text value, string name) { return key.tostring() + "/" + name; } @override protected text generateactualkey(text key, text value) { return null; } } the returned value from this method replaces the key that’s supplied to the underlying recordwriter , so if you return null as in the above example, no key will be written to the file. $ hadoop fs -lsr /output/cupertino/* apple pear $ hadoop fs -lsr /output/sunnyvale/* banana you can achieve the same result for the output value by overriding the generateactualvalue method. changing the recordwriter in our final step we’ll look at how you can leverage multiple recordwriter classes for different output files. this is accomplished by overriding the getrecordwriter method. in the example below we’re leveraging the same textoutputformat for all the files, but it gives you a sense of what can be accomplished. static class keybasedmultipletextoutputformat extends multipletextoutputformat { @override protected string generatefilenameforkeyvalue(text key, text value, string name) { return key.tostring() + "/" + name; } @override public recordwriter getrecordwriter(filesystem fs, jobconf job, string name, progressable prog) throws ioexception { if (name.startswith("apple")) { return new textoutputformat().getrecordwriter(fs, job, name, prog); } else if (name.startswith("banana")) { return new textoutputformat().getrecordwriter(fs, job, name, prog); } return super.getrecordwriter(fs, job, name, prog); } } conclusion when using multipleoutputformat , give some thought to the number of distinct files that each reducer will create. it would be prudent to plan your bucketing so that you have a relatively small number of files. in this post we extended multipletextoutputformat , which is a simple extension of multipleoutputformat that supports text outputs. multiplesequencefileoutputformat also exists to support sequencefiles in a similar fashion. so what are the shortcomings with the multipleoutputformat class? if you have a job that uses both map and reduce phases, then multipleoutputformat can’t be used in the map-side to write outputs. of course, multipleoutputformat works fine in map-only jobs. all recordwriter classes must support exactly the same output record types. for example, you wouldn’t be able to support a recordwriter that emitted for one output file, and have another recordwriter that emitted . multipleoutputformat exists in the mapred package, so it won’t work with a job that requires use of the mapreduce package. all is not lost if you bump into either one of these issues, as you’ll discover in the next blog post.

How to get visibility into an important component of any complex system: the messaging queue Here at AppNeta, we get to see a lot about how people build their web applications. From simple PHP scripts to heavily service-oriented Java clouds to monolithic Django apps, everybody’s product is architected a little differently. We’re still out to trace everything, and today I want to talk how to get visibility into an important component of any complex system: the messaging queue. Specifically, let’s look at how to trace a job from Rails using Resque. Messaging Queues If you haven’t used a messaging queue in your app, the idea is simple. Instead of forcing all the work to happen during the request, while the user is waiting, you can delay some of the more time-consuming tasks. You can do anything in these tasks, ranging from a simple insert to kicking off a series of user analytics that touch all parts of your infrastructure. The advantage is that you can return a speedy response to the user, or, if they are actually waiting on the task results, give them a better loading interface than a white screen and browser loading bar. A Quick Resque Tutorial In Ruby, Resque is a task runner, which by default stores the task descriptions in Redis (though other options are available). Resque jobs are just Ruby classes, with a single mandatory method perform. Resque will call perform with the arguments given in the task description. Let’s look at a minimal task, that takes a single argument and prints it. (Useless, I know.) The @queue variable defines a name that a worker can bind to, in case you want to spread different types of jobs across different machines. To create a task that this worker could run, we just call it from our request: And that’s our job! Maybe not the most interesting job, and probably not prone to performance issues, but we don’t know that yet. So let’s measure it! Tracing a Resque Task Now that we’ve added this to our system, we should have monitoring around it. The easiest way to do this would be to just measure the time each task takes, and log that information: Unfortunately, the data presentation here leaves a bit to be desired, so I’m going to use TraceView to log this information instead. This also has the benefit of logging any SQL queries, cache accesses, or service calls that we might do in a more complex task, as well as reporting errors. To start a trace fresh, we can wrap this call in the start_trace block: That’s a start! We’ve now got some visibility into our Resque jobs, and we can rest easy knowing that this is running smoothly in production. Tracing a Resque Job (with multiple tasks!) For cron-style jobs, the approach of tracing each task individually works fine. For reference, let’s look at the events we’re generating with that code: Pretty straightforward. Now let’s consider a more complicated set of tasks: a document-processing pipeline. That code might look like this: In this case, our first task takes a document, and the second one archives it. If we have multiple tasks, each one gets logged separately, and we can figure out same statistics for each — average, std. dev., percentiles, and the like. But what if you have a job that spans multiple tasks? We can further aggregate the stats, but we might be starting to miss things, like large inputs that cause the entire pipeline to slow down. What we’d really like is to correlate the related tasks, so instead of timing the each task, we’re timing the entire job. Under the hood, TraceView generates a token for each request. If we pass this ID (generally stored in xtrace, after the X-Trace header it’s passed around in) to each task, we can correlate those timings before storing them, and retrieve them all together. To do this, we can modify each task to take this token, and trace using that ID. ProcessDoc then becomes: Now we need to start the trace somewhere, but we’re not doing it in the job. We could start it in the first task, or we could link this one step further up the chain and tie it back to the web request that started it in the first place. In a default rails stack, that request generates the following events: To add in the task queue call to the logged request, we can call the following function: We have to force a fork in the execution path to indicate that we’re running an asynchronous task, possibly in parallel, with the rest of the web request, which is done with the call to fromString. Aside from that, this is the same underlying call as is done by start_trace above — log that we’re entering a named block of code, and start timing it. When we put it all together, we get a secondary execution path attached to the web request, and the logged events look like this: Now we’ve got everything: the original request, all tasks, individual timing information, and a global view of how the process performed. Not that we now have an additional timing measurement here: the delay before starting the task at all. In this case, we waited a full 500ms between queuing the job actually executing it! Once we were in the pipeline, the tasks happened much faster (only 25ms between processing and archiving). Caveats Lest you think that everything was easy, there’s a couple things to keep in mind when you use this in your own application. Because we’re starting the timing in the web request and ending it in a task queue, we’re relying on those two processes to have an identical clock. If they’re on the same machine, it won’t be a problem, but on different machines, any clock skew will effect the timing. I’ve quietly assumed everything in this system is reliable, which is almost certainly wrong. Whatever your error handling is, make sure you always log the exit event for ‘job’, or you may never know that you have errors! As long as I haven’t totally dissuaded you from trying this out, all the code is available in one place in this gist, and you can try in out in your application today with our free version of TraceView! (Source) Related Articles Ruby 2.0 Released: Let The Tracing Begin! AppNeta Rubygems Verified Relieve Event Binding Aches in Backbone.js

A couple of weeks ago we looked at how to do a quick "health check" of an agile team. We saw that a great deal can be learned just by attending one of their daily stand-ups and by inspecting the state of their Scrum and Kanban boards. Of course these are nothing more than cursory examinations, and serious ailments can lie behind an apparently robust façade of agile practice. If you have reason to believe that a team is dysfunctional, you might have to dig deeper than the superficial evidence suggested by its apparent morphology. In my experience the next thing to examine is the team's "Definition of Done". This is the standard to which all work is put before it can be considered to be complete. Each team is collectively responsible for its own Definition of Done. It's up to them to make sure that it is adequate, and that it is applied by all members to all of the work they do. Without such professional oversight there can be no assurance that any deliverable will truly be fit for release. A spiffy stand-up and a cracker of a Kanban board might suggest rude team health, but they are no guarantee that the Definition of Done is solid, or that it isn't being undercut by someone along the way. Technical debt and rework are the main symptoms to look for. The consequences of backsliding on a Definition of Done might not become apparent until long after the events that caused it. By then, that rework or debt can be difficult to trace to the specific behaviors of those who cheated the system. You see, unfortunately a Definition of Done is a bit like personal hygiene. If there is no oversight, everyone can pretend that they are following the rules for the sake of the team, even though the presence of E. Coli on the office keyboards will tell its own tale about compliance. Everyone knows that it has to be coming from somewhere, but won't admit to their own liability or involvement, perhaps not even to themselves. Just as team vomiting will follow one member's dubious hand-washing practices, a short-changed Definition of Done will lead to rework by the team or the creation of technical debt. This is why team ownership and enforcement of what "Done" means is so important. An effective Definition of Done has to be founded on a healthy balance between due diligence and professional trust. What does this mean for agile development? You can think of a Definition of Done as the key defensive bulwark in software development epidemiology. If you balance the right level of team oversight with the right level of trust, severe outbreaks of technical debt or rework will become rare. High levels of oversight may be needed to start with, since team members might not have bought in to the idea of "done" yet. Once people become conditioned to do the right thing and see themselves as stakeholders in the team and its success, the balance can swing more towards trust. People become reluctant to renege on a team investment if they can see that it adds value for everyone including them. What's more, a Definition of Done improves the more it is respected, and becomes better respected the more it improves. In terms of agile best practice a Definition of Done will be used to determine whether or not User Story implementations are release-ready. However, each team can implement many User Stories over the course of a sprint, and making sure that all of these stories meet the Definition of Done can be challenging. Teams that are new to agile methods often have more modest ambitions. For example, their Definition of Done may only extend as far as delivery into a pre-production environment. Of course, anything less than "fully release ready" incurs the risk of technical debt and the need to pay it back later. Yet like a sloppy approach to hand-washing, it has to be admitted that something is better than nothing at all. Applying a Definition of Done consistently to even a sub-optimal standard will at least permit the delivery of each User Story to a known level of quality. It might not be great but at least it's there, and it's something that can be built upon and improved. The Lessons of Lean-Kanban Lean-Kanban methodologies have an instructive relationship with the Definition of Done. In these approaches the optimization of the value stream is of great significance. Naturally though, if a value stream is to be optimized it must first be understood. This means breaking the stream down into multiple discrete steps that can be studied for bottlenecks and any other occurrences of waste. For example, "Work in Progress" can be broken down into finer-grained stations such as "In Development", "Peer Review", "QA Test", "Knowledge Transfer", and "In Deployment". Team members will be cross-trained and will move freely across those stations in order to expedite as smooth a workflow as possible. Now here's where it gets interesting. If a Lean-Kanban operation has multiple well-defined stations, the case for having a Definition of Done can seem rather harder to make. After all, by the time a User Story gets to "Done", you already know that it has gone through the key steps you care about in the development process. What value can a Definition of Done really add in such a situation? Doesn't it just become waste itself? To find the answer, we need to look back to the manufacturing roots of Lean-Kanban. In a car plant for example, the steps of construction are exceptionally well defined and team members can move freely over several dozen stations. Some of those stations will be for the chassis, others for the interior, others for the engine block and electrics and so on. Yet despite this the Definition of Done will be an absolute corker, and much of the process of verification will be automated. Each station might even have its own Definition of Done so inspection can occur as close as possible to the point of action and potential remedy. The total number of checks that happen before each vehicle leaves the factory will be exhaustive. Why is this thought to be necessary? Because the manufacturers know perfectly well that the verification of "done" adds value. Merely having well-defined stations is no guarantee that everything will be done well. Quality is built in and validated by inspection. One thing's for sure: no-one in IT should accuse car manufacturers of having a weak understanding of what "done" means. The Definition of Done versus Acceptance Criteria However, software projects have a wild-card to deal with that car manufacturers don't have to worry about. Unlike the car doors and carburettors that roll down an assembly line, each User Story is different and has to be treated as a special case. To deal with this, each User Story has Acceptance Criteria that are agreed by the team members and the Product Owner as part of a Sprint Planning Session. Acceptance Criteria have to be quite specific to particular User Stories, because each story can be unique. The Definition of Done, on the other hand, applies to all of the User Stories being worked on by a team. The associated conditions must be invariant. For example, if all work has to be peer reviewed and subjected to QA testing prior to release, then those criteria would be enumerated in the Definition of Done rather than being repeated in each User Story's Acceptance Criteria. If the definition is enforced properly, a developer could never claim that a User Story was “Done” if it hadn't both been reviewed and passed QA testing. Writing a Definition of Done The Scrum Guide describes a Definition of Done as a "shared understanding of what it means for work to be complete". No process is suggested for writing a Definition of Done, nor in fact is there any suggestion that one should be written down at all. However, a documented definition may go some way towards providing that shared understanding. Here's how you can set about eliciting one: Review Acceptance Criteria: Gather the Acceptance Criteria for work completed so far Look for common criteria that can be abstracted out and applied across work in general Use these common criteria as the basis for a Definition of Done Assess Technical Debt Identify any rework that needs to be done Identify the reasons why it wasn't done properly the first time Identify what measures can be put in place to stop similar rework from occurring Add these measures to the Definition of Done (DoD) Continually update the DoD In each Sprint Review, identify which (if any) work was rejected or which caused rework to be done, then In each Sprint Retrospective, challenge the DoD for relevance and completeness There isn't a prescribed format for a Definition of Done, but it can be beneficial to use the same as that which is used for Acceptance Criteria, either in whole or in part. This allows a flexible definition based on story type. Alternatively they can be written as simple lists. Here are some examples: Example of a Definition of Done in Acceptance Criteria Format Given that a user story has required a code change When BDD and unit tests have been written for the story and the code change has been reviewed and the code change has been approved by a peer and all BDD and unit tests have been run and no BDD or unit tests have broken (green bar) and the code change has been committed to the repository and QA testing has passed satisfactorily and the Product Owner has approved the change Then the user story will be deployed to production and it will be considered Done. Given that a user story has required the authoring of documentation When the documentation has been reviewed and approved by a peer and the documentation has been approved by the Product Owner Then a new version of the documentation will be committed and the user story will be considered Done. Example of a Definition of Done in List Format Code changes: BDD tests written and pass Unit tests written and pass Code peer reviewed & approved Code committed to repository QA testing done Product Owner signed off Documentation: Documentation has been peer reviewed & approved Documentation approved by Product Owner Version number updated Definitions of Done for IT Infrastructure Support We've seen that having a good Definition of Done is important, although in IT we also need Acceptance Criteria that address the particulars of each User Story. When used in combination they can approach the levels of rigor that have been shown to be possible in manufacturing. Those working in software development can adopt a similar commitment to quality. Now we need to turn our attention to another function within the IT department...Infrastructure Support. Infrastructure support teams are increasingly expected to work in an agile way. As part of an enterprise transformation that does not seem unreasonable. After all, the rest of the organization is highly dependent upon them. Their scope includes such things as deploying new workstations and laptops (possibly across entire sites), installing networks, performing miscellaneous diagnostics and repairs, and maintaining and upgrading local server resources. Clearly they will also need Definitions of Done and Acceptance Criteria if they are to be stakeholders in a joined-up agile enterprise. The question is, how on earth can a meaningful Definition of Done be abstracted across wildly different physical tasks? How can a Definition of Done cover everything from a phone installation to a printer driver upgrade or a memory enhancement, or a firewall configuration to a keyboard replacement? The answer is to focus on the value chain that is represented by each user story. Work is not "released" as such, but rather it is handed over to someone who can derive benefit from it (i.e. the person occupying the user story role). This is the key to understanding "done" in an infrastructure context. If you can identify the parties who derive value, and demonstrably pass that value on to them, then your work is done. Here's an example of a Definition of Done that might be used to close out a support ticket: The receiver of the work has been identified Handover instructions have been completed and given to the receiver The receiver has been notified of the intention to close the ticket, and has not raised an objection A security assessment has been conducted and approved There are three things to point out here. The absence of any reference to a Product Owner. This is because infrastructure teams have to support multiple products, and prioritization of work is traditionally handled not through any sense of ownership of those products, but rather through help-desk triage. It's certainly possible for work to be represented by Product Owners, but it would have to be ownership of the business support function rather than ownership of the actual products being supported. The need to identify and work with the actual receivers of value is still there. The "acceptance by default" position. Receivers typically have little incentive to sign work off as being complete. On the contrary, their incentive is to defer acceptance as long as possible, for potential use as a "banker" in case a requirement for additional unforeseen work transpires. They might hope to ride this new work on an existing ticket instead of having to raise a new one. Receivers can be expected to care about their own support needs, not about the big picture of enterprise delivery. If a Product Owner can be identified - even if it is just the most likely owner of the business support function - then this situation can be improved. A Product Owner can apply leverage for appropriate and timely sign-off, such as by not accepting new work from certain parties while their approval (or justified rejection) of prior work remains outstanding. The elicitation of solid Acceptance Criteria can help the Product Owner immensely. Security implications need to be given careful consideration. The reworking of organizational infrastructure offers great potential for security to be compromised. Approval from Information Security should be obtained for all work, either directly or through authorized agents. One approach is for each team to have a designated "security champion" who provides this function. Conclusion Teams that appear to be healthy can still incur rework and technical debt. A poor understanding of what "done" means often underlies such problems, and this should be one of the first things to be looked at if problems are suspected. Having a meaningful Definition of Done encourages a team's sense of ownership of their own process, and helps instil self-discipline into its members to follow agile best practices. The application of this standard requires finding the right balance between team oversight and trust.

I recently stumbled upon an article Synchronising Multithreaded Integration Tests on Captain Debug's Blog. That post emphasizes the problem of designing integration tests involving class under test running business logic asynchronously. This contrived example was given (I stripped some comments): public class ThreadWrapper { public void doWork() { Thread thread = new Thread() { @Override public void run() { System.out.println("Start of the thread"); addDataToDB(); System.out.println("End of the thread method"); } private void addDataToDB() { // Dummy Code... try { Thread.sleep(4000); } catch (InterruptedException e) { e.printStackTrace(); } } }; thread.start(); System.out.println("Off and running..."); } } This is only an example of common pattern where business logic is delegated to some asynchronous job pool we have no control over. Roger Hughes (the author) enumerates few techniques of testing such code, including: arbitrary ("long enough") sleep() in test method to make sure background logic finishes refactoring doWork() so that it accepts CountDownLatch and agrees to notify it when job is done making the method above package private and @VisibleForTesting only "The" solution - refactoring doWork() so that it accepts arbitrary Runnable. In test we can wrap this Runnable (decorator pattern) and wait for inner Runnable to complete Last solution is not bad but it changes the responsibilities of ThreadWrapper significantly. Now it's up to the caller to decide what kind of job should be executed asynchronously while previously ThreadWrapper was encapsulating business logic completely. I am not saying it's a bad design, but it's drastically different from original method. Awaitility Can we write a test without such a massive refactoring? First solution involves handy library called Awaitility. This library is not a silver bullet, it simply evaluates given condition periodically and makes sure it's fulfilled within given time. It's the kind of code you probably wrote once or twice - wrapped in a library with well designed API. So here is our initial approach: import static com.jayway.awaitility.Awaitility.await; import static java.util.concurrent.TimeUnit.SECONDS; //... await().atMost(10, SECONDS).until(recordInserted()); //... private Callable recordInserted() { return new Callable() { @Override public Boolean call() throws Exception { return dataExists(); } }; } I think there is nothing to explain here. dataExists() is simply a boolean method that initially returns false but will eventually return true once the background task (addDataToDB()) is done. In other words we assume that background task introduces some side effect and dataExists() can detect that side effect. BTW I happened to have JDK 8 with Lambda support installed and IntelliJ IDEA gives me this nice tooltip: Suddenly I get this Java 8-compatible alternative suggested: private Callable recordInserted() { return () -> dataExists(); } But there's more: Which transforms my code to: private Callable recordInserted() { return this::dataExists; } this:: prefix means that recordInsterted is a method of current object. Just as well we can say someDao::dataExists. Simply put this syntax turns method into a function object we can pass around (this process is called eta expansion in Scala). By now recordInsterted() method is no longer that needed so I can inline it and remove it completely: await().atMost(10, SECONDS).until(this::dataExists); I am not sure what I love more - the new lambda syntax or how IntelliJ IDEA takes pre-Java 8 code and retrofits it for me automatically (well, it's still a bit experimental, just reported IDEA-106670). I can run this intention in IntelliJ project-wide, Lambda-enabling my whole code base in seconds. Sweet! But back to original problem. Awaitility helps a lot by providing decent API and some handy features. I use it extensively in combination with FluentLenium. But periodically polling for state changes feels a bit like a workaround and still introduces minimal latency. But notice that running and synchronizing on asynchronous tasks is quite common and JDK already provides necessary facilities: Future abstraction! java.util.concurrent.Future To limit the scope of refactoring I will leave the original new Thread() approach for now and use SettableFuture from Guava. It is a Future implementation that allows triggering completion or failure at any time, from any thread (see DeferredResult - asynchronous processing in Spring MVC for more advanced usage). As you can see the changes are quite small: public class ThreadWrapper { public ListenableFuture doWork() { final SettableFuture future = SettableFuture.create(); Thread thread = new Thread() { @Override public void run() { addDataToDB() //... //last instruction future.set(null); } private void addDataToDB() { // Dummy Code... // ... } }; thread.start(); return future; } } doWork() now returns ListenableFuture with lifecycle controlled inside asynchronous task. We use Void but in reality you might want to return some asynchronous result instead. future.set(null) invocation in the end is crucial. It signals that future is fulfilled and all threads waiting for that future will be notified. Once again, in practice you would use e.g. Future and then instead of null we would say future.set(someInteger). Here null is just a placeholder for Void type. How does this help us? Test code can now rely on future completion: final ListenableFuture future = wrapper.doWork(); future.get(10, SECONDS); future.get() blocks until future is done (with timeout), i.e. until we call future.set(...). BTW I use ListenableFuture from Guava but Java 8 introduces equivalent and standard CompletableFuture - I will write about it soon. So, we are getting somewhere. Future is a useful abstraction for waiting and signalling completion of background jobs. But there is also one immense advantage of Future which are not taking, ekhm, advantage from - exception handling and propagation. Future.get() will block until future is complete and return asynchronous result or throw an exception initially thrown from our job. This is really useful for asynchronous tests. Currently if Thread.run() throws an exception it may or may not be logged or visible to us and future will never be completed. With Awaitility it's slightly better - it will timeout without any meaningful reason, which have to be tracked down manually in console/logs. But with minor modification our test is much more verbose: public void run() { try { addDataToDB() //... future.set(null); } catch (Exception e) { future.setException(e); } } If some exception occurs in asynchronous job, it will pop-up and be shown as JUnit/TestNG failure reason. (Listening)ExecutorService That's it. If addDataToDB() throws an exception it will not be lost. Instead our future.get() in test will re-throw that exception for us. Our test won't simply timeout leaving us with no clue what went wrong. Great, but do we really have to create this special SettableFuture instance, can't we just use existing libraries that already give us Future with correct underlying implementation? Of course! By this requires further refactoring: import com.google.common.util.concurrent.ListeningExecutorService; import com.google.common.util.concurrent.MoreExecutors; import java.util.concurrent.Executors; import java.util.concurrent.Future; public class ThreadWrapper { private final ListeningExecutorService executorService = MoreExecutors.listeningDecorator( Executors.newSingleThreadExecutor() ); public ListenableFuture doWork() { Runnable job = new Runnable() { @Override public void run() { //... } }; return executorService.submit(job); } } This is what you've all been waiting for. Don't start new Thread all the time, use thread pool! I actually went one step further by using ListeningExecutorService - an extension to ExecutorService that returns ListenableFuture instances (see why you want that). But the solution doesn't require this, I just spread good practices. As you can see Future instance is now created and managed for us. The test is exactly the same but production code is cleaner and more robust. MoreExecutors.sameThreadExecutor() The final trick I want to show you involves dependency injection. First let's externalize the creation of a thread pool from ThreadWrapper class: private final ListeningExecutorService executorService; public ThreadWrapper() { this(Executors.newSingleThreadExecutor()); } public ThreadWrapper(ExecutorService executorService) { this.executorService = MoreExecutors.listeningDecorator(executorService); } We can now optionally supply custom ExecutorService. This is good for various other reasons, but for us it opens brand new testing opportunity: MoreExecutors.sameThreadExecutor(). This time we modify our test slightly: final ThreadWrapper wrapper = new ThreadWrapper(MoreExecutors.sameThreadExecutor()); wrapper.doWork().get(); See how we pass custom ExecutorService? It's a very special implementation that doesn't really maintain thread pool of any kind. Every time you submit() some task to that "pool" it will be executed in the same thread in a blocking manner. This means that we no longer have asynchronous test, even though the production code wasn't changed that much! wrapper.doWork() will block until "background" job finishes. The extra call to get() is still needed to make sure exceptions are propagated, but is guaranteed to never block (because the job is already done). Using the same thread to execute asynchronous task instead of a thread pool might have an unexpected results if you somehow depend on thread-based properties, e.g. transactions, security, ThreadLocal. However if you use standard ThreadPoolExecutor with CallerRunsPolicy, JDK already behaves this way if thread pool is overflowed. So it's not that unusual. Summary Testing asynchronous code is hard, but you have options. Several options. But one conclusion that strikes me is the side effect of our efforts. We refactored original code in order to make it testable. But the final production code is not only testable, but also much better structured and robust. Surprisingly it's even source-code compatible with previous version as we barely changed return type from void to Future. It seems to be a rule - testable code is often better designed and implemented. Unit test is the first client code using our library. It naturally forces us to to think more about consumers, not the implementation.

If you think that in year 2012 all companies which produce software and IT divisions in our world have already their optimized software development process, you are wrong. It seems that we - software architects, software developers or whatever your title is - still need to optimize the software development process in many software companies and IT divisions. So what do you do if you enter a software company or IT division and you see following things: 1. There is a perfect project management process to handle all those development of software but it is a pure project management without a context to software development. So basically you only take care of cost, time, budget and quality factors. In the software development you still use the old fashioned waterfall process. 2. From the tooling point of view: you have a project management planning and controlling tool but you are still in the beginning of Wiki (almost no collaboration tool) and you don't use issues tracking system to handle all the issues for the development of your software components and applications. You use Winword and Excel to define your requirements and you cannot transform them to your software products since you don't have any isssues tracking system. No chance to have traceability from your requirements down to your issues to be done in your software components and applications. 3. Maven is already used but with a lot customization and not intuitively used. The idea of using a concrete already released version of dependencies was not implemented. Instead you always open all the dependently projects in Eclipse. You can imagine how slow Eclipse works since you need to open a lot of projects at once although you only work for one project. Versioning in Maven is also not used correctly e.g.: no SNAPSHOT for development versions. 4. As you work with webapp you always need to redeploy to the application server. No possibility to hot deploy the webapp. Use ctrl-s, see your changes and continue to work without new deployment is just a dream and not available. Luckily as an experienced software architect and developer we know that we can optimize the two main software development processes: 1. Software Development Macro Process (SDMaP): this is the overall software development lifecycle. In this process model we define our requirements, we execute analysis, design, implementation, test and we deploy the software into production. Waterfall process model and agile process model like RUP and Scrum are examples of SDMaP. 2. Software Development Micro Process (SDMiP): this is the daily work of a software developer. How a software developer works to develop the software. A software developer codes, refactors, compiles, tests, runs, debugs, packages and deploys the software. More information on SDMaP and SDMiP: You can find the definition of SDMaP and SDMiP in the context of analysis and design in the book Object-Oriented Analysis and Design with Applications from Grady Booch, et. al. Unifying Microprocess and Macroprocess Research Effects of Architecture and Technical Development Process on Micro-Process The picture below shows the SDMaP and SDMiP in combination. The macro (SDMaP) and micro (SDMiP) process meet at the implementation phase and activity. So changing and optimizing one has definitely side effects on the other one and vice versa. At the example of organization mentioned above it is important that we optimize both processes since they work hand in hand. So how can the optimization for macro and micro process looks like? 1. SDMaP: Introduce Wiki for IT divisions and software companies. You can use WikIT42 to make the structure of your Wiki and use Confluence as your Wiki platform. Introduce Wiki with issue tracking like JIRA and combine both of them to track your requirements. Refine the requirements into issues (features, tasks, bugs, etc.) to the level of the software components and applications, because at the end you will implement all the requirements using your software components and applications. Introduce iterative software development lifecycle instead of waterfall process. This is a long way to go since you need to change the culture of the company and you need a full support from your management. 2. SDMiP Update the Maven projects to use the standard Maven mechanism and best practices with no exception. Transform the structure of the old Maven to the new standard Maven using frameworks like MoveToMaven. Use Maven release plugin to standardize the release mechanism of all Maven projects. Use m2e Eclipse plugin to optimize your daily work as a software developer under Eclipse and Maven. Use Mylyn to integrate your issue tracking system like JIRA into your Eclipse IDE. Introduce JRebel to be able to hot deploy quickly your webapps into the application server. Optimizing macro and micro process for software development is not an easy task. In the macro process you need to handle all those relationships with other divisions like Business Requirements, Quality Assurance and Project Management divisions. You need to convince them that your SDMaP optimization is the best way to go. This is more an organizational challenge and changes than the micro process optimization. The micro process is also not easy to optimize, since you need to convince all developers that they can be more productive with the new way of working than before. You need to show them that it is a lot more faster if you don't open a lot of Java projects within your Eclipse workspace. Also using JRebel to deploy your webapp to your application server is the best way to go. Normally developers are technical oriented, so if you can show them the cool things to make, they will join your way.

The longer I spend working as an agile coach, the more I find myself in disagreement with Hamlet. To estimate, or not to estimate? That is the question. Out of all of the agile practices which have been adopted in recent years, few have proven more controversial than this one. The battle for and against rages like Shakespearean armies set against each other's teeth. (Free Estimation Ebook) At first blush there doesn't seem to be any reasonable cause for disagreement. The rationale for making estimates is ostensibly straightforward. If a team is to work in Sprints, and to deliver something at the end of each one, then the work must surely be estimated. Otherwise how can the team know if it is even possible to do the work within the Sprint? How can they commit to deliver something by the end of that time-box if the effort involved is of uncertain magnitude? Well, there are two things that we need to draw out at this point. Firstly, the above rationale assumes that Sprints will be used, and that delivery will therefore be time-boxed. That's a very Scrum oriented philosophy...but Scrum isn't the only agile way of working. Lean-Kanban teams, for example, don't use Sprints and rarely make use of estimates. Secondly, Scrum itself says nothing about estimation. It only says that each item in a backlog must be sized - how that sizing happens is up to the team. It should also be remembered that a Scrum team commits to a Sprint Goal that delivers value, not to the delivery of a certain number of estimated points. So then...to estimate, or not to estimate? Let's listen in at the camp-fires of each side, and pick out in more detail the arguments they make for and against. For (Ye Scrum Brigade of Sprinte and Stande-uppe) "Estimates allow us to predict when a Sprint Goal will be met, and therefore when a substantial increment of value will be delivered" "Our estimates help our stakeholders plan ahead. They are part of the value we provide" "Estimates help us to de-risk scope of uncertain size and complexity" "Estimated work can be traded in and out of scope for other work of similar size. Without estimates you can't trade" "The very process of estimation adds value. When we estimate we discuss requirements in more detail, and gain a better understanding of what is needed" Against (Ye Lean Kanban Brigade of Boarde Pullers) "Estimates are rarely accurate. All you are doing when you estimate a piece of work is to set false expectations" "In practice, estimation is seen as a commitment, not as a best guess. Every time you make an estimate, you make a rod for your own back" "Estimation is time consuming. The time a team spends playing planning poker or whatever is time that could have been spent on delivery. Estimation is waste." "It's the actuals that matter, not estimates. Agility requires metrics, and the only metrics that count are those that reflect actual delivery" Both sides are right If you see this debate in terms of whether a Scrum or Lean-Kanban process is being followed, then both sides are right. A Scrum process is optimized for project work where scope risk is high and an entire system is represented. The requirements tend to be uncertain, complex, and very heavily intertwined. By committing to a Sprint Goal and to the delivery of a substantial increment of value, that risk can be managed. Uncertain and interdependent requirements are batched together into a Sprint and dealt with as a group. When this is done well, you have a clear Sprint Goal and a coherent Sprint Backlog. When it is done badly, you have a vague or disjointed Sprint Goal, a mishmash of requirements that command no sense of team purpose, and no team commitment towards the delivery of an increment. A Lean-Kanban process, on the other hand, is usually focused on "Business As Usual" (BAU) activities. The diet of a Lean-Kanban operation should consist of small and repeatable changes. They don't have to be related at all...in fact they shouldn't be. Things like bug fixes, minor enhancements, and administrative tasks are representative of this kind of work. Scope risk is low because the process of making such changes is well understood. Estimates are generally held to be unnecessary because there is very little uncertainty to deal with. There is no need for work to be batched...each change can be actioned and delivered independently of all others. Work is enqueued and actioned according to priority and the required quality of service. Predictions are based on the actual rate of delivery, not on estimates. In a Lean-Kanban way of working, the actuals are indeed everything. Methods of estimation So then, estimates add value where scope is uncertain and there are associated risks to be managed. That's why Scrum teams engaged on projects typically make use of them, but Lean-Kanban BAU teams generally don't. Now let's look at three simple methods of estimation that Scrum teams, or other teams doing project work, can make use of. Planning Poker This is a well established technique popularised by Mike Cohn, and variations on his Planning Poker cards can be found in offices across the world. A typical Planning Poker set has cards with the following numbers: ½ 1 2 3 5 8 13 20 40 100. Nerds will observe and be irritated by the fact that this is roughly (but not quite) in line with the Fibonacci sequence. Here's how it's played: An identical hand of cards is given to each team member. Each team member will have a set of cards with numbers on the above pseudo-Fibonacci scale. The Product Owner describes the piece of work to be estimated. Normally this is a user story with acceptance criteria. Each team member mentally estimates the size of it on the scale. They can ask the Product Owner questions to clarify any points, but for the moment they will keep their estimate to themselves. Each team member places the card that corresponds to their mental estimate face down in front of them. At the facilitators instruction - usually the Scrum Master - the team turn their cards over. In an ideal case the cards will all have the same value, suggesting that the team have a common understanding of the requirements and the likely effort that will be involved. If the values are different, the team then need to discuss their estimates and their reasoning behind them. They need to understand each other’s thinking, and from that reach a consensus. It may be necessary to replay the cards several times before agreement is reached. By convention, estimates are written on the corner of a User Story card before being placed on a Scrum board. A variation of this takes from a suite of regular playing cards. The Ace (1), 2, 3, 5, 8, Jack, Queen, and King might be used. The Jack signifies that no or negligible work needs doing (jack all). The Queen indicates a larger story that should be broken down in the planning session and reconsidered, while the King indicates an epic that will need greater analysis and cannot be brought into scope for this Sprint. The Joker can be played if anyone wants a coffee break. As an estimation method, Planning Poker has the advantage of being fairly democratic. Every team member gets a hand of cards and is allowed to play, and has a clear opportunity to explain their reasoning to the others. The disadvantage of Planning Poker is that it can be rather time consuming in comparison with other methods. It can also encourage novice teams to estimate in terms of time, as they are often initially prejudiced to correlate points to hours or days. This prejudice must be challenged and eroded if the relative sizing of estimates is to be achieved. Team Sort (T-Shirt Sizing) This is a good way of doing team estimates if no planning poker cards are available. All you need are six scraps of paper and a set of index cards with the requirements (e.g. user stories) written on them. Normally these will be the same index cards that go on the Scrum board. Write one of the following sizes on each of the scraps of paper: Extra Small (XS), Small (S), Medium (M), Large (L), Extra Large (XL), and Extra Extra Large (XXL) Arrange the sizes in a horizontal line on a table, ordered from XS on the left to XXL on the right. Put the pile of index cards on the table in front of the sizing line. The team then collaborate to organise the requirements on the cards under the headings XS to XXL. They can ask the Product Owner to clarify any questions that they may have while doing so. Once the cards have all been sorted, story points can be allocated to each of them by mapping each T-Shirt size to a value. This allows metrics to be gathered about the flow of work, and used to populate a velocity or burndown chart. T Shirt Size Suggested Story Point Value XS 1 S 2 M 3 L 5 XL 8 XXL 13 An advantage of the team sort is that it is quick and easy to do. The complete set of requirements is estimated in one sweep. Also, it is a fairly direct way of achieving relative sizing. There is no temptation to correlate points to hours. The disadvantage is that it is potentially undemocratic, in that assertive team members can dominate meeker ones with their opinions. There is a variant of the team sort which encourages more egalitarian behavior. Each team member takes it in turns to move one card by one position. They also have the option to pass, i.e. to not move a card. Eventually a consensus should be reached and no more cards will be moved. However this is a more time consuming method and deadlocks can occur. These deadlocks can be difficult to spot if multiple card shifts are caught in the cycle. One Point One Card This method is a spin on the Lean-Kanban approach of tracking actuals. Instead of estimating the relative effort required for each story card, the team estimates how many stories it is likely to complete in the Sprint being planned. This can be as straightforward as using the yesterday's weather analogy for velocity estimation. Just as the weather today is most likely to resemble the weather yesterday, the velocity that will be achieved by a team in the upcoming Sprint will most likely match the velocity of its predecessor. So if two dozen cards were completed in the last sprint, approximately two dozen can be expected in the one that follows. The budget can be adjusted to allow for holiday, foreseeable absences, and other such changes that will impact the team's commitment. The advantage of this system is its raw simplicity. The estimation overhead is almost negligible. Also, it encourages the authoring of small user stories that will spend little time in progress and that stand little chance of being impeded. The liquidity of the board is therefore increased and further requirements analysis is encouraged. Some variation in size will be inevitable, and there will be statistical outliers, but the effects of these will average out as the flow rate increases. The disadvantage of this technique lies in the separation of fine-grained user stories from business value. There is a significant risk that they will become excessively technically focused and task-like. Conclusion Agile estimation is often seen as being invaluable, yet others dismiss it as waste. The reasons for this disagreement can be traced to disparities in Scrum and Lean-Kanban ways of working, and to the fundamental differences between project work and Business As Usual. When seen in the context of Scrum projects, some form of estimation process is valuable. Yet regardless of the method chosen, it must be acknowledged that a Scrum Team is responsible for its own estimates. No-one else can make a team's estimates for them. Going through that process of estimation, and understanding the size and scope of the work, is fundamental to the team's sense of Sprint Backlog ownership and to their commitment to a Sprint Goal.

Richard Hundhausen has put together a great list of funny Scrum/Agile related videos. Some of these are classics such as High Moon Studios: Portrait of Scrum and Adam Weisbart’s Shit Bad Scrum Masters say. Be warned, not all of these are actually that funny. I’ve never found The Downfall of Agile Hitler to be funny, because the original film is harrowing and difficult to watch. I also find the Scrum Haka to be trite and unwatchable … this is what a real haka should look like. So here’s the list and, once you’re done (in the words of Adam Weisbart), “Get back to work!” I want to run an agile project http://www.youtube.com/watch?v=4u5N00ApR_k I want to run an agile project (part 2) http://www.youtube.com/watch?v=lAf3q13uUpE The Power of Scrum (Ian Sense Scrum Master) http://www.youtube.com/watch?v=P6v-I9VvTq4The making-of http://www.youtube.com/watch?v=ncjdtqf1gSg Developer Abuse http://www.youtube.com/watch?v=LYlhCGng5Mk Spooning and pair programming http://www.youtube.com/watch?v=dYBjVTMUQY0 Improving Sprint Reviews (is that Jeroen?) http://www.youtube.com/watch?v=fpBQ5yxrR_c The Downfall of Agile Hitler http://www.youtube.com/watch?v=l1wKO3rID9g High moon studios: Portrait of Scrum http://www.youtube.com/watch?v=UT4giM9mxHk Shit Bad Scrum Masters Say http://www.youtube.com/watch?v=GGbsgs611MM The Scrum Haka (hideous) http://www.youtube.com/watch?v=Qvqq97unS2w Joe Justice Team WikiSpeed http://www.youtube.com/watch?v=x8jdx-lf2Dw Deathstar Project Deployment Meeting http://www.youtube.com/watch?v=2T5QNcb_Z8g Raking Leaves – A Scrum/Agile Approach http://www.youtube.com/watch?v=StBS-loIIz4 I Need Agile Methodology http://www.youtube.com/watch?v=nvks70PD0Rs

I have been thinking about institutions that strive for change. Sometimes we call them communities or organizations, sometimes we call them alliances or parties. But whatever their nature, these institutions are usually led and managed by a small group of people. I see two kinds of leading groups: coalitions and councils. coalition A temporary alliance of distinct parties, persons, or states for joint action council A group elected or appointed as an advisory or legislative body Coalitions A coalition is a self-selecting team. The persons seek each other out because they want to be active agents for change, and by working together they can be more successful in achieving a common goal. In his change management books John Kotter referred to them as guiding coalitions. They are not elected. They are not appointed. They select each other because they want to. And they can even work undercover, because their goal is to influence, not to govern. The allied powers in World War II were a coalition. The Google founders were a coalition. The originators of the Stoos Network were a coalition. Councils A council is a group of representatives. These people also want to be active agents for change. But, their primary concern is to have buy-in from the larger group of people they are representing within the institute (community, organization, or party). The concept of democracy has led to many different versions of these councils. Sometimes we call them a government. Sometimes a committee. And everything has to be out in the open, because if it’s not, we call them cronies. Their goal is primarily to govern or advise the institute. The United Nations has a council. My former students society had a council. And many workplaces have management teams acting as councils. And you? If you have a group of people who all desire change, do you lead with a coalition or with a council? This is the big problem with some alliances and consortiums for change. They have directors who try to be both. It is a recipe for disaster. Maybe the best institutions have both: a coalition and a council. (image from Veni Markovski)

Application Services Governance is a necessary step towards building a responsive IT organization and achieving business agility. By guiding teams through a streamlined application services development process, Application Services Governance Platforms optimize IT effectiveness, raise software quality, and reduce delivery timeframes. Governance relies on policy, people, process and technology to guide business activity and consistently deliver positive outcomes. Effective governance channels business activity towards the ‘right’ path; by making the right actions the path of least resistance. To efficiently guide teams and demonstrate policy compliance benefits, Application Services Governance Platforms provide policy management, developer portals, repositories, service integration and composition, and business value dashboards. Effective governance encompasses the entire IT solution spanning APIs, services, business processes, data, and application delivery. While most governance solutions focus on web services, leading Application Services Governance Platforms bridge API governance, SOA governance, Cloud deployment governance, data governance, and application delivery governance. Additionally, the governance experience must be tailored for the participant’s project role. Portals may be personalized to present notifications, tasks, actions, and reports suitable for application service creators, publishers, subscribers, consumers, or business managers. Application delivery governance segments participants into developers, quality assurance testers, operations, project managers, and application users. End-user Application Services Governance priorities are evolving toward bridging service governance with API governance, extending application lifecycle management to embrace cloud deployment environments, and focusing on visualizing asset business value. Key governance challenges include meeting mobile application demands, implementing efficient self-service provisioning, right-sizing governance practices (not too heavy or light), and defining appropriate policy tiers. Governance Components To efficiently guide teams and demonstrate policy compliance benefits, Application Services Governance Platforms provide policy management, developer portals, repositories, service integration and composition, and business value dashboards. Figure 1 Application Services Governance Components Policy Management Policy management is used to specify the correct behavior, detail exception thresholds, and define corrective actions or notifications. Leading application services governance platforms deliver advanced policy management by conforming to a flexible architecture, addressing relevant policy categories, and spanning all lifecycle phases. A comprehensive Application Services Governance Platform manages: Design-time Policy Run-time Policy Security Policy Developer access Policy Service and API Lifecycle Management Policy Application Lifecycle Management Policy Within these six broad categories, application services governance commonly encompasses service level policies, usage policies, version policies, subscription policies, and access control policies. Registries serve as policy stores for many types of runtime policies including security policies, lifecycle management workflow policies, API policies, service description, service contracts, service consumption, service usage, service lifecycle management, service level agreements (SLAs) and XACML authorization policies. Leading platforms have built-in support for a number of policy standards including WS-Policy, XACML 3.0, and SCXML. Cloud foundation and cloud middleware components deliver sophisticated run-time policy enforcement for tenant partitioning, service level management, application provisioning, tenant access, and resource management. All run-time infrastructure products should serve as well-integrated policy enforcement points that may delegate policy decisions to external decision points or internally cache and process policy assertions. Identity Management infrastructure components serve as a policy decision point and a policy manager for sophisticated security policies encoded in XACML. The Application Service Governance Platforms use workflow engines to execute governance workflow, present task lists, and manage approvals. Complex Event Processor components can be configured as policy decision points, which use time-based policy pattern matching to evaluate run-time service, message, REST resource, and event traffic. For more information on policy management, read the detailed policy management blog post. Developer Portal and Repository Portals serve as the viewport into policy management, service integration and composition, and business value dashboards. The Application Service Governance portals should deliver an application service governance experience tuned for self-service, on-demand access, and safe API usage. Developer portals are often contextually personalized to fit the project and user’s role. For example, a developer portal may fit the needs of API creators and API publishers who are defining, documenting, and publishing APIs. The portal’s user experience may enable API creators and publishers to monitor, manage, and analyze API usage. A developer portal may also be personalized to deliver a user experience tailored for API consumers. API developers who are consuming APIs can find, explore, subscribe and evaluate APIs. Developer portals are often tuned to facilitate service meta-data and lifecycle management for service creators. Service and integration developers who are consuming services can find and explore services. A developer portal should guide teams toward effective and efficient governance when building service implementation and service consumption code. Advanced developer portals capabilities include overlaying build management governance, test governance (i.e. unit, integration, performance), implementation lifecycle governance, and deployment governance. An Application Services Governance Platform should enable flexible organization, classification & documentation of services, APIs, and any IT asset. Key repository capabilities include governing and managing: Any type of metadata in any structure Service, API, or artifact associations and relationships Schema definitions and namespaces Users and Roles User subscriptions Service level agreements Developer documentation Social taxonomies (e.g. ratings, comments, tags) Implementation artifacts (i.e. code, test cases) Service Integration and Composition Service integration and composition for APIs, web services, or business process are often implemented using tools provided by the run-time infrastructure vendor. Application Services Governance components must integrate into diverse run-time infrastructure containers and development tooling. Synchronizing policy, development artifacts, and deployment packages requires tight integration between design-time tools, development tools, run-time management consoles, and application services governance portals and repositories. Business Value Dashboards To gauge governance effectiveness and enhanced business value, analytic dashboards assess policy compliance, quality of service, service usage, architecture coherence, and team performance. The Application Services Governance platform should capture service tier subscription information, collects usage statistics, and integrate with billing and payment systems that deliver show-back or charge-back reports. Subscription and usage reports help teams understand asset adoption (by version, by service) and usage (by version, by service). By understanding adoption and usage, business owners and architects can intelligently invest future development resources, properly plan infrastructure scale, and rationalize the portfolio. Dashboards also present a service overview, number of services, service lifecycle stage, schema re-use, service dependencies, upgrade impacts, development team productivity, and project progress. Governance Lifecycle Phases API management portals and SOA Governance Registries must work together to keep API lifecycle stages synchronized with backend service implementation stages. An API Governance experience may provide a straightforward set of lifecycle stages (e.g., created, published, deprecated, retired, blocked) that may be customized by the development team. SOA Governance Registries facilitates service metadata management and governance across design, implementation, test, and run-time operations. Figure 2 below depicts the intersection of the two governance views. Figure 2: API and Service Lifecycle Views Application delivery governance usually relies on ad hoc tools and processes, knitted together by end-user delivery managers. Application Services Governance Platforms should span project inception, development, quality assurance, production deployment, production management, maintenance, and retirement. Figure 3 illustrates service implementation activities governed by an application delivery governance product. Figure 3: Implementation activities governed by application services delivery governance Application Services Governance Drivers The IT focus on API, DevOps, and Cloud scale is driving resurgent interest in Application Services Governance. As development teams support mobile applications by fielding web APIs, they are creating a new ‘demand layer’ in front of existing service implementations. Both API and SOA success requires creating loosely coupled consumer-provider connections, enforcing a separation of concerns between consumer and provider, and exposing a set of re-usable, shared services, and gaining service consumer adoption. With traditional SOA Governance, many development teams publish services, yet struggle to create a service architecture that is widely shared, re-used, and adopted across internal development teams. In today’s connected business world, API and SOA are the business. An effective governance approach must address human collaboration stumbling blocks. By publishing managed APIs, establishing API manager and publisher roles, extending the governance registry, facilitating API management practices (e.g self-service key management, self-service provisioning, service tier management, and usage visualization),and offering APIs through developer portal, organizations can overcome collaboration, trust, and adoption hurdles while enhancing SOA success. By publishing managed APIs, establishing API manager and publisher roles, extending the governance registry, and offering APIs through an API Store, team have a new opportunity to increase service re-use and enhance IT business value. For more information on how teams can complement SOA Governance with API Governance, read the promoting services with API Management white paper. Because services are often imbedded in application solutions, leading Application Services Governance platforms wrap services governance inside application delivery governance. When operation team members use traditional point tools (i.e. Puppet, Chef, Jenkins,Selenium) to achieve DevOps benefits, the teams spend a considerable amount of time and effort creating agile workflow, effective governance, seamless activity transitions, and on-demand self-service access. A configurable DevOps PaaS can implement governance best practices and be readily adopted by teams without extensive implementation effort. Effective application delivery governance presents a simplified and unified user experience to complex development tools, processes, and team hand-offs. By integrating software promotion best practices, test automation, continuous integration, and issue tracking, application delivery governance raises software quality while reducing delivery timeframes. For more information, read about how to accelerate agility and maintain governance with DevOps PaaS. Recommended Reading Policy Management for Application Services Governance Application Services Governance Requires More Than a SOA Registry API and SOA Convergence Promoting services with API Management white paper Accelerate agility and maintain governance with DevOps PaaS Governance Registry Brings Integrity to SaaS Platform Gartner’s analysis of WSO2 SOA Governance

Here are 10 articles from 10 different authors that provide valuable advice for Scrum teams. These articles are in no particular order, so feel free to skim down the list and start with the ones that are most relevant to you. 10 Tips for a Great Daily Scrum Meeting by Platinum Edge – The daily Scrum meeting is a powerful tool that keeps your project moving. At the same time, it is also easy for the meetings to not bring any added value. Tips for Effective Backlog Grooming by Charles Bradley – Are you wasting time in your Sprint Planning Meetings? Increase the value of your team’s Sprint Planning Meetings by grooming your Product Backlog. Yoda’s top 10 tips for a new Scrum Master by Nigel Steane – As a new Scrum Master, you face unfamiliar challenges and your success is very much based on your ability to utilise coaching and soft skills to gently guide your team and colleagues. Top ten tips for distributed Scrum team teleconferences By Jon Archer – After acting as a Scrum Master for several months on a distributed team with people in six different locations, three different countries, learn ten tips to help get past those inevitable awkward silences. 10 tips for adopting Scrum to save your project by Matthew Hodgson – Are you interested in adopting Scrum for your next project? Here are 10 tips from his experience with moving a number of projects from their existing project management frameworks to Scrum. Five Tips for Impediment Resolution with Scrum by Stefan Roock – Impediments can slow down or even halt the progress of an otherwise well-functioning Scrum team. Take a look at the most common challenges that crop up on teams and what steps you can take to resolve them. 10 Tips for Succeeding with Enterprise Agile Development by Tools Journal – Many enterprises are experimenting with agile development approaches like Scrum, Kanban, Lean, and XP hoping that introducing a new development approach will help. Yet, agile development has struggled to achieve critical mass in large enterprises. 6 Tips for Good Scrum by Martin Harris – If you are doing these 6 tips, then you are doing very well and are likely to get better over time. 9 Tips for Creating a Good Sprint Backlog by Luciano Felix – Giving attention to the sprint backlog creation process is fundamental to the team’s understanding of what should be done and how to better plan during the sprint. 7 Tips for a More Effective Daily Scrum by Richard Lawrence – The main purpose of the Daily Scrum is for team members to make and follow-up on commitments to one another that work towards the team’s shared sprint commitment. Here are seven ways to get your Daily Scrum back on focus If your it has become unfocused, too long, or otherwise ineffective. If you have any other good articles related to agile, please share them in the comments. Thanks.

I've been working as a Scrum Master and as an Agile Coach for a good few years now, mainly as a contractor. Each time I am interviewed for a new contract I always like to ask if I can meet the teams I’d be working with. You see, time and again it will be a manager who does the interviewing, while the team members themselves are left with little say in whether I should be hired. I think it’s important that they reckon they can get along with me. Of course, it also gives me an opportunity to see them, and to gain a fuller understanding of the situation I’d really be walking into. As we head towards the desks of my prospective team, one of the first things I look for is the board, whether it be a Scrum task board or a Kanban board. Most teams with agile aspirations…or agile pretensions…will have set up a board of some kind. A board is the "grand old dame" of information radiators. No matter how much the details of a sordid past are glossed over, the truth always seems to come out. It's in the nature of a board to tell the truth, since any untruths can be quickly exposed. The story I can piece together from dubious lanes and columns, misplaced or missing tickets, misplaced or missing avatars, and a host of other shibboleths can be far more telling than anything I get to hear from people in an interview situation. Another of the things I look for is a "fast track" lane on a Scrum Team's board. These are very common; you could say it is almost unusual not to see them. From a certain perspective they are good things to have, and they can imply a level of maturity - or at least of pragmatism - on the part of a team. They suggest that the team accepts that not everything can be predicted in Sprint planning. A fast track lane is a nod to the fact that emergencies happen, that support work and unforeseen defect fixes still need to be done, and most importantly, that the team has a way of dealing with all of this. However it also shows that they aren't doing Scrum. There...I've said it. Fast track lanes aren't part of Scrum. It's that simple. I don't mean to say that they are bad practice, or in some sense un-agile. On the contrary, they are part of the Lean Kanban approach to varying the Quality of Service provided to certain backlog items. That's what a fast track lane is...a way of varying the quality of service that a Scrum team gives to certain items. When something hits a fast track lane, a well-trained team will swarm over it and decide who is best qualified to progress the matter. While they do this, their own tasks will be marked as impeded or blocked. Then, the decision made, all others return to their work in progress. So if fast track lanes are a widely understood and practical way of managing operational issues, what is wrong with them, Scrum-wise? The answer is that Scrum - unlike Lean Kanban - doesn't provide for variations in quality of service. Each piece of work is prioritized and negotiated into a Sprint backlog. The team then self-organizes to deliver a corresponding increment of functionality. The team will plan with the Product Owner what it intends to do during a sprint, and the sprint backlog they agree to belongs to them. No-one, not even the CEO of the organization, can override their sprint backlog by introducing work to be "fast tracked". The team wholly owns their sprint backlog. That's Scrum. When I point this out, teams can become crestfallen or even defensive. “What else are we supposed to do”, they say. “We aren’t dedicated 100% to doing project work. We still have support work to do, and serious issues always trump development. We have to fix them and put project work on hold.” My answer to that is that under the circumstances the team is facing, it may indeed be right to vary the quality of service by fast-tracking support work. It just isn’t Scrum, that’s all. It’s a type of "Scrumban", a Scrum variant that includes Kanban characteristics. This is no fault of the team, but it could suggest a problem higher up. Perhaps a dedicated Kanban support team hasn’t been properly resourced and trained so that Scrum development can proceed unimpeded. Perhaps the Product Owner is being undermined by other managers who have separate interests impacting the development. Whatever the situation, it needs to be made transparent and acknowledged by all stakeholders. So, the next step…and the one I’ll often indicate as the interview progresses…is to account for fast track work as impediments against product burndown or velocity. Moreover, these are impediments which are external to the team. It’s essentially a type of waste, or unplanned work, being generated from outside. It needs to be made quite transparent where this waste is coming from and what can be done to mitigate it. What can be done about those other teams, or workflows, or managers, who are undercutting this Scrum team’s ability to plan out their Sprints? Often, the source of these impediments will be the people interviewing me...and that’s when things can start to get really interesting!

With unit tests you can check that your code behaviours just as you expect it to. When writing your unit tests you shouldn't need to worry about if any other area of the application is working correctly. The benefits of unit testing are: Decouples your code Write more modular classes Functions are smaller and more focused Your functions are more defensive Quality of code becomes higher You will find it easier to reuse code. When writing unit tests you just need to test this one method of your application, if your method relies on another class/variable there should be a way you can inject this into the method. This is where dependency injection in your code comes in handy, it will allow you to inject objects into your classes to change the output of the class. There are a few things you need to do to make a method unit testable, methods will need an input from a parameter or a class variable and it will need a return or set a class variable in the method. If the method hasn't got these things then the method can not be unit testable. If there isn't a return of the method then there is no way in knowing how the method performs. Dependency Injection Dependency injection is when your object has a dependency on another object. The simplest form to understand what dependency injection is to think of a setter method. A setter method will take one parameter and set a class variable from this parameter. This is using code injection to pass in a parameter to be used as the class variable value. public function setValue( $val ) { $this->val = $val; } Without dependency injection this method will look like this. public function setValue() { $this->val = 10; } For unit testing you need to be aware of any classes that your class is dependent on. For example if you have a login class that will connect to a database. class login { private $db = false; public function __construct() { $this->db = new Database(); } public function loginUser( $user, $password ) { $this->db->checkLogin( $user, $password ); } } This login class has a dependency of the class Database in the constructor, which means that we can't unit test this correctly. If we want to unit test this then the database class has to be development and tested. If the database class is broken and we try to unit test the loginUser() method the test will always fail and we won't know that it's the database class which is broke or the loginUser() method that is broke. If the database class is finished development, tested and data is in the database then we can use this for the loginUser() function. But now our tests are dependent on data being correct in the database. If we pass in a username and password it must be in the database for our test to pass. Our code could be correct but if the data isn't there then our unit tests will fail. This isn't correct use of unit tests and is more suited to be an integration test. To fix this problem we can use dependency injection to pass in a database connector which will set the database class variable. There are 2 ways we can inject a variable into a class, it can either be in the constructor of the class or by using a setter method. I tend to use constructor for all required dependences and use the setter method if there is a default value for the class variable. class login { private $db = false; public function __construct( $db ) { $this->db = $db; } public function loginUser( $user, $password ) { $this->db->checkLogin( $user, $password ); } } Now this class isn't dependant on a certain database class we can pass in the database class by using the parameter on the login class constructor. We can unit test this loginUser() method by first setting the $this->db class variable. We don't want to rely on a real database as the data can change so we can either create a test harness database class or you can mock the database class. A test harness class will allow you to create your database class and hardcode any data that you need. In the example above we can create a method checkLogin(), in our test harness we can then hardcode a successful login username and password to make the loginUser() method pass. Or you can use a PHP mocking framework to mock a class/method/return value. Both methods have their benefits but mocking is normally quicker to code, but there are times when you want to hardcode certain variables in a class. Mocking Objects In TDD With PHP Mocking objects in test driven development allows you create objects to act as a certain class, if your test depends on another method to return a value, you can mock this method and make it return any value you want. In the example we used above you can mock the database class and choose what value we are expecting back from the checkLogin() method. When mocking a method you can choose what you want to return from this method, therefore we can write tests to see what will happen when checkLogin() returns TRUE and then we can write another test to see what happens when checkLogin() returns FALSE. Mocking objects means that you can run your unit tests without depending on another class returning the values you are expecting, ao you can test just your code in this one method. Here are some of the most popular PHP mocking frameworks: Mocking with PHPUnit - http://www.phpunit.de/manual/3.0/en/mock-objects.html Mocking with Phake - http://phake.digitalsandwich.com/docs/html/ Mocking with Mockery - https://github.com/padraic/mockery Mocking with Enchane PHP - https://github.com/Enhance-PHP/Enhance-PHP Mocking with FBMock - https://github.com/facebook/FBMock Dependency Injection With Interfaces If we are going to pass in a database connector in a constructor of the login class, then this database connector will always have to have a method of checkLogin(). This is why we should code our dependences by using interfaces to make sure that we are always passing in the correct type of class. class login { private $db = false; public function __construct( IDatabase $db ) { $this->db = $db; } } class database implements IDatabase { public function checkLogin( $username, $password ) { // check the login credentials } } interface IDatabase { public function checkLogin( $username, $password ); } This will make sure that the class we pass into the constructor is a type of IDatabase, so if our database class doesn't implement IDatabase then the code will fail and therefore our unit tests will fail. This means whatever we pass into the constructor we know that this class will be able to run the methods it needs for the unit tests to run.

n this post i’m going to cover how to do job chaining in quartz versus obsidian scheduler . both are java job schedulers, but they have different approaches so i thought i’d highlight them here and give some guidance to users using both options. it’s very common when using a job scheduler to need to chain one job to another. chaining in this case refers to executing a specific job after a certain job completes (or maybe even fails). often we want to do this conditionally, or pass on data to the target job so it can receive it as input from the original job. we’ll start with demonstrating how to do this in quartz, which will take a fair bit of work. obsidian will come after since it’s so simple. chaining in quartz quartz is the most popular job scheduler out there, but unfortunately it doesn’t provide any way to give you chaining without you writing some code. quartz is a low-level library at heart, and it doesn’t try to solve these types of problems for you, which in my mind is unfortunate since it puts the onus on developers. but despite this, many teams still end up using quartz, so hopefully this is useful to some of you. i’m going to outline probably the most basic way to perform chaining. it will allow a job to chain to another, passing on its jobdatamap (for state). this is simpler than using listeners, which would require extra configuration, but if you want to take a look, check out this listener for a starting point. sample code this will rely on an abstract class that will provided basic flow and chaining functionality to any subclasses. it acts as a very simple template class. first, let’s create the abstract class that gives us chaining behaviour: import static org.quartz.jobbuilder.newjob; import static org.quartz.triggerbuilder.newtrigger; import org.quartz.*; import org.quartz.impl.*; public abstract class chainablejob implements job { private static final string chain_job_class = "chainedjobclass"; private static final string chain_job_name = "chainedjobname"; private static final string chain_job_group = "chainedjobgroup"; @override public void execute(jobexecutioncontext context) throws jobexecutionexception { // execute actual job code doexecute(context); // if chainjob() was called, chain the target job, passing on the jobdatamap if (context.getjobdetail().getjobdatamap().get(chain_job_class) != null) { try { chain(context); } catch (schedulerexception e) { e.printstacktrace(); } } } // actually schedule the chained job to run now private void chain(jobexecutioncontext context) throws schedulerexception { jobdatamap map = context.getjobdetail().getjobdatamap(); @suppresswarnings("unchecked") class jobclass = (class) map.remove(chain_job_class); string jobname = (string) map.remove(chain_job_name); string jobgroup = (string) map.remove(chain_job_group); jobdetail jobdetail = newjob(jobclass) .withidentity(jobname, jobgroup) .usingjobdata(map) .build(); trigger trigger = newtrigger() .withidentity(jobname + "trigger", jobgroup + "trigger") .startnow() .build(); system.out.println("chaining " + jobname); stdschedulerfactory.getdefaultscheduler().schedulejob(jobdetail, trigger); } protected abstract void doexecute(jobexecutioncontext context) throws jobexecutionexception; // trigger job chain (invocation waits for job completion) protected void chainjob(jobexecutioncontext context, class jobclass, string jobname, string jobgroup) { jobdatamap map = context.getjobdetail().getjobdatamap(); map.put(chain_job_class, jobclass); map.put(chain_job_name, jobname); map.put(chain_job_group, jobgroup); } } there’s a fair bit of code here, but it’s nothing too complicated. we create the basic flow for job chaining by creating an abstract class which calls a doexecute() method in the child class, then chains the job if it was requested by calling chainjob() . so how do we use it? check out the job below. it actually chains to itself to demonstrate that you can chain any job and that it can be conditional. in this case, we will chain the job to another instance of the same class if it hasn’t already been chained, and we get a true value from new random().nextboolean() . import java.util.*; import org.quartz.*; public class testjob extends chainablejob { @override protected void doexecute(jobexecutioncontext context) throws jobexecutionexception { jobdatamap map = context.getjobdetail().getjobdatamap(); system.out.println("executing " + context.getjobdetail().getkey().getname() + " with " + new linkedhashmap(map)); boolean alreadychained = map.get("jobvalue") != null; if (!alreadychained) { map.put("jobtime", new date().tostring()); map.put("jobvalue", new random().nextlong()); } if (!alreadychained && new random().nextboolean()) { chainjob(context, testjob.class, "secondjob", "secondjobgroup"); } } } the call to chainjob() at the end will result in the automatic job chaining behaviour in the parent class. note that this isn’t called immediately, but only executes after the job completes its doexecute() method. here’s a simple harness that demonstrates everything together: import org.quartz.*; import org.quartz.impl.*; public class test { public static void main(string[] args) throws exception { // start up scheduler stdschedulerfactory.getdefaultscheduler().start(); jobdetail job = jobbuilder.newjob(testjob.class) .withidentity("firstjob", "firstjobgroup").build(); // trigger our source job to triggers another trigger trigger = triggerbuilder.newtrigger() .withidentity("firstjobtrigger", "firstjobbtriggergroup") .startnow() .withschedule( simpleschedulebuilder.simpleschedule().withintervalinseconds(1) .repeatforever()).build(); stdschedulerfactory.getdefaultscheduler().schedulejob(job, trigger); thread.sleep(5000); // let job run a few times stdschedulerfactory.getdefaultscheduler().shutdown(); } } sample output executing firstjob with {} chaining secondjob executing secondjob with {jobvalue=5420204983304142728, jobtime=sat mar 02 15:19:29 pst 2013} executing firstjob with {} executing firstjob with {} chaining secondjob executing secondjob with {jobvalue=-2361712834083016932, jobtime=sat mar 02 15:19:31 pst 2013} executing firstjob with {} chaining secondjob executing secondjob with {jobvalue=7080718769449337795, jobtime=sat mar 02 15:19:32 pst 2013} executing firstjob with {} chaining secondjob executing secondjob with {jobvalue=7235143258790440677, jobtime=sat mar 02 15:19:33 pst 2013} executing firstjob with {} deficiencies well, we’re up and chaining, but there are some problems with this approach: it doesn’t integrate with a container like spring to use configured jobs. more code would be required. it forces you to know up front which jobs you want to chain, and write code for it. configuration is fixed, unless, once again, you write more code. no real-time changes (unless you write more code). a fair bit of code to maintain , and high likelihood you will have to expand it for more functionality. the theme here is that it’s doable, but it’s up to you to do the work to make it happen. obsidian avoids these problems by making chaining configurable, instead of it being a feature of the job itself. read on to find out how. chaining in obsidian in contrast to quartz, chaining in obsidian requires no code and no up-front knowledge of which jobs will chain or how you might want to chain them later. chaining is a form of configuration, and like all job-related configuration in obsidian, you can make live changes at any time without a build or any code at all. job configuration can use a native rest api or the web ui that’s included with obsidian. the following chaining features are available for free: no code and no redeploy to add or remove chains. you can chain specific configurations of job classes. you can chain only on certain states, including failure. chain conditionally based on source job saved state (equivalent to quartz’s jobdatamap), including multiple conditions. regexp/equals/greater than, etc. chain only when matching a schedule. check out the feature and ui documentation to find out more. now that we know what’s possible, let’s see an example. once you have your jobs configured , just create a new chain using the ui. rest api support will be here shortly but as of 1.5.1 chaining isn’t included in the api. if you need to script this right now, we can provide pointers . in the ui, it looks like the following: easy, huh? all configuration is stored in a database, so it’s easy to replicate it in various environments or to automate it via scripting. as a bonus, obsidian tracks and shows you all chaining state including what job triggered a chained job. it will even tell you why a job chain didn’t fire, whether it’s because the job status didn’t match, or one of your conditions didn’t. conclusion that summarizes how you can go about chaining in quartz and obsidian. quartz definitely has a minimalist approach, but that leaves developers with a lot of work to do. meanwhile, obsidian provides rich functionality out of the box to keep developers working on their own rich functionality, instead of the plumbing that so often seems to consume their time. if you have any suggestions or feature requests for obsidian, drop us a note by leaving a comment or by contacting us .

I Programmer - Anime Scrum - An Overview If you are an anime fan, and perhaps even if you are not, then you might like a new poster about Scrum - in anime style. As long as you find anime cute or something then seeing the different people involved in the Scrum methodology as anime characters might help you convey the ideas to others. ... Scrum Primer - Scrum Overview - Anime version High-resolution versions of the overview: Scrum Overview - Blue Scrum Overview - Pink Scrum Overview - Green Feel free to use it in your own material. ..." Come on! You KNOW that's awesome! :)

Many enterprise applications require batch processing to process billions of transactions every day. These big transaction sets have to be processed without performance problems. Spring Batch is a lightweight and robust batch framework to process these big data sets. Spring Batch offers ‘TaskletStep Oriented’ and ‘Chunk Oriented’ processing style. In this article, TaskletStep Oriented Processing Model is explained. Let us investigate fundamental Spring Batch components : Job : An entity that encapsulates an entire batch process. Step and Tasklets are defined under a Job Step : A domain object that encapsulates an independent, sequential phase of a batch job. JobInstance : Batch domain object representing a uniquely identifiable job run – it’s identity is given by the pair Job and JobParameters. JobParameters : Value object representing runtime parameters to a batch job. JobExecution : A JobExecution refers to the technical concept of a single attempt to run a Job. An execution may end in failure or success, but the JobInstance corresponding to a given execution will not be considered complete unless the execution completes successfully. JobRepository : An interface which responsible for persistence of batch meta-data entities. In the following sample, an in-memory repository is used via MapJobRepositoryFactoryBean. JobLauncher : An interface exposing run method, which launches and controls the defined jobs. TaskLet : An interface exposing execute method, which will be a called repeatedly until it either returns RepeatStatus.FINISHED or throws an exception to signal a failure. It is used when both readers and writers are not required as the following sample. Let us take a look how to develop Tasklet-Step Oriented Processing Model. Used Technologies : JDK 1.7.0_09 Spring 3.1.3 Spring Batch 2.1.9 Maven 3.0.4 STEP 1 : CREATE MAVEN PROJECT A maven project is created as below. (It can be created by using Maven or IDE Plug-in). STEP 2 : LIBRARIES Firstly, dependencies are added to Maven’ s pom.xml. 3.1.3.RELEASE 2.1.9.RELEASE org.springframework spring-core ${spring.version} org.springframework spring-context ${spring.version} org.springframework.batch spring-batch-core ${spring-batch.version} log4j log4j 1.2.16 maven-compiler-plugin(Maven Plugin) is used to compile the project with JDK 1.7 org.apache.maven.plugins maven-compiler-plugin 3.0 1.7 1.7 The following Maven plugin can be used to create runnable-jar, org.apache.maven.plugins maven-shade-plugin 2.0 package shade 1.7 1.7 com.onlinetechvision.exe.Application META-INF/spring.handlers META-INF/spring.schemas STEP 3 : CREATE SuccessfulStepTasklet TASKLET SuccessfulStepTasklet is created by implementing Tasklet Interface. It illustrates business logic in successful step. package com.onlinetechvision.tasklet; import org.apache.log4j.Logger; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; /** * SuccessfulStepTasklet Class illustrates a successful job * * @author onlinetechvision.com * @since 27 Nov 2012 * @version 1.0.0 * */ public class SuccessfulStepTasklet implements Tasklet { private static final Logger logger = Logger.getLogger(SuccessfulStepTasklet.class); private String taskResult; /** * Executes SuccessfulStepTasklet * * @param StepContribution stepContribution * @param ChunkContext chunkContext * @return RepeatStatus * @throws Exception * */ @Override public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { logger.debug("Task Result : " + getTaskResult()); return RepeatStatus.FINISHED; } public String getTaskResult() { return taskResult; } public void setTaskResult(String taskResult) { this.taskResult = taskResult; } } STEP 4 : CREATE FailedStepTasklet TASKLET FailedStepTasklet is created by implementing Tasklet Interface. It illustrates business logic in failed step. package com.onlinetechvision.tasklet; import org.apache.log4j.Logger; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; /** * FailedStepTasklet Class illustrates a failed job. * * @author onlinetechvision.com * @since 27 Nov 2012 * @version 1.0.0 * */ public class FailedStepTasklet implements Tasklet { private static final Logger logger = Logger.getLogger(FailedStepTasklet.class); private String taskResult; /** * Executes FailedStepTasklet * * @param StepContribution stepContribution * @param ChunkContext chunkContext * @return RepeatStatus * @throws Exception * */ @Override public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { logger.debug("Task Result : " + getTaskResult()); throw new Exception("Error occurred!"); } public String getTaskResult() { return taskResult; } public void setTaskResult(String taskResult) { this.taskResult = taskResult; } } STEP 5 : CREATE BatchProcessStarter CLASS BatchProcessStarter Class is created to launch the jobs. Also, it logs their execution results. A Completed Job Instance can not be restarted with the same parameter(s) because it already exists in job repository and JobInstanceAlreadyCompleteException is thrown with “A job instance already exists and is complete” description. It can be restarted with different parameter. In the following sample, different currentTime parameter is set in order to restart FirstJob. package com.onlinetechvision.spring.batch; import org.apache.log4j.Logger; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.core.JobParametersInvalidException; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.JobRestartException; /** * BatchProcessStarter Class launches the jobs and logs their execution results. * * @author onlinetechvision.com * @since 27 Nov 2012 * @version 1.0.0 * */ public class BatchProcessStarter { private static final Logger logger = Logger.getLogger(BatchProcessStarter.class); private Job firstJob; private Job secondJob; private Job thirdJob; private JobLauncher jobLauncher; private JobRepository jobRepository; /** * Starts the jobs and logs their execution results. * */ public void start() { JobExecution jobExecution = null; JobParametersBuilder builder = new JobParametersBuilder(); try { builder.addLong("currentTime", new Long(System.currentTimeMillis())); getJobLauncher().run(getFirstJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getFirstJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); getJobLauncher().run(getSecondJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getSecondJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); getJobLauncher().run(getThirdJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getThirdJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); builder.addLong("currentTime", new Long(System.currentTimeMillis())); getJobLauncher().run(getFirstJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getFirstJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); } catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException | JobParametersInvalidException e) { logger.error(e); } } public Job getFirstJob() { return firstJob; } public void setFirstJob(Job firstJob) { this.firstJob = firstJob; } public Job getSecondJob() { return secondJob; } public void setSecondJob(Job secondJob) { this.secondJob = secondJob; } public Job getThirdJob() { return thirdJob; } public void setThirdJob(Job thirdJob) { this.thirdJob = thirdJob; } public JobLauncher getJobLauncher() { return jobLauncher; } public void setJobLauncher(JobLauncher jobLauncher) { this.jobLauncher = jobLauncher; } public JobRepository getJobRepository() { return jobRepository; } public void setJobRepository(JobRepository jobRepository) { this.jobRepository = jobRepository; } } STEP 6 : CREATE applicationContext.xml Spring Configuration file, applicationContext.xml, is created. It covers Tasklets and BatchProcessStarter definitions. STEP 7 : CREATE jobContext.xml Spring Configuration file, jobContext.xml, is created. Jobs’ flows are the following : FirstJob’ s flow : 1) FirstStep is started. 2) After FirstStep is completed with COMPLETED status, SecondStep is started. 3) After SecondStep is completed with COMPLETED status, ThirdStep is started. 4) After ThirdStep is completed with COMPLETED status, FirstJob execution is completed with COMPLETED status. SecondJob’ s flow : 1) FourthStep is started. 2) After FourthStep is completed with COMPLETED status, FifthStep is started. 3) After FifthStep is completed with COMPLETED status, SecondJob execution is completed with COMPLETED status. ThirdJob’ s flow : 1) SixthStep is started. 2) After SixthStep is completed with COMPLETED status, SeventhStep is started. 3) After SeventhStep is completed with FAILED status, ThirdJob execution is completed FAILED status. FirstJob’ s flow is same with the first execution. STEP 8 : CREATE Application CLASS Application Class is created to run the application. package com.onlinetechvision.exe; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import com.onlinetechvision.spring.batch.BatchProcessStarter; /** * Application Class starts the application. * * @author onlinetechvision.com * @since 27 Nov 2012 * @version 1.0.0 * */ public class Application { /** * Starts the application * * @param String[] args * */ public static void main(String[] args) { ApplicationContext appContext = new ClassPathXmlApplicationContext("jobContext.xml"); BatchProcessStarter batchProcessStarter = (BatchProcessStarter)appContext.getBean("batchProcessStarter"); batchProcessStarter.start(); } } STEP 9 : BUILD PROJECT After OTV_SpringBatch_TaskletStep_Oriented_Processing Project is built, OTV_SpringBatch_TaskletStep-0.0.1-SNAPSHOT.jar will be created. STEP 10 : RUN PROJECT After created OTV_SpringBatch_TaskletStep-0.0.1-SNAPSHOT.jar file is run, the following console output logs will be shown : First Job’ s console output : 25.11.2012 21:29:19 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=firstJob]] launched with the following parameters: [{currentTime=1353878959462}] 25.11.2012 21:29:19 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=0, version=0, startTime=null, endTime=null, lastUpdated=Sun Nov 25 21:29:19 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN; exitDescription=, job=[JobInstance: id=0, version=0, JobParameters=[{currentTime=1353878959462}], Job=[firstJob]] 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:135) - Resuming state=firstJob.firstStep with status=UNKNOWN 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.firstStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [firstStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=1 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : First Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:209) - Step execution success: id=1 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=1, version=3, name=firstStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.firstStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.secondStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [secondStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=2 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Second Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:209) - Step execution success: id=2 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=2, version=3, name=secondStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.secondStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.thirdStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [thirdStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=3 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Third Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=3, version=3, name=thirdStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.thirdStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.end3 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.end3 with status=COMPLETED 25.11.2012 21:29:20 DEBUG (AbstractJob.java:294) - Job execution complete: JobExecution: id=0, version=1, startTime=Sun Nov 25 21:29:19 GMT 2012, endTime=null, lastUpdated=Sun Nov 25 21:29:19 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=0, version=0, JobParameters=[{currentTime=1353878959462}], Job=[firstJob]] 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:121) - Job: [FlowJob: [name=firstJob]] completed with the following parameters: [{currentTime=1353878959462}] and the following status: [COMPLETED] 25.11.2012 21:29:20 DEBUG (BatchProcessStarter.java:44) - JobExecution: id=0, version=2, startTime=Sun Nov 25 21:29:19 GMT 2012, endTime=Sun Nov 25 21:29:20 GMT 2012, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=0, version=0, JobParameters=[{currentTime=1353878959462}], Job=[firstJob]] Second Job’ s console output : 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=secondJob]] launched with the following parameters: [{currentTime=1353878959462}] 25.11.2012 21:29:20 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=1, version=0, startTime=null, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=1, version=0, JobParameters=[{currentTime=1353878959462}], Job=[secondJob]] 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:135) - Resuming state=secondJob.fourthStep with status=UNKNOWN 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=secondJob.fourthStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [fourthStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=4 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Fourth Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=4, version=3, name=fourthStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=secondJob.fourthStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=secondJob.fifthStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [fifthStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=5 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Fifth Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=5, version=3, name=fifthStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=secondJob.fifthStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=secondJob.end5 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=secondJob.end5 with status=COMPLETED 25.11.2012 21:29:20 DEBUG (AbstractJob.java:294) - Job execution complete: JobExecution: id=1, version=1, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=1, version=0, JobParameters=[{currentTime=1353878959462}], Job=[secondJob]] 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:121) - Job: [FlowJob: [name=secondJob]] completed with the following parameters: [{currentTime=1353878959462}] and the following status: [COMPLETED] 25.11.2012 21:29:20 DEBUG (BatchProcessStarter.java:48) - JobExecution: id=1, version=2, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=Sun Nov 25 21:29:20 GMT 2012, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=1, version=0, JobParameters=[{currentTime=1353878959462}], Job=[secondJob]] Third Job’ s console output : 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=thirdJob]] launched with the following parameters: [{currentTime=1353878959462}] 25.11.2012 21:29:20 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=2, version=0, startTime=null, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=2, version=0, JobParameters=[{currentTime=1353878959462}], Job=[thirdJob]] 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:135) - Resuming state=thirdJob.sixthStep with status=UNKNOWN 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=thirdJob.sixthStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [sixthStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=6 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Sixth Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=6, version=3, name=sixthStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=thirdJob.sixthStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=thirdJob.seventhStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [seventhStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=7 25.11.2012 21:29:20 DEBUG (FailedStepTasklet.java:33) - Task Result : Error occurred! 25.11.2012 21:29:20 DEBUG (TaskletStep.java:456) - Rollback for Exception: java.lang.Exception: Error occurred! 25.11.2012 21:29:20 DEBUG (TransactionTemplate.java:152) - Initiating transaction rollback on application exception ... 25.11.2012 21:29:20 DEBUG (AbstractPlatformTransactionManager.java:821) - Initiating transaction rollback 25.11.2012 21:29:20 DEBUG (ResourcelessTransactionManager.java:54) - Rolling back resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@40874c04] 25.11.2012 21:29:20 DEBUG (RepeatTemplate.java:291) - Handling exception: java.lang.Exception, caused by: java.lang.Exception: Error occurred! 25.11.2012 21:29:20 DEBUG (RepeatTemplate.java:251) - Handling fatal exception explicitly (rethrowing first of 1): java.lang.Exception: Error occurred! 25.11.2012 21:29:20 ERROR (AbstractStep.java:222) - Encountered an error executing the step ... 25.11.2012 21:29:20 DEBUG (ResourcelessTransactionManager.java:34) - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@66a7d863] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=7, version=2, name=seventhStep, status=FAILED, exitStatus=FAILED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=0, rollbackCount=1 25.11.2012 21:29:20 DEBUG (ResourcelessTransactionManager.java:34) - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@156f803c] 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=thirdJob.seventhStep with status=FAILED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=thirdJob.fail8 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=thirdJob.fail8 with status=FAILED 25.11.2012 21:29:20 DEBUG (AbstractJob.java:294) - Job execution complete: JobExecution: id=2, version=1, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=FAILED, exitStatus=exitCode=FAILED;exitDescription=, job=[JobInstance: id=2, version=0, JobParameters=[{currentTime=1353878959462}], Job=[thirdJob]] 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:121) - Job: [FlowJob: [name=thirdJob]] completed with the following parameters: [{currentTime=1353878959462}] and the following status: [FAILED] 25.11.2012 21:29:20 DEBUG (BatchProcessStarter.java:52) - JobExecution: id=2, version=2, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=Sun Nov 25 21:29:20 GMT 2012, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=FAILED, exitStatus=exitCode=FAILED; exitDescription=, job=[JobInstance: id=2, version=0, JobParameters=[{currentTime=1353878959462}], Job=[thirdJob]] First Job’ s console output after restarting : 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=firstJob]] launched with the following parameters: [{currentTime=1353878960660}] 25.11.2012 21:29:20 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=3, version=0, startTime=null, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=3, version=0, JobParameters=[{currentTime=1353878960660}], Job=[firstJob]] 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:135) - Resuming state=firstJob.firstStep with status=UNKNOWN 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.firstStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [firstStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=8 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : First Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:209) - Step execution success: id=8 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=8, version=3, name=firstStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.firstStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.secondStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [secondStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=9 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Second Task is executed... 25.11.2012 21:29:20 DEBUG (TaskletStep.java:417) - Applying contribution: [StepContribution: read=0, written=0, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:209) - Step execution success: id=9 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=9, version=3, name=secondStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.secondStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.thirdStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [thirdStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=10 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Third Task is executed... 25.11.2012 21:29:20 DEBUG (TaskletStep.java:417) - Applying contribution: [StepContribution: read=0, written=0, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:209) - Step execution success: id=10 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=10, version=3, name=thirdStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.thirdStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.end3 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.end3 with status=COMPLETED 25.11.2012 21:29:20 DEBUG (AbstractJob.java:294) - Job execution complete: JobExecution: id=3, version=1, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=3, version=0, JobParameters=[{currentTime=1353878960660}], Job=[firstJob]] 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:121) - Job: [FlowJob: [name=firstJob]] completed with the following parameters: [{currentTime=1353878960660}] and the following status: [COMPLETED] 25.11.2012 21:29:20 DEBUG (BatchProcessStarter.java:57) - JobExecution: id=3, version=2, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=Sun Nov 25 21:29:20 GMT 2012, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=3, version=0, JobParameters=[{currentTime=1353878960660}], Job=[firstJob]] STEP 11 : DOWNLOAD https://github.com/erenavsarogullari/OTV_SpringBatch_TaskletStep REFERENCES : Spring Batch – Reference Documentation Spring Batch – API Documentation

From an original article on Wakaleo.com Conventional test reports, generated by tools such as JUnit or TestNG, naturally focus on what tests have been executed, and whether they passed or failed. While this is certainly useful from a testing perspective, these reports are far from telling the whole picture. BDD reporting tools like Cucumber and JBehave take things a step further, introducing the concept of "pending" tests. A pending test is one that has been specified (for example, as an acceptance criteria for a user story), but which has not been implemented yet. In BDD, we describe the expected behaviour of our application using concrete examples, that eventually form the basis of the "acceptance criteria" for the user stories we are implementing. BDD tools such as Cucumber and JBehave not only report on test results: they also report on the user stories that these tests validate. However this reporting is still limited for large projects, where the numbers of user stories can become unwieldy. User stories are not created in isolation: rather, user stories help describe features, which support capabilities that need to be implemented to achieve the business goals of the application. So it makes sense to be able to report on test results not only at the user story level, but also at higher levels, for example in terms of features and capabilities. This makes it easier to report on not only what stories have been implemented, but also what features and capabilities remain to be done. An example of such a report is shown in Figure 1 (or see the full report here). Figure 1: A test coverage report listing both tested and untested requirements. In agile projects, it is generally considered that a user story is not complete until all of its automated acceptance tests pass. Similarly, a feature cannot be considered ready to deliver until all of the acceptance criteria for the underlying user stories have been specified and implemented. However, sensible teams shy away from trying to define all of the acceptance criteria up-front, leaving this until the "last responsible moment", often shortly before the user story is scheduled to be implemented. For this reason, reports that relate project progress and status only in terms of test results are missing out on the big picture. To get a more accurate idea of what features have been delivered, which ones are in progress, and what work remains to be done, we must think not in terms of test results, but in terms of the requirements as we currently understand them, matching the currently implemented tests to these requirements, but also pointing out what requirements currently have no acceptance criteria defined. And when graphs and reports illustrate how much progress has been made, the requirements with no acceptance criteria must also be part of the picture. Requirements-level BDD reporting with Thucydides Thucydides is an open source tool that puts some of these concepts into practice. Building on top of BDD tools such as JBehave, or using just ordinary JUnit tests, Thucydides reports not only on how the tests did, but also fits them into the broader picture, showing what requirements have been tested and, just as importantly, what requirements haven't. You can learn more about Thucydides in this tutorial or on the Thucydides website. During the rest of this article, we will see how to report on both your requirements and your test results using Thucydides, using a very simple directory-based approach. You can follow along with this example by cloning the Github project at https://github.com/thucydides-webtests/thucydides-simple-demo Simple requirements in Thucydides - a directory-based approach Thucydides can integrate with many different requirement management systems, and it is easy to write your own plugin to tailor the integration to suite your particular environment. A popular approach, for example, is to store requirements in JIRA and to use Thucydides to read the requirements hierarcy directly from the JIRA cards. However the simplest approach, which uses a directory-based approach, is probably the easiest to use to get started, and it is that approach that we will be looking at here. Requirements can usually be organized in a hierarchial structure. By default, Thucydides uses a three-level hierarchy of requirements. At the top level, capabilities represent a high-level capacity that the application must provide to meet the application's business goals. At the next level down, features help deliver these capabilities. To make implementation easier, a feature can be broken up into user stories, each of which in turn can contain a number of acceptance criteria. Figure 2: JUnit test directories mirror the requirements hierarchy. Of course, you don't have to use this structure if it doesn't suit you. You can override the thucydides.capability.types system property to provide your own hierarchy. For example, if you wanted a hierarchy with modules,epics, and features, you would just set thucydides.capability.types to "module,epic,feature". When we use the default directory-based requirements strategy in Thucydides, the requirements are stored in a hierarchial directory structure that matches the requirements hierarchy. At the lowest level, a user story is represented by a JBehave *.story file, an easyb story, or a JUnit test. All of the other requirements are represented as directories (see Figure 2 for an example of such a structure). In each requirements directory, you can optionally place a file called narrative.txt, which contains a free-text summary of the requirement. This will appear in the reports, with the first line appearing as the requirement title. A typical narrative text is illustrated in the following example: Learn the meaning of a word In order to learn the meaning of a word that I don't know As an online reader I want to be able to find out the meaning of the word If you are implementing the acceptance criteria as JUnit tests, just place the JUnit tests in the package that matches the correspoinding requirement. You need to use the thucydides.test.root system property to specify the root package of your requirements. For the example in Figure 2, this value should be set to nz.govt.nzqa.lssu.stories. Figure 3: The narrative.txt file appears in the reports to describe a requirement. If you are using JBehave, just place the *.story files in the src/test/resources/stories directory, again respecting a directory structure that corresponds to your requirement hierarchy. The narrative.txt files also work for JBehave requirements. Progress is measured by the total number of passing, failing or pending acceptance criteria, either for the whole project (at the top level), or within a particular requirement as you drill down the requirements hierarchy. For the purposes of reporting, a requirement with no acceptance criteria is attributed an arbitrary number of "imaginary" pending acceptance criteria. Thucydides considers that you need 4 tests per requirement by default, but you can override this value using the thucydides.estimated.tests.per.requirement system property. Figure 3: For JBehave, everything goes under src/test/resources/stories. Conclusion BDD is an excellent approach for communicating with, and reporting back to, stakeholders. However, for accurate acceptance test reporting on real-world projects, you need to go beyond the story level, and cater for the whole requirements hierarchy. In particular, you need to not only report on tests that have been executed, but also allow for the tests that haven't been written yet. Thucydides puts these concepts into practice: using a simple directory-based convention, you can easily integrate your requirements hierarcy into your acceptance tests.

In many Agile training programs and conferences, a common question that gets raised is, does Agile/Scrum work in maintenance projects? I always say "YES" and the team needs to tweak or invent the practices to suit their needs. Maintenance projects could be enhancement projects OR pure defect fixing projects. Enhancement projects involve new set of developments over existing one. Since the developers get a new set of requirements at the end of each iteration, one can apply the standard set of Scrum practices with little or no modification. Defect Fixing projects involve fixing defects on closed or current projects not in development. Sometimes these projects are boring, especially if a new team has been hired for defect fixing purposes only. The customer sends a set of defects on a daily basis or weekly basis with a deadline to deliver. The development team needs to fix them ASAP and send the patch for further testing. While coaching one of such a defect fixing projects, I found that the following Scrum practices can be applied without much modification: 1. Daily Scrum meetings 2. A Scrum of Scrum 3. Modeling days while solving complex defects 4. Information radiators displaying InProgress, completed, reopened, closed defects and other information 5. Usage of Wiki for collaborating with the customer 6. Requirement workshop while understanding complex defects 7. Review and Retrospective A common problem that I have found in defect fixing projects is setting the iteration length. Especially if the defects are given on a day to day basis without prior knowledge of what you are going to get, it makes the life of the development team bit difficult. This can be solved by collaborating with the customer and coming up with a plan to have 1 or 2 weeks of iteration length.

While lying in bed recovering from an injury a few years ago, I was stumbling around through the myriad of video podcasts I subscribe to and decided to take a look at some of the videos in The George Lucas Educational Foundation Integrated Studies series. That's where I came across this gem featuring Pixar's Randy Nelson who is the Dean of Pixar University. It has had an extremely profound impact on how I think and collaborate. He's giving a short talk entitled Learning and Working in the Collaborative Age at the Apple Education Leadership Summit in April of 2008. Take a look: In his very casual and easy style, Nelson starts off by talking about how PIxar uses improv as a method of collaboration. In that method, two principles have surfaced that have guided Pixar: Accept every offer. You don't know where that offer is going to go. But one thing is for sure: If you don't accept that offer, it's going nowhere! So you have a sure thing on one hand: a dead end. And you have possibility on the other. Make you partner look good. That means that everybody on your team is going to try to make you look good and vice versa. And it's not about judgement or saying "This is pretty good. How can I make it better?". It's about saying "Here's where I'm starting. What can I do with this?". Nelson calls this "plus-ing". I passed this video along to my friend Bert Decker, CEO of Decker Communications, to get his take on this as it is right up his alley. Here's what Bert had to say: "Randy talks about ‘plus-ing’. Sue [Walden ofImprovWorks] calls it “yes, and...” What we mention in our advanced course is two essential rules of improv that you can apply to all communications, (and life for that matter) is: Always positive (yes, and...) Support your partner And of course there’s ‘forward lean’ but that comes even before improv...." Based on those two principles, Pixar looks to find people who are really good at something. And Pixar is really good at being innovative. So, how do you find people who are really good at being innovative? If something has never been done before and it's truly innovative, how do you find the people to do it. According to Nelson "You look for people who have seen failure and figured out how to make something from it. The core skill of innovators is error recovery not failure avoidance. We're looking for resiliency and adaptability." Wow, how many places think like this? I mean really think this way and not just pay the lip service. Not many trust me. It's so great to see a hugely successful organization express this attitude out loud and really mean it. What Pixar has realized is that a great predictor of innovation is mastery of something. It could be mastery of anything. The important thing is the personality that goes along with mastery. It's that sense of "I'm going to get to the top of that mountain" that you can use in your enterprise. It's called depth. Nelson goes on to say that given the fast pace of business these days, there's very little chance that people are going to achieve mastery on the job. You want them to be masters coming in the door. Another predictor of success is breadth. No one-trick ponies. We want to find people with lots of experiences (not necessarily "experience"). People with a breadth of experiences are deeply interested in many things. My favorite quote from Nelson: "We're looking for people who are interested...not interesting." Interested is tough, interesting is easy. Interested is a real skill. If you say "I've got a problem", interested people lean in. They amplify you. They want to know what YOU want to know. The notion of breadth leads to Nelson's third predictor, communication. Another awesome quote, especially for all of you developers and techies out there: "Communication involves translation." If you just emit tech, nobody really hears you. The translation gets pushed to the receiving end of the conversation and gets garbled. Do the translation at the SENDING end so that it doesn't have to be done at the receiving end and the listener can say, "I understand". So, no non-communicative techies! Nelson says that "Communication is not something the emitter can measure." You can't declare yourself as articulate or a good communicator...only your listener can. People who are interested are more likely to view communication as a destination rather than as a source. Nelson postulates that breadth and a broad range of experiences is the thing that fuels that. To me, this notion of communication as a destination not a source is extremely crucial to the success of teams comprised of so many different skillsets and levels of technical expertise. According to Nelson though, the most important predictor of success and innovation is collaboration. But what is collaboration? Real collaboration? It's not cooperation. We've been conditioned to jump to this answer very quickly. We all think "We have to cooperate to get our jobs done. That's collaboration." But, all this really means is we're not getting in each other's way. Nelson says that the things that get done in a cooperative enterprise could, in effect, all be done by one person if we had enough time and resources. He says that there is nothing in a cooperative workplace that job one does that can make job two better. Job one can prevent job two from getting done, but there's nothing job one can do to make job two better. Collaboration is not a synonym for cooperation. So what does collaboration mean if it's not about cooperation? Nelson says that collaboration for Pixar means AMPLIFICATION. It means connecting a group of individuals that are INTERESTED in each other, that bring separate DEPTH to the problem and that bring a BREADTH that gives them interest in the entire solution. And most importantly, it allows them to COMMUNICATE on multiple different levels: verbally, in writing, feeling, acting, pictures. In all of these ways, Nelson says: "They find the most articulate way to get a high fidelity notion across to a broad range of people so they can each pull on the right lever." I absolutely love this definition of collaboration and it's all rooted in a collective vision that everyone understands and can relate to. After listening to Nelson walk through these four points with passion and enthusiasm, it's no wonder why Pixar has been immensely successful in their endeavors. After a little digging and emailing, I found that indeed, Pixar's HR department uses all four of these predictors for the basis of their hires. They don't just look at a candidate's experience or resume. In a 2006 New York Times interview, Nelson said: "The problem with the Hollywood model is that it’s generally the day you wrap production that you realize you’ve finally figured out how to work together," Mr. Nelson said. "We’ve made the leap from an idea-centered business to a people-centered business. Instead of developing ideas, we develop people. Instead of investing in ideas, we invest in people. We’re trying to create a culture of learning, filled with lifelong learners. It’s no trick for talented people to be interesting, but it’s a gift to be interested. We want an organization filled with interested people." The things Nelson describes are intangible, you can't write them down. But when you talk with and work with people who possess these traits, you know who they are right away. And they're the kind of people you want on your team. Give me 10 people like this over 100 people with years of experience and you can do incredible things.

At my new job we’re reigniting an effort to move to continuous delivery for our software releases. We figured that we could learn a thing or two from Facebook, so we reached out to Chuck Rossi, Facebook’s first release engineer and the head of their release engineering team. He generously gave us an hour of his time, offering insights into how Facebook releases software, as well as specific improvements we could make to our existing practice. This post describes several highlights of that conversation. What’s so good about Facebook release engineering? The core capability my company wants to reproduce is Facebook’s ability to release its frontend web UI on demand, invisibly and with high levels of control and quality. In fact Facebook does a traditional-style large weekly release each Tuesday, as well as not-so-traditional two daily pushes on all other weekdays. They are also able to release on demand as needed. This capability is impressive in any context; it’s all the more impressive when you consider Facebook’s incredible scale: Over 1B users worldwide About 700 developers committing against their frontend source code repo Single frontend code binary about 1.5GB in size Pushed out to many thousands of servers (the number is not public) Changes can go from check-in to end users in as quickly as 40 minutes Release process almost entirely invisible to the users Holy cow. While the release engineering problem for my company is considerably smaller than the one confronting Facebook, it’s not by any means small. (Facebook is so massive that user bases orders of magnitude smaller than Facebook can still have nontrivial scale.) We don’t have to contend with the 1B users, 700 developers, 1.5GB binary or many thousands of servers. But we do want to be able to release on demand, quickly, reliably and invisibly to our users. How Facebook pushes twice daily to over 1B users The common thread running through the practices below is that they reject the supposed tradeoff between speed and quality. Releases are going to happen twice a day, and this needs to occur without sacrificing quality. Indeed, the quality requirements are very high. So any approach to quality incompatible with the always-be-pushing requirement is a non-starter. Here are some of the key themes and techniques. Empower your release engineers Chuck mentioned early on that the whole thing rides on having an empowered release engineering team. Ultimately release engineers have to strike a balance between development’s desire to ship software and operations’ desire to keep everything running smoothly. Release engineers therefore need access to the information that tells them whether a given change is a good risk for some upcoming push, as well as the authority to reject changes that aren’t in fact good risks. At the same time, we want release engineers that “get it” when it comes to software development. We don’t want them blocking changes just because they don’t understand them, or just because they can. Facebook’s release engineers are all programmers, so they understand the importance of shipping software, and they know how to look at test plans, stack traces and the code itself should the need arise. Empowerment is part cultural, part process and part tool-related. On the cultural side, Chuck introduces new hires to the release process, and makes it clear that the release engineering team makes the decision. As part of that presentation, he explains how the development, test and review processes generate data about the risk associated with a change. The highly integrated toolset, based largely around Facebook’s open source Phabricator suite, provides visibility into that change risk data. Just to give you an idea of the expectation on the developers, there are a number of factors that determine whether a change will go through: The size of the diff. Bigger = more risky. The quality of the test plan. The amount of back-and-forth that occurred in the code review (see below). The more back-and-forth, the more rejections, the more requests for change—the more risk. The developer’s “push karma”. Developers with a history of pushing garbage through get more scrutiny. They track this, though any given developer’s push karma isn’t public. The day of the week. Mondays are for small, not-risky changes because they don’t want to wreck Tuesday’s bigger weekly release. Wednesdays allow the bigger changes that were blocked for Monday. Thursdays allow normal changes. Changes for Friday can’t be too risky, partly because weekend traffic tends to be heavier than Friday traffic (so they don’t want any nasty weekend surprises), and partly because developers can be harder to reach on weekends. The release engineers evaluate every change against these criteria, and then decide accordingly. They process 30-300 changes per day. Test suite should take no longer than the slowest test When you’re releasing code twice a day, you have to take testing very seriously. Part of this is making sure that developers write tests, and part of this is running the full test suite—including integration and acceptance tests—against every change before pushing it. In some development organizations, one major challenge with doing this is that integration tests are slow, and so running a full regression against every change becomes impractical. Such organizations—especially those that practice a lot of manual regression testing—often handle this by postponing full regression testing until late in the release cycle. This makes regression testing more cost-feasible because it happens only once per release. But if we’re trying to push twice daily, the run-regression-at-the-end-of-the-release-cycle approach doesn’t work. And neither does truncating the test suite. We can’t give up the quality. Facebook’s alternative is simple: apply extreme parallelization such that it’s the slowest integration test that limits the performance of the overall suite. Buy as many machines as are required to make this real. Now we can run the full battery of tests quickly against every single change. No more speed/quality tradeoff. Code review EVERYTHING Chuck was at Google before he joined Facebook, and apparently at both Google and Facebook they review every code change, no matter how small. Whereas some development shops either practice code review only in limited contexts or else not at all, pre-push code reviews are fundamental to Facebook’s development and release process. The process flat out doesn’t work without them. As the session progressed, I came to understand some reasons why. One key reason is that it promotes the right-sizing of changes so they can be developed, tested, understood and cherry-picked appropriately. Since Facebook releases are based on sets of cherry picks, commits need to be smallish and coherent in a way that reviews promote. And (as noted above) the release engineers depend upon the review process to generate data as to any given change’s riskiness so they can decide whether to perform the cherry pick. Another important benefit is that pre-push code reviews can make it feasible to pursue a single monolithic code repo strategy (often favored for frontend applications involving multiple components that must be tested together), because breaking changes are much less likely to make it into the central, upstream repo. Facebook has about 700 developers committing against a single source repository, so they can’t afford to have broken builds. Facebook uses Phabricator (specifically, Differential and Arcanist) for code reviews. Practice canary releases Testing and pre-push reviews are critical, but they aren’t the entire quality strategy. The problem is that testing and reviews don’t (and can’t) catch everything. So there has to be a way to detect and limit the impact of problems that make their way into the production environment. Facebook handles this using “canary releases”. The name comes from the practice of using canaries to test coal mines for the presence of poisonous gases. Facebook starts by pushing to six internal servers that their employees see. If no problems surface, they push to 2% of their overall server fleet and once again watch closely to see how it goes. If that passes, they release to 100% of the fleet. There’s a bunch of instrumentation in place to make sure that no fatal errors, performance issues and other such undesirables occur during the phased releases. Decouple stuff Chuck made a number of suggestions that I consider to fall under the general category “decouple stuff”. Whereas many of the previous suggestions were more about process, the ones below are more architectural in nature. Decouple the user from the web server. Sessions are stateless, so there’s no server affinity. This makes it much easier to push without impacting users (e.g., downtime, forcing them to reauthenticate, etc.). It also spreads the pain of a canary-test-gone-wrong across the entire user population, thus thinning it out. Users who run into a glitch can generally refresh their browser to get another server. Decouple the UI from the service. Facebook’s operational environment is extremely large and dynamic. Because of this, the environment is never homogeneous with respect to which versions of services and UI are running on the servers. Even though pushes are fast, they’re not instantaneous, so there has to be an accommodation for that reality. It becomes very important for engineers to design with backward and forward compatibility in mind. Contracts can evolve over time, but the evolution has to occur in a way that avoids strong assumptions about which exact software versions are operating across the contract. Decouple pushes from feature activation. Facebook uses dark launches and feature flags to decouple binary pushes from the activation of features. The general concept is for the features to exist in latent form in the production environment, with a means to activate and deactivate them at will. Dark launches and feature flags further erode the speed/quality tradeoff. You can release code without activating it, giving you a way to get it out the door without impacting users. And when you do activate it, you have a way to turn it off immediately should a problem arise. These techniques also simplify source code management because you can just manage everything on trunk instead of having a bunch of branches sitting around waiting to be merged. Facebook uses an internally-developed tool called Gatekeeper to manage feature flags. Gatekeeper allows Facebook to turn feature flags on and off, and to do that in a flexibly segmented fashion. Recap and concluding thoughts I mentioned earlier that Facebook rejects the apparent tradeoff between speed and quality. At their core, the practices above amount to ways to maintain quality in the face of rapid fire releases. As the overall release practice and infrastructure matures, opportunities for further speedups and quality enhancements emerge. As you can see, our one hour conversation was packed with a lot of outstanding information. I hope that others might benefit from this material in the way that I know my company will. Thanks Chuck! Additional resources for Facebook release engineering Facebook publishes a great deal of useful information about their release engineering processes. Here are some good resources to learn more, mostly directly from Chuck himself. Push: Tech Talk – May 26, 2011 (video): This is a class that Chuck gives to new developers when they join Facebook. It’s just slightly out of date as Facebook now does two daily pushes instead of one. Outstanding information about release schedule, branching strategy, cultural norms, tools and more. Just under an hour but well worth the watch. Release engineering and push karma: Chuck Rossi: Interview covering some highlights of the Facebook release process and its supporting culture. Ship early and ship twice as often: Chuck explains how Facebook moved from a once-per-day push schedule to a twice-per-day schedule. Release Engineering at Facebook: Secondary source with highlights on the Facebook release process. Hammering Usernames: Facebook explains how they use dark launches to mitigate risk. Girish Patangay keynote Velocity Europe 2012 “Move Fast and Ship Things” (video) – Keynote by Facebook’s Girish Patangay describing some additional elements of the Facebook release process, including its use of a BitTorrent-based system to push a large binary very quickly out to many thousands of servers.