We’re all works in progress, honey. And believe me when I tell you that I’ve had to work harder than most. ― Susan Elizabeth Phillips, "Ain't She Sweet" It's pretty well understood that limiting Work In Progress - or WIP as it is often abbreviated - is a good thing. Ideally, WIP should be limited to one item in progress at a time. Having multiple pieces of inventory on-hand is a form of waste, since each incurs a handling cost, and any work done on one of them will depreciate while another is being worked on. In theory at least, restricting WIP to one item at a time will reduce this waste and get value out of the door as quickly as possible. This principle of Single Piece Flow (SPF) is central to Lean-Kanban ways of working, especially in a manufacturing context. In a software context the accepted WIP limits tend to be rather higher. It is often limited to one item per developer, such as by allowing each developer only one avatar to place on an item, and it can be reduced further if pair-programming is in use. As such, software teams might not often achieve SPF but the value of limiting WIP as far as possible is still understood. There are however problems in interpreting limited WIP in Scrum. This is because a Scrum board will often take the form of a task board ... not a Kanban board. In other words, the work being limited by Scrum teams is not always a user story or similar requirement. It is often a task. Task-limited WIP allows developers to progress tasks from any user story in any order. They could potentially limit themselves to one or two tasks from a story, complete them, then move on to a task from a different story and maybe a task from a third. In effect multiple stories - perhaps even the entire Sprint Backlog of stories - can be in progress before so much as one story gets completed. None of this breaks Scrum rules. There's nothing to stop a team, in Sprint Planning, from organizing the Sprint Backlog into any number of tasks which can be progressed in any order they choose, and from delivering all of the user stories in one go at the end of the Sprint. The Sprint Goal can of course be met by this approach, and there should still be a nice task burn-down to show the associated technical risks being managed. The problem is that it defers approval of each user story to the end of the Sprint (i.e. the Sprint Review), when it is best-practice to get continual sign-off by a Product Owner throughout the iteration. On-going inspection allows the business risks of delivery to be managed well, and not just the technical risks. This is an issue that all Scrum teams must consider when they formulate a Sprint Plan. Is it important to limit WIP in terms of user stories rather than tasks, and thereby facilitate early approval of those stories by a Product Owner? Or would this compromise the team's principle of incremental delivery ... and amount to Lean-Kanban by the back door?

Remembrance and reflection, how allied; What thin partitions sense from thought divide. - Alexander Pope Retrospectives, and why you need them A couple of months ago we looked at how to conduct a Sprint Review. We saw that a Review considers what work was done, and distinguished it from a Sprint Retrospective which reflects upon how work is being done. The distinction between the two can appear to be rather academic, and slurring a Review and a Retrospective together is a mistake that is often made by immature teams. After all, both take a reflective view of a Sprint that has just finished, and both can be said to fulfill a remit of historical inquiry. Yet while the separation of concerns might seem to be a narrow one, it is nonetheless quite precise, and there is great value to be had in maintaining the appropriate focus. A Review looks candidly at what has been achieved, and soberly at what remains to be achieved, with regard to product completion. A Retrospective on the other hand is an opportunity for the Scrum Team to inspect and adapt their actual implementation of the Scrum process. The rationale behind this inspection is methodological but it is in no sense abstract. It is grounded firmly in the desire to achieve worthwhile and practical reform. Perhaps there are certain working practices which the team can make more efficient, or which can otherwise be improved upon. If so, a Retrospective presents the ideal opportunity for those improvements to be discussed and brought into action. Failing to inspect and adapt in this manner will condemn a team to perpetual infancy and the repetition of past mistakes. Sprint Retrospectives help keep a Scrum team on the road to continual improvement. When these sessions are done well, team members will not be afraid to challenge the status quo, and will do so in a constructive and informed manner. The result will be an improved delivery of value – in fact, the sort of productivity gain that might well be identified in the Sprint Review we considered earlier. In this article we’ll switch our attention fully to Retrospectives, and examine the matter of how they should be conducted. Setting up a Retrospective As any event manager will tell you, the key to a successful gig lies in the preparation. Okay…I’ll concede that a holding a Retrospective isn’t as mammoth an undertaking as hosting the Thinking Digital conference, nor can it be said to demand the organizational skills of Bruce Springsteen’s road manager. Nevertheless it’s still important to get a few ducks in a row. Let’s start by lining them up and giving them some admittedly rather unimaginative names: Why, Who, Where, When, and What. We’ve just covered the issue of why a Retrospective needs to be held…that duck’s down. Let’s pop the rest. Who should attend a Sprint Retrospective? The invitation list for a Sprint Retrospective should be simple and uncontroversial. According to the Scrum Guide all Scrum Team members are expected to attend. That’s the Developers, the Scrum Master (who may facilitate the session), and the Product Owner. No others are expected. In fact, it would be quite irregular to extend the invitation to other people, even if they consider themselves to be important players or stakeholders. That’s because it is the Scrum Team who are responsible for the way they have implemented the Scrum Framework. Only they are in a position to inspect and adapt their very own ways of working. For this reason, all members have a duty to be present, to contribute, and to help make each Retrospective a success. Some teams exclude the Product Owner from this activity, arguing that if he or she was present, the team would not be able to have an open and frank discussion. This is a common issue and we’ll return to it later. For now though, just take it as read that a good Retrospective must include all Scrum Team members, and will give each a voice in molding the processes and working environment that they collectively own. Where should a Retrospective be held? Let’s answer this one with another question. If all of the Scrum Team members are co-located, and if they have the necessary equipment to hand (such as their Scrum board, plus a whiteboard for notes), why not hold the retrospective in situ? In other words, why not just hold the session at the team’s desks? Well, although this might sound like a capital idea, there can be problems. Perhaps it would create too much of a disturbance and annoy other teams within earshot? Then again, perhaps the physical layout of the working area is simply not conducive to holding a meeting. Perhaps the team is not entirely co-located in the first place. Any one of these things can tip the balance in favour of booking an actual meeting room, and getting everyone to decamp there for a Sprint Retrospective. If so, remember to book such a room in advance…if possible as a recurring appointment for the anticipated duration of the project. Make sure it has sufficient capacity and the resources needed. When should a Retrospective happen? The glib answer is to say that a Retrospective should happen “at the end of each Sprint”. A more useful answer would say whether or not it should precede or follow the Sprint Review. In my experience it is generally better to do the Review first, because that helps to establish a context within which the Retrospective can happen. The next thing to consider is how long to allow for the session. As with all Scrum events, a Sprint Retrospective is time-boxed. This means that it isn’t allowed to exceed a set length. The rules of Scrum are exact: for a one month Sprint the limit for a Retrospective is 3 hours, which is reduced to one-and-a-half hours for a two week Sprint. You should adjust this value by the same ratio if needed. Note that if a Retrospective finishes before the time-box expires, that’s fine and dandy. You aren’t obliged to use all of the available time. The rule is simply that the time-box must never be exceeded. Scrum is not a philosophy in which matters are allowed to drag on. What topics should the Retrospective cover? This is the biggest duck in the row, and it’ll take a few pings to knock it down. What we have to do is to establish a suitable agenda for a Sprint Retrospective. We have to formulate it in such a way that the inspection of the team’s Scrum implementation does indeed happen. We also have to make sure that any recommendations for its adaptation are elicited, agreed, and turned into achievable action items. The Scrum Guide provides us with something of a starting point. It isn’t much, but I reckon that if you look at it through a beer glass with your head sideways and one eye closed, you can just about discern a notional agenda for holding a Sprint Retrospective. A notional agenda The Scrum Guide is sparing in the advice it gives on how to conduct a Retrospective. We are told that a Scrum Team must: Inspect how the last Sprint went with regards to people, relationships, process, and tools; Identify and order the major items that went well and potential improvements; and, Create a plan for implementing improvements to the way the Scrum Team does its work…[including]…ways to increase product quality by adapting the Definition of “Done” as appropriate. Yes, I know that’s not much to go on, but each of these items is clearly significant. They seem to address the very rubric of agile practice; we can recognize in them a succinct appeal to the three legs of Transparency, Inspection, and Adaptation. In them, we can see not only a notional agenda, but also how critical a Sprint Retrospective is to the Scrum process. A Retrospective is arguably the most important time-boxed event that any agile process can have. If we want to turn these points into a more formal agenda for the session, we’ll have to make sure that each of them is addressed carefully. Towards a canonical format Scrum has been around for well over a decade now, and a fairly standard agenda for conducting a Sprint Retrospective has emerged. Here’s what it looks like. Set the scene. Ways to do this can include any or all of the following: Sketching out a timeline of significant events that occurred in the Sprint, so its historical context can be established Holding the Sprint Review shortly beforehand, so the project context is fresh in attendees’ minds Declaring the Prime Directive in order to define a professional context of mutual respect and openness Assess prior action items. Unless this is the first sprint, there will have been an earlier retrospective in which some improvements will have been proposed. Look back over each of them. Have they been followed through? In short, has the process actually been adapted following that earlier inspection? If any action items remain undone, make a note of them. They’ll have to be considered when determining actions for the future. Set up a Retrospective Board. This can be a whiteboard, or even a large sheet of paper stuck to a wall. Divide it into four quadrants and label each in the following manner. The precise terminology does tend to vary a bit. There can be subtle and not-so-subtle differences in meaning (consider the difference between “good points” and “things to continue doing”). Be aware of these differences, as they will shape the responses and ultimately the results. “What went well” (or “good points”, or “things to continue doing”) “What didn’t go well” (or “bad points”, or “things to stop doing”) “Ideas for improvement” (or things to “start doing”) “Shout-outs” (i.e. recognition of noteworthy individual contributions) Storm the Board. There are several ways in which this can be done. Here are some of the more common ones: Sticky notes. This method is fairly democratic in that each attendee gets a clear say by putting sticky notes on a board. Assertive individuals are therefore less able to dominate others. However, it can be disjointed as attention shifts from one person’s topics to another person’s. As such, it can be hard to develop a line of thought for group discussion. Here’s the process: Blocks of notes are distributed to the attendees. They are given a small time-box (5 or 10 minutes) to jot down their ideas…good points, bad points, improvements, and shouts. Each attendee should write one point per sticky note. There is no limit to the number of points they can make. After the time is up, attendees take it in turn to put their notes on the board and in the relevant quadrants As an attendee puts their sticky note on the board, they briefly state what the point is to the rest of the team Once the last attendee has finished, duplicate points will be identified by the group and removed. Facilitator-as-arbitrator. In this approach a facilitator will act as a scribe for the group, and write their ideas on the board. Group discussion of ideas is encouraged, and the facilitator can arbitrate in the event of disagreement. The downside is that it can favor the more assertive type of individual who ends up doing most of the talking. Here’s how it’s done: The facilitator stands in front of the board with a marker pen Any attendee who has a suggestion to make will make it – a good point, bad point, idea, or shout-out The facilitator writes each suggestion into the appropriate quadrant, disallowing any duplicates. The group discuss the merits of each suggestion The facilitator will erase, keep, or revise each suggestion according to group opinion Hybrid. This uses a mix of techniques, such as a facilitated session for identifying good points and bad points, and a sticky-note approach in order to elicit ideas for improvement. Changing the techniques used in a Retrospective every now and then can help keep the sessions fresh, and is certainly a good idea if you reckon they are getting a bit stale. Propose actions. I have five rules that I apply when “storming the board” with a team: For every bad point there must be an idea for improvement. In other words, for everything that people are being asked to stop doing an alternative and better course of action must be proposed. This rule helps to keep attendees focused on the need to adapt the process constructively, and not to use the session for mere complaint. If you have been storming for “good points” rather than for things to “start doing”, make sure that each of those points is matched with an idea for further improvement. It isn’t enough to look back appreciatively whenever something positive has happened. Your challenge is to translate that observation into an even bigger future win. Re-assess undone action items from the previous Retrospective. If any remain undone, ask if they are worth bringing forward. Ask why they weren’t implemented, with a view to finding out what really needs to happen to expedite them. If these outstanding actions are impractical, or are no longer relevant, jettison them and concentrate on those improvements which are valuable and achievable. Ask the “Five Whys”. For each action item you produce, you need to be sure that you have understood the root cause and that the action will be appropriate. A shallow retrospective is no retrospective at all. It has to be deep and probing. Improve the Definition of Done. The Scrum Guide doesn’t provide much advice about holding Retrospectives, but it is quite clear about the need to revisit the Definition of Done. This is something that many teams, including some quite experienced ones, forget or otherwise fail to do. So be careful to identify any room for improvement in the team’s understanding of what “done” means, and what it should take for work to be considered potentially releasable. Vote. It’s quite possible that the list of proposed actions will be extensive. In aggregate they could amount to too much change if all were to be implemented in the forthcoming Sprint. You can resolve this by getting team members to vote on action items, so that only the most important ones are taken forward. For example, if the team can take forward five items, allow each attendee to vote for five of them. The most popular can then be actioned. Other observations Here are some other things to consider when conducting a Sprint Retrospective. Decide whether or not to precede it with the Scrum “Prime Directive”. This is an assertion which is meant to be said, in earnest, before each and every Retrospective. It isn’t mentioned in the Scrum Guide, but it is widely recognized and some teams do choose to recite it. “Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand” We considered the significance of this assertion in an earlier article on Agile Teamwork in Practice, so I’m not going to say much more about it here. However, Martin Fowler has expressed his thoughts on the Prime Directive, and I suggest you read his opinion piece in full. All I’ll add is that I am in agreement with his observations and that I share his sense of revulsion. Determine what to do about Product Owner representation. According to the Scrum Primer the Product Owner may attend a Sprint Retrospective. Only “Development Team” members are actually required to be there. Yet according to the Scrum Guide, all “Scrum Team” members must attend. The Scrum Team is a wider group than the Development team and includes the Product Owner. The reason for this discrepancy probably lies in the interpretation of process ownership. If we see the Development Team as owning the process through which iterative and incremental value will be delivered to a Product Owner, then the PO would not indeed have a say in the adaptation of that process. He or she would merely be a consumer of its outputs, and would therefore be a stakeholder in a Sprint Review but not in a Sprint Retrospective. However, if we view the process as a more collaborative one, in which the Development Team works with the Product Owner to deliver potentially releasable increments of value every Sprint, then the PO would indeed be a stakeholder in how that process is managed, and must therefore attend. It’s therefore important to determine what relationship the Development Team has, or should have, with the Product Owner. It’s unquestionably best if a Product Owner is on-side as a team player, and can handle root cause analysis and the exposure of potentially uncomfortable truths. Whether or not that is the case though is only something that the team can decide. Remember they’re human. Bring snacks and drinks to keep attendees refreshed, and allow enough time for breaks – at least 10 minutes every hour. Consider wrapping up the session with a “touchy feely graph” of some sort, which captures the mood and confidence of the team. Allow everyone to mark a dot or cross on a chart to show how positive or negative they feel about things, and then see how the mood changes…hopefully for the better…from one Sprint to the next. Conclusion A Sprint Retrospective is arguably the most important event that a team can hold. It provides the means to inspect and adapt the team’s actual implementation of the Scrum framework. In this article we’ve looked at how to create an agenda for the session and how to facilitate it, and at the issues of when and where it should be held, and who should attend. Those who cannot remember the past are condemned to repeat it. - George Santayana

Story points are a common name for sizing stories in agile projects. Combined with XpVelocity they provide a technique to aid planning by providing a forecast of when stories can be completed. When estimating work, a common approach is to estimate in terms staff-hours, such as a programmer saying "this will take me two days to do". Many people in the early days of agile, especially those in the ExtremeProgramming community, found that teams struggled to come up with useful estimates using this approach, even when they applied an approach of IdealTime. We found the most effective way to estimate was to size stories relative to each other, and then use past experience to determine how much could be done in an iteration. [1] To determine the points for a story, we compare rough relative sizes. If we are estimating the "fibble the foobar" story, we look for a story of similar size that we've already estimated. We sense it's about the same size as "flipping the synergy bit". Then we look at the story point score for "flipping the synergy bit" and score the "fibble the foobar" the same amount. A team using story points uses a small range of story points to work with. Common examples might be 1,2,4,8 or 1,2,3,5,8 [2]. Often the top number in the series represents "too big" and should be broken down further. [3] Allocating story points should be rapid activity. Discussion should only break out when people have contrasting views on the estimate, in which case its useful to have a discussion as it usually means that something about the story isn't clear. Using a ThrownEstimate is a good technique to move things along quickly. To form a plan with time, you use XpVelocity. Some teams don't like using story points, preferring instead to use StoryCounting. I don't have a preference between the two - both seem to work equally well so it's up to the team to try out and go with whichever suits them best. Further Reading The ThoughtWorks ebook on estimation provides includes a good Q&A on story points. Kent and I discussed them in more depth in the tasteful green book. Most books that talk about planning and estimation in an agile context discuss story points in more detail. Notes 1: "Story Points" is the most common name that I hear these days, but various terms have been used over the years, often with whimsical names that emphasized their arbitrary nature. I particularly like Joseph Pelrine's gummi bears and Josh Kerievsky's: NUTs (Nebulous Units of Time). 2: This is a Fibonacci sequence 3: Using the top number as too big is saying that a story sized at '8' really means '8 or more'. If you do this beware of using this top number when making forecasts of things like completion time, since '8' can turn into all sorts of numbers when it finally gets broken down. It's usually better to explicitly say its too big to be estimated rather than use a false marker number.

"A whole leisure day before you, a good novel in hand, and the backlog only just beginning to kindle..." - Backlog Studies, by Charles Dudley Warner A Recap on Backlogs A few weeks ago we took a critical look at Product Backlogs. We considered their purpose, how they are meant to be used, and why the aspirations they represent can so easily fall into a state of "Lost Remembrance". We also saw that a Product Backlog is an ordered list of requirements that are in scope, and if a project is to deliver value, then certain portions of that scope must be delivered in a timely manner. The Product Backlog is an instrument for managing this process. In short it is a queue, and one that is constantly tended and revised by a Product Owner. It is an artifact of diligent curation in which some requirements are determined to be more important than others, and which therefore ought to be delivered first. On the other hand some requirements will be observed to depend upon others, and must therefore be delivered afterwards. Introducing the Sprint Backlog In a very simple agile process - such as an elementary Kanban implementation - there will only be one backlog. Team members will action each item from the backlog in turn. They will be careful to draw only from the top of the queue, in order of priority. More sophisticated methods can include refinements such as “fast track” lanes in which the Quality of Service will be varied. We've already seen how this approach works in the context of managing critical incidents, and also in the context of hybrid agile methods such as Scrumban. Yet when we consider Scrum itself, we see that the Product Backlog is complemented by another of these queues...the Sprint Backlog. The idea is that if the team deliver something of value at regular intervals then the risks of the project can be better managed, and metrics can be generated that show progress towards its completion. Those regular intervals are known as Sprints. The chunk of requirements that the team agrees to work on during Sprint Planning is the Sprint Backlog. All of this is well known to agile developers, and the chances are that most of you reading this will have been working along these lines for years. So now let's challenge some common assumptions that are made about Sprint Backlogs and how a Scrum team is meant to handle them. Have any of these assumptions been made by your team? Assumption: The Sprint Backlog is a subset of the Product Backlog During Sprint Planning, a team will agree with the Product Owner which requirements from the Product Backlog will be worked on and met by the end of the forthcoming iteration. This has lead to the widespread practice of placing corresponding index cards into the Sprint Backlog on the Scrum board. In effect, it's a subset of the Product Backlog. What many teams fail to realize is that although the identification of an appropriate subset of Product Backlog requirements may be fine as a statement of intent, it can hardly be said to represent an actual plan for delivery. Admittedly a suitable plan doesn't have to be documented; it can live entirely in the developers' heads. A Scrum board's Sprint Backlog may indeed only show that subset of Product Backlog requirements which have been chosen for the Sprint. In fact the whole thing may look very like a Kanban board, even to the point that a casual observer might not be able to tell whether Scrum or Kanban rules are in force just by looking at it. The important thing is that a Sprint plan is agreed upon, shared, and understood by the team. Alternatively a task board may be used. Each selected requirement will be planned into tasks, and these will in turn be transcribed onto index cards or sticky notes. The tasks will move across the board in horizontal swim lanes that align each one to its parent requirement. In this model the Sprint Backlog is not represented by a subset of the Product Backlog, but rather by the corresponding tasks that have been planned for delivery. Assumption: A Sprint Backlog consists of tasks If we can see that each User Story has been broken down into tasks, it implies that some attempt has been made at Sprint Planning. It doesn't prove it of course. For all we know, each one of those tasks could have been identified by one person in the back of the pub last night. In other words, the tasks themselves do not amount to a plan. They merely infer by their presence that a team planning session is likely to have occurred, and that a team understanding regarding the delivery of the Sprint Goal has been reached. This means that a Sprint Backlog doesn't have to consist of tasks. It could be that “clean subset” of the Product Backlog we mentioned earlier, and therefore it might consist of User Stories. What matters is whether or not the team have a plan. While tasks imply that such a plan may have been formulated, they are not conclusive evidence of this, and they are certainly not the only way to compose a Sprint Backlog. Assumption: The Sprint Backlog is the Sprint Goal Identifying a meaningful Sprint Goal is usually the hardest part of Sprint Planning. Deciding how many User Stories can be accommodated, and what they should be, is comparatively straightforward. After all the team should know their budget. Time and again, Sprint Planning will boil down to horse-trading with the Product Owner over how many story points can be absorbed. “We've got 13 points left”, is a common refrain in Planning Poker. “We can't do that 20 pointer”. “OK”, says the Product Owner. “I'll bring forward a 5 and an 8 from the next Sprint”. While this satisfies the brutal arithmetic of planning, it does little to help create an increment of value. When the Sprint Backlog consists of disjointed requirements that don't play together as part of a cohesive potential release, the business value you might expect from such a release can hardly be delivered. Product Owners who expect otherwise are doing themselves and the product a disservice, and team members should not be party to such shenanigans. So, can each one of your team members articulate the goal for their current Sprint? Or is the “goal” just to deliver everything that's on the Sprint Backlog? A Sprint Goal is more than the sum of stories to be delivered or the tasks to be performed. It's about releasing business value incrementally and continually. Without that, the Product Owner probably has no idea when the project will reach completion. The common question “When will the project be done” is most often heard when incremental delivery is weak and the corresponding Sprint Goals are shoddy. Assumption: The Product Owner puts the Sprint Backlog in order This assumption is commonly held, but in Scrum terms it's plain wrong. The Development Team wholly own their Sprint Backlog, and it's up to them how they choose to order it. All the Product Owner should care about is whether or not the Sprint Goal is likely to be met by the end of the Sprint. This assumption is commonly held because Scrum is sometimes conflated with Kanban practice. In Kanban, there will normally be just one backlog and a Product Owner might well put it in order, and thereby exercise fine control over what gets delivered and when. Scrum is a different agile method and a different deal. In Scrum, value will be released at the end of the Sprint, not at discrete or arbitrary points within it. Granted, the Development Team should engage with the Product Owner throughout the Sprint, including on such matters as review and signoff, but the schedule for this is up to them. They decide, by creating their Sprint Plan, how the Sprint Backlog will be structured and how the corresponding work will be actioned. Assumption: Developers shouldn't cherry-pick from the Sprint Backlog This is a very good rule, but it is also one that is subject to misunderstanding. The underlying principle is a sound one. Agile teams should be fully cross-trained, and they should action work from a backlog as a team. Kanban team members, for example, should always take the next highest priority item from the backlog, assuming that there is no other work in progress or which is impeded and needs their attention. No team member should ever try and “pre-book” an item on the backlog, regardless of whether they simply want it or because they think they are best qualified to handle it. Each team member should go to where the work is, whatever that work is, and exactly when it needs doing. Scrum fully supports this principle but there is a further consideration that has to be borne in mind...a Scrum Development Team will have a Sprint Plan. When formulating this plan, they will self-organize to meet the Sprint Goal. That means it's quite possible for the team to decide up front, during Sprint Planning and subsequently during each daily Stand-Up, who will do what. It's important to be able to distinguish this behavior from cherry picking. It's also important for a Scrum Master to be able to smell a rat, and to sense when teams genuinely have a good plan or have started to cherry pick or to form undesirable skill silos. Assumption: A team commits to deliver everything in the Sprint Backlog This is a tricky assumption to deal with because until recently it was seen as being quite valid. For a long time, commitment-based planning was pivotal to a Scrum based way of working. Then, in 2011, the Scrum Guide was revised and the Sprint Backlog was positioned as a forecast of what a team could reasonably be expected to deliver. Clearly, a “forecast” is a weaker use of language than “commitment”. The rationale underlying this change is sensible. There are many things that can change during a Sprint, including requirements understanding or the perception of business value. Moreover, estimates are precisely that – estimates. There's something else we have to remember. The Development Team wholly own their Sprint Backlog. Regardless of whether they forecast their deliverables or commit to them, the content of this backlog is up to them and they can revise it at any time. It's the Sprint Goal, and the concomitant potential release of functionality, that is either committed to or forecast for delivery. Assumption: The Sprint Backlog cannot be changed once the Sprint has started This assumption is incorrect, although it is true that the Product Owner can't change the Sprint Backlog unilaterally. Only the Development Team can make such a change, because they wholly own it. If a Product Owner wishes to change something on the Sprint Backlog then that must be negotiated with the team. Now, let's also bear in mind that Scrum does not prescribe how the requirements within a Sprint Backlog are enumerated. User Stories, or the tasks to realize such stories, are the most common form of expression. Since User Stories do not document requirements exhaustively, but are placeholders for a future conversation, it follows that a change in understanding does not necessarily mean a change in the Sprint Backlog itself. Conclusion Sprint Backlogs mean different things to different teams. Some may populate them with tasks, while others may simply transfer over agreed User Stories from the Product Backlog. Either approach is acceptable given that the Development Team wholly own the Sprint Backlog. The important thing is that the team should have a plan for meeting a well defined Sprint Goal that has been agreed with the Product Owner, and they should form their Sprint Backlog in accordance with that plan. The backlog itself should never be mistaken for, or used in lieu of, a coherent goal for delivering a potentially releasable increment of value.

The following are some highlights from Dan North‘s excellent, inspiring, and insightful talk Patterns of Effective Delivery at RootConf 2011. North has a unique take on what agile development is, going beyond the established (and rather rigid) views. I really recommend this talk to learn more about effective teams, about North’s “shocking,” beyond-agile experience, and for great ideas on improving your team. The talk challenges the dogma of some widely accepted principles of “right” software development such as TDD, naming, and the evilness of copy/paste. However the challenge is in a positive way: it makes us think in which contexts these principles really help (in many) and when it might be more effective to (temporarily) postpone them. The result is a much more balanced view giving you a better understanding of their value. A lot of it is inspired by the theory (and practice) of Real Options. What are Patterns of Effective Delivery? Patterns – Strategies that work in a particular context – and not in another (too often we forget the context and to consider the context where a strategy doesn’t work / is counter-productive). Beware: a part of the context is the experience of the developer. For inexperienced devs it might be better to just stick to a process and apply TDD all the time instead of trying to guess when they it is appropriate and when it is not. Effective – Optimize for something: Volume of software produced? Time to market? Learning/discovery? Certanity? User experience? Any of these will work. Delivery – Get stuff that is useful out of the door. Software is not important, the utility it provides is. Know why you write the software. Some of the patterns take years to master and require significant investment before you start seeing the benefits. You might need to fail a few times before getting them right. Disclaimer: These are notes that make sense to me. They will likely make only limited or no sense to people that haven’t heard the talk. It would be best to go and listen to it instead. Selected patterns Spike and Stabilize (or throw away): traditionally we decide whether we are writing production-grade code (with high rigor such as TDD) or just a throw-away spike before we start coding – i.e. at the moment when we know the least about it. We should not decide this up front but “exercise the option of investing in the quality” later, based on experience. Start as a spike and if the code proves valuable, stabilize it, refactor, test etc. Evolve the code based on experience (good naming, quality). Defer the commitment to the quality of the code and optimize for learning An example of spike-and-stabilize regarding test naming: take a test originally named blah – you don’t know what it should do yet but you're experimenting. When the code evolves into something meaningful, name it properly, according to that. Ginger Cake – Copy and paste code, rip irrelevant things out until only the important things are left, then write tests around it. You may end up with code that is similar, but not in the ways you expected. If you started with abstracting, it would be the wrong abstraction. The pattern says: “We know and respect DRY but are not slaves to it.” Short Software Half-Life: 1) We don’t care about the software but the utility it gives us. If writing it gives us better ideas, we can delete it and do the better thing. 2) How would you write the code if 1/2 of it – but you don’t know which half – would be gone in a few weeks? The answer is, start simple (see Spike & St.), extract commonalities, improve quality, etc. For code that has already been around for a while and has proven itself useful; Some architecture styles lend themselves better to such quick evolution – such as small, focused services, popularly known as micro services (see slides, esp. p.42+). “Look at the code as it evolves and decide what to invest in.” (The investment includes thinking about the design.) All code is not equal. Create Urgency – To change a paradigm, the way of thinking, people must be desperate and have no more options along with the knowledge of what to do. Apply this pattern when learning something new. Do it on something real, under self-inflicted pressure. For example, you could commit to do an app with a crazy deadline using some new tech. This would give you a sense of urgency, with no more options. It forces you to learn only the parts you really need. Socratic Testing (coaching style) – Don’t tell the team what’s wrong with their code, which is threatening and thus hard to accept. Pair with them on writing a test and to support the test, make “helper” classes etc. that you’d like to see in the production code. If they really are useful, they will spot it and decide to pull them into the production code. Make them the hero. Respond to their questions with another question. Fits In My Head – we need code that we can understand and reason out (big classes, methods, complex models, etc.). Keep the code simple, optimize for understandability, readability, and obviousness. Build Shared Idioms in the team so that the team members would, given the same context, arrive at the same decisions/design. Something should only differ from the usual way of doing it when there is a good reason for it. Thus a difference provides a hint, difference is data. For example, putting all communication over ZeroMQ at only one place through shared memory. This indicates there is some (most likely performance) reason for it. Communication strategies shouldn’t be picked randomly or ad-hoc. TDD – A pattern that, in a particular context, may make you much more effective. Most of you reading this know what TDD is. Bonus: Micro Services Rough notes from James Lewis’s talk Micro Services: Java, the Unix Way (2013) – especially slides 42+: Use web, do not bypass it – REST, JSON; standardized application protocols and message semantics Small with a single responsibilities (does one thing, fits into one’s head, small enough to rewrite and throw away rather than maintain) Containerless and installed as well-behaved Unix services (executable jar with embedded Jetty + rc.d start scripts and config files) Avoid unnecessary coupling; Domains in different bounded contexts should be distinct; It's ok to have duplication with physical separation to enforce it; There will be common code, but it should be library and infrastructure code; Leverage Conway’s Law to support decoupling Provisioned automatically: “The way to manage the complexity of many small applications is declarative provisioning” (including instance count, scaling, load balancing) Status aware and auto-scaling; In-app status pages; monitoring to autoscaling Each service is entirely decoupled from its clients, scalable, testable and deployable individually Decomposition: This technique goes from product to a set of capabilities (e.g. monitoring, reporting, fulfillment, user) and then to each capability being implemented by a set of small apps/services exposing a uniform interface of Atom Collections. The capabilities form the project by interacting via a uniform interface – HTTP (reverse proxies etc.), HATEOS (link relations drive state changes; its an anti-corruption layer that allows the capability to evolve independently of its clients), and standard media types (usable by many types of clients). Explicit tips from the talk: Divide and conquer – Start on the outside and model business capabilities. Use Conway’s Law to structure teams (and enforce decoupling). The Last Responsible Moment – Don’t decide everything at the point when you know the least. Be of the web, not behind the web. If something is important, make it an explicit part of your design (reify) – an exampoe would be an instance of services creating users by posting to /user. They post a user creation request and get response immediately. The user is then created eventually (reminds me of futures). Favor service choreography over orchestration. Use hypermedia controls to decouple services. Some tools used: SimpleWeb/Jetty, Abdera for Atom, Smoothie charts (JS charts for streaming data), Coda Hale’s metrics, Graphite. Ops: Fabric with boto, AWS, Puppet, … . Remember there are NO SILVER BULLETS. This stuff is hard. Versioning, Integration, Testing, Deployment and eventual consistency are hard for people to wrap their heads around. Note: Comoyo.com, powered by a number of ex-googlers and other smart people, does the same thing. So does Netflix, I believe. Related If you liked this, you might also like Dan North's presentations Accelerating Agile: hyper-performing teams without the hype and Patterns of Effective Teams at NDC Oslo 2013.

How should a team respond to change? The simple answer is “they should respond by being agile”. If there’s one concept about agility that sceptical managers have caught onto it’s this one. When change happens, they expect that a truly agile team will be able to turn on a dime. You can hardly blame them, it sounds like a great idea. It suggests that perhaps managers don’t need to stabilise the working environment. They just need to pass change on. The teams will be able to deal with the impact…if they’re any good. After all, aren’t they meant to be agile? Of course, team members will have a rather different interpretation of this. They’ll tell you that agility isn’t about being reactive – it’s about responding to change in a controlled manner. With seemingly limitless demands on the team, and clearly finite resources, prioritisation becomes essential. Agile teams will work from an ordered backlog, and they’ll plan to deliver value by drawing work requests out of that queue. In other words they plan to follow an agile process…and that means things like “Sprint Planning” can still happen. So let’s ask the question again – how should a team respond to change? A better answer is “they should respond by following agile rules”. It isn’t about turning on a dime, it’s about following rules, and it’s important to understand how those rules differ between agile methods. Nowhere is this made more clear than in the way agile teams respond to a “P1 Incident”. It’s common in service management to rank incidents by priority. A P1 (Priority 1) is considered to be the highest – the business itself is threatened. When a P1 happens the expectation is that all hands will man the pumps. So what does that mean for an agile team that plays by the rules of backlog management? Well, in the case of a Lean-Kanban team, the response model is fairly straightforward. Priority incidents can be moved to the top of the backlog so they are actioned as soon as the next team member becomes available. Alternatively the quality of service can be varied. As soon as a P1 is raised a ticket (card) will be placed in a fast-track lane on the Kanban board. The team will stop what they are working on, mark their tickets as impeded, and swarm over the P1. Once a response has been planned those members who won’t be involved can return to their original work. A Scrum team has a different agile response. They’re rigged for the incremental de-risking of project scope, and plan to meet a Sprint Goal each iteration. If they have to drop everything for a P1, then that goal may no longer be achievable and the iteration could have to be terminated. Clearly they aren’t geared to be as immediate in their response as a Lean Kanban, but not all managers will understand or appreciate that point. Interestingly, some companies have dedicated “incident rooms” to which key personnel are expected to decamp should a P1 occur. These are clearly modelled on the incident rooms of the emergency services, the idea being that if responders are co-located then the crisis should be handled more efficiently. In an agile context however, they are something of an anti-pattern. In a genuinely agile organisation the responders will already be co-located along with the resources needed, and information radiators will be in place to keep stakeholders updated. As long as the agile models in use are understood a P1 incident can be handled without recourse to special measures.

"Don't tell people how to do things, tell them what to do and let them surprise you with their results" - General George S. Patton What's the best way to encourage agile teamwork? It's a tricky question, because so much of Scrum and Kanban practice is predicated on the assumption that collaborative behavior will "happen". Empowerment is often presented as the mechanism for achieving this success. If you just press the empowerment button, developers will then choose to self-organize and will go on to deliver sterling results. Patently however, that isn't the case. I'm sure that many of us will have experienced teams that are actually less than the sum of their parts. Technically skilled people can be more focused on stack traces than on individuals and interactions, and may view each other as unwanted complications or impediments. All too often the social graces that underpin effective agile teamwork have to be elicited painfully, like drawing teeth. Whenever I consider this matter, the above quote by Patton often comes into my mind. It isn't the perspicacity of his argument that I find compelling, or even that it was said so long ago. I suppose that these days we have just become more accepting of such observations. No...to me the interesting thing about this quote is that someone of Patton's background and temperament said it. You see, George Smith Patton was arguably the most hard-boiled U.S. General in World War 2. He was spit-and-polish to the core, and an absolute stickler for discipline. Even tiny misdemeanours would incur his wrath. His idea of a touchy-feely management style was to kick people in the pants after slapping them about the chops, and he frequently railed against "malingerers" who he reckoned ought to be court-martialed and shot. We have yet to hear Esther Derby or Johanna Rothman prescribe such remedies for disaffected team members. Perhaps the most politically correct thing we can do is to categorize his beliefs as an alternative viewpoint. Anyway, it's difficult to imagine anyone less likely than Patton to be sympathetic to agile principles, nor anyone more likely to try and micro-manage those they might consider to be their sub-ordinates. It seems we need a deeper insight if we are to explain this unlikely patronage of a central maxim of agile development. I suspect that Patton knew that if a team is to self-organize and deliver value successfully, then discipline will be key. It can't really be about empowerment, because an empowered team can still be sloppy and never cut the mustard. While good management isn't about telling people how to do their jobs, it is about making sure that they understand the rules of best practice and are competent to follow them, preferably with very little oversight. Strangely perhaps, this is a route to freedom rather than constraint. It releases individual initiative. I think that's what Patton was getting at. Who are we to empower others, after all? What gift is that? Where is the transfer of value? How much better it is to instil the best practices that make people more effective, and thereby become more valued themselves. Development Team Membership Now, a development team is made up of individuals, so when we talk about the rules of team membership we are largely talking about what those individuals do. More specifically, it's about what they do in respect to themselves, and with respect to the wider team of stakeholders including the Scrum Master and Product Owner. So before we go any further, let's look at the behaviors that we can expect a disciplined agile developer to exhibit. What a good team member will do: Agree with other team members and the Product Owner to deliver a valuable and achievable piece of work every Sprint Understand the Sprint Backlog and how it correlates to the Sprint Goal Participate fully and actively in daily standups, planning sessions, reviews, and retrospectives Work with the rest of the team to meet each Sprint Goal (self-organize) Help other team members and the Product Owner to clarify requirements, such as by writing user stories and acceptance criteria Pro-actively remove skill silos, such as by pair programming or cross training, and without being told to do so Work with the Product Owner on an ongoing basis, so that work is understood, reviewed, and approved continually Make sure that the work done is transparent, such as by updating Scrum and Kanban boards Understand that they, and all team members, are stakeholders in the agile process Estimate work so that the Product Owner and other stakeholders can plan ahead (e.g. for release planning) Fully support and encourage the elicitation of metrics, and be able to interpret them and act on them Resolve outstanding or impeded work before actioning new work from the backlog Limit work in progress so as to maximize throughput Act immediately on impediments by appraising other team members and the Scrum Master of any issues, and help to resolve them Accept personal responsibility for the team's success Accept personal responsibility for their work meeting the team’s Definition of Done What a good team member doesn’t do… Fail to give the best unpadded estimates that can be provided at the time. Estimates should be given and received in good faith. Cherry pick work from the Sprint Backlog. The backlog is owned by the team and must be actioned in accordance with the team's Sprint Plan. Attempt to work on more than one item at a time. A good team member will pro-actively limit work in progress. Expect somebody else, such as a Scrum Master, to update the Scrum board or Kanban boards. Information radiators are owned by the team. Work in a "skills silo". A good team member does not view their work as a speciality that only he or she is able to work on. Claim that work has been completed if it does not satisfy the team’s Definition of Done Claim that work is complete if it does not meet the specific acceptance criteria that have been agreed for it Shot at Dawn: Teamwork and the Prime Directive If we were to apply the Patton philosophy in extremis, I suppose that an agile team would shoot its own malingerers following a retrospective, the Scrum Master standing by to deliver the coup-de-grace if needed. Although this lurid concept is absurd, how many experienced Scrum Masters have never secretly wished for a revolver in their desks, even for just a fleeting moment? It highlights a problem that the agile community is often evasive about. What should actually be done about a developer who causes problems for the rest of the team? Is it possible, or even desirable, to correlate the occurrences of those problems to the individual concerned? In a Sprint Retrospective, for example, no blame is ever meant to be directed towards any one team member. In fact the format of the session precludes the establishment of such a correlation, or even the inference that a particular individual may have been remiss in some way. Known as the "Prime Directive", this article of faith is meant to be recanted at the beginning of each retrospective session, and it has to be said in earnest. "Regardless of what we discover, we understand and truly believe that everyone did the best job he or she could, given what was known at the time, his or her skills and abilities, the resources available, and the situation at hand." The question is: what if we don't believe it though? What if all the evidence in the world is stacked against it? Should we go along with the directive anyway, and just kid ourselves for the duration of the session? If so, how can it possibly help? Where is the transparency, which we covet in agile practice, if we subscribe to this devil's credo that makes a mockery of the truth? The answer is potentially quite shocking, and certainly little understood. Don't think of the Prime Directive as a creed, or even as the temporary suspension of disbelief for the sake of the meeting. Think of it as a pre-condition that must hold, and genuinely be true, before a retrospective can happen at all. The underlying principle is that all of the attendees must be fully able to participate. All are expected to be professionals who can fulfil their duty to each other and to the Scrum process, and inspect and adapt their working practices accordingly. It isn't enough just to leave your knives at the door. You actually have to trust the people you are working with. Really trust them. Given that most developers are assigned to their teams by managers, and not by each other, this expectation of trust is indeed potentially shocking. It gets even scarier than that. Think about what all of this really means should trust be absent, or somehow lost. It means that you can't have a Sprint Retrospective at all until the issues around trust are resolved. It means that if a team member must be removed, then that should happen beforehand. Scrum does not go so far as to prescribe a mechanism for this, but it is established that a team will self-organize to remove its own problems. Perhaps they will have to make collective representations to a line manager, or petition for a member's removal through the Scrum Master. It might even mean that the team can deselect a team member by their own consensus. Yet however it is done, it appears that the team aren't too far removed from assembling a firing squad after all. If this all seems very draconian, let's reassert the key principle here: when Scrum is done properly a team will solve its own problems, including distasteful matters like this. Now, it has to be admitted that most Scrum teams across industry today don't get to operate at such a high level of proficiency. The consequences of this cut both ways. On the one hand a team may not be allowed to get on with their jobs without interference from management, while on the other hand they usually don't have to deal with the nastiness of putting a sick dog down. A few conversations with that same pointy-haired boss could be enough to get him to do the deed. Yet as the industry transitions more fully towards agile practice, this "remedy" will no longer be sustainable. Problems regarding a team member's competence won't be someone else's responsibility; rather, it will be incumbent upon the team to find a solution. In an agile world, greater responsibility falls on self-managing teams, along with their greater rights. Professionalism: from Team Discipline to Self Discipline In this article we've identified a range of behaviors that typify good team membership, and we've looked squarely at what should happen when things go wrong. In short, it's up to the team to sort out its own problems when a team member doesn't measure up. Yet this is only part of what disciplined agile practice is about. It isn't enough to put the focus on punitive measures and the threat of sanction, even if the exercising of authority is driven entirely by the team itself. What we need to do is to take things a step further. We don't really want discipline to be enforced by the team, even though they should be the ultimate arbiters. What we want is to encourage a self-discipline that wells up from each individual team member, and which serves as an inspiration to others. Disciplined teamwork isn't about empowerment. It's about cascading the release of potential through the clear demonstration of value. I look at it this way. There is only one person in this world any of us can change. I don't think I need to spell out who that person is. So, wherever you and your team may be on your agile journey, there should always be at least one person who can be relied upon. If that person does their bit, then they are helping to make the team more than the sum of its parts. "Don't empower me. Release me. I'll find my own power, and it will be far greater than anything you can bestow on me" - Tobias Mayer

There are many good ideas and practices in Agile development, ideas and practices that definitely work: breaking projects into Small Releases to manage risk and accelerate feedback; time-boxing to limit WIP and keep everyone focused; relying only on working software as the measure of progress; simple estimating and using velocity to forecast team performance; working closely and constantly with the customer; and Continuous Integration – and Continuous Delivery – to ensure that code is always working and stable. But there are other commonly accepted ideas and best practices that aren’t important: if you don’t follow them, nothing bad will happen to you and your project will still succeed. And there are a couple that you are better off not following at all. Test-Driven Development Teams that need to move quickly need to depend on a fast, efficient testing safety net. With Test First Development or Test-Driven Development (TDD), there’s no excuse for not writing tests – after all, you have to write a failing test before you write the code. So you end up with a good set of working automated tests that ensure a high level of coverage and regression protection. TDD is not only a way of ensuring that developers test their code. It is also advocated as a design technique that leads to better quality code and a simpler, cleaner design. A study of teams at Microsoft and IBM (Realizing Quality Improvement through Test Driven Development, Microsoft Research, 2008) found that while TDD increased upfront development costs between 15-35% (TDD demands developers change the way that they think and work, which slows developers down, at least at first), it reduced defect density by 40% (IBM) or as much as 60-90% (Microsoft) over teams that did not follow disciplined unit testing. But in Making Software Chapter 12 “How Effective is Test-Driven Development” researchers led by Burak Turhan found that while TDD improves external quality (measured by one or more of test cases passed, number of defects, defect density, defects per test, effort required to fix defects, change density, % of preventative changes) and can improve the quality of the tests (fewer mistakes in the tests, tests that are easier to maintain), TDD does not consistently improve the quality of the design. TDD seems to reduce code complexity and improve reuse, however it also negatively impacts coupling and cohesion. And while method and class-level complexity is better in code developed using TDD, project/package level complexity is worse. People who like TDD like it a lot, so if you like it, do it. And even if you are not TDD-infected, there are times when working test first is natural – when you have to solve a specific problem in a specific way, or if you’re fixing a bug where the failing test case is already written up for you. But the important thing is that you write a good set of tests and keep them up to date and run them frequently – it doesn't matter if you write them before, or after, you write the code. Pair Programming According to the VersionOne State of Agile Development Survey 2012, almost 1/3 of teams follow pair programming – a surprisingly high number, given how disciplined pair programming is, and how few teams follow XP (2%) or Scrum/XP Hybrid (11%) methods where pair programming would be prescribed. There are good reasons for pairing: information sharing and improving code quality through continuous, informal code reviews as developers work together. And there are natural times to pair developers, or sometimes developers and testers, together: when you’re working through a hard design problem; or on code that you’ve never seen before and somebody who has worked on it is available to help; or when you’re over your head in troubleshooting a high-pressure problem; or testing a difficult part of the system; or when a new person joins the team and needs to learn about the code and coding practices. Some (extroverted) people enjoy pairing up, the energy it creates and the opportunities it provides to get to know others on the team. But forcing people who prefer working on their own or who don’t like each other to work closely together is definitely not a good idea. There are real social costs in pairing: you have to be careful to pair people up by skill, experience, style, personality type and work ethic. And sustained pair programming can be exhausting, especially over the long term – one study (Vanhanen and Lassenius 2007) found that people only pair between 1.5 and 4 hours a day on average, because it’s too intense to do all day long. In Pair Programming Considered Harmful? Jon Evans says that pairing can have also negative effects on creativity: Research strongly suggests that people are more creative when they enjoy privacy and freedom from interruption … What distinguished programmers at the top-performing companies wasn’t greater experience or better pay. It was how much privacy, personal workspace and freedom from interruption they enjoyed,” says a New York Times article castigating “the new groupthink”. And in “Still Questioning Extreme Programming” Pete McBreen points out some other disadvantages and weaknesses of pair programming: Exploration of ideas is not encouraged, pairing makes a developer focus on writing the code, so unless there is time in the day for solo exploration the team gets a very superficial level of understanding of the code. Developers can come to rely too much on the unit tests, assuming that if the tests pass then the code is OK. (This follows on from the lack of exploration.) Corner cases and edge cases are not investigated in detail, especially if they are hard to write tests for. Code that requires detail thinking about the design is hard to do when pairing unless one partner completely dominates the session. With the usual tradeoff between partners, it is hard to build technically complex designs unless they have been already been worked out in a solo session. Personal styles matter when pairing, and not all pairings are as productive as others. Pairs with different typing skills and proficiencies often result in the better typist doing all of the coding with the other partner being purely passive. And of course pairing in distributed teams doesn't work well if at all (depending on distance, differences in time zones, culture, working styles, language), although some people still try. While pairing does improve code quality over solo programming, you can get the same improvements in code quality, and at least some of the information sharing advantages, through code reviews, at less cost. Code reviews – especially lightweight, offline reviews – are easier to schedule, less expensive and less intrusive than pairing. And as Jason Cohen points out even if developers are pair programming, you may still need to do code reviews, because pair programming is really about joint problem solving, and doesn’t cover all of the issues that a code review would. Back to Jon Evans for the final word on pair programming: The true answer is that there is no one answer; that what works best is a dynamic combination of solitary, pair, and group work, depending on the context, using your best judgement. Paired programming definitely has its place. (Betteridge’s Law strikes again!) In some cases that place may even be “much of most days.” But insisting on 100 percent pairing is mindless dogma, and like all mindless dogma, ultimately counterproductive. Emergent Design and Metaphor Incremental development works, and trying to keep design simple makes good sense, but attempting to define an architecture on the fly is foolish and impractical. There’s a reason that almost nobody actually follows Emergent Design: it doesn't work. Relying on a high-level metaphor (the system is an "assembly line" or a "bill of materials" or a "hive of bees") shared by the team as some kind of substitute for architecture is even more ridiculous. Research from Carnegie Mellon University found that … natural language metaphors are relatively useless for either fostering communication among technical and non-technical project members or in developing architecture. Almost no one understands what a system metaphor is any ways, or how it is to be used, or how to choose a meaningful metaphor or how to change it if you got it wrong (and how you would know if you got it wrong), including one of the people who helped come up with the idea: Okay I might as well say it publicly - I still haven't got the hang of this metaphor thing. I saw it work, and work well on the C3 project, but it doesn't mean I have any idea how to do it, let alone how to explain how to do it. Martin Fowler, Is Design Dead? Agile development methods have improved development success and shown better ways to approach many different software development problems – but not architecture and design. Daily Standups When you have a new team and everyone needs to get to know each other and more time to understand what the project is about; or when the team is working under emergency conditions trying to fix something or finish something under extreme pressure, then getting everyone together in regular meetings, maybe even more than once a day, is necessary and valuable. But whether everyone stands up or sits down and what they end up talking about in a meeting should be up to you. If your team has been working well together for a while and everyone knows each other and knows what they are working on, and if developers update cards on a task board or a Kanban board or the status in an electronic system as they get things done, and if they are grown up enough to ask for help when they need it, then you don’t need to make them all stand up in a room every morning. Collective Code Ownership Letting everyone work on all of the code isn't always practical (because not everyone on the team has the requisite knowledge or experience to work on every problem) and collective code ownership can have negative effects on code quality. Share code where it makes sense to do so, but realize that not everybody can – or should – work on every part of the system. Writing All Requirements as Stories The idea that every requirement specification can be written as User Stories in 1 or 2 lines on cards, that requirements should be too short on purpose (so that the developer has to talk to someone to explain what’s really needed) and insisting that they should all be in the same template form “As a type of user I want some goal so that some reason…” is silly and unnecessary. This is the same kind of simple minded orthodoxy that led everyone to try to capture all requirements in UML Use Case format with stick men and bubbles 15 years ago. There are many different ways to effectively express requirements. Sometimes requirements need to be specified in detail (when you have to meet regulatory compliance or comply with a standard or integrate with an existing system or implement a specific algorithm or…). Sometimes it’s better to work from a test case or a detailed use case scenario or a wire frame or some other kind of model, because somebody who knows what’s going on has already worked out the details for you. So pick the format and level of detail that works best and get to work. Relying on a Product Owner Relying on one person as the Product Owner, as the single solitary voice of the customer and the “one throat to choke” when the project fails, doesn't scale, doesn't last, and puts the team and the project and eventually the business at risk. It’s a naïve, dangerous approach to designing a product and to managing a development project, and it causes more problems than it solves. Many teams have realized this and are trying to work around the Product Owner idea because they have to. To succeed, a team needs real and sustained customer engagement at multiple levels, and they should take responsibility themselves for making sure that they get what they need, rather than relying on one person to do it all.

A couple of weeks ago we looked at how to do a quick "health check" of an agile team. We saw that a great deal can be learned just by attending one of their daily stand-ups and by inspecting the state of their Scrum and Kanban boards. Of course these are nothing more than cursory examinations, and serious ailments can lie behind an apparently robust façade of agile practice. If you have reason to believe that a team is dysfunctional, you might have to dig deeper than the superficial evidence suggested by its apparent morphology. In my experience the next thing to examine is the team's "Definition of Done". This is the standard to which all work is put before it can be considered to be complete. Each team is collectively responsible for its own Definition of Done. It's up to them to make sure that it is adequate, and that it is applied by all members to all of the work they do. Without such professional oversight there can be no assurance that any deliverable will truly be fit for release. A spiffy stand-up and a cracker of a Kanban board might suggest rude team health, but they are no guarantee that the Definition of Done is solid, or that it isn't being undercut by someone along the way. Technical debt and rework are the main symptoms to look for. The consequences of backsliding on a Definition of Done might not become apparent until long after the events that caused it. By then, that rework or debt can be difficult to trace to the specific behaviors of those who cheated the system. You see, unfortunately a Definition of Done is a bit like personal hygiene. If there is no oversight, everyone can pretend that they are following the rules for the sake of the team, even though the presence of E. Coli on the office keyboards will tell its own tale about compliance. Everyone knows that it has to be coming from somewhere, but won't admit to their own liability or involvement, perhaps not even to themselves. Just as team vomiting will follow one member's dubious hand-washing practices, a short-changed Definition of Done will lead to rework by the team or the creation of technical debt. This is why team ownership and enforcement of what "Done" means is so important. An effective Definition of Done has to be founded on a healthy balance between due diligence and professional trust. What does this mean for agile development? You can think of a Definition of Done as the key defensive bulwark in software development epidemiology. If you balance the right level of team oversight with the right level of trust, severe outbreaks of technical debt or rework will become rare. High levels of oversight may be needed to start with, since team members might not have bought in to the idea of "done" yet. Once people become conditioned to do the right thing and see themselves as stakeholders in the team and its success, the balance can swing more towards trust. People become reluctant to renege on a team investment if they can see that it adds value for everyone including them. What's more, a Definition of Done improves the more it is respected, and becomes better respected the more it improves. In terms of agile best practice a Definition of Done will be used to determine whether or not User Story implementations are release-ready. However, each team can implement many User Stories over the course of a sprint, and making sure that all of these stories meet the Definition of Done can be challenging. Teams that are new to agile methods often have more modest ambitions. For example, their Definition of Done may only extend as far as delivery into a pre-production environment. Of course, anything less than "fully release ready" incurs the risk of technical debt and the need to pay it back later. Yet like a sloppy approach to hand-washing, it has to be admitted that something is better than nothing at all. Applying a Definition of Done consistently to even a sub-optimal standard will at least permit the delivery of each User Story to a known level of quality. It might not be great but at least it's there, and it's something that can be built upon and improved. The Lessons of Lean-Kanban Lean-Kanban methodologies have an instructive relationship with the Definition of Done. In these approaches the optimization of the value stream is of great significance. Naturally though, if a value stream is to be optimized it must first be understood. This means breaking the stream down into multiple discrete steps that can be studied for bottlenecks and any other occurrences of waste. For example, "Work in Progress" can be broken down into finer-grained stations such as "In Development", "Peer Review", "QA Test", "Knowledge Transfer", and "In Deployment". Team members will be cross-trained and will move freely across those stations in order to expedite as smooth a workflow as possible. Now here's where it gets interesting. If a Lean-Kanban operation has multiple well-defined stations, the case for having a Definition of Done can seem rather harder to make. After all, by the time a User Story gets to "Done", you already know that it has gone through the key steps you care about in the development process. What value can a Definition of Done really add in such a situation? Doesn't it just become waste itself? To find the answer, we need to look back to the manufacturing roots of Lean-Kanban. In a car plant for example, the steps of construction are exceptionally well defined and team members can move freely over several dozen stations. Some of those stations will be for the chassis, others for the interior, others for the engine block and electrics and so on. Yet despite this the Definition of Done will be an absolute corker, and much of the process of verification will be automated. Each station might even have its own Definition of Done so inspection can occur as close as possible to the point of action and potential remedy. The total number of checks that happen before each vehicle leaves the factory will be exhaustive. Why is this thought to be necessary? Because the manufacturers know perfectly well that the verification of "done" adds value. Merely having well-defined stations is no guarantee that everything will be done well. Quality is built in and validated by inspection. One thing's for sure: no-one in IT should accuse car manufacturers of having a weak understanding of what "done" means. The Definition of Done versus Acceptance Criteria However, software projects have a wild-card to deal with that car manufacturers don't have to worry about. Unlike the car doors and carburettors that roll down an assembly line, each User Story is different and has to be treated as a special case. To deal with this, each User Story has Acceptance Criteria that are agreed by the team members and the Product Owner as part of a Sprint Planning Session. Acceptance Criteria have to be quite specific to particular User Stories, because each story can be unique. The Definition of Done, on the other hand, applies to all of the User Stories being worked on by a team. The associated conditions must be invariant. For example, if all work has to be peer reviewed and subjected to QA testing prior to release, then those criteria would be enumerated in the Definition of Done rather than being repeated in each User Story's Acceptance Criteria. If the definition is enforced properly, a developer could never claim that a User Story was “Done” if it hadn't both been reviewed and passed QA testing. Writing a Definition of Done The Scrum Guide describes a Definition of Done as a "shared understanding of what it means for work to be complete". No process is suggested for writing a Definition of Done, nor in fact is there any suggestion that one should be written down at all. However, a documented definition may go some way towards providing that shared understanding. Here's how you can set about eliciting one: Review Acceptance Criteria: Gather the Acceptance Criteria for work completed so far Look for common criteria that can be abstracted out and applied across work in general Use these common criteria as the basis for a Definition of Done Assess Technical Debt Identify any rework that needs to be done Identify the reasons why it wasn't done properly the first time Identify what measures can be put in place to stop similar rework from occurring Add these measures to the Definition of Done (DoD) Continually update the DoD In each Sprint Review, identify which (if any) work was rejected or which caused rework to be done, then In each Sprint Retrospective, challenge the DoD for relevance and completeness There isn't a prescribed format for a Definition of Done, but it can be beneficial to use the same as that which is used for Acceptance Criteria, either in whole or in part. This allows a flexible definition based on story type. Alternatively they can be written as simple lists. Here are some examples: Example of a Definition of Done in Acceptance Criteria Format Given that a user story has required a code change When BDD and unit tests have been written for the story and the code change has been reviewed and the code change has been approved by a peer and all BDD and unit tests have been run and no BDD or unit tests have broken (green bar) and the code change has been committed to the repository and QA testing has passed satisfactorily and the Product Owner has approved the change Then the user story will be deployed to production and it will be considered Done. Given that a user story has required the authoring of documentation When the documentation has been reviewed and approved by a peer and the documentation has been approved by the Product Owner Then a new version of the documentation will be committed and the user story will be considered Done. Example of a Definition of Done in List Format Code changes: BDD tests written and pass Unit tests written and pass Code peer reviewed & approved Code committed to repository QA testing done Product Owner signed off Documentation: Documentation has been peer reviewed & approved Documentation approved by Product Owner Version number updated Definitions of Done for IT Infrastructure Support We've seen that having a good Definition of Done is important, although in IT we also need Acceptance Criteria that address the particulars of each User Story. When used in combination they can approach the levels of rigor that have been shown to be possible in manufacturing. Those working in software development can adopt a similar commitment to quality. Now we need to turn our attention to another function within the IT department...Infrastructure Support. Infrastructure support teams are increasingly expected to work in an agile way. As part of an enterprise transformation that does not seem unreasonable. After all, the rest of the organization is highly dependent upon them. Their scope includes such things as deploying new workstations and laptops (possibly across entire sites), installing networks, performing miscellaneous diagnostics and repairs, and maintaining and upgrading local server resources. Clearly they will also need Definitions of Done and Acceptance Criteria if they are to be stakeholders in a joined-up agile enterprise. The question is, how on earth can a meaningful Definition of Done be abstracted across wildly different physical tasks? How can a Definition of Done cover everything from a phone installation to a printer driver upgrade or a memory enhancement, or a firewall configuration to a keyboard replacement? The answer is to focus on the value chain that is represented by each user story. Work is not "released" as such, but rather it is handed over to someone who can derive benefit from it (i.e. the person occupying the user story role). This is the key to understanding "done" in an infrastructure context. If you can identify the parties who derive value, and demonstrably pass that value on to them, then your work is done. Here's an example of a Definition of Done that might be used to close out a support ticket: The receiver of the work has been identified Handover instructions have been completed and given to the receiver The receiver has been notified of the intention to close the ticket, and has not raised an objection A security assessment has been conducted and approved There are three things to point out here. The absence of any reference to a Product Owner. This is because infrastructure teams have to support multiple products, and prioritization of work is traditionally handled not through any sense of ownership of those products, but rather through help-desk triage. It's certainly possible for work to be represented by Product Owners, but it would have to be ownership of the business support function rather than ownership of the actual products being supported. The need to identify and work with the actual receivers of value is still there. The "acceptance by default" position. Receivers typically have little incentive to sign work off as being complete. On the contrary, their incentive is to defer acceptance as long as possible, for potential use as a "banker" in case a requirement for additional unforeseen work transpires. They might hope to ride this new work on an existing ticket instead of having to raise a new one. Receivers can be expected to care about their own support needs, not about the big picture of enterprise delivery. If a Product Owner can be identified - even if it is just the most likely owner of the business support function - then this situation can be improved. A Product Owner can apply leverage for appropriate and timely sign-off, such as by not accepting new work from certain parties while their approval (or justified rejection) of prior work remains outstanding. The elicitation of solid Acceptance Criteria can help the Product Owner immensely. Security implications need to be given careful consideration. The reworking of organizational infrastructure offers great potential for security to be compromised. Approval from Information Security should be obtained for all work, either directly or through authorized agents. One approach is for each team to have a designated "security champion" who provides this function. Conclusion Teams that appear to be healthy can still incur rework and technical debt. A poor understanding of what "done" means often underlies such problems, and this should be one of the first things to be looked at if problems are suspected. Having a meaningful Definition of Done encourages a team's sense of ownership of their own process, and helps instil self-discipline into its members to follow agile best practices. The application of this standard requires finding the right balance between team oversight and trust.

The longer I spend working as an agile coach, the more I find myself in disagreement with Hamlet. To estimate, or not to estimate? That is the question. Out of all of the agile practices which have been adopted in recent years, few have proven more controversial than this one. The battle for and against rages like Shakespearean armies set against each other's teeth. (Free Estimation Ebook) At first blush there doesn't seem to be any reasonable cause for disagreement. The rationale for making estimates is ostensibly straightforward. If a team is to work in Sprints, and to deliver something at the end of each one, then the work must surely be estimated. Otherwise how can the team know if it is even possible to do the work within the Sprint? How can they commit to deliver something by the end of that time-box if the effort involved is of uncertain magnitude? Well, there are two things that we need to draw out at this point. Firstly, the above rationale assumes that Sprints will be used, and that delivery will therefore be time-boxed. That's a very Scrum oriented philosophy...but Scrum isn't the only agile way of working. Lean-Kanban teams, for example, don't use Sprints and rarely make use of estimates. Secondly, Scrum itself says nothing about estimation. It only says that each item in a backlog must be sized - how that sizing happens is up to the team. It should also be remembered that a Scrum team commits to a Sprint Goal that delivers value, not to the delivery of a certain number of estimated points. So then...to estimate, or not to estimate? Let's listen in at the camp-fires of each side, and pick out in more detail the arguments they make for and against. For (Ye Scrum Brigade of Sprinte and Stande-uppe) "Estimates allow us to predict when a Sprint Goal will be met, and therefore when a substantial increment of value will be delivered" "Our estimates help our stakeholders plan ahead. They are part of the value we provide" "Estimates help us to de-risk scope of uncertain size and complexity" "Estimated work can be traded in and out of scope for other work of similar size. Without estimates you can't trade" "The very process of estimation adds value. When we estimate we discuss requirements in more detail, and gain a better understanding of what is needed" Against (Ye Lean Kanban Brigade of Boarde Pullers) "Estimates are rarely accurate. All you are doing when you estimate a piece of work is to set false expectations" "In practice, estimation is seen as a commitment, not as a best guess. Every time you make an estimate, you make a rod for your own back" "Estimation is time consuming. The time a team spends playing planning poker or whatever is time that could have been spent on delivery. Estimation is waste." "It's the actuals that matter, not estimates. Agility requires metrics, and the only metrics that count are those that reflect actual delivery" Both sides are right If you see this debate in terms of whether a Scrum or Lean-Kanban process is being followed, then both sides are right. A Scrum process is optimized for project work where scope risk is high and an entire system is represented. The requirements tend to be uncertain, complex, and very heavily intertwined. By committing to a Sprint Goal and to the delivery of a substantial increment of value, that risk can be managed. Uncertain and interdependent requirements are batched together into a Sprint and dealt with as a group. When this is done well, you have a clear Sprint Goal and a coherent Sprint Backlog. When it is done badly, you have a vague or disjointed Sprint Goal, a mishmash of requirements that command no sense of team purpose, and no team commitment towards the delivery of an increment. A Lean-Kanban process, on the other hand, is usually focused on "Business As Usual" (BAU) activities. The diet of a Lean-Kanban operation should consist of small and repeatable changes. They don't have to be related at all...in fact they shouldn't be. Things like bug fixes, minor enhancements, and administrative tasks are representative of this kind of work. Scope risk is low because the process of making such changes is well understood. Estimates are generally held to be unnecessary because there is very little uncertainty to deal with. There is no need for work to be batched...each change can be actioned and delivered independently of all others. Work is enqueued and actioned according to priority and the required quality of service. Predictions are based on the actual rate of delivery, not on estimates. In a Lean-Kanban way of working, the actuals are indeed everything. Methods of estimation So then, estimates add value where scope is uncertain and there are associated risks to be managed. That's why Scrum teams engaged on projects typically make use of them, but Lean-Kanban BAU teams generally don't. Now let's look at three simple methods of estimation that Scrum teams, or other teams doing project work, can make use of. Planning Poker This is a well established technique popularised by Mike Cohn, and variations on his Planning Poker cards can be found in offices across the world. A typical Planning Poker set has cards with the following numbers: ½ 1 2 3 5 8 13 20 40 100. Nerds will observe and be irritated by the fact that this is roughly (but not quite) in line with the Fibonacci sequence. Here's how it's played: An identical hand of cards is given to each team member. Each team member will have a set of cards with numbers on the above pseudo-Fibonacci scale. The Product Owner describes the piece of work to be estimated. Normally this is a user story with acceptance criteria. Each team member mentally estimates the size of it on the scale. They can ask the Product Owner questions to clarify any points, but for the moment they will keep their estimate to themselves. Each team member places the card that corresponds to their mental estimate face down in front of them. At the facilitators instruction - usually the Scrum Master - the team turn their cards over. In an ideal case the cards will all have the same value, suggesting that the team have a common understanding of the requirements and the likely effort that will be involved. If the values are different, the team then need to discuss their estimates and their reasoning behind them. They need to understand each other’s thinking, and from that reach a consensus. It may be necessary to replay the cards several times before agreement is reached. By convention, estimates are written on the corner of a User Story card before being placed on a Scrum board. A variation of this takes from a suite of regular playing cards. The Ace (1), 2, 3, 5, 8, Jack, Queen, and King might be used. The Jack signifies that no or negligible work needs doing (jack all). The Queen indicates a larger story that should be broken down in the planning session and reconsidered, while the King indicates an epic that will need greater analysis and cannot be brought into scope for this Sprint. The Joker can be played if anyone wants a coffee break. As an estimation method, Planning Poker has the advantage of being fairly democratic. Every team member gets a hand of cards and is allowed to play, and has a clear opportunity to explain their reasoning to the others. The disadvantage of Planning Poker is that it can be rather time consuming in comparison with other methods. It can also encourage novice teams to estimate in terms of time, as they are often initially prejudiced to correlate points to hours or days. This prejudice must be challenged and eroded if the relative sizing of estimates is to be achieved. Team Sort (T-Shirt Sizing) This is a good way of doing team estimates if no planning poker cards are available. All you need are six scraps of paper and a set of index cards with the requirements (e.g. user stories) written on them. Normally these will be the same index cards that go on the Scrum board. Write one of the following sizes on each of the scraps of paper: Extra Small (XS), Small (S), Medium (M), Large (L), Extra Large (XL), and Extra Extra Large (XXL) Arrange the sizes in a horizontal line on a table, ordered from XS on the left to XXL on the right. Put the pile of index cards on the table in front of the sizing line. The team then collaborate to organise the requirements on the cards under the headings XS to XXL. They can ask the Product Owner to clarify any questions that they may have while doing so. Once the cards have all been sorted, story points can be allocated to each of them by mapping each T-Shirt size to a value. This allows metrics to be gathered about the flow of work, and used to populate a velocity or burndown chart. T Shirt Size Suggested Story Point Value XS 1 S 2 M 3 L 5 XL 8 XXL 13 An advantage of the team sort is that it is quick and easy to do. The complete set of requirements is estimated in one sweep. Also, it is a fairly direct way of achieving relative sizing. There is no temptation to correlate points to hours. The disadvantage is that it is potentially undemocratic, in that assertive team members can dominate meeker ones with their opinions. There is a variant of the team sort which encourages more egalitarian behavior. Each team member takes it in turns to move one card by one position. They also have the option to pass, i.e. to not move a card. Eventually a consensus should be reached and no more cards will be moved. However this is a more time consuming method and deadlocks can occur. These deadlocks can be difficult to spot if multiple card shifts are caught in the cycle. One Point One Card This method is a spin on the Lean-Kanban approach of tracking actuals. Instead of estimating the relative effort required for each story card, the team estimates how many stories it is likely to complete in the Sprint being planned. This can be as straightforward as using the yesterday's weather analogy for velocity estimation. Just as the weather today is most likely to resemble the weather yesterday, the velocity that will be achieved by a team in the upcoming Sprint will most likely match the velocity of its predecessor. So if two dozen cards were completed in the last sprint, approximately two dozen can be expected in the one that follows. The budget can be adjusted to allow for holiday, foreseeable absences, and other such changes that will impact the team's commitment. The advantage of this system is its raw simplicity. The estimation overhead is almost negligible. Also, it encourages the authoring of small user stories that will spend little time in progress and that stand little chance of being impeded. The liquidity of the board is therefore increased and further requirements analysis is encouraged. Some variation in size will be inevitable, and there will be statistical outliers, but the effects of these will average out as the flow rate increases. The disadvantage of this technique lies in the separation of fine-grained user stories from business value. There is a significant risk that they will become excessively technically focused and task-like. Conclusion Agile estimation is often seen as being invaluable, yet others dismiss it as waste. The reasons for this disagreement can be traced to disparities in Scrum and Lean-Kanban ways of working, and to the fundamental differences between project work and Business As Usual. When seen in the context of Scrum projects, some form of estimation process is valuable. Yet regardless of the method chosen, it must be acknowledged that a Scrum Team is responsible for its own estimates. No-one else can make a team's estimates for them. Going through that process of estimation, and understanding the size and scope of the work, is fundamental to the team's sense of Sprint Backlog ownership and to their commitment to a Sprint Goal.

Richard Hundhausen has put together a great list of funny Scrum/Agile related videos. Some of these are classics such as High Moon Studios: Portrait of Scrum and Adam Weisbart’s Shit Bad Scrum Masters say. Be warned, not all of these are actually that funny. I’ve never found The Downfall of Agile Hitler to be funny, because the original film is harrowing and difficult to watch. I also find the Scrum Haka to be trite and unwatchable … this is what a real haka should look like. So here’s the list and, once you’re done (in the words of Adam Weisbart), “Get back to work!” I want to run an agile project http://www.youtube.com/watch?v=4u5N00ApR_k I want to run an agile project (part 2) http://www.youtube.com/watch?v=lAf3q13uUpE The Power of Scrum (Ian Sense Scrum Master) http://www.youtube.com/watch?v=P6v-I9VvTq4The making-of http://www.youtube.com/watch?v=ncjdtqf1gSg Developer Abuse http://www.youtube.com/watch?v=LYlhCGng5Mk Spooning and pair programming http://www.youtube.com/watch?v=dYBjVTMUQY0 Improving Sprint Reviews (is that Jeroen?) http://www.youtube.com/watch?v=fpBQ5yxrR_c The Downfall of Agile Hitler http://www.youtube.com/watch?v=l1wKO3rID9g High moon studios: Portrait of Scrum http://www.youtube.com/watch?v=UT4giM9mxHk Shit Bad Scrum Masters Say http://www.youtube.com/watch?v=GGbsgs611MM The Scrum Haka (hideous) http://www.youtube.com/watch?v=Qvqq97unS2w Joe Justice Team WikiSpeed http://www.youtube.com/watch?v=x8jdx-lf2Dw Deathstar Project Deployment Meeting http://www.youtube.com/watch?v=2T5QNcb_Z8g Raking Leaves – A Scrum/Agile Approach http://www.youtube.com/watch?v=StBS-loIIz4 I Need Agile Methodology http://www.youtube.com/watch?v=nvks70PD0Rs

Application Services Governance is a necessary step towards building a responsive IT organization and achieving business agility. By guiding teams through a streamlined application services development process, Application Services Governance Platforms optimize IT effectiveness, raise software quality, and reduce delivery timeframes. Governance relies on policy, people, process and technology to guide business activity and consistently deliver positive outcomes. Effective governance channels business activity towards the ‘right’ path; by making the right actions the path of least resistance. To efficiently guide teams and demonstrate policy compliance benefits, Application Services Governance Platforms provide policy management, developer portals, repositories, service integration and composition, and business value dashboards. Effective governance encompasses the entire IT solution spanning APIs, services, business processes, data, and application delivery. While most governance solutions focus on web services, leading Application Services Governance Platforms bridge API governance, SOA governance, Cloud deployment governance, data governance, and application delivery governance. Additionally, the governance experience must be tailored for the participant’s project role. Portals may be personalized to present notifications, tasks, actions, and reports suitable for application service creators, publishers, subscribers, consumers, or business managers. Application delivery governance segments participants into developers, quality assurance testers, operations, project managers, and application users. End-user Application Services Governance priorities are evolving toward bridging service governance with API governance, extending application lifecycle management to embrace cloud deployment environments, and focusing on visualizing asset business value. Key governance challenges include meeting mobile application demands, implementing efficient self-service provisioning, right-sizing governance practices (not too heavy or light), and defining appropriate policy tiers. Governance Components To efficiently guide teams and demonstrate policy compliance benefits, Application Services Governance Platforms provide policy management, developer portals, repositories, service integration and composition, and business value dashboards. Figure 1 Application Services Governance Components Policy Management Policy management is used to specify the correct behavior, detail exception thresholds, and define corrective actions or notifications. Leading application services governance platforms deliver advanced policy management by conforming to a flexible architecture, addressing relevant policy categories, and spanning all lifecycle phases. A comprehensive Application Services Governance Platform manages: Design-time Policy Run-time Policy Security Policy Developer access Policy Service and API Lifecycle Management Policy Application Lifecycle Management Policy Within these six broad categories, application services governance commonly encompasses service level policies, usage policies, version policies, subscription policies, and access control policies. Registries serve as policy stores for many types of runtime policies including security policies, lifecycle management workflow policies, API policies, service description, service contracts, service consumption, service usage, service lifecycle management, service level agreements (SLAs) and XACML authorization policies. Leading platforms have built-in support for a number of policy standards including WS-Policy, XACML 3.0, and SCXML. Cloud foundation and cloud middleware components deliver sophisticated run-time policy enforcement for tenant partitioning, service level management, application provisioning, tenant access, and resource management. All run-time infrastructure products should serve as well-integrated policy enforcement points that may delegate policy decisions to external decision points or internally cache and process policy assertions. Identity Management infrastructure components serve as a policy decision point and a policy manager for sophisticated security policies encoded in XACML. The Application Service Governance Platforms use workflow engines to execute governance workflow, present task lists, and manage approvals. Complex Event Processor components can be configured as policy decision points, which use time-based policy pattern matching to evaluate run-time service, message, REST resource, and event traffic. For more information on policy management, read the detailed policy management blog post. Developer Portal and Repository Portals serve as the viewport into policy management, service integration and composition, and business value dashboards. The Application Service Governance portals should deliver an application service governance experience tuned for self-service, on-demand access, and safe API usage. Developer portals are often contextually personalized to fit the project and user’s role. For example, a developer portal may fit the needs of API creators and API publishers who are defining, documenting, and publishing APIs. The portal’s user experience may enable API creators and publishers to monitor, manage, and analyze API usage. A developer portal may also be personalized to deliver a user experience tailored for API consumers. API developers who are consuming APIs can find, explore, subscribe and evaluate APIs. Developer portals are often tuned to facilitate service meta-data and lifecycle management for service creators. Service and integration developers who are consuming services can find and explore services. A developer portal should guide teams toward effective and efficient governance when building service implementation and service consumption code. Advanced developer portals capabilities include overlaying build management governance, test governance (i.e. unit, integration, performance), implementation lifecycle governance, and deployment governance. An Application Services Governance Platform should enable flexible organization, classification & documentation of services, APIs, and any IT asset. Key repository capabilities include governing and managing: Any type of metadata in any structure Service, API, or artifact associations and relationships Schema definitions and namespaces Users and Roles User subscriptions Service level agreements Developer documentation Social taxonomies (e.g. ratings, comments, tags) Implementation artifacts (i.e. code, test cases) Service Integration and Composition Service integration and composition for APIs, web services, or business process are often implemented using tools provided by the run-time infrastructure vendor. Application Services Governance components must integrate into diverse run-time infrastructure containers and development tooling. Synchronizing policy, development artifacts, and deployment packages requires tight integration between design-time tools, development tools, run-time management consoles, and application services governance portals and repositories. Business Value Dashboards To gauge governance effectiveness and enhanced business value, analytic dashboards assess policy compliance, quality of service, service usage, architecture coherence, and team performance. The Application Services Governance platform should capture service tier subscription information, collects usage statistics, and integrate with billing and payment systems that deliver show-back or charge-back reports. Subscription and usage reports help teams understand asset adoption (by version, by service) and usage (by version, by service). By understanding adoption and usage, business owners and architects can intelligently invest future development resources, properly plan infrastructure scale, and rationalize the portfolio. Dashboards also present a service overview, number of services, service lifecycle stage, schema re-use, service dependencies, upgrade impacts, development team productivity, and project progress. Governance Lifecycle Phases API management portals and SOA Governance Registries must work together to keep API lifecycle stages synchronized with backend service implementation stages. An API Governance experience may provide a straightforward set of lifecycle stages (e.g., created, published, deprecated, retired, blocked) that may be customized by the development team. SOA Governance Registries facilitates service metadata management and governance across design, implementation, test, and run-time operations. Figure 2 below depicts the intersection of the two governance views. Figure 2: API and Service Lifecycle Views Application delivery governance usually relies on ad hoc tools and processes, knitted together by end-user delivery managers. Application Services Governance Platforms should span project inception, development, quality assurance, production deployment, production management, maintenance, and retirement. Figure 3 illustrates service implementation activities governed by an application delivery governance product. Figure 3: Implementation activities governed by application services delivery governance Application Services Governance Drivers The IT focus on API, DevOps, and Cloud scale is driving resurgent interest in Application Services Governance. As development teams support mobile applications by fielding web APIs, they are creating a new ‘demand layer’ in front of existing service implementations. Both API and SOA success requires creating loosely coupled consumer-provider connections, enforcing a separation of concerns between consumer and provider, and exposing a set of re-usable, shared services, and gaining service consumer adoption. With traditional SOA Governance, many development teams publish services, yet struggle to create a service architecture that is widely shared, re-used, and adopted across internal development teams. In today’s connected business world, API and SOA are the business. An effective governance approach must address human collaboration stumbling blocks. By publishing managed APIs, establishing API manager and publisher roles, extending the governance registry, facilitating API management practices (e.g self-service key management, self-service provisioning, service tier management, and usage visualization),and offering APIs through developer portal, organizations can overcome collaboration, trust, and adoption hurdles while enhancing SOA success. By publishing managed APIs, establishing API manager and publisher roles, extending the governance registry, and offering APIs through an API Store, team have a new opportunity to increase service re-use and enhance IT business value. For more information on how teams can complement SOA Governance with API Governance, read the promoting services with API Management white paper. Because services are often imbedded in application solutions, leading Application Services Governance platforms wrap services governance inside application delivery governance. When operation team members use traditional point tools (i.e. Puppet, Chef, Jenkins,Selenium) to achieve DevOps benefits, the teams spend a considerable amount of time and effort creating agile workflow, effective governance, seamless activity transitions, and on-demand self-service access. A configurable DevOps PaaS can implement governance best practices and be readily adopted by teams without extensive implementation effort. Effective application delivery governance presents a simplified and unified user experience to complex development tools, processes, and team hand-offs. By integrating software promotion best practices, test automation, continuous integration, and issue tracking, application delivery governance raises software quality while reducing delivery timeframes. For more information, read about how to accelerate agility and maintain governance with DevOps PaaS. Recommended Reading Policy Management for Application Services Governance Application Services Governance Requires More Than a SOA Registry API and SOA Convergence Promoting services with API Management white paper Accelerate agility and maintain governance with DevOps PaaS Governance Registry Brings Integrity to SaaS Platform Gartner’s analysis of WSO2 SOA Governance

Here are 10 articles from 10 different authors that provide valuable advice for Scrum teams. These articles are in no particular order, so feel free to skim down the list and start with the ones that are most relevant to you. 10 Tips for a Great Daily Scrum Meeting by Platinum Edge – The daily Scrum meeting is a powerful tool that keeps your project moving. At the same time, it is also easy for the meetings to not bring any added value. Tips for Effective Backlog Grooming by Charles Bradley – Are you wasting time in your Sprint Planning Meetings? Increase the value of your team’s Sprint Planning Meetings by grooming your Product Backlog. Yoda’s top 10 tips for a new Scrum Master by Nigel Steane – As a new Scrum Master, you face unfamiliar challenges and your success is very much based on your ability to utilise coaching and soft skills to gently guide your team and colleagues. Top ten tips for distributed Scrum team teleconferences By Jon Archer – After acting as a Scrum Master for several months on a distributed team with people in six different locations, three different countries, learn ten tips to help get past those inevitable awkward silences. 10 tips for adopting Scrum to save your project by Matthew Hodgson – Are you interested in adopting Scrum for your next project? Here are 10 tips from his experience with moving a number of projects from their existing project management frameworks to Scrum. Five Tips for Impediment Resolution with Scrum by Stefan Roock – Impediments can slow down or even halt the progress of an otherwise well-functioning Scrum team. Take a look at the most common challenges that crop up on teams and what steps you can take to resolve them. 10 Tips for Succeeding with Enterprise Agile Development by Tools Journal – Many enterprises are experimenting with agile development approaches like Scrum, Kanban, Lean, and XP hoping that introducing a new development approach will help. Yet, agile development has struggled to achieve critical mass in large enterprises. 6 Tips for Good Scrum by Martin Harris – If you are doing these 6 tips, then you are doing very well and are likely to get better over time. 9 Tips for Creating a Good Sprint Backlog by Luciano Felix – Giving attention to the sprint backlog creation process is fundamental to the team’s understanding of what should be done and how to better plan during the sprint. 7 Tips for a More Effective Daily Scrum by Richard Lawrence – The main purpose of the Daily Scrum is for team members to make and follow-up on commitments to one another that work towards the team’s shared sprint commitment. Here are seven ways to get your Daily Scrum back on focus If your it has become unfocused, too long, or otherwise ineffective. If you have any other good articles related to agile, please share them in the comments. Thanks.

I've been working as a Scrum Master and as an Agile Coach for a good few years now, mainly as a contractor. Each time I am interviewed for a new contract I always like to ask if I can meet the teams I’d be working with. You see, time and again it will be a manager who does the interviewing, while the team members themselves are left with little say in whether I should be hired. I think it’s important that they reckon they can get along with me. Of course, it also gives me an opportunity to see them, and to gain a fuller understanding of the situation I’d really be walking into. As we head towards the desks of my prospective team, one of the first things I look for is the board, whether it be a Scrum task board or a Kanban board. Most teams with agile aspirations…or agile pretensions…will have set up a board of some kind. A board is the "grand old dame" of information radiators. No matter how much the details of a sordid past are glossed over, the truth always seems to come out. It's in the nature of a board to tell the truth, since any untruths can be quickly exposed. The story I can piece together from dubious lanes and columns, misplaced or missing tickets, misplaced or missing avatars, and a host of other shibboleths can be far more telling than anything I get to hear from people in an interview situation. Another of the things I look for is a "fast track" lane on a Scrum Team's board. These are very common; you could say it is almost unusual not to see them. From a certain perspective they are good things to have, and they can imply a level of maturity - or at least of pragmatism - on the part of a team. They suggest that the team accepts that not everything can be predicted in Sprint planning. A fast track lane is a nod to the fact that emergencies happen, that support work and unforeseen defect fixes still need to be done, and most importantly, that the team has a way of dealing with all of this. However it also shows that they aren't doing Scrum. There...I've said it. Fast track lanes aren't part of Scrum. It's that simple. I don't mean to say that they are bad practice, or in some sense un-agile. On the contrary, they are part of the Lean Kanban approach to varying the Quality of Service provided to certain backlog items. That's what a fast track lane is...a way of varying the quality of service that a Scrum team gives to certain items. When something hits a fast track lane, a well-trained team will swarm over it and decide who is best qualified to progress the matter. While they do this, their own tasks will be marked as impeded or blocked. Then, the decision made, all others return to their work in progress. So if fast track lanes are a widely understood and practical way of managing operational issues, what is wrong with them, Scrum-wise? The answer is that Scrum - unlike Lean Kanban - doesn't provide for variations in quality of service. Each piece of work is prioritized and negotiated into a Sprint backlog. The team then self-organizes to deliver a corresponding increment of functionality. The team will plan with the Product Owner what it intends to do during a sprint, and the sprint backlog they agree to belongs to them. No-one, not even the CEO of the organization, can override their sprint backlog by introducing work to be "fast tracked". The team wholly owns their sprint backlog. That's Scrum. When I point this out, teams can become crestfallen or even defensive. “What else are we supposed to do”, they say. “We aren’t dedicated 100% to doing project work. We still have support work to do, and serious issues always trump development. We have to fix them and put project work on hold.” My answer to that is that under the circumstances the team is facing, it may indeed be right to vary the quality of service by fast-tracking support work. It just isn’t Scrum, that’s all. It’s a type of "Scrumban", a Scrum variant that includes Kanban characteristics. This is no fault of the team, but it could suggest a problem higher up. Perhaps a dedicated Kanban support team hasn’t been properly resourced and trained so that Scrum development can proceed unimpeded. Perhaps the Product Owner is being undermined by other managers who have separate interests impacting the development. Whatever the situation, it needs to be made transparent and acknowledged by all stakeholders. So, the next step…and the one I’ll often indicate as the interview progresses…is to account for fast track work as impediments against product burndown or velocity. Moreover, these are impediments which are external to the team. It’s essentially a type of waste, or unplanned work, being generated from outside. It needs to be made quite transparent where this waste is coming from and what can be done to mitigate it. What can be done about those other teams, or workflows, or managers, who are undercutting this Scrum team’s ability to plan out their Sprints? Often, the source of these impediments will be the people interviewing me...and that’s when things can start to get really interesting!

With unit tests you can check that your code behaviours just as you expect it to. When writing your unit tests you shouldn't need to worry about if any other area of the application is working correctly. The benefits of unit testing are: Decouples your code Write more modular classes Functions are smaller and more focused Your functions are more defensive Quality of code becomes higher You will find it easier to reuse code. When writing unit tests you just need to test this one method of your application, if your method relies on another class/variable there should be a way you can inject this into the method. This is where dependency injection in your code comes in handy, it will allow you to inject objects into your classes to change the output of the class. There are a few things you need to do to make a method unit testable, methods will need an input from a parameter or a class variable and it will need a return or set a class variable in the method. If the method hasn't got these things then the method can not be unit testable. If there isn't a return of the method then there is no way in knowing how the method performs. Dependency Injection Dependency injection is when your object has a dependency on another object. The simplest form to understand what dependency injection is to think of a setter method. A setter method will take one parameter and set a class variable from this parameter. This is using code injection to pass in a parameter to be used as the class variable value. public function setValue( $val ) { $this->val = $val; } Without dependency injection this method will look like this. public function setValue() { $this->val = 10; } For unit testing you need to be aware of any classes that your class is dependent on. For example if you have a login class that will connect to a database. class login { private $db = false; public function __construct() { $this->db = new Database(); } public function loginUser( $user, $password ) { $this->db->checkLogin( $user, $password ); } } This login class has a dependency of the class Database in the constructor, which means that we can't unit test this correctly. If we want to unit test this then the database class has to be development and tested. If the database class is broken and we try to unit test the loginUser() method the test will always fail and we won't know that it's the database class which is broke or the loginUser() method that is broke. If the database class is finished development, tested and data is in the database then we can use this for the loginUser() function. But now our tests are dependent on data being correct in the database. If we pass in a username and password it must be in the database for our test to pass. Our code could be correct but if the data isn't there then our unit tests will fail. This isn't correct use of unit tests and is more suited to be an integration test. To fix this problem we can use dependency injection to pass in a database connector which will set the database class variable. There are 2 ways we can inject a variable into a class, it can either be in the constructor of the class or by using a setter method. I tend to use constructor for all required dependences and use the setter method if there is a default value for the class variable. class login { private $db = false; public function __construct( $db ) { $this->db = $db; } public function loginUser( $user, $password ) { $this->db->checkLogin( $user, $password ); } } Now this class isn't dependant on a certain database class we can pass in the database class by using the parameter on the login class constructor. We can unit test this loginUser() method by first setting the $this->db class variable. We don't want to rely on a real database as the data can change so we can either create a test harness database class or you can mock the database class. A test harness class will allow you to create your database class and hardcode any data that you need. In the example above we can create a method checkLogin(), in our test harness we can then hardcode a successful login username and password to make the loginUser() method pass. Or you can use a PHP mocking framework to mock a class/method/return value. Both methods have their benefits but mocking is normally quicker to code, but there are times when you want to hardcode certain variables in a class. Mocking Objects In TDD With PHP Mocking objects in test driven development allows you create objects to act as a certain class, if your test depends on another method to return a value, you can mock this method and make it return any value you want. In the example we used above you can mock the database class and choose what value we are expecting back from the checkLogin() method. When mocking a method you can choose what you want to return from this method, therefore we can write tests to see what will happen when checkLogin() returns TRUE and then we can write another test to see what happens when checkLogin() returns FALSE. Mocking objects means that you can run your unit tests without depending on another class returning the values you are expecting, ao you can test just your code in this one method. Here are some of the most popular PHP mocking frameworks: Mocking with PHPUnit - http://www.phpunit.de/manual/3.0/en/mock-objects.html Mocking with Phake - http://phake.digitalsandwich.com/docs/html/ Mocking with Mockery - https://github.com/padraic/mockery Mocking with Enchane PHP - https://github.com/Enhance-PHP/Enhance-PHP Mocking with FBMock - https://github.com/facebook/FBMock Dependency Injection With Interfaces If we are going to pass in a database connector in a constructor of the login class, then this database connector will always have to have a method of checkLogin(). This is why we should code our dependences by using interfaces to make sure that we are always passing in the correct type of class. class login { private $db = false; public function __construct( IDatabase $db ) { $this->db = $db; } } class database implements IDatabase { public function checkLogin( $username, $password ) { // check the login credentials } } interface IDatabase { public function checkLogin( $username, $password ); } This will make sure that the class we pass into the constructor is a type of IDatabase, so if our database class doesn't implement IDatabase then the code will fail and therefore our unit tests will fail. This means whatever we pass into the constructor we know that this class will be able to run the methods it needs for the unit tests to run.

I Programmer - Anime Scrum - An Overview If you are an anime fan, and perhaps even if you are not, then you might like a new poster about Scrum - in anime style. As long as you find anime cute or something then seeing the different people involved in the Scrum methodology as anime characters might help you convey the ideas to others. ... Scrum Primer - Scrum Overview - Anime version High-resolution versions of the overview: Scrum Overview - Blue Scrum Overview - Pink Scrum Overview - Green Feel free to use it in your own material. ..." Come on! You KNOW that's awesome! :)

From an original article on Wakaleo.com Conventional test reports, generated by tools such as JUnit or TestNG, naturally focus on what tests have been executed, and whether they passed or failed. While this is certainly useful from a testing perspective, these reports are far from telling the whole picture. BDD reporting tools like Cucumber and JBehave take things a step further, introducing the concept of "pending" tests. A pending test is one that has been specified (for example, as an acceptance criteria for a user story), but which has not been implemented yet. In BDD, we describe the expected behaviour of our application using concrete examples, that eventually form the basis of the "acceptance criteria" for the user stories we are implementing. BDD tools such as Cucumber and JBehave not only report on test results: they also report on the user stories that these tests validate. However this reporting is still limited for large projects, where the numbers of user stories can become unwieldy. User stories are not created in isolation: rather, user stories help describe features, which support capabilities that need to be implemented to achieve the business goals of the application. So it makes sense to be able to report on test results not only at the user story level, but also at higher levels, for example in terms of features and capabilities. This makes it easier to report on not only what stories have been implemented, but also what features and capabilities remain to be done. An example of such a report is shown in Figure 1 (or see the full report here). Figure 1: A test coverage report listing both tested and untested requirements. In agile projects, it is generally considered that a user story is not complete until all of its automated acceptance tests pass. Similarly, a feature cannot be considered ready to deliver until all of the acceptance criteria for the underlying user stories have been specified and implemented. However, sensible teams shy away from trying to define all of the acceptance criteria up-front, leaving this until the "last responsible moment", often shortly before the user story is scheduled to be implemented. For this reason, reports that relate project progress and status only in terms of test results are missing out on the big picture. To get a more accurate idea of what features have been delivered, which ones are in progress, and what work remains to be done, we must think not in terms of test results, but in terms of the requirements as we currently understand them, matching the currently implemented tests to these requirements, but also pointing out what requirements currently have no acceptance criteria defined. And when graphs and reports illustrate how much progress has been made, the requirements with no acceptance criteria must also be part of the picture. Requirements-level BDD reporting with Thucydides Thucydides is an open source tool that puts some of these concepts into practice. Building on top of BDD tools such as JBehave, or using just ordinary JUnit tests, Thucydides reports not only on how the tests did, but also fits them into the broader picture, showing what requirements have been tested and, just as importantly, what requirements haven't. You can learn more about Thucydides in this tutorial or on the Thucydides website. During the rest of this article, we will see how to report on both your requirements and your test results using Thucydides, using a very simple directory-based approach. You can follow along with this example by cloning the Github project at https://github.com/thucydides-webtests/thucydides-simple-demo Simple requirements in Thucydides - a directory-based approach Thucydides can integrate with many different requirement management systems, and it is easy to write your own plugin to tailor the integration to suite your particular environment. A popular approach, for example, is to store requirements in JIRA and to use Thucydides to read the requirements hierarcy directly from the JIRA cards. However the simplest approach, which uses a directory-based approach, is probably the easiest to use to get started, and it is that approach that we will be looking at here. Requirements can usually be organized in a hierarchial structure. By default, Thucydides uses a three-level hierarchy of requirements. At the top level, capabilities represent a high-level capacity that the application must provide to meet the application's business goals. At the next level down, features help deliver these capabilities. To make implementation easier, a feature can be broken up into user stories, each of which in turn can contain a number of acceptance criteria. Figure 2: JUnit test directories mirror the requirements hierarchy. Of course, you don't have to use this structure if it doesn't suit you. You can override the thucydides.capability.types system property to provide your own hierarchy. For example, if you wanted a hierarchy with modules,epics, and features, you would just set thucydides.capability.types to "module,epic,feature". When we use the default directory-based requirements strategy in Thucydides, the requirements are stored in a hierarchial directory structure that matches the requirements hierarchy. At the lowest level, a user story is represented by a JBehave *.story file, an easyb story, or a JUnit test. All of the other requirements are represented as directories (see Figure 2 for an example of such a structure). In each requirements directory, you can optionally place a file called narrative.txt, which contains a free-text summary of the requirement. This will appear in the reports, with the first line appearing as the requirement title. A typical narrative text is illustrated in the following example: Learn the meaning of a word In order to learn the meaning of a word that I don't know As an online reader I want to be able to find out the meaning of the word If you are implementing the acceptance criteria as JUnit tests, just place the JUnit tests in the package that matches the correspoinding requirement. You need to use the thucydides.test.root system property to specify the root package of your requirements. For the example in Figure 2, this value should be set to nz.govt.nzqa.lssu.stories. Figure 3: The narrative.txt file appears in the reports to describe a requirement. If you are using JBehave, just place the *.story files in the src/test/resources/stories directory, again respecting a directory structure that corresponds to your requirement hierarchy. The narrative.txt files also work for JBehave requirements. Progress is measured by the total number of passing, failing or pending acceptance criteria, either for the whole project (at the top level), or within a particular requirement as you drill down the requirements hierarchy. For the purposes of reporting, a requirement with no acceptance criteria is attributed an arbitrary number of "imaginary" pending acceptance criteria. Thucydides considers that you need 4 tests per requirement by default, but you can override this value using the thucydides.estimated.tests.per.requirement system property. Figure 3: For JBehave, everything goes under src/test/resources/stories. Conclusion BDD is an excellent approach for communicating with, and reporting back to, stakeholders. However, for accurate acceptance test reporting on real-world projects, you need to go beyond the story level, and cater for the whole requirements hierarchy. In particular, you need to not only report on tests that have been executed, but also allow for the tests that haven't been written yet. Thucydides puts these concepts into practice: using a simple directory-based convention, you can easily integrate your requirements hierarcy into your acceptance tests.

In many Agile training programs and conferences, a common question that gets raised is, does Agile/Scrum work in maintenance projects? I always say "YES" and the team needs to tweak or invent the practices to suit their needs. Maintenance projects could be enhancement projects OR pure defect fixing projects. Enhancement projects involve new set of developments over existing one. Since the developers get a new set of requirements at the end of each iteration, one can apply the standard set of Scrum practices with little or no modification. Defect Fixing projects involve fixing defects on closed or current projects not in development. Sometimes these projects are boring, especially if a new team has been hired for defect fixing purposes only. The customer sends a set of defects on a daily basis or weekly basis with a deadline to deliver. The development team needs to fix them ASAP and send the patch for further testing. While coaching one of such a defect fixing projects, I found that the following Scrum practices can be applied without much modification: 1. Daily Scrum meetings 2. A Scrum of Scrum 3. Modeling days while solving complex defects 4. Information radiators displaying InProgress, completed, reopened, closed defects and other information 5. Usage of Wiki for collaborating with the customer 6. Requirement workshop while understanding complex defects 7. Review and Retrospective A common problem that I have found in defect fixing projects is setting the iteration length. Especially if the defects are given on a day to day basis without prior knowledge of what you are going to get, it makes the life of the development team bit difficult. This can be solved by collaborating with the customer and coming up with a plan to have 1 or 2 weeks of iteration length.

According to Steve McConnell in Code Complete (data from 1975-1992) most bugs don’t take long to fix. About 85% of errors can be fixed in less than a few hours. Some more can be fixed in a few hours to a few days. But the rest take longer, sometimes much longer – as I talked about in an earlier post. Given all of these factors and uncertainty, how to you estimate a bug fix? Or should you bother? Block out some time for bug fixing Some teams don’t estimate bug fixes upfront. Instead they allocate a block of time, some kind of buffer for bug fixing as a regular part of the team’s work, especially if they are working in time boxes. Developers come back with an estimate only if it looks like the fix will require a substantial change – after they’ve dug into the code and found out that the fix isn’t going to be easy, that it may require a redesign or require changes to complex or critical code that needs careful review and testing. Use a rule of thumb placeholder for each bug fix Another approach is to use a rough rule of thumb, a standard place holder for every bug fix. Estimate ½ day of development work for each bug, for example. According to this post on Stack Overflow the ½ day suggestion comes from Jeff Sutherland, one of the inventors of Scrum. This place holder should work for most bugs. If it takes a developer more than ½ day to come up with a fix, then they probably need help and people need to know anyways. Pick a place holder and use it for a while. If it seems too small or too big, change it. Iterate. You will always have bugs to fix. You might get better at fixing them over time, or they might get harder to find and fix once you’ve got past the obvious ones. Or you could use the data earlier from Capers Jones on how long it takes to fix a bug by the type of bug. A day or half day works well on average, especially since most bugs are coding bugs (on average 3 hours) or data bugs (6.5 hours). Even design bugs on average only take little more than a day to resolve. Collect some data – and use it Steve McConnell, In Software Estimation: Demystifying the Black Art says that it’s always better to use data than to guess. He suggests collecting time data for as little as a few weeks or maybe a couple of months on how long on average it takes to fix a bug, and use this as a guide for estimating bug fixes going forward. If you have enough defect data, you can be smarter about how to use it. If you are tracking bugs in a bug database like Jira, and if programmers are tracking how much time they spend on fixing each bug for billing or time accounting purposes (which you can also do in Jira), then you can mine the bug database for similar bugs and see how long they took to fix – and maybe get some ideas on how to fix the bug that you are working on by reviewing what other people did on other bugs before you. You can group different bugs into buckets (by size – small, medium, large, x-large – or type) and then come up with an average estimate, and maybe even a best case, worst case and most likely for each type. Use Benchmarks For a maintenance team (a sustaining engineering or break/fix team responsible for software repairs only), you could use industry productivity benchmarks to project how many bugs your team can handle. Capers Jones in Estimating Software Costs says that the average programmer (in the US, in 2009), can fix 8-10 bugs per month (of course, if you’re an above-average programmer working in Canada in 2012, you’ll have to set these numbers much higher). Inexperienced programmers can be expected to fix 6 a month, while experienced developers using good tools can fix up to 20 per month. If you’re focusing on fixing security vulnerabilities reported by a pen tester or a scan, check out the remediation statistical data that Denim Group has started to collect, to get an idea on how long it might take to fix a SQL injection bug or an XSS vulnerability. So, do you estimate bug fixes, or not? Because you can’t estimate how long it will take to fix a bug until you’ve figured out what’s wrong, and most of the work in fixing a bug involves figuring out what’s wrong, it doesn’t make sense to try to do an in-depth estimate of how long it will take to fix each bug as they come up. Using simple historical data, a benchmark, or even a rough guess place holder as a rule-of-thumb all seem to work just as well. Whatever you do, do it in the simplest and most efficient way possible, don’t waste time trying to get it perfect – and realize that you won’t always be able to depend on it. Remember the 10x rule – some outlier bugs can take up to 10x as long to find and fix than an average bug. And some bugs can’t be found or fixed at all – or at least not with the information that you have today. When you’re wrong (and sometimes you’re going to be wrong), you can be really wrong, and even careful estimating isn’t going to help. So stick with a simple, efficient approach, and be prepared when you hit a hard problem, because it's gonna happen.

Test-Driven Development is a code-level practice, based on running automated tests that are written before the production code they exercise. But practices can be applied only in the context where they were developed: when some premises are not present is difficult to apply TDD as-is. Automated specification For example, consider the premise of assertion automation: it is possible to write a (hopefully) small algorithm that is able to check the result of running production code and return true or false. In the case the problem is: Draw an antialiased circle on this blank canvas. -- Carlo Pescio it is not immediately clear how to define automated tests for this behavior. We could check that some pixels are still blank inside or outside the circle, or that there is a bound number of pixels of black color; or even that they are contiguous. An opinion I've heard (that I try not to misrepresent) is that we only need to write some looser tests in these cases, checking only a few pixels of the circle. This process will give us a little feedback on the API of our Canvas or Circle object, but not much on the algorithm we are implementing inside it. Are we going in the right direction? Have new test cases correctly been satisfied without a large intervention on the existing code? Are we painting some unrelated pixels due to an hidden bug? What I argument here is instead that we should change the nature of the feedback mechanism. Speaking in control theory terms, change the block that acquires the output and influences the input to our design process. Develop in the browser When I was developing a Couchapp, a kind of web application served directly from a CouchDB database, I was appaled by the difficulty of testing it. While the production code was composed of ~100 lines, it was a complex mix of technologies: HTML and CSS code, client-side JavaScript for managing user events and some server-side JavaScript for the "queries" (actually the server-side only consists of the database in Couchapps.) Some of this logic could be tested in automation, like the result of queries over views. Yet much of it was related to a user interface, and as such requiring a large time investment to automate. Instead of waking up my Selenium server and start to manipulate a browser with code, I noticed that this UI was almost read-only; there were a few cases where a new document would have to be inserted, but a manual test of them was short and did not even required to reload the page. The whole application state was observable. Summing it up, I performed a frequent manual test that took a few seconds instead of trying to define complex and brittle automation logic for testing the UI. Now that I've been introduced to a simple qualitative ROI model by Carlo Pescio's article, I would do the same for every context where: a large time investment is needed for automating tests. it is possible to perform manual tests quickly. as the only logic conclusion. A word of caution TDD has many benefits (including catching regressions early) so I'm not prepared to give it up just because it is difficult to test. These are technical scenarios where I have successfully followed TDD by the book: multithreaded and multiprocess code applications distributed over multiple machines computer vision (object recognition and tracking) image manipulation code (via comparison testing) development of browser bindings for Selenium And even in the case the big picture is not easy to test-first (like in the case of image manipulation), we can benefit from TDD the pieces of the solution. For example, in the computer vision case I wasn't able to write a test beforehand for tracking a car inside a movie. But I was able to TDD the objects that the algorithmic solution to the problem called for: Patch, Area, Cluster, Movement, and so on. End-to-end TDD is not always cheap but unit level TDD can often be, if it considers testability as a relevant property (while regression testing even at the end-to-end level is always possible, in the worst case with record and replay.) End-to-end specifications If we can't define automated assertions for our "big picture" problem, it doesn't mean that we cannot apply the TDD approach, by substituting a manual step. Going back to the circle problem, I would define manual test cases on an inspection page seen by a human. I've seen this done with layouts and multiple browsers to catch CSS rendering bugs, for example: It would be very difficult to check these screenshots automatically, as each browser renders pages a bit differently from the others. The iterative process becomes: Define a cheap manual test, automating the arrange and act phases but not the assertion. Write only the code necessary to make it pass. Refactor. As long as the number of tests does not increase without limit and the manual check can be performed quickly, this approach does not slow you down with respect to TDD by-the-book. You'll have to take care of regression with other means; but at least you define a set of manual test cases. Feedback! TDD is an instrument of feedback: if feedback cannot be gathered in an automated way, we have to resort to manual checking of the specifications. Here are other examples of manual tools for generating feedback: Read-Eval-Print Loops: you can experimenting with existing classes and functions, and easily repeat steps thanks to history. the browser refresh button: the fastest way to transform a PSD into an HTML and CSS template. MongoDB console for learning the database API; other kinds of consoles like Firebug and Chrome's, or Clojure's.