"Don't tell people how to do things, tell them what to do and let them surprise you with their results" - General George S. Patton What's the best way to encourage agile teamwork? It's a tricky question, because so much of Scrum and Kanban practice is predicated on the assumption that collaborative behavior will "happen". Empowerment is often presented as the mechanism for achieving this success. If you just press the empowerment button, developers will then choose to self-organize and will go on to deliver sterling results. Patently however, that isn't the case. I'm sure that many of us will have experienced teams that are actually less than the sum of their parts. Technically skilled people can be more focused on stack traces than on individuals and interactions, and may view each other as unwanted complications or impediments. All too often the social graces that underpin effective agile teamwork have to be elicited painfully, like drawing teeth. Whenever I consider this matter, the above quote by Patton often comes into my mind. It isn't the perspicacity of his argument that I find compelling, or even that it was said so long ago. I suppose that these days we have just become more accepting of such observations. No...to me the interesting thing about this quote is that someone of Patton's background and temperament said it. You see, George Smith Patton was arguably the most hard-boiled U.S. General in World War 2. He was spit-and-polish to the core, and an absolute stickler for discipline. Even tiny misdemeanours would incur his wrath. His idea of a touchy-feely management style was to kick people in the pants after slapping them about the chops, and he frequently railed against "malingerers" who he reckoned ought to be court-martialed and shot. We have yet to hear Esther Derby or Johanna Rothman prescribe such remedies for disaffected team members. Perhaps the most politically correct thing we can do is to categorize his beliefs as an alternative viewpoint. Anyway, it's difficult to imagine anyone less likely than Patton to be sympathetic to agile principles, nor anyone more likely to try and micro-manage those they might consider to be their sub-ordinates. It seems we need a deeper insight if we are to explain this unlikely patronage of a central maxim of agile development. I suspect that Patton knew that if a team is to self-organize and deliver value successfully, then discipline will be key. It can't really be about empowerment, because an empowered team can still be sloppy and never cut the mustard. While good management isn't about telling people how to do their jobs, it is about making sure that they understand the rules of best practice and are competent to follow them, preferably with very little oversight. Strangely perhaps, this is a route to freedom rather than constraint. It releases individual initiative. I think that's what Patton was getting at. Who are we to empower others, after all? What gift is that? Where is the transfer of value? How much better it is to instil the best practices that make people more effective, and thereby become more valued themselves. Development Team Membership Now, a development team is made up of individuals, so when we talk about the rules of team membership we are largely talking about what those individuals do. More specifically, it's about what they do in respect to themselves, and with respect to the wider team of stakeholders including the Scrum Master and Product Owner. So before we go any further, let's look at the behaviors that we can expect a disciplined agile developer to exhibit. What a good team member will do: Agree with other team members and the Product Owner to deliver a valuable and achievable piece of work every Sprint Understand the Sprint Backlog and how it correlates to the Sprint Goal Participate fully and actively in daily standups, planning sessions, reviews, and retrospectives Work with the rest of the team to meet each Sprint Goal (self-organize) Help other team members and the Product Owner to clarify requirements, such as by writing user stories and acceptance criteria Pro-actively remove skill silos, such as by pair programming or cross training, and without being told to do so Work with the Product Owner on an ongoing basis, so that work is understood, reviewed, and approved continually Make sure that the work done is transparent, such as by updating Scrum and Kanban boards Understand that they, and all team members, are stakeholders in the agile process Estimate work so that the Product Owner and other stakeholders can plan ahead (e.g. for release planning) Fully support and encourage the elicitation of metrics, and be able to interpret them and act on them Resolve outstanding or impeded work before actioning new work from the backlog Limit work in progress so as to maximize throughput Act immediately on impediments by appraising other team members and the Scrum Master of any issues, and help to resolve them Accept personal responsibility for the team's success Accept personal responsibility for their work meeting the team’s Definition of Done What a good team member doesn’t do… Fail to give the best unpadded estimates that can be provided at the time. Estimates should be given and received in good faith. Cherry pick work from the Sprint Backlog. The backlog is owned by the team and must be actioned in accordance with the team's Sprint Plan. Attempt to work on more than one item at a time. A good team member will pro-actively limit work in progress. Expect somebody else, such as a Scrum Master, to update the Scrum board or Kanban boards. Information radiators are owned by the team. Work in a "skills silo". A good team member does not view their work as a speciality that only he or she is able to work on. Claim that work has been completed if it does not satisfy the team’s Definition of Done Claim that work is complete if it does not meet the specific acceptance criteria that have been agreed for it Shot at Dawn: Teamwork and the Prime Directive If we were to apply the Patton philosophy in extremis, I suppose that an agile team would shoot its own malingerers following a retrospective, the Scrum Master standing by to deliver the coup-de-grace if needed. Although this lurid concept is absurd, how many experienced Scrum Masters have never secretly wished for a revolver in their desks, even for just a fleeting moment? It highlights a problem that the agile community is often evasive about. What should actually be done about a developer who causes problems for the rest of the team? Is it possible, or even desirable, to correlate the occurrences of those problems to the individual concerned? In a Sprint Retrospective, for example, no blame is ever meant to be directed towards any one team member. In fact the format of the session precludes the establishment of such a correlation, or even the inference that a particular individual may have been remiss in some way. Known as the "Prime Directive", this article of faith is meant to be recanted at the beginning of each retrospective session, and it has to be said in earnest. "Regardless of what we discover, we understand and truly believe that everyone did the best job he or she could, given what was known at the time, his or her skills and abilities, the resources available, and the situation at hand." The question is: what if we don't believe it though? What if all the evidence in the world is stacked against it? Should we go along with the directive anyway, and just kid ourselves for the duration of the session? If so, how can it possibly help? Where is the transparency, which we covet in agile practice, if we subscribe to this devil's credo that makes a mockery of the truth? The answer is potentially quite shocking, and certainly little understood. Don't think of the Prime Directive as a creed, or even as the temporary suspension of disbelief for the sake of the meeting. Think of it as a pre-condition that must hold, and genuinely be true, before a retrospective can happen at all. The underlying principle is that all of the attendees must be fully able to participate. All are expected to be professionals who can fulfil their duty to each other and to the Scrum process, and inspect and adapt their working practices accordingly. It isn't enough just to leave your knives at the door. You actually have to trust the people you are working with. Really trust them. Given that most developers are assigned to their teams by managers, and not by each other, this expectation of trust is indeed potentially shocking. It gets even scarier than that. Think about what all of this really means should trust be absent, or somehow lost. It means that you can't have a Sprint Retrospective at all until the issues around trust are resolved. It means that if a team member must be removed, then that should happen beforehand. Scrum does not go so far as to prescribe a mechanism for this, but it is established that a team will self-organize to remove its own problems. Perhaps they will have to make collective representations to a line manager, or petition for a member's removal through the Scrum Master. It might even mean that the team can deselect a team member by their own consensus. Yet however it is done, it appears that the team aren't too far removed from assembling a firing squad after all. If this all seems very draconian, let's reassert the key principle here: when Scrum is done properly a team will solve its own problems, including distasteful matters like this. Now, it has to be admitted that most Scrum teams across industry today don't get to operate at such a high level of proficiency. The consequences of this cut both ways. On the one hand a team may not be allowed to get on with their jobs without interference from management, while on the other hand they usually don't have to deal with the nastiness of putting a sick dog down. A few conversations with that same pointy-haired boss could be enough to get him to do the deed. Yet as the industry transitions more fully towards agile practice, this "remedy" will no longer be sustainable. Problems regarding a team member's competence won't be someone else's responsibility; rather, it will be incumbent upon the team to find a solution. In an agile world, greater responsibility falls on self-managing teams, along with their greater rights. Professionalism: from Team Discipline to Self Discipline In this article we've identified a range of behaviors that typify good team membership, and we've looked squarely at what should happen when things go wrong. In short, it's up to the team to sort out its own problems when a team member doesn't measure up. Yet this is only part of what disciplined agile practice is about. It isn't enough to put the focus on punitive measures and the threat of sanction, even if the exercising of authority is driven entirely by the team itself. What we need to do is to take things a step further. We don't really want discipline to be enforced by the team, even though they should be the ultimate arbiters. What we want is to encourage a self-discipline that wells up from each individual team member, and which serves as an inspiration to others. Disciplined teamwork isn't about empowerment. It's about cascading the release of potential through the clear demonstration of value. I look at it this way. There is only one person in this world any of us can change. I don't think I need to spell out who that person is. So, wherever you and your team may be on your agile journey, there should always be at least one person who can be relied upon. If that person does their bit, then they are helping to make the team more than the sum of its parts. "Don't empower me. Release me. I'll find my own power, and it will be far greater than anything you can bestow on me" - Tobias Mayer

There are many good ideas and practices in Agile development, ideas and practices that definitely work: breaking projects into Small Releases to manage risk and accelerate feedback; time-boxing to limit WIP and keep everyone focused; relying only on working software as the measure of progress; simple estimating and using velocity to forecast team performance; working closely and constantly with the customer; and Continuous Integration – and Continuous Delivery – to ensure that code is always working and stable. But there are other commonly accepted ideas and best practices that aren’t important: if you don’t follow them, nothing bad will happen to you and your project will still succeed. And there are a couple that you are better off not following at all. Test-Driven Development Teams that need to move quickly need to depend on a fast, efficient testing safety net. With Test First Development or Test-Driven Development (TDD), there’s no excuse for not writing tests – after all, you have to write a failing test before you write the code. So you end up with a good set of working automated tests that ensure a high level of coverage and regression protection. TDD is not only a way of ensuring that developers test their code. It is also advocated as a design technique that leads to better quality code and a simpler, cleaner design. A study of teams at Microsoft and IBM (Realizing Quality Improvement through Test Driven Development, Microsoft Research, 2008) found that while TDD increased upfront development costs between 15-35% (TDD demands developers change the way that they think and work, which slows developers down, at least at first), it reduced defect density by 40% (IBM) or as much as 60-90% (Microsoft) over teams that did not follow disciplined unit testing. But in Making Software Chapter 12 “How Effective is Test-Driven Development” researchers led by Burak Turhan found that while TDD improves external quality (measured by one or more of test cases passed, number of defects, defect density, defects per test, effort required to fix defects, change density, % of preventative changes) and can improve the quality of the tests (fewer mistakes in the tests, tests that are easier to maintain), TDD does not consistently improve the quality of the design. TDD seems to reduce code complexity and improve reuse, however it also negatively impacts coupling and cohesion. And while method and class-level complexity is better in code developed using TDD, project/package level complexity is worse. People who like TDD like it a lot, so if you like it, do it. And even if you are not TDD-infected, there are times when working test first is natural – when you have to solve a specific problem in a specific way, or if you’re fixing a bug where the failing test case is already written up for you. But the important thing is that you write a good set of tests and keep them up to date and run them frequently – it doesn't matter if you write them before, or after, you write the code. Pair Programming According to the VersionOne State of Agile Development Survey 2012, almost 1/3 of teams follow pair programming – a surprisingly high number, given how disciplined pair programming is, and how few teams follow XP (2%) or Scrum/XP Hybrid (11%) methods where pair programming would be prescribed. There are good reasons for pairing: information sharing and improving code quality through continuous, informal code reviews as developers work together. And there are natural times to pair developers, or sometimes developers and testers, together: when you’re working through a hard design problem; or on code that you’ve never seen before and somebody who has worked on it is available to help; or when you’re over your head in troubleshooting a high-pressure problem; or testing a difficult part of the system; or when a new person joins the team and needs to learn about the code and coding practices. Some (extroverted) people enjoy pairing up, the energy it creates and the opportunities it provides to get to know others on the team. But forcing people who prefer working on their own or who don’t like each other to work closely together is definitely not a good idea. There are real social costs in pairing: you have to be careful to pair people up by skill, experience, style, personality type and work ethic. And sustained pair programming can be exhausting, especially over the long term – one study (Vanhanen and Lassenius 2007) found that people only pair between 1.5 and 4 hours a day on average, because it’s too intense to do all day long. In Pair Programming Considered Harmful? Jon Evans says that pairing can have also negative effects on creativity: Research strongly suggests that people are more creative when they enjoy privacy and freedom from interruption … What distinguished programmers at the top-performing companies wasn’t greater experience or better pay. It was how much privacy, personal workspace and freedom from interruption they enjoyed,” says a New York Times article castigating “the new groupthink”. And in “Still Questioning Extreme Programming” Pete McBreen points out some other disadvantages and weaknesses of pair programming: Exploration of ideas is not encouraged, pairing makes a developer focus on writing the code, so unless there is time in the day for solo exploration the team gets a very superficial level of understanding of the code. Developers can come to rely too much on the unit tests, assuming that if the tests pass then the code is OK. (This follows on from the lack of exploration.) Corner cases and edge cases are not investigated in detail, especially if they are hard to write tests for. Code that requires detail thinking about the design is hard to do when pairing unless one partner completely dominates the session. With the usual tradeoff between partners, it is hard to build technically complex designs unless they have been already been worked out in a solo session. Personal styles matter when pairing, and not all pairings are as productive as others. Pairs with different typing skills and proficiencies often result in the better typist doing all of the coding with the other partner being purely passive. And of course pairing in distributed teams doesn't work well if at all (depending on distance, differences in time zones, culture, working styles, language), although some people still try. While pairing does improve code quality over solo programming, you can get the same improvements in code quality, and at least some of the information sharing advantages, through code reviews, at less cost. Code reviews – especially lightweight, offline reviews – are easier to schedule, less expensive and less intrusive than pairing. And as Jason Cohen points out even if developers are pair programming, you may still need to do code reviews, because pair programming is really about joint problem solving, and doesn’t cover all of the issues that a code review would. Back to Jon Evans for the final word on pair programming: The true answer is that there is no one answer; that what works best is a dynamic combination of solitary, pair, and group work, depending on the context, using your best judgement. Paired programming definitely has its place. (Betteridge’s Law strikes again!) In some cases that place may even be “much of most days.” But insisting on 100 percent pairing is mindless dogma, and like all mindless dogma, ultimately counterproductive. Emergent Design and Metaphor Incremental development works, and trying to keep design simple makes good sense, but attempting to define an architecture on the fly is foolish and impractical. There’s a reason that almost nobody actually follows Emergent Design: it doesn't work. Relying on a high-level metaphor (the system is an "assembly line" or a "bill of materials" or a "hive of bees") shared by the team as some kind of substitute for architecture is even more ridiculous. Research from Carnegie Mellon University found that … natural language metaphors are relatively useless for either fostering communication among technical and non-technical project members or in developing architecture. Almost no one understands what a system metaphor is any ways, or how it is to be used, or how to choose a meaningful metaphor or how to change it if you got it wrong (and how you would know if you got it wrong), including one of the people who helped come up with the idea: Okay I might as well say it publicly - I still haven't got the hang of this metaphor thing. I saw it work, and work well on the C3 project, but it doesn't mean I have any idea how to do it, let alone how to explain how to do it. Martin Fowler, Is Design Dead? Agile development methods have improved development success and shown better ways to approach many different software development problems – but not architecture and design. Daily Standups When you have a new team and everyone needs to get to know each other and more time to understand what the project is about; or when the team is working under emergency conditions trying to fix something or finish something under extreme pressure, then getting everyone together in regular meetings, maybe even more than once a day, is necessary and valuable. But whether everyone stands up or sits down and what they end up talking about in a meeting should be up to you. If your team has been working well together for a while and everyone knows each other and knows what they are working on, and if developers update cards on a task board or a Kanban board or the status in an electronic system as they get things done, and if they are grown up enough to ask for help when they need it, then you don’t need to make them all stand up in a room every morning. Collective Code Ownership Letting everyone work on all of the code isn't always practical (because not everyone on the team has the requisite knowledge or experience to work on every problem) and collective code ownership can have negative effects on code quality. Share code where it makes sense to do so, but realize that not everybody can – or should – work on every part of the system. Writing All Requirements as Stories The idea that every requirement specification can be written as User Stories in 1 or 2 lines on cards, that requirements should be too short on purpose (so that the developer has to talk to someone to explain what’s really needed) and insisting that they should all be in the same template form “As a type of user I want some goal so that some reason…” is silly and unnecessary. This is the same kind of simple minded orthodoxy that led everyone to try to capture all requirements in UML Use Case format with stick men and bubbles 15 years ago. There are many different ways to effectively express requirements. Sometimes requirements need to be specified in detail (when you have to meet regulatory compliance or comply with a standard or integrate with an existing system or implement a specific algorithm or…). Sometimes it’s better to work from a test case or a detailed use case scenario or a wire frame or some other kind of model, because somebody who knows what’s going on has already worked out the details for you. So pick the format and level of detail that works best and get to work. Relying on a Product Owner Relying on one person as the Product Owner, as the single solitary voice of the customer and the “one throat to choke” when the project fails, doesn't scale, doesn't last, and puts the team and the project and eventually the business at risk. It’s a naïve, dangerous approach to designing a product and to managing a development project, and it causes more problems than it solves. Many teams have realized this and are trying to work around the Product Owner idea because they have to. To succeed, a team needs real and sustained customer engagement at multiple levels, and they should take responsibility themselves for making sure that they get what they need, rather than relying on one person to do it all.

A couple of weeks ago we looked at how to do a quick "health check" of an agile team. We saw that a great deal can be learned just by attending one of their daily stand-ups and by inspecting the state of their Scrum and Kanban boards. Of course these are nothing more than cursory examinations, and serious ailments can lie behind an apparently robust façade of agile practice. If you have reason to believe that a team is dysfunctional, you might have to dig deeper than the superficial evidence suggested by its apparent morphology. In my experience the next thing to examine is the team's "Definition of Done". This is the standard to which all work is put before it can be considered to be complete. Each team is collectively responsible for its own Definition of Done. It's up to them to make sure that it is adequate, and that it is applied by all members to all of the work they do. Without such professional oversight there can be no assurance that any deliverable will truly be fit for release. A spiffy stand-up and a cracker of a Kanban board might suggest rude team health, but they are no guarantee that the Definition of Done is solid, or that it isn't being undercut by someone along the way. Technical debt and rework are the main symptoms to look for. The consequences of backsliding on a Definition of Done might not become apparent until long after the events that caused it. By then, that rework or debt can be difficult to trace to the specific behaviors of those who cheated the system. You see, unfortunately a Definition of Done is a bit like personal hygiene. If there is no oversight, everyone can pretend that they are following the rules for the sake of the team, even though the presence of E. Coli on the office keyboards will tell its own tale about compliance. Everyone knows that it has to be coming from somewhere, but won't admit to their own liability or involvement, perhaps not even to themselves. Just as team vomiting will follow one member's dubious hand-washing practices, a short-changed Definition of Done will lead to rework by the team or the creation of technical debt. This is why team ownership and enforcement of what "Done" means is so important. An effective Definition of Done has to be founded on a healthy balance between due diligence and professional trust. What does this mean for agile development? You can think of a Definition of Done as the key defensive bulwark in software development epidemiology. If you balance the right level of team oversight with the right level of trust, severe outbreaks of technical debt or rework will become rare. High levels of oversight may be needed to start with, since team members might not have bought in to the idea of "done" yet. Once people become conditioned to do the right thing and see themselves as stakeholders in the team and its success, the balance can swing more towards trust. People become reluctant to renege on a team investment if they can see that it adds value for everyone including them. What's more, a Definition of Done improves the more it is respected, and becomes better respected the more it improves. In terms of agile best practice a Definition of Done will be used to determine whether or not User Story implementations are release-ready. However, each team can implement many User Stories over the course of a sprint, and making sure that all of these stories meet the Definition of Done can be challenging. Teams that are new to agile methods often have more modest ambitions. For example, their Definition of Done may only extend as far as delivery into a pre-production environment. Of course, anything less than "fully release ready" incurs the risk of technical debt and the need to pay it back later. Yet like a sloppy approach to hand-washing, it has to be admitted that something is better than nothing at all. Applying a Definition of Done consistently to even a sub-optimal standard will at least permit the delivery of each User Story to a known level of quality. It might not be great but at least it's there, and it's something that can be built upon and improved. The Lessons of Lean-Kanban Lean-Kanban methodologies have an instructive relationship with the Definition of Done. In these approaches the optimization of the value stream is of great significance. Naturally though, if a value stream is to be optimized it must first be understood. This means breaking the stream down into multiple discrete steps that can be studied for bottlenecks and any other occurrences of waste. For example, "Work in Progress" can be broken down into finer-grained stations such as "In Development", "Peer Review", "QA Test", "Knowledge Transfer", and "In Deployment". Team members will be cross-trained and will move freely across those stations in order to expedite as smooth a workflow as possible. Now here's where it gets interesting. If a Lean-Kanban operation has multiple well-defined stations, the case for having a Definition of Done can seem rather harder to make. After all, by the time a User Story gets to "Done", you already know that it has gone through the key steps you care about in the development process. What value can a Definition of Done really add in such a situation? Doesn't it just become waste itself? To find the answer, we need to look back to the manufacturing roots of Lean-Kanban. In a car plant for example, the steps of construction are exceptionally well defined and team members can move freely over several dozen stations. Some of those stations will be for the chassis, others for the interior, others for the engine block and electrics and so on. Yet despite this the Definition of Done will be an absolute corker, and much of the process of verification will be automated. Each station might even have its own Definition of Done so inspection can occur as close as possible to the point of action and potential remedy. The total number of checks that happen before each vehicle leaves the factory will be exhaustive. Why is this thought to be necessary? Because the manufacturers know perfectly well that the verification of "done" adds value. Merely having well-defined stations is no guarantee that everything will be done well. Quality is built in and validated by inspection. One thing's for sure: no-one in IT should accuse car manufacturers of having a weak understanding of what "done" means. The Definition of Done versus Acceptance Criteria However, software projects have a wild-card to deal with that car manufacturers don't have to worry about. Unlike the car doors and carburettors that roll down an assembly line, each User Story is different and has to be treated as a special case. To deal with this, each User Story has Acceptance Criteria that are agreed by the team members and the Product Owner as part of a Sprint Planning Session. Acceptance Criteria have to be quite specific to particular User Stories, because each story can be unique. The Definition of Done, on the other hand, applies to all of the User Stories being worked on by a team. The associated conditions must be invariant. For example, if all work has to be peer reviewed and subjected to QA testing prior to release, then those criteria would be enumerated in the Definition of Done rather than being repeated in each User Story's Acceptance Criteria. If the definition is enforced properly, a developer could never claim that a User Story was “Done” if it hadn't both been reviewed and passed QA testing. Writing a Definition of Done The Scrum Guide describes a Definition of Done as a "shared understanding of what it means for work to be complete". No process is suggested for writing a Definition of Done, nor in fact is there any suggestion that one should be written down at all. However, a documented definition may go some way towards providing that shared understanding. Here's how you can set about eliciting one: Review Acceptance Criteria: Gather the Acceptance Criteria for work completed so far Look for common criteria that can be abstracted out and applied across work in general Use these common criteria as the basis for a Definition of Done Assess Technical Debt Identify any rework that needs to be done Identify the reasons why it wasn't done properly the first time Identify what measures can be put in place to stop similar rework from occurring Add these measures to the Definition of Done (DoD) Continually update the DoD In each Sprint Review, identify which (if any) work was rejected or which caused rework to be done, then In each Sprint Retrospective, challenge the DoD for relevance and completeness There isn't a prescribed format for a Definition of Done, but it can be beneficial to use the same as that which is used for Acceptance Criteria, either in whole or in part. This allows a flexible definition based on story type. Alternatively they can be written as simple lists. Here are some examples: Example of a Definition of Done in Acceptance Criteria Format Given that a user story has required a code change When BDD and unit tests have been written for the story and the code change has been reviewed and the code change has been approved by a peer and all BDD and unit tests have been run and no BDD or unit tests have broken (green bar) and the code change has been committed to the repository and QA testing has passed satisfactorily and the Product Owner has approved the change Then the user story will be deployed to production and it will be considered Done. Given that a user story has required the authoring of documentation When the documentation has been reviewed and approved by a peer and the documentation has been approved by the Product Owner Then a new version of the documentation will be committed and the user story will be considered Done. Example of a Definition of Done in List Format Code changes: BDD tests written and pass Unit tests written and pass Code peer reviewed & approved Code committed to repository QA testing done Product Owner signed off Documentation: Documentation has been peer reviewed & approved Documentation approved by Product Owner Version number updated Definitions of Done for IT Infrastructure Support We've seen that having a good Definition of Done is important, although in IT we also need Acceptance Criteria that address the particulars of each User Story. When used in combination they can approach the levels of rigor that have been shown to be possible in manufacturing. Those working in software development can adopt a similar commitment to quality. Now we need to turn our attention to another function within the IT department...Infrastructure Support. Infrastructure support teams are increasingly expected to work in an agile way. As part of an enterprise transformation that does not seem unreasonable. After all, the rest of the organization is highly dependent upon them. Their scope includes such things as deploying new workstations and laptops (possibly across entire sites), installing networks, performing miscellaneous diagnostics and repairs, and maintaining and upgrading local server resources. Clearly they will also need Definitions of Done and Acceptance Criteria if they are to be stakeholders in a joined-up agile enterprise. The question is, how on earth can a meaningful Definition of Done be abstracted across wildly different physical tasks? How can a Definition of Done cover everything from a phone installation to a printer driver upgrade or a memory enhancement, or a firewall configuration to a keyboard replacement? The answer is to focus on the value chain that is represented by each user story. Work is not "released" as such, but rather it is handed over to someone who can derive benefit from it (i.e. the person occupying the user story role). This is the key to understanding "done" in an infrastructure context. If you can identify the parties who derive value, and demonstrably pass that value on to them, then your work is done. Here's an example of a Definition of Done that might be used to close out a support ticket: The receiver of the work has been identified Handover instructions have been completed and given to the receiver The receiver has been notified of the intention to close the ticket, and has not raised an objection A security assessment has been conducted and approved There are three things to point out here. The absence of any reference to a Product Owner. This is because infrastructure teams have to support multiple products, and prioritization of work is traditionally handled not through any sense of ownership of those products, but rather through help-desk triage. It's certainly possible for work to be represented by Product Owners, but it would have to be ownership of the business support function rather than ownership of the actual products being supported. The need to identify and work with the actual receivers of value is still there. The "acceptance by default" position. Receivers typically have little incentive to sign work off as being complete. On the contrary, their incentive is to defer acceptance as long as possible, for potential use as a "banker" in case a requirement for additional unforeseen work transpires. They might hope to ride this new work on an existing ticket instead of having to raise a new one. Receivers can be expected to care about their own support needs, not about the big picture of enterprise delivery. If a Product Owner can be identified - even if it is just the most likely owner of the business support function - then this situation can be improved. A Product Owner can apply leverage for appropriate and timely sign-off, such as by not accepting new work from certain parties while their approval (or justified rejection) of prior work remains outstanding. The elicitation of solid Acceptance Criteria can help the Product Owner immensely. Security implications need to be given careful consideration. The reworking of organizational infrastructure offers great potential for security to be compromised. Approval from Information Security should be obtained for all work, either directly or through authorized agents. One approach is for each team to have a designated "security champion" who provides this function. Conclusion Teams that appear to be healthy can still incur rework and technical debt. A poor understanding of what "done" means often underlies such problems, and this should be one of the first things to be looked at if problems are suspected. Having a meaningful Definition of Done encourages a team's sense of ownership of their own process, and helps instil self-discipline into its members to follow agile best practices. The application of this standard requires finding the right balance between team oversight and trust.

The longer I spend working as an agile coach, the more I find myself in disagreement with Hamlet. To estimate, or not to estimate? That is the question. Out of all of the agile practices which have been adopted in recent years, few have proven more controversial than this one. The battle for and against rages like Shakespearean armies set against each other's teeth. (Free Estimation Ebook) At first blush there doesn't seem to be any reasonable cause for disagreement. The rationale for making estimates is ostensibly straightforward. If a team is to work in Sprints, and to deliver something at the end of each one, then the work must surely be estimated. Otherwise how can the team know if it is even possible to do the work within the Sprint? How can they commit to deliver something by the end of that time-box if the effort involved is of uncertain magnitude? Well, there are two things that we need to draw out at this point. Firstly, the above rationale assumes that Sprints will be used, and that delivery will therefore be time-boxed. That's a very Scrum oriented philosophy...but Scrum isn't the only agile way of working. Lean-Kanban teams, for example, don't use Sprints and rarely make use of estimates. Secondly, Scrum itself says nothing about estimation. It only says that each item in a backlog must be sized - how that sizing happens is up to the team. It should also be remembered that a Scrum team commits to a Sprint Goal that delivers value, not to the delivery of a certain number of estimated points. So then...to estimate, or not to estimate? Let's listen in at the camp-fires of each side, and pick out in more detail the arguments they make for and against. For (Ye Scrum Brigade of Sprinte and Stande-uppe) "Estimates allow us to predict when a Sprint Goal will be met, and therefore when a substantial increment of value will be delivered" "Our estimates help our stakeholders plan ahead. They are part of the value we provide" "Estimates help us to de-risk scope of uncertain size and complexity" "Estimated work can be traded in and out of scope for other work of similar size. Without estimates you can't trade" "The very process of estimation adds value. When we estimate we discuss requirements in more detail, and gain a better understanding of what is needed" Against (Ye Lean Kanban Brigade of Boarde Pullers) "Estimates are rarely accurate. All you are doing when you estimate a piece of work is to set false expectations" "In practice, estimation is seen as a commitment, not as a best guess. Every time you make an estimate, you make a rod for your own back" "Estimation is time consuming. The time a team spends playing planning poker or whatever is time that could have been spent on delivery. Estimation is waste." "It's the actuals that matter, not estimates. Agility requires metrics, and the only metrics that count are those that reflect actual delivery" Both sides are right If you see this debate in terms of whether a Scrum or Lean-Kanban process is being followed, then both sides are right. A Scrum process is optimized for project work where scope risk is high and an entire system is represented. The requirements tend to be uncertain, complex, and very heavily intertwined. By committing to a Sprint Goal and to the delivery of a substantial increment of value, that risk can be managed. Uncertain and interdependent requirements are batched together into a Sprint and dealt with as a group. When this is done well, you have a clear Sprint Goal and a coherent Sprint Backlog. When it is done badly, you have a vague or disjointed Sprint Goal, a mishmash of requirements that command no sense of team purpose, and no team commitment towards the delivery of an increment. A Lean-Kanban process, on the other hand, is usually focused on "Business As Usual" (BAU) activities. The diet of a Lean-Kanban operation should consist of small and repeatable changes. They don't have to be related at all...in fact they shouldn't be. Things like bug fixes, minor enhancements, and administrative tasks are representative of this kind of work. Scope risk is low because the process of making such changes is well understood. Estimates are generally held to be unnecessary because there is very little uncertainty to deal with. There is no need for work to be batched...each change can be actioned and delivered independently of all others. Work is enqueued and actioned according to priority and the required quality of service. Predictions are based on the actual rate of delivery, not on estimates. In a Lean-Kanban way of working, the actuals are indeed everything. Methods of estimation So then, estimates add value where scope is uncertain and there are associated risks to be managed. That's why Scrum teams engaged on projects typically make use of them, but Lean-Kanban BAU teams generally don't. Now let's look at three simple methods of estimation that Scrum teams, or other teams doing project work, can make use of. Planning Poker This is a well established technique popularised by Mike Cohn, and variations on his Planning Poker cards can be found in offices across the world. A typical Planning Poker set has cards with the following numbers: ½ 1 2 3 5 8 13 20 40 100. Nerds will observe and be irritated by the fact that this is roughly (but not quite) in line with the Fibonacci sequence. Here's how it's played: An identical hand of cards is given to each team member. Each team member will have a set of cards with numbers on the above pseudo-Fibonacci scale. The Product Owner describes the piece of work to be estimated. Normally this is a user story with acceptance criteria. Each team member mentally estimates the size of it on the scale. They can ask the Product Owner questions to clarify any points, but for the moment they will keep their estimate to themselves. Each team member places the card that corresponds to their mental estimate face down in front of them. At the facilitators instruction - usually the Scrum Master - the team turn their cards over. In an ideal case the cards will all have the same value, suggesting that the team have a common understanding of the requirements and the likely effort that will be involved. If the values are different, the team then need to discuss their estimates and their reasoning behind them. They need to understand each other’s thinking, and from that reach a consensus. It may be necessary to replay the cards several times before agreement is reached. By convention, estimates are written on the corner of a User Story card before being placed on a Scrum board. A variation of this takes from a suite of regular playing cards. The Ace (1), 2, 3, 5, 8, Jack, Queen, and King might be used. The Jack signifies that no or negligible work needs doing (jack all). The Queen indicates a larger story that should be broken down in the planning session and reconsidered, while the King indicates an epic that will need greater analysis and cannot be brought into scope for this Sprint. The Joker can be played if anyone wants a coffee break. As an estimation method, Planning Poker has the advantage of being fairly democratic. Every team member gets a hand of cards and is allowed to play, and has a clear opportunity to explain their reasoning to the others. The disadvantage of Planning Poker is that it can be rather time consuming in comparison with other methods. It can also encourage novice teams to estimate in terms of time, as they are often initially prejudiced to correlate points to hours or days. This prejudice must be challenged and eroded if the relative sizing of estimates is to be achieved. Team Sort (T-Shirt Sizing) This is a good way of doing team estimates if no planning poker cards are available. All you need are six scraps of paper and a set of index cards with the requirements (e.g. user stories) written on them. Normally these will be the same index cards that go on the Scrum board. Write one of the following sizes on each of the scraps of paper: Extra Small (XS), Small (S), Medium (M), Large (L), Extra Large (XL), and Extra Extra Large (XXL) Arrange the sizes in a horizontal line on a table, ordered from XS on the left to XXL on the right. Put the pile of index cards on the table in front of the sizing line. The team then collaborate to organise the requirements on the cards under the headings XS to XXL. They can ask the Product Owner to clarify any questions that they may have while doing so. Once the cards have all been sorted, story points can be allocated to each of them by mapping each T-Shirt size to a value. This allows metrics to be gathered about the flow of work, and used to populate a velocity or burndown chart. T Shirt Size Suggested Story Point Value XS 1 S 2 M 3 L 5 XL 8 XXL 13 An advantage of the team sort is that it is quick and easy to do. The complete set of requirements is estimated in one sweep. Also, it is a fairly direct way of achieving relative sizing. There is no temptation to correlate points to hours. The disadvantage is that it is potentially undemocratic, in that assertive team members can dominate meeker ones with their opinions. There is a variant of the team sort which encourages more egalitarian behavior. Each team member takes it in turns to move one card by one position. They also have the option to pass, i.e. to not move a card. Eventually a consensus should be reached and no more cards will be moved. However this is a more time consuming method and deadlocks can occur. These deadlocks can be difficult to spot if multiple card shifts are caught in the cycle. One Point One Card This method is a spin on the Lean-Kanban approach of tracking actuals. Instead of estimating the relative effort required for each story card, the team estimates how many stories it is likely to complete in the Sprint being planned. This can be as straightforward as using the yesterday's weather analogy for velocity estimation. Just as the weather today is most likely to resemble the weather yesterday, the velocity that will be achieved by a team in the upcoming Sprint will most likely match the velocity of its predecessor. So if two dozen cards were completed in the last sprint, approximately two dozen can be expected in the one that follows. The budget can be adjusted to allow for holiday, foreseeable absences, and other such changes that will impact the team's commitment. The advantage of this system is its raw simplicity. The estimation overhead is almost negligible. Also, it encourages the authoring of small user stories that will spend little time in progress and that stand little chance of being impeded. The liquidity of the board is therefore increased and further requirements analysis is encouraged. Some variation in size will be inevitable, and there will be statistical outliers, but the effects of these will average out as the flow rate increases. The disadvantage of this technique lies in the separation of fine-grained user stories from business value. There is a significant risk that they will become excessively technically focused and task-like. Conclusion Agile estimation is often seen as being invaluable, yet others dismiss it as waste. The reasons for this disagreement can be traced to disparities in Scrum and Lean-Kanban ways of working, and to the fundamental differences between project work and Business As Usual. When seen in the context of Scrum projects, some form of estimation process is valuable. Yet regardless of the method chosen, it must be acknowledged that a Scrum Team is responsible for its own estimates. No-one else can make a team's estimates for them. Going through that process of estimation, and understanding the size and scope of the work, is fundamental to the team's sense of Sprint Backlog ownership and to their commitment to a Sprint Goal.

Richard Hundhausen has put together a great list of funny Scrum/Agile related videos. Some of these are classics such as High Moon Studios: Portrait of Scrum and Adam Weisbart’s Shit Bad Scrum Masters say. Be warned, not all of these are actually that funny. I’ve never found The Downfall of Agile Hitler to be funny, because the original film is harrowing and difficult to watch. I also find the Scrum Haka to be trite and unwatchable … this is what a real haka should look like. So here’s the list and, once you’re done (in the words of Adam Weisbart), “Get back to work!” I want to run an agile project http://www.youtube.com/watch?v=4u5N00ApR_k I want to run an agile project (part 2) http://www.youtube.com/watch?v=lAf3q13uUpE The Power of Scrum (Ian Sense Scrum Master) http://www.youtube.com/watch?v=P6v-I9VvTq4The making-of http://www.youtube.com/watch?v=ncjdtqf1gSg Developer Abuse http://www.youtube.com/watch?v=LYlhCGng5Mk Spooning and pair programming http://www.youtube.com/watch?v=dYBjVTMUQY0 Improving Sprint Reviews (is that Jeroen?) http://www.youtube.com/watch?v=fpBQ5yxrR_c The Downfall of Agile Hitler http://www.youtube.com/watch?v=l1wKO3rID9g High moon studios: Portrait of Scrum http://www.youtube.com/watch?v=UT4giM9mxHk Shit Bad Scrum Masters Say http://www.youtube.com/watch?v=GGbsgs611MM The Scrum Haka (hideous) http://www.youtube.com/watch?v=Qvqq97unS2w Joe Justice Team WikiSpeed http://www.youtube.com/watch?v=x8jdx-lf2Dw Deathstar Project Deployment Meeting http://www.youtube.com/watch?v=2T5QNcb_Z8g Raking Leaves – A Scrum/Agile Approach http://www.youtube.com/watch?v=StBS-loIIz4 I Need Agile Methodology http://www.youtube.com/watch?v=nvks70PD0Rs

Application Services Governance is a necessary step towards building a responsive IT organization and achieving business agility. By guiding teams through a streamlined application services development process, Application Services Governance Platforms optimize IT effectiveness, raise software quality, and reduce delivery timeframes. Governance relies on policy, people, process and technology to guide business activity and consistently deliver positive outcomes. Effective governance channels business activity towards the ‘right’ path; by making the right actions the path of least resistance. To efficiently guide teams and demonstrate policy compliance benefits, Application Services Governance Platforms provide policy management, developer portals, repositories, service integration and composition, and business value dashboards. Effective governance encompasses the entire IT solution spanning APIs, services, business processes, data, and application delivery. While most governance solutions focus on web services, leading Application Services Governance Platforms bridge API governance, SOA governance, Cloud deployment governance, data governance, and application delivery governance. Additionally, the governance experience must be tailored for the participant’s project role. Portals may be personalized to present notifications, tasks, actions, and reports suitable for application service creators, publishers, subscribers, consumers, or business managers. Application delivery governance segments participants into developers, quality assurance testers, operations, project managers, and application users. End-user Application Services Governance priorities are evolving toward bridging service governance with API governance, extending application lifecycle management to embrace cloud deployment environments, and focusing on visualizing asset business value. Key governance challenges include meeting mobile application demands, implementing efficient self-service provisioning, right-sizing governance practices (not too heavy or light), and defining appropriate policy tiers. Governance Components To efficiently guide teams and demonstrate policy compliance benefits, Application Services Governance Platforms provide policy management, developer portals, repositories, service integration and composition, and business value dashboards. Figure 1 Application Services Governance Components Policy Management Policy management is used to specify the correct behavior, detail exception thresholds, and define corrective actions or notifications. Leading application services governance platforms deliver advanced policy management by conforming to a flexible architecture, addressing relevant policy categories, and spanning all lifecycle phases. A comprehensive Application Services Governance Platform manages: Design-time Policy Run-time Policy Security Policy Developer access Policy Service and API Lifecycle Management Policy Application Lifecycle Management Policy Within these six broad categories, application services governance commonly encompasses service level policies, usage policies, version policies, subscription policies, and access control policies. Registries serve as policy stores for many types of runtime policies including security policies, lifecycle management workflow policies, API policies, service description, service contracts, service consumption, service usage, service lifecycle management, service level agreements (SLAs) and XACML authorization policies. Leading platforms have built-in support for a number of policy standards including WS-Policy, XACML 3.0, and SCXML. Cloud foundation and cloud middleware components deliver sophisticated run-time policy enforcement for tenant partitioning, service level management, application provisioning, tenant access, and resource management. All run-time infrastructure products should serve as well-integrated policy enforcement points that may delegate policy decisions to external decision points or internally cache and process policy assertions. Identity Management infrastructure components serve as a policy decision point and a policy manager for sophisticated security policies encoded in XACML. The Application Service Governance Platforms use workflow engines to execute governance workflow, present task lists, and manage approvals. Complex Event Processor components can be configured as policy decision points, which use time-based policy pattern matching to evaluate run-time service, message, REST resource, and event traffic. For more information on policy management, read the detailed policy management blog post. Developer Portal and Repository Portals serve as the viewport into policy management, service integration and composition, and business value dashboards. The Application Service Governance portals should deliver an application service governance experience tuned for self-service, on-demand access, and safe API usage. Developer portals are often contextually personalized to fit the project and user’s role. For example, a developer portal may fit the needs of API creators and API publishers who are defining, documenting, and publishing APIs. The portal’s user experience may enable API creators and publishers to monitor, manage, and analyze API usage. A developer portal may also be personalized to deliver a user experience tailored for API consumers. API developers who are consuming APIs can find, explore, subscribe and evaluate APIs. Developer portals are often tuned to facilitate service meta-data and lifecycle management for service creators. Service and integration developers who are consuming services can find and explore services. A developer portal should guide teams toward effective and efficient governance when building service implementation and service consumption code. Advanced developer portals capabilities include overlaying build management governance, test governance (i.e. unit, integration, performance), implementation lifecycle governance, and deployment governance. An Application Services Governance Platform should enable flexible organization, classification & documentation of services, APIs, and any IT asset. Key repository capabilities include governing and managing: Any type of metadata in any structure Service, API, or artifact associations and relationships Schema definitions and namespaces Users and Roles User subscriptions Service level agreements Developer documentation Social taxonomies (e.g. ratings, comments, tags) Implementation artifacts (i.e. code, test cases) Service Integration and Composition Service integration and composition for APIs, web services, or business process are often implemented using tools provided by the run-time infrastructure vendor. Application Services Governance components must integrate into diverse run-time infrastructure containers and development tooling. Synchronizing policy, development artifacts, and deployment packages requires tight integration between design-time tools, development tools, run-time management consoles, and application services governance portals and repositories. Business Value Dashboards To gauge governance effectiveness and enhanced business value, analytic dashboards assess policy compliance, quality of service, service usage, architecture coherence, and team performance. The Application Services Governance platform should capture service tier subscription information, collects usage statistics, and integrate with billing and payment systems that deliver show-back or charge-back reports. Subscription and usage reports help teams understand asset adoption (by version, by service) and usage (by version, by service). By understanding adoption and usage, business owners and architects can intelligently invest future development resources, properly plan infrastructure scale, and rationalize the portfolio. Dashboards also present a service overview, number of services, service lifecycle stage, schema re-use, service dependencies, upgrade impacts, development team productivity, and project progress. Governance Lifecycle Phases API management portals and SOA Governance Registries must work together to keep API lifecycle stages synchronized with backend service implementation stages. An API Governance experience may provide a straightforward set of lifecycle stages (e.g., created, published, deprecated, retired, blocked) that may be customized by the development team. SOA Governance Registries facilitates service metadata management and governance across design, implementation, test, and run-time operations. Figure 2 below depicts the intersection of the two governance views. Figure 2: API and Service Lifecycle Views Application delivery governance usually relies on ad hoc tools and processes, knitted together by end-user delivery managers. Application Services Governance Platforms should span project inception, development, quality assurance, production deployment, production management, maintenance, and retirement. Figure 3 illustrates service implementation activities governed by an application delivery governance product. Figure 3: Implementation activities governed by application services delivery governance Application Services Governance Drivers The IT focus on API, DevOps, and Cloud scale is driving resurgent interest in Application Services Governance. As development teams support mobile applications by fielding web APIs, they are creating a new ‘demand layer’ in front of existing service implementations. Both API and SOA success requires creating loosely coupled consumer-provider connections, enforcing a separation of concerns between consumer and provider, and exposing a set of re-usable, shared services, and gaining service consumer adoption. With traditional SOA Governance, many development teams publish services, yet struggle to create a service architecture that is widely shared, re-used, and adopted across internal development teams. In today’s connected business world, API and SOA are the business. An effective governance approach must address human collaboration stumbling blocks. By publishing managed APIs, establishing API manager and publisher roles, extending the governance registry, facilitating API management practices (e.g self-service key management, self-service provisioning, service tier management, and usage visualization),and offering APIs through developer portal, organizations can overcome collaboration, trust, and adoption hurdles while enhancing SOA success. By publishing managed APIs, establishing API manager and publisher roles, extending the governance registry, and offering APIs through an API Store, team have a new opportunity to increase service re-use and enhance IT business value. For more information on how teams can complement SOA Governance with API Governance, read the promoting services with API Management white paper. Because services are often imbedded in application solutions, leading Application Services Governance platforms wrap services governance inside application delivery governance. When operation team members use traditional point tools (i.e. Puppet, Chef, Jenkins,Selenium) to achieve DevOps benefits, the teams spend a considerable amount of time and effort creating agile workflow, effective governance, seamless activity transitions, and on-demand self-service access. A configurable DevOps PaaS can implement governance best practices and be readily adopted by teams without extensive implementation effort. Effective application delivery governance presents a simplified and unified user experience to complex development tools, processes, and team hand-offs. By integrating software promotion best practices, test automation, continuous integration, and issue tracking, application delivery governance raises software quality while reducing delivery timeframes. For more information, read about how to accelerate agility and maintain governance with DevOps PaaS. Recommended Reading Policy Management for Application Services Governance Application Services Governance Requires More Than a SOA Registry API and SOA Convergence Promoting services with API Management white paper Accelerate agility and maintain governance with DevOps PaaS Governance Registry Brings Integrity to SaaS Platform Gartner’s analysis of WSO2 SOA Governance

Here are 10 articles from 10 different authors that provide valuable advice for Scrum teams. These articles are in no particular order, so feel free to skim down the list and start with the ones that are most relevant to you. 10 Tips for a Great Daily Scrum Meeting by Platinum Edge – The daily Scrum meeting is a powerful tool that keeps your project moving. At the same time, it is also easy for the meetings to not bring any added value. Tips for Effective Backlog Grooming by Charles Bradley – Are you wasting time in your Sprint Planning Meetings? Increase the value of your team’s Sprint Planning Meetings by grooming your Product Backlog. Yoda’s top 10 tips for a new Scrum Master by Nigel Steane – As a new Scrum Master, you face unfamiliar challenges and your success is very much based on your ability to utilise coaching and soft skills to gently guide your team and colleagues. Top ten tips for distributed Scrum team teleconferences By Jon Archer – After acting as a Scrum Master for several months on a distributed team with people in six different locations, three different countries, learn ten tips to help get past those inevitable awkward silences. 10 tips for adopting Scrum to save your project by Matthew Hodgson – Are you interested in adopting Scrum for your next project? Here are 10 tips from his experience with moving a number of projects from their existing project management frameworks to Scrum. Five Tips for Impediment Resolution with Scrum by Stefan Roock – Impediments can slow down or even halt the progress of an otherwise well-functioning Scrum team. Take a look at the most common challenges that crop up on teams and what steps you can take to resolve them. 10 Tips for Succeeding with Enterprise Agile Development by Tools Journal – Many enterprises are experimenting with agile development approaches like Scrum, Kanban, Lean, and XP hoping that introducing a new development approach will help. Yet, agile development has struggled to achieve critical mass in large enterprises. 6 Tips for Good Scrum by Martin Harris – If you are doing these 6 tips, then you are doing very well and are likely to get better over time. 9 Tips for Creating a Good Sprint Backlog by Luciano Felix – Giving attention to the sprint backlog creation process is fundamental to the team’s understanding of what should be done and how to better plan during the sprint. 7 Tips for a More Effective Daily Scrum by Richard Lawrence – The main purpose of the Daily Scrum is for team members to make and follow-up on commitments to one another that work towards the team’s shared sprint commitment. Here are seven ways to get your Daily Scrum back on focus If your it has become unfocused, too long, or otherwise ineffective. If you have any other good articles related to agile, please share them in the comments. Thanks.

I've been working as a Scrum Master and as an Agile Coach for a good few years now, mainly as a contractor. Each time I am interviewed for a new contract I always like to ask if I can meet the teams I’d be working with. You see, time and again it will be a manager who does the interviewing, while the team members themselves are left with little say in whether I should be hired. I think it’s important that they reckon they can get along with me. Of course, it also gives me an opportunity to see them, and to gain a fuller understanding of the situation I’d really be walking into. As we head towards the desks of my prospective team, one of the first things I look for is the board, whether it be a Scrum task board or a Kanban board. Most teams with agile aspirations…or agile pretensions…will have set up a board of some kind. A board is the "grand old dame" of information radiators. No matter how much the details of a sordid past are glossed over, the truth always seems to come out. It's in the nature of a board to tell the truth, since any untruths can be quickly exposed. The story I can piece together from dubious lanes and columns, misplaced or missing tickets, misplaced or missing avatars, and a host of other shibboleths can be far more telling than anything I get to hear from people in an interview situation. Another of the things I look for is a "fast track" lane on a Scrum Team's board. These are very common; you could say it is almost unusual not to see them. From a certain perspective they are good things to have, and they can imply a level of maturity - or at least of pragmatism - on the part of a team. They suggest that the team accepts that not everything can be predicted in Sprint planning. A fast track lane is a nod to the fact that emergencies happen, that support work and unforeseen defect fixes still need to be done, and most importantly, that the team has a way of dealing with all of this. However it also shows that they aren't doing Scrum. There...I've said it. Fast track lanes aren't part of Scrum. It's that simple. I don't mean to say that they are bad practice, or in some sense un-agile. On the contrary, they are part of the Lean Kanban approach to varying the Quality of Service provided to certain backlog items. That's what a fast track lane is...a way of varying the quality of service that a Scrum team gives to certain items. When something hits a fast track lane, a well-trained team will swarm over it and decide who is best qualified to progress the matter. While they do this, their own tasks will be marked as impeded or blocked. Then, the decision made, all others return to their work in progress. So if fast track lanes are a widely understood and practical way of managing operational issues, what is wrong with them, Scrum-wise? The answer is that Scrum - unlike Lean Kanban - doesn't provide for variations in quality of service. Each piece of work is prioritized and negotiated into a Sprint backlog. The team then self-organizes to deliver a corresponding increment of functionality. The team will plan with the Product Owner what it intends to do during a sprint, and the sprint backlog they agree to belongs to them. No-one, not even the CEO of the organization, can override their sprint backlog by introducing work to be "fast tracked". The team wholly owns their sprint backlog. That's Scrum. When I point this out, teams can become crestfallen or even defensive. “What else are we supposed to do”, they say. “We aren’t dedicated 100% to doing project work. We still have support work to do, and serious issues always trump development. We have to fix them and put project work on hold.” My answer to that is that under the circumstances the team is facing, it may indeed be right to vary the quality of service by fast-tracking support work. It just isn’t Scrum, that’s all. It’s a type of "Scrumban", a Scrum variant that includes Kanban characteristics. This is no fault of the team, but it could suggest a problem higher up. Perhaps a dedicated Kanban support team hasn’t been properly resourced and trained so that Scrum development can proceed unimpeded. Perhaps the Product Owner is being undermined by other managers who have separate interests impacting the development. Whatever the situation, it needs to be made transparent and acknowledged by all stakeholders. So, the next step…and the one I’ll often indicate as the interview progresses…is to account for fast track work as impediments against product burndown or velocity. Moreover, these are impediments which are external to the team. It’s essentially a type of waste, or unplanned work, being generated from outside. It needs to be made quite transparent where this waste is coming from and what can be done to mitigate it. What can be done about those other teams, or workflows, or managers, who are undercutting this Scrum team’s ability to plan out their Sprints? Often, the source of these impediments will be the people interviewing me...and that’s when things can start to get really interesting!

With unit tests you can check that your code behaviours just as you expect it to. When writing your unit tests you shouldn't need to worry about if any other area of the application is working correctly. The benefits of unit testing are: Decouples your code Write more modular classes Functions are smaller and more focused Your functions are more defensive Quality of code becomes higher You will find it easier to reuse code. When writing unit tests you just need to test this one method of your application, if your method relies on another class/variable there should be a way you can inject this into the method. This is where dependency injection in your code comes in handy, it will allow you to inject objects into your classes to change the output of the class. There are a few things you need to do to make a method unit testable, methods will need an input from a parameter or a class variable and it will need a return or set a class variable in the method. If the method hasn't got these things then the method can not be unit testable. If there isn't a return of the method then there is no way in knowing how the method performs. Dependency Injection Dependency injection is when your object has a dependency on another object. The simplest form to understand what dependency injection is to think of a setter method. A setter method will take one parameter and set a class variable from this parameter. This is using code injection to pass in a parameter to be used as the class variable value. public function setValue( $val ) { $this->val = $val; } Without dependency injection this method will look like this. public function setValue() { $this->val = 10; } For unit testing you need to be aware of any classes that your class is dependent on. For example if you have a login class that will connect to a database. class login { private $db = false; public function __construct() { $this->db = new Database(); } public function loginUser( $user, $password ) { $this->db->checkLogin( $user, $password ); } } This login class has a dependency of the class Database in the constructor, which means that we can't unit test this correctly. If we want to unit test this then the database class has to be development and tested. If the database class is broken and we try to unit test the loginUser() method the test will always fail and we won't know that it's the database class which is broke or the loginUser() method that is broke. If the database class is finished development, tested and data is in the database then we can use this for the loginUser() function. But now our tests are dependent on data being correct in the database. If we pass in a username and password it must be in the database for our test to pass. Our code could be correct but if the data isn't there then our unit tests will fail. This isn't correct use of unit tests and is more suited to be an integration test. To fix this problem we can use dependency injection to pass in a database connector which will set the database class variable. There are 2 ways we can inject a variable into a class, it can either be in the constructor of the class or by using a setter method. I tend to use constructor for all required dependences and use the setter method if there is a default value for the class variable. class login { private $db = false; public function __construct( $db ) { $this->db = $db; } public function loginUser( $user, $password ) { $this->db->checkLogin( $user, $password ); } } Now this class isn't dependant on a certain database class we can pass in the database class by using the parameter on the login class constructor. We can unit test this loginUser() method by first setting the $this->db class variable. We don't want to rely on a real database as the data can change so we can either create a test harness database class or you can mock the database class. A test harness class will allow you to create your database class and hardcode any data that you need. In the example above we can create a method checkLogin(), in our test harness we can then hardcode a successful login username and password to make the loginUser() method pass. Or you can use a PHP mocking framework to mock a class/method/return value. Both methods have their benefits but mocking is normally quicker to code, but there are times when you want to hardcode certain variables in a class. Mocking Objects In TDD With PHP Mocking objects in test driven development allows you create objects to act as a certain class, if your test depends on another method to return a value, you can mock this method and make it return any value you want. In the example we used above you can mock the database class and choose what value we are expecting back from the checkLogin() method. When mocking a method you can choose what you want to return from this method, therefore we can write tests to see what will happen when checkLogin() returns TRUE and then we can write another test to see what happens when checkLogin() returns FALSE. Mocking objects means that you can run your unit tests without depending on another class returning the values you are expecting, ao you can test just your code in this one method. Here are some of the most popular PHP mocking frameworks: Mocking with PHPUnit - http://www.phpunit.de/manual/3.0/en/mock-objects.html Mocking with Phake - http://phake.digitalsandwich.com/docs/html/ Mocking with Mockery - https://github.com/padraic/mockery Mocking with Enchane PHP - https://github.com/Enhance-PHP/Enhance-PHP Mocking with FBMock - https://github.com/facebook/FBMock Dependency Injection With Interfaces If we are going to pass in a database connector in a constructor of the login class, then this database connector will always have to have a method of checkLogin(). This is why we should code our dependences by using interfaces to make sure that we are always passing in the correct type of class. class login { private $db = false; public function __construct( IDatabase $db ) { $this->db = $db; } } class database implements IDatabase { public function checkLogin( $username, $password ) { // check the login credentials } } interface IDatabase { public function checkLogin( $username, $password ); } This will make sure that the class we pass into the constructor is a type of IDatabase, so if our database class doesn't implement IDatabase then the code will fail and therefore our unit tests will fail. This means whatever we pass into the constructor we know that this class will be able to run the methods it needs for the unit tests to run.

I Programmer - Anime Scrum - An Overview If you are an anime fan, and perhaps even if you are not, then you might like a new poster about Scrum - in anime style. As long as you find anime cute or something then seeing the different people involved in the Scrum methodology as anime characters might help you convey the ideas to others. ... Scrum Primer - Scrum Overview - Anime version High-resolution versions of the overview: Scrum Overview - Blue Scrum Overview - Pink Scrum Overview - Green Feel free to use it in your own material. ..." Come on! You KNOW that's awesome! :)

From an original article on Wakaleo.com Conventional test reports, generated by tools such as JUnit or TestNG, naturally focus on what tests have been executed, and whether they passed or failed. While this is certainly useful from a testing perspective, these reports are far from telling the whole picture. BDD reporting tools like Cucumber and JBehave take things a step further, introducing the concept of "pending" tests. A pending test is one that has been specified (for example, as an acceptance criteria for a user story), but which has not been implemented yet. In BDD, we describe the expected behaviour of our application using concrete examples, that eventually form the basis of the "acceptance criteria" for the user stories we are implementing. BDD tools such as Cucumber and JBehave not only report on test results: they also report on the user stories that these tests validate. However this reporting is still limited for large projects, where the numbers of user stories can become unwieldy. User stories are not created in isolation: rather, user stories help describe features, which support capabilities that need to be implemented to achieve the business goals of the application. So it makes sense to be able to report on test results not only at the user story level, but also at higher levels, for example in terms of features and capabilities. This makes it easier to report on not only what stories have been implemented, but also what features and capabilities remain to be done. An example of such a report is shown in Figure 1 (or see the full report here). Figure 1: A test coverage report listing both tested and untested requirements. In agile projects, it is generally considered that a user story is not complete until all of its automated acceptance tests pass. Similarly, a feature cannot be considered ready to deliver until all of the acceptance criteria for the underlying user stories have been specified and implemented. However, sensible teams shy away from trying to define all of the acceptance criteria up-front, leaving this until the "last responsible moment", often shortly before the user story is scheduled to be implemented. For this reason, reports that relate project progress and status only in terms of test results are missing out on the big picture. To get a more accurate idea of what features have been delivered, which ones are in progress, and what work remains to be done, we must think not in terms of test results, but in terms of the requirements as we currently understand them, matching the currently implemented tests to these requirements, but also pointing out what requirements currently have no acceptance criteria defined. And when graphs and reports illustrate how much progress has been made, the requirements with no acceptance criteria must also be part of the picture. Requirements-level BDD reporting with Thucydides Thucydides is an open source tool that puts some of these concepts into practice. Building on top of BDD tools such as JBehave, or using just ordinary JUnit tests, Thucydides reports not only on how the tests did, but also fits them into the broader picture, showing what requirements have been tested and, just as importantly, what requirements haven't. You can learn more about Thucydides in this tutorial or on the Thucydides website. During the rest of this article, we will see how to report on both your requirements and your test results using Thucydides, using a very simple directory-based approach. You can follow along with this example by cloning the Github project at https://github.com/thucydides-webtests/thucydides-simple-demo Simple requirements in Thucydides - a directory-based approach Thucydides can integrate with many different requirement management systems, and it is easy to write your own plugin to tailor the integration to suite your particular environment. A popular approach, for example, is to store requirements in JIRA and to use Thucydides to read the requirements hierarcy directly from the JIRA cards. However the simplest approach, which uses a directory-based approach, is probably the easiest to use to get started, and it is that approach that we will be looking at here. Requirements can usually be organized in a hierarchial structure. By default, Thucydides uses a three-level hierarchy of requirements. At the top level, capabilities represent a high-level capacity that the application must provide to meet the application's business goals. At the next level down, features help deliver these capabilities. To make implementation easier, a feature can be broken up into user stories, each of which in turn can contain a number of acceptance criteria. Figure 2: JUnit test directories mirror the requirements hierarchy. Of course, you don't have to use this structure if it doesn't suit you. You can override the thucydides.capability.types system property to provide your own hierarchy. For example, if you wanted a hierarchy with modules,epics, and features, you would just set thucydides.capability.types to "module,epic,feature". When we use the default directory-based requirements strategy in Thucydides, the requirements are stored in a hierarchial directory structure that matches the requirements hierarchy. At the lowest level, a user story is represented by a JBehave *.story file, an easyb story, or a JUnit test. All of the other requirements are represented as directories (see Figure 2 for an example of such a structure). In each requirements directory, you can optionally place a file called narrative.txt, which contains a free-text summary of the requirement. This will appear in the reports, with the first line appearing as the requirement title. A typical narrative text is illustrated in the following example: Learn the meaning of a word In order to learn the meaning of a word that I don't know As an online reader I want to be able to find out the meaning of the word If you are implementing the acceptance criteria as JUnit tests, just place the JUnit tests in the package that matches the correspoinding requirement. You need to use the thucydides.test.root system property to specify the root package of your requirements. For the example in Figure 2, this value should be set to nz.govt.nzqa.lssu.stories. Figure 3: The narrative.txt file appears in the reports to describe a requirement. If you are using JBehave, just place the *.story files in the src/test/resources/stories directory, again respecting a directory structure that corresponds to your requirement hierarchy. The narrative.txt files also work for JBehave requirements. Progress is measured by the total number of passing, failing or pending acceptance criteria, either for the whole project (at the top level), or within a particular requirement as you drill down the requirements hierarchy. For the purposes of reporting, a requirement with no acceptance criteria is attributed an arbitrary number of "imaginary" pending acceptance criteria. Thucydides considers that you need 4 tests per requirement by default, but you can override this value using the thucydides.estimated.tests.per.requirement system property. Figure 3: For JBehave, everything goes under src/test/resources/stories. Conclusion BDD is an excellent approach for communicating with, and reporting back to, stakeholders. However, for accurate acceptance test reporting on real-world projects, you need to go beyond the story level, and cater for the whole requirements hierarchy. In particular, you need to not only report on tests that have been executed, but also allow for the tests that haven't been written yet. Thucydides puts these concepts into practice: using a simple directory-based convention, you can easily integrate your requirements hierarcy into your acceptance tests.

In many Agile training programs and conferences, a common question that gets raised is, does Agile/Scrum work in maintenance projects? I always say "YES" and the team needs to tweak or invent the practices to suit their needs. Maintenance projects could be enhancement projects OR pure defect fixing projects. Enhancement projects involve new set of developments over existing one. Since the developers get a new set of requirements at the end of each iteration, one can apply the standard set of Scrum practices with little or no modification. Defect Fixing projects involve fixing defects on closed or current projects not in development. Sometimes these projects are boring, especially if a new team has been hired for defect fixing purposes only. The customer sends a set of defects on a daily basis or weekly basis with a deadline to deliver. The development team needs to fix them ASAP and send the patch for further testing. While coaching one of such a defect fixing projects, I found that the following Scrum practices can be applied without much modification: 1. Daily Scrum meetings 2. A Scrum of Scrum 3. Modeling days while solving complex defects 4. Information radiators displaying InProgress, completed, reopened, closed defects and other information 5. Usage of Wiki for collaborating with the customer 6. Requirement workshop while understanding complex defects 7. Review and Retrospective A common problem that I have found in defect fixing projects is setting the iteration length. Especially if the defects are given on a day to day basis without prior knowledge of what you are going to get, it makes the life of the development team bit difficult. This can be solved by collaborating with the customer and coming up with a plan to have 1 or 2 weeks of iteration length.

According to Steve McConnell in Code Complete (data from 1975-1992) most bugs don’t take long to fix. About 85% of errors can be fixed in less than a few hours. Some more can be fixed in a few hours to a few days. But the rest take longer, sometimes much longer – as I talked about in an earlier post. Given all of these factors and uncertainty, how to you estimate a bug fix? Or should you bother? Block out some time for bug fixing Some teams don’t estimate bug fixes upfront. Instead they allocate a block of time, some kind of buffer for bug fixing as a regular part of the team’s work, especially if they are working in time boxes. Developers come back with an estimate only if it looks like the fix will require a substantial change – after they’ve dug into the code and found out that the fix isn’t going to be easy, that it may require a redesign or require changes to complex or critical code that needs careful review and testing. Use a rule of thumb placeholder for each bug fix Another approach is to use a rough rule of thumb, a standard place holder for every bug fix. Estimate ½ day of development work for each bug, for example. According to this post on Stack Overflow the ½ day suggestion comes from Jeff Sutherland, one of the inventors of Scrum. This place holder should work for most bugs. If it takes a developer more than ½ day to come up with a fix, then they probably need help and people need to know anyways. Pick a place holder and use it for a while. If it seems too small or too big, change it. Iterate. You will always have bugs to fix. You might get better at fixing them over time, or they might get harder to find and fix once you’ve got past the obvious ones. Or you could use the data earlier from Capers Jones on how long it takes to fix a bug by the type of bug. A day or half day works well on average, especially since most bugs are coding bugs (on average 3 hours) or data bugs (6.5 hours). Even design bugs on average only take little more than a day to resolve. Collect some data – and use it Steve McConnell, In Software Estimation: Demystifying the Black Art says that it’s always better to use data than to guess. He suggests collecting time data for as little as a few weeks or maybe a couple of months on how long on average it takes to fix a bug, and use this as a guide for estimating bug fixes going forward. If you have enough defect data, you can be smarter about how to use it. If you are tracking bugs in a bug database like Jira, and if programmers are tracking how much time they spend on fixing each bug for billing or time accounting purposes (which you can also do in Jira), then you can mine the bug database for similar bugs and see how long they took to fix – and maybe get some ideas on how to fix the bug that you are working on by reviewing what other people did on other bugs before you. You can group different bugs into buckets (by size – small, medium, large, x-large – or type) and then come up with an average estimate, and maybe even a best case, worst case and most likely for each type. Use Benchmarks For a maintenance team (a sustaining engineering or break/fix team responsible for software repairs only), you could use industry productivity benchmarks to project how many bugs your team can handle. Capers Jones in Estimating Software Costs says that the average programmer (in the US, in 2009), can fix 8-10 bugs per month (of course, if you’re an above-average programmer working in Canada in 2012, you’ll have to set these numbers much higher). Inexperienced programmers can be expected to fix 6 a month, while experienced developers using good tools can fix up to 20 per month. If you’re focusing on fixing security vulnerabilities reported by a pen tester or a scan, check out the remediation statistical data that Denim Group has started to collect, to get an idea on how long it might take to fix a SQL injection bug or an XSS vulnerability. So, do you estimate bug fixes, or not? Because you can’t estimate how long it will take to fix a bug until you’ve figured out what’s wrong, and most of the work in fixing a bug involves figuring out what’s wrong, it doesn’t make sense to try to do an in-depth estimate of how long it will take to fix each bug as they come up. Using simple historical data, a benchmark, or even a rough guess place holder as a rule-of-thumb all seem to work just as well. Whatever you do, do it in the simplest and most efficient way possible, don’t waste time trying to get it perfect – and realize that you won’t always be able to depend on it. Remember the 10x rule – some outlier bugs can take up to 10x as long to find and fix than an average bug. And some bugs can’t be found or fixed at all – or at least not with the information that you have today. When you’re wrong (and sometimes you’re going to be wrong), you can be really wrong, and even careful estimating isn’t going to help. So stick with a simple, efficient approach, and be prepared when you hit a hard problem, because it's gonna happen.

Test-Driven Development is a code-level practice, based on running automated tests that are written before the production code they exercise. But practices can be applied only in the context where they were developed: when some premises are not present is difficult to apply TDD as-is. Automated specification For example, consider the premise of assertion automation: it is possible to write a (hopefully) small algorithm that is able to check the result of running production code and return true or false. In the case the problem is: Draw an antialiased circle on this blank canvas. -- Carlo Pescio it is not immediately clear how to define automated tests for this behavior. We could check that some pixels are still blank inside or outside the circle, or that there is a bound number of pixels of black color; or even that they are contiguous. An opinion I've heard (that I try not to misrepresent) is that we only need to write some looser tests in these cases, checking only a few pixels of the circle. This process will give us a little feedback on the API of our Canvas or Circle object, but not much on the algorithm we are implementing inside it. Are we going in the right direction? Have new test cases correctly been satisfied without a large intervention on the existing code? Are we painting some unrelated pixels due to an hidden bug? What I argument here is instead that we should change the nature of the feedback mechanism. Speaking in control theory terms, change the block that acquires the output and influences the input to our design process. Develop in the browser When I was developing a Couchapp, a kind of web application served directly from a CouchDB database, I was appaled by the difficulty of testing it. While the production code was composed of ~100 lines, it was a complex mix of technologies: HTML and CSS code, client-side JavaScript for managing user events and some server-side JavaScript for the "queries" (actually the server-side only consists of the database in Couchapps.) Some of this logic could be tested in automation, like the result of queries over views. Yet much of it was related to a user interface, and as such requiring a large time investment to automate. Instead of waking up my Selenium server and start to manipulate a browser with code, I noticed that this UI was almost read-only; there were a few cases where a new document would have to be inserted, but a manual test of them was short and did not even required to reload the page. The whole application state was observable. Summing it up, I performed a frequent manual test that took a few seconds instead of trying to define complex and brittle automation logic for testing the UI. Now that I've been introduced to a simple qualitative ROI model by Carlo Pescio's article, I would do the same for every context where: a large time investment is needed for automating tests. it is possible to perform manual tests quickly. as the only logic conclusion. A word of caution TDD has many benefits (including catching regressions early) so I'm not prepared to give it up just because it is difficult to test. These are technical scenarios where I have successfully followed TDD by the book: multithreaded and multiprocess code applications distributed over multiple machines computer vision (object recognition and tracking) image manipulation code (via comparison testing) development of browser bindings for Selenium And even in the case the big picture is not easy to test-first (like in the case of image manipulation), we can benefit from TDD the pieces of the solution. For example, in the computer vision case I wasn't able to write a test beforehand for tracking a car inside a movie. But I was able to TDD the objects that the algorithmic solution to the problem called for: Patch, Area, Cluster, Movement, and so on. End-to-end TDD is not always cheap but unit level TDD can often be, if it considers testability as a relevant property (while regression testing even at the end-to-end level is always possible, in the worst case with record and replay.) End-to-end specifications If we can't define automated assertions for our "big picture" problem, it doesn't mean that we cannot apply the TDD approach, by substituting a manual step. Going back to the circle problem, I would define manual test cases on an inspection page seen by a human. I've seen this done with layouts and multiple browsers to catch CSS rendering bugs, for example: It would be very difficult to check these screenshots automatically, as each browser renders pages a bit differently from the others. The iterative process becomes: Define a cheap manual test, automating the arrange and act phases but not the assertion. Write only the code necessary to make it pass. Refactor. As long as the number of tests does not increase without limit and the manual check can be performed quickly, this approach does not slow you down with respect to TDD by-the-book. You'll have to take care of regression with other means; but at least you define a set of manual test cases. Feedback! TDD is an instrument of feedback: if feedback cannot be gathered in an automated way, we have to resort to manual checking of the specifications. Here are other examples of manual tools for generating feedback: Read-Eval-Print Loops: you can experimenting with existing classes and functions, and easily repeat steps thanks to history. the browser refresh button: the fastest way to transform a PSD into an HTML and CSS template. MongoDB console for learning the database API; other kinds of consoles like Firebug and Chrome's, or Clojure's.

in working with development teams at organizations which are adopting continuous delivery , i have found there can be friction over practices that many developers have come to consider as the right way for agile teams to work. i believe the root of conflicts between what i’ve come to think of as traditional agile and cd is the approach to making software “ready for release”. evolution of software delivery a usefully simplistic view of the evolution of ideas about making software ready for release is this: waterfall believes a team should only start making its software ready for release when all of the functionality for the release has been developed (i.e. when it is “feature complete”). agile introduces the idea that the team should get their software ready for release throughout development. many variations of agile (which i refer to as “traditional agile” in this post) believe this should be done at periodic intervals. continuous delivery is another subset of agile which in which the team keeps its software ready for release at all times during development. it is different from “traditional” agile in that it does not involve stopping and making a special effort to create a releasable build. continuous delivery is not about shorter cycles going from traditional agile development to continuous delivery is not about adopting a shorter cycle for making the software ready for release. making releasable builds every night is still not continuous delivery. cd is about moving away from making the software ready as a separate activity, and instead developing in a way that means the software is always ready for release. ready for release does not mean actually releasing a common misunderstanding is that continuous delivery means releasing into production very frequently. this confusion is made worse by the use of organizations that release software multiple times every day as poster children for cd. continuous delivery doesn’t require frequent releases, it only requires ensuring software could be released with very little effort at any point during development. (see jez humble’s article on continuous delivery vs. continuous deployment .) although developing this capability opens opportunities which may encourage the organization to release more often, many teams find more than enough benefit from cd practices to justify using it even when releases are fairly infrequent. friction points between continuous delivery and traditional agile as i mentioned, there are sometimes conflicts between continuous delivery and practices that development teams take for granted as being “proper” agile. friction point: software with unfinished work can still be releasable one of these points of friction is the requirement that the codebase not include incomplete stories or bugfixes at the end of the iteration. i explored this in my previous post on iterations . this requirement comes from the idea that the end of the iteration is the point where the team stops and does the extra work needed to prepare the software for release. but when a team adopts continuous delivery, there is no additional work needed to make the software releasable. more to the point, the cd team ensures that their code could be released to production even when they have work in progress, using techniques such as feature toggles . this in turn means that the team can meet the requirement that they be ready for release at the end of the iteration even with unfinished stories. this can be a bit difficult for people to swallow. the team can certainly still require all work to be complete at the iteration boundary, but this starts to feel like an arbitrary constraint that breaks the team’s flow. continuous delivery doesn’t require non-timeboxed iterations, but the two practices are complementary. friction point: snapshot/release builds many development teams divide software builds into two types, “snapshot” builds and “release” builds. this is not specific to agile, but has become strongly embedded in the java world due to the rise of maven, which puts the snapshot/build concept at the core of its design. this approach divides the development cycle into two phases, with snapshots being used while software is in development, and a release build being created only when the software is deemed ready for release. this division of the release cycle clearly conflicts with the continuous delivery philosophy that software should always be ready for release. the way cd is typically implemented involves only creating a build once, and then promoting it through multiple stages of a pipeline for testing and validation activities, which doesn’t work if software is built in two different ways as with maven. it’s entirely possible to use maven with continuous delivery, for example by creating a release build for every build in the pipeline. however this leads to friction with maven tools and infrastructure that assume release builds are infrequent and intended for production deployment. for example, artefact repositories such as nexus and artefactory have housekeeping features to delete old snapshot builds, but don’t allow release builds to be deleted. so an active cd team, which may produce dozens of builds a day, can easily chew through gigabytes and terabytes of disk space on the repository. friction point: heavier focus on testing deployability a standard practice with continuous delivery is automatically deploying every build that passes basic continuous integration to an environment that emulates production as closely as possible, using the same deployment process and tooling. this is essential to proving whether the code is ready for release on every commit, but this is more rigorous than many development teams are used to having in their ci. for example, pre-cd continuous integration might run automated functional tests against the application by deploying it to an embedded application server using a build tool like ant or maven. this is easier for developers to use and maintain, but is probably not how the application will be deployed in production. so a cd team will typically add an automated deployment to an environment will more fully replicates production, including separated web/app/data tiers, and deployment tooling that will be used in production. however this more production-like deployment stage is more likely to fail due to its added complexity, and may be may be more difficult for developers to maintain and fix since it uses tooling more familiar to system administrators than to developers. this can be an opportunity to work more closely with the operations team to create a more reliable, easily supported deployment process. but it is likely to be a steep curve to implement and stabilize this process, which may impact development productivity. is cd worth it? given these friction points, what benefit is there to moving from traditional agile to continuous delivery worthwhile, especially for a team that is unlikely to actually release into production more often than every iteration? decrease risk by uncovering deployment issues earlier, increase flexibility by giving the organization the option to release at any point with minimal added cost or risk, involves everyone involved in production releases - such as qa, operations, etc. - in making the full process more efficient. the entire organization must identify difficult areas of the process and find ways to fix them, through automation, better collaboration, and improved working practices, by continuously rehearsing the release process, the organization becomes more competent at doing it, so that releasing becomes autonomic, like breathing, rather than traumatic, like giving birth, improves the quality of the software, by forcing the team to fix problems as they are found rather than being able to leave things for later. dealing with the friction the friction points i’ve described seem to come up fairly often when continuous delivery is being introduced. my hope is that understanding the source of this friction will be helpful in discussing it when it comes up, and working through the issues. if developers who are initially uncomfortable with breaking with the “proper” way of doing things, or find a cd pipeline overly complex or difficult understand the aims and value of these practices, hopefully they will be more open to giving them a chance. once these practices become embedded and mature in an organization, team members often find it’s difficult to go back to the old ways of doing them. edit: i’ve rephrased the definition of the “traditional agile” approach to making software ready for release. this definition is not meant to apply to all agile practices, but rather applies to what seems to me to be a fairly mainstream belief that agile means stopping work to make the software releasable.

Options Thinking lead us to invest time and money in delaying decisions to a time where we know the most about it; the extreme application of the Decide as late as possible principle is the concept of Last Responsible Moment, the optimal point of the trade-off between the available time for a decision and the need to complete a story or a task. The last responsible moment is the instant in which the cost of the delay of a decision surpasses the benefit of delay; or the moment when failing to take a decision eliminates an important alternative. For example, failing to provide a public HTTP API may make you lose an important customer, forcing you to publish an unfinished work. Tactics Mary Poppendiesk describes several tactics for delaying decisions until the last responsible moment: share partial design information, before it is freezed or released. The irreversible decisions, like freezing an api, are made later after feedback has been gathered; at the same time, the rest of the team can start to work with it. improve the response time for new stories. If you want to make a decision later, you still will have to respect the deadline. The faster you are, the later you can take important decisions. The adjectives lean and agile usually connotates lightweight approaches where decisions can be taken later for maximum flexibility. absorb changes by delaying the commitments to particular implementations, tools, and libraries. Modularization, interfaces, configuration parameters and any kind of abstraction are welcome investments in any case where there is the possibility of change in the future. By the way, the *no extra features* XP mantra recognizes that simple design, which minimizes duplication and moving parts, is the best response to the need for evolution. Real Options The Real Option (still the financial option metaphor) concept motivates Agile practices as for their ability to improve our options for deciding at the last responsible moment. For example, tests give us more options for a design by preserving its ability to change; and pairing give us more options for who should develop a feature, as knowledge of that particular part of the code base is spread across the team instead of being concentrated in a few people. It's all about risk management. Delaying decisions lets us able to make them in conditions of less uncertainty, when we can only know more about the domain and the project. Criticism Alistair Cockburn criticizes the concept of last responsible moment for several reasons. First, since the characterization as a single instant is not so close to reality. Cost and benefits of a decisions are soft functions that vary continuously, so it's difficult to think of a precise moment where a decision must be taken. In most cases, the *moment* spans for days. Second, this concept is not actionable, in the sense that you don't know the point in time where it will take place until after it has passed. Knowing that there is a deadline for a decision is different from knowing it with absolute precision. Finally, Cockburn views it as simple not good advice as trade-offs between cost and benefits should only apply to critical decisions, like a database with an high cost or lock-in, or the hardware architecture of the application. From the Extreme Programming point of view, it is correct to delay commitment to the last responsible moment, but not to overengineer a system to postpone every possible design decision. Choices like the programming language to write code in must be taken at the start of the project; the set of classes and methods should be kept minimal as long as duplication is eliminated. After all, this is a series on tools and it's up to us to pick up the right tool in the right context. The last responsible moment makes sense for decisions which are costly to change, but everything that can be rolledback thanks to encapsulation and information hiding is already abstracted away enough. In fact, iterative development is based on starting with a large set of assumptions and removing them one by one according to priority, evolving the code towards a more general picture. For example I have no problem hardcoding business rules, database drivers choices inside Repositories (but not credentials of course), and web application routes. As long as I can go back to the code in the future, they are not final decision; instead, I try to reserve the delaying of commitment to published interfaces and HTTP APIs... Conclusions We have learned to try to postpone decisions which are not immediately required, and even to invest in finding solutions for postponing some of them even when they should ordinarily be taken at the present time. The last responsible moment is a concept not to be taken literally, but when applied to difficult design and business decisions sets a goal for gathering all the needed information to take a choice when the time comes. Don't worry about what you can still change: worry about what will be carved in stone and delay the related decision as long as it does not damage you.

i recently answered this question in stackoverflow : what is an utility tree and what is it’s purpose in case of architecture tradeoff analysis method(atam)? i did answer the question there but here’s a better explanation with lots of examples based on the initial version for chapter 1 of soa patterns (which didn’t make it into the final version of the book). there are two types of requirements for software projects: functional and non-functional requirements. functional requirements are the requirements for what the solution must do (which are usually expressed as use cases or stories). the functional requirements are what the users (or systems) that interact with the system do with the system (fill in an order, update customer details, authorize a loan etc.). non-functional requirements are attributes the system is expected to have or manifest. these usually include requirements in areas such as performance, security, availability etc. a better name for non-functional requirements is “quality attributes” . below are some formal definitions from ieee standad 1061 “standard for a software quality metrics methodology” for quality attributes and related terms: quality attribute a characteristic of software, or a generic term applying to quality factors, quality subfactors, or metric values. quality factor a management-oriented attribute of software that contributes to its quality. quality subfactor a decomposition of a quality factor or quality subfactor to its technical components. metric value a metric output or an element that is from the range of a metric. software quality metric a function whose inputs are software data and whose output is a single numerical value that can beinterpreted as the degree to which software possesses a given attribute that affects its quality. most of the requirements that drive the design of a software architecture comes from system’s quality attributes. the reason for this is that that the effect of quality attributes is usually system-wide (e.g. you wouldn’t want your system to have good performance only in the ui – you want the system to perform well no matter what) – which is exactly what software architecture is concerned with. note however, that few requirements might still come from functional requirements) [1] . the question is how do we find out what those requirements are? the answer to that is also in the software architecture definition. the source for quality attributes are the stakeholders. so what or who are these “stakeholders”? well, a stakeholder is just about anyone who has a vested interest in the project. a typical system has a lot of stakeholders starting from the (obvious) customer, the end-users (those people in the customer organization/dept that will actually use the software) and going to the operations personnel (it – those who will have to keep the solution running), the development team, testers, maintainers, management. in some systems the stakeholders can even be the shareholders or even the general public (imagine for example, that you build a new dispatch system for a 911 center). one of the architect’s roles is to analyze the quality attributes and define an architecture that will enable delivering all the functional requirements while supporting the quality attributes. as can be expected ,sometimes quality attributes are in conflict with each other – the most obvious examples are performance vs. security or flexibility vs. simplicity and the architect’s role is to strike a balance between the different quality attributes (and the stakeholders) to make sure the overall quality of the system is maximized. contextual solutions (e.g. patterns) can be devised to solve specific quality attributes need. however saying that a system needs to have “good performance” or that it needs to be “testable” doesn’t really help us know what to do. in order for us to be able to discern which patterns apply to specific quality attribute , we need a better understanding of quality attributes besides the formal definition, something that is more concrete. the way to get that concrete understanding of the effect of quality attributes is to use scenarios. scenarios are short, “user story”-like proses that demonstrate how a quality attribute is manifested in the system using a functional situation quality attributes scenarios originated as a way to evaluate software architecture. the software engineering institute developed several evaluation methodologies, like architecture tradeoff analysis method (clements, kazman and klein, 2002) that heavily build on scenarios to contrast and compare how the different quality attributes are met by candidate architectures. atam (and similar evaluation methods like laaam which is part of msf 4.0) suggest building a “utility tree” which represent the overall usefulness of the system. the scenarios serve as the leafs of the utility tree and the architecture is evaluated by considering how the architecture makes the scenarios possible. i found that using scenarios and the utility tree approach early in the design of the architecture (see writings about saf ) can greatly enhance the quality of the architecture that is produced. when you examine the scenarios you can also prioritize them and better balance conflicting attributes. the scenarios can be used as an input to make sure the quality attributes are actually met. furthermore you can use the scenarios to help identify the strategies or patterns applicable to make the scenarios possible (and thus ensure the quality attributes are met) within the system. we usually group scenarios into a “utility tree” which is a representation of the total usefulness (“utility”) of a system . as you can see in the diagram below we have the key quality attributes (performance, security etc.). each of the quality attributes has sub categories (e.g. performance is broken into latency, data loss etc.). each sub category is demonstrated by a scenario that we expect the system to manifest. the tree representation helps get the whole picture but the important bits here are the scenarios so let’s explore them some more. scenarios are expressed as statements that have 3 parts: a stimulus , a context and a response . the stimulus is the action taken (by the system / user/ other system / any other person); response is how the system is expected to behave when the stimulus occur, and the context specifies the environment or conditions under which we expect the to get the response. for example in the following scenario: “when you perform a database operation , under normal condition, it should take less than 100 miliseconds.” “under normal condition” is the context “when you perform a database operation” is the stimulus “it should take less than 100 millisecond” is the response expected from the system. here are a couple of additional examples for quality attribute scenarios: performance –>latency -> under normal conditions a client consuming multiple services should have latency less than 5 seconds. security->authentications -> under all conditions, any call to a service should be authenticated using x.509 certificate you can also check out this document for a few more scenario examples from a system i worked on in the past [1] design has the ratios reversed i.e. most of the requirements for design come from functional requirements and a few requirements might come from the quality attributes. illustration by epsos.de

We now have finished exploring the Lean tools for amplifying learning like feedback, iterations and set-based development. We enter the real of the 3rd Lean principle, Decide as late as possible. This principle is oriented to postpone decisions as long as the delay does not impact the product, in order to gain more flexibility instead of becoming locked in with some initial design decisions. Software is easy to rebuild from source code, but its architecture is not always malleable by default as non-technical people would think. Moreover, there are some changes which will always happen, like upgrade of libraries and operating systems, which complements change in requirements or integration ports. The easiest decision to change is the one that has not been made yet. Options Thinking The first tool that helps in postponing decisions is Options Thinking: the introduction of mechanisms whose specific purpose is to enable delaying decisions. In the financial domain, an option is the right to buy a good at a certain price before a future date comes - effectively transferring the decision of buying shares or products some time in the future, as options can expire without being exercised. A simpler instance of Options Thinking cited by Mary Poppendieck is an hotel reservation: you invest a small sum of money (the reservation fee) to book a room; exercising the option means actually going to the hotel, a decision which is made only when the time comes. Trains and airlines often use the same pricing model for seats (even if we do not consider the rise of prices as a flight is being filled). There are multiple types of tickets for each combination of flight and date: some basic and not transferrable or refundable, some more costly that provide the option of changing the date or to get a partial or total refund. Agile Mary Poppendieck adds the insight that Agile software development is a process that creates many options by introducing a very flexible plan and only prescribing more detailed actions after several inspect and adapt loops. It's not bad to delay a commitment until you know more about a problem: forced early decisions are the mark of waterfall (actually of the mainstream version of waterfall). But options do not come for free: for example, in order to simplify a technical decision, XP suggests to create throwaway code. These spikes are the exploration of each potential solution, which in a certain sense are a waste of development time as their final result is of low quality and usually thrown away. However, spikes produces knowledge about the solution that results in a better estimate for its full development or in its abandonment. The decision to adopt a technology or of which solution to adopt is delayed until the end of a spike, but this option pay itself quickly as uncertainty is removed and decisions "get it right" with an higher probability. Real world examples Almost any application I have been involved with in the last two years has had the separation of a persistence layer as one of the goals: Active Record has been progressively abandoned in the PHP world to favor Data Mappers like the Doctrine ORM and ODMs. As for all options that can be bought, this separation does not come for free: development is a little slower when Repositories are objects that have to be designed instead of just a bunch of static calls to the Entity class like User::find() (although there are benefits of the Data Mapper approach that go beyond keeping options open.) An isolated persistence layer, however, allows us to postpone fundamental decisions about the database to use: it's a rough time for many of them as licenses change (MySQL) or new NoSQL solutions come out and evolve. Every month of development where you're not tied to a specific database is a month where the hype goes down and we move towards more mature solutions that we can choose with a greater knowledge of the requirements of our data. Do we need relational database consistency? Or a schema-less store? Moreover, the investment in persistence adapters separated from the core of the application let us able to choose different databases for different bounded contexts of an application; for example, storing views in a relational database and the primary database as a set of aggregates in Couch or Mongo. Conclusion I will never advocate to invest in an option just for the sake of the technical challenge, nor that they come for free; but once you recognize postponing a decision freezing is valuable for the project, there should be really no issue in go and buying it.

when working with agile teams the daily stand up meeting provides a heart beat to the day and an opportunity for team members to share information. stand up meetings work best when they are short and balance the inputs across all the people in the team. a common problem is stand ups that start running too long. when two hands are raised then it's a signal to move the stand up meeting on the two hands rule sometimes the conversations at stand up can get too detailed or go on too long. for these situations we’ve introduced the “two hands” rule; if anyone thinks the current conversation has gone off topic, or is no longer effective, then they raise a hand. once a second person raises a hand then that’s a sign to stop the conversation and continue with the rest of the stand up. those speaking can continue the conversation after the stand up has finished. this approach makes it easy for people to share their view on the effectiveness of the conversation in a way that reduces the risk of causing offence. it also provides a way for the team to detect and correct its own behaviour. i introduced this idea recently to a team who agreed to give it a try. in a stand up a few days later i was talking with a team member and didn’t realise that our conversation had got too detailed and gone on too long. i missed seeing that two other team members had put their hands up. it wasn’t until one of them spoke up that i noticed! this is one of the characteristics of difficult conversations; we often become blind to signs, easily spotted by others, that the conversation has become ineffective. by agreeing with the team to use the “two hands” rule they helped me detect when they thought i’d become ineffective. the technique can have some downsides though. it can feel direct or confrontational, especially when people first experience it. it’s important to discuss any issues after the stand up and consider reviewing the practice in a retrospective. i’d like to hear your thoughts. have you had stand up meetings that have taken too long? what approaches have you used? if you’ve tried something like the “two hands” rule, how did it go?

Everyone knows from reading The Mythical Man Month that as you add more people to a software development project you will see diminishing marginal returns. When you add a person to a team, there’s a short-term hit as the rest of the team slows down to bring the new team member up to speed and adjusts to working with another person, making sure that they fit in and can contribute. There’s also a long-term cost. More people means more people who need to talk to each other (n x n-1 / 2), which means more opportunities for misunderstandings and mistakes and misdirections and missed handoffs, more chances for disagreements and conflicts, more bottleneck points. As you continue to add people, the team needs to spend more time getting each new person up to speed and more time keeping everyone on the team in synch. Adding more people means that the team speeds up less and less, while people costs and communications costs and overhead costs keep going up. At some point negative returns set in – if you add more people, the team’s performance will decline and you will get less work done, not more. Diminishing Returns from any One Practice But adding too many people to a project isn’t the only case of diminishing returns in software development. If you work on a big enough project, or if you work in maintenance for long enough, you will run into problems of diminishing returns everywhere that you look. Pushing too hard in one direction, depending too much on any tool or practice, will eventually yield diminishing returns. This applies to: - Manual functional and acceptance testing - Test automation - Any single testing technique - Code reviews - Static analysis bug finding tools - Penetration tests and other security reviews Aiming for 100% code coverage on unit tests is a good example. Building a good automated regression safety net is important – as you wire in tests for key areas of the system, programmers get more confidence and can make more changes faster. How many tests are enough? In Continuous Delivery, Jez Humble and David Farley set 80% coverage as a target for each of automated unit testing, functional testing and acceptance testing. You could get by with lower coverage in many areas, higher coverage in core areas. You need enough tests to catch common and important mistakes. But beyond this point, more tests get more difficult to write, and find fewer problems. Unit testing can only find so many problems in the first place. In Code Complete, Steve McConnell explains that unit testing can only find between 15% and 50% (on average 30%) of the defects in your code. Rather than writing more unit tests, people’s time would be better spent on other approaches like exploratory system testing and code reviews or stress testing or fuzzing to find different kinds of errors. Too much of anything is bad, but too much whiskey is enough. Mark Twain, as quoted in Code Complete Refactoring is important for maintaining and improving the structure and readability of code over time. It is intended to be a supporting practice – to help make changes and fixes simpler and clearer and safer. When refactoring becomes an end in itself or turns into Obsessive Refactoring Disorder, it not only adds unnecessary costs as programmers waste time over trivial details and style issues, it can also add unnecessary risks and create conflict in a team. Make sure that refactoring is done in a disciplined way, and focus refactoring on those areas that need it the most: on code that is frequently changed, routines that are too big, too hard to read, too complex and error-prone. Putting most of your attention refactoring (or if necessary rewriting) this code will get you the highest returns. Less and Less over Time Diminishing returns also set in over time. The longer that you spend working the same way and with the same tools, the less benefits you will see. Even core practices that you’ve grown to depend on don’t pay back over time, and at some point may cost more than they are worth. It’s time again for New Year’s resolutions – time to sign up at a gym and start lifting weights. If you stick with the same routine for a couple of months, you will start to see good results. But after a while your body will get used to the work – if you keep doing the same things the same way your performance will plateau and you will stop seeing gains. You will get bored and stop going to the gym, which will leave more room for people like me. If you do keep going, trying to push harder for returns, you will overtrain and injure yourself. The same thing happens to software teams following the same practices, using the same tools. Some of this is due to inertia. Teams, organizations reach an equilibrium point and they want to stay there. Because it is comfortable, and it works – or at least they understand it. And because the better the team is working, the harder it is to get better – all the low-hanging fruit has been picked. People keep doing what worked for them in the past. They stop looking beyond their established routines, stop looking for new ideas. Competence and control lead to complacency and acceptance. Instead of trying to be as good as possible, they settle for being good enough. This is the point of inspect-and-adapt in Scrum and other time boxed methods – asking the team to regularly re-evaluate what they are doing and how they are doing it, what’s going well and what isn’t, what they should do more of or less of, challenging the status quo and finding new ways to move forward. But even the act of assessing and improving is subject to diminishing returns. If you are building software in 2-week time boxes, and you’ve been doing this for 3, 4 or 5 years, then how much meaningful feedback should you really expect from so many superficial reviews? After a while the team finds themselves going over the same issues and problems and coming up with the same results. Reviews become an unnecessary and empty ritual, another waste of time. The same thing happens with tools. When you first start using a static analysis bug checking tool for example, there’s a good chance that you will find some interesting problems that you didn’t know were in the code – maybe even more problems than you can deal with. But once you triage this and fix up the code and use the tool for a while, the tool will find fewer and fewer problems until it gets to the point where you are paying for insurance – it isn’t finding problems any more, but it might someday. In "Has secure software development reached its limits?” William Jackson argues that SDLCs – all of them – eventually reach a point of diminishing returns from a quality and security standpoint, and that Microsoft and Oracle and other big shops are already seeing diminishing returns from their SDLCs. Their software won’t get any better – all they can do is to keep spending time and money to stay where they are. The same thing happens with Agile methods like Scrum or XP – at some point you’ve squeezed everything that you can from this way or working, and the team’s performance will plateau. What can you do about diminishing returns? First, understand and expect returns to diminish over time. Watch for the signs, and factor this into your expectations – that even if you maintain discipline and keep spending on tools, you will get less and less return for your time and money. Watch for the team’s velocity to plateau or decline. Expect this to happen and be prepared to make changes, even force fundamental changes on the team. If the tools that you are using aren’t giving returns any more, then find new ones, or stop using them and see what happens. Keep reviewing how the team is working, but do these reviews differently: review less often, make the reviews more focused on specific problems, involve different people from inside and outside of the team. Use problems or mistakes as an opportunity to shake things up and challenge the status quo. Dig deep using Root Cause Analysis and challenge the team’s way of thinking and working, look for something better. Don’t settle for simple answers or incremental improvements. Remember the 80/20 rule. Most of your problems will happen in the same small number of areas, from a small number of common causes. And most of your gains will come from a few initiatives. Change the team’s driving focus and key metrics, set new bars. Use Lean methods and Lean Thinking to identify and eliminate bottlenecks, delays and inefficiencies. Look at the controls and tests and checks that you have added over time, question whether you still need them, or find steps and checks that can be combined or automated or simplified. Focus on reducing cycle time and eliminating waste until you have squeezed out what you can. Then change your focus to quality and eliminating bugs, or to simplifying the release and deployment pipeline, or some other new focus that will push the team to improve in a meaningful way. And keep doing this and pushing until you see the team slowing down and results declining. Then start again, and push the team to improve again along another dimension. Keep watching, keep changing, keep moving ahead. Source: http://swreflections.blogspot.com/2011/11/diminishing-returns-in-software.html