DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Culture and Methodologies Topics

article thumbnail
A Template for Formulating Great Sprint Goals
I find it helpful to consider three questions when choosing a sprint goal: Why do we carry out the sprint? How do we reach its goal? And how do we know that the goal has been met? My sprint goal template therefore consists of three main parts: the actual goal, the method employed to reach the goal, and the metrics to determine if the goal has been met. It additionally provides a header section that allows you to state to which product and sprint the goal belongs, as the picture below shows. You can download the template as a PDF from romanpichler.com/tools/sprint-goal-template/ or by clicking on the image below. The template above has grown out of my experience of working with Scrum for more than ten years, and it is inspired by the scientific method and Lean Startup. Let’s have a look at the template sections in more detail. The Goal Section The goal section states why it is worthwhile to undertake the sprint. Examples are: Test an assumption about the user interaction and learn what works best for the user, for instance: “Will users be willing to register before using the product features?” Address a technical risk such as: “Does the architecture enable the desired performance?” Release a feature, for instance: “Get the reporting feature for general release.” The sprint goal hence differs from listing the user stories that should be implemented. It communicates the reason for carrying out the work, and it provides a motivation for running the sprint. The sprint goal should be shared: The product owner and the development team should believe that working towards the goal is the right thing to do. To choose the right sprint goal I find it helpful to consider the amount of uncertainty present. In the early sprints, addressing risks and testing assumptions allows me to learn about what the product should look like and do and how it is built. Once the key risks and critical assumptions have been dealt with, I like to focus on completing and optimising features, as the following picture shows: The Method Section This section addresses the question of how the goal is met. The default Scrum answer is simple: Create a (potentially shippable) product increment using the high-priority product backlog items, and demo it to the stakeholders in the sprint review meeting. But writing software and employing a product demo are not always the best methods to achieve the goal! A paper prototype can be good enough to test a visual design idea or an assumption about the user interaction, for instance. What’s more, other methods such as carrying out a usability test or releasing software to run an A/B test may well be more effective than a product demo. You should therefore carefully choose the right method and state it in this section. But don’t stop there. Determine the test group, the people who should provide feedback and data. Who these individuals are depends on the sprint goal: If you are validating an assumption about the visual design, the user interaction or the product functionality, then you probably want to collect feedback and data from the users. But if you are addressing a technical risk, then users may not be able to help you. Consider inviting a senior developer or architect from another team instead. Stating the test group clarifies who “the stakeholders” are, who is required to provide feedback so that the right product is developed. The Metrics Section The metrics section communicates how you determine if the goal has been met. Which metrics you use depends on the method chosen. For a product demo, you may state that at least two thirds of the stakeholders present should respond positively to the new feature, for instance; for a usability test, at least three of the five testers are complete the task successfully in less than a minute; and for the release of a new feature, you might say that at least 80% of the users use the new functionality at least once within five days after launching the feature. Whichever metrics you choose, make sure that they allow you to understand if and to which extent you have met the goal. The Header Section The header section consists of the two subsections “Product” and “Sprint”. They simply allow you to state which product and which sprint the goal belongs to. Customise this section according to your needs. If you work for an agencies or an IT solution provider, you could replace “Product” with “Project”, for instance. User Stories and the Sprint Goal You may be wondering how the template relates to the user stories. Let me first reiterate that your sprint goal should differ from your user stories. The goal explains the why it is a good idea to carry out the sprint an implement the stories. The user stories enable you to reach the goal. It’s a common mistake to confuse the two. To connect the template and the stories you have two options: You can state the relevant user stories in the template’s method section, or you can list them separately on the sprint backlog, as the following picture illustrates. In the picture above, the sprint goal is stated on the left to the sprint backlog, which lists the user stories and the tasks required to meet the goal in form of a task board. Learn more You can learn more about choosing effective sprint gaols and applying the sprint goal template by attending my Certified Scrum Product Owner training course. I have written in more detail about sprint planning in my book “Agile Product Management with Scrum”. Please contact me for onsite and virtual product owner training.
March 12, 2014
by Roman Pichler
· 14,223 Views · 1 Like
article thumbnail
Choosing Columns for Agile Team Boards
"And let Reform her columns roll. With thunder peal, and lightning flash..." - Ignis, "The Genius of Liberty" Vol III No. 2 Introduction In the past couple of articles we've seen how a Kanban board is able to help in the attainment of transparency and the stabilization of an agile team. Today we'll see if we can resolve one of the most common queries that result from this usage: how does a team decide which columns should appear on the board for tracking the progress of work items? The simplest case...and why it may not be enough When we set up a Kanban board in the last article, there were only three columns - or to use the correct term, "stations". These were a "Backlog" station (essentially a "To Do" list of work that has not yet been started), a station for showing which work is "In Progress", and a finally a station for representing the work that has been completed. You can't get much simpler than that, and it begs the question as to why you would want to make it more complicated. In practice however, there are at least two situations in which this minimalist approach will be found wanting: A team isn't cross-trained, and its members effectively work in skill silos. Consequently we can expect dependencies between team members, some of whom may become blocked while waiting for others to complete their part of the work. The incurral of this wasted time will not be apparent if it is all considered to be "work in progress". For example, the team may be split into developers and testers, and bottlenecks may arise as work passes between them. We may need to break Work In Progress down into further stations in order to expose this waste more fully. Bottlenecks arise due to constraints in the workflow, which is a different problem. In this situation a team might be fully cross-trained, and none of its members become blocked waiting for another. Rather, waste arises because the work itself is inefficiently staged. This often happens with activities like development, review, and test. For example if two people are required for a review, but only one is needed for development and test, then a bottleneck may well occur. Work will build-up awaiting review due to contention for these resources and the value of the investment in effort will start to depreciate. Again the incurral of this waste will not be apparent if it is all considered to be "work in progress". More stations are needed to expose it. Adding further stations These then are the two key things to consider when choosing additional stations. We're out to expose waste caused by work silos, or by the inappropriate staging of activities. Either of these can introduce constraints and become the source of bottlenecks in a value stream. Sometimes blockages can occur due to a dependency on something that must be done outside of the team. When this happens, it implies that the team are not fully in control of their own process, and consequently are unable to meet their own Definition of Done. They don't have all of the skills or resources needed. This is a problem and a contra-indication to agile practice. If it happens it's essential to make the dependency clear so that it can be challenged and removed. We may therefore choose to have an "externally blocked" column on the board to expose problems of this nature. It isn't really a station, because it doesn't represent a state in which value is added. Rather, it shows that an item has stalled within the value stream and that the team are not in a position to provide remedy. Another option is to place a red, day-glo sticky note on the ticket highlighting the seriousness of the problem. This is a clear signal that an impediment has occurred...that is to say, in Lean-Kanban terminology, it is an andon flag. In this case the flag shows that a major blockage has arisen and needs resolving. Challenging the boundaries Now we need to turn our attention to the boundaries of the board. There are two principal areas we should look at. Firstly, on the leftmost side of the board, we can see the work that a team inducts into its "Backlog" prior to actioning it. Secondly, on the rightmost side, we can see the work that the team considers to be "Done". These two boundaries are very often a source of waste. To understand why, just consider how backlogs are often allowed to grow without effective limit, and at how completed work may be permitted to accumulate in a "Done" column. These stations may not represent work in progress as far as the team is concerned, but it would be foolish to deny that they are batches too. After all, they are still part of the value stream. They represent inventory that is depreciating in value, or relevance, until something useful is done with it all. It behooves us to query the waste that is incurred, and to ask how the size of these batches may be constrained. Specifically: How can work be inducted into a backlog with minimal accumulation and delay? How can value be delivered to consumers as soon as work is completed? In short, what can be done to "lean" these process boundaries, so that inventory in the team's part of the value stream enters and exits in a "just-in-time" fashion? We can answer these questions by improving transparency still further. This can mean the refinement of the "Backlog" and "Done" columns into other, more finely-grained stations. For example, work might be building up in a Product Backlog because it is not being triaged appropriately, or perhaps because acceptance criteria are insufficiently well defined. We might be able to expose these problems by replacing a backlog with "Triaged", "Accepted", and "Ready" stations. At the other side of the board, completed work may be building up in the "Done" column because a release cannot yet be made. Additional stations such as "System Integrated", "In User Acceptance", and "Awaiting Release" could add clarity here. Removing stations The simple, 3 column board we started has now exploded into a behemoth of perhaps ten columns or more. This may seem like an excessively complex structure for a workflow and a casual observer may criticize it for being fundamentally unagile. After all, inventory should either be work in progress by an agile team, or it will be awaiting their attention or have already been completed. The criticism is a valid one but we need to bear one thing in mind: these stations are there to expose problems. Only once transparency has been attained can we hope to provide remedy. The bottlenecks, along with the diagnostic stations we added to reveal them, can then be removed. Conclusion Knowing how many "columns" to include on an agile board, and what they should be, is something of a black art to many agile teams. In this article we've looked at the issues involved in making this decision. The board of a fully cross-trained team should be elegant in its simplicity, but when problems arise we must be prepared to do some digging in order to root out their causes.
February 25, 2014
by $$anonymous$$
· 12,566 Views · 2 Likes
article thumbnail
SPNego Authentication with JBoss
Background SPNego is RFC 4178 used for negotiation either NTLM or Kerberos based SSO. A typical use case is for web applications to reuse the authentication used by Desktops such as Windows or Linux. In this article, we will explore approaches for SPNego authentication with JBoss Enterprise Application Platform. JBoss Negotiation is the library that provides the SPNego authentication support in JBoss. This library has been integrated in JBoss EAP and WildFly Application Server. Checklist Obtain JBoss EAP from jboss.org. Enable your JavaEE Web Application for SPNego Authentication. Configure JBoss EAP for SPNego. Configure your Browsers for SPNego. Start JBoss EAP. Test your web application. Obtain JBoss EAP from jboss.org Download JBoss EAP 6.2 or newer from http://www.jboss.org/products/eap You can also use WildFly Application Server from http://www.wildfly.org. Your configuration may vary slightly. Enable your JavaEE Web Application for SPNego Authentication It is easier to use a demo web application as a starting point. You can then modify your web application for SPNego authentication. The demo web application we use for this article is called spnego-demo, by my colleague, Josef Cazek. The demo web application is available at https://github.com/kwart/spnego-demo . You can also download the spnego-demo.war from here . Fully configured web application spnego-demo.war can be obtained from this location . Copy the spnego-demo.war in your jboss-eap-6.2/standalone/deployments directory. Configure JBoss EAP for SPNego Authentication You will need to configure a couple of security domains and system properties in JBoss EAP6. There are two ways by which you can configure: either manual editing or using CLI tool. Manual Editing of configuration file standalone.xml in jboss-eap-6.2/standalone/configuration Add system properties to this file. Remember to put this block right after the extensions block (around line 25 of the configuration file). Add security domains to this file. Remember to put these blocks in the block. Using Command Line Interface to update JBoss EAP Go to the bin directory of JBoss EAP 6.2 and run the following. $ cat << EOT > $SPNEGO_TEST_DIR/cli-commands.txt /subsystem=security/security-domain=host:add(cache-type=default) /subsystem=security/security-domain=host/authentication=classic:add(login-modules=[{"code"=>"Kerberos", "flag"=>"required", "module-options"=>[ ("debug"=>"true"),("storeKey"=>"true"),("refreshKrb5Config"=>"true"),("useKeyTab"=>"true"),("doNotPrompt"=>"true"),("keyTab"=>"$SPNEGO_TEST_DIR/http.keytab"),("principal"=>"HTTP/[email protected]")]}]) {allow-resource-service-restart=true} /subsystem=security/security-domain=SPNEGO:add(cache-type=default) /subsystem=security/security-domain=SPNEGO/authentication=classic:add(login-modules=[{"code"=>"SPNEGO", "flag"=>"required", "module-options"=>[("serverSecurityDomain"=>"host")]}]) {allow-resource-service-restart=true} /subsystem=security/security-domain=SPNEGO/mapping=classic:add(mapping-modules=[{"code"=>"SimpleRoles", "type"=>"role", "module-options"=>[("[email protected]"=>"Admin"),("[email protected]"=>"User")]}]) {allow-resource-service-restart=true} /system-property=java.security.krb5.conf:add(value="$SPNEGO_TEST_DIR/krb5.conf") /system-property=java.security.krb5.debug:add(value=true) /system-property=jboss.security.disable.secdomain.option:add(value=true) :reload() EOT $ ./jboss-cli.sh -c --file=$SPNEGO_TEST_DIR/cli-commands.txt This is explained in https://github.com/kwart/spnego-demo/blob/master/README.md We will need a keytab file. In this example, we will use the Kerberos Server using ApacheDS (as explained in Appendix A). $ java -classpath kerberos-using-apacheds.jar org.jboss.test.kerberos.CreateKeytab HTTP/[email protected] httppwd http.keytab Note that the http.keytab has been configured in the security domain called "host" in standalone.conf. So place the keytab file appropriately while correcting the path defined in security domain. More information is available at https://github.com/kwart/kerberos-using-apacheds/blob/master/README.md JBoss EAP will need a keytab file. In this example we use a keytab called as http.keytab Different tools such as ktutil exist to create keytab files. Keytab files contain Kerberos Principals and encrypted keys. It is important to safeguard keytab files. It is very important that JBoss EAP configuration for keytab in the security domain "host" refers to the actual path of the keytab file. Configure your Browsers for SPNego The browsers such as Microsoft IE, Mozilla Firefox, Google Chrome, Apple Safari have different settings for enabling SPNego or Integrated Authentication. Start JBoss EAP Go to the bin directory of JBoss EAP 6.2 and either use standalone.sh (Unix/Linux) or standalone.bat to start your instance. Test your Web Application Assuming you have followed Appendix A steps to start the kerberos server and done kinit, you are ready to test the web application. In this article we have used spnego-demo, we can test that by going to http://localhost:8080/spnego-demo/ You can click on the "User Page" link and you should be able to see the principal name as "[email protected]" Appendix A Local Kerberos Server Download the zip file https://github.com/kwart/kerberos-using-apacheds/archive/master.zip Unzip the zip file into a directory. Build the package using maven. $ mvn clean package Start the Kerberos Server as $ java -jar target/kerberos-using-apacheds.jar test.ldif A krb5.conf file has been created. Login now using [email protected] $ kinit [email protected] Password for [email protected]: secret Launch Firefox via command line from where the kinit was run On MacOSX $open -a firefox http://localhost:8080/spnego-demo/ Appendix B Kerberos Command Line Utilities klist can be used to see the current kerberos tickets. $ klist Credentials cache: API:501:10 Principal: [email protected] Issued Expires Principal Feb 9 21:19:30 2014 Feb 10 07:19:27 2014 krbtgt/[email protected] kdestroy can be used to clear the current kerberos tickets. References SPNego Demo Web Application: https://github.com/kwart/spnego-demo Kerberos Server using ApacheDS: https://github.com/kwart/kerberos-using-apacheds JBoss EAP 6 http://www.jboss.org/products/eap PicketLink Open Source Project: http://www.picketlink.org Troubleshooting https://docs.jboss.org/author/display/PLINK/SPNego+Support+Questions Remember krb5.conf is important for client side kerberos interactions. You can use a environment variable on Unix/Linux/Mac systems called KRB5_CONFIG to point to your krb5.conf Acknowledgement Darran Lofthouse for the wonderful JBoss Negotiation Project and Josef Czacek for the SPNego-demo and Kerberos_using_Apache DS projects.
February 12, 2014
by Anil Saldanha
· 19,688 Views
article thumbnail
Hunting for an SWT Test Framework? Say Hello to Red Deer
This is the first in a series of posts on the new “Red Deer” (https://github.com/jboss-reddeer/reddeer) open source testing framework for Eclipse. In this post, we’ll introduce Red Deer, and take a look at the some of the advantages that it offers by building a sample test program from scratch. Some of the features that Red Deer automated offers are: An easy to use, high-level API for testing standard Eclipse components Support for creating custom extensions for your own applications A requirements validation mechanism to assist you in configuring complex tests Eclipse Tooling to Assist in Creating new Projects A record and playback tool to enable you to quickly create automated tests An integration with Selenium for testing web based applications Support for running tests in a Jenkins CI environment Note that as of this writing, Red Deer is in an incubation stage. The current release is at level 0.5. The target date for the 1.0 release of Red Deer is late 2014. But, as a community-based, open source project, now is a great time to try Red Deer and make suggestions or even contribute code! A Look at Red Deer’s Architecture The Red Deer project itself is comprised of utilities and the API that supports the development and execution of automated tests. The API (the parts of the above diagram that are enclosed in dashed line boxes) can be thought of as having three layers: The top layer consists of extensions to Red Deer’s abstract classes or implementations for Eclipse components such as Views, Editors, Wizards, or Shells. For example, if you are writing tests for a feature that uses a custom Eclipse View, you can extend Red Deer’s View class by adding support for the specific functions of the feature. The advantage that this API layer gives you is that your test programs do not have to focus on manipulating the individual UI elements directly to perform operations. Your programs can instead instantiate an instance of an Eclipse component such as a View, and then use that instance’s methods to perform operations on the View. This layer of abstraction makes your test programs easier to write, understand, and maintain. The middle layer consists of the Red Deer implementations for SWT UI elements such as: Button, Combo, Label, Menu, Shell, TabItem, Table, ToolBar, Tree. This API layer supports the API’s higher level by providing the building blocks for the API’s Views, Editors, Shells, and WIzards. This middle layer of the API also provides Red Deer packages that enable your tests to enforce requirements, so that necessary setup tasks are performed before a test is run. The bottom layer consists of Red Deer packages that support the execution of tests such as: Conditions, Matchers, Widgets, Workbench, and Red Deer extensions to JUnit. What Makes Red Deer different from other Tools? A Layer of Abstraction The top-most layer of the API enables you to instantiate Eclipse UI elements as objects, and then manipulate them through their methods. The resulting code is easier to read and maintain, instead of being brittle and subject to failures when the UI changes. For example, for a test that has to open a view and press a button, without Red Deer, the test would have to navigate the top level menu, find the view menu, then the view type in that menu, then find the view open dialog, then locate the “OK” button, etc. Your test would have to spend a lot of time navigating through the UI elements before it could even begin to perform the test’s steps. With Red Deer, the code to open a view (in this case, the servers view) is simply: ServersView view = new ServersView(); view.open(); Furthermore, within that ServersView, your test program can perform operations on the View through methods which are defined in the view (and are incidentally also well debugged by the Red Deer team), instead of having to explicitly locate and manipulate the UI elements directly. For example, to obtain a list of all the servers, instead of locating the UI tree that contains the server list, and extracting that list of servers into an array, your Red Deer program can simply call the “getServers()” method. Likewise, the code to open a PackageExplorer, and then select a project within that PackageExplorer is as follows: PackageExplorer packageExplorer = new PackageExplorer(); packageExplorer.open(); packageExplorer.getProject("myTestProject").select(); And, the code to retrieve all the projects within that PackageExplorer is simply: packageExplorer.getProjects(); The result are that your tests are easier to write and maintain and you can focus on testing your application’s logic instead of writing brittle code to navigate through the application. Installing Red Deer The only prerequisites to using Red Deer are Eclipse and Java. In this post, we’ll use Eclipse Kepler and OpenJDK 1.7, running on Red Hat Enterprise Linux (RHEL) 6. To install Red Deer 0.4 (this is the latest stable milestone version as of this writing) follow these steps: Open up Eclipse Navigate to: Help->Install New Software Define a new download site using the Red Deer update site URL: http://download.jboss.org/jbosstools/updates/stable/kepler/core/reddeer/0.4.0/ Select Red Deer, click on the Finish button and Red Deer will install Now that you have Red Deer installed, let’s move onto building a new Red Deer test. Building your First Red Deer Test To create a new Red Deer test project, you make use of the Red Deer UI tooling and select New->Project->Other->Red Deer Test: Before we move on, let’s take a look at the WEB-INF/MANIFEST.MF file that is created in the project: Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: com.example.reddeer.sample Bundle-SymbolicName: com.example.reddeer.sample;singleton:=true Bundle-Version: 1.0.0.qualifier Bundle-ActivationPolicy: lazy Bundle-Vendor: Sample Co Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Require-Bundle: org.junit, org.jboss.reddeer.junit, org.jboss.reddeer.swt, org.jboss.reddeer.eclipse The line we’re interested in is the final line in the file. These are the bundles that are required by Red Deer. After the empty project is created by the wizard, you can define a package and create a test class. Here's the code for a minimal functional test. The test will verify that the eclipse configuration is not empty. package com.example.reddeer.sample; import static org.junit.Assert.assertFalse; import java.util.List; import org.jboss.reddeer.swt.api.TreeItem; import org.jboss.reddeer.swt.impl.button.PushButton; import org.jboss.reddeer.swt.impl.menu.ShellMenu; import org.jboss.reddeer.swt.impl.tree.DefaultTree; import org.junit.Test; import org.junit.runner.RunWith; import org.jboss.reddeer.junit.runner.RedDeerSuite; @RunWith(RedDeerSuite.class) public class SimpleTest { @Test public void TestIt() { new ShellMenu("Help", "About Eclipse Platform").select(); new PushButton("Installation Details").click(); DefaultTree ConfigTree = new DefaultTree(); List ConfigItems = ConfigTree.getAllItems(); assertFalse ("The list is empty!", ConfigItems.isEmpty()); for (TreeItem item : ConfigItems) { System.out.println ("Found: " + item.getText()); } } } After you save the test's source file, you can run the test. To run the test, select the Run As->Red Deer Test option: And - there's the green bar! Simplifying Tests with Requirements Red Deer requirements enable you to define actions that you want happen before a test is executed. The advantage to using requirements is that you define the actions with annotations instead of using a @BeforeClass method. The result is that your test code is easier to read and maintain. The biggest difference between a Red Deer requirement and the the @BeforeClass annotation from the JUnit framework is that if a requirement cannot be fulfilled the test is not executed. Like everything else in Red Deer, you can make use of predefined requirements, or you can extend the feature by adding your own custom requirements. These custom requirements can be made complex and for convenience can be stored in external properties files. (We’ll take a look at defining custom requirements in a later post in this series when we examine how to create and contribute extensions to Red Deer.) The current milestone release of Red Deer provides predefined requirements that enable you to clean out your current workspace and open a perspective. Let’s add these to our example. To do this, we need to add these import statements: import org.jboss.reddeer.eclipse.ui.perspectives.JavaBrowsingPerspective; import org.jboss.reddeer.requirements.cleanworkspace.CleanWorkspaceRequirement.CleanWorkspace; import org.jboss.reddeer.requirements.openperspective.OpenPerspectiveRequirement.OpenPerspective; And these annotations: @CleanWorkspace @OpenPerspective(JavaBrowsingPerspective.class) And, we also have to a reference to org.jboss.reddeer.requirements to the required bundle list in our example’s MANIFEST.MF file: Require-Bundle: org.junit, org.jboss.reddeer.junit, org.jboss.reddeer.swt, org.jboss.reddeer.eclipse, org.jboss.reddeer.requirements When we’re done, our example looks like this: package com.example.reddeer.sample; import static org.junit.Assert.assertFalse; import java.util.List; import org.jboss.reddeer.swt.api.TreeItem; import org.jboss.reddeer.swt.impl.button.PushButton; import org.jboss.reddeer.swt.impl.menu.ShellMenu; import org.jboss.reddeer.swt.impl.tree.DefaultTree; import org.junit.Test; import org.junit.runner.RunWith; import org.jboss.reddeer.junit.runner.RedDeerSuite; import org.jboss.reddeer.eclipse.ui.perspectives.JavaBrowsingPerspective; import org.jboss.reddeer.requirements.cleanworkspace.CleanWorkspaceRequirement.CleanWorkspace; import org.jboss.reddeer.requirements.openperspective.OpenPerspectiveRequirement.OpenPerspective; @RunWith(RedDeerSuite.class) @CleanWorkspace @OpenPerspective(JavaBrowsingPerspective.class) public class SimpleTest { @Test public void TestIt() { new ShellMenu("Help", "About Eclipse Platform").select(); new PushButton("Installation Details").click(); DefaultTree ConfigTree = new DefaultTree(); List ConfigItems = ConfigTree.getAllItems(); assertFalse ("The list is empty!", ConfigItems.isEmpty()); for (TreeItem item : ConfigItems) { System.out.println ("Found: " + item.getText()); } } } Notice how we were able to add those functions to the test code, while only adding a very small amount of actual new code? Yes, it can pay to be a lazy programmer. ;-) What’s Next? What’s next for Red Deer is its continued development as it progresses through its incubation stage until its 1.0 release. What’s next for this series of posts will be discussions about: The Red Deer Recorder - To enable you to capture manual actions and convert them into test programs How you can Extend Red Deer - To provide test coverage for your plugins’ specific functions. And How you can Contribute these extensions to the Red Deer project. How you can Define Complex Requirements - To enable you to perform setup tasks for your tests. Red Deer’s Integration with Selenium - To enable you to test web interfaces provided by your plugins. Running Red Deer tests with Jenkins - To enable you to take advantage of Jenkins’ Continuous Integration (CI) test framework. Author’s Acknowledgements I’d like to thank all the contributors to Red Deer for their vision and contributions. It’s a new project, but it is growing fast! The contributors (in alphabetic order) are: Stefan Bunciak, Radim Hopp, Jaroslav Jankovic, Lucia Jelinkova, Marian Labuda, Martin Malina, Jan Niederman, Vlado Pakan, Jiri Peterka, Andrej Podhradsky, Milos Prchlik, Radoslav Rabara, Petr Suchy, and Rastislav Wagner.
January 7, 2014
by Len DiMaggio
· 7,783 Views
article thumbnail
MongoDB and its locks
Sometimes, you need your jobs to be persisted to a database. Existing solutions such as Gearman only used relational or file-based persistence, so they were a no-go for us and we went with MongoDB. Fast-forward a few months, and we have some problems with the database load. However, it's not that workers are pestering it too much: the problem was related to locks. MongoDB locking model As of 2.4, MongoDB holds write locks on an entire database for each write operation. Since atomicity is guaranteed only on a single document, this isn't usually a problem because even if you are inserting thousands of documents you are doing so in thousands of different operations that can be interleaved with queries and other inserts with a fair policy. This sometimes results in count() queries being inconsistent as documents are moved and indexes are asynchronously updated. However, write corruption is inexistent as documents are a very cohesive entity. However, atomic operations over a single document still lock the whole database, as in the case of findAndModify(), which looks for a document matching a certain query and updates it with a $set operation before returning it; all in a single shot and with the guarantee no other process will be able to perform the same operation of reading and writing at the same time. You can see this operation is ideal for implementing workers based on a pull model, each asking the database for a new job to do and locking it with '$set: {locked: true}'. However, after the number of workers increases a little bit, locks become a problem. Lock duration We cleaned up the working space collection of our MongoDB database by keeping in it only the unfinished jobs, and moving all the rest (completed or failed) to a different collection for archival. As the load increases due to new contracts, we saw the locking time increase as well: the application and the workers were insisting on the same database. The first of the problems was that after reducing the specs of our primary server, we started seeing timeouts of unrelated code even if the CPU and IO usage were low. The locks taken by workers to pick jobs were starting to take seconds or tens of seconds. Moreover, the MongoDB server started filling the logs with: Fri Dec 6 00:01:07 [conn280998] warning: ClientCursor::yield can't unlock b/c of recursive lock... I'm a user, not MongoDB guru but that seems not very good, especially given hundreds of these messages were written every day (although the queues continued to work correctly.) We did not find any explanation for these messages in the documentation, but I suppose they mean some operations are taking so long that they have to yield to make room for others, but in the case of atomic operations they can't to preserve consistency. An easy solution Since MongoDB does not have collection-wide locks yet, we decided to move the job pool and the completed job collections to a different database. In this way, we had a main database with the usual collections and one containing just these two, named with a '_queue' suffix. Note that we're still writing to the same database server: there is still the same number of connections being created by each process. This solution preallocates more space given two databases are involved, but as you know space is cheap nowadays. Both insertion of jobs and worker reads must take place on the same database. Here is where we discovered cohesion pays: if you have this information in a single place it is very easy to change configuration. If you have a singleton database, because "we should only have one database in this application, it will never change" this feature would cost you a lot. Fortunately, in our case it was about 10 lines of code, including the refactoring on the Factory Methods that created MongoDB database objects. Long term This solution is not for the long term, as we know the numbers of machines and their workers pool will increase in the future; a sufficiently high number of workers will saturate the connections available on the MongoDB server and lock the common collection until a pick of a job takes dozens of seconds. The design towards which we are moving includes one "foreman" to each machine, and many workers under his control; only the foreman polls the database and may lock the common collection. Distributing the job pool is not what we want for ease of retrieval of a job in case something goes bad (ever done a query on multiple databases?). Also, we don't want a push solution as it will involve the registration of workers or foremen to a central point of failure that assignes them their jobs. Since most of our servers are shutdown and rebooted according to the user load, we prefer a dynamic solution where a server can start picking jobs whenever it wants and stop without notifying remote machines.
December 6, 2013
by Giorgio Sironi
· 27,648 Views
article thumbnail
The GO Product Roadmap – a New Agile Product Management Tool
A product roadmap is a high-level, strategic plan, which provides a longer-term outlook on the product. This creates a continuity of purpose, and it helps product managers and owners acquire funding for their product; it sets expectations, aligns stakeholders, and facilitates prioritization; it makes it easier to coordinate the development and launch of different products, and it provides reassurance to the customers (if the product roadmap is made public). Unfortunately, I find that many product managers and product owners struggle with their roadmaps, as they are dominated by features: There are too many features, and the features are often too detailed. This turns a roadmap into a tactical planning tool that competes with the Product Canvas or product backlog. What’s more, the features are sometimes regarded as a commitment by senior management than part of a high-level plan that is likely to change. The GO Product Roadmap Explained Faced with this situation, I have developed a new goal-oriented agile roadmap — the GO product roadmap, or “GO” for short. GO is based on my experience of teaching and coaching product managers and product owners, as well as using product roadmaps in my own business. The following pictures shows what the GO product roadmap looks like. You can download a PDF and Excel template by simply clicking on the picture. The first row of the GO roadmap depicted above contains the date or timeframe for the upcoming releases. You can work with a specific date such as 1st of March, or a period such as the first or second quarter. The second row states the name or version of the releases, for instance, iOS 7 or Windows 8.1. The third row provides the goal of each release, the reason why it is worthwhile to develop and launch it. Sample goals are to acquire or to activate users, to retain users by enhancing the user experience, or to accelerate development by removing technical debt. Working with goals shifts the conversation from debating individual features to agreeing on desired benefits making strategic product decisions. The development team, the stakeholders, and the management sponsor should all buy into the goals. The fourth row provides the features necessary to reach the goal. The features are means to an end, but not an end in themselves: They serve to create value and to reach the goal. Try to limit the number of features for each release to three, but do not state more than five. Refrain from detailing the features, and focus on the product capabilities that are necessary to meet the goal. Your product roadmap should be a high-level plan. The details should be covered in the Product Canvas or product backlog, and commitments should be limited to individual sprints. The last row states the metrics, the measurements or key performance indicators (KPIs) that help determine if the goal has been met, and if the release was successful. Make sure that the metrics you select allow you to measure if and to which extent you have met the goal. A Sample GO Product Roadmap To illustrate how the GO template can be applied, imagine we are about to develop a new dance game for girls aged eight to 12 years. The app should be fun and educational allowing the players to modify the characters, change the music, dance with remote players, and choreograph new dances. Here is what the corresponding GO roadmap could look like: While the roadmap above will have to be updated and refined at a later stage (particularly the metrics), I find it good enough to show how the product may evolve and make an investment decision. When creating your GO roadmap make sure you determine the goal of each release before you identify the features. This ensures that the features do serve the goal. Filling in the roadmap template from top to bottom and from left to right works well for me. Wrap-up The GO product roadmap provides a new, powerful way to do product roadmapping. Rather than focussing on features, GO emphasizes the importance of shared goals. This makes it easier to communicate the roadmap, create alignment, and use it as a strategic planning tool that provides an umbrella for the Product Canvas and the product backlog. The metrics provided by the tool ensure that the goals are measurable rather than lofty and fuzzy ideas. Download the template now, and try it out! You can learn more about creating effective product roadmap and working with the GO product roadmap by attending my Agile Product Planning training course. I would love to hear your questions about the roadmap and your experiences of creating product roadmaps. Please leave a comment below, or contact me.
December 3, 2013
by Roman Pichler
· 15,341 Views
article thumbnail
Wet agile or agile waterfall?
The second blog post in the Exploring the landscape of large scale agile frameworks - series “Wet agile”, “the agile waterfall” and “the Agile-Waterfall Hybrid” … this controversial, mixed-method baby has as many names as formats. Some have received a lot of dedicated thought, are fit-for-purpose and manage to preserve the main benefits of the more pure methods. Other hybrid models have resulted by accident; either as a consequence of… … a half-way-stopped agile transitioning program that dropped dead somewhere between waterfall and scrum in an unsuccessful change initiative … a compromise between method-promoters of different ideologies (often management being on the more waterfall friendly side vs. developers generally promoting more agility) The result of the latter two cases can be horrible to witness and worse to experience for everyone involved regardless if they sit within the development organization or are on the receiving end waiting for the product. However, I do not think the frequent occurrence of poor agile-waterfall hybrids we can witness in the industry is a reason to consistently argue against mixing the two arts as many agilists do. Rather, these cases highlight the need to bring the successful and proven versions of agile-waterfall hybrids into the light and reserve a place for them among their purer cousins in the landscape for large scale agile frameworks. Founders of more successful hybrid versions often come from product development companies that deal with both software and hardware. The driver behind the method mixing in these cases is, if we simplify to the extreme, the realization that many aspects of hardware development benefit from waterfall processes, whereas software development has much to gain from an agile approach. The Agile-Waterfall Hybrid described by Erick Bergmann and Andy Hamilton from Schneider Electric is a great example of a model that merges Agile and Project Management Process/waterfall in a good way without compromising the methods’ core principles to much. Erick Bergmann and Andy Hamilton presented the Agile-Waterfall Hybrid at Agile 2013 and the key concepts of the model can be well understood from the presenting material; https://submissions.agilealliance.org/system/sessions/attachments/000/000/760/original/Agile-Waterfall_Hybrid_01AUG2013.pdf The model allows software teams to adapt agile practices while hardware development and overall product management is handled through a traditional PMP/waterfall approach The overall PMP process has a well-defined interface to the agile software development which is continuous right from the start of the concept/feasibility study up until validation and production. The interface has the format of close collaboration for all activities ranging from definition of requirements and scoping to continuous software deliveries and feedback between the agile side and the waterfallish PMP side The model accommodates development where the same/similar software is used in several products with separate POs. Eg., the software teams need to deliver features to multiple stakeholders continuously in a similar fashion as component teams typically need to act towards feature teams. The very challenging backlog management situation resulting for teams in environments like this is addressed in a good way by the model. In short, it describes how you can achieve swift product releases by putting effort into the feature releases planning on the software side There is no perfect alternative. Just like any other hybrid model, this Agile-Waterfall Hybrid compromises with some of the principles of the non-hybrid methods. The waterfall side of development is forced to live with the flexibility surrounding the software requirements and the agile teams must commit to time-fixed delivery dates, cost forecasts and risk assessment. Hybrid development models face many challenges. It is not easy to combine the dependency tracking and clarity of waterfall with the flexibility and openness to change of Agile development without diluting the benefits and create complex work processes. However, examples like the one referred to above prove that it is possible and as the industry need for these types of models is evident I hope to see more of these being published in the future.
November 4, 2013
by Ebba Kraemer
· 25,922 Views
article thumbnail
Securing Docker’s Remote API
One piece to Docker that is interesting AMAZING is the Remote API that can be used to programatically interact with docker. I recently had a situation where I wanted to run many containers on a host with a single container managing the other containers through the API. But the problem I soon discovered is that at the moment when you turn networking on it is an all or nothing type of thing… you can’t turn networking off selectively on a container by container basis. You can disable IPv4 forwarding, but you can still reach the docker remote API on the machine if you can guess the IP address of it. One solution I came up with for this is to use nginx to expose the unix socket for docker over HTTPS and utilize client-side ssl certificates to only allow trusted containers to have access. I liked this setup a lot so I thought I would share how it’s done. Disclaimer: assumes some knowledge of docker! Generate The SSL Certificates We’ll use openssl to generate and self-sign the certs. Since this is for an internal service we’ll just sign it ourselves. We also remove the password from the keys so that we aren’t prompted for it each time we start nginx. # Create the CA Key and Certificate for signing Client Certs openssl genrsa -des3 -out ca.key 4096 openssl rsa -in ca.key -out ca.key # remove password! openssl req -new -x509 -days 365 -key ca.key -out ca.crt # Create the Server Key, CSR, and Certificate openssl genrsa -des3 -out server.key 1024 openssl rsa -in server.key -out server.key # remove password! openssl req -new -key server.key -out server.csr # We're self signing our own server cert here. This is a no-no in production. openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt # Create the Client Key and CSR openssl genrsa -des3 -out client.key 1024 openssl rsa -in client.key -out client.key # no password! openssl req -new -key client.key -out client.csr # Sign the client certificate with our CA cert. Unlike signing our own server cert, this is what we want to do. openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out client.crt Another option may be to leave the passphrase in and provide it as an environment variable when running a docker container or through some other means as an extra layer of security. We’ll move ca.crt, server.key and server.crt to /etc/nginx/certs. Setup Nginx The nginx setup for this is pretty straightforward. We just listen for traffic on localhost on port 4242. We require client-side ssl certificate validation and reference the certificates we generated in the previous step. And most important of all, set up an upstream proxy to the docker unix socket. I simply overwrote what was already in /etc/nginx/sites-enabled/default. upstream docker { server unix:/var/run/docker.sock fail_timeout=0; } server { listen 4242; server localhost; ssl on; ssl_certificate /etc/nginx/certs/server.crt; ssl_certificate_key /etc/nginx/certs/server.key; ssl_client_certificate /etc/nginx/certs/ca.crt; ssl_verify_client on; access_log on; error_log /dev/null; location / { proxy_pass http://docker; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 10m; client_body_buffer_size 128k; proxy_connect_timeout 90; proxy_send_timeout 120; proxy_read_timeout 120; proxy_buffer_size 4k; proxy_buffers 4 32k; proxy_busy_buffers_size 64k; proxy_temp_file_write_size 64k; } } One important piece to make this work is you should add the user nginx runs as to the docker group so that it can read from the socket. This could be www-data, nginx, or something else! Hack It Up! With this setup and nginx restarted, let’s first run a curl command to make sure that this setup correctly. First we’ll make a call without the client cert to double check that we get denied access then a proper one. # Is normal http traffic denied? curl -v http://localhost:4242/info # How about https, sans client cert and key? curl -v -s -k https://localhost:4242/info # And the final good request! curl -v -s -k --key client.key --cert client.crt https://localhost:4242/info For the first two we should get some run of the mill 400 http response codes before we get a proper JSON response from the final command! Woot! But wait there’s more… let’s build a container that can call the service to launch other containers! For this example we’ll simply build two containers: one that has the client certificate and key and one that doesn’t. The code for these examples are pretty straightforward and to save space I’ll leave the untrusted container out. You can view the untrusted container on github (although it is nothing exciting). First, the node.js application that will connect and display information: https = require 'https' fs = require 'fs' options = host: 172.42.1.62 port: 4242 method: 'GET' path: '/containers/json' key: fs.readFileSync('ssl/client.key') cert: fs.readFileSync('ssl/client.crt') headers: { 'Accept': 'application/json'} # not required, but being semantic here! req = https.request options, (res) -> console.log res req.end() And the Dockerfile used to build the container. Notice we add the client.crt and client.key as part of building it! FROM shykes/nodejs MAINTAINER James R. Carr ADD ssl/client* /srv/app/ssl ADD package.json /srv/app/package.json ADD app.coffee /srv/app/app.coffee RUN cd /srv/app && npm install . CMD cd /srv/app && npm start That’s about it. Run docker build . and docker run -n >IMAGE ID< and we should see a json dump to the console of the actively running containers. Doing the same in the untrusted directory should present us with some 400 error about not providing a client ssl certificate. I’ve shared a project with all this code plus a vagrant file on github for your own prusual. Enjoy!
October 31, 2013
by James Carr
· 14,313 Views
article thumbnail
Adding Appsec to Agile: Security Stories, Evil User Stories and Abuse(r) Stories
Because Agile development teams work from a backlog of stories, one way to inject application security into software development is by writing up application security risks and activities as stories, making them explicit and adding them to the backlog so that application security work can be managed, estimated, prioritized and done like everything else that the team has to do. Security Stories SAFECode has tried to do this by writing a set of common, non-functional Security Stories following the well-known “As a [type of user] I want {something} so that {reason}” template. These stories are not customer- or user-focused: not the kind that a Product Owner would understand or care about. Instead, they are meant for the development team (architects, developers and testers). Example: As a(n) architect/developer, I want to ensure AND as QA, I want to verify that sensitive data is kept restricted to actors authorized to access it. There are stories to prevent/check for the common security vulnerabilities in applications: XSS, path traversal, remote execution, CSRF, OS command injection, SQL injection, password brute forcing. Checks for information exposure through error messages, proper use of encryption, authentication and session management, transport layer security, restricted uploads and URL redirection to un-trusted sites; and basic code quality issues: NULL pointer checking, boundary checking, numeric conversion, initialization, thread/process synchronization, exception handling, use of unsafe/restricted functions. SAFECode also includes a list of secure development practices (operational tasks) for the team that includes making sure that you’re using the latest compiler, patching the run-time and libraries, static analysis, vulnerability scanning, code reviews of high-risk code, tracking and fixing security bugs; and more advanced practices that require help from security experts like fuzzing, threat modeling, pen tests, environmental hardening. Altogether this is a good list of problems that need to be watched out for and things that should be done on most projects. But although SAFECode’s stories look like stories, they can’t be used as stories by the team. These Security Stories are non-functional requirements (NFRs) and technical constraints that (like requirements for scalability and maintainability and supportability) need to be considered in the design of the system, and may need to be included as part of the definition of done and conditions of acceptance for every user story that the team works on. Security Stories can’t be pulled from the backlog and delivered like other stories and removed from the backlog when they are done, because they are never “done”. The team has to keep worrying about them throughout the life of the project and of the system. As Rohit Sethi points out, asking developers to juggle long lists of technical constraints like this is not practical: If you start adding in other NFR constraints, such as accessibility, the list of constraints can quickly grow overwhelming to developers. Once the list grows unwieldy, our experience is that developers tend to ignore the list entirely. They instead rely on their own memories to apply NFR constraints. Since the number of NFRs continues to grow in increasingly specialized domains such as application security, the cognitive burden on developers’ memories is substantial. OWASP Evil User Stories – Hacking the Backlog Someone at OWASP has suggested an alternative, much smaller set of non-functional Evil User Stories that can be "hacked" into the backlog: A way for a security guy to get security on the agenda of the development team is by “hacking the backlog”. The way to do this is by crafting Evil User Stories, a few general negative cases that the team needs to consider when they implement other stories. Example #1. "As a hacker, I can send bad data in URLs, so I can access data and functions for which I'm not authorized." Example #2. "As a hacker, I can send bad data in the content of requests, so I can access data and functions for which I'm not authorized." Example #3. "As a hacker, I can send bad data in HTTP headers, so I can access data and functions for which I'm not authorized." Example #4. "As a hacker, I can read and even modify all data that is input and output by your application." Thinking like a Bad Guy – Abuse Cases and Abuser Stories Another way to beef up security in software development is to get the team to carefully look at the system they are building from the bad guy's perspective. In “Misuse and Abuse Cases: Getting Past the Positive”, Dr. Gary McGraw at Cigital talks about the importance of anticipating things going wrong, and thinking about behaviour that the system needs to prevent. Assume that the customer/user is not going to behave, or is actively out to attack the application. Question all of the assumptions in the design (the can’ts and won’ts), especially trust conditions – what if the bad guy can be anywhere along the path of an action (for example, using an attack proxy between the client and the server)? Abuse Cases are created by security experts working with the team as part of a critical review – either of the design or of an existing application. The goal of a review like this is to understand how the system behaves under attack/failure conditions, and document any weaknesses or gaps that need to be addressed. At Agile 2013 Judy Neher presented a hands-on workshop on how to write Abuser Stories, a lighter-weight, Agile practice which makes “thinking like a bad guy” part of the team’s job of defining and refining user requirements. Take a story, and as part of elaborating the story and listing the scenarios, step back and look at the story through a security lens. Don’t just think of what the user wants to do and can do - think about what they don’t want to do and can’t do. Get the same people who are working on the story to “put their black hats on” and think evil for a little while, brainstorm to come up with negative cases. As {some kind of bad guy} I want to {do some bad thing}… The {bad guy} doesn’t have to be a hacker. They could be an insider with a grudge or a selfish customer who is willing to take advantage of other users, or an admin user who needs to be protected from making expensive mistakes, or an external system that may not always function correctly. Ask questions like: How do I know who the user is and that I can trust them? Who is allowed to do what, and where are the authorization checks applied? Look for holes in multi-step workflows – what happens if somebody bypasses a check or tries to skip a step or do something out of sequence? What happens if an action or a check times-out or blocks or fails – what access should be allowed, what kind of information should be shown, what kind shouldn’t be? Are we interacting with children? Are we dealing with money? With dangerous command-and-control/admin functions? With confidential or pirvate data? Look closer at the data. Where is it coming from? Can I trust it? Is the source authenticated? Where is it validated – do I have to check it myself? Where is it stored (does it have to be stored)? If it has to be stored, should it be encrypted or masked (including in log files)? Who should be able to see it? Who shouldn’t be able to see it? Who can change it, and to the changes need to be audited? Do we need to make sure the data hasn't been tampered with (checksum, HMAC, digital signature)? Use this exercise to come up with refutation criteria (user can do this, but can’t do that; they can see this but they can’t see that), instead of, or as part of the conditions of acceptance for the story. Prioritize these cases based on risk, add the cases that you agree need to be taken care of as scenarios to the current story, or as new stories to the backlog if they are big enough. “Thinking like a bad guy” as you are working on a story seems more useful and practical than other story-based approaches. It doesn’t take a lot of time, and it’s not expensive. You don’t need to write Abuser Stories for every user Story and the more Abuser Stories that you do, the easier it will get – you'll get better at it, and you’ll keep running into the same kinds of problems that can be solved with the same patterns. You end up with something concrete and functional and actionable, work that has to be done and can be tested. Concrete, actionable cases like this are easier for the team to understand and appreciate – including the Product Owner, which is critical in Scrum, because the Product Owner decides what is important and what gets done. And because Abuser Stories are done in phase, by the people who are working on the stories already (rather than a separate activity that needs to be setup and scheduled) they are more likely to get done. Simple, quick, informal threat modeling like this isn’t enough to make a system secure – the team won’t be able to find and plug all of the security holes in the system this way, even if the developers are well-trained in secure software development and take their work seriously. Abuser Stories are good for identifying business logic vulnerabilities, reviewing security features (authentication, access control, auditing, password management, licensing) improving error handling and basic validation, and keeping onside of privacy regulations. Effective software security involves a lot more work than this: choosing a good framework and using it properly, watching out for changes to the system's attack surface, carefully reviewing high-risk code for design and coding errors, writing good defensive code as much as possible, using static analysis to catch common coding mistakes, and regular security testing (pen testing and dynamic analysis). But getting developers and testers to think like a bad guy as they build a system should go a long way to improving the security and robustness of your app.
October 31, 2013
by Jim Bird
· 21,934 Views · 1 Like
article thumbnail
A MindMap for Java Developer Interviews
Over the years I have been a panelist in many of the interviews for Java Developers. I have previously written a post titled Top 7 tips for succeeding in a technical interview for software engineers which covers few of the general guidelines. In this post I will share a mind map containing general topics covered in a Java developer interview. I have prepared this as a general reference for myself to remember the pointers and to keep a common standard across the multiple interviews. XMind gives a nice listing of the map. You can find the map here. Here is Image which you can download and use. Finally here is a old fashioned tabbed content list which is easier to copy paste. Java-Topics OOPs Encapsulation Abstraction Inheritance Interface - Abstract Class Casting IS-A vs HAS-A Relationships Aggregation vs Composition Plymorphism Method overloading vs Method Overloading Compile time vs Runtime Threads Creating threads Multitasking Synchronization Thread Transitions Marker Interface Serialization Clonnable Shallow copy vs Deep Copy Collections Map, List and Set Equals - Hashcode Legacy - Synchronized Classes JVM Stack vs Heap Memory Garbage Collection JRE, JVM, JDK Class loaders Exception Checked Vs Unchecked Exceptions Exception handling best practices try, catch, finally, throw, throws APIs Files String - StringBuffer - String Builder Java IO XML SAX Based & DOM Based JAXB - Java API for XML Binding Access specifier Access modifier public protected deafult private final static synchronized abstract transient volatile Inner/Nested Classes JavaEE Basics Packaging the Applications WAR EAR Basics MVC Servlets Listeners Lifecycle JSPs APIs JPA JAX-WS SOAP, WSDL Webservices basics Contract first vs JAX-RS RESTful and its advantages JSF This is a work in progress and I hope to refine it further. Let me know if you have any comments. - See more at: http://jyops.blogspot.ie/2013/10/a-mindmap-for-java-developer-interviews.html#sthash.K0A5wDAz.dpuf
October 27, 2013
by Manu Pk
· 20,396 Views · 1 Like
article thumbnail
Scrum to Lean Kanban: Some Problems and Pitfalls
Some months ago I wrote an article on how to transition between Scrum and a Lean Kanban operation. It's an important capability for an organization to have, because when a Scrum project finishes it is likely to enter a "leaner" BAU (Business As Usual) support phase. There are consequences arising from such a move which experienced Scrum hands may find surprising, and perhaps even a little off-putting. In this article we'll look at the shift in mindset that is required to do this. "Whoa! Something screwy has happened to our task board, it looks different" Kanban boards are subtly different to the task boards commonly used in Scrum. At first blush they might look similar. Both have columns showing the progress of user story "tickets" from a backlog through states such as in progress, peer review, in test, and done. In either case there might also be a blocked column, although it is equally acceptable to add a "blocked" sticker, or to simply invert the ticket on the board. As the name suggests, a task board will show the progress of the tasks that are needed to complete user stories. Often these tasks will be kept within horizontal swim lanes - one lane per user story. When all of the tasks are done, the user story will also move into done. Each user story therefore "chases" its tasks across the board. A Kanban board on the other hand - which is meant to deal with smaller and finer-grained pieces of work - will typically track the progress of user stories themselves across the board. The requirements should be well understood and there should be little appreciable depth to the solutioning; there will be few if any explicit tasks associated with the user stories. There is therefore no need for horizontal swim lanes to keep tasks and user stories aligned. You might also notice that Work in Progress limits are given particular emphasis in Lean Kanban. This is because scope is not timeboxed into sprints. The only way to throttle the rate of ticket throughput, and to keep it to manageable levels, is therefore by making sure that WIP limits are rigorously enforced. These are often annotated to the column headers on a Kanban board. For example, if there are 3 developers and 1 tester, the WIP for in progress would be 3, and 1 for in test. "Hey…there's just one backlog" That's right. Since there are no sprints in Lean Kanban, there can be no meaningful separation between a "sprint backlog" and a "product backlog". Instead there's just a single backlog of enqueued work items being brought into progress. This has repercussions for product ownership because you no longer have a clear separation between the prioritization that a team does for itself on a sprint backlog, and the prioritization done by a Product Owner on the product backlog. In effect you've just got a product backlog. In this situation clear product ownership can become more important then ever…or it can become a complete non-issue. "The Product Owner has too much power, he keeps jerking our chain" Since there is only one backlog, the Product Owner (or customer representative) must constantly reprioritize the user stories within it. The Product Owner needs to have more operational control in Lean Kanban than in Scrum. Developers can action tickets from the backlog on a daily or even hourly basis. There is no notion of getting a product backlog in shape before "the next sprint starts". Product Owners are therefore much more closely involved in day-to-day delivery than they would be in Scrum, and their involvement in daily standups becomes much more important. Note that the extent of a Product Owner's decision making should not extend beyond the backlog, and a good Kanban Leader will protect the team and its work in progress just like a good ScrumMaster would. "Now the Product Owner has disappeared altogether" Business as Usual work often boils down to the maintenance of existing systems post-delivery. Depending upon the level of demand, it's quite plausible to have one Lean-Kanban team responsible for the maintenance of multiple systems. In this situation there is no product being delivered as such, and consequently there is no clear product ownership. Instead, work items are raised as change requests and triaged by the team who then manage and prioritize their own backlog. This means that the team needs a strong and shared sense of direction and purpose. "There's no vision for this project" That's because a Lean Kanban operation typically isn't a project at all. A defined end point is likely to be missing… remember that it's covering "Business as Usual work". These are small, repeatable changes that may affect diverse systems and without any sort of narrative to bind them together. There'll certainly be a purpose and a rationale for operating a Lean Kanban… but don't expect a project vision. "We don't even seem to have decent sprint goals any more" Yep, they've gone too. Since there is no project vision and no sprints on a Lean Kanban, we won't have any "sprint goals" either. What we might get is a grouping of work requests that fall within a larger epic of changes…but if we do, it could well be a cause for concern. We must ask: are those related changes really representative of "Business as Usual" work, or are they too high risk? Do they constitute a project? "Lean Kanban work seems very bitty. I can't get a decent chunk to chew on" The diet of a Lean Kanban should consist of small, "digestible" pieces of work that do not require much breaking down in order to action them. By definition they must be well-understood and low-risk. A team must know how to handle them without the need for impact analysis or de-scoping. You're unlikely to get a meaty piece of work; you're more likely to be sucking these things up through a straw. Velocity and lead times are particularly significant metrics in Lean Kanban. Having said that, substantial and time consuming pieces of work can be taken on board if they satisfy the criteria of low risk and clear scope. An example would be the sort of work that conforms to a templated change. Of course, this sort of work might not appeal to an agile developer. So let's be clear: it takes a different temperament to do Lean Kanban BAU work than project work in Scrum. They are different skill sets. Agile developers who are happy doing one can find it unsettling, or even unrewarding, if they are switched to the other. "Why aren't we doing planning poker any more?" Without a sprint backlog there is no budget of story points to be brought into a sprint. This in turn means that estimation exercises such as planning poker lose much of their significance. In a Lean Kanban operation velocity can be measured not in terms of story points - either estimated or actual - but simply as the number of tickets actioned over a set period. This also provides an indication of the lead time before a ticket is handled. If tickets are of too variable a size - for example, if they include small ones as well as larger templated changes - then they can be awarded points for how long, or how much effort, they took. T-Shirt sizes is one approach. Remember that these points should represent the actuals, not estimates, so there's still no need for planning poker. Velocity can be averaged for each size. Alternatively the sizes can be mapped to points (e.g. small = 1, medium = 3, large = 7) and an aggregate velocity calculated. "Some of the BAU work that's been coming through looks like project work to me" You could well be right. It's important that you raise your suspicions with your team lead. There's often politics involved, but here's the lowdown. In many organizations "Business as Usual" work is classed - you could almost say "written off" - as an operational expenditure (OpEx), and is not drawn from the capital expenditure (CapEx) assigned to projects. Internal customers often have an incentive to sneak through initiatives as BAU work so as not to incur capital expense on their departmental budgets. This is indeed a political issue. But be on your guard otherwise your team could be hobbled with project work being slipped in on the sly. Be particularly wary of significant numbers of related changes, large changes, a seemingly high level of risk with any work items, or changes of uncertain scope. These suggest, but do not prove, that a fast one might be being pulled. Your team lead (who is analagous to a ScrumMaster) should try and defend against this, so if you as a team member have your suspicions, it's important to bring them to your lead's attention. Conclusion, and what's next In this post we've looked at the important differences between Lean Kanban and Scrum, and what that means for a team. We've also reviewed how a reasonably informed choice can be made between them. In my next post we'll look at a hybrid approach known as ScrumBan which can potentially address both project and BAU work. ScrumBan is becoming increasingly popular and has significant ramifications for project scalability.
October 16, 2013
by $$anonymous$$
· 13,671 Views · 1 Like
article thumbnail
The Blogging Programmer's Style Guide: Front-End or Frontend?
Even among the large IT/development publications, I see inconsistencies in the use of the word front-end. Is it hyphenated or not?
October 1, 2013
by Mitch Pronschinske
· 55,791 Views · 4 Likes
article thumbnail
3 Styles of Agile: Iterative, Incremental, and Evolutionary
When I’m teaching training courses (as I was this week at Skills Matter) or advising clients on the requirements-side of software development (which I’m doing a lot of just now), I talk about a model I call the “3 Styles of Agile.” Incredibly, I’ve never blogged about this -- although the model is hidden inside a couple of articles over the years. So now the day has come… I don’t claim the “3 Styles Model” is the way it should be, I only claim that it is the way I find the world. While “doing Agile” on the code side of software development always comes back to the same things (stand-up meetings, test/behavior driven development, code review/pair programming, stories, boards, etc.) the requirements side is very very variable. The advice that is given is very variable and the degree to which that advice is compatible with corporate structures and working is very variable. However, I find three reoccurring styles in which the requirements side operates and interfaces to development. I call these styles: Iterative, Incremental and Evolutionary, and I usually draw this diagram: I say style because I’m looking for a neutral word. I think you can use Scrum, XP and Kanban (or any other method) in any of the three styles. That said, I believe Kanban is a better fit for evolutionary while Scrum/XP are a better fit for Iterative and Incremental. I try not to be judgmental, I know a lot of Agile folk will see Evolutionary as superior, they may even consider Evolutionary to be the only True Agile but actually I don’t think that is always the case. There are times when the other styles are “right.” Let me describe the three styles: Iterative In this style the development team is doing lots of good stuff like: stand up meetings, planning meetings, short iterations or Kanban flow, test driven development, code review, refactoring, continuous integration and so on. I say they are doing it but it might be better to say “I hope they are doing” because quite often some bit or other is missing. That’s not important for this model. The key thing is the dev team are doing it! In this model, requirements arrive in a requirements document en mass. In fact, the rest of the organization carries on as if nothing has changed, indeed this may be what the organization wants. In this model you hear people say things like “Agile is a delivery mechanism” and “Agile is for developers." The requirement document may even have been written by a consultant or analyst who is now gone. The document is “thrown over the fence” to another analyst or project manager who is expected to deliver everything (scope, features) within some fixed time frame for some budget. Delivery is most likely one “big bang” at the end of the project (when the team may be dissolved). In order to do this they use a bacon slicer. I’ve written about this before and called it Salami Agile. The requirements document exists and the job of the “Product Owner” is to slice off small pieces for the team to do every iteration. The development team is insulated from the rest of the organization. There is probably still a change review board and any increase scope is seen as a problem. I call this iterative because the team is iterating but that’s about it. This is the natural style of large corporations, companies with annual budgets, senior managers who don’t understand IT and in particular banks. Incremental This style is mostly the same as Iterative, it looks similar to start with. The team are still (hopefully) doing good stuff and iterating. There is still a big requirements document, the organization still expects it all delivered and it is still being salami sliced. However, in this model, the team is delivering the software to customers. At the very least, they are demonstrating the software and listening to feedback. More likely, they are deploying the software and (potential) users can start using it today. As a result, the customer/users give feedback about what they want in the software. Sometimes this is an extra feature and functionality (scope creep!) and sometimes it is about removing things that were requested (scope retreat!). The “project” is still done in the traditional sense that everything in the document is “done,” but now some things are crossed out rather than ticked. Plus some additional stuff might be done over and above the requirements document. I call this incremental because the customers/users/stakeholders are seeing the thing grow in increments -- and hopefully early value is being delivered. I actually believe this is the most common style of software development -- whether that work is called Agile, waterfall or anything else. However, in some environments this is seen as wrong, wrong because the upfront requirements are “wrong” or because multiple deliveries need to be made, or because the team aren’t delivering everything they were originally asked to deliver. Evolutionary Here again the development team are iterating much as before. However, this time there is no requirements document. Work has begun with just an idea. Ideally I would want to see a goal, an objective, an aim, which will guide work and help inform what should be done -- and this goal should be stated in a single sentence, a paragraph at most. But sometimes even this is missing, for better or worse. In this model the requirements guy and developers both start at the beginning. They brainstorm some ideas and select something to do. While Mr. Requirements runs off to talk to customers and stakeholders about what the problem is and what is needed, the tech team (maybe just one person) gets started on the best idea so far. Sometime soon (2 weeks tops) they get back together. Mr. Requirements talks about what he has found and the developers demonstrate what they have built. They talk some more and decide what to do next. With that done, the developers gets on with building and Mr. Requirements gets on his bike again, he shows what has been built and talks to people -- some people again and some new people. As soon as possible the team starts to push software out to users and customers to use. This delivers value and provides feedback. And so it goes. It finishes, if it finishes, when the goal is met to the organization decided to use its resources somewhere else. Evolutionary style is most at home in Palo Alto, Mountain View, and anywhere else that start-ups are the norm. Evolutionary is actually a lot more common than is recognized but it is called maintenance or “bug fixing” and seen as something that shouldn’t exist. Having set out the three styles I’ll leave discussion of how to use the model and why you might use each style to another entry. If you want to know more about each model and how I see Agile as spectrum have a look my 2011 “The Agile Spectrum” from ACCU Overload or the recently revised (expanded but unfinished) version by the same title: “Agile Spectrum” (the 2013 version I suppose, online only).
October 1, 2013
by Allan Kelly
· 25,232 Views
article thumbnail
The Real Cost of Change in Software Development
There are two widely opposed (and often misunderstood) positions on how expensive it can be to change or fix software once it has been designed, coded, tested and implemented. One holds that it is extremely expensive to leave changes until late, that the cost of change rises exponentially. The other position is that changes should be left as late as possible, because the cost of changing software is – or at least can be – essentially flat (that’s why we call it software). Which position is right? Why should we care? And what can we do about it? Exponential Cost of Change Back in the early 1980s, Barry Boehm published some statistics (Software Engineering Economics, 1981) which showed that the cost of making a software change or fix increases significantly over time – you can see the original curve that he published here. Boehm looked at data collected from Waterfall-based projects at TRW and IBM in the 1970s, and found that the cost of making a change increases as you move from the stages of requirements analysis to architecture, design, coding, testing and deployment. A requirements mistake found and corrected while you are still defining the requirements costs almost nothing. But if you wait until after you've finished designing, coding and testing the system and delivering it to the customer, it can cost up to 100 times as much. A few caveats here. First, the cost curve is much higher in large projects (in smaller projects, the cost curve is more like 1:4 instead of 1:100). Those cases when the cost of change rises up to 100 times are rare, what Boehm calls Architecture-Breakers, where the team gets a fundamental architectural assumption wrong (scaling, performance, reliability) and doesn't find out until after customers are already using the system and running into serious operational problems. This analysis was all done on a small data sample from more than 30 years ago, when developing code was much more expensive and time-consuming and paperworky, and the tools sucked. A few other studies have been done since then that mostly back up Boehm's findings – at least the basic idea that the longer it takes for you to find out that you made a mistake, the more expensive it is to correct it. These studies have been widely referenced in books like Steve McConnell’s Code Complete, and used to justify the importance of early reviews and testing: Studies over the last 25 years have proven conclusively that it pays to do things right the first time. Unnecessary changes are expensive. Researchers at Hewlett-Packard, IBM, Hughes Aircraft, TRW, and other organizations have found that purging an error by the beginning of construction allows rework to be done 10 to 100 times less expensively than when it's done in the last part of the process, during system test or after release (Fagan 1976; Humphrey, Snyder, and Willis 1991; Leffingwell 1997; Willis et al. 1998; Grady 1999; Shull et al. 2002; Boehm and Turner 2004). In general, the principle is to find an error as close as possible to the time at which it was introduced. The longer the defect stays in the software food chain, the more damage it causes further down the chain. Since requirements are done first, requirements defects have the potential to be in the system longer and to be more expensive. Defects inserted into the software upstream also tend to have broader effects than those inserted further downstream. That also makes early defects more expensive. There’s some controversy over how accurate and complete this data is, how much we can rely on it, and how relevant it is today when we have much better development tools and many teams have moved from heavyweight sequential Waterfall development to lightweight iterative, incremental development approaches. Flattening the Cost of Changing Code The rules of the game should change with iterative and incremental development – because they have to. Boehm realized back in the 1980s that we could catch more mistakes early (and therefore reduce the cost of development) if we think about risks upfront and design and build software in increments, using what he called the Spiral Model, rather than trying to define, design and build software in a Waterfall sequence. The same ideas are behind more modern, lighter Agile development approaches. In Extreme Programming Explained (the first edition, but not the second) Kent Beck states that minimizing the cost of change is one of the goals of Extreme Programming, and that a flattened change cost curve is “the technical premise of XP”: Under certain circumstances, the exponential rise in the cost of changing software over time can be flattened. If we can flatten the curve, old assumptions about the best way to develop software no longer hold … You would make big decisions as late in the process as possible, to defer the cost of making the decisions and to have the greatest possible chance that they would be right. You would only implement what you had to, in hopes that the needs you anticipate for tomorrow wouldn't come true. You would introduce elements to the design only as they simplified existing code or made writing the next bit of code simpler. It’s important to understand that Beck doesn't say that with XP the change curve is flat. He says that these costs can be flattened if teams work toward this, leveraging key practices and principles in XP, such as: Simple Design, doing the simplest thing that works, and deferring design decisions as late as possible (YAGNI), so that the design is easy to understand and easy to change Continuous, disciplined refactoring to keep the code easy to understand and easy to change Test-First Development – writing automated tests upfront to catch coding mistakes immediately, and to build up a testing safety net to catch mistakes in the future Developers collaborating closely and constantly with the customer to confirm their understanding of what they need to build and working together in pairs to design solutions and solve problems, and catch mistakes and misunderstandings early Relying on working software over documentation to minimize the amount of paperwork that needs to be done with each change (write code, not specs) The team’s experience working incrementally and iteratively – the more that people work and think this way, the better they will get at it. All of this makes sense and sounds right, although there are no studies that back up these assertions, which is why Beck dropped this change curve discussion from the second edition of his XP book. But, by then, the idea that change could be flat with Agile development had already become accepted by many people. The Importance of Feedback Scott Amber agrees that the cost curve can be flattened in Agile development, not because of Simple Design, but because of the feedback loops that are fundamental to iterative, incremental development. Agile methods optimize feedback within the team, developers working closely together with each other and with the customer and relying on continuous face-to-face communications. Following technical practices like test-first development, pair programming and continuous integration makes these feedback loops even tighter. But what really matters is getting feedback from the people using the system – it’s only then that you know if you got it right or what you missed. The longer that it takes to design and build something and get feedback from real users, the more time and work that is required to get working software into a real customer’s hands, the higher your cost of change really is. Optimizing and streamlining this feedback loop is what is driving the lean startup approach to development: defining a minimum viable product (something that just barely does the job), getting it out to customers as quickly as you can, and then responding to user feedback through continuous deployment and A/B testing techniques until you find out what customers really want. Even Flat Change Can Still Be Expensive Even if you do everything to optimize these feedback loops and minimize your overheads, this still doesn’t mean that change will come cheap. Being fast isn’t good enough if you make too many mistakes along the way. The Post Agilist uses the example of painting a house: Assume that it costs $1,000 each time you paint the house, whether you paint it blue, red or white. The cost of change is flat. But if you have to paint it blue first, then red, then white before everyone is happy, you’re wasting time and money. “No matter how expensive or cheap the "cost of change" curve may be, the fewer changes that are made, the cheaper and faster the result will be … Planning is not a four letter word.” (However, I would like to point out that “plan” is.) Spending too much time upfront in planning and design is waste. But not spending enough time upfront to find out what you should be building and how you should be building it before you build it, and not taking the care to build it carefully, is also a waste. Change Gets More Expensive Over Time You also have to accept that the incremental cost of change will go up over the life of a system, especially once a system is being used. This is not just a technical debt problem. The more people using the system, the more people who might be impacted by the change if you get it wrong, the more careful you have to be. This means that you need to spend more time on planning and communicating changes, building and testing a roll-back capability, and roll changes out slowly using canary releases and dark launching – which add costs and delays to getting feedback. There are also more operational dependencies that you have to understand and take care of, and more data that you have to change or fix up, making changes even more difficult and expensive. If you do things right, keep a good team together and manage technical debt responsibly, these costs should rise gently over the life of a system – and if you don’t, that exponential change curve will kick in. What is the real cost of change? Is the real cost of change exponential, or is it flat? The truth is somewhere in between. There’s no reason that the cost of making a change to software has to be as high as it was 30 years ago. We can definitely do better today, with better tools and better, cheaper ways of developing software. The keys to minimizing the costs of change seem to be: Get your software into customer hands as quickly as you can. I am not convinced that any organization really needs to push out software changes 10 to 50 to 100 times a day, but you don’t want to wait months or years for feedback, either. Deliver less, but more often. And because you’re going to deliver more often, it makes sense to build a continuous delivery pipeline so that you can push changes out efficiently and with confidence. Use ideas from lean software development and maybe Kanban to identify and eliminate waste and to minimize cycle time. We know that, even with lots of upfront planning and design thinking, we won’t get everything right upfront -- this is the Waterfall fallacy. But it’s also important not to waste time and money iterating when you don’t need to. Spending enough time upfront in understanding requirements and in design to get it at least mostly right the first time can save a lot later on. Whether you’re working incrementally and iteratively, or sequentially, it makes good sense to catch mistakes early when you can, whether you do this through test-first development and pairing, or requirements workshops and code reviews -- whatever works for you.
September 20, 2013
by Jim Bird
· 22,330 Views
article thumbnail
This is how Facebook develops and deploys software. Should you care?
A recently published academic paper by Prof. Dror Feitelson at Hebrew University, Eitan Frachtenberg a research scientist at Facebook, and Kent Beck (who is also doing something at Facebook), describes Facebook’s approach to developing and deploying its front-end software. While it would be more interesting to understand how back-end development is done (this is where the real heavy lifting is done scaling up to handle hundreds of millions of users), there are a few things in the paper that are worth knowing about. Continuous Deployment at Facebook is Not Continuous Deployment Rather than planning work out into projects or breaking work into time-boxed Sprints, Facebook developers do most of their work in independent, small changes that are released frequently. This makes sense in Facebook’s online business model, everyone constantly tuning the platform and trying out new options and applications in different user communities, seeing what sticks. It’s a credit to their architecture that so many small, independent changes can actually be done independently and cheaply. Facebook says that it follows Continuous Deployment, but it’s not Continuous Deployment the way that IMVU made popular where every change is pushed out to customers immediately, or even how a company like Etsy does Continuous Deployment. At Facebook, code can be released twice a day, but this is done mostly for bug fixes and internal code. New production code is released once per week: thousands of changes by hundreds of developers are packaged up by their small release team on Sundays, run through automated regression testing, and released on Tuesday if the developers who contributed the changes are present. Release engineers assess the risk of changes based on the size of the change, the amount of discussion done in code reviews (which is recorded through an internal code review tool), and on each developer’s “push karma”: how many problems they have seen from code by this developer before. A tool called “Gatekeeper” controls what features are available to which customers to support dark launching, and all code is released incrementally – to staging, then a subset of users, and so on. Changes can be rolled-back if necessary – individually, or, as a last resort, an entire code release. However, like a lot of Silicon Valley DevOps shops, they mostly follow the “Real Men only Roll Forward” motto. Code Ownership A key to the culture at Facebook is that developers are individually responsible for the code that they wrote, for testing it and supporting it in production. This is reflected in their code ownership model: Developers must also support the operational use of their software — a combination that’s become known as “DevOps.” This further motivates writing good code and testing it thoroughly. Developers’ personal stake in keeping the system running smoothly complements the engineering procedures and lets the system maintain quality at scale. Methodologies and tools aren’t enough by themselves because they can always be misused. Thus, a culture of personal responsibility is critical. Consequently, most source files are modified by only a few engineers. Although at least one other engineer reviews all changes before they’re committed, a third of the source files have only been edited by one engineer, and another quarter by two. Only 10 percent of the files are handled by more than seven engineers. On the other hand, the distribution of engineers per file has a heavy tail, with the most widely shared file handled by no fewer than 870 distinct engineers. These widely shared files are predominantly library files and also include major configuration and top-level PHP files. Testing? We don’t need no stinking testing … Facebook doesn't have an independent test team, because, it says, doesn'tneed one. First, they depend a lot on code reviews to find bugs: At Facebook, code review occupies a central position. Every line of code that’s written is reviewed by a different engineer than the original author. This serves multiple purposes: the original engineer is motivated to ensure that the code is of high quality, the reviewer comes with a fresh mind and might find defects or suggest alternatives, and, in general, knowledge about coding practices and the code itself spreads throughout the company. Developers are also responsible for writing unit tests and their own regression tests – they have “tens of thousands of regression tests” (which doesn't sound like nearly enough for 10+ million lines of mostly PHP code compiled into C++, in both of which languages coding mistakes are easy to make) and automated performance tests. And developers also test the software by using the development version of Facebook for their personal Facebook use. According to the authors, “this is just one aspect of the departure from traditional software development”. But Facebook developers using their own software internally (and passing this off as “testing”) is no different than the early days at Microsoft where employees were supposed to “eat their own dog food”, a practice that did little if anything to improve the quality of Microsoft products. Facebook also depends on customers to test the software for it. Software is released in steps for A/B testing and “live experimentation” on subsets of the user base, whether customers want to participate in this testing or not. Because its customer base is so large, it can get meaningful feedback from testing with even a small percentage of users, which at least minimizes the risk and inconvenience to customers. Security??? While performance is an important consideration for developers at Facebook, there is no mention of security checks or testing anywhere in this description of how Facebook develops and deploys software. No static analysis, dynamic analysis/scanning, pen testing or explanation of how the security team and developers work together, not even for “privacy sensitive code” – although this code is “held to a higher standard” it doesn’t explain what this “higher standard” is. Presumably it relies on the use of libraries and frameworks to handle at least some AppSec problems, and possibly to look for security bugs in its code reviews, but it doesn't say. There isn’t much information available on Facebook’s AppSec program anywhere. The security team at Facebook seems to spend a lot of time educating people on how to use Facebook safely and how to develop Facebook apps safely and running their bug bounty program which pays outsiders to find security bugs for them. A search on security on Facebook mostly comes back with a long list of public security failures, privacy violations and application security vulnerabilities found over the years and continuing up to the present day. Maybe the lack of an effective AppSec program is the reason for this. This is the way Facebook is Developed. Should you care? While it’s interesting to get a look inside a high-profile organization like Facebook and how it approaches development at scale, it’s not clear why this paper was written. There is little about what Facebook is doing (on its front-end development at least) that is unique or innovative, except maybe the way it uses BitTorrent to push code changes out to thousands of servers like Twitter does, something that I already heard about a few years ago at Velocity and that has been written about before. I like the idea of developers being responsible for their work, all the way into production, which is a principle that we also follow. Code reviews are good. Dark launching features is a good practice and has been a common practice in systems for a long time (even before it was called "dark launching"). Not having testers or doing AppSec is not good. Otherwise, I'm not sure what the rest of us can learn from or would want to use from this.
September 4, 2013
by Jim Bird
· 43,052 Views · 1 Like
article thumbnail
XP Values: Courage
In a complex system such as a software development team, it's easy for fear to arise.
August 28, 2013
by Giorgio Sironi
· 6,844 Views
article thumbnail
Kanban Paper Airplane Factory
i went to the local capital kanban meetup yesterday evening. it was a bunch of project managers discussing kanban and waste in it. seemed completely out of my comfort zone and a way to meet new people in tech here in town so i attended. it turned out to be really cool and way more interesting than my expectations were. i wanted to mention some of those here, specifically some of the it wastes that were mentioned i see all the time, the insights i got from the paper airplane factory game, and some after meeting talk that changed my perspective on what i perceive as problems in our industry with good software east of california (hah, trick question, there is no good software done east of california…). it wastes andrea ross did a presentation about waste in it. kanban, the production process used by toyota then turned into a project management and software development, has 7 or 8 forms of what it calls waste. these are primarely in the factory line production process, so you have to draw your own metaphors and similes, and that’s what andrea’s presentation extrapolated on. from a high level, these are: defects / rework overproduction waiting non-standard over processing transportation / logistics intellect motion excess inventory her slides that have bullet point examples for each one are pretty self-explanatory. what was interesting to me was the sheer volume of bullet points i see all the time, together, in the same projects i work on. some can’t be avoided, nature of the business and all that. still, it was pretty eye opening to see that a traditional factory production process has identified these items as the core waste items, and software development has plenty of them with just about the same meanings. i won’t cover them in detail here as her slides do. kanban, bottlenecks, and waste the concept of kanban is to quickly identify bottlenecks in the existing production process, and iterate to improve the process to fix them. notice i said “existing” and “process”. the existing part is where kanban has been easier to market than say six sigma which is bought into wholesale, hence why it’s easier to be a six sigma consultant than a kanban one. kanban you basically overlay on top of what you have and it surfaces the problems your existing process has pretty clearly. meaning, if you see a bunch of cards on a kanban board that are in the “analysis” column, and very few in the rest, it’s pretty clear where the bottle neck is located. now the “process” part is analogous to the production line; in this case all that goes into making software from the traditional waterfall perspective: design, development, deployment. however, the key here is you aren’t fixing the “bottleneck”, but rather the process itself. that is what i learned through our paper airplane factory exercise quite clearly. there are a series of games like this that can be modified, but the point is they help teach the bottleneck vs. process modification process extremely clearly. the key takeaway for me was fixing the bottleneck, like the 5 developers + 1 manager in a war room during a troubling moment during a software project, is actually a form of waste. yes, it’s great teams rise up to tackle these problems in the moment. however, it’s important to note that it’s the project manager’s job to both recognize this as waste and fix the actual process problem. i’ll explain this below. note: if you’re concerned about spoilers, please be aware of 2 things. first, there are more than just the airplane factory game that you can find online. second, if you do read the following section and later participate in the exercise, please either let the teacher/presenter know, or try not to modify the process too much to allow others to learn. airplane factory the game is like so (abridged version, you can find the full instructions here ): divide your people into 4 groups, each sitting adjacent to each other. circle or semi-circular seating arrangements works best to encourage intentional bottleneck adjustment engagement. cut up the airplane folding instructions alone the designated lines. give the first part of the instructions to the first group in the line. give the second part of the instructions to the second group, and repeat on down the line. some people may not necessarily have designated jobs beyond passing papers, etc. this is intentional to illustrate the intellect waste of not using human ip… and also to note how they’ll often become efficient passers, helpers, or even qa. ensure the group/person who’s last in line is aware of how far the plane must fly as a metric of defining a successful plane. setup a 5 minute timer, start it, and yell “go!” after 5 minutes, identify how many successfully flown airplanes were made as well as how much waste (crumpled papers, non-flying planes, etc) were created. that’s the round 1 score. iterate. the iterate step is where you reflect on what just happened and attempt to modify the process. i’ll go over how ours went down so you get an idea. round 1 line setup : we had 8 people in our line. round 1, we had 1 lady do the half fold, the 2nd guy do the additional 2 folds + paper sides cut, andrea and i pass the paper to our left, another lady handle the 1st wing folds, and a gentleman at the end to make the wings and throw it. our last person was a lady who handled qa and scoring. process : very quickly we had a bottleneck with the lady at my stations left. the instructions weren’t very clear and she struggled to learn how to do the first one. both andrea and i quickly went to help; andrea attempting to do it with her, me taking a picture of the instructions with my iphone, and attempting to duplicate at my desk while ensuring i kept passing the planes to my left into an ever growing pile. once i figured it out (i had actually built the exact same plane last week for my daughters birthday present which has an electronic propeller you attach), i told the ladies to ignore the “requirements” as they were crap and i walked them through how to successfully complete their step. i then quickly returned to my desk which had a pile of unmoved inventory. the only person who didn’t struggle with their assembly was the 1st in the line who had to fold paper in half. i believe we ended up with 2 planes and 1 waste. takeaway : our teacher quickly pointed how we went “downstream” to fix the production line process. this is a reactionary, and completely normal mode, to fix production line problems. it’s also wrong. you’re supposed to identify the upstream problem causing it and fix that. additionally, we didn’t stop the line to ensure we fixed this problem first before continuing, also wrong. car companies like toyota do this via a chain that’s pulled to stop the line so they can ensure the problem is resolved. sometimes they even take part of their line off the main line to ensure things keep moving. as a side note, apparently gm used to keep going. door doesn’t fit right? keep going; jam the mofo on there. they’d end up with a lot full of cars pretty quickly… even if they were low quality. ford was similar, but they’d ensure the cars were actually sold first before they sold them, thus not resulting in lots full of inventory they couldn’t sell like gm… even if the quality of pre-sold cars was still low. we also noted various other problems such as no training in each plane’s building instructions, no one stopping if the station after them go overwhelmed with their ever growing stack, and sometimes idle resources (people with not much to do). round 2 improvements : first, the teacher actually implemented designated stack areas with a piece of paper on each station, and then wrote a number on the paper; this was the max amount of completed planes you could place for the next station to build so you didn’t exacerbate a bottleneck. second, i become a designated passer to my left while my partner moved to the left station to have a 2 women team doing the complicated folding. process : my job was pretty easy; pass, and ensure i don’t bottleneck it. every single station was faster since they had practiced their portion. the guy to my right had actually done his first 3 paper cuts wrong in round 1 which caused confusion to my left station, but had it down pat in round 2. the last station still experimented with various angles of folding to see how far the plane could actually fly. we actually had the last women in the line, qa, send a messed up plane back through the line as an unfolded piece of paper because it didn’t fly right; w00t, less waste! a bottleneck, again, formed to my left, but the girls found a way to divide up the 2 step process between them to be more efficient. as our 5 minutes progressed, they got faster and eventually started making progress on the backlog. we made 5 planes, 1 waste. takeaway : we built 4 planes with 1 waste. the first person, as usual, was too fast. the guy to my right had an inefficient process because he’d have to fold, pick up the scissors, cut, then put ‘em down again. we had all abandoned our airplane instructions by this point. round 3 improvements : everything else was fine, so we decided to give me the cutting job and the guy to my right would just fold. the girls to my left on-the-fly modification was good and we kept it. process : my first 3 were slow, but once i practiced, i was uber fast and we were humming. the girls to my left were killing it. i managed to keep my right stack always below or at 2 in the pile. very quickly it became apparent the 1st person was too fast; she was constantly folding and then waiting before she was allowed to make another. we made 8 airplanes, no waste. takeaway : those of with spouses were already getting texted like mad to leave, but we all wanted to see round 3 succeed better than 2, and see if we made the process perfect. we didn’t; it went the complete opposite direction to the front of the line needing minor modifications. overall, though, our output increased a lot, our waste went down, and it was very clear that the adjustments we made + the teachers maximum stack amounts were working well. my overall takeaways i went to this meeting to both meet new people in town to network with as well as to get out of my comfort zone. i find when i do the latter, i learn a lot and sometimes get a new perspective. it gave me a new appreciation for project managers who have not just 1, but 5 projects they have to manage to make an attempt to do this on. this also assumes they get enough time to really learn about each teams issues, where those bottlenecks are, and what the best ways are to address them. not by just fixing the bottlenecks, but by fixing the process itself, ensuring stop guards are in place not as many items/cards in a column, etc. it also made me intimately aware of how i, as a consultant, immediately want to fix the bottleneck, and have learned ways (such as the war room) to solve them… when in reality, it’s a pm issue for a greater process problem. the other thing that makes it more complex is the whole “all things being equal”. for example, a kanban board a pm would use on the whole process vs. just the kanban board my software team would use. if my team fails to do tdd and ends up with a variety of bugs in the system because we’re forced to develop quickly and produce bad work, this show up on hers as us being the bottleneck. without time to talk to us and really empower us to change our process, nothing will change. i see this time and time again. the excuses, which are sometimes valid, range from “the software’s good enough even with the bugs”, or “tdd is too much work for not enough value” or “we can’t write a test suite for a huge mess that isn’t even testable”. …and that’s just a small portion of what i’ve seen gone wrong. if you’ve ever worked for a design agency, or even a large firm that has a huge new client, it’s very apparent many teams have a hard time getting sign off from clients which causes a bottleneck in the analysis column on the kanban board because the items either pile up, or priority constantly shifts… yet they never actually make it out of their column. a pm there who works with the government offered his strategies for dealing with the strange qa cycles government agencies will have where it goes into a black hole for 6 weeks thus really screwing up his kanban metrics. overall, it was neat to be in a room with people who were geeking out on improving process. you see a lot of software developers get bored with programming or frustrated with how their lack of process is going, so they read up on xp and agile. when you look at what these pm’s deal with, it makes you feel like just a small part in a larger overall process. more importantly, my preconceptions about leadership being the problem 99% of my problem projects really had a wrench thrown in. i was bitching about it to one of the pm’s, and quickly explained, in great detail, why some big companies which don’t have a hard line metric such as money to predict performance will often use lean methodologies since “ensuring customer satisfaction” is hard to measure depending on your business, and requires a more exploratory way of doing business. that said, it was great to hear that the common problems i experience in software dev with solutions were the same, just 1 of many that pm’s have to deal with. i highly encourage software developers to partake in one of these exercises, even if you do scrum vs. kanban. really eye opening stuff.
August 14, 2013
by James Warden
· 12,131 Views · 1 Like
article thumbnail
Limiting WIP: Stories vs. Tasks
We’re all works in progress, honey. And believe me when I tell you that I’ve had to work harder than most. ― Susan Elizabeth Phillips, "Ain't She Sweet" It's pretty well understood that limiting Work In Progress - or WIP as it is often abbreviated - is a good thing. Ideally, WIP should be limited to one item in progress at a time. Having multiple pieces of inventory on-hand is a form of waste, since each incurs a handling cost, and any work done on one of them will depreciate while another is being worked on. In theory at least, restricting WIP to one item at a time will reduce this waste and get value out of the door as quickly as possible. This principle of Single Piece Flow (SPF) is central to Lean-Kanban ways of working, especially in a manufacturing context. In a software context the accepted WIP limits tend to be rather higher. It is often limited to one item per developer, such as by allowing each developer only one avatar to place on an item, and it can be reduced further if pair-programming is in use. As such, software teams might not often achieve SPF but the value of limiting WIP as far as possible is still understood. There are however problems in interpreting limited WIP in Scrum. This is because a Scrum board will often take the form of a task board ... not a Kanban board. In other words, the work being limited by Scrum teams is not always a user story or similar requirement. It is often a task. Task-limited WIP allows developers to progress tasks from any user story in any order. They could potentially limit themselves to one or two tasks from a story, complete them, then move on to a task from a different story and maybe a task from a third. In effect multiple stories - perhaps even the entire Sprint Backlog of stories - can be in progress before so much as one story gets completed. None of this breaks Scrum rules. There's nothing to stop a team, in Sprint Planning, from organizing the Sprint Backlog into any number of tasks which can be progressed in any order they choose, and from delivering all of the user stories in one go at the end of the Sprint. The Sprint Goal can of course be met by this approach, and there should still be a nice task burn-down to show the associated technical risks being managed. The problem is that it defers approval of each user story to the end of the Sprint (i.e. the Sprint Review), when it is best-practice to get continual sign-off by a Product Owner throughout the iteration. On-going inspection allows the business risks of delivery to be managed well, and not just the technical risks. This is an issue that all Scrum teams must consider when they formulate a Sprint Plan. Is it important to limit WIP in terms of user stories rather than tasks, and thereby facilitate early approval of those stories by a Product Owner? Or would this compromise the team's principle of incremental delivery ... and amount to Lean-Kanban by the back door?
August 6, 2013
by $$anonymous$$
· 5,592 Views
article thumbnail
Sprint Retrospectives in Practice
Remembrance and reflection, how allied; What thin partitions sense from thought divide. - Alexander Pope Retrospectives, and why you need them A couple of months ago we looked at how to conduct a Sprint Review. We saw that a Review considers what work was done, and distinguished it from a Sprint Retrospective which reflects upon how work is being done. The distinction between the two can appear to be rather academic, and slurring a Review and a Retrospective together is a mistake that is often made by immature teams. After all, both take a reflective view of a Sprint that has just finished, and both can be said to fulfill a remit of historical inquiry. Yet while the separation of concerns might seem to be a narrow one, it is nonetheless quite precise, and there is great value to be had in maintaining the appropriate focus. A Review looks candidly at what has been achieved, and soberly at what remains to be achieved, with regard to product completion. A Retrospective on the other hand is an opportunity for the Scrum Team to inspect and adapt their actual implementation of the Scrum process. The rationale behind this inspection is methodological but it is in no sense abstract. It is grounded firmly in the desire to achieve worthwhile and practical reform. Perhaps there are certain working practices which the team can make more efficient, or which can otherwise be improved upon. If so, a Retrospective presents the ideal opportunity for those improvements to be discussed and brought into action. Failing to inspect and adapt in this manner will condemn a team to perpetual infancy and the repetition of past mistakes. Sprint Retrospectives help keep a Scrum team on the road to continual improvement. When these sessions are done well, team members will not be afraid to challenge the status quo, and will do so in a constructive and informed manner. The result will be an improved delivery of value – in fact, the sort of productivity gain that might well be identified in the Sprint Review we considered earlier. In this article we’ll switch our attention fully to Retrospectives, and examine the matter of how they should be conducted. Setting up a Retrospective As any event manager will tell you, the key to a successful gig lies in the preparation. Okay…I’ll concede that a holding a Retrospective isn’t as mammoth an undertaking as hosting the Thinking Digital conference, nor can it be said to demand the organizational skills of Bruce Springsteen’s road manager. Nevertheless it’s still important to get a few ducks in a row. Let’s start by lining them up and giving them some admittedly rather unimaginative names: Why, Who, Where, When, and What. We’ve just covered the issue of why a Retrospective needs to be held…that duck’s down. Let’s pop the rest. Who should attend a Sprint Retrospective? The invitation list for a Sprint Retrospective should be simple and uncontroversial. According to the Scrum Guide all Scrum Team members are expected to attend. That’s the Developers, the Scrum Master (who may facilitate the session), and the Product Owner. No others are expected. In fact, it would be quite irregular to extend the invitation to other people, even if they consider themselves to be important players or stakeholders. That’s because it is the Scrum Team who are responsible for the way they have implemented the Scrum Framework. Only they are in a position to inspect and adapt their very own ways of working. For this reason, all members have a duty to be present, to contribute, and to help make each Retrospective a success. Some teams exclude the Product Owner from this activity, arguing that if he or she was present, the team would not be able to have an open and frank discussion. This is a common issue and we’ll return to it later. For now though, just take it as read that a good Retrospective must include all Scrum Team members, and will give each a voice in molding the processes and working environment that they collectively own. Where should a Retrospective be held? Let’s answer this one with another question. If all of the Scrum Team members are co-located, and if they have the necessary equipment to hand (such as their Scrum board, plus a whiteboard for notes), why not hold the retrospective in situ? In other words, why not just hold the session at the team’s desks? Well, although this might sound like a capital idea, there can be problems. Perhaps it would create too much of a disturbance and annoy other teams within earshot? Then again, perhaps the physical layout of the working area is simply not conducive to holding a meeting. Perhaps the team is not entirely co-located in the first place. Any one of these things can tip the balance in favour of booking an actual meeting room, and getting everyone to decamp there for a Sprint Retrospective. If so, remember to book such a room in advance…if possible as a recurring appointment for the anticipated duration of the project. Make sure it has sufficient capacity and the resources needed. When should a Retrospective happen? The glib answer is to say that a Retrospective should happen “at the end of each Sprint”. A more useful answer would say whether or not it should precede or follow the Sprint Review. In my experience it is generally better to do the Review first, because that helps to establish a context within which the Retrospective can happen. The next thing to consider is how long to allow for the session. As with all Scrum events, a Sprint Retrospective is time-boxed. This means that it isn’t allowed to exceed a set length. The rules of Scrum are exact: for a one month Sprint the limit for a Retrospective is 3 hours, which is reduced to one-and-a-half hours for a two week Sprint. You should adjust this value by the same ratio if needed. Note that if a Retrospective finishes before the time-box expires, that’s fine and dandy. You aren’t obliged to use all of the available time. The rule is simply that the time-box must never be exceeded. Scrum is not a philosophy in which matters are allowed to drag on. What topics should the Retrospective cover? This is the biggest duck in the row, and it’ll take a few pings to knock it down. What we have to do is to establish a suitable agenda for a Sprint Retrospective. We have to formulate it in such a way that the inspection of the team’s Scrum implementation does indeed happen. We also have to make sure that any recommendations for its adaptation are elicited, agreed, and turned into achievable action items. The Scrum Guide provides us with something of a starting point. It isn’t much, but I reckon that if you look at it through a beer glass with your head sideways and one eye closed, you can just about discern a notional agenda for holding a Sprint Retrospective. A notional agenda The Scrum Guide is sparing in the advice it gives on how to conduct a Retrospective. We are told that a Scrum Team must: Inspect how the last Sprint went with regards to people, relationships, process, and tools; Identify and order the major items that went well and potential improvements; and, Create a plan for implementing improvements to the way the Scrum Team does its work…[including]…ways to increase product quality by adapting the Definition of “Done” as appropriate. Yes, I know that’s not much to go on, but each of these items is clearly significant. They seem to address the very rubric of agile practice; we can recognize in them a succinct appeal to the three legs of Transparency, Inspection, and Adaptation. In them, we can see not only a notional agenda, but also how critical a Sprint Retrospective is to the Scrum process. A Retrospective is arguably the most important time-boxed event that any agile process can have. If we want to turn these points into a more formal agenda for the session, we’ll have to make sure that each of them is addressed carefully. Towards a canonical format Scrum has been around for well over a decade now, and a fairly standard agenda for conducting a Sprint Retrospective has emerged. Here’s what it looks like. Set the scene. Ways to do this can include any or all of the following: Sketching out a timeline of significant events that occurred in the Sprint, so its historical context can be established Holding the Sprint Review shortly beforehand, so the project context is fresh in attendees’ minds Declaring the Prime Directive in order to define a professional context of mutual respect and openness Assess prior action items. Unless this is the first sprint, there will have been an earlier retrospective in which some improvements will have been proposed. Look back over each of them. Have they been followed through? In short, has the process actually been adapted following that earlier inspection? If any action items remain undone, make a note of them. They’ll have to be considered when determining actions for the future. Set up a Retrospective Board. This can be a whiteboard, or even a large sheet of paper stuck to a wall. Divide it into four quadrants and label each in the following manner. The precise terminology does tend to vary a bit. There can be subtle and not-so-subtle differences in meaning (consider the difference between “good points” and “things to continue doing”). Be aware of these differences, as they will shape the responses and ultimately the results. “What went well” (or “good points”, or “things to continue doing”) “What didn’t go well” (or “bad points”, or “things to stop doing”) “Ideas for improvement” (or things to “start doing”) “Shout-outs” (i.e. recognition of noteworthy individual contributions) Storm the Board. There are several ways in which this can be done. Here are some of the more common ones: Sticky notes. This method is fairly democratic in that each attendee gets a clear say by putting sticky notes on a board. Assertive individuals are therefore less able to dominate others. However, it can be disjointed as attention shifts from one person’s topics to another person’s. As such, it can be hard to develop a line of thought for group discussion. Here’s the process: Blocks of notes are distributed to the attendees. They are given a small time-box (5 or 10 minutes) to jot down their ideas…good points, bad points, improvements, and shouts. Each attendee should write one point per sticky note. There is no limit to the number of points they can make. After the time is up, attendees take it in turn to put their notes on the board and in the relevant quadrants As an attendee puts their sticky note on the board, they briefly state what the point is to the rest of the team Once the last attendee has finished, duplicate points will be identified by the group and removed. Facilitator-as-arbitrator. In this approach a facilitator will act as a scribe for the group, and write their ideas on the board. Group discussion of ideas is encouraged, and the facilitator can arbitrate in the event of disagreement. The downside is that it can favor the more assertive type of individual who ends up doing most of the talking. Here’s how it’s done: The facilitator stands in front of the board with a marker pen Any attendee who has a suggestion to make will make it – a good point, bad point, idea, or shout-out The facilitator writes each suggestion into the appropriate quadrant, disallowing any duplicates. The group discuss the merits of each suggestion The facilitator will erase, keep, or revise each suggestion according to group opinion Hybrid. This uses a mix of techniques, such as a facilitated session for identifying good points and bad points, and a sticky-note approach in order to elicit ideas for improvement. Changing the techniques used in a Retrospective every now and then can help keep the sessions fresh, and is certainly a good idea if you reckon they are getting a bit stale. Propose actions. I have five rules that I apply when “storming the board” with a team: For every bad point there must be an idea for improvement. In other words, for everything that people are being asked to stop doing an alternative and better course of action must be proposed. This rule helps to keep attendees focused on the need to adapt the process constructively, and not to use the session for mere complaint. If you have been storming for “good points” rather than for things to “start doing”, make sure that each of those points is matched with an idea for further improvement. It isn’t enough to look back appreciatively whenever something positive has happened. Your challenge is to translate that observation into an even bigger future win. Re-assess undone action items from the previous Retrospective. If any remain undone, ask if they are worth bringing forward. Ask why they weren’t implemented, with a view to finding out what really needs to happen to expedite them. If these outstanding actions are impractical, or are no longer relevant, jettison them and concentrate on those improvements which are valuable and achievable. Ask the “Five Whys”. For each action item you produce, you need to be sure that you have understood the root cause and that the action will be appropriate. A shallow retrospective is no retrospective at all. It has to be deep and probing. Improve the Definition of Done. The Scrum Guide doesn’t provide much advice about holding Retrospectives, but it is quite clear about the need to revisit the Definition of Done. This is something that many teams, including some quite experienced ones, forget or otherwise fail to do. So be careful to identify any room for improvement in the team’s understanding of what “done” means, and what it should take for work to be considered potentially releasable. Vote. It’s quite possible that the list of proposed actions will be extensive. In aggregate they could amount to too much change if all were to be implemented in the forthcoming Sprint. You can resolve this by getting team members to vote on action items, so that only the most important ones are taken forward. For example, if the team can take forward five items, allow each attendee to vote for five of them. The most popular can then be actioned. Other observations Here are some other things to consider when conducting a Sprint Retrospective. Decide whether or not to precede it with the Scrum “Prime Directive”. This is an assertion which is meant to be said, in earnest, before each and every Retrospective. It isn’t mentioned in the Scrum Guide, but it is widely recognized and some teams do choose to recite it. “Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand” We considered the significance of this assertion in an earlier article on Agile Teamwork in Practice, so I’m not going to say much more about it here. However, Martin Fowler has expressed his thoughts on the Prime Directive, and I suggest you read his opinion piece in full. All I’ll add is that I am in agreement with his observations and that I share his sense of revulsion. Determine what to do about Product Owner representation. According to the Scrum Primer the Product Owner may attend a Sprint Retrospective. Only “Development Team” members are actually required to be there. Yet according to the Scrum Guide, all “Scrum Team” members must attend. The Scrum Team is a wider group than the Development team and includes the Product Owner. The reason for this discrepancy probably lies in the interpretation of process ownership. If we see the Development Team as owning the process through which iterative and incremental value will be delivered to a Product Owner, then the PO would not indeed have a say in the adaptation of that process. He or she would merely be a consumer of its outputs, and would therefore be a stakeholder in a Sprint Review but not in a Sprint Retrospective. However, if we view the process as a more collaborative one, in which the Development Team works with the Product Owner to deliver potentially releasable increments of value every Sprint, then the PO would indeed be a stakeholder in how that process is managed, and must therefore attend. It’s therefore important to determine what relationship the Development Team has, or should have, with the Product Owner. It’s unquestionably best if a Product Owner is on-side as a team player, and can handle root cause analysis and the exposure of potentially uncomfortable truths. Whether or not that is the case though is only something that the team can decide. Remember they’re human. Bring snacks and drinks to keep attendees refreshed, and allow enough time for breaks – at least 10 minutes every hour. Consider wrapping up the session with a “touchy feely graph” of some sort, which captures the mood and confidence of the team. Allow everyone to mark a dot or cross on a chart to show how positive or negative they feel about things, and then see how the mood changes…hopefully for the better…from one Sprint to the next. Conclusion A Sprint Retrospective is arguably the most important event that a team can hold. It provides the means to inspect and adapt the team’s actual implementation of the Scrum framework. In this article we’ve looked at how to create an agenda for the session and how to facilitate it, and at the issues of when and where it should be held, and who should attend. Those who cannot remember the past are condemned to repeat it. - George Santayana
August 4, 2013
by $$anonymous$$
· 19,339 Views
article thumbnail
Jersey Client: Testing External Calls
Jim and I have been doing a bit of work over the last week which involved calling neo4j’s HA status URI to check whether or not an instance was a master/slave and we’ve been using jersey-client. The code looked roughly like this: class Neo4jInstance { private Client httpClient; private URI hostname; public Neo4jInstance(Client httpClient, URI hostname) { this.httpClient = httpClient; this.hostname = hostname; } public Boolean isSlave() { String slaveURI = hostname.toString() + ":7474/db/manage/server/ha/slave"; ClientResponse response = httpClient.resource(slaveURI).accept(TEXT_PLAIN).get(ClientResponse.class); return Boolean.parseBoolean(response.getEntity(String.class)); } } While writing some tests against this code we wanted to stub out the actual calls to the HA slave URI so we could simulate both conditions and a brief search suggested that mockito was the way to go. We ended up with a test that looked like this: @Test public void shouldIndicateInstanceIsSlave() { Client client = mock( Client.class ); WebResource webResource = mock( WebResource.class ); WebResource.Builder builder = mock( WebResource.Builder.class ); ClientResponse clientResponse = mock( ClientResponse.class ); when( builder.get( ClientResponse.class ) ).thenReturn( clientResponse ); when( clientResponse.getEntity( String.class ) ).thenReturn( "true" ); when( webResource.accept( anyString() ) ).thenReturn( builder ); when( client.resource( anyString() ) ).thenReturn( webResource ); Boolean isSlave = new Neo4jInstance(client, URI.create("http://localhost")).isSlave(); assertTrue(isSlave); } which is pretty gnarly but does the job. I thought there must be a better way so I continued searching and eventually came across this post on the mailing list which suggested creating a custom ClientHandler and stubbing out requests/responses there. I had a go at doing that and wrapped it with a little DSL that only covers our very specific use case: private static ClientBuilder client() { return new ClientBuilder(); } static class ClientBuilder { private String uri; private int statusCode; private String content; public ClientBuilder requestFor(String uri) { this.uri = uri; return this; } public ClientBuilder returns(int statusCode) { this.statusCode = statusCode; return this; } public Client create() { return new Client() { public ClientResponse handle(ClientRequest request) throws ClientHandlerException { if (request.getURI().toString().equals(uri)) { InBoundHeaders headers = new InBoundHeaders(); headers.put("Content-Type", asList("text/plain")); return createDummyResponse(headers); } throw new RuntimeException("No stub defined for " + request.getURI()); } }; } private ClientResponse createDummyResponse(InBoundHeaders headers) { return new ClientResponse(statusCode, headers, new ByteArrayInputStream(content.getBytes()), messageBodyWorkers()); } private MessageBodyWorkers messageBodyWorkers() { return new MessageBodyWorkers() { public Map> getReaders(MediaType mediaType) { return null; } public Map> getWriters(MediaType mediaType) { return null; } public String readersToString(Map> mediaTypeListMap) { return null; } public String writersToString(Map> mediaTypeListMap) { return null; } public MessageBodyReader getMessageBodyReader(Class tClass, Type type, Annotation[] annotations, MediaType mediaType) { return (MessageBodyReader) new StringProvider(); } public MessageBodyWriter getMessageBodyWriter(Class tClass, Type type, Annotation[] annotations, MediaType mediaType) { return null; } public List getMessageBodyWriterMediaTypes(Class tClass, Type type, Annotation[] annotations) { return null; } public MediaType getMessageBodyWriterMediaType(Class tClass, Type type, Annotation[] annotations, List mediaTypes) { return null; } }; } public ClientBuilder content(String content) { this.content = content; return this; } } If we change our test to use this code it now looks like this: @Test public void shouldIndicateInstanceIsSlave() { Client client = client().requestFor("http://localhost:7474/db/manage/server/ha/slave"). returns(200). content("true"). create(); Boolean isSlave = new Neo4jInstance(client, URI.create("http://localhost")).isSlave(); assertTrue(isSlave); } Is there a better way? In Ruby I’ve used WebMock to achieve this and Ashok pointed me towards WebStub which looks nice except I’d need to pass in the hostname + port rather than constructing that in the code.
August 1, 2013
by Mark Needham
· 10,834 Views
  • Previous
  • ...
  • 133
  • 134
  • 135
  • 136
  • 137
  • 138
  • 139
  • 140
  • 141
  • 142
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×