Why Some Projects Are Doomed from the Start
All seasoned programmers have lived through and can tell the tale of the lost project. Some of these projects were conscious decisions while others were secret skunk projects. They all suffered from the terminal affliction of "uncertainty" and its polarizing effects. Although each project varies, most follow the same path. In the early stages, knowledge is sparse, estimates are rough, and there is a large amount of uncertainty. As the project progresses, additional research, feedback, and refinement helps reduce the project's risks and uncertainty. In project management this is known as the Cone of Uncertainty. Construx has an excellent diagram illustrating the cone.
Unfortunately, many projects suffer from a lack of respect for uncertainty. Most projects perform fast, rudimentary estimating of tasks or features. This is typically a one time event and is perceived as a necessary evil as it takes away from development time. This is the wrong mindset as the cone only reduces with effort. Projects must be actively and continuously groomed to reduce uncertainty. Each additional review of the process brings with it new insights and provides further refinement. Sometimes these refinements bring to life additional tasks, reduce task load, or help to further outline a project's scope. In the development world, the phrase scope creep is synonymous with frustration and other negative emotions. This can be easily avoided with a continuous drive toward the accuracy of a project's estimates.
Programming methodologies such as Agile's Scrum provides an infrastructure for maintaining proper project focus. Features and ideas are placed in a larger product backlog, then proceed to a short iteration where work is completed. Recurring meetings are held to groom the estimates in the product backlog as new information arrives, feedback is received, or requirements change. It is encouraged to estimate as a team to further vet issues, concerns, and questions. Additionally, estimating is encouraged through numbering/sizing methods such as t-shirt sizes or the Fibonacci sequence. These methods allow for relative "person independent" estimating. This is an excellent framework for any project, Agile or otherwise.
The Cone of Uncertainty is another excellent tool in any developer's toolbox. It can be very helpful in detailing the current status of a project. The simple action of drawing the cone alone can be enough communication to project veterans. For others, it provides an opportunity to educate.