Being in the software development industry, I worked on multiple projects and on the way met people with diverse mindsets. I interacted with people from the same industry but from different sectors like marketing, operations, R&D, development, and people from other industries mainly as clients, who had unique points view on software development projects. In the early years of my career, I used to go to the client places to study their requirements, accompanied by the senior members of my team. The initial discussions were around topics like the scope of work, features of the system to be built, preferable technologies, an approximate estimation of the work, and required span of time to complete the project.
I remember a specific situation where a similar conversation was going on with a couple of representatives from the client company. The topic was on estimation and we proposed a rough estimation for the work as 30 man months (well I can't recollect the actual figure right now). One of the associates from the client's side said, “When you say 30 man months, does that mean 30 men can finish the project within 1 month and vice versa?”
From his point of view, it’s quite a rational question, to us it was utterly unexpected. In an adverse situation like this, one of our senior members attempted to explain to him a bit about the complexity and dependencies involved in the execution of a project. I roughly remember the project initially estimated around six months which got extended to two years; an illustration of charismatic management.
Why Can’t We?
A couple of months back, I saw in the news that some people built a playground in a day in the city of Sanford, Florida. If the chap who asked the question to us years ago had read this news before participating in the conversation, I am sure he would’ve had an, even more, stalwart argument. Building a playground includes months of preparation, heavy equipment, and, most importantly, manpower, but somehow they did it.
Can we do this for software projects? So that someday I could go back and tell the chap that, yes, we can, we can use the following equation:
30 man-months = 30 men X 1 month or 1 man X 30 months or even roughly 600 men X 1 day (assuming 20 working days a month).
The debate becomes absolutely baseless if we consider this figure as only the manpower effort. Should we revisit our estimation standards, methods, and techniques once more? Rather, we can provide an estimate including another factor. Apart from manpower, it engages lots of other resources and depends on multiple factors at various phases of a project, like:
- Stakeholders’ interest in the scope, cost, motivation, and foreseeing the future of the product.
- Availability of required infrastructure to execute a project.
- Availability of appropriate skills and experiences.
- The rate of attrition of the human resources.
- A Proper strategy to mitigate the risks caused by controlled or uncontrolled conditions.
- Effective project management plan and monitoring.
- Effective communication to come to a consensus on the requirement.
- Proper analysis of the requirement to map elements from the problem to the solution domain.
- Time spent on research to find technical solutions to unpredicted problems during execution.
- Suitable design of the proposed system.
- Time spent on testing and rework.
- Time spent on documentation and user training.
- Constraints in deployment.
Pros and Cons
A lack of any of the above factors can lead to a delay in the route to delivery. So far, we are far away from the miraculous day when we can have all the factors we need at our fingertips while executing a project. Certain factors stand to be uncontrolled, for example, the clients’ financial interests, the rate of attrition, and sometimes the unexpected technical hazards of the infrastructure, especially when you are working across different geographical locations. The rest of the factors are more or less controllable.
From my past experiences, I noticed most projects introduce delays while deciding on the scope and cost, from the stakeholders’ perspective. Once we are in a hurry we end up rushing into the next phase, missing out on essential information, and forgetting about the big picture, which might turn out to be a big obstacle in the testing phase. A good place to lose out on time.
Experience and Skill
The human resource and skill set problem does not interfere so much when everyone is able to participate in any of the phases of a project, but that happens in very few cases. Work distribution is done on the basis of experience levels and skill types. You can have a more aggressive project schedule based on how efficient your team is, and that’s been a universal truth since the early ages of software development. With the advent of new technologies, frameworks, and, of course, a wide variety of resources available on the internet, the requirement of the skill sets are not so bad as they used to be, but in that case, we need time to learn new things. The rate of attrition is one of the natural calamities hindering the execution of a project, particularly when you are lacking a proper backup plan. The risk mitigation plan should take care of that. Documentation, well that’s, again, an activity that could take up a fair bit of time, though it will help in terms of knowledge transfer to new members.
Project Management and Monitoring
Project management plans should address these dependencies and mark out a critical path to carry out the project. To allocate resources, the plan needs to take care of team members and their skills. This is a critical activity since all the team members do not possess identical skills and experiences. Agile methodologies will surely help in closely monitoring and recovering from misdirection fast. The advantage of a member acting in versatile roles surely keeps Agile processes one step ahead. This is a vital strength of small companies with a small set of resources. Sometimes team members working on a small part of the product can get bogged down by its requirements, and lose sight of the bigger picture. This can restrict the future oriented thinking for a team of developers. Members of a team following Agile sometimes end up working in silos.
So, should we try mixing and matching our old methodologies (e.g. Waterfall, Spiral, Iterative, RUP, etc.) and the Agile ones (e.g. XP, Scrum, Crystal, etc.)?
Study and Analysis
Proper communication between team members during the requirement gathering and analysis phases could save a considerable amount of rework during the testing and implementation phase. A little bit of experience, psychology, and technical expertise helps the players in this phase to get to the heart of the problem. After all, it’s a human to human communication; thank God we don’t have to deal with psychology with a computer. An expressive analysis document saves a lot of time in design.
The construction phase is usually constituted of architecture planning, design, development, and testing. It involves researching new technologies, designing and constructing components, and then testing these components. With the coming storm of ideas, products, open-source tools, and buzzwords it’s hard to select the appropriate technology these days. A project should consider the time it takes to research and decide on what technology to use, and try to minimize this process as much as possible. The proper technical guidance of an architect toward a structured approach of development can speed up the development process. The selection of the appropriate technology, a birds-eye view of the design, recognition of patterns and a proper introduction of the same, the reusability of pieces, and adherence to standards during development can be well controlled by an architect, so that you don’t get face-to-face with refactoring or re-designing quite so often. Another important thing to figure out is the expectations on non-functional requirements of the system and make out a plan to address those. People without prior experiences with automation are often reluctant about discussing the non-functional requirements at the initial phase of the project.
Designers are, of course, confined within the scope sketched out by the analysis, but if they try, they can still foresee the feasible enhancements of a product and design it to be flexible enough to accommodate necessary changes in the future. You never know, some of the enhancements you need could be right under your nose during the testing and deployment phases. They can make it expressive enough to ease the implementation for developers, which minimizes time.
Testing and rework is the most important phase where the product actually gets weighed against the requirement and that’s where the time to completion gets pushed out more and more. This is a normal phenomenon in most projects and the target is to eliminate the rework caused by a misinterpretation of the initial requirements. A preparation of test plans parallel to the requirement analysis often helps.
Sometimes a gap is noticed in development and deployment environments (where your product is installed) causing a delay due to rework or configuration in the deployment phase. The preparation of a concrete deployment plan during the architecture or design phase helps in trimming down the time this takes. Especially among geographically diverse teams, on heterogeneous platforms, or a scenario where multiple pre-existing systems are involved.
Tangible and Intangible Hurdles
We understand that there are multiple of reasons why the time it takes to deliver on a project cannot be trimmed down as desired. But, we do have control over the tangible elements involved in the process of execution, for example:
- Setting up infrastructure.
- The use of tools in various phases of design, development, testing, and deployment.
But intangible factors are not entirely under our control. These factors include:
- Stakeholders’ interest.
- The skill of other team members.
- The rate of attrition.
- The time needed to map the requirements into features of the proposed system.
- The time devoted to research for finding a suitable software/technology to help solve a given problem.
- The time to think of and create a proper design which draws on the team's experiences.
- The time it takes to understand how to use a new technology or open-source product so that it can be applied during the development phase.
Do we have to evolve with an agile method of thinking? Can we minimize the time it takes for the user's thoughts on a product to reach the analyst and from there get distributed to the architects, the designers, the developers, and so on?
Honey, I Shrunk The Kids
The Agile method of thinking is not yet in the picture, but, being in the software industry, we are quite familiar with abstraction. From machine languages (first generation) to assembly languages (second generation), from assembly languages to high-level programming languages (third generation, e.g. C/C++, Java, PL/I, etc., and then to English-like natural languages like SQL (fourth generation) and finally to the graphical interfaces (fifth generation) like integrated development interfaces (IDEs) for planning, design, development and most of other areas like requirement capturing and testing. These tools intake the thoughts from the human behind the keyboard and translate them to other languages like C/C++, SQL, Java, etc. The pains of programming in machine or assembly languages have been removed by third generation languages. IDEs made the writing of C/C++/Java code quite a bit easier. We are even thinking about Domain Specific Languages (DSL) to keep the movement toward more readable languages going forward. I look at SQL being a DSL for database operations. The history of software development shows us an approach toward reducing the time for faster and easier development processes, while at the same time standardizing solutions.
So now we have our helper friends, Abstraction and Code Generation. Though the use of the term “code generation” has picked up during the last couple of years with the conceptualization of Model Driven Architecture (MDA), the basic concept was there from the beginning when we learned to design compilers. The only thing is that we are thinking in a step-by-step manner to reduce the gap between the user’s requirements and the end product, thus shrinking development time. Open sources are playing the lead role in the reduction of cost and speeding up the process by creating reusable, standard solutions.
And now, we, as an industry, are even thinking of ways to implement automation throughout the entire process, leading to a transformation of thoughts at different phases of development and researching better ways to do business process management (BPM) and modeling. BPM enables us to grasp the thoughts right from the minds of the user and turn this into code (well not exactly, we still have to have an architect, a designer, a developer, and a tester). I don’t understand why we are trying to address the domain and the business processes separately, from different angles of BPM and DSL. If you are given a graphic interface to build, you would process model and generate code out of it, but would you take the time to write the same thing using a DSL? A good use of DSL could be defining the bits of the business process after you have modeled and generated it using a BPM tool. A BPM tool is, of course, a good tool to address the issue of integrating your new system with old ones, irrespective of the platform on which they are built. Reusing the existing systems is now far easier, with the advent of Service Oriented Architecture (SOA).
We can certainly imagine the day when one can generate the entire solution in a single day sitting along at his/her desk. I will have to repent on that day for not keeping the contact number of the chap who asked the question years ago.