Whether an IT project is a development, integration, or transformation, Technology Selection has always been a critical activity. It is critical because the technologies selected are used during the entire lifecycle of the project and remain as the main drivers for successful completion of the project with the desired Quality of Service (QoS) of the product. This activity requires an intense study of the business, functional and non-functional requirements, expected QoS, user requirements, and the existing system, if any.
Usually this process falls in the area of the technical lead or architect. The selection of the right kind of software, technologies, frameworks, and tools becomes a boon to a project, provided you have right hands to operate these instruments efficiently. However the main objective is to deliver high quality (QoS) software with desired outcomes, which can never be compromised during this process.
There are well-known project management issues that screw up projects, which are the subjects of project management experts. However, technology selection is a subject just for techies. Gil Edelman provided a nice and brief overview of technology selection process in his article “How to Choose Your Tech Stack” (http://svsg.co/how-to-choose-your-tech-stack/). Primary factors that drive the technology selection process are:
Scope of project
Functional and non-functional requirements
Budget for licenses
Availability of skills and experience
Strategic vision and management buy-in
There are very few technical pros and cons that really matter. For example, PHP, C#, Python, and Java — all of them work perfectly. The choice is usually driven by budget, strategic vision, political reasons, or available experience more than anything else.
Quality standards, e.g. ISO or CMMi, ensure that things are done properly e.g. design, requirements, budget, testing, attitude, etc. These standards can ensure whether the process of software manufacturing, testing, and delivery are done properly or not. The quality parameters of the actual software system (e.g. Quality of Service) and technology are not the primary focus for them, while Quality of Service is the primary focus area for the technology selection process.
However, there are situations where the wrong technology can be disastrous. Having all cutting edge technologies together doesn’t always help projects fly, especially when the technology is under research and experimentation.
1. Poor Statement of Work
When no one is sure about what needs to be accomplished, things start to get fragile from the beginning. One of the key ways to screw up a project is to avoid creating a roadmap, definition of project requirements, and expectations of all stakeholders in a documented format at the beginning of the project and get approval from all stakeholders. This is a common approach, as mentioned by Jennifer Lonoff Schiff in her article “15 Ways to Screw Up an IT Project” (http://www.cio.com/article/2384088/project-management/15-ways-to-screw-up-an-it-project.html).
2. Lack of Clarity On Requirements
When the functional requirements of a system under development are not understood, we can’t decide whether a web server or portal server (for example) is required. This is a typical decision point on whether an application is going to be offered over the web or not. There are several basic questions and checklists which may drive us to different kinds of software types e.g. infrastructure services, network services, messaging services, security services, etc.
An application connecting with external systems raises more questions on which transport techniques, protocols, and security measures to be used, which might drive us toward an ESB or set of connectors or adapters.
If the business logic is expected to be configurable such as a rule, a business rule engine is helpful.
Certain questions usually get their answers from the Application Architecture of a system. TOGAF has rightly depicted the dependency of Technology Architecture on Application Architecture (http://pubs.opengroup.org/architecture/togaf9-doc/arch/chap12.html). Without diving deep into the functional requirements, arriving at decisions in these areas is next to impossible.
Apart from functional requirements, identification of Quality of Services (QoS) and non-functional requirements play a critical role in technology selection.
The top priorities of systems like Google, Twitter, and Facebook have been speed, parallel processing, security, and big data processing. If we have similar requirements, technologies like big data and faster real time processing can be considered, otherwise it is an overkill of effort, such as putting a Jet Engine in a bicycle.
Finally the end user requirements mean a lot. I have seen systems with full functionalities being rejected by users due to the lack of user friendliness. A study of user communities, their requirements, ease of use, and comfort helps in deciding whether to go for desktop, tablet, mobile, or other devices. Sometimes experienced users with mainframe backgrounds might not like the idea of playing with their mouse on a beautiful web responsive GUIs. Popular user interfaces can be a way to screw up the success here. Managing comfort and expectation is the key to success.
3. Too Many Stakeholders
Projects with multiple stakeholders (including customers, management, end-users, and technical architects or leads) really need the best of luck to survive. In this situation, applying technical governance is a big challenge. Starting projects of this type without a technical governance framework is the surest way to screw up.
The presence of multiple stakeholders usually requires the technology stack to be decided and reviewed by many.
Stakeholders with their various backgrounds and experiences possess different perspectives and try to help out during technology selection by contributing their thoughts. Managing and convincing them can obviously save the project from being doomed. This is a sensitive task which should be handled with care by considering the sentiments, interests, importance, and political angles. TOGAF nicely describes Stakeholders Management as one of the key areas (http://pubs.opengroup.org/architecture/togaf9-doc/arch/chap24.html).
4. Personal Aspirations and Technology Madness
Developers always enjoy learning new technologies. Most of us are or were developers. Using young developers’ perspectives during technology selection is another way to screw up the project. Learning new technologies for fun and applying them in real project deliveries each have different risks involved. Not all cutting edge technologies are required to achieve the desired outcome from the system under construction. Sometimes they might appear in your nightmares when you are suffering from lack of support and newly reported bugs.
In 2010, Twitter’s search engine was re-developed using Apache Lucene by replacing MySQL. Again in 2011, they shifted from Ruby-on-Rails to JVM for better performance and scalability. So by analyzing the pros and cons, finding out the availability of support is mandatory activities before jumping for any new technology. After all, the customer will not wait till you find a new solution or fix a bug in your tech stack. Therefore, technology madness is a strong driver behind immature decisions.
5. Poor Communication
Mr. Architect has defined all technology layers with the latest technology components, held meetings, and prepared slide decks to share his thoughts. But designers and developers are not sure how to use them in code.
Well I might love to use Spring Framework annotations for dependency injection (DI) to avoid the pain of defining the beans in appContext.xml, but another developer might not have that knowledge. Hence, we end up with some beans defined in xml while some are auto wired with annotations in the code. Eventually the team which gets to manage the code later will be in a soup.
Similar situations can arise while using a message oriented middleware (MoM). I am using my SOAP objects over HTTP transport, while my friend likes JSON over HTTP. Therefore, defining a descriptive document with all areas covered is the primary task of an architect. Apart from that, the architect should hang around with the developers to ensure the architecture standards, patterns, technology best practices are being followed and enforced properly.
6. Lack of Skills and Experience
A very crucial point: lack of skills and experience to define and use technologies can work as a knife through butter when it comes to screwing up a project. This applies not only to developers, but also to architects. An architect without knowledge of the latest technologies and their impact on the target system is really helpless in the technology selection process. Upgrading skills and knowledge can be a savior in situations like this.
The architect must provide enough rationale and logically proof the suitability, capability, and applicability of the selected stack with respect to the requirements of the system under construction. She should also list out all architectural goals and constraints (aligning with the technical vision and strategy) and show how the selected stack is supporting the goals and constraints.
Lack of developers with required skills is the most critical issue that can easily screw up a project. Proper screening, recruitment of technical members, and training them can help to a large extent, however to touch the stars, experienced professionals are your only instruments. This is where the technology stars and heroes are born to save the project.
7. Compromising On Quality of Service
Quality of Service (QoS) is a significant factor which drives the technology architecture of a system, especially at the age of component or service based architecture. The parameters of QoS e.g. performance, availability, scalability, security, serviceability, reliability, maintainability, evolvability, survivability, adaptability, reusability, etc. are considered as the measure of a highly efficient and robust system.
Decisions to apply architectural styles such as SOA or Microservices can help in reusability, serviceability, and adaptability.
Cassandra database is a fail-safe, highly scalable, specialized in fast insertions, however query executions with joins (such as in SQL) are not sweet spots for Cassandra. For new queries, you might need to copy data to a matching data structure on a SQL database. Shifting the design paradigm from a relational background to the world of name/value or column families is not an easy job.
To avoid bumps ahead, sometimes the QoS of the system is compromised with the technologies or tools in our comfort zone. The usage of legacy technologies sometimes restricts the system to leverage improved features from the cutting edge technologies and speedy development lifecycle.
On the other hand, using all the latest technologies without experienced developers and lack of support will surely screw up the maintainability of a project.
8. Technology Life Cycle
The architect can easily ensure the birth of a system with a bunch of outdated technologies by selecting the tech stack with technologies which are out of life (such as legacy ones from history which can only find a few old hands, e.g. Visual Basic, Power Builder, COBOL, Sybase, etc.).
On the other hand, selecting the alpha, beta, early release, or latest versions of software tools can put the system in great risk due to lack of support and unknown bugs.
Using a previous version of software rather than the latest can be a safer option. You can check out the defect lists from the release notes before doing so.
9. Lack of Compatibility With Strategic Vision
Business or enterprise strategies are usually established as a result of political influences, business vision, governance policies, and achieve specific business targets such as profitability, cost cutting, promotions, etc. Technology strategies are driven by enterprise strategies. In fact, most of the software development, transformation, integration, and implementation projects are kicked off as per business strategies and plans. Most of the large scale transformation, migration, rationalization, and modernization projects are aids to cutting costs on maintenance.
One will surely not migrate to an IPv4 network when the strategic vision demands for a rapid growth preferably to an IPv6 network.
Selection of the right kind of technology stacks is a sensitive issue. It requires the study of the entire enterprise landscape, business processes, features, interfaces, and standards to be able to define the target technology stack so that redundant applications are decommissioned or consolidated to the strategic stack. Poor decision-making on tech stacks takes its toll on these projects.
10. Budget and Cost
The budget and cost of software licenses and cost of human resources are two major factors that influence the technology selection process. For example, Open Source software is a popular instrument for cost cutting however maintaining them can be a pain due to lack of managed services. We need to arrive at an optimum balanced figure between cost and maintainability here.
Finally, it is suggested considering the Technology Selection process with required care by realizing its importance and effectiveness at the beginning of IT projects. It should be initiated by identifying significant factors, drivers and stakeholders followed by a proper study and evaluation of technologies. Preferably performing proof-of-concepts of technologies (once decided) can give us better peace of mind.