Why Software Engineering is complex?

Writing a quality software is a very complex process. It must not only meet all the functional requirements, but also should address non-functional requirements like robustness, responsiveness, maintainability, testability, scalability, security, supportability, monitorability, and disaster recoverability to satisfy not only the immediate business needs, but also flexible enough to adapt to growing and changing business needs. Anyone who has been in the software industry for a while will vouch for how quickly the business priorities can change. They also strongly understand that the following aspects need to be properly thought through from the beginning of a software project.

• Capture and document the functional and non-functional requirements. Conduct work shops and brain storming sessions to fully understand and close any gaps. It is also important to look at the big picture to ascertain if existing services or processes can be either leveraged or enhanced. In larger organizations, expertise are scattered and proper engagement and facilitation is of paramount importance.
• Choice of right development, dependency management, build, and deployment tools, frameworks, libraries, and processes.
• Hardware (CPU, memory, storage, etc), networking (latency, routing, clustering, DNS look-ups, load balancing, SNMP traps, etc), and access control requirements. For example, mission critical systems with high availability requirements like 99.99% may need to use a relevant hardware like a HP non-stop server.
• Good design, performance, transaction, memory, resource management , thread-safety, data integrity, security, exception handling, scalability, best practices, and coding standards considerations to build a robust application.
• Adequate logging, auditing, and alarms to provide better problem diagnostics and customer support.
• Data retention, fail over, and disaster recovery strategies.
• Service assurance requirements like being up 24x7, performance metrics, and adherence to SLAs (Service Level Agreements) like peak and off-peak response times, service timeouts, etc to help improve customer trust and experience in the system.
• Taking the project through the full software development life cycle (SDLC) process by adopting waterfall and agile development practices, and most importantly liaising with multidisciplinary teams to get things done with proper documentation and hand over procedures.

It is also important to understand the alternatives, pros/cons of each alternative, potential risks, likelyhood of those risks and their impacts. A number of tradeoffs need to be considered between design decisions, time to market, strategic versus tactical solution, weighing up amount of effort expended in security against significance of damage in terms dollars and reputation, etc.

So, it really pays to have the right people in your team. Not just techies or certified professionals, but well balanced professionals with good technical skills, great soft skills, and the right attitude to manage complexities and get things done. It is also imperative to have a well balanced team with varying strenths to complement each other. You need people who have the technical know-how in the following 16 key-areas

1. Language Fundamentals
2. Specification Fundamentals
3. Platform Fundamentals
4. Design Concepts
5. Design Patterns
6. Concurrency Management
7. Performance Considerations
8. Memory/Resource Management
9. Transaction Management
10. Scalability
11. Exception Handling
12. Best Practices
13. Software Development Processes
14. Security
15. Coding
16. Quality of Service

I will elaborate on these key areas in my next post.

Java/JEE Career Companions