All About Vertical IT Teams: Part II

In Part I, we talked about aligning vertical IT teams with core business services. Let's talk more about what vertical IT teams are and how they differ from DevOps teams.

What Is a Vertical Information Technology Team?

Let's discuss how they're organized. Organizing for Big Data calls for cohesive, ring-fenced (dedicated), and even co-located vertical teams that have most of the core IT functions, working together to realize new business requirements in an agile fashion. It starts by having a team that in this case comprises business analysts, data engineers, developers, QA engineers, project managers, product managers, and technical operations professionals.

The teams are orders of magnitude smaller than enterprise-wide organizations since they are focused on a specific area of business core services. Furthermore, requirements for faster time to market that are dictated by the competition in Big Data and Fast Data and IOT necessitate more creative ways of organizing IT teams. For Big Data and Fast Data, it makes more sense to have the vertical IT organizations focus on specific current and future business use cases that are closely tied to the relevant business core services. Such teams are in charge of their own Big Data and Fast Data cluster(s) and related applications.

The vertical team will have its own business operations sub-team that works closely with business and understands the data. The business operations team is composed of both data analysts and business analysts, championing business goals and collaborating with development and QA. This internal team collaboration takes place in an agile fashion to move business ideas into acceptance and integration environments in a span of days. The technical operations sub-team, consisting of Big Data and Fast Data specialists (i.e., Hadoop, Kafka, Spark engineers), deal with the complexities of implementation and maintenance in a dedicated fashion that will ensure quick turnaround.

Technical operations will also handle production support levels one to three in close collaboration with the development team. The vertical IT team also has its own technical Project Management subteam who, in addition to their agile PM functions, owns IT budget management, supplier management, and release engineering. A close collaboration between release engineering and technical operations facilitates a quick production implementation and even production testing. The whole model could also take advantage of ITIL best practices and arrive at a viable Big Data and Fast Data service model for the business.

As discussed above, the IT Vertical organization consists of various teams who collaborate on developing business application software services (or products). Figures 1 and 2 show how a vertical IT team is organized and how it collaborates with enterprise-wide centers of excellence.

Here's a list of various teams and roles of a vertical IT organization in a software business application environment.

Business Operations Team

• Business Operations Manager

• Senior Business Analyst – Team Lead

• Business Analyst

• Data Analyst

• Data Engineer

Development Team

• Development Manager

• Senior Developer – Team Lead

• Developer

QA Team

• QA Manager

• Senior QA Engineer – Team Lead

• QA Engineer

Technical Operations and Production Support Team

• Technical Operations Manager

• Senior System Engineer – Team Lead

• System Engineer

• Senior Production Support Engineer – Team Lead

• Production Support Engineer

• Hadoop Cluster and Big Data Infrastructure Engineer

Project Management Team

• Manager of Project Management

• Senior Project Manager – Team Lead

• Project Manager

• Technical Project Manager

• Release Manager

Up to this point, we have discussed the concept of vertical IT teams from the perspective of a software business application development environment where a software service or product is developed in-house intended for internal customers. The same concept could also be applied to a product development environment where software service or product is developed to be sold to external customers.

Even though these two settings share the majority of the roles and responsibilities, there are slight differences. The main difference is the role of Product Manager, who is responsible for the product roadmap. This responsibility includes working with business organizations such as sales and marketing to arrive at business requirements as defined by the needs of the market and external customers. Product management then works with the development and project management to arrive at delivery schedules of new versions of the product.

As you can see, there is a clear overlap between this role and Business Operations Manager and it makes sense that in product development environments we replace the role of Business Operations manager by Product Manager. Naturally, the user acceptance testing function is replaced by Beta testing by a select list of customers and so on.

What do we mean when we say the vertical IT team is ring-fenced? Well, it is absolutely essential for the vertical IT team to be dedicated on purpose to serving the goals of the business organization.

That is what ring-fenced means. The enterprise IT model of having ‘generic’ or cross-functional business analysts, developers, QA engineers, technical operations and even PMs does not work in this case. What distinguishes these IT professionals from ‘generic’ IT resources is that they become Subject Matter Experts (SMEs) of the business services to which they are dedicated. This is where executive buy-in becomes essential, so that a change in management will not cause a break-up of the team or reallocation of its resources to other teams. The ‘generic’ model (sometimes called factory model) works for commodity services discussed earlier, but will only cause delays and lack of quality when IT professionals do not have the necessary in-depth understanding of the business services and business requirements associated with the on-going projects.

How Is This Approach Distinct From DevOps?

This approach could be called an extension of DevOps and is not limited to organizations that use Agile methodologies. One challenge of DevOps is that IT organizations try to adopt it while ignoring the relationship to business and core services of the organization. While that is a subject for another article, there are effective ways of tying other methodologies such as SDLC to vertical IT teams. Also, DevOps does not put any limitations on the size of the team, while the personal experience of the authors has shown that the overall size of the IT vertical team should not exceed 30 (the optimal size would be closer to 20).

Last but not least, DevOps does not bring technical project management into the equation, including budget management and gaining the support of executive leadership. This is crucial in creatively traversing the swim lanes of the corporate world; without funding and executive buy-in, there is no future of vertical IT teams, regardless of their success in delivering results.

An Expanded Role for Software Quality Assurance

The expanded role of SQA is another key aspect of the vertical IT team. What does Software Quality Assurance mean in the high-velocity world of Big Data, Fast Data, and IOT? It means moving SQA beyond traditional concepts. Most IT organizations only think in terms of functional, system, integration, and regression testing when it comes to SQA. In the vertical IT team model, SQA is not only in charge of all aspects of testing, it also takes over data quality, data completeness, performance, compliance, and security. The SQA team works closely with business and data analysts, developers, and technical operations to make sure that functional, data quality, performance, compliance, and security requirements are defined early as business requirements.

This requires a major culture change and turns the SQA team into the quality guardian of all aspects of software: embracing data quality, performance, and security beyond functional requirements. Concepts like the security of data-at-rest, data-in-motion, and data-in-use, plus customer and user privacy, become an integral part of SQA work. As stated before, it is important to emphasize that the SQA team should be ring-fenced and a QA factory model will not work. Our experience has specifically shown that ideas like enterprise shared QA services will reduce quality and agility to such an extent that such resources become a quality impeding factor rather than the opposite. We have observed that such QA resources not only will be limited to testing but they also end up relying on developers to define their test cases. This goes against the basic best practice of keeping SQA independent, so they can think “outside the box” of the developers.

A New Role for Operations and Production Support

Another key benefit of using vertical IT teams is the expanded role of operations and production support. Again and again, and contrary to many beliefs that sharing enterprise IT operational services will save costs, in the case of core business services we have seen in practice that dedicating technical operations and production support to specific core business services actually saves more money and increases the quality of the service at the same time. We have devised a whole new approach called Operational User Acceptance testing (Operational UAT) that allows for a higher level of agility and quality of production releases.

Dedicated (ring-fenced) resources allow the technical operations team to be involved in the software development lifecycle from the start and raise the importance of the operational requirements to the level of business requirements. Our experience shows that operations teams should be treated at the same level of importance as business users because both are actual users of the application. Engagement of operations teams and their alertness is key to the success of the business application in production. If their requirements for monitoring and supporting the application are ignored, then supportability and consequently stability of the application in production will be jeopardized. Operational requirements become the basis of Operational UAT.

Another key to the success of Operational UAT is having an acceptance (and if possible, integration) environment that resembles the production environment. The advent of the Cloud and dynamic environment management makes it possible to create such environments at a fraction of the cost of building them in-house. Operational UAT is a much more effective approach compared to using “production readiness checklists,” which are usually done only a few days before the production date and are non-verifiable, compared to Operational UAT planning and execution based on initial operational requirements.

Vertical Teams and Technology Trends

Team cohesion and collaborative nature of the vertical teams at both internal and external levels result in significant flexibility to adopt new technology trends to increase productivity. It is important to note that we are not saying that enterprise-wide teams are unable to take advantage of these trends. We are saying though that Vertical IT teams are better positioned to fully utilize these trends as a result of their business focus and nimble size and structure. Here are a few technology trends that could be fully or partially utilized by vertical teams.

Microservices

"Do one thing and do it well" is a common principle of microservices. Vertical teams are best positioned to take full advantage of Microservices. Internally, they could be in charge of multiple core services that have a well-defined common API to talk to each other. Externally, they could have a variety of perimeter services that use the principle of common API to talk to external Microservices or other external applications for that matter. This is yet another advantage that smaller size and flexibility bring to Vertical IT Teams.

Continuous Integration

Since vertical teams control their own IT environments, Development, QA, and Technical Operations could work together closely to utilize Continuous Integration. The adoption of CI could vary from the protracted case where automatic build and delivery to the development and integration and QA environments takes place after every check-in. Automated integration testing that is executed after every build is a definite requirement for CI. Of course, there are many ways to implement CI. Although Agile development is not a prerequisite to CI, it is hard to imagine an Agile development team that doesn’t take advantage of CI.

Agile Cross-Functional Teams

Vertical teams are also best positioned to take advantage of Agile Development. Team cohesion and collaboration facilitate cross-functional Agile teams of Business Analysts, Developers, QA Engineers, and Technical Operations Engineers to work closely together in an Agile cross-functional team.

Continuous Delivery

Both CI and Agile frameworks along with Dev/Ops could also facilitate Continuous Delivery of functionality either to integration or production environment. Functionality is delivered in small releases in one to four-week time frames to users. Vertical teams are again best suited for utilizing this framework since they are dedicated to specific business teams and business goals thereby controlling the delivery of new functionality to their user community.

A/B User Testing

A/B testing of UI(sometimes called Split Testing), where a certain percentage of the users are seeing A version and rest see the B version, is a mechanism used by many web-based applications to expose users to a new version in a piecemeal fashion. This testing is also widely used for Mobile apps. This could help for a gradual transition and better risk management.

Conclusion

While there is a lot more to be said about this topic, it is important to note that organizing IT around business goals via vertical teams is a transformative change that requires a strong commitment by both business and IT executive leadership. It is a multi-year commitment, and in many cases will also include cultural change. At the same time, if we agree that what is happening in the areas of Big Data and Fast Data, IOT and customer digital experience (that come with significant security and privacy concerns) is a revolutionary wave, then a transformative approach is needed to ride it. While this approach is no panacea, it is a strong step in the right direction. Such an undertaking will only be successful when it is handled in a continuum of calculated risk taking and learning from our mistakes.