Environment Strategy for ESB Projects

During the planning phase of a greenfield integration project, while setting up the groundwork for the project there are a number of challenges and questions a tech lead/software architect faces, some of which relate to how the project will fit into the organization's structure and align with enterprise IT. The way to tackle these questions is not straightforward and there is no golden rule of software processes defined to handle them. Even the modern SDLC methodologies such as Agile, Scrum or Kanban do not define any concrete steps to tackle this issue. We might get some help from the good old ITIL framework, though. The key here in the consulting world is the science of analysis and agreements. As long as the current state of the IT architecture, IT processes, approval chains, and the do's and don'ts of an organization are known and properly analyzed there is absolute certainty that your IT project will be approved/succeed. 

In this article, we are not going to move the mountain as a whole but handle two of the main problem sets an implementing partner faces and some of the standard ways to tackle them in an integration world.

Environment Strategy

There is no holy grail of environment strategies for an integration project – most of the time implementing partners go with whatever standards the current organization follows so as to align with IT systems and processes and not get stuck in the approval process for starting something from scratch. This approach works most of the time, especially when you have a standard software implementation such as a CRM System, ERP, or a web portal setup, which are basically the "end system" or the "standalone" software in the integration world. These might not always be feasible for an integration project, as integrations tend to work a little differently than a traditional "end system." There are a few key points to keep in mind, however:

• Global Consistency – Ensure the environments you chose are globally consistent with the organization's defined standards. These may include keeping a separate environment for load and performance testing; a dedicated environment for disaster recovery; etc. 
• Deployment Model – Whether you deploy your application to SaaS or on-prem plays an important role in which environment strategy you should pick. Environments that host cloud-based integration tools, such as MuleSoft’s CloudHub, tend to be more fluid since there is just one demarcation for prod vs. non-prod cores. The same goes with Dell Boomi, where you can create ‘fluid’ environments and classify them as production and non-production. 

For on-prem deployments to physical servers organizational standards are usually followed, but at the same time project needs should not be forgotten. 

• Alignment to End Systems – This doesn’t mean an ESB has to be tightly coupled with the environments of the end system, but a degree of alignment should be there. This is done to support easy integration into the system and throughout the software development lifecycle. 

Examples of Environments

• Development (Dev, Sandbox, DevBox):
  • Environments are created either in a SaaS setup or on-prem setup to provide an environment for developers to develop and unit test their code and its stability in a controlled setup. Dev environments are usually not strongly controlled and have limited resource capacity for deployments. There is no requirement for high availability or reliability. This may or may not be connected with CI/CD processes based on the type of deployment used. Some SaaS tools, such as MuleSoft, provide a shared Development environment, whereas some offer a dedicated Dev environment per developer.
• Testing (SIT, UAT, L&P, Pre-Prod):
  • Environments in a B2B setup are usually the ones which are used after the development process is complete and ready to be tested. These are controlled environments with limited access to a subset of users. They may or may not be as performant and reliable as production depending on the requirements. For example, an SIT (System Integration Testing) and UAT (User Acceptance Testing) environment is used by a subset of the testing team and is not as performant as Production. Whereas an L&P (Load and Performance) is usually a replica of the production setup and is as performant and reliable as a production environment. This environment is always enabled in the CI/CD processes.
• Production (Prod/Live, DR):
  • As the name suggests, this environment is the live environment. This environment is kept highly reliable and highly available by means of load balancers, clusters, and disaster recovery servers to provide nearly 100% uptime.

Some Tips for an Effective Environment Strategy

• Always bear in mind the cost of maintenance, hardware, subscription fees, etc., while planning to create new environments. My suggestion would be to keep the number of environments you run as low as possible and focus on the DevOps process more. 
• Maintain an environment mapping sheet. Create a sheet which aligns different environments from different systems and shows how they align with the other system’s environment. 
• Keep production and higher testing environments tightly in control and limit access to only the ‘required users.’ 
• Do load and performance testing from time-to-time to ensure the current infrastructure aligns with the organizational load.
• Keep the disaster recovery environment ready and in-sync with production so that the switchover is smooth in case of a disaster.