What's the Biggest Challenge When Implementing Microservices Architecture?

To gather insights on the state of microservices today, we spoke with 19 executives who are familiar with the current state of microservices architecture. We asked them, "What are the most common issues you see affecting the implementation of microservices?" Here's what they told us:

Change

• Without a doubt, microservices are a radical departure from the status quo. Our customers frequently need new employees, new process models, and new hosting infrastructure to get the most out of a new microservice architecture. As a result, our job is frequently to show them what life is like on the other side of such a transition and to build a compelling, cost-effective roadmap to reach it. Ironically, while the bulk of the visible costs to microservice implementation is technological, the invisible price we pay in culture change and user adoption is much more significant. Because the change represents such an investment of work and confidence in a new way of running software operations, we focus on educating our customers on the options that best fit their situation, otherwise, the costs and benefits will remain too vague to continue. 
• The stated benefits come at a cost of having to build out the supporting architecture, infrastructure, and processes. This means the upfront investment in both time and money is higher before you begin to see the benefits. 
• Microservices are yet another operational and developmental paradigm shift. These shifts always present challenges to implementation. The architectural maturity of an organization is often the greatest hindrance to adoption and implementation. If an organization is not in a place to facilitate the exposure of microservices, for example, due to legacy systems not supporting open messaging, it will hinder implementation.

Legacy Culture

• The shift from big “A” apps and the old way of doing things. The end user wants their information fast, quick, and easy. The consumer internet is enabling consumers to consume data feeds with push notifications of what you’re interested in. Employees want the same ease of access to information.
• Organizationally, development teams take time to take ownership of the production of services. Coordination of different microservices and their connections. This improves over time with a steady state. Microservices do not offer a “free pass” in terms of overbuilding. Do not overbuild to accommodate every use case. 
• The most common issue we see typically involves organizations integrating microservices with their legacy back-end systems. Many organizations erroneously focus more on the back end than the microservice itself, where this is a very backwards approach when building microservices as part of an organization’s API. Organizations must focus on making the API as developer friendly as possible, especially when the developer is the customer and your business depends on it. In an API economy, many organizations will live and die where the quality of their APIs is a significant factor. 
• DevOps trinity of challenges to overcome: 1) people/culture; 2) process/practices; and, 3) tools/technology. Not just a new architectural paradigm. Understand the promise – it’s not a silver bullet. Misdirected adoption – focus on the business value being provided. Eighty percent of apps are still monolithic. Able to restructure the team to benefit.

Other

• Developers throw everything they think they'll need into their container manifest. It makes them bloated and can pose security threats to put stuff in that you don't need. Often this is because they used a bad boilerplate or just simply didn't know what they were doing. I've seen so many hundreds of containers using a customized version of the Ubuntu base container that it's nauseating. Use standardized and official containers from the registry as often as you can.
• We don’t trust synchronous HTTP operations among services and we use queues everywhere so that we can stop and start a microservice whenever we need to. To get an easier development, you have to face initial configuration and driver setup costs to connect your service to different protocols. Queues come with a cost. I see some companies are implementing a Configuration microservice with the only purpose of sharing configuration strings across microservices: this is a good example of which kind of extra-effort you find when you move to microservices, as your configuration needs to be properly distributed too.
• 1) Information – distributed environment is still hard to debug and troubleshoot with latency and disconnections. Need to learn the fallacies. 2) Data management – monolithic apps typically came from one database. When the data is distributed across many small components, how do you maintain data concurrency across two or more services?
• Application configuration around fault tolerance, fall back, health checks, and metrics.
• API conventions are the wild west for standards make integration challenging. Come up with your own conventions for what you want to do. Evolve to mix and match conventions that are right for your business. Understand the versioning strategy and ensure everyone is on the same page or you’ll end up with elaborate API gateways. Microservices should always be Restful and use JSON.
• How to break out the code base – what’s the right size. Think about how to structure the code base while maintaining synchronous dependency. Could you be reducing reliability and latency? Networking considerations with distribution systems. Able to log and track metrics across all microservices.
• Networking between microservices, in addition to making sure everything stays up to date and compatible can be a challenge.
• You would like to use microservices for all of your products but you have to figure out how to architect microservices to handle multiple products. Need to determine how microservices will work across a siloed organization. What is the balance between products? Map all potential services out there and think about what could be combined. How to orchestrate everything together in a synchronized fashion. You can use Kubernetes but it requires more plumbing. Testing in your own language. Thinking about bigger business objectives.
• A mix of cloud and microservices coming together in the application stack. How to get visibility over customers and see the data in its entirety.

What are the most significant challenges to the implementation of a microservices architecture from your perspective?

Here’s who we spoke to:

• Thomas Butt, CTO, CardCash
• Matt McLarty, Vice President, API Academy, CA Technologies
• Brian Dawson, DevOps Evangelist, CloudBees
• Lucas Vogel, Founder, Endpoint Systems
• Ali Hodroj, V.P. Products and Strategy, GigaSpaces
• Job van der Voort, VP Product, GitLab
• Kevin Sutter, MicroProfile and Java EE Architect, IBM
• Sandeep Singh Kohli, Director of Marketing, MuleSoft
• Karl McGuinness, Senior Director of Identity, Okta
• Ross Smith, Chief Architect, PITSS America
• Mike LaFleur, Director of Solution Architecture, Provenir
• Gianni Fiore, CTO, Rebrandly
• Peter Yared, CTO, Sapho
• Sha Ma, V.P. Software Engineering, SendGrid
• Keshav Vasudevan, Product Marketing Manager, Swagger/SwaggerHub, SmartBear
• Chris McFadden, V.P. Engineering and Operations, SparkPost
• Christian Beedgen, Co-founder and CTO, Sumo Logic
• Todd Millecam, CEO, SWYM Systems, Inc.
• Tim Jarret, Senior Director of Product Marketing, Veracode