Microservices

Distributed caching is an important aspect of cloud-based applications, be it for on-premises, public, or hybrid cloud environments. It facilitates incremental scaling, allowing the cache to grow and incorporate the data growth. In this blog, we will explore distributed caching on the cloud and why it is useful for environments with high data volume and load. This blog will cover the following: Traditional Caching Challenges What is Distributed Caching Benefits of Distributed Caching on cloud Recommended Distributed Caching Database Tools Ways to Deploy Distributed Caching on Hybrid Cloud Traditional Caching Challenges Traditional caching servers are usually deployed with limited storage and CPU speed. Often these caching infrastructures reside in data centers that are on-premises. I am referring to a non-distributed caching server. Traditional distributed caching comes with numerous challenges: Hard-to-scale cache storage and CPU speed on non-cloud node servers. High operational cost to manage infrastructure and unutilized hardware resources. Inability to scale and manage traditional distributed caching (since it is non-containerized). Possibility of servers crashing if client load is higher than actual. Chances of stale data during programmatic sync-up with multiple data center servers. Slow data synchronization between servers and various data centers. What Is Distributed Caching? Caching is a technique to store the state of data outside of the main storage and store it in high-speed memory to improve performance. In a microservices environment, all apps are deployed with their multiple instances across various servers/containers on the hybrid cloud. A single caching source is needed in a multicluster Kubernetes environment on the cloud to persist data centrally and replicate it on its own caching cluster. It will serve as a single point of storage to cache data in a distributed environment. Benefits of Distributed Caching on Cloud Periodic caching of frequently used read REST APIs' response ensures faster API read performance. Reduced database network calls by accessing cached data directly from distributed caching databases. Resilience and fault tolerance by maintaining multiple copies of data at various caching databases in a cluster. High availability by auto-scaling the cache databases based on load or client requests. Storage of secret session tokens like JSON Web Token (ID/JWT) for authentication and authorization purposes for microservices apps containers. Faster read and write access in-memory if it's used as a dedicated database solution for high-load mission-critical applications. Avoid unnecessary roundtrip data calls to persistent databases. Auto-scalable cloud infrastructure deployment. Containerization of distributed caching libraries/solutions. Consistent read data from any synchronized connected caching data centers. Minimal to no outage, high availability of caching data. Faster data synchronization between caching data servers. Recommended Distributed Caching Database Tools The following are popular industry-recognized caching servers: Redis Memcached GemFire Hazelcast databases Redis It's one of the most popular distributed caching services. It supports different data structures. It's an open-source, in-memory data store used by millions of developers as a database, cache, streaming engine, and message broker. It also has an enterprise version. It can be deployed in containers on private, public, and hybrid clouds, providing consistent and faster data synchronization between different data centers. Memcached It is an open-source, high-performance, distributed memory object caching system. It is generic in nature but intended for use in speeding up dynamic web applications by alleviating database load. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from the results of database calls, API calls, or page rendering. Memcached is simple yet powerful. Its simple design promotes easy, quick deployment and development. It solves many data-caching problems, and the API is available in various commonly used languages. GemFire It provides distributed in-memory data-grid cache powered by Apache Geode open source. It scales data services on demand to support high performance. It's a key-value store that performs read and write operations at fast speeds. In addition, it offers highly available parallel message queues, continuous availability, and an event-driven architecture to scale dynamically with no downtime. It provides multisite replication. As data size requirements increase to support high-performance, real-time apps, they can scale linearly with ease. Applications get low-latency responses to data-access requests and always return fresh data. It maintains transaction integrity across distributed nodes and supports high-concurrency, low-latency data operations of the application. It also provides node failover and cross-data center or multi-datacenter replication to ensure applications are resilient, whether on-premises or in the cloud. Hazelcast Hazelcast is a distributed computation and storage platform for consistent low-latency querying, aggregation, and stateful computation against event streams and traditional data sources. It allows you to quickly build resource-efficient, real-time applications. You can deploy it at any scale, from small-edge devices to a large cluster of cloud instances. A cluster of Hazelcast nodes share both the data storage and computational load, which can dynamically scale up and down. When you add new nodes to the cluster, the data is automatically rebalanced across the cluster. The computational tasks (jobs) that are currently in a running state snapshot their state and scale with a processing guarantee. Ways to Deploy Distributed Caching on Hybrid Cloud These are recommended ways to deploy, and setup distributed caching, be it on the public cloud or hybrid cloud: Open source distributed caching on traditional VM instances. Open source distributed caching on Kubernetes container. I would recommend deploying on a Kubernetes container for high availability, resiliency, scalability, and faster performance. Enterprise commercial off-the-shelf distributed caching deployment on VM and container. I would recommend the enterprise version because it will provide additional features and support. The public cloud offers managed services of distributed caching for open-source and enterprise tools like Redis, Hazelcast and Memcached, etc. Caching servers can be deployed on multiple sources like on-premises and public cloud together, public servers, or only one public server in different availability zones. Conclusion Distributed caching is now a de-facto requirement for distributed microservices applications in a distributed deployment environment on a hybrid cloud. It addresses concerns in important use cases like maintaining user sessions when cookies are disabled on the web browser, improving API query read performance, avoiding operational costs and database hits for the same type of requests, managing secret tokens for authentication and authorization, etc. Distributed cache syncs data on hybrid clouds automatically without any manual operation and always gives the latest data. I would recommend industry-standard distributed caching solutions like Redis, Hazelcast, and Memcached. We need to choose a better distributed caching technology in the cloud based on use cases.

This is an article from DZone's 2022 Enterprise Application Security Trend Report.For more: Read the Report According to a 2020 Gartner report, it is estimated that by 2023, 75 percent of cybersecurity incidents will result from inadequate management of identities and excessive privileges. To a large extent, this is attributable to the increased number of identities used by modern cloud infrastructures. Applications run as microservices in fully virtualized environments that consist of dynamically orchestrated clusters of multiple containers in the cloud. The security requirements in such environments are significantly different compared to monolithic applications running on-premises. First, the concept of the perimeter does not exist in the cloud. Second, organizations are now handling thousands of dynamically created workloads and identities. Applying traditional IAM tools to manage the dynamic nature of these identities is not adequate. Using static, long-lived, and often excessive access permissions enables attackers to perform lateral movement. To address these issues, a security model is needed that better satisfies today's application security and identity requirements. Zero-trust security is a proactive security model that uses continuous verification and adaptive security controls to protect endpoints and access to applications as well as the data that flows between them. Zero trust replaces the outdated assumption that everything running inside an organization's network can be implicitly trusted. This security model has proven to minimize the attack surface, offer threat protection against internal and external attackers, reduce the lateral movement of attackers, increase operational efficiency, and help support continuous compliance with regulations such as PCI-DSS and the White House's 2021 Cybersecurity Executive Order. Since its inception, zero trust has evolved and expanded, touching almost every corner of the enterprise. This article will provide an overview of how the zero-trust principles can be applied in a microservices environment and what security controls should be implemented on the back end. Zero-Trust Principles Zero trust is primarily based on the concepts of "never trust, always verify" and "assume everything is hostile by default." It is driven by three core principles: assume breach, verify explicitly, and the principle of least privilege. Assume Breach Always assume that cyber attacks will happen, the security controls have been compromised, and the network has been infiltrated. This requires using redundant and layered security controls, constant monitoring, and collection of telemetry to detect anomalies and respond in real-time. Verify Explicitly No network traffic, component, action, or user is inherently trusted within a zero-trust security model, regardless of location, source, or identity. Trust only to the extent that you verify the identity, authenticity, permissions, data classification, etc. Principle of Least Privilege Always grant the least number of privileges. Only give access for the time that it is needed and remove access when it is not needed anymore. Least privilege access is essential to reduce the attack surface, limit the "blast radius," and minimize an attacker's opportunity to move laterally within an environment in case of compromise. Zero-Trust Security in a Microservices Environment When a microservice is compromised, it may maliciously influence other services. By applying the principles of zero trust to a microservices environment, the trust between services, components, and networks is eliminated or minimized. Identity and Access Management Identity and access management is the backbone of zero trust, which requires strong authentication and authorization of end-user identities, services, functions, workloads, and devices. To enable authentication and authorization, we must first ensure that each workload is automatically assigned a cryptographically secure identity that is validated on every request. Importantly, ensure that there is an automated mechanism to reliably distribute, revoke in case of compromise, and frequently rotate the services' certificates and secrets. Use a cloud-neutral identity for workloads, such as SPIFFE for authentication and OPA for unified authorization across the stack. Secure Service-To-Service Communications In zero trust, it is fundamental to treat the network as adversarial. Thus, all communication between services, APIs, and storage layers must be encrypted. The standard way of protecting data in transit is to use HTTPS and strict mTLS everywhere. Similarly, a strong authentication mechanism should be enforced across all microservices. It must be understood that not every service that can be authenticated should be authorized. Authorization must be done based on the authentication context and on access control policies, and it should be performed at the edge of each microservice — not at the network edge. To achieve this, use a service mesh, like Istio or Linkerd, for: Automatic certificate management Traffic interception Secure service-to-service communication without application code changes Micro-segmentation (via authorization policies) This reduces the blast radius of an attack and prevents attackers from pivoting from one compromised service into other parts of the infrastructure. In a container orchestration environment, such as Kubernetes, define network policies for egress and ingress isolation at a granular level. Enforce zero trust for all traffic (east-west and north-south) by specifying network policies and service-to-service level RBAC policies that limit access per cluster and per source, following the need-to-know principle. Secure Access to Resources External entities must not access the microservices environment directly. Instead, use an API gateway as a single entry point to the microservices deployment. To pass the user context or the identity of the caller, implement a pattern, such as the phantom token pattern (API Security in Action, part 11.6.1) or the passport pattern. Validate the external access token and user context at the edge and generate a new short-lived token that represents the external entity identity and is cryptographically signed by the trusted issuer and propagated to back-end microservices. Ensure that the new token's scope of access is as limited as the scope of the identity of the external entity. Most importantly, assume that access tokens can be stolen and create access tokens with a short lifespan on a resource-by-resource basis. Use a service mesh to verify the validity of the access tokens at the microservice edge. In all cases, access to resources should be granted using fine-grained role-based access controls with the least privileges. Figure 1: Data in-transit, data at-rest, and data in-use encryption Data Security It is essential to ensure that all data is classified according to their secrecy and confidentiality. Create a data registry to know which microservice handles what data. Then, implement multiple layers of data encryption, depending on the data classification. Do not trust only the encryption of external components (including databases and messaging systems like Kafka). Use application-level encryption (ALE) to transfer personally identifiable information (PII) and highly confidential data between microservices. To mitigate the risk of unauthorized data modification, perform data integrity checksums throughout the data lifecycle. Infrastructure Security Adopting an immutable infrastructure has become standard. Use Infrastructure as Code to provision components upfront and never change them after deployment. Do not trust the storage mediums (persistent or temporary) and do not store any sensitive data or secrets in an unencrypted form. All secrets, certificates, and API keys should be securely stored in access-controlled centralized key vaults. Zero trust always assumes that the network is compromised. To contain a possible compromise and prevent lateral spreading through the rest of the network, implement network micro-segmentation, create software-defined perimeters in each segment, and place microservices in each segment according to their functionality, business domain, and data classification. Communication between segments should be well-defined and controlled through API gateways. Consider adopting a cell-based architecture for inter-segment communication. Container and Cluster Security Zero trust requires the explicit verification of container images, containers, and cluster nodes. Thus, use container images that are signed only from trusted issuers and registries. Allow images to be used only if they are scanned in the DevSecOps pipeline and have no vulnerabilities. To reduce the risk of privilege escalation, run the Docker daemon and all containers without root privileges. One standard way is to run Docker in rootless mode. Logically isolate high-risk applications and workloads in the same cluster for the least number of privileges. Runtime Security Consider running security-sensitive microservices on confidential virtual machines in hardware-based trusted execution environments with encrypted memory. To reduce the risk of rogue or compromised nodes in the cluster, verify the integrity of nodes, VMs, and containers by running them on instances enabled with Secure Boot and Virtual Trusted Platform Module. Also, by running containers in read-only mode, filesystem integrity is achieved and attackers are prevented from making modifications. Finally, we can reduce our trust for the runtime by adopting a RASP solution that inspects all code executed by the runtime and dynamically stops the execution of malicious code. Figure 2: Zero-trust runtime via confidential computing and RASP Image adapted from "Application enclave support with Intel SGX based confidential computing nodes on AKS," Microsoft Azure Documentation Conclusion Implementing a zero-trust architecture is a critical defense-in-depth strategy and has become a mandatory security model in modern IT infrastructures. It is important to understand that implementing a zero-trust architecture does not mean zero security incidents. The goal is to continually layer security controls to increase the cost of attacks. As we introduce more friction into the cyber-attack kill chain, the attacker's value proposition will be reduced, and potential attacks will be disrupted. The key to a successful implementation of a zero-trust architecture is to follow the guidance of whitepapers such as NIST's "Planning for a Zero Trust Architecture" and the U.S. Office of Management and Budget's "Moving the U.S. Government Towards Zero Trust Cybersecurity Principles." In this article, we provided an overview of how to apply the core principles of the zero-trust model in a microservices environment, and we examined the critical areas and the zero-trust security goals of microservices that need to be achieved. The highly distributed and heterogeneous nature of a microservice deployment and its complex communication patterns has increased the number of different components and the volume of data that is exposed on the network. This provides a broader attack surface compared to a traditional deployment of a monolithic application. Because the security of a system is as good as its weakest link, applying the zero-trust core principles to proactively secure all layers and components of a microservices deployment is fundamental for a modern, reliable, and mature cybersecurity strategy. With a proper zero-trust strategy for microservices, the risk of compromised clusters, lateral movement, and data breaches in most cases can be eliminated. Zero trust is a necessary evolution to security; however, its implementation should not be a destination. It is a continuous journey and an organization-wide commitment. Since its inception, zero trust has become a widely deployed security model and a business-critical cybersecurity priority. Microsoft's 2021 Zero Trust Adoption Report confirms that point on page 11, indicating that 76 percent of organizations have started adopting a zero-trust strategy. The industry is rapidly adopting zero trust across the whole infrastructure and not just on end-user access. This is an article from DZone's 2022 Enterprise Application Security Trend Report.For more: Read the Report

In this blog, you will learn some Docker best practices mainly focussed on Spring Boot applications. You will learn these practices by applying them to a sample application. Enjoy! 1. Introduction This blog continues where the previous blog about Docker Best Practices left off. However, this blog can be read independently from the previous one. The goal is to provide some best practices that can be applied to Dockerized Spring Boot applications. The Dockerfile that will be used as a starting point is the following: Dockerfile FROM eclipse-temurin:17.0.5_8-jre-alpine@sha256:02c04793fa49ad5cd193c961403223755f9209a67894622e05438598b32f210e WORKDIR /opt/app RUN addgroup --system javauser && adduser -S -s /usr/sbin/nologin -G javauser javauser ARG JAR_FILE COPY target/${JAR_FILE} app.jar RUN chown -R javauser:javauser . USER javauser ENTRYPOINT ["java", "-jar", "app.jar"] This Dockerfile is doing the following: FROM: Take eclipse-temurin:17 Java Docker image as base image; WORKDIR: Set /opt/app as the working directory; RUN: Create a system group and system user; ARG: provide an argument JAR_FILE so that you do not have to hard code the jar file name into the Dockerfile; COPY: Copy the jar file into the Docker image; RUN: Change the owner of the WORKDIR to the previously created system user; USER: Ensure that the previously created system user is used; ENTRYPOINT: Start the Spring Boot application. In the next sections, you will change this Dockerfile to adhere to best practices. The resulting Dockerfile of each paragraph is available in the git repository in the directory Dockerfiles. At the end of each paragraph, the name of the corresponding final Dockerfile will be mentioned where applicable. The code being used in this blog is available on GitHub. 2. Prerequisites The following prerequisites apply to this blog: Basic Linux knowledge Basic Java and Spring Boot knowledge Basic Docker knowledge 3. Sample Application A sample application is needed in order to demonstrate the best practices. Therefore, a basic Spring Boot application is created containing the Spring Web and Spring Actuator dependencies. The application can be run by invoking the following command from within the root of the repository: Shell $ mvn spring-boot:run Spring Actuator will provide a health endpoint for your application. By default, it will always return the UP status. Shell $ curl http://localhost:8080/actuator/health {"status":"UP"} In order to alter the health status of the application, a custom health indicator is added. Every 5 invocations, the health of the application will be set to DOWN. Java @Component public class DownHealthIndicator implements HealthIndicator { private int counter; @Override public Health health() { counter++; Health.Builder status = Health.up(); if (counter == 5) { status = Health.down(); counter = 0; } return status.build(); } } For building the Docker image, a fork of the dockerfile-maven-plugin of Spotify will be used. The following snippet is therefore added to the pom file. XML <plugin> <groupId>com.xenoamess.docker</groupId> <artifactId>dockerfile-maven-plugin</artifactId> <version>1.4.25</version> <configuration> <repository>mydeveloperplanet/dockerbestpractices</repository> <tag>${project.version}</tag> <buildArgs> <JAR_FILE>${project.build.finalName}.jar</JAR_FILE> </buildArgs> </configuration> </plugin> The advantage of using this plugin is that you can easily reuse the configuration. Creating the Docker image can be done by a single Maven command. Building the jar file is done by invoking the following command: Shell $ mvn clean verify Building the Docker image can be done by invoking the following command: Shell $ mvn dockerfile:build Run the Docker image: Shell $ docker run --name dockerbestpractices mydeveloperplanet/dockerbestpractices:0.0.1-SNAPSHOT Find the IP-address of the running container: Shell $ docker inspect dockerbestpractices | grep IPAddress "SecondaryIPAddresses": null, "IPAddress": "172.17.0.3", "IPAddress": "172.17.0.3" In the above example, the IP-address is 172.17.0.3. The application also contains a HelloController which just responds with a hello message. The Hello endpoint can be invoked as follows: Shell $ curl http://172.17.0.3:8080/hello Hello Docker! Everything is now explained to get started! 4. Best Practices 4.1 Healthcheck A healthcheck can be added to your Dockerfile in order to expose the health of your container. Based on this status, the container can be restarted. This can be done by means of the HEALTHCHECK command. Add the following healthcheck: Dockerfile HEALTHCHECK --interval=30s --timeout=3s --retries=1 CMD wget -qO- http://localhost:8080/actuator/health/ | grep UP || exit 1 This healthcheck is doing the following: interval: Every 30 seconds the healthcheck is executed. For production use, it is better to choose something like five minutes. In order to do some tests, a smaller value is easier. This way you do not have to wait for five minutes each time. timeout: A timeout of three seconds for executing the health check. retries: This indicates the number of consecutive checks which have to be executed before the health status changes. This defaults to three which is a good number for in-production. For testing purposes, you set it to one, meaning that after one unsuccessful check, the health status changes to unhealthy. command: The Spring Actuator endpoint will be used as a healthcheck. The response is retrieved and piped to grep in order to verify whether the health status is UP. It is advised not to use curl for this purpose because not every image has curl available. You will need to install curl in addition to the image and this enlarges the image with several MBs. Build and run the container. Take a closer look at the status of the container. In the first 30 seconds, the health status indicates starting because the first health check will be done after the interval setting. Shell $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ddffb5a9cbf0 mydeveloperplanet/dockerbestpractices:0.0.1-SNAPSHOT "java -jar /opt/app/…" 8 seconds ago Up 6 seconds (health: starting) dockerbestpractices After 30 seconds, the health status indicates healthy. Shell $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ddffb5a9cbf0 mydeveloperplanet/dockerbestpractices:0.0.1-SNAPSHOT "java -jar /opt/app/…" 33 seconds ago Up 32 seconds (healthy) dockerbestpractices After 2-5 minutes, the health status indicates unhealthy because of the custom health indicator you added to the sample application. Shell $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ddffb5a9cbf0 mydeveloperplanet/dockerbestpractices:0.0.1-SNAPSHOT "java -jar /opt/app/…" 2 minutes ago Up 2 minutes (unhealthy) dockerbestpractices Again, 30 seconds after the unhealthy status, the status reports healthy. Did you notice that the container did not restart due to the unhealthy status? That is because the Docker engine does not do anything based on this status. A container orchestrator like Kubernetes will do a restart. Is it not possible to restart the container when running with the Docker engine? Yes, it can: you can use the autoheal Docker image for this purpose. Let’s start the autoheal container. Shell docker run -d \ --name autoheal \ --restart=always \ -e AUTOHEAL_CONTAINER_LABEL=all \ -v /var/run/docker.sock:/var/run/docker.sock \ willfarrell/autoheal Verify whether it is running. Shell $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ddffb5a9cbf0 mydeveloperplanet/dockerbestpractices:0.0.1-SNAPSHOT "java -jar /opt/app/…" 10 minutes ago Up 10 minutes (healthy) dockerbestpractices d40243eb242a willfarrell/autoheal "/docker-entrypoint …" 5 weeks ago Up 9 seconds (healthy) autoheal Wait until the health is unhealthy again or just invoke the health actuator endpoint in order to speed it up. When the status reports unhealthy, the container is restarted. You can verify this in the STATUS column where you can see the uptime of the container. Shell $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ddffb5a9cbf0 mydeveloperplanet/dockerbestpractices:0.0.1-SNAPSHOT "java -jar /opt/app/…" 12 minutes ago Up 6 seconds (health: starting) dockerbestpractices You have to decide for yourself whether you want this or whether you want to monitor the health status yourself by means of a monitoring tool. The autoheal image provides you the means to automatically restart your Docker container(s) without manual intervention. The resulting Dockerfile is available in the git repository with the name 6-Dockerfile-healthcheck. 4.2 Docker Compose Docker Compose gives you the opportunity to start multiple containers at once with a single command. Besides that, it also enables you to document your services, even when you only have one service to manage. Docker Compose used to be installed separately from Docker, but nowadays it is part of Docker itself. You need to write a compose.yml file that contains this configuration. Let’s see what this looks like for the two containers you used during the healthcheck. YAML services: dockerbestpractices: image: mydeveloperplanet/dockerbestpractices:0.0.1-SNAPSHOT autoheal: image: willfarrell/autoheal:1.2.0 restart: always environment: AUTOHEAL_CONTAINER_LABEL: all volumes: - type: bind source: /var/run/docker.sock target: /var/run/docker.sock Two services (read: containers) are configured. One for the dockerbestpractices image and one for the autoheal image. The autoheal image will restart after a reboot, has an environment variable defined, and has a volume mounted. Execute the following command from the directory where the compose.yml file can be found: Shell $ docker compose up In the logging, you will see that both containers are started. Open another terminal window and navigate to the directory where the compose.yml can be found. A lot of commands can be used in combination with Docker Compose. E.g. show the status of the running containers. Shell $ docker compose ps NAME COMMAND SERVICE STATUS PORTS mydockerbestpracticesplanet-autoheal-1 "/docker-entrypoint …" autoheal running (healthy) mydockerbestpracticesplanet-dockerbestpractices-1 "java -jar /opt/app/…" dockerbestpractices running (healthy) Or stop the containers: Shell $ docker compose stop [+] Running 2/2 ⠿ Container mydockerbestpracticesplanet-autoheal-1 Stopped 4.3s ⠿ Container mydockerbestpracticesplanet-dockerbestpractices-1 Stopped 0.3s Or easily remove the containers: Shell $ docker compose rm ? Going to remove mydockerbestpracticesplanet-dockerbestpractices-1, mydockerbestpracticesplanet-autoheal-1 Yes [+] Running 2/0 ⠿ Container mydockerbestpracticesplanet-autoheal-1 Removed 0.0s ⠿ Container mydockerbestpracticesplanet-dockerbestpractices-1 Removed As you can see, Docker Compose provides quite some advantages and you should definitely consider using it. 4.3 Multi-Stage Builds Sometimes it can be handy to build your application inside a Docker container. The advantage is that you do not need to install a complete development environment onto your system and that you can interchange the development environment more easily. However, there is a problem with building the application inside your container. Especially when you want to use the same container for running your application. The sources and the complete development environment will be available in your production container and this is not a good idea from a security perspective. You could write separate Dockerfiles to circumvent this issue: one for the build and one for running the application. But this is quite cumbersome. The solution is to use multi-stage builds. With multi-stage builds, you can separate the building stage from the running stage. The Dockerfile looks as follows: Dockerfile FROM maven:3.8.6-eclipse-temurin-17-alpine@sha256:e88c1a981319789d0c00cd508af67a9c46524f177ecc66ca37c107d4c371d23b AS builder WORKDIR /build COPY . . RUN mvn clean package -DskipTests FROM eclipse-temurin:17.0.5_8-jre-alpine@sha256:02c04793fa49ad5cd193c961403223755f9209a67894622e05438598b32f210e WORKDIR /opt/app RUN addgroup --system javauser && adduser -S -s /usr/sbin/nologin -G javauser javauser COPY --from=builder /build/target/mydockerbestpracticesplanet-0.0.1-SNAPSHOT.jar app.jar RUN chown -R javauser:javauser . USER javauser HEALTHCHECK --interval=30s --timeout=3s --retries=1 CMD wget -qO- http://localhost:8080/actuator/health/ | grep UP || exit 1 ENTRYPOINT ["java", "-jar", "app.jar"] As you can see, this Dockerfile contains two FROM statements. The first one is used for building the application: FROM: A Docker image containing Maven and Java 17, this is needed for building the application; WORKDIR: Set the working directory; COPY: copy the current directory to the working directory into the container; RUN: The command in order to build the jar file. Something else is also added to the FROM statement. At the end, AS builder is added. This way, this container is labeled and can be used for building the image for running the application. The second part is identical to the Dockerfile you used to have before, except for two lines. The following lines are removed: Dockerfile ARG JAR_FILE COPY target/${JAR_FILE} app.jar These lines ensured that the jar file from our local build was copied into the image. These are replaced with the following line: Dockerfile COPY --from=builder /build/target/mydockerbestpracticesplanet-0.0.1-SNAPSHOT.jar app.jar With this line, you indicate that you want to copy a file from the builder container into the new image. When you build this Dockerfile, you will notice that the build container executes the build and finally, the image for running the application is created. During building the image, you will also notice that all Maven dependencies are downloaded. The resulting Dockerfile is available in the git repository with the name 7-Dockerfile-multi-stage-build. 4.4 Spring Boot Docker Layers A Docker image consists of layers. If you are not familiar with Docker layers, you can check out a previous post. Every command in a Dockerfile will result in a new layer. When you initially pull a Docker image, all layers will be retrieved and stored. If you update your Docker image and you only change for example the jar file, the other layers will not be retrieved anew. This way, your Docker images are stored more efficiently. However, when you are using Spring Boot, a fat jar is created. Meaning that when you only change some of your code, a new fat jar is created with unchanged dependencies. So each time you create a new Docker image, megabytes are added in a new layer without any necessity. For this purpose, Spring Boot Docker layers can be used. A detailed explanation can be found here. In short, Spring Boot can split the fat jar into several directories: /dependencies /spring-boot-loader /snapshot-dependencies /application The application code will reside in the directory application, whereas for example, the dependencies will reside in directory dependencies. In order to achieve this, you will use a multi-stage build. The first part will copy the jar file into a JDK Docker image and will extract the fat jar. Dockerfile FROM eclipse-temurin:17.0.4.1_1-jre-alpine@sha256:e1506ba20f0cb2af6f23e24c7f8855b417f0b085708acd9b85344a884ba77767 AS builder WORKDIR application ARG JAR_FILE COPY target/${JAR_FILE} app.jar RUN java -Djarmode=layertools -jar app.jar extract The second part will copy the split directories into a new image. The COPY commands replace the jar file. Shell FROM eclipse-temurin:17.0.4.1_1-jre-alpine@sha256:e1506ba20f0cb2af6f23e24c7f8855b417f0b085708acd9b85344a884ba77767 WORKDIR /opt/app RUN addgroup --system javauser && adduser -S -s /usr/sbin/nologin -G javauser javauser COPY --from=builder application/dependencies/ ./ COPY --from=builder application/spring-boot-loader/ ./ COPY --from=builder application/snapshot-dependencies/ ./ COPY --from=builder application/application/ ./ RUN chown -R javauser:javauser . USER javauser HEALTHCHECK --interval=30s --timeout=3s --retries=1 CMD wget -qO- http://localhost:8080/actuator/health/ | grep UP || exit 1 ENTRYPOINT ["java", "org.springframework.boot.loader.JarLauncher"] Build and run the container. You will not notice any difference when running the container. The main advantage is the way the Docker image is stored. The resulting Dockerfile is available in the git repository with the name 8-Dockerfile-spring-boot-docker-layers. 5. Conclusion In this blog, some best practices are covered when creating Dockerfiles for Spring Boot applications. Learn to apply these practices and you will end up with much better Docker images.

Microservices today are often deployed on a platform such as Kubernetes, which orchestrates the deployment and management of containerized applications. Microservices, however, don't exist in a vacuum. They typically communicate with other services, such as databases, message brokers, or other microservices. Therefore, an application usually consists of multiple services that form a complete solution. But, as a developer, how do you develop and test an individual microservice that is part of a larger system? This article examines some common inner-loop development cycle challenges and shows how Quarkus and other technologies help solve some of these challenges. What Is the Inner Loop? Almost all software development is iterative. The inner loop contains everything that happens on a developer's machine before committing code into version control. The inner loop is where a developer writes code, builds and tests it, and perhaps runs the code locally. In today's world, the inner loop could also include multiple commits to a Git pull request, where a developer may commit multiple times against a specific feature until that feature is deemed complete. Note: The word local is also up for debate in industry today as more and more remote development environments, such as Red Hat OpenShift Dev Spaces, Gitpod, and GitHub Codespaces are available. This article does not differentiate between a developer machine and any of these kinds of environments. They are all viewed as local in this article. Inner loop shifts to outer loop when code reaches a point in source control where it needs to be built, tested, scanned, and ultimately deployed by automated continuous integration and deployment (CI/CD) processes. Figure 1 illustrates a simple inner loop and outer loop. Figure 1: The inner loop takes place on a developer's local machine, whereas the outer loop takes place within CI/CD processes. Challenges of Inner Loop Development Developing a single microservice in isolation is challenging enough without worrying about additional downstream services. How do you run a microservice in isolation on your local machine if it depends on other services for it to function properly? Using various mocking techniques, you can to some extent get around the absence of required services when writing and running tests. Mocking techniques generally work great for testing. You can also use in-memory replacements for required services, such as an H2 database instead of a separate database instance. Beyond that, if you want or need to run the application locally, you need a better solution. Of course, you could try to reproduce your application's entire environment on your development machine. But even if you could, would you really want to? Do you think a developer at Twitter or Netflix could reproduce their environment on their development machine? Figure 2 shows the complexity of their architectures. Lyft also tried this approach and found it wasn't feasible or scalable. Figure 2 (courtesy of https://future.com/the-case-for-developer-experience): Major services such as Netflix and Twitter can easily have more than 500 microservices. Container-based Inner Loop Solutions Using containers can help speed up and improve the inner loop development lifecycle. Containers can help isolate and provide a local instance of a dependent service. We'll look at a few popular tools and technologies for the inner loop. Docker Compose One common pattern is to use Docker Compose to run some of your microservice's dependent services (databases, message brokers, etc.) locally while you run, debug, and test your microservice. With Docker Compose, you define a set of containerized services that provide the capabilities required by your microservice. You can easily start, stop, and view logs from these containerized services. However, there are a few downsides to using Docker Compose. First, you must maintain your Docker Compose configuration independently of your application's code. You must remember to make changes to the Docker Compose configuration as your application evolves, sometimes duplicating configuration between your application and your Docker Compose configuration. Second, you are locked into using the Docker binary. For Windows and macOS users, Docker Desktop is no longer free for many non-individual users. You are also prevented from using other container runtimes, such as Podman. Podman does support Docker Compose, but it doesn't support everything you can do with Docker Compose, especially on non-Linux machines. Testcontainers Testcontainers is an excellent library for creating and managing container instances for various services when applications run tests. It provides lightweight, throwaway instances of common databases, message brokers, or anything else that can run in a container. But Testcontainers is only a library. That means an application must incorporate it and do something with it to realize its benefits. Generally speaking, applications that use Testcontainers do so when executing unit or integration tests, but not in production. A developer generally won't include the library in the application's dependencies because Testcontainers doesn't belong as a dependency when the application is deployed into a real environment with real services. Quarkus Quarkus is a Kubernetes-native Java application framework focusing on more than just feature sets. In addition to enabling fast startup times and low memory footprints compared to traditional Java applications, Quarkus ensures that every feature works well, with little to no configuration, in a highly intuitive way. The framework aims to make it trivial to develop simple things and easy to develop more complex ones. Beyond simply working well, Quarkus aims to bring Developer Joy, specifically targeting the inner loop development lifecycle. Dev Mode The first part of the Quarkus Developer Joy story, live coding via Quarkus dev mode, improves and expedites the inner loop development process. When Quarkus dev mode starts, Quarkus automatically reflects code changes within the running application. Therefore, Quarkus combines the Write Code, Build, and Deploy/Run steps of Figure 1's inner loop into a single step. Simply write code, interact with your application, and see your changes running with little to no delay. Dev Services A second part of the Quarkus Developer Joy story, Quarkus Dev Services, automatically provisions and configures supporting services, such as databases, message brokers, and more. When you run Quarkus in dev mode or execute tests, Quarkus examines all the extensions present. Quarkus then automatically starts any unconfigured and relevant service and configures the application to use that service. Quarkus Dev Services uses Testcontainers, which we've already discussed, but in a manner completely transparent to the developer. The developer does not need to add the Testcontainers libraries, perform any integration or configuration, or write any code. Furthermore, Dev Services does not affect the application when it is deployed into a real environment with real services. Additionally, if you have multiple Quarkus applications on your local machine and run them in dev mode, by default, Dev Services attempts to share the services between the applications. Sharing services is beneficial if you work on more than one application that uses the same service, such as a message broker. Let's use the Quarkus Superheroes sample application as an example. The application consists of several microservices that together form an extensive system. Some microservices communicate synchronously via REST. Others are event-driven, producing and consuming events to and from Apache Kafka. Some microservices are reactive, whereas others are traditional. All the microservices produce metrics consumed by Prometheus and export tracing information to OpenTelemetry. The source code for the application is on GitHub under an Apache 2.0 license. The system's architecture is shown in Figure 3. Figure 3: The Quarkus Superheroes application has many microservices and additional dependent services. In the Quarkus Superheroes sample application, you could start both the rest-fights and event-statistics services locally in dev mode. The rest-fights dev mode starts a MongoDB container instance, an Apicurio Registry container instance, and an Apache Kafka container instance. The event-statistics service also requires an Apicurio Registry instance and an Apache Kafka instance, so the instance started by the rest-fights dev mode will be discovered and used by the event-statistics service. Continuous Testing A third part of the Quarkus Developer Joy story, continuous testing, provides instant feedback on code changes by immediately executing affected tests in the background. Quarkus detects which tests cover which code and reruns only the tests relevant to that code as you change it. Quarkus continuous testing combines testing with dev mode and Dev Services into a powerful inner loop productivity feature, shrinking all of Figure 1's inner loop lifecycle steps into a single step. Other Inner Loop Solutions The solutions we've outlined thus far are extremely helpful with local inner loop development, especially if your microservice requires only a small set of other services, such as a database, or a database and message broker. But when there are lots of dependent services, trying to replicate them all on a local machine probably won't work well, if at all. So what do you do? How do you get the speed and agility of inner loop development for an application when it depends on other services that you either can't or don't want to run locally? One solution could be to manage an environment of shared services. Each developer would then configure those services in their local setup, careful not to commit the configuration into source control. Another solution could be to use Kubernetes, giving each developer a namespace where they can deploy what they need. The developer could then deploy the services and configure their local application to use them. Both of these solutions could work, but in reality, they usually end up with a problem: The microservice the developer is working on is somewhere in the graph of services of an overall system. How does a developer trigger the microservice they care about to get called as part of a larger request or flow? Wouldn't a better solution be to run the application locally, but make the larger system think the application is actually deployed somewhere? This kind of remote + local development model is becoming known as remocal. It is an extremely powerful way to get immediate feedback during your inner loop development cycle while ensuring your application behaves properly in an environment that is close to or matches production. Quarkus Remote Development Another part of the Quarkus Developer Joy story, remote development, enables a developer to deploy an application into a remote environment and run Quarkus dev mode in that environment while doing live coding locally. Quarkus immediately synchronizes code changes to the remotely deployed instance. Quarkus remote development allows a developer to develop the application in the same environment it will run in while having access to the same services it will have access to. Additionally, this capability greatly reduces the inner feedback loop while alleviating the "works on my machine" problem. Remote development also allows for quick and easy prototyping of new features and capabilities. Figure 4 illustrates how the remote development mode works. Figure 4: Quarkus remote dev mode incrementally synchronizes local code changes with a remote Quarkus application. First, the application is deployed to Kubernetes, a virtual machine, a container, or just some Java virtual machine (JVM) somewhere. Once running, the developer runs the remote development mode on their local machine, connecting their local machine to the remote instance. From there, development is just like live coding in Quarkus dev mode. As the developer makes code changes, Quarkus automatically compiles and pushes the changes to the remote instance. Let's continue with the Quarkus Superheroes example from before. Let's assume the entire system is deployed into a Kubernetes cluster. Let's also assume you want to make changes to the rest-fights microservice. As shown in Figure 5, you start the rest-fights microservice in remote dev mode on your local machine. The rest-fights application running on the cluster connects to the MongoDB, Apicurio Registry, and Apache Kafka instances on the Kubernetes cluster. Figure 5: Changes to the local application in remote development mode continuously send updates to the remote instance. You can then interact with the system through its user interface. Quarkus incrementally synchronizes the changes with the remote instance on the Kubernetes cluster as you make changes to the rest-fights microservice. If you want, you could even use breakpoints within your IDE on your local machine to assist with debugging. Skupper Skupper is a layer 7 service interconnect that enables secure communication across Kubernetes clusters without VPNs or special firewall rules. Using Skupper, an application can span multiple cloud providers, data centers, and regions. Figure 6 shows a high-level view of Skupper. Figure 6: Logically, Skupper connects services on different sites together to exist as a single site. With Skupper, you can create a distributed application comprised of microservices running in different namespaces within different Kubernetes clusters. Services exposed to Skupper are subsequently exposed to each namespace as if they existed in the namespace. Skupper creates proxy endpoints to make a service available within each of the namespaces where it is installed. Figure 7 shows a logical view of this architecture. Figure 7: Logically, Skupper can span multiple Kubernetes clusters and make remote services appear as local ones. Why do we mention Skupper in an article about Kubernetes native inner loop development? Because in addition to bridging applications across Kubernetes clusters, a Skupper proxy can run on any machine, enabling bidirectional communication between the machine and the other Kubernetes clusters. Logically, this is like a local machine inserted into the middle of a set of Kubernetes clusters. Services exposed to Skupper on the clusters can discover services exposed to the Skupper proxy on the local machine and vice versa. Skupper can make our Quarkus Superheroes example even more interesting, taking it further from the remote development scenario we described earlier. With Skupper, rather than continuously synchronizing changes to the rest-fights service from a local instance to a remote instance, you could completely replace the remote rest-fights instance with a local instance running Quarkus dev mode and continuous testing. Skupper would then redirect traffic on the Kubernetes cluster into the rest-fights service running on your local machine. Any outgoing requests made by the rest-fights service, such as connections to the MongoDB, Apicurio registry, and Apache Kafka instances, and even the rest-heroes and rest-villains services, would then be redirected back to the Kubernetes cluster. Figure 8 shows a logical view of what this architecture might look like. Figure 8: Logically, Skupper can make it look like a local developer machine is inside a Kubernetes cluster. You could even use Quarkus dev services to allow the rest-fights microservice to provide its own local MongoDB instance rather than using the instance on the cluster, yet continue to let traffic to Kafka flow onto the cluster. This setup would enable other Kafka consumers listening on the same topic to continue functioning. In this scenario, Quarkus continuously runs the tests of the rest-fights microservice while a developer makes live code changes, all while traffic is continually flowing through the whole system on the Kubernetes cluster. The services could even be spread out to other Kubernetes clusters on different cloud providers in other regions of the world while traffic continues to flow through a developer's local machine. A Better Developer Experience, Whether Local or Distributed Parts two and three of the previously mentioned article series at Lyft show Lyft's approach to solving this problem, albeit using different technologies. As more and more services came to life, Lyft saw that what they were doing wasn't scaling and that they, therefore, needed a kind of "remocal" environment. Quarkus was designed with many of these Developer Joy characteristics in mind. Quarkus helps developers iterate faster and contains built-in capabilities that alleviate many of these challenges and shorten the development lifecycles. Developers can focus on writing code.

Quarkus Quarkus is an open-source CDI-based framework introduced by Red Hat. It supports the development of fully reactive microservices, provides a fast startup, and has a small memory footprint. Below was our overall experience using Quarkus: It helped with a quicker and more pleasant development process. Optimized Serverless deployments for low memory usage and fast startup times Allowed us to utilize both blocking (imperative) and non-blocking (reactive) libraries and APIs Worked well with continuous testing to facilitate test-driven development Allowed support to test the JUnit test cases, which we have developed using test-driven development approach Quarkus Supports Native Builds Quarkus supports native builds for an application deployment which contains the application code, required libraries, Java APIs, and a reduced version of a VM. The smaller VM base improves the startup time of the application. To generate a native build using a Java Maven project, one can leverage Docker or podman with GraalVM: mvn clean install -Dnative -Dquarkus.native.container-build=true -Dmaven.test.skip=true The native executable is lightweight and performance optimized. Common Build and Runtime Errors Quarkus, being fairly new, lacks sufficient support in the community and documentation. Below were some of the errors/issues we encountered during development and their resolution. Build Errors UnresolvedElementException com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved type during parsing: xxx This error is caused by missing classes at the image build time. Since the native image runtime does not include the facilities to load new classes, all code needs to be available and compiled at build time. So any class that is referenced but missing is a potential problem at run time. Solution The best practice is to provide all dependencies to the build process. If you are absolutely sure that the class is 100% optional and will not be used at run time, then you can override the default behavior of failing the build process by finding a missing class with the — allow-incomplete-classpath option to native-image. Runtime Errors Random/SplittableRandom com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected an instance of Random/SplittableRandom class in the image heap These errors are caused when you try to initialize these classes in a static block. Embedding instances of Random and SplittableRandom in native images cause these errors. These classes are meant to provide random values and are typically expected to get a fresh seed in each run. Embedding them in a native image results in the seed value that was generated at build-time to be cached in the native image, thus breaking that expectation. Solution We were able to resolve these by using below different ways: By avoiding build time initialization of classes holding static fields that reference (directly or transitively) instances of Random or SplittableRandomclasses. The simplest way to achieve this is to pass — initialize-at-run-time=<ClassName>to native-image and see if it works. Note that even if this works, it might impact the performance of the resulting native image since it might prevent other classes from being build-time initialized as well. Register classes holding static fields that directly reference instances of Random or SplittableRandom classes to be reinitialized at run-time. This way, the referenced instance will be re-created at run-time, solving the issue. Reset the value of fields (static or not) referencing (directly or transitively) instances of Random or SplittableRandom to null in the native-image heap. ClassNotFoundException/InstantiationException/IllegalArgumentException These errors can occur when a native image builder is not informed about some reflective calls or a resource to be loaded at run time. Or if there is a third-party/custom library that includes some, ahead-of-time incompatible code. Solution In order to resolve these exceptions, add the complaining class in reflect-config.json JSON { { "name": "com.foo.bar.Person", "allDeclaredMethods": true, "allDeclaredConstructors": true } } Reflection Issues With Native Builds When building a native executable, GraalVM operates with a closed-world assumption. Native builds with GraaVM analyzes the call tree and remove all the classes/methods/fields that are not used directly. The elements used via reflection are not part of the call tree, so they are dead code eliminated. In order to include these elements in the native executable, we’ll need to register them for reflection explicitly. JSON libraries typically use reflection to serialize the objects to JSON, and not registering these classes for reflection causes errors like the below: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class org.acme.jsonb.Person and no properties discovered to create BeanSerializer (to avoid an exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) We resolved these by adding below annotation: Java @RegisterForReflection public class MyClass { ... } If the class is in a third-party jar, you can do it by using an empty class that will host the @RegisterForReflection for it: Java @RegisterForReflection(targets={ MyClassRequiringReflection.class, MySecondClassRequiringReflection.class}) public class MyReflectionConfiguration { ... } Note that MyClassRequiringReflection and MySecondClassRequiringReflection will be registered for reflection but not MyReflectionConfiguration. This feature is handy when using third-party libraries using object mapping features (such as Jackson or GSON): Java @RegisterForReflection(targets = {User.class, UserImpl.class}) public class MyReflectionConfiguration { ... } We can use a configuration file to register classes for reflection. As an example, in order to register all methods of class com.test.MyClass for reflection, we create reflection-config.json (the most common location is within src/main/resources). JSON [ { "name" : "com.test.MyClass", "allDeclaredConstructors" : true, "allPublicConstructors" : true, "allDeclaredMethods" : true, "allPublicMethods" : true, "allDeclaredFields" : true, "allPublicFields" : true } ] Integration With DynamoDB-Enhanced Client Another aspect of using Serverless architecture was the use of DynamoDB. Although there are ways to connect simple DynamoDB clients to do all operations, it does require a lot of code writing which brings verbosity and a lot of boilerplate code to the project. We considered using DynamoDBMapper but figured we couldn't use it with Quarkus since it doesn't support Java SDK1. Enhanced DynamoDB Client in Java SDK2 is the substitute Java SDK1 DynamoDBMapper, which worked well with Quarkus, although there were a few issues setting it up for classes when using native images. Annotated Java beans for creating TableSchema apparently didn’t work with native images. Mappings got lost in translation due to reflection during the native build. To resolve this, we used static table schema mappings using builder pattern, which actually is faster compared to bean annotations since it doesn't require costly bean introspection: Java TableSchema<Customer> customerTableSchema = TableSchema.builder(Customer.class) .newItemSupplier(Customer::new) .addAttribute(String.class, a -> a.name("id") .getter(Customer::getId) .setter(Customer::setId) .tags(primaryPartitionKey())) .addAttribute(Integer.class, a -> a.name("email") .getter(Customer::getEmail) .setter(Customer::setEmail) .tags(primarySortKey())) .addAttribute(String.class, a -> a.name("name") .getter(Customer::getCustName) .setter(Customer::setCustName) .addAttribute(Instant.class, a -> a.name("registrationDate") .build(); Quarkus has extensions for commonly used libraries which simplifies the use of that library in an application by providing some extra features which help in development, testing, and configuration for a native build. Recently, Quarkus released an extension for an enhanced client. This extension will resolve the above-mentioned issue related to native build and annotated Java beans for creating TableSchema caused by the use of reflection in AWS SDK. To use this extension in the Quarkus project, add the following dependency in the pom file: XML <dependency> <groupId>io.quarkiverse.amazonservices</groupId> <artifactId>quarkus-amazon-dynamodb</artifactId> </dependency> <dependency> <groupId>io.quarkiverse.amazonservices</groupId> <artifactId>quarkus-amazon-dynamodb-enhanced</artifactId> </dependency> We have three options to select an HTTP client 2 for a sync DynamoDB client and 1 for an async DynamoDB client; the default is a URL HTTP client, and for that need to import the following dependency. XML <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>url-connection-client</artifactId> </dependency> If we want an Apache HTTP client instead of a URL client, we can configure it by using the following property and dependencies: Properties files quarkus.dynamodb.sync-client.type=apache XML <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>apache-client</artifactId> </dependency> <dependency> <groupId>io.quarkus</groupId> <artifactId>quarkus-apache-httpclient</artifactId> </dependency> For an async client, the following dependency can be used: XML <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>netty-nio-client</artifactId> </dependency> Dev and test services for DynamoDB are enabled by default which uses docker to start and stop those dev services; these services help in dev and test by running a local DynamoDB instance; we can configure the following property to stop them if we don’t want to use them or don’t have Docker to run them: Properties files quarkus.dynamodb.devservices.enabled=false We can directly inject enhanced clients into our application, annotate the model with corresponding partitions, sort, and secondary partitions, and sort keys if required. Java @Inject DynamoDbEnhancedClient client; @Produces @ApplicationScoped public DynamoDbTable<Fruit> mappedTable() { return client.table("Fruit", TableSchema.fromClass(Fruit.class)) } Java @DynamoDbBean public class Fruit { private String name; private String description; @DynamoDbPartitionKey public String getName() { return name; } public void setName(String name) { this.name = name; } public String getDescription() { return description; } public void setDescription(String description) { this.description = description; } } Achieving Security With Native Builds — SSL connections Developing microservices architecture will, at some point, require you to call a microservice from another microservice. Quarkus provides its own Rest client configuration to efficiently do so but calling those services securely (using SSL) becomes complicated when using native images. By default, Quarkus will embed the default trust store ($GRAALVM_HOME/Contents/Home/lib/security/cacerts) in the native Docker image. The function.zip generated with the native build won’t embed the default trust store, which causes issues when the function is deployed to AWS. We had to add an additional directory zip.native under src/main to add the certificates to function.zip. zip.native contains cacerts and custom bootstrap.sh (shown below) cacerts is the trust store and it holds all the certificates needed. bootstrap.sh should embed this trust store within the native function.zip. Shell # bootstrap.sh #!/usr/bin/env bash ./runner -Djava.library.path=./ -Djavax.net.ssl.trustStore=./cacerts -Djavax.net.ssl.trustStorePassword=changeit Health Checks and Fault Tolerance with Quarkus Health Checks and Fault Tolerance are crucial for microservices since these help in enabling Failover strategy in applications. We leveraged the quarkus-smallrye-health extension, which provides support to build health checks out of the box. We had overridden the HealthCheck class and added dependency health checks for dependent AWS components like DynamoDB to check for health. Below is one of the sample responses from the health checks with HTTP status 200: JSON { "status": "UP", "checks": [ { "name": "Database connections health check", "status": "UP" } ] } Along with these health checks, we used fault tolerance for microservice-to-microservice calls. In a case called microservice is down or not responding, max retry and timeouts were configured using quarks-small rye-fault-tolerance. After max retries, if the dependent service still doesn't respond, we used method fallbacks to generate static responses. Java import org.eclipse.microprofile.faulttolerance.Retry; import org.eclipse.microprofile.faulttolerance.Timeout; import org.eclipse.microprofile.faulttolerance.Fallback; ... public class FooResource { ... @GET @Retry(maxRetries = 3) @Timeout(1000) public List<Foo> getFoo() { ... } @Fallback(fallbackMethod = "fallbackRecommendations") public List<Foo> fallbackforGetFoo() { ... } public List<Foo> fallbackRecommendations() { ... return Collections.singletonList(fooRepository.getFooById(1)); } } Conclusion Overall, Quarkus is an excellent growing framework that offers a lot of options for developing serverless microservices using native images. It optimizes Java and makes it efficient for containers, cloud, and serverless environments with memory consumption optimization and a fast first response time.

I recently started studying styles of software architecture in different ways: by reading books by renowned architects and by trying to go a step further in my professional career. I have seen how different it is to be an architect from a developer; although these two roles have many points in common, the approach is different. I don't want to describe what it means to be a software architect in this article. I will summarize what I have been reading and learning about the different styles of software architecture categorized as monolithic or distributed. Monolithic vs. Distributed Definitions When we talk about a monolithic or distributed architecture, we are referring to the type of deployment that the application has, that is, should the application be deployed as a whole, in a unitary way (monolithic), or do we deploy several isolated artifacts independently (distributed)? As a developer, I have always thought that a distributed architecture is a solution that comes after having a monolithic architecture. With the passage of time and the increase of application users, you need to isolate certain parts of your code because they make the rest collapse. I was not wrong, but it is not the only way to get there; moreover, it may be the case that you must start with some distributed architecture for different requirements such as security, elasticity, and legal issues (pe PCI), although it is not usual. Styles of Software Architecture Monolithic Architecture This software architecture style has important benefits, especially when starting an application where the domain is at a very early stage and the boundaries between contexts are very blurred. Something important to keep in mind is that regardless of how long the application has been in production, changes in the domain must be carried out. It’s not something you can leave out; we must update our implementation as often as possible to faithfully represent the domain, and no architecture will save you from this. The benefit that a monolithic software architecture style can give you is to make change faster. This does not mean that with a distributed architecture, it cannot be done; we are talking about the time it takes and the security when doing it (this implies development cost). At the development level, it also simplifies a lot in the initial stage by only talking about a single repository in a single language. At the deployment level, it is a bit more complex. At an early stage of the application, simplifying the deployment could help you a lot to focus on the domain. If you introduce microservices at this stage, you are going to have to deal not only with the domain but also with all the infrastructure that goes with it. However, as your application grows, deployments tend to be slower and spaced out over time due to security issues. For example: in e-commerce, wanting to change a small part of the product list implies a deployment not only of that part but of the rest, including the checkout system or other more vital parts, to avoid something that provides benefit being broken by a minuscule change, grouping several small changes in the same one during low-traffic hours. At the scalability level, I think the same as with the deployment; at an early stage, scale the whole project equally while you collect metrics so that later you can see which part of your project needs to scale more or less (scaling a monolith has its risks but Stack Overflow has already demonstrated that it is very viable). As time goes by, you see that it is not necessary to scale the whole project; it is likely that the product list is something that is visited much more than the checkout or the user profile page, so we could consider some changes when deploying this application (or maybe just adding an elastic, who knows). Distributed Architecture When we talk about a distributed architecture, we refer to that type of application where the modules are deployed independently, isolated from the rest, and with communication between them with remote access protocols (e.g., HTTP, grpc, queues, etc.). They are widely known due to their scalability and availability, among other factors. Performance is usually one of their strong points, especially depending on the granularity of the services. In addition, this type of architecture usually relies heavily on eventual consistency, so events are a widely used means of communication between services. This means that by not waiting for a response from the service to which we send a message, we process the requests we receive faster. Sending events opens up a wide range of possibilities for communication between services, but we still need transactions between services. This is where one of the weaknesses of distributed architectures comes in: distributed transactions. The alternative to the distributed transaction is ACID transactions, something impossible to achieve according to the way we dimension our services. In a service-based architecture or microservices where it is more domain-driven (DDD), where services are more coarse-grained, transactions will be ACID (and if they are not, it is very likely that you have a design problem). While in other architectures, for example, EDA, the transactions will be distributed, and there we will need extra pieces (e.g., SAGA, 2PC) to complete, either satisfactorily or not, the transaction. In the event that any part fails, the rest will have to be compensated. When to Use a Monolithic or a Distributed Architecture Although it is something that is asked a lot, the answer will always be “it depends," since each project will have different requirements, and depending on these requirements, a decision will be made, but it is not a decision that should be made quickly, it requires certain criteria, experience, and consideration. It is here where a developer makes the difference, and it is having the ability to see the project from another point of view, a much more global view. Developers will always tend to think more in our day-to-day and not so much in the long term; we will make decisions based on our experience and knowledge, or by mere learning, following the currents of hype. Developers who have the ability to shift their focus to look at the project will make that decision. They will define the characteristics of the application based on the business requirements, whether explicit (they are going to appear in a TV commercial) or implicit (it is an e-commerce, and you know about sales, black Friday, and so on); they will evaluate the different architectures and their tradeoffs and decide which one is more convenient. That said, as we have already mentioned, monolithic architectures are usually good, among other factors, because of the ease of change, and that is why it is a recommended architecture when you are starting a development. This is not to say that you should not consider the possibility of isolating specific components of your monolith in the future. When we usually talk about styles of software architecture, we are referring to top-level styles, that is, the architectures that define the first level. A clear example would be Modular Monolith Architecture, where at the top level, we see a monolithic component divided into modules separated by domain. Still, if we zoom into each domain, we see the layered architecture clearly identified, which would separate the layers by technical components (presentation, domain, or infrastructure, among others). Monolith or “Big Ball of Mud”? It is common to come across projects where a “monolith” is harshly criticized. When you dig a little deeper into the project, you realize that the criticism is not directed at the type of architecture but at what is known precisely as the “lack of” architecture, known as the “Big Ball Of Mud.” This leads to demonizing architecture, something that should not fall into subjectivity. The decision as to whether a monolithic architecture is good or not for the project should be made with weighty arguments, weighing all the trade-offs of both the discarded architecture and the one to be introduced. The fact that subjectively speaking, monolithic architectures have lost a lot of weight leads to complete project restructurings to move from the big ball of mud to the chosen architecture, microservices, leading us to a new architecture known as the “Distributed Big Ball Of Mud”. Needless to say, there will be teams dedicated exclusively to this, stopping business (or trying to) while such restructuring is being done. Far from fixing anything, we’ve made our system even more complicated, making it not only difficult to modify, but we’ve added many more points of failure and latency along the way. Oh, and a slightly larger DevOps team. A much better approach would be to attack the “big ball of mud” in a more direct way while delivering business value by following certain patterns that will help us move to another architecture in a healthier way. Conclusion Lack of attention to architecture leads many companies to make decisions that are not the best ones. Even though they will work, they will not work as they should. Mark Richards once said: “There are no wrong answers in architecture, only expensive ones.” And he is absolutely right: when decisions are made based on a single problem and not from a global point of view, the results are not as expected; you solve one problem by adding ten more to the list. The project will progress, but the delivery to the business will slow down little by little, causing friction between “teams” until reaching a point of no return where the delivery of value to the project will not be affordable, and a possible rewrite will have to be considered, with all that this entails. Here are some of my conclusions: Make decisions as objectively as possible, based on metrics, such as response latency or throughput, if performance is a feature to take into account or the level of coupling based on the instability of our code. Read or listen to others’ experiences through books, articles, or a simple conversation with a colleague. And for me, the most important thing is to be self-critical and leave the ego aside; most likely, the error is not in how we deploy our code but in a bad design.

Microservices are the trend of the hour. Businesses are moving towards cloud-native architecture and breaking their large applications into smaller, self-independent modules called microservices. This architecture gives a lot more flexibility, maintainability, and operability, not to mention better customer satisfaction. With these added advantages, architects and operations engineers face many new challenges as well. Earlier, they were managing one application; now they have to manage many instead. Each application again needs its own support services, like databases, LDAP servers, messaging queues, and so on. So the stakeholders need to think through different strategies for deployment where the entire application can be well deployed while maintaining its integrity and providing optimal performance. Deployment Patterns The microservices architects suggest different types of patterns that could be used to deploy microservices. Each design provides solutions for diverse functional and non-functional requirements. So microservices could be written in a variety of programming languages or frameworks. Again, they could be written in different versions of the same programming language or framework. Each microservice comprises several different service instances, like the UI, DB, and backend. The microservice must be independently deployable and scalable. The service instances must be isolated from each other. The service must be able to quickly build and deploy itself. The service must be allocated proper computing resources. The deployment environment must be reliable, and the service must be monitored. Multiple Service Instances per Host To meet the requirements mentioned at the start of this section, we can think of a solution with which we can deploy service instances of multiple services on one host. The host may be physical or virtual. So, we are running many service instances from different services on a shared host. There are different ways we could do it. We can start each instance as a JVM process. We can also start multiple instances as part of the same JVM process, kind of like a web application. We can also use scripts to automate the start-up and shutdown processes with some configurations. The configuration will have different deployment-related information, like version numbers. With this kind of approach, the resources could be used very efficiently. Service Instance per Host In many cases, microservices need their own space and a clearly separated deployment environment. In such cases, they can’t share the deployment environment with other services or service instances. There may be a chance of resource conflict or scarcity. There might be issues when services written in the same language or framework but with different versions can’t be co-located. In such cases, a service instance could be deployed on its own host. The host could either be a physical or virtual machine. In such cases, there wouldn’t be any conflict with other services. The service remains entirely isolated. All the resources of the VM are available for consumption by the service. It can be easily monitored. The only issue with this deployment pattern is that it consumes more resources. Service Instance per VM In many cases, microservices need their own, self-contained deployment environment. The microservice must be robust and must start and stop quickly. Again, it also needs quick upscaling and downscaling. It can’t share any resources with any other service. It can’t afford to have conflicts with other services. It needs more resources, and the resources must be properly allocated to the service. In such cases, the service could be built as a VM image and deployed in a VM. Scaling could be done quickly, as new VMs could be started within seconds. All VMs have their own computing resources that are properly allocated according to the needs of the microservice. There is no chance of any conflict with any other service. Each VM is properly isolated and can get support for load balancing. Service Instance per Container In some cases, microservices are very tiny. They consume very few resources for their execution. However, they need to be isolated. There must not be any resource sharing. They again can’t afford to be co-located and have a chance of conflict with another service. It needs to be deployed quickly if there is a new release. There might be a need to deploy the same service but with different release versions. The service must be capable of scaling rapidly. It also must have the capacity to start and shut down in a few milliseconds. In such a case, the service could be built as a container image and deployed as a container. In that case, the service will remain isolated. There would not be any chance of conflict. Computing resources could be allocated as per the calculated need of the service. The service could be scaled rapidly. Containers could also be started and shut down quickly. Serverless Deployment In certain cases, the microservice might not need to know the underlying deployment infrastructure. In these situations, the deployment service is contracted out to a third-party vendor, who is typically a cloud service provider. The business is absolutely indifferent about the underlying resources; all it wants to do is run the microservice on a platform. It pays the service provider based on the resources consumed from the platform for each service call. The service provider picks the code and executes it for each request. The execution may happen in any executing sandbox, like a container, VM, or whatever. It is simply hidden from the service itself. The service provider takes care of provisioning, scaling, load-balancing, patching, and securing the underlying infrastructure. Many popular examples of serverless offerings include AWS Lambda, Google Functions, etc. The infrastructure of a serverless deployment platform is very elastic. The platform scales the service to absorb the load automatically. The time spent managing the low-level infrastructure is eliminated. The expenses are also lowered as the microservices provider pays only for the resources consumed for each call. Service Deployment Platform Microservices can also be deployed on application deployment platforms. By providing some high-level services, such platforms clearly abstract out the deployment. The service abstraction can handle non-functional and functional requirements like availability, load balancing, monitoring, and observability for the service instances. The application deployment platform is thoroughly automated. It makes the application deployment quite reliable, fast, and efficient. Examples of such platforms are Docker Swarm, Kubernetes, and Cloud Foundry, which is a PaaS offering. Conclusion Microservices deployment options and offerings are constantly evolving. It's possible that a few more deployment patterns will follow suit. Many of these patterns mentioned above are very popular and are being used by most microservice providers. They are very successful and reliable. But with changing paradigms, administrators are thinking of innovative solutions.

As a software lead and feature developer, I witnessed a team learn an important lesson about resource scaling and the cloud. Spoiler alert: the lesson was that if you’re not careful, scaling can get expensive! The team felt they had no choice but to keep increasing the number of instances for a given service that was under heavy load. When they finished scaling up, the number of instances was several times larger than the number of instances that were configured by default. What they didn’t realize at the time was that—despite the load returning back to a normal state—their extra instances remained in place. Everyone seemed to be okay with this “scale up as much as needed” approach… until they received their next invoice from the cloud provider. This situation got me thinking about Render, a platform I’ve been adopting more and more for some of my projects. It made me wonder just how easy it could be to implement scaling in cloud-based applications and services with Render. Another spoiler alert: it’s easy. The Concept of Cloud-Based Scaling Consumers of your application or service have one common expectation: all of their requests should be handled in a reasonable amount of time. At the same time, solution owners have expectations, which include: Ensuring that customer expectations are met Keeping costs within a planned budget Minimizing downtime and outages—especially those related to performance All of these expectations are easy to meet when the level of demand is less than the max capacity of the technology used to handle each request. When demand begins to exceed those levels, that’s when things get interesting. The challenge is finding that sweet spot that meets expectations and keeps costs reasonable. That’s where the concept of cloud-based scaling comes into play. With cloud-based scaling, the focus is on scaling up the number of service instances to meet the current demand but scaling back down when the demand subsides. A Trio of Scenarios There are three use cases for auto-scaling that we will talk about: Manual scaling Auto-scaling up Auto-scaling down Let’s explore each use case with a scenario example. Manual Scaling The manual scaling concept is for teams with a strong understanding of the demand for their applications or services. As an example, consider an income-tax-related service that answers customers’ questions as they populate their tax returns. The team supporting this service may have decades of information regarding usage patterns, allowing them to determine how many service instances are required throughout the entire year. With this information in hand, the manual scaling approach will allow consumers to be satisfied because the team always knows how many instances ought to be available. Solution owners are pleased because their monthly spend is completely on budget. Of course, this information does not consider major changes to the expected usage patterns. For example, maybe a press release on the service comes out, suddenly impacting demand positively or negatively. Auto-Scaling Up The auto-scaling up approach puts the number of instances into the hands of pre-defined thresholds created by the service owner but enforced by the cloud provider. As those thresholds are exceeded, the number of instances will grow until demand falls to an expected level. Most providers allow users to set a maximum number of instances to cap the number of instances which can ultimately be spawned. While there is some uncertainty regarding the impact on the monthly budget, solution owners might use the rationale that increased demand for their service is often related to new or upgraded subscriptions, which result in additional income. This is where the concept of “you have to spend money to make money” comes into play. When implementing auto-scaling up policies, I always recommend doing the same for auto-scaling down, too. Auto-Scaling Down The auto-scaling down approach is similar to auto-scaling up, except the number of services decreases to coincide with a reduction in demand. While the auto-scaling up functionality is very quick to introduce new instances, auto-scaling down is often delayed to avoid scaling down too soon. Thinking back to the team mentioned in my introduction, had they employed auto-scaling down for the service I mentioned, they would not have encountered the sticker shock of leaving all of those instances running well after the peak demand had subsided. Cloud providers that offer auto-scaling are now getting into the practice of combining auto-scaling up with auto-scaling down since this is the more common implementation of this functionality. Scaling with Render I have written about the Render platform several times this year. Here are links to some of my other publications on the subject: Using Render and Go for the First Time Under the Hood: Render Unified Cloud Purpose-Driven Microservice Design Launch Your Startup Idea in a Day I’ve learned that they take their Zero DevOps promise very seriously. As one might expect, scaling with Render is easy and driven by a simple user interface. With a service running at the Starter plan (or above), the ability to manually scale the number of instances is as simple as sliding to the required level in the Scaling menu of the Render Dashboard: If you’re interested in using auto-scaling with Render, simply enable Autoscaling, then: Select the number of instances Enable and set a target CPU utilization Enable and set a target memory utilization Keep in mind: it is possible to limit auto-scaling to depend on only CPU or memory utilization (instead of both). After implementing auto-scaling, the Render Dashboard communicates as changes are made to the number of instances running: Additionally, metrics are provided to justify the auto-scaling implementation: From a billing perspective, changes to the cost structure are based upon the amount of time the new instances are in place in a given month. This means if you double your number of instances for seven hours on a single day of the billing cycle, the cost for that billing cycle will not double; instead, it will only double for those seven hours where the number of instances doubled. Other Available Integrations Services deployed with Render can also integrate with the following solutions: Datadog: Provides Postgres metrics and log streams into the Datadog observatory platform Scout APM: Provides application performance monitoring (APM) for Ruby, PHP, Python, Node.js, and Elixir-based services These integrations provide insights, which can be helpful to larger, enterprise-scale applications and solutions running on the Render platform. Conclusion Technologists who have worked for less than 13 years have been fortunate not to have to worry about the side effects of a global recession. Present-day economists suggest that the next recession will begin soon, and some economic indicators already justify such claims. This means that corporations are likely to be more conservative with their spending to preserve their bottom line. One area of scrutiny for corporations is cloud spending. I still believe that cloud-based products and services can strongly outweigh the costs to support and maintain similar configurations within a dedicated data center. That said, there are certain aspects that can significantly impact the periodic costs related to cloud-based technology: Having a good understanding of each incurred cost Knowing how and when to scale applications and services to meet demand For those using cloud services from Amazon, Google, or Microsoft, firms like CleanSlate Technology Group offer services to help you with these concerns. Since 2021, I have been trying to live by the following mission statement, which I feel can apply to any technology professional: “Focus your time on delivering features/functionality that extends the value of your intellectual property. Leverage frameworks, products, and services for everything else.” - J. Vester In the time I have been utilizing Render for my own applications and services, I have been able to focus on delivering features strongly because of its Zero DevOps model. For those looking to simplify their cloud architecture, Render provides mission-critical scalability without becoming an expert in the technologies adopted by their competitors. Have a really great day!

This is an article from DZone's 2022 Microservices and Containerization Trend Report.For more: Read the Report Does your organization use a microservices-style architecture to implement its business functionality? What approaches to microservices communication and orchestration do you use? Microservices have been a fairly dominant application architecture for the last few years and are usually coupled with the adoption of a cloud platform (e.g., containers, Kubernetes, FaaS, ephemeral cloud services). Communication patterns between these types of services vary quite a bit. Microservices architectures stress independence and the ability to change frequently, but these services often need to share data and initiate complex interactions between themselves to accomplish their functionality. In this article, we'll take a look at patterns and strategies for microservices communication. Problems in the Network Communicating over the network introduces reliability concerns. Packets can be dropped, delayed, or duplicated, and all of this can contribute to misbehaving and unreliable service-to-service communication. In the most basic case — service A opening a connection to service B — we put a lot of trust in the application libraries and the network itself to open a connection and send a request to the target service (service B in this case). Figure 1: Simple example of service A calling service B But what happens if that connection takes too long to open? What if that connection times out and cannot be open? What if that connection succeeds but then later gets shut down after processing a request, but before a response? We need a way to quickly detect connection or request issues and decide what to do. Maybe if service A cannot communicate with service B, there is some reasonable fallback (e.g., return an error message, static response, respond with a cached value). Figure 2: More complicated example of calling multiple services In a slightly more complicated case, service A might need to call service B, retrieve some values from the service B response, and use it to call service C. If the call to service B succeeds but the call to service C fails, the fallback option may be a bit more complicated. Maybe we can fall back to a predefined response, retry the request, pull from a cache based on some of the data from the service B response, or maybe call a different service? Problems within the network that cause connection or request failures can, and do, happen intermittently and must be dealt with by the applications. These problems become more likely and more complicated with the more service calls orchestrated from a given service, as is seen in Figure 3. Figure 3: Trying to orchestrate multiple service calls across read/write APIs These problems become even more troubling when these calls between services aren't just "read" calls. For example, if service A calls service B, which performs some kind of data mutation that must be coordinated with the next call to service C (e.g., service A tells service B that customer Joe’s address is updated but must also tell service C to change the shipping because of the address change), then these failed calls are significant. This can result in inconsistent data and an inconsistent state across services. Network errors like this impact resilience, data consistency, and likely service-level objectives (SLOs) and service-level agreements (SLAs). We need a way to deal with these network issues while taking into account other issues that crop up when trying to account for failures. Helpful Network Resilience Patterns Building APIs and services to be resilient to network unreliability is not always easy. Services (including the frameworks and libraries used to build a service) can fail due to the network in sometimes unpredictable ways. A few patterns that go a long way to building resilient service communication are presented here but are certainly not the only ones. These three patterns can be used as needed or together to improve communication reliability (but each has its own drawbacks): Retry/backoff retry – if a call fails, send the request again, possibly waiting a period of time before trying again Idempotent request handling – the ability to handle a request multiple times with the same result (can involve de-duplication handling for write operations) Asynchronous request handling – eliminating the temporal coupling between two services for request passing to succeed Let’s take a closer look at each of these patterns. Retries With Backoff Handling Network unreliability can strike at any time and if a request fails or a connection cannot be established, one of the easiest things to do is retry. Typically, we need some kind of bounded number of retries (e.g., "retry two times" vs. just retry indefinitely) and potentially a way to backoff the retries. With backoffs, we can stagger the time we spend between a call that fails and how long to retry. One quick note about retries: We cannot just retry forever, and we cannot configure every service to retry the same number of times. Retries can contribute negatively to "retry storm" events where services are degrading and the calling services are retrying so much that it puts pressure on, and eventually takes down, a degraded service (or keeps it down as it tries to come back up). A starting point could be to use a small, fixed number of retries (say, two) higher up in a call chain and don’t retry the deeper you get into a call chain. Idempotent Request Handling Idempotent request handling is implemented on the service provider for services that make changes to data based on an incoming request. A simple example would be a counter service that keeps a running total count and increments the count based on requests that come in. For example, a request may come in with the value "5," and the counter service would increment the current count by 5. But what if the service processes the request (increments of 5), but somehow the response back to the client gets lost (network drops the packets, connection fails, etc.)? The client may retry the request, but this would then increment the count by 5 again, and this may not be the desired state. What we want is the service to know that it’s seen a particular request already and either disregard it or apply a "no-op." If a service is built to handle requests idempotently, a client can feel safe to retry the failed request with the service able to filter out those duplicate requests. Asynchronous Request Handling For the service interactions in the previous examples, we've assumed some kind of request/response interaction, but we can alleviate some of the pains of the network by relying on some kind of queue or log mechanism that persists a message in flight and delivers it to consumers. In this model, we remove the temporal aspect of both a sender and a receiver of a request being available at the same time. We can trust the message log or queue to persist and deliver the message at some point in the future. Retry and idempotent request handling are also applicable in the asynchronous scenario. If a message consumer can correctly apply changes that may occur in an "at-least once delivery" guarantee, then we don't need more complicated transaction coordination. Essential Tools and Considerations for Service-to-Service Communication To build resilience into service-to-service communication, teams may rely on additional platform infrastructure, for example, an asynchronous message log like Kafka or a microservices resilience framework like Istio service mesh. Handling tasks like retries, circuit breaking, and timeouts can be configured and enforced transparently to an application with a service mesh. Since you can externally control and configure the behavior, these behaviors can be applied to any/all of your applications — regardless of the programming language they've been written in. Additionally, changes can be made quickly to these resilience policies without forcing code changes. Another tool that helps with service orchestration in a microservices architecture is a GraphQL engine. GraphQL engines allow teams to fan out and orchestrate service calls across multiple services while taking care of authentication, authorization, caching, and other access mechanisms. GraphQL also allows teams to focus more on the data elements of a particular client or service call. GraphQL started primarily for presentation layer clients (web, mobile, etc.) but is being used more and more in service-to-service API calls as well. Figure 4: Orchestrating service calls across multiple services with a GraphQL engine GraphQL can also be combined with API Gateway technology or even service mesh technology, as described above. Combining these provides a common and consistent resilience policy layer — regardless of what protocols are being used to communicate between services (REST, gRPC, GraphQL, etc.). Conclusion Most teams expect a cloud infrastructure and microservices architecture to deliver big promises around service delivery and scale. We can set up CI/CD, container platforms, and a strong service architecture, but if we don’t account for runtime microservices orchestration and the resilience challenges that come with it, then microservices are really just an overly complex deployment architecture with all of the drawbacks and none of the benefits. If you’re going down a microservices journey (or already well down the path), make sure service communication, orchestration, security, and observability are at front of mind and consistently implemented across your services. This is an article from DZone's 2022 Microservices and Containerization Trend Report.For more: Read the Report

This is an article from DZone's 2022 Kubernetes in the Enterprise Trend Report.For more: Read the Report For some time, microservices have drawn interest across the architecture and software engineering landscape, and now, applications comprised of microservices have become commonplace. So what exactly is the definition of a microservice? That is somewhat of a loaded question as there is plenty of debate on granularity, segmentation, and what designates a microservice. For the purposes of this discussion, a microservices-based architecture is segmenting an application's units of work into discrete, interoperable components. This is a broad definition, but it is workable in that it identifies two foundational microservice concepts: discrete and interoperable. Along with the technical and business benefits, a microservices-based application architecture brings its own set of challenges. These challenges have been met with solutions ranging from new architectural patterns to the evolution of tech stacks themselves. Kubernetes has become one of the technologies in the tech stack evolution. Deploying microservices using Kubernetes enhances and enforces key principles and patterns while offering additional benefits. It's an Evolution, Not a Revolution As with any technical evolution, taking the next step improves upon what has already been shown to be successful while removing barriers to adoption or execution. Kubernetes is not going to address all microservices challenges, but it does address several pain points. Best Practices Remain In many cases, the development and packaging of microservices destined for Kubernetes deployment is no different than a non-Kubernetes deployment. Non-Kubernetes deployments include bare metal servers, virtual machines, and containerized applications. Applications already packaged for containerized deployment make the step to adopt Kubernetes-managed microservices straightforward. All key microservices patterns, development, and deployment best practices are applied. Application tech stacks and components are unchanged. Continuous integration/continuous delivery (deployment) systems remain intact. Operating system platforms and versions can be tightly controlled. Differences The differences between Kubernetes and non-Kubernetes microservices architectures focus less on the task performed by the microservices and more on the deployment of non-functional requirements. Satisfying non-functional requirements is not a new concept introduced by Kubernetes or even by a microservices architecture. However, through a combination of leveraging the services offered by Kubernetes itself as well as defining cross-cutting application support services, supporting many nonfunctional requirements becomes transparent to an application. The following are two examples. Kubernetes Ingress A Kubernetes Ingress is an example of a configurable service that auto-configures external access to microservices. When a microservice is deployed, it can define whether and how it is to be externally accessed. If a microservice specifies that it is to be externally accessible, the Ingress services within the Kubernetes cluster automatically configure external access, including details such as virtual host definitions and SSL certificates. Figure 1: An Ingress definition supporting two services Here, a Kubernetes Ingress accepts HTTP(S) requests external to the Kubernetes cluster and, based on the request path, routes requests to specific services within the cluster. Operators Kubernetes Operators are a Cloud Native Computing Foundation (CNCF) specification outlining a pattern that supports cross-cutting application services. They behave similarly to a Kubernetes Ingress in that a service is auto-configured based on application specification. The primary difference is that Kubernetes Operators present an abstraction where any type of service is automatically configured to extend the behavior of a Kubernetes cluster. There are Kubernetes Operators that connect applications to logging and metrics systems with the application knowing little of the specifics regarding those systems' implementation. There are also Kubernetes Operators that will build and deploy complete database instances. Figure 2: Kubernetes Operator flow In the diagram above, an application requests that a service be made available for its use. The Kubernetes Operator monitors and watches for requests. When a request is made, the Kubernetes Operator instructs the Kubernetes cluster to deploy or configure a cross-cutting service specific to the application's request. Abstractions Kubernetes provides and supports abstractions over many systems required to satisfy non-functional components. Successful Kubernetes microservices architectures are comprehensive beyond application architecture, considering a strategy to not only address interoperability across microservices but coordination with common services. Applying Kubernetes Constructs to a Microservices Architecture Kubernetes deploys container-based applications; this implies that an artifact of a microservice build and packaging process is a Docker (or suitable alternative) image. In Kubernetes, the basic deployment unit for an image is a Pod. Often there is a one-to-one relationship between a deployed image and a Pod. However, Kubernetes Pods can support multiple deployed images within a single Pod. While the deployed containers do not share a file system, they can reference each other using localhost. Within a Kubernetes cluster, deployed Pods can provide their services to other Pods. This is like a deployed microservice on bare metal or a virtual machine, although this deployment doesn't provide access to the Pod's service from an external resource, high availability, or scalability. As discussed, Kubernetes helps applications meet non-functional requirements. A general rule of thumb is when "-ility" is used to describe a function, it often means a non-functional requirement. Using high availability and scalability as examples, Kubernetes provides these with relative ease. There are a few Kubernetes constructs that support these functions. Two are presented here: Services and Deployments. Kubernetes provides a construct called a Service. A Kubernetes Service specifies ports that a microservice wishes to expose and how they are to be exposed. Services provide two powerful features. First, a Kubernetes Service integrates with the internal Kubernetes DNS service to provide a consistent hostname by which the microservices are accessed within the Kubernetes cluster. In addition, if there are multiple instances of the same microservice Pod, a Kubernetes Service can act as a load balancer across the Pod instances, providing high availability. While Pod instances can be individually deployed, manually monitoring their status is impractical. A common pattern for adding automation to Pod "-ilities" is Kubernetes Deployments. Kubernetes Deployments specify details surrounding Pod definitions and provide several features that support the production deployment of microservices, including: The number of replicas to be maintained Updating the state of declared Pods Rollback to earlier versions Scaling up or down the number of Pods With Pod, Service, and Deployment definitions, a solid microservices architecture is in place. In this microcosm, one piece remains — that is, auto-scaling. With Deployments, scalability is available, but like direct Pod deployments, they are manually controlled. The final component to this architectural pattern is using a HorizontalPodAutoscaler to automatically scale the number of Pod instances based on certain criteria (e.g., CPU usage). This example demonstrates how Kubernetes can take any containerized microservice and, using Kubernetes constructs, satisfy the critical non-functional requirements that most applications require. Assembling the patterns discussed here, the following diagram presents a high-level visual of a Kubernetes microservices deployment pattern: Figure 3: Putting it all together The diagram portrays two microservices, "greensvc" and "bluesvc." Each microservice utilizes a Kubernetes Service to expose its functionality. In addition to providing high availability by load balancing multiple Kubernetes Pods per microservice, the Kubernetes Service maps expose Pod ports to port 80. The definition of a Kubernetes Service also creates DNS entries internal to the Kubernetes cluster (greensvc.ns.cluster.local and bluesvc.ns.cluster.local) that can allow microservices to interoperate. Both microservices are exposed outside the Kubernetes cluster through a Kubernetes Ingress. The configured ingress routes incoming requests to their respective services. Microservices Deployment Patterns Kubernetes provides many constructs and abstractions to support service and application Deployment. While applications differ, there are foundational concepts that help drive a well-defined microservices deployment strategy. Well-designed microservices deployment patterns play into an often-overlooked Kubernetes strength. Kubernetes is independent of runtime environments. Runtime environments include Kubernetes clusters running on cloud providers, in-house, bare metal, virtual machines, and developer workstations. When Kubernetes Deployments are designed properly, deploying to each of these and other environments is accomplished with the same exact configuration. In grasping the platform independence offered by Kubernetes, developing and testing the deployment of microservices can begin with the development team and evolve through to production. Each iteration contributes to the overall deployment pattern. A production deployment definition is no different than a developer's workstation configuration. This pattern provides a level of validation that is difficult to reproduce in any previous pattern and can lead to rapid maturity of an application's delivery cycle. The Kubernetes ecosystem offers tools that support these patterns. The most predominant tool is Helm, which orchestrates the definition, installation, and upgrade of Kubernetes applications. It's through tools such as Helm that the same deployment definition can be executed across multiple runtime environments by simply supplying a set of parameters specific to a runtime environment. These parameters don't change the deployment pattern; rather, they configure the deployment pattern to meet the runtime environment (e.g., configuring the amount of memory to allocate to a process). To learn more about Helm charts, check out the article, "Advanced Guide to Helm Charts for Package Management in Kubernetes." Microservices Deployment in Kubernetes Makes Sense Deploying microservices in Kubernetes is an evolution of microservices architectures. Kubernetes addresses many pain points and challenges in developing and deploying microservices-based applications. Being an evolution implies that it's not a revolution. It's not a rewrite. When designing microservices, in many ways, Kubernetes addresses the question that needs to be answered. Rather than waiting, good Kubernetes design and deployment patterns encourage tackling non-functional requirements early in the development process, leading to an application that will mature much faster. Whether it's Kubernetes or a different deployment platform, the same issues that need to be considered will need to be addressed upfront or later. In software engineering, it's almost always best to consider issues upfront. Kubernetes directly helps in addressing many microservices architectures and deployment challenges. This is an article from DZone's 2022 Kubernetes in the Enterprise Trend Report.For more: Read the Report