Microservices

Distributed tracing is now a staple in the modern observability stack. With the shift to microservices, we needed a new way to observe how our services interacted. Distributed Tracing provides that view by allowing us to do request tracing - i.e., trace a request across the components in our distributed system. Today, Distributed Tracing is used for identifying performance bottlenecks, debugging issues, and understanding how our systems are interacting in production. However, implementing Distributed Tracing is complex, and how much value teams get from it depends a fair bit on how it is implemented. Implementation mechanics like which components are instrumented, the sampling rate, and the quality of trace visualization all influence the value companies get from tracing, which in turn influences developer adoption. Additionally, this space is continuously evolving, with new tools and techniques emerging all the time. In this article, let us look at best practices for distributed tracing in 2023. What Is Distributed Tracing? Distributed Tracing refers to a mechanism that allows us to track a single request as it traverses multiple services in a distributed environment. Why we need distributed tracing To enable this, distributed tracing tools insert a unique trace context (trace ID) into each request's header and implement mechanisms to ensure that the trace context is propagated throughout the request path. Each network call made in the request's path is captured and represented as a span. A span is a basic unit of a trace - it represents a single event within the trace, and a trace can have one or multiple spans. A span consists of log messages, time-related data, and other attributes to provide information about the operation it tracks. Anatomy of a distributed trace Through its unique view, Distributed Tracing unlocks several new use cases/improve existing use cases. It allows us to understand service interdependencies (for example, who is calling my service), identify performance bottlenecks (which specific DB call is degrading my latency?), quickly identify failure points for debugging (which API is causing this 500 issue?) and also have more granular SLOs. Components of a Distributed Tracing System To implement any distributed tracing system, we install four distinct components: Instrumentation library Collector (pre-processor) Storage back-end Visualization layer Today, there are several options available for each of these components - you could use one single platform that does all four above or piece together your distributed tracing framework by using different solutions for different components. Components of a tracing system Instrumentation Library This is the part that is integrated into each application or service. When an application executes, the instrumentation library ensures that traceIDs are added into each request or that trace context (trace ID) is propagated into the next span. The library sends this data to a collector. Collector The collector is an intermediary between the instrumentation library and the storage back-end. It gathers traces, processes them (e.g., aggregating spans, sampling), and prepares them for storage. Storage back-end The storage back-end persists and indexes trace data. It typically uses a distributed storage system capable of handling large volumes of data and allows for efficient querying and retrieval. Visualization Layer This is the user interface of the distributed tracing system. It allows developers and operators to interact with trace data. This layer provides tools for querying, searching, and filtering trace data based on various criteria. It presents the trace data in a visually meaningful way, often as a trace graph or timeline, allowing users to analyze the sequence of events and identify bottlenecks or issues. Implementing Distributed Tracing Systems Is Complex While there are several benefits, implementing distributed tracing systems (especially well) is not yet an easy, "solved" task. It requires that the implementing team make several decisions, and those decisions meaningfully impact the amount of value the rest of the engineering team gets from tracing. It’s not uncommon for companies to implement distributed tracing and pay half a million dollars annually, only to have the average developer use it only twice a year. See below for some best practices in how to implement tracing well. Best Practices for Distributed Tracing Pick OTel for Instrumentation There are several popular open-source tracing frameworks, such as OpenTelemetry, Jaeger, and Zipkin. Today, in 2023, OTel has become somewhat of an obvious choice for the following reasons: Wide coverage: OTel has instrumentation libraries and SDKs for different programming languages and frameworks and has broad coverage now. See here for what OTel supports. Is vendor-neutral: By now, most vendors support OTel instrumentation. So you could instrument with OTel and push the data to any vendor of your choice. You'd have vendor interoperability and portability over time (should you choose to change vendors). This is a list of observability vendors that natively support OTel data, and here's a registry of libraries and plugins for connecting OTel with other vendors. Maturity and stability: OTel has been maturing for several years, with wide community support. It is now the 2nd largest project in the CNCF ecosystem in terms of contributors, next only to Kubernetes itself. The strong community ensures it continues to evolve and add support to new technologies rapidly Leverage Automatic Instrumentation Where Possible OpenTelemetry provides two ways to instrument code into applications and components - manual instrumentation and automation instrumentation. If you're on Kubernetes and if most of your services are on Java, NodeJS, or Python, leverage automatic instrumentation extensively as it reduces implementation effort. Manual instrumentation The OTel code has to be added to the application by the developer, so this requires a code change. Manual instrumentation allows for more customization in terms of spans and traces. Most languages are covered for manual instrumentation - C++, .NET, Go, Java, Python, etc. Refer here for the latest list. Automatic instrumentation This is a way to instrument applications/ services without making code changes or having to recompile the application. An intelligent agent gets attached to an application, reads its activity, and extracts the traces. This is possible if you are on Kubernetes. OTel today supports automatic instrumentation for Java, NodeJS, Python, etc. (refer here for the latest list). Customization of spans and traces is limited with automatic instrumentation (vs. manual instrumentation) but is sufficient for most use cases. Start With Critical Paths and Expand From There It is impractical to instrument every service/ component in large distributed systems in one go, so it is important to thoughtfully pick out which paths to instrument first and how to expand from there. Some guidelines/ principles to follow here: Go Outside-In/ Start Close to the Users It is often best to begin from the outside and move inward. This means starting at the points where a request enters your application, incoming requests from users, or external clients. By starting at the entry points, it is easier to get a holistic view of how requests flow through the system. Pick the Most Critical Paths in the System and Instrument Them First The general guideline is to identify the most important request paths in your system; these may be the ones that are most frequently accessed or have the most significant impact on overall application performance monitoring. Start by instrumenting these critical paths first so you can demonstrate value to the overall organization and then expand from there. Always Instrument Request Paths End-To-End So a Trace Doesn’t Break Whatever paths you choose, ensure that the path is instrumented end-to-end - which means each service and component in the request path is instrumented to propagate the context (TraceID) and generate spans as required. Any gaps result in incomplete or broken traces, which negate the effort invested to instrument upstream services. Be Intentional About Sampling In 99% of the cases, companies want to sample their traces. This is because if you store every single trace, you might be storing and managing a massive amount of data. Let's take an example. Assume each span is 500 bytes (including tagging and logging). If your application is serving 2000 requests per second and has 20 different services, it ends up generating 20MB of data every second, or 72 GB per hour, or 1 TB each day, for a simple 20-service setup. This is why most companies end up storing a sample of the distributed traces. It is important to select the right sampling strategy so you still get visibility into what you care about while having control over costs. Broadly, there are two categories of sampling: 1. Upfront/ Head-Based Sampling This is a simple way to decide which spans to keep before any spans have been generated for a given request. This is called head-based sampling, as the decision is made at the beginning or “head” of the request. Sometimes, it is referred to as unbiased sampling when decisions are made without even looking at the request. Within head-based sampling, there are several mechanisms commonly in use, like below. Probabilistic or fixed rate sampling: Randomly selecting a subset of traces to keep based on a fixed sampling rate - say 1% Rate-Limiting Sampling: Setting a fixed limit on the number of requests to be traced per unit of time. For instance, if the rate limit is set to 100 requests per minute, only the first 100 requests in that minute will be traced. Priority-Based Sampling: Priority-based sampling assigns different priorities to requests, and the sampling rate is adjusted accordingly. Requests with higher priority (e.g., critical transactions) have a higher rate of sampling, and lower priority requests have a lower rate. 2. Tail-Based Sampling Tail sampling is where the decision to sample takes place based on the responses within the trace, e.g., high latency and errors. This method ensures that "interesting" requests are traced, even when overall sampling rates are low. However, tail-based sampling is much harder to implement (vs other simpler methods), as one would have to store in a buffer all traces until the response comes back. This guide covers tail-based sampling in some depth. Most organizations typically resort to a simple head-based probabilistic sampling mechanism, with a rate of 1-3% sampling. See here for how to configure fixed-rate sampling at OTel. Be Selective in Implementing Custom Tracing Distributed tracing is powerful in that it allows us to report custom tracing spans. Custom spans allow us to enrich distributed traces with additional, domain-specific information, making tracing data more meaningful. It’s possible to capture and log error states as part of a span or create child spans that further describe the functioning of a service. Effectively tagged spans can, in turn, significantly reduce the amount of logging statements required by your code. In the context of tracing, breadth refers to the number of services or components being instrumented, while depth refers to the level of detail captured within each span. Striking the right balance between breadth and depth is crucial in implementing an effective tracing mechanism while also controlling costs. In general, it is a good idea to go as broad as possible and to be selective in where you go deep. Integrate Tracing With Your Monitoring and Logging Systems Make sure to connect tracing with existing monitoring and logging systems to make it easier for developers to correlate across the three datasets while troubleshooting. Typically, this is done through: Log Injection: Inject trace IDs/ span IDs directly into logs using log frameworks or libraries. This way, each log message has a traceID that can be used to easily query specific logs. Metrics Tagging: Trace-related tags or labels can be included when recording metrics. These tags can be traceIDs span names or other trace-specific metadata. This enables developers to filter and aggregate metrics around tracing data and makes it easier to understand distributed systems. Protocols like OpenTelemetry already allow you to do this easily. Pick a Modern Trace Visualization Front-End There's a meaningful difference across solutions in terms of the front end. After collecting tracing data, you need to be able to visualize it. A good tracing visualization will allow you to see the flow of tracing requests through a system and identify performance bottlenecks. However, all tracing solutions do not provide an intuitive and user-friendly way to visualize and analyze this data directly. Some tools excel at the collection and storage of tracing data but have basic visualization (e.g., Jaeger, Zipkin, AWS XRay), while others are more focused on providing insights from tracing data and, as a result, have invested in more sophisticated visualization and analytics (e.g., Honeycomb, Lighstep, Helios). Good visualization tools should offer out-of-the-box dashboards that automatically give you service dependency maps, have Gantt and waterfall trace visualizations, and allow for detailed querying and filtering of traces. This article is a well-rounded PoV on visualization in distributed tracing. Explore Next-Generation Tools That Combine AI and Tracing With OTel maturing rapidly, instrumentation has become pretty standardized. Similarly, storage and querying have also become broadly commoditized across the observability industry over the last few years. Today, there is some differentiation in the visualization and analytics layer, although even that is not meaningful. There is an emerging class of solutions that use AI on distributed tracing data to generate inferences on the causes of issues. These solutions also have the most modern tracing stack and make implementation and management dramatically simpler. For example, solutions like ZeroK allow you to do the following: Install distributed tracing across all your components in one go without any code change; all services, DBs, and queues are covered right away using OTel and eBPF. They eliminate the need for sampling- they process 100% of traces and use AI to automatically identify anomalous/"interesting" ones to store (e.g., error traces, high-latency traces). Append the anomalous traces with additional context (e.g., logs) to aid debugging as required. Apply LLMs to these traces to automatically identify likely causes of your production issues. Invest in Developer Onboarding This is an often overlooked but critical factor that will drive the success of distributed tracing in your organization. Remember that distributed tracing is complex, and it is difficult for new developers to get up to speed on how to use it effectively. It is not at all uncommon for companies to have just a handful of power users using the tracing platform, and that to a couple of times a quarter. Developers need to be taught how to interpret trace data, understand the relationships between different microservices, and troubleshoot problems using distributed tracing tools. They must be guided on best practices such as consistent naming conventions, proper instrumentation, and understanding trace context propagation. Planning developer onboarding for distributed tracing is a strategic investment. It not only accelerates the integration of tracing within the system but fosters a culture where developers are active participants in the continuous improvement of system visibility, reliability, and performance. Conclusion We looked at distributed tracing best practices and what you can do to make the journey easier. Distributed tracing is no longer a novelty; it has evolved into a crucial part of the observability stack.

The rise of microservices architecture has changed the way developers build and deploy applications. Spring Cloud, a part of the Spring ecosystem, aims to simplify the complexities of developing and managing microservices. In this comprehensive guide, we will explore Spring Cloud and its features and demonstrate its capabilities by building a simple microservices application. What Is Spring Cloud? Spring Cloud is a set of tools and libraries that provide solutions to common patterns and challenges in distributed systems, such as configuration management, service discovery, circuit breakers, and distributed tracing. It builds upon Spring Boot and makes it easy to create scalable, fault-tolerant microservices. Key Features of Spring Cloud Configuration management: Spring Cloud Config provides centralized configuration management for distributed applications. Service discovery: Spring Cloud Netflix Eureka enables service registration and discovery for better load balancing and fault tolerance. Circuit breaker: Spring Cloud Netflix Hystrix helps prevent cascading failures by isolating points of access between services. Distributed tracing: Spring Cloud Sleuth and Zipkin enable tracing requests across multiple services for better observability and debugging. Building a Simple Microservices Application With Spring Cloud In this example, we will create a simple microservices application consisting of two services: a user-service and an order-service. We will also use Spring Cloud Config and Eureka for centralized configuration and service discovery. Prerequisites Ensure that you have the following installed on your machine: Java 8 or later Maven or Gradle An IDE of your choice Dependencies XML <!-- maven --> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-config-server</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-config</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> OR Groovy //Gradle implementation 'org.springframework.cloud:spring-cloud-config-server' implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-client' implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-server' implementation 'org.springframework.cloud:spring-cloud-starter-config' implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-client' implementation 'org.springframework.boot:spring-boot-starter-web' Step 1: Setting up Spring Cloud Config Server Create a new Spring Boot project using Spring Initializr (https://start.spring.io/) and add the Config Server and Eureka Discovery dependencies. Name the project config-server. Add the following properties to your application.yml file: YAML server: port: 8888 spring: application: name: config-server cloud: config: server: git: uri: https://github.com/your-username/config-repo.git # Replace with your Git repository URL eureka: client: serviceUrl: defaultZone: http://localhost:8761/eureka/ Enable the Config Server and Eureka Client by adding the following annotations to your main class: Java import org.springframework.cloud.config.server.EnableConfigServer; import org.springframework.cloud.netflix.eureka.EnableEurekaClient; @EnableConfigServer @EnableEurekaClient @SpringBootApplication public class ConfigServerApplication { public static void main(String[] args) { SpringApplication.run(ConfigServerApplication.class, args); } } Step 2: Setting up Spring Cloud Eureka Server Create a new Spring Boot project using Spring Initializr and add the Eureka Server dependency. Name the project eureka-server. Add the following properties to your application.yml file: YAML server: port: 8761 spring: application: name: eureka-server eureka: client: registerWithEureka: false fetchRegistry: false Enable the Eureka Server by adding the following annotation to your main class: Java import org.springframework.cloud.netflix.eureka.server.EnableEurekaServer; @EnableEurekaServer @SpringBootApplication public class EurekaServerApplication { public static void main(String[] args) { SpringApplication.run(EurekaServerApplication.class, args); } } Step 3: Creating the User Service Create a new Spring Boot project using Spring Initializr and add the Config Client, Eureka Discovery, and Web dependencies. Name the project user-service. Add the following properties to your bootstrap.yml file: YAML spring: application: name: user-service cloud: config: uri: http://localhost:8888 eureka: client: serviceUrl: defaultZone: http://localhost:8761/eureka/ Create a simple REST controller for the User Service: Java import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.RestController; @RestController public class UserController { @GetMapping("/users/{id}") public String getUser(@PathVariable("id") String id) { return "User with ID: " + id; } } Step 4: Creating the Order Service Create a new Spring Boot project using Spring Initializr and add the Config Client, Eureka Discovery, and Web dependencies. Name the project order-service. Add the following properties to your bootstrap.yml file: YAML spring: application: name: order-service cloud: config: uri: http://localhost:8888 eureka: client: serviceUrl: defaultZone: http://localhost:8761/eureka/ Create a simple REST controller for the Order Service: Java import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.RestController; @RestController public class OrderController { @GetMapping("/orders/{id}") public String getOrder(@PathVariable("id") String id) { return "Order with ID: " + id; } } Step 5: Running the Application Start the config-server, eureka-server, user-service, and order-service applications in the following order. Once all services are running, you can access the Eureka dashboard at http://localhost:8761 and see the registered services. You can now access the User Service at http://localhost:<user-service-port>/users/1 and the Order Service at http://localhost:<order-service-port>/orders/1. Conclusion In this comprehensive guide, we explored Spring Cloud and its features and demonstrated its capabilities by building a simple microservices application. By leveraging the power of Spring Cloud, you can simplify the development and management of your microservices, making them more resilient, scalable, and easier to maintain. Embrace the world of microservices with Spring Cloud and elevate your applications to new heights.

Low Latency? In computing, latency is defined as the length of time to perform some task. This could be the time it takes to respond to an interrupt from hardware or the time it takes for a message sent by one component to be available to its recipient. In many cases, latency is not seen as a primary non-functional concern when designing an application, even when considering performance. Most of the time, after all, computers seem to do their work at speeds that are well beyond human perception, typically using scales of milliseconds, microseconds, or even nanoseconds. The focus is often more on throughput - a measure of how many events can be handled within a given time period. However, basic arithmetic tells us that if a service can handle an event with low latency (for example, microseconds), then it will be able to handle far more events within a given time period, say 1 second, than a service with millisecond event handling latency. This can allow us to avoid, in many cases, the need to implement horizontal scaling (starting new instances) of a service, a strategy that introduces significant complexity into an application and may not even be possible for some workloads. Additionally, there are many application domains where consistently low latency is a critical element of an application’s success, for example: Electronic trading systems must be able to respond to changes in event loads based on market conditions fast enough to take advantage of these before competitors in the market - huge sums of money may be gained by being able to do this (or lost by missing such opportunities). There is not enough time to respond to these load “spikes” by scaling horizontally — which could take up to a second — before opportunities are lost. Systems that monitor equipment, such as those found in the IoT space, need to be able to react to indications from that equipment with minimal delays. Alarms, for example, security or environmental alarms, must be notified and responded to as quickly as possible. The overhead introduced by monitoring itself must be minimal to avoid becoming a factor that affects the data being recorded. Some machine learning or AI algorithms need to react to input data as it arrives or as near to it as possible, making them more effective in areas such as pricing, threat detection, sentiment analysis, or buy/sell decisions. Online gaming software must be able to react to input from potentially large numbers of users, adjusting feedback and strategies in as near real-time as possible. At Chronicle Software, our primary focus is to develop software that minimizes latency. It’s often felt that Java is not a suitable language to use for such software; however, it is possible to achieve latency figures that approach those of lower-level languages such as C++ and Rust. Challenges in Building Low-Latency Software Modern applications tend to be implemented using architectural approaches based on loosely coupled components (microservices) that interact with each other based on asynchronous message passing. Several toolkits and frameworks exist that help in implementing such microservices in Java. However, it is not straightforward to build truly low-latency software that follows this approach. Latency creeps in at many different levels. Existing microservice toolkits tend to focus on the quality of the abstractions provided in order to protect their users from lower-level APIs and features. This higher level of abstraction often comes at the price of the creation of large numbers of intermediate objects, placing a significant load on the memory management subsystem of the JVM — something that is anathema to low-latency coding. Other approaches lean towards stripping away almost all abstractions, exposing developers to the lowest level of detail. While this clearly dispenses with overhead, it pushes more complexity into the application-level code, making it more error-prone and significantly more difficult to maintain and evolve. Even at this level of detail, however, it is often necessary to understand and be able to tune operating system level parameters to achieve consistent low latency. Chronicle Tune is a product that can be used to perform this level of analysis and configuration based on Chronicle’s extensive knowledge and experience in this area. Introducing Chronicle Services Over many years, Chronicle Software has been involved in building libraries, applications, and systems that operate in environments where low latency is critical, primarily in the financial sector. Based on the experience gained in this work, we have developed an architectural approach for constructing low-latency applications based on event-driven microservices. We have created the Chronicle Services framework to support this approach, taking care of necessary software infrastructure and enabling developers to focus on implementing business logic based on their functional requirements. Chronicle Services presents an opinionated view of several of the specialized libraries we have developed to support low-latency applications. Philosophy A key requirement in achieving the strict requirements of minimal latency is the elimination of accidental complexity. Frameworks such as Spring Boot, Quarkus, and Micronaut offer rich sets of abstractions to support the construction of microservices and patterns such as event sourcing and CQRS. These are useful parts of frameworks that are necessarily designed to support general-purpose applications, but they can introduce complexity that should be avoided when building highly focused, low-latency components. Chronicle Services offers a smaller set of abstractions, leading to considerable simplification in the framework, less load on the underlying JVM, and, hence, much smaller overhead in processing events. This leads to a throughput of 1 million events per second for a single service. We have also been able to help customers refactor systems that were required to be run on multiple servers to run on a single server (plus one server for continuity in the event of failure). How It Works There are two key concepts in Chronicle Services: Services and Events. A Service is a self-contained processing component that accepts input from one or more sources and outputs to a single sink. Service input and output are in the form of Events, where an Event is an indication that something has happened. By default, events are transmitted between services using Chronicle Queue, a persisted low-latency messaging framework offering the ability to send messages with latencies of under 1 microsecond. Events are transmitted in a compact proprietary binary format. Encoding and decoding are extremely efficient in terms of both time and space and require no additional code generation on the sending or receiving side. Building a Service The public interface of a Service is defined by the types of Events it expects as input and the types of Events that it outputs. The Service implementation itself provides implementations of handlers for each of the input events. There is a clean separation of the Service from the underlying infrastructure for event delivery, so the developer can focus on implementing the business logic encapsulated in the event handlers. A Service handles all incoming events in a single thread, removing the need for dealing with concurrency, another common source of accidental complexity. Detailed functional testing is available through a powerful testing framework, where input events are supplied as YAML, together with expected output events. Configuration of Services is available through APIs or using a declarative approach based on external files, or even dynamic configuration updates through events. An example of the static configuration file for a simple Services application is shown below: YAML !ChronicleServicesCfg { queues: { sumServiceIn: { path: data/sumServiceIn }, sumServiceOut: { path: data/sumServiceOut }, sumServiceSink: { path: data/sumServiceSink }, }, services: { sumService: { inputs: [ sumServiceIn ], output: sumServiceOut, implClass: !type software.chronicle.services.ex1.services.SumServiceImpl, }, sumUpstream: { inputs: [ ], output: sumServiceIn, implClass: !type software.chronicle.services.ex1.services.SumServiceUpstream, }, sumDownstream: { inputs: [ sumServiceOut ], output: sumServiceSink, implClass: !type software.chronicle.services.ex1.services.SumServiceDownstream, } } } Each Service is defined in terms of its implementation class and the Chronicle Queues that are used for the transmission of Events. There is enough information here for the Chronicle Services runtime to create and start each service. Diagrammatically, the application described in the above file would appear like this: Deploying a Service Chronicle Services supports many options for deploying Services. Multiple services can share a single thread, can be run on multiple threads, or spread across multiple processes. Chronicle Queue is a shared memory-based IPC mechanism, so message exchange between Services in different processes is extremely fast. Services can be further packaged into containers, which can simplify deployment, especially in Cloud environments. Enterprise Class Features Chronicle Services is based on the Enterprise edition of Chronicle Queue, which offers cluster-based replication of event storage, along with other Enterprise features. Replication is based on the single leader/multiple followers model, with both Active/Passive and Active/Active approaches to high availability available in the event of failure of the cluster leader. Chronicle Services applications can also integrate with industry-standard monitoring and observability components such as Prometheus and Grafana to provide visualizations of their operation. For example, we can have a snapshot of the overall state of an application: or specific latency statistics from individual services: Monitoring solutions are described in more detail in this article. Conclusion In order to achieve the best latency figures from an application, it is often necessary to depart from idiomatic techniques for developing in the chosen language. It takes time to acquire the skills to do this effectively, and even if it can be done in the business logic layers of code, supporting frameworks do not always provide the same level of specialization. Chronicle Services is a highly opinionated framework that leverages concepts implemented in libraries that have been developed by Chronicle Software to support the development of asynchronous message-passing applications with market-leading latency performance. It does not aim to compete with general-purpose microservice frameworks like Spring Boot or Quarkus. Instead, it provides a low-latency platform on which business logic can be layered using a simple computational model, bringing the benefits of low latency without the pain.

Building APIs and their related components is often a tedious task, especially when dealing with complex data models and architectures like microservices. Repetitive coding, setting up the configurations, and the overhead of building unit tests can quickly become time-consuming. Here's how AI tools, like ChatGPT, can be a game-changer. Harnessing AI Tools (Chat GPT) In API Development To understand the capabilities of ChatGPT, let's dive into a hands-on example. We're tasked with developing a REST API to manage users' Personally Identifiable Information (PII). The process entails: Database object creation (Stored procedure) Repository (Interface for Database Interaction and Data Access Object) Service (Business Logic Layer) REST Controller (Exposing Endpoints) Database Objects To build a stored procedure, first provide the table structure to ChatGPT. After that, supply a sample stored procedure. This will allow ChatGPT to anchor to the stored procedure pattern and construct a new procedure for the specified table. Below is the table we provided: MySQL CREATE TABLE `user_pii_data_repo` ( `id` bigint NOT NULL AUTO_INCREMENT, `user_id` int DEFAULT NULL, `loan_id` int DEFAULT NULL, `ext_partner_name` varchar(30) NOT NULL, `ext_partner_source` varchar(30) NOT NULL, `phase` varchar(20) NOT NULL, `session_uuid` varchar(36) DEFAULT NULL, `first_name` varchar(300) DEFAULT NULL, `last_name` varchar(300) DEFAULT NULL, `address1` varchar(300) DEFAULT NULL, `address2` varchar(300) DEFAULT NULL, `city` varchar(300) DEFAULT NULL, `state` char(2) DEFAULT NULL, `zip` varchar(300) DEFAULT NULL, `ssn` varchar(300) DEFAULT NULL, `dob` date DEFAULT NULL, `phone` varchar(300) DEFAULT NULL, `payload_path` varchar(500) DEFAULT NULL, PRIMARY KEY (`id`) ); Here is the stored procedure that has been generated: MySQL CREATE PROCEDURE `p_get_user_pii_data`(IN v_user_id INT, IN v_loan_id INT) BEGIN SELECT id, user_id, loan_id, ext_partner_name, ext_partner_source, phase, session_uuid, first_name, last_name, address1, address2, city, state, zip, ssn, dob, phone, payload_path FROM user_pii_data_repo WHERE user_id = v_user_id AND loan_id = v_loan_id; END; Here is the conversation link. Repository (Interface for Database Interaction and Data Access Object) In our endeavor to construct the repository layer, we employed a dedicated ChatGPT thread. We supplied it with the previously mentioned stored procedure and table definition. Leveraging ChatGPT's capabilities, we successfully generated: A JPA entity. A comprehensive Repository class encompassing both the interface and its implementation. Post-generation, we tasked ChatGPT with the creation of unit test cases. Impressively, the outcomes were near-perfect. A few minor adjustments were necessary, predominantly related to namespaces and some other minor details. Below are the generated classes and code: Repository Interface and Implementation Java package com.demo.app.api.core.repository.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import java.util.List; public interface UserPiiDataRepository { List<UserPiiData> getUserPiiData(Integer userId, Integer loanId); } Java package com.demo.app.api.core.repository.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.repository.EntityManagerThreadLocalHelper; import com.demo.app.api.core.services.common.CryptoService; import com.demo.app.api.utils.Utility; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.stereotype.Repository; import javax.persistence.*; import java.util.List; @Repository @Qualifier("userPiiDataRepository") public class UserPiiDataRepositoryImpl implements UserPiiDataRepository { private final Logger logger = LoggerFactory.getLogger(this.getClass()); @Autowired EntityManagerThreadLocalHelper threadLocalHelper; @Autowired CryptoService cryptoService; @Override public List<UserPiiData> getUserPiiData(Integer userId, Integer loanId) { List<UserPiiData> results = null; try { EntityManager entityManager = threadLocalHelper.getEntityManager(); StoredProcedureQuery query = entityManager.createStoredProcedureQuery(UserPiiDataDbConstants.PROC_UW_AP_GET_USER_PII_DATA, UserPiiData.class); query.registerStoredProcedureParameter(UserPiiDataDbConstants.V_PARAM_USER_ID, Integer.class, ParameterMode.IN); query.registerStoredProcedureParameter(UserPiiDataDbConstants.V_PARAM_LOAN_ID, Integer.class, ParameterMode.IN); query.setParameter(UserPiiDataDbConstants.V_PARAM_USER_ID, userId); query.setParameter(UserPiiDataDbConstants.V_PARAM_LOAN_ID, loanId); query.execute(); results = query.getResultList(); for (UserPiiData userPiiData : results) { if (userPiiData != null) { if (userPiiData.getFirstName() != null) { String decryptedFirstName = this.cryptoService.decrypt(userPiiData.getFirstName()); userPiiData.setFirstName(decryptedFirstName); } ..... if (userPiiData.getDob() != null) { String formattedDate = Utility.getFormattedDobMMDDYYYY(userPiiData.getDob()); userPiiData.setFormattedDob(formattedDate); } } entityManager.detach(userPiiData); } threadLocalHelper.closeEntityManager(); } catch(Exception ex) { logger.error("UserPiiDataRepositoryImpl::getUserPiiData - an error occurred while fetching user pii data", ex); threadLocalHelper.closeEntityManager(); } return results; } } Java package com.demo.app.api.core.repository.user_pii; public interface UserPiiDataDbConstants { String PROC_UW_AP_GET_USER_PII_DATA = "p_uw_ap_get_user_pii_data"; String V_PARAM_USER_ID = "v_user_id"; String V_PARAM_LOAN_ID = "v_loan_id"; } Unit Tests Java import static org.junit.jupiter.api.Assertions.*; import static org.mockito.Mockito.*; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.repository.EntityManagerReadOnlyThreadLocalHelper; import com.demo.app.api.core.services.common.CryptoService; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.InjectMocks; import org.mockito.Mock; import org.mockito.MockitoAnnotations; import javax.persistence.EntityManager; import javax.persistence.StoredProcedureQuery; import java.util.Collections; import java.util.Date; import java.util.List; public class UserPiiDataRepositoryImplTest { @InjectMocks private UserPiiDataRepositoryImpl userPiiDataRepository; @Mock private EntityManagerReadOnlyThreadLocalHelper threadLocalHelper; @Mock private CryptoService cryptoService; @Mock private EntityManager entityManager; @Mock private StoredProcedureQuery storedProcedureQuery; @BeforeEach public void setUp() { MockitoAnnotations.openMocks(this); } @Test public void testGetUserPiiData() { // Arrange UserPiiData sampleData = new UserPiiData(); sampleData.setFirstName("encryptedFirstName"); sampleData.setLastName("encryptedLastName"); sampleData.setAddress1("encryptedAddress1"); sampleData.setCity("encryptedCity"); sampleData.setZip("encryptedZip"); sampleData.setSsn("encryptedSsn"); sampleData.setPhone("encryptedPhone"); sampleData.setDob(new Date()); when(threadLocalHelper.getEntityManager()).thenReturn(entityManager); when(entityManager.createStoredProcedureQuery(UserPiiDataDbConstants.PROC_UW_AP_GET_USER_PII_DATA, UserPiiData.class)).thenReturn(storedProcedureQuery); when(storedProcedureQuery.getResultList()).thenReturn(Collections.singletonList(sampleData)); when(cryptoService.decrypt("encryptedFirstName")).thenReturn("John"); when(cryptoService.decrypt("encryptedLastName")).thenReturn("Doe"); when(cryptoService.decrypt("encryptedAddress1")).thenReturn("123 Main St."); when(cryptoService.decrypt("encryptedCity")).thenReturn("Test City"); when(cryptoService.decrypt("encryptedZip")).thenReturn("12345"); when(cryptoService.decrypt("encryptedSsn")).thenReturn("123-45-6789"); when(cryptoService.decrypt("encryptedPhone")).thenReturn("123-456-7890"); // Act List<UserPiiData> result = userPiiDataRepository.getUserPiiData(1, 100); // Assert assertEquals(1, result.size()); UserPiiData fetchedData = result.get(0); assertEquals("John", fetchedData.getFirstName()); assertEquals("Doe", fetchedData.getLastName()); assertEquals("123 Main St.", fetchedData.getAddress1()); assertEquals("Test City", fetchedData.getCity()); assertEquals("12345", fetchedData.getZip()); assertEquals("123-45-6789", fetchedData.getSsn()); assertEquals("123-456-7890", fetchedData.getPhone()); } } During our initial execution, we encountered challenges with the code generated for the repository implementation. The data stored in our database was encrypted, necessitating decryption of the persisted values. We refined the repository layer's code to address this, integrating decryption mechanisms. After implementing the modifications, we supplied ChatGPT with the updated repository code, seeking a corresponding unit test case. Remarkably, ChatGPT furnished us with a perfectly functional unit test case. For those interested in the detailed dialogue, here's the link to our ChatGPT conversation thread. Building Business Service layer To deepen our exploration, we extended our requirements to an existing ChatGPT service sample, specifying dependencies like the JPA entity and the repository interface. The AI tool promptly generated classes. The Service Interface outlined a clear contract for fetching user PII data based on user and loan IDs. Its implementation, UserPIIDataServiceImpl, utilized the JPA repository to fetch the required data, and if null, would return an empty list — a thoughtful touch for avoiding potential null pointer exceptions. The unit test, UserPIIDataServiceImplTest, was comprehensive. Using Mockito for mock objects, the test verified the service's functionality by arranging mock data, calling the service, and asserting the expected results. Below are the generated classes and code: Service Interface and Implementation Java package com.demo.app.api.core.services.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import java.util.List; public interface UserPIIDataService { List<UserPiiData> getUserPiiData(Integer userId, Integer loanId); } Java package com.demo.app.api.core.services.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.repository.user_pii.UserPiiDataRepository; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import java.util.Collections; import java.util.List; @Service public class UserPIIDataServiceImpl implements UserPIIDataService { @Autowired UserPiiDataRepository userPiiDataRepository; @Override public List<UserPiiData> getUserPiiData(Integer userId, Integer loanId) { List<UserPiiData> piiData = userPiiDataRepository.getUserPiiData(userId, loanId); return piiData != null ? piiData : Collections.emptyList(); } } Unit Tests Java package com.demo.app.api.core.services.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.repository.user_pii.UserPiiDataRepository; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.InjectMocks; import org.mockito.Mock; import org.mockito.MockitoAnnotations; import java.util.Arrays; import java.util.List; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.mockito.Mockito.when; public class UserPIIDataServiceImplTest { @Mock private UserPiiDataRepository userPiiDataRepository; @InjectMocks private UserPIIDataServiceImpl userPIIDataService; @BeforeEach public void setup() { MockitoAnnotations.openMocks(this); } @Test public void testGetUserPiiData() { // Arrange Integer userId = 1; Integer loanId = 101; UserPiiData data1 = new UserPiiData(); data1.setUserId(userId); data1.setLoanId(loanId); data1.setFirstName("John"); UserPiiData data2 = new UserPiiData(); data2.setUserId(userId); data2.setLoanId(loanId); data2.setFirstName("Jane"); List<UserPiiData> mockResponse = Arrays.asList(data1, data2); when(userPiiDataRepository.getUserPiiData(userId, loanId)).thenReturn(mockResponse); // Act List<UserPiiData> result = userPIIDataService.getUserPiiData(userId, loanId); // Assert assertEquals(2, result.size()); assertEquals("John", result.get(0).getFirstName()); assertEquals("Jane", result.get(1).getFirstName()); } } Here is the chat thread for building the service layer. Final Step: REST Controller Generation Having set up our foundational layers, it was time to build our REST Controller. For this, we leveraged a previously existing REST Controller sample as a base template. We provided ChatGPT with this sample alongside the service interface and the necessary entity object. In response, ChatGPT generated a RestController that was tailored to our requirements. Below are the generated classes and code: Java package com.demo.app.api.controller.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.services.user_pii.UserPIIDataService; import io.swagger.annotations.Api; import io.swagger.annotations.ApiOperation; import io.swagger.annotations.ApiResponse; import io.swagger.annotations.ApiResponses; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.*; import java.util.List; @CrossOrigin(origins = "*") @RestController @RequestMapping("/api/mis/v1/ap") @Api(tags = "Controller for fetching User's PII data") public class UserPIIDataController { @Autowired UserPIIDataService userPIIDataService; @GetMapping(path = "/pii-data/user/{userId}/loan/{loanId}", produces = "application/json") @ApiOperation("Fetch user's PII data by user ID and loan ID") @ApiResponses(value = { @ApiResponse(code = 200, message = "Successfully fetched user's PII data"), @ApiResponse(code = 400, message = "Bad request"), @ApiResponse(code = 404, message = "Not found"), @ApiResponse(code = 500, message = "Internal server error") }) public List<UserPiiData> getUserPiiDataByUserIdAndLoanId( @PathVariable(name = "userId") Integer userId, @PathVariable(name = "loanId") Integer loanId) { return this.userPIIDataService.getUserPiiData(userId, loanId); } } A standout feature of the auto-generated code was the inclusion of annotations, specifically ApiResponse and ApiOperation. Without ChatGPT's intervention, these essential annotations might have been inadvertently overlooked, which can affect the clarity and documentation quality of the API. Subsequent to the REST Controller generation, we sought ChatGPT's assistance in producing unit test cases. Initially, the generated tests were designed as if they were calling the actual API. However, our intention was to strictly test the controller's code. After providing the necessary instructions to narrow down the focus, ChatGPT effectively delivered the tailored unit test code. Here's a snapshot of the generated unit test: Java package com.demo.app.api.controller.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.services.user_pii.UserPIIDataService; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.InjectMocks; import org.mockito.Mock; import org.mockito.MockitoAnnotations; import java.util.Collections; import java.util.List; import static org.mockito.Mockito.when; import static org.junit.jupiter.api.Assertions.assertEquals; public class UserPIIDataControllerTest { @Mock private UserPIIDataService userPIIDataService; @InjectMocks private UserPIIDataController userPIIDataController; @BeforeEach public void setup() { MockitoAnnotations.openMocks(this); } @Test public void testGetUserPiiDataByUserIdAndLoanId() { Integer userId = 1; Integer loanId = 1; UserPiiData userPiiData = new UserPiiData(); userPiiData.setUserId(userId); userPiiData.setLoanId(loanId); userPiiData.setFirstName("John"); userPiiData.setLastName("Doe"); List<UserPiiData> expectedUserPiiDataList = Collections.singletonList(userPiiData); when(userPIIDataService.getUserPiiData(userId, loanId)).thenReturn(expectedUserPiiDataList); List<UserPiiData> actualUserPiiDataList = userPIIDataController.getUserPiiDataByUserIdAndLoanId(userId, loanId); assertEquals(expectedUserPiiDataList, actualUserPiiDataList); } } Finally, to ensure everything was functioning as anticipated, we conducted a verification of the API endpoint using Postman. Delightfully, the API behaved exactly as expected, showcasing the practical applicability and precision of the auto-generated code. Here is the chat thread. The above example might seem straightforward, but we've also applied this approach to build API endpoints for inserting data observing a consistent pattern. The real advantage shines through when dealing with extensive tables, say with 30 columns. Manually defining stored procedure parameters and constructing entities with a multitude of attributes — each requiring accurate column mapping — can be a tedious and error-prone process. However, leveraging tools like ChatGPT or other AI utilities eliminates these repetitive tasks. As a result, developers can produce more efficient, well-documented code with reduced effort. Conclusion The technological realm is evolving rapidly. With the advent of AI tools like ChatGPT, developers now possess powerful allies in their coding endeavors. By automating the more tedious and repetitive tasks in API development, these tools not only streamline the process but also drastically improve code quality and accuracy. The experiences shared in this article are a testament to the fact that AI's potential to revolutionize software development isn't mere speculation — it's a reality we're beginning to embrace. As we forge ahead, such collaborations between humans and machines will undoubtedly redefine the landscape of software engineering, opening doors to new possibilities and efficiency levels previously thought unattainable.

What Is Monorepo? A monorepo is a single repository with multiple related services, projects, and components, which different teams can use to store code for related or unrelated projects. The term monorepo comes from mono, meaning single, and repo is short for the repository. Benefits of Monorepo Here are some key benefits of using monorepo: Code sharing: The projects share standard code, libraries, or utilities. Reusability: The components need to be reused across different projects. Easier code review: Code reviews are more efficient in monorepo since reviewers can easily see the context of changes across related projects, which can improve code quality and catch potential issues earlier. Simplify CI/CD processes: Releasing multiple projects simultaneously becomes more straightforward with a mono repo. Consistent dependency management: The projects share similar or overlapping dependencies, and you want to manage dependencies centrally. Team collaboration: Teams working on related projects can collaborate more effectively within a monorepo, sharing knowledge, insights, and resources. Microservices architecture: When dealing with a suite of closely related microservices, a monorepo can simplify code sharing, dependency management, and testing across services. Version consistency: When all projects can share a standard versioning schema, simplifying communication and understanding. Libraries and Tools Specifically Designed To Manage Monorepos Using Node.js Lerna: A widely used tool for managing JavaScript projects with multiple packages. Nx: Focused on Angular but adaptable to other frameworks, Nx offers powerful development tools for monorepos, emphasizing efficient workflows, code reuse, and testing. Yarn Workspaces: Yarn's built-in monorepo feature allows managing multiple packages in a single codebase. Rush: A scalable monorepo manager developed by Microsoft, suitable for large codebases. Bolt: A monorepo management tool that focuses on performance and can be faster for certain operations than Lerna. Monorepo Manager: This tool simplifies the creation and maintenance of monorepos, offering a user-friendly interface for managing packages, dependencies, and scripts. pnpm: Like Yarn, pnpm also supports monorepo setups through its workspace feature, reducing duplication and improving disk space utilization with shared dependencies. Each tool provides specific benefits and features, so the choice depends on your project's requirements and preferences. Why Lerna? Lerna is a tool designed for managing repositories that contain multiple npm packages. It simplifies handling dependencies, releases, and publishing packages within multi-package repositories housed in a single git repository. Lerna is specifically useful for monorepo, as it enables efficient code sharing and collaboration among developers working on different npm packages within the same repository. It allows developers to treat projects with multiple packages as a single entity, thereby improving the development lifecycle management. Prerequisite Before Installing Lerna Git: Download and install Git Git Bash (Terminal): If you're using Windows, Git Bash is included with the Git installation; for macOS and Linux, use your system's terminal. Node.js: Download and install Node.js npm: npm is included with Node.js, so once Node.js is installed, npm is available in your terminal. Verify by opening your terminal and typing. npm -v. We're creating a monorepo that includes a payment service utilized by the backend server. Furthermore, the backend server and the payment service will share a logging service. Logging Service: Designed for efficient logging across various services. Payment Service: Responsible for handling payment-related functionalities. Backend Server: Executes payment processing and integrates logging service for seamless operations. Let's now dive into implementing the Monorepo using Lerna. Step 1: Make a Directory and Initialize Lerna Navigate to your project's root directory and initialize Lerna: Shell mkdir monorepo # create a repo directory of the monorepo cd monorepo npx lerna@latest init # initalize the repo The above npx command will create a new Lerna-managed repository.lerna.json : Configuration file contains settings for Lerna's behavior, such as versioning mode, package location, etc package.json: The root package.json file for the entire repository. Shell git config user.name ${username} git config user.email ${email} Step 2: Generate Back-End Package Make sure you are on the root folder of the project. Lerna command for creating package: npx lerna create #{packageName} #{directory} Here, the directory is by default: packages Shell npx lerna create back-end //or //this will skip the questionnaire related to package npx lerna create back-end -y The above command, without -y will prompt you with various questions, such as overwriting the package name, adding a description, and more. However, these details are not crucial for this example, so press "Enter" for each. After running, the package back-end will look like the below: Step 3: Generate Packages for Payment and Logging Services Follow the same procedure again, but specify the directory of the service you want to create since we want the package to be located in the "services/" directory. In the root package.json file, you must also inform Lerna about the packages in the services/ directory. Edit the package.json workspace configuration and add "services/*" to it. The configuration should resemble the following: In the main package.json file at the root level, and you must inform Lerna about the packages within the services/ directory. Modify the workspace configuration package.json and include "services/*". The configuration should appear like this: Shell npx lerna create payment services -y npx lerna create logging services -y Step 4: Set Up Logging Service Within the services/logging directory, set up the logging service using the Bunyan library with a simple configuration. Install the Buyan library within the logging service and Mocha as dev dependency in the root directory for testing all the services. Shell // root folder install test dependencies npm install mocha --save-dev //inside logging cd services/logging npm install bunyan Replace the content of the logging functionality file services/logging/lib/logging.js JavaScript const bunyan = require('bunyan'); const logger = bunyan.createLogger({ name: 'my-logging-service', level: 'info', }); module.exports = logger; Test cases for logging (testing logger): Replace the content of the test file services/logging/__tests__/logging.test.js JavaScript const loggingService = require('../lib/logging'); // Import the logging service describe('Logging Service', () => { it('should log messages', () => { loggingService.info('Test log message'); }); }); Update the test script in package.json of services/logging. Shell "test": "mocha ./__tests__/logging.test.js" package.json should look as the image attached. It's time to run the test using lerna npx lerna run test --scope="logging" With the logging service implementation now in place, let's develop the payment service. Step 5: Set Up Payment Service The Payment Service features a function called makePayment, which accepts a single argument as amount and utilizes the logger service to log the activity. Within the services/payment directory, and set up the payment service with a simple function. Replace the existing scripts with mocha, the provided code snippet for testing purposes. To use the logging service in the payment service, add its dependency to the payment service's package.json, as mentioned below. Then, run npm i in the services/payment directory to install it. JSON "scripts": { "test": "mocha ./__tests__/payment.test.js" }, "dependencies": { "logging": "file:../logging" } package.json should look as the image attached Replace the content of the payment file. services/payment/lib/payment.js JavaScript const loggingService = require('logging'); const paymentService = { makePayment: (amount) => { loggingService.info('Payment processing initiated'); // Implement payment logic here loggingService.info('Payment processed successfully'); return `Payment of ${amount} processed successfully`; }, }; module.exports = paymentService; Test cases for makePayment the function of payment service. Replace the content of the test file services/payment/__tests__/payment.test.js JavaScript const chai = require('chai'); const paymentService = require('../lib/payment'); // Import the payment service const expect = chai.expect; describe('Payment Service', () => { it('should make a payment successfully', () => { const paymentResult = paymentService.makePayment(100); expect(paymentResult).to.equal('Payment of 100 processed successfully'); }); }); It's time to run the test using lerna npx lerna run test --scope="payment" We have completed the implementation of the payment services. Now, let's move on to creating the back-end service. Step 4: Set Up Backend Server We will configure a server with a basic GET API that leverages both the logger and payment services. This setup will facilitate making payments and logging the corresponding activities. Install an Express server and implement functionality to use both services. Shell //from root cd packages/back-end npm install express Replace the content of the logging functionality file packages/back-end/lib/back-end.js We want to use payment and logging service in the server, so let's add the following snippet in dependencies in package.json of packages/back-end JSON "logging": "file:../services/logging", "payment": "file:../services/payment" Replace the scripts block to run the server and test as below: JSON "scripts": { "start": "node ./lib/back-end.js", "test": "mocha ./__tests__/back-end.test.js --exit" } package.json should look as the image attached Now, update the dependencies by executing npm update. Replace the content of packages/back-end/lib/back-end.jswith the following code: We will create a server with get / API on port 3000 and uses both logging and payment services. JavaScript const express = require('express'); const loggingService = require('logging'); const paymentService = require('payment'); const app = express(); app.get('/', (req, res) => { // Use of logging service loggingService.info('Backend server received a request'); // Use the payment service const paymentResult = paymentService.makePayment(100); loggingService.info('Payment result:', paymentResult); res.send('Backend Server: Running!'); }); app.listen(3000, () => { console.log('Backend server is running on port 3000'); }); Install chai-http to unit-test the API on the packages/back-enddirectory. npm i chai-http --save-dev Replace the content of the test file where we will test the API if it is working as expected. JavaScript const chai = require('chai'); const chaiHttp = require('chai-http'); const app = require('../lib/back-end'); // Import the Express app // using request server as chaiHttp; chai.use(chaiHttp); const expect = chai.expect; describe('Backend Server', () => { it('should log a request and process payment', (done) => { chai.request(app) .get('/') .end((err, res) => { expect(res).to.have.status(200); expect(res.text).to.equal('Backend Server: Running!'); done(); }); }); }); Congratulations! This concludes the concise and effective implementation of the three microservices within the monorepo. Step 5: Running the App Great! Now, let's begin the server and observe how all the services come together and works. Execute lerna run start in the root folder. This will initiate the server on port 3000. Open your browser and navigate to localhost:3000/ . You'll observe the output "Backend Server: Running!" displayed in the browser. Inspect the logs in the terminal, and you'll encounter a result similar to what's illustrated in the image. Execute lerna run test, this will run all tests in all microservices as all the microservices has test command in the scripts. Conclusion Creating a monorepo with a backend server, payment service, and logging service highlights the benefits of a unified development approach. This setup facilitates efficient code management and sharing by consolidating related components in a single repository. Integrating the logging service into both the payment service and backend server exemplifies the power of code reusability and consistent logging practices across services. Adopting a monorepo architecture leads to an organized and collaborative development environment. Modularity streamlines development, improving efficiency and long-term maintenance. It provides a sturdy foundation for complex applications featuring transparent communication, code reusability, and effective testing.

Starters are an integral part of the Spring Boot application. In addition to dependency versioning, they provide the ability to describe the configuration for a particular functionality. They gained their popularity due to the development of microservice architecture. When we have dozens or hundreds of services within one application, some of the functionality is not duplicated. For example, we need to connect the cache, database, logging etc. In order not to duplicate the same configuration every time, we simply add the starter as a dependency to our microservice, and Spring recognizes all our configurations. We will try to create our own simple starter on Spring Boot 3. There have been some changes in it compared to the second version. Our starter will have the simplest functionality, namely to output a message to the console when our application starts. We create a project through spring initializer, which we will call according to the custom-spring-boot-starter convention We connect it as a module in our application. In real projects, of course, it will be connected as an external dependency, but it is more convenient for us to do this in one project so that all the code lies in one place. Plug in the module in the settings. gradle file In order for our application to send a message to the console, you can create a Listener that, when updating the context, will output the line “Our spring boot starter is working!”. We also need to implement the onApplicationEvent method, where we will send the message. Java import org.springframework.context.ApplicationListener; import org.springframework.context.event.ContextRefreshedEvent; public class CustomListener implements ApplicationListener<ContextRefreshedEvent> { @Override public void onApplicationEvent(ContextRefreshedEvent event) { System.out.println("Our spring boot starter is working!"); } } Add Listener to the configuration Java import org.springframework.boot.autoconfigure.AutoConfiguration; import org.springframework.context.annotation.Bean; @AutoConfiguration public class CustomConfiguration { @Bean public CustomListener customListener() { return new CustomListener(); } } In order for this configuration to be pulled into our main application, we need to add the META-INF/org. spring framework. boot. autoconfigure file.AutoConfiguration.imports, where the path for the configuration file will be specified. In the Spring 2 version, this file was called Spring. factories and was described in a slightly different way. Thus, we get an inversion of control, since we do not need to connect our starters in the code. We add all our configurations to the file from a new line We add our starter depending on the build file. Gradle We launched the application and it seems that the message was displayed at startup. Our starter has worked successfully. How does it work? We have an annotation @SpringBootApplication that includes @EnableAutoConfiguration. When scanning packages, it finds all the imported files and connects the configurations that are described there. If you want to migrate the starters from SB 2. x to SB 3, you should consider a few additional points. You have to use Java 17+ Replace java import.* on Jakarta.* For declared autoconfiguration, change @Configuration to @AutoConfiguration. Trailing slash in controllers was removed by default The mapping of properties has changed from spring. Redis to spring. data.redist Spring-cloud-sleuth is absorbed by the micrometre project. Now the tracking of requests in the actuator is automatically enabled (that is, health checks, samples, etc. begin to be included in the trace). Prometheus. The properties for enabling the metrics endpoint have changed. Was management: metrics: export: prometheus: enabled: true Become management: prometheus: metrics: export: enabled: true

Spring Cloud is a versatile framework for building Java applications in various cloud environments. Today, we'll explore how to use two components of the framework - Spring Cloud Gateway and Discovery Service (aka Spring Cloud Netflix) - for easy routing of user requests between your Java microservices. We'll build two microservices, register them with a Discovery Service instance, and use the Cloud Gateway for routing requests to a specific microservice instance. The cool thing is that the Cloud Gateway will also be registered with the Discovery Service and will use the latter to resolve a microservice name into an actual connection endpoint. So, whether you prefer reading or watching, let’s walk through this practical example: Creating Sample Microservices Imagine we’re creating an online service for a pizza company. There are two basic capabilities the service needs to support - customers can order a pizza online and then track the order status. To achieve this, let's introduce two microservices - the Kitchen and the Tracker. Kitchen Microservice The Kitchen microservice allows customers to place pizza orders. Once an order is placed, it'll hit the kitchen, and the chef will start cooking. Let's create a basic implementation for the purpose of testing Spring Cloud Gateway with the Discovery Service. This service is a Spring Boot web application with a REST controller that simply acknowledges an order. Java @RestController @RequestMapping("/kitchen") public class KitchenController { @PostMapping("/order") public ResponseEntity<String> addNewOrder(@RequestParam("id") int id) { return ResponseEntity.ok("The order has been placed!"); } } The service will be listening on port 8081, which is set in the application.properties file: Properties files server.port=8081 Once the microservice is started you can use curl or HTTPie to test that the REST endpoint works. We’ll be using HTTPie throughout the article: Shell http POST localhost:8081/kitchen/order id==1 HTTP/1.1 200 Connection: keep-alive Content-Length: 26 Content-Type: text/plain;charset=UTF-8 Date: Thu, 03 Aug 2023 18:45:26 GMT Keep-Alive: timeout=60 The order has been placed! Tracker Microservice Customers use the second microservice, the Tracker, to check their order status. We'll go the extra mile with this service implementation by supporting several order statuses, including ordered, baking, and delivering. Our mock implementation will randomly select one of these statuses: Java @RestController @RequestMapping("/tracker") public class TrackerController { @GetMapping("/status") public ResponseEntity<String> getOrderStatus(@RequestParam("id") int orderId) { String[] status = { "Ordered", "Baking", "Delivering" }; Random rand = new Random(); return ResponseEntity.ok(status[rand.nextInt(status.length)]); } } The Tracker will be listening on port 8082, which is configured in the application.properties file: Properties files server.port=8082 Once the microservice is started, we can test it by sending the following GET request: Shell http GET localhost:8082/tracker/status id==1 HTTP/1.1 200 Connection: keep-alive Content-Length: 10 Content-Type: text/plain;charset=UTF-8 Date: Thu, 03 Aug 2023 18:52:45 GMT Keep-Alive: timeout=60 Delivering Registering Microservices With Spring Cloud Discovery Service Our next step is to register these two microservices with the Spring Cloud Discovery Service. But what exactly is a Discovery Service? Discovery Service The Discovery Service lets your microservices connect to each other using only their names. For instance, if Tracker needs to connect to Kitchen, the Discovery Service gives Tracker the IP addresses of Kitchen's available instances. This list can change - you can add or remove Kitchen instances as needed, and the Discovery Service always keeps the updated list of active endpoints. There are several ways to start a Discovery Service server instance. One of the options is to use the Spring Initializr website to generate a Spring Boot project with the Eureka Server dependency. If you choose that method, the generated project will come with the following class that initiates a server instance of the Discovery Service: Java @SpringBootApplication @EnableEurekaServer public class DiscoveryServerApplication { public static void main(String[] args) { SpringApplication.run(DiscoveryServerApplication.class, args); } } By default, the server listens on port 8761. So, once we start the server, we can visit localhost:8761 to view the Discovery Service dashboard: Currently, the Discovery Service is running, but no microservices are registered with it yet. Now, it's time to register our Kitchen and Tracker microservices. Update the Kitchen Microservice To register the Kitchen service with the Discovery Service, we need to make the following changes: 1. Add the Discovery Service’s client library to the Kitchen’s pom.xml file: XML <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> 2. Annotate the Kitchen’s application class with @EnableDiscoveryClient: Java @SpringBootApplication @EnableDiscoveryClient public class KitchenApplication { public static void main(String[] args) { SpringApplication.run(KitchenApplication.class, args); } } 3. Update the application.properties file by adding these two parameters: Properties files # The microservice will be registered under this name # with the Discovery Service spring.application.name=kitchen-service # Discovery Service address eureka.client.service-url.defaultZone=http://localhost:8761/eureka Update the Tracker Microservice We need to follow the same steps to update the Tracker microservice. There's only one difference for Tracker: its name, which is provided via the spring.application.name property in the application.properties file: Properties files # The microservice will be registered under this name with the Discovery Service spring.application.name=tracker-service Register Microservices Finally, restart the microservices to confirm their registration with the Discovery Service. As expected, both the Kitchen and Tracker microservices successfully register with the Discovery Service! Now, it's time to focus on the Spring Cloud Gateway. Spring Cloud Gateway Spring Cloud Gateway is used to resolve user requests and forward them to the appropriate microservices or API endpoints for further processing. You can generate a Spring Boot project with Cloud Gateway support using the same Spring Initializr website. Simply add the Gateway and Eureka Discovery Client libraries, then click the generate button: The Gateway will serve as a one-stop solution for directing requests to Kitchen and Tracker instances. Its implementation is simple yet powerful: Java @SpringBootApplication @EnableDiscoveryClient public class ApiGatewayApplication { public static void main(String[] args) { SpringApplication.run(ApiGatewayApplication.class, args); } @Bean public RouteLocator routeLocator(RouteLocatorBuilder builder) { return builder.routes() .route("kitchen-route", r -> r.path("/kitchen/**").uri("lb://kitchen-service")) .route("tracker-route", r -> r.path("/tracker/**").uri("lb://tracker-service")) .build(); } } The Gateway supports two routes: The kitchen-route is for all requests beginning with the /kitchen/** path. The tracker-route is for requests starting with the /tracker/** path. The most interesting part of this route configuration is how we define the destination endpoint (the uri(...) part of the configuration). Each destination starts with lb:, followed by a microservice name. When the Gateway encounters such a destination URL, it will use the Discovery Service to resolve a microservice name into an IP address of a microservice instance and establish a connection to it. Furthermore, lb stands for load balancer, which means the Gateway will distribute requests evenly if the Discovery Service returns several instances of the same microservice. Finally, once we initiate the Gateway, it begins to monitor port 8080 for incoming traffic. We can then use the following HTTP requests to confirm that the Gateway is successfully routing our requests to the Kitchen and Tracker microservices! Shell http POST localhost:8080/kitchen/order id==2 HTTP/1.1 200 OK Content-Length: 26 Content-Type: text/plain;charset=UTF-8 Date: Thu, 03 Aug 2023 20:11:26 GMT The order has been placed! http GET localhost:8080/tracker/status id==2 HTTP/1.1 200 OK Content-Length: 6 Content-Type: text/plain;charset=UTF-8 Date: Thu, 03 Aug 2023 20:11:41 GMT Baking Summary With Spring Cloud, building robust, scalable Java applications for the cloud has never been easier. Say goodbye to the hassles of tracking IP addresses and instances, and say hello to intelligent, effortless request routing. Enjoy!

Enterprises nowadays are keen on adopting a microservices architecture, given its agility and flexibility. Containers and the rise of Kubernetes — the go-to container orchestration tool — made the transformation from monolith to microservices easier for them. However, a new set of challenges emerged while using microservices architecture at scale: It became hard for DevOps and architects to manage traffic between services As microservices are deployed into multiple clusters and clouds, data goes out of the (firewall) perimeter and is vulnerable; security becomes a big issue Getting overall visibility into the network topology became a nightmare for SREs. Implementing new security tools, or tuning existing API gateway or Ingress controllers, is just a patchwork and not a complete solution to solve the above problems. What architects need is a radical implementation of their infrastructure to deal with their growing network, security, and observability challenges. And that is where the concept of service mesh comes in. What Is a Service Mesh? A service mesh decouples the communication between services from the application layer to the infrastructure layer. The abstraction at the infrastructure level happens by proxying the traffic between services (see Fig. A). Fig A — Service-to-service communication before and after service mesh implementation The proxy is deployed alongside the application as a sidecar container. The traffic that goes in and out of the service is intercepted by the proxy, and it provides advanced traffic management and security features. On top of it, service mesh provides observability into the overall network topology. In a service mesh architecture, the mesh of proxies is called the data plane, and the controller responsible for configuring and managing the data plane proxies is called the control plane. Why Do You Need a Service Mesh for Kubernetes? While starting off, most DevOps only have a handful of services to deal with. As the applications scale and the number of services increases, managing the network and security becomes complex. Tedious Security Compliance Applications deployed in multiple clusters from different cloud vendors talk to each other over the network. It is essential for such traffic to comply with certain standards to keep out intruders and to ensure secure communication. The problem is that security policies are typically cluster-local and do not work across cluster boundaries. This points to a need for a solution that can enforce consistent security policies across clusters. Chaotic Network Management DevOps engineers would often need to control the traffic flow to services — to perform canary deployments, for example. And they also would want to test the resiliency and reliability of the system by injecting faults and implementing circuit breakers. Achieving such kinds of granular controls over the network requires DevOps engineers to create a lot of configurations and scripting in Kubernetes and the cloud environment. Lack of Visualization Over the Network With applications distributed over a network and communications happening between them, it becomes hard for SREs to keep track of the health and performance of the network infrastructure. This severely impedes their ability to identify and troubleshoot network issues. Implementing service solves the above problems by providing features that make managing applications deployed to Kubernetes painless. Key Features of Service Mesh in Kubernetes Service mesh acts as a centralized platform for networking, security, and observability for microservices deployed into Kubernetes. Centralized Security With a service mesh, security compliance is easier to achieve as it can be done from a central plane instead of configuring it per service. A service mesh platform can enforce consistent security policies that work across cluster boundaries. Service mesh provides granular authentication, authorization, and access control for applications in the mesh. Authentication: mTLS implementation, JWT Authorization: Policies can be set to allow, deny, or perform custom actions against an incoming request Access control: RBAC policies that can be set on method, service, and namespace levels Advanced Networking and Resilience Testing Service mesh provides granular control over the traffic flow between services. DevOps engineers can split traffic between services or route them based on certain weights. Besides, service mesh provides the following features to test the resiliency of the infrastructure with little work: Fault injection Timeouts Retries Circuit breaking Mirroring Unified Observability Implementing service mesh helps SREs and Ops teams to have centralized visibility into the health and performance of applications within the mesh. Service mesh provides the following telemetry for observability and real-time visibility: Metrics: To monitor performance and see latency, traffic, errors, and saturation. Distributed tracing: To understand requests’ lifecycle and analyze service dependency and traffic flow. Access logs: To audit service behavior. Top Service Mesh Software To Consider for Kubernetes One may find various service mesh software such as Istio, Linkerd, HashiCorp Consul, Kong KUMA, Google Anthos (built on Istio), VMware Tanzu, etc., in the market. However, over 90% of the users either use Istio or Linkerd service mesh software because of their strong and vibrant open-source ecosystem for innovation and support. Istio Istio is the most popular, CNCF-graduated open-source service mesh software available. It uses Envoy proxy as sidecars, while the control plane is used to manage and configure them (see Fig.B). Fig B — Istio sidecar architecture Istio provides networking, security, and observability features for applications at scale. Developers from Google, Microsoft, IBM, and others actively contribute to the Istio project. Linkerd Linkerd is a lightweight, open-source service mesh software developed by Buoyant. It provides the basic features of a service mesh and has a destination service, identity service, and proxy injector (see Fig.C). Fig C — Linkerd architecture More than 80% of the contributions to Linkerd are by the founders, Buoyant, itself. (To see a detailed comparison between the two and choose one for your Kubernetes deployments, head to Istio vs Linkerd: The Best Service Mesh for 2023.) Benefits of Service Mesh for Kubernetes Below are some benefits enterprises would reap by implementing service mesh in Kubernetes. 100% Network and Data Security Service mesh helps in maintaining a zero-trust network where requests are constantly authenticated and authorized before processing. DevOps engineers can implement features such as mTLS, which works cluster-wide and across cluster boundaries (see Fig.D). Fig D — Zero trust network implementation A zero-trust network helps in maintaining a secure infrastructure in the current dynamic threat landscape filled with attacks, like man-in-the-middle (MITM) and denial of service (DoS). (If you are interested to learn more, check out this article: Zero Trust Network for Microservices with Istio.) 80% Reduction in Change Failure Rate Nowadays, enterprises release applications to a small set of live users before a complete rollout. It helps DevOps and SREs to analyze the application performance, identify any bugs, and thus avoid potential downtime.Canary and blue/green deployments are two such deployment strategies. The fine-grained traffic controls — including splitting traffic based on weights (see Fig.F) — provided by service mesh make it easier for DevOps engineers to perform them. Fig F — Canary deployment with Istio service mesh 99.99% Available and Resilient Infrastructure The telemetry data provided by service mesh software helps SREs and Ops teams to identify and respond to bugs/threats quickly. Most service mesh software integrates with monitoring tools like Prometheus, Grafana, Kiali, Jaeger, etc., and the dashboard (see Fig.G) provided by them helps operators visualize the health, performance, and behavior of the services. Fig G — Kiali service graph 5X Improvement in Developer Experience Most application developers do not enjoy configuring the network and security logic in their applications. Their focus tends to be on business logic and building features. Implementing service mesh reduces developer toil as they are left alone with the application code. They can offload network and security configurations completely to the service mesh at the infrastructure level. The separation helps developers focus on their core responsibilities, i.e., delivering the business logic. Three Pillars for Successful Implementation of Service Mesh Since service mesh is a radical concept, it can be overwhelming for enterprises to implement and realize its value successfully. If you are an architect or CIO, you would want to consider the following three pillars for successful service mesh implementation. 1. Technology Support It is important to evaluate a service mesh software from the technology support perspective. If you are a mature DevOps organization using various open source and open standards in your CI/CD process, ensure that service mesh software integrates well with your CI/CD tools (of whatever versions). For example, if you are using Argo CD for GitOps deployment, or Prometheus for monitoring, then a service mesh software must be able to integrate with less intervention. 2. Enterprise Support Open-source software adoption is on the rise. But the support for software will be a prime necessity for enterprises to make sure their IT is available for the business. Evaluate a service mesh software that is backed by a large community member (very good for support), and also, there are 3rd party vendor ecosystems that can provide 24*7 support with fixed SLA. 3. Training and Onboarding Support Ensure there are adequate reading materials, documents, and videos available, which will supplement the learning of users of service mesh software because it would not make sense if internal employees such as DevOps and SREs are not able to adopt it. Finally, a service mesh is not just software but is an application operation pattern. Do not hasten your project. Rather, research and evaluate the best service mesh that suits your organizational requirements and needs. Service Mesh Is the Way Forward for Kubernetes Workloads The goal of service mesh implementation is to make managing applications deployed in Kubernetes easier. Adopting it will become a necessity as services start to overflow from a single cluster to multiple clusters. With Kubernetes adoption on the rise, service mesh will eventually become a critical component in most organizations.

A quiet revolution has occurred in the software industry, with many organizations backing away from cloud computing and microservices. These decisions are mainly influenced by cost control and performance. In this post, we look at: The repatriation and consolidation trends. Why it's a step forward, not a step back. The core idea of cloud-nomad architecture. The Discovery Process When a new technology or technique arrives in the software industry, you must imagine its impact on the software you create. You can discover the benefits, limits, and costs only after using the new technology in many different scenarios. We shouldn't call this a hype cycle. It's simply part of the discovery process. Cloud computing and microservices have shared their discovery timeline. Both ideas were around for some time before they caught on, and they each grew rapidly between 2010 and 2020. The result of all that growth is that the industry has developed a stronger sense of where they work best and, most importantly, where they don't work well. This has led to an increasing number of stories where organizations: Move applications away from the cloud to on-premises infrastructure, known as repatriation. Consolidate microservices into fewer macroservices. Crucially, this trend doesn't mean we were wrong to use cloud and microservices. In some cases, we just went too far. Now it's time to rebalance. The Repatriation Trend A survey by Virtana found that 95% of organizations had started a cloud migration, but 72% went on to repatriate applications. One of the key drivers is cost. Not only can on-premises infrastructure work out significantly cheaper over five years, but the costs are also more predictable. The cloud offers a seemingly infinite ability to scale. However, this can lead to unpredictable spending, especially with volatile costs such as egress charges. Though cost is the top reason for repatriation, organizations often report improved performance running their applications on their on-premises infrastructure. A crucial consideration for repatriation is that it's now possible to use cloud-native technology on your private infrastructure, like Infrastructure as Code (IaC) and containers. Moving applications back to on-premises infrastructure or private clouds is a further step forward rather than returning to old ways. Dropbox was an early example of this trend, repatriating over 600 petabytes of data in 2016 for their US-based customers. At the time, they decided to continue using the cloud for European customers to satisfy data residency requirements. More recently, 37signals, the creators of Basecamp and Hey, repatriated workloads with an estimated cost saving of $7 million over five years. They also found significant performance benefits as a result of this move. You don't need to pick a single hosting strategy for all workloads. You can use a mixture of public cloud, private cloud, and on-premises infrastructure, just as you can use multiple cloud vendors. The Microservices Consolidation Trend With microservices, many companies are opting to consolidate them either into fewer macroservices or into a single monolithic application. When organizations return to a monolith, they are taking care to ensure a loosely coupled architecture. When organizations move to macroservices, they pay attention to Conway's Law by organizing teams and services around business capabilities. A sprinkling of domain-driven design and team topologies is crucial here. Mel Conway wrote a 1967 paper titled How Do Committees Invent, where he made a social observation that you can summarize as: Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. Fred Brooks shared this idea in The Mythical Man Month and named it Conway's Law. This book is in our DevOps reading list. Microservices aim to trade some performance in exchange for operational benefits. You can deploy and scale small services independently, but they must pass data out-of-process to other services. There's some sophistication needed to monitor, debug, and troubleshoot a microservice architecture. Organizations must know the trade-offs and take action when the benefits evaporate. For example, Segment (Twilio) found microservices were making it harder to change the code. Dependency management was a nightmare; they found it hard to manage the scaling. Amazon Prime's Video Quality Analysis team consolidated the stream monitoring into a single service. This was a closer match to the team design. Within the service, the architecture is still loosely coupled, but they no longer need to pass the data over the wire to different services. Many people will argue that macroservices represent "microservices done correctly." It might be more appropriate to say they are service-oriented architecture done right. What Is Cloud-Nomad Architecture? With organizations drastically rethinking their approach to architecture and infrastructure, a cloud-nomad architecture provides a way to maintain maximum options. A cloud-native architecture encourages application design that maximizes the benefits of cloud environments. Cloud-nomad architecture takes this further, requiring the software to be easily movable. To be cloud-native, you use technologies like containers, microservices, and immutable infrastructure to build loosely-coupled systems that are easy to manage and operate. Cloud-nomad means you can shift the application between cloud vendors, private clouds, and traditional data centers. Cloud-nomad architecture encompasses the trend toward fewer services you can host anywhere, thanks to modern cloud-native technology. Portable Shelters Ancient hunter-gatherers learned to move to where the food was. Rather than having a fixed home, they migrated based on the seasonal availability of water, plants, and animals. In more modern times, tinkers and traders moved to where they could find new customers. To maintain mobility, nomads developed portable dwellings or temporary shelters like goahti, tipis, and wickiups. The ability to move easily to a new location is central to cloud-nomad architecture. A Step or Two Further Than Cloud-Native The CNCF definition of cloud-native encompasses public and private clouds. You should be able to run your cloud-native application on the public cloud, in a data center, or using on-premises infrastructure. Taking this a step further, to say that it should also be easy to move between these options gives us cloud-nomad architecture. To achieve this, you must avoid depending on vendor-specific features and embrace ephemeral infrastructure. You need infrastructure automation that works across different hosting scenarios. Cloud-nomad architecture also encourages you to balance your microservice architecture against Conway's Law. The complexity will likely outweigh the benefits if you have five teams in your organization and 100 microservices. Many teams found that excessive numbers of services cost too much to run, caused performance problems, and made it harder to change their applications. A cloud-nomad architecture has all the properties of cloud-native architecture. Additionally, it: Is minimally complex Values portability by avoiding vendor-specific dependencies. Automatically provisions infrastructure in a vendor-agnostic way. Like nomadic shelters, your application should be easy to pack up, move, and set up in a new location. When the shelter is too heavy, has too many parts, or depends on finding highly specific replacement parts, it becomes less portable. Conclusion When designing software, it's common for developers to create an abstraction to make it easier to change the selected database. In many cases, the database switch never happens. A far more common scenario is changing where software runs. The core idea of cloud-nomad architecture is ensuring you can move the application without major changes. A key measure of success should be how easy it is to move your workloads to different cloud providers or back to your on-premises infrastructure. Happy deployments!

2023 has seen rapid growth in native-cloud applications and platforms. Organizations are constantly striving to maximize the potential of their applications, ensure seamless user experiences, and drive business growth. The rise of hybrid cloud environments and the adoption of containerization technologies, such as Kubernetes, have revolutionized the way modern applications are developed, deployed, and scaled. In this digital arena, Kubernetes is the platform of choice for most cloud-native applications and workloads, which is adopted across industries. According to a 2022 report, 96% of companies are already either using or evaluating the implementation of Kubernetes in their cloud system. This popular open-source utility is helpful for container orchestration and discovery, load balancing, and other capabilities. However, with this transformation comes a new set of challenges. As the complexity of applications increases, so does the need for robust observability solutions that enable businesses to gain deep insights into their containerized workloads. Enter Kubernetes observability—a critical aspect of managing and optimizing containerized applications in hybrid cloud environments. In this blog post, we will delve into Kubernetes observability, exploring six effective strategies that can empower businesses to unlock the full potential of their containerized applications in hybrid cloud environments. These strategies, backed by industry expertise and real-world experiences, will equip you with the tools and knowledge to enhance the observability of your Kubernetes deployments, driving business success. Understanding Observability in Kubernetes Let us first start with the basics. Kubernetes is a powerful tool for managing containerized applications. But despite its powerful features, keeping track of what's happening in a hybrid cloud environment can be difficult. This is where observability comes in. Observability is collecting, analyzing, and acting on data in a particular environment. In the context of Kubernetes, observability refers to gaining insights into the behavior, performance, and health of containerized applications running within a Kubernetes cluster. Kubernetes Observability is based on three key pillars: 1. Logs: Logs provide valuable information about the behavior and events within a Kubernetes cluster. They capture important details such as application output, system errors, and operational events. Analyzing logs helps troubleshoot issues, understand application behavior, and identify patterns or anomalies. 2. Metrics: Metrics are quantitative measurements that provide insights into a Kubernetes environment's performance and resource utilization. They include CPU usage, memory consumption, network traffic, and request latency information. Monitoring and analyzing metrics help identify performance bottlenecks, plan capacity, and optimize resource allocation. 3. Traces: Traces enable end-to-end visibility into the flow of requests across microservices within a Kubernetes application. Distributed tracing captures timing data and dependencies between different components, providing a comprehensive understanding of request paths. Traces help identify latency issues, understand system dependencies, and optimize critical paths for improved application performance. Kubernetes observability processes typically involve collecting and analyzing data from various sources to understand the system's internal state and provide actionable intelligence. By implementing the right observability strategies, you can gain a deep understanding of your applications and infrastructure, which will help you to: Detect and troubleshoot problems quickly Improve performance and reliability Optimize resource usage Meet compliance requirements Observability processes are being adopted at a rapid pace by IT teams. By 2026, 70% of organizations will have successfully applied observability to achieve shorter latency for decision-making while increasing distributed, organized, and simplified data management processes. 1. Use Centralized Logging and Log Aggregation For gaining insights into distributed systems, centralized logging is an essential strategy. In Kubernetes environments, where applications span multiple containers and nodes, collecting and analyzing logs from various sources becomes crucial. Centralized logging involves consolidating logs from different components into a single, easily accessible location. The importance of centralized logging lies in its ability to provide a holistic view of your system's behavior and performance. With Kubernetes logging, you can correlate events and identify patterns across your Kubernetes cluster, enabling efficient troubleshooting and root-cause analysis. To implement centralized logging in Kubernetes, you can leverage robust log aggregation tools or cloud-native solutions like Amazon CloudWatch Logs or Google Cloud Logging. These tools provide scalable and efficient ways to collect, store, and analyze logs from your Kubernetes cluster. 2. Leverage Distributed Tracing for End-to-End Visibility In a complex Kubernetes environment with microservices distributed across multiple containers and nodes, understanding the flow of requests and interactions between different components becomes challenging. This is where distributed tracing comes into play, providing end-to-end visibility into the execution path of requests as they traverse through various services. Distributed tracing allows you to trace a request's journey from its entry point to all the microservices it touches, capturing valuable information about each step. By instrumenting your applications with tracing libraries or agents, you can generate trace data that reveals each service's duration, latency, and potential bottlenecks. The benefits of leveraging distributed tracing in Kubernetes are significant. Firstly, it helps you understand the dependencies and relationships between services, enabling better troubleshooting and performance optimization. When a request experiences latency or errors, you can quickly identify the service or component responsible and take corrective actions. Secondly, distributed tracing allows you to measure and monitor the performance of individual services and their interactions. By analyzing trace data, you can identify performance bottlenecks, detect inefficient resource usage, and optimize the overall responsiveness of your system. This information is invaluable with regard to capacity planning and ensuring scalability in your Kubernetes environment. Several popular distributed tracing solutions are available. These tools provide the necessary instrumentation and infrastructure to effectively collect and visualize trace data. By integrating these solutions into your Kubernetes deployments, you can gain comprehensive visibility into the behavior of your microservices and drive continuous improvement. 3. Integrate Kubernetes With APM Solutions To achieve comprehensive observability in Kubernetes, it is essential to integrate your environment with Application Performance Monitoring (APM) solutions. APM solutions provide advanced monitoring capabilities beyond traditional metrics and logs, offering insights into the performance and behavior of individual application components. One of the primary benefits of APM integration is the ability to detect and diagnose performance bottlenecks within your Kubernetes applications. With APM solutions, you can trace requests as they traverse through various services and identify areas of high latency or resource contention. Armed with this information, you can take targeted actions to optimize critical paths and improve overall application performance. Many APM solutions offer dedicated Kubernetes integrations that streamline the monitoring and management of containerized applications. These integrations provide pre-configured dashboards, alerts, and instrumentation libraries that simplify capturing and analyzing APM data within your Kubernetes environment. 4. Use Metrics-Based Monitoring Metrics-based monitoring forms the foundation of observability in Kubernetes. It involves collecting and analyzing key metrics that provide insights into your Kubernetes clusters and applications' health, performance, and resource utilization. When it comes to metrics-based monitoring in Kubernetes, there are several essential components to consider: Node-Level Metrics: Monitoring the resource utilization of individual nodes in your Kubernetes cluster is crucial for capacity planning and infrastructure optimization. Metrics such as CPU usage, memory usage, disk I/O, and network bandwidth help you identify potential resource bottlenecks and ensure optimal allocation. Pod-Level Metrics: Pods are the basic units of deployment in Kubernetes. Monitoring metrics related to pods allows you to assess their resource consumption, health, and overall performance. Key pod-level metrics include CPU and memory usage, network throughput, and request success rates. Container-Level Metrics: Containers within pods encapsulate individual application components. Monitoring container-level metrics helps you understand the resource consumption and behavior of specific application services or processes. Metrics such as CPU usage, memory usage, and file system utilization offer insights into container performance. Application-Specific Metrics: Depending on your application's requirements, you may need to monitor custom metrics specific to your business logic or domain. These metrics could include transaction rates, error rates, cache hit ratios, or other relevant performance indicators. Metric-based monitoring architecture diagram 5. Use Custom Kubernetes Events for Enhanced Observability Custom events communicate between Kubernetes components and between Kubernetes and external systems. They can signal important events, such as deployments, scaling operations, configuration changes, or even application-specific events within your containers. By leveraging custom events, you can achieve several benefits in terms of observability: Proactive Monitoring: Custom events allow you to define and monitor specific conditions that require attention. For example, you can create events to indicate when resources are running low, when pods experience failures, or when specific thresholds are exceeded. By capturing these events, you can proactively detect and address issues before they escalate. Contextual Information: Custom events can include additional contextual information that helps troubleshoot and analyze root causes. You can attach relevant details, such as error messages, timestamps, affected resources, or any other metadata that provides insights into the event's significance. This additional context aids in understanding and resolving issues more effectively. Integration with External Systems: Kubernetes custom events can be consumed by external systems, such as monitoring platforms or incident management tools. Integrating these systems allows you to trigger automated responses or notifications based on specific events. This streamlines incident response processes and ensures the timely resolution of critical issues. To leverage custom Kubernetes events, you can use Kubernetes event hooks, custom controllers, or even develop your event-driven applications using the Kubernetes API. By defining event triggers, capturing relevant information, and reacting to events, you can establish a robust observability framework that complements traditional monitoring approaches. 6. Incorporating Synthetic Monitoring for Proactive Observability Synthetic monitoring simulates user journeys or specific transactions that represent everyday interactions with your application. These synthetic tests can be scheduled to run regularly from various geographic locations, mimicking user behavior and measuring key performance indicators. There are several key benefits to incorporating synthetic monitoring in your Kubernetes environment: Proactive Issue Detection: Synthetic tests allow you to detect issues before real users are affected. By regularly simulating user interactions, you can identify performance degradations, errors, or unresponsive components. This early detection enables you to address issues proactively and maintain high application availability. Performance Benchmarking: Synthetic monitoring provides a baseline for performance benchmarking and SLA compliance. You can measure response times, latency, and availability under normal conditions by running consistent tests from different locations. These benchmarks serve as a reference for detecting anomalies and ensuring optimal performance. Geographic Insights: Synthetic tests can be configured to run from different geographic locations, providing insights into the performance of your application from various regions. This helps identify latency issues or regional disparities that may impact user experience. By optimizing your application's performance based on these insights, you can ensure a consistent user experience globally. You can leverage specialized tools to incorporate synthetic monitoring into your Kubernetes environment. These tools offer capabilities for creating and scheduling synthetic tests, monitoring performance metrics, and generating reports. An approach for gaining Kubernetes observability for traditional and microservice-based applications is by using third-party tools like Datadog, Splunk, Middleware, and Dynatrace. This tool captures metrics and events, providing several out-of-the-box reports, charts, and alerts to save time. Wrapping Up This blog explored six practical strategies for achieving Kubernetes observability in hybrid cloud environments. By utilizing centralized logging and log aggregation, leveraging distributed tracing, integrating Kubernetes with APM solutions, adopting metrics-based monitoring, incorporating custom Kubernetes events, and synthetic monitoring, you can enhance your understanding of the behavior and performance of your Kubernetes deployments. Implementing these strategies will provide comprehensive insights into your distributed systems, enabling efficient troubleshooting, performance optimization, proactive issue detection, and improved user experience. Whether you are operating a small-scale Kubernetes environment or managing a complex hybrid cloud deployment, applying these strategies will contribute to the success and reliability of your applications.