Open Source

Asynchronous programming is a cornerstone of modern game development, enabling developers to execute multiple tasks simultaneously without blocking the main thread. This is crucial for maintaining smooth gameplay and enhancing the overall user experience. One of the most powerful yet often misunderstood features for achieving this in Unity is Coroutines. In this article, we will demystify Unity's Coroutines, starting with a foundational understanding of what coroutines are in the broader context of programming. We'll then narrow our focus to Unity's specific implementation of this concept, which allows for simplified yet robust asynchronous programming. By the end of this article, you'll have a solid grasp of how to start, stop, and effectively utilize coroutines in your Unity projects. Whether you're new to Unity or an experienced developer looking to deepen your understanding of this feature, this article aims to provide you with the knowledge you need to make your game development journey smoother and more efficient. Stay tuned as we delve into the ABCs of Unity's Coroutines: from basics to implementation. Understanding Coroutines Coroutines are a fascinating and powerful construct in the world of programming. They allow for a more flexible and cooperative form of multitasking compared to the traditional methods. Before diving into Unity's specific implementation, let's first understand what coroutines are in a general programming context. Coroutines are a type of control structure where the flow control is cooperatively passed between two different routines without returning. In simpler terms, imagine you have two tasks, A and B. Normally, you would complete task A before moving on to task B. However, with coroutines, you can start task A, pause it, switch to task B, and then resume task A from where you left off. This is particularly useful for tasks that don't need to be completed in one go and can yield control to other tasks for better efficiency. Traditional functions and methods have a single entry point and a single exit point. When you call a function, it runs to completion before returning control back to the caller. Coroutines, on the other hand, can have multiple entry and exit points. They can pause their execution at specific points, yield control back to the calling function, and then resume from where they left off. In Unity, this ability to pause and resume makes coroutines extremely useful for a variety of tasks such as animations, timing, and handling asynchronous operations without the complexity of multi-threading. Unity's Take on Coroutines Unity's implementation of coroutines is a bit unique but very much in line with the engine's overall design philosophy of simplicity and ease of use. Unity uses C#'s IEnumerator interface for its coroutine functionality. This allows you to use the `yield` keyword to pause and resume the coroutine's execution. Here's a simple Unity C# example to demonstrate a coroutine: C# using System.Collections; using UnityEngine; public class CoroutineExample : MonoBehaviour { // Start is called before the first frame update void Start() { StartCoroutine(SimpleCoroutine()); } IEnumerator SimpleCoroutine() { Debug.Log("Coroutine started"); yield return new WaitForSeconds(1); Debug.Log("Coroutine resumed"); yield return new WaitForSeconds(1); Debug.Log("Coroutine ended"); } } In this example, the SimpleCoroutine method is defined as an IEnumerator. Inside the coroutine, we use Debug.Log to print messages to the Unity console. The yield return new WaitForSeconds(1); line pauses the coroutine for one second. After the pause, the coroutine resumes and continues its execution. This is a very basic example, but it demonstrates the core concept of how Unity's coroutines work. They allow you to write asynchronous code in a more straightforward and readable manner, without getting into the complexities of multi-threading or callback hell. In summary, Unity's coroutines offer a powerful yet simple way to handle asynchronous programming within the Unity engine. They leverage the IEnumerator interface and the yield keyword to provide a flexible mechanism for pausing and resuming tasks, making it easier to create smooth and responsive games. Unity's Implementation of Coroutines Unity's approach to coroutines is both elegant and practical, fitting well within the engine's broader design philosophy. The implementation is rooted in C#'s IEnumerator interface, which provides the necessary methods for iteration. This allows Unity to use the yield keyword to pause and resume the execution of a coroutine, making it a powerful tool for asynchronous programming within the Unity environment. Explanation of How Unity Has Implemented Coroutines In Unity, coroutines are essentially methods that return an IEnumerator interface. The IEnumerator interface is part of the System.Collections namespace and provides the basic methods for iterating over a collection. However, Unity cleverly repurposes this interface to control the execution flow of coroutines. Here's a simple example to illustrate how Unity's coroutines work: C# using System.Collections; using UnityEngine; public class CoroutineDemo : MonoBehaviour { void Start() { StartCoroutine(MyCoroutine()); } IEnumerator MyCoroutine() { Debug.Log("Coroutine started at time: " + Time.time); yield return new WaitForSeconds(2); Debug.Log("Coroutine resumed at time: " + Time.time); } } In this example, the MyCoroutine method returns an IEnumerator. Inside the coroutine, we log the current time, then use yield return new WaitForSeconds(2); to pause the coroutine for 2 seconds. After the pause, the coroutine resumes, and we log the time again. The IEnumerator interface is the cornerstone of Unity's coroutine system. It provides the methods MoveNext(), Reset(), and the property Current, which are used internally by Unity to control the coroutine's execution. When you use the yield keyword in a coroutine, you're essentially providing a point where the MoveNext() method will pause and later resume the execution. The yield return statement can take various types of arguments to control the coroutine's behavior. For example: yield return null: Waits until the next frame. yield return new WaitForSeconds(float): Waits for a specified time in seconds. yield return new WaitForEndOfFrame(): Waits until the frame's rendering is done. yield return new WWW(string): Waits for a web request to complete. Here's an example that combines multiple yield statements: C# using System.Collections; using UnityEngine; public class MultipleYields : MonoBehaviour { void Start() { StartCoroutine(ComplexCoroutine()); } IEnumerator ComplexCoroutine() { Debug.Log("Started at frame: " + Time.frameCount); yield return null; Debug.Log("One frame later: " + Time.frameCount); Debug.Log("Waiting for 2 seconds..."); yield return new WaitForSeconds(2); Debug.Log("Resumed after 2 seconds."); yield return new WaitForEndOfFrame(); Debug.Log("Waited for end of frame."); } } In this example, the ComplexCoroutine method uses different types of yield statements to control its execution flow. This showcases the flexibility and power of using the IEnumerator interface in Unity's coroutines. In summary, Unity's implementation of coroutines via the IEnumerator interface provides a robust and flexible way to handle asynchronous tasks within your game. Whether you're animating characters, loading assets, or making network calls, coroutines offer a straightforward way to perform these operations without blocking the main thread, thereby keeping your game smooth and responsive. Starting and Stopping Coroutines Understanding how to start and stop coroutines is crucial for effectively managing asynchronous tasks in Unity. The engine provides simple yet powerful methods to control the lifecycle of a coroutine, allowing you to initiate, pause, and terminate them as needed. In this section, we'll delve into these methods and their usage. Starting a coroutine in Unity is straightforward. You use the StartCoroutine() method, which is a member of the MonoBehaviour class. This method takes an IEnumerator as an argument, which is the coroutine you want to start. Here's a basic example: C# using System.Collections; using UnityEngine; public class StartCoroutineExample : MonoBehaviour { void Start() { StartCoroutine(MyCoroutine()); } IEnumerator MyCoroutine() { Debug.Log("Coroutine has started."); yield return new WaitForSeconds(2); Debug.Log("Two seconds have passed."); } } In this example, the Start() method calls StartCoroutine(MyCoroutine()), initiating the coroutine. The MyCoroutine method is defined as an IEnumerator, fulfilling the requirement for the StartCoroutine() method. You can also start a coroutine by passing a string name of the method: C# StartCoroutine("MyCoroutine"); However, this approach is generally less efficient and more error-prone, as it relies on reflection and won't be checked at compile-time. How to Stop a Coroutine Using StopCoroutine() and StopAllCoroutines() Methods Stopping a coroutine is just as important as starting one, especially when you need to manage resources or change the flow of your game dynamically. Unity provides two methods for this: StopCoroutine() and StopAllCoroutines(). StopCoroutine(): This method stops a specific coroutine. You can pass the coroutine's IEnumerator or the string name of the coroutine method to stop it. C# using System.Collections; using UnityEngine; public class StopCoroutineExample : MonoBehaviour { IEnumerator MyCoroutine() { while (true) { Debug.Log("Coroutine is running."); yield return new WaitForSeconds(1); } } void Start() { StartCoroutine(MyCoroutine()); } void Update() { if (Input.GetKeyDown(KeyCode.Space)) { StopCoroutine(MyCoroutine()); Debug.Log("Coroutine has been stopped."); } } } In this example, pressing the spacebar will stop the `MyCoroutine` coroutine, which is running in an infinite loop. StopAllCoroutines(): This method stops all coroutines running on the current MonoBehaviour script. C# using System.Collections; using UnityEngine; public class StopAllCoroutinesExample : MonoBehaviour { IEnumerator Coroutine1() { while (true) { Debug.Log("Coroutine1 is running."); yield return new WaitForSeconds(1); } } IEnumerator Coroutine2() { while (true) { Debug.Log("Coroutine2 is running."); yield return new WaitForSeconds(1); } } void Start() { StartCoroutine(Coroutine1()); StartCoroutine(Coroutine2()); } void Update() { if (Input.GetKeyDown(KeyCode.Space)) { StopAllCoroutines(); Debug.Log("All coroutines have been stopped."); } } } In this example, pressing the spacebar will stop all running coroutines (Coroutine1 and Coroutine2) on the current MonoBehaviour. In summary, Unity provides a straightforward yet powerful set of tools for managing the lifecycle of coroutines. The StartCoroutine() method allows you to initiate them, while StopCoroutine() and StopAllCoroutines() give you control over their termination. These methods are essential for writing efficient and manageable asynchronous code in Unity. Conclusion Coroutines in Unity serve as a powerful tool for handling a variety of tasks in a more manageable and efficient manner. They offer a simplified approach to asynchronous programming, allowing developers to write cleaner, more readable code. By understanding the basics, you've laid the groundwork for diving into more complex and nuanced aspects of using coroutines in Unity. In this article, we've covered the foundational elements: Understanding Coroutines: We started by defining what coroutines are in a general programming context, emphasizing their ability to pause and resume execution, which sets them apart from conventional functions and methods. Unity's Implementation of Coroutines: We delved into how Unity has uniquely implemented coroutines using the `IEnumerator` interface. This implementation allows for the use of the yield keyword, which is central to pausing and resuming coroutine execution. Starting and Stopping Coroutines: Finally, we explored the methods Unity provides for controlling the lifecycle of a coroutine. We discussed how to initiate a coroutine using StartCoroutine() and how to halt its execution using StopCoroutine() and StopAllCoroutines(). As we move forward, there are several advanced topics to explore: Concept of Yielding: Understanding the different types of yield instructions can help you control your coroutines more effectively. This includes waiting for seconds, waiting for the end of the frame, or even waiting for asynchronous operations to complete. Execution Flow: A deeper look into how coroutines affect the overall execution flow of your Unity project can provide insights into optimizing performance and resource management. Practical Use-Cases: Coroutines are versatile and can be used in a myriad of scenarios like animations, AI behavior, procedural generation, and network calls, among others. In the next article, we will delve into these advanced topics, providing you with the knowledge to leverage the full power of Unity's coroutines in your projects. Whether you're a beginner just getting your feet wet or a seasoned developer looking to optimize your code, understanding coroutines is a valuable skill that can elevate your Unity development experience.

Apache Kafka has emerged as a clear leader in corporate architecture for moving from data at rest (DB transactions) to event streaming. There are many presentations that explain how Kafka works and how to scale this technology stack (either on-premise or cloud). Building a microservice using ChatGPT to consume messages and enrich, transform, and persist is the next phase of this project. In this example, we will be consuming input from an IoT device (RaspberryPi) which sends a JSON temperature reading every few seconds. Consume a Message As each Kafka event message is produced (and logged), a Kafka microservice consumer is ready to handle each message. I asked ChatGPT to generate some Python code, and it gave me the basics to poll and read from the named "topic." What I got was a pretty good start to consume a topic, key, and JSON payload. The ChatGPT created code to persist this to a database using SQLAlchemy. I then wanted to transform the JSON payload and use API Logic Server (ALS - an open source project on GitHub) rules to unwarp the JSON, validate, calculate, and produce a new set of message payloads based on the source temperature outside a given range. Shell ChatGPT: “design a Python Event Streaming Kafka Consumer interface” Note: ChatGPT selected Confluent Kafka libraries (and using their Docker Kafka container)- you can modify your code to use other Python Kafka libraries. SQLAlchemy Model Using API Logic Server (ALS: a Python open-source platform), we connect to a MySQL database. ALS will read the tables and create an SQLAlchemy ORM model, a react-admin user interface, safrs-JSON Open API (Swagger), and a running REST web service for each ORM endpoint. The new Temperature table will hold the timestamp, the IoT device ID, and the temperature reading. Here we use the ALS command line utility to create the ORM model: Shell ApiLogicServer create --project_name=iot --db_url=mysql+pymysql://root:password@127.0.0.1:3308/iot The API Logic Server generated class used to hold our Temperature values. Python class Temperature(SAFRSBase, Base): __tablename__ = 'Temperature' _s_collection_name = 'Temperature' # type: ignore __bind_key__ = 'None' Id = Column(Integer, primary_key=True) DeviceId = Column(Integer, nullable=False) TempReading = Column(Integer, nullable=False) CreateDT = Column(TIMESTAMP, server_default=text("CURRENT_TIMESTAMP"), nullable=False) KafkaMessageSent = Column(Booelan, default=text("False")) Changes So instead of saving the Kafka JSON consumer message again in a SQL database (and firing rules to do the work), we unwrap the JSON payload (util.row_to_entity) and insert it into the Temperature table instead of saving the JSON payload. We let the declarative rules handle each temperature reading. Python entity = models.Temperature() util.row_to_entity(message_data, entity) session.add(entity) When the consumer receives the message, it will add it to the session which will trigger the commit_event rule (below). Declarative Logic: Produce a Message Using API Logic Server (an automation framework built using SQLAlchemy, Flask, and LogicBank spreadsheet-like rules engine: formula, sum, count, copy, constraint, event, etc), we add a declarative commit_event rule on the ORM entity Temperature. As each message is persisted to the Temperature table, the commit_event rule is called. If the temperature reading exceeds the MAX_TEMP or less than MIN_TEMP, we will send a Kafka message on the topic “TempRangeAlert”. We also add a constraint to make sure we receive data within a normal range (32 -132). We will let another event consumer handle the alert message. Python from confluent_kafka import Producer conf = {'bootstrap.servers': 'localhostd:9092'} producer = Producer(conf) MAX_TEMP = arg.MAX_TEMP or 102 MIN_TEMP = arg.MIN_TTEMP or 78 def produce_message( row: models.KafkaMessage, old_row: models.KafkaMessage, logic_row: LogicRow): if logic_row.isInserted() and row.TempReading > MAX_TEMP: produce(topic="TempRangeAlert", key=row.Id, value=f"The temperature {row.TempReading}F exceeds {MAX_TEMP}F on Device {row.DeviceId}") row.KafkaMessageSent = True if logic_row.isInserted() and row.TempReading < MIN_TEMP: produce(topic="TempRangeAlert", key=row.Id, value=f"The temperature {row.TempReading}F less than {MIN_TEMP}F on Device {row.DeviceId}") row.KafkaMessageSent = True Rules.constraint(models.Temperature, as_expression= lambda row: row.TempReading < 32 or row.TempReading > 132, error_message= "Temperature {row.TempReading} is out of range" Rules.commit_event(models.Temperature, calling=produce_message) Only produce an alert message if the temperature reading is greater than MAX_TEMP or less than MIN_TEMP. Constraint will check the temperature range before calling the commit event (note that rules are always unordered and can be introduced as specifications change). TDD Behave Testing Using TDD (Test Driven Development), we can write a Behave test to insert records directly into the Temperature table and then check the return value KafkaMessageSent. Behave begins with a Feature/Scenario (.feature file). For each scenario, we write a corresponding Python class using Behave decorators. Feature Definition Plain Text Feature: TDD Temperature Example Scenario: Temperature Processing Given A Kafka Message Normal (Temperature) When Transactions normal temperature is submitted Then Check KafkaMessageSent Flag is False Scenario: Temperature Processing Given A Kafka Message Abnormal (Temperature) When Transactions abnormal temperature is submitted Then Check KafkaMessageSent Flag is True TDD Python Class Python from behave import * import safrs db = safrs.DB session = db.session def insertTemperature(temp:int) -> bool: entity = model.Temperature() entity.TempReading = temp entity.DeviceId = 'local_behave_test' session.add(entity) return entity.KafkaMessageSent @given('A Kafka Message Normal (Temperature)') def step_impl(context): context.temp = 76 assert True @when('Transactions normal temperature is submitted') def step_impl(context): context.response_text = insertTemperature(context.temp) @then('Check KafkaMessageSent Flag is False') def step_impl(context): assert context.response_text == False Summary Using ChatGPT to generate the Kafka message code for both the Consumer and Producer seems like a good starting point. Install Confluent Docker for Kafka. Using API Logic Server for the declarative logic rules allows us to add formulas, constraints, and events to the normal flow of transactions into our SQL database and produce (and transform) new Kafka messages is a great combination. ChatGPT and declarative logic is the next level of "paired programming."

This is an article from DZone's 2023 Kubernetes in the Enterprise Trend Report.For more: Read the Report Cloud-native architecture is a transformative approach to designing and managing applications. This type of architecture embraces the concepts of modularity, scalability, and rapid deployment, making it highly suitable for modern software development. Though the cloud-native ecosystem is vast, Kubernetes stands out as its beating heart. It serves as a container orchestration platform that helps with automatic deployments and the scaling and management of microservices. Some of these features are crucial for building true cloud-native applications. In this article, we explore the world of containers and microservices in Kubernetes-based systems and how these technologies come together to enable developers in building, deploying, and managing cloud-native applications at scale. The Role of Containers and Microservices in Cloud-Native Environments Containers and microservices play pivotal roles in making the principles of cloud-native architecture a reality. Figure 1: A typical relationship between containers and microservices Here are a few ways in which containers and microservices turn cloud-native architectures into a reality: Containers encapsulate applications and their dependencies. This encourages the principle of modularity and results in rapid development, testing, and deployment of application components. Containers also share the host OS, resulting in reduced overhead and a more efficient use of resources. Since containers provide isolation for applications, they are ideal for deploying microservices. Microservices help in breaking down large monolithic applications into smaller, manageable services. With microservices and containers, we can scale individual components separately. This improves the overall fault tolerance and resilience of the application as a whole. Despite their usefulness, containers and microservices also come with their own set of challenges: Managing many containers and microservices can become overly complex and create a strain on operational resources. Monitoring and debugging numerous microservices can be daunting in the absence of a proper monitoring solution. Networking and communication between multiple services running on containers is challenging. It is imperative to ensure a secure and reliable network between the various containers. How Does Kubernetes Make Cloud Native Possible? As per a survey by CNCF, more and more customers are leveraging Kubernetes as the core technology for building cloud-native solutions. Kubernetes provides several key features that utilize the core principles of cloud-native architecture: automatic scaling, self-healing, service discovery, and security. Figure 2: Kubernetes managing multiple containers within the cluster Automatic Scaling A standout feature of Kubernetes is its ability to automatically scale applications based on demand. This feature fits very well with the cloud-native goals of elasticity and scalability. As a user, we can define scaling policies for our applications in Kubernetes. Then, Kubernetes adjusts the number of containers and Pods to match any workload fluctuations that may arise over time, thereby ensuring effective resource utilization and cost savings. Self-Healing Resilience and fault tolerance are key properties of a cloud-native setup. Kubernetes excels in this area by continuously monitoring the health of containers and Pods. In case of any Pod failures, Kubernetes takes remedial actions to ensure the desired state is maintained. It means that Kubernetes can automatically restart containers, reschedule them to healthy nodes, and even replace failed nodes when needed. Service Discovery Service discovery is an essential feature of a microservices-based cloud-native environment. Kubernetes offers a built-in service discovery mechanism. Using this mechanism, we can create services and assign labels to them, making it easier for other components to locate and communicate with them. This simplifies the complex task of managing communication between microservices running on containers. Security Security is paramount in cloud-native systems and Kubernetes provides robust mechanisms to ensure the same. Kubernetes allows for fine-grained access control through role-based access control (RBAC). This certifies that only authorized users can access the cluster. In fact, Kubernetes also supports the integration of security scanning and monitoring tools to detect vulnerabilities at an early stage. Advantages of Cloud-Native Architecture Cloud-native architecture is extremely important for modern organizations due to the evolving demands of software development. In this era of digital transformation, cloud-native architecture acts as a critical enabler by addressing the key requirements of modern software development. The first major advantage is high availability. Today's world operates 24/7, and it is essential for cloud-native systems to be highly available by distributing components across multiple servers or regions in order to minimize downtime and ensure uninterrupted service delivery. The second advantage is scalability to support fluctuating workloads based on user demand. Cloud-native applications deployed on Kubernetes are inherently elastic, thereby allowing organizations to scale resources up or down dynamically. Lastly, low latency is a must-have feature for delivering responsive user experiences. Otherwise, there can be a tremendous loss of revenue. Cloud-native design principles using microservices and containers deployed on Kubernetes enable the efficient use of resources to reduce latency. Architecture Trends in Cloud Native and Kubernetes Cloud-native architecture with Kubernetes is an ever-evolving area, and several key trends are shaping the way we build and deploy software. Let's review a few important trends to watch out for. The use of Kubernetes operators is gaining prominence for stateful applications. Operators extend the capabilities of Kubernetes by automating complex application-specific tasks, effectively turning Kubernetes into an application platform. These operators are great for codifying operational knowledge, creating the path to automated deployment, scaling, and management of stateful applications such as databases. In other words, Kubernetes operators simplify the process of running applications on Kubernetes to a great extent. Another significant trend is the rise of serverless computing on Kubernetes due to the growth of platforms like Knative. Over the years, Knative has become one of the most popular ways to build serverless applications on Kubernetes. With this approach, organizations can run event-driven and serverless workloads alongside containerized applications. This is great for optimizing resource utilization and cost efficiency. Knative's auto-scaling capabilities make it a powerful addition to Kubernetes. Lastly, GitOps and Infrastructure as Code (IaC) have emerged as foundational practices for provisioning and managing cloud-native systems on Kubernetes. GitOps leverages version control and declarative configurations to automate infrastructure deployment and updates. IaC extends this approach by treating infrastructure as code. Best Practices for Building Kubernetes Cloud-Native Architecture When building a Kubernetes-based cloud-native system, it's great to follow some best practices: Observability is a key practice that must be followed. Implementing comprehensive monitoring, logging, and tracing solutions gives us real-time visibility into our cluster's performance and the applications running on it. This data is essential for troubleshooting, optimizing resource utilization, and ensuring high availability. Resource management is another critical practice that should be treated with importance. Setting resource limits for containers helps prevent resource contention and ensures a stable performance for all the applications deployed on a Kubernetes cluster. Failure to manage the resource properly can lead to downtime and cascading issues. Configuring proper security policies is equally vital as a best practice. Kubernetes offers robust security features like role-based access control (RBAC) and Pod Security Admission that should be tailored to your organization's needs. Implementing these policies helps protect against unauthorized access and potential vulnerabilities. Integrating a CI/CD pipeline into your Kubernetes cluster streamlines the development and deployment process. This promotes automation and consistency in deployments along with the ability to support rapid application updates. Conclusion This article has highlighted the significant role of Kubernetes in shaping modern cloud-native architecture. We've explored key elements such as observability, resource management, security policies, and CI/CD integration as essential building blocks for success in building a cloud-native system. With its vast array of features, Kubernetes acts as the catalyst, providing the orchestration and automation needed to meet the demands of dynamic, scalable, and resilient cloud-native applications. As readers, it's crucial to recognize Kubernetes as the linchpin in achieving these objectives. Furthermore, the takeaway is to remain curious about exploring emerging trends within this space. The cloud-native landscape continues to evolve rapidly, and staying informed and adaptable will be key to harnessing the full potential of Kubernetes. Additional Reading: CNCF Annual Survey 2021 CNCF Blog "Why Google Donated Knative to the CNCF" by Scott Carey Getting Started With Kubernetes Refcard by Alan Hohn "The Beginner's Guide to the CNCF Landscape" by Ayrat Khayretdinov This is an article from DZone's 2023 Kubernetes in the Enterprise Trend Report.For more: Read the Report

By Saurabh Dashora CORE

IT teams have been observing applications for their health and performance since the beginning. They observe the telemetry data (logs, metrics, traces) emitted from the application/microservice using various observability tools and make informed decisions regarding scaling, maintaining, or troubleshooting applications in the production environment. If observability is not something new and there are a plethora of monitoring and observability tools available in the market, why bother about OpenTelemetry? What makes it special such that it is getting widely adopted? And most importantly, what is in it for developers, DevOps, and SRE folks? Well, let us find out. What Is OpenTelemetry? OpenTelemetry (OpenTelemetry) provides open-source standards and formats for collecting and exporting telemetry data from microservices for observability purposes. The standardized way of collecting data helps DevOps and SRE engineers use any compatible observability backend of their choice to observe services and infrastructure, without being vendor locked-in. OpenTelemetry diagram for microservices deployed in a Kubernetes cluster OpenTelemetry is both a set of standards and an open-source project that provides components, such as collectors and agents, for its implementation. Besides, OpenTelemetry offers APIs, SDKs, and data specifications for application developers to standardize instrumenting their application code. (Instrumentation is the process of adding observability libraries/dependencies to the application code so that it emits logs, traces, and metrics.) Why Is OpenTelemetry Good News for DevOps and SREs? The whole observability process starts with application developers. Typically, they instrument application code with the proprietary library/agent provided by the observability backend tool that IT teams plan to go with. For example, let us say IT teams want to use Dynatrace as the observability tool. Then, application developers use code/SDKs from Dynatrace to instrument (i.e., to generate and export telemetry data) all the applications in the system. It helps to fetch and feed data in the format Dynatrace is compatible with. But this is where the problem lies. The observability requirements of DevOps and SREs seldom stay the same. They will have to switch between vendors providing observability tools or may want to use more than one tool, as their needs evolve. But, since all the applications are instrumented with the proprietary code from the current vendor, switching becomes a nightmare: The new vendor may prefer collecting telemetry data in a format (tracing format, for example) not compatible with the existing vendor. It means developers will have to rewrite the instrumentation code for all applications. This will have severe overhead in terms of cost, developer effort, and potential service disruptions, depending on the deployments and infrastructure. Non-compatible formats also cause problems with historical data while switching vendors. That is, it becomes hard for DevOps and SREs to analyze the performance before and after the migration. This is where OpenTelemetry proves helpful, and this the reason it is being widely adopted. OpenTelemetry prevents such vendor lock-in by standardizing telemetry data collection and exportation. With OpenTelemetry, developers can send the data to one or more observability backends, be it open-source or proprietary, as it supports most of the leading observability tools. OpenTelemetry Components and Workflow OpenTelemetry provides certain vendor-agnostic components that work together to fetch, process, and export telemetry data to various backends. There are three major components: Instrumentation library, OpenTelemetry Collector, and Exporters. Instrumentation Library OpenTelemetry provides SDKs and libraries for application developers to instrument their code manually or automatically. They support many popular programming languages, such as Java, Python, Ruby, Rust, JavaScript, and more.The instrumentation library is evolving, and developers should check the status of the telemetry data component in the instrumentation library, specific to the programming language they use. OpenTelemetry docs update them frequently. The status at the time of writing this piece is given below: Status of programming language-specific telemetry data support in OpenTelemetry For Kubernetes workloads, OpenTelemetry Operator for Kubernetes can be used to inject auto-instrumentation libraries. OpenTelemetry Collector (OTC) The collector has receiver, processor, and exporter components, which gather, process, and export telemetry data from instrumented applications or infrastructure to observability backends for visualization (refer to the image below). It can receive and export data in various formats, such as its native format (OpenTelemetry Protocol or OTLP), Prometheus, Jaeger, and more. OpenTelemetry Collector components and workflow OTC can be deployed as an agent — either as a sidecar container that runs alongside the application container or as a DaemonSet that runs on each node. And it can be scaled in or out depending on the data throughput. OpenTelemetry Collector is not mandatory since OpenTelemetry is designed to be modular and flexible. IT teams can pick components of their choice as receivers, processors, and exporters or even add custom ones. Exporters They allow developers to configure any compatible backend they want to send the processed telemetry data to. There are open-source and vendor-specific exporters available. Some of them are Apache Skywalking, Prometheus, Datadog, and Dynatrace, which are part of the contrib projects. You can see the complete list of vendors who provide exporters here. The difference Between Trace Data Collected by OpenTelemetry and Istio In a distributed system, tracing is the process of monitoring and recording the lifecycle of a request as it goes through different services in the system. It helps DevOps and SREs visualize the interaction between services and troubleshoot issues, like latency. Istio is one of the most popular service mesh software that provides distributed tracing for observability purposes. In Istio, application containers accompany sidecar containers, i.e., Envoy proxies. The proxy intercepts traffic between services and provides telemetry data for observability (refer to the image below). Istio sidecar architecture and observability Although both OpenTelemetry and Istio provide tracing data, there is a slight difference between them. Istio focuses on the lifecycle of a request as it traverses through multiple services in the system (networking layer) while OpenTelemetry — given that the application is instrumented with the OpenTelemetry library — focuses on the lifecycle of a request as it flows through an application (application layer), interacting with various functions and modules. For example, let us say service A is talking to service B, and the communication has latency issues. Istio can show you which service causes latency and by how much. While this information is enough for DevOps and SREs, it will not help developers debug the part of the application that is causing the problem. This is where OpenTelemetry tracing helps. Since the application is instrumented with the OpenTelemetry library, OpenTelemetry tracing can provide details regarding the specific function of the application that causes latency here. To put it another way, Istio gives traces from outside the application, while OpenTelemetry tracing provides traces from within the application. Istio tracing is good for troubleshooting problems at the networking layer, while OpenTelemetry tracing helps to troubleshoot problems at the application level. OpenTelemetry for Microservices Observability and Vendor Neutrality Enterprises adopting microservices architecture have applications distributed across the cloud, with respective IT teams maintaining them. By instrumenting applications with OpenTelemetry libraries and SDKs, the IT teams are free to choose any compatible observability backend of their choice. The choice will not affect the Ops/SRE teams’ ability to have central visibility into the entire services in the system. OpenTelemetry supports a variety of data formats and seamlessly integrates with most of the open-source and vendor-specific monitoring and observability tools. This also makes switching between vendors painless. Get Started With OpenTelemetry for Istio Service Mesh Watch the following video to learn how to get started with OpenTelemetry for Istio service mesh to achieve observability-in-depth: Additionally, you can go through the blog post, "Integrate Istio and Apache Skywalking for Kubernetes Observability," where the OpenTelemetry collector is used to scrape Prometheus endpoints.

This is an article from DZone's 2023 Kubernetes in the Enterprise Trend Report.For more: Read the Report Kubernetes celebrates its ninth year since the initial release this year, a significant milestone for a project that has revolutionized the container orchestration space. During the time span, Kubernetes has become the de facto standard for managing containers at scale. Its influence can be found far and wide, evident from various architectural and infrastructure design patterns for many cloud-native applications. As one of the most popular and successful open-source projects in the infrastructure space, Kubernetes offers a ton of choices for users to provision, deploy, and manage Kubernetes clusters and applications that run on them. Today, users can quickly spin up Kubernetes clusters from managed providers or go with an open-source solution to self-manage them. The sheer number of these options can be daunting for engineering teams deciding what makes the most sense for them. In this Trend Report article, we will take a look at the current state of the managed Kubernetes offerings as well as options for self-managed clusters. With each option, we will discuss the pros and cons as well as recommendations for your team. Overview of Managed Kubernetes Platforms Managed Kubernetes offerings from the hyperscalers (e.g., Google Kubernetes Engine, Amazon Elastic Kubernetes Service, Azure Kubernetes Service) remain one of the most popular options for administering Kubernetes. The 2019 survey of the Kubernetes landscape from the Cloud Native Computing Foundation (CNCF) showed that these services from each of the cloud providers make up three of the top five options that enterprises use to manage containers. More recent findings from CloudZero illustrating increased cloud and Kubernetes adoption further solidifies the popularity of managed Kubernetes services. All of the managed Kubernetes platforms take care of the control plane components such as kube-apiserver, etcd, kubescheduler, and kube-controller-manager. However, the degree to which other aspects of operating and maintaining a Kubernetes cluster are managed differs for each cloud vendor. For example, Google offers a more fully-managed service with GKE Autopilot, where Google manages the cluster's underlying compute, creating a serverless-like experience for the end user. They also provide the standard mode where Google takes care of patching and upgrading of the nodes along with bundling autoscaler, load balancer controller, and observability components, but the user has more control over the infrastructure components. On the other end, Amazon's offering is more of a hands-off, opt-in approach where most of the operational burden is offloaded to the end user. Some critical components like CSI driver, CoreDNS, VPC CNI, and kube-proxy are offered as managed add-ons but not installed by default. Figure 1: Managed Kubernetes platform comparison By offloading much of the maintenance and operational tasks to the cloud provider, managed Kubernetes platforms can offer users a lower total cost of ownership (especially when using something like a per-Pod billing model with GKE Autopilot) and increased development velocity. Also, by leaning into cloud providers' expertise, teams can reduce the risk of incorrectly setting Kubernetes security settings or fault-tolerance that could lead to costly outages. Since Kubernetes is so complex and notorious for a steep learning curve, using a managed platform to start out can be a great option to fast-track Kubernetes adoption. On the other hand, if your team has specific requirements due to security, compliance, or even operating environment (e.g., bare metal, edge computing, military/medical applications), a managed Kubernetes platform may not fit your needs. Note that even though Google and Amazon have on-prem products (GKE on-prem and EKS anywhere), the former requires VMware's server virtualization software, and the latter is an open-source, self-managed option. Finally, while Kubernetes lends itself to application portability, there is still some degree of vendor lock-in by going with a managed option that you should be aware of. Overview of Self-Managed Kubernetes Options Kubernetes also has a robust ecosystem of self-managing Kubernetes clusters. First, there's the manual route of installing "Kubernetes the Hard Way," which walks through all the steps needed for bootstrapping a cluster step by step. In practice, most teams use a tool that abstracts some of the setup such as kops, kubeadm, kubespray, or kubicorn. While each tool behaves slightly differently, they all automate the infrastructure provisioning, support maintenance functions like upgrades or scaling, as well as integrate with cloud providers and/or bare metal. The biggest advantage of going the self-managed route is that you have complete control over how you want your Kubernetes cluster to work. You can opt to run a small cluster without a highly available control plane for less critical workloads and save on cost. You can customize the CNI, storage, node types, and even mix and match across multiple cloud providers if need be. Finally, self-managed options are more prevalent in non-cloud environments, namely edge or on-prem. On the other hand, operating a self-managed cluster can be a huge burden for the infrastructure team. Even though open-source tools have come a long way to lower the burden, it still requires a non-negligible amount of time and expertise to justify the cost against going with a managed option. PROS AND CONS OF MANAGED vs. SELF-MANAGED KUBERNETES Options Pros Cons Managed Lower TCO Increased development velocity Lean on security best practices Inherit cloud provider's expertise Less maintenance burden Fully customizable to satisfy compliance requirements Can use latest features Flexible deployment schemes Self-managed May not be available on-prem or on the edge Not open to modification Requires support from service provider in case of outage Requires significant Kubernetes knowledge and expertise Maintenance burden can be high Table 1 Considerations for Managed vs. Self-Managed Kubernetes For most organizations running predominantly on a single cloud, going with the managed offering makes the most sense. While there is a cost associated with opting for the managed service, it is a nominal fee ($0.10 per hour per cluster) compared to the engineer hours that may be required for maintaining those clusters. The rest of the cost is billed the same way as using VMs, so cost is usually a non-factor. Also, note that there will still be a non-negligible amount of work to do if you go with a vendor who provides a less-managed offering. There are few use cases where going with a self-managed Kubernetes option makes sense: If you need to run on-prem or on the edge, you may decide that the on-prem offerings from the cloud providers may not fit your needs. If you are running on-prem, likely this means that either cost was a huge factor or there is a tangible need to be on-prem (i.e., applications must run closer to where it's deployed). In these scenarios, you likely already have an infrastructure team with significant Kubernetes experience or the luxury of growing that team in-house. Even if you are not running on-prem, you may consider going with a self-managed option if you are running on multiple clouds or a SaaS provider that must offer a flexible Kubernetes-as-a-Service type of product. While you can run different variants of Kubernetes across clouds, it may be desirable to use a solution like Cluster API to manage multiple Kubernetes clusters in a consistent manner. Likewise, if you are offering Kubernetes as a Service, then you may need to support more than the managed Kubernetes offerings. Also, as mentioned before, compliance may play a big role in the decision. You may need to support an application in regions where major US hyperscalers do not operate in (e.g., China) or where a more locked-down version is required (e.g., military, banking, medical). Finally, you may work in industries where there is a need for either cutting-edge support or massive modifications to fit the application's needs. For example, for some financial institutions, there may be a need for confidential computing. While the major cloud providers have some level of support for them at the time of writing, it is still limited. Conclusion Managing and operating Kubernetes at scale is no easy task. Over the years, the community has continually innovated and produced numerous solutions to make that process easier. On one hand, we have massive support from major hyperscalers for production-ready, managed Kubernetes services. Also, we have more open-source tools to self-manage Kubernetes if need be. In this article, we went through the pros and cons of each approach, breaking down the state of each option along the way. While most users will benefit from going with a managed Kubernetes offering, opting for a self-managed option is not only valid but sometimes necessary. Make sure your team either has the expertise or the resources required to build it in-house before going with the self-managed option. Additional Reading: CNCF Survey 2019: Deployments Are Getting Larger as Cloud Native Adoption Becomes Mainstream "101+ Cloud Computing Statistics That Will Blow Your Mind (Updated 2023)" by Cody Slingerland, Cloud Zero This is an article from DZone's 2023 Kubernetes in the Enterprise Trend Report.For more: Read the Report

By Yitaek Hwang CORE

This is an article from DZone's 2023 Kubernetes in the Enterprise Trend Report.For more: Read the Report Kubernetes, a true game-changer in the domain of modern application development, has revolutionized the way we manage containerized applications. Some people tend to think that Kubernetes is an opposing approach to serverless. This is probably because of the management bound in deploying applications to Kubernetes — the node management, service configuration, load management, etc. Serverless computing, celebrated for its autoscaling power and cost-efficiency, is known for its easy application development and operation. Yet, the complexities Kubernetes introduces have led to a quest for a more automated approach — this is precisely where serverless computing steps into Kubernetes. In this exploration, we'll delve into the serverless trend advantages and highlight key open-source solutions that bridge the gap between serverless and Kubernetes, examining their place in the tech landscape. Factors Driving Kubernetes' Popularity Kubernetes has experienced a meteoric rise in popularity among experienced developers, driven by several factors: Extensibility – Kubernetes offers custom resource definitions (CRDs) that empower developers to define and manage complex application architectures according to their requirements. Ecosystem – Kubernetes fosters a rich ecosystem of tools and services, enhancing its adaptability to various cloud environments. Declarative configuration – Kubernetes empowers developers through declarative configuration, which allows developers to define desired states and lets the system handle the rest. Kubernetes Challenges: Understanding the Practical Complexities That being said, experienced developers navigating the intricate landscape of Kubernetes are familiar with the complexities of setting up, configuring, and maintaining Kubernetes clusters. One of the common challenges is scaling. While manual scaling is becoming a thing of the past, autoscaling has become the de facto standard, with organizations who deploy in Kubernetes benefiting from native autoscaling capabilities such as horizontal pod autoscaling (HPA) and vertical pod autoscaling (VPA). Figure 1: HorizontalPodAutoscaler Nonetheless, these solutions are not without their constraints. HPA primarily relies on resource utilization metrics (e.g., CPU and memory) for scaling decisions. For applications with unique scaling requirements tied to specific business logic or external events, HPA may not provide the flexibility needed. Furthermore, consider the challenge HPA faces in scaling down to zero Pods. Scaling down to zero Pods can introduce complexity and safety concerns. It requires careful handling of Pod termination to ensure that in-flight requests or processes are not disrupted, which can be challenging to implement safely in all scenarios. Understanding Serverless Computing Taking a step back in time to 2014, AWS introduced serverless architectures, which fully exemplified the concept of billing accordingly and using resources only when, how much, and for as long as they're needed. This approach offers two significant benefits: Firstly, it frees up teams from worrying about how applications run, enabling them to concentrate solely on business matters; secondly, it minimizes the hardware, environmental impact, and thus has a positive financial impact on running applications to the absolute minimum. It's essential to understand that "serverless" doesn't imply that there is no server. Instead, it means you don't have to concern yourself with the server responsible for executing tasks; your focus remains solely on the tasks themselves. Serverless Principles in Kubernetes Serverless computing is on the rise, and in some ways, it's getting along with the growing popularity of event-driven architecture, which makes this pairing quite potent. Figure 2: Serverless advantages Event-driven designs are becoming the favored method for creating robust apps that can respond to real-world events in real time. In an event-driven pattern, the crucial requirement is the capability to respond to varying volumes of events at different rates and to dynamically scale your application accordingly. This is where serverless technology perfectly aligns and dynamically scales the application infrastructure accordingly. When you combine the event-driven approach with serverless platforms, the benefits are twofold: You not only save on costs by paying only for what you need, but you also enhance your app's user experience and gain a competitive edge as it syncs with real-world happenings. Who Needs Serverless in Kubernetes? In practical terms, serverless integration in Kubernetes is beneficial for software development teams aiming to simplify resource management and reduce operational complexity. Additionally, it offers advantages to organizations looking to optimize infrastructure costs while maintaining agility in deploying and scaling applications. Explore a Real-World Scenario To illustrate its practicality, imagine a data processing pipeline designed around the producer-consumer pattern. The producer-consumer pattern allows independent operation of producers and consumers, efficient resource utilization, and scalable concurrency. By using a buffer and coordination mechanisms, it optimizes resource usage and ensures orderly data processing. In this architectural context, Kubernetes and KEDA demonstrate significant potential. Producer-Consumer Serverless Architecture With KEDA The system works as the following — producers generate data that flows into a message queue, while consumers handle this data asynchronously. KEDA dynamically fine tunes the count of consumer instances in response to changes within the message queue's activity, ensuring optimal resource allocation and performance. Figure 3: Producer-consumer architecture based on KEDA This efficient serverless architecture includes: Message queue – Selected message queue system that is compatible with Kubernetes. Once chosen, it has to be configured to enable accessibility for both producers and consumers. Producer – The producer component is a simple service that is responsible for generating the tasks and pushing data into the message queue. Consumer– Consumer applications are capable of pulling data asynchronously from the message queue. These applications are designed for horizontal scalability to handle increased workloads effectively. The consumers are deployed as Pods in Kubernetes and utilized by KEDA for dynamic scaling based on queue activity. It's essential to note that while the application operates under KEDA's management, it remains unaware of this fact. Other than that, in this kind of dynamic scale, it is important to highly prioritize robust error handling, retries, and graceful shutdown procedures within the consumer application to ensure reliability and fault tolerance. KEDA– The KEDA system contains scalers that are tailored to the message queue and scaling rules that cater to the system's unique requirements. KEDA offers multiple options to configure the events delivery to the consumers on various metrics such as queue length, message age, or other relevant indicators. For example, if we choose setting the queueLength as target and if one Pod can effectively process 10 messages, you can set the queueLength target to 10. In practical terms, this means that if the actual number of messages in the queue exceeds this threshold, say it's 50 messages, the scaler will automatically scale up to five Pods to efficiently handle the increased workload. Other than that, an upper limit can be configured by the maxReplicaCount attribute to prevent excessive scaling. The triggers are configured by the following format: triggers: - type: rabbitmq metadata: host: amqp://localhost:5672/vhost protocol: auto mode: QueueLength value: "100.50" activationValue: "10.5" queueName: testqueue unsafeSsl: true Let's go over this configuration: It sets up a trigger for RabbitMQ queue activity. This monitors the testqueue and activates when the queue length exceeds the specified threshold of 100.50. When the queue length drops below 10.5, the trigger deactivates. The configuration includes the RabbitMQ server's connection details, using the auto protocol detection and potentially unsafe SSL settings. This setup enables automated scale in response to queue length changes. The architecture achieves an effortlessly deployable and intelligent solution, allowing the code to concentrate solely on essential business logic without the distraction of scalability concerns. This was just an example; the producer-consumer serverless architecture can be implemented through a variety of robust tools and platforms other than KEDA. Let's briefly explore another solution using Knative. Example: Architecture Based on Knative The implementation of the Knative-based system distinguishes itself by assuming the responsibility for data delivery management, in contrast to KEDA, which does not handle data delivery and requires you to set up data retrieval. Prior to deployment of the Knative-based system, it is imperative to ensure its environment is equipped with Knative Serving and Eventing components. Figure 4: Producer-consumer architecture based on Knative The architecture can include: Message broker – Selected message queue that seamlessly integrates as a Knative Broker like Apache Kafka or RabbitMQ. Producer – The producer component is responsible for generating the tasks and dispatching them to a designated message queue within the message broker, implemented as Knative Service. Trigger – The Knative trigger establishes the linkage between the message queue and the consumer, ensuring a seamless flow of messages from the broker to the consumer service. Consumer – The consumer component is configured to efficiently capture these incoming messages from the queue through the Knative trigger, implemented as Knative Service. All of this combined results in an event-driven data processing application that leverages Knative's scaling capabilities. The application automatically scales and adapts to the ever-evolving production requirements of the real world. Indeed, we've explored solutions that empower us to design and construct serverless systems within Kubernetes. However, the question that naturally arises is: What's coming next for serverless within the Kubernetes ecosystem? The Future of Serverless in Kubernetes The future of serverless in Kubernetes is undeniably promising, marked by recent milestones such as KEDA's acceptance as a graduated project and Knative's incubating project status. This recognition highlights the widespread adoption of serverless concepts within the Kubernetes community. Furthermore, the robust support and backing from major industry players underscores the significance of serverless in Kubernetes. Large companies have shown their commitment to this technology by providing commercial support and tailored solutions. It's worth highlighting that the open-source communities behind projects like KEDA and Knative are the driving force behind their success. These communities of contributors, developers, and users actively shape the projects' futures, fostering innovation and continuous improvement. Their collective effort ensures that serverless in Kubernetes remains dynamic, responsive, and aligned with the ever-evolving needs of modern application development. In short, these open-source communities promise a bright and feature-rich future for serverless within Kubernetes, making it more efficient, cost-effective, and agile. This is an article from DZone's 2023 Kubernetes in the Enterprise Trend Report.For more: Read the Report

The Ballerina GraalVM image, a performance-enhanced version of a Ballerina application that can be run as a native executable, offers improved performance and reduced memory consumption. However, like any application in production, issues can arise that impact its performance and reliability. In this article, we will explore how to analyze and resolve production issues that can occur with the Ballerina GraalVM image by delving into essential tools. We will discuss generating heap dumps and connecting a Ballerina GraalVM native application to Java Flight Recorder (JFR) for effective issue diagnosis and resolution. Ballerina GraalVM Image is a native executable of a Ballerina application generated with the GraalVM native image tool. It leverages the capabilities of the GraalVM platform to compile the Ballerina code into a standalone executable, resulting in optimized performance and reduced memory footprint. To fully utilize the benefits of the Ballerina GraalVM Image, it is essential to be aware of potential production issues that can affect its performance. In a production environment, various unexpected issues can impact the performance and reliability of the Ballerina GraalVM Image. Some common issues include memory leaks, deadlocks, and performance bottlenecks. Understanding and resolving these issues is crucial for ensuring the smooth operation of Ballerina GraalVM Image applications. To analyze and resolve production issues, we rely on specific tools. Two essential tools for diagnosing and resolving issues with Ballerina GraalVM Image are heap dumps and Java Flight Recorder (JFR). Heap dumps provide snapshots of an application’s memory, capturing information about object allocation and memory usage, while JFR is a powerful profiling and diagnostic tool that provides detailed runtime information. In this article, we will explore how to effectively use these tools with a Ballerina GraalVM Image. We will discuss generating heap dumps and demonstrate how to connect a Ballerina GraalVM native application to Java Flight Recorder. To illustrate the usage of these tools, we will work with a simple Ballerina program that can cause an out-of-memory issue, allowing us to explore the diagnosis and resolution process in practical scenarios. Follow the following steps to create the application: Download and install Ballerina Swan Lake 2201.7.0 or greater Install GraalVM and configure it appropriately Install Visual Studio Code with Ballerina extension Open the command terminal in VS code and execute the following command to create a Ballerina application : $ bal new ballerina_oom Go to the main.bal file inside the ballerina_oom folder and replace the content with the following: Java public function main() { string[] arr = []; while true { arr.push("value"); } } For this sample, we will discuss the following topics: Generating heap dumps Connecting to Java Flight Recorder Generating Heap Dumps With the Executable We can create a heap dump of a running Ballerina GraalVM executable to monitor its execution. Similar to the heap dump of a Java application, it can be opened with the VisualVM tool. To enable heap dump support, the Ballerina GraalVM executable should be built with the --enable-monitoring=heapdump option. Currently, heap dumps can be created in two different ways: Create heap dumps sending a SIGUSR1 signal at run time. Dump the initial heap of the GraalVM image using the -XX:HeapDumpAndExit run-time option. (The GraalVM native image consists of two parts: the required libs and an initial heap, which is built at the native image build process.) Create heap dumps with VisualVM. Create Heap Dumps Sending a SIGUSR1 Signal at Run Time GraalVM native image does not support generating heap dumps using Attach API yet. Currently, it is not possible to obtain heap dumps using JVM tools like jmap or jcmd . But we can send a SIGUSR1 signal to the process, which will automatically generate heap dumps for us. Let’s build the Ballerina application with the --enable-monitoring=heapdump option. Run the following command inside the ballerina_oom folder. $ bal build --graalvm --graalvm-build-options="--enable-monitoring=heapdump" Note: The native image options can also be added in the Ballerina.toml . For more information, see Configure GraalVM native image build options. Run the executable built in the target/bin directory. $ ./target/bin/ballerina_oom Find the process id of the application by running the following command. $ ps PID TTY TIME CMD 4116 ttys001 0:11.62 ./target/bin/ballerina_oom Send the SIGUSR1 signal to the process using the following command. $ kill -SIGUSR1 4116 The heap dump will be created with the following name while the application continues to run: svm-heapdump-<process-id>-<timestamp>.hprof. The heap dump can be opened using VisualVM or Eclipse Memory Analyzer. Dump the Initial Heap of the Executable Now, let’s try to obtain the initial heap of the native image to see which objects are generated during the native image build process. Run the following command: $ ./target/bin/ballerina_oom -XX:+DumpHeapAndExit The initial heap will be dumped as ballerina_oom.hprof in the current directory. Heap dump opened in VisualVM Create Heap Dumps With Visual VM A convenient way to generate heap dumps is to use VisualVM. For this, you need to build with the --enable-monitoring=heapdump,jvmstat native image option. The additional jvmstat option will allow VisualVM to pick up and list running native executable processes. We can request the heap dumps just like Java processes(right-click on the process, then select Heap Dump). Let’s build the Ballerina GraalVM executable by enabling heap dump and jvmstat. Run the following command inside the ballerina_oom folder: $ bal build --graalvm --graalvm-build-options="--enable-monitoring=heapdump,jvmstat" Run the executable and check the process in the VisualVM window. $ ./target/bin/ballerina_oom Create the heap dump from the running process For more information on generating a heap dump for a GraalVM native image, see Create heap dump. Note: Generating heap dump for a GraalVM native image is not supported on Windows. Generating heap dump when there is an Out-Of-Memory error is not yet supported with GraalVM images. For more information, see this GitHub issue. Ballerina GraalVM executable supports a set of run time arguments. To see all run the native executable with the following argument: -XX:PrintFlags=. Connecting to Java Flight Recorder(JFR) To build a Ballerina GraalVM executable with the JFR event support, we have to build the Ballerina GraalVM executable with the --enable-monitoring=jfr option. Furthermore, at run time, we can start a recording, and configure logging using the following options: -XX:+FlightRecorder: use to enable JFR at run time -XX:StartFlightRecording: use to start a recording on the application’s startup -XX:FlightRecorderLogging: use to configure the log output for the JFR system Let's build the Ballerina GraalVM executable by enabling VM inspection. Run the following command inside the ballerina_oom folder: $ bal build --graalvm --graalvm-build-options="--enable-monitoring=jfr" Let's run the Ballerina executable while starting a recording on the startup (to stop the process with OOM, we can set the max heap size at the runtime using the -XX:MaxHeapSize= option). $ ./target/bin/ballerina_oom -XX:+FlightRecorder -XX:StartFlightRecording="filename=ballerina_recording.jfr" -XX:MaxHeapSize=250 The recording will be created in the current directory with the name specified. To view the contents of the recording file, run this command: $ jfr print ballerina_recording.jfr The recording can be opened using VisualVM as well. We can also incorporate JDK Mission control with JFR to monitor events, start flight recorders and generate heap dumps by specifying the -- enable-montioring=all option at build-time. This option will enable heapdump, jvmstat and jfr. Monitoring the Native-Image Process in JDK Mission Control For more information on running a GraalVM native image with JFR, see Build and Run Native Executables with JFR. In conclusion, the Ballerina GraalVM Image offers developers the ability to unlock enhanced performance and reduced memory consumption for their Ballerina applications. However, like any production environment, issues can arise that impact the performance and reliability of these applications. In this article, we have explored common production issues that can occur with the Ballerina GraalVM Image and introduced essential tools for analyzing and resolving these issues. With a comprehensive understanding of production issues and the utilization of tools like heap dumps and JFR, developers can overcome challenges, optimize performance, and ensure the success of their Ballerina GraalVM Image applications.

Building complex container-based architectures is not very different from programming in terms of applying design best practices and principles. The goal of this article is to present three popular extensibility architectural patterns from a developer's perspective using well-known programming principles. Let's start with the Single Responsibility Principle. According to R. Martin, "A class should have only one reason to change." But classes are abstractions used to simplify real-world problems and represent software components. Hence, a component should have only one reason to change over time. Software services and microservices in particular are also components (runtime components) and should have only one reason to change. Microservices are supposed to be a single deployable unit, meaning they are deployed independently of other components and can have as many instances as needed. But is that always true? Are microservices always deployed as a single unit? In Kubernetes, the embodiment of a microservice is a Pod. A Pod is defined as a group of containers that share resources like file systems, kernel namespaces, and an IP address. The Pod is the atomic unit of scheduling in a Kubernetes cluster and each Pod is meant to run a single instance of a given application. According to the documentation, "Pods are designed to support multiple cooperating processes (as containers) that form a cohesive unit of service. The containers in a Pod are automatically co-located and co-scheduled on the same physical or virtual machine in the cluster. Scaling an application horizontally means replicating Pods. According to the Kubernetes documentation, Pods can be configured using two strategies: Pods that run a single container: The "one-container-per-Pod" model is the most common Kubernetes use case; the Pod is a wrapper around a single container and Kubernetes manages Pods rather than managing the containers directly. Pods that run multiple containers working together: A Pod can encapsulate an application composed of multiple co-located containers that are tightly coupled and need to share resources. These co-located containers form a single cohesive unit of service—for example, one container serving data stored in a shared volume to the public, while a separate sidecar container refreshes or updates those files. The Pod wraps these containers, storage resources, and an ephemeral network identity together as a single unit." The answer is: NO! Microservices are NOT always deployed as a single unit! Next to some popular architectural patterns for the cloud like scalability patterns, deployment and reliability patterns are extensibility architectural patterns. We will have a closer look at the three most popular extensibility patterns for cloud architectures: Sidecar pattern Ambassador pattern Adapter pattern Sidecar Pattern Problem Each deployable service/application has its own "reason to change," or responsibility. However, in addition to its core functionality it needs to do other things called in the software developer terminology "cross-cutting concerns." One example is the collection of performance metrics that need to be sent to a monitoring service. Another one is logging events and sending them to a distributed logging service. I called them cross-cutting concerns, as they do not directly relate to business logic and are needed by multiple services, they basically represent reusable functionality that needs to be part of each deployed unit. Solution The solution to that problem is called the sidecar pattern and imposes the creation of an additional container called a sidecar container. Sidecar containers are an extension of the main container following the Open-Closed design principle (opened for extension, closed for modification). They are tightly coupled with the "main" container in terms of deployment as they are deployed as part of the same Pod but are still easy to replace and do not break the single responsibility of the extended container. Furthermore, the achieved modularity allows for isolated testing of business-related functionality and additional helper services like event logging or monitoring. The communication of the two containers is fast and reliable and they share access to the same resources enabling the helper component to provide reusable infrastructure-related services. In addition, it is applicable to many types of services solving the issue with heterogeneity in terms of different technologies used. The upgrade of the sidecar components is also straightforward as it usually means the upgrade of a Docker container version and redeploying using for example the no-down-time Kubernetes strategies. Ambassador Containers Problem Deployed services do not function in isolation. They usually communicate over the network to other services even outside the application or software platform controlled by a single organization. Integrations between components in general imply integration with external APIs and also dealing with failures or unavailability of external systems. A common practice for external systems integration is to define the so-called API Facade, an internal API that hides the complexity of external system APIs. The role of the API Facades is to isolate the external dependencies providing an implementation of the internal API definition and taking care of security and routing if needed. In addition, failures and unavailability of external systems are usually handled using some common patterns like the Retry Pattern, Circuit Breaker Pattern, and sometimes backed by Local Caching. All these technicalities would complicate the main service and appear to be candidates for a helper container. Solution The solution to that problem is called Ambassador Pattern and implies the creation of an additional container called an Ambassador container. Ambassador containers proxy a local connection to the world, they are basically a type of Sidecar container. This composition of containers is powerful, not just because of the separation of concerns and the fact that different teams can easily own the components but it also allows for an easy mocking of external services for local development environments. Adapter Containers Problem There are still many monolith systems planned for migration to more lightweight architectures. Migrations, though, can not happen in one pass, and it is also risky to wait for the rewriting of a whole system for years while also supporting the addition of new features in both versions of the system. Migrations should happen in small pieces publishing separate services and integrating them one by one. That process repeats until the legacy monolith system is gone. So we have a new part of the system supporting new APIs and an old part that still supports old APIs. For example, we might have newly implemented REST services and still have some old SOAP-based services. We need something that takes care of exposing the old functionality as if all the services were migrated and can be integrated by the clients' systems. Solution The solution to that problem is called Adapter or Anti-Corruption pattern. The Adapter container takes care of translating from one communication protocol to another and from one data model to another while hiding the actual service from the external world. Furthermore, the Adapter container can provide two-way communication. If the legacy system needs to communicate with the new services it could also be the adapting component for that communication serving as a kind of an Ambassador container until the migration is finalized. In this article, we saw how container composition provides an extensibility mechanism without an actual change of the main application container providing stability and reusability by allowing the composite pod to be treated as any other simple pod exposing a single and simple service in a microservice architecture. One would ask why not use a library and share it across many containers. Well, that is also a solution but then we are facing the shared responsibility problem of introducing coupling between all the services using it. In addition, heterogeneous services would require rewriting the libraries using all the supported languages. That also breaks the Single Responsibility Principle, which we would in any case like to keep.

By Daniela Kolarova CORE

The ELK stack is an abbreviation for Elasticsearch, Logstash, and Kibana, which offers the following capabilities: Elasticsearch: a scalable search and analytics engine with a log analytics tool and application-formed database, perfect for data-driven applications. Logstash: a log-processing tool that collects logs from various sources, parses them, and sends them to Elasticsearch for storage and analysis. Kibana: A powerful visualization tool that allows you to explore and analyze the data stored in Elasticsearch using interactive charts, graphs, and dashboards. The Infrastructure of Elasticsearch Before we dive into deploying the ELK Stack, let's first understand the critical components of Elasticsearch's infrastructure: Nodes: elasticsearch runs on dedicated servers called nodes, which operate as binaries for search and analytics tasks. Shards: the database space is logically divided into shards, enabling faster data accessibility and distribution. Indices: elasticsearch organizes the stored data into indices, facilitating efficient data management. Configuring the ELK stack: you'll need a Kubernetes cluster to deploy the ELK Stack on Kubernetes. If you already have one, you can proceed with the deployment. Alternatively, you can use the provided GitHub repository with Terraform files to set up a Kubernetes cluster. Deploying elasticsearch: utilizing Helm charts, we can efficiently deploy Elasticsearch. Modify the values file to match your specific requirements, such as adjusting the number of replicas or turning certain features on/off. Download them from Artifactory Hub. values-elasticsearch.yaml YAML clusterName: "itsyndicateblog" replicas: 1 minimumMasterNodes: 1 createCert: true secret: enabled: true password: "" # generated randomly if not defined image: "docker.elastic.co/elasticsearch/elasticsearch" imageTag: "8.5.1" resources: requests: cpu: "200m" memory: "500Mi" limits: cpu: "300m" memory: "1Gi" ingress: enabled: false # enable ingress only if you need external access to elasticsearch cluster hosts: - host: elastic.itsyndicate.org paths: - path: / Once you've customized the values, use the Helm chart to install Elasticsearch: Shell helm install elasticsearch -f elasticsearch-values.yaml <chart-name> Note: Ensure you have configured the drivers (EBS or EFS) for persistent volumes. Deploying Kibana Kibana deployment is straightforward using Helm charts. In the values file, specify the URL and port of the Elasticsearch service: values-kibana.yaml YAML elasticsearchHosts: "https://elasticsearch-master:9200" enterpriseSearch: host: "https://elasticsearch-master:9200" Shell helm install kibana -f kibana-values.yaml <chart-name> Check if Kibana is installed correctly, port forward the container’s port to the local network (I am using K8s Lens) Deploying Logstash and Filebeat To manage logs effectively, we use Logstash and Filebeat. Filebeat collects records from various sources and Logstash processes and sends them to Elasticsearch. Deploy Logstash Clone repository with configs: logstash-k8s Move to tf-modules/eks/manifests/logstash-k8s Edit configmap.yaml file add elasticsearch host, user and password(you can take them from “Secrets” Kubernetes resource) Apply templates: Shell kubectl apply -f logstash-k8s -n $CHANGE_TO_ELASTIC_NS Deploy Filebeat Ensure Filebeat's configuration points to the correct log files on your nodes. Usually, in EKS, it’s the /var/log/containers folder. To check it, log in to one of your nodes and move to the /var/log/containers directory; if there are no files, try to change the directory. In case everything is correct, apply Kubernetes templates: Shell kubectl apply -f filebeat-k8s Deploy a Simple Application to Check How Logs Are Streaming Into Elasticsearch Enter the eks/manifests folder from the cloned repository. Execute command: Shell kubeclt apply -f app -n default After installation is complete, revisit Kibana and create an elasticsearch index. Creating an index: Create logstash index pattern: logstash-[namespace]* Now, you should see logs from the deployed application. If not, make some requests to this app and try to troubleshoot the issue; refer to the video guide in case help is required. Conclusion You've successfully deployed the ELK Stack on Kubernetes, empowering your applications with robust log analysis and data-driven insights. Elasticsearch, Logstash, and Kibana seamlessly handle large data streams and provide meaningful visualizations. Now that you have a robust logging solution, you can efficiently manage your logs and gain valuable insights. Happy analyzing! Thank you for reading this guide on deploying the ELK Stack. Feel free to reach out if you have any questions or require further assistance. Happy coding!

This is an article from DZone's 2023 Kubernetes in the Enterprise Trend Report.For more: Read the Report Kubernetes streamlines cloud operations by automating key tasks, specifically deploying, scaling, and managing containerized applications. With Kubernetes, you have the ability to group hosts running containers into clusters, simplifying cluster management across public, private, and hybrid cloud environments. AI/ML and Kubernetes work together seamlessly, simplifying the deployment and management of AI/ML applications. Kubernetes offers automatic scaling based on demand and efficient resource allocation, and it ensures high availability and reliability through replication and failover features. As a result, AI/ML workloads can share cluster resources efficiently with fine-grained control. Kubernetes' elasticity adapts to varying workloads and integrates well with CI/CD pipelines for automated deployments. Monitoring and logging tools provide insights into AI/ML performance, while cost-efficient resource management optimizes infrastructure expenses. This partnership streamlines the AI/ML development process, making it agile and cost-effective. Let's see how Kubernetes can join forces with AI/ML. The Intersection of AI/ML and Kubernetes The partnership between AI/ML and Kubernetes empowers organizations to deploy, manage, and scale AI/ML workloads effectively. However, running AI/ML workloads presents several challenges, and Kubernetes addresses those challenges effectively through: Resource management – This allocates and scales CPU and memory resources for AI/ML Pods, preventing contention and ensuring fair distribution. Scalability – Kubernetes adapts to changing AI/ML demands with auto-scaling, dynamically expanding or contracting clusters. Portability – AI/ML models deploy consistently across various environments using Kubernetes' containerization and orchestration. Isolation – Kubernetes isolates AI/ML workloads within namespaces and enforces resource quotas to avoid interference. Data management – Kubernetes simplifies data storage and sharing for AI/ML with persistent volumes. High availability – This guarantees continuous availability through replication, failover, and load balancing. Security – Kubernetes enhances security with features like RBAC and network policies. Monitoring and logging – Kubernetes integrates with monitoring tools like Prometheus and Grafana for real-time AI/ML performance insights. Deployment automation – AI/ML models often require frequent updates. Kubernetes integrates with CI/CD pipelines, automating deployment and ensuring that the latest models are pushed into production seamlessly. Let's look into the real-world use cases to better understand how companies and products can benefit from Kubernetes and AI/ML. REAL-WORLD USE CASES Use Case Examples Recommendation systems Personalized content recommendations in streaming services, e-commerce, social media, and news apps Image and video analysis Automated image and video tagging, object detection, facial recognition, content moderation, and video summarization Natural language processing (NLP) Sentiment analysis, chatbots, language translation, text generation, voice recognition, and content summarization Anomaly detection Identifying unusual patterns in network traffic for cybersecurity, fraud detection, and quality control in manufacturing Healthcare diagnostics Disease detection through medical image analysis, patient data analysis, drug discovery, and personalized treatment plans Autonomous vehicles Self-driving cars use AI/ML for perception, decision-making, route optimization, and collision avoidance Financial fraud detection Detecting fraudulent transactions in real-time to prevent financial losses and protect customer data Energy management Optimizing energy consumption in buildings and industrial facilities for cost savings and environmental sustainability Customer support AI-powered chatbots, virtual assistants, and sentiment analysis for automated customer support, inquiries, and feedback analysis Supply chain optimization Inventory management, demand forecasting, and route optimization for efficient logistics and supply chain operations Agriculture and farming Crop monitoring, precision agriculture, pest detection, and yield prediction for sustainable farming practices Language understanding Advanced language models for understanding and generating human-like text, enabling content generation and context-aware applications Medical research Drug discovery, genomics analysis, disease modeling, and clinical trial optimization to accelerate medical advancements Table 1 Example: Implementing Kubernetes and AI/ML As an example, let's introduce a real-world scenario: a medical research system. The main purpose is to investigate and find the cause of Parkinson's disease. The system analyzes graphics (tomography data and images) and personal patient data (which allows the use of the data). The following is a simplified, high-level example: Figure 1: Parkinson's disease medical research architecture The architecture contains the following steps and components: Data collection – gathering various data types, including structured, unstructured, and semi-structured data like logs, files, and media, in Azure Data Lake Storage Gen2 Data processing and analysis – utilizing Azure Synapse Analytics, powered by Apache Spark, to clean, transform, and analyze the collected datasets Machine learning model creation and training – employing Azure Machine Learning, integrated with Jupyter notebooks, for creating and training ML models Security and authentication – ensuring data and ML workload security and authentication through the Key Cloak framework and Azure Key Vault Container management – managing containers using Azure Container Registry Deployment and management – using Azure Kubernetes Services to handle ML model deployment, with management facilitated through Azure VNets and Azure Load Balancer Model performance evaluation – assessing model performance using log metrics and monitoring provided by Azure Monitor Model retraining – retraining models as required with Azure Machine Learning Now, let's examine security and how it lives in Kubernetes and AI/ML. Data Analysis and Security in Kubernetes In Kubernetes, data analysis involves processing and extracting insights from large datasets using containerized applications. Kubernetes simplifies data orchestration, ensuring data is available where and when needed. This is essential for machine learning, batch processing, and real-time analytics tasks. Kubernetes ML analyses require a strong security foundation, and robust security practices are essential to safeguard data in AI/ML and Kubernetes environments. This includes data encryption at rest and in transit, access control mechanisms, regular security audits, and monitoring for anomalies. Additionally, Kubernetes offers features like role-based access control (RBAC) and network policies to restrict unauthorized access. To summarize, here is an AL/ML for Kubernetes security checklist: Access control Set RBAC for user permissions Create dedicated service accounts for ML workloads Apply network policies to control communication Image security Only allow trusted container images Keep container images regularly updated and patched Secrets management Securely store and manage sensitive data (Secrets) Implement regular Secret rotation Network security Segment your network for isolation Enforce network policies for Ingress and egress traffic Vulnerability scanning Regularly scan container images for vulnerabilities Last but not least, let's look into distributed ML in Kubernetes. Distributed Machine Learning in Kubernetes Security is an important topic; however, selecting the proper distributed ML framework allows us to solve many problems. Distributed ML frameworks and Kubernetes provide scalability, security, resource management, and orchestration capabilities essential for efficiently handling the computational demands of training complex ML models on large datasets. Here are a few popular open-source distributed ML frameworks and libraries compatible with Kubernetes: TensorFlow – An open-source ML framework that provides tf.distribute.Strategy for distributed training. Kubernetes can manage TensorFlow tasks across a cluster of containers, enabling distributed training on extensive datasets. PyTorch – Another widely used ML framework that can be employed in a distributed manner within Kubernetes clusters. It facilitates distributed training through tools like PyTorch Lightning and Horovod. Horovod – A distributed training framework, compatible with TensorFlow, PyTorch, and MXNet, that seamlessly integrates with Kubernetes. It allows for the parallelization of training tasks across multiple containers. These are just a few of the many great platforms available. Finally, let's summarize how we can benefit from using AI and Kubernetes in the future. Conclusion In this article, we reviewed real-world use cases spanning various domains, including healthcare, recommendation systems, and medical research. We also went into a practical example that illustrates the application of AI/ML and Kubernetes in a medical research use case. Kubernetes and AI/ML are essential together because Kubernetes provides a robust and flexible platform for deploying, managing, and scaling AI/ML workloads. Kubernetes enables efficient resource utilization, automatic scaling, and fault tolerance, which are critical for handling the resource-intensive and dynamic nature of AI/ML applications. It also promotes containerization, simplifying the packaging and deployment of AI/ML models and ensuring consistent environments across all stages of the development pipeline. Overall, Kubernetes enhances the agility, scalability, and reliability of AI/ML deployments, making it a fundamental tool in modern software infrastructure. This is an article from DZone's 2023 Kubernetes in the Enterprise Trend Report.For more: Read the Report

By Boris Zaikin CORE