DZone Spotlight

The time for rapid technology development and cloud computing is perhaps the most sensitive time when security issues are of great importance. It is here that security will have to be injected into a process right from the beginning — be it software development or cloud infrastructure deployment. Two concepts that are very influential in doing so are CSPM and DevSecOps. Don't worry if these terms seem complicated — all they really mean is the inclusion of security within how companies build and manage their cloud environments and software pipelines. So, let's break down what CSPM and DevSecOps are, how they fit together, and how they can assist with keeping systems secure in this article. What Is Cloud Security Posture Management? Imagine that there is this huge cloud environment, like a giant digital warehouse, containing data, services, and software. Keeping everything in such a huge environment secure is very difficult. It is here that companies find Cloud Security Posture Management or CSPM. CSPM assists the companies in the following. Track everything: There is now a bird's eye view of an entire cloud infrastructure for companies, which enables them to easily point out something that may be risky, such as misconfiguration or vulnerability. Being compliant: CSPM tools support the idea that everything in the cloud is governed whether it is in line with company policy or with regulatory compliance such as GDPR and HIPAA. Remediate issues in record time: If a problem arises, it will either automatically remediate the issue or suggest remediation. CSPM acts like a thorough security guard in the cloud, ever vigilant and watchful, ensuring everything stays safe and sound. Understanding DevSecOps We'll introduce DevSecOps in simple terms. As the name suggests, we're describing an intersection of three core domains: Dev: The activity of writing and testing software Sec: The protection of software and infrastructure against malicious activities Ops: Ensuring that the software works well and reliably once it goes live Security, before DevSecOps, tended to be an afterthought added simply at the very end of development. This meant that it had delays and would make the system more vulnerable, but with DevSecOps, security is actually integrated all the way through from when you first write a line of code to running the software in production. Key Benefits of DevSecOps Catches issues early: Security checks happen throughout development, catching issues while they are still small problems rather than waiting until they develop into major issues. Delivers fast: Without security, it only tends to the end, so software will come faster and faster. Improves collaboration: Developers, security experts, and operations teams interact with each other more closely to minimize misunderstandings and delays. How Does CSPM Relate to DevSecOps? CSPM tools serve as the security guard for your cloud. When infused into DevSecOps, they ensure that every change in the cloud or during development is made with the best security practices from day one. In a nutshell, here is the integration of CSPM and DevSecOps: Continuous security monitoring: These CSPM tools continuously scan into their cloud environment for risk-readiness. Integration of this into the DevSecOps pipeline ensures security checks occur each and every time new infrastructure is deployed or updated. Automated compliance checks: As more features are added to their cloud infrastructure, CSPM automatically scans whether the concerned infrastructure is compliant with security rules and industry standards in real time. Infrastructure as Code security: DevSecOps teams use tools like Terraform to IaC, or automatically deploy cloud infrastructure. CSPM can scan the IaC templates before anything is live to ensure that configurations are secure from the get-go. The below diagram shows stages of DevSecOps (development, testing, deployment) with continuous CSPM monitoring at each stage. Empowering DevSecOps With CSPM Here's why CSPM is so powerful when added to DevSecOps pipelines: Proactive security: The security solution will be proactive scanning continuously for risks. You don't have to wait till something breaks; you fix issues before they become a problem Speeder compliance: Instead of waiting for time to run checks through, CSPM automates checks to ensure newly deployed software and applications are meeting the security standards at an instance. Higher transparency: The teams of DevSecOps have visibility into all types of cloud assets, their configurations, and the risks. It is such transparency that it makes it easier to manage the cloud environment. Lesser manual patches: Some of the CSPM tools also include an auto-fix feature for most common security issues which saves time and effort for your team. Common Challenges With DevSecOps in Implementing CSPM Even though the benefits are clearly visible, implementing CSPM in DevSecOps pipelines is not very straightforward sometimes. Some of the frequent problems arising in this process are listed below. Complexity of tools: DevSecOps involves a large number of tools for development and deployment purposes. Hence, adding on the CSPM sometimes complicates things if not done very well. Too many alerts: Some of the tools used in CSPM often send too many notifications, which results in "alert fatigue." Thus, the alerts must be fine-tuned in order to make them meaningful. Team collaboration: DevSecOps is truly effective if and only if proper communication between development, security, and operations teams takes place; otherwise, implementing CSPM is going to be pretty challenging. Multi-cloud setups: In most organizations, a multi-cloud environment is implemented. Ensuring consistency in security across multiple clouds might be challenging, but that's exactly what CSPM tools are built for, given the right configurations in place. Infrastructure as Code (IaC) and Pre-Certified Modules The role of CSPM in IaC tools like Terraform is pretty important by scanning the code that expresses the cloud infrastructure. In one practical way, making sure that the deployment is secured can make use of pre-certified modules. Here again, the modules come with baked-in security best practices that enable DevSecOps to build environments from scratch securely. Compliance modules are only deployed here, and they will be continuously monitored. CSPM Tools Here’s a list of CSPM tools: IBM Cloud Security and Compliance Center (SCC) - Provides continuous compliance monitoring, risk management, and policy enforcement for IBM Cloud environments with in-depth audit capabilities Palo Alto Networks Prisma Cloud - Offers multi-cloud security posture management with threat detection, visibility, and automated compliance checks AWS Security Hub - A native AWS service that aggregates security alerts and enables compliance checks across AWS accounts Microsoft Defender for Cloud - Secures workloads across Azure and hybrid cloud environments by assessing security posture and providing real-time threat protection Check Point CloudGuard - Provides posture management, threat intelligence, and automated compliance enforcement for cloud-native applications and multi-cloud environments Aqua Security - Combines CSPM with container and Kubernetes security, offering end-to-end visibility and risk management for cloud infrastructures Wiz - A fast-growing CSPM solution offering deep security insights, prioritizing vulnerabilities and compliance risks across cloud platforms Orca Security - An agentless CSPM tool that provides real-time risk assessment and cloud workload protection for multiple cloud environments CSPM and Beyond In addition to CSPM, there are several other cloud security tools and frameworks designed to ensure the safety, compliance, and efficiency of cloud environments. Here are some of the key tools commonly used alongside or as alternatives to CSPM: Cloud Workload Protection Platform (CWPP) Secures cloud-based workloads, including virtual machines (VMs), containers, and serverless functions Includes vulnerability management, system integrity monitoring, runtime protection, and network segmentation Cloud Access Security Broker (CASB) Acts as a gatekeeper between users and cloud service providers, ensuring secure access to cloud services Provides visibility, compliance, data security, and threat protection for cloud applications. Cloud Infrastructure Entitlement Management (CIEM) Focuses on managing and securing permissions and access to cloud resources Helps with least privilege enforcement, identity governance, and mitigating risks of misconfigurations Cloud-Native Application Protection Platform (CNAPP) Provides a comprehensive suite that integrates CSPM, CWPP, and more to secure applications across development and production Encompasses vulnerability management, runtime security, and compliance for cloud-native applications like containers and Kubernetes Security Information and Event Management (SIEM) Centralized logging and analysis of security events from cloud infrastructure and applications Enables threat detection, incident response, and compliance reporting Runtime Application Self-Protection (RASP) Provides real-time protection for applications while they are running in the cloud Detects and mitigates attacks by monitoring the behavior of an application and blocking malicious activity Security Orchestration, Automation, and Response (SOAR) Automates security operations and workflows to reduce manual effort in threat detection and response. Coordinates multiple security tools to streamline threat management and incident response. Conclusion: The Force of Security From the Start This enables companies to build secure, compliant, and fast cloud environments. Companies are able to move fast while staying ahead of security threats by integrating security throughout every stage of development and cloud management. Tools like CSPM make sure no cloud misconfiguration slips through and with this approach, DevSecOps carries out this process — that of being collaborative and fast. The integration of security is essentially a core part of every decision. If you're into cloud infrastructure, think about what kinds of such tools and practices you might bring into your processes. By putting security into your applications at the beginning, you save time, decrease risks, and give a more solid environment for your applications.

By Josephine Eskaline Joyce CORE

This is another article in the series related to supporting the Postgres JSON functions in a project using the Hibernate framework with version 6. The topic for the article is modification operations on JSON records. As in the previous article, it is worth mentioning that Postgres might now have such comprehensive operations as other NoSQL databases like MongoDB for JSON modification (although, with the proper function constructions, it is possible to achieve the same effect). It still suits most projects that require JSON modification. Plus, with transaction support (not support in a NoSQL database at such a level), it is a pretty good idea to use Postgres with JSON data. Of course, NoSQL databases have other benefits that might suit better projects. There are generally many articles on Postgres' support for JSON. This article focuses on integrating this support with the Hibernate 6 library. In case someone is interested in querying JSON data or text search using Postgres and Hibernate, please see the below links: Postgres Full-Text Search With Hibernate 6 Postgres JSON Functions With Hibernate 6 Test Data For the article, let's assume that our database has a table called the item, which has a column with JSON content, like in the below example: SQL create table item ( id int8 not null, jsonb_content jsonb, primary key (id) ) We also might have some test data: SQL INSERT INTO item (id, jsonb_content) VALUES (1, '{"top_element_with_set_of_values":["TAG1","TAG2","TAG11","TAG12","TAG21","TAG22"]}'); INSERT INTO item (id, jsonb_content) VALUES (2, '{"top_element_with_set_of_values":["TAG3"]}'); -- item without any properties, just an empty json INSERT INTO item (id, jsonb_content) VALUES (6, '{}'); -- int values INSERT INTO item (id, jsonb_content) VALUES (7, '{"integer_value": 132}'); -- double values INSERT INTO item (id, jsonb_content) VALUES (10, '{"double_value": 353.01}'); INSERT INTO item (id, jsonb_content) VALUES (11, '{"double_value": -1137.98}'); -- enum values INSERT INTO item (id, jsonb_content) VALUES (13, '{"enum_value": "SUPER"}'); -- string values INSERT INTO item (id, jsonb_content) VALUES (18, '{"string_value": "the end of records"}'); Native SQL Execution Like in other Java frameworks, with Hibernate, you can run native SQL queries — which is well documented and there are a lot of examples on the internet. That is why in this article, we won't focus on native SQL operation execution. However, there will be examples of what kind of SQL the JPA operations generate. Because Hibernate is a JPA implementation, it makes sense to show how JPA API can modify JSON data in the Postgres database. Modify JSON Object Properties and Not the Entire JSON Object (Path) Setting the whole JSON payload for one column is easy and does not require much explanation. We just set the value for the property in our Entity class, which represents a column with JSON content. It is similar to setting single or multiple properties for JSON for one database row. We just read the table row, deserialize the JSON value to a POJO representing a JSON object, set values for particular properties, and update the database records with the whole payload. However, such an approach might not be practical when we want to modify JSON properties for multiple database rows. Suppose we have to do batch updates of particular JSON properties. Fetching from the database and updating each record might not be an effective method. It would be much better to do such an update with one update statement where we set values for particular JSON properties. Fortunately, Postgres has functions that modify JSON content and can be used in the SQL update statement. Posjsonhelper Hibernate has better support for JSON modification in version 7, including most of the functions and operators mentioned in this article. Still, there are no plans to add such support in version 6. Fortunately, the Posjsonhelper project adds such support for Hibernate in version 6. All the examples below will use the Posjsonhelper library. Check this link to find out how to attach a library to your Java project. You will also have to attach FunctionContributor. All examples use Java entity class that represents the item table, whose definition was mentioned above: Java import jakarta.persistence.Column; import jakarta.persistence.Entity; import jakarta.persistence.Id; import jakarta.persistence.Table; import org.hibernate.annotations.JdbcTypeCode; import org.hibernate.annotations.Type; import org.hibernate.type.SqlTypes; import java.io.Serializable; @Entity @Table(name = "item") public class Item implements Serializable { @Id private Long id; @JdbcTypeCode(SqlTypes.JSON) @Column(name = "jsonb_content") private JsonbContent jsonbContent; public Long getId() { return id; } public void setId(Long id) { this.id = id; } public JsonbContent getJsonbContent() { return jsonbContent; } public void setJsonbContent(JsonbContent jsonbContent) { this.jsonbContent = jsonbContent; } } jsonb_set Function Wrapper The jsonb_set function is probably the most helpful function when modifying JSON data is required. It allows specific properties for JSON objects and specific array elements to be set based on the array index. For example, the below code adds the property "birthday" to the inner property "child". Java // GIVEN Long itemId = 19L; String property = "birthday"; String value = "1970-01-01"; String expectedJson = "{\"child\": {\"pets\" : [\"dog\"], \"birthday\": \"1970-01-01\"}"; // when CriteriaUpdate<Item> criteriaUpdate = entityManager.getCriteriaBuilder().createCriteriaUpdate(Item.class); Root<Item> root = criteriaUpdate.from(Item.class); // Set the property you want to update and the new value criteriaUpdate.set("jsonbContent", new JsonbSetFunction((NodeBuilder) entityManager.getCriteriaBuilder(), root.get("jsonbContent"), new JsonTextArrayBuilder().append("child").append(property).build().toString(), JSONObject.quote(value), hibernateContext)); // Add any conditions to restrict which entities will be updated criteriaUpdate.where(entityManager.getCriteriaBuilder().equal(root.get("id"), itemId)); // Execute the update entityManager.createQuery(criteriaUpdate).executeUpdate(); // then Item item = tested.findById(itemId); assertThat((String) JsonPath.read(item.getJsonbContent(), "$.child." + property)).isEqualTo(value); JSONObject jsonObject = new JSONObject(expectedJson); DocumentContext document = JsonPath.parse((Object) JsonPath.read(item.getJsonbContent(), "$")); assertThat(document.jsonString()).isEqualTo(jsonObject.toString()); This code would generate such a SQL statement: SQL update item set jsonb_content=jsonb_set(jsonb_content, ?::text[], ?::jsonb) where id=? Hibernate: select i1_0.id, i1_0.jsonb_content from item i1_0 where i1_0.id=? Concatenation Operator Wrapper "||" The wrapper for the concatenation operator (||) concatenates two JSONB values into a new JSONB value. Based on Postgres documentation, the operator behavior is as follows: Concatenating two arrays generates an array containing all the elements of each input. Concatenating two objects generates an object containing the union of their keys, taking the second object's value when there are duplicate keys. All other cases are treated by converting a non-array input into a single-element array, and then proceeding as for two arrays. Does not operate recursively: only the top-level array or object structure is merged. Here is an example of how to use this wrapper in your code: Java // GIVEN Long itemId = 19l; String property = "birthday"; String value = "1970-01-01"; // WHEN CriteriaUpdate<Item> criteriaUpdate = entityManager.getCriteriaBuilder().createCriteriaUpdate(Item.class); Root<Item> root = criteriaUpdate.from(Item.class); JSONObject jsonObject = new JSONObject(); jsonObject.put("child", new JSONObject()); jsonObject.getJSONObject("child").put(property, value); criteriaUpdate.set("jsonbContent", new ConcatenateJsonbOperator((NodeBuilder) entityManager.getCriteriaBuilder(), root.get("jsonbContent"), jsonObject.toString(), hibernateContext)); criteriaUpdate.where(entityManager.getCriteriaBuilder().equal(root.get("id"), itemId)); entityManager.createQuery(criteriaUpdate).executeUpdate(); // THEN Item item = tested.findById(itemId); assertThat((String) JsonPath.read(item.getJsonbContent(), "$.child." + property)).isEqualTo(value); JSONObject expectedJsonObject = new JSONObject().put(property, value); DocumentContext document = JsonPath.parse((Object) JsonPath.read(item.getJsonbContent(), "$.child")); assertThat(document.jsonString()).isEqualTo(expectedJsonObject.toString()); Code merge a JSON object with the child property with the already stored JSON object in the database. This code generates such a SQL query: SQL update item set jsonb_content=jsonb_content || ?::jsonb where id=? Hibernate: select i1_0.id, i1_0.jsonb_content from item i1_0 where i1_0.id=? Delete the Field or Array Element Based on the Index at the Specified Path "#-" The Posjsonhelper has a wrapper for the delete operation (#-). It deletes the field or array element based on the index at the specified path, where path elements can be either field keys or array indexes. For example, the below code removes from the JSON object property based on the "child.pets" JSON path. Java // GIVEN Item item = tested.findById(19L); JSONObject jsonObject = new JSONObject("{\"child\": {\"pets\" : [\"dog\"]}"); DocumentContext document = JsonPath.parse((Object) JsonPath.read(item.getJsonbContent(), "$")); assertThat(document.jsonString()).isEqualTo(jsonObject.toString()); // WHEN CriteriaUpdate<Item> criteriaUpdate = entityManager.getCriteriaBuilder().createCriteriaUpdate(Item.class); Root<Item> root = criteriaUpdate.from(Item.class); // Set the property you want to update and the new value criteriaUpdate.set("jsonbContent", new DeleteJsonbBySpecifiedPathOperator((NodeBuilder) entityManager.getCriteriaBuilder(), root.get("jsonbContent"), new JsonTextArrayBuilder().append("child").append("pets").build().toString(), hibernateContext)); // Add any conditions to restrict which entities will be updated criteriaUpdate.where(entityManager.getCriteriaBuilder().equal(root.get("id"), 19L)); // Execute the update entityManager.createQuery(criteriaUpdate).executeUpdate(); // THEN entityManager.refresh(item); jsonObject = new JSONObject("{\"child\": {}"); document = JsonPath.parse((Object) JsonPath.read(item.getJsonbContent(), "$")); assertThat(document.jsonString()).isEqualTo(jsonObject.toString()); The generated SQL would be: SQL update item set jsonb_content=(jsonb_content #- ?::text[]) where id=? Delete Multiple Array Elements at the Specified Path By default, Postgres (at least in version 16) does not have a built-in function that allows the removal of array elements based on their value. However, it does have the built-in operator, -#, which we mentioned above, that helps to delete array elements based on the index but not their value. For this purpose, the Posjsonhelper can generate a function that must be added to the DDL operation and executed on your database. SQL CREATE OR REPLACE FUNCTION {{schema}.remove_values_from_json_array(input_json jsonb, values_to_remove jsonb) RETURNS jsonb AS $$ DECLARE result jsonb; BEGIN IF jsonb_typeof(values_to_remove) <> 'array' THEN RAISE EXCEPTION 'values_to_remove must be a JSON array'; END IF; result := ( SELECT jsonb_agg(element) FROM jsonb_array_elements(input_json) AS element WHERE NOT (element IN (SELECT jsonb_array_elements(values_to_remove))) ); RETURN COALESCE(result, '[]'::jsonb); END; $$ LANGUAGE plpgsql; One of the wrappers will use this function to allow the deletion of multiple values from the JSON array. This code removes a "mask" and "compass" elements for the "child.inventory" property. Java // GIVEN Item item = tested.findById(24L); DocumentContext document = JsonPath.parse((Object) JsonPath.read(item.getJsonbContent(), "$")); assertThat(document.jsonString()).isEqualTo("{\"child\":{\"pets\":[\"crab\",\"chameleon\"]},\"inventory\":[\"mask\",\"fins\",\"compass\"]}"); CriteriaUpdate<Item> criteriaUpdate = entityManager.getCriteriaBuilder().createCriteriaUpdate(Item.class); Root<Item> root = criteriaUpdate.from(Item.class); NodeBuilder nodeBuilder = (NodeBuilder) entityManager.getCriteriaBuilder(); JSONArray toRemoveJSONArray = new JSONArray(Arrays.asList("mask", "compass")); RemoveJsonValuesFromJsonArrayFunction deleteOperator = new RemoveJsonValuesFromJsonArrayFunction(nodeBuilder, new JsonBExtractPath(root.get("jsonbContent"), nodeBuilder, Arrays.asList("inventory")), toRemoveJSONArray.toString(), hibernateContext); JsonbSetFunction jsonbSetFunction = new JsonbSetFunction(nodeBuilder, (SqmTypedNode) root.get("jsonbContent"), new JsonTextArrayBuilder().append("inventory").build().toString(), deleteOperator, hibernateContext); // Set the property you want to update and the new value criteriaUpdate.set("jsonbContent", jsonbSetFunction); // Add any conditions to restrict which entities will be updated criteriaUpdate.where(entityManager.getCriteriaBuilder().equal(root.get("id"), 24L)); // WHEN entityManager.createQuery(criteriaUpdate).executeUpdate(); // THEN entityManager.refresh(item); document = JsonPath.parse((Object) JsonPath.read(item.getJsonbContent(), "$")); assertThat(document.jsonString()).isEqualTo("{\"child\":{\"pets\":[\"crab\",\"chameleon\"]},\"inventory\":[\"fins\"]}"); Here is the SQL generated by the above code: SQL update item set jsonb_content=jsonb_set(jsonb_content, ?::text[], remove_values_from_json_array(jsonb_extract_path(jsonb_content, ?), ?::jsonb)) where id=? Hibernate6JsonUpdateStatementBuilder: How To Combine Multiple Modification Operations With One Update Statement All the above examples demonstrated the execution of a single operation that modifies JSON data. Of course, we can have update statements in our code that use many of the wrappers mentioned in this article together. However, being aware of how those operations and functions will be executed is crucial because it makes the most sense when the result of the first JSON operation is an input for the following JSON modification operations. The output for that operation would be an input for the next operation, and so on, until the last JSON modification operation. To better illustrate that, check the SQL code. SQL update item set jsonb_content= jsonb_set( jsonb_set( jsonb_set( jsonb_set( ( (jsonb_content #- ?::text[]) -- the most nested #- operator #- ?::text[]) , ?::text[], ?::jsonb) -- the most nested jsonb_set operation , ?::text[], ?::jsonb) , ?::text[], ?::jsonb) , ?::text[], ?::jsonb) where id=? This assumes that we have four jsonb_set function executions and two delete operations. The most nested delete operation is a first JSON modification operation because the original value from a column that stores JSON data is passed as a parameter. Although this is the correct approach, and the existing wrapper allows the creation of such an UPDATE statement, it might not be readable from a code perspective. Fortunately, Posjsonhelper has a builder component that makes building such a complex statement easy. The Hibernate6JsonUpdateStatementBuilder type allows the construction of update statements with multiple operations that modify JSON and rely on each other. Below is a code example: Java // GIVEN Item item = tested.findById(23L); DocumentContext document = JsonPath.parse((Object) JsonPath.read(item.getJsonbContent(), "$")); assertThat(document.jsonString()).isEqualTo("{\"child\":{\"pets\":[\"dog\"]},\"inventory\":[\"mask\",\"fins\"],\"nicknames\":{\"school\":\"bambo\",\"childhood\":\"bob\"}"); CriteriaUpdate<Item> criteriaUpdate = entityManager.getCriteriaBuilder().createCriteriaUpdate(Item.class); Root<Item> root = criteriaUpdate.from(Item.class); Hibernate6JsonUpdateStatementBuilder hibernate6JsonUpdateStatementBuilder = new Hibernate6JsonUpdateStatementBuilder(root.get("jsonbContent"), (NodeBuilder) entityManager.getCriteriaBuilder(), hibernateContext); hibernate6JsonUpdateStatementBuilder.appendJsonbSet(new JsonTextArrayBuilder().append("child").append("birthday").build(), quote("2021-11-23")); hibernate6JsonUpdateStatementBuilder.appendJsonbSet(new JsonTextArrayBuilder().append("child").append("pets").build(), "[\"cat\"]"); hibernate6JsonUpdateStatementBuilder.appendDeleteBySpecificPath(new JsonTextArrayBuilder().append("inventory").append("0").build()); hibernate6JsonUpdateStatementBuilder.appendJsonbSet(new JsonTextArrayBuilder().append("parents").append(0).build(), "{\"type\":\"mom\", \"name\":\"simone\"}"); hibernate6JsonUpdateStatementBuilder.appendJsonbSet(new JsonTextArrayBuilder().append("parents").build(), "[]"); hibernate6JsonUpdateStatementBuilder.appendDeleteBySpecificPath(new JsonTextArrayBuilder().append("nicknames").append("childhood").build()); // Set the property you want to update and the new value criteriaUpdate.set("jsonbContent", hibernate6JsonUpdateStatementBuilder.build()); // Add any conditions to restrict which entities will be updated criteriaUpdate.where(entityManager.getCriteriaBuilder().equal(root.get("id"), 23L)); // WHEN entityManager.createQuery(criteriaUpdate).executeUpdate(); // THEN entityManager.refresh(item); document = JsonPath.parse((Object) JsonPath.read(item.getJsonbContent(), "$")); assertThat(document.jsonString()).isEqualTo("{\"child\":{\"pets\":[\"cat\"],\"birthday\":\"2021-11-23\"},\"parents\":[{\"name\":\"simone\",\"type\":\"mom\"}],\"inventory\":[\"fins\"],\"nicknames\":{\"school\":\"bambo\"}"); The SQL statement that was mentioned previously was generated by this code. To know more about how the builder works, please check the documentation. Conclusion Postgres database has a wide range of possibilities regarding JSON data modification operations. This leads us to consider Postgres a good document storage solution choice. So if our solution does not require higher read performance, better scaling, or sharding (although all those things could be achieved with Postgres database, especially with solutions provided by cloud providers like AWS), then is it worth considering storing your JSON documents in Postgres database — not to mention transaction support with databases like Postgres.

By Szymon Tarnowski CORE

Full stack development is often likened to an intricate balancing act, where developers are expected to juggle multiple responsibilities across the front end, back end, database, and beyond. As the definition of full-stack development continues to evolve, so too does the approach to debugging. Full stack debugging is an essential skill for developers, as it involves tracking issues through multiple layers of an application, often navigating domains where one’s knowledge may only be cursory. In this blog post, I aim to explore the nuances of full-stack debugging, offering practical tips and insights for developers navigating the complex web of modern software development. Notice that this is an introductory post focusing mostly on the front-end debugging aspects. In the following posts, I will dig deeper into the less familiar capabilities of front-end debugging. Full Stack Development: A Shifting Definition The definition of full stack development is as fluid as the technology stacks themselves. Traditionally, full-stack developers were defined as those who could work on both the front end and back end of an application. However, as the industry evolves, this definition has expanded to include aspects of operations (OPS) and configuration. The modern full-stack developer is expected to submit pull requests that cover all parts required to implement a feature — backend, database, frontend, and configuration. While this does not make them an expert in all these areas, it does require them to navigate across domains, often relying on domain experts for guidance. I've heard it said that full-stack developers are: Jack of all trades, master of none. However, the full quote probably better represents the reality: Jack of all trades, master of none, but better than a master of one. The Full Stack Debugging Approach Just as full-stack development involves working across various domains, full-stack debugging requires a similar approach. A symptom of a bug may manifest in the front end, but its root cause could lie deep within the backend or database layers. Full stack debugging is about tracing these issues through the layers and isolating them as quickly as possible. This is no easy task, especially when dealing with complex systems where multiple layers interact in unexpected ways. The key to successful full-stack debugging lies in understanding how to track an issue through each layer of the stack and identifying common pitfalls that developers may encounter. Frontend Debugging: Tools and Techniques It Isn't "Just Console.log" Frontend developers are often stereotyped as relying solely on Console.log for debugging. While this method is simple and effective for basic debugging tasks, it falls short when dealing with the complex challenges of modern web development. The complexity of frontend code has increased significantly, making advanced debugging tools not just useful, but necessary. Yet, despite the availability of powerful debugging tools, many developers continue to shy away from them, clinging to old habits. The Power of Developer Tools Modern web browsers come equipped with robust developer tools that offer a wide range of capabilities for debugging front-end issues. These tools, available in browsers like Chrome and Firefox, allow developers to inspect elements, view and edit HTML and CSS, monitor network activity, and much more. One of the most powerful, yet underutilized, features of these tools is the JavaScript debugger. The debugger allows developers to set breakpoints, step through code, and inspect the state of variables at different points in the execution. However, the complexity of frontend code, particularly when it has been obfuscated for performance reasons, can make debugging a challenging task. We can launch the browser tools on Firefox using this menu: On Chrome we can use this option: I prefer working with Firefox, I find their developer tools more convenient but both browsers have similar capabilities. Both have fantastic debuggers (as you can see with the Firefox debugger below); unfortunately, many developers limit themselves to console printing instead of exploring this powerful tool. Tackling Code Obfuscation Code obfuscation is a common practice in frontend development, employed to protect proprietary code and reduce file sizes for better performance. However, obfuscation also makes the code difficult to read and debug. Fortunately, both Chrome and Firefox developer tools provide a feature to de-obfuscate code, making it more readable and easier to debug. By clicking the curly brackets button in the toolbar, developers can transform a single line of obfuscated code into a well-formed, debuggable file. Another important tool in the fight against obfuscation is the source map. Source maps are files that map obfuscated code back to its original source code, including comments. When generated and properly configured, source maps allow developers to debug the original code instead of the obfuscated version. In Chrome, this feature can be enabled by ensuring that “Enable JavaScript source maps” is checked in the developer tools settings. You can use code like this in the JavaScript file to point at the sourcemap file: //@sourceMappingURL=myfile.js.map For this to work in Chrome we need to ensure that “Enable JavaScript source maps” is checked in the settings. Last I checked it was on by default, but it doesn’t hurt to verify: Debugging Across Layers Isolating Issues Across the Stack In full-stack development, issues often manifest in one layer but originate in another. For example, a frontend error might be caused by a misconfigured backend service or a database query that returns unexpected results. Isolating the root cause of these issues requires a methodical approach, starting from the symptom and working backward through the layers. A common strategy is to reproduce the issue in a controlled environment, such as a local development setup, where each layer of the stack can be tested individually. This helps to narrow down the potential sources of the problem. Once the issue has been isolated to a specific layer, developers can use the appropriate tools and techniques to diagnose and resolve it. The Importance of System-Level Debugging Full stack debugging is not limited to the application code. Often, issues arise from the surrounding environment, such as network configurations, third-party services, or hardware limitations. A classic example of this that we ran into a couple of years ago was a production problem where a WebSocket connection would frequently disconnect. After extensive debugging, Steve discovered that the issue was caused by the CDN provider (CloudFlare) timing out the WebSocket after two minutes — something that could only be identified by debugging the entire system, not just the application code. System-level debugging requires a broad understanding of how different components of the infrastructure interact with each other. It also involves using tools that can monitor and analyze the behavior of the system as a whole, such as network analyzers, logging frameworks, and performance monitoring tools. Embracing Complexity Full stack debugging is inherently complex, as it requires developers to navigate multiple layers of an application, often dealing with unfamiliar technologies and tools. However, this complexity also presents an opportunity for growth. By embracing the challenges of full-stack debugging, developers can expand their knowledge and become more versatile in their roles. One of the key strengths of full-stack development is the ability to collaborate with domain experts. When debugging an issue that spans multiple layers, it is important to leverage the expertise of colleagues who specialize in specific areas. This collaborative approach not only helps to resolve issues more efficiently but also fosters a culture of knowledge sharing and continuous learning within the team. As tools continue to evolve, so too do the tools and techniques available for debugging. Developers should strive to stay up-to-date with the latest advancements in debugging tools and best practices. Whether it’s learning to use new features in browser developer tools or mastering system-level debugging techniques, continuous learning is essential for success in full-stack development. Video Conclusion Full stack debugging is a critical skill for modern developers, we mistakenly think it requires a deep understanding of both the application and its surrounding environment. I disagree: By mastering the tools and techniques discussed in this post/upcoming posts, developers can more effectively diagnose and resolve issues that span multiple layers of the stack. Whether you’re dealing with obfuscated frontend code, misconfigured backend services, or system-level issues, the key to successful debugging lies in a methodical, collaborative approach. You don't need to understand every part of the system, just the ability to eliminate the impossible.

By Shai Almog CORE

Equality in Java Object equality is often a hot topic for assessing concepts and one of the pillars (the other is- hashCode()) of how many of the implementations of Collection Frameworks work. We check equality by providing our own implementation for the method public booleanjava.lang.Object#equals(java.lang.Object other). According to Oracle documentation, the following mandates should be adhered to: It is reflexive: For any non-null reference value x, x.equals(x) should return true. It is symmetric: For any non-null reference values x and y, x.equals(y) should return true if and only if y.equals(x) returns true. It is transitive: For any non-null reference values x, y, and z, if x.equals(y) returns true and y.equals(z) returns true, then x.equals(z) should return true. It is consistent: For any non-null reference values x and y, multiple invocations of x.equals(y) consistently return true or consistently return false, provided no information used in equals comparisons on the objects are modified. For any non-null reference value x, x.equals(null) should return false. Please note that there exist a few more related to using it along with hashCode(), but we do not discuss them here for brevity, assuming the readers are already aware of them. Reference Equality or Content Equality? The term "equality" can itself be ambiguous, since we can either talk about reference equality or be interested in content equality. Let us illustrate both with a simple example. However, the reader may choose to skip this section and jump into the main topic of discussion at one's own discretion. Assume a class (POJO), LaptopCharger: Java package com.yourcompany.model; /** * An AC-to-DC converter LaptopCharger * */ public class LaptopCharger { private String manufacturer; private int wattage; // Consumption: Volt times Amp. private float outputCurrent; // output Amp. private float outputVoltage; // output Volt private double price; private String connectorJackType; // E.g. USB-C, pin etc. // Setters and Getters follow here } Note that we did not override any method of java.lang.Object (which, is inherited by any Java class); the default implementations, therefore, apply here. The below code snippet outputs false false: Java LaptopCharger charger_A = new LaptopCharger(65, 3.25f, 19.0f, 100, "usb-c"); LaptopCharger charger_B =new LaptopCharger(65, 3.25f, 19.0f, 100, "usb-c"); boolean refEqulas=charger_A==charger_B; boolean equals=charger_A.equals(charger_B); System.out.println(refEqulas+" "+equals); We see that reference equality is the default return value of the equals method. However, consider that Bob was searching a popular e-commerce site for a charger for his laptop. His laptop requires a 65-watt/19.8Volt type-C charger, but he finds that the one given by his laptop manufacturer is not going to reach him anytime soon. He, therefore, searches for a close alternative. The meaning of equality, in this case, is content equality as shown below: Java @Override public boolean equals(Object obj) { if (null == obj) return false; if (obj == this) return true; if (!(obj instanceof LaptopCharger)) return false; LaptopCharger other = (LaptopCharger) obj; return this.wattage == other.wattage && this.outputCurrent == other.outputCurrent && this.connectorJackType.equals(this.connectorJackType); } The output is: false true. However, the equals method can be overridden if these conditions are met: The code, i.e. LaptopCharger is open to us. This logic is accepted across the business domain. Otherwise, we can use Objects.compare(..) somewhat like the following:(Important note: Unless we are certain about ordering the objects, it may be against the prescribed contract to use Comprator<T> for just checking content equality.) Java Comparator<LaptopCharger> specificationComparator=(x,y)->{ if(x.wattage == y.wattage && x.outputCurrent == y.outputCurrent && x.connectorJackType.equals(y.connectorJackType)) return 0; else return 1; }; int t=Objects.compare(charger_A, charger_B, specificationComparator); System.out.println(t); How Much Equal Is an Object to Another?: Degree of Equality So far, we talked about content equality and it was all in black and white. However, what if we needed to check the degree of equality beyond just false and true? To elaborate on this point, let us assume the following fields: Equality of wattage, outputCurrent, and outputVoltage Equality of charger connectivity : connectorJackType Brand: manufacturer Price of the item Hypothetical business requirements are: If all 4 points above are the same, we consider 100% equality. [2] must be the same. A small variation in output current and voltage may be permissible. (Alert: in real life, this may not be the best practice!) The manufacturer of the charger is not required to be the same as the laptop's but is recommended to be. Price: Customers always hunt for low prices, discounts, and of course, value for money! A small compromise for a few other constraints is granted. Restricting the discussion to Java SE 17, we can address this scenario using third-party libraries like Fuzzy-Matcher, etc. However, would this not just be great if Java itself handled this by using a utility method in java.util.Objects? Note that it does not until this version. I just wish it were a part of Java SE and here itself! Below is a small and coarse prototype to illustrate what would have been good to have: Java /** * @param t1 * @param t2 * @param fuzzyComparator * @return R the result. No type is enforced to provide more flexibility */ public static <T, R> R fuzzyEquals(T t1, T t2, BiFunction<? super T, ? super T, R> fuzzyComparator) { return fuzzyComparator.apply(t1, t2); } The first two parameters are of type T and last one, the comparator itself is a BiFunction<? super T, ? super T, R>. In this example, I did not enforce a return type for the method, leveraging the power of generics and functional programming to provide more flexibility. This eliminates the need for a strict return type such as double as well as a dedicated functional interface like FuzzyComprator which would otherwise have looked somewhat like this: Java @FunctionalInterface public interface Comparator<T>{ // other stuff like static, default methods etc. double compare(T o1, T o2) } Below is a simple illustration using it: Java BiFunction<LaptopCharger, LaptopCharger, OptionalDouble> mySimpleFuzzyCompartor = (x, y) -> { if (x.connectorJackType.equals(y.connectorJackType)) { if (x.wattage == y.wattage && x.outputCurrent == y.outputCurrent && x.manufacturer.equals(y.manufacturer) && x.price == y.price) return OptionalDouble.of(1.0D); // Full match if (x.wattage == y.wattage && x.outputCurrent == y.outputCurrent && x.manufacturer.equals(y.manufacturer)) return OptionalDouble.of(1.0 - (x.price - y.price) / x.price);// Price based match if (x.wattage == y.wattage && x.outputCurrent == y.outputCurrent) return OptionalDouble.of(1.0 - 0.2 - (x.price - y.price) / x.price); // if (x.wattage == y.wattage && Math.abs(x.outputCurrent - y.outputCurrent) < 0.15) return OptionalDouble .of(1.0 - 0.2 - Math.abs((x.outputCurrent - y.outputCurrent) / x.outputCurrent)); return OptionalDouble.empty(); } else { return OptionalDouble.empty(); } }; OptionalDouble fuzzyEquals = fuzzyEquals(charger_A, charger_B, mySimpleFuzzyCompartor); System.out.println(fuzzyEquals); We used OptionalDouble as the return type of the fuzzyEquals. Readers are strongly encouraged to introduce the method, fuzzyEquals, in java.util.Objects and use it and get it benchmarked. Once we have that satisfactory, Collection Frameworks might be made to undergo relevant contract upgradation to strongly support beyond-the-Boolean comparison!

Distributed Application Runtime (Dapr) is a portable and event-driven runtime that commoditizes some of the problems developers face with distributed systems and microservices daily. Imagine there are 3-4 different microservices. As part of communication between these services, developers must think about: Handling timeouts and failures Metrics and traces Controlling and restricting communication between services. These challenges are recurring, but with Dapr's Service-to-Service Invocation building block, they are seamlessly abstracted. Dapr divides such capabilities into components that can be invoked using a building block, aka API. Components Overview Below mentioned are a subset of components that Dapr supports. Component Description Service-to-Service Facilitates communication between microservices: It encapsulates handling failures, observability, and applying policies (responsible for enforcing restrictions on who is allowed to call) Secrets Facilitate communication with cloud secrets and Kubernetes secrets provider stores Workflows With the Workflows component, developers can run long-running workloads distributed across nodes. Publish/Subscribe Similar to the producer/consumer pattern, with this component messages can be produced to a topic and listeners can consume from the subscribed topic. Let's dive into the workflow component. Workflow Component Problem An example of a simple Workflow can be a scheduled job that moves data between data sources. The complexity increases when child workflows must be triggered as part of the parent workflow and the workflow author also becomes responsible for saving, resuming, and maintaining the state and the schema. With the Dapr Workflow component, most of the state management is abstracted out, allowing developers to focus only on the business logic. Key Terms Workflow: Contains a set of tasks that need to be executed Activities: Tasks that need to be executed; For example, in the previous work where data must be moved from source to destination: Activity 1: Reads data from Source Activity 2: Writes to the destination Workflow will compromise both these activities. Benefits Using Workflow Replays we inherently get checkpointing mechanism. For example, in the C# async/await model, Dapr automatically checkpoints at each await call. This allows the system to recover from the most recent I/O operation during a failure, making recovery less costly. Built-in retry strategies for the workflows and activities are customizable to suit specific workflows. Workflow Patterns Pattern 1 The parent workflow parallelly schedules multiple child activities. Pattern 2 In this scenario, the workflow schedules Activity 1 and then passes its output to Activity 2 for further processing. Pattern 3 Here, the parent workflow schedules another child workflow which in turn schedules some activities. Example Let's explore an example using C# and Dapr to schedule workflows that read data from Blob storage. Step 1 Import the Dapr packages into csproj. XML <ItemGroup> # https://www.nuget.org/packages/Dapr.AspNetCore <PackageReference Include="Dapr.AspNetCore" Version="1.14.0" ></PackageReference> # https://www.nuget.org/packages/Dapr.Workflow <PackageReference Include="Dapr.Workflow" Version="1.14.0" ></PackageReference> </ItemGroup> Step 2: Configuring Workflow and Activity Add workflow and activities to the Dapr Workflow extension. "Register Workflow" is used to register workflows. "Register Activity" is used to register activity. C# /// <summary> /// Configure workflow extension. /// </summary> public static class DaprConfigurationExtension { /// <summary> /// Configure Workflow extension. /// </summary> /// <param name="services">services.</param> /// <returns>IServiceCollection.</returns> public static IServiceCollection ConfigureDaprWorkflows(this IServiceCollection services) { services.AddDaprWorkflow(options => { // Note that it's also possible to register a lambda function as the workflow // or activity implementation instead of a class. options.RegisterWorkflow<BlobOrchestrationWorkflow>(); // These are the activities that get invoked by the Dapr workflow(s). options.RegisterActivity<BlobDataFetchActivity>(); }); return services; } } Step 3: Writing the First Workflow The Blob Orchestration Workflow implements Workflow coming from Dapr NuGet with input and output parameters. The input here is the name of the blob, which is a string, and the output is content from the blob, nothing but a list of lines. C# /// <summary> /// Dapr workflow responsible for peforming operations on blob. /// </summary> public class BlobOrchestrationWorkflow : Workflow<string, List<string>> { /// <inheritdoc/> public async override Task<List<string>> RunAsync(WorkflowContext context, string input) { ArgumentNullException.ThrowIfNull(context); ArgumentNullException.ThrowIfNull(input); List<string> identifiers = await context.CallActivityAsync<List<string>>( name: nameof(BlobDataFetchActivity), input: input).ConfigureAwait(false); // state is saved return identifiers; } } Step 4: Writing the First Activity Like Workflow, Activity also takes input and output. In this case, input is the blob name, and output is the list of lines from the blob. C# /// <summary> /// Fetch identifiers from Blob. /// </summary> public class BlobDataFetchActivity : WorkflowActivity<string, List<string>> { private readonly IBlobReadProcessor readProcessor; /// <summary> /// Initializes a new instance of the <see cref="BlobDataFetchActivity"/> class. /// </summary> /// <param name="blobReadProcessor">read blob data.</param> public BlobDataFetchActivity(IBlobReadProcessor blobReadProcessor) { this.readProcessor = blobReadProcessor; } /// <inheritdoc/> public override async Task<List<string>> RunAsync(WorkflowActivityContext context, string input) { return await this.readProcessor.ReadBlobContentAsync<List<string>>(input).ConfigureAwait(false); // state is saved } } Step 5: Scheduling the First Workflow Use the Workflow Client schedule workflows. The "instance id" must be unique to each workflow. Using the same ID can cause indeterministic behavior. Each workflow has an input and an output. For example, if the workflow is going to take a blob name as input and return a list of lines in the blob, the input is a string, and the output is a List<string>. Workflow is tracked using the workflow ID and once it is completed, the "Execute Workflow Async" method completes execution. C# public class DaprService { // Workflow client injected using Dependency Injection. private readonly DaprWorkflowClient daprWorkflowClient; /// <summary> /// Initializes a new instance of the <see cref="QueuedHostedService{T}"></see> class. /// </summary> /// <param name="daprWorkflowClient">Dapr workflow client.</param> public QueuedHostedService(DaprWorkflowClient daprWorkflowClient) { this.daprWorkflowClient = daprWorkflowClient; } /// <summary> /// Execute Dapr workflow. /// </summary> /// <param name="message">string Message.</param> /// <returns>Task.</returns> public async Task ExecuteWorkflowAsync(string message) { string id = Guid.NewGuid().ToString(); // Schedule the Dapr Workflow. await this.daprWorkflowClient.ScheduleNewWorkflowAsync( name: nameof(NetworkRecordIngestionWorkflow), instanceId: id, input: message).ConfigureAwait(false); WorkflowState state = await this.daprWorkflowClient.GetWorkflowStateAsync( instanceId: id, getInputsAndOutputs: true).ConfigureAwait(false); // Track the workflow state until completion. while (!state.IsWorkflowCompleted) { state = await this.daprWorkflowClient.GetWorkflowStateAsync( instanceId: id, getInputsAndOutputs: true).ConfigureAwait(false); } } } Best Practices Each time Dapr encounters an "await," it saves the workflow state. Leveraging this feature is important for ensuring workflows can resume efficiently and cost-effectively after interruptions. In addition to the above, the input and output must be deterministic for the Workflow replay pattern to work correctly. For example, Assume below is the first input to the workflow. The workflow then pulls the data from the blob, saves it to the state, and for some reason crashes. JSON { "blobName": "dapr-blob", "createdOn": "2024-12-11T23:00:00.11212Z" } After a restart, we resend the input with a different "created on" timestamp. Even though we’ve already saved the output for the blob name, the new timestamp qualifies this as a new payload, prompting the output to be recomputed. If the "created on" timestamp was omitted, we could retrieve the state from the state store without making an additional I/O call. JSON { "blobName": "dapr-blob", "createdOn": "2024-12-11T23:01:00.11212Z" } Workflow interaction with data other than the state must happen through Activities only.

Editor's Note: The following is an article written for and published in DZone's 2024 Trend Report, Kubernetes in the Enterprise: Once Decade-Defining, Now Forging a Future in the SDLC. Kubernetes has become a cornerstone in modern infrastructure, particularly for deploying, scaling, and managing artificial intelligence and machine learning (AI/ML) workloads. As organizations increasingly rely on machine learning models for critical tasks like data processing, model training, and inference, Kubernetes offers the flexibility and scalability needed to manage these complex workloads efficiently. By leveraging Kubernetes' robust ecosystem, AI/ML workloads can be dynamically orchestrated, ensuring optimal resource utilization and high availability across cloud environments. This synergy between Kubernetes and AI/ML empowers organizations to deploy and scale their ML workloads with greater agility and reliability. This article delves into the key aspects of managing AI/ML workloads within Kubernetes, focusing on strategies for resource allocation, scaling, and automation specific to this platform. By addressing the unique demands of AI/ML tasks in a Kubernetes environment, it provides practical insights to help organizations optimize their ML operations. Whether handling resource-intensive computations or automating deployments, this guide offers actionable advice for leveraging Kubernetes to enhance the performance, efficiency, and reliability of AI/ML workflows, making it an indispensable tool for modern enterprises. Understanding Kubernetes and AI/ML Workloads In order to effectively manage AI/ML workloads in Kubernetes, it is important to first understand the architecture and components of the platform. Overview of Kubernetes Architecture Kubernetes architecture is designed to manage containerized applications at scale. The architecture is built around two main components: the control plane (coordinator nodes) and the worker nodes. Figure 1. Kubernetes architecture For more information, or to review the individual components of the architecture in Figure 1, check out the Kubernetes Documentation. AI/ML Workloads: Model Training, Inference, and Data Processing AI/ML workloads are computational tasks that involve training machine learning models, making predictions (inference) based on those models, and processing large datasets to derive insights. AI/ML workloads are essential for driving innovation and making data-driven decisions in modern enterprises: Model training enables systems to learn from vast datasets, uncovering patterns that power intelligent applications. Inference allows these models to generate real-time predictions, enhancing user experiences and automating decision-making processes. Efficient data processing is crucial for transforming raw data into actionable insights, fueling the entire AI/ML pipeline. However, managing these computationally intensive tasks requires a robust infrastructure. This is where Kubernetes comes into play, providing the scalability, automation, and resource management needed to handle AI/ML workloads effectively, ensuring they run seamlessly in production environments. Key Considerations for Managing AI/ML Workloads in Kubernetes Successfully managing AI/ML workloads in Kubernetes requires careful attention to several critical factors. This section outlines the key considerations for ensuring that your AI/ML workloads are optimized for performance and reliability within a Kubernetes environment. Resource Management Effective resource management is crucial when deploying AI/ML workloads on Kubernetes. AI/ML tasks, particularly model training and inference, are resource intensive and often require specialized hardware such as GPUs or TPUs. Kubernetes allows for the efficient allocation of CPU, memory, and GPUs through resource requests and limits. These configurations ensure that containers have the necessary resources while preventing them from monopolizing node capacity. Additionally, Kubernetes supports the use of node selectors and taints/tolerations to assign workloads to nodes with the required hardware (e.g., GPU nodes). Managing resources efficiently helps optimize cluster performance, ensuring that AI/ML tasks run smoothly without over-provisioning or under-utilizing the infrastructure. Handling resource-intensive tasks requires careful planning, particularly when managing distributed training jobs that need to run across multiple nodes. These workloads benefit from Kubernetes' ability to distribute resources while ensuring that high-priority tasks receive adequate computational power. Scalability Scalability is another critical factor in managing AI/ML workloads in Kubernetes. Horizontal scaling, where additional Pods are added to handle increased demand, is particularly useful for stateless workloads like inference tasks that can be easily distributed across multiple Pods. Vertical scaling, which involves increasing the resources available to a single Pod (e.g., more CPU or memory), can be beneficial for resource-intensive processes like model training that require more power to handle large datasets. In addition to Pod autoscaling, Kubernetes clusters benefit from cluster autoscaling to dynamically adjust the number of worker nodes based on demand. Karpenter is particularly suited for AI/ML workloads due to its ability to quickly provision and scale nodes based on real-time resource needs. Karpenter optimizes node placement by selecting the most appropriate instance types and regions, taking into account workload requirements like GPU or memory needs. By leveraging Karpenter, Kubernetes clusters can efficiently scale up during resource-intensive AI/ML tasks, ensuring that workloads have sufficient capacity without over-provisioning resources during idle times. This leads to improved cost efficiency and resource utilization, especially for complex AI/ML operations that require on-demand scalability. These autoscaling mechanisms enable Kubernetes to dynamically adjust to workload demands, optimizing both cost and performance. Data Management AI/ML workloads often require access to large datasets and persistent storage for model checkpoints and logs. Kubernetes offers several persistent storage options to accommodate these needs, including PersistentVolumes (PVs) and PersistentVolumeClaims (PVCs). These options allow workloads to access durable storage across various cloud and on-premises environments. Additionally, Kubernetes integrates with cloud storage solutions like AWS EBS, Google Cloud Storage, and Azure Disk Storage, making it easier to manage storage in hybrid or multi-cloud setups. Handling large volumes of training data requires efficient data pipelines that can stream or batch process data into models running within the cluster. This can involve integrating with external systems, such as distributed file systems or databases, and using tools like Apache Kafka for real-time data ingestion. Properly managing data is essential for maintaining high-performance AI/ML pipelines, ensuring that models have quick and reliable access to the data they need for both training and inference. Deployment Automation Automation is key to managing the complexity of AI/ML workflows, particularly when deploying models into production. CI/CD pipelines can automate the build, test, and deployment processes, ensuring that models are continuously integrated and deployed with minimal manual intervention. Kubernetes integrates well with CI/CD tools like Jenkins, GitLab CI/CD, and Argo CD, enabling seamless automation of model deployments. Tools and best practices for automating AI/ML deployments include using Helm for managing Kubernetes manifests, Kustomize for configuration management, and Kubeflow for orchestrating ML workflows. These tools help standardize the deployment process, reduce errors, and ensure consistency across environments. By automating deployment, organizations can rapidly iterate on AI/ML models, respond to new data, and scale their operations efficiently, all while maintaining the agility needed in fast-paced AI/ML projects. Scheduling and Orchestration Scheduling and orchestration for AI/ML workloads require more nuanced approaches compared to traditional applications. Kubernetes excels at managing these different scheduling needs through its flexible and powerful scheduling mechanisms. Batch scheduling is typically used for tasks like model training, where large datasets are processed in chunks. Kubernetes supports batch scheduling by allowing these jobs to be queued and executed when resources are available, making them ideal for non-critical workloads that are not time sensitive. Kubernetes Job and CronJob resources are particularly useful for automating the execution of batch jobs based on specific conditions or schedules. On the other hand, real-time processing is used for tasks like model inference, where latency is critical. Kubernetes ensures low latency by providing mechanisms such as Pod priority and preemption, ensuring that real-time workloads have immediate access to the necessary resources. Additionally, Kubernetes' HorizontalPodAutoscaler can dynamically adjust the number of pods to meet demand, further supporting the needs of real-time processing tasks. By leveraging these Kubernetes features, organizations can ensure that both batch and real-time AI/ML workloads are executed efficiently and effectively. Gang scheduling is another important concept for distributed training in AI/ML workloads. Distributed training involves breaking down model training tasks across multiple nodes to reduce training time, and gang scheduling ensures that all the required resources across nodes are scheduled simultaneously. This is crucial for distributed training, where all parts of the job must start together to function correctly. Without gang scheduling, some tasks might start while others are still waiting for resources, leading to inefficiencies and extended training times. Kubernetes supports gang scheduling through custom schedulers like Volcano, which is designed for high-performance computing and ML workloads. Latency and Throughput Performance considerations for AI/ML workloads go beyond just resource allocation; they also involve optimizing for latency and throughput. Latency refers to the time it takes for a task to be processed, which is critical for real-time AI/ML workloads such as model inference. Ensuring low latency is essential for applications like online recommendations, fraud detection, or any use case where real-time decision making is required. Kubernetes can manage latency by prioritizing real-time workloads, using features like node affinity to ensure that inference tasks are placed on nodes with the least network hops or proximity to data sources. Throughput, on the other hand, refers to the number of tasks that can be processed within a given time frame. For AI/ML workloads, especially in scenarios like batch processing or distributed training, high throughput is crucial. Optimizing throughput often involves scaling out workloads horizontally across multiple Pods and nodes. Kubernetes' autoscaling capabilities, combined with optimized scheduling, ensure that AI/ML workloads maintain high throughput — even as demand increases. Achieving the right balance between latency and throughput is vital for the efficiency of AI/ML pipelines, ensuring that models perform at their best while meeting real-world application demands. A Step-by-Step Guide: Deploying TensorFlow Sentiment Analysis Model on AWS EKS In this example, we demonstrate how to deploy a TensorFlow-based sentiment analysis model using AWS Elastic Kubernetes Service (EKS). This hands-on guide will walk you through setting up a Flask-based Python application, containerizing it with Docker, and deploying it on AWS EKS using Kubernetes. Although many tools are suitable, TensorFlow was chosen for this example due to its popularity and robustness in developing AI/ML models, while AWS EKS provides a scalable and managed Kubernetes environment that simplifies the deployment process. By following this guide, readers will gain practical insights into deploying AI/ML models in a cloud-native environment, leveraging Kubernetes for efficient resource management and scalability. Step 1: Create a Flask-based Python app setup Create a Flask app (app.py) using the Hugging Face transformers pipeline for sentiment analysis: Shell from flask import Flask, request, jsonify from transformers import pipeline app = Flask(__name__) sentiment_model = pipeline("sentiment-analysis") @app.route('/analyze', methods=['POST']) def analyze(): data = request.get_json() result = sentiment_model(data['text']) return jsonify(result) if __name__ == '__main__': app.run(host='0.0.0.0', port=5000) Step 2: Create requirements.txt Shell transformers==4.24.0 torch==1.12.1 flask jinja2 markupsafe==2.0.1 Step 3: Build Docker image Create a Dockerfile to containerize the app: Shell FROM python:3.9-slim WORKDIR /app COPY requirements.txt requirements.txt RUN pip install -r requirements.txt COPY . . CMD ["python", "app.py"] Build and push the Docker image: Shell docker build -t brainupgrade/aiml-sentiment:20240825 . docker push brainupgrade/aiml-sentiment:20240825 Step 4: Deploy to AWS EKS with Karpenter Create a Kubernetes Deployment manifest (deployment.yaml): Shell apiVersion: apps/v1 kind: Deployment metadata: name: sentiment-analysis spec: replicas: 1 selector: matchLabels: app: sentiment-analysis template: metadata: labels: app: sentiment-analysis spec: containers: - name: sentiment-analysis image: brainupgrade/aiml-sentiment:20240825 ports: - containerPort: 5000 resources: requests: aws.amazon.com/neuron: 1 limits: aws.amazon.com/neuron: 1 tolerations: - key: "aiml" operator: "Equal" value: "true" effect: "NoSchedule" Apply the Deployment to the EKS cluster: Shell kubectl apply -f deployment.yaml Karpenter will automatically scale the cluster and launch an inf1.xlarge EC2 instance based on the resource specification (aws.amazon.com/neuron: 1). Karpenter also installs appropriate device drivers for this special AWS EC2 instance of inf1.xlarge, which is optimized for deep learning inference, featuring four vCPUs, 16 GiB RAM, and one Inferentia chip. Reference Karpenter spec as follows: Shell apiVersion: karpenter.sh/v1alpha5 kind: Provisioner metadata: name: default spec: limits: resources: cpu: "16" provider: instanceProfile: eksctl-KarpenterNodeInstanceProfile-<cluster-name> securityGroupSelector: karpenter.sh/discovery: <cluster-name> subnetSelector: karpenter.sh/discovery: <cluster-name> requirements: - key: karpenter.sh/capacity-type operator: In values: - spot - key: node.kubernetes.io/instance-type operator: In values: - inf1.xlarge - key: kubernetes.io/os operator: In values: - linux - key: kubernetes.io/arch operator: In values: - amd64 ttlSecondsAfterEmpty: 30 Step 5: Test the application Once deployed and exposed via an AWS Load Balancer or Ingress, test the app with the following cURL command: Shell curl -X POST -H "Content-Type: application/json" -d '{"text":"I love using this product!"}' https://<app-url>/analyze This command sends a sentiment analysis request to the deployed model endpoint: https://<app-url>/analyze. Challenges and Solutions Managing AI/ML workloads in Kubernetes comes with its own set of challenges, from handling ephemeral containers to ensuring security and maintaining observability. In this section, we will explore these challenges in detail and provide practical solutions to help you effectively manage AI/ML workloads in a Kubernetes environment. Maintaining State in Ephemeral Containers One of the main challenges in managing AI/ML workloads in Kubernetes is handling ephemeral containers while maintaining state. Containers are designed to be stateless, which can complicate AI/ML workflows that require persistent storage for datasets, model checkpoints, or intermediate outputs. For maintaining state in ephemeral containers, Kubernetes offers PVs and PVCs, which enable long-term storage for AI/ML workloads, even if the containers themselves are short-lived. Ensuring Security and Compliance Another significant challenge is ensuring security and compliance. AI/ML workloads often involve sensitive data, and maintaining security at multiple levels — network, access control, and data integrity — is crucial for meeting compliance standards. To address security challenges, Kubernetes provides role-based access control (RBAC) and NetworkPolicies. RBAC ensures that users and services have only the necessary permissions, minimizing security risks. NetworkPolicies allow for fine-grained control over network traffic, ensuring that sensitive data remains protected within the cluster. Observability in Kubernetes Environments Additionally, observability is a key challenge in Kubernetes environments. AI/ML workloads can be complex, with numerous microservices and components, making it difficult to monitor performance, track resource usage, and detect potential issues in real time. Monitoring and logging are essential for observability in Kubernetes. Tools like Prometheus and Grafana provide robust solutions for monitoring system health, resource usage, and performance metrics. Prometheus can collect real-time metrics from AI/ML workloads, while Grafana visualizes this data, offering actionable insights for administrators. Together, they enable proactive monitoring, allowing teams to identify and address potential issues before they impact operations. Conclusion In this article, we explored the key considerations for managing AI/ML workloads in Kubernetes, focusing on resource management, scalability, data handling, and deployment automation. We covered essential concepts like efficient CPU, GPU, and TPU allocation, scaling mechanisms, and the use of persistent storage to support AI/ML workflows. Additionally, we examined how Kubernetes uses features like RBAC and NetworkPolicies and tools like Prometheus and Grafana to ensure security, observability, and monitoring for AI/ML workloads. Looking ahead, AI/ML workload management in Kubernetes is expected to evolve with advancements in hardware accelerators and more intelligent autoscaling solutions like Karpenter. Integration of AI-driven orchestration tools and the emergence of Kubernetes-native ML frameworks will further streamline and optimize AI/ML operations, making it easier to scale complex models and handle ever-growing data demands. For practitioners, staying informed about the latest Kubernetes tools and best practices is crucial. Continuous learning and adaptation to new technologies will empower you to manage AI/ML workloads efficiently, ensuring robust, scalable, and high-performance applications in production environments. This is an excerpt from DZone's 2024 Trend Report, Kubernetes in the Enterprise: Once Decade-Defining, Now Forging a Future in the SDLC.Read the Free Report

Understanding some of the concepts of functional programming that form the basis for the functions within the itertools module helps in understanding how such functions work. These concepts provide insight into the way the module functions operate and their conformance with regard to the paradigm that makes them powerful and efficient tools in Python. This article is going to explain some concepts related to functional programming through specific functions of the itertools module. The article can't possibly talk about all the methods in detail. Instead, it will show how the ideas work in functions like: takewhile dropwhile groupby partial Higher-Order Functions (HOF) A higher-order function is a function that does at least one of the following: Accepts one or more functions as an argument Returns a function as a result All other functions are first-order functions. Example 1: HOF Accepting a Function In the code below, the apply_operation function accepts another function named operation that can be any mathematical operation like add, subtract, or multiply and applies it to variables x and y: Python def apply_operation(operation, x, y): return operation(x, y) def add(a, b): return a + b def multiply(a, b): return a * b print(apply_operation(add, 5, 3)) # 8 print(apply_operation(multiply, 5, 3)) # 15 Example 2: HOF Returning a Function Python def get_func(func_type: str): if func_type == 'add': return lambda a, b: a + b elif func_type == 'multiply': return lambda a, b: a * b else: raise ValueError("Unknown function type") def apply_operation(func, a, b): return func(a, b) func = get_func('add') print(apply_operation(func, 2, 3)) # 5 Advantages of Higher-Order Functions Reusability Higher-order functions help avoid code duplication. In the apply_operation example, the function is reusable as it currently accepts add and multiply; similarly, we can pass the subtract function to it without any changes. Python def subtract(a, b): return a – b print(apply_operation(subtract, 5, 3)) # 2 Functional Composition Since higher-order functions can return functions that can help in function composition, my other article also discusses it. This is useful for creating flexible, modular code. Python def add_one(x): return x + 1 def square(x): return x * x def compose(f, g): return lambda x: f(g(x)) composed_function = compose(square, add_one) print(composed_function(2)) # 9 Here, add_one is applied first, and then the square is applied to the result, producing 9 (square(add_one(2))). Lazy Evaluation Lazy evaluation is about delaying the evaluation of an expression until its value is actually needed. This allows for optimized memory usage and can handle very large datasets efficiently by only processing elements on demand. In some cases, you may only need a few elements from an iterable before a condition is met or a result is obtained. Lazy evaluation allows you to stop the iteration process as soon as the desired outcome is achieved, saving computational resources. In the itertools module, functions like takeWhile, dropWhile, chain, etc. all support lazy evaluation. Currying Currying is all about breaking a function that takes multiple arguments into a sequence of functions, each of which takes one argument. This enables such a function to be partially applied and forms the basis of the partial function in the itertools module. Python does not natively support currying like Haskell, but we can emulate currying in Python by either using lambda functions or functools.partial. Python def add_three(a, b, c): return a + b + c add_curried = lambda a: lambda b: lambda c: a + b + c result = add_curried(1)(2)(3) # Output: 6 Currying breaks down a function into smaller steps, making it easier to reuse parts of a function in different contexts. Partial Functions A partial function fixes a certain number of arguments to a function, producing a new function with fewer arguments. This is similar to currying, but in partial functions, you fix some arguments of the function and get back a function with fewer parameters. The benefits of both currying and partial application help with code reusability and modularity, allowing functions to be easily reused in different contexts. These techniques facilitate function composition, where simpler functions can be combined to build more complex ones. This makes it easier to create modular and adaptable systems, as demonstrated in the article through the use of the partial function. takewhile and dropwhile Both takewhile and dropwhile are lazy evaluation functions from the itertools module, which operate on iterables based on a predicate function. They are designed to either include or skip elements from an iterable based on a condition. 1. takewhile The takewhile function returns elements from the iterable as long as the predicate function returns True. Once the predicate returns False, it stops and does not yield any more elements, even if subsequent elements would satisfy the predicate. Python from itertools import takewhile numbers = [1,2,3,4,5,6,7] list(takewhile(lambda x: x < 3, numbers)) # [1,2] 2. dropwhile The dropwhile function is the opposite of takewhile. It skips elements as long as the predicate returns True, and once the predicate returns False, it yields the remaining elements (without further checking the predicate). Python from itertools import dropwhile numbers = [1,2,3,4,5,6,7] list(dropwhile(lambda x: x < 3, numbers)) # [3, 4, 5, 6, 7] Functional Programming Concepts Both takewhile and dropwhile are higher-order functions because they take a predicate function ( a lambda function) as an argument, demonstrating how functions can be passed as arguments to other functions. They also support lazy evaluation; in takewhile, the evaluation stops as soon as the first element fails the predicate. For example, when 3 is encountered, no further elements are processed. In dropwhile, elements are skipped while the predicate is True. Once the first element fails the predicate, all subsequent elements are yielded without further checks. groupby The groupby function from the itertools module groups consecutive elements in an iterable based on a key function. It returns an iterator that produces groups of elements, where each group shares the same key (the result of applying the key function to each element). Unlike database-style GROUP BY operations, which group all similar elements regardless of their position, groupby only groups consecutive elements that share the same key. If non-consecutive elements have the same key, they will be in separate groups. Python from itertools import groupby people = [ {"name": "Alice", "age": 30}, {"name": "Bob", "age": 30}, {"name": "Charlie", "age": 25}, {"name": "David", "age": 25}, {"name": "Eve", "age": 35} ] grouped_people = groupby(people, key=lambda person: person['age']) for age, group in grouped_people: print(f"Age: {age}") for person in group: print(f" Name: {person['name']}") Functional Programming Concepts Higher-order function: groupby accepts a key function as an argument, which determines how elements are grouped, making it a higher-order function. Lazy evaluation: Like most itertools functions, groupby yields groups lazily as the iterable is consumed. partial As explained above, partial allows you to fix a certain number of arguments in a function, returning a new function with fewer arguments. Python from functools import partial def create_email(username, domain): return f"{username}@{domain}" create_gmail = partial(create_email, domain="gmail.com") create_yahoo = partial(create_email, domain="yahoo.com") email1 = create_gmail("alice") email2 = create_yahoo("bob") print(email1) # Output: alice@gmail.com print(email2) # Output: bob@yahoo.com partial is used to fix the domain part of the email (gmail.com or yahoo.com), so you only need to provide the username when calling the function. This reduces redundancy when generating email addresses with specific domains. Functional Programming Concepts Function currying: partial is a form of currying, where a function is transformed into a series of functions with fewer arguments. It allows pre-setting of arguments, creating a new function that "remembers" the initial values. Higher-order function: Since partial returns a new function, it qualifies as a higher-order function. Conclusion Exploring concepts like higher-order functions, currying, and lazy evaluation can help Python developers make better use of the itertools functions. These fundamental principles help developers understand the workings of functions such as takewhile, dropwhile, groupby, and partial, enabling them to create more organized and streamlined code.

By Sameer Shukla CORE

Developers may be aware of the lifecycle of service instances when using dependency injection, but many don’t fully grasp how it works. You can find numerous articles online that clarify these concepts, but they often just reiterate definitions that you might already know. Let me illustrate with a detailed example that simplifies the explanation. When implementing dependency injection, developers have three options that determine the lifecycle of the instances: Singleton Scoped Transient While most developers recognize these terms, a significant number struggle to determine which option to choose for a service's lifetime. Definitions Let me start with definitions: Singleton lifetime service instances are created once per application from the service container. A single instance will serve all subsequent requests. Singleton services are disposed of at the end of the application (i.e., upon application restart). Transient lifetime service instances are created per request from the service container. Transient services are disposed of at the end of the request. Scoped lifetime service instances are created once per client request. Transient services are disposed of at the end of the request. When to Use Singleton - When you want to use single instances of services throughout the life cycle of the application Transient - When you want to use individual instances of services within the client request Scoped - When you want to use a single instance of service for each request What is a client request? In very simple words, you can consider it as an API/REST call coming to your application by button clicks of the user to get the response. Don’t worry, let's understand with an example. Example First, let's create interfaces/services and classes: C# // we are declaring 3 services as below Public interface ISingleton Public interface ITransient Public interface IScoped Now let's write the implementation for each service Interface/service created above. We will try to understand the concept by trying to update the callMeSingleton, callMeTransient, and callMeScoped variable. Singleton class implementation: C# class SingletonImplementation: ISingleton { var callMeSingleton = "" // other implementation public SetSingleton(string value) { callMeSingleton = value; } // other implementation } Transient class implementation: C# class TransientImplementation: ITransient { var callMeTransient = "" // other implementation public SetTransient(string value) { callMeTransient = value; } // other implementation } Scoped class implementation: C# class ScopedImplementation: IScoped { var callMeScoped = "" //other implementation public SetScoped(string value) { callMeScoped = value; } //other implementation } Let's register (ConfigureServices) with DI (Dependency Injection) to decide the life cycle of each service instance: C# services.AddSingleton<ISingleton, SingletonImplementation>(); services.AddTransient<ITransient , TransientImplementation>(); services.AddScoped<IScoped , ScopedImplementation>(); Let's use/call these services from 3 different classes (ClassA, ClassB, and ClassC) to understand the life cycle of each service: ClassA: C# public class ClassA { private ISingleton _singleton; //constructor to instantiate 3 different services we creates public ClassA(ISingleton singleton, ITransient _transient, IScoped _scoped) { _singleton = singleton; } public void UpdateSingletonFromClassA() { _singleton.SetSingleton("I am from ClassA"); } public void UpdateTransientFromClassA() { _transient.SetTransient("I am from ClassA"); } public void UpdateScopedFromClassA() { _scoped.SetScoped("I am from ClassA"); } // other implementation } ClassB: C# public class ClassB { private ISingleton _singleton; //constructor to instantiate 3 different services we creates public ClassB(ISingleton singleton, ITransient _transient, IScoped _scoped) { _singleton = singleton; } public void UpdateSingletonFromClassB() { _singleton.SetSingleton("I am from ClassB"); } public void UpdateTransientFromClassB() { _transient.SetTransient("I am from ClassB"); } public void UpdateScopedFromClassB() { _scoped.SetScoped("I am from ClassB"); } // other implementation } ClassC: C# public class ClassC { private ISingleton _singleton; //constructor to instantiate 3 different services we creates public ClassC(ISingleton singleton, ITransient _transient, IScoped _scoped) { _singleton = singleton; } public void UpdateSingletonFromClassC() { _singleton.SetSingleton("I am from ClassC"); } public void UpdateTransientFromClassC() { _transient.SetTransient("I am from ClassC"); } public void UpdateScopedFromClassC() { _scoped.SetScoped("I am from ClassC"); } // other implementation } Analysis Let's analyze the results and behavior for each life cycle one by one from the above implementation: Singleton All the classes (ClassA, ClassB, and ClassC) will use the same single instance of the SingletonImplementation class throughout the lifecycle of the application. This means that properties, fields, and operations of the SingletonImplementation class will be shared among instances used on all calling classes. Any updates to properties or fields will override previous changes. For example, in the code above, ClassA, ClassB, and ClassC are all utilizing the SingletonImplementation service as a singleton instance and calling SetSingleton to update the callMeSingleton variable. In this case, there will be a single value of the callMeSingleton variable for all requests trying to access this property. Whichever class accesses it last to update will override the value of callMeSingleton. ClassA - It will have its same instance as other classes for service TransientImplementation. ClassB - It will have its same instance as other classes for service TransientImplementation. ClassC - It will have its same instance as other classes for service TransientImplementation. ClassA, ClassB, and ClassC are updating the same instance of the SingletonImplementation class, which will override the value of callMeSingleton. Therefore, be careful when setting or updating properties in the singleton service implementation. Singleton services are disposed of at the end of the application (i.e., upon application restart). Transient All the classes (ClassA, ClassB, and ClassC) will use their individual instances of the TransientImplementation class. This means that if one class calls for properties, fields, or operations of the TransientImplementation class, it will only update or override its individual instance values. Any updates to properties or fields are not shared among other instances of TransientImplementation. Let's understand: ClassA - It will have its own instance of service of TransientImplementation. ClassB - It will have its own instance of service of TransientImplementation. ClassC - It will have its own instance of service of TransientImplementation. Let's say you have a ClassD which is calling transient service from ClassA, ClassB, and ClassC instances. In this case, each class instance would be treated as different/separate instance and each class would have its own value of callMeTransient. Read the inline comments below for ClassD: C# public ClassD { // other implementation // Below line of code will update the value of callMeTransient to "I am from ClassA" for the intance of ClassA only. // And it will not be changed by any next calls from Class B or B class ClassA.UpdateTransientFromClassA(); // Below line of code will update the value of callMeTransient to "I am from ClassB" for the intance of ClassB only. // And it will neither override the value for calssA instance nor will be changed by next call from Class C ClassB.UpdateTransientFromClassB(); // Below line of code will update the value of callMeTransient to "I am from ClassC" for the intance of ClassC only. // And it will neither override the value for calssA and classB instance nor will be changed by any next call from any other class. ClassC.UpdateTransientFromClassC(); // other implementation } Transient services are disposed at the end of each request. Use Transient when you want a state less behavior within the request. Scoped All the classes (ClassA, ClassB, and ClassC) will be using single instances of ScopedImplementation class for each request. This means that calls for properties/fields/operations on ScopedImplementation class will happen on single instance with in the scope of request. Any updates of properties/fields will be shared among other classes. Let's understand: ClassA - It will have its instance of service of TransientImplementation. ClassB - It will have its same instance of service of TransientImplementation as ClassA. ClassC - It will have its same instance of service of TransientImplementation as ClassA and ClassB. Let's say you have a ClassD which is calling scoped service from ClassA, ClassB, and ClassC instances. In this case, each class will have single instance of ScopedImplementation class. Read the inline comments for ClassD below. C# public class ClassD { // other implementation // Below code will update the value of callMeScoped to "I am from ClassA" for the instance of ClassA // But as it is Scoped life cycle so it is holding single instance ScopedImplementation of // Then it can be overridden by next call from ClassB or ClassC ClassA.UpdateScopedFromClassA(); // Below code will update the value of callMeScoped to "I am from ClassB" for single instance ScopedImplementation // And it will override the value of callMeScoped for classA instance too. ClassB.UpdateScopedFromClassB(); // Now if Class A will perform any operation on ScopedImplementation, // it will use the latest properties/field values which are overridden by classB. // Below code will update the value of callMeScoped to "I am from ClassC" // And it will override the value of callMeScoped for classA and ClassB instance too. ClassC.UpdateScopedFromClassC(); // now if Class B or Class A will perform any operation on ScopedImplementation , it will use the latest properties/field values which are overridden by classC // other implementation } Scoped services are disposed at the end of each request. Use Scoped when you want a stateless behavior between individual requests. Trivia Time The lifecycle of a service can be overridden by a parent service where it gets initialized. Confused? Let me explain: Let's take the same example from above classes and initialize the Transient and Scoped services from SingletonImplementation (which is a singleton) as below. That would initiate the ITransient and IScoped services and overwrite the lifecycle of these to singleton life cycle as parent service. In this case your application would not have any Transient or Scoped services (considering you just have these 3 services we were using in our examples). Read through the lines in the below code: C# public class SingletonImplementation: ISingleton { // constructor to add initialize the services. private readonly ITransient _transient private readonly IScoped _scoped SingletonImplementation(ITransient transient, IScoped scoped) { _transient = transient; // Now _transient would behave as singleton service irrespective of how it was registered as Transient _scoped = scoped; // now scoped would behave as singleton service irrespective of it being registered as Scoped } var callMeSingleton = "" // other implementation } Summary I hope the above article is helpful in understanding the topic. I would recommend try it yourself with the context set above and you will never be confused again. Singleton is the easiest to understand because once you create its instance, it will be shared across applications throughout the lifecycle of the application. On the similar lines of Singleton, Scoped instances mimic the same behavior but only throughout the lifecycle of a request across application. Transient is totally stateless, for each request and each class instance will hold its own instance of serivice.

Editor's Note: The following is an article written for and published in DZone's 2024 Trend Report, Kubernetes in the Enterprise: Once Decade-Defining, Now Forging a Future in the SDLC. Kubernetes is driving the future of cloud computing, but its security challenges require us to adopt a full-scale approach to ensure the safety of our environments. Security is not a one-size-fits-all solution; security is a spectrum, influenced by the specific context in which it is applied. Security professionals in the field rarely declare anything as entirely secure, but always as more or less secure than alternatives. In this article, we are going to present various methods to brace the security of your containers. Understanding and Mitigating Container Security Threats To keep your containerized systems secure, it's important to understand the threats they face. Just like a small leak can sink a ship, even a tiny vulnerability can cause big issues. This section will help you gain a deeper understanding of container security and will provide guidance on how to mitigate the threats that come with it. Core Principles of Container Security Attackers often target containers to hijack their compute power — a common example is to gain access for unauthorized cryptocurrency mining. Beyond this, a compromised container can expose sensitive data, including customer information and workload details. In more advanced attacks, the goal is to escape the container and infiltrate the underlying node. If the attacker succeeds, they can move laterally across the cluster, gaining ongoing access to critical resources such as user code, processing power, and valuable data across other nodes. One particularly dangerous attack method is container escape, where an attacker leverages the fact that containers share the host's kernel. If they gain elevated privileges within a compromised container, they could potentially access data or processes in other containers on the same host. Additionally, the Kubernetes control plane is a prime target. If an attacker compromises one of the control plane components, they can manipulate the entire environment, potentially taking it offline or causing significant disruption. Furthermore, if the etcd database is compromised, attackers could alter or destroy the cluster, steal secrets and credentials, or gather enough information to replicate the application elsewhere. Defense in Depth Maintaining a secure container environment requires a layered strategy that underscores the principle of defense in depth. This approach involves implementing multiple security controls at various levels. By deploying overlapping security measures, you create a system where each layer of defense reinforces the others. This way, even if one security measure is breached, the others continue to protect the environment. Figure 1. Defense-in-depth strategy Understanding the Attack Surface Part of the security strategy is understanding and managing the attack surface, which encompasses all potential points of exploitation, including container images, runtime, orchestration tools, the host, and network interfaces. Reducing the attack surface means simplifying the system and minimizing unnecessary components, services, and code. By limiting what is running and enforcing strict access controls, you decrease the opportunities for vulnerabilities to exist or be exploited, making the system more secure and harder for attackers to penetrate. Common Threats and Mitigation Strategies Let's shift our focus to the everyday threats in container security and discover the tools you can immediately put to work to safeguard your systems. Vulnerable Container Images Relying on container images with security vulnerabilities poses significant risks as these vulnerable images often include outdated software or components with publicly known vulnerabilities. A vulnerability, in this context, is essentially a flaw in the code that malicious actors can leverage to trigger harmful outcomes. An example of this is the infamous Heartbleed flaw in the OpenSSL library, which allowed attackers to access sensitive data by exploiting a coding error. When such flaws are present in container images, they create opportunities for attackers to breach systems, leading to potential data theft or service interruptions. Best practices to secure container images include the following: To effectively reduce the attack surface, start by using minimal base imagesthat include only the essential components required for your application. This approach minimizes potential vulnerabilities and limits what an attacker can exploit. Tools like Docker's FROM scratch or distroless images can help create these minimal environments. Understanding and managing container image layers is crucial as each layer can introduce vulnerabilities. By keeping layers minimal and only including what is necessary, you reduce potential attack vectors. Use multi-stage builds to keep the final image lean and regularly review and update your Dockerfiles to remove unnecessary layers. It's important to avoid using unverified or outdated images. Unverified images from public repositories may contain malware, backdoors, or other malicious components. Outdated images often have unpatched vulnerabilities that attackers can exploit. To mitigate these risks, always source images from trusted repositories and regularly update them to the latest versions. Insecure Container Runtime An insecure container runtime is a critical threat as it can lead to privilege escalation, allowing attackers to gain elevated access within the system. With elevated access, attackers can disrupt services by modifying or terminating critical processes, causing downtime and impacting the availability of essential applications. They can gain full control over the container environment, manipulating configurations to deploy malicious containers or introduce malware, which can be used as a launchpad for further attacks. Best practices for hardening the container runtime include the following: Implementing strict security boundaries and adhering to the principle of least privilege are essential for protecting the container runtime. Containers should be configured to run with only the permissions they need to function, minimizing the potential impact of a security breach. This involves setting up role-based access controls. Admission control is a critical aspect of runtime security that involves validating and regulating requests to create or update containers in the cluster. By employing admission controllers, you can enforce security policies and ensure that only compliant and secure container configurations are deployed. This can include checking for the use of approved base images, ensuring that security policies are applied, and verifying that containers are not running as root. Tools like Open Policy Agent (OPA) can be integrated into your Kubernetes environment to provide flexible and powerful admission control capabilities. Here's an example for OPA policy that acts as a gatekeeper, ensuring no container runs with root privileges: Shell package kubernetes.admission deny[msg] { input.request.kind.kind == "Pod" input.request.object.spec.containers[_].securityContext.runAsUser == 0 msg = "Containers must not run as root." } There are a few practices to avoid when securing container runtime: If a container running as root is compromised, an attacker can gain root-level access to the host system, potentially leading to a full system takeover. When containers have unrestricted access to host resources, like the file system, network, or devices, a compromised container could exploit this access to then tamper with the host system, steal sensitive data, or disrupt other services. To prevent such scenarios, use tools like seccomp and AppArmor. These tools can restrict the system calls that containers make and enforce specific security policies. By applying these controls, you can confine containers to their intended operations, protecting the host system from potential breaches or unauthorized activities. Misconfigured Kubernetes Settings Misconfigured Kubernetes settings are a significant threat as they expose the cluster to attacks through overly permissive network policies, weak access controls, and poor secrets management: Overly permissive network policies enable attackers to intercept and tamper with data. Weak access controls allow unauthorized users to perform administrative tasks, disrupt services, and alter configurations. Poor secrets management exposes sensitive information like API keys and passwords, enabling attackers to escalate privileges. Best practices for secure Kubernetes configuration are as follows: The risk of transmitting sensitive information without protection is that it can be intercepted or tampered with by malicious actors during transit. To mitigate this risk, secure all communication channels with transport layer security (TLS). Kubernetes offers tools like cert-manager to automate the management and renewal of TLS certificates. This ensures that communication between services remains encrypted and secure, thereby protecting your data from interception or manipulation. Network policies control the traffic flow between Pods and services in a Kubernetes cluster. By defining network policies, you can isolate sensitive workloads and reduce the risk of lateral movement in case of a compromise. Use Kubernetes' native NetworkPolicy resource to create rules that enforce your desired network security posture. On the other hand, it's important to avoid exposing unnecessary application ports. Exposure of ports provides multiple entry points for attackers, making the cluster more vulnerable to exploits. CI/CD Security CI/CD pipelines are granted extensive permissions, ensuring they can interact closely with production systems and manage updates. However, this extensive access also makes CI/CD pipelines a significant security risk. If compromised, attackers can exploit these broad permissions to manipulate deployments, introduce malicious code, gain unauthorized access to critical systems, steal sensitive data, or create backdoors for ongoing access. There are several best practices to implement when securing CI/CD. The first best practice is ensuring that once a container image is built and deployed, it is immutable. We always want to make sure the Pod is running on exactly what we intended. It also helps in quickly identifying and rolling back to previous stable versions if a security issue arises, maintaining a reliable and predictable deployment process. Implementing immutable deployments involves several key steps to ensure consistency and security: Assign unique version tags to each container image build, avoiding mutable tags like "latest," and use Infrastructure-as-Code tools like Terraform or Ansible to maintain consistent setups. Configure containers with read-only file systems to prevent changes post-deployment. Implement continuous monitoring with tools like Prometheus and runtime security with Falco to help detect and alert to unauthorized changes, maintaining the security and reliability of your deployments. Another best practice is implementing image vulnerability scanning in CI/CD. Vulnerability scanners meticulously analyze the components of container images, identifying known security flaws that could be exploited. Beyond just examining packages managed by tools like DNF or apt, advanced scanners also inspect additional files added during the build process, such as those introduced through Dockerfile commands like ADD, COPY, or RUN. It's important to include both third-party and internally created images in these scans as new vulnerabilities are constantly emerging. To guarantee that images are thoroughly scanned for vulnerabilities before deployment, scanning tools like Clair or Trivy can be directly embedded into your CI/CD pipeline. Do not store sensitive information directly in the source code (e.g., API keys, passwords) as this increases the risk of unauthorized access and data breaches. Use secrets management tools like SOPS, AWS Secrets Manager, or Google Cloud Secret Manager to securely handle and encrypt sensitive information. Conclusion Regularly assessing and improving Kubernetes security measures is not just important — it's essential. By implementing the strategies we introduced above, organizations can protect their Kubernetes environments, ensuring that containerized applications are more secure and resilient against challenges. In the future, we anticipate that attackers will develop more sophisticated methods to specifically bypass Kubernetes' built-in security features. As organizations increasingly rely on Kubernetes for critical workloads, attackers will likely invest time in uncovering new vulnerabilities or weaknesses in Kubernetes' security architecture, potentially leading to breaches that are more difficult to detect and mitigate. The path to a secure Kubernetes environment is clear, and the time to act is now. Prioritize security to safeguard your future. This is an excerpt from DZone's 2024 Trend Report, Kubernetes in the Enterprise: Once Decade-Defining, Now Forging a Future in the SDLC.Read the Free Report

Editor's Note: The following is an article written for and published in DZone's 2024 Trend Report, Kubernetes in the Enterprise: Once Decade-Defining, Now Forging a Future in the SDLC. In recent years, observability has re-emerged as a critical aspect of DevOps and software engineering in general, driven by the growing complexity and scale of modern, cloud-native applications. The transition toward microservices architecture as well as complex cloud deployments — ranging from multi-region to multi-cloud, or even hybrid-cloud, environments — has highlighted the shortcomings of traditional methods of monitoring. In response, the industry has standardized utilizing logs, metrics, and traces as the three pillars of observability to provide a more comprehensive sense of how the application and the entire stack is performing. We now have a plethora of tools to collect, store, and analyze various signals to diagnose issues, optimize performance, and respond to issues. Yet anyone working with Kubernetes will still say that observability in Kubernetes remains challenging. Part of it comes from the inherent complexity of working with Kubernetes, but the fact of the matter is that logs, metrics, and traces alone don't make up observability. Also, the vast ecosystem of observability tooling does not necessarily equate to ease of use or high ROI, especially given today's renewed focus on cost. In this article, we'll dive into some considerations for Kubernetes observability, challenges of and some potential solutions for implementing it, and the oft forgotten aspect of developer experience in observability. Considerations for Kubernetes Observability When considering observability for Kubernetes, most have a tendency to dive straight into tool choices, but it's advisable to take a hard look at what falls under the scope of things to "observe" for your use case. Within Kubernetes alone, we already need to consider: Cluster components – API server, etcd, controller manager, scheduler Node components – kublect, kube-proxy, container runtime Other resources – CoreDNS, storage plugins, Ingress controllers Network – CNI, service mesh Security and access – audit logs, security policies Application – both internal and third-party applications And most often, we inevitably have components that run outside of Kubernetes but interface with many applications running inside. Most notably, we have databases ranging from managed cloud offerings to external data lakes. We also have things like serverless functions, queues, or other Kubernetes clusters that we need to think about. Next, we need to identify the users of Kubernetes as well as the consumers of these observability tools. It's important to consider these personas as building for an internal-only cluster vs. a multi-tenant SaaS cluster may have different requirements (e.g., privacy, compliance). Also, depending on the team composition, the primary consumers of these tools may be developers or dedicated DevOps/SRE teams who will have different levels of expertise with not only these tools but with Kubernetes itself. Only after considering the above factors can we start to talk about what tools to use. For example, if most applications are already on Kubernetes, using a Kubernetes-focused tool may suffice, whereas organizations with lots of legacy components may elect to reuse an existing observability stack. Also, a large organization with various teams mostly operating as independent verticals may opt to use their own tooling stack, whereas a smaller startup may opt to pay for an enterprise offering to simplify the setup across teams. Challenges and Recommendations for Observability Implementation After considering the scope and the intended audience of our observability stack, we're ready to narrow down the tool choices. Largely speaking, there are two options for implementing an observability stack: open source and commercial/SaaS. Open-Source Observability Stack The primary challenge with implementing a fully open-source observability solution is that there is no single tool that covers all aspects. Instead, what we have are ecosystems or stacks of tools that cover different aspects of observability. One of the more popular tech stacks from Prometheus and Grafana Lab's suite of products include: Prometheus for scraping metrics and alerting Loki for collecting logs Tempo for distributed tracing Grafana for visualization While the above setup does cover a vast majority of observability requirements, they still operate as individual microservices and do not provide the same level of uniformity as a commercial or SaaS product. But in recent years, there has been a strong push to at least standardize on OpenTelemetry conventions to unify how to collect metrics, logs, and traces. Since OpenTelemetry is a framework that is tool agnostic, it can be used with many popular open-source tools like Prometheus and Jaeger. Ideally, architecting with OpenTelemetry in mind will make standardization of how to generate, collect, and manage telemetry data easier with the growing list of compliant open-source tools. However, in practice, most organizations will already have established tools or in-house versions of them — whether that is the EFK (Elasticsearch, Fluentd, Kibana) stack or Prometheus/Grafana. Instead of forcing a new framework or tool, apply the ethos of standardization and improve what and how telemetry data is collected and stored. Finally, one of the common challenges with open-source tooling is dealing with storage. Some tools like Prometheus cannot scale without offloading storage with another solution like Thanos or Mimir. But in general, it's easy to forget to monitor the observability tooling health itself and scale the back end accordingly. More telemetry data does not necessarily equal more signals, so keep a close eye on the volume and optimize as needed. Commercial Observability Stack On the commercial offering side, we usually have agent-based solutions where telemetry data is collected from agents running as DaemonSets on Kubernetes. Nowadays, almost all commercial offerings have a comprehensive suite of tools that combine into a seamless experience to connect logs to metrics to traces in a single user interface. The primary challenge with commercial tools is controlling cost. This usually comes in the form of exposing cardinality from tags and metadata. In the context of Kubernetes, every Pod has tons of metadata related to not only Kubernetes state but the state of the associated tooling as well (e.g., annotations used by Helm or ArgoCD). These metadata then get ingested as additional tags and date fields by the agents. Since commercial tools have to index all the data to make telemetry queryable and sortable, increased cardinality from additional dimensions (usually in the form of tags) causes issues with performance and storage. This directly results in higher cost to the end user. Fortunately, most tools now allow the user to control which tags to index and even downsample data to avoid getting charged for repetitive data points that are not useful. Be aggressive with filters and pipeline logic to only index what is needed; otherwise, don't be surprised by the ballooning bill. Remembering the Developer Experience Regardless of the tool choice, one common pitfall that many teams face is over-optimizing for ops usage and neglecting the developer experience when it comes to observability. Despite the promise of DevOps, observability often falls under the realm of ops teams, whether that be platform, SRE, or DevOps engineering. This makes it easy for teams to build for what they know and what they need, over-indexing on infrastructure and not investing as much on application-level telemetry. This ends up alienating developers to invest less time or become too reliant on their ops counterparts for setup or debugging. To make observability truly useful for everyone involved, don't forget about these points: Access. It's usually more of a problem with open-source tools, but make sure access to logs, dashboards, and alerts are not gated by unnecessary approvals. Ideally, having quick links from existing mediums like IDEs or Slack can make tooling more accessible. Onboarding. It's rare for developers to go through the same level of onboarding in learning how to use any of these tools. Invest some time to get them up to speed. Standardization vs. flexibility. While a standard format like JSON is great for indexing, it may not be as human readable and is filled with extra information. Think of ways to present information in a usable format. At the end of the day, the goals of developers and ops teams should be aligned. We want tools that are easy to integrate, with minimal overhead, that produce intuitive dashboards and actionable, contextual information without too much noise. Even with the best tools, you still need to work with developers who are responsible for generating telemetry and also acting on it, so don't neglect the developer experience entirely. Final Thoughts Observability has been a hot topic in recent years due to several key factors, including the rise of complex, modern software coupled with DevOps and SRE practices to deal with that complexity. The community has moved past the simple notion of monitoring to defining the three pillars of observability as well as creating new frameworks to help with generation, collection, and management of these telemetry data. Observability in a Kubernetes context has remained challenging so far given the large scope of things to "observe" as well as the complexity of each component. With the open source ecosystem, we have seen a large fragmentation of specialized tools that is just now integrating into a standard framework. On the commercial side, we have great support for Kubernetes, but cost control has been a huge issue. And to top it off, lost in all of this complexity is the developer experience in helping feed data into and using the insights from the observability stack. But as the community has done before, tools and experience will continue to improve. We already see significant research and advances in how AI technology can improve observability tooling and experience. Not only do we see better data-driven decision making, but generative AI technology can also help surface information better in context to make tools more useful without too much overhead. This is an excerpt from DZone's 2024 Trend Report, Kubernetes in the Enterprise: Once Decade-Defining, Now Forging a Future in the SDLC.Read the Free Report

By Yitaek Hwang CORE