DZone Spotlight

Wednesday, April 30 View All Articles »

Why I Started Using Dependency Injection in Python

By Haymang Ahuja

When I first started building real-world projects in Python, I was excited just to get things working. I had classes calling other classes, services spun up inside constructors, and everything somehow held together. But deep down, I knew something was off. Why? My code felt clunky. Testing was a nightmare. Changing one part broke three others. Adding a small feature would make me change 10 different files in the package. I couldn't quite explain why - until a seasoned software developer reviewed my code and asked, "Have you heard of dependency injection?" I hadn't. So I went down the rabbit hole. And what I found changed the way I think about structuring code. In this post, I'll walk you through what dependency injection is, how it works in Python, and why you might want to start using a library like dependency-injector. We'll use real-world examples, simple language, and practical code to understand the concept. If you're early in your dev journey or just looking to clean up messy code, this one's for you. So, What the Heck Is Dependency Injection? Let's break it down. Dependency injection (DI) is just a fancy term for "give me what I need, don't make me create it myself." Instead of a class or function creating its own dependencies (like other classes, database, clients, etc), you inject those dependencies from the outside. This makes it easier to swap them out, especially in tests, and promotes loose coupling. A Quick Real-World Analogy Imagine you're running a coffee shop. Every time someone orders coffee, you don't go build a coffee machine from scratch. You already have one, and you pass it to the barista. That's DI. The barista (your class) doesn't create the coffee machine (the dependency to serve coffee); it receives it from outside. Without DI: A Painful Example Python class DataProcessor: def __init__(self): # DataProcessor creates its own database connection self.db = MySQLDatabase(host="localhost", user="root", password="secret") def process_data(self): data = self.db.query("SELECT * from data") # Process the data... return processed_data See the problem? This works, but here's the problem. Our DataProcessor is tightly coupled to a specific MySQL database implementation. Want to switch databases? You'll have to modify this class.Want to test it with a mock database? Good luck with that. With DI: Clean and Flexible Now, here's the same code with dependency injection. Python class DataProcessor: def __init__(self, database): # Database is injected from outside self.db = database def process_data(self): data = self.db.query("SELECT * from data") # Process the data ... return processed_data # Creating and injecting dependencies db = MySQLDatabase(host="localhost", user="root", password="secret") processor = DataProcessor(db) That's it! The processor now takes its database dependency as a parameter instead of creating it internally. This simple change makes a world of difference. Using a DI Library in Python The simplest form of DI in Python is just passing dependencies through constructors (as we saw above). This is called constructor injection and works great for simple cases. But in larger applications, manually writing dependencies gets messy. That's where the dependency-injector library shines. It gives you containers, providers, and a nice way to manage your dependencies. First, let's install the package: Plain Text pip install dependency-injector Now, let's see it in action in the real world. Imagine we're building a data engineering pipeline that: Extracts data from different sources.Transforms it.Loads it into a data warehouse. Here's how we'd structure it with DI: Python from dependency-injector import containers, providers from database import PostgresDatabase, MongoDatabase from services import DataExtractor, DataTransformer, DataLoader class Container(containers.DeclarativeContainer): config = providers.Configuration() # Database dependencies postgres_db = providers.Singleton( PostgresDatabase, host=config.postgres.host, username=config.postgres.username, password=config.postgres.password ) mongo_db = providers.Singleton( MongoDatabase, connection_string=config.mongo.connection_string ) # Service dependencies extractor = providers.Factory( DataExtractor, source_db=mongo_db ) transformer = providers.Factory( DataTransformer ) loader = providers.Factory( DataLoader, target_db=postgres_db ) # Main application etl_pipeline = providers.Factory( ETLpipeline, extractor=extractor, transfomer=transformer, loader=loader ) Let's break down what's happening here: Containers A container is like a registry of your application's dependencies. It knows how to create each component and what dependencies each component needs. Providers Providers tell the container how to create objects. There are different types: Factory: Creates a new instance each time (good for most services)Singleton: Creates only one instance (good for connections, for example)Configuration: Provides configuration valuesAnd many more... (check out the complete list following the link above) Using the Container Python # Load configuration from a file container = Container() container.config.from_yaml('config.yaml') # Create the pipeline with all dependencies resolved automatically pipeline = container.etl_pipeline() pipeline.run() The magic here is that our ETLPipeline class doesn't need to know how to create extractors, transformers, or loaders. It just uses what it's given: Python class ETLPipeline: def __init__(self, extractor, transformer, loader): self.extractor = extractor self.transformer = transformer self.loader = loader def run(self): data = self.extractor.extract() transformed_data = self.transformer.transform(data) self.loader.load(transformed_data) Wrapping Up Dependency injection might sound like something only big enterprise apps need. But if you've ever tried to write tests or refactor your code and wanted to scream, DI can save your sanity. The dependency-injector library in Python gives you an elegant way to structure your code, keep things loosely coupled, and actually enjoy testing. Final Thoughts Next time you're writing a class that needs "stuff" to work, pause and ask: Should I be injecting this instead? It's a simple shift in mindset, but it leads to cleaner code, better tests, and happier developers (you included). If you found this helpful, give it a clap or share it with a teammate who's knee-deep in spaghetti code. And if you want more articles like this — real talk, real examples — follow me here on DZone. Let's write better Python, one clean service at a time. More

Filtering Messages With Azure Content Safety and Spring AI

By Tamas Kiss

As AI-powered applications like chatbots and virtual assistants become increasingly integrated into our daily lives, ensuring that they interact with users in a safe, respectful, and responsible manner is more important than ever. Unchecked user input or AI-generated content can lead to the spread of harmful language, including hate speech, sexually explicit content, or content promoting violence or self-harm. This can negatively affect the user experience and may also lead to legal or ethical problems. Microsoft's Azure Content Safety service provides a powerful toolset for automatically classifying and evaluating text (and image) content for various harm categories. By using it, developers can build safer, more reliable AI-driven applications without implementing complex moderation logic from scratch. In this tutorial, we'll walk through how to integrate Azure Content Safety into a Spring Boot application using Spring AI and filter user messages in real time. The app will check incoming messages against Azure’s classification engine, and only forward them to an OpenAI model (via Spring AI) if they meet safety thresholds. Azure Content Safety Deployment Before integrating Azure Content Safety into the application, you need to deploy the service to your Azure subscription. For this tutorial, we’ll use Terraform to automate the setup. The provided Terraform template will create the following: A resource groupAn Azure Content Safety instance (with free F0 tier by default) Prerequisites To run the Terraform deployment, make sure you have the following tools installed: Azure CLITerraform You will also need to authenticate Terraform to Azure. One recommended way is to use a service principal. You can create one with the following command: Shell az ad sp create-for-rbac --name "terraform-sp" --role="Contributor" --scopes="/subscriptions/<your-subscription-id>" --output json > terraform-sp.json Then export the required environment variables: Shell export ARM_CLIENT_ID="appId from terraform-sp.json" export ARM_CLIENT_SECRET="password from terraform-sp.json" export ARM_TENANT_ID="tenant from terraform-sp.json" export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv) Deploying With Terraform The deployment is defined in a set of Terraform configuration files. The Terraform template (main.tf) looks like this: Plain Text terraform { required_providers { azurerm = { source = "hashicorp/azurerm" version = ">=4.1.0" } } } provider "azurerm" { features {} } resource "azurerm_resource_group" "rg" { name = var.resource_group_name location = var.location } resource "azurerm_cognitive_account" "content_safety" { name = var.cognitive_account_name location = azurerm_resource_group.rg.location resource_group_name = azurerm_resource_group.rg.name kind = "ContentSafety" sku_name = var.sku_name identity { type = "SystemAssigned" } tags = { environment = "dev" } } output "content_safety_endpoint" { value = azurerm_cognitive_account.content_safety.endpoint } output "content_safety_primary_key" { value = azurerm_cognitive_account.content_safety.primary_access_key sensitive = true } resource "local_file" "content_safety_output" { filename = "${path.module}/outputs.json" content = jsonencode({ endpoint = azurerm_cognitive_account.content_safety.endpoint key = azurerm_cognitive_account.content_safety.primary_access_key }) file_permission = "0600" } The variables.tf file looks like this: Plain Text variable "resource_group_name" { description = "The name of the resource group" type = string default = "rg-content-safety" } variable "location" { description = "Azure region" type = string default = "westeurope" } variable "cognitive_account_name" { description = "Globally unique name for the Cognitive Services account" type = string default = "contentsafetydemo123" # must be globally unique } variable "sku_name" { description = "SKU name for the account (e.g., F0 for Free Tier)" type = string default = "F0" } And here is an example terraform.tfvars for deploying to the free tier: Plain Text resource_group_name = "rg-my-content-safety" location = "westeurope" cognitive_account_name = "contentsafetyexample456" sku_name = "F0" After defining these files, you can initialize and apply the deployment as follows: Shell terraform init terraform apply At the end of the process, Terraform will output the Azure Content Safety endpoint and key. These values will also be saved to a local outputs.json file, which will be used later during the application configuration. Dependencies Since spring-ai is still only available as a milestone release, you need to add the Spring Milestone repository to your Maven configuration. You can do this by including the following snippet in your pom.xml: XML <repositories> <repository> <id>central</id> <name>Central</name> <url>https://repo1.maven.org/maven2/</url> </repository> <repository> <id>spring-milestones</id> <name>Spring Milestones</name> <url>https://repo.spring.io/milestone</url> <snapshots> <enabled>false</enabled> </snapshots> </repository> </repositories> You also need to include the spring-ai-bom in your dependency management section: XML <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-bom</artifactId> <version>1.0.0-M6</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> After that, you can add the following dependencies: spring-boot-starter-web – for building the REST APIspring-ai-openai-spring-boot-starter – for integrating with OpenAI via Spring AIazure-ai-contentsafety – the official Azure Content Safety client library for Java Here's how to include them in your pom.xml: XML <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-openai-spring-boot-starter</artifactId> </dependency> <dependency> <groupId>com.azure</groupId> <artifactId>azure-ai-contentsafety</artifactId> <version>1.0.11</version> </dependency> </dependencies> Setting Up the Chat Client To get an OpenAI ChatClient working, the first step is to provide the required API key. This can be done using the following property: YAML spring: ai: openai: api-key: ${OPENAI_API_KEY} The ChatClient itself, which serves as the interface to communicate with the underlying chat model, can be configured like this: Java @Bean("mainChatClient") ChatClient mainChatClient(ChatClient.Builder clientBuilder, List<Advisor> advisors) { return clientBuilder .defaultAdvisors(advisors) .build(); } Note: By default, the ChatClient will use the gpt-4o-mini model, but this can be overridden using the spring.ai.openai.chat.options.model property. If you wire the ChatClient into a RestController like this, you can start interacting with the chat model through the /chat/send endpoint: Java @RequestMapping("/chat") @RestController public class ChatController { private final ChatClient chatClient; public ChatController(@Qualifier("mainChatClient") ChatClient chatClient) { this.chatClient = chatClient; } public record UserMessage(String message) { } public record AiMessage(String message) { } @PostMapping(value = "/send") public ResponseEntity<AiMessage> sendMessage(@RequestBody UserMessage userMessage) { return ResponseEntity.ok(new AiMessage(chatClient.prompt(userMessage.message()).call().content())); } } As shown in the ChatClient configuration above, you can register so-called advisors to a given client. But what are these advisors? Advisors in Spring AI are components that participate in a chain, allowing you to customize how chat requests are built and how responses are handled. Each advisor in the chain can add logic before the request is sent to the model or after the response is returned. They are useful for injecting system instructions, adding safety checks, or modifying the prompt dynamically without putting this logic directly into the main application code. So we have the ability to create our own Advisor implementation that can check each user prompt using the Azure Content Safety service, based on its harm categories, before the prompt reaches the ChatModel. Harm Categories Azure Content Safety can analyze a user message and classify it into one or more predefined harm categories. These categories represent different types of potentially dangerous or inappropriate content. The classification is multi-label, meaning a single message can be associated with multiple categories at once. There are four built-in harm categories: Hate – Targets or discriminates against individuals or groups based on attributes such as race, ethnicity, nationality, gender identity, sexual orientation, religion, immigration status, ability status, personal appearance, or body size.Sexual – Covers content related to sexual acts, anatomy, romantic or erotic expressions, sexual assault, pornography, or abuse.Violence – Describes or encourages physical harm, injury, or the use of weapons, including violent threats or references to violent acts.Self-harm – Includes language about hurting oneself or suicide. Each harm category comes with a severity level rating. These levels indicate how serious the content is and help you decide what kind of action to take. The severity score ranges from 0 to 7. By default, Azure groups these values into four simplified levels: Scores between 0 and 1 → level 0Scores between 2 and 3 → level 2Scores between 4 and 5 → level 4Scores between 6 and 7 → level 6 However, if you need more fine-grained detail, you can configure the API request to return the original 0–7 scale instead of the simplified version. To configure the Azure Content Safety client and enable harm category–based filtering, I defined the following properties in the application: YAML contentsafety: azure-content-safety: enabled: true endpoint: ${AZURE_CONTENT_SAFETY_ENDPOINT:} key: ${AZURE_CONTENT_SAFETY_KEY:} category-thresholds: -"Hate": 1 -"SelfHarm": 1 -"Sexual": 1 -"Violence": 1 In the above configuration: enabled: Controls whether the Azure Content Safety check is active.endpoint: URL of the deployed Azure Content Safety resource. This value can be found in the outputs.json file generated by the Terraform deployment.key: Access key for authenticating requests. This is also available in the outputs.json file.category-thresholds: Defines the severity level threshold for each harm category. If the severity of any category in a user message exceeds the configured threshold, the message will be rejected. Instead of simply returning a static response like "Rejected message!" when a prompt violates one or more harm category thresholds, the ContentSafetyAdvisor used in this application provides a short explanation. This explanation is generated using a separate ChatClient, which receives the original user message along with the harm category results returned by Azure Content Safety. This secondary client is configured with system instructions that help it generate a polite and informative response. Here's how the rejectExplanationChatClient is set up: Java @Bean("rejectExplanationChatClient") ChatClient rejectExplanationChatClient(ChatClient.Builder clientBuilder) { return clientBuilder .defaultSystem(""" You are a helpful assistant who can give a polite message rejection explanation. All the user messages are checked for content safety by a dedicated service. If the message is rejected, you should explain why the message is rejected. You will get the original message and the results of the content safety service where the content safety result will show the severity of the message in different categories. The severity values are on a scale from 0 to 7, where 0 is the least severe and 7 is the most severe. """) .build(); } The ContentSafetyAdvisor, which is responsible for checking incoming user messages and returning a rejection message if the harm category thresholds are exceeded, looks like this: Java public class ContentSafetyAdvisor implements CallAroundAdvisor { private static final Logger LOGGER = LoggerFactory.getLogger(ContentSafetyAdvisor.class); private static final PromptTemplate REJECTION_PROMPT_TEMPLATE = new PromptTemplate(""" Explain politely why the message is rejected. The rejected message is: {message} The content safety result is: {contentSafetyResult} """); private final ChatClient rejectExplanationChatClient; private final ContentSafetyClient contentSafetyClient; private final Map<String, Integer> categoryThresholds; private final ObjectMapper objectMapper; private final int order; public ContentSafetyAdvisor(ContentSafetyClient contentSafetyClient, ChatClient rejectExplanationChatClient, Map<String, Integer> categoryThresholds, ObjectMapper objectMapper, int order) { this.contentSafetyClient = contentSafetyClient; this.rejectExplanationChatClient = rejectExplanationChatClient; this.categoryThresholds = categoryThresholds; this.objectMapper = objectMapper; this.order = order; } @Override public AdvisedResponse aroundCall(AdvisedRequest advisedRequest, CallAroundAdvisorChain chain) { try { var analyzeTextResult = analyzeText(advisedRequest.userText()); if (!isMessageSafe(analyzeTextResult)) { var rejectExplanation = provideRejectExplanation(advisedRequest.userText(), analyzeTextResult); return createResponse(rejectExplanation, advisedRequest.adviseContext()); } } catch (Exception e) { return createResponse("I'm sorry, I can't answer you now.", advisedRequest.adviseContext()); } return chain.nextAroundCall(advisedRequest); } @Override public String getName() { return "ContentSafetyAdvisor"; } @Override public int getOrder() { return this.order; } private AnalyzeTextResult analyzeText(String text) throws Exception { var request = new AnalyzeTextOptions(text); // request.setOutputType(AnalyzeTextOutputType.EIGHT_SEVERITY_LEVELS); Severity levels from 0 to 7 // request.setOutputType(AnalyzeTextOutputType.FOUR_SEVERITY_LEVELS); Severity levels in {0, 2, 4, 6} (the default). var result = contentSafetyClient.analyzeText(request); LOGGER.info("AnalyzeTextResult of message '{}': {}", text, objectMapper.writeValueAsString(result)); return result; } private boolean isMessageSafe(AnalyzeTextResult analyzeTextResult) { for (var categoryAnalysis : analyzeTextResult.getCategoriesAnalysis()) { if (categoryAnalysis.getSeverity() > categoryThresholds.getOrDefault(categoryAnalysis.getCategory().getValue(), Integer.MAX_VALUE)) { return false; } } return true; } private String provideRejectExplanation(String message, AnalyzeTextResult analyzeTextResult) throws JsonProcessingException { return rejectExplanationChatClient.prompt(REJECTION_PROMPT_TEMPLATE .create(Map.of( "message", message, "contentSafetyResult", objectMapper.writeValueAsString(analyzeTextResult)))) .call().content(); } private AdvisedResponse createResponse(String responseMessage, Map<String, Object> adviseContext) { return new AdvisedResponse(ChatResponse.builder() .generations(List.of(new Generation(new AssistantMessage(responseMessage)))) .build(), adviseContext); } } Message Filtering As shown in the configuration above, I set the default category-threshold for each harm category to a low value of 1. This allows me to demonstrate both an accepted and a rejected message without needing to include anything truly offensive. Below are two simple examples: An accepted message A rejected message Conclusion I hope this short article on integrating Azure Content Safety was helpful for you. The full source code is available here on GitHub. While this tutorial focused on the built-in harm categories provided by Azure, it's also possible to train and use custom categories. But maybe that’s a topic for another time. More

Trend Report

Generative AI

AI technology is now more accessible, more intelligent, and easier to use than ever before. Generative AI, in particular, has transformed nearly every industry exponentially, creating a lasting impact driven by its (delivered) promises of cost savings, manual task reduction, and a slew of other benefits that improve overall productivity and efficiency. The applications of GenAI are expansive, and thanks to the democratization of large language models, AI is reaching every industry worldwide.Our focus for DZone's 2025 Generative AI Trend Report is on the trends surrounding GenAI models, algorithms, and implementation, paying special attention to GenAI's impacts on code generation and software development as a whole. Featured in this report are key findings from our research and thought-provoking content written by everyday practitioners from the DZone Community, with topics including organizations' AI adoption maturity, the role of LLMs, AI-driven intelligent applications, agentic AI, and much more.We hope this report serves as a guide to help readers assess their own organization's AI capabilities and how they can better leverage those in 2025 and beyond.

Refcard #158

Machine Learning Patterns and Anti-Patterns

By Tuhin Chattopadhyay

CORE

Machine Learning Patterns and Anti-Patterns

Refcard #269

Getting Started With Data Quality

By Miguel Garcia

CORE

*You* Can Shape Trend Reports: Join DZone's Software Supply Chain Security Research

Hey, DZone Community! We have an exciting year of research ahead for our beloved Trend Reports. And once again, we are asking for your insights and expertise (anonymously if you choose) — readers just like you drive the content we cover in our Trend Reports. Check out the details for our research survey below. Software Supply Chain Security Research Supply chains aren't just for physical products anymore; they're a critical part of how software is built and delivered. At DZone, we're taking a closer look at the state of software supply chain security to understand how development teams are navigating emerging risks through smarter tooling, stronger practices, and the strategic use of AI. Take our short research survey (~10 minutes) to contribute to our upcoming Trend Report. We're exploring key topics such as: SBOM adoption and real-world usageThe role of AI and ML in threat detectionImplementation of zero trust security modelsCloud and open-source security posturesModern approaches to incident response Join the Security Research We’ve also created some painfully relatable memes about the state of software supply chain security. If you’ve ever muttered “this is fine” while scanning dependencies, these are for you! Over the coming month, we will compile and analyze data from hundreds of respondents; results and observations will be featured in the "Key Research Findings" of our Trend Reports. Your responses help inform the narrative of our Trend Reports, so we truly cannot do this without you. Stay tuned for each report's launch and see how your insights align with the larger DZone Community. We thank you in advance for your help! —The DZone Content and Community team

By Lauren Forbes

Building Data Pipelines With Jira API

I’ve spent years building data pipelines and connecting project management to technical workflows. Disconnected systems lead to manual errors and delays, problems that Jira’s API helps solve. This tool lets code interact directly with project boards, automating tasks such as creating tickets when data checks fail or updating statuses after ETL. For data engineers, the API bridges Jira and databases. Extract issue details into warehouses, build dashboards linking pipeline performance to project effort, or trigger data workflows from Jira events. It’s about seamless integration. In this guide, I want to walk you through how I use the Jira API in my day-to-day data engineering work. We'll cover everything from basic authentication to creating issues automatically, querying data using JQL, and even syncing custom fields with databases. My goal is to give you practical, hands-on examples you can adapt for your own pipelines, helping you connect your project management directly to your database and data processing workflows. Let me show you how to automate tasks, extract the data you need, and integrate Jira smoothly into your data engineering stack. Setting Up Your Jira API Environment First, create an API token in Jira Cloud: Go to your Atlassian account settings.Under Security, generate an API token. Here’s how to authenticate using Python: Python import requests auth = ("your-email@domain.com", "API_TOKEN") For cURL: Python curl -u "email:API_TOKEN" https://your-domain.atlassian.net/rest/api/3/issue I always test connectivity by fetching basic project data: Python response = requests.get("https://your-domain.atlassian.net/rest/api/3/project", auth=auth) print(response.json()) Creating Issues Programmatically Use this JSON template to create tickets. Replace PROJECT_KEY and ISSUE_TYPE_ID with your project’s values (found via Jira’s metadata API): JSON { "fields": { "project": { "key": "PROJECT_KEY" }, "summary": "Data pipeline failure", "issuetype": { "id": "10001" }, "description": { "type": "doc", "content": [{"type": "text", "text": "Alert from our monitoring system"}] } } } Send it via Python: Python url = "https://your-domain.atlassian.net/rest/api/3/issue" headers = { "Content-Type": "application/json" } response = requests.post(url, json=issue_data, headers=headers, auth=auth) Querying Data for Analytics Extract ticket data using JQL (Jira Query Language). This example fetches all bugs from the last 7 days: Python jql = "project = PROJECT_KEY AND issuetype = Bug AND created >= -7d" response = requests.get( f"https://your-domain.atlassian.net/rest/api/3/search?jql={jql}", auth=auth ) Store results in a PostgreSQL database: Python import psycopg2 data = response.json()["issues"] conn = psycopg2.connect("dbname=etl user=postgres") cur = conn.cursor() for issue in data: cur.execute("INSERT INTO jira_issues VALUES (%s, %s)", (issue["key"], issue["fields"]["summary"])) conn.commit() Syncing Custom Fields With Databases Jira’s database schema (source) uses tables like customfield and jiraissue. While direct database access isn’t recommended, here’s how to map API data to SQL: 1. Fetch custom field metadata: Python custom_fields = requests.get("https://your-domain.atlassian.net/rest/api/3/field", auth=auth).json() 2. Create a database table dynamically: Python columns = ["issue_key VARCHAR PRIMARY KEY"] for field in custom_fields: columns.append(f"{field['id']} VARCHAR") cur.execute(f"CREATE TABLE jira_custom_fields ({', '.join(columns)})") Automating Workflows Trigger data pipeline runs when Jira tickets update. Use webhooks: 1. Set up a Flask endpoint: Python from flask import Flask, request app = Flask(__name__) @app.route("/webhook", methods=["POST"]) def handle_webhook(): data = request.json if data["issue"]["fields"]["status"] == "Done": # Start your ETL job here return "OK" 2. Configure the webhook in Jira’s settings. When to Work With Jira Consultants While the API is powerful, complex setups like custom schema migrations or large-scale automation might require expertise. Jira consultants often help teams design these systems, like optimizing how ticket data flows into data lakes or aligning Jira workflows with CI/CD pipelines. Troubleshooting Common Issues API Rate Limits Jira Cloud allows 100 requests/minute. Use exponential backoff: Python import time def make_request(url): for _ in range(5): response = requests.get(url, auth=auth) if response.status_code != 429: return response time.sleep(2 ** _) Data Type Mismatches Jira’s timeworked field stores seconds, while many databases use intervals. Convert during ingestion: Python timeworked_seconds = issue["fields"]["worklog"]["total"] timeworked_interval = f"{timeworked_seconds // 3600}:{(timeworked_seconds % 3600) // 60}:{timeworked_seconds % 60}" What I've shown you here really just gets you started. Think of it like learning the basic chords on a guitar; you can play some simple songs now, but there's a whole world of music still out there. These pieces — authenticating, creating issues, searching data, using webhooks – are the essential building blocks. With these under your belt, you can start building some really useful connections between Jira and your data world. For example, you can absolutely turn Jira into a live, real-time data source for your monitoring and reporting. Imagine dashboards that don't just show database performance, but also display the open engineering tickets related to that specific database. You could pull data on how long issues stay in different statuses and feed that into Grafana or Kibana to visualize bottlenecks in your team's workflow. By regularly fetching data via the API, you get a constantly updated picture, much more alive than static reports. And triggering your data pipelines directly from Jira events? That opens up serious automation possibilities. We touched on using webhooks for this. Think about it: a Jira issue moves to 'Ready for Deployment'. A webhook fires, triggering your CI/CD pipeline to deploy a new data transformation script. Or, a new bug ticket is created with a specific tag like 'Data Quality'. A webhook could automatically trigger a diagnostic script to gather more information about the failure and add it to the ticket description. This links project decisions directly to technical actions, cutting out manual steps and delays. Don't overlook the potential locked away in your past Jira tickets either. There are often years of history sitting there. Using the API, you can extract all that historical data — things like how long tasks actually took versus their estimates, which components had the most bugs, or how quickly critical issues were resolved. This historical data is gold dust for analysis and even machine learning. You could train models to better predict how long future tasks might take, identify patterns that lead to recurring problems, or even forecast potential support load based on recent activity. The API is your key to unlocking that historical treasure chest. But the most important piece of advice I can give is "start small". Seriously. Don't try to build a massive, all-singing, all-dancing integration on day one. You'll likely get overwhelmed. Pick one simple, concrete pain point you have. Maybe it's manually creating tickets for ETL failures. Automate just that one thing first. Get it working reliably. See the time it saves you. Feel that little win. Then, pick the next small thing. Maybe it's pulling a weekly report of completed tasks into a database table. Build that. This step-by-step approach keeps things manageable. You learn as you go, build confidence, and gradually create a more connected system that truly works for you and your team. Expand outwards from those small successes. That’s how you make real progress.

By Saeed Ashif Ahmed

How to Convert Between PDF and TIFF in Java

PDF and TIFF: Converting Between Document and Image in Java We rarely encounter just one document format in enterprise applications. The longer a system has been in production, the more likely it is that file interoperability becomes a real concern. That’s especially true for file types that sit at the intersection of document and image processing, like PDF and TIFF. TIFF and PDF are both widely used in healthcare, insurance, and legal services (among other industries) where a premium is placed upon long-term file fidelity and visual accuracy. While PDF has a much wider range of use-cases, TIFF holds ground in archival contexts and systems that prefer image-based representations over embedded formatting. Introduction In this article, we’ll look at what it means to convert between PDF and TIFF formats, in both directions. We’ll first break down how these formats store content, why that matters when we’re making a conversion between each format, and what kinds of considerations Java developers should keep in mind when switching between them. Finally, we’ll explore some open-source and third-party APIs that Java developers can use to streamline these conversions in production code. Understanding the (Significant) Structural Differences Between PDF and TIFF PDF and TIFF are both designed for viewing static content, but they couldn’t be more different in terms of composition. TIFF is raster-based —meaning it stores image data pixel by pixel — while PDF is a container that can mix text, images, and vector graphics all in one file. A PDF can be one or more static, raster-based images, which is one way it overlaps with TIFF in terms of real-world functionality. PDFs are designed to support layers, fonts, compression types, and even embedded scripts. TIFF files, on the other hand, lean heavily into visual fidelity and long-term preservation. Many TIFFs use CCITT or LZW compression and are commonly leaned upon for precision, such as single-bit black-and-white image scanning or medical imaging. When we convert between TIFF and PDF, we’re not just changing file extensions — we’re changing how the file's internal contents are represented. That means we must choose how to properly convert PDFs into valid TIFF images, and how to encapsulate TIFF images into a valid PDF structure that still renders correctly across viewers. Converting From PDF to TIFF: Rendering Pages as Image Data When converting a PDF to TIFF, we’re effectively rendering each page of the PDF as an image, then encoding that image in lossless TIFF format. Under the hood, that involves selecting a rendering resolution (DPI), deciding whether to use grayscale, RGB, or black-and-white output, and finding a way to manage multipage TIFF creation if the original PDF has more than one page. Probably the biggest consideration in this direction of our conversion is resolution. A higher DPI results in better visual fidelity, but also larger file sizes. For documents like scanned contracts or forms, a DPI of 200–300 is typically sufficient. For more detailed visual content, we might need to go higher. Converting From TIFF to PDF: Wrapping Image Data in a Page Structure When converting in the other direction — TIFF to PDF — the process flips: we’re taking big, detailed raster images and wrapping them in a valid PDF page structure. This means we’re making decisions about page sizes, margins, orientation, and lossy or lossless compression algorithms (e.g., using JPEG or Flate inside the PDF). If the TIFF we’re converting from is a multipage image, we’ll need to create a separate PDF page for each frame of the TIFF. It's important to understand that we’re not embedding a TIFF inside a PDF — we’re re-encoding its contents into a different format entirely. Open-Source Libraries for PDF-TIFF Conversion For those looking to handle conversions to and from PDF and TIFF with open-source solutions, there are several established libraries that stand out. PDF to TIFF With Apache PDFBox and JAI Apache PDFBox is a robust and well-maintained library for working with PDF files in Java. Most importantly, it includes built-in support for rendering PDF pages to BufferedImage instances, which can then be encoded into TIFF format using the Java Advanced Imaging (JAI) API or standard ImageIO plugins. PDFBox gives us control over rendering DPI, color models, and page iteration, which makes it a solid choice for export workflows. TIFF to PDF With iText or OpenPDF To go from TIFF to PDF, we'll need a library that can build PDFs from image data. To that end, iText (and its fork, OpenPDF) offers a clean way to insert image content into new PDF documents, one image per page. With a multipage TIFF, we'd extract each frame as a separate image and append them sequentially to our resulting PDF. These libraries handle PDF layout and compression settings, which takes a lift off our hands. We can fine-tune the output as needed. These tools don’t abstract away all the complexity, but they’re reliable, well-documented, and used in many production environments. They'll take us where we need to go. Converting PDF to TIFF, TIFF to PDF With Web APIs If we’re not looking to get intimately involved in open-source documentation, we can alternatively try a pair of free plug-and-play conversion APIs that integrate with our Java application using minimal code. Code examples are provided below in this case. These aren’t open-source, which may be a no-go for some, but the idea here is to abstract and simplify all the cumbersome aspects of a high-fidelity conversion away from our environment entirely, including the overhead processing that would otherwise fall on our resources to support. While not particularly relevant here, it's also worth noting that each of these APIs also supports conversions from a variety of other image formats (like WEBP, JPG, PNG, etc.). PDF to TIFF The first thing we'll do is install the SDK with Maven. We'll first add a reference to the repository in pom.xml: XML <repositories> <repository> <id>jitpack.io</id> <url>https://jitpack.io</url> </repository> </repositories> And then we'll add a reference to the dependency in pom.xml: XML <dependencies> <dependency> <groupId>com.github.Cloudmersive</groupId> <artifactId>Cloudmersive.APIClient.Java</artifactId> <version>v4.25</version> </dependency> </dependencies> Following that, we'll add the import classes to the top of our file: Java // Import classes: //import com.cloudmersive.client.invoker.ApiClient; //import com.cloudmersive.client.invoker.ApiException; //import com.cloudmersive.client.invoker.Configuration; //import com.cloudmersive.client.invoker.auth.*; //import com.cloudmersive.client.ConvertApi; Next, we'll initialize the API client, set an API key for authorization (we can get one of these for free), create a conversion API instance, and call the API with our PDF file input: Java ApiClient defaultClient = Configuration.getDefaultApiClient(); // Configure API key authorization: Apikey ApiKeyAuth Apikey = (ApiKeyAuth) defaultClient.getAuthentication("Apikey"); Apikey.setApiKey("YOUR API KEY"); // Uncomment the following line to set a prefix for the API key, e.g. "Token" (defaults to null) //Apikey.setApiKeyPrefix("Token"); ConvertApi apiInstance = new ConvertApi(); File imageFile = new File("/path/to/inputfile"); // File | Image file to perform the operation on. Common file formats such as PNG, JPEG are supported. try { byte[] result = apiInstance.convertToTiff(imageFile); System.out.println(result); } catch (ApiException e) { System.err.println("Exception when calling ConvertApi#convertToTiff"); e.printStackTrace(); } We'll get our TIFF image back as a byte array, which we can then write to a new TIFF file. TIFF to PDF We'll follow similar instructions here. We'll once again install the SDK with our pom.xml reference: XML <repositories> <repository> <id>jitpack.io</id> <url>https://jitpack.io</url> </repository> </repositories> And our pom.xml dependency: XML <dependencies> <dependency> <groupId>com.github.Cloudmersive</groupId> <artifactId>Cloudmersive.APIClient.Java</artifactId> <version>v4.25</version> </dependency> </dependencies> Then we'll add our imports: Java // Import classes: //import com.cloudmersive.client.invoker.ApiClient; //import com.cloudmersive.client.invoker.ApiException; //import com.cloudmersive.client.invoker.Configuration; //import com.cloudmersive.client.invoker.auth.*; //import com.cloudmersive.client.ConvertDocumentApi; Finally, we'll initialize conversion iteration following the same overarching structure as the prior: Java ApiClient defaultClient = Configuration.getDefaultApiClient(); // Configure API key authorization: Apikey ApiKeyAuth Apikey = (ApiKeyAuth) defaultClient.getAuthentication("Apikey"); Apikey.setApiKey("YOUR API KEY"); // Uncomment the following line to set a prefix for the API key, e.g. "Token" (defaults to null) //Apikey.setApiKeyPrefix("Token"); ConvertDocumentApi apiInstance = new ConvertDocumentApi(); File inputFile = new File("/path/to/inputfile"); // File | Input file to perform the operation on. try { byte[] result = apiInstance.convertDocumentAutodetectToPdf(inputFile); System.out.println(result); } catch (ApiException e) { System.err.println("Exception when calling ConvertDocumentApi#convertDocumentAutodetectToPdf"); e.printStackTrace(); } Similar to our other conversion, this will return a PDF byte array, which we can write to a PDF with the proper extension. Conclusion In this article, we explored the relationship between PDF and TIFF formats and how files can be programmatically converted between them. We suggested open-source and non-open-source API solutions to simplify the implementation of this conversion workflow in a Java environment.

By Brian O'Neill

CORE

Scalability 101: How to Build, Measure, and Improve It

In this post, I'd like to talk a little about scalability from a system design perspective. In the following paragraphs, I'll cover multiple concepts related to scalability—from defining what it is, to the tools and approaches that help make the system more scalable, and finally, to the signs that show whether a system is scaling well or not. What Is Scalability? First things first: I’m pretty sure you know what scalability is, but let’s have a brief look at the definition just to be safe. Scalability is the ability—surprise—of a system or application to scale. It is probably one of the most crucial non-business features of every modern-day piece of code. After all, we all want our systems to be able to handle increasing traffic and not just crash the first chance they can. Besides just making our system work properly under increased load, good scalability offers other benefits: Stable User Experience - We can ensure the same level of experience for an increased number of users trying to use our system.Future Proof - The more we think about the scalability and extensibility of our system during the design phase, the smaller the chance that we will need an architecture rework in the near future.Competitive Advantage - Stable user experience under an extensive increase in traffic can shortly evolve into a business advantage, especially when some competitors’ sites are down while our site is up and running. In the most simple way, when we’re looking at scaling, there are two ways to handle that task—Horizontal and Vertical. Horizontal scaling is focused on adding more nodes to the system, while Vertical scaling is about adding more resources to a single machine that is responsible for hosting our application. However, this is only the beginning, or the end, depending on the perspective. Adding more resources to the physical machines has its physical limits. On the other hand, spawning multiple instances also has its limitations, not to mention significant complexity penalties with the possible architecture redesigns. That is why today we will dive deeper into tools and concepts that will help us make our system more scalable. Before that, let’s take a look at how we can measure the scalability of our system. How To Measure Scalability There are a couple of methods to do that. We can go from an experimental method and set up some stress tests, for example using Gatling, that will show us pretty visible results on how far our systems can scale. On the other hand, you can go for a more quantitative approach and try to calculate the limits of your system. You can read more about the quantitative approach here. As for the experimental methods, I recommend you taking a look at the Gatling Academy. It looks like a nice introduction into writing performance tests. Signs Your System Is Not Scaling Well, there are a couple of metrics and behaviors we can notice, which indicate possible scalability problems within our system: Steadily increasing response times - The response time starts to increase steadily as new users join, after a certain threshold.Higher error rate - The number of erroneous responses, timeouts, dropped messages or connections starts to increase.High cost to low performance - Gaining actual performance benefits (for example by adding more instances) gives little to no results despite investing a relatively high amount of money.Backlog growth - Processing queues, task managers, thread pools, schedulers start failing to keep up with incoming load, extending processing times, possibly later ending up as timeouts.Direct feedback - An important client is calling CTO/CIO or whoever else and complaining that everything is down, alerts start spinning, everything explodes, and other funny events. The Scalability Game Changer There is one key concept that can impact the scalability of our system quite significantly. The concept in question is statelessness, or stateless services/processing. The more stateless services or steps in our workflow we have, the easier it is to scale up the system. We can basically keep spawning new and new instances infinitely—there are some theoretical and of course practical limits, but they are highly dependent on your architecture. You can read more about them here. Of course, this approach has some potential drawbacks. For example, we have to store the state somewhere, since most of the systems are, by definition, stateful. We can offload it to some external dependencies like caches or databases, but this also has its drawbacks. While usually quite scalable, such tools can impose some further penalties on our systems and, what is more, impose some architectural requirements on our systems. That is why we should do in-depth research before actually picking the third-party solutions we want to offload our state to. Nevertheless, in my opinion, the potential benefits of having stateless services overcome their drawbacks, or at least make it an approach worth considering. With this somewhat long dive into the basics of scalability done, we can move to the main course of today's article: The tools and concepts that will make our application scale. Tools For Scalability We can skip captain obvious and move right away past the things like adding more resources to the server or spawning more instances of our service. Caching Creating a cache layer in front of our application is probably the simplest method to increase the scalability of our application. We can relatively easily offload part of the incoming traffic to a cache service that can store frequently queried data. Then, our service can focus either on handling incoming write requests while only handling read requests in case of cache misses and invalidations. Unfortunately, caching also has a couple of drawbacks and considerations that we have to think of before going head-on into adding it to our service. Firstly, cache invalidation is probably one of the biggest problems in the software world. Picking and tuning it correctly may require substantial time and effort. Next, we can take a look at cache inconsistencies and all the possible problems related to handling them correctly. There are a couple more problems related to caching, but these should be enough for a start. I promise to take a more detailed look at caching and all its quirks in a separate blog. Database Optimizations It is not a tool per se. However, it is probably one of the most important things we can do to make our system more scalable. What I exactly mean by this is: Well, sometimes the database can be the limiting factor. It is not a common case; nevertheless, after reaching a certain processing threshold, the chances for it to happen increase exponentially. On the other hand, if you do a lot of database queries (like 10–100 thousands) or your queries are suboptimal, then the chances also increase quite significantly. In such cases, performance tuning of your database can be a good idea. In every database engine there are switches and knobs that we can move up or down to change its behavior. There are also concepts like sharding and partitioning that may come in handy with processing more work. What is more, there is a concept called replication that is particularly useful for read-heavy workflows based on plain SQL databases. We can spawn multiple replicas of our database and forward all the incoming read traffic to them, while our primary node will take care of writes. As a last resort, you can try to exchange your database for some other solution. Just please do proper research beforehand, as this change is neither easy nor simple, and may result in degradation, not the up-scaling, of your database performance. Not to mention all the time and money spent along the way. Usually, even plain old databases like Postgres and MySQL (Uber) can handle quite a load, and with some tuning and good design, should be able to meet your requirements. Content Delivery Network (CDN) Setting up a CDN is somewhat similar to setting up a cache layer despite having quite a different purpose and setup process. While the caching layer can be used to store more or less anything we want, in the case of a CDN it is not that simple. They are designed to store and then serve mostly static content, like images, video, audio, and CSS files. Additionally, we can store different types of web pages. Usually, the CDN is a paid service managed and run by third-party companies like Cloudflare or Akamai. Nevertheless, if you are big enough, you can set up your own CDN, but the threshold is pretty high for this to make sense. For example, Netflix has its own CDN, while Etsy uses Akamai. The CDN consists of multiple servers spread across the globe. They utilize location-based routing to direct the end users to the nearest possible edge node that contains the requested data. Such an approach greatly reduces wait time for users and moves a lot of the traffic off the shoulders of our services. Load Balancing Load balancing is more of a tool for helping with availability than with the scalability of the system. Nonetheless, it can also help with the scaling part. As the name suggests, it is about balancing the incoming load equally across available services. There are different types of load balancers (4th and 7th level) that can support various load balancing algorithms. The point for load balancers and scalability is that we can route requests based on request processing time, routing new requests to less busy services. We can also use location-based routing similar to the case of a CDN. What is more, Load Balancers can cache some responses, integrate with CDNs, and manage the on-demand scaling of our system. On the other hand, using Load Balancers requires at least some degree of statelessness in the connected service. Otherwise, we have to use sticky sessions, which is not desired behavior nor recommended practice. Another potential drawback of using load balancers is that they may introduce a single point of failure into our systems. We all know what the result of such an action may be. The desired approach would be to use some kind of distributed load balancers, where some nodes may actually fail without compromising the whole application. Yet doing so adds additional complexity to our design. As you see, it is all about the trade-offs. Async Communication Here, the matter is pretty straightforward: Switching to async communication frees up processing power in our services. Services do not have to wait for responses and may switch to other tasks while the targets process the requests we already sent. While the exact numbers or performance increase may vary significantly from use case to use case, the principle remains. Besides the additional cognitive load put upon the maintainers and some debugging complexity, there are no significant drawbacks related to switching to an async communication model. This will get more interesting in the next topic. Just to clarify, in this point I thought mostly of approaches like async HTTP calls, WebSockets, async gRPC, and SSE. We will move into the messaging space in the next paragraph. Messaging It is a type of asynchronous communication that uses message queues, message brokers to exchange data between services. The most important change that it brings is the decoupling of sender and consumer. We can just put the message on the queue and everyone interested can read it and act as they want. Message brokers are usually highly scalable and resilient. They will probably handle your workload and not even break a sweat while doing so. Unfortunately, they introduce a few drawbacks: Additional complexity - Messaging is totally different on a conceptual level than the plain request-response model, even if done in an async manner. There may be problems with logging, tracing, debugging, message ordering, messages being lost, and possible duplication.Infrastructure overhead - Setting up and running a platform for exchanging messages can be a challenging undertaking, especially for newcomers.Messages backlog - If the producers send messages faster than the consumer is able to process them, then the backlog grows, and if we do not handle this matter, it may lead to system-wide failures. There is a concept of Backpressure that aims to address this issue. Service Oriented Architecture (SOA) Last but not least, I wanted to spend some time describing any type of service-oriented architecture and its potential impact on how our application scales. While this type of architecture is inherently complex and hard to implement correctly, it can bring our system’s scalability to a totally different level: Decoupling the components enables scaling components in isolation. If one service becomes a bottleneck, we can focus optimization efforts solely on that service.We can find the root cause of potential performance bottlenecks in a more efficient and timely manner, as we can gather more insights into the behavior of different services.Our autoscaling becomes more granular, allowing us to apply tailored scaling strategies on a per-service basis. All of these features improve our efficiency, which can result in reduced costs. We should be able to act faster and more adequately. We also avoid potential over-provisioning of resources across the entire application. However, the sheer amount of potential problems related to this architecture can easily make it a no-go. Why We Fail To Scale After what, how, and why, it is time for why we fail. In my opinion and experience, there are a few factors that lead to our ultimate failure in making an application scalable. Poor understanding of tools we use - We can have state-of-the-art tools; but we have to use them correctly. Otherwise, we should better drop using them—at least we will not increase complexity.Poor choice of tools - While the previous point addresses the improper use of state-of-the-art tools, this one addresses the issue of choosing incorrect tools and approaches for a job, either on a conceptual or implementation level. I kind of tackled the problem with the quantity of different tools for a single task here.Ignoring the trade-offs - Every decision we make has short- and long-lasting consequences we have to be aware of. Of course, we can ignore them; still, we have to know them first and be conscious as to why we are ignoring some potential drawbacks.Bad architecture solutions - Kind of similar point as the one above. This is a point that addresses what happens if we ignore one too many trade-offs—the answer is that our architecture will be flawed in a fundamental way, and we will have to change it sooner or later.Under-provisioning / Over-provisioning - It is the result of loopholes in our data while we are researching what we would need to handle incoming traffic. While over-provisioning is not a scalability failure per se, as we were able to meet the demand, someone may ask, but at what cost? Summary We walked through a number of concepts and approaches to make our application more scalable. Here they are: ConceptDescriptionProsConsCachingIntroduces a cache layer to store frequently accessed data and offload traffic from the application.- Reduces load on the backend - Improves response time - Handles read-heavy workflows efficiently- Cache invalidation is complex - Risk of inconsistenciesDatabase OptimizationsImproves database performance through tuning, sharding, replication, and partitioning.- Handles high read/write workloads - Scalability with replication for read-heavy workflows - Efficient query optimization- Complex setup (e.g., sharding) - High migration costs if switching databases - Risk of degraded performanceCDNDistributes static content (e.g., images, CSS, videos) across globally distributed servers.- Reduces latency - Offloads traffic from core services- Primarily for static content - Can be quite costlyLoad BalancingDistributes incoming traffic across multiple instances.- Balances traffic efficiently - Supports failover and redundancy - Works well with auto-scaling- Requires stateless services - SPOF (if centralized)Async CommunicationFrees up resources by processing requests asynchronously, allowing services to perform other tasks in parallel.- Increases throughput - Reduces idle waiting time- Adds cognitive load for maintainers - Debugging and tracing can be complexMessagingUses message queues to decouple services.- Decouples sender/consumer - Highly scalable and resilient - Handles distributed workloads effectively- Infrastructure complexity - Risks like message backlog, duplication, or loss - Conceptually harder to implementService-Oriented Architecture (SOA)Breaks down the application into independent services that can scale and be optimized individually.- Granular auto-scaling - Simplifies root-cause analysis - Enables tailored optimizations - Reduces resource wastage- High complexity - Inter-service communication adds overhead - Requires robust observability and monitoring Also, here is the table of concrete tools we can use to implement some of the concepts from above. The missing ones are either too abstract or require multiple tools and techniques to implement. ConceptToolCachingRedis, MemcachedCDNCloudFlare, AkamaiLoad BalancingHAProxy, NGINXAsync CommunicationAsync HTTP, WebSockets, async gRPC, SSEMessagingRabbitMQ, Kafka I think that both these tables are quite a nice summary of this article. I wish you luck on your struggle with scalability, and thank you for your time.

By Bartłomiej Żyliński

CORE

SQL Server Index Optimization Strategies: Best Practices with Ola Hallengren’s Scripts

Index maintenance is a critical component of database administration as it helps ensure the ongoing efficiency and performance of a Structured Query Language (SQL) Server environment. Over time, as data is added, updated, and deleted, index fragmentation can occur, where the logical and physical ordering of index pages becomes misaligned. This fragmentation can lead to increased disk I/O, decreased query performance, and overall system inefficiency. Running index maintenance jobs, such as those provided by the Ola Hallengren SQL Server Maintenance Solution, allows DBAs to proactively address this fragmentation and optimize the indexes for better performance. By regularly monitoring index fragmentation levels and executing maintenance operations like index reorganizations and rebuilds, DBAs can keep their databases running at peak efficiency. This is especially important for large, mission-critical databases, where any degradation in performance can have a significant business impact. Maintaining optimal index health helps ensure fast, reliable data access, reduced resource consumption, and an overall improvement in the user experience. Consequently, implementing a well-designed index maintenance strategy is a crucial responsibility for any DBA managing a complex SQL Server environment. Ola Hallengren's SQL Server Maintenance Solution The SQL Server Maintenance Solution, developed by Ola Hallengren, is a widely adopted and trusted set of scripts used by database administrators worldwide. This comprehensive solution automates various maintenance tasks, including index optimization, database integrity checks, and statistics updates. Ola's scripts have become the industry standard for proactive database maintenance. The IndexOptimize procedure from the Maintenance Solution provides extensive customization and configuration options to tailor the index maintenance process for specific environments and requirements. Many database administrators rely on these scripts as the foundation for their index management strategy, as they offer a robust and efficient way to keep indexes in an optimal state. You can download the latest SQL Server Maintenance Solution version from Ola Hallengren's website. The scripts are released under the MIT License, allowing users to freely use, modify, and distribute them as needed. Core IndexOptimize Parameters and Their Impact The `IndexOptimize` stored procedure provides extensive customization through numerous parameters. Understanding these is critical for effective implementation: Essential Parameters ParameterDescriptionImpact`@Databases`Target databasesControls scope of operation`@FragmentationLow`Action for low fragmentationTypically NULL (no action)`@FragmentationMedium`Action for medium fragmentationUsually REORGANIZE`@FragmentationHigh`Action for high fragmentationREBUILD or REORGANIZE`@FragmentationLevel1`Low/medium threshold (%)Typically 5-15%`@FragmentationLevel2`Medium/high threshold (%)Typically 30-40%`@PageCountLevel`Minimum index size to processExcludes small indexes`@SortInTempdb`Use tempdb for sortingReduces production database I/O`@MaxDOP`Degree of parallelismControls CPU utilization`@FillFactor`Index fill factorControls free space in pages`@PadIndex`Apply fill factor to non-leaf levelsAffects overall index size`@LOBCompaction`Compact LOB dataReduces storage for LOB columns`@UpdateStatistics`Update statistics after rebuild'ALL', 'COLUMNS', 'INDEX', NULL`@OnlyModifiedStatistics`Only update changed statisticsReduces unnecessary updates`@TimeLimit`Maximum execution time (seconds)Prevents runaway jobs`@Delay`Pause between operations (seconds)Reduces continuous resource pressure`@Indexes`Specific indexes to maintainAllows targeted maintenance`@MinNumberOfPages`Minimum size thresholdAlternative to PageCountLevel`@MaxNumberOfPages`Maximum size thresholdLimits operation to smaller indexes`@LockTimeout`Lock timeout (seconds)Prevents blocking`@LogToTable`Log operations to tableEnables tracking/troubleshootingParameterDescriptionRecommended Setting`@AvailabilityGroups`Target specific AGsLimit scope when needed`@AvailabilityGroupReplicas`Target specific replicas'PRIMARY' to limit AG impact`@AvailabilityGroupDatabases`Target specific databasesFocus on critical databases Availability Group-Specific Parameters ParameterDescriptionRecommended Setting`@AvailabilityGroups`Target specific AGsLimit scope when needed`@AvailabilityGroupReplicas`Target specific replicas'PRIMARY' to limit AG impact`@AvailabilityGroupDatabases`Target specific databasesFocus on critical databases Implementation Strategies by Index Size Large Indexes (>10GB) EXECUTE dbo.IndexOptimize @Databases = 'PRODUCTION_DB', @FragmentationLow = NULL, @FragmentationMedium = 'INDEX_REORGANIZE', @FragmentationHigh = 'INDEX_REORGANIZE,INDEX_REBUILD_ONLINE', @FragmentationLevel1 = 15, @FragmentationLevel2 = 40, @PageCountLevel = 10000, -- Only process substantial indexes @MaxDOP = 4, -- Limit CPU utilization @TimeLimit = 7200, -- 2-hour limit per operation @Delay = '00:00:45', -- 45-second pause between operations @SortInTempdb = 'Y', -- Reduce database file I/O @MaxNumberOfPages = NULL, -- No upper limit @MinNumberOfPages = 10000, @LockTimeout = 300, -- 5-minute lock timeout @LogToTable = 'Y', @Execute = 'Y'; Special considerations: Prefer REORGANIZE for large indexes to minimize transaction log growthUse REBUILD selectively when reorganize is insufficientImplement larger `@Delay`to allow transaction log processingSchedule during low-activity periodsConsider smaller batches using `@Indexes` parameter Medium Indexes (1GB-10GB) EXECUTE dbo.IndexOptimize @Databases = 'PRODUCTION_DB', @FragmentationLow = NULL, @FragmentationMedium = 'INDEX_REORGANIZE', @FragmentationHigh = 'INDEX_REBUILD_ONLINE', @FragmentationLevel1 = 10, @FragmentationLevel2 = 30, @PageCountLevel = 1000, @MaxDOP = 2, @TimeLimit = 3600, -- 1-hour limit @Delay = '00:00:20', -- 20-second pause @SortInTempdb = 'Y', @MinNumberOfPages = 1000, @MaxNumberOfPages = 10000, @LockTimeout = 180, -- 3-minute lock timeout @LogToTable = 'Y', @Execute = 'Y'; Special considerations: Balance between REORGANIZE and REBUILD operationsModerate `@Delay` value to manage resource impactCan run more frequently than large index maintenance Small Indexes (<1GB) EXECUTE dbo.IndexOptimize @Databases = 'PRODUCTION_DB', @FragmentationLow = NULL, @FragmentationMedium = 'INDEX_REORGANIZE', @FragmentationHigh = 'INDEX_REBUILD_ONLINE', @FragmentationLevel1 = 5, @FragmentationLevel2 = 30, @PageCountLevel = 100, @MaxDOP = 0, -- Use server default @TimeLimit = 1800, -- 30-minute limit @Delay = '00:00:05', -- 5-second pause @SortInTempdb = 'Y', @MaxNumberOfPages = 1000, @MinNumberOfPages = 100, @LockTimeout = 60, -- 1-minute lock timeout @LogToTable = 'Y', @Execute = 'Y'; Special considerations: Can be more aggressive with rebuild operations.Minimal `@Delay` needed between operations.Can run during regular business hours with minimal impact. Availability Group-Specific Configurations Environment: Large, mission-critical OLTP database with multiple replicas in an Availability Group (AG) configured for synchronous commit. Maintenance Objectives: Minimize impact on production workload and log shipping.Avoid exhausting storage resources due to log growth.Ensure high availability and minimal downtime. Synchronous AG Environment EXECUTE dbo.IndexOptimize @Databases = 'PRODUCTION_DB', @FragmentationLow = NULL, @FragmentationMedium = 'INDEX_REORGANIZE', @FragmentationHigh = 'INDEX_REORGANIZE', -- Avoid rebuilds in sync AGs @FragmentationLevel1 = 15, @FragmentationLevel2 = 40, @PageCountLevel = 5000, @MaxDOP = 2, @TimeLimit = 3600, @Delay = '00:01:00', -- Longer delay for sync replicas @AvailabilityGroupReplicas = 'PRIMARY', @LockTimeout = 300, @LogToTable = 'Y', @Execute = 'Y'; Synchronous AG considerations: Minimize rebuilds - Transaction logs must be synchronized before the operation completes.Implement longer delays between operations to allow synchronization.Monitor replica lag and suspend jobs if lag exceeds thresholds.Increase log backup frequency during maintenance windows.Split maintenance across multiple days for very large environments. Asynchronous AG Environment Environment: Large, multi-terabyte data warehouse database with asynchronous AG replicas. Maintenance Objectives: Perform comprehensive index and statistics maintenanceMinimize the impact on the reporting workload during the maintenance windowEnsure optimal performance for the upcoming quarter EXECUTE dbo.IndexOptimize @Databases = 'PRODUCTION_DB', @FragmentationLow = NULL, @FragmentationMedium = 'INDEX_REORGANIZE', @FragmentationHigh = 'INDEX_REBUILD_ONLINE', -- Rebuilds more acceptable @FragmentationLevel1 = 10, @FragmentationLevel2 = 30, @PageCountLevel = 2000, @MaxDOP = 4, @TimeLimit = 5400, @Delay = '00:00:30', -- Moderate delay @AvailabilityGroupReplicas = 'PRIMARY', @LockTimeout = 240, @LogToTable = 'Y', @Execute = 'Y'; Asynchronous AG considerations: More liberal with rebuilds - operations don't wait for secondary synchronization.Still monitor send queue to prevent overwhelming secondaries.Consider network bandwidth and adjust `@Delay` accordingly.Implement send queue size alerts during maintenance. Preventing Storage and IOPS Pressure Pre-Maintenance Preparation Expand transaction log files proactively: ALTER DATABASE [YourDatabase] MODIFY FILE (NAME = LogFileName, SIZE = ExpandedSizeInMB); Configure TempDB properly: -- Verify TempDB configuration SELECT name, size/128.0 AS [Size_MB] FROM tempdb.sys.database_files; Implement pre-maintenance checks: -- Create helper procedure to validate environment readiness CREATE PROCEDURE dbo.ValidateMaintenanceReadiness AS BEGIN DECLARE @IssuesFound BIT = 0; -- Check log space IF EXISTS ( SELECT 1 FROM sys.databases d CROSS APPLY sys.dm_db_log_space_usage() l WHERE d.database_id = DB_ID() AND l.log_space_used_percent > 30 ) BEGIN RAISERROR('Log usage exceeds 30%. Backup logs before proceeding.', 16, 1); SET @IssuesFound = 1; END -- Check AG health IF EXISTS ( SELECT 1 FROM sys.dm_hadr_availability_replica_states ars JOIN sys.availability_replicas ar ON ars.replica_id = ar.replica_id WHERE ars.is_local = 0 AND ars.synchronization_health <> 2 -- Not HEALTHY ) BEGIN RAISERROR('Availability Group replicas not in healthy state.', 16, 1); SET @IssuesFound = 1; END RETURN @IssuesFound; END; GO Operational Techniques Implement dynamic index selection based on business impact: -- Create index priority categories CREATE TABLE dbo.IndexMaintenancePriority ( SchemaName NVARCHAR(128), TableName NVARCHAR(128), IndexName NVARCHAR(128), Priority INT, -- 1=High, 2=Medium, 3=Low MaintenanceDay TINYINT -- Day of week (1-7) ); -- Use with dynamic execution DECLARE @IndexList NVARCHAR(MAX); SELECT @IndexList = STRING_AGG(CONCAT(DB_NAME(), '.', SchemaName, '.', TableName, '.', IndexName), ',') FROM dbo.IndexMaintenancePriority WHERE Priority = 1 AND MaintenanceDay = DATEPART(WEEKDAY, GETDATE()); EXEC dbo.IndexOptimize @Databases = 'PRODUCTION_DB', @Indexes = @IndexList, -- other parameters Implement I/O throttling techniques: Use Resource Governor to limit I/O (SQL Server Enterprise).Set lower `@MaxDOP` values during business hours.Implement longer `@Delay` values during peak periods. Database-level I/O tuning: -- Consider trace flag 1117 for uniform file growth DBCC TRACEON(1117, -1); -- Consider trace flag 1118 for reducing SGAM contention DBCC TRACEON(1118, -1); -- For SQL Server 2016+, use proper tempdb configuration ALTER DATABASE [tempdb] MODIFY FILE (NAME = 'tempdev', SIZE = 8GB); Advanced Scheduling Strategies Workload-Aware Batching -- Create helper procedure for smart batching CREATE PROCEDURE dbo.ExecuteIndexMaintenanceBatch @BatchSize INT = 5, @MaxRuntime INT = 7200 -- 2 hours in seconds AS BEGIN DECLARE @StartTime DATETIME = GETDATE(); DECLARE @EndTime DATETIME = DATEADD(SECOND, @MaxRuntime, @StartTime); DECLARE @CurrentTime DATETIME; DECLARE @IndexBatch NVARCHAR(MAX); WHILE (1=1) BEGIN SET @CurrentTime = GETDATE(); IF @CurrentTime > @EndTime BREAK; -- Get next batch of indexes based on priority and fragmentation SELECT TOP (@BatchSize) @IndexBatch = STRING_AGG(CONCAT(DB_NAME(), '.', s.name, '.', t.name, '.', i.name), ',') FROM sys.indexes i JOIN sys.tables t ON i.object_id = t.object_id JOIN sys.schemas s ON t.schema_id = s.schema_id JOIN sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, 'LIMITED') ps ON ps.object_id = i.object_id AND ps.index_id = i.index_id WHERE i.type_desc = 'NONCLUSTERED' AND ps.avg_fragmentation_in_percent > 30 AND ps.page_count > 1000 AND NOT EXISTS ( -- Skip indexes we've already processed SELECT 1 FROM dbo.CommandLog WHERE DatabaseName = DB_NAME() AND SchemaName = s.name AND ObjectName = t.name AND IndexName = i.name AND StartTime > DATEADD(DAY, -7, GETDATE()) ) ORDER BY ps.avg_fragmentation_in_percent DESC; IF @IndexBatch IS NULL BREAK; -- No more work to do -- Execute maintenance for this batch EXEC dbo.IndexOptimize @Databases = DB_NAME(), @Indexes = @IndexBatch, @FragmentationLow = NULL, @FragmentationMedium = 'INDEX_REORGANIZE', @FragmentationHigh = 'INDEX_REORGANIZE', @FragmentationLevel1 = 10, @FragmentationLevel2 = 30, @MaxDOP = 2, @TimeLimit = 1800, -- 30 minutes per batch @Delay = '00:00:30', @LogToTable = 'Y', @Execute = 'Y'; -- Pause between batches WAITFOR DELAY '00:01:00'; END END; GO Monitoring Framework -- Create monitoring stored procedure CREATE PROCEDURE dbo.MonitorIndexMaintenance AS BEGIN -- Check transaction log usage SELECT DB_NAME(database_id) AS DatabaseName, log_space_in_use_percentage FROM sys.dm_db_log_space_usage WHERE log_space_in_use_percentage > 50; -- Check AG send queue size SELECT ar.replica_server_name, drs.database_name, drs.log_send_queue_size, drs.log_send_rate, drs.redo_queue_size, drs.redo_rate FROM sys.dm_hadr_database_replica_states drs JOIN sys.availability_replicas ar ON drs.replica_id = ar.replica_id WHERE drs.log_send_queue_size > 10000 OR drs.redo_queue_size > 10000; -- Check ongoing index operations SELECT r.session_id, r.command, r.status, r.wait_type, r.wait_time, OBJECT_NAME(p.object_id) AS ObjectName, p.index_id, i.name AS IndexName FROM sys.dm_exec_requests r CROSS APPLY sys.dm_exec_sql_text(r.sql_handle) t LEFT JOIN sys.partitions p ON p.hobt_id = r.statement_id LEFT JOIN sys.indexes i ON i.object_id = p.object_id AND i.index_id = p.index_id WHERE t.text LIKE '%INDEX_REBUILD%' OR t.text LIKE '%INDEX_REORGANIZE%'; END; GO Best Practices Summary For synchronous AG environments: Prioritize REORGANIZE over REBUILD, especially for large indexes.Implement longer delays between operations (45-90 seconds). Schedule maintenance during the least active periods.Consider partitioning very large tables for incremental maintenance. For asynchronous AG environments: More liberal use of REBUILD for critical indexes. Implement moderate delays (15-45 seconds). Monitor send queue and redo queue sizes closely. General IOPS reduction techniques: Leverage `@SortInTempdb = 'Y'` to spread I/O load. Use `@MaxDOP` to control parallelism (lower values reduce I/O).Implement `@Delay` parameters appropriate to your environment. Use `@TimeLimit` to prevent runaway operations. Storage pressure mitigation: Pre-allocate transaction log space before maintenance.Increase log backup frequency during maintenance (every 5-15 minutes). Use Resource Governor to limit I/O impact. Implement batched approaches with appropriate pauses. Comprehensive maintenance approach: Different strategies for different index sizes. Business-hour vs. off-hour configurations. Prioritization based on business impact.Regular verification of fragmentation levels post-maintenance. By implementing these guidelines and adapting the provided scripts to your specific environment, you can maintain optimal SQL Server index performance while minimizing production impact, even in complex Availability Group configurations.

By arvind toorpu

CORE

Java's Quiet Revolution: Thriving in the Serverless Kubernetes Era

Along with the rise of Kubernetes, there is another shift that is happening under the hood - the rise of serverless architecture, which is quietly rewriting the way we deploy and scale applications, with Java taking a lead. Java, which is usually associated with legacy code and monolithic enterprise applications, has been slowly but steadily evolving into a microservices architecture and is now evolving into a leaner, serverless-ready world. With the availability of tools like Knative and frameworks like Quarkus, Java has been transforming from a heavyweight language into a zero-management, Kubernetes-ready ready. In this article, we will reflect on this promising transformation in Java and where it can take us in 2025 and beyond. The Promise of Serverless Kubernetes Architecture Kubernetes is really good at handling containers; however, the management of the underlying infrastructure, like nodes, scaling, and patching, is still difficult. However, serverless Kubernetes frameworks like Knative, KEDA, and OpenFaaS remove this difficulty; they let applications scale to zero and back up without the need for any human intervention. This approach not only enhances developer productivity but also helps with cost efficiency and is in sync with the needs for a modern event-driven system. Java has also been slowly catching up on this. Historically, Java has been associated with resource-intensive applications, where there were challenges with startup speed and being a memory-intensive language. However, with recent advancements, Java is becoming a viable option for serverless deployments. Java's Technical Evolution Java's adaptation to a serverless environment is primarily driven by optimized frameworks and runtime enhancements like, 1. Quarkus Quarkus was introduced by Red Hat. It is designed for Kubernetes native deployment and integrates well with GraalVM. This enables native compilation, which reduces startup times and lowers memory usage, making applications built with Java suitable for serverless scenarios. This framework also simplifies the developer experience by offering a unified approach to both imperative and reactive coding styles. 2. Spring Boot Enhancements Recent versions of Spring Boot have incorporated several features that are aimed at improving performance in cloud native environments. This includes features like support for reactive programming models, integration with Kubernetes, among others. These updates allow developers to seamlessly bridge traditional Java applications with modern, distributed systems. 3. Project Loom Project Loom is an initiative to introduce lightweight, user-mode threads to the Java Virtual Machine (JVM), which aims to enhance concurrency handling. It does so by enabling the JVM to manage numerous virtual threads efficiently, which improves the scalability of Java applications in I/O-bound operations. This can be beneficial for serverless workloads. This innovation promises to make Java a go-to choice for highly concurrent workloads without the complexity of traditional threading models. Practical Use Cases and Applications Java's applicability in a serverless Kubernetes environment can be understood with different use cases as discussed below: 1. Event-Driven Architecture Java frameworks like Quarkus can be used to build event-driven applications that respond to various triggers, such as HTTP requests or messaging queues. This capability allows Java to power dynamic systems where responsiveness to unpredictable events is critical. 2. Microservices Java has a rich ecosystem that supports the development of microservices, which can be deployed in serverless environments and benefit from automatic scaling and reduced operational overhead. Developers can now build modular, independent services that integrate smoothly with existing enterprise solutions. 3. Batch Processing and AI Inference Java can be employed in batch processing for large-scale batch jobs and AI inference tasks, taking advantage of serverless platforms that auto-scale with workload demands and minimize idle resource costs. Its robust libraries make it particularly effective for processing intricate datasets or deploying machine learning models in real time. Challenges and Limitations Although Java is making significant advancements, there are several challenges, 1. Cold Start Latency Even though there are optimizations happening, Java applications can still experience higher cold start latencies compared to other languages like NodeJS, which can be an issue in applications where latency is monitored. This issue often stems from the inherent overhead of initializing the JVM in transient environments. 2. Build Complexity Native compilation with tools like GraalVM can introduce additional build complexities. Also, if there is a misconfiguration, it can lead to runtime issues or an increase in the size of a binary file. Developers may also find debugging these native images trickier than traditional Java setups. 3. Learning Curve Frameworks like Knative can be difficult to learn initially and can have a learning curve, particularly for advanced features like eventing, which may require significant investment in understanding and its implementation. This complexity can slow adoption in teams accustomed to simpler deployment models. Competitive Edge Java has an established ecosystem that offers a wealth of libraries and frameworks, such as JPA for data persistence and Spring Security for authentication and authorization. This maturity in the framework allows developers to leverage existing tools and practices, which facilitates an easy transition to serverless architecture. Java's use across various industries would facilitate easy adaptability of existing skills with the new paradigm due to the vast community of developers who use Java. Additionally, Java’s long-standing reputation for reliability ensures it remains a trusted choice for mission-critical applications in serverless contexts. Future Implications With Java continuously evolving through projects like Loom and Quarkus, its role and usage for applications built for serverless environments are expected to grow. The combination of various factors, including scaling up from zero, simplicity in operations, and agility in deployments, positions Java as a strong contender for future cloud-native applications. As serverless adoption expands, Java’s ability to integrate with emerging tools and platforms will keep it relevant in an ever-evolving tech landscape. Its ongoing evolution suggests it could redefine how enterprises approach scalable, cost-effective solutions well into the future. Conclusion The transformation of Java in the context of serverless Kubernetes not only reflects a strategic evolution, but it also leverages modern frameworks and runtime enhancements to meet the demands of cloud-native development. Though there has been some progress, some challenges remain. However, with the trajectory Java is taking, it is clear that it is not only adapting to new demands but also thriving in this paradigm. With the strong march, Java has positioned itself such that developers and architects can confidently consider Java as a robust and scalable foundation for serverless applications.

By Alok Tibrewala

Building Event-Driven Go Applications With Azure Cosmos DB and Azure Functions

The Go programming language is a great fit for building serverless applications. Go applications can be easily compiled to a single, statically linked binary, making deployment simple and reducing external dependencies. They start up quickly, which is ideal for serverless environments where functions are frequently invoked from a cold start. Go applications also tend to use less memory compared to other languages, helping optimize resource usage and reduce costs in serverless scenarios. Azure Functions supports Go using custom handlers, and you can use triggers and input and output bindings via extension bundles. Azure Functions is tightly integrated with Azure Cosmos DB using bindings (input, output) and triggers. This blog post will walk you through how to build Azure Functions with Go that make use of these Azure Cosmos DB integrations. Bindings allow you to easily read and write data to Cosmos DB, while triggers are useful for building event-driven applications that respond to changes in your data in Cosmos DB. Part 1 of this blog starts off with a function that gets triggered by changes in a Cosmos DB container and simply logs the raw Azure Functions event payload and the Cosmos DB document. You will learn how to run the function and also test it with Cosmos DB locally, thanks to the Cosmos DB emulator and Azure Functions Core Tools. If this is your first time working with Go and Azure Functions, you should find it helpful to get up and running quickly. Although you can deploy it to Azure, we will save that for the next part of this blog. Part 2 dives into another function that generates embeddings for the documents in the Cosmos DB container. This example will use an Azure OpenAI embedding model to generate embeddings for the documents in the container and then store the embeddings back in the container. This is useful for building applications that require semantic search or other generative AI applications. Check out the GitHub repository for the complete code. Part 1: Build a Simple Cosmos DB Trigger-Based Function and Run It Locally Just as the Cosmos DB emulator lets you run Cosmos DB locally, Azure Functions Core Tools lets you develop and test your functions locally. Start by installing the Azure Functions Core Tools; refer to the documentation for instructions specific to your OS. For example, on Linux, you can: Shell sudo apt-get update sudo apt-get install azure-functions-core-tools-4 Next, start the Cosmos DB emulator. The commands below are for Linux and use the Docker container-based approach. Refer to the documentation for other options. You need to have Docker installed and running on your machine. If you don't have it installed, please refer to the Docker installation guide. YAML docker pull mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest docker run \ --publish 8081:8081 \ --name linux-emulator \ -e AZURE_COSMOS_EMULATOR_PARTITION_COUNT=1 \ mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest Make sure to configure the emulator SSL certificate as well. For example, for the Linux system I was using, I ran the following command to download the certificate and regenerate the certificate bundle: Shell curl --insecure https://localhost:8081/_explorer/emulator.pem > ~/emulatorcert.crt sudo update-ca-certificates Use the following URL to navigate to the Cosmos DB Data Explorer using your browser: http://localhost:8081/_explorer/index.html. Create the following resources: A databaseA container with a partition key /id – this is the source containerA lease container with the name leases and partition key /id – it is used by the trigger to keep track of the changes in the source container. Clone the GitHub repository with the code for the function: Shell git clone https://github.com/abhirockzz/golang_cosmosdb_azure_functions.git cd golang_cosmosdb_azure_functions/getting_started_guide Create a local.settings.json file with the Cosmos DB-related info. Use the same database and container names as you created in the previous step. The local.settings.json file is used to store the configuration settings for your function app when running locally: JSON { "IsEncrypted": false, "Values": { "AzureWebJobsStorage": "", "FUNCTIONS_WORKER_RUNTIME": "custom", "COSMOS_CONNECTION": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==;", "COSMOS_DATABASE_NAME": "test", "COSMOS_CONTAINER_NAME": "tasks" } } COSMOS_CONNECTION has a static value for the connection string for the Cosmos DB emulator — do not change it. Build the Go function binary using the following command. This will create a binary file named main in the current directory: Go go build -o main main.go Start the function locally: Go func start This will start the function app and listen for incoming requests. You should see output similar to this: Plain Text [2025-04-25T07:44:53.921Z] Worker process started and initialized. Functions: processor: cosmosDBTrigger For detailed output, run func with --verbose flag. [2025-04-25T07:44:58.809Z] Host lock lease acquired by instance ID '0000000000000000000000006ADD8D3E'. //... Add data to the source container in Cosmos DB. You can do this by navigating to Data Explorer in the emulator. For example, add a document with the following JSON: JSON { "id": "42", "description": "test" } The function should be triggered automatically when the document is added to the container. You can check the logs of the function app to see if it was triggered successfully: Plain Text [2025-04-25T07:48:10.559Z] Executing 'Functions.processor' (Reason='New changes on container tasks at 2025-04-25T07:48:10.5593689Z', Id=7b62f8cf-683b-4a5b-9db0-83d049bc4c86) [2025-04-25T07:48:10.565Z] processor function invoked... [2025-04-25T07:48:10.565Z] Raw event payload: {{"[{\"id\":\"42\",\"description\":\"test\",\"_rid\":\"AxI2AL1rrFoDAAAAAAAAAA==\",\"_self\":\"dbs/AxI2AA==/colls/AxI2AL1rrFo=/docs/AxI2AL1rrFoDAAAAAAAAAA==/\",\"_etag\":\"\\\"00000000-0000-0000-b5b6-6123f4d401db\\\"\",\"_attachments\":\"attachments/\",\"_ts\":1745567285,\"_lsn\":4}]"} {{processor 2025-04-25T07:48:10.560243Z 4f29b3f3-ba95-4043-9b67-2856a43b4734}} [2025-04-25T07:48:10.566Z] Cosmos DB document: {42 AxI2AL1rrFoDAAAAAAAAAA== dbs/AxI2AA==/colls/AxI2AL1rrFo=/docs/AxI2AL1rrFoDAAAAAAAAAA==/ "00000000-0000-0000-b5b6-6123f4d401db" attachments/ 1745567285 4} [2025-04-25T07:48:10.566Z] Executed 'Functions.processor' (Succeeded, Id=7b62f8cf-683b-4a5b-9db0-83d049bc4c86, Duration=6ms) //..... How It Works Here is a very high-level overview of the code: main.go: Implements an HTTP server with a processor endpoint. When triggered, it reads a Cosmos DB trigger payload from the request, parses the nested documents, logs information, and returns a structured JSON response. It uses types and helpers from the commonpackage.common package: Contains shared types and utilities for Cosmos DB trigger processing: payload.go: Defines data structures for the trigger payload, documents, and response.parse.go: Provides a Parse function to extract and unmarshal documents from the trigger payload’s nested JSON structure. Part 2: Use Azure OpenAI to Generate Embeddings for the Documents in the Cosmos DB Container In addition to its low-latency, high-performance, and scalability characteristics, its support for vector (semantic/similarity), full-text, and hybrid search makes Azure Cosmos DB a great fit for generative AI applications. Consider a use case for managing a product catalog for an e-commerce platform. Each time a new product is added to the system (with a short description like “Bluetooth headphones with noise cancellation”), we want to immediately make that item searchable semantically. As soon as the product document is written to Cosmos DB, an Azure Function is triggered. It extracts the product description, generates a vector embedding using Azure OpenAI, and writes the embedding back to the same document using an output binding. With the embedding in place, the product is now indexed and ready for semantic and hybrid search queries, without any additional effort. Prerequisites You will run this example in Azure, so you need to have an Azure account. If you don't have one, you can create a free account. Create an Azure Cosmos DB for NoSQL account. Enable the vector indexing and search feature – this is a one-time operation. Just like before, you will need to create the following resources: A databaseA container with partition key /id – this is the source containerA lease container with the name leases and partition key /id – it is used by the trigger to keep track of the changes in the source container. The lease container needs to be created in advance since we have configured Azure Functions to use managed identity to access the Cosmos DB account – you don't need to use keys or connection strings. Create an Azure OpenAI Service resource. Azure OpenAI Service provides access to OpenAI's models, including GPT-4o, GPT-4o mini (and more), as well as embedding models. Deploy an embedding model of your choice using the Azure AI Foundry portal (for example, I used the text-embedding-3-small model). Just like the Cosmos DB account, the Azure Function app uses a managed identity to access the Azure OpenAI Service resource. Deploy Resources Move into the right directory: Shell cd ../embeddings_generator To simplify the deployment of the function app along with the required resources and configuration, you can use the deploy.sh script. At a high level, it: Sets up environment variables for Azure resources.Creates an Azure resource group, storage account, and function app plan.Deploys a custom Go-based Azure Function App.Builds the Go binary for Windows.Publishes the function app to Azure.Enables the function app system identity and provides it the required roles for Cosmos DB and Azure OpenAI resource access. Before you deploy the solution, update the local.settings.json. Use the same database and container names as you created in the previous step: JSON { "IsEncrypted": false, "Values": { "AzureWebJobsStorage": "", "FUNCTIONS_WORKER_RUNTIME": "custom", "COSMOS_CONNECTION__accountEndpoint": "https://ENTER_COSMOSDB_ACCOUNT_NAME.documents.azure.com:443/", "COSMOS_DATABASE_NAME": "name of the database", "COSMOS_CONTAINER_NAME": "name of the container", "COSMOS_HASH_PROPERTY": "hash", "COSMOS_VECTOR_PROPERTY": "embedding", "COSMOS_PROPERTY_TO_EMBED": "description", "OPENAI_DEPLOYMENT_NAME": "enter the embedding model deployment name e.g. text-embedding-3-small", "OPENAI_DIMENSIONS": "enter the dimensions e.g. 1536", "OPENAI_ENDPOINT": "https://ENTER_OPENAI_RESOURCE_NAME.openai.azure.com/" } } COSMOS_CONNECTION_accountEndpoint: Endpoint URL for the Azure Cosmos DB account.COSMOS_DATABASE_NAME: Name of the Cosmos DB database to use.COSMOS_CONTAINER_NAME: Name of the Cosmos DB container to use.COSMOS_HASH_PROPERTY: Name of the property used as a hash in Cosmos DB documents (no need to modify this).COSMOS_VECTOR_PROPERTY: Name of the property storing vector embeddings in Cosmos DB.COSMOS_PROPERTY_TO_EMBED: Name of the property whose value will be embedded. Change this based on your document structure.OPENAI_DEPLOYMENT_NAME: Name of the Azure OpenAI model deployment to use for embeddings.OPENAI_DIMENSIONS: Number of dimensions for the embedding vectors.OPENAI_ENDPOINT: Endpoint URL for the Azure OpenAI resource. Run the deploy.sh script: Shell chmod +x deploy.sh ./deploy.sh As part of the azure functionapp publish command that's used in the script, you will be prompted to overwrite the value of the existing AzureWebJobsStorage setting in the local.settings.json file to Azure – choose "no". Run the End-to-End Example Add data to the source container in Cosmos DB. For example, add a document with the following JSON: JSON { "id": "de001c6d-4efe-4a65-a59a-39a0580bfa2a", "description": "Research new technology" } The function should be triggered automatically when the document is added to the container. You can check the logs of the function app to see if it was triggered successfully: Shell func azure functionapp logstream <FUNCTION_APP_NAME> You should see logs similar to this (the payload will be different depending on the data you add): Plain Text 2025-04-23T05:34:41Z [Information] function invoked 2025-04-23T05:34:41Z [Information] cosmosVectorPropertyName: embedding 2025-04-23T05:34:41Z [Information] cosmosVectorPropertyToEmbedName: description 2025-04-23T05:34:41Z [Information] cosmosHashPropertyName: hash 2025-04-23T05:34:41Z [Information] Processing 1 documents 2025-04-23T05:34:41Z [Information] Processing document ID: de001c6d-4efe-4a65-a59a-39a0580bfa2a 2025-04-23T05:34:41Z [Information] Document data: Research new technology 2025-04-23T05:34:41Z [Information] New document detected, generated hash: 5bb57053273563e2fbd4202c666373ccd48f86eaf9198d7927a93a555aa200aa 2025-04-23T05:34:41Z [Information] Document modification status: true, hash: 5bb57053273563e2fbd4202c666373ccd48f86eaf9198d7927a93a555aa200aa 2025-04-23T05:34:41Z [Information] Created embedding for document: map[description:Research new technology id:de001c6d-4efe-4a65-a59a-39a0580bfa2a] 2025-04-23T05:34:41Z [Information] Adding 1 document with embeddings 2025-04-23T05:34:41Z [Information] Added enriched documents to binding output 2025-04-23T05:34:41Z [Information] Executed 'Functions.cosmosdbprocessor' (Succeeded, Id=91f4760f-047a-4867-9030-46a6602ab179, Duration=128ms) //.... Verify the data in Cosmos DB. You should see an embedding for the description property of the document stored in the embedding property. It should look something like this: JSON { "id": "de001c6d-4efe-4a65-a59a-39a0580bfa2a", "description": "Research new technology", "embedding": [ 0.028226057, -0.00958694 //.... ], "hash": "5bb57053273563e2fbd4202c666373ccd48f86eaf9198d7927a93a555aa200aa" } Once the embeddings are generated, you can integrate them with generative AI applications. For example, you can use the vector search feature of Azure Cosmos DB to perform similarity searches based on the embeddings. How It Works Here is a very high-level overview of the code: main.go: Implements an HTTP server with a cosmosdbprocessorendpoint. When triggered, it reads a Cosmos DB trigger payload from the request, parses the nested documents, generates embeddings using Azure OpenAI, and writes the enriched documents back to the Cosmos DB container. Exposes the cosmosdbprocessor endpoint, which processes incoming Cosmos DB documents.For each document, checks if it is new or modified (using a hash), generates an embedding (vector) using Azure OpenAI, and prepares enriched documents for output.Handles logging and error reporting for the function execution.common package: Contains shared utilities and types for processing Cosmos DB documents embedding.go: Handles creation of embeddings using Azure OpenAI.parse.go: Parses and extracts documents from the Cosmos DB trigger payload.payload.go: Defines data structures for payloads and responses used across the project. The function uses a hash property to check if the document has already been processed. If the hash value is different from the one stored in Cosmos DB, it means that the document has been modified and needs to be reprocessed. In this case, the function will generate a new embedding and update the document with the new hash value. This ensures that the function does not get stuck in an infinite loop. If the hash value is the same, it means that the document has not been modified and does not need to be re-processed. In this case, the function will log that the document is unchanged and will not generate a new embedding. You should see logs similar to this: Plain Text 2025-04-23T05:34:42Z [Information] function invoked 2025-04-23T05:34:42Z [Information] cosmosVectorPropertyName: embedding 2025-04-23T05:34:42Z [Information] cosmosVectorPropertyToEmbedName: description 2025-04-23T05:34:42Z [Information] cosmosHashPropertyName: hash 2025-04-23T05:34:42Z [Information] Processing 1 document 2025-04-23T05:34:42Z [Information] Processing document ID: de001c6d-4efe-4a65-a59a-39a0580bfa2a 2025-04-23T05:34:42Z [Information] Document data: Research new technology 2025-04-23T05:34:42Z [Information] Document unchanged, hash: 5bb57053273563e2fbd4202c666373ccd48f86eaf9198d7927a93a555aa200aa 2025-04-23T05:34:42Z [Information] Document modification status: false, hash: 2025-04-23T05:34:42Z [Information] Executed 'Functions.cosmosdbprocessor' (Succeeded, Id=f0cf039a-5de5-4cc1-b29d-928ce32b294e, Duration=6ms) //.... Delete Resources Be sure to clean up the resources you created in Azure. You can do this using the Azure portal or the Azure CLI. For example, to delete the resource group and all its resources, run: Shell az group delete --name <resource-group-name> This will delete the resource group and all its resources, including the Cosmos DB account, function app, and storage account. Conclusion In this blog post, you learned how to build Azure Functions with Go that use Cosmos DB triggers and bindings. You started with a simple function that logs the raw event payload and the Cosmos DB document, and then moved on to a more complex function that generates embeddings for the documents in the Cosmos DB container using Azure OpenAI. You also learned how to run the functions locally using the Cosmos DB emulator and Azure Functions Core Tools, and how to deploy them to Azure. You can use these examples as a starting point for building your own serverless applications with Go and Azure Functions. The combination of Go's performance and simplicity, along with Azure Functions' scalability and integration with Cosmos DB, makes it a powerful platform for building modern applications.

By Abhishek Gupta

CORE

CRDTs Explained: How Conflict-Free Replicated Data Types Work

In today's post, I would like to dive deeper into one of the newest—relatively speaking— topics in the distributed systems domain. As you may have guessed already, the spotlight is on Conflict-free Replicated Data Types or CRDTs for short. I will explain what they are and what role they play in the larger landscape of distributed systems. Let’s start our journey from explaining what Strong Eventual Consistency (SEC) means in this context. Why SEC Matters Consistency is one of the most important—if not the most important—traits in any system. However, the original strong consistency model imposes a significant toll on performance. It also limits the scalability and availability of our systems. As a result, “weaker” consistency models became more and more popular and widely adopted. Eventual consistency promises to solve some of the issues created by strong consistency models. However, it also introduces some totally new types of problems—conflict resolution is one of them. SEC aims to tackle this particular issue. It is a consistency model built atop an eventual consistency that aims to provide a conflict-free environment to ensure availability in the face of failure. It also reduces the cognitive load put on system architects by removing the need for implementing complex conflict-resolution and rollback logic. The theoretical base for SEC is simple mathematics properties like monotonicity, commutativity, and associativity. As such, it is only valid for very specific data types and operations. These data types are commonly denoted as CRDT. It's not surprising, taking into consideration SEC was introduced in the original, as far as I know, CRDT paper. What are CRDTs? CRDTs are a data structure designed to ensure that data on different computers (replicas) will eventually converge—and will be merged—into a consistent state. All of that, no matter what modifications were made and without any special conflict resolution code or user intervention. Additionally, CRDTs are decentralized, and thus, they do not need any coordination between the replicas. Particular replicas exchange data between each other. This trait makes them quite interesting and different from algorithms used in most online gaming and Distributed File Systems (DFS). We can differentiate two basic types of CRDTs: object-based and state-based. There is also a delta-based type which is an extension on top of the state-based CRDTs family. Convergent Replicated Data Types (CvRDTs) CvRDTs are state-based CRDTs. They rely on continuous exchanges of current states between particular replicas. By the way, this is a classic use case for the gossip protocol. When a replica receives the new version of the state, it uses a predefined merge function, effectively updating its own state. In such a setting, when updates stop coming, all the replicas will reach the same, consistent state. Keep in mind that the key here is that the replicas exchange the total state each time. Thus, the size of messages may become quite big. Commutative Replicated Data Types (CmRDTs) CmRDTs (also called operation-based CRDT) are an alternative to state-based types. Contrary to state-based types, they do not have a merge method. Instead, they split the update operations into two steps: prepare update and effect-update. The first phase is executed locally at a particular replica, and it is directly followed by the second phase that executes across all other replicas, effectively equalizing the state across the whole deployment. However, for the second phase to work correctly, CmRDTs require a reliable communication protocol that provides causal ordering of messages. While it is not a very complex problem, because such tools are very common nowadays, it adds another layer of complexity. Equivalence There is one interesting fact regarding both CvRDTs and CmRDTs. They are equivalent to each other, at least from a mathematical perspective. In the previously linked paper, there is an entire subsection (3.2) explaining in great detail why this statement holds true. I will not be copy-pasting the same text here—TLDR, it is based on emulating one type with the other. Delta-State Conflict-Free Replicated Data Types (δ-CRDT) Paulo Sérgio Almeida et al., in their paper Efficient State-based CRDTs by Delta-Mutation, proposed δ-CRDT. It is an extension on top of classic state-based CRDTs which addresses its biggest weakness; the continuous exchange of messages containing the full state of the object. There are two key concepts used to achieve this, namely, the delta-mutator and a delta-state. The δ-state is a representation of changes applied by the mutator to the current state. This delta is later sent to other replicas, effectively reducing the size of messages. Additionally, to reduce the number of messages exchanged between replicas, we can group multiple deltas into a delta-group. I do not want to get too much into the details of different types; there is much more here to uncover. If you are interested in all the math behind CRDTs, you can find all of these details here. CRDTs Fault Tolerance In terms of classic availability and fault tolerance, CRDTs are quite an interesting case. Their base consistency model—SEC—promises to provide very high resilience. It is possible, mostly thanks to the eventual consistency of SEC itself, but also the resilient nature of the CRDTs algorithms themselves. In case of state-based CRDTs, they exchange full state between each other, so besides the case of total failure, sooner or later the replicas should be able to converge into a consistent state. On the other hand, in the case of operation-based (op-based) CRDTs, the update-effect is cumulative, so again no matter the order of messages spread throughout the replicas, they will also be able to converge on an equivalent state. With delta CRDTs, the situation is similar, as it is built upon both op- and state-based types. There are 3 traits of CRDTs that make them especially resilient: Decentralized CRDTs operate without a central coordinator, eliminating single points of failure. Thus, they naturally handle network partitions. Updates are applied locally and propagate when communication is restored. Asynchronous Communication CRDTs utilize only async communication, either via a gossip-based protocol or some broadcasting protocols. Nodes do not need to wait for any type of acknowledgments, nor do they use any type of consensus algorithm. The CRDT-wide state convergence happens asynchronously. Node Failures and Recovery Nodes continue to store and process their local state even in case of network failure. Upon recovery, they can synchronize with other replicas to merge any missed updates. Byzantine fault tolerance Despite all the traits above and in general, very high fault tolerance, CRDTs are not fully invincible. There is a very particular type of failures which CRDTs cannot easily recover from—Byzantine faults. Ironically, the exact same thing that makes CRDTs so highly available—decentralization—is also the main factor of them being susceptible to Byzantine fault. Byzantine faults occur when nodes in a distributed system behave maliciously or send malformed states, potentially leading to inconsistencies. In such a situation, reaching a consistent state across all the replicas through a gossip-based protocol or broadcast can be highly problematic. Unfortunately, at least in this case, CRDTs heavily rely on exactly these approaches. Making CRDTs Byzantine fault-tolerant is a relatively new and hot topic among researchers focused on distributed systems, with Martin Kleppmann’s paper Making CRDTs Byzantine Fault Tolerant being one of the most cited CRDTs papers ever. CRDTs vs CAP CAP Theorem describes the spectrum of availability and consistency while stating that having both of them at the same time is not possible. CRDTs put this claim into question to some extent, at least part of it, as CAP is more nuanced than just consistency vs availability. CRDTs promise very high availability and eventual consistency. CRDT replicas are always available for reads and writes no matter the network partition or failures, and what is more, any subset of communicating replicas will eventually be consistent. While it is not the same as the lineralization required by CAP, it still gives strong guarantees as to the eventual state consistency. CRDTs show that CAP is more of a spectrum than an exact choice, and that we can balance both availability and consistency throughout our system. Types of CRDT The full list of all existing CRDTs is very, very long and would require multiple pages to list, not to mention describe. Here I will cover only some basic types which can later be used to build more complex structures. Let’s start with a simple register. Register Register is the simplest CRDT structure. It is responsible for holding a single value, like a variable. There are two basic semantics for building CRDT registers, depending on how they approach the resolving of concurrent writes: Multi-value Register - Stores and returns all concurrently written values, effectively returning a multi-set. Requires a conflict resolution mechanism on a higher level.Last-write-wins Register (LWW) - As the name suggests, only the newest value will be stored in the register. Counter A counter is similar to a register in the fact that it stores only one value, to be precise, a numeric type. In the case of the counter, we can also differentiate two basic types: Grow-only counter (GCounter) - The simplest counter that only supports an increment operation. In this counter, each replica holds its own state, and the global state is the sum of all local counters.Positive-Negative Counter (PN-Counter) - Somewhat more complex counter; it supports both increment and decrement operations. It tracks increments and decrements as two counters (GCounters in particular). The result is computed by counting the difference between them. Global state, similarly as in the case of GCounter, is the total sum of all counters across the nodes. Set Surprising as it may be, this is just a normal set, but distributed in a CRDT manner. We have multiple different set-like CRDTs. Grow-only set (GSet) is one of the most basic ones. It works almost the same way as GCounter, so I will not spend too much time on it.Another one is USet that works in a similar fashion to PN-Counter, using GSet to handle adds and removes. The USet returns the set difference between them.We also have Add-wins sets that favor the add operation while resolving conflicts between addition and removal of a particular element in the set.There is Remove-wins set that works in the directly opposite manner to Add-wins set and favors removal operations during conflict resolution.Later, we have even more CRDTs like Last-write-wins set, ORSet (observable-removal), and many more. Sequence Sequence CRDTs are a very specialized type of structure. They are extensively used in the field of collaborative editing—documents shared and edited in Google Docs. There are multiple open-source implementations of this type of CRDT. Here are a few examples, with Yjs probably being the most popular one (over 17k stars on GitHub), followed by Automerge (4k stars on GitHub), and many, many more. Map The case of Map is very similar to the Set CRDTs. We have Add-wins Map, Remove-wins Map, Last-write-wins Map. All of these structures have similar behavior as their set counterparts but with one difference: the conflict resolution is handled on a per-key basis. An interesting case is the Multi-value Map, similar to Multi-value Register, where the result of each concurrent put operation is stored within the same key, and conflict resolution needs to be handled on a higher level. A more advanced case of a Map-based structure is a Map of CRDTs, for example a PN-Counter Map that holds PN-Counters as entry values. There is some more nuance behavior when we want to update such entries, but in the end, composing CRDTs is a relatively easy task. This is just a simplified and shortened list of all available basic CRDTs, probably not even all high-level types are covered above. For example, we also have graph-based CRDTs which can be implemented using a GSet of GSets. As to the full one, I’m not sure if it even exists, however the list available here is somewhat lengthier. Moreover, as you could see above, with PN-Counter you can build more complex CRDTs from simpler ones as building blocks. Summary You could read a quite comprehensive introduction to the subject of CRDTs above. You now know what CvRDTs mean and what the difference is between them and delta CRDTs. You also have some insight on how they behave when put in unfavorable situations. Moreover, you know some of the basic CRDT types, what they are, and where you can use them. If you would like to read more about CRDTs, here is a very good page. It is run by Martin Kleppmann and aggregates a lot of data around CRDTs, like white papers and actual implementations. Thank you for your time.

By Bartłomiej Żyliński

CORE

Failure Handling Mechanisms in Microservices and Their Importance

Microservices architecture has gained significant popularity due to its scalability, flexibility, and modular nature. However, with multiple independent services communicating over a network, failures are inevitable. A robust failure-handling strategy is crucial to ensure reliability, resilience, and a seamless user experience. In this article, we will explore different failure-handling mechanisms in microservices and understand their importance in building resilient applications. Why Failure Handling Matters in Microservices? Without proper failure-handling mechanisms, these failures can lead to system-wide disruptions, degraded performance, or even complete downtime. Failure scenarios commonly occur due to: Network failures (e.g., DNS issues, latency spikes)Service unavailability (e.g., dependent services down)Database outages (e.g., connection pool exhaustion)Traffic spikes (e.g., unexpected high load) In Netflix, if the recommendation service is down, it shouldn’t prevent users from streaming videos. Instead, Netflix degrades gracefully by displaying generic recommendations. Key Failure Handling Mechanisms in Microservices 1. Retry Mechanism Sometimes, failures are temporary (e.g., network fluctuations, brief server downtime). Instead of immediately failing, a retry mechanism allows the system to automatically reattempt the request after a short delay. Use cases: Database connection timeoutsTransient network failuresAPI rate limits (e.g., retrying failed API calls after a cooldown period) For example, Amazon’s order service retries fetching inventory from a database before marking an item as out of stock. Best practice: Use Exponential Backoff and Jitter to prevent thundering herds. Using Resilience4j Retry: Java @Retry(name = "backendService", fallbackMethod = "fallbackResponse") public String callBackendService() { return restTemplate.getForObject("http://backend-service/api/data", String.class); } public String fallbackResponse(Exception e) { return "Service is currently unavailable. Please try again later."; } 2. Circuit Breaker Pattern If a microservice is consistently failing, retrying too many times can worsen the issue by overloading the system. A circuit breaker prevents this by blocking further requests to the failing service for a cooldown period. Use cases: Preventing cascading failures in third-party services (e.g., payment gateways)Handling database connection failuresAvoiding overloading during traffic spikes For example, Netflix uses circuit breakers to prevent overloading failing microservices and reroutes requests to backup services. States used: Closed → Calls allowed as normal.Open → Requests are blocked after multiple failures.Half-Open → Test limited requests to check recovery. Below is an example using Circuit Breaker in Spring Boot (Resilience4j). Java @CircuitBreaker(name = "paymentService", fallbackMethod = "fallbackPayment") public String processPayment() { return restTemplate.getForObject("http://payment-service/pay", String.class); } public String fallbackPayment(Exception e) { return "Payment service is currently unavailable. Please try again later."; } 3. Timeout Handling Slow service can block resources, causing cascading failures. Setting timeouts ensures a failing service doesn’t hold up other processes. Use cases: Preventing slow services from blocking threads in high-traffic applicationsHandling third-party API delaysAvoiding deadlocks in distributed systems For example, Uber’s trip service times out requests if a response isn’t received within 2 seconds, ensuring riders don’t wait indefinitely. Below is an example of how to set timeouts in Spring Boot (RestTemplate and WebClient). Java @Bean public RestTemplate restTemplate() { var factory = new SimpleClientHttpRequestFactory(); factory.setConnectTimeout(3000); // 3 seconds factory.setReadTimeout(3000); return new RestTemplate(factory); } 4. Fallback Strategies When a service is down, fallback mechanisms provide alternative responses instead of failing completely. Use cases: Showing cached data when a service is downReturning default recommendations in an e-commerce app Providing a static response when an API is slow For example, YouTube provides trending videos when personalized recommendations fail. Below is an example for implementing Fallback in Resilience4j. Java @Retry(name = "recommendationService") @CircuitBreaker(name = "recommendationService", fallbackMethod = "defaultRecommendations") public List<String> getRecommendations() { return restTemplate.getForObject("http://recommendation-service/api", List.class); } public List<String> defaultRecommendations(Exception e) { return List.of("Popular Movie 1", "Popular Movie 2"); // Generic fallback } 5. Bulkhead Pattern Bulkhead pattern isolates failures by restricting resource consumption per service. This prevents failures from spreading across the system. Use cases: Preventing one failing service from consuming all resourcesIsolating failures in multi-tenant systemsAvoiding memory leaks due to excessive load For example, Airbnb’s booking system ensures that reservation services don’t consume all resources, keeping user authentication operational. Java @Bulkhead(name = "inventoryService", type = Bulkhead.Type.THREADPOOL) public String checkInventory() { return restTemplate.getForObject("http://inventory-service/stock", String.class); } 6. Message Queue for Asynchronous Processing Instead of direct service calls, use message queues (Kafka, RabbitMQ) to decouple microservices, ensuring failures don’t impact real-time operations. Use cases: Decoupling microservices (Order Service → Payment Service)Ensuring reliable event-driven processing Handling traffic spikes gracefully For example, Amazon queues order processing requests in Kafka to avoid failures affecting checkout. Below is an example of using Kafka for order processing. Java @Autowired private KafkaTemplate<String, String> kafkaTemplate; public void placeOrder(Order order) { kafkaTemplate.send("orders", order.toString()); // Send order details to Kafka } 7. Event Sourcing and Saga Pattern When a distributed transaction fails, event sourcing ensures that each step can be rolled back. Banking applications use Saga to prevent money from being deducted if a transfer fails. Below is an example of a Saga pattern for distributed transactions. Java @SagaOrchestrator public void processOrder(Order order) { sagaStep1(); // Reserve inventory sagaStep2(); // Deduct balance sagaStep3(); // Confirm order } 8. Centralized Logging and Monitoring Microservices are highly distributed, without proper logging and monitoring, failures remain undetected until they become critical. In a microservices environment, logs are distributed across multiple services, containers, and hosts. A log aggregation tool collects logs from all microservices into a single dashboard, enabling faster failure detection and resolution. Instead of storing logs separately for each service, a log aggregator collects and centralizes logs, helping teams analyze failures in one place. Below is an example of logging in microservices using the ELK stack (Elasticsearch, Logstash, Kibana). YAML logging: level: root: INFO org.springframework.web: DEBUG Best Practices for Failure Handling in Microservices Design for Failure Failures in microservices are inevitable. Instead of trying to eliminate failures completely, anticipate them and build resilience into the system. This means designing microservices to recover automatically and minimize user impact when failures occur. Test Failure Scenarios Most systems are only tested for success cases, but real-world failures happen in unexpected ways. Chaos engineering helps simulate failures to test how microservices handle them. Graceful Degradation In high-traffic scenarios or service failures, the system should prioritize critical features and gracefully degrade less essential functionalities. Prioritize essential services over non-critical ones. Idempotency Ensure retries don’t duplicate transactions. If a microservice retries a request due to a network failure or timeout, it can accidentally create duplicate transactions (e.g., charging a customer twice). Idempotency ensures that repeated requests have the same effect as a single request. Conclusion Failure handling in microservices is not optional — it’s a necessity. By implementing retries, circuit breakers, timeouts, bulkheads, and fallback strategies, you can build resilient and fault-tolerant microservices.

By Arunkumar Kallyodan

Why Testing is a Long-Term Investment for Software Engineers

In the world of software engineering, we’re constantly racing against the clock—deadlines, deployments, and decisions. In this rush, testing often gets sidelined. Some developers see it as optional, or something they’ll “get to later.” But that’s a costly mistake. Because just like documentation, testing is a long-term investment—one that pays off in quality, safety, and peace of mind. Testing is crucial. It’s about ensuring quality, guaranteeing expected behavior, and enabling safe refactoring. Without tests, every change becomes a risk. With tests, change becomes an opportunity to improve. Testing doesn’t just prevent bugs. It shapes the way we build software. It enables confident change, unlocks collaboration, and acts as a form of executable documentation. Tests are a Guarantee of Behavior At its core, a test is a contract. It tells the system—and anyone reading the code—what should happen when given specific inputs. This contract helps ensure that as the software evolves, its expected behavior remains intact. A system without tests is like a building without smoke detectors. Sure, it might stand fine for now, but the moment something catches fire, there’s no safety mechanism to contain the damage. Testing Supports Safe Refactoring Over time, all code becomes legacy. Business requirements shift, architectures evolve, and what once worked becomes outdated. That’s why refactoring is not a luxury—it’s a necessity. But refactoring without tests? That’s walking blindfolded through a minefield. With a reliable test suite, engineers can reshape and improve their code with confidence. Tests confirm that behavior hasn’t changed—even as the internal structure is optimized. This is why tests are essential not just for correctness, but for sustainable growth. Tests Help Teams Move Faster There’s a common myth: tests slow you down. But seasoned engineers know the opposite is true. Tests speed up development by reducing time spent debugging, catching regressions early, and removing the need for manual verification after every change. They also allow teams to work independently, since tests define and validate interfaces between components. The ROI of testing becomes especially clear over time. It’s a long-term bet that pays exponential dividends. When to Use Mocks (and When not to) Not every test has to touch a database or external service. That’s where mocks come in. A mock is a lightweight substitute for an absolute dependency—valid when you want to isolate logic, simulate failures, or verify interactions without relying on complete integration. Use mocks when: You want to test business logic in isolationYou need to simulate rare or hard-to-reproduce scenariosYou want fast, deterministic tests that don’t rely on external state But be cautious: mocking too much can lead to fragile tests that don’t reflect reality. Always complement unit tests with integration tests that use real components to validate your system holistically. A Practical Stack for Java Testing If you're working in Java, here's a battle-tested stack that combines readability, power, and simplicity: JUnit Jupiter JUnit is the foundation for writing structured unit and integration tests. It supports lifecycle hooks, parameterized tests, and extensions with ease. AssertJ This is a fluent assertion library that makes your tests expressive and readable. Instead of writingassertEquals(expected, actual), you write assertThat(actual).isEqualTo(expected)—much more human-friendly. Testcontainers These are perfect for integration tests. With Testcontainers, you can spin up real databases, message brokers, or services in Docker containers as part of your test lifecycle—no mocks, no fakes—just the real thing, isolated and reproducible. Here’s a simple example of combining all three: Java @Test void shouldPersistGuestInDatabase() { Guest guest = new Guest("Ada Lovelace"); guestRepository.save(guest); List<Guest> guests = guestRepository.findAll(); assertThat(guests).hasSize(1).extracting(Guest::getName).contains("Ada Lovelace"); } This kind of test, when paired with Testcontainers and a real database, gives you confidence that your system works, not just in theory, but in practice. Learn More: Testing Java Microservices For a deeper dive into testing strategies—including contract testing, service virtualization, and containerized tests—check out Testing Java Microservices. It’s an excellent resource that aligns with modern practices and real-world challenges. Understanding the Value of Metrics in Testing Once tests are written and passing, a natural follow-up question arises: how do we know they're doing their job? In other words, how can we be certain that our tests are identifying genuine problems, rather than merely giving us a false sense of security? This is where testing metrics come into play—not as final verdicts, but as tools for better judgment. Two of the most common and impactful metrics in this space are code coverage and mutation testing. Code coverage measures how much of your source code is executed when your tests run. It’s often visualized as a percentage and can be broken down by lines, branches, methods, or even conditions. The appeal is obvious: it gives a quick sense of how thoroughly the system is being exercised. But while coverage is easy to track, it’s just as easy to misunderstand. The key limitation of code coverage is that it indicates where the code executes, but not how effectively it is being executed. A line of code can be executed without a single meaningful assertion. This means a project with high coverage might still be fragile underneath—false confidence is a real risk. That’s where mutation testing comes in. This approach works by introducing small changes—known as mutants—into the code, such as flipping a conditional or changing an arithmetic operator. The test suite is then rerun to see whether it detects the change. If the tests fail, the mutant is considered “killed,” indicating that the test is practical. If they pass, the mutant “survives,” exposing a weakness in the test suite. Mutation testing digs into test quality in a way coverage cannot. It challenges the resilience of your tests and asks: Would this test catch a bug if the logic were to break slightly? Of course, this comes with a cost. Mutation testing is slower and more computationally intensive. On large codebases, it can take considerable time to run, and depending on the granularity and mutation strategy, the results can be noisy or overwhelming. That’s why it’s best applied selectively—used on complex business logic or critical paths where the risk of undetected bugs is high. Now here’s where things get powerful: coverage and mutation testing aren’t competing metrics—they’re complementary. Coverage helps you identify what parts of your code aren't being tested at all. Mutation testing indicates how well the tested parts are protected. Used together, they offer a fuller picture: breadth from coverage, and depth from mutation. But even combined, they should not become the ultimate goal. Metrics exist to serve understanding, not to replace it. Chasing a 100% mutation score or full coverage can lead to unrealistic expectations or, worse, wasted effort on tests that don’t matter. What truly matters is having enough coverage and confidence in the parts of the system that are hard to change or essential to your business. In the end, the most valuable metric is trust: trust that your system behaves as expected, trust that changes won’t break things silently, and trust that your test suite is more than a checkbox—it’s a safety net that allows you to move fast without fear. Coverage and mutation testing, when used wisely, help you build and maintain that trust. Final Thoughts: Test Like a Professional Testing is more than a safety net; it’s a form of engineering craftsmanship. It’s how we communicate, refactor, scale, and collaborate without fear. So, treat tests like you treat production code—because they are. They’re your guarantee that what works today still works tomorrow. And in the ever-changing world of software, that’s one of the most valuable guarantees you can have.

By Otavio Santana

CORE