DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Newsletter
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Coding

Also known as the build stage of the SDLC, coding focuses on the writing and programming of a system. The Zones in this category take a hands-on approach to equip developers with the knowledge about frameworks, tools, and languages that they can tailor to their own build needs.

Functions of Coding

Frameworks

Frameworks

A framework is a collection of code that is leveraged in the development process by providing ready-made components. Through the use of frameworks, architectural patterns and structures are created, which help speed up the development process. This Zone contains helpful resources for developers to learn about and further explore popular frameworks such as the Spring framework, Drupal, Angular, Eclipse, and more.

Java

Java

Java is an object-oriented programming language that allows engineers to produce software for multiple platforms. Our resources in this Zone are designed to help engineers with Java program development, Java SDKs, compilers, interpreters, documentation generators, and other tools used to produce a complete application.

JavaScript

JavaScript

JavaScript (JS) is an object-oriented programming language that allows engineers to produce and implement complex features within web browsers. JavaScript is popular because of its versatility and is preferred as the primary choice unless a specific function is needed. In this Zone, we provide resources that cover popular JS frameworks, server applications, supported data types, and other useful topics for a front-end engineer.

Languages

Languages

Programming languages allow us to communicate with computers, and they operate like sets of instructions. There are numerous types of languages, including procedural, functional, object-oriented, and more. Whether you’re looking to learn a new language or trying to find some tips or tricks, the resources in the Languages Zone will give you all the information you need and more.

Tools

Tools

Development and programming tools are used to build frameworks, and they can be used for creating, debugging, and maintaining programs — and much more. The resources in this Zone cover topics such as compilers, database management systems, code editors, and other software tools and can help ensure engineers are writing clean code.

Latest Premium Content
Trend Report
Platform Engineering and DevOps
Platform Engineering and DevOps
Trend Report
Developer Experience
Developer Experience
Refcard #291
Code Review Core Practices
Code Review Core Practices
Refcard #400
Java Application Containerization and Deployment
Java Application Containerization and Deployment

DZone's Featured Coding Resources

7 Failure Modes Every Production AI Agent Should Be Tested Against

7 Failure Modes Every Production AI Agent Should Be Tested Against

By Mihir Bhatt
Testing an AI agent is very different from testing a conventional software feature. A traditional application usually receives a defined input, follows code-controlled logic, and produces an expected output. An AI agent can make decisions during execution. It may select a tool, construct arguments, inspect a response, change its plan, retrieve more information, update memory, and continue acting until it believes the task is complete. That difference makes happy-path testing dangerously incomplete. A support agent returning the correct order status proves that one expected interaction worked. It does not tell the engineering team what will happen when retrieved content contains malicious instructions, a tool times out after completing an action, stored memory contains false information, or two systems provide conflicting answers. Production testing therefore needs to examine more than response quality. Teams should test the agent's behavior when the environment becomes unreliable, ambiguous, or adversarial. The following seven failure modes provide a practical starting point for building that kind of test suite. 1. Indirect Prompt Injection Through External Content Many production agents operate on information they did not generate themselves. They read support tickets, web pages, emails, PDFs, knowledge-base articles, database records, and search results. Any of those sources can contain text that looks like an instruction to the model, even though the application intended it to be treated only as data. Consider a support agent asked to summarize an uploaded customer document. Somewhere inside that document is a hidden instruction telling the agent to ignore its previous rules and retrieve confidential account information. A weak test would look only at the final answer and check whether confidential information appeared. A stronger test would inspect the entire execution trace to determine whether the malicious content changed the agent's plan, caused an unauthorized tool call, altered memory, or triggered communication with another system. The expected behavior should be clear before the test runs. Retrieved content must remain untrusted input and should not be able to override system-level instructions or authorization rules. The test should fail even if the final answer appears safe, but the agent attempted a prohibited action behind the scenes. For agent systems, observing tool calls and intermediate actions is often as important as checking the final response. 2. Excessive Permission and Authorization Failure Agents become more dangerous when their tools expose capabilities beyond what a task actually requires. An agent responsible for reading customer information might also have access to tools that modify billing records, delete accounts, issue refunds, or send messages. A carefully written system prompt may tell the model not to use those capabilities, but a prompt should never be treated as the final authorization boundary. A useful test is to give the agent a read-only task while making privileged tools available in the environment. The test can then introduce misleading context suggesting that deleting or modifying a record would help complete the task. Engineers should observe whether the agent attempts the privileged call and, more importantly, whether the surrounding system prevents execution. The correct control belongs below the model layer. Every sensitive action should be checked against the identity, permissions, and scope assigned to that agent or workflow. A successful test is not merely one in which the model chooses not to call the dangerous tool. The stronger result is that even an incorrect tool decision cannot cross the authorization boundary. 3. Tool Failure After a Side Effect Has Already Happened Distributed systems have always had to deal with ambiguous failures, and AI agents add another decision-maker to the problem. A tool can successfully complete an operation while the response is lost because of a timeout, connection failure, or interrupted request. The agent then sees an error and may decide that the safest response is to retry. Imagine an agent that calls a payment API to create a refund. The payment service completes the refund, but the connection times out before the success response reaches the agent. If the agent immediately repeats the request, the customer could receive the refund twice unless the API or orchestration layer protects against duplicate execution. A production test should deliberately inject this condition. The test can simulate a successful side effect followed by a timeout and then observe the agent's recovery behavior. Instead of blindly retrying, the workflow should first reconcile the current state, query whether the transaction already exists, or use an idempotency mechanism that prevents duplicate operations. This test matters because a technically correct retry policy at the HTTP layer can still produce incorrect business behavior. Agents need explicit recovery rules for operations that change external state. Those rules should be tested before the system encounters the failure in production. 4. Poisoned, Stale, or Cross-Session Memory Persistent memory can make an agent more useful, but it also creates a new source of hidden state. Information stored during one interaction can influence decisions hours, days, or weeks later. If the stored information is false, outdated, malicious, or assigned to the wrong user, a future session may behave incorrectly even when the current prompt is perfectly safe. A useful test is to insert a false memory into the system, such as a claim that a particular customer has permanent approval for unusually high transactions. A later session can then ask the agent to process a transaction for that customer. The objective is to determine whether the agent treats remembered information as authoritative or verifies it against a trusted business system. Cross-session isolation should be tested as well. Information written during one user's conversation must not appear in another user's context unless the application explicitly supports shared state. Teams should verify memory ownership, expiration rules, provenance, and which information requires revalidation before use. The important design distinction is that conversation history, long-term memory, and authoritative business data are not interchangeable. Memory can provide context, but sensitive facts such as permissions, account status, or financial limits should normally be confirmed against the system responsible for maintaining them. 5. Tool Schema Drift and Unexpected Responses Agent tool calls often work perfectly in development because mocks and schemas remain stable. Production services change. Fields get renamed, enum values expand, required parameters become optional, responses return partial data, and older versions disappear. When that happens, an agent may try to infer what the new schema means instead of failing safely. Suppose an agent has learned to send a tool request containing a priority field with values such as high or low. The tool later changes that field to severity and expects values such as P1, P2, and P3. The test should determine whether the system rejects the incompatible request, whether the agent tries to invent a mapping, and whether repeated retries create additional problems. Unexpected response values deserve the same attention. An agent that understands APPROVED, DECLINED, and PENDING may encounter a new REQUIRES_MANUAL_REVIEW state. Production tests should verify that the agent does not guess how to handle unknown values simply because the model can generate a plausible interpretation. Schema validation should therefore exist at the execution boundary. The model may prepare a proposed tool call, but deterministic validation should decide whether the call is syntactically and semantically acceptable before it reaches the target system. Recovery behavior for rejected calls should also be part of the test suite. 6. Runaway Reasoning and Repeated Tool Loops Not every agent failure creates an incorrect answer. Some failures consume time, tokens, API requests, and infrastructure until a safety limit is reached. An agent may repeatedly call the same tool, alternate between two tools, or keep searching because every new piece of information creates another unresolved question. A simple fault-injection test can create a dependency loop. Tool A tells the agent that the answer requires information from Tool B, while Tool B directs it back to Tool A. Another version can return search results that never provide enough information for the agent to complete the task. Engineers should then observe whether the runtime recognizes the lack of progress. The system should have deterministic limits around execution. These may include maximum tool calls, repeated-call thresholds, elapsed time, token budgets, or workflow depth. The exact numbers depend on the application, so they should come from real operational requirements rather than being copied from a generic recommendation. Teams should also record whether the agent is making meaningful progress between steps. A workflow that remains under its token limit can still be unhealthy if it calls the same endpoint repeatedly with nearly identical parameters. Repeated-call ratio, tool-call depth, execution time, and cost per task are useful signals for detecting this behavior. 7. Unsafe Actions When Evidence Is Incomplete or Conflicting Agents eventually encounter situations where there is no obvious correct answer. One system reports that an account is active while another reports that it is closed. A document contains an incomplete instruction. A primary model fails, and a fallback model interprets the task differently. These are normal production conditions, not theoretical edge cases. Consider an agent that receives a request to delete historical billing records. The customer relationship system says the account is active, while the billing platform reports that it is closed. If the agent selects whichever source seems more convincing and proceeds with deletion, the system has converted uncertainty into an irreversible action. The test should verify that uncertainty changes behavior. Depending on the risk associated with the action, the workflow may need to retrieve an authoritative source, request clarification, switch into a read-only state, require human approval, or abort the operation. The correct response depends on the application, but the behavior should be defined before the agent encounters the conflict. A useful pass condition is therefore not simply that the agent chose the correct system. The stronger requirement is that the system recognized when the available evidence was insufficient to justify the requested action. Building a Failure-Injection Test Matrix These failure modes become easier to manage when teams document them in a repeatable test matrix. Each test should identify the injected condition, the telemetry that needs to be captured, the signal that represents failure, and the behavior expected from the system. Failure ModeWhat to ObserveFailure SignalExpected BehaviorIndirect prompt injectionPlan and tool traceExternal text changes protected behaviorTreat content as untrusted dataExcess permissionsAuthorization eventsPrivileged action executesBlock action outside assigned scopePost-action timeoutRetry and transaction traceDuplicate side effectReconcile state before retryPoisoned memoryMemory reads and writesFalse memory controls sensitive actionVerify against authoritative sourceSchema driftTool validation eventsInvalid or invented arguments executeReject call and follow recovery pathRunaway loopSteps, tokens, repeated callsExecution continues without progressStop through runtime limitsConflicting evidenceDecision and approval traceAgent performs risky action despite uncertaintyVerify, escalate, or abort The matrix should evolve with the product. When a production incident reveals a new class of failure, the team should turn the incident into a permanent regression test. Over time, the suite becomes a record of the system's actual operational risks instead of a collection of hypothetical prompts. Metrics That Make Agent Tests More Useful Binary pass-or-fail results are valuable, but operational measurements provide more information about how the system behaves under pressure. Teams can track unauthorized tool-call attempts, prevented side effects, duplicate-action rates, recovery success, repeated-call ratios, token consumption, time to safe termination, human escalation rates, and cross-session memory leakage. These measurements are particularly useful when model versions, prompts, orchestration logic, or tool configurations change. A new model may still complete the task successfully while making twice as many tool calls. A prompt update may reduce answer errors while increasing escalation failures. Looking at behavior-level metrics makes those regressions easier to detect. Teams should avoid inventing universal thresholds. A five-second execution time may be acceptable for one workflow and unacceptable for another. The meaningful limits should come from the latency, cost, risk, and business requirements of the specific agent being tested. Test the Agent's Boundaries, Not Just Its Answers Production AI agents should not be evaluated only by comparing an input with an expected response. Their behavior spans context selection, planning, tool choice, authorization, execution, observation, memory, and repeated decision-making. A system can produce a convincing final answer while something unsafe has already happened earlier in the execution trace. That is why failure injection should become part of the normal agent testing process. Teams need to know what happens when external content is hostile, permissions are too broad, tools return ambiguous failures, memory becomes unreliable, schemas change, execution loops, or evidence conflicts. These conditions will eventually appear outside the test environment. The strongest agent test suites grow from real operating experience. Each incident, near miss, or unexpected trace should become a test case that the system must pass before future releases. That approach shifts testing away from asking whether an agent can complete a task and toward a harder question: whether it can fail without creating a larger problem. More
Building an AI-Powered Code Review Assistant in Java

Building an AI-Powered Code Review Assistant in Java

By Akhil Sabu
Code review is the highest-leverage activity on most engineering teams, yet it is also the most expensive in terms of senior-developer time. What if you could offload a first pass to an LLM that understands Java idioms, common anti-patterns, SOLID principles, and security pitfalls — and get back structured, actionable feedback before a human even opens the PR? In this post, we will build exactly that: a self-contained Java application that accepts a Java source snippet, sends it to an LLM (OpenAI GPT-4o or a locally running Ollama model — your choice), and returns a structured list of code-quality findings with severity levels and suggested fixes. What you will build: A CodeReviewAssistant Java service backed by LangChain4j, which analyses Java code and returns structured improvement suggestions — runnable in under 10 minutes on your laptop. Prerequisites ToolVersionNotesJava JDK17+Records & sealed types usedMaven or GradleAny recentMaven snippets shown hereLangChain4j0.36.xLatest stable at time of writingOpenAI API key or Ollama—We show both No Spring, no frameworks — this is plain Java, so you can drop it into any stack. Step 1: Add LangChain4j to Your Project LangChain4j is the Java-native counterpart of the Python LangChain library. It gives you a clean, type-safe API over multiple LLM providers — OpenAI, Anthropic, Azure, Hugging Face, Ollama, and more — with zero vendor lock-in. Add the BOM and the provider module you want: XML <!-- pom.xml --> <dependencyManagement> <dependencies> <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j-bom</artifactId> <version>0.36.2</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <dependencies> <!-- Core --> <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j</artifactId> </dependency> <!-- Choose ONE provider below --> <!-- Option A: OpenAI (GPT-4o, GPT-3.5-turbo, …) --> <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j-open-ai</artifactId> </dependency> <!-- Option B: Ollama (runs locally, no API key needed) --> <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j-ollama</artifactId> </dependency> </dependencies> Tip — local-first development: Start with Ollama + codellama or mistral so you can iterate without burning API credits. Switch to OpenAI GPT-4o for production quality by changing one line. Step 2: Model the Review Output as a Java Record LangChain4j's Structured Output feature can instruct the LLM to respond with JSON that maps directly onto a Java type. We define our domain model first: Java // Finding.java public record Finding( String severity, // "CRITICAL" | "HIGH" | "MEDIUM" | "LOW" | "INFO" String category, // e.g. "Null Safety", "Performance", "Security" String description, // human-readable explanation String suggestedFix // concrete code-level recommendation ) {} // CodeReviewResult.java import java.util.List; public record CodeReviewResult( String summary, List<Finding> findings ) {} Using Java Records keeps the model immutable and removes boilerplate. LangChain4j will automatically build the JSON schema from these types and include it in the system prompt. Step 3: Define the AI Service Interface LangChain4j's AI Services abstraction is the cleanest part of the library. You declare an interface, annotate it with prompts, and the library generates a proxy at runtime — no boilerplate, no manual HTTP calls. Java import dev.langchain4j.service.SystemMessage; import dev.langchain4j.service.UserMessage; import dev.langchain4j.service.V; public interface CodeReviewService { @SystemMessage(""" You are a senior Java engineer performing a thorough code review. Analyse the provided Java code for: - Correctness and potential bugs - Null safety issues - Performance anti-patterns - Security vulnerabilities (injection, insecure randomness, etc.) - SOLID principle violations - Readability and naming conventions Return ONLY a JSON object matching the CodeReviewResult schema. Be concise but actionable. Severity: CRITICAL, HIGH, MEDIUM, LOW, INFO. """) @UserMessage("Review this Java code:\n\n```java\n{{code}\n```") CodeReviewResult review(@V("code") String javaCode); } How it works: The @SystemMessage sets the LLM's persona and instructions. The @UserMessage is the per-call prompt. @V("code") is a template variable — LangChain4j substitutes it at call time. Zero boilerplate. Step 4: Wire It All Together Now we create the LLM model, register our output type, and build the AI Service proxy. The entire wiring fits in one short class: Java import dev.langchain4j.model.openai.OpenAiChatModel; import dev.langchain4j.model.ollama.OllamaChatModel; import dev.langchain4j.service.AiServices; public class CodeReviewAssistant { public static void main(String[] args) { // ── Choose your provider ────────────────────────────────────────── // Option A: OpenAI (set env var OPENAI_API_KEY) var model = OpenAiChatModel.builder() .apiKey(System.getenv("OPENAI_API_KEY")) // never hardcode keys! .modelName("gpt-4o") .responseFormat("json_object") // enforce JSON mode .temperature(0.2) // low temp = consistent output .build(); // Option B: Ollama running locally on port 11434 // var model = OllamaChatModel.builder() // .baseUrl("http://127.0.0.1:11434") // .modelName("codellama") // .build(); // ───────────────────────────────────────────────────────────────── CodeReviewService reviewer = AiServices.builder(CodeReviewService.class) .chatLanguageModel(model) .build(); // ── Sample code to review ───────────────────────────────────────── String javaCode = """ public class UserService { private static Map<String, String> cache = new HashMap<>(); public String getUser(String id) { if (cache.containsKey(id)) return cache.get(id); String sql = "SELECT * FROM users WHERE id = " + id; // ... execute query return null; } } """; CodeReviewResult result = reviewer.review(javaCode); // ── Print results ───────────────────────────────────────────────── System.out.println("Summary: " + result.summary()); System.out.println(); for (Finding f : result.findings()) { System.out.printf("[%s] %s — %s%n", f.severity(), f.category(), f.description()); System.out.printf(" Fix: %s%n%n", f.suggestedFix()); } } } Step 5: Sample Output Running the above against that deliberately flawed snippet produces output like: Plain Text Summary: The code has a critical SQL injection vulnerability, a thread-safety issue with the shared HashMap cache, and returns null without documentation. [CRITICAL] Security — String concatenation builds a raw SQL query, making the method vulnerable to SQL injection attacks. Fix: Use PreparedStatement with parameterised queries: "SELECT * FROM users WHERE id = ?" with stmt.setString(1, id) [HIGH] Concurrency — HashMap is not thread-safe. Concurrent reads/writes will cause data corruption in multi-threaded environments. Fix: Replace with ConcurrentHashMap<>() or add explicit synchronisation. [MEDIUM] Null Safety — getUser() returns null without documentation. Callers have no contract to guard against NPEs. Fix: Return Optional<String> and document the empty-case semantics. [LOW] Performance — cache.containsKey() + cache.get() performs two lookups. Fix: Use cache.computeIfAbsent(id, k -> fetchFromDb(k)) instead. Going Further: Production Hardening 1. Streaming Responses for Large Files For large classes, swap the return type of your AI Service method to TokenStream and use onNext() / onComplete() to stream findings incrementally rather than waiting for the full response. 2. Memory and Conversation History LangChain4j's MessageWindowChatMemory lets you maintain multi-turn conversations — useful if you want the developer to ask follow-up questions about a specific finding. Java CodeReviewService reviewer = AiServices.builder(CodeReviewService.class) .chatLanguageModel(model) .chatMemory(MessageWindowChatMemory.withMaxMessages(10)) .build(); 3. RAG: Ground Findings in Your Own Style Guide Use EmbeddingStore + ContentRetriever to inject excerpts from your team's coding standards into every review. The LLM will then apply your own rules rather than falling back on generic advice. 4. CI/CD Integration Wrap the service in a small CLI tool (using picocli), wire it into your GitHub Actions or Jenkins pipeline, and post findings as PR comments via the GitHub REST API. This enables a first-pass AI review before any human reviewer opens the PR. Key Takeaways LangChain4j AI Services let you express LLM interactions as plain Java interfaces — no HTTP plumbing, no JSON parsing.Structured Output (records + JSON schema) eliminates fragile string parsing; you get strongly typed results directly.Provider-agnostic design means you swap one builder line to move between OpenAI, Ollama, Anthropic, or Azure with zero business logic changes.Keep API keys in environment variables — never in source code.Start local (Ollama) to iterate fast, then promote to a hosted model for production quality. Further Reading LangChain4j official docsAI Services tutorialOllama — CodeLlama modelOpenAI Structured Outputs guide More
Modernizing a Legacy Java Application With OpenRewrite: What We Automated and What We Didn't
Modernizing a Legacy Java Application With OpenRewrite: What We Automated and What We Didn't
By Kirankumar Phirangi
Spring Boot's Hidden Secret for Configuration: EnvironmentPostProcessor
Spring Boot's Hidden Secret for Configuration: EnvironmentPostProcessor
By Mikhail Polivakha
Choosing a BBD Framework
Choosing a BBD Framework
By Himanshu Mandhyan
Controller and Data Access: Understanding Two-Layer Architecture Through Abstraction and Encapsulation
Controller and Data Access: Understanding Two-Layer Architecture Through Abstraction and Encapsulation

Not every real-world application needs full Clean Architecture with its multiple layers, dependency inversion, and strict domain boundaries. Many production systems — particularly internal enterprise applications like HRMS and Payroll platforms — run successfully on a simpler, pragmatic two-layer architecture: a Controller layer and a Data Access layer. Understanding why this simpler structure works, and how it relates to the same OOP principles behind Clean Architecture, makes it easier to work confidently in either kind of codebase. The Two Layers A two-layer architecture splits an application into exactly what the name suggests: Layer 1 — Controller Layer: handles incoming HTTP requests and delegates the actual work elsewhere. C# [ApiController] [Route("api/[controller]")] public class LeaveRequestController : ControllerBase { private readonly ILeaveRequestDataAccess _dataAccess; public LeaveRequestController(ILeaveRequestDataAccess dataAccess) { _dataAccess = dataAccess; } [HttpGet("get-emp-details")] public IActionResult GetEmployeeDetails(string empId) { var result = _dataAccess.GetEmployeeDetails(empId); return Ok(result); } } Layer 2 — Data Access Layer: handles the actual database interaction — connection strings, parameters, stored procedure or package calls. C# public interface ILeaveRequestDataAccess { EmployeeDto GetEmployeeDetails(string empId); } public class LeaveRequestDataAccess : ILeaveRequestDataAccess { public EmployeeDto GetEmployeeDetails(string empId) { // ADO.NET/Oracle-specific logic lives here: // connection string, OracleCommand, package name, parameter types return employeeData; } } The Controller never touches ADO.NET directly. The Data Access layer never touches HTTP concerns like status codes or routing. Each layer has exactly one job. Why This Is Abstraction and Encapsulation in Practice This separation isn't just organizational tidiness — it's a direct, practical application of two core OOP principles. Abstraction: The Controller calls _dataAccess.GetEmployeeDetails(empId) without knowing or caring how that data actually gets retrieved. It doesn't know whether the underlying call uses a stored procedure, a package function, or raw SQL. It only needs to know that calling this method returns the data it asked for. The internal complexity of connection handling, parameter binding, and query execution is hidden behind a simple method signature. Encapsulation: The Data Access layer protects the details of how data is fetched from being scattered across the codebase. If every controller method directly opened its own OracleConnection and wrote its own SQL, any change to the database — a renamed package, a new parameter, a switched provider — would require hunting down and editing code in dozens of places. By encapsulating that logic inside one dedicated layer, a change only needs to happen once. The Practical Test: What Happens When the Database Changes A useful way to confirm this separation is working correctly is to ask: if the underlying database technology changed — say, from Oracle to SQL Server — which layer would need to change? The answer should be: only the Data Access layer. The connection strings, parameter types, and query syntax living inside LeaveRequestDataAccess would need rewriting. The Controller, which only knows about the ILeaveRequestDataAccess interface and calls GetEmployeeDetails(empId), wouldn't need to change at all — it has no idea, and no need to know, what database sits behind that interface. If a database change ever required touching Controller code too, that would be a signal the separation isn't clean — some data-access detail has leaked into a layer that shouldn't know about it. How This Relates to Clean Architecture Clean Architecture takes this same underlying idea — separating concerns so that change in one area doesn't ripple through the whole system — and extends it further, typically into more layers: a Domain layer (core business rules, independent of any framework), an Application layer (use cases, orchestration), an Infrastructure layer (database, external services), and a Presentation layer (API or UI). The core principle is identical to the two-layer setup: outer layers depend on inner layers, never the other way around, and each layer only knows what it strictly needs to know. A two-layer Controller/Data Access split is, in effect, a simplified version of this same idea — fewer layers, less ceremony, but the same underlying discipline of not letting database-specific details leak into request-handling code, and not letting request-handling concerns leak into data-access code. For many internal enterprise applications, a full Clean Architecture setup — with separate Domain, Application, and Infrastructure projects, dependency inversion containers, and strict boundary enforcement — is more structure than the project actually needs. A well-maintained two-layer architecture, with a clear interface between Controller and Data Access, delivers much of the same practical benefit — testability, easier maintenance, isolated change — without the additional overhead. When Two Layers Stop Being Enough The two-layer pattern works well until business logic starts accumulating inside the Controller itself — validation rules, calculations, workflow decisions that have nothing to do with HTTP handling or database access. When that happens, it's usually a sign that a third layer — often called a Service or Business Logic layer — is needed to hold that logic separately, keeping the Controller thin and focused purely on request/response handling. This is often the natural next step toward something closer to Clean Architecture, added incrementally as a project's complexity genuinely justifies it, rather than adopted wholesale from day one. Why This Split Also Makes Testing Easier A practical benefit of separating Controller and Data Access that often goes unmentioned until it's needed: this structure makes unit testing significantly more straightforward. Because the Controller depends on ILeaveRequestDataAccess — an interface — rather than a concrete class tied to Oracle, it becomes possible to test the Controller's logic in isolation, without touching a real database at all. A test can supply a fake or mock implementation of the interface: C# public class FakeLeaveRequestDataAccess : ILeaveRequestDataAccess { public EmployeeDto GetEmployeeDetails(string empId) { return new EmployeeDto { EmpId = empId, Name = "Test Employee", Salary = 50000 }; } } With this fake in place, a test can verify that GetEmployeeDetails on the Controller returns the correct HTTP response, handles a missing employee correctly, or triggers the right status code — all without a live Oracle connection, without test data cluttering a real database, and without the test slowing down because it's waiting on network or database round-trips. This is only possible because the Controller was written to depend on an abstraction (ILeaveRequestDataAccess) rather than directly instantiating new LeaveRequestDataAccess() inside itself. If the Controller created its own Data Access instance internally, there would be no way to substitute a fake version during testing — the real database dependency would be unavoidable. This is the same underlying idea, applied practically: depending on an interface rather than a concrete implementation isn't just an architectural preference; it's what makes isolated, fast, reliable testing possible at all. Takeaway A two-layer Controller/Data Access architecture isn't a simplified shortcut that skips "real" architecture — it's a direct, practical application of Abstraction and Encapsulation at the system level, just as those same principles apply within a single class. Understanding it this way makes the reasoning behind Clean Architecture's more elaborate layering easier to grasp too: both are solving the same underlying problem — isolating change, hiding implementation detail, and keeping each part of the system focused on exactly one responsibility — just at different levels of complexity depending on what the project actually needs.

By Dhana PAMULA
From Database Changes to Real-Time Actions at Scale: Oracle GoldenGate and Java
From Database Changes to Real-Time Actions at Scale: Oracle GoldenGate and Java

DB changes can be streamed in real time via Oracle GoldenGate. Java applications can then consume them to validate/act on/route events without polling the DB. Goal Listen to real-time transactions in the DB, act on all operations within transactions, and maintain transaction boundaries with near real-time SLAs. Component Details 1. Source DB This is the source DB from which the Java service listens for transaction changes in real time. It can be any DB supported by the GoldenGate service (e.g., MySql/ Oracle). 2. OCI Source GoldenGate Service Deployment Input: DB Redo logsOutput: Trail file The GoldenGate service has two primary components: Extract and Replicat. Extract reads changes from the source DB in the form of redo and archive logs. It commits each transaction's DML operations until either COMMIT or ROLLBACK is identified. If it encounters a COMMIT, it stores the transaction in TRAIL files. If ROLLBACK is encountered, it discards those changes. The output of this deployment is a TRAIL file with an ordered list of all transactions that occurred in the source DB. 3. OCI Target GoldenGate Big Data Deployment Input: Trail files generated by traditional GG deploymentOutput: Ordered stream of operations within a transaction This is another deployment type that can be used for transaction management. We can use the DATA STREAMS feature instead, but this uses the AsyncAPI publish/subscribe interface, whereas big data deployment is a managed integration runtime for CDC delivery to Kafka. 4. Streaming Service Input: Trail files generated by traditional GG deploymentOutput: Ordered stream of operations within a transaction Ideally, Kafka keeps up with the scale of GoldenGate and downstream scalability. Kafka provides an option to add partitions while the partitions are receiving data. This feature is particularly useful for handling traffic bursts or batch SQL jobs on the source DB. 5. Java Service Kubernetes Deployment Input – Kafka messages, in this case individual operationsOutput – Application logic A Java application instance that can scale horizontally depending on the Kafka partition backlog. Use Case Java service is interested in listening to the entire transaction in the order that it is happening for a predefined set of tables.It does not want to listen to all the source table changesIt also does not want all columns of the changed row. It is interested in listening to only specific set of columns from given list of tables. Component Setup The detailed setup guide here can be used as general guidance, but I will focus on the specific changes needed to achieve the use case above. 1. Source DB Here we want to listen to redo logs from the source DB. To enable supplemental logging on the source DB, use the command below. SQL ALTER DATABASE ADD SUPPLEMENTAL LOG DATA In our use case, we also want to make sure we always get only the columns from the table that the Java service is interested in, and not all the columns. To enable table-specific supplemental logs, use the command below. SQL ALTER TABLE HR.EMPLOYEE ADD SUPPLEMENTAL LOG GROUP emp_group_loc (EMPLOYEE_ID, EMP_NAME, EMP_DEPARTMENT_ID); 2. OCI Source GG Deployment In this step, OCI source GG deployment is configured to listen to redo logs of the source DB. As per our use case the important step is to configure only required tables in Extract's parameter file. The list of tables basically makes sure transactions happening on only those tables will be identified by GG and will be collected in trail files. You can use this article to set up Extract. Example of parameter file for extract: SQL table HR.EMPLOYEE; Along with extract, a distribution path needs to be created within the same source deployment. This distribution path makes sure the target big data deployment can access the trail file records easily. The details to create the distribution path can be found in TASK 3 of the article. 3. OCI Target GoldenGate Big Data Deployment In this deployment, a process called 'replicat' is created; replicat reads transaction data from the distribution path created in step 2 above, then publishes those transactions. Replicat works using two configuration files. Parameter File Similar to Extract, Replicat also has a parameter file. This file contains mapping information on how operations from the source DB map to the target schema and table. Since we are not replicating transactions in the DB, a JSON with the target table mapped in the parameter file gets written to kafka topic instead. Example for a complete setup guide: use this guide. SQL MAP HR.EMPLOYEE, TARGET *.*; Properties File Three important parameters that decide how the source transactions get written to the target are 1. TopicMappingTemplate – Tells replicat about the target, where source transactions will be written, 2. opMode – How the source transaction will be written. Complete transaction or one operation at a time.3. keyMappingTemplate – This parameter is the hashing key used by the replicat Kafka producer. For all the parameters that are available in the properties file, please refer to section "11.2.10.1.3.2.2 Kafka Handler Configuration" in this guide. 4. Streaming Service and Java Consumer Service Since GG hashes based on transaction ID, it is guaranteed that all operations in a transaction will always be in the same partition. The Java service needs a consumer group used by all pods. At bootstrap, all pods register their client ID with the Kafka broker and reserve a partition. This mechanism makes sure a single pod always listens to one partition until a rebalance happens due to a pod going down or other reasons. This setup ensures the Kafka consumer in the Java service can poll and collect transactions while maintaining operation boundaries and sequence for downstream application logic within the same service. Separating polling logic from processing logic in the Java service helps avoid various timeout scenarios. We can dive deep into this mechanism in a follow-up article. Summary With this setup, we can guarantee that transactions at the source DB are accessed by a Java service in order and can run within a single transaction at a time. This workflow is useful for industries that need an eventing mechanism, such as fraud detection and financial applications, telecommunications and real-time billing, Retail and real-time inventory management, and AI and semantic search pipelines.

By Sushant Sakolkar
7 Swift Features That Will Let You Delete Code Tomorrow
7 Swift Features That Will Let You Delete Code Tomorrow

Every year Apple’s “What’s new in Swift” session drops thirty minutes of language features, and every year the recaps list them in the order they appeared on stage. That order is optimized for a keynote, not for your sprint. This one is sorted differently. The question for every feature is the same: how far is it from a diff I can open tomorrow? Some of these are one-line changes that delete a hack you’ve been apologizing for since 2021. Others are powerful, well-designed, and will not touch your app code for a year. Both deserve to be understood. Only one deserves a ticket. One thing to fix in your mental model before we start: this year’s session covers two releases, Swift 6.3 and 6.4. Some of what follows is already in your toolchain. Some is waiting for the next Xcode. I’ll flag which is which where Apple did. Each feature gets three lines: Lands in, Reach for it when, and a Verdict. Let’s go. Tier 1: Monday-Morning Changes 1. XCTest and Swift Testing Finally Speak the Same Language Lands in: Swift 6.4 If you’ve stalled on migrating to Swift Testing, it was almost certainly because of this: every test target has a pile of helpers built on XCTAssert*, and rewriting them all at once was never going to get prioritized. That blocker is gone. XCTest assertion failures, when called from a Swift Testing test, are now reported as test issues. And it works the other way too: #expect runs correctly inside an XCTestCase. Swift // Written in 2019. Still fine. func assertValidCart(_ cart: Cart) { XCTAssertFalse(cart.items.isEmpty, "cart must not be empty") XCTAssertGreaterThan(cart.total, 0) } Swift // A brand-new Swift Testing test can call it directly. @Test func checkoutBuildsCart() throws { let cart = try CartBuilder() .add(.sku("SNK-001"), quantity: 2) .build() assertValidCart(cart) // XCTest assertion, reported as an issue #expect(cart.total == 3_998) // Swift Testing expectation }// And the reverse: a legacy XCTestCase using #expect. final class LegacyCartTests: XCTestCase { func testWelcomeCouponApplies() throws { let cart = try CartBuilder() .add(.sku("SNK-001")) .apply(coupon: "WELCOME10") .build() #expect(cart.discount == 0.10) } } The practical consequence: you can write one set of assertion helpers and have them behave identically regardless of which runner calls them. Migration becomes a per-file decision instead of a per-target big bang. Reach for it when: You have more than a handful of XCTest-based helpers and want to move to Swift Testing incrementally. Verdict: Ship it. This is the single highest-ROI change in the session for any team with a real test suite. There’s a trap here, though. See the pitfalls section before you flip the switch. 2. @diagnose: Warnings You Control Per Declaration, Not Per Target Lands in: The 6.3/6.4 cycle Deprecation warnings have a lifecycle problem. The API gets deprecated. You can’t migrate yet, because the replacement is a different shape and the migration is a quarter of the work. So you either live with yellow triangles in the build log until everyone stops reading them, or you disable the warning for the whole module and lose it forever. @diagnose gives you a third option: change how a specific warning group behaves inside a specific declaration. Swift // Silence a known deprecation in exactly one place, with a paper trail. @diagnose(DeprecatedDeclaration, as: ignored, reason: "PaymentsKit v3 lands next quarter, tracked in PAY-4821") func tokenize(_ card: Card) -> Token { LegacyPayments.tokenize(card) // deprecated, and we know } Swift // Turn ON a check that's off by default, only where it matters. @diagnose(StrictMemorySafety, as: warning) func decodeFrame(_ bytes: UnsafeRawBufferPointer) -> Frame { // every unsafe access here now gets audited by the compiler }// Promote future errors to errors today, so nobody adds new instances. @diagnose(ErrorInFutureSwiftVersion, as: error) func parsePrice(_ raw: String) -> Decimal { // ... } Three modes, three very different uses: ignored for scoped suppression, warning for opting into stricter checks in security-sensitive code, error for ratcheting a migration forward one function at a time. Reach for it when: You’re mid-migration to Swift 6 language mode, or you have a security boundary you’d like the compiler to be paranoid about. Verdict: Ship it. Then add a lint rule that rejects as: ignored without a reason:. That parameter exists for a reason of its own. 3. Concurrency Stops Nagging You About the Wrong Things Lands in: Swift 6.4 This is a bundle of small fixes, and each one deletes a workaround you’ve probably written. Silently swallowed Task errors now warn. This one catches real bugs. A fire-and-forget task whose closure throws would just... drop the error. Now the compiler tells you. Swift // Before 6.4: compiles, error vanishes into the void. // 6.4: warning — you're ignoring a thrown error. Task { try await syncEngine.push(pendingChanges) } Swift // Option A: handle it inside the task. Task { do { try await syncEngine.push(pendingChanges) } catch { logger.error("push failed: \(error)") } }// Option B: keep the handle and check later. let push = Task { try await syncEngine.push(pendingChanges) } defer { await telemetry.flush() } // async calls in defer are allowed now try await push.value Note the defer in Option B. Calling async functions from a defer block was previously forbidden. Not anymore. weak let exists, and it unblocks Sendable. Any class holding a weak var delegate couldn't be Sendable without @unchecked. That was a lie you told the compiler to make it stop talking. Now you can make the weak reference an immutable binding, and Sendable checking is fine with it. Swift // Before: a lie. final class ImagePrefetcher: @unchecked Sendable { weak var delegate: PrefetchDelegate? } Swift // After: honest. final class ImagePrefetcher: Sendable { weak let delegate: PrefetchDelegate? init(delegate: PrefetchDelegate?) { self.delegate = delegate } } The reference can still go nil when the delegate deallocates. What changed is that the binding is immutable, and that's what Sendable cares about. ~Sendable says the quiet part out loud. You can now declare a type explicitly non-Sendable. Subclasses are free to opt back in. Swift class Screen: ~Sendable { } final class SplashScreen: Screen, @unchecked Sendable { } A second memberwise initializer for mixed-visibility structs. A struct with internal stored properties and a private one with a default used to get a single private memberwise init, which meant hand-writing the public one. Now the compiler generates both. Swift struct OrderDraft { var items: [LineItem] var address: Address private var validationCache: [String: Bool] = [:] } Swift // From another file in the same module — this now just works: let draft = OrderDraft(items: items, address: address) And some/any next to an optional no longer needs parentheses. Small. Satisfying. Reach for it when: You grep for @unchecked Sendable and find more than zero results. Verdict: Ship it. Run the 6.4 compiler over your codebase and treat every new “ignored error” warning as a bug report you didn’t have to write. 4. withTaskCancellationShield: Finish the Write Before You Bail Lands in: The 6.3/6.4 cycle (standard library) Checking Task.isCancelled before expensive work is the right default. Except for the moment when you're halfway through writing a file, committing a transaction, or flushing an analytics journal. Bailing there doesn't save work. It corrupts state. Inside the shield, every cancellation check reads false, all the way down the call stack. Swift extension AnalyticsJournal { func flush(_ batch: [Event]) { // Everything before this point may honor cancellation. withTaskCancellationShield { // Inside: Task.isCancelled == false, so lower layers // won't abort mid-write. storage.append(batch) storage.commit() } } } Apple’s guidance is worth repeating: keep the shielded region short. This is for “finish or roll back what I already started,” not for “ignore cancellation because it’s inconvenient.” Reach for it when: You have a write path that’s already had a bug caused by cancellation landing between two dependent operations. Verdict: Ship it, surgically. One shield per write path, none anywhere else. 5. ProgressManager: Progress for the Async/Await Era Lands in: The 6.3/6.4 cycle (Foundation) Foundation’s old Progress was designed around KVO and implicit parent/child registration via thread-local state, which is exactly the kind of thing that stops making sense once your code is structured concurrency. ProgressManager separates composition (who owns which share of the work) from reporting (who's watching), and it does it with types instead of magic. Swift let manager = ProgressManager(totalCount: 100) Swift // Hand out a slice of the total to a child operation. try await exporter.exportOrders(manager.subprogress(assigningCount: 100))extension OrderExporter { func exportOrders(_ progress: consuming Subprogress? = nil) async throws { let steps = progress?.start(totalCount: 3) try await fetch(); steps?.complete(count: 1) try await encode(); steps?.complete(count: 1) try await upload(); steps?.complete(count: 1) } }// Observe from the UI side. Task { for await fraction in Observations({ manager.fractionCompleted }) { exportButton.title = "Exporting \(Int(fraction * 100))%" } } Two details to notice. The child function takes consuming Subprogress?, so the caller can pass nil when nobody's watching and the function doesn't care. And the manager supports typed metadata, so you can attach domain-specific values (bytes transferred, items processed) and summarize them across the tree without stringly-typed user-info dictionaries. Reach for it when: Any multi-stage async flow that shows a progress bar. Uploads, exports, media processing, initial sync. Verdict: Ship it for new flows. Don’t rewrite working Progress code just to use it. 6. anyAppleOS: One Availability Line Instead of Five Lands in: The 6.3/6.4 cycle Apple aligned OS version numbers last year (everything is 26, then 27). This year Swift cashes that in: a single platform name covers all of them. Swift // Before @available(macOS 27, iOS 27, watchOS 27, tvOS 27, visionOS 27, *) func showLiveActivityGlance() { } Swift // After @available(anyAppleOS 27, *) func showLiveActivityGlance() { }// Set the default, then carve out exceptions. @available(anyAppleOS 27, *) @available(tvOS, unavailable) func startCheckoutHandoff() { }// Works in conditional compilation too. #if os(anyAppleOS) import WidgetKit #endif Reach for it when: You ship a framework or app extension across three or more Apple platforms. Verdict: Ship it if your minimum versions line up across platforms. If they don’t, this does nothing for you. 7. Module Selectors (::): Name Collisions, Resolved for Real Lands in: Swift 6.3 Two modules export a type with the same name. You disambiguate with ModuleName.TypeName. This works until the module also contains a type with the same name as the module, at which point Swift prefers the type, looks inside it, finds nothing, and errors. The new :: syntax makes the left side always a module name. Swift import SwiftUI import DesignSystem // also exports a `View` type Swift struct ProductCard: SwiftUI::View { var body: some SwiftUI::View { DesignSystem::Card { /* ... */ } } } It works on members too, which matters when two modules add same-named methods to one type via extensions: Swift // Module Fulfillment adds `cancel()` to Order. // Module Payments also adds `cancel()` to Order. order.Payments::cancel() // refund, don't un-ship Apple’s own note on this: it’s for collisions between modules you don’t control, and for macro expansions where you can’t predict what else got imported. Do not design APIs that rely on it. Reach for it when: The View-vs-View problem, or you write macros. Verdict: Ship it as a fix, never as a design. Tier 2: The Testing Toolbox Got Sharper Lands in: Swift 6.4 Beyond the XCTest bridge, Swift Testing picked up three things that address real CI pain: Swift @Test(arguments: Storefront.all) func priceFormatting(storefront: Storefront) throws { // Skip *this argument*, not the whole test. if storefront.currency == nil { try Test.cancel("no currency configured for \(storefront.id)") } Swift let text = PriceFormatter(storefront: storefront).string(from: 1_999) // Worth a look, not worth blocking the merge queue. if text.count > 12 { Issue.record("\(storefront.id) renders unusually long", severity: .warning) } #expect(text.contains("19")) } Test.cancel is the interesting one in parameterized tests: you cancel individual arguments instead of either running them to a meaningless completion or failing the whole test. Issue.record(severity: .warning) surfaces things you want a human to notice without turning the pipeline red. And swift test gained a repeat mode: rerun until pass or until fail, with a cap on repetitions. In "until pass" mode only the failing tests rerun. Flaky-test hunting, built in. Verdict: Ship it. Test.cancel alone will clean up a lot of guard ... else { return } at the top of parameterized tests. Tier 3: Performance, With Receipts Everything here is real, well-designed, and will not show up in most app code this year. It will show up in the frameworks your app depends on. Know it exists; reach for it when the profiler tells you to. 8. borrow/mutate Accessors: Computed Properties Without the Copy Lands in: Swift 6.4 A computed property with get/set moves the value by copy. For an Int, who cares. For an InlineArray of 256 Ints, that's a two-kilobyte struct copied out and back just to touch one element. borrow gives read access to the storage in place. mutate gives exclusive write access in place. Swift struct FrameBuffer: ~Copyable { private let storage: UnsafeMutablePointer<[1024 of UInt8]> Swift var pixels: [1024 of UInt8] { borrow { storage.pointee } mutate { &storage.pointee } } }buffer.pixels[512] = 0xFF // in place. No 1 KB round trip. As a bonus, the property can now hold non-copyable values, which get/set fundamentally couldn't. Verdict: Reach for it in wrappers around large value types. This is the most practical thing in the ownership block. 9. Continuation: Resume-Once, Checked at Compile Time Lands in: The 6.3/6.4 cycle (standard library) Every codebase bridging callbacks to async has the same two options: CheckedContinuation (safe, runtime cost) or UnsafeContinuation (fast, and one double-resume away from a crash in production). The new Continuation type verifies at compile time that you resume it exactly once. Safer than checked, as cheap as unsafe. The session names the type and the guarantee without walking through the call site, so check the standard library docs for the exact spelling before you write it in a code review. Verdict: Adopt it in new bridging code. Sweep the old UnsafeContinuation sites when you have a quiet week. 10. MutableRef: Hoist the Dictionary Lookup Out of the Loop Lands in: The 6.3/6.4 cycle (standard library) You have a loop that increments counts[key] on every iteration. The lookup is the same every time. Until now, holding a dictionary slot "open" across a loop required moving the loop into a nested function and passing the slot as inout. Obscure, and everyone who's found it has felt slightly dirty. Swift func countMentions(of tag: Tag, in posts: [Post], into counts: inout [Tag: Int]) { var slot = MutableRef(&counts[tag, default: 0]) // one lookup Swift for post in posts where post.tags.contains(tag) { slot.value += 1 // no re-hash } } Ref (read) and MutableRef (write) are non-escapable, so the compiler knows the access ends when the variable leaves scope. They can be stored, passed, returned, and used in generics, which means APIs that hand out a live reference to one of their properties are now expressible. Verdict: Reach for it in hot loops the profiler has already pointed at. 11. Iterable: for Loops That Borrow Instead of Copy Lands in: Swift 6.4 Sequence copies each element out to the loop variable. Iterable lets the loop borrow them, in batches, via Span. No retain/release for reference types or copy-on-write containers, non-copyable elements are supported, and the loop can throw. The constraints matter: you can’t mutate the collection while iterating (exclusivity is enforced), and for prefers Sequence when a type conforms to both. So adopting Iterable on a type that already has Sequence changes nothing. Verdict: Know it exists. Author it in collection-heavy library code. Don’t expect free speedups elsewhere. 12. @inline(always) and @specialized: Telling the Optimizer What You Know Lands in: @inline(always) in 6.4, @specialized in 6.3 @inline(never) has existed forever. It now has a twin that forces inlining even when the optimizer's heuristics say no. For class methods, pair it with final or dynamic dispatch will still get in the way. @specialized asks the compiler to emit a concrete version of a generic function for a type you know is hot, which matters most in libraries where the compiler can't see the call sites. Swift @inline(always) func clamp01(_ x: Float) -> Float { min(max(x, 0), 1) } Swift @specialized(where Bytes == [UInt8]) func checksum<Bytes: Sequence<UInt8>>(_ bytes: Bytes) -> UInt32 { // ... } Verdict: Measure first. Both can make a binary bigger and slower when applied on a hunch. The Traps Nobody Puts in the Recap The XCTest bridge reports failures as warnings by default. An XCTAssert that fails inside a Swift Testing test does not fail the test until you opt in via the Xcode build settings. Flip that switch on day one, or you'll have "green" runs that are quietly losing signal.@diagnose(..., as: ignored) is debt with a nicer syntax. Require reason: in code review. Grep for it quarterly.@inline(always) and @specialized are not free. Apple describes them as tools you "usually won't need." Believe that. Profile, apply, re-profile.Iterable won't replace Sequence automatically. If your type has both, for picks Sequence. No silent speedup.weak let doesn't make the reference permanent. It can still go nil. What you gain is that the immutable binding no longer blocks Sendable.anyAppleOS requires aligned minimums. Different minimum versions per platform means you're back to the long form. What I’d Do This Week Open a branch. Compile with 6.4. Fix every new “ignored thrown error” warning. That’s your free bug sweep.Replace @unchecked Sendable on delegate-holding classes with weak let.Turn on XCTest-failure promotion in Swift Testing, then migrate one test file.Put a cancellation shield around your one scariest write path.Bookmark borrow/mutate and MutableRef for the next time Instruments shows you a copy you didn't ask for. Everything else in this session (Swift-to-C via @C, the official Android SDK, Wasm at 35–40x faster bridging, embedded Swift growing its subset) is genuinely exciting and belongs in a different article, one about where Swift is going rather than what lands in your app.

By Pavel Andreev
Detecting Duplicates With Machine Learning in 2026: From String Metrics to Embeddings and LLMs
Detecting Duplicates With Machine Learning in 2026: From String Metrics to Embeddings and LLMs

The more data companies gather on their customers, products, and code, the more duplicated information accumulates, confusing the teams who depend on it. You don't go through millions of records manually. So deduplication is now a machine learning problem. This article looks at the current state of duplicate detection in ML systems where the old techniques still work, and what embeddings and large language models have brought to the table. How Do ML Systems Detect Duplicates? It is possible for a person to look at two pictures or two strings and know if they are the same. Teaching a machine to do the same raises difficult questions. What is 'like'? There are degrees of resemblance. Is “Jon Smith, 12 Oak St.” the same customer as “Jonathan Smith, 12 Oak Street?” In practice, modern deduplication pipelines work like a funnel that begins with cheap, fast methods and introduces intelligence (and cost) only where necessary: Normalize block. Clean the data (case, punctuation, abbreviations, etc.) and group records into blocks where they share something in common (e.g., postal code or the first letters of a name). Only records within the same block are compared.Score candidate pairs using string metrics. Fast deterministic similarity measures catch typos, transpositions, and formatting differences.Embeddings for meaning. Dense vectors and approximate nearest neighbor search can discover pairs that are worded differently but mean the same.Push borderline cases up. If the earlier stages cannot be confidently settled, an LLM or a human reviewer resolves the pairs. Each stage decreases the number of pairs that the next, more expensive stage has to consider. This makes the approach affordable at scale. String Metrics Still Matter String metrics are the first line of defense. They are fast, interpretable, and do not need a model. Some common options are: Hamming distance: The number of positions at which two strings of equal length differ, or the number of substitutions needed to change one string into the other.Levenshtein distance: Generalizes Hamming to allow insertions and deletions as well as substitutions, so it works on strings of different lengths.Jaro-Winkler similarity: Measures the number of matched characters and transpositions, with more weight for a common prefix. It works particularly well with short strings, such as personal names.Sørensen–Dice coefficient: A measure of similarity based on the number of common bigrams (adjacent letter pairs).Jaccard similarity and TF-IDF cosine similarity: Comparing sets of tokens or weighted vectors of words; the latter is more suitable for longer text.MinHash with locality-sensitive hashing (LSH): a cheap way to estimate set similarity, so you can find near-duplicate documents without comparing them all.Learnable distance: understands that different domains have different weights for edit operations and learns those weights from labeled data. In a modern pipeline, these scores are typically features that go into a classifier, not the final answer. For example, a model can learn that a mismatch in a phone number is much more important than a mismatch in a middle initial. Vector Search and Embeddings String metrics break down when two texts say the same thing in different words. Take two questions from a Q&A site. How can I get cheap hotel rooms?What are the best ways to get cheap hotel deals? They have very few words in common, and thus have a rather high edit distance rating. Sentence-embedding models, by contrast, map each text into a vector space where similar texts are near each other. Then finding duplicates becomes a nearest-neighbor search: Python from sentence_transformers import SentenceTransformer import faiss questions = [ "How do I book cheap hotel rooms?", "What are the best ways to find cheap hotel deals?", "How do I renew my passport?", ] model = SentenceTransformer("all-MiniLM-L6-v2") vectors = model.encode(questions, normalize_embeddings=True) # Inner product on normalized vectors equals cosine similarity index = faiss.IndexFlatIP(vectors.shape[1]) index.add(vectors) scores, neighbors = index.search(vectors, k=2) for i, (row_scores, row_ids) in enumerate(zip(scores, neighbors)): for score, j in zip(row_scores, row_ids): if j != i and score > 0.7: # tune this threshold on labeled data print(f"Possible duplicate: {questions[i]!r} ~ {questions[j]!r} ({score:.2f})") The flat index above is fine for thousands of records. For millions, switch to an approximate index such as HNSW or IVF, which trades a little recall for large speedups. Before reaching for approximate search, check whether simple blocking already shrinks your comparison space enough. The similarity threshold is not universal. Tune it on a labeled sample of your own data, because the right cutoff for support tickets will differ from the right cutoff for customer names. LLMs for the Tough Cases Embeddings provide you with candidates, not certainty. Two product listings might differ only in a model number that matters, but otherwise be nearly the same vector. That’s where big language models are earning their place now, as a pair resolver that looks at two candidate records and decides whether they refer to the same real-world thing. Open-source tooling has started to package this pattern. One example is cocoindex, which uses FAISS embedding similarity to find near-duplicate entity names, and then queries an LLM-based resolver to confirm matches and choose the better canonical name. Academic work is moving in the same direction. A SIGMOD 2026 paper on in-context, clustering-based entity resolution with LLMs includes ways to reduce hallucinations and API calls. There are two caveats: Cost and delay. Send LLM calls only for the pairs the cheaper stages couldn't resolve, as LLM calls are much more expensive than string comparison. That’s why the funnel is there.Matches that have been hallucinated. An LLM can confidently speak of a wrong merge. Keep confidence scores, log reasoning, and have a human in the loop for high-stakes merges. Real World Examples Questions on Q&A Sites: Duplicates Sites such as Quora and Reddit are constantly bombarded with the same questions over and over again because users don't know someone else has already asked them. The most famous benchmark here is the Quora Question Pairs dataset, which came out in 2017 with about 404,000 labeled training pairs [check exact counts; the original paper reported 404,290 training pairs and 2,345,795 test pairs]. The benchmark is hard because duplicates are meaning-based, not word-based; capitalization, abbreviations, and context all matter. The above hotel example is a good first test for any approach. Embedding models do a much better job on this task than edit distance. Marketplace Duplicate Listings On classified-ad and marketplace sites, sellers will often re-post or slightly edit a listing if it's not performing. The same funnel applies: block by category and location, score titles and descriptions with string metrics, use embeddings to catch rewording, and flag the clusters for removal or consolidation. Another signal could be added for image similarity when listings have photos. Copy Code The biggest example of how much duplication builds up at scale is GitHub. Its Octoverse 2025 report says the platform now has 630 million total repositories, with 121 million added in 2025 alone, and 395 million of them are public or open source. It has more than 180 million developers. At that size, duplicated code is a must. Code duplication can be exact copies or clones that behave the same, but are written differently. Research on the topic (including a widely cited 2017 study of code duplication on GitHub, DéjàVu) found that a large share of JavaScript files were copies [verify the study’s details and the 93% figure before citing it, and make clear that this was academic research rather than something GitHub itself runs]. The newer story is AI-assisted coding. GitClear research of 211 million changed lines of code in 2025 found an eightfold increase in 2024 in code blocks that duplicate adjacent code with five or more lines. It also found for the first time that copy-pasted lines outnumbered moved (refactored) lines. GitClear sells code-review analytics, and its data shows correlation with AI adoption, not proof of cause, so treat it as vendor research. But it hints that code deduplication will grow in importance as AI tools make it easier to generate new code than to consolidate it. Duplicate Records in CRM Software Traditional CRM de-duplication relies on matching rules, and rules can’t cover all fuzzy cases. A learning-based approach, however, identifies fuzzy duplicates by combining multiple string metrics and learning from human feedback: the system shows you a pair of records, you mark them as duplicate or distinct, and it adjusts its matching accordingly. You can learn more about this in my previous article. The second big advantage is scalability. Comparing every record with every other record is quadratic. Now suppose you have 50,000 records and add 5,000 new records. Which means 50,000 x 5,000 = 250,000,000 comparisons to compare each new record with each existing record. That is almost 7 hours, 10,000 comparisons per second: 250 000 000 / 10000 / 3600 ≈ 6.94 hours If you compare every record with every other record in the combined 55,000 records, you get roughly 1.5 billion pairs or about 42 hours at the same speed. Blocking does this by grouping records with common characteristics and only comparing within each group, which reduces the amount of work to be done dramatically. Vector search does something similar, returning only the nearest neighbors of each record. AI Pipelines Duplicates Deduplication has also become a core step in the artificial intelligence systems themselves: Training data: Removing near-duplicate documents, often using MinHash and LSH, so models don't over-weight repeated text.RAG corpora: Collapsing repeated passages so retrieval doesn’t fill the context window with copies of the same content.Knowledge graphs: Marrying differently named entities like “Mr. Edwards,” “Brad Edwards,” and “Detective Edwards,” so relationships don’t break across duplicate nodes. Some tools now leverage a combination of deterministic rules, semantic similarity, LLM reasoning, and human review to do this.Pitfalls to plan for: False merges are worse than missed duplicates. Merging two different customers can corrupt the records and break downstream processes. Choose the more costly error and set the limits accordingly Just measure it. Evaluate on a labeled sample of pairs with precision, recall, and F1. Re-check as your data changes. Keep Humans in the Loop Low-confidence pairs are passed to reviewers, and their decisions are fed back into the system. A few considerations remain important: Data privacy: Records about people often contain sensitive information. Confirm what you send to hosted embedding or LLM services, and apply your organization's data-handling policies.Provenance: Keep original records and a merge log so that a bad merge can be undone. Machine learning will play an increasingly important role in deduplication, but it doesn't eliminate the need for human judgment. Any CRM, database, or codebase has a real price for duplicate data, and hand-written filters cannot handle the variety of ways duplicates show up. What the last few years have taught us in practice is not that one technique has replaced another. It’s that the best systems layer them: string metrics and blocking for speed, embeddings for meaning, and LLMs or people for the hardest calls. Start with the cheapest stage that works on your data, evaluate it, and only add the next stage if the results justify the cost.

By Ilya Dudkin DZone Core CORE
Persisting Cookies Across JAX-WS Service Calls Using a Custom SOAP Handler
Persisting Cookies Across JAX-WS Service Calls Using a Custom SOAP Handler

The Cookie That Never Came Back The first call authenticated the user. The second failed. I was integrating with a customer-provided SOAP service exposed through a single WSDL, with JAX-WS client classes generated by IBM Rational Application Developer from the WSDL and its XSDs. The first call authenticated the user and came back with a Set-Cookie header. Every call after it needed that cookie. The generated client sent none of it. Each invocation behaved as an independent conversation, and the workflow broke on its second step. No exception, no warning. Just a server that had forgotten who you are. The Built-in Answer, and Why It Failed The JAX-WS specification anticipated this: every proxy generated from a WSDL implements BindingProvider, and the spec defines a request-context property for exactly this purpose: Java CheckPort port = new Check().getCheckPort(); ((BindingProvider) port).getRequestContext() .put(BindingProvider.SESSION_MAINTAIN_PROPERTY, Boolean.TRUE); Set that flag, and the runtime is supposed to maintain session state. In most tutorials, this is where the article ends. Mine broke in deployment for three reasons. First is scope. The property is set on one proxy instance, and the runtime's cookie store lives with that proxy. RAD-generated code encourages pulling a fresh port per unit of work. Every fresh port gets a fresh store, so the authentication session is discarded when the method returns. Holding one proxy for the workflow fixes it, but pins the session lifetime to the object lifetime. Second is opacity. The spec does not say where the runtime keeps the cookie. The JDK's bundled implementation delegates to java.net.CookieHandler.getDefault(), null unless somebody installed one. IBM's runtime, which RAD-generated clients typically run on under WebSphere, does it its own way. Same client class, different behavior per container, and the failure is silent. Third is the obvious fix: install a default CookieManager inline. `CookieHandler.setDefault(new CookieManager(null, CookiePolicy.ACCEPT_ALL));` A JVM-wide static. Fine in a standalone client, but inside an application server it is shared by every deployed application and every outbound HTTP client in the JVM. It is also the wrong model for a server-side client: a CookieManager is built for a browser, where one cookie jar belongs to one user. A server handling concurrent requests would pool every user's session cookies into one store, and CookieManager applies RFC domain and path matching, so if the service sits behind a load balancer, the cookie is quietly dropped. The Options on the Table I wanted cookie handling scoped to the call, visible in the code, and independent of the container. The alternatives fell short. Holding one proxy for the whole workflow is the simplest change, but it marries the session to an object's lifetime, and the strain shows across components or threads. Installing a default CookieManager is one line and works in a standalone client, but inside an app server it becomes shared mutable state across applications and users, which rules it out for anything concurrent. Hand-rolling the client on plain HTTP gives full header control with no container surprises, but you lose the WSDL tooling: generated types, binding, error handling. Too much machinery to rebuild for several operations. A custom SOAP handler was the fourth option: roughly sixty lines you own, sitting in the message path of the existing generated client. Nothing JVM-wide, nothing container-specific, nothing hand-edited in generated classes. The costs: you match the cookie by name instead of trusting RFC rules, and you own the stored value's lifecycle. Acceptable for this integration. How the Handler Works The contract is SOAPHandler<SOAPMessageContext>; the direction comes from a boolean on the context. The listing below is illustrative, adapted from the class I used, and not presented as tested code. Java public boolean handleMessage(SOAPMessageContext ctx) { if (Boolean.TRUE.equals(ctx.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY))) { attachCookie(ctx); } else { captureCookie(ctx); } return true; } @SuppressWarnings("unchecked") private void captureCookie(SOAPMessageContext ctx) { Map<String, List<String>> headers = (Map<String, List<String>>) ctx.get(MessageContext.HTTP_RESPONSE_HEADERS); if (headers == null) { return; } for (Map.Entry<String, List<String>> entry : headers.entrySet()) { if (!"Set-Cookie".equalsIgnoreCase(entry.getKey())) { continue; } for (String raw : entry.getValue()) { String pair = raw.split(";", 2)[0].trim(); if (pair.startsWith("JSESSIONID=")) { SESSION_COOKIE.set(pair); } } } } @SuppressWarnings("unchecked") private void attachCookie(SOAPMessageContext ctx) { String cookie = SESSION_COOKIE.get(); if (cookie == null) { return; } Map<String, List<String>> existing = (Map<String, List<String>>) ctx.get(MessageContext.HTTP_REQUEST_HEADERS); Map<String, List<String>> headers = (existing == null) ? new HashMap<>() : new HashMap<>(existing); headers.put("Cookie", Collections.singletonList(cookie)); ctx.put(MessageContext.HTTP_REQUEST_HEADERS, headers); } public boolean handleFault(SOAPMessageContext ctx) { return true; } public void close(MessageContext ctx) { } public Set<QName> getHeaders() { return Collections.emptySet(); } } Three details in that listing exist because earlier versions broke. Copying the existing request headers before adding Cookie keeps the handler additive: putting a fresh map into HTTP_REQUEST_HEADERS replaces whatever was already there, and my first version did exactly that, until another handler down the chain set a custom authorization header that then vanished. Matching the header name case-insensitively is for portability, since a case-insensitive map is an implementation choice, not a spec guarantee. Splitting on the first semicolon keeps response attributes such as Path, Expires, and HttpOnly out of the request; echoing them back produces a Cookie value the server will not parse. Iterating over every value rather than taking index zero covers responses that set more than one cookie, where the session cookie is not guaranteed to be first. Why Threadlocal, and Where It Bites The integration ran several workflows concurrently, so a static field would leak one user's session into another's request. A ThreadLocal gives each thread its own value, which fits a workflow that runs start to finish on one thread. The cost is that thread pools reuse threads. If a workflow ends without clearing the value, the next request on that thread inherits a stale session cookie, and in the worst case sends one user's session identifier on another user's behalf. Clearing it when the workflow ends closes that hole: SESSION_COOKIE.remove(); Note that close() is intentionally empty: the container calls it at the end of each message exchange, not the workflow, so clearing there would discard the cookie from the authentication call. Cleanup belongs at the logout step or in a finally block around the workflow. And if the workflow hands off between threads, ThreadLocal is the wrong container: the cookie then belongs in whatever object already represents the conversation. Registering the handler JAX-WS lets handlers be declared in an external XML file: XML <?xml version="1.0" encoding="UTF-8"?> <handler-chains xmlns="http://java.sun.com/xml/ns/javaee"> <handler-chain> <handler> <handler-class>com.company.cc.handler.SessionCookieHandler</handler-class> </handler> </handler-chain> </handler-chains> The file sits in the same package as the generated service class, since the annotation resolves relative to that class: Java @WebServiceClient( name = "check", targetNamespace = "http://ws.example.com/", wsdlLocation = "META-INF/wsdl/spellingCheck.wsdl" ) @HandlerChain(file = "sessionCookieHandler.xml") public class Check extends Service { } Every proxy from that service now runs through the handler. One caveat: regenerating from an updated WSDL overwrites the service class and removes the annotation, so keep it in a small subclass or script the regeneration step. Edge Cases Cookie rotation needs no special handling: capture runs on every inbound response, so a new cookie mid-workflow replaces the old value on the next exchange. Expiry is different: a ThreadLocal cannot know when the server-side session times out, so an idle workflow keeps attaching a dead cookie. That fix belongs at the workflow level: catch the session-expired response and re-authenticate. Testing It without a Container A test can build a stub message context, inject HTTP_RESPONSE_HEADERS, run handleMessage, and check the Cookie header on the next outbound call. Cover the cases that burned earlier versions: two Set-Cookie values in one response, a header name in unexpected case, a pre-existing request map that must survive the copy, and the remove() at the end of the workflow. For integration confidence, run the real client against a stub endpoint that returns Set-Cookie on the first call and asserts the Cookie header on the second. The Takeaway JAX-WS has a session mechanism. It is scoped to a proxy instance, implemented differently across runtimes, and backed by a JVM-wide singleton that does not fit a server-side client handling concurrent users. A custom SOAP handler replaces it with a small class you own, an explicit scope you choose, and behavior identical wherever the application deploys. For teams maintaining legacy SOAP integrations that will not be modernized soon, that trade is usually the right one.

By Vishal Rameshchandra Shah
Building Enterprise Authentication with Angular, Spring Boot, SSO, and MFA
Building Enterprise Authentication with Angular, Spring Boot, SSO, and MFA

Authentication Isn’t a Feature, It’s Frontend Architecture Whereas the authentication process can typically be implemented as an afterthought for any development team through just creating a login interface that will be connected to some sort of identity management solution, it will not be as straightforward when there is a need to develop a fully functional enterprise solution with the use of Angular and Spring Boot. From implementing single sign-on and multi-factor authentication in such cases with the use of such a front-end/back-end combination, it became evident that authentication is more of a constraint than a feature. Why SSO Changes More Than the Login Screen However, the idea of single sign-on appears to be fairly straightforward – you visit the identity provider and get your token, then that’s all. But that implies that the application will have to start working differently from within. From now on, the routes should have a guard to check if the user is authorized before any component gets initialized. Deep links have to be maintained during the redirect and return from the identity provider, which essentially means that the app needs to maintain the intended destination state in a process it does not control. Plus, since the token comes from an external provider on their own schedule, it needs to be treated by the frontend as an ordinary and expected situation where a token expires. In our platform, we implemented route guards checking token validity before resolving a protected route and token refreshing in the background mode without any interruptions to the user’s workflow. This part was more important for the whole implementation than the single sign-on flow itself, since a technically correct single sign-on flow that constantly interrupts users is still considered broken. The refresh mechanism brought its own nuances. In case multiple requests failed due to token expiration, an oblivious interceptor would spawn multiple refresh actions, while the majority of identity providers will revoke all previous refresh tokens upon the generation of a new one. It was resolved by having a request queue, where the first error will cause a single refresh action to be fired, while the rest of the pending requests would wait for this result and get re-fired with a fresh token. Should the refresh action fail, the queue would be cleared, and the user would go through the entire authentication process with his destination saved. Where Multi-Factor Authentication Adds Real Complexity One of the features MFA brings to the table is an additional level of state that the frontend has to handle properly: users might be authenticated but not yet fully verified. Confusing these two states is what makes most MFA setups break: they allow access to routes only available to fully authenticated users or require users to go through a verification step that they have already done in the current session. We considered the authentication state to be a state machine rather than a mere boolean value due to the existence of different states like the state of unauthenticated, the state of authenticated and waiting for verification, and the state of authenticated. Different permissions were applied in each of those states, and different transitions were possible between the different states. It allowed us to easily predict the results of the authentication code analysis and especially testing since we could test each transition between the different states. Step-up verification introduced yet another complication. Certain activities, like processing an important transaction, needed more recent verification than merely logging in. We simulated this by modeling it as a level of verification tied to a period of time within the session, thus allowing the back-end to request re-verification during certain important tasks while retaining the user’s work at the front-end. The Spring Boot Side of the Contract All of that required the backend to implement the exact same model that the frontend expected. The validation of tokens was done by our Spring Boot services on every single request instead of relying only on the frontend, and we were sending different and clearly recognizable error messages for token expiry, invalid tokens, and insufficient verification level instead of simply sending 401 all the time. And these differences are extremely important on the frontend – token expiry causes silent refreshing of tokens, invalid tokens cause the full authentication process, and insufficient verification level causes the user to be forced to pass through the MFA challenge once again, and not start the whole authentication from scratch. It took us way more coordination and planning to get this contract right than either side expected initially. The backend team could have considered all types of authentication errors to be the same thing and the frontend team could have thought that the error codes would translate perfectly into frontend states without any further checks. We recognized this problem at the very early stages mainly thanks to our SDET testing this boundary explicitly. What This Means for Compliance-Heavy Environments As the banking system is a highly regulated one, authentication was not only about the quality of the user experience; it was also an audit problem. With every state transition done, we needed to be able to explain and justify our choice of actions rather than just copying what worked before. This way we had to move toward using the explicit state machine rather than implicit, since the latter was difficult to prove. The Takeaway It is the choices about the design of the authentication procedure, the model of the state, semantic error management in the frontend-backend communication, token refreshing, and many more that will define the level of opposition that our team needs to overcome in order to make the app survive. The fact that our approach to the design of the authentication procedure as part of the frontend architecture from the very beginning was the reason for our success.

By Vaishnavi Khosla
Best of Three Worlds: AI Agents Writing In Rust and Java Beat the C Library
Best of Three Worlds: AI Agents Writing In Rust and Java Beat the C Library

Every app that accepts photo uploads meets HEIC sooner or later. iPhones save in it by default. Neither Java nor Rust can read it out of the box, and the usual answer, libheif, is a C library: native packages in every container image, a C toolchain in the build, and a parser written in C reading files that strangers upload. I had the problem twice, in a Java product and in a Rust one. So I went the other way: decoders written in the language itself, built and improved by AI coding agents, with me steering. Here is how it went. 1. Make It Work, In Each Project One agent per codebase, each starting from an existing open-source decoder in its own language: Openize.HEIC for Java, HEIC decoder for Rust. The first versions lived inside each app. They read the photos, and that was enough to ship. They were ready in no time, given there were upstream libraries to start from, even if some features had to be added to match libheif. It was tempting to stop there. Nobody complains about an upload that works. 2. Extract and Iterate Next, I had each decoder pulled out as a standalone library, open source, with its own tests and releases, aligned with the best practices for each language (and mine). Then I asked each agent to iterate on its own: measure, improve, measure again. That alone went a long way. The Java decoder went from 19 seconds to 3.2 for our ten test photos. 3. Compare Everything With Everything Working alone, each agent eventually ran out of obvious ideas. What changed the game was putting both decoders next to libheif on the same machine with the same photos, and then comparing them stage by stage. Every gap became a lead. If another implementation did a step fast, there was a way to do it even faster, and the size of the gap told me whether it was worth chasing. A gap is a hint with a number attached. The comparison also kept us honest. It showed where we were ahead, and it showed that part of libheif's lead came from doing less work: it copied color samples where our decoder interpolated them. 4. Make the Agents Talk Here is the part that led to further improvements. Each agent, working in its own session, was told to talk with the other agent. I had the Rust agent brief the Java agent on what it had found, and the other way around. A gap one side had already closed became a recipe for the other. The Java decoder took the Rust decoder's entropy decoding approach; the Rust decoder took the Java decoder's parallel tiles and transform tricks. Later, the two agents, who were then aware of each other, even coordinated on their own, communicating which CPU cores each could benchmark on. Connecting them was the most useful step in the project. 5. Repeat On the Gaps That Remain Then we ran the same loop again: compare, find the largest remaining gap, close it, then measure. One more round on the Rust side closed the last single-core gap with libheif. The Java side tried the JVM's vector instructions and gained another 9 to 14%. Not every idea survived. The trick the C library is famous for, wide vector instructions, gained almost nothing in Rust: its slow parts were elsewhere. We measured it, wrote it down, and dropped it. Be Strict About Evidence A computer is never idle. Browsers, chat apps, other agents: everything competes for the CPU, and the numbers move with it. When we measured the very same code twice, back to back, the second run came out about 0.4% slower. Noise with a direction looks exactly like progress, or like a regression. So I asked for proof, not impressions: count CPU work instead of wall-clock time, run old and new in alternating order, split results by type of photo, and drop any "gain" smaller than the noise. Several changes didn't make it. One that looked good overall turned out to slow down rotated photos, which are most of what phones upload, until the agent found why. Where They Stand Today Measured together on the same machine, the same ten photos, all three doing the same work: 8 cores: Rust 0.46 s, Java 0.55 s, libheif 0.81 s1 core: Rust 1.38 s, libheif 1.45 s, Java 1.96 sRust can also interpolate color for sharper edges and still matches libheif: 1.45 s Memory is on par with libheif or below, and a long way from where the two decoders started. What I Take Away Iterating alone gets you far. Comparing gets you further. The gap between two implementations is the best to-do list.Cross-pollinating agents helps a lot. The ideas were there, in two separate sessions. Making them talk to each other triggered more optimizations.Be strict about evidence. Ask for measurements that hold up against noise, and be ready to throw away a change that doesn't.Two implementations in two languages beat one. Each made the other faster. Getting a third as reference (libheif) helped even more.Java is fast, too. “Java is slow” is a meme, not a measurement. Optimized the right way, it comes close to the C library on one core and beats it on eight. Both decoders are open source: Rust: github.com/Valuya/heic-decoder-rustJava: github.com/Valuya/openize-heic-java

By Yannick Majoros
Microsoft Foundry Agent Debugging Dashboard with Microsoft Fabric Eventhouse, KQL, Docker and Grafana
Microsoft Foundry Agent Debugging Dashboard with Microsoft Fabric Eventhouse, KQL, Docker and Grafana

An agent built in Microsoft Foundry makes many decisions in a single run. It calls a model, selects a tool, reads the tool result, and calls the model again. When the answer is slow or wrong, the cause sits somewhere inside that sequence, and the final response gives no hint about which step failed. Foundry records this sequence as OpenTelemetry traces. By default, a hosted agent sends them to Application Insights, which is a good place to start. Two gaps remain for a team that builds agents every day. The first gap appears during development. A developer who changes a prompt or a tool wants to see the spans of that one run within seconds, on a laptop, without a deployed project. The second gap appears after deployment. Agent owners want a shared dashboard with run counts, latency percentiles, token use, tool latency, and errors, and they want to query raw spans with the same language they use for the rest of their data. This article builds a debugging dashboard that covers both gaps with one OpenTelemetry Collector configuration. In development, the collector runs in Docker Compose next to the Aspire dashboard, which displays each trace as a waterfall. In production, the same collector forwards spans to a Microsoft Fabric Eventstream. An Eventhouse stores them, and a Real-Time Dashboard reads them with KQL. Teams that already run Grafana can read the same Eventhouse there as well, and a later section shows that option. A note on scope before the build starts. The collector configuration and the Compose file in this article were checked for syntax only, and the Fabric steps follow the product documentation. Run each stage against a real project and compare the result with the screenshots described here. The dashboard images in this article use illustrative data. What a Foundry Agent Emits Foundry follows the OpenTelemetry semantic conventions for generative AI. These conventions currently have Development status, so attribute names can change between releases. The agent tracing documentation names several span types, which the following table summarizes together with the attributes this article reads. Span nameWhat it coversAttributes used in this articleinvoke_agentOne complete run of an agentgen_ai.agent.name, gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, error.typeexecute_toolOne tool call made by the agentgen_ai.tool.call.arguments, gen_ai.tool.call.resultplan and invoke_workflowPlanning and workflow stepsOperation specific The Foundry documentation does not list every attribute that the service writes on every span. For that reason, the KQL later in this article is a starting point. Open one real span in the Aspire dashboard first, read its attribute names, and adjust the queries if they differ. Message content is the second point of attention. The environment variable OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT controls whether prompts and responses are written into spans. Enable it on a laptop when the content is needed to debug a prompt. Leave it disabled for anything that leaves the laptop, because spans sent to a provider other than Microsoft can contain prompt and tool content. Trace One Agent Run Locally The client-side tracing path uses the azure-ai-projects package at version 2.0.0 or later, together with the OpenTelemetry SDK and an exporter. Tracing for generative AI is an opt-in feature, so the environment variable AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING must be set to true before the instrumentor starts. Shell pip install "azure-ai-projects>=2.0.0" azure-identity opentelemetry-sdk \ azure-core-tracing-opentelemetry opentelemetry-exporter-otlp The documentation shows a console exporter and describes the OTLP exporter as a replacement without a full sample. The script below applies that replacement. It creates a prompt agent, sends one request through the Responses API, and exports the spans to localhost:4317, which is the port that the collector will expose. Python import os os.environ["AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING"] = "true" os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "false" from azure.identity import DefaultAzureCredential from azure.ai.projects import AIProjectClient from azure.ai.projects.models import PromptAgentDefinition from azure.ai.projects.telemetry import AIProjectInstrumentor from opentelemetry import trace from opentelemetry.sdk.resources import Resource from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter provider = TracerProvider(resource=Resource.create({"service.name": "foundry-agent-local"})) provider.add_span_processor( BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4317", insecure=True)) ) trace.set_tracer_provider(provider) AIProjectInstrumentor().instrument() tracer = trace.get_tracer(__name__) project = AIProjectClient( endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], credential=DefaultAzureCredential(), ) with tracer.start_as_current_span("local-agent-run"): agent = project.agents.create_version( agent_name="trace-demo-agent", definition=PromptAgentDefinition( model=os.environ["FOUNDRY_MODEL_NAME"], instructions="Answer in one sentence.", ), ) client = project.get_openai_client() response = client.responses.create( input="What does a scheduler heartbeat tell an operator?", extra_body={"agent": {"name": agent.name, "type": "agent_reference"}, ) print(response.output_text) provider.force_flush() Set FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL_NAME in the shell, and sign in with the Azure CLI so that DefaultAzureCredential can find an identity. The script runs correctly before the collector exists, but it prints an export warning because nothing listens on port 4317 yet. Start the Collector and the Aspire Dashboard The Aspire dashboard is a standalone container that accepts OTLP and displays traces, logs, and metrics. It listens for OTLP over gRPC on container port 18889 and serves its interface on port 18888. The collector forwards to it, so the same spans can later go to a second destination. YAML services: otel-collector: image: otel/opentelemetry-collector-contrib:latest command: ["--config=/etc/otelcol/config.yaml"] environment: FABRIC_KAFKA_BOOTSTRAP: ${FABRIC_KAFKA_BOOTSTRAP:-localhost:9093} FABRIC_KAFKA_TOPIC: ${FABRIC_KAFKA_TOPIC:-agent-traces} FABRIC_KAFKA_CONNECTION_STRING: ${FABRIC_KAFKA_CONNECTION_STRING:-unset} volumes: - ./otel-collector-config.yaml:/etc/otelcol/config.yaml:ro ports: - 4317:4317 - 4318:4318 depends_on: - aspire-dashboard aspire-dashboard: image: mcr.microsoft.com/aspire/dashboard:latest environment: ASPIRE_DASHBOARD_UNSECURED_ALLOW_ANONYMOUS: "true" ports: - 18888:18888 # Optional. Start with: docker compose --profile grafana up -d # The Azure Data Explorer plugin is a Grafana Enterprise plugin and needs a license. grafana: profiles: ["grafana"] image: grafana/grafana-enterprise:latest environment: GF_INSTALL_PLUGINS: grafana-azure-data-explorer-datasource GF_ENTERPRISE_LICENSE_TEXT: ${GRAFANA_LICENSE_TEXT:-} EVENTHOUSE_QUERY_URI: ${EVENTHOUSE_QUERY_URI:-} AZURE_TENANT_ID: ${AZURE_TENANT_ID:-} AZURE_CLIENT_ID: ${AZURE_CLIENT_ID:-} AZURE_CLIENT_SECRET: ${AZURE_CLIENT_SECRET:-} volumes: - ./grafana/provisioning/datasources:/etc/grafana/provisioning/datasources:ro ports: - 3000:3000 The anonymous access variable removes the login prompt, which is acceptable for a dashboard that is reachable only from the local machine. The grafana service belongs to a Compose profile, so it does not start until a later section asks for it. The collector configuration defines two pipelines, one for each destination. YAML receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: batch: timeout: 5s exporters: # Local trace viewer otlp/aspire: endpoint: aspire-dashboard:18889 tls: insecure: true # Fabric Eventstream, through its Kafka-compatible custom endpoint kafka/fabric: brokers: - ${env:FABRIC_KAFKA_BOOTSTRAP} traces: topic: ${env:FABRIC_KAFKA_TOPIC} encoding: otlp_json tls: insecure: false auth: sasl: mechanism: PLAIN username: $$ConnectionString password: ${env:FABRIC_KAFKA_CONNECTION_STRING} service: pipelines: traces/local: receivers: [otlp] processors: [batch] exporters: [otlp/aspire] traces/fabric: receivers: [otlp] processors: [batch] exporters: [kafka/fabric] Several details in this file deserve attention. The exporter name otlp/aspire points at the service name aspire-dashboard on the Compose network, and TLS is disabled for that hop only. The SASL username contains a literal dollar sign, so the collector configuration escapes it as $$ConnectionString. The encoding value otlp_json makes each Kafka message a readable JSON document, which the Eventhouse can ingest without a custom decoder. Start the stack with docker compose up -d, then run the Python script. A new trace appears at http://localhost:18888 under the service name foundry-agent-local. Select it to see the waterfall, with invoke_agent at the top and its child spans below. The first run is the moment to record the real attribute names for the KQL stage. Without Fabric values in the environment, the Kafka pipeline fails to connect, and the collector logs retry messages. The local pipeline continues to work, because each pipeline has its own queue. Comment out the traces/fabric pipeline if the log noise is distracting. Create the Fabric Eventstream and Eventhouse The Fabric side has three items. An Eventstream receives the spans, an Eventhouse stores them, and a Real-Time Dashboard displays them. The Eventstream accepts Kafka clients through a custom endpoint source, which is the reason the collector can use its standard Kafka exporter. Create an Eventhouse named agent-telemetry. Fabric creates a default KQL database with the same name.Create an Eventstream and add a Custom endpoint source.Open the source, select the Kafka tab, and choose SAS Key Authentication. The page shows the bootstrap server, the connection string for the primary and secondary keys, and the topic name.Copy these values into a .env file next to the Compose file. Plain Text # Copy to .env and fill in from the Kafka tab of the Eventstream custom endpoint. FABRIC_KAFKA_BOOTSTRAP=<bootstrap-server-from-sas-key-authentication-page> FABRIC_KAFKA_TOPIC=<topic-name-from-the-kafka-tab> FABRIC_KAFKA_CONNECTION_STRING=<connection-string-primary-key> The Fabric documentation describes the authentication settings as security.protocol of SASL_SSL, sasl.mechanism of PLAIN, a username equal to the literal text $ConnectionString, and the connection string as the password. The collector configuration above maps each of these to its own keys. Copy the bootstrap server exactly as the page shows it, including the port. Before the Eventstream can write to the Eventhouse, the tables must exist. Run the following script in the KQL query set of the database. It creates a landing table, a typed table for analysis, a function that flattens the OTLP structure, and an update policy that runs the function for every ingested batch. Plain Text .create table RawTraces (resourceSpans: dynamic) .create table AgentSpans ( Timestamp: datetime, TraceId: string, SpanId: string, ParentSpanId: string, Service: string, SpanName: string, Operation: string, AgentName: string, Model: string, DurationMs: real, InputTokens: long, OutputTokens: long, ErrorType: string ) .create-or-alter function ExpandAgentSpans() { RawTraces | mv-expand rs = resourceSpans | mv-apply a = rs.resource.attributes on ( where a.key == "service.name" | project Service = tostring(a.value.stringValue) ) | mv-expand ss = rs.scopeSpans | mv-expand sp = ss.spans | mv-apply a = sp.attributes on ( summarize attrs = make_bag(bag_pack(tostring(a.key), a.value)) ) | project Timestamp = unixtime_nanoseconds_todatetime(tolong(sp.startTimeUnixNano)), TraceId = tostring(sp.traceId), SpanId = tostring(sp.spanId), ParentSpanId = tostring(sp.parentSpanId), Service, SpanName = tostring(sp.name), Operation = tostring(attrs["gen_ai.operation.name"].stringValue), AgentName = tostring(attrs["gen_ai.agent.name"].stringValue), Model = tostring(attrs["gen_ai.request.model"].stringValue), DurationMs = (tolong(sp.endTimeUnixNano) - tolong(sp.startTimeUnixNano)) / 1000000.0, InputTokens = tolong(attrs["gen_ai.usage.input_tokens"].intValue), OutputTokens = tolong(attrs["gen_ai.usage.output_tokens"].intValue), ErrorType = tostring(attrs["error.type"].stringValue) } .alter table AgentSpans policy update @'[{"IsEnabled": true, "Source": "RawTraces", "Query": "ExpandAgentSpans()", "IsTransactional": false, "PropagateIngestionProperties": false}]' .alter-merge table RawTraces policy retention softdelete = 1d Each Kafka message holds one OTLP export request, so each row in RawTraces holds one resourceSpans array. The function expands that array level by level, reads the service name from the resource attributes, and converts the span attributes into a property bag. OTLP JSON stores every attribute as a typed object, for instance {"intValue": "412"}, which explains why the function reads .intValue and .stringValue separately. The retention policy on RawTraces keeps raw data for one day, because the typed table holds the long-term copy. Back in the Eventstream, add the Eventhouse as a destination. Select direct ingestion, choose the RawTraces table, and set the data format to JSON. When the mapping page appears, map the top-level resourceSpans field to the single column of the same name. Publish the Eventstream, then restart the collector so that it picks up the .env values. Python docker compose up -d --force-recreate otel-collector python agent_trace_demo.py Within a minute, the query AgentSpans | take 10 should return rows. When it does not, query RawTraces | take 1 first. An empty landing table points to the Kafka connection, and a populated landing table with an empty AgentSpans points to the update policy or to an attribute name that differs from the one in the function. Build the Real-Time Dashboard Create a real-time dashboard in the same workspace and add the Eventhouse database as a data source. Each tile runs one KQL query. The queries below cover the five questions that agent owners ask most often. The first compares load between agents, and the second shows whether latency has changed. The third and fourth show cost and tool health, and the last lists failures. Plain Text // Agent runs per 15 minutes AgentSpans | where Operation == "invoke_agent" and Timestamp > ago(6h) | summarize Runs = count() by bin(Timestamp, 15m), AgentName | render timechart // Run latency, median and 95th percentile AgentSpans | where Operation == "invoke_agent" and Timestamp > ago(6h) | summarize p50 = percentile(DurationMs, 50), p95 = percentile(DurationMs, 95) by bin(Timestamp, 15m) | render timechart // Token use per hour AgentSpans | where Operation == "invoke_agent" and Timestamp > ago(24h) | summarize InputTokens = sum(InputTokens), OutputTokens = sum(OutputTokens) by bin(Timestamp, 1h) | render columnchart // Tool calls and their latency AgentSpans | where Operation == "execute_tool" and Timestamp > ago(6h) | summarize Calls = count(), p95 = percentile(DurationMs, 95) by SpanName | order by Calls desc // Errors by type AgentSpans | where isnotempty(ErrorType) and Timestamp > ago(24h) | summarize Errors = count() by ErrorType | order by Errors desc Two choices in these queries are worth explaining. Latency uses percentiles, because the mean hides slow runs when most runs are fast. Token counts are summed from invoke_agent spans only, so a model call inside a run is not counted twice. Set the dashboard to refresh every 30 seconds. A tile that shows errors can also be connected to a Fabric Activator rule, which sends a message to a Teams channel when the count exceeds a threshold. That step is optional, and it works on the same AgentSpans table. Read the Same Data in Grafana The Real-Time Dashboard is the shortest path, because it needs no extra software. Many teams already operate Grafana for other services, and they prefer to keep agent panels next to those. Grafana can query the Eventhouse with the Azure Data Explorer data source plugin, because an Eventhouse accepts the same KQL queries as an Azure Data Explorer cluster. One condition applies before any setup. The plugin is a Grafana Enterprise plugin. Its installation page states that it needs an Enterprise license for self-managed Grafana, or a Pro or Advanced plan on Grafana Cloud. If no license is available, the Real-Time Dashboard from the previous section remains the way to read the data. The Compose file already contains a grafana service in a profile. It uses the Enterprise image, installs the plugin at startup through GF_INSTALL_PLUGINS, and reads a provisioning file that defines the data source. YAML apiVersion: 1 datasources: - name: Eventhouse type: grafana-azure-data-explorer-datasource access: proxy jsonData: clusterUrl: ${EVENTHOUSE_QUERY_URI} tenantId: ${AZURE_TENANT_ID} clientId: ${AZURE_CLIENT_ID} defaultDatabase: agent-telemetry secureJsonData: clientSecret: ${AZURE_CLIENT_SECRET} version: 1 The data source authenticates with an app registration. Create one in Microsoft Entra ID and add it as a viewer on the agent-telemetry KQL database in Fabric. Copy the Query URI from the Eventhouse page into EVENTHOUSE_QUERY_URI, and add the tenant ID, client ID, and client secret to the .env file. Whether a service principal can reach an Eventhouse depends on the Fabric tenant settings, so ask a Fabric administrator to confirm that setting before the first attempt. Shell docker compose --profile grafana up -d Open http://localhost:3000, select the Eventhouse data source, and use Save and test. A panel then takes the same queries as the Real-Time Dashboard. Replace the fixed ago(6h) filter with the Grafana time macro so that the dashboard time picker controls the range, for example where $__timeFilter(Timestamp). Check the macro names against the plugin documentation for the installed version. Both dashboards read one table, so the numbers agree. The choice between them depends on where the team already works. The Real-Time Dashboard fits a team that works in Fabric, and Grafana fits a team with an existing alerting and on-call setup. Send Spans From a Hosted Agent A hosted agent in Foundry sends telemetry to Application Insights through the connection string that the platform injects. It also exports OTLP when OTEL_EXPORTER_OTLP_ENDPOINT is set. The related variables are OTEL_EXPORTER_OTLP_PROTOCOL, which accepts http/protobuf or grpc, OTEL_EXPORTER_OTLP_HEADERS, and OTEL_TRACES_SAMPLER. These variables belong in the environment_variables section of the agent definition. YAML environment_variables: OTEL_EXPORTER_OTLP_ENDPOINT: https://<collector-host> OTEL_EXPORTER_OTLP_PROTOCOL: http/protobuf OTEL_EXPORTER_OTLP_HEADERS: ${{connections.otel-secrets.credentials.otlp_headers} The headers value usually carries a credential, so the documentation stores it in a custom keys connection and references it, as the last line shows. Check the hosted agent telemetry page for the exact agent definition schema before the first deployment. Several constraints shape the design. Environment variables are fixed for each agent version, so a change to the endpoint requires a new version. The service.name value always equals the agent name. OTLP export does not disable Application Insights, and project monitoring must be turned off for an OTLP-only setup. The collector must also be reachable from Foundry. A container app with HTTPS ingress is one option, and the same otel-collector-config.yaml works there with the receiver on port 4318 and the Aspire exporter removed. Add authentication to the receiver before exposing it, because an open OTLP endpoint accepts spans from any sender. Where This Leaves You The local stack answers the question a developer asks after every change: what happened inside this run. The Fabric stack answers the question an owner asks every morning: how the agent behaved overnight. Both read the same spans from the same collector, so a pattern found on the dashboard can be reproduced on a laptop with a matching input. Two limits apply. The attribute names come from a convention still in development, so the flattening function is the only place to adjust after a platform update. Eventstream and Eventhouse also consume Fabric capacity, so review the sampler setting for high-volume agents. References Trace an agent in Microsoft Foundry (client-side tracing)Configure telemetry for hosted agents in Microsoft FoundryAgent tracing concepts in Microsoft FoundryConsume events from a Fabric Eventstream with the Kafka endpointOpenTelemetry Collector Contrib: Kafka exporterAspire standalone dashboardAzure Data Explorer data source for Grafana: installationAzure Data Explorer data source for Grafana: configuration

By Jubin Soni, FBCS DZone Core CORE
Gitember: Building a Git Client Beyond Commit, Push, and Pull
Gitember: Building a Git Client Beyond Commit, Push, and Pull

Git plays a key role in modern software development. However, most Git GUIs still focus on the same basic operations: stage, commit, branch, merge, pull, and push. These actions matter, but they cover only part of what developers do every day. When working on a real project, developers also need to answer questions such as: What changed across several repositories?When did this piece of code or configuration exist?How do I compare two versions?Where did a particular change disappear?How do I review a change without switching between several tools? Gitember is an open-source desktop Git client designed to address these issues while supporting the full cycle of daily Git workflows. The most interesting aspect here is not just another interface for executing git commit, but the functionality built around Git. Of Course, Gitember Is a Complete Everyday Git Client Gitember covers the typical Git workflow, focusing on the following areas: Repository and remote operations – cloning/initializing, fetch, pull, push, working with remote repositories, and Git LFS.Changes and branches – staging changes, creating commits, editing and undoing commits, branch management, merging, rebasing, and interactive rebasing.History and repository management – commit history, modified files, comparing versions and branches, reflog, Git notes, worktrees, and submodules.Conflict resolution and security – stash management, three-way merging, commit signing, and signature verification. The goal is simple: day-to-day Git work shouldn't require constantly switching between the command line, a diff viewer, a merge tool, and a browser. However, it is not these features that make Gitember particularly interesting. Workspaces: Working With Several Repositories as One Project Modern applications are often distributed across multiple repositories. A single product might consist of a frontend and backend, a set of services, APIs and their implementations, and so on. Git treats these as independent repositories, which is the right approach from a version-control perspective. The challenge arises because developers often view them as a single, unified system. Workspaces in Gitember provide an abstraction layer over individual repositories. A workspace allows you to group multiple independent Git repositories and view their aggregate state. Git operations — such as pull, push, fetch, and managing branches and commits — can be performed across all these repositories simultaneously. It is important to note that this does not turn them into a monorepo; each repository retains its own history and remote configurations. There are also deliberate limitations: operations within a workspace are executed sequentially, and a commit spanning multiple repositories is not an atomic transaction. Thus, a workspace serves more as a coordination mechanism than a distributed transaction system. Submodules are supported as well. Search Git History by Content, Not Only by Commit Metadata Typically, searching Git history is done by commit messages, authors, dates, or file paths. But in most cases, you remember the content, not the commit itself. For example: “There was a configuration value somewhere around two years ago.” Or: “At some point we had this API endpoint.” Or: “Which version contained this particular text?” The Extended Search function in Gitember indexes the selected repository history and allows searching for file content in the history. This changes the workflow from: Find the commit and then inspect its contents. to: Find the content and then navigate to the commit. Search is not limited to source code. The documentation describes searching within the content of documents in various supported formats, including Office files, CAD files, archives, audio recordings, and other materials. The Git history effectively transforms into a searchable array of historical data. Integrated Diff and Merge Workflow Another common developer workflow involves jumping between several applications: Git client -> diff tool -> merge tool -> Git client Gitember tries to keep this workflow together. It provides: side-by-side diffsversion comparisonbranch comparisonfile and folder comparisonvisual three-way merge The three-way merge editor is particularly useful when conflicts need to be understood rather than simply resolved by choosing “ours” or “theirs.” The file and folder comparison function can be used independently. Repository Statistics The Git commit graph shows exactly what happened. Statistics, however, help answer other questions: How active is this repository?How has development activity changed over time?Who is making the biggest contribution?What does the repository's history look like in terms of trends? Gitember includes tools for analyzing statistics, allowing you to view the repository not just as a collection of Git objects, but as a dataset regarding development activity. This is particularly useful when exploring an unfamiliar repository. Instead of immediately browsing through hundreds of commits, you can first get a general overview of the project. Local AI Instead of Another Cloud Dependency Gitember includes experimental AI-powered features that utilize locally running large language models (LLMs). The described use cases include: Detecting potential secrets in staged files;Generating descriptions of differences between branches;Creating commit messages. A key architectural feature is that the AI model runs locally. This means that using the AI does not require adding another mandatory cloud service to the developer's workflow. This feature is intentionally positioned as an auxiliary tool rather than a replacement for comprehensive security solutions. Why Build This in Java? Gitember is a Java desktop application using Swing and JGit. One reason this is practical is that the Java ecosystem already provides many of the building blocks required for this type of application: JGit, mature search/indexing technologies, document extraction libraries, etc. The result can be a fairly self-contained developer tool rather than a thin GUI that depends on a collection of external applications. There is also an interesting misconception about Java desktop applications and memory consumption. Java is not automatically the biggest memory consumer simply because it is Java. Many modern desktop developer tools are Electron-based and bring a browser runtime with them. In practical measurements, Gitember can consume less memory than GitHub Desktop despite providing a broader set of Git functionality. The important metric is therefore not simply the implementation language, but how efficiently the complete application performs its workload. Location Gitember is open source under GPL-3.0 and available for Windows, macOS, and Linux. Project: https://gitember.orgSource code: https://github.com/iazarny/gitember

By Igor Azarny
Kubernetes Rolling Updates Are Lying to You — and Your Users Are Paying for It
Kubernetes Rolling Updates Are Lying to You — and Your Users Are Paying for It

There is a moment during a rolling update when everything looks fine. The old pods are draining. The new pods are coming up. The readiness probes are passing. The deployment controller is happy. You watch the replica counts shift in your terminal and think this is working exactly as designed. Then your on-call phone buzzes. In a 5G signaling deployment I worked on, this happened during what should have been a routine update. Every infrastructure signal was green. The Kubernetes event log was clean. The new pods passed their readiness checks within a few seconds of starting up. Session establishment failures jumped 18%. The pods were running. They just weren’t ready. And our probes had no way to tell the difference. The Problem With HTTP Probes Readiness probes are one of Kubernetes’ better ideas. Before they existed, you’d route traffic to a pod and hope the application had finished initializing. Probes let you say: Don't send traffic until the pod tells you it’s ready. The default implementation looks something like this: Plain Text readinessProbe: httpGet: path: /healthz port: 8080 initialDelaySeconds: 5 periodSeconds: 10 Kubernetes hits /healthz. Gets a 200. Marks the pod ready. Routes traffic. For a stateless web service, this is fine. The HTTP endpoint responding and the application being ready to handle requests are essentially the same thing. Load a config file, spin up a thread pool, and you’re good. The assumption breaks the moment your application has to integrate with something before it can serve real traffic. What Integration Actually Means In telecom environments, network functions don’t just start — they register. A SIP proxy needs to complete handshakes with upstream session border controllers. A Diameter node needs to establish peer connections with authentication servers. A GTP function needs to synchronize state with its neighbors. These aren’t instantaneous. Depending on the environment, full registration can take anywhere from ten seconds to over a minute after the process has started. During that window, here’s what happens: Pod starts./healthz returns 200 (process is up).Readiness probe passes.Kubernetes routes traffic to the pod.Pod receives a request that requires upstream registration.Registration isn’t complete yet.Request fails. From Kubernetes’ perspective, everything worked. The probe passed. Traffic was routed. From your user’s perspective, their session failed. The problem isn’t telecom-specific. Any of these patterns hit the same issue: gRPC services that need to warm connection pools before handling loadMessage queue consumers that need to establish broker connections and claim partitionsDatabase-backed services that need to run migrations or seed connection poolsService mesh participants that need sidecar handshakes before traffic can flow The HTTP probe can only tell you the first thing. It says nothing about the second. What the Incident Actually Looked Like In the 5G deployment, the readiness probe was checking a lightweight internal endpoint that returned 200 as soon as the main process thread started. Two seconds after startup, at most. What it wasn’t checking: whether the pod had completed SIP registration with the upstream proxy and re-established session state with the session management function. That took closer to thirty seconds. Sometimes longer under load. So for that thirty-second window during each pod’s rollout, we were routing live signaling traffic to pods that were alive but unregistered. Requests that hit those pods during registration would fail. The failures were silent—no crash, no obvious error, just session establishment attempts that didn’t complete. The 18 percent failure rate during the update window wasn’t from a bug in the new code. It was from routing traffic to pods before they were actually ready to handle it. The first time it happened, it resolved on its own, and we wrote it off as a fluke. The second time, it happened during a peak traffic period, and the impact was hard to ignore. The third time never happened because we’d fixed the probe by then. How to Fix It The fix is to replace the HTTP probe with something that checks actual readiness — not process readiness, but integration readiness. For the signaling service, that meant an exec probe that called a script checking registration state: YAML readinessProbe: exec: command: - /bin/sh - -c - "check-registration-status.sh && exit 0 || exit 1" initialDelaySeconds: 15 periodSeconds: 5 failureThreshold: 18 The script did two things: it verified that the SIP registration with the upstream proxy had been completed and confirmed that the session management peer connection was established. Until both returned healthy, the pod stayed out of rotation. The tuning matters: initialDelaySeconds: 15 – gives registration time to begin before Kubernetes starts checkingperiodSeconds: 5 – checks frequently enough to catch completion quicklyfailureThreshold: 18 – allows up to 90 seconds for registration to complete before Kubernetes gives up and restarts the pod For non-telecom workloads, the same pattern applies. A gRPC service might check whether its connection pool is warmed: YAML readinessProbe: exec: command: - /bin/sh - -c - "grpc_health_probe -addr=:9090 && pool-ready-check.sh" initialDelaySeconds: 10 periodSeconds: 5 failureThreshold: 12 The principle is always the same: don’t ask Kubernetes to decide your application is ready based on evidence that doesn’t reflect what ready actually means for your workload. One More Thing Worth Knowing If you fix your readiness probe and your rolling updates get slower, that’s correct behavior—not a regression. Before the fix, pods were being marked ready before they actually were. Updates appeared fast because Kubernetes was moving quickly through the rollout. After the fix, updates take longer because Kubernetes now waits for real readiness before routing traffic. The slowdown is the probe working. Your update velocity was an illusion. The Short Version If your readiness probe is an HTTP check against a /healthz endpoint, ask yourself what that endpoint actually knows about your application’s readiness state. If the answer is "it knows the process started," you have a process-up probe, not a readiness probe. The distinction matters only when something goes wrong during a rollout — exactly when you need your probes to be accurate. Related Research The production autoscaling findings referenced in this article informed peer-reviewed research on workload-aware Kubernetes autoscaling for telecom infrastructure, published in IEEE Xplore: https://ieeexplore.ieee.org/document/11501289

By Pallavi Priya Patharlagadda

The Latest Coding Topics

article thumbnail
When Jev Says It's Sure, Is It Right? A Fraud Case Study in Neo4j
Jev’s confidence exceeds its accuracy, and only its top range holds up, mostly on easy cases. Use confidence for triage and error rates for automation thresholds.
November 17, 2026
by Akmal Chaudhri DZone Core CORE
· 410 Views
article thumbnail
Pitching Kotlin: An Engineering Leader's Guide to Legacy Modernization
Most Kotlin pitches stall on developer preference. This reviewer's framework prices the friction, phases the rollout behind gates, and wins approval.
November 17, 2026
by Yuri Geronimus
· 376 Views
article thumbnail
Reading GC Logs: Memory Leak or Undersized Heap?
A large live set after repeated full GCs doesn't prove a leak. Rising post-GC floor = accumulation; stable high floor = undersized heap.
November 17, 2026
by Amit Sonar
· 355 Views
article thumbnail
Using Middleware in LangChain: Recovering From Tool Failures
LangChain Middleware provides ways to recover from Tool Failures. This article helps in understanding how to use ToolRetryMiddleware and @wrap_tool_call to handle errors.
November 17, 2026
by Ruchi Saini
· 450 Views · 2 Likes
article thumbnail
7 Failure Modes Every Production AI Agent Should Be Tested Against
Learn seven AI agent failure modes and learn how to test prompt injection, tool failures, memory, authorization, and unsafe actions.
November 16, 2026
by Mihir Bhatt
· 470 Views · 1 Like
article thumbnail
Spring Boot's Hidden Secret for Configuration: EnvironmentPostProcessor
Learn how to use Spring Boot's EnvironmentPostProcessor to load, merge, and prioritize configuration from shared libraries.
November 16, 2026
by Mikhail Polivakha
· 613 Views · 1 Like
article thumbnail
Building an AI-Powered Code Review Assistant in Java
Wire up LangChain4j to an LLM, submit Java code snippets, and get structured improvement suggestions back, all from plain Java.
November 16, 2026
by Akhil Sabu
· 776 Views
article thumbnail
Modernizing a Legacy Java Application With OpenRewrite: What We Automated and What We Didn't
A practical look at OpenRewrite for large-scale Java/Spring refactoring, including automated transformations, SQL limitations, and runtime issues.
November 16, 2026
by Kirankumar Phirangi
· 917 Views
article thumbnail
Debugging senro Pipelines: Replay a Run, Shell Into a Live Step, Audit a Cache
Replay a run, open a shell inside a live step, and explain cache misses component by component: the field guide to debugging senro pipelines.
November 16, 2026
by Xavier Portilla Edo DZone Core CORE
· 422 Views
article thumbnail
Choosing a BBD Framework
Learn how to choose a behavior-driven development framework, compare Cucumber, Reqnroll, Behave, and Godog, and avoid common BDD mistakes teams make.
November 13, 2026
by Himanshu Mandhyan
· 888 Views
article thumbnail
Controller and Data Access: Understanding Two-Layer Architecture Through Abstraction and Encapsulation
Learn how a two-layer Controller and Data Access architecture applies abstraction and encapsulation to simplify maintenance and testing in C# applications.
November 13, 2026
by Dhana PAMULA
· 903 Views
article thumbnail
From Database Changes to Real-Time Actions at Scale: Oracle GoldenGate and Java
Learn how to stream database changes through Oracle GoldenGate and Kafka to Java services while preserving transaction order and boundaries.
November 13, 2026
by Sushant Sakolkar
· 943 Views
article thumbnail
7 Swift Features That Will Let You Delete Code Tomorrow
Explore the most practical Swift 6.3 and 6.4 features, from testing and concurrency improvements to performance and memory safety.
November 13, 2026
by Pavel Andreev
· 794 Views
article thumbnail
Detecting Duplicates With Machine Learning in 2026: From String Metrics to Embeddings and LLMs
Explore how machine learning uses string metrics, embeddings, and LLMs to detect duplicate records, documents, and code at scale.
November 12, 2026
by Ilya Dudkin DZone Core CORE
· 1,218 Views
article thumbnail
Persisting Cookies Across JAX-WS Service Calls Using a Custom SOAP Handler
A custom SOAP handler preserves JAX-WS session cookies across calls without JVM-wide state, using ThreadLocal isolation and explicit cleanup to prevent session leaks.
November 12, 2026
by Vishal Rameshchandra Shah
· 947 Views
article thumbnail
Building Enterprise Authentication with Angular, Spring Boot, SSO, and MFA
Implementing SSO and MFA for an enterprise Angular and Spring Boot platform taught me that authentication is frontend architecture, not a feature you bolt on at the end.
November 12, 2026
by Vaishnavi Khosla
· 973 Views
article thumbnail
Best of Three Worlds: AI Agents Writing In Rust and Java Beat the C Library
How two pure-language image decoders: one in Rust and one in Java, ended up beating libheif, and what I learned about getting AI agents to improve each other's work.
November 12, 2026
by Yannick Majoros
· 1,313 Views · 1 Like
article thumbnail
Microsoft Foundry Agent Debugging Dashboard with Microsoft Fabric Eventhouse, KQL, Docker and Grafana
Build a Microsoft Foundry agent debugging dashboard using OpenTelemetry, Aspire, Microsoft Fabric Eventhouse, KQL, and Grafana.
November 11, 2026
by Jubin Soni, FBCS DZone Core CORE
· 1,379 Views · 1 Like
article thumbnail
Kafka Consumers on EKS: Designing for a Platform That Restarts Your Consumer Group
On EKS, pod restarts look like Kafka failures. This tutorial gives a baseline config and eight failure modes, with fixes for timing, rebalances and shutdown.
November 11, 2026
by Charan Reddy Rachumalla
· 889 Views
article thumbnail
Gitember: Building a Git Client Beyond Commit, Push, and Pull
Explore Gitember, an open-source Git client with multi-repository workspaces, historical content search, visual diff and merge tools, and local AI features.
November 11, 2026
by Igor Azarny
· 1,076 Views
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×