Integration Resources

DZone's Featured Integration Resources

Global Setup in Playwright TypeScript for API Testing

By Faisal Khatri

CORE

If you need to generate prerequisite test data, such as adding dummy records through an API, before executing your test suite, using Global Setup in Playwright is a clean, maintainable, and scalable approach. It ensures all required data is prepared upfront, keeping your test execution structured and reliable. What Is Global Setup? The Global Setup in Playwright runs once before the entire test suite begins execution. It is typically used for activities such as creating test data, seeding the database, generating authentication tokens, preparing the test environment, or cleaning up old data. Since it executes only once at the start of the run, it helps ensure efficient and consistent test initialization across the suite. How to Create a Global Setup in a Playwright TypeScript Project The following steps need to be followed to create a globalSetup in the Playwright project: Step 1: Create a global-setup.ts file in the Root Folder of the Project TypeScript import orders from './test_data/orders.json' with {type: 'json'}; async function globalSetup() { console.log('Running Global Setup...'); const apiContext = await request.newContext(); const response = await apiContext.post("http://localhost:3004/addOrder/", { data: orders, }); if(response.status() !=201) { throw new Error ("Failed to create setup test data!"); } console.log('Test data created successfully'); } export default globalSetup; The globalSetup() function will insert test data into the RESTful e-commerce demo application. The test data for creating new orders is supplied using the following orders.json file. JSON [ { "user_id": "1", "product_id": "79", "product_name": "5 star 10gm Chocobar", "product_amount": 5, "qty": 1, "tax_amt": 0.5, "total_amt": 5.5 } ] The data is required to validate the GET API tests, which will execute immediately after the Playwright test command is triggered. Step 2: Register globalSetup in playwright.config.ts TypeScript import { defineConfig } from '@playwright/test'; export default defineConfig({ globalSetup: './global-setup', }); The globalSetup property tells Playwright to execute the file once before the entire test suite runs. This setup file is typically used to perform pre-test activities such as creating test data or preparing the test environment, ensuring that all tests start with the required initial state. tsconfig.json: JSON { "compilerOptions": { "module": "NodeNext", "moduleResolution": "NodeNext", "resolveJsonModule": true, "types": ["node", "@playwright/test"] } } In this case, “require.resolve()” is not used because the project is configured to use ES Modules ("module": "NodeNext" and "moduleResolution": "NodeNext" in tsconfig.json). Since the project runs in ESM mode, Playwright can directly resolve the setup file using a string path ('./global-setup') instead of relying on CommonJS-based require.resolve(). Step 3: Run the Test The tests are already in place; only global-setup.ts needs to run to feed data into the system before running them. The script for running the tests is updated in the package.json file: JSON "scripts": { "test": "npx playwright test" }, Let’s run the tests by executing the following command in the terminal: Plain Text npm run test The following is a screenshot of the output after running the tests: It can be seen that the test data was created using the global setup before running the tests. How This Helps in Your API Testing Instead of creating the order in the first test and relying on test execution order, we can use a more robust approach by setting up the required data beforehand. By creating test data once during global setup, the tests remain independent, reducing flaky dependencies and unintended failures. This approach aligns with Playwright best practices and helps build a stable, maintainable automation test suite. Important Points to Note About the Global Setup It runs once per test run.It does NOT share variables directly with tests.Use files or environment variables to pass data.Assertions should not be put in the global setup, except for setup validation. Check out how to verify response data with Playwright TypeScript for a detailed understaning on assertions. Summary Global setup in Playwright runs once before the entire test suite and is primarily used for preparing the test environment before running the tests. It helps keep tests independent and avoids relying on execution order. However, since it runs only once and does not directly share variables with tests, any required data must be passed through files, environment variables, or external storage mechanisms. Happy testing! More

How Cache Invalidation Caused Memory Pressure and API Failures at Scale

By Semyon Slepov

Most production systems do not fail loudly. They degrade quietly. This case started with an API that was technically “healthy.” It was serving traffic, success rates were high, and no critical alerts were firing. Yet recurring degradation periods caused the service to miss its service-level agreement (SLA) target for request success. During the worst degradation periods, request success could fall to approximately 98%, against a target of 99.9%. Some requests were simply failing. Because the API sat directly in the purchase path, the gap between 98% and 99.9% meant real checkout failures, abandoned transactions, and lost product revenue. What followed was not a quick fix or a scaling exercise. It was a deep investigation into how memory behavior, caching strategy, and system design interacted under load. This article walks through that process, not as a theoretical exercise, but as a practical approach to diagnosing and fixing degradation in a live production system. The Problem: Stable Metrics, Unstable System At first glance, nothing looked catastrophically wrong, which actually made the investigation significantly harder. Average latency was acceptable. Error rates were low. Infrastructure scaling policies were functioning as expected. The issue appeared only when looking at tail behavior. 95th- and 99th-percentile (P95 and P99) latencies were inconsistent. During traffic spikes, request failures increased. Retries amplified the load, which further stressed the system. Predictably, the worst incidents happened during high-traffic events like Black Friday and Christmas sales. EEE Example of Request Success Degradation During Traffic Spike Observability First, Always The investigation began with correlating signals across metrics, traces, and logs. Memory utilization was the first thing to check. Under peak load, memory usage would spike sharply, followed by increased garbage collection activity. These GC cycles introduced latency pauses that aligned closely with SLA drops. To make this visible, memory and latency were tracked together: Pascal def record_metrics(latency_ms, heap_used, gc_pause_ms): metrics.emit("api.latency", latency_ms) metrics.emit("api.heap_used_mb", heap_used) metrics.emit("api.gc_pause_ms", gc_pause_ms) (This and the following snippets are simplified pseudocode. Implementation details have been generalized.) Distributed traces added more context. Slow requests were not uniformly distributed. They clustered around specific execution paths tied to cache interactions. Structured logs filled in the missing detail. Pascal def log_cache_event(key, action, size): logger.info({ "event": "cache", "key": key, "action": action, "size_bytes": size }) By logging cache events and request metadata, it became clear that latency spikes often followed bursts of cache invalidation activity. The timing was the strongest clue. Latency degradation did not appear randomly or simply whenever traffic increased. It consistently followed spikes in cache invalidation activity. Those invalidations triggered recomputation, which increased memory allocations and garbage collection pressure shortly afterward. That made a simple capacity shortage less likely as the primary cause: adding more capacity might have provided additional headroom, but it would not have explained why degradation repeatedly followed the same application-level event. For the same reason, we treated increased GC activity as a consequence of the problem rather than the root cause. The system was not randomly degrading. It was reacting to specific internal behavior. Root Cause: Memory Pressure From Invalidation The API used a result invalidation mechanism to ensure freshness. When upstream data changed, cached results were aggressively invalidated and recomputed. On paper, the design made perfect sense. In production, it became the source of instability. Frequent invalidation caused repeated recomputation, and each recomputation allocated new objects. Under load, this led to memory churn and extra heap pressure. A simplified version of the logic looked roughly like this: Pascal def get_result(key): if cache.is_invalidated(key): result = recompute(key) cache.set(key, result) return result return cache.get(key) The issue was not correctness. It was frequency. Hot keys were invalidated too often, and the cache never stabilized. The system kept rebuilding the same data repeatedly. Simplified Request Flow The system was operating correctly, but it was still unstable under load. Designing the Experiment The working hypothesis was relatively straightforward. The invalidation strategy was too aggressive and was the primary driver of memory pressure. Rather than rewriting the mechanism, the team introduced controlled changes. The core experiment was to disable result invalidation before serving responses, allowing cached data to persist longer until the cache naturally evicted older entries. Pascal def get_result_no_invalidation(key): result = cache.get(key) if result is not None: return result result = recompute(key) cache.set(key, result) return result This change was not deployed globally. It was introduced behind a feature flag and rolled out gradually. Pascal def get_result_with_flag(key, user_id): if feature_flags.is_enabled("disable_invalidation", user_id): return get_result_no_invalidation(key) return get_result(key) Before rollout, baseline metrics were captured. Memory usage, GC frequency, cache hit rate, and latency distributions were measured to establish a comparison point. Evaluating the Tradeoff Disabling invalidation introduces risk. Data may become stale. The question was whether that staleness was actually a problem. It turned out that the underlying data did not require strict freshness. Close to 99% of the underlying data changed no more frequently than once every 10 minutes. Recomputing cached results much more often than the source data itself changed was simply wasting resources. To bound the amount of staleness while reducing the load, we introduced time-based expiry: Pascal def get_result_with_ttl(key, ttl_seconds=300): entry = cache.get_with_metadata(key) if entry and not entry.is_expired(ttl_seconds): return entry.value result = recompute(key) cache.set(key, result) return result This ensured that data would eventually refresh without triggering excessive recomputation. Rollout and Validation To prevent destabilizing the system even further, we decided to roll out the change gradually. A small percentage of traffic was routed through the new logic. The effect was visible almost immediately. Heap graphs flattened, GC pauses dropped, and the latency spikes that had dominated dashboards during peak traffic largely disappeared. Retry rates decreased. The system stopped amplifying its own load. User-facing request failures dropped significantly. As confidence grew, we expanded the feature flag to cover all traffic. Measurable Impact We saw a clear improvement after the change was implemented and rolled out. After the rollout, request success returned to approximately 99.9%, and the degradation periods that had pushed it toward 98% largely disappeared. The API powered a public-facing e-commerce system and sat directly on the purchase request path, so latency spikes and failed requests translated into abandoned purchase flows and failed transactions. During the rollout, we also measured an increase in product revenue. This is often the missing link in engineering discussions. Reliability work is not just about writing postmortems and creating action items. It can also have a visible monetary impact. Lessons From the Fix The most important lesson is that degradation rarely comes from a single failure. It emerges from the complex interactions of multiple system components. In this case, caching, memory management, and retry behavior combined to create instability. Another key insight is that correctness and reliability are often in tension. A system that is perfectly fresh but unstable is less useful than one that is slightly stale but consistently available. Finally, observability was essential to diagnosing this problem. Without high-quality signals, this issue would have been misdiagnosed as a capacity problem. The fix would have been scaling infrastructure, not addressing the root cause. Closing Thoughts Restoring the API to its 99.9% SLA target was not about incremental tuning. It requires identifying and correcting fundamental inefficiencies. In this case, the turning point was recognizing that a well-intentioned mechanism was harming the system under real-world conditions. The broader takeaway is that production systems rarely collapse because of a single catastrophic failure. When systems degrade without failing, the answer is rarely more infrastructure. It is often better visibility, disciplined experimentation, and the willingness to challenge default design choices. More

How to Test PUT, PATCH, and DELETE API Requests Using Playwright TypeScript

By Faisal Khatri

CORE

Threat Modeling Context-Aware AI: A STRIDE Walkthrough

By Sandeep Gadde

Gossips on Cryptography: Part 4

By Sahil Aggarwal

Build Software Faster With Three Simple Principles

Development in small companies and startups often slows down at the boundaries between people and teams. A developer waits for a product decision. Frontend work stalls because the API response is unclear. QA discovers that two teams interpreted the same requirement differently. Three practical habits can reduce these delays: document the feature, discuss its technical risks, and agree on interface contracts before dependent implementations diverge. Keep each step proportional to the size and uncertainty of the task. Start With the Biggest Uncertainty Before distributing work, identify what the team needs to learn first. Unclear user experience? Start with a UI prototype. Frontend developers can use mocks to explore the flow while the team clarifies requirements.Uncertain business logic, performance, or integration? Start with a focused backend investigation or technical prototype.Unclear user problem or business value? Start with product discovery, involving users, product stakeholders, and technical specialists as needed. Having a UI does not automatically make frontend work the first priority. Choose the starting point that resolves the most important uncertainty, then agree on enough shared detail to let work proceed in parallel. 1. Maintain Feature Documentation and Define Responsibilities A concise Product Requirements Document (PRD) gives the team a shared explanation of what to build, why it matters, and how to evaluate it. For a small feature, a short page may be enough. QuestionWhat to recordWhat are we building?The intended behavior, MVP scope, and explicit exclusions.Who is it for?The users and the problem they face.Why is it needed?The expected benefit to users and the business.How will we evaluate it?Acceptance criteria, success metrics, a baseline or comparison group, and a measurement period. Name the person responsible for product decisions and identify the people needed to implement and validate the feature. These are responsibilities; a small team may combine several of them in one role. ResponsibilityTypical ownerSet goals, decide scope, and resolve product questionsProduct ownerAssess feasibility, design interfaces, and implement the featureDevelopers and technical leadDefine test scenarios and verify behaviorQA and developersDesign the user experienceDesigner, where neededDefine instrumentation and evaluate impactAnalyst or another explicitly assigned team member Example: A Post Recommendation System The following is an illustrative PRD. Its numerical targets are examples to agree on for a particular product, rather than measured results or universal benchmarks. SectionExampleProblem and hypothesisUsers may struggle to find relevant posts in a chronological feed. We expect recommendations based on their interests to improve content discovery.UsersReaders discovering posts; creators whose eligible posts can be recommended.MVP behaviorReturn up to 20 eligible posts, ranked by popularity within topics inferred from likes and subscriptions. Recompute lists every 24 hours. Exclude deleted posts and posts the requesting user cannot access when serving the response. Use a general popularity list when there is insufficient interest history.Experiment supportAssign eligible users to stable control and treatment groups and record recommendation impressions and clicks. Include this in the MVP so the first version can be evaluated.Out of scopeML models, similarity between users, and advertising recommendations. Decide whether to add these after evaluating the MVP.PerformanceIllustrative target: server-side p95 response time below 200 ms for serving precomputed lists at 1,000 requests per second on a representative test dataset. Measure the batch recomputation job separately and require it to finish within the 24-hour refresh window.Acceptance criteriaTests verify ranking on a fixed dataset, the popularity fallback, access filtering, and feedback recording. Likes and subscription changes affect the next scheduled recomputation. Load tests meet the stated latency target.Success measurementIllustrative primary target: a 10% relative increase in recommendation CTR versus the control group. Define CTR as clicks divided by recorded impressions. Plan an initial two-week experiment; estimate sample size before launch and report uncertainty if the result is inconclusive. Monitor seven-day retention and API error rate as guardrails, with acceptable thresholds agreed before launch.RisksWeak relevance, overexposure of already popular posts, and load spikes. Inspect recommendation diversity, provide a fallback, and test capacity before rollout. Technical notes can accompany the PRD, with implementation decisions reviewed during technical refinement. For example, a Go service could use PostgreSQL for source data and Redis for precomputed lists. A draft API might expose GET /recommendations for the authenticated user and POST /feedback for interactions. The team still needs to define schemas, error responses, authorization behavior, and pagination before implementation. Keep release acceptance separate from business success. A correctly implemented feature can fail to improve the chosen metric. That result should inform the next product decision. What if the Product Has No Documentation? Start with the workflow you are changing and the business rules most likely to be misunderstood. Link the relevant code, record open questions, and expand the documentation as the team learns. Documenting the entire system does not need to become a prerequisite for the next useful change. After release, compare outcomes with the original hypothesis. A weak result is a reason to investigate the feature, measurement, and assumptions. Some work provides value through reliability, lower operating costs, or reduced risk, and its evaluation should reflect that purpose. 2. Conduct Technical Refinement Technical refinement, sometimes called technical grooming, connects product expectations with implementation decisions. Its output should be a workable approach and a clear record of remaining questions. Ask a developer or technical lead to review the PRD and identify: Existing constraints and integration dependencies.Performance, security, and operational risks relevant to the change.Decisions that require input from product or another team.Unknowns that need a short investigation or prototype. Bring the relevant people together to resolve those questions. If the implementation cost changes the original assumptions, revisit the scope with the product owner. Keep the Process Proportional Agree on a timebox based on scope and uncertainty. For example, a modest feature might receive two days of initial technical analysis, with a named owner and deadline for each product question. A complex migration may require several investigations. These are planning choices to review as new information appears. Record decisions, tradeoffs, unresolved questions, and owners. There is usually no need to transcribe every discussion. Finish with a small implementation plan: what can run in parallel, what must happen first, and what evidence will show that a risky assumption holds. This process can reduce avoidable rework. It does not guarantee that every issue will be discovered in advance, so leave room to revise the plan during development. 3. Develop Using the Specification-First Principle For work that crosses an API boundary, agree on the contract early. An OpenAPI description can capture HTTP operations, request and response schemas, and other interface details. The contract should also be supported by examples and documented behavior where a schema alone is insufficient. Backend and frontend engineers review the contract together, including empty states, errors, and compatibility expectations.Frontend developers build against mocks that reflect the agreed contract.Backend developers implement the API and verify that its behavior matches the contract.QA and developers prepare API contract checks and derive end-to-end scenarios from the PRD, user flows, and business rules. API contract checks and end-to-end tests serve different purposes. Matching a response schema does not establish that a complete user journey works correctly. How This Reduces Waiting Consider an illustrative recommendation feature. If frontend developers invent a response shape while waiting for the backend, they may need to rewrite rendering and error handling during integration. Agreeing on the response schema, empty-list behavior, and refresh semantics first lets both sides work against the same assumptions. Mocks still need to match the implementation. Run compatibility checks in CI and update the specification, mocks, and tests together when the contract changes. Integrate early enough to expose mismatches before release. Specification-first development also leaves room for exploratory code. A short prototype may be necessary to discover whether a proposed interface is feasible. The aim is to agree on the contract before teams invest heavily in dependent implementations. For more on the approach, see Boost Efficiency With the Specification-First Principle. Keep Documentation Useful as the Project Evolves Documentation helps future team members understand behavior and the reasons behind earlier decisions. DORA's research on documentation quality links high-quality internal documentation with organizational performance and finds that it strengthens the impact of technical practices. Keep a small set of maintained resources close to the work: The PRD and the results of the feature's evaluation.Architecture decisions, API contracts, and relevant data models.Deployment and recovery instructions.Links to changes and the decisions behind them, in GitLab, a README, or another shared system.Test scenarios and acceptance checklists. Assign ownership and update these resources when behavior changes. Outdated documentation can mislead the next person just as missing documentation can leave them guessing. Conclusion These three habits address common sources of delay: Concise feature documentation gives the team a shared goal, scope, and definition of success.Technical refinement surfaces constraints and assigns owners to unresolved questions.Agreed API contracts support parallel development and reduce integration rework. Try them on one feature. Track time spent waiting for decisions, integration rework, and time from an agreed scope to release, alongside quality and product outcomes. Use what you learn to adjust the process. The practical goal is a team that can make decisions, build, and validate changes with less avoidable waiting.

By Ilia Ivankin

Beyond HTTP Handoffs: Build Durable Agent-to-Agent Services With Temporal Nexus

Agent systems are increasingly decomposed into specialized services: a planning agent delegates research, a research agent invokes retrieval and synthesis, and a compliance agent validates the result before an action is allowed. Open standards such as A2A formalize communication between independent agents, but transport interoperability is only part of the production problem. Requests can be lost after acceptance, retries can duplicate expensive work, callers can disappear while remote tasks continue, and multi-agent chains can become difficult to reconstruct. Temporal Nexus addresses a different layer. It connects Temporal applications through durable service contracts, allowing an agent capability to behave less like a fragile HTTP handoff and more like a reliable distributed operation. When an Agent Call Becomes a Distributed Transaction Boundary A conventional agent handoff often looks like ordinary RPC: serialize a task, send it to another service, wait for a response, and retry failures. That model works for short, stateless interactions. It becomes fragile when delegated work lasts minutes or hours, crosses ownership boundaries, depends on databases or model providers, or triggers side effects. Retry behavior, deduplication, cancellation, timeout ownership, and result delivery then become part of the application protocol. Nexus moves those concerns into Temporal’s execution model. A Nexus Endpoint routes a request to a target Namespace and Task Queue while hiding those implementation details from the caller. The caller depends on a named service contract rather than the handler’s Workflow type, queue, or deployment topology. Temporal describes the relationship as peer-to-peer: caller and handler Workflows remain independent executions while Nexus provides the durable boundary between them. This distinction matters for agent platforms because a durable agent service should expose a business capability, not internal orchestration mechanics. A “research” operation can remain stable even if its implementation changes from one Workflow to a graph of Activities, model calls, human approval, or additional Nexus calls. Temporal supports multi-level Nexus composition, with each hop represented as a separate durable operation. A Nexus Contract Fits the Agent Capability Boundary In the Java SDK, a Nexus Service defines the callable contract and Nexus Operations define individual capabilities. A compact contract for a research agent can remain deliberately narrow: Java @Service interface ResearchAgentService { @Operation AgentResult research(AgentTask task); } The contract carries only the input and output required by the collaboration boundary. Temporal’s Java guidance recommends service APIs containing operation names plus serializable input and output types; JSON and Protobuf are practical choices when multiple SDK languages are involved. The caller therefore does not need access to the handler’s prompts, memory representation, tool graph, or Workflow implementation. Long-running agent work should normally use an asynchronous Nexus Operation backed by a Workflow. Temporal reserves synchronous operations for reliable, predictably low-latency paths that finish within the ten-second handler deadline. Asynchronous operations can represent long-running work and return an operation token for tracking completion without holding a conventional request open. In Temporal Cloud, the maximum Nexus Schedule-to-Close timeout is 60 days. A handler can expose an agent Workflow without introducing an HTTP controller or polling API: Java @OperationImpl public OperationHandler<AgentTask, AgentResult> research() { return WorkflowRunOperation.fromWorkflowMethod( (ctx, details, task) -> Nexus.getOperationContext() .getWorkflowClient() .newWorkflowStub( ResearchAgentWorkflow.class, WorkflowOptions.newBuilder() .setWorkflowId("research-" + details.getRequestId()) .build()) ::run); } WorkflowRunOperation.fromWorkflowMethod maps the Nexus Operation to a Workflow execution. Temporal documents the Nexus request ID as stable across retries, making it suitable for constructing a deduplication-oriented Workflow ID when no stronger business identifier exists. A business identifier is generally preferable because it aligns duplicate suppression and operational correlation with domain semantics. Durability Changes the Meaning of Retry and Cancellation The most important Nexus behavior is execution semantics. Once a caller Workflow schedules an operation, Temporal atomically hands the command to Nexus machinery. Delivery uses at-least-once execution, automatic retry, rate limiting, concurrency limiting, load balancing, and circuit breaking. A handler can therefore be invoked more than once for the same operation, which makes idempotency essential for external side effects. Temporal also documents stronger duplicate protection by backing the operation with a Workflow whose ID reuse policy rejects duplicates. That behavior is especially relevant to agents because model inference may be nondeterministic and repeated tool calls can duplicate actions such as ticket creation, notifications, or database mutations. Durable execution does not remove the need for idempotency keys at external boundaries. Instead, it provides a stable execution context in which prior decisions and completed steps can be recorded and replayed consistently. Cancellation also becomes a protocol-level feature rather than a best-effort HTTP convention. Canceling a caller Workflow propagates cancellation to pending Nexus Operations and their backing handler Workflows. Termination is different: terminating the caller abandons pending operations and does not send cancellation to the handler Namespace, so remote work can continue. Graceful cancellation is therefore the safer control path for agent chains that may need cleanup or compensation. Timeouts express separate service-level expectations. Schedule-to-Start bounds how long an operation may wait to begin, Start-to-Close bounds asynchronous execution after start, and Schedule-to-Close bounds the complete lifecycle. Nexus retries retryable failures within those limits, while non-retryable failures resolve the operation and become visible to the caller. The Caller Stays Simple While the Runtime Carries State From a caller Workflow, the remote agent looks like a typed service stub. Retry loops, callback plumbing, and result polling do not need to appear in business code: Java private final ResearchAgentService research = Workflow.newNexusServiceStub( ResearchAgentService.class, NexusServiceOptions.newBuilder() .setOperationOptions( NexusOperationOptions.newBuilder() .setScheduleToCloseTimeout(Duration.ofHours(2)) .build()) .build()); public AgentResult delegate(AgentTask task) { return research.research(task); } The endpoint mapping is configured when the caller Workflow implementation is registered, allowing deployment configuration to bind a service name to a Nexus Endpoint without changing business logic. Temporal recommends a collocated pattern by default, where Nexus handlers live beside the Workflows they expose, and a router-queue pattern when routing requires separate scaling, permissions, or deployment ownership. Operationally, Nexus creates a stronger debugging surface than an opaque chain of HTTP requests. Caller histories record Nexus scheduling, start, completion, failure, timeout, and cancellation events. Bidirectional links connect caller events to handler Workflow histories, pending operations expose retry state, and OpenTelemetry integration can visualize call graphs across Nexus Operations, Activities, and Child Workflows. Security follows the service boundary. In Temporal Cloud, Nexus Endpoints use caller-Namespace allowlists, Workers authenticate with mTLS or API keys, and cross-Namespace Nexus traffic is secured by the platform. Nexus payloads use the same Data Converter model as Workflows and Activities, including codec-based encryption. Cloud Nexus Endpoints are not general public HTTP endpoints; they are reached through Temporal SDK execution inside the account. Durable Agent Services, Not Just Durable Requests Temporal Nexus does not replace agent interoperability protocols such as A2A. A2A standardizes how independent agentic applications discover capabilities, delegate tasks, and exchange results, while Nexus connects Temporal applications through durable operations. The boundary is complementary: an external A2A-facing layer can provide ecosystem interoperability, while Temporal Workflows and Nexus provide durable execution for agent services running inside Temporal. The deeper shift is from treating agent delegation as message delivery to treating it as durable work. HTTP can move a request across a network, but production agent collaboration also needs remembered state, bounded retries, deduplication, cancellation propagation, durable completion, access control, and traceable execution across service boundaries. Nexus puts those semantics directly into the invocation model. For long-running or side-effecting agent systems, that changes the handoff from an unreliable gap between services into a first-class execution boundary that can survive process crashes, worker outages, and delayed completion without losing the thread of the operation.

By Akhil Madineni

CORE

Why Databricks and Snowflake Speak the Kafka Protocol: Ingestion vs Architecture

Kafka-compatible ingestion is showing up everywhere now, including in analytics platforms that have nothing to do with running Kafka. Databricks added Kafka-compatible APIs to its Zerobus Ingest service. Snowflake went a step further at its Summit and announced Datastream, a native, fully Kafka-compatible streaming service. In both cases, existing Kafka producers can stream straight into the platform with no code changes, just a config change. This is good news for the ecosystem. It also confirms a point I have made for years: the Kafka API has become the de facto standard for moving events around, the same way the Amazon S3 API became the standard for object storage. But the trend hides an important distinction. Using the Kafka API for ingestion into an analytics platform is a very different thing from building an event-driven enterprise architecture on Kafka. And in practice, most companies do both: they run Kafka as the operational backbone, and one of the most common sink connectors in that setup feeds the lakehouse. This post is about that difference, ingestion versus architecture, why it matters, and why the two approaches are complementary rather than competing. What Is Apache Kafka? Messaging, Storage, Connect, and Streams Most people know Kafka as messaging. Pub-sub. A way to send events from A to B in real time. That is the part everyone learns first, and it is real, but it is only a quarter of the picture. Apache Kafka (the open source framework) is four things in one platform: Messaging is the real-time publish-and-subscribe layer that decouples producers from consumers. Storage is the durable, replayable commit log. A Kafka topic is not a transient buffer that forgets data the moment it is read. Events are persisted, ordered, and can be replayed by any consumer at any time. This is what makes Kafka a system of record for events, not just a pipe. Kafka Connect is the integration framework. Hundreds of connectors move data into and out of Kafka from databases, message queues, SaaS applications, cloud storage, and yes, lakehouses. Kafka Streams and stream processing handle continuous processing of data while it is in motion. You filter, join, aggregate, and enrich events as they flow, rather than landing everything first and processing it later in batch. The point is simple: Kafka is a platform, not a message queue. That combination of storage, integration, and processing on top of messaging is exactly what lets it serve as a backbone for an entire enterprise, not just a transport between two systems. The persistent log also decouples producers from consumers in time, so each system reads the same events at its own pace, whether real-time, batch, or request-response. That is what makes Kafka a tool for data consistency across the whole estate, not only real-time speed and scale. Kafka Protocol vs. Kafka Framework: What 'Kafka-Compatible' Really Means Here is the distinction that explains the whole "Kafka is everywhere" trend, and it is one people constantly blur. The Kafka API, also called the Kafka protocol, is the open wire protocol licensed under Apache 2.0. It defines how clients talk to brokers: the requests, the binary format, the rules. It is open, and anyone can implement against it. The Kafka framework is the actual open-source software: the brokers, Kafka Connect, Kafka Streams. It is one implementation of that protocol, the original one. Because the protocol is open, other systems can speak it without using the framework underneath. Snowflake is refreshingly direct about this with Datastream: it is compatible with the Kafka wire protocol but does not use Kafka under the hood. The technology is pure Snowflake. That is the protocol-versus-framework split in a single sentence, stated by the vendor. And it is precisely why the Kafka API became the de facto standard for event streaming. But there is a crucial caveat. Protocol-compatible does not mean feature-complete. Many systems that advertise "Kafka-compatible" implement only the messaging core, and even there they often miss features. They often skip the storage semantics, connectors, stream processing, and exactly-once guarantees. So when a product says it supports the Kafka API, that tells you the on-ramp works. It does not tell you that you have a streaming platform underneath. The devil is in the details, and the details are usually everything except the basic produce-and-consume path. Two Jobs for the Kafka API: Ingestion vs. Architecture Once you separate the protocol from the platform, two very different use cases come into focus. The first is Kafka as the operational backbone. Event-driven microservices. Real-time applications. The central nervous system that connects operational and analytical systems across the business. Kafka as the strategic integration layer that replaces the old ESB, ETL, and iPaaS middleware. In this world, data is processed in motion, consumed by many systems at once, and the workloads are mission-critical, stateful, and continuous. The second is Kafka as an ingestion path into analytics. Here the Kafka API is a convenient, standardized on-ramp to get events into a warehouse or lakehouse, where they are processed analytically, often in micro-batches. One producer, one destination, analytics at the end. The first is an event-driven architecture. The second is a smarter pipe into an analytics platform. And here is the part that matters most: most companies do not pick one. They do both. They deploy Kafka for the event-driven architecture, run their operational workloads on it, and then one of the most common sink connectors in that whole setup is the one feeding the lakehouse. Kafka runs the live business, and Snowflake or Databricks gets a continuous feed of those same events for analytics, reporting, and AI. The two are complementary, not competing. Streaming Platform vs. Lakehouse Ingestion: Confluent, Databricks, Snowflake This is where the recent vendor announcements fit in, and it helps to see the two approaches side by side. On one side are the data streaming platforms. Whether you run open-source Apache Kafka yourself or work with a vendor such as Confluent, the goal is the same: build the full platform around the event-driven architecture. Operational and analytical workloads, Connect for integration, stream processing for data in motion, and the durable log as a system of record. There is a rich and fast-moving market here, with strong options well beyond the obvious names. For an overview of how it is evolving, see my Data Streaming Landscape 2026. On the other side are the analytics platforms adding Kafka-compatible ingestion. Databricks Zerobus and Snowflake Datastream are two clear examples. Their goal is narrower and perfectly reasonable: get producer data into their platform without requiring a separate Kafka cluster or a separate streaming vendor. Snowflake is explicit that Datastream is purpose-built for teams that want to replace their Kafka infrastructure with a native Snowflake service, landing topics directly as governed Snowflake or Iceberg tables. Worth noting on maturity: Databricks has rolled out its Kafka-compatible APIs in beta, with the rest of Zerobus generally available, while Snowflake Datastream is still heading into private preview. Both implement the protocol as an ingest interface into their own platform, not as a general-purpose streaming backbone. Both approaches are valid. They solve different problems. When to Use Kafka vs. Lakehouse Ingestion If all you need is to land events in the lakehouse, then Kafka-compatible ingestion built directly into the analytics platform is a fine choice. There is nothing wrong with it. It is one less system to run, one less vendor to manage, and the config-change migration story is real. No need to overthink it. But be clear-eyed about what most enterprises actually do. They do not use Kafka only for ingestion into analytics. They use it for operational use cases, for a strategic event-driven enterprise architecture, and as the integration platform that replaces legacy middleware. For that, an analytics platform's ingest service is not a substitute. It was never designed to be one. It is the on-ramp, or at most a replacement for the on-ramp, not the backbone. So choose based on the use case, not the headline. If you only need the pipe into the lakehouse, take the pipe. If you are building the operational nervous system of your business, you need the platform, and the lakehouse becomes one important consumer of it rather than the center of it. More vendors adopting the Kafka API is a win for everyone. It confirms the protocol is the standard. The only thing to stay sharp on is which problem you are solving: ingestion into analytics, or the event-driven enterprise architecture that runs the business.

By Kai Wähner

CORE

How to Test POST API Requests With Playwright TypeScript

Testing POST API requests is an important skill for modern QA and automation engineers working with backend services and microservices. In this article, we’ll explore how to test POST API requests with Playwright and TypeScript, focusing on sending a request body using different approaches. By the end, you will learn how to send POST API requests using the following approaches for adding a request body: JSON Object/Array(data)Stringified JSONJSON FileFaker library Application Under Test We will use the POST /addOrder API of the RESTful e-commerce demo application for this demo. The API schema is provided below: JSON { "user_id": "string", "product_id": "string", "product_name": "string", "product_amount": 0, "qty": 0, "tax_amt": 0, "total_amt": 0 } ] Testing POST API Requests With Playwright TypeScript Playwright provides a powerful request API that allows us to create and manage HTTP request contexts. Let’s walk through how to send POST API requests step-by-step using different approaches for passing the request body: Request Body as JSON Object/Array Let’s send a POST API request with a static JSON Array and verify that the response status code is 201. TypeScript test("POST order details API with static JSON Array", async ({ request }) => { const response = await request.post("http://localhost:3004/addOrder/", { data:[{ user_id: "1", product_id: "82", product_name: "Cadbury Bar", product_amount: 12, qty: 2, tax_amt: 1, total_amt: 25 }, { user_id: "2", product_id: "80", product_name: "MilkyBar", product_amount: 10, qty: 1, tax_amt: 1, total_amt: 11 } ], }); expect(response.status()).toBe(201); }); Code Walkthrough A Playwright test case for testing the POST API request is created using the request API, which is a built-in Playwright APIRequestContext fixture used to send HTTP requests. Sending the POST Request: The request.post() sends a POST request to the “http://localhost:3004/addOrder/” endpoint. The response received after sending a POST request is stored in the response variable.Passing the Request Body (Static JSON Array): The data property is used to send the request body, which accepts data in JSON Array and JSON object formats. Since the POST /addOrder API accepts an array of order details, allowing multiple orders to be submitted within a single JSON array, the test provides the data in JSON array format.Validating the Response: The response.status() retrieves the HTTP status code. The assertion ensures that the status code 201 is returned, indicating that the orders were successfully created. Request Body Using JSON.Stringify() JSON.stringify() can be used while sending raw payloads, testing malformed JSON, or sending custom-formatted JSON. However, the Content-Type header should be set to application/json so the server correctly interprets the request body as JSON data. Let’s write the same POST /addOrder test using JSON.stringify() and verify that a status code 201 is returned in the response. TypeScript test("POST order details API using JSON.Stringify", async ({ request }) => { const orderData = [{ user_id: "5", product_id: "64", product_name: "Cadbury Mini", product_amount: 5, qty: 3, tax_amt: 1, total_amt: 16 }]; const response = await request.post("http://localhost:3004/addOrder/", { data: JSON.stringify(orderData), headers: { "Content-Type": "application/json", }, }); expect(response.status()).toBe(201); }); The orderData array is defined to contain one order object. Each field represents the order details such as user_id, product_id, product_name, and so on. This is a normal JavaScript object at this stage, not JSON yet. The JSON.stringify(orderData )converts the JavaScript object into a JSON string. As we manually stringify the request payload, we should explicitly set the Content-Type header to application/json. It tells the server to treat the request body as JSON data. Playwright sends the POST API request using the post() method and then validates that the response returns a 201 status code. Request Body as a JSON File Using a JSON file as a request body is a handy approach when testing POST API requests with Playwright TypeScript. It supports large payloads and allows the same file to be easily reused across multiple tests. The following orders.json file will be used as a request payload in the POST /addOrder API: JSON [ { "user_id": "1", "product_id": "79", "product_name": "5 star 10gm Chocobar", "product_amount": 5, "qty": 1, "tax_amt": 0.5, "total_amt": 5.5 }, { "user_id": "2", "product_id": "71", "product_name": "Lindt Milk Chocolate", "product_amount": 15, "qty": 3, "tax_amt": 2.5, "total_amt": 47.5 } ] The following configurations should be in place before we proceed to write the test using a JSON file as the request payload. tsconfig.json JSON { "compilerOptions": { "module": "NodeNext", "moduleResolution": "NodeNext", "resolveJsonModule": true } } We need to ensure that the JSON file is imported first and then attached to the data parameter while sending the POST request. TypeScript import orders from '../test_data/orders.json' with {type: 'json'}; test("POST order details API using JSON file", async ({ request }) => { const response = await request.post("http://localhost:3004/addOrder/", { data: orders, }); expect(response.status()).toBe(201); }); This test imports test data from an external orders.json file and uses it as the request body for a POST API call in Playwright. The request.post() method sends the imported JSON directly as the payload, and the test verifies that the API returns a 201 status code. Using a JSON file makes it easier to manage, reuse, and update request payloads without modifying the test logic, improving the maintainability and readability of API tests. Request Body With Faker Library Using the Faker library to create a request body for a POST API helps generate realistic and dynamic test data, reduces hardcoded values, and improves test coverage. It is especially useful for simulating real-world scenarios and avoiding duplicate data issues. However, it also has limitations. Since Faker generates random data, it may lead to inconsistent test results if not properly handled, and debugging failures could become more difficult without fixed or reproducible inputs. To use the Faker library, we need to install it first using the following command: Plain Text npm install --save-dev @faker-js/faker Next, let’s create two helper functions: one to create order objects with the required order details, and another to generate an array of orders based on the count provided by the user. TypeScript import { faker } from '@faker-js/faker'; export function createOrderDetails() { const productAmount:number = faker.number.int({ min: 1, max: 100 }); const qty:number = faker.number.int({ min: 1, max: 5 }); const taxAmt:number = faker.number.int({ min: 2, max: 10 }); const totalAmt:number = (productAmount*qty)+taxAmt; return { user_id: faker.number.int({ min: 1, max: 50 }), product_id: faker.number.int({ min: 1, max: 100 }), product_name: faker.commerce.productName(), product_amount: productAmount, qty:qty, tax_amt: taxAmt, total_amt: totalAmt }; } The createOrderDetails() function returns the order details with the required fields. It also calculates the total amount by summing the total product value and tax amount. The values in all fields are updated randomly using the appropriate methods provided by the Faker library. TypeScript import { faker } from '@faker-js/faker'; export function createRandomOrders(count:number) { return faker.helpers.multiple(createOrderDetails, {count}); } The createRandomOrders(count) method accepts a count parameter and returns randomly generated orders that can be directly used in the test. TypeScript return faker.helpers.multiple(createOrderDetails, {count}); This is where the work happens. multiple() is a helper function provided by Faker. Its job is to execute another function multiple times and collect all the results into an array. It takes two arguments: A function to execute repeatedly.An options object specifying how many times to execute it. It asks Faker to call createOrderDetails() exactly count times, collect all the generated orders into an array, and return that array to the caller. TypeScript test('POST order details API using Faker library', async({request}) => { const orderData = createRandomOrders(5); const response = await request.post("http://localhost:3004/addOrder/", { data: orderData, headers: { "Content-Type": "application/json", }, }); expect(response.status()).toBe(201); }); This test sends a POST request to the /addOrder endpoint API using dynamically generated test data from the Faker library. The createRandomOrders(5) function generates an array of 5 random order objects, which are passed as the request body using the data property. The test then verifies that the API responds with a 201 status code, confirming that the orders were successfully created. Summary In this tutorial, we explored multiple approaches to testing POST API requests using Playwright with TypeScript, including sending static JSON payloads, using JSON.stringify, importing data from external JSON files, and generating dynamic test data with the Faker library. We also covered how to handle headers correctly and validate API responses using status code assertions. In my experience, using external JSON files is a practical approach for testing POST API requests because it lets us send bulk data from a single file. Similarly, the Faker library can also be used to generate dynamic test data; however, in some cases, using a third-party library may not be permitted. Ultimately, the software team chooses the approach that best fits their testing strategy. Happy testing!

By Faisal Khatri

CORE

Jakarta Batch in Practice: Reliable Chunk-Oriented Processing for Enterprise Workloads

Batch processing remains vital because many business operations aren't suited to interactive requests. Tasks such as recalculating prices, reconciling transactions, migrating records, generating reports, processing invoices, reclassifying customers, or applying rules across millions of records may require considerable time. Handling these as standard requests leads to fragile systems, increased user wait times, frequent timeouts, challenging retries, and possible data inconsistencies. A batch model handles large workloads predictably, incrementally, and with control over progress and recovery. Rather than processing a massive operation as a single loop, batch processing uses jobs, steps, chunks, checkpoints, filtering, and restartability. This approach separates long-running data tasks from the user experience while delivering a structured execution model. In this article, we will focus on Jakarta Batch and examine its sustained relevance for modern enterprise applications. Why Batch Processing Still Matters in Enterprise Systems Modern applications offer various methods for background processing, such as message queues, event-driven architectures, schedulers, reactive pipelines, and distributed stream-processing platforms. While each addresses specific needs, batch processing is most effective when operations have a defined start and end, involve a known or discoverable dataset, and require controlled execution, progress tracking, restartability, or periodic processing. Batch processing remains essential in enterprise systems. Workloads such as financial reconciliation, billing, payroll, reporting, data migration, regulatory processing, catalog updates, and large-scale reclassification are still prevalent. In these scenarios, the priority is to process large volumes of work safely and predictably, rather than responding to individual events quickly. Batch provides a model specifically designed for these requirements. How Jakarta Batch Works Jakarta Batch organizes background processing into jobs and steps. A job defines the overall batch operation, while each step represents a specific stage. In chunk-oriented processing, a step follows a simple pipeline: read, process, write, and repeat until it processes all input. The Jakarta Batch runtime manages this lifecycle so application code can focus on reading, transforming, and persisting data. A job is the top-level unit of execution and represents a complete business operation, such as importing records, recalculating customer classifications, processing invoices, or reconciling transactions. Jobs can accept parameters at startup, allowing the same batch definition to run with different inputs or business rules. A job consists of one or more steps, each representing a distinct phase of the workload. Simple jobs may have a single step, while complex processes can use multiple steps in sequence, such as importing data, validating it, and generating a final report. Within a chunk-oriented step, the ItemReader supplies data to the runtime one item at a time, from sources such as a database or file. The reader only retrieves the next item and does not need to know how it will be processed or persisted.The ItemProcessor receives each item and applies business rules, such as validation, transformation, classification, enrichment, or filtering. It may return a modified item or null if the item should be excluded from writing.The ItemWriter receives processed items and persists or exports them. Unlike the reader and processor, which handle items individually, the writer typically receives a group of items from the current chunk. This enables more efficient database or bulk operations. Jakarta Batch adds features around this pipeline to support enterprise workloads. The runtime manages chunk boundaries, transactions, checkpoints, execution status, failures, and restart behavior. Chunk size determines how much work is grouped before a write and checkpoint, making it a key parameter for balancing throughput, memory usage, database cost, and recovery. The core model is straightforward: Job → Step → Read → Process → Write → Repeat Jakarta Batch keeps the business pipeline simple while the runtime manages the execution mechanics needed for reliable, long-running data processing. The Sample: Customer Segmentation with Jakarta Batch This example demonstrates the Jakarta Batch model using an e-commerce customer segmentation scenario. Customers are assigned to tiers such as Bronze, Silver, Gold, and Platinum based on configurable spending thresholds. When thresholds change, the application reevaluates the customer base and updates only customers whose classification has changed. The full application includes MongoDB integration, a Jakarta Faces UI, a preview workflow, validation, and supporting services. The complete source code is available at https://github.com/soujava/mongodb-jakarta-batch. This section focuses on the classes directly involved in Jakarta Batch execution. Starting the Batch Job The application initiates the batch process through CustomerSegmentationService. Unlike the reader, processor, and writer, this class is not a batch artifact. Instead, it is an application service that retrieves Jakarta Batch’s JobOperator from BatchRuntime to start and monitor job executions. Java @ApplicationScoped public class CustomerSegmentationService { public static final String JOB_NAME = "customer-segmentation"; private volatile CustomerSegmentationPolicy currentPolicy; // initialization and status methods omitted public long start(CustomerSegmentationPolicy policy) { if (isRunning()) { throw new IllegalStateException( "A customer segmentation batch is already running"); } Properties parameters = new Properties(); parameters.setProperty( CustomerSegmentationPolicy.JOB_PARAMETER, policy.toJson()); long executionId = BatchRuntime.getJobOperator() .start(JOB_NAME, parameters); currentPolicy = policy; return executionId; } public boolean isRunning() { // implementation omitted } } The key API here is JobOperator, which Jakarta Batch provides as the interface for starting, stopping, restarting, and inspecting jobs. In this example, the segmentation policy is serialized into the job parameters to ensure each execution gets the correct business rules. Reading the Input The first batch artifact, CustomerItemReader, extends Jakarta Batch’s AbstractItemReader to implement a chunk-oriented reader. Java @Named("customerItemReader") @Dependent public class CustomerItemReader extends AbstractItemReader { @Inject private CustomerRepository customerRepository; private List<Customer> customers = List.of(); private int nextIndex; @Override public void open(Serializable checkpoint) { try (Stream<Customer> customerStream = customerRepository.findAll()) { customers = customerStream .sorted(Comparator.comparing(Customer::getId)) .toList(); } nextIndex = checkpoint instanceof Integer index ? index : 0; } @Override public Customer readItem() { if (nextIndex >= customers.size()) { return null; } return customers.get(nextIndex++); } @Override public Serializable checkpointInfo() { return nextIndex; } } These methods are part of the Jakarta Batch reader lifecycle defined by AbstractItemReader. open() prepares the reader and accepts a previous checkpoint if available. readItem() provides the next item to the runtime; returning null indicates there is no more input. checkpointInfo() reports the reader’s current position for checkpointing. For simplicity, this sample loads customers into memory. For larger workloads, the implementation might use pagination or a MongoDB cursor without changing the Jakarta Batch model. Processing Each Customer The next artifact implements Jakarta Batch’s ItemProcessor interface. Java @Named("customerTierProcessor") @Dependent public class CustomerTierProcessor implements ItemProcessor { @Inject @BatchProperty( name = CustomerSegmentationPolicy.JOB_PARAMETER) private String thresholdsJson; private CustomerSegmentationPolicy policy; @PostConstruct void initialize() { policy = CustomerSegmentationPolicy.fromJson( thresholdsJson); } @Override public Customer processItem(Object item) { if (!(item instanceof Customer customer)) { throw new IllegalArgumentException( "Expected a Customer item"); } CustomerTier calculatedTier = policy.tierFor(customer.getTotalSpent()); if (calculatedTier == customer.getTier()) { return null; } return Customer.builder() .id(customer.getId()) .name(customer.getName()) .totalSpent(customer.getTotalSpent()) .tier(calculatedTier) .build(); } } Here the Jakarta Batch contract is explicit: ItemProcessor defines processItem(). The runtime calls that method for every item produced by the reader. The processor applies the segmentation rule and either returns the transformed customer or null. Returning null has a specific meaning in Jakarta Batch: the item is filtered and does not continue to the writer. The @BatchProperty is also part of the Batch integration. It receives the thresholds property defined for this job execution, allowing the processor to reconstruct the CustomerSegmentationPolicy before processing begins. Writing the Results The final artifact extends AbstractItemWriter, Jakarta Batch’s base implementation for writing a chunk. Java @Named("customerItemWriter") @Dependent public class CustomerItemWriter extends AbstractItemWriter { @Inject private CustomerRepository customerRepository; @Override public void writeItems(List<Object> items) { List<Customer> customers = items.stream() .map(this::toCustomer) .toList(); customerRepository.saveAll(customers); } private Customer toCustomer(Object item) { if (item instanceof Customer customer) { return customer; } throw new IllegalArgumentException( "Expected a Customer item"); } } writeItems() is defined by the Jakarta Batch writer contract inherited from AbstractItemWriter. Unlike the processor, which receives one item at a time, the writer receives a collection of processed items. In this case, the collection contains only customers whose classification changed, as the processor has already filtered the others. At this point, the Java components of the pipeline are as follows: Plain Text CustomerItemReader extends AbstractItemReader ↓ CustomerTierProcessor implements ItemProcessor ↓ CustomerItemWriter extends AbstractItemWriter These types are what connect the application code to the Jakarta Batch runtime. Connecting the Artifacts With JSL The Java classes define the behavior, but Jakarta Batch requires explicit mapping of the reader, processor, and writer to each job. This orchestration is described in JSL: XML <?xml version="1.0" encoding="UTF-8"?> <job id="customer-segmentation" xmlns="https://jakarta.ee/xml/ns/jakartaee" version="2.0"> <step id="recalculate-customer-tiers"> <chunk item-count="20"> <reader ref="customerItemReader"/> <processor ref="customerTierProcessor"> <properties> <property name="thresholds" value="#{jobParameters['thresholds']}"/> </properties> </processor> <writer ref="customerItemWriter"/> </chunk> </step> </job> The ref values correspond directly to the names declared with @Named in the Java classes: Java @Named("customerItemReader") @Named("customerTierProcessor") @Named("customerItemWriter") The XML therefore tells the Jakarta Batch runtime: for this step, use this reader, then this processor, and finally this writer. It also maps the thresholds job parameter into the processor property. The item-count="20" sets the chunk size for this sample. Jakarta Batch coordinates reading and processing, periodically invoking the writer according to the chunk lifecycle and establishing transaction and checkpoint boundaries. The value 20 is for demonstration; real applications should tune chunk size based on processing cost, database behavior, transaction size, throughput, and recovery requirements. This structure is recommended for the article: present the class declaration first, then describe the lifecycle methods inherited from or required by Jakarta Batch. This approach helps the sample teach the API rather than simply presenting isolated methods. Conclusion Jakarta Batch is valuable because it transforms large-scale data processing into a structured execution model, eliminating the need for custom loops and ad hoc background logic. By separating reading, processing, and writing, and introducing runtime concepts such as jobs, steps, checkpoints, restartability, and chunk-oriented execution, it provides enterprise applications with a predictable approach to handling workloads involving thousands or millions of records. This allows implementations to focus on business logic, while the Batch runtime manages repetitive execution concerns, making the model easier to understand, optimize, and scale as workloads increase.

By Otavio Santana

CORE

The Request Timed Out, But the Payment Succeeded: Building Retry-Safe Mobile APIs

Mobile networks are notoriously unreliable. A common scenario is when a user taps "Pay Now" in an app: the payment request reaches the server and is processed, but the network response never reaches the phone. The client assumes the request failed and retries, leading to the charge running twice. This is precisely the kind of bug that idempotency solves. Idempotency means that repeating the same operation has no additional effect, and the second attempt should recognize it's a duplicate and do nothing new. In practice, mobile engineers must treat payment or order APIs as idempotent by attaching unique operation identifiers to requests and deduplicating them on the backend. With this approach, even if the network drops a response or the user double-taps a button, the user is charged only once. Achieving this involves coordination between the app and the server. On the client side, every payment or mutation request is given a persistent unique ID. For example, the app might generate a new UUID when the user submits a payment, save that operation in a local "outbox" or queue, and include the ID in the HTTP request: Swift let opID = UUID().uuidString var request = URLRequest(url: URL(string: "/api/payments")!) request.httpMethod = "POST" request.setValue(opID, forHTTPHeaderField: "Idempotency-Key") request.httpBody = /* JSON payload of the payment */ This custom header (Idempotency-Key) carries the operation's identity to the server. (Modern APIs often expect this header to ensure that POST is treated safely.) The client's logic must be persistent; before sending, it writes the operation (ID and payload) into local storage (e.g., SQLite or SharedPreferences) so that it can recover and retry if the app closes or the network is down. This pattern, sometimes called the outbox pattern, means the user's intent is recorded immediately. A background dispatcher can then drain the queue on network availability or app restart; it tries each pending request. If a send fails (timeout, no internet, 5xx error), the entry remains in the queue for a later retry. This ensures at-least-once delivery and the server will eventually see the request, even if the app crashes or the network is flaky. But at-least-once alone would cause duplicates, so the server must be ready. When the backend receives the request with its Idempotency-Key, it first checks a deduplication store (for example, a database table keyed by this ID). Java String key = request.getHeader("Idempotency-Key"); PaymentResponse prev = idempotencyStore.lookup(key); if (prev != null) { // We have processed this request before - return the original response return prev; } // No record of this key; proceed with processing PaymentResponse result = processPayment(request.getBody()); // Store the result before returning it idempotencyStore.insert(key, result); return result; If the key already exists, the server simply returns the stored result without charging again. This ensures that the second (or third) time the client re-sends, the user doesn't get double-charged. Stripe's API, for instance, works exactly this way: it saves the outcome of the first request for a given idempotency key, and any retry with the same key returns the same result. In effect, the combination of at-least-once delivery (the client keeps retrying) plus idempotent handling on the server yields an effectively-once outcome. The server's idempotency store can be implemented with a simple database table that records each key and the operation's result. For instance, a processed_payments table might use the idempotency key as a primary key or unique constraint. The service then does an atomic INSERT ... ON CONFLICT DO NOTHING (PostgreSQL syntax) or equivalent. If the insert succeeds, the code proceeds with the payment and stores the result; if it fails because the key already exists, it knows this is a duplicate and can fetch the prior result. Wrapping the insert and the business operation in one database transaction avoids a race condition, as either both the key and payment record are written, or neither is. In SQL terms: SQL BEGIN; INSERT INTO payments(idempotency_key, user_id, amount) VALUES (:key, :userId, :amount) ON CONFLICT (idempotency_key) DO NOTHING; -- Check how many rows were inserted: IF (INSERT was successful) THEN -- This is the first time seeing this key; perform the payment CALL process_payment(...); -- The payment service may record a transaction ID, etc. COMMIT; ELSE -- Key already existed: rollback any partial work ROLLBACK; -- Retrieve and return the original payment result END IF; Even if two identical requests arrive concurrently, the unique constraint ensures only one succeeds in its insert. The other can detect the conflict and simply return the saved response. The system design sandbox guide describes this approach as "The database enforces uniqueness and no separate check needed. This works well when the idempotency record belongs in the same database as the business data, since you can wrap both in a single transaction." On the mobile side, it's also wise to guard against duplicates before the request is even sent. A simple in-memory or on-disk set of "seen" IDs can help reject retry loops after a crash or double tap. In Swift: Swift final class OperationDeduplicator { private var seen: Set<String> = [] func shouldProcess(_ id: String) -> Bool { return seen.insert(id).inserted } } This OperationDeduplicator returns true only the first time an ID appears. Persisting this set across app launches (for example in Core Data or a file) makes the app resilient to a crash after the payment is sent but before the response arrives. On relaunch, the app knows it already handled that operation and won't enqueue it again. It's important to integrate these pieces smoothly. A typical mobile flow might look something like this: the user submits a payment form, the app immediately generates a new opID (a UUID) and creates an operation record { id: opID, payload: {amount, items, ...} }. This record is saved locally. Then a background task picks it up, attaches opID as the Idempotency-Key header, and sends it. If the network call times out, the record stays queued. When the app regains connectivity or restarts, the dispatcher tries again. Because the same opID is used each time, the server knows to treat all retries as one. Only after the server successfully processes the payment does the app remove the operation from its queue. This pattern ensures retries and crashes do not cause duplicate side effects. Some systems even use more granular controls. For example, if the backend involves multiple microservices, one service might call others, and each service should propagate the same idempotency key or a related correlation ID so that the entire transaction remains idempotent. Distributed tracing can help debug how a request flowed through the system. Ultimately, the goal is to capture the entire user action from UI tap through backend processing and make sure it's only applied once globally. This often means also having the backend return the same HTTP status and response body on every retry, so the client never gets an unexpected error. Developers should simulate network failures and verify that retries do not cause double effects. Most important is to observe real production behavior, as logs or traces with the operation ID can tie multiple client attempts to a single transaction. If everything is correct, the system achieves effectively-once behavior where the payment occurs exactly once no matter how many times the client tries. As systemdesignsandbox summarizes, "at-least-once delivery + idempotent consumer = effective exactly-once". In practice, this means mobile apps can assume failures are not fatal and they can safely retry with the same key, knowing the server will protect against duplicates. Conclusion In summary, retry-safe mobile operations require treating each user action as an idempotent transaction. The client must persist a unique operation key and reuse it across retries, while the server must detect previously processed keys and prevent duplicate side effects. Combining durable client operations with server-side idempotency allows payments and other critical transactions to survive timeouts, crashes, and unreliable networks without being executed twice.

By Uthej Mopathi

CORE

How to Verify Response Data in API Testing With Playwright TypeScript

One of the most important parts of API test automation is validating the response body to ensure data integrity. This step plays a key role in functional API testing, as it helps confirm that the API is returning the right data in the expected format. Response body validation isn’t limited to a specific request type; it applies equally to POST, GET, PUT, and PATCH APIs. The same validation approach can be used for any API response to verify the data returned by the service. Playwright offers multiple ways to validate response bodies. In this tutorial, I’ll walk you through these approaches to help you efficiently perform assertions on the response data using best practices. Checkout the previous tutorial blog to learn about Installation, the demo application, and how to send GET API requests with Playwright. How to Verify the Response Structure Response structure checks ensure that an API consistently returns data in the expected format, protecting the contract between backend services and their consumers. They help catch breaking changes early, such as missing or renamed fields, even when the API still returns a successful status code. TypeScript test("GET Order details and perform structure check", async ({ request }) => { const response = await request.get("http://localhost:3004/getOrder/", { params: { user_id: "1", }, failOnStatusCode: true, }); const responseBody = await response.json(); expect(responseBody).toHaveProperty("message"); expect(responseBody).toHaveProperty("orders"); expect(responseBody.orders[0]).toHaveProperty("id"); expect(responseBody.orders[0]).toHaveProperty("product_name"); }); This test focuses on validating the structure of the API response. It validates that the response body contains the expected top-level keys and that each order object includes the required fields. Basic Assertions The basic assertions validate API success and data presence, making them a good first layer of verification before deeper structure or data-level checks. TypeScript test("Get order details and perform basic level verification", async ({ request, }) => { const response = await request.get("http://localhost:3004/getOrder/", { params: { user_id: 1, }, failOnStatusCode: true, }); const responseBody = await response.json(); expect(responseBody.message).toBe("Order found!!"); expect(Array.isArray(responseBody.orders)).toBeTruthy(); expect(responseBody.orders.length).toBeGreaterThan(0); }); This test performs a basic level check to confirm that the endpoint works as expected and returns the expected data in the response. After parsing the response body, the assertions focus on the following essential basic-level checks: TypeScript expect(responseBody.message).toBe("Order found!!"); The above line of code verifies that the API returns the expected message text in the response body. TypeScript expect(Array.isArray(responseBody.orders)).toBeTruthy(); This line of code ensures that the orders field in the response is an array, validating the basic response format. TypeScript expect(responseBody.orders.length).toBeGreaterThan(0); This part of the test confirms that at least one order is returned in the orders array, ensuring the response contains required data. How to Verify Response Data With Details Validating the actual data returned in the response is essential to ensure that the API response contains the correct values. TypeScript test("Get order and verify order details", async ({ request }) => { const response = await request.get("http://localhost:3004/getOrder/", { params: { user_id: "1", }, failOnStatusCode: true, }); const responseBody = await response.json(); const order = responseBody.orders[0]; expect(order.id).not.toBeNull(); expect(order.id).toBeDefined(); expect(order.user_id).toEqual("1"); expect(order.product_id).toEqual("79"); expect(order.product_name).toEqual("5 star 10gm Chocobar"); }); The following code ensures that the response has a valid identifier and it is not missing or empty. TypeScript expect(order.id).not.toBeNull(); expect(order.id).toBeDefined(); This check is required because the API generates the order ID when a new order is created in the system. It ensures that the “id” field has a valid value generated and assigned to it, since this “id” is used to retrieve, update, or delete order data. TypeScript expect(order.user_id).toEqual("1"); expect(order.product_id).toEqual("79"); expect(order.product_name).toEqual("5 star 10gm Chocobar"); These statements assert that the order details are retrieved correctly for the respective request. The “user_id” - “1” was sent in the request, and verifying it in the response, along with the other order details such as “product_id” and “product_name,” ensures that the correct data is returned. How to Verify Response Data by Matching Objects and Arrays Playwright allows response data verification by matching objects and arrays partially within the API response. This approach is useful because it makes tests more flexible and confirms that the API returns the correct data structure and values. TypeScript test("Get order and verify matching object and array", async ({ request }) => { const response = await request.get("http://localhost:3004/getOrder/", { params: { user_id: 1, }, failOnStatusCode: true, }); const responseBody = await response.json(); expect(responseBody).toMatchObject({ message: "Order found!!", orders: expect.arrayContaining([ expect.objectContaining({ product_id: "79", product_name: "5 star 10gm Chocobar", product_amount: 5, qty: 1, tax_amt: 0.5, total_amt: 5.5, }), ]), }); }); In this test, the toMatchObject assertion verifies that the response contains a “message” with the expected value “Order found!!” and an orders array. Within the array, "expect.arrayContaining" ensures that at least one order matches the expected data, while "expect.objectContaining" verifies only the values in the specified fields of that order. Using Best Practices to Perform Assertions Best practices create stable, maintainable API automation tests by combining basic checks with flexible data matching. TypeScript test("Get Order details API test with best practice", async ({ request }) => { const response = await request.get("http://localhost:3004/getOrder/", { params: { user_id: "1", }, failOnStatusCode: true, }); const responseBody = await response.json(); expect(responseBody.message).toBe("Order found!!"); expect(responseBody.orders.length).toBeGreaterThan(0); expect(responseBody.orders).toEqual( expect.arrayContaining([ expect.objectContaining({ id: 1, product_name: "5 star 10gm Chocobar", }), ]) ); }); The test sends a GET request to fetch order details for “user_id”-“1". The use of failOnStatusCode: true ensures the test fails immediately if the API does not return a 2xx status code. The response is then parsed into a JSON object for validation. The assertions are structured in layers: TypeScript expect(responseBody.message).toBe("Order found!!"); This assertion verifies the message text, confirming that the API returns the correct message when an order is found. TypeScript expect(responseBody.orders.length).toBeGreaterThan(0); This statement ensures meaningful data is returned and avoids false positives when the array is empty. TypeScript expect(responseBody.orders).toEqual( expect.arrayContaining([ expect.objectContaining({ id: 1, product_name: "5 star 10gm Chocobar", }), ]) ); The final part of the code performs the final assertion using arrayContaining and objectContaining to verify that at least one order has the expected “id” and “product_name”, without asserting every field. These layered validations improve clarity by verifying structure, data presence, and key data values in sequence. Extracting Data From the Response Extracting data from the API response is a common and widely used pattern in API test automation. It is important in multiple ways, such as reusing the data in further tests for dynamic testing and end-to-end validation. TypeScript test('Get order details and extract the order id', async({request}) => { const response = await request.get("http://localhost:3004/getOrder/", { params: { id: 1, }, failOnStatusCode: true, }); const responseBody = await response.json(); expect(responseBody.message).toBe("Order found!!"); expect(responseBody.orders.length).toBeGreaterThan(0); expect(responseBody.orders).toEqual( expect.arrayContaining([ expect.objectContaining({ id: 1, product_name: "5 star 10gm Chocobar", }), ]) ); const order = responseBody.orders[0]; expect(order.id).not.toBeNull(); const order_id= order.id; console.log(order_id); const product_name = order.product_name console.log(product_name) }); This test sends a GET API request and performs basic validations to ensure the API response is reliable. TypeScript const order = responseBody.orders[0]; expect(order.id).not.toBeNull(); const order_id= order.id; console.log(order_id); The code above extracts the “order_id” from the order object in the response. Before accessing it, an assertion is made to verify that the value is not null. Finally, the value of the order_id is printed in the console. TypeScript const product_name = order.product_name console.log(product_name) Similarly, other values, such as product_name, can also be extracted. Attaching the Response Body to the Playwright Report The Playwright report, by default, shows the steps executed, the number of tests run, pass/fail status, and time taken to run the tests. However, it does not attach the response body to the test report. Attaching the response body to the report improves visibility and makes the test report more informative and transparent. The following code shows how to extract the required metadata and attach it to the Playwright report. TypeScript test("Get order details API and attach the response details to the report", async ({ request, }, testInfo) => { const response = await request.get("http://localhost:3004/getOrder/", { params: { user_id: "1", }, }); expect(response.status()).toBe(200); const status = response.status(); const statusText = response.statusText(); const headers = response.headers(); const body = await response.json(); const fullResponse = { status, statusText, headers, body, }; await testInfo.attach("Full API Response", { body: JSON.stringify(fullResponse, null, 2), contentType: "application/json", }); }); The testInfo is a built-in Playwright fixture and provides utilities to manage and inspect test execution, such as attaching files to reports, updating test timeouts, and identifying the currently running test. The following lines of code extract the response metadata, such as the status code, status text, headers, and response body. TypeScript const status = response.status(); const statusText = response.statusText(); const headers = response.headers(); const body = await response.json(); Next, let’s combine all response details and create a single object containing: Status codeStatus textHeadersResponse body TypeScript const fullResponse = { status, statusText, headers, body, }; Finally, let’s attach these details to the report using the testInfo.attach() method as shown below: TypeScript await testInfo.attach("Full API Response", { body: JSON.stringify(fullResponse, null, 2), contentType: "application/json", }); The testInfo.attach() adds an attachment to the Playwright report. The attach() method has 3 parameters: Name of the attachment: The first parameter is the name, “Full API Response”, that will be shown for the attachment.Body of the attachment: The second parameter is for the body of the attachment. The JSON.stringify(fullResponse, null, 2) has 3 arguments. The first argument converts the fullResponse object into a readable, pretty-formatted JSON. The second argument is the replacer, which is null. It ensures that all properties from the fullResponse object are included as they are, without modifying anything. The third argument controls pretty-printing. Here, “2” means indent nested JSON by 2 spaces.Content type: This parameter ensures that the report treats the attachment as JSON. The following screenshot is generated after the tests are run: Test Execution Running the tests in Playwright is simple and easy. We can run the following command from the terminal: Plain Text npx playwright test To generate the report, the following command can be used: Plain Text npx playwright show-report Summary Playwright provides multiple approaches, including structure checks and matching objects and arrays for verifying response data. The right strategy should be chosen based on your project’s requirements. Based on my experience, combining response structure checks with response data validation, including the matching object and array strategy, can be used as an effective approach for validating API responses. Happy testing!

By Faisal Khatri

CORE

6 Techniques To Reduce LLM API Costs With the Python Library

Before diving into solutions, it helps to understand the scale of the problem. Take a common production pattern: a customer support bot that processes 10,000 messages per day, each with a 2,000-token system prompt and a 200-token user message. Cost breakdown by the numbers: scenariomodeldaily costNo optimizationClaude Opus ($5/1M)$11.00Right model for taskClaude Haiku ($1/1M)$2.20Add prompt cachingHaiku + caching ($0.10/1M cached)$0.42Combined savings96% reduction That's not a benchmark; it's arithmetic. The techniques don't require magic; they require applying what providers already offer. Technique 1: Prompt Caching, Up to 90% Off Repeated Content The Problem Most LLM applications send the same system prompt on every request. If your system prompt is 2,000 tokens and you make 10,000 requests per day, you're paying for 20 million input tokens daily, even though the content never changes. How It Works Anthropic's prompt caching lets you mark content blocks with cache_control. The first request pays full price and writes to cache. Every subsequent request that hits the same cached content pays 10% of the normal input price. Cache entries last 5 minutes and reset on each hit. The key insight: cache the most stable content first. Your base instructions change rarely. Your few-shot examples change occasionally. Your per-request context changes every time. Structure your prompt from most stable to least stable. Python from llm_optimizer import OptimizedClient, build_cached_system_prompt import anthropic client = OptimizedClient(anthropic_client=anthropic.Anthropic()) # Build an optimally structured cached system prompt system = client.build_cached_system( base_instructions=""" You are an expert customer support agent for a SaaS company. You have deep knowledge of our product, billing, and technical issues. Always be empathetic, clear, and solution-focused. [... 1,500 more tokens of stable instructions ...] """, # ← cached after first request — 10% cost on all subsequent calls few_shot_examples=""" Example 1: Billing question → here's how to handle it Example 2: Technical issue → here's the escalation path [... 500 tokens of examples ...] """, # ← also cached separately ) # First call: pays full price, writes to cache response1 = client.complete(messages=[{"role": "user", "content": "How do I cancel?"}], system=system) # Second+ calls: system prompt served from cache at 10% cost response2 = client.complete(messages=[{"role": "user", "content": "Where's my invoice?"}], system=system) What the Library Does llm-optimizer automatically injects cache_control breakpoints at optimal positions, system prompt, few-shot examples, and long conversation history, respecting Anthropic's 4-breakpoint limit. You don't touch the API directly. Savings Calculation Shell 2,000 token system prompt × 10,000 requests/day = 20M tokens/day Without caching: 20M × $3.00/1M (Sonnet) = $60.00/day With caching: 2M × $3.00 + 18M × $0.30 = $11.40/day Savings: $48.60/day = $17,739/year Technique 2: Model Routing, 60% to 80% Off by Using the Right Model The Problem Routing every request to your best model is the most common and most expensive mistake. Claude Opus costs 5x more than Claude Haiku. For tasks that Haiku handles perfectly, such as classification, extraction, translation, and simple Q&A, you're paying a 500% premium for no benefit. The Naive Approach and Why It Fails The obvious solution is to route by keyword: if the prompt contains "classify," use Haiku; if it contains "analyze," use Sonnet. This works until it doesn't. A prompt like "Explain the constitutional implications of this clause" is 8 words. Short, simple-looking. A keyword router sees no complexity signals and routes it to Haiku. But the task requires expert-level legal reasoning. This class of error intent-heavy short prompts is the primary failure mode of heuristic routing. The Better Approach: Use a Classifier llm-optimizer solves this by optionally using Haiku itself to classify task complexity before routing. The cost is approximately 15 tokens, about $0.000015. If that classification prevents one wrong Opus call (2,000 tokens × $5/1M = $0.01), it pays for itself 666 times over. Python from llm_optimizer import OptimizedClient, Provider client = OptimizedClient( anthropic_client=anthropic.Anthropic(), enable_llm_classifier=True, # uses Haiku to assess complexity — ~$0.000015/call preferred_provider=Provider.ANTHROPIC, ) # Short prompt, complex intent → correctly routed to Opus response = client.complete( messages=[{"role": "user", "content": "Explain the constitutional implications of this clause"}] ) # You can audit the routing decision from llm_optimizer import ModelRouter router = ModelRouter(enable_llm_classifier=False) print(router.explain("classify this email as spam or not")) # { # "detected_complexity": "simple", # "routed_model": "claude-haiku-4-5", # "keyword_signals_fired": {"simple": ["classify"]}, # "token_count": 8 # } Complexity Tiers A closer look at complexity: Tierexamplesdefault modelSimpleClassification, extraction, yes/no, translationClaude HaikuMediumSummarization, paraphrasing, short Q&AClaude HaikuComplexCode generation, analysis, evaluationClaude SonnetExpertLegal reasoning, research, system design, math proofsClaude Opus Shell 1,000 requests/day — mixed complexity Without routing: all → Opus ($5/1M input) 1,000 × 500 tokens = 500K tokens × $5 = $2.50/day With routing: 70% Haiku, 20% Sonnet, 10% Opus 700 × 500 × $1 + 200 × 500 × $3 + 100 × 500 × $5 = $1.05/day Savings: 58% reduction Technique 3: Prompt Optimization, 5% to 20% Off Token Count The Problem Prompts written by humans, especially in collaborative or enterprise settings, accumulate filler. Phrases like "please note that", "it is important to note that", "in order to", and "due to the fact that" add tokens without adding meaning. At scale, this is a measurable cost. What the Library Strips Python from llm_optimizer import PromptOptimizer opt = PromptOptimizer() result = opt.optimize(""" In order to complete this task, please note that you should carefully analyze the following text. It is important to note that accuracy matters. Please be aware that your response should be concise. """) print(result.optimized_text) # "To complete this task, carefully analyze the following text. # Accuracy matters. Your response should be concise." print(f"Saved {result.tokens_saved} tokens ({result.savings_pct}%)") # Saved 18 tokens (31%) What is never touched: Code blocks, factual content, user-specified phrasing. The optimizer is conservative by default. It only removes patterns with no semantic value. Conversation history trimming: In long conversations, the library keeps the last N turns and drops older context, preventing unbounded token grow. Technique 4: Batch Processing, 50% Off Non-Urgent Requests The Problem Not every LLM call needs an immediate response. Nightly report generation, document indexing, data enrichment pipelines, and offline classification jobs all of these run fine with a delay. But most teams send them as real-time requests anyway, paying full price. How Anthropic's Batch API Works Anthropic's Message Batch API processes up to 10,000 requests per batch at 50% of the normal price. Results are available within minutes to hours. The trade-off is explicit: cost for latency. Python client = OptimizedClient( anthropic_client=anthropic.Anthropic(), enable_batching=True, ) # Queue 1,000 document summaries throughout the day for doc in documents: client.queue( custom_id=doc["id"], messages=[{"role": "user", "content": f"Summarize: {doc['text']}"}], max_tokens=200, ) # Submit as one batch — 50% cheaper than 1,000 individual calls batch_id = client.submit_batch() # Poll when ready — minutes to hours depending on load results = client.poll_batch(batch_id, wait=True) for r in results: print(f"{r.custom_id}: {r.content}") When to Use It Nightly data processing pipelinesDocument indexing and enrichment Offline classification and tagging Report generation Technique 5: Document Compression to Reduce Context Before Sending The Honest Tradeoff This technique requires a direct warning: document compression is lossy. Removing content from a document to reduce token count means the model works with less information. For some tasks this is fine; for others it produces wrong answers. Use it only when: You've verified empirically that compression doesn't hurt your answer quality.You're doing rough extraction where completeness isn't required.You have re-ranking downstream (e.g., RAG pipelines). Do not use it for legal documents, compliance reviews, or any task where every sentence may be relevant. TF-IDF Extractive Compression When you do use compression, naive truncation (cutting from the end) is the worst strategy. llm-optimizer implements TF-IDF paragraph scoring where each paragraph is scored by its term overlap with your query, weighted by how unique those terms are across the document. The most relevant paragraphs fill the token budget; the rest are dropped. Python from llm_optimizer import DocumentCompressor # ⚠️ Read the accuracy warning before using in production comp = DocumentCompressor( max_tokens=4000, strategy="extractive", # TF-IDF scoring — best accuracy ) compressed, tokens_saved = comp.compress( document=long_contract, # 50,000 tokens query="payment terms and termination clauses" # focus compression here ) print(f"Compressed to {4000} tokens, saved {tokens_saved} tokens") # All compressed output includes a visible [⚠️ COMPRESSION WARNING] marker There are three strategies available: Strategy options: StrategyHow it worksbest forExtractiveTF-IDF scoring against queryWhen you have a specific querySmartKeeps first 60% + last 20%Structured documents with summariesTruncateHard cutoffWhen you need predictable behavior Technique 6: Cost Tracking That Observes Before You Optimize Why This Matters You can't optimize what you don't measure. Before applying any of the above techniques, you need to know: Which models you're actually usingWhere your token spend is goingWhether your optimizations are working Python client = OptimizedClient( anthropic_client=anthropic.Anthropic(), persist_tracking="usage.jsonl", # survives restarts ) # ... run your application ... client.print_summary() # ═══════════════════════════════════════════════════════ # LLM Cost Optimizer — Usage Summary # ═══════════════════════════════════════════════════════ # Total Requests : 1,247 # Total Cost : $0.8432 # Total Saved : $7.2180 (89.5% savings) # Cached Tokens : 8,432,000 # By Model : haiku: 891 reqs ($0.12) | sonnet: 312 ($0.58) # Optimizations : prompt_caching: 1247x | model_routing: 1247x # ═══════════════════════════════════════════════════════ The tracker records every request's tokens, cost, cached tokens, savings, latency, and which optimizations fired. Data persists to JSONL so you can analyze it across sessions or pipe it to your observability stack. Architecture: Why Not Just Use LiteLLM? The obvious question. LiteLLM is excellent and covers a lot of ground, including unified provider API, routing, cost tracking, batch processing. If you're not already using it, you should evaluate it. llm-optimizer does three things LiteLLM doesn't: Automatic cache_control injection: LiteLLM passes caching headers through but doesn't inject breakpoints at optimal positions automatically.Prompt filler stripping: LiteLLM has no token-level prompt optimization.TF-IDF document compression: LiteLLM has no query-aware document compression. The intended use is actually as a complement: llm-optimizer can sit on top of a LiteLLM setup, handling the prompt-level optimizations that LiteLLM doesn't touch. Streaming Support For user-facing applications, the library supports streaming: Python with client.stream( messages=[{"role": "user", "content": "Explain quantum entanglement"}], system="You are a physics tutor.", max_tokens=512, ) as stream: for chunk in stream: print(chunk, end="", flush=True) # Access token usage after stream completes usage = stream.usage() All optimizations, including caching, routing, prompt, and optimization, apply identically to streaming requests. Error Handling Production LLM applications need to handle rate limits and model overloads gracefully. The library handles this automatically: Python client = OptimizedClient( anthropic_client=anthropic.Anthropic(), max_retries=3, # retry on rate limit with exponential backoff retry_base_delay=1.0, # 1s, 2s, 4s ) # Rate limit (429) → retried with backoff # Model overloaded (529) → falls back to next capable model automatically # Non-retriable error → raises immediately Limitations Honest about what this doesn't do yet: Not org-scale validated: v0.4.0 is tested against ai-core Bedrock and Anthropic direct (14 live tests, all passing). Not yet run against production workloads at team scale. The pilot measures this.No async support: complete() and stream() are synchronous. Async support planned for a future release.OpenAI and Google partially tested: Anthropic and AWS Bedrock are the validated providers. OpenAI is implemented but not end-to-end tested in CI. Google Gemini streaming is not yet implemented.Token counting is approximate: The default estimator is within ~20% of the actual count. Install tiktoken for exact counts: pip install llm-optimizer[tiktoken].Pricing data can go stale: Stored in pricing.json with a version stamp. The library warns automatically if data is older than 30 days.Model allowlist: Only models listed in pricing.json can be routed to. Mythos and Fable 5 are not in the registry and cannot be called. Adding a new model requires a deliberate update to pricing.json. Python # Basic install pip install llm-optimizer # With exact token counting pip install llm-optimizer[tiktoken] # All providers pip install llm-optimizer[all] Python import anthropic from llm_optimizer import OptimizedClient client = OptimizedClient( anthropic_client=anthropic.Anthropic(), # All optimizations on by default except compression (lossy — opt-in) ) response = client.complete( messages=[{"role": "user", "content": "Your prompt here"}], system="Your system prompt here", ) client.print_summary() Links: PyPI: https://pypi.org/project/llm-optimizeGitHub: https://github.com/banerjeeso/llm-optimiz What's Next Async support (acomplete(), astream())LiteLLM adapterBudget guard: raise before a request exceeds a cost thresholdReal production benchmarks once I've run this against a live workload. Feedback welcome, especially from anyone who works with LLM APIs in production and can stress-test the routing logic or compression accuracy. Published on PyPI as llm-optimizer. MIT license. Contributions welcome.

By Somnath Banerjee

Building a Secure MCP Server for File Processing: Auth, Rate Limiting, and Idempotency

Most write-ups on building an MCP server focus on the protocol itself: defining tools, handling requests, wiring up a client. That part is genuinely straightforward. What gets skipped over far more often is what changes when the tool you are exposing operates on files rather than returning data. File processing introduces a specific set of security and reliability problems that a typical read-only API does not have to think about, and getting them wrong is easy to miss until something goes badly. This is a rundown of the decisions that mattered most while building an MCP server that exposes document processing tools, merge, convert, OCR, and similar operations, and why a few of the obvious approaches turned out to be the wrong ones. Why File-Processing Tools Are a Different Security Case A typical MCP tool that queries a database or calls a read-only API has a bounded, predictable attack surface. A tool that accepts a file, or worse, a URL pointing to a file, and processes it does not. Two problems show up immediately that a simpler API rarely has to deal with. First, any tool parameter that accepts a URL is a potential SSRF vector. An MCP client could be tricked, directly or through a compromised upstream model response, into passing a URL pointing at an internal service, a cloud metadata endpoint, or an otherwise unreachable internal address. If the server naively fetches whatever URL it is given, that request happens from inside your infrastructure with whatever network access your server has. Treating every incoming URL as untrusted input, resolving it before fetching, and explicitly blocking private IP ranges and metadata endpoints is not optional for a tool like this, it is baseline. Second, file processing is expensive relative to a typical API call. Merging PDFs, running OCR, converting between formats, these all consume real CPU and memory per request in a way that a database lookup does not. That changes how rate limiting needs to work, which is worth its own section below. Auth: Why API Keys Plus JWT, Not Just One or the Other A single long-lived API key is simple to implement and simple to leak. Once issued, it is valid until manually revoked, and if a key ends up in a log file, a committed config, or a client-side integration by accident, there is no time-boxing to limit the damage. The approach that held up better in practice: bcrypt-hashed API keys for the initial authentication step, then a short-lived JWT issued from that exchange for the actual session. The API key never gets passed around on every request, only at the start, and it is never stored in plaintext server-side, so a database compromise does not directly expose usable credentials. The JWT that follows has a real expiry, which bounds how long a leaked token stays useful and gives you a natural mechanism for revocation without needing to invalidate the underlying key. This is not a novel pattern. It is standard practice in plenty of API design. The point worth making is that it is easy to skip for an MCP server specifically, because the tooling and examples in most MCP documentation default to a single static key for simplicity, and that default quietly becomes the shipped implementation if nobody revisits it. Idempotency: The Requirement Everyone Forgets Until It Bites MCP clients retry. Network hiccups, timeouts, a model deciding to re-invoke a tool call, all of these mean the same logical request can arrive at your server more than once. For a read-only tool, that is harmless, you just return the same data twice. For a tool that processes and charges against a file, a duplicate request means duplicate processing, potentially duplicate output files, and depending on your billing model, duplicate charges for a single user action. The fix is an idempotency key attached to each request, generated client-side and checked server-side before any processing begins. If a request with a given idempotency key has already been handled, the server returns the cached result rather than reprocessing. This sounds obvious once stated, but it is very easy to build a working MCP server that passes every test in development, where retries are rare, and only discover the gap once it is handling real, occasionally flaky client connections in production. Rate Limiting That Doesn't Punish Legitimate Use Because file processing is CPU and memory intensive per request, generic per-minute rate limits borrowed from a typical REST API tend to either allow abuse or block legitimate batch workflows, and it is hard to tune a single number that avoids both. Someone processing twenty files in a genuine batch workflow looks identical, from a naive rate limiter's perspective, to a script hammering the endpoint. What worked better was tracking limits per API key with enough granularity to distinguish sustained high-frequency abuse from a legitimate burst of activity, rather than a single flat request-per-minute ceiling applied uniformly. This is a harder problem to get exactly right than it sounds, and it is one area worth revisiting periodically as real usage patterns become clearer, rather than treating the initial configuration as final. Audit Logging as a Design Decision, Not an Afterthought It is tempting to treat logging as something you bolt on once a security question actually comes up. For a tool that processes user files, that is backwards. Knowing which API key touched which file, when, and what operation was performed needs to exist from the first deployment, not added retroactively after an incident makes it obvious it should have been there. This matters for debugging as much as for security, since a surprising number of support questions end up being answerable directly from audit logs rather than requiring back-and-forth with the user. What Would Have Saved Time in Hindsight Two things, if starting over. The first is deciding on the auth pattern, API key exchange plus short-lived JWT versus a single static key, before writing a single tool handler, rather than starting with the simpler static key for speed and migrating later. The migration is not hard technically, but it touches every existing integration and every piece of client documentation, so the cost of delaying the decision is mostly organizational rather than technical. The second is building the idempotency check in from the first tool, rather than adding it once a duplicate-processing report surfaces. It is a small amount of code, a lookup and a cache write around the start of request handling, but retrofitting it means auditing every existing tool for where duplicate execution would actually cause a visible problem versus where it is harmless, which takes longer than just building it in from the start would have. Putting It Together None of these individually are exotic ideas. Short-lived tokens over static keys, treating URL inputs as untrusted, idempotency keys for retryable operations, audit logging from day one, all of these are well-understood patterns in API design generally. What is specific to building an MCP server for file processing is that the combination matters more here than it does for a typical read-only integration, because the failure modes are more expensive: a duplicated file, a leaked key with no expiry, an SSRF hole reachable through a tool parameter, or an untracked operation on a user's document. If you are building or evaluating an MCP server that touches files rather than just data, these are the questions worth asking early, before the first real client connects to it, rather than after.

By Peter Ndumia

When Production Stops Moving: Running Claude Code Across a Distributed Enterprise Integration Team

Enterprise platforms don't fail gracefully. A stalled integration, a malformed payload, a service that silently drops a field — these aren't abstract bugs; they're business processes that stop moving and stakeholders who start calling. Leading a multinational engineering team across two regions that keeps a large platform's web service integrations running, I've spent the last several months evaluating where an agentic coding tool like Claude Code actually earns its place in that world — not as a novelty, but as infrastructure. Here's what that looks like in practice. Development: From Description to Diff The obvious use case is still the most valuable one. Claude Code reads an entire codebase — including the mix of legacy and modern service layers that most enterprise platforms actually run on — and can trace a bug from a symptom description down to the root cause across files it wasn't explicitly pointed at. Describe the integration you want built, and it plans an approach, writes the code, and verifies its own work before handing it back. For a legacy-adjacent stack paired with modern services, this matters more than it would in a greenfield project. Institutional knowledge about why an integration was built a certain way often lives in people's heads, not in comments. A CLAUDE.md file at the project root — read at the start of every session — becomes a place to encode that: architecture decisions, naming conventions, which endpoints are safe to retry and which aren't. Claude also builds its own memory as it works, picking up build commands and debugging patterns without being told to. Support: Turning Log Noise Into Signal Enterprise integration incidents rarely announce themselves cleanly. They show up as a batch job that partially completed, or a spike in errors from a downstream system. Claude Code's headless mode (claude -p) lets you pipe raw log output straight into a triage step: Shell tail -200 integration.log | claude -p "flag anything that looks like a failed payload" That same pattern scripts into runbooks, scheduled checks, or CI jobs — useful when your support rotation spans time zones, and nobody wants to be the one manually grepping logs at 2 a.m. local time. The Distributed Team Problem Leading engineers across multiple regions means standards drift is a constant risk — not from lack of skill, but from lack of shared context. Two teams solving the same class of integration problem several time zones apart will converge on different patterns unless something forces alignment. This is where Claude Code's team-facing features do real work: Skills package a repeatable workflow — a /review-integration command that checks a new service against your platform's conventions — so the process isn't re-explained every time it's needed.GitHub Actions or GitLab CI/CD integration lets Claude respond to @claude mentions on a pull request or run automated code review with severity-tagged findings, which matters when reviewers in one region are asleep while another region is shipping.Managed settings push CLAUDE.md files, permission rules, and MCP server allowlists centrally, so a Tech Lead sets the guardrails once instead of hoping every developer's local config agrees. MCP: The Part That Matters Most for Integration Work If your job is service integrations, the Model Context Protocol (MCP) is arguably more relevant than the coding assistance itself. MCP lets Claude Code connect directly to the systems around your codebase — ticketing tools, internal databases, Slack — so it can pull incident context or check a downstream system's schema as part of diagnosing a fix, instead of an engineer manually assembling that context first. For a platform with a web of service dependencies, that's the difference between "here's a plausible fix" and "here's a fix that accounts for what the downstream service actually expects today." Governance Isn't Optional Here Enterprise data often brings compliance weight that a typical greenfield codebase doesn't carry. Before rolling this out past a pilot, the governance layer needs real attention: Permission modes and sandboxing control what Claude can execute autonomously versus what needs a human in the loop — worth tuning tighter than the defaults for anything touching production data.Data handling and Zero Data Retention options are worth understanding fully if your platform has regulatory constraints on where data can transit.Analytics (on Team/Enterprise plans) gives adoption and PR-attribution data, which is useful less for surveillance and more for showing leadership that a pilot is actually paying off before expanding it. Where to Start Don't roll this out platform-wide on day one. Pick a single integration workstream, write the CLAUDE.md that captures your legacy-plus-modern conventions, and run a two-week pilot with one automated review pipeline (GitHub Actions or GitLab CI/CD, whichever matches where your repos already live). Measure whether incident triage time drops and whether code review turnaround improves across the time-zone gap. Expand from there. The tooling is capable enough now that the harder problem isn't whether it can help — it's building the guardrails and shared conventions that let a distributed enterprise team trust it with production-critical integrations.

By Balaji Venkatasubramaniyar

CORE

Integration

DZone's Featured Integration Resources

Monthly Top Integration Experts

The Latest Integration Topics