Integration Resources

DZone's Featured Integration Resources

The Permission Was Valid. The Action Was Not Authorized.

By Igboanugo David Ugochukwu

CORE

Every incident review has a moment where someone asks how the thing was even possible. For agentic AI, the uncomfortable answer is often that everything worked as designed. The credentials were real. The API accepted them. The identity system confirmed the agent was allowed to call that endpoint. And the organization still ended up with an action nobody wanted. That gap is what Salt Security's State of Agentic AI and API Security 2H 2026 puts numbers on. The report surveys more than 380 security and technology professionals and adds telemetry from Salt Labs, the company's research arm. I read it as a security argument, not a scare statistic, so here is what the data supports and where it needs care. The Headline, Read Carefully The number everyone will quote is 49.8%. That share of respondents confirmed or suspected that an AI agent in their organization took an action in the past 12 months that the organization did not intend, expect, or authorize. Read the fine print and the picture gets more interesting. Only 25.3% report confirmed instances. Another 24.5% suspect it happened but could not prove it. A further 12.3% said they would not currently be able to determine whether it happened at all, and 15.4% did not know. Only 22.5% said it had not happened. So the cautious reading is that one in four organizations has a confirmed case, and a large share cannot rule one out. That second group may be the more important finding, because an organization that cannot answer the question has a different problem from one that answered yes. What These Incidents Actually Look Like Among respondents with a confirmed or suspected event, the report breaks down what the action involved. Respondents could pick several outcomes, so the figures overlap: 53.9% reported an external API call or transaction.49.7% reported a configuration or permission change.41.9% reported modified or deleted data.37.2% reported contact with a customer, partner, or third party.32.5% reported access to data the agent should not have had.Only 9.9% said there was no material impact. These are changes in state, made in real systems. When a chatbot gets something wrong, someone reads a wrong answer. When an agent gets something wrong, money moves, records change, or a customer receives a message. I think this is the line security teams should draw: the risk is no longer in what the model says but in what the system around it is allowed to do. Why Valid Access Is Not the Same as Authorization Authentication tells you who is calling. Permission tells you what that caller may touch. Neither tells you whether this specific action, at this moment, serves what the humans actually asked for. Salt Security's report makes a related point about identity. Only 26.1% of respondents say their agents get dedicated identities with scoped permissions. Another 24.3% use dedicated identities with broad permissions. The rest is a patchwork: 8.1% use shared service accounts, 5.2% use human user credentials, and 7.6% rely on embedded credentials or API keys. About 17.5% say it varies by application. The report is blunt that a dedicated identity is necessary but not sufficient. A clearly labeled agent with too much privilege can still do serious damage. The coverage data sharpens the point. Respondents report dedicated controls for AI agents (56.4%) and APIs (55.9%), but only 26.6% for agent identities and 24.8% for agent tools. Those two are exactly where an agent gains the ability to act. The Chain Nobody Designed Part of why this is hard is architectural. The report finds that 76.8% of organizations have agents that invoke other agents or tools that themselves call APIs. For 20.4% this happens routinely. By the time a request reaches the system that changes something, the original instruction can be several steps behind it. A security control that evaluates each API call in isolation sees a legitimate request from a legitimate caller. It cannot see that the intent behind it drifted three hops ago. This is why the report defines the agentic path as the full sequence from instruction, through the model's reasoning and the tools and servers it invokes, to the API call and the outcome. I find that framing useful because it gives defenders a unit of analysis that matches how the system actually behaves. Can You Reconstruct What Happened? Here the numbers are hard to wave away. Only 12.5% can consistently trace an agent's complete path across all agents, and 38.6% can do so for most. Asked how long it would take to determine every action a specific agent took yesterday, only 15.7% said minutes with automated tooling. 36% said hours of manual log correlation, and 28.5% said days and multiple teams. That means 64.5% are looking at hours or days. Hours matter when the actor is software. An agent can repeat an action, call several services, and change configurations without waiting for anyone's workday to start. Accountability looks somewhat better than visibility: 20.9% can name the responsible human or team for all agents, and 51.2% for most. But as the report notes, knowing whose agent it was does not tell you what it did. Stopping It Is Mostly a Human Job Only 17.5% can automatically stop an agent that begins behaving unexpectedly in production. About 56.1% can stop one manually within minutes. 12% can only take the whole application down, 3.1% cannot stop it, and 11.2% do not know. So 85.6% have some way to intervene, but most of that way runs through a person noticing first. The report's recommendation here is the part I would underline. A kill switch is useful, but it should not be the only thing standing between an odd decision and an irreversible action. The stronger design is preventive and layered: least privilege on the agent identity, explicit policy at the tool and API boundary, step-up human approval for sensitive operations, transaction limits, auditable action records, and fast credential or session revocation. The Connective Layer Is Now an Attack Surface The Model Context Protocol, or MCP, gives AI applications a standard way to discover and use external tools. The report is careful to say the protocol is not inherently unsafe. The concern is that it concentrates powerful capabilities behind an interface an agent can invoke. In the survey, 32.9% confirmed finding an unauthorized, shadow, or misconfigured MCP server in the past year, and 54.3% confirmed or suspected one. Salt Labs also reports detecting MCP-related infrastructure in 21.6% of customer environments in August 2026, up from 16.7% in January. The report itself cautions that the customer population grew between those measurements, so this is a change in detected prevalence rather than growth in a fixed group. The same restraint applies to the attack data. Salt Labs observed attacker activity on agentic connected API paths in 26.3% of the environments where it looked, but calls the analysis experimental, limited, and directional. I would treat it as a signal that these endpoints are already on attackers' radar, not as a market-wide rate. Separately, the broader telemetry covers 5,319,583 API endpoints and 104,339 attacker instances over twelve months, and 95.8% of those attacker instances were authenticated. The report warns against assuming those are agents. The takeaway is narrower and still important: authentication alone does not establish trust. A Framework I Would Use: Sort Actions by Consequence This part is my analysis, not the report's. Do not treat every agent action as one risk class. I would sort them into three: Read. Retrieving information. Often fine to run autonomously with logging.Write. Changing a record, a configuration, or a permission. Needs policy checks tied to the resource and the sensitivity of the change.Destructive or external. Deleting data, moving money, altering access, or contacting a third party. Needs the strongest gate, which may mean human approval or hard transaction limits. Notice that the survey's own breakdown maps onto these tiers. External transactions, permission changes, deletions, and third-party contact are all in the top half of the list. A read-only assistant and a write-capable agent should never share the same trust assumptions just because they share a model. What I Would Do on Monday The report's recommendations point in a consistent direction, and I would compress them into five moves: Build a living inventory that maps agents, MCP servers, tools, APIs, and data flows together, not just a flat list of endpoints.Give each production agent its own non-human identity with narrow scope, a named owner, and credentials that can be revoked quickly.Put policy checks at the action boundary, before the high-impact call executes, instead of relying on detection after the fact.Preserve a correlated trail from the initiating instruction to the final state change, and aim to make reconstruction a minutes-long task.Measure the controls: scoped identity coverage, path trace coverage, time to reconstruct, time to contain, and how quickly unknown MCP servers are discovered. A Note on Sourcing This is a vendor survey. Salt Security sells agentic and API security products, and the report's own methodology states that it is not a fixed respondent panel, so changes from the prior report are directional. Treat the figures as informed signal rather than audited fact. The two executive comments in the release reflect that vendor perspective. Salt CEO Roey Eliyahu said the data shows a gap "already producing real operational consequences." Yaniv Balmas, Salt's Head of Research, said the company's Manus research and the survey are "two views of the same problem." Per the release, Salt Labs showed that a single malicious email could hijack the Manus agentic platform and reach connected accounts, and that the platform's guardrail detected the attack but did not stop it before the malicious code ran. I have not independently verified that research, so I am attributing it to Salt. The Real Question The report closes on a test I find hard to improve: can your organization say what an agent did, why it was allowed, and whether you can stop it? Most cannot yet answer all three. The sensible response is not to slow agents down out of fear. It is to treat their authority as a privileged interface, engineered and monitored like every other one. Source: Salt Security, State of Agentic AI and API Security 2H 2026 More

Batching Strategies: Throughput vs Latency in Practice

By Md Sharique

Production inference is fundamentally a scheduling problem. Once you've addressed serving architecture and memory, the next question isn’t whether the GPU can do the work; it's how to schedule it. That decision begins with understanding the workload: how requests arrive, how long they run, which are latency-sensitive, and which can tolerate delay. This is where many production deployments encounter their next engineering challenge. Serving architecture establishes how an inference system is organized. Memory management determines how efficiently it uses GPU memory. Batching answers a different question: How do you keep expensive GPUs doing useful work while still meeting latency service-level objectives (SLOs)? Production workloads introduce competing objectives that don't exist during experimentation. Production LLM systems serve requests that arrive unpredictably, vary dramatically in length, and compete for the same hardware resources. The engineering challenge isn’t batching requests together; it's deciding how work should be scheduled as those requests enter and leave the system. Production Traffic Is Inherently Uneven Most proof-of-concept environments present an idealized view of inference. Requests arrive at predictable intervals. Prompt lengths are similar. Response lengths are relatively consistent. Production looks nothing like that. Prompt lengths, output lengths, arrival patterns, and latency expectations differ. But production schedulers have to account for all of them simultaneously. That heterogeneity is what transforms batching from a simple grouping exercise into a scheduling problem. One user may ask a simple question that requires only a few tokens. Another may submit a lengthy document that requires a large context window and an extended response. Some requests are interactive and latency-sensitive. Others are background jobs that can tolerate additional delay. Treating all of those requests as if they were identical creates unnecessary inefficiencies. That is why batching strategies that work well during experimentation often become liabilities in production. Static Batching Breaks Under Real Workloads Static batching is straightforward: Collect a group of requests, execute them together, and wait until the batch finishes before beginning the next one. That approach works well for many traditional AI workloads because inputs generally require a similar amount of computation. Autoregressive generation behaves differently. Each request generates tokens at its own pace. Some complete almost immediately. Others continue generating for hundreds or even thousands of decoding iterations. The result is that the shortest requests spend much of their time waiting for the longest one to finish. This is the convoy effect. The GPU is effectively synchronized to the slowest request in the batch. The GPU is occupied, but completed or short-running requests cannot release their resources until the longest request in the batch finishes. As production workloads become more diverse, those idle cycles accumulate. Throughput falls, latency increases, and expensive GPU resources spend more time waiting than performing useful work. Bigger Batches Are Not Always Better The natural response is often to increase batch size. Larger batches generally improve GPU utilization, but they also introduce a tradeoff. Every request must wait until enough additional work arrives to fill the batch, increasing queueing delay before execution even begins. Once execution starts, larger batches can also increase Time Per Output Token (TPOT). A configuration that delivers acceptable TPOT at one batch size may exceed the application's latency SLO as the batch grows. Higher GPU utilization provides little benefit if response times no longer meet application requirements. This is the central balancing act of production inference. Higher throughput often comes at the cost of higher latency. The engineering challenge is determining where that operating point belongs. An internal document-processing pipeline may prioritize maximum throughput because latency is relatively unimportant. A customer-facing assistant may willingly sacrifice some throughput to preserve fast responses. Neither choice is universally correct. The workload determines the objective. The batching strategy should support it. Continuous Batching Changes the Scheduling Model Modern serving systems approach the problem differently. Rather than waiting for an entire batch to complete, continuous batching schedules work at each decode iteration. Finished requests leave immediately. New requests enter immediately. The scheduler is no longer managing batches. It is managing decoding iterations. Requests enter the batch as capacity becomes available. Completed requests leave immediately. New requests replace them without waiting for the entire batch to finish. This seemingly simple change fundamentally alters GPU utilization. Instead of waiting for the longest request to complete, the scheduler continuously replenishes the batch with new work. GPU utilization becomes constrained primarily by queue depth rather than by the slowest request in the system. The scheduler is no longer waiting for an entire batch to complete. It's continuously looking for the next unit of useful work. The objective shifts from building larger batches to keeping the GPU continuously productive. Long Prompts Introduce Another Challenge The prefill phase processes the entire prompt in a single forward pass, while the decode phase generates one token at a time. These phases place very different demands on the hardware. Prefill is compute-bound. Decode is memory-bandwidth-bound. Long prompts can monopolize GPU compute, delaying interactive requests that are already in the decode phase. Chunked prefill addresses this by breaking large prompts into smaller segments that can be interleaved with ongoing decode operations. Rather than allowing one long prefill operation to dominate the GPU, the resource scheduler alternates between prompt processing and token generation, allowing engineers to balance Time to First Token (TTFT) for new requests against Time Per Output Token (TPOT) for active ones. The result is a more balanced system. Time to First Token (TTFT) for new requests remains reasonable while Time Per Output Token (TPOT) for active requests stays within acceptable service-level objectives. Once again, the objective is balance — not maximization. Scheduling Becomes Policy As production systems mature, batching becomes less about grouping requests and more about deciding which work should execute next. Interactive applicationsBackground batch jobsHigh-priority usersLow-priority workloads Each may require different treatment. Some workloads don't require interactive latency at all. If a business process can tolerate scheduled execution, maximizing throughput may be the better engineering decision than minimizing response time. Modern serving systems increasingly make resource scheduling decisions at the token level rather than at the request level. GPU time becomes a shared resource allocated according to policy instead of simple arrival order. That policy also extends to admission control. Available GPU capacity is not the same thing as available latency headroom. Accepting every incoming request may maximize utilization for a short period, but it can also push latency beyond acceptable SLOs for every user in the system. Well-designed serving systems recognize that preserving predictable latency is often more valuable than accepting one additional request. In production, saying "not yet" is frequently better than saying "yes" to everyone. Throughput and Latency Are Negotiated — Not Maximized One theme runs consistently through every batching strategy. There is no universally optimal configuration. The appropriate batch size, scheduling policy, and admission thresholds all depend on the workload and the service-level objectives the application must satisfy. There is no universally optimal configuration because every production deployment is balancing a different set of constraints. A nightly batch pipeline should not be engineered like an interactive coding assistant. Likewise, a customer support chatbot should not inherit the scheduling strategy of an offline document-processing system simply because both happen to use the same foundation model. The model may remain unchanged. The serving behavior should not. The objective isn't maximum throughput. It's the highest sustainable throughput that still satisfies the application's latency SLOs. Looking Ahead Serving architecture organizes the inference system. Memory management ensures efficient GPU memory use. Batching strategies determine how available resources are converted into useful work. The final layer focuses on extracting even more performance from the same hardware. In the next article, we'll examine inference optimization techniques – including quantization, speculative decoding, FlashAttention, and distillation – and how carefully sequencing those optimizations can dramatically reduce cost and latency without changing the underlying model. More

Why AI Agents Need a Shared Enterprise Ontology

By Josephine Eskaline Joyce, Ph.D

CORE

Adding Graph Retrieval to a RAG Pipeline With Python

By Sriharsha Makineni

When RabbitMQ Accepts a Message but Does Not Route It to a Queue

By Vamsi Krishna Gude

Build a Voice Assistant With Gemini Live API

I wanted a Nest Hub that I could actually program. Something on the kitchen wall that shows the time, the weather, my teams, what's cooking and what's playing, and that answers when I talk to it. That is Dash: an open-source ambient display for any tablet or screen in kiosk mode, one config.json, six languages, free data sources, MIT. Most of Dash needs no account at all. The part that does is the one I want to talk about here: the "Hey Dash" voice assistant, which runs on Google's real-time voice model, Gemini 3.8 Live, through the Gemini Live API. If you have ever wanted to build a voice assistant without stitching speech-to-text, an LLM, and text-to-speech together yourself, this is the article. You will see how the Live API works at the protocol level, how Dash keeps the API key off the device, how tool calls turn speech into shopping list items and timers, and what a question costs. Everything below is the real code from the repository, as of October 2026, against gemini-3.8-live. What Dash Is Dash is an Express server plus a Vite and vanilla TypeScript front end, built to be left alone on a wall for weeks. Panels rotate every minute: Home with a "Today" strip (weather warnings, holidays, your teams playing today, reminders, bin day, the cheapest electricity hour in Spain, the moon), Agenda from any iCal link, news, markets with sparklines, Spotify with synced lyrics, a kitchen that suggests six recipes a day and walks you through them one step at a time, and live sports boards that take over the screen when one of your teams plays, with a celebration when they score. Everything on screen also has an HTTP endpoint, so an iOS Shortcut or Home Assistant can add to the shopping list or start a timer. And then there is the voice layer, which is where Gemini comes in. Why the Live API and Not a Pipeline The classic way to build a voice assistant is a pipeline: a wake word engine, a speech-to-text service, an LLM call, a text-to-speech service, and a lot of glue to make the whole thing feel responsive. You can build Dash that way. You will spend most of your time on the glue, and it will still feel like a walkie-talkie. The Gemini Live API collapses that pipeline into one stateful WebSocket. You stream raw microphone audio in, and the model streams spoken audio back, with transcripts of both sides, tool calls in the middle, and Google Search when it needs a current fact. This is what that buys you: ConcernPipeline (STT + LLM + TTS)Gemini Live APILatencyThree sequential network calls per turnOne bidirectional stream, audio starts before the sentence is finishedInterruptionsHand-rolled: detect speech, cancel TTS, re-promptBuilt in: barge-in, interrupted event, playback stopsLanguageConfigure STT and TTS per language, detect yourselfReply in whatever language the user spoke, 70+ languagesTranscriptsTwo separate services, two formatsinputTranscription and outputTranscription on the same socketToolsParse LLM output, dispatch, re-prompt, re-synthesizetoolCall and toolResponse messages inside the sessionCurrent factsYour own search integrationgoogleSearch tool with groundingVoicePick a TTS vendor and voicevoiceName in the session config (Dash ships with Kore) The trade-off is control: you do not get to inspect the text between STT and TTS, because there is no text in between. For a kitchen assistant, that is the right deal. How "Hey Dash" Works Here is the whole shape of it. Three parties: the browser running the display, the Dash server on the same machine, and the Gemini Live API. The two rules that drive the design: The API key never leaves the server. The browser never sees it. For each conversation, the server mints a single-use ephemeral token with the entire session setup locked in (model, voice, system instructions, tools), and the browser connects to Gemini with that token.Audio never touches the server. The browser talks to Gemini directly over WebSocket: 16 kHz PCM up, 24 kHz PCM down. The server's only job during a conversation is to run the tools Gemini asks for. Dash is on the kitchen wall, so a third rule follows from the first two: Gemini is only on the line while you are talking to it. Waiting for the wake word is free and runs in the browser. What You Need Node.js 22+ (Dash uses node --test, native fetch, and the WebSocket global)A Gemini API key from Google AI Studio, in a .env file next to config.jsonA browser with the Web Speech API for the wake word (Edge or Chrome; Firefox has no speech recognition, so you tap the microphone button)A tablet or screen in kiosk mode, with microphone access allowed for http://localhost:3000 Shell git clone https://github.com/xavidop/dash && cd dash npm install cp config.example.json config.json echo "GEMINI_API_KEY=your-key" > .env npm run build && npm start # http://localhost:3000 Then switch on Settings → General → Voice assistant (or set features.assistant to true in config.json). The rest of this article walks through what happens from the moment you say "Dash". Step 1: Wait for the Wake Word Without Paying for It A Live session is billed per minute of audio, so you do not want it open all day. Dash uses the browser's own speech recognition to wait for the wake word, and only opens a Gemini session once it hears it. This is the relevant part of web/src/assistant.ts: TypeScript function startWake() { if (!cfg() || wakeBlocked || phase !== 'off' || wakeOn) return; const SR = (window as any).SpeechRecognition || (window as any).webkitSpeechRecognition; if (!SR) return; startMic(); // keeps the pre-roll buffer filling rec = new SR() as Recognition; rec.lang = getState().config?.locale || defaultLocale(); rec.continuous = true; rec.interimResults = true; const re = wakeRegex(); rec.onresult = (e: any) => { for (let i = e.resultIndex; i < e.results.length; i++) { // Cooking: short commands ("next", "siguiente paso") act at once, without Gemini. if (e.results[i].isFinal && cookingCommand(norm(e.results[i][0].transcript))) return; if (re.test(` ${norm(e.results[i][0].transcript)} `)) { restartDelay = 300; open(true); return; } } }; // ... } Two details matter here. The wake regex is forgiving. A recognizer set to Spanish or German hears "Dash" as "das", "dach" or "dasch". Exact wake words match anywhere; sound-alikes only count after "hey", "oye", "ok" and friends, because "das" is just a word in German: TypeScript function wakeRegex() { const words = (cfg()?.wakeWords || ['dash']).map(norm).map((w) => w.trim()).filter(Boolean); const alike = words.flatMap((w) => (w === 'dash' ? ['dasch', 'dach', 'dache', 'dass', 'dahs', ...(getState().config?.language === 'de' ? [] : ['das'])] : [])); const exact = words.map(esc).join('|'); const loose = alike.map(esc).join('|'); return new RegExp(`(?:^|\\s)(?:${exact})(?:\\s|$)${loose ? `|(?:^|\\s)(?:hey|hei|ei|ey|oye|ok|okay|hola|hallo|ehi|eh)\\s+(?:${loose})(?:\\s|$)` : ''}`); } The microphone keeps a 2.5-second ring buffer in memory. People say "Hey Dash, what's the weather?" in one breath. By the time the recognizer fires, the question is already half spoken. An AudioWorklet converts the mic to 16-bit PCM at 16 kHz in 100 ms chunks, and the last 25 chunks are kept so the whole sentence reaches Gemini when the session opens: TypeScript const IN_RATE = 16000; const CHUNK = 1600; // 100 ms const PREROLL_CHUNKS = 25; // 2.5 s of audio before the wake word fired node.port.onmessage = (e) => { ring.push(e.data); if (ring.length > PREROLL_CHUNKS) ring.shift(); onChunk?.(e.data); }; Nothing in that buffer leaves the device before the wake word. What the browser's own speech service hears while waiting is a different matter, and Dash is explicit about it in docs/voice-assistant.md: in Edge and Chrome, recognition runs on the browser vendor's online service, not on Gemini, and Dash stores none of it. Cooking mode adds a shortcut: while a recipe is open, "next", "back", "repeat", "start the timer," and "stop cooking" (in each of the six languages) are matched locally and act instantly, without opening a Gemini session at all. Step 2: Mint an Ephemeral Token on the Server This is the part of the Live API that most examples skip, and the one that makes a client-side connection safe. Instead of handing the browser your API key, your backend calls the token endpoint and gets back a short-lived credential that is locked to a full session setup. The browser can connect with it, but it cannot change the model, the voice, the instructions, or the tools. server/services/assistant.js: JavaScript const API = 'https://generativelanguage.googleapis.com/v1beta'; export const LIVE_WS = 'wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContentConstrained'; /** Mints a single-use token valid for 30 min (a new session must start within 1 min). */ async session() { if (!this.enabled) throw Object.assign(new Error('assistant disabled'), { status: 404 }); if (Date.now() - this.lastSession < 2000) throw Object.assign(new Error('too many sessions'), { status: 429 }); this.lastSession = Date.now(); const iso = (min) => new Date(Date.now() + min * 60e3).toISOString(); const res = await fetch(`${API}/auth_tokens`, { method: 'POST', headers: { 'x-goog-api-key': this.key, 'content-type': 'application/json' }, body: JSON.stringify({ uses: 1, expireTime: iso(30), newSessionExpireTime: iso(1), bidiGenerateContentSetup: this.setup() }), signal: AbortSignal.timeout(10000), }); const d = await res.json().catch(() => ({})); if (!res.ok || !d.name) throw Object.assign(new Error(d.error?.message || `token request failed (HTTP ${res.status})`), { status: 502 }); return { url: `${LIVE_WS}?access_token=${encodeURIComponent(d.name)}`, idleSeconds: this.config.assistant.idleSeconds }; } The fields, one by one: uses: 1: the token starts exactly one session.expireTime: how long messages can flow over a connection opened with it (30 minutes here; the API's default is also 30).newSessionExpireTime: how long you have to open that session (1 minute; again the default). The browser requests a token and connects immediately, so a minute is plenty.bidiGenerateContentSetup: the whole setup message the client would normally send on the WebSocket, baked into the token. In the SDKs this is called liveConnectConstraints; on the REST endpoint it is bidiGenerateContentSetup. The response is tiny, and it is all the browser ever gets from the server: JSON { "name": "auth_tokens/<redacted>" } Two things follow from this. First, the browser connects to the constrained WebSocket endpoint, BidiGenerateContentConstrained, with the token as access_token. That endpoint ignores any setup the client sends, so the browser cannot repurpose the token for something else. Second, the token endpoint is the only place that sees the API key, and Dash puts a guard in front of it: only the display itself (requests from localhost) may ask for a token unless assistant.allowRemote is set, so other devices on your Wi-Fi cannot spend your quota. The setup locked into the token is where the assistant gets its personality: JavaScript setup() { const a = this.config.assistant; return { model: `models/${a.model}`, // gemini-3.8-live generationConfig: { responseModalities: ['AUDIO'], speechConfig: { voiceConfig: { prebuiltVoiceConfig: { voiceName: a.voice } } }, // Kore }, systemInstruction: { parts: [{ text: [ /* persona, rules, and this.context() */ ].join('\n') }] }, tools: this.spotify?.enabled && this.spotify.connected ? [TOOLS[0], { functionDeclarations: [...TOOLS[1].functionDeclarations, ...SPOTIFY_TOOLS] }] : TOOLS, inputAudioTranscription: {}, outputAudioTranscription: {}, }; } responseModalities: ['AUDIO'] is the only modality Gemini 3.8 Live speaks on this path; the two empty transcription objects ask for text transcripts of both sides, which is what the on-screen chat is built from. Note how the Spotify tools are only declared when Spotify is actually connected: fewer tools means fewer wrong tool picks. Step 3: Open the WebSocket and Stream Audio Back in the browser, the session is one WebSocket and a handful of message types. This is the core of open() in web/src/assistant.ts: TypeScript const res = await fetch('/api/assistant/session', { method: 'POST' }); const d = await res.json(); ws = new WebSocket(d.url); ws.onopen = () => ws?.send(JSON.stringify({ setup: {} })); The setup message is empty on purpose: everything is already in the token. Gemini answers with setupComplete, and from then on the browser pushes audio: TypeScript const sendAudio = (pcm: ArrayBuffer) => ws?.readyState === WebSocket.OPEN && ws.send(JSON.stringify({ realtimeInput: { audio: { data: b64(pcm), mimeType: `audio/pcm;rate=${IN_RATE}` } } })); if (m.setupComplete) { setPhase('listening'); if (prompt) ws?.send(JSON.stringify({ realtimeInput: { text: prompt } })); // "[briefing]" for (const c of pending) sendAudio(c); // the 2.5 s pre-roll onChunk = (pcm) => { if (!isSpeaking()) sendAudio(pcm); }; maxTimer = window.setTimeout(close, MAX_SESSION_MS); armIdle(); return; } Three things worth noticing: The pre-roll goes first. Those 25 buffered chunks are flushed before live audio starts, so the question you began before the wake word fired arrives intact.A text prompt can open the conversation. The scheduled morning briefing sends [briefing] as realtimeInput.text, and the system instruction tells the model what a briefing is. Same session, same voice, no audio needed to start it.Half-duplex. onChunk drops microphone audio while Dash is speaking. A wall tablet plays through speakers, and without reliable echo cancellation, the model would hear itself. You can still interrupt: tap the card and playback stops. The server's side of the conversation is a few message shapes, all handled in one onmessage: TypeScript if (m.toolCall) return runTools(m.toolCall.functionCalls || []); if (m.goAway) return close(); const sc = m.serverContent; if (!sc) return; if (sc.inputTranscription?.text) say('user', sc.inputTranscription.text); // what you said for (const p of sc.modelTurn?.parts || []) { if (p.inlineData?.data) play(p.inlineData.data); // 24 kHz PCM, base64 } if (sc.outputTranscription?.text) say('dash', sc.outputTranscription.text); // what Dash said if (sc.interrupted) stopPlayback(); if (sc.turnComplete) { /* arm the idle timer, or close if the model said goodbye */ } MessageMeaningsetupCompleteSession is ready; start sending realtimeInputserverContent.inputTranscriptionStreaming transcript of the user's speechserverContent.modelTurn.parts[].inlineDataChunks of 16-bit PCM audio at 24 kHz, base64serverContent.outputTranscriptionStreaming transcript of what the model is sayingserverContent.interruptedThe user barged in; stop playing what you have queuedserverContent.turnCompleteThe model finished its turntoolCall.functionCalls[]The model wants you to run functions and reply with toolResponsegoAwayThe server is about to close the connection Playback is plain Web Audio: decode the base64 into Int16, scale to floats, schedule each buffer right after the previous one on a 24 kHz AudioContext. The playHead bookkeeping is what lets Dash know how long it will still be speaking, which drives both the half-duplex gate and the idle timer. TypeScript function play(b64: string) { outCtx ??= new AudioContext({ sampleRate: OUT_RATE }); const bin = atob(b64); const pcm = new Int16Array(bin.length / 2); for (let i = 0; i < pcm.length; i++) pcm[i] = (bin.charCodeAt(i * 2) | (bin.charCodeAt(i * 2 + 1) << 8)) << 16 >> 16; const buf = outCtx.createBuffer(1, pcm.length, OUT_RATE); const ch = buf.getChannelData(0); for (let i = 0; i < pcm.length; i++) ch[i] = pcm[i] / 0x8000; const src = outCtx.createBufferSource(); src.buffer = buf; src.connect(outCtx.destination); playHead = Math.max(playHead, outCtx.currentTime + 0.05); src.start(playHead); playHead += buf.duration; speakingUntil = Date.now() + (playHead - outCtx.currentTime) * 1000; } Step 4: Let Gemini Act Through Tools A voice assistant that can only talk is a radio. The useful part is that Gemini can change what is on the screen. Dash declares its tools once, in the setup locked into the token, and runs them on the server. The declarations are plain function schemas, plus Google Search as a built-in tool: JavaScript const S = (description) => ({ type: 'STRING', description }); export const TOOLS = [ { googleSearch: {} }, { functionDeclarations: [ { name: 'add_to_shopping_list', description: 'Add one or more items to the household shopping list shown on the display.', parameters: { type: 'OBJECT', properties: { items: { type: 'ARRAY', items: S('One item, short, e.g. "milk" or "2 kg of oranges"') } }, required: ['items'] }, }, { name: 'start_timer', description: 'Start a kitchen timer on the display: for a duration ("10 minutes") or until a clock time ("at 18:30", an alarm). Up to 6 at once.', parameters: { type: 'OBJECT', properties: { minutes: { type: 'NUMBER', description: 'Duration in minutes (may be fractional). Omit when `at` is given.' }, at: S('Local clock time "HH:MM" to ring at (today, or tomorrow if already past). Omit when `minutes` is given.'), label: S('Optional label, e.g. "pasta"'), }, }, }, { name: 'remember', description: 'Save a fact the household wants Dash to remember (people, phone numbers, codes, preferences, where things are). Store it as one self-contained sentence.', parameters: { type: 'OBJECT', properties: { fact: S('E.g. "The plumber is Juan, phone 600 123 456"') }, required: ['fact'] }, }, { name: 'start_recipe', description: 'Open cooking mode on the display with a recipe: big step-by-step instructions the user follows while cooking. ... Every step that takes time must say how long in minutes, with a number ("cook the spaghetti for 9 minutes"): the display turns those into one-tap timers. Never write "until al dente" without a time.', parameters: { type: 'OBJECT', properties: { title: S('Dish name'), servings: S('E.g. "4 servings"'), ingredients: { type: 'ARRAY', items: S('One ingredient with quantity') }, steps: { type: 'ARRAY', items: S('One short step, with its time in minutes when it takes time') }, }, required: ['title', 'ingredients', 'steps'], }, }, { name: 'end_conversation', description: 'Close the assistant when the user is done (e.g. "thanks", "that is all"). Say a short goodbye first.' }, // ... remove_from_shopping_list, get_shopping_list, add_note, list_timers, add_time_to_timer, // cancel_timer, forget, open_saved_recipe, recipe_step, start_step_timer, stop_cooking ], }, ]; The full list is eighteen functions, plus three Spotify ones when it is connected. The start_recipe description is my favorite: it is a prompt disguised as a schema. Telling the model that every timed step needs a number of minutes is what makes the "Timer 8 min" button in cooking mode appear by itself. When the model decides to act, the browser receives toolCall, forwards each call to the server, and sends the results back as toolResponse: TypeScript async function runTools(calls: { id: string; name: string; args?: Record<string, unknown> }[]) { setPhase('thinking'); const functionResponses = []; for (const c of calls) { let response: unknown; try { response = await (await fetch('/api/assistant/tool', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ name: c.name, args: c.args || {} }), })).json(); } catch { response = { error: 'tool failed' }; } if (c.name === 'end_conversation') ending = true; const done = actionText(c.name, c.args || {}, response as Record<string, unknown>); if (done) log.push({ role: 'action', text: done }); // the green receipt on screen functionResponses.push({ id: c.id, name: c.name, response }); } ws?.send(JSON.stringify({ toolResponse: { functionResponses } })); } On the server, tool() is a switch over the names, working against the local JSON database and the timers service. The responses are deliberately rich: add_to_shopping_list returns both what was added and the full list afterward, so the model can say "milk is already on it" instead of pretending it added a duplicate: JavaScript case 'add_to_shopping_list': { const added = this.db.addShopping(args.items); this.onChange('shopping'); return { added: added.map((x) => x.text), list: this.db.shopping().filter((x) => !x.done).map((x) => x.text) }; } case 'start_timer': { let seconds = Math.round(Number(args.minutes) * 60); const at = String(args.at || '').match(/^(\d{1,2}):(\d{2})$/); if (at) { const today = new Date().toLocaleDateString('en-CA', { timeZone: this.config.timezone }); let when = localToEpoch(`${today}T${at[1].padStart(2, '0')}:${at[2]}`, this.config.timezone); if (when != null && when <= Date.now()) when += 864e5; seconds = when == null ? NaN : Math.ceil((when - Date.now()) / 1000); // ring on the minute, never early } if (!(seconds >= 5 && seconds <= 24 * 3600)) return { error: 'give minutes (up to 1440) or a clock time "HH:MM"' }; const tm = this.timers.add(seconds, String(args.label || '').slice(0, 40)); return { started: true, minutes: Math.round(seconds / 6) / 10, endsAt: tm.endsAt, label: tm.label || null }; } Errors go back to the model as data, not as exceptions. "Which timer? say its label" is a tool response, and the model turns it into a question. One Gemini 3.8 Live detail you should know: function calls are asynchronous by default (NON_BLOCKING). The model can keep talking while a tool runs, and schedule the result for when it is idle or interrupt itself with it. For Dash's tools, which take milliseconds, the difference is invisible; for a slow tool you would set BLOCKING on the declaration. Step 5: Give Gemini the Household Context The system instruction is rebuilt for every session, and the second half of it is a snapshot of what Dash already knows. That is why "when does Arsenal play next?" is answered from the display's own data instead of a web search, and why "add it to the list" works when there is only one list. JavaScript /** What Dash knows right now, so "when do Valencia play?" doesn't need a web search. */ context() { const tz = this.config.timezone; const lines = [`Now: ${now.toLocaleString('en-GB', { timeZone: tz, dateStyle: 'full', timeStyle: 'short' })} (${tz}).`]; const w = this.store.get('weather')?.data; if (w) lines.push(`Weather now: ${w.current.temp}°C, ${w.current.desc}; today ${w.today.min}–${w.today.max}°C, rain chance ${w.today.precip}%.`); for (const c of teams.slice(0, 10)) { /* last result and next game per followed team */ } if (events.length) lines.push(`Calendar: ${events.map((e) => `${e.title || 'event'} (${e.allDay ? e.start : fmt(e.start)})`).join('; ')}.`); if (alerts.length) lines.push(`Weather warnings: ...`); if (hol?.today) lines.push(`Today is a public holiday: ${hol.today.name}.`); if (reminders.length) lines.push(`Reminders today: ...`); if (bins.length) lines.push(`Bins: ${bins.join('; ')}.`); if (news.length) lines.push(`Top headlines: ${news.map((n) => n.title).join(' | ')}.`); const cook = this.cooking?.current(); if (cook) { lines.push(`COOKING MODE is open: "${cook.title}" (${cook.servings || 'servings not given'}), on step ${cook.step + 1} of ${cook.steps.length}: "${cook.steps[cook.step].text}".`); lines.push(`Ingredients: ${cook.ingredients.join('; ')}.`); } if (music?.item) lines.push(`Spotify: ${music.playing ? 'playing' : 'paused'} "${music.item.name}" by ${music.item.by}...`); const memories = this.db.memories(); if (memories.length) lines.push(`Things the household asked you to remember:\n${memories.map((m) => `- ${m.text}`).join('\n')}`); lines.push(shopping.length ? `Shopping list: ${shopping.map((x) => x.text).join(', ')}.` : 'The shopping list is empty.'); if (timers.length) lines.push(`Running timers: ...`); return lines.join('\n'); } The memories deserve a word. "Remember that the plumber is Juan, 600 123 456" calls the remember tool, which stores one sentence locally. Every later session gets those sentences in the context, and the instruction tells the model to quote numbers and codes exactly. They live in Dash's own data/db.json, and Settings lists them so you can delete any. It is a simple mechanism, and it is enough: a kitchen does not need a vector database. The persona rules are short and shaped by the fact that the output is spoken: no lists, no markdown, no URLs, one or two sentences, answer first and then maybe a follow-up, never claim a tool ran without calling it, and reply in the language the user is speaking. Step 6: End the Conversation on Purpose A per-minute API needs a clear end. Dash closes a session in four ways: Silence. After the model finishes a turn, an idle timer waits for Dash to finish speaking plus idleSeconds (8 by default). If nothing was heard since, the session closes."Thanks." The end_conversation tool exists so the model can hang up gracefully: say goodbye, call the tool, and the browser closes once the goodbye has played.Five minutes. A hard cap, MAX_SESSION_MS, so a kitchen full of chatter cannot keep the meter running.goAway. When Gemini announces it is about to drop the connection, Dash closes cleanly instead of waiting for the error. TypeScript function armIdle() { clearTimeout(idleTimer); // Wait for Dash to finish talking, then give the user a few seconds to follow up. const wait = Math.max(0, speakingUntil - Date.now()) + (cfg()?.idleSeconds ?? 8) * 1000; idleTimer = window.setTimeout(() => (ending || !heardSinceTurn ? close() : armIdle()), wait); } Closing plays a short down-chime (two oscillators, no audio file), stops playback, restarts the wake-word recognizer a second and a half later, and tells the server to restore the Spotify volume. While the assistant is open, Dash lowers the music ("ducking") so the model does not have to shout over it. What Gemini 3.8 Live Changes Dash ships pinned to gemini-3.8-live, the stable Live model Google updated in September 2026. If you are coming from gemini-3.1-flash-live-preview, these are the differences that affect a voice assistant: Gemini 3.8 LiveInputs / outputsText, image, audio, video in; text and audio outContext131,072 input tokens, 65,536 output tokensFunction callingSupported, asynchronous by default (NON_BLOCKING); BLOCKING, SILENT, WHEN_IDLE, INTERRUPTED scheduling availableGoogle Search groundingSupported, 5,000 free requests a month shared across Gemini 3+ modelsThinkingInterleaved reasoning built in; thinking_level is not a setup option any moreProactive audioAlways on (the model decides when a sound is addressed to it); setting it to false is an errorAffective dialogRemoved from the APITranscriptionInput and output transcription, opt in per sessionVariantgemini-3.8-live-extended-thinking for multi-step reasoning alongside speech Two of those matter for Dash's design. Proactive audio being always on is why the pre-roll buffer and the half-duplex gate are enough: the model is already good at ignoring background noise and its own echo. And asynchronous function calling is why the "thinking" phase on screen is so short: the model does not stall while a tool runs. What It Costs and Where Your Audio Goes Gemini is on the line only during a conversation, and this is what each state costs: StateWhat runsGemini costWaiting for "Dash"The browser's speech recognitionnoneTalkingA live Gemini sessionper minuteClosedAfter 8 s of silence, on "thanks" or ✕, or after 5 minutes at mostnone On the paid tier, gemini-3.8-live audio is priced at $0.005 per minute in and $0.018 per minute out, with the first 5,000 Google searches a month included. A typical question is a few seconds each way, around half a cent. Twenty questions a day comes to roughly $3 a month. The free tier costs nothing, with rate limits, and Google may use free-tier data to improve its products, which is the main reason to pay. On privacy, the three facts you need are the three design rules from the architecture section: the key stays on the server, audio goes from the browser straight to Gemini only during a session, and the wake word runs on the browser's speech service. Memories are stored locally and sent to Gemini as context in each conversation, so do not store anything you would not say to it. Gemini Beyond the Voice: Recipes and Translation The Live API is the headline, but Dash uses plain generateContent on gemini-3.8-flash in two quieter places, both in the kitchen. Any recipe link, turned into steps. Most recipe sites embed a schema.org Recipe card, and that is what Dash parses first. For pages without one, Gemini reads the page text and returns the same shape, with structured output enforced by a response schema: JavaScript async function recipeWithGemini(text, url, { key, model, language }) { const res = await fetch(`https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent`, { method: 'POST', headers: { 'x-goog-api-key': key, 'content-type': 'application/json' }, signal: AbortSignal.timeout(40000), body: JSON.stringify({ contents: [{ parts: [{ text: `Extract the recipe from this web page. Keep the original language (${language} if unclear). Steps are short and in order. If there is no recipe, return an empty title.\n\nURL: ${url}\n\n${text.slice(0, 30000)}` }] }], generationConfig: { responseMimeType: 'application/json', responseSchema: { type: 'OBJECT', properties: { title: { type: 'STRING' }, servings: { type: 'STRING' }, ingredients: { type: 'ARRAY', items: { type: 'STRING' } }, steps: { type: 'ARRAY', items: { type: 'STRING' } }, }, required: ['title', 'ingredients', 'steps'], }, }, }), }); const d = await res.json().catch(() => ({})); if (!res.ok) throw new Error(d.error?.message || `Gemini HTTP ${res.status}`); return normalizeRecipe(JSON.parse(d.candidates?.[0]?.content?.parts?.[0]?.text || '{}'), url); } "What to cook" in your language. The daily ideas come from TheMealDB, which is English only. With a key set and the display in another language, titles and full recipes are translated in one structured call, with the same responseSchema trick. No SDK in either case. Dash's whole Gemini integration is fetch against the REST API and one WebSocket, which is a good reminder of how small the surface actually is. Turning It On The assistant is a few lines of configuration on top of a running Dash: JSON "features": { "assistant": true }, "assistant": { "model": "gemini-3.8-live", "voice": "Kore", "wakeWords": ["dash"], "idleSeconds": 8, "allowRemote": false }, "briefing": { "time": "07:30", "weekdays": [1, 2, 3, 4, 5] } Any Gemini prebuilt voice works in voice (Kore, Puck, Charon, and the rest). wakeWords can be anything the browser's recognizer can hear reliably. briefing.time opens a session by itself at that time and asks for the morning summary: weather, calendar, your teams, bins, one headline. On a Windows kiosk, windows/edge-kiosk-policy.reg allows the microphone for http://localhost:3000 without a prompt. Then say "Dash", or "Hey Dash, what's the weather?" in one breath, or tap the microphone button at the bottom left. A blue dot on that button means the wake word is listening. Conclusion The Gemini Live API turns a voice assistant from a systems-integration project into an application project. The protocol is small: one token request on your server, one WebSocket in the client, PCM in, PCM out, and toolCall and toolResponse for everything that changes the world. Gemini 3.8 Live adds the pieces that used to be the hard part, interruptions, language detection, asynchronous tools and search grounding, and leaves you with the decisions that are actually yours: when to open a session, what the model is allowed to do, and what it knows about your house. Dash makes those decisions one way: free wake word in the browser, ephemeral tokens with the setup locked in, tools on the server, memories on disk, and a session that ends when you say thanks. The code is at github.com/xavidop/dash. Put it on a tablet, add your teams, and talk to your kitchen. Further Reading Dash on GitHub, and the voice assistant doc with everything it can doGemini Live API overviewEphemeral tokens for the Live APIGemini 3.8 Live model page, including the migration notes from the 3.1 previewGemini API pricingDash architecture and HTTP API Last updated: October 2026. Model versions referenced: gemini-3.8-live for the assistant, gemini-3.8-flash for recipes. Dash at commit 60e909f.

By Xavier Portilla Edo

CORE

Spring AI on Local LLMs

Why Running It Locally The usual reason given is privacy, and it’s a real one. A prompt sent to a local model never leaves the machine, which changes the conversation with your security people from a negotiation into a non-event. But three other reasons matter more day to day. Cost goes to zero at the margin. Iterating on a prompt means running it fifty times. Against a hosted API, it's a line item and a rate limit. Against a local model, it’s free.Tests stop depending on the Internet. A build that reaches a hosted endpoint is a build that fails when someone modifies a key or a region has a bad afternoon.You learn where the abstraction leaks. Spring AI presents one ChatClient over every provider. Running a small local model is the fastest way to find out which parts of that promise hold and which parts quietly assume you are talking to a frontier model. The honest cost is quality and speed. An 8-billion-parameter model on a laptop isn’t a frontier model, and the numbers later in this article are not flattering. That trade is fine for development, prompt iteration, and tests, but it’s a decision to deliberately make for production. The Starting Point The initial set-up is the following: Java 25 (Temurin)Spring Boot 4.1.1Spring AI 2.0.1Local LLM runtime and model (details in the next section) Additionally, a freshly generated Spring Boot project, the state you get from start.spring.io before anything interesting has happened. Web MVC, Actuator, DevTools, their test companions and the Spring AI BOM: XML <properties> <java.version>25</java.version> <spring-ai.version>2.0.1</spring-ai.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-webmvc</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-webmvc-test</artifactId> <scope>test</scope> </dependency> </dependencies> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-bom</artifactId> <version>${spring-ai.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> One note on the parent: Spring AI 2.0 is built against Spring Boot 4.x and Spring Framework 7. If you are coming from Spring AI 1.x on Boot 3, this is not a version bump you can do in isolation; package names moved on both sides. Getting a Model onto the Machine In this article, Ollama is the choice: it downloads models, manages their lifecycle, and serves them over HTTP on port 11434. Let’s take it step-by-step. Install Ollama The Windows installer is a one-liner in PowerShell: PowerShell irm https://ollama.com/install.ps1 | iex # >>> Downloading Ollama for Windows... # ######################################## 100.0% # >>> Installing Ollama... # >>> Install complete. Run 'ollama' from the command line. Run Ollama the First Time ollama run pulls the model if it is missing, loads it, and takes you into an interactive prompt. The daemon keeps serving on port 11434 after you leave it: PowerShell ollama run llama3.1 # pulling manifest # pulling 667b0c1932bc: 100% ▕████████████████████████████████████████▏ 4.9 GB # pulling 0ba8f0e314b4: 100% ▕████████████████████████████████████████▏ 12 KB # pulling 455f34728c9b: 100% ▕████████████████████████████████████████▏ 487 B # verifying sha256 digest # writing manifest # success The one-line Windows command below confirms the model is present: PowerShell ollama list # NAME ID SIZE MODIFIED # llama3.1:latest 46e0c10c039e 4.9 GB 12 days ago The detail that matters for Spring AI is that Ollama serves two APIs at once. Its native API lives under /api and an OpenAI-compatible one lives under /v1. Prove the second is alive before you write any Java, as it saves you from debugging a Spring context when the problem is a daemon: PowerShell curl http://localhost:11434/v1/models # {"object":"list","data":[{"id":"llama3.1:latest","object":"model","created":1788341287,"owned_by":"library"}]} Two Ways to Wire It Up Spring AI ships both a dedicated Ollama starter and an OpenAI one that will successfully talk to anything speaking the OpenAI protocol. Both work against the same daemon. Putting both on the classpath and choosing between them with a property is a good way to feel the difference: XML <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-starter-model-openai</artifactId> </dependency> <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-starter-model-ollama</artifactId> </dependency> Two starters on the classpath would normally mean two competing ChatModel beans. Spring AI resolves that with spring.ai.model.chat property, which names the provider that is currently chosen. The other starter’s auto-configuration backs off. Properties files # Specifies the used starter spring.ai.model.chat=ollama # Native Ollama starter (active when spring.ai.model.chat=ollama) spring.ai.ollama.base-url=http://localhost:11434 spring.ai.ollama.chat.model=llama3.1 spring.ai.ollama.chat.temperature=0.7 # OpenAI-compatible route (active when spring.ai.model.chat=openai) # Ollama does not check the API key, but the starter requires one to be set. spring.ai.openai.api-key=ollama # The /v1 must be part of the base-url: the underlying openai-java client # appends only /chat/completions, so omitting it yields a 404. spring.ai.openai.base-url=http://localhost:11434/v1 spring.ai.openai.chat.model=llama3.1 spring.ai.openai.chat.temperature=0.7 A note is worth making on spring.ai.openai.api-key=ollama. Ollama ignores the value entirely, but the starter refuses to configure itself without one, so it needs a placeholder. Any non-empty string does the job. Another aspect worth noting is the base URL needs its own /v1. The natural thing to write is spring.ai.openai.base-url=http://localhost:11434, on the assumption that the client appends the full /v1/chat/completions path. It does not. Spring AI 2.0 delegates to the official openai-java SDK (4.49.0 here), which appends only /chat/completions. Get this wrong and every call comes back as: Plain Text com.openai.errors.NotFoundException: 404: 404 No hostname, no path, no hint, just the status code twice. The fix is to make /v1 part of the base URL, as above. The same applies to any other OpenAI-compatible server you point this at. Flipping to the OpenAI-compatible route needs no code change at all: PowerShell ./mvnw spring-boot:run -Dspring-boot.run.arguments=--spring.ai.model.chat=openai Both spring.ai.ollama.chat.temperature and spring.ai.ollama.chat.options.temperature are valid, although the latter is deprecated. The starter’s configuration metadata registers both spellings, so your IDE will not flag either. Pick one and be consistent, because a typo in this family of properties fails silently rather than at startup. One Client, Built Once With a provider configured, Spring AI auto-configures a ChatClient.Builder. The temptation is to build a client inside the controller. Resist it; the client carries defaults that belong to the application, not to one endpoint and ChatClient is thread-safe and meant to be built once and shared. So it becomes a bean: Java @Configuration public class ChatConfig { @Bean public ChatMemory chatMemory() { return MessageWindowChatMemory.builder() .maxMessages(10) .build(); } @Bean public ChatClient chatClient(ChatClient.Builder builder, @Value("classpath:/prompts/system-prompt.st") Resource systemPrompt, ChatMemory chatMemory) { return builder .defaultSystem(systemPrompt) .defaultAdvisors(new SimpleLoggerAdvisor(Ordered.LOWEST_PRECEDENCE), MessageChatMemoryAdvisor.builder(chatMemory).build()) .build(); } } Three decisions are packed into those twenty lines. Prompts Live in Files defaultSystem() accepts a Resource, so the system prompt is a StringTemplate file under src/main/resources/prompts rather than a string literal in Java. It reads better, diffs better, and can be edited without recompiling: Plain Text You are a generalist. Answer as briefly and precisely as you can both general and specific questions. A system prompt is worth setting even for a toy. Small local models are chatty by default, and a brief system message is the difference between a sentence and five paragraphs of preamble. Memory Is an Advisor Every ChatClient call is stateless unless something makes it otherwise. MessageChatMemoryAdvisor is that something. Before each call, it loads the history for a conversation and appends it to the prompt, and after each call it stores the new user and assistant messages. The store behind it here is MessageWindowChatMemory, an in-memory map keyed by conversation id that keeps the last ten messages and drops the oldest when the window fills. Which conversation a request belongs to is decided per call, in the next chapter. Logging Sits Closest to the Model SimpleLoggerAdvisor logs the request and the response at DEBUG. Its position in the chain is set by the order argument and Ordered.LOWEST_PRECEDENCE puts it last, after the memory advisor has done its work, so what it logs is the prompt the model actually receives, history and format instructions included. It stays quiet until you switch its logger on: Properties files logging.level.org.springframework.ai.chat.client.advisor=DEBUG That one line is the most useful debugging aid in this article. When a model misbehaves, the first question is always “what did it actually see?” and this answers it. The Endpoint The controller is thin. It resolves a conversation ID from an optional header, delegates to a service, and echoes the ID back so the caller can continue the same conversation on the next request: Java @RestController @RequestMapping("/api") class ChatController { private final ChatService chatService; public ChatController(ChatService chatService) { this.chatService = chatService; } @GetMapping("/v{version}/chat") public ResponseEntity<Response> chat(@RequestParam String message, @RequestHeader(value = "X-Conversation-Id", required = false) String conversationId) { String id = conversationId == null ? UUID.randomUUID().toString() : conversationId; var response = chatService.respond(message, id); return ResponseEntity.ok() .header("X-Conversation-Id", id) .body(response); } } The v{version} segment is not a hand-rolled convention. Spring Framework 7 has first-class API versioning and Boot 4 configures it from properties: Properties files spring.mvc.apiversion.use.path-segment=1 spring.mvc.apiversion.supported=1.0 spring.mvc.apiversion.default=1.0 The first line tells MVC that the version is the path segment at index 1, in /api/v1/chat that is v1, and the default parser understands the leading v. The second lists what the application supports, so a request for /api/v2/chat is rejected before any controller runs. The third is the version assumed when a request carries none. The service is where the ChatClient fluent API earns its keep. One chained call covers the prompt template, the conversation, the tools and the output type: Java @Service public class ChatService { private final ChatClient chatClient; private final Resource userPrompt; private final TenantService tenantService; public ChatService(ChatClient chatClient, @Value("classpath:/prompts/user-prompt.st") Resource userPrompt, TenantService tenantService) { this.chatClient = chatClient; this.userPrompt = userPrompt; this.tenantService = tenantService; } public Response respond(String message, String conversationId) { return chatClient.prompt() .user(userSpec -> userSpec.text(userPrompt) .param("message", message)) .advisors(advisorSpec -> advisorSpec.param(ChatMemory.CONVERSATION_ID, conversationId)) .tools(tenantService) .call() .entity(Response.class); } } Line by line, the user message is a template with a {message} placeholder, filled with .param(), which keeps user input out of the template’s structure and the template itself readable: Plain Text Provide a short, to-the-point answer to the below message: {message} The advisors() block hands the conversation id to the previously mentioned memory advisor, under the well-known key ChatMemory.CONVERSATION_ID. Two requests with the same X-Conversation-Id header share history. A request without the header starts a fresh conversation and is told its id in the response. tools() and entity() get their own chapters. Plain Text GET /api/v1/chat?message=In one sentence, what is quantization of an LLM? LLAMA3.1: Quantization of a large language model (LLM) refers to the process of approximating its weights and activations with lower-precision data types (such as integers or fixed-point numbers) to reduce memory usage and computational overhead, while minimizing the impact on model performance. If you are running with DevTools and edit application.properties or a prompt file under src/main/resources, nothing happens until the file is copied into target/classes. DevTools watches the build output, not the source tree. A rebuild in your IDE triggers it; editing the file in a plain text editor doesn’t, and you’ll spend a while wondering why your fixed base URL is still returning 404. Answers as Java Records Text is informative, but typed objects are the reason to put a model behind a Spring service. entity() takes a class, derives a JSON schema from it, appends format instructions to the prompt, and parses what comes back. The endpoint returns this: Java public record Response( @JsonPropertyDescription(""" The category of the request message. \ SPECIFIC when the message refers to a particular tenant, identified by its unique identifier, \ so that answering requires the tenant information tool. \ GENERAL for any other message, such as general knowledge or technical questions \ that can be answered without tenant data.""") ResponseCategory category, @JsonPropertyDescription("The actual response message text") String text) { } public enum ResponseCategory { GENERAL, SPECIFIC } The @JsonPropertyDescription annotations are not decoration. They flow into the generated schema and the schema is what the model reads, so a description is the only place a field’s meaning gets explained. The enum becomes an enum constraint in the schema, which is why the model reliably answers with one of the two constants rather than a synonym. The category description is deliberately a paragraph, not a label. Giving the Model a Tool A model knows nothing about your data specifics (tenants’ information as part of this experiment). Tool calling is how it asks. You describe a method; the model decides whether to call it and with what arguments, Spring AI executes it and feeds the result back, and the model writes its answer from that. The description is the interface; it’s all the model sees: Java @Service public class TenantService { private static final Logger logger = LoggerFactory.getLogger(TenantService.class); private final Map<String, String> tenantInfo = Map.of( "TNG", "Tenant TNG is a well known TEM services provider.", "FED", "Tenant FED is Roger Federer's foundation.", "DJO", "Tenant DJO is Novak Djokovic's foundation." ); @Tool(name = "get-tenant-info-by-id", description = "Retrieve additional information by tenant id") public String infoByTenantId(@ToolParam(description = "The unique identifier of a tenant") String tenantId) { logger.info("Retrieving additional info for tenant with id {}", tenantId); return tenantId == null ? "N/A" : tenantInfo.getOrDefault(tenantId, "N/A"); } } Passing the bean to .tools(tenantService) in the service is all the registration there is. Spring AI scans it for @Tool methods and builds the JSON schema for their parameters from the @ToolParam descriptions. The log line is deliberate. Whether the model decided to call the tool and what it passed, is otherwise invisible. The null guard matters too. A model that decides to call the tool but fails to extract an argument sends nothing for tenantId and a Map.getOrDefault(null, …) on an immutable map throws. Returning "N/A" instead lets the model explain the gap in its own words, which is exactly what happens in the first test case below. Testing Without a Model A local model makes tests possible offline, but it does not make them good tests. The output varies run to run, and every assertion becomes a guess about phrasing. For anything that checks your code rather than the model’s, replace the ChatModel bean with a mock. Spring AI‘s auto-configuration builds the ChatClient.Builder from whatever ChatModel is in the context, so a @MockitoBean is enough to take the daemon out of the picture entirely: Java @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.MOCK) @AutoConfigureMockMvc class ChatControllerTests { @MockitoBean private ChatModel chatModel; @Autowired private MockMvcTester mvc; @BeforeEach void init() { // ChatClient copies the model's defaults into every request, so a bare mock does not work when(chatModel.getOptions()) .thenReturn(ChatOptions.builder().build()); } @Test void chat() { var message = "Local models keep your prompts on your own machine."; var modelOutput = """ {"category": "GENERAL", "text": "%s"} """.formatted(message); var response = new ChatResponse(List.of(new Generation(new AssistantMessage(modelOutput)))); when(chatModel.call(any(Prompt.class))) .thenReturn(response); assertThat(mvc.get() .uri("/api/v1/chat") .param("message", "Why run locally?")) .hasStatusOk() .bodyJson() .convertTo(Response.class) .isEqualTo(new Response(ResponseCategory.GENERAL, message)); } } The mock returns the JSON a model would return, and the test asserts that your controller, service, template and record mapping serve it back as a Response. The model’s ability to produce that JSON is a separate question and not one a unit test should be asking. Two things are quietly bypassed here, and it is worth knowing which. Tool execution happens inside the concrete ChatModel implementation, so a mocked model never calls TenantService. The tool is registered but inert. The memory advisor, on the other hand, is real; it runs, stores the exchange in the in-memory window under a freshly generated conversation ID, and does no harm. A note on the chatModel mock is worth making. If only mocking ChatModel and stubbing just call(), the first request fails with: Plain Text NullPointerException: Cannot invoke "ChatOptions.mutate()" because the return value of "ChatModel.getOptions()" is null ChatClient reads the model’s default options and merges them into every request. Mockito returns null for the unstubbed getter and the merge breaks. One line in a @BeforeEach fixes it, but the stack trace points into Spring AI internals rather than at your test, so it costs more time than it should. Testing the Integration With the application started, four requests on the same conversation exercise the whole chain: template, memory, tool, structured output. The X-Conversation-Id header ties them together. What each one did is known from the log line in TenantService and from the advisor log previously switched on. Use Case 1: Ask by “id” PowerShell curl -G http://localhost:8080/api/v1/chat -H "X-Conversation-Id: 123-1bc-981" --data-urlencode "message=Provide info for tenant with id FED" # {"category":"GENERAL","text":"Tenant information not available"} The tool was invoked, but with no tenantId. The model recognized that the question needed the tool and then failed to map "FED" onto the parameter, so the method returned N/A and the model paraphrased it. Use Case 2: Same Question, Phrased With the Parameter’s Own Description PowerShell curl -G http://localhost:8080/api/v1/chat -H "X-Conversation-Id: 123-1bc-981" --data-urlencode "message=Provide info for tenant with unique identifier FED" # {"category":"GENERAL","text":"Tenant FED is Roger Federer's foundation."} Tool invoked, tenantId mapped to FED, correct answer. The only change was “id” becoming “unique identifier,” the exact words in the @ToolParam description. Use Case 3. The Winning Phrasing, Different Tenant PowerShell curl -G http://localhost:8080/api/v1/chat -H "X-Conversation-Id: 123-1bc-981" --data-urlencode "message=Provide info for tenant with unique identifier DJO" # {"category":"GENERAL","text":"Tenant DJO is an abbreviation for DJ Okaya."} The tool was not invoked. The model answered from nothing, fluently and wrongly. Use Case 4: The Identical Request, Sent Again PowerShell curl -G http://localhost:8080/api/v1/chat -H "X-Conversation-Id: 123-1bc-981" --data-urlencode "message=Provide info for tenant with unique identifier DJO" # {"category":"GENERAL","text":"Tenant DJO is Novak Djokovic's foundation."} This time the tool was invoked, tenantId mapped to DJO, and the answer is right. Nothing changed between use cases 3 and 4 except the conversation history being one exchange longer. Four requests, one tool, three different behaviors: called without arguments, called correctly, and not called at all, the last two for the same input. The temperature of 0.7 explains some of the variance, but the deeper issue is that an 8-billion-parameter model is weak at the two judgment calls tool use requires: should I call this, and what goes in the parameter. Two mitigations are visible in the results. Wording the user’s request in the same vocabulary as the @ToolParam description made the difference between use cases 1 and 2, so descriptions should use the words users will use. And the null guard in TenantService turned a would-be exception into a polite non-answer. Neither makes use case 3 go away, that probably needs a lower temperature, a system prompt that instructs the model to always use the tool for tenant questions, or a bigger model. What It Actually Feels Like Measured on a laptop, Intel Core Ultra 7 155H, 32 GB RAM, RTX 500 Ada, running llama3.1 8B through the endpoint above: 14.2s – first request after startup, three-bullet answer11.3s – same request, model already warm Double-digit seconds for a paragraph and a tool call adds a second round trip to the model on top. That’s workable for development and unacceptable for a request a user is waiting on, which is the honest headline: local models are a development and testing tool at this size. The quality gap is sharper than the speed gap, and it’s worth seeing rather than being told about. Asked a question about the very tool serving it: PowerShell GET /api/v1/chat?message=In one sentence: what is Ollama? LLAMA3.1: Ollama is a brand of vegan, plant-based egg substitute made from a combination of pea protein, potato starch and other ingredients. Confident, fluent, well-formed and entirely invented. The model’s training data predates the tool it’s running inside. Put that next to "DJ Okaya" from before and the pattern is clear. This is the most useful thing about developing against a small local model: the failure modes you will eventually have to design around are loud here instead of rare. Whatever guardrails, validation, and grounding your application needs in production, you will notice you need them on day one. Resources [1] – POC code is here [2] – The picture was taken in Zakynthos, Greece

By Horatiu Dan

CORE

Data Integration Landscape 2026: Event Streaming, API, and Batch in the Era of Agentic AI

Data integration used to be treated as back-office plumbing. Difficult, yes, and always operationally critical, but rarely seen as strategic. That has changed. The Data Integration Landscape 2026 maps this shift across three communication paradigms and every major vendor. The largest software companies in the world have paid a fortune to own the integration layer: IBM acquired Confluent for $11 billion, Salesforce acquired Informatica for $8 billion, Qlik absorbed Talend, TIBCO folded into Cloud Software Group, and Fivetran merged with dbt Labs. When integration vendors command prices like these, the integration layer has become a strategic asset, not a piece of infrastructure to take for granted. The pressure behind this has been building for a long time. Data volumes keep exploding, digitalization keeps pushing more of the business onto real-time rails, and the demand for fresh data has been rising for years. AI is the latest and most acute driver. Models and autonomous agents need data that is current, governed, and continuously flowing. A nightly batch job and an API that returns yesterday's state cannot support a system making decisions in the moment. An agent working from outdated data does not fail loudly. It confidently does the wrong thing. That raised the stakes on a problem enterprises have been wrestling with for a decade. What Are the Three Data Integration Paradigms? The three data integration paradigms are request-response, batch, and event streaming. The landscape is best understood not by vendor but by how data actually moves. Choosing which paradigm sits at the center is among the most consequential architectural decisions an enterprise makes. Request-response is the most familiar. A system asks another for data and waits. REST APIs, GraphQL, and API management and gateway products work this way. It suits transactional interactions like fetching a record or submitting an order, but it couples the caller to the callee: you must know where the other system is and wait for it to answer. Batch moves data in bulk on a schedule. It is reliable and well understood for historical loads, analytics, regulatory reporting, and AI model training. Its weakness is freshness. The data is always minutes or hours old, which breaks down the moment a system needs to act on what is happening right now. Event streaming moves data continuously as events occur, and Apache Kafka has become its de facto standard. When a transaction completes or a record changes, that event is published to a stream, and any system that needs it consumes independently, without asking the source for anything. It is decoupled, persistent, and real-time, which is why it has become the foundation for both modern integration and agentic AI. Apache Flink has emerged as the de facto standard for stateful stream processing on top of Kafka, covering real-time transformation, aggregation, and AI-enriched data pipelines. The Data Integration Landscape below maps every major vendor across these three paradigms. Filled bubbles are platform leaders; outlined bubbles are specialists and emerging players. Grouped by paradigm, not by ranking. The full analysis is available as a free PDF without registration. Why Should Event-Driven Architecture Be the Center of Data Integration? The central argument of this landscape is simple. Event-driven architecture belongs at the center, with request-response and batch as consumer interfaces on top of it. The reason is decoupling. In an event-driven model, the system that produces data does not need to know who consumes it, when, or how. Producers and consumers are independent, which means a slow or unavailable system does not block everything connected to it. This is most commonly implemented with event streaming platforms like Apache Kafka, but message brokers such as IBM MQ and Solace serve event-driven patterns too, particularly where guaranteed delivery and fine-grained routing matter more than high-throughput streaming. Most enterprises use all three paradigms at once, and that is correct. The mistake is treating the choice as either/or. A business unit running iPaaS workflows, an analytics team pulling batch exports, and an AI agent querying live state can all be served from the same event-driven backbone without coupling to each other or to the underlying systems. Each connects in the way that suits it: some consume the stream directly, others reach it through an API gateway that exposes the stream as an endpoint or pushes events to a webhook or function. No large enterprise runs a single integration platform, so the real question is never which one product to buy. It is which paradigm sits at the center, and how the rest connect to it without recreating the tangle of point-to-point links it was meant to replace. The full landscape works through the market vendor by vendor: Data Integration Landscape. Is Batch Integration Dead? No. Batch is still the right tool for historical loads, analytical refresh cycles, regulatory reporting, and AI model training. What has changed is where the new thinking happens. The innovation has moved to the streaming layer, where data is processed continuously and acted on at the right time and context, often in real time. The batch column of the landscape has no emerging row because new entrants are building for streaming and change data capture, not for scheduled pipelines. Batch endures. It is just no longer where the architecture is evolving. Which Architecture Decisions Matter More Than Vendor Selection? The consolidation wave points to three decisions that outweigh which vendor you pick. The three-paradigm suite is the new baseline. The major platform vendors now each cover all three paradigms in one ecosystem. Choosing one of them increasingly means committing to that ecosystem across API management, data integration, and event streaming at once. That lock-in is far more expensive to unwind than any single product swap. If you are making a five-year platform decision, map which paradigms each candidate actually covers before you compare feature lists. The market is moving toward event-driven architecture. Databricks added Kafka-compatible APIs to its Zerobus Ingest service, allowing existing Kafka producers to stream data directly into Databricks with a configuration change rather than code changes. Snowflake announced Datastream, its own Kafka-compatible service, now heading into private preview. Batch ETL vendors are adding change data capture. iPaaS vendors are adding event-driven capabilities. The direction is consistent because agents and operational systems cannot wait for batch cycles. Legacy integration shapes the architecture more than vendor selection does. Same-vendor shortcuts like Zero-ETL work inside one ecosystem, but they do nothing for the SAP iDoc pipeline, the mainframe, the EDIFACT exchange, or the hospital's HL7 feed. Real enterprise architecture absorbs legacy formats at the edge for years. A platform that connects cleanly to the future but needs custom work to reach the past is not a complete platform for most enterprises. How Does Data Integration Connect to Process Intelligence and Agentic AI? Data integration does not stand alone. It is the foundation beneath process intelligence and trusted agentic AI, the three layers I think of as the Trinity of modern data architecture. The data layer moves and governs the data. Process intelligence understands how the organization actually runs. Trusted agentic AI acts on both. Each layer depends on the one beneath it. The full vendor-by-vendor analysis covers the platform leaders, the specialists, and the emerging players across all three paradigms, plus the legacy integration challenge, industrial IoT, and the trends shaping the next two years. If you are making an integration decision this year, the Data Integration Landscape 2026 is the starting point. Begin with the paradigms, not the vendor list. Map which ones you actually need at the center and which belong at the edges. The vendor choice gets much easier once the architecture is clear. About this landscape: This is an independent analyst perspective. No vendor paid for inclusion or placement. No vendor reviewed its section before publication. I run an advisory practice through Kai Waehner GmbH and have held roles at Talend, TIBCO, and Confluent. I also serve as Global Field CTO at Kestra, a workflow orchestration vendor outside this landscape's scope. The complete methodology note is in the full landscape.

By Kai Wähner

CORE

OwlVigil Open-Sources Its Go SDK for Enterprise AI

Since the launch of OwlVigil, I have been contemplating a key objective: fully unlocking its capabilities for developers, enabling seamless integration of AI management functionalities, and fostering the creation of innovative products built on top of these capabilities. We have developed a comprehensive set of APIs and SDKs for various programming languages, making nearly all of OwlVigil’s core capabilities available. Developers can use a subset of these features via the SDKs, or build a comprehensive AI gateway and management platform on our APIs. Today, we are officially open-sourcing the first OwlVigil Software Development Kit (SDK) — the Go SDK (https://github.com/Syrovex/owlvigil_sdk_go). Subsequently, we will implement versions for Java, Node.js, and Python. This aligns with my vision for founding Syrovex and developing OwlVigil: providing enterprises and developers with foundational capabilities for AI management. These capabilities include handling complex yet essential tasks such as model integration, cost management, access control, security compliance, and observability. However, for most developers, the complexity of the integration process can still pose a significant challenge, regardless of the platform’s feature richness. This is particularly relevant for enterprise systems; once deployed in a production environment, the considerations extend far beyond the initial success of “successfully calling a model once.” Therefore, I have always wanted OwlVigil to be “lightweight” enough to integrate seamlessly into enterprises’ existing technical ecosystems. Enterprises are not required to begin development from the beginning, nor do they need to redevelop existing applications simply to introduce an AI governance platform. Developers can continue using their familiar languages, frameworks, and models; they simply need to integrate OwlVigil into their existing call chain to gradually gain capabilities such as multi-model provider integration, cost management, access control, log monitoring, and security compliance. Our objective is straightforward: liberate enterprises and developers so they can prioritize their core business. OwlVigil: An AI Gateway and Governance Platform When people first encounter OwlVigil, they may see it as an AI Gateway or a “relay station” that forwards model requests. This understanding is incomplete. An AI Gateway is indeed part of OwlVigil. Rather than starting from scratch, we leverage established open-source technologies and industry standards to ensure a seamless integration of these capabilities. This is because simply forwarding a model request is not the most difficult part. The real complexity lies in how enterprises should manage models, providers, permissions, budgets, policies, logs, and audits once AI enters the production environment. This is the problem that OwlVigil aims to solve. In our design, OwlVigil strategically positions itself between enterprise applications, workflows, agents, and various model providers. Applications are only required to submit models and requests; they are not required to maintain the keys and interface differences for each provider within their business code. Meanwhile, enterprises can determine behind the scenes whether to permit requests, where to route them, how much budget they can consume, and how to trace issues when they arise. From an API perspective, we have divided these capabilities into two independent components: the Gateway Runtime API and the Management API. These are not merely two features of OwlVigil, but rather two API surfaces designed for different scenarios with distinct permission boundaries. Gateway Runtime API for AI Model Requests The Gateway Runtime API is designed for applications and agents and is responsible for processing actual model requests. It currently supports model queries, chat completions, responses, embeddings, streaming, and Anthropic-compatible messages. Developers will find a relatively simple and familiar model interface. Before a request reaches the upstream Provider, OwlVigil handles authentication, permission checks, model routing, policy enforcement, budget verification, and access restrictions. After the request completes, OwlVigil also logs tokens, costs, status, logs, and the call chain. In summary, it addresses the following aspects: first, the “how to invoke a model”; second, the control of the process before invocation; third, the routing of requests during invocation; and fourth, the monitoring and auditing after invocation. Management API: Making AI Governance Capabilities Available to Developers The Management API is responsible for controlling and managing OwlVigil. This tool empowers developers to automate tasks that previously required manual execution in the Dashboard, including: The following actions are required to create and isolate different workspaces and runtime environments: Configuring providers, models, routing, and fallback candidatesCreating, rotating, limiting, and revoking gateway keysManaging teams, members, roles, and access permissionsConfiguring request policies, budgets, quotas, and cost alertsQuerying usage, request logs, audit logs, and invocation tracesManaging webhooks, event subscriptions, retries, and redeliveryIntegrating with subscription, billing, top-up, order, and account workflows These capabilities may be especially crucial for internal enterprise platforms, automation systems, and SaaS products, perhaps even more so than simple model calls. For example, enterprises can automatically generate gateway keys and budgets for different teams within internal developer platforms. SaaS products can establish independent workspaces, permissions, and usage quotas for each customer. And operations systems can monitor abnormal requests and cost fluctuations through logs, traces, and webhooks. How the OwlVigil SDK Simplifies API Integration The SDK is built on top of these two sets of APIs, addressing many engineering challenges, including typed requests, authentication, timeouts, streaming, pagination, request metadata, structured errors, retries, and Webhook signature verification. While these capabilities are not complex on their own, implementing them from scratch would be a waste of time and risk introducing security and reliability issues in production environments. The Software Development Kit (SDK) unifies the handling of these common capabilities, enabling developers to build their own applications, internal enterprise AI platforms, and even customer-facing AI management products directly on top of OwlVigil. For a more comprehensive overview of capabilities and usage, please visit the OwlVigil Developer Center and Core Concepts. This section is exclusively designed to provide readers with instructions on installation, making calls, and accessing documentation. It does not reiterate the product capabilities previously covered. The following revisions have been made: Getting Started With the OwlVigil Go SDK The OwlVigil Go SDK is open-source and can be installed directly via Go Modules: Go go get github.com/Syrovex/owlvigil_sdk_go Once installed, create a Client using the Gateway Key to call the OwlVigil Gateway. Below is an example of querying the models available in the current Workspace: Go client := gateway.NewClient( owlvigil.WithAPIKey( os.Getenv("OWLVIGIL_GATEWAY_KEY"), ), ) Go models, meta, err := client.ListModels( context.Background(), ) if err != nil { log.Fatal(err) } Go fmt.Println(meta.RequestID, len(models.Data)) If you need to call the Management API, you can use management.NewClientand pass in OWLVIGIL_API_KEY: Go client := management.NewClient( owlvigil.WithAPIKey( os.Getenv("OWLVIGIL_API_KEY"), ), ) The Gateway Key and the Management API Key are separate sets of credentials. The former is used for applications to call models, while the latter is used to manage resources such as workspaces, routing, members, policies, and budgets. It is important to note that these two functions should not be used interchangeably. In production environments, please refrain from hardcoding them in source code or committing them to a Git repository. The repository already provides ready-to-run examples for model invocation, streaming, Gateway Keys, usage queries, team management, webhooks, and multi-environment configuration. These can be referenced directly based on the specific use cases of the user. Please refer to the following resources for more information: Go SDK DocumentationGitHub Open-Source Repository Upcoming OwlVigil SDKs for Java, Node.js, and Python The Go SDK represents our inaugural release, and we have more planned for the future. The next step in our company’s strategic plan is the gradual release of Java, Node.js, and Python SDKs. These will cover the most common development languages and scenarios found in backend services, AI applications, data processing, and enterprise systems. The Software Development Kit (SDK) is a living, breathing project that never truly ends once it is released. In addition to continuously integrating new OwlVigil capabilities, we will also enhance the sample projects, development guides, and Recipes. The Developer Center, Quickstart, API Reference, and Go SDK documentation are currently accessible at https://owlvigil.com/developer/.If you are developing AI applications, agents, or internal enterprise tools, or if you wish to integrate model routing, budgeting, permissions, and auditing capabilities into your own systems, we encourage you to try it out. We will continue to enhance the project. Should you encounter any issues while using it, or have suggestions regarding existing features or future SDKs, please feel free to contact us directly via a GitHub issue or other channels. This article was originally published on my blog. For the latest version and future updates, please visit the original post: https://jaketao.com/language/en/owlvigil-open-source-go-sdk/.

By Jake Tao

Why 200 OK Is the Most Dangerous Lie in Enterprise Integration

A customer changes her email address in a mobile app on Friday afternoon. The app displays a green confirmation message: Your profile has been updated. The API response was 200 OK. The mobile team sees no error. The API dashboard counts the request as successful. On Monday, the customer receives an account statement at her old email address. The update never reached Salesforce. This is the kind of failure that frustrates everyone because nobody is obviously wrong. The mobile app did what the API told it to do. The API gateway recorded a successful response. The Mule application may even have logged the Salesforce timeout. But the business action did not happen. The customer did not care that the JSON payload was valid or that MuleSoft accepted the request. She cared that her email address changed. That is the gap many integration APIs miss: technical success is not the same as business completion. A 200 OK response can be completely valid at the HTTP level while being dangerously misleading at the business level. This article is about fixing that gap. The Real Problem: Green Dashboards, Failed Business Operations Here is the common sequence. The mobile app calls a customer-profile API. MuleSoft validates the request and invokes Salesforce. Salesforce is slow or temporarily unavailable. The Mule flow catches the error, logs it, and continues so that the client does not receive an ugly error message. The caller gets 200 OK. The update never happens. Figure 1 — The Green Dashboard Failure A support ticket usually appears much later: “The customer says she updated her email, but Salesforce still has the old one. Can someone check?” At that point, the support engineer has to search logs, identify the correlation ID, discover a suppressed connector error, and explain why the API told the customer the opposite of what happened. That is not a monitoring problem. It is an API-contract problem. How MuleSoft Can Accidentally Turn Failure into Success The risky part is often a well-intentioned error handler. A developer does not want to expose Salesforce exceptions, stack traces, or internal system names to a mobile application. That is the right instinct. But hiding technical details is different from hiding the fact that the operation failed. Consider this Mule error handler: XML <error-handler> <on-error-continue type="HTTP:CONNECTIVITY"> <logger level="ERROR" message="#['Salesforce update failed: ' ++ error.description]" /> <set-payload value='#[{ status: "SUCCESS", message: "Customer updated successfully" }]' /> </on-error-continue> </error-handler> The log captures the failure. The client receives success. The word that changes everything is continue. In Mule 4, on-error-continue treats the error as handled and allows the owning flow to complete. If the flow owns an HTTP request, that can result in a successful response even though a downstream call failed. MuleSoft’s own error-handling documentation shows this behavior. on-error-continue is not wrong. It is useful when the failed step is optional. For example: A customer order is created, but a noncritical analytics event fails.A product API succeeds, but an optional cache refresh fails.A notification preference update succeeds, but a marketing enrichment call fails. In those cases, continuing can be reasonable. It is not reasonable when the failed call is the business action the API promised to perform. If the API says “update customer email,” and Salesforce is the system of record for customer email, then a Salesforce failure means the update is not complete. The response must say so. Stop Asking “Did the API Work?” Ask a better question: “When this API returns success, what business fact can the caller rely on?” For a customer-email update, the answer should be specific: “The customer’s email address has been updated in the system of record.” If that cannot be guaranteed before the response is sent, the API has two honest choices: Return a failure because the requested action did not complete.Return 202 Accepted because the action was safely accepted for later processing. What it should not do is return 200 OK because the request reached MuleSoft. Figure 2 — Three Very Different Meanings of “Success” This is the key design decision for command APIs: API responseWhat it should meanExample200 OK or 201 CreatedThe requested business action completedSalesforce confirmed the email update202 AcceptedThe request was durably saved and will be processed laterThe update command was written to a queue or database400 Bad RequestThe caller must fix the requestInvalid email format or required field missing409 ConflictThe request conflicts with known stateSame idempotency key used with different payload503 Service UnavailableA required dependency prevented completionSalesforce is unavailable and no durable request was created A successful HTTP response should not be a polite way of saying, “We will hope for the best.” The Synchronous Contract: Return 200 Only After the System of Record Confirms For a small, immediate update such as a customer email change, a synchronous API can be appropriate. The call waits for Salesforce. If Salesforce confirms the update, return 200 OK. If Salesforce is unavailable, return a truthful temporary failure. Plain Text sequenceDiagram participant App as Mobile App participant Mule as MuleSoft API participant CRM as Salesforce App->>Mule: PUT /customers/4821/email Mule->>CRM: Update email address alt Salesforce confirms update CRM-->>Mule: 204 No Content Mule-->>App: 200 OK<br/>{status: COMPLETED} else Salesforce unavailable CRM--xMule: Timeout Mule-->>App: 503 Service Unavailable<br/>{status: NOT_COMPLETED} end Here is the Mule flow shape: XML <flow name="update-customer-email"> <http:listener config-ref="HTTP_Listener" path="/customers/{customerId}/email" allowedMethods="PUT"> <http:response statusCode="#[vars.httpStatus default 200]" /> </http:listener> <try> <http:request config-ref="Salesforce_Request" method="PATCH" path="/services/data/v61.0/sobjects/Contact/#{attributes.uriParams.customerId}"> <error-mapping sourceType="HTTP:CONNECTIVITY" targetType="CUSTOMER:CRM_UNAVAILABLE" /> <error-mapping sourceType="HTTP:TIMEOUT" targetType="CUSTOMER:CRM_UNAVAILABLE" /> </http:request> <set-variable variableName="httpStatus" value="200" /> <set-payload value='#[{ status: "COMPLETED", customerId: attributes.uriParams.customerId, correlationId: correlationId }]' /> <error-handler> <on-error-propagate type="CUSTOMER:CRM_UNAVAILABLE"> <set-variable variableName="httpStatus" value="503" /> <set-payload value='#[{ status: "NOT_COMPLETED", code: "CRM_UNAVAILABLE", message: "The customer update could not be completed. Retry later using the same idempotency key.", correlationId: correlationId }]' /> </on-error-propagate> </error-handler> </try> </flow> The exact XML varies by project, but the design should not: Translate connector errors into meaningful application errors.Do not expose raw Salesforce or Mule exceptions.Do not convert a required downstream failure into success.Include a correlation ID in every outcome.Tell callers whether retrying is appropriate. MuleSoft supports custom error mappings precisely so a connectivity failure from one dependency can be handled differently from a failure in another. See the Mule error model. A 503 Can Be a Better Customer Experience Than a Fake 200 At first, returning 503 Service Unavailable sounds harsh. But compare the two customer experiences. Fake success Plain Text Friday: “Your email was updated.” Monday: Sensitive statement sent to old email. Tuesday: Customer contacts support. Truthful temporary failure Plain Text Friday: “We could not update your email right now. Please try again.” Friday, 30 seconds later: Customer retries. Friday: Update succeeds and is confirmed. A truthful failure is inconvenient. A false success is expensive. For retryable failures, return a clear response: HTTP HTTP/1.1 503 Service Unavailable Retry-After: 30 Idempotency-Key: 7a88e9dc-e1bd-4f38-a47b-0fc009392c78 X-Correlation-ID: 49d7d0a2-e771-43b7-ae4d-a9055ab21f3d { "status": "NOT_COMPLETED", "code": "CRM_UNAVAILABLE", "message": "The customer update could not be completed. Retry after 30 seconds using the same idempotency key.", "correlationId": "49d7d0a2-e771-43b7-ae4d-a9055ab21f3d" } This response gives the client application something useful: A reason category, without leaking internal implementation detailsA retry intervalA stable idempotency key for safe retryA correlation ID for support investigation That is far more useful than a green checkmark built on a hidden error. When 202 Accepted Is the Better Design Not every operation should wait for the downstream system before responding. Imagine a new employee onboarding request: Plain Text HR system → MuleSoft → Identity provider → Active Directory → Laptop workflow → Payroll This is not a two-second API call. It is a business process. Returning 200 OK after the HR request reaches MuleSoft would be misleading. But returning 503 because Active Directory takes time would also be wrong. The honest model is asynchronous. The API stores the request durably, assigns a tracking ID, and returns 202 Accepted. HTTP HTTP/1.1 202 Accepted Location: /employee-provisioning-requests/a0b6ef54-91d6-4a3c-ae76-e2d326f73ad5 { "requestId": "a0b6ef54-91d6-4a3c-ae76-e2d326f73ad5", "status": "PENDING", "correlationId": "49d7d0a2-e771-43b7-ae4d-a9055ab21f3d" } The client can then retrieve the real outcome: HTTP GET /employee-provisioning-requests/a0b6ef54-91d6-4a3c-ae76-e2d326f73ad5 HTTP { "requestId": "a0b6ef54-91d6-4a3c-ae76-e2d326f73ad5", "status": "COMPLETED", "completedAt": "2026-08-29T16:45:12Z" } Or, if something fails: JSON { "requestId": "a0b6ef54-91d6-4a3c-ae76-e2d326f73ad5", "status": "FAILED", "code": "IDENTITY_VALIDATION_ERROR", "message": "The employee username conflicts with an existing account." } Figure 3 — The Honest Asynchronous Contract The critical rule is simple: Return 202 Accepted only when the request has been safely stored somewhere durable. A request held only in memory is not accepted. A message that has not been committed to a queue is not accepted. A flow that merely intends to run later is not accepted. The API must be able to prove that it owns the request before it tells the caller to stop waiting. Measure Completion, Not Just API Availability A dashboard showing “99.99% of requests returned 2xx” can be dangerously comforting. For command APIs, measure three separate things: metricwhat it tells youHTTP response-code rateWhat the API told clientsDownstream completion rateWhether the target business system completed the actionTime to completionHow long customers wait for the real outcome For asynchronous work, also monitor: Pending requests by business operationAge of the oldest pending requestFailures by downstream systemRetry attempts and retry exhaustionRequests requiring manual interventionDuplicate requests prevented through idempotency A useful alert is not: “The API has 500 errors.” A useful alert is: “Customer updates accepted more than 10 minutes ago have not reached COMPLETED or FAILED.” That alert tells the operations team what is actually at risk. The Question to Ask in Every API Design Review Before approving an integration API, ask one question: “If this endpoint returns 200 OK, which system has definitely completed which business action?” If the answer is “MuleSoft received the request,” the contract is not finished. If the answer is “Salesforce confirmed the customer update,” then 200 OK is meaningful. If the answer is “the request was safely stored for later processing,” return 202 Accepted. If the answer is “we do not know because the dependency timed out,” return a truthful retryable failure—not success. That one question catches a surprising number of production defects before they become support tickets. Final Takeaway The goal is not to return more errors. The goal is to stop lying with success. A 200 OK should mean the business operation completed. A 202 Accepted should mean the operation was safely accepted for later completion. A 503 should mean the operation did not complete and can be retried safely. Anything else creates the worst kind of integration failure: one that is technically green, operationally invisible, and painfully real for the customer.

By Manjeera Chanda

Post-Quantum TLS for Cloud APIs and Microservices

Why Every TLS Termination Is a Separate Migration Boundary A cloud application rarely has one TLS connection. A single API request can be decrypted and re-encrypted at a CDN, a web application firewall, an API gateway, a load balancer, a service-mesh proxy, an application runtime, and a managed cloud endpoint. Each termination negotiates a fresh session. Upgrading only the public edge can leave every downstream hop protected by classical key exchange. That architectural fact changes how teams should approach post-quantum TLS. The right unit of migration is not the hostname, service, cluster, or application. It is the directional TLS link between two adjacent termination points, complete with an owner, a policy, a negotiated group, and runtime evidence. Figure 1. Hybrid negotiation must be configured and proved independently on every TLS link. An edge-only upgrade does not protect downstream hops. The Unit of Migration Is the TLS Link Treat the request path as a graph. The nodes are clients, gateways, proxies, workloads, and managed services. Every edge is an independently negotiated TLS connection. For each edge, record the source and destination, the team or provider that owns it, the data's confidentiality lifetime, the full-handshake rate, the selected key-establishment group, the authentication algorithm, the policy that produced the configuration, and the evidence that proves the result. This link-level model prevents an attractive but misleading label such as "PQC enabled." A client can advertise a hybrid group without the server selecting it. A gateway can negotiate hybrid on ingress and classical TLS to the backend. A service mesh can provide pervasive mTLS while its certificates still use classical signatures. Readiness must be measured from observed behavior, not configuration intent. Design rule: Count a link as post-quantum protected only when telemetry proves that it successfully negotiated an approved hybrid or post-quantum group. What X25519MLKEM768 Protects, and What It Does Not TLS 1.3 makes three largely independent cryptographic choices: a symmetric record cipher and hash, a key-establishment group, and an authentication mechanism. X25519MLKEM768 changes the second choice. It combines classical X25519 with NIST-standardized ML-KEM-768 and feeds both 32-byte shared secrets into the TLS 1.3 key schedule. The hybrid construction is designed so the session secret remains protected if either component remains secure, subject to the combiner and transcript assumptions. That is useful against a harvest-now, decrypt-later adversary recording traffic today. It does not make an RSA- or ECDSA-signed certificate quantum resistant, and it does not change AES-GCM record protection. TLS PROPERTY WHAT HYBRID PROVIDES WHAT STILL REMAINS Recorded-traffic confidentiality A combined X25519 and ML-KEM session secret Every link carrying the data must negotiate hybrid Server authentication Normal TLS transcript authentication Classical RSA or ECDSA remains quantum vulnerable Client authentication in mTLS Conventional certificate-based mTLS Client certificate and trust chain need a separate signature migration Bulk encryption Unchanged TLS 1.3 record protection Key establishment changes; AES-GCM does not Fallback Classical peers may still connect when policy allows Fallback must be visible, bounded, and eventually retired Resumption A PSK can inherit security from the original session PSK-only resumption has no fresh key exchange Use honest labels: Publish post-quantum key-establishment coverage and post-quantum authentication coverage as separate fields. A hybrid key exchange is not complete post-quantum TLS. The Benchmark: Modest Compute, Much Larger First Flights The research used OpenSSL 3.6.3 and paired in-memory BIOs to isolate protocol encoding, allocation, signature work, and key-establishment CPU from network latency. Across 120,000 successful full TLS 1.3 handshakes on an Apple M3 Pro, the only intended variable was X25519 versus X25519MLKEM768. Session tickets and caches were disabled, and each configuration used the same ECDSA P-256 certificate and TLS_AES_128_GCM_SHA256. SINGLE-WORKER MEASURE X25519 X25519MLKEM768 CHANGE TLS-record bytes 1,172 3,438 +2,266 (+193.3%) Process CPU per handshake 332.5 us 416.8 us +84.3 us (+25.3%) Throughput 2,977 hs/s 2,389 hs/s -19.7% Median elapsed time 320.3 us 405.0 us +84.7 us The raw hybrid key shares add 2,272 bytes: a 1,184-byte ML-KEM encapsulation key in the ClientHello and a 1,088-byte ML-KEM ciphertext in the ServerHello. In the measured handshake, surrounding encoding reduced the observed TLS-record delta slightly to 2,266 bytes. The important deployment effect is that a previously small ClientHello can cross packet boundaries and expose brittle middleboxes, parsers, or firewalls before cryptographic CPU becomes the bottleneck. Figure 2. Throughput plateaus beyond 16 workers on the 11-core host. At saturation, hybrid capacity is about 13.4 thousand full handshakes per second versus 16.0 thousand for X25519, while tail latency rises under oversubscription. Connection Reuse Determines the Real Cost The isolated benchmark is intentionally a worst case for connection churn: every request creates a fresh connection. In real cloud systems, HTTP/2, long-lived gRPC streams, SDK connection pools, and keep-alive amortize the handshake across many application requests. Aggressive idle timeouts, autoscaling bursts, DNS churn, unbalanced pools, serverless cold starts, and per-request clients can erase that advantage. REQUESTS PER CONNECTION FULL HANDSHAKES/S AT 50,000 REQUESTS/S ESTIMATED EXTRA CPU ESTIMATED EXTRA BANDWIDTH 1 50,000 4.214 cores 906.4 Mb/s 10 5,000 0.421 core 90.6 Mb/s 100 500 0.042 core 9.1 Mb/s 1,000 50 0.004 core 0.9 Mb/s Operational implication: Measure requests per connection by source-destination pair. Capacity tests should include cold starts, failover, and connection-pool disruption, not only steady-state request traffic. Cloud Controls Are Directional, Versioned, and Explicit The implementation landscape is uneven. A product may support the hybrid group on one direction, behind one policy name, or only when linked with a recent cryptographic library. The control plane therefore needs a capability registry that records provider, product version, binary build, direction, supported group names, policy provenance, and verification method. Managed Gateways and Edges Amazon API Gateway documents enhanced security policies with PQ for incoming REST API connections, while its integration egress remains a separate TLS link. Treat ingress and backend connectivity as different inventory records.Application Load Balancer policies can expose hybrid key exchange and log the selected group. Set the listener policy explicitly in infrastructure as code because console and automation defaults can differ.Cloudflare can extend hybrid key establishment from its edge to the origin, but the origin connection remains a distinct boundary that must be tested and observed. Service Meshes Envoy and Istio provide central levers for internal mTLS, but a recent proxy image alone is not evidence that ordinary mesh traffic negotiated X25519MLKEM768. Default curve lists, BoringSSL builds, FIPS profiles, and rolling-upgrade revisions can differ. Canary a preferred policy by namespace or proxy revision, inspect configuration dumps, and probe both inbound and outbound paths. Plain Text meshConfig: meshMTLS: minProtocolVersion: TLSV1_3 ecdhCurves: - X25519MLKEM768 - X25519 Application and Egress TLS Database drivers, message brokers, SDKs, and application-managed HTTPS clients may bypass the mesh or establish TLS after a CONNECT tunnel. Their linked runtime matters. OpenSSL 3.5 and later support ML-KEM and hybrid groups, while NGINX passes group configuration to its linked OpenSSL library. Pin patched versions, record the provider build, and observe selected-group distributions after upgrades. Infrastructure rule: Every managed listener and every application-controlled TLS client must receive an explicit, versioned policy. An omitted default should fail review. Roll Out Hybrid TLS as an Observable State Machine A Boolean flag such as pqc_enabled hides peer compatibility, mixed proxy versions, middlebox failures, classical fallback, and policy drift. Use explicit states whose transitions depend on evidence and whose rollback changes policy rather than rebuilding software. Figure 3. Each state transition is gated by interoperability, performance, capacity, and rollback evidence. A failed promotion returns the scope to the last approved policy without removing hybrid support from the software image. Observe. Upgrade clients, proxies, and libraries so they can advertise the hybrid group without requiring it. Inventory directional links and record offered groups, selected group, TLS version, resumption state, certificate signature algorithm, failure stage, build, and policy ID. Prefer. Put X25519MLKEM768 first while retaining time-bounded X25519 fallback. Roll out by listener, namespace, mesh revision, client cohort, or traffic percentage. Compare the canary with a matched classical baseline. Require. Move only capability-complete scopes to hybrid-only. Internal mTLS segments and controlled partner endpoints are usually better candidates than broad public APIs with unknown older clients. Retire. Remove classical groups from required scopes, deny legacy policy creation, close exceptions, and verify that health checks, direct pod paths, backup regions, and administrative endpoints do not bypass the policy. Promote on Evidence, Not Configuration The minimum useful dashboard describes both the security result and the operational cost. Every measurement must be attributable to a directional link, proxy or library build, and policy digest so the team can separate peer incapability from middlebox failure, local misconfiguration, or capacity exhaustion. SIGNAL REQUIRED DIMENSIONS EXAMPLE PROMOTION GATE Negotiated group Source class, destination, listener, build, policy ID >= 99.99% hybrid in a controlled scope; every remainder explained Handshake outcome Error stage, alert, timeout, HRR, retransmit, path class No statistically significant success regression Resource cost Full/resumed, CPU, queue time, p50/p95/p99, saturation p95 within budget and at least 30% CPU headroom Connection reuse Requests/connection, connection age, idle-close reason Reuse ratio meets a workload-specific target Authentication state Server/client certificate algorithm, chain, issuer Authentication coverage reported separately Configuration provenance Canonical policy, provider translation, digest, deploy ID 100% of observed links map to an approved digest Exception state Owner, reason, risk weight, expiry, compensating control No expired exception; weighted coverage increases each release Choose the First Rollout by Risk, Not Convenience Three rollout patterns are useful, and most programs will combine them. Order the work by confidentiality lifetime and capture feasibility so that long-lived, internet-facing health data, credentials, source code, and secrets outrank low-risk endpoints that are merely easy to upgrade. Edge-first reaches the most capturable public segment quickly and produces compatibility data, but downstream re-encryption can remain classical.Mesh-first can move a large controlled workload population through one proxy fleet and provide consistent telemetry, but proxy revisions can diverge, and workload certificates may still use classical signatures.SDK-first protects direct calls to capable cloud services, including selected secrets and key-management paths, but support varies by language runtime and connection pooling can hide whether fresh negotiations occur. Preproduction Checklist Every in-scope TLS link has a source, destination, owner, provider, current group, certificate algorithm, and replacement path.Active probes and runtime telemetry confirm the selected group; capability is not inferred from a library version or policy name.Hybrid key establishment and post-quantum authentication are reported separately.ClientHello fragmentation, HelloRetryRequest behavior, retransmits, and representative middlebox paths have been tested.Handshake success, fallback, p95/p99 latency, full-handshake rate, connection reuse, and CPU headroom gates are defined before rollout.Rollback restores the previous approved policy state and has been exercised in the same environment.Every classical fallback or provider exception has an owner, reason, blast radius, compensating control, and expiration date.Infrastructure as code sets an explicit TLS policy and blocks omitted or legacy defaults.The full trust path - leaf, intermediate, root, and distributed trust bundle - has been validated independently of the key-establishment rollout. Conclusion Post-quantum TLS for cloud APIs is deployable, but it is not an application checkbox. X25519MLKEM768 adds modest compute on modern hardware and a much larger first flight. Connection pooling can reduce the per-request cost by orders of magnitude, while churn, autoscaling, and regional failover can expose it again. The decisive engineering problem is TLS termination. A credible migration inventories every directional link, separates key establishment from authentication, prefers hybrid with observable fallback, tests packet-size and middlebox behavior, enforces explicit policy, and advances only when SLO and rollback gates pass. That operating model turns post-quantum TLS from a library experiment into a measurable cloud security control. Practical milestone: The first definition of readiness is not that every workload is hybrid-only. It is that every important TLS link is visible, owned, risk-ranked, measurable, and reachable through a controlled migration path. Research Basis and Further Reading NIST FIPS 203: Module-Lattice-Based Key-Encapsulation Mechanism StandardNIST SP 800-227: Recommendations for Key-Encapsulation MechanismsIETF draft: Post-quantum hybrid ECDHE-MLKEM Key Agreement for TLS 1.3OpenSSL 3.6 supported-group configurationAWS post-quantum confidentiality readinessCloudflare: Post-quantum between Cloudflare and origin servers.

By Nikhil Papisetty

When Retries Become Failure Amplifiers: The Hidden Cost of Recovery at Scale

The City Traffic Analogy Imagine a city's traffic system during rush hour. A minor accident blocks one lane on a major highway. Traffic slows down, but the road remains partially functional. Now imagine every driver, upon encountering the slowdown, decides to: Take an alternate route through residential streetsCircle back and try the highway againCall friends to warn them, who then also reroute What happens? The residential streets become congested. Side roads jam up. Intersections gridlock. The alternate routes become slower than the original highway. Drivers circle back, creating even more congestion. What started as a single blocked lane cascades into city-wide traffic paralysis. The problem isn't the initial accident. The problem is the collective response to it. This is exactly what happens in distributed systems when retry logic becomes aggressive during degradation. A slow downstream service is like a partially blocked highway. Blind retries are like drivers flooding alternate routes without coordination. The recovery mechanism — designed to help — amplifies the original problem. Traffic engineers solve this with adaptive signals, ramp metering to control highway entry, real-time routing that reflects current conditions, and occasionally closing certain routes entirely to let the system recover. Distributed systems need the same discipline: not just a mechanism to retry, but a policy for when to hold back. The Illusion of "Retry Until It Works" Distributed systems plan for failure using timeouts, circuit breakers, message queues, and dead-letter topics. Among these tools, retries are the most common because the logic seems simple: A call fails.Wait a moment.Try again. For an isolated network drop or server reboot, this works well. But when a dependency struggles under heavy load, a retry is never free. Every retry: Sends another request over the wire.Consumes worker or execution capacity while the request is in flight.Consumes connection-pool capacity while the request remains in flight.Adds another item to process in the queue. When thousands of clients hit the same hiccup and retry together, they create an accidental traffic storm. The recovery code stops being a safety net and becomes the main driver of the outage. How Small Glitches Snowball Consider a hypothetical event platform delivering millions of webhook notifications every hour. Suppose a downstream API slows down under sudden pressure. Instead of returning in 50 milliseconds, responses stretch to 4 seconds. Some calls time out; others return 503 errors. In this scenario, if 100,000 requests fail and each client is configured for 4 retries, the recovery mechanism can generate up to 400,000 additional retry attempts against a target that is already drowning. Plain Text Downstream slows down │ ▼ Requests time out │ ▼ Clients retry │ ▼ Extra inbound traffic │ ▼ Higher downstream latency │ ▼ Widespread failure │ ▼ Retry storm This creates a self-reinforcing loop: the system reacts to failure by generating more work, adding latency, triggering more timeouts, and generating even more retries. The Six-Layer Amplification Framework Most engineers view retry amplification as simple request volume—turning one failed request into several. In reality, the blast radius cuts across multiple infrastructure layers: Plain Text Request Amplification ──► 1 failed call multiplies into 3–5x outbound requests Worker Amplification ──► Saturated execution capacity reduces fresh-work capacity Connection Amplification ──► In-flight requests consume connection-pool capacity Queue Amplification ──► Repeated redelivery increases queue pressure and lag Latency Amplification ──► Shared resource contention slows healthy paths Neighbor Amplification ──► Failing targets starve unrelated platform tenants Requests: One failing transaction multiplies into multiple outgoing calls.Workers: Long-running retries tie up execution capacity, reducing throughput for fresh work.Connections: In-flight requests consume connection-pool capacity while waiting on responses.Queues: Repeated processing and redelivery increase queue pressure and can drive consumer lag.Latency: Resource contention across shared CPU and network paths slows down healthy endpoints.Neighbors: A single failing downstream target hogs shared platform resources, degrading unrelated tenants. The important insight is that retry amplification is not merely a traffic problem; it is a resource-propagation problem. Why Queues Cannot Prevent the Storm Message queues like Kafka, RabbitMQ, and AWS SQS provide durable buffering and asynchronous decoupling, helping systems preserve messages while consumers are unavailable or degraded. However, a queue answers one specific question: "Is the message saved safely in persistent storage?" It cannot answer: "Is it operationally safe to deliver this message right now?" If workers blindly pull messages off a topic and blast them against an overloaded API, the queue acts like a firehose pointed at a fragile target. Queues provide buffering; they do not determine whether delivery should happen now. Failure Is Not Just "Up" or "Down" Basic retry policies assume a binary world: a service is either working or offline. Real production outages are rarely that clean. A downstream dependency might be: Returning 200s, but running ten times slower than normal.Dropping 20% of requests because its database connection pool is full.Rejecting calls because memory crossed a critical limit.Throttling traffic because a tenant hit their API rate limit. If your system treats a slow, memory-starved server like a brief network blip, it will keep hammering that server until it crashes. Matching the Recovery to the Failure Different failure modes require completely different responses: Transient Blips: Dropped packets or DNS timeouts. Retry with exponential backoff and randomized jitter so clients do not hit the server simultaneously.Rate Limits (HTTP 429): The service asks you to slow down. Read Retry-After, back off, cut concurrency, and resume gradually.Downstream Overload: Requests time out under load. Trip circuit breakers, lower concurrency, and pause retries until latency recovers.Non-Retryable Client Errors: Invalid payloads, unsupported operations, or permanent business-rule rejections should not be retried. Route them directly to a dead-letter queue (DLQ) for inspection. Knowing When to Stop Resilient architectures know when doing nothing is safer than trying again. SituationUnsafe ResponseSafe ResponseHTTP 429 Rate LimitImmediate retry loopThrottle affected traffic and respect the limit Retry-AfterCircuit Breaker OpenKeep pinging targetFast-fail or park messages in DLQHigh Queue BacklogRetry all failed jobsDrop or delay low-priority retriesInvalid Message SchemaRetry 5 timesRoute directly to DLQ without retryingDownstream Latency > SLAFire aggressive retriesCut concurrency and shed non-critical load Adaptive Recovery: Check System State First A static policy like retries = 3, backoff = 2s has no awareness of the surrounding environment. Consider two situations handling the same error: Scenario A: Queue depth is 150, CPU is at 20%, downstream latency is 40ms. Retrying is safe.Scenario B: Queue depth is 700,000, workers are 98% busy, downstream latency is 9s. Retrying will trigger an outage. A resilient system checks platform health before firing another request: Plain Text Request Fails │ ▼ Check Failure Type ──► Permanent error? ──► Send to DLQ │ (Transient) ▼ Check Downstream Health (Circuit breaker / Latency) │ (Degraded) ▼ Check Local Load (Queue depth / Worker pool) │ ├─► System healthy ──► Exponential backoff + Jitter ──► Retry │ └─► System stressed ─► Park message / Throttle rate ─► Delay Protecting Shared Resources To keep one broken dependency from pulling down the entire platform: Enforce a Retry Budget: Limit the capacity recovery traffic can consume based on normal workload traffic and available system headroom. When exhausted, delay, throttle, or park retries.Isolate Worker Pools: Process retries on dedicated worker pools so they cannot starve fresh incoming requests.Partition by Target: Apply distinct concurrency limits per downstream endpoint so healthy routes run unaffected.Prioritize Critical Work: When queues back up, retry critical jobs (payments, security alerts) and defer low-priority ones (analytics).Warm Up Slowly: When a failed service recovers, release queued traffic in controlled batches rather than dumping the backlog all at once. What to Measure in Production Standard metrics like error rate and average CPU hide retry problems until damage is done. Track these indicators instead: Retry-to-Primary Ratio: Track how much outgoing capacity is consumed by retries. A sustained increase signals that recovery traffic is becoming a significant source of system load.Retries per Recovery: How many attempts does a message need before succeeding? Rising numbers mean wasted compute.Worker Occupancy: How much worker capacity is spent processing retries instead of fresh work?Circuit Breaker Trip Rate: How often and how long breakers stay open across each downstream target. Summary: Recovery Has a Cost Retrying is not just an error-handling convenience — it is a resource allocation decision. Every recovery attempt consumes bandwidth, threads, memory, and downstream capacity. When systems fail under load, retrying harder often becomes the primary mechanism of collapse. To build genuinely resilient distributed architectures: Treat recovery actions as operations with real infrastructure costs.Filter errors early so permanent failures are never retried.Respect downstream throttling signals and use circuit breakers when defined failure or latency thresholds are reached.Use randomized jitter and exponential backoff to prevent self-inflicted traffic spikes.Check operational system state before deciding whether another attempt will help or harm. True system resilience is not about trying indefinitely until a call succeeds. It is about recognizing that recovery consumes resources, and ensuring every recovery decision accounts for system state before taking action.

By Sayan Chatterjee

Global Setup in Playwright TypeScript for API Testing

If you need to generate prerequisite test data, such as adding dummy records through an API, before executing your test suite, using Global Setup in Playwright is a clean, maintainable, and scalable approach. It ensures all required data is prepared upfront, keeping your test execution structured and reliable. What Is Global Setup? The Global Setup in Playwright runs once before the entire test suite begins execution. It is typically used for activities such as creating test data, seeding the database, generating authentication tokens, preparing the test environment, or cleaning up old data. Since it executes only once at the start of the run, it helps ensure efficient and consistent test initialization across the suite. How to Create a Global Setup in a Playwright TypeScript Project The following steps need to be followed to create a globalSetup in the Playwright project: Step 1: Create a global-setup.ts file in the Root Folder of the Project TypeScript import orders from './test_data/orders.json' with {type: 'json'}; async function globalSetup() { console.log('Running Global Setup...'); const apiContext = await request.newContext(); const response = await apiContext.post("http://localhost:3004/addOrder/", { data: orders, }); if(response.status() !=201) { throw new Error ("Failed to create setup test data!"); } console.log('Test data created successfully'); } export default globalSetup; The globalSetup() function will insert test data into the RESTful e-commerce demo application. The test data for creating new orders is supplied using the following orders.json file. JSON [ { "user_id": "1", "product_id": "79", "product_name": "5 star 10gm Chocobar", "product_amount": 5, "qty": 1, "tax_amt": 0.5, "total_amt": 5.5 } ] The data is required to validate the GET API tests, which will execute immediately after the Playwright test command is triggered. Step 2: Register globalSetup in playwright.config.ts TypeScript import { defineConfig } from '@playwright/test'; export default defineConfig({ globalSetup: './global-setup', }); The globalSetup property tells Playwright to execute the file once before the entire test suite runs. This setup file is typically used to perform pre-test activities such as creating test data or preparing the test environment, ensuring that all tests start with the required initial state. tsconfig.json: JSON { "compilerOptions": { "module": "NodeNext", "moduleResolution": "NodeNext", "resolveJsonModule": true, "types": ["node", "@playwright/test"] } } In this case, “require.resolve()” is not used because the project is configured to use ES Modules ("module": "NodeNext" and "moduleResolution": "NodeNext" in tsconfig.json). Since the project runs in ESM mode, Playwright can directly resolve the setup file using a string path ('./global-setup') instead of relying on CommonJS-based require.resolve(). Step 3: Run the Test The tests are already in place; only global-setup.ts needs to run to feed data into the system before running them. The script for running the tests is updated in the package.json file: JSON "scripts": { "test": "npx playwright test" }, Let’s run the tests by executing the following command in the terminal: Plain Text npm run test The following is a screenshot of the output after running the tests: It can be seen that the test data was created using the global setup before running the tests. How This Helps in Your API Testing Instead of creating the order in the first test and relying on test execution order, we can use a more robust approach by setting up the required data beforehand. By creating test data once during global setup, the tests remain independent, reducing flaky dependencies and unintended failures. This approach aligns with Playwright best practices and helps build a stable, maintainable automation test suite. Important Points to Note About the Global Setup It runs once per test run.It does NOT share variables directly with tests.Use files or environment variables to pass data.Assertions should not be put in the global setup, except for setup validation. Check out how to verify response data with Playwright TypeScript for a detailed understaning on assertions. Summary Global setup in Playwright runs once before the entire test suite and is primarily used for preparing the test environment before running the tests. It helps keep tests independent and avoids relying on execution order. However, since it runs only once and does not directly share variables with tests, any required data must be passed through files, environment variables, or external storage mechanisms. Happy testing!

By Faisal Khatri

CORE

How to Test PUT, PATCH, and DELETE API Requests Using Playwright TypeScript

API testing is the practice of verifying that an API behaves as expected. In Agile environments, teams often perform it early in the SDLC as part of shift-left testing, enabling faster feedback and helping them identify and fix defects early. Playwright is a modern test automation framework that’s popular among testers and developers. Built by Microsoft, it makes it easy to automate both web and API testing across multiple programming languages, all within a single tool. In this tutorial, we will focus on testing APIs using Playwright TypeScript. By the end of this article, you should have a clear understanding of writing automation tests for PUT, PATCH, and DELETE API requests with Playwright TypeScript. Check out the previous articles to learn how to test GET API request and verify response data with Playwright TypeScript. What are PUT Requests? PUT requests are commonly used to update an existing resource by replacing its current data with the data provided in the request body. They are similar to POST requests; the Content-Type header plays an important role because it tells the server how to process the data being sent. When a PUT request is successful, it typically returns a 200 status along with the updated resource in the response. However, this can vary by API design; some APIs may return only a status code without a response body. How to Test PUT API Requests With Playwright TypeScript Playwright offers a request API that lets us create and manage HTTP request contexts easily. We’ll use the PUT /updateOrder API that updates the available order using its order ID. This API is a part of the RESTful e-commerce application available on GitHub. This API requires an authentication token to validate the request and authorize the update; if the token is missing or invalid, the request will fail. The order ID needs to be passed as a path parameter so the correct order can be identified, and the updated details should be included in the request body in JSON format. Since this is a PUT request, you must send the entire order object, not just the field being modified. Even for a small change, the full set of order data is required. We’ll use the following test scenario to demonstrate testing the PUT API request with Playwright TypeScript: Markdown ## Test Scenario Title: Update the existing orders in the system. ### Pre-condition: Valid orders are available in the system #### Test 1. Update all the order details of order_id "1" 2. Verify that the Status Code 200 is returned in the response 3. Assert that the order details have been updated correctly To implement this test scenario, we’ll need to perform the following steps: Hit the Auth API and extract the token.Write a test to send the PUT API request and pass the token extracted in step 1. Step 1: Create a Fixture to Generate and Extract the Token We’ll need the auth token for PUT, PATCH, and DELETE API requests, so let’s create a fixture that hits the POST /auth API and extracts the token for use across other tests. Let’s create a new fixture file auth.fixture.ts, in the fixtures folder in the project’s root directory. TypeScript import { test as base, expect } from "@playwright/test"; import dotenv from "dotenv"; dotenv.config(); type MyFixtures = { token: string; }; export const test = base.extend<MyFixtures>({ token: async ({ request }, use) => { const response = await request.post("http://localhost:3004/auth", { data: JSON.stringify({ username: process.env.USERNAME, password: process.env.PASSWORD, }), headers: { "Content-Type": "application/json", }, }); expect(response.status()).toBe(201); const responseBody = await response.json(); await use(responseBody.token); }, }); export { expect } from '@playwright/test'; A custom Playwright test setup is created by extending the base test object and adding a reusable token fixture. It uses the dotenv library to load environment variables like USERNAME and PASSWORD, ensuring sensitive data is not hardcoded, and defines a TypeScript type to make the token available with proper type safety. A .env file is created containing the credentials and placed in the project’s root folder. This ensures best practices are followed and sensitive information, such as usernames and passwords, is not exposed. As this is a demo API, the credentials used with the POST /auth endpoint are provided below. In a real-time project, the .env file should be added to the .gitignore file to ensure it is not committed to the remote repository. Plain Text #.env USERNAME=admin PASSWORD=secretPass123 Inside the fixture, an authentication API is called to generate a token; the response is validated, and the token is extracted and passed to the tests using the use() function. Finally, expect is re-exported so that both test and expect can be imported from this single fixture file, keeping test imports clean and consistent. Step 2: Updating the Order With a PUT Request and Verifying the Response Let’s create a new spec file to update the order using the PUT /updateOrder API request. TypeScript # 5_put-test.spec.ts import { test, expect } from "../fixtures/auth.fixture.js"; test("Update Order using PUT API Request", async ({ request, token }) => { const updatedOrder = { user_id: "89", product_id: "124", product_name: "Lindt Choclate", product_amount: 50, qty: 3, tax_amt: 10, total_amt: 160, }; const id = 1; const response = await request.put( `http://localhost:3004/updateOrder/${id}`, { data: JSON.stringify(updatedOrder), headers: { "Content-Type": "application/json", Authorization: `${token}`, }, }, ); const responseBody = await response.json(); expect(response.status()).toBe(200); expect(responseBody).toEqual( expect.objectContaining({ message: "Order updated successfully!", order: expect.objectContaining({ product_id: "124", product_name: "Lindt Choclate", }), }), ); }); Code Walkthrough Using the auth fixture for getting the token: The test imports a custom test from the auth fixture, which automatically provides the token as part of the test context. This token is generated before the test runs and injected via Playwright fixtures, eliminating the need to manually authenticate in each test.Request body: The updatedOrder object represents the payload sent in the request to update an existing order. It contains the required fields for the order update request body. This object is later serialized into JSON format before being sent to the API.Sending the PUT request: The request.put() function is used to send an HTTP PUT request to update the order with a specific ID. The request includes the serialized request body and headers such as Content-Type and Authorization, where the token is passed for authentication. This step performs the actual API interaction to update the order in the system.Dynamic endpoint handling: The order ID is stored in a variable and dynamically appended to the API endpoint using template literals. This makes the test more maintainable and reusable.Verifying the response: The response is first converted to JSON, and the status code is validated to be 200, indicating a successful update. Then, assertions are made using expect.ObjectContaining that partially validates the response body, ensuring that the success message and updated order details match expected values. Test Execution Before running the test, the RESTful e-commerce application should be started using Docker by running the following command from the terminal: Plain Text docker compose -f docker-compose-ecommerce.yml up -d The following command should be run from the terminal: Plain Text npx playwright test tests/5_put-test.spec.ts The following test output shows that the test was executed successfully: What Are PATCH Requests? A PATCH request is used when we want to update only part of a resource, not the whole thing. It’s similar to a PUT request, but the main difference is that PUT expects the entire payload, whereas PATCH lets us send only the fields we want to change. How to Test PATCH API Requests With Playwright TypeScript We’ll use the PATCH /partialUpdateOrder API from the RESTful e-commerce application that updates the available order using its order ID. Similar to the PUT API request discussed in the earlier section of this article, the PATCH API also needs an authentication token to update the order. As it is a PATCH request, we don’t need to send the full payload. Only the fields that require updates should be included in the request body. Test Scenario: Markdown ## Test Scenario Title: Partially update an existing orders in the system. ### Pre-condition: Valid orders are available in the system #### Test 1. Update the user_id and product_id for the order ID - 1 2. Verify that the Status Code 200 is returned in the response. 3. Assert that the order details have been updated correctly. The PATCH API request requires an authorization token. We’ll use the token from the auth.fixture.ts file that we generated in the earlier section while discussing the PUT API request. Let’s create a new spec file to partially update an order using the PATCH /partialUpdateOrder API. TypeScript # 6_patch-test.spec.ts import { test, expect } from "../fixtures/auth.fixture.js"; test("Update Partial Order using PATCH API Request", async ({ request, token, }) => { const updatedOrder = { user_id: "98", product_id: "214", }; const id = 1; const response = await request.patch( `http://localhost:3004/partialUpdateOrder/${id}`, { data: JSON.stringify(updatedOrder), headers: { "Content-Type": "application/json", Authorization: `${token}`, }, }, ); const responseBody = await response.json(); expect(response.status()).toBe(200); expect(responseBody).toEqual( expect.objectContaining({ message: "Order updated successfully!", order: expect.objectContaining({ user_id: "98", product_id: "214", }), }), ); }); Code Walkthrough Using the auth fixture for getting the token: The test uses a custom fixture that automatically provides a token, so we don’t have to call the POST /auth API every time. The token is generated before test execution and injected into the test context, ensuring secure and reusable authentication across multiple tests.Request body: The updatedOrder object contains only the fields user_id and product_id that need to be modified. As it is a PATCH request, there is no need to send the complete order payload; this makes the request lightweight and focused on partial updates.Sending the PATCH request: The request.patch() function is used to send a partial update request to the API endpoint, with the order ID dynamically added to the URL. The request includes the JSON-stringified body along with the Content-Type and Authorization headers. The token is passed in the Authorization header.Verifying the response: The response is parsed into JSON and validated to ensure a 200 status code is returned after the PATCH API is hit successfully. This verification ensures that the operation was successful. Then, expect.objectContaining is used to validate the response body, confirming that the success message and updated fields match the expected values. Test Execution To run the test, the following command should be executed from the terminal: Plain Text npx playwright test tests/6_patch-test.spec.ts The following screenshot of the test execution confirms that the test was run successfully: What Are DELETE Requests? A DELETE API request is used for removing a specific resource from the server. In most cases, it doesn’t return a response body. The resource is identified using a URI, and once the request is processed, it’s permanently deleted. Because of this, DELETE operations can have side effects, like removing records from a database. A DELETE request should be handled carefully because: Data deleted through a DELETE request is permanently removed and cannot be restored.It directly impacts the server data; hence, it is not considered a safe method.It may lead to inconsistencies if not managed appropriately.DELETE isn’t always idempotent in real-world scenarios. For example, the first DELETE request might return a 204 (No Content) indicating success, while calling it again on the same resource could return 404 (Not Found) because the resource has already been removed. How to Test DELETE API Requests With Playwright TypeScript We’ll use the DELETE /deleteOrder API endpoint from the RESTful e-commerce application for this demonstration. This API requires an order_id as a path parameter to identify the order to delete. A valid authentication token must be included in the headers. On successful execution, the order is removed, and a 204 (No Content) status is returned. Want to learn more about API Testing? Check out this tutorial to get hands-on API Testing Test Scenario: Markdown ## Test Scenario Title: Delete existing order from the system. ### Precondition - There should be some valid orders available in the system. #### Test Steps: 1. Delete the order with order_id "6" using the DELETE request. 2. Verify that 204 Status Code is returned in the response. To implement this test scenario, we need valid orders in the system, which can be created using the POST /addOrder API. Refer to the earlier tutorial on testing POST requests with Playwright TypeScript to explore different ways to generate request payloads and test the POST API request. Test Implementation Let’s create a new spec file to delete an order using the DELETE /deleteOrder API. TypeScript import { test, expect } from "../fixtures/auth.fixture.js"; test("Delete an order using DELETE API Request", async ({ request, token }) => { const id = 6; const response = await request.delete( `http://localhost:3004/deleteOrder/${id}`, { headers: { Authorization: `${token}`, }, }, ); expect(response.status()).toBe(204); }); Code Walkthrough Using the auth fixture for getting the token: The test imports a custom test from the auth fixture, which automatically provides a valid authentication token. This token is generated before the test runs and injected into the test context, so there’s no need to call the POST /auth API explicitly before the DELETE API.Sending the DELETE request: The request.delete() function is used to send a DELETE request to the API endpoint, with the ID dynamically passed in the URL to identify the order to be deleted. The request includes the Authorization header, where the token is sent to authenticate the request. No request body is required for this operation.Assertion: The test verifies that the response status is 204, which indicates that the request was successfully processed and the resource was deleted. Since DELETE APIs often don’t return a response body, validating the status code is sufficient to confirm success. Test Execution To run the test, run the following command from the terminal: Plain Text npx playwright test tests/7_delete-test.spec.ts The following screenshot confirms that the test ran successfully: A simple, practical approach is to call the GET API after running a DELETE request to confirm the resource has been removed from the system. Summary Testing PUT, PATCH, and DELETE APIs with Playwright and TypeScript helps ensure that the application handles updates and deletions correctly. While challenges such as managing test data, handling authentication, and maintaining test independence can arise, Playwright’s fixtures and built-in request handling make it easier to create clean and reusable tests. The main advantage is the ability to validate end-to-end API behavior with clear assertions and strong TypeScript support. In my experience, the tests should be isolated, avoid unnecessary dependencies between them, and validate both status codes and response data to make the API tests more robust and maintainable. Happy testing!!

By Faisal Khatri

CORE

Builder Pattern + Faker in API Test Automation Using Playwright With TypeScript

Ever felt stuck writing Playwright tests because of messy or hardcoded test data? It usually starts simple, but as your tests grow, like testing an e-commerce or finance-related application, managing realistic data quickly becomes confusing and repetitive. More time is spent fixing test data than actually testing. That’s where the Builder Pattern, combined with Faker, can really help. It lets us create clean, flexible, and realistic test data with ease. In this article, we’ll learn how this approach can make the API tests with Playwright TypeScript easier to write, read, and maintain. What Is the Builder Design Pattern? The Builder Design Pattern is a creational design pattern that allows constructing complex objects step by step, allowing different representations of the same object. It separates object-creation logic from structure, making the code more readable and maintainable. This pattern is especially useful for objects with many optional parameters, as it avoids messy constructors and improves test data creation. What Is Faker? Faker is a library that generates realistic-looking fake data such as names, emails, addresses, numbers, and more. It is commonly used in testing to avoid hardcoding values and to simulate real-world data scenarios. By combining the Builder Pattern with Faker, we can generate realistic test data on the fly and perform testing without worrying about test data management, duplication, or maintenance. Faker can be installed using npm by running the following command in the terminal: Plain Text npm install @faker-js/faker --save-dev After installation, we can use the Faker library directly in the code by using the following import: TypeScript import {faker} from '@faker-js/faker'; How to Use the Builder Pattern With Faker in Playwright With TypeScript To use the Builder design pattern with Faker in Playwright TypeScript, the setup usually includes two main parts: ModelBuilder class The Model defines how the data should look, while the Builder class handles creating and customizing that data using Faker. Application Under Test We’ll use the POST /addOrder API of the RESTful e-commerce demo application for the demonstration. The following is the request payload of the API: JSON [ { "user_id": "string", "product_id": "string", "product_name": "string", "product_amount": 0, "qty": 0, "tax_amt": 0, "total_amt": 0 } ] Creating a Model for the Order The following model interface should be created for the Order payload: TypeScript export interface Order { user_id: string, product_id: string, product_name: string, product_amount: number, qty: number, tax_amt: number, total_amt: number } Code Walkthrough The Order interface represents the structure of an order object, where each field in the interface specifies the expected property and its data type: user_id, product_id, and product_name are strings.product_amount, qty, tax_amt, and total_amt are numbers. This interface ensures that any order object created in the code follows this exact structure, helping catch errors early and making the code more predictable and easier to maintain. Creating the Builder Class The following builder class is created to generate a new order with realistic fake values using the Faker library. TypeScript import { Order } from "../models/order.model.js"; import {faker} from '@faker-js/faker'; export class OrderBuilder { private order:Order; constructor() { const product_amount = faker.number.int({ min: 1, max: 100 }); const qty = faker.number.int({ min: 1, max: 5 }); const tax_amt = faker.number.int({ min: 2, max: 10 }); this.order = { user_id: this.generateUserId(), product_id: this.generateProductId(), product_name: faker.commerce.productName(), product_amount, qty, tax_amt, total_amt: (product_amount*qty)+tax_amt } } private generateUserId():string { return faker.number.int({min: 1, max: 50}).toString(); } private generateProductId():string { return faker.number.int({ min: 1, max: 100 }).toString(); } withUserID(user_id:string) { this.order.user_id = String(user_id); return this; } withProductId(product_id:string) { this.order.product_id=String(product_id); return this; } withProductName(product_name:string) { this.order.product_name = String(product_name); return this; } withProductAmount(product_amount:number) { this.order.product_amount= Number(product_amount); return this; } withQty(qty:number) { this.order.qty=Number(qty); return this; } withTaxAmt(tax_amt:number){ this.order.tax_amt=Number(tax_amt); return this; } withTotalAmt(total_amt:number) { this.order.total_amt= Number(total_amt); return this; } build():Order{ return this.order; } } Code Walkthrough The OrderBuilder class implements the Builder Pattern to create an Order object using Faker. The following is a detailed explanation of the code: Constructor(): The OrderBuilder class internally holds an Order object, which is based on the Order interface. In the constructor, it generates default random data using the Faker library.Private functions: The private functions, namely, generateUserId() and generateProductId(), are helper methods that generate random user IDs and product IDs, respectively.with() functions: The withX() functions, such as withUserId(), withProductId(), etc., allow overriding specific fields. Each method updates a value and “returns this,” enabling method chaining.Build functions: The build() function method finally returns the fully constructed Order object. In short, this class generates realistic test data by default while allowing selective customization, resulting in cleaner and more reusable test code. Writing the API Automation Test Using Playwright The following Playwright test demonstrates how to send a POST request using an order request payload generated through the Builder Pattern with the Faker library. TypeScript import { test, expect } from "@playwright/test"; import { OrderBuilder } from "../builders/order.builder.js"; import { Order } from "../models/order.model.js"; test("Add Order details using Builder Design Pattern", async ({ request }) => { const orderDetails = new OrderBuilder().build(); const orders: Order [] = [orderDetails]; console.log(orders); const response = await request.post("http://localhost:3004/addOrder/", { data: orders, headers: { "Content-Type": "application/json", }, }); expect(response.status()).toBe(201); }); Code Walkthrough The test imports OrderBuilder to create test data, and the Order interface to ensure type safety.Inside the test, new OrderBuilder().build() generates a complete order object with realistic values using Faker.The generated order is wrapped in an array, orders: Order[] = [orderDetails], as the API expects a list of orders. A console log statement is added to display the generated order in the console.A POST request is sent to the endpoint using Playwright’s request fixture, with the order payload passed in the data field and the appropriate Content-Type header.The test validates that the response status is 201, confirming that the order was successfully created. Test Execution Before running the test, start the RESTful e-commerce application using Docker by running the following command from the terminal: Plain Text docker compose -f docker-compose-ecommerce.yml up -d The test can be executed by running the following command from the terminal: Plain Text npx playwright test tests/post-api-builder-test.spec.ts The following is the output of the test execution: The order details generated using the Builder design pattern are printed on the console with fake data and are used as the request payload in the POST /addOrder test. Writing automated tests by combining generated fake data with selectively updated static values. In the previous test scenario, all order details were generated using the Faker library. However, in real testing scenarios, certain fields may require static values. The Builder Pattern supports this by allowing specific fields to be overridden with static data, while the remaining fields continue to be generated dynamically using Faker. TypeScript test("Add Specific Order details using Builder Design Pattern", async ({ request }) => { const orderDetails = new OrderBuilder().withUserID("56").withProductName("Donuts").build(); const orders: Order [] = [orderDetails]; console.log(orderDetails); const response = await request.post("http://localhost:3004/addOrder/", { data: orders, headers: { "Content-Type": "application/json", }, }); expect(response.status()).toBe(201); }); In this test, the user_id and product_name fields are explicitly set using the withUserID() and withProductName() functions. While the Builder generates the rest of the order details using the Faker library, these two fields are overridden with the static value provided in the test. Let’s execute the test again and check the output: In the above screenshot, it can be seen that the order details are generated successfully using the Faker library; however, the value for user_id is set to "56" and product_name is set to "Donuts," as these were explicitly provided as static values for those fields. Learn how to use the Builder Design Pattern in Java with a Datafaker library to generate test data for automated testing. Summary The Builder Design Pattern is really helpful for creating clean, flexible, and maintainable test data in API automation. When combined with Faker, it enables the generation of realistic data while still allowing specific fields to be customized as needed. There are multiple ways to supply test data in API automation for POST API testing, such as using JSON files or directly generating data with the Faker library in the request payload without the Builder Pattern. However, the choice of approach depends on the application under test, the level of flexibility required, and how the test framework is designed. Happy testing!

By Faisal Khatri

CORE

Integration

DZone's Featured Integration Resources

Monthly Top Integration Experts

The Latest Integration Topics