AI/ML

Most Foundry writeups assume you're all in on Microsoft's stack end to end: the Agent Framework for orchestration, the Foundry Agent Service for hosting, and the Responses API wrapped in Microsoft's own client. That's a reasonable default, but it's not the only shape this can take. Microsoft Foundry hosts OpenAI's own models behind an OpenAI-compatible endpoint, and Foundry IQ exposes every knowledge base as a plain MCP server. Put those two facts together, and you get a genuinely different setup: OpenAI's own Agents SDK, unmodified, orchestrating a model that happens to be running on Foundry, grounded by a knowledge base that happens to be Foundry IQ, with MCP as the only thing that has to agree between them. This is a hands-on guide to building exactly that. Not because you should always prefer OpenAI's SDK over Microsoft's own tooling, but because knowing this path exists changes how you think about lock-in. If your orchestration layer is a thin, protocol-based client, swapping the model host or the knowledge layer underneath it is a config change, not a rewrite. The Mental Model First Three pieces, from three different places, held together by two protocols: An OpenAI model, hosted on Microsoft Foundry. Foundry deploys OpenAI's models behind an endpoint that speaks the same wire format as OpenAI's own API, including the Responses API. Point any OpenAI-compatible client at that endpoint with a different base_url and it has no idea it's not talking to OpenAI directly.A Foundry IQ Knowledge Base, doing the same job it always does: chunking, embedding, indexing, and agentic retrieval over your sources. What matters here is that every knowledge base speaks MCP natively. It doesn't care what called it.The OpenAI Agents SDK, running as your orchestration layer, in your own process, not inside Foundry at all. It calls the Foundry-hosted model for reasoning and generation, and calls the Foundry IQ Knowledge Base as an MCP tool for grounding. Neither call requires Microsoft-specific code. The thing worth sitting with here: nothing about this setup is a workaround or an unsupported hack. Foundry explicitly documents the OpenAI SDK as the recommended client when you want maximum OpenAI compatibility or the lowest latency path to a Foundry-hosted model. Foundry IQ explicitly exposes MCP as a first-class interface, not an afterthought. This guide is just connecting two things that were each already built to be connected this way. Prerequisites You'll need: A Microsoft Foundry project with an OpenAI model deployed (a gpt-5.1 or similar deployment, created through the Foundry portal or the Foundry SDK).A Foundry IQ Knowledge Base already built and populated. If you haven't done this before, the short version is a Knowledge Source pointed at your data plus a knowledge base wrapping it, both created through azure-search-documents. The full walkthrough, chunking strategy, semantic configuration, and all, is worth its own read if you're starting from zero.Python 3.10+ with the OpenAI SDK and the Agents SDK installed. Shell pip install openai openai-agents azure-identity Step 1: Point a Plain OpenAI Client at Your Foundry Deployment Before bringing the Agents SDK into it, confirm the basic connection works with the plain OpenAI client. This is the part that trips people up the least, but it's worth isolating as its own step, because if it doesn't work here, nothing built on top of it will either. Python from openai import OpenAI from azure.identity import DefaultAzureCredential, get_bearer_token_provider token_provider = get_bearer_token_provider( DefaultAzureCredential(), "https://ai.azure.com/.default" ) client = OpenAI( base_url="https://YOUR-RESOURCE-NAME.services.ai.azure.com/api/projects/YOUR-PROJECT/openai", api_key=token_provider, ) response = client.responses.create( model="gpt-5.1", input="Say hello in one sentence.", ) print(response.output_text) Two things to get right here. The base_url is your Foundry project endpoint with /openai on the end, not the raw resource endpoint, and not the older /openai/v1/ Azure OpenAI-specific path (that one still works for Azure OpenAI resources, but the project endpoint is the current recommended shape for Foundry). And api_key accepts a callable token provider, not just a string, which is how Entra ID authentication slots in without you having to manually refresh anything. Step 2: Swap in the Token Provider and Hand the Client to the Agents SDK The Agents SDK doesn't have its own concept of Azure authentication. It just needs an AsyncOpenAI client, and it doesn't care where that client points. Python from openai import AsyncOpenAI from agents import set_default_openai_client, set_tracing_disabled async_client = AsyncOpenAI( base_url="https://YOUR-RESOURCE-NAME.services.ai.azure.com/api/projects/YOUR-PROJECT/openai", api_key=token_provider, ) set_default_openai_client(async_client) One gotcha worth flagging immediately: the Agents SDK ships with built-in tracing that exports run traces to OpenAI's own platform dashboard by default. That's a sensible default when you're calling OpenAI directly, but it's an odd one once your model calls are routed through Foundry instead, since your traces would still be leaving through a separate, OpenAI-direct path that doesn't share your Foundry project's auth or data boundary. If that matters for your compliance posture, disable it or point it at your own collector: Python set_tracing_disabled(True) # or, to keep tracing but redirect it, register a custom trace processor instead This is easy to miss because nothing breaks if you leave it on. It just quietly sends run metadata somewhere your Foundry-hosted setup otherwise never touches. Step 3: Connect to the Knowledge Base Over MCP Every Foundry IQ Knowledge Base exposes itself at a predictable MCP endpoint. The Agents SDK's MCPServerStreamableHttp class is built for exactly this kind of self-managed, HTTP-based MCP server. Python from agents.mcp import MCPServerStreamableHttp, create_static_tool_filter kb_server = MCPServerStreamableHttp( name="foundry-iq-kb", params={ "url": "https://YOUR-SEARCH-SERVICE.search.windows.net/knowledgebases/team-kb/mcp?api-version=2026-05-01-preview", "headers": {"api-key": "YOUR-SEARCH-ADMIN-KEY"}, "timeout": 15, }, cache_tools_list=True, tool_filter=create_static_tool_filter(allowed_tool_names=["knowledge_base_retrieve"]), ) cache_tools_list=True is worth defaulting to here. A knowledge base publishes exactly one tool, knowledge_base_retrieve, and that isn't going to change between requests, so there's no reason to pay a tools/list round trip on every single agent turn. The tool_filter is mostly redundant given there's only one tool to begin with, but it's cheap insurance if the knowledge base ever grows to a second tool you don't want this particular agent touching. Step 4: Build the Agent and Run It With the client and the MCP server both wired up, the agent itself is short. Python import asyncio from agents import Agent, Runner async def main(): async with kb_server as server: agent = Agent( name="support-agent", instructions=( "Answer questions using the knowledge_base_retrieve tool. " "Always call it before answering. Preserve [ref_id:N] citations " "from the tool's response in your final answer." ), model="gpt-5.1", mcp_servers=[server], ) result = await Runner.run(agent, "What's our current rate limit on the export API?") print(result.final_output) asyncio.run(main()) The model string here is the Foundry deployment name, not an OpenAI model ID, since every call now routes through the client you registered in Step 2. If you want to stream the response instead of waiting for the full turn, Runner.run_streamed gives you the same event-based streaming interface regardless of which backend is actually generating the tokens: Python result = Runner.run_streamed(agent, "What's our current rate limit on the export API?") async for event in result.stream_events(): if event.type == "raw_response_event" and hasattr(event.data, "delta"): print(event.data.delta, end="", flush=True) Nothing in either of these two blocks is Foundry-specific or Azure-specific. That's the point. The vendor-specific work all happened in Steps 1 through 3, in the client and connection setup, not in how you define or run the agent. Where the Credentials Actually Live Two separate credentials are doing two separate jobs here, and it's worth being precise about which is which, because they fail differently. The model credential is whatever you passed as api_key on the AsyncOpenAI client, a Foundry project token from DefaultAzureCredential, or a static API key if you're using key-based auth on the resource. This is checked on every responses.create() call the Agents SDK makes internally when the agent reasons or generates a final answer. Scope this to the project, not the whole Foundry resource, using the same RBAC roles you'd use for any other Foundry SDK client (Cognitive Services User is usually sufficient for inference-only access). The knowledge base credential is the api-key header on the MCP server's params, and it's checked independently by Azure AI Search when the knowledge_base_retrieve tool gets called. These two credentials can be, and generally should be, scoped to completely different principals. A key that can call your Foundry model deployment shouldn't automatically be able to query every knowledge base on your Search service, and the reverse is just as true. If you're building anything past a prototype, put each behind its own least-privilege identity rather than reusing one Foundry project's admin key for both. If your knowledge base sits over permission-sensitive content, this is also where the on-behalf-of pattern from Foundry IQ's own permission model applies unchanged: thread the requesting user's token through as an additional header on the MCP params, since the KB's enforcement of ingestionPermissionOptions doesn't know or care that the caller this time is the OpenAI Agents SDK instead of the Foundry Agent Service. Production Considerations Before You Commit Decide on tracing deliberately, not by default. Leaving the Agents SDK's tracing on means run metadata leaves through an OpenAI-direct path that bypasses your Foundry project's boundary entirely. Turn it off or replace it with a custom processor as a first-day decision, not something you notice in a security review months later.Cache the tool list, but know when to invalidate it. cache_tools_list=True avoids a redundant round trip, but if you ever change what a knowledge base exposes, which is rare but not impossible as Foundry IQ's MCP surface evolves, a long-lived process holding a stale cached tool list will keep calling the old shape until it's restarted.Separate the model deployment's quota from the knowledge base's query load. These are billed and throttled independently. A burst of retrieval-heavy queries against the knowledge base won't show up as pressure on your model deployment's tokens-per-minute limit, and the reverse is also true, so alert on both rather than assuming one is a proxy for the other.Pin the MCP API version. The api-version=2026-05-01-preview query parameter on the knowledge base's MCP URL is still a preview surface as of this writing. Track it the same way you'd track any other preview dependency, and don't assume a bare /mcp URL without a version pin will behave identically across a Foundry IQ update.Keep the instructions honest about tool use. The Agents SDK does not force a tool call. If your instructions say "always call knowledge_base_retrieve" but the model decides a question doesn't need it, you'll get an ungrounded answer with no error. Log whether the tool was actually invoked on each run, not just what the final answer said, if grounding is a correctness requirement rather than a nice-to-have. Where This Leaves You The interesting thing this setup demonstrates isn't that OpenAI's SDK can technically reach a Foundry endpoint. It's that both vendors built their integration points — an OpenAI-compatible inference endpoint on one side, an MCP-native knowledge base on the other — generally enough that they compose without either one knowing the other exists. That's a genuinely different bet than the usual platform story, where the value proposition is staying inside one vendor's tooling end to end. If you're already committed to the OpenAI Agents SDK for orchestration, whether for its tracing, its handoff model, or just team familiarity, you don't have to give that up to use Foundry-hosted models or Foundry IQ's retrieval layer. The protocol boundary is the only thing that has to hold, and both sides are already built to it. References Microsoft Learn. "Get started with Microsoft Foundry SDKs and endpoints." learn.microsoft.com/en-us/azure/foundry/how-to/develop/sdk-overviewMicrosoft Learn. "Use the Azure OpenAI Responses API." Microsoft Foundry. learn.microsoft.com/en-us/azure/foundry/openai/how-to/responsesMicrosoft Learn. "How to migrate from Azure AI Inference SDK to OpenAI SDK." Microsoft Foundry. learn.microsoft.com/en-us/azure/foundry/how-to/model-inference-to-openai-migrationOpenAI. "Configuration." OpenAI Agents SDK documentation. openai.github.io/openai-agents-python/configMicrosoft Learn. "What is Foundry IQ?" learn.microsoft.com/en-us/azure/foundry/agents/concepts/what-is-foundry-iq

By Jubin Soni, FBCS CORE

Picture a checkout page throwing the dreaded 500 error at 2 a.m. Someone opens an AI agent and asks it to fix things. The instinct is to be generous. Paste in the runbooks. Drop in three dashboards. Attach a pile of customer complaints. Let the model sort it out. More context should mean a smarter answer. Right? Not really. Researchers who studied how language models actually use long inputs found something inconvenient for anyone who pastes first and thinks later. "Performance can degrade significantly when changing the position of relevant information" Source: Liu et al., "Lost in the Middle: How Language Models Use Long Contexts," arXiv:2307.03172 In plain English, an AI agent buried under ten documents isn't automatically wiser than one working with the two documents that actually matter. The fix for agentic AI isn't a bigger context window. It's giving the agent the right kind of knowledge, delivered the right way, at the right moment. There are four main ways to do that: skills, MCP, RAG, and memory. Each one solves a different problem. Mixing them up is where a lot of enterprise AI projects quietly go wrong. Skills: The Onboarding Manual Nobody Wrote A skill is a set of instructions for doing one specific job, sometimes with a bit of code attached. Think of it as the manual you'd hand a sharp new hire on day one. Check the error rate first. Then check recent deployments. If neither explains the problem, stop guessing and escalate to a human. Without that manual, a capable model will improvise, and improvisation is exactly what you don't want during an outage. A good skill doesn't just list steps. It carries judgment about when to follow them and when to stop. The trick is that skills only load when they're relevant, which is what keeps them cheap. Anthropic, which built this pattern into Claude, puts it plainly. Only relevant content occupies the context window at any given time (Source: Anthropic, Agent Skills documentation, ) A library of fifty skills doesn't cost fifty skills worth of context. It costs one: the one the agent actually needed for this task. But a skill can't reach outside itself. It can tell an agent to check the error rate. It can't get the agent to the dashboard. MCP: Giving the Agent Hands That's where the Model Context Protocol, or MCP, comes in. MCP is a standard way for an agent to connect to outside systems: logging tools, databases, ticketing systems, whatever a company already runs. The agent is the "host." Each connected system sits behind an "MCP server" that knows how to talk to it. Before MCP, wiring an AI assistant into five internal tools meant writing five custom integrations, then doing it again for the next assistant. Anthropic built MCP to close exactly that gap. An open standard that enables developers to build secure, two-way connections (Source: Anthropic, "Introducing the Model Context Protocol," ) Back to the checkout error. With MCP wired up, the agent doesn't just know it should check the error rate. It can go pull the number from the logging stack and the metrics dashboard itself. That solves the access problem. It doesn't solve judgment. MCP hands the agent raw numbers. It has no opinion about whether last month's number was normal for this particular system. RAG: The Library Card That's where retrieval-augmented generation, or RAG, earns its keep. Instead of stuffing every manual and dependency map into the prompt up front, RAG lets the agent search a collection of documents and pull back only the passages that match the question, using semantic search rather than a keyword match. The original RAG paper, published by Facebook AI researchers back in 2020, was blunt about the problem it set out to solve. Language models are good at sounding confident. They're less reliable at being precise. Their ability to access and precisely manipulate knowledge is still limited (Source: Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks") For the checkout bug, RAG might point the agent to the exact paragraph in the payment gateway's integration guide that explains why timeouts spike under load, buried in a document nobody on the team has fully read in a year. RAG is knowledge a person deliberately wrote down and filed away. It's only as good as what gets filed. Memory: What the Agent Remembers From Last Time Memory looks a lot like RAG from a distance. Both retrieve relevant information on demand. The difference is where the knowledge comes from. RAG pulls from documents a person wrote. Memory pulls from what the agent picked up itself while working. Say this exact 500 error happened three months ago, and the real cause turned out to be a stale feature flag nobody had documented anywhere. Memory is what lets the agent recall that the hard way this time, then write the fix back down for next time. A recent academic survey framed the shift well. Memory is increasingly the substrate through which agents self-evolve (Source: "A Survey of Agent Memory in the Second Half") That's the piece a pure RAG setup or a static skill file can't give you. An agent that gets measurably better at your specific systems the longer it works on them, instead of starting fresh every time. Four Tools, One Simple Rule None of these four replace each other. None of them do much alone beyond a demo. A rough rule of thumb, borrowed from how production teams tend to use them: Knowledge someone wrote down on purpose: RAGKnowledge the agent picked up from experience: memoryA repeatable procedure with judgment attached: a skillA way to reach the outside world without custom glue code: MCP Most serious agentic systems end up using all four together. The skill tells the agent what steps to follow and when to escalate. MCP gets it into the logging and metrics tools. RAG surfaces the relevant page from the documentation nobody memorized. Memory remembers that this exact error showed up before, and what actually fixed it. Skip any one layer and the agent falls back to guessing. That's the same problem as throwing everything into the context window in the first place, just with better manners. Choosing Which One to Build First The four rarely get built at once, and deciding where to start is one of the harder calls in enterprise AI architecture. Get the skill and the connective tissue right before reaching for a memory layer, and the sequencing tends to hold up. Build memory first, on top of a shaky procedure, and the agent will remember the wrong lessons very efficiently. This is the kind of tradeoff Faisal Feroz works through regularly as a Chief Technical Architect and Fractional CTO, helping enterprise teams turn legacy platforms into AI-first, event-driven systems. Readers weighing the same decisions on their own stack can find more of his writing on enterprise AI architecture at fferoz.medium.com, or connect with him on LinkedIn at linkedin.com/in/faisalferoz to talk through where skills, MCP, RAG, or memory actually fit.

Context engineering is becoming essential as AI agents take on more software development work. An agent can plan, code, test, investigate incidents, trigger CI, and help deploy software. But none of that matters if it is operating without the right information. This is the main problem I keep seeing. We connect an LLM to a few tools, give it a good prompt, and expect magic. Then the agent has to figure out which service we mean, who owns it, what repository it belongs to, whether it is healthy, what incidents are open, and whether a deployment is safe. That is a lot of disconnected information to reconstruct every single time. Context engineering is the discipline of structuring, surfacing, and governing the information an AI agent needs to act reliably. It is how we give agents the right facts, rules, tools, and boundaries so they can make better decisions without hallucinating or wasting time hopping between systems. Key Takeaways Context engineering gives AI agents structured access to instructions, knowledge, memory, examples, tools, and guardrails.A context layer reduces tool switching and prevents agents from wasting effort interpreting disconnected SDLC data.Service catalogs, reusable skills, and human approval gates make agentic workflows more reliable and governable.Deployment recommendations should be grounded in visible evidence such as ownership, health, test coverage, runbooks, and incidents. Step 1: Understand What Context Engineering Actually Means AI agents are powered by LLMs. The LLM is basically the brain, but it does not automatically know the current state of your engineering organization. It does not know your service ownership, deployment history, runbooks, incident status, infrastructure, or internal policies unless you provide that information. That is where Context Engineering comes in. Instead of leaving an agent to guess, I give it access to relevant, organized context. This helps it plan properly, use tools properly, and take actions with much more accuracy. A simple way to think about it is this: Without context: An agent guesses what a service is, where its data lives, and what action is safe.With context: An agent can retrieve the service record, ownership, health, repository, runbook, deployment data, and guardrails before responding or acting. Context engineering is not just about putting more tokens into a prompt. It is about making the right information accessible at the moment an agent needs it. The goal is grounded actions, not longer conversations. Step 2: Identify Why Your Engineering Team Needs a Context Layer Most engineering ecosystems are distributed by design. Source code might live in GitHub, documentation in Notion, incidents in PagerDuty, conversations in Slack, infrastructure in AWS, observability in Datadog, and deployments in Kubernetes. Each tool is useful. The issue is that the knowledge is fragmented. For a developer, that fragmentation creates constant context switching. To understand one service, I may need to open a repository, find its owner, inspect deployment history, search incident records, locate the runbook, and check infrastructure health. That slows down delivery and increases the chance of missing something important. For an AI agent, the problem becomes even bigger. If I ask it to analyze bottlenecks, delivery velocity, quality gaps, and patterns across the SDLC, it may need to fetch and interpret data from every one of those disconnected systems. It spends tokens trying to understand the environment before it can solve the actual task. A context layer sits between the engineering ecosystem and the agent. It connects services, teams, workflows, documentation, policies, and operational data in one place. With that layer in place, context engineering can improve: Deployment speed and confidenceAccuracy in agent responses and actionsSecurity and policy enforcementOperational reliabilityCollaboration across teamsDeveloper productivity by reducing tool switching The point is not to replace every engineering tool. The point is to let humans and agents access the relevant context without manually rebuilding the story every time. Step 3: Fix Engineering Chaos Before It Turns Into Agentic Chaos The software development lifecycle has many stages: planning, coding, building, testing, securing, deploying, operating, learning, and improving. Teams commonly introduce specialized tools at every stage. Over time, that creates tool sprawl, duplicate data, fragmented workflows, and unclear ownership. I call that engineering chaos. It hurts quality, security, compliance, productivity, and operational excellence. Now add AI agents on top of that environment. If every agent is independently connected to different tools and given incomplete instructions, the chaos gets multiplied. Agents may have no shared visibility, no human approval points, inconsistent decisions, and no meaningful safeguards. This is why context engineering should begin with a simple question: What does an agent need to know before it can safely answer or act? For example, if I ask an agent, “Analyze my SDLC data and surface bottlenecks, velocity, quality gaps, and interesting patterns,” the agent needs more than a prompt. It may need: Repository and pull request data from GitHubInfrastructure context from AWSIncident data from PagerDutyOperational discussions from SlackDeployment state from Kubernetes Without a unified context model, the agent must interpret isolated facts from every system. That consumes tokens and can lead to weak, incomplete, or incorrect conclusions. Context Engineering gives that agent a better starting point. Step 4: Build the Six Types of Agent Context When I design context for an AI agent, I think in six categories. Each category answers a different part of the agent’s decision-making problem. 1. Instructions Instructions define rules, goals, and boundaries. They tell an agent what its job is and what it should not do. For example, an incident investigation agent may be instructed to gather evidence, summarize findings, and avoid triggering production actions. 2. Knowledge Knowledge includes documents, architecture diagrams, service metadata, domain data, repositories, and runbooks. This is the factual material an agent needs to understand the environment. 3. Memory Memory holds session logs, previous decisions, and persistent state. It lets an agent maintain continuity across multi-step workflows rather than treating every action as a completely new task. 4. Examples Examples provide short demonstrations and reference patterns. They show an agent what a useful output or a correct workflow looks like. This is especially useful when a task needs a consistent format. 5. Tools Tools include APIs, scripts, CI systems, and external services. Tools turn an agent from a chat interface into something that can retrieve current data and execute approved tasks. 6. Guardrails Guardrails are the hard constraints: safety rules, checklists, policy requirements, and approval gates. They are critical when an agent can do more than just answer a question. Instructions, knowledge, and memory are generally more static forms of context. Examples, tools, and guardrails are dynamic because they can change with the workflow, the service, and the current situation. Effective Context Engineering brings all six together instead of relying on a single prompt. Step 5: Separate Prompt Engineering From Context Engineering Prompt engineering and context engineering work together, but they solve different problems. Prompt engineering is about what to say. It focuses on the instructions and examples used to guide an interaction. It is useful for optimizing a single request or response. Context engineering is about what the agent gets to see. It focuses on managing accessible information across a workflow: the service data, connected tools, policies, history, ownership, and real-time status the agent needs. A great prompt cannot compensate for missing operational facts. If an agent does not know the owning team, service tier, runbook, open incidents, or deployment policy, no clever wording will make its production decision trustworthy. Step 6: Create a Service Catalog That Gives Agents Grounded Context To make Context Engineering practical, I need a system that represents the services in my environment and connects their information. In the demo, I use Port.io as a context layer for an agentic SDLC. A service catalog can hold details such as: Service name and identifierEnvironment, such as staging or productionOwning teamRepository associationRunbook URLSlack channelService tier and visibilityObservability linksOn-call rotation status Once this context is registered, an agent can answer a question like “Share everything about the shipment service” by retrieving a unified service overview. In the example, that overview includes the owning team, language, repository, branch, recent code activity, runbook, on-call status, health information, deployments, pull requests, and scorecard data. This is the practical value of context engineering. Instead of manually gathering facts from several tools, I can ask once and get a contextual answer built from the connected service record. Step 7: Turn Repeated Agent Instructions Into Reusable Skills Agents often perform repeated tasks: investigate an incident, assess deployment risk, review a pull request, measure DORA metrics, run CI, or deploy a service to production. Repeating the full instructions every time is not scalable. That is where agent skills are useful. A skill packages the context and logic needed for a repeatable operation. For example, I can define skills for: Incident responsePort readiness checksRunning CIDeploying a serviceDeploying to production When I ask an agent to run CI for the shipment service, it can load the relevant CI skill and combine it with the shipment service context. The agent is not starting from zero. It knows the service, the intended workflow, and the constraints around execution. This makes Context Engineering reusable. Skills reduce repeated setup work, standardize workflows, and help agents perform the same task in a predictable way across services. Step 8: Add Human Gates to Agentic SDLC Workflows Automation does not mean removing human control. In an agentic SDLC workflow, agents can gather requirements, plan work, generate code, test changes, and run continuous integration. But important actions should still include approval or rejection points. For example, a workflow can fetch service context first, then proceed through: Requirements gatheringPlanningCodingTestingContinuous integrationHuman approval before sensitive actions Human gates are part of good context engineering because they provide governance. The agent can recommend, prepare, and trigger approved workflows, but a person can still decide whether a proposed action should proceed. Step 9: Use Context to Make Better Deployment Decisions The final demo makes the value very clear. A simple application loads context for a selected service and gives a production-readiness verdict. For a healthy payment service, the context shows a clear picture: ownership is assigned, the Slack channel is configured, the runbook is documented, on-call rotation is active, test coverage is 94%, health status is healthy, the service was deployed recently, and there are no open incidents. Based on that connected information, the service is marked ready to deploy. For another service, the result is completely different. It is marked as not ready because key context is missing. There is no owning team, no runbook, and several other readiness requirements are incomplete. The system identifies the gaps instead of making a blind recommendation. That is what a production decision should look like. Not “yes” or “no” based on a vague prompt, but a verdict grounded in explicit evidence: Identity and ownershipHealth and operational statusRunbook availabilityOn-call coverageTest coverageRecent deployment historyOpen incidentsRequired scorecard checks When the context indicates risk, the result can say to proceed with caution and explain why. This is far more useful than an agent giving an unverified deployment recommendation. Step 10: Treat Context Engineering as an Engineering Discipline Context engineering is important because AI agents are only as reliable as the environment they can understand. If an agent has scattered data, unclear ownership, missing policies, and unrestricted tools, it will struggle no matter how advanced the model is. The practical path is straightforward: Map the tools and data sources that define your SDLC.Define the service-level context agents need to retrieve.Centralize ownership, health, repositories, runbooks, incidents, and policies.Create reusable skills for common workflows.Use tools for live data and approved execution.Add guardrails and human approvals around consequential actions.Make agent verdicts explainable through visible context. That is how I move from disconnected AI experiments to reliable agentic engineering workflows. Context Engineering reduces unnecessary token use, reduces confusion, and gives agents the facts they need to help build, test, operate, and deploy software with more control. Context Engineering FAQs What Is Context Engineering for AI agents? Context Engineering is the practice of organizing and governing the information an AI agent can access, including instructions, service data, memory, tools, examples, and safety constraints. It helps the agent make grounded decisions rather than guessing. How Is Context Engineering Different From Prompt Engineering? Prompt engineering focuses on how to phrase instructions for an interaction. Context Engineering focuses on the information the agent can retrieve and use throughout a workflow, such as ownership, repositories, incidents, deployment data, and policies. What Context Should an SDLC Agent Have? An SDLC agent should have the context needed for its task, which can include service ownership, repository details, environment, runbooks, on-call status, deployment history, test coverage, incident status, relevant tools, and hard safety rules. Why Are Human Approval Gates Important for AI Workflows? Human gates preserve control over consequential actions. Agents can retrieve context, prepare work, and recommend or trigger an approved workflow, while a person retains the ability to approve or reject sensitive changes.

By Pavan Belagatti CORE

In this article, I'll try to give practical insights for choosing the right AI architecture for impact, not just experimentation. Companies are spending heavily on AI. Many are still struggling to show clear business returns. The most common reason is not the model; it is the architecture. Teams often jump straight to multi-agent systems or "autonomous AI" because those terms sound advanced. In reality, a well-designed decision intelligence system or a focused single-agent architecture often delivers faster, more reliable ROI than a complex multi-agent setup that no one can debug or govern. This article maps the five AI architectures that are actually driving measurable business value. For each one, you will see: What the architecture looks likeWhen you should use itWhy it works from a business perspectivePractical risks and success factors The goal is simple: help you choose the right level of architectural complexity for the outcome you need. 1. AI Decision Intelligence Architecture What it is: This is the classic "data -> insight -> decision -> action" loop, now powered by stronger models. Data from operational systems flows into an analytics layer, an AI model produces predictions or scores, a decision engine applies business rules and thresholds, and actions are triggered (often still with human oversight). When to use it: Strategy, forecasting, pricing, demand planning, risk scoring, inventory optimization, and any domain where the primary value is better decisions at scale. Why it works: It directly connects data to decisions that affect revenue, cost, or risk. The architecture is relatively mature, easier to govern, and usually has clear KPIs (forecast accuracy, reduction in stock-outs, improved conversion, lower credit losses, etc.). Practical notes: Success depends more on data quality, feature engineering, and decision policy design than on the latest foundation model. Many organizations already have 70% of this architecture in place and only need to modernize the model and decision layers. 2. AI Personalization Engine Architecture What it is: User data and behavioral tracking feed a feature store. An AI model (recommendation, ranking, or generative) produces personalized outputs: product recommendations, content, offers, or next-best-action. The system continuously learns from engagement. When to use it: Marketing, e-commerce, media, customer experience, and any product surface where relevance directly drives engagement and revenue. Why it works: Personalization has one of the most proven ROI profiles in AI. Even modest lifts in click-through, conversion, or average order value compound quickly at scale. The architecture is well understood and has mature tooling (feature stores, real-time inference, experimentation platforms). Practical notes: The biggest failures come from poor cold-start handling, lack of real-time features, or treating personalization as a pure model problem instead of a full-stack system (data-> features -> model -> delivery -> feedback). 3. Single-Agent AI Architecture What it is: A single agent receives a goal, maintains memory, reasons about the next step, uses tools, and executes. It operates in a loop until the task is complete. This is the architecture behind many of today’s coding assistants, research helpers, and internal automation agents. When to use it: Task automation, structured multi-step workflows, coding, document processing, customer support escalation, and any problem that can be owned by one competent agent with good tools. Why it works: It handles multi-step work with context and logic in a way that pure predictive models or simple RPA cannot. It is significantly simpler to build, observe, and govern than multi-agent systems, while still delivering real autonomy on well-scoped tasks. Practical notes: Most organizations should master single-agent systems before moving to multi-agent. The limiting factors are usually tool quality, memory design, evaluation harnesses, and clear task boundaries, not the choice of foundation model. Key insight: A reliable single-agent system with excellent tools and evaluation often outperforms a poorly coordinated multi-agent system in both speed of delivery and actual business results. 4. Multi-Agent AI Architecture What it is: A planner (or meta-agent) decomposes a complex user goal into sub-tasks. Specialized task agents execute those sub-tasks, often in parallel, using shared or private memory. Results are aggregated into a final output. This is the architecture used in advanced research systems and complex enterprise workflows. When to use it: Complex workflows that genuinely require different skills (research + analysis + writing + coding), long-horizon projects, or situations where parallelism and specialization produce clear gains in quality or speed. Why it works: It distributes cognitive load. Different agents can be optimized (or even use different models) for different sub-problems. When designed well, the system scales in capability without making any single agent monolithic. Practical notes: Coordination cost is real. Handoff failures, inconsistent memory, and unclear ownership of the final result are common. Multi-agent systems require stronger observability, evaluation, and governance than single-agent systems. Do not adopt this architecture just because it sounds more advanced. 5. Autonomous AI System Architecture What it is: A closed-loop system: Input -> Perception-> Reasoning-> Planning-> Execution -> Feedback. The system continuously senses its environment, updates its understanding, plans, acts, and learns from outcomes with minimal human intervention. This is the most ambitious architecture on the spectrum. When to use it: End-to-end automation of well-understood business processes, self-optimizing systems, and domains where continuous operation without constant human oversight is both possible and desirable (certain supply-chain, infrastructure, or trading systems, for example). Why it works: When the feedback loops are high-quality and the environment is sufficiently stable or well-modeled, the system can improve over time and operate at a scale and speed humans cannot match. Practical notes: This is the highest-risk architecture. Failures can be expensive and hard to contain. Most organizations should treat full autonomy as a long-term destination, not a starting point. Strong guardrails, human oversight points, and kill switches are mandatory. How to Choose the Right Architecture ArchitectureComplexityTime to ValueBest ForMain RiskDecision IntelligenceLow–MediumFastForecasting, optimization, riskPoor data or unclear decision policiesPersonalization EngineMediumFast–MediumEngagement, conversion, CXWeak feedback loops or cold startSingle-AgentMediumMediumTask automation, coding, researchBad tools or weak evaluationMulti-AgentHighSlowerComplex multi-skill workflowsCoordination and observability failuresAutonomous SystemVery HighSlowestFully automated closed-loop processesUncontrolled behavior and high blast radius Simple decision rules: If the primary value is better decisions from data, then start with decision intelligence.If the primary value is relevance at scale, then build a personalization engine.If you need multi-step task completion with tools, then master single-agent first.Only move to multi-agent when you have clear specialization and coordination benefits.Treat autonomous systems as a maturity goal, not a first project. Common Mistakes That Destroy ROI Jumping to multi-agent or autonomous too early: complexity without corresponding process maturity.Treating architecture as a model problem: the model is rarely the bottleneck; tools, data, evaluation, and governance usually are.No clear success metrics: if you cannot define what "good" looks like in business terms, you cannot steer the system.Ignoring observability: agentic and autonomous systems that cannot be inspected become impossible to improve or trust.Building technology in search of a problem: the architecture must serve a real workflow and a real economic outcome. Closing The organizations that extract real ROI from AI are not necessarily the ones using the most advanced architecture. They are the ones that match the architecture to the problem, keep the design as simple as the use case allows, and invest heavily in data quality, tools, evaluation, and governance. Start with the architecture that solves the actual business problem with the least unnecessary complexity. Prove value. Then, and only then, increase architectural sophistication where the returns justify the cost and risk. Decision intelligence and personalization still deliver some of the clearest and fastest returns. Single-agent systems are currently the highest-leverage step-change for knowledge work and automation. Multi-agent and fully autonomous systems are powerful... but only when the organization is ready to operate them with discipline. Choose deliberately. Measure ruthlessly. Scale what works.

By Ram Ghadiyaram CORE

Most enterprises have all three pieces. A process automation layer. A data integration strategy. An AI initiative. Yet critical decisions still fail, agents still hallucinate, and workflows still run on yesterday's data. The investments exist. The convergence does not. The problem is not a lack of technology, but a lack of architectural thinking. Process intelligence, event-driven integration, and trusted agentic AI are being built in isolation, by different teams, with different goals, on different timelines. The result is an architecture that looks complete on a slide and breaks in production. This post argues that these three capabilities form a Trinity. They only deliver their full value when they are designed to work together. Three Capabilities, One Architectural Commitment Process intelligence, event-driven integration, and trusted agentic AI each solve a real problem. Each one also creates new risks when it operates alone. The following architecture shows how the three layers connect into a single, converged system. Process Intelligence: The Layer That Gives Agentic AI Its Boundaries Process intelligence is the evolution of classic Business Process Management (BPM) into something adaptive, event-aware, and AI-ready. It is the layer where technology maps directly to business value. Every workflow connects to a concrete business outcome: a loan approved, a shipment rerouted, a fraud case resolved. Process mining observes how business processes actually run, identifies where decisions fail, and surfaces where automation would deliver the most value. Vendors like Celonis have built entire platforms around this capability. Process orchestration executes workflows, enforces business rules, and produces the audit trails that compliance teams depend on. Camunda is a leading example. Agentic process orchestration goes one step further: it allows AI agents to participate directly in workflow execution, taking autonomous actions within defined boundaries while the process layer maintains control. Automation is the business driver. Organizations adopt process intelligence to automate more, faster, with less manual intervention, while keeping humans in control of the decisions that matter. But agentic automation only works safely when the process layer defines the operational envelope: what the agent can decide alone, what requires human approval, and what must be escalated regardless of what the model recommends. This is where guardrails live in practice. Not as theoretical constraints inside a model, but as concrete workflow gates that stop, route, or escalate before an action is executed. Process intelligence is what makes automation trustworthy at scale. Event-Driven Integration: From Scheduled Batches to Live Events Event-driven integration is the architectural principle that connects operational systems continuously, based on what happens rather than when a scheduler runs. An event from a payment system, a sensor, a CRM update, or a logistics platform travels in real time to whatever system needs to act on it. Apache Kafka has become the de facto standard for event-driven integration at enterprise scale. Other options exist, including cloud-native messaging services and specialized event brokers, but Kafka is where the ecosystem has converged. What matters in any case is the commitment to events as the primary integration primitive to ensure true decoupling, scalability, and data consistency across real-time and batch systems. The market reflects this shift. Process orchestration engines have rearchitected their core runtimes to be event-driven from the ground up, built for real-time throughput and horizontal scale. Camunda's Zeebe is a leading example. Zeebe is itself an event-driven engine, which means organizations can implement event-driven workflows and lightweight integration patterns without Kafka as a prerequisite. For broader enterprise integration at scale, Apache Kafka complements the process orchestration layer, connecting the full landscape of operational systems, SaaS platforms, and data infrastructure into a single event-driven backbone. Core business applications and SaaS platforms followed. SAP S/4HANA, Salesforce CRM, and ServiceNow have all added eventing interfaces and Change Data Capture (CDC) capabilities alongside their traditional API-based request-response integrations. The direction is clear: even systems that were designed around synchronous HTTP are moving toward event-driven models. Process engines receive live state. Agentic AI systems receive current context. Decisions are made on what is actually happening, not on what happened last night. Trusted Agentic AI: Safety Is an Architecture, Not a Setting Trusted agentic AI is an architectural property, not a product feature. Agentic AI systems do not just generate responses. They take actions, trigger workflows, and interact with operational systems. That autonomy is what makes trust and safety an architectural concern rather than a model configuration. It operates at two levels. The first is the model itself. Vendors like Anthropic and Mistral build alignment, constitutional constraints, and refusal behaviors directly into their models. This provides a baseline. The second level is the process intelligence layer. A well-aligned model can still be manipulated through prompt injection or adversarial inputs. It can still hallucinate when the surrounding data is stale or incomplete. Model-level safety defines how the agent behaves within a given context. Process-level safety defines the operational envelope: what the agent is allowed to do, which decisions require human approval, and what the fallback is when the agent is wrong. Both levels are necessary. Neither is sufficient alone. When the Trinity Splits: Three Agentic AI Failure Scenarios Three short failure scenarios make this concrete. Process intelligence without event-driven integration. A workflow engine automates a credit decision. The data feeding it comes from a nightly batch export. The process runs correctly. The decision is based on a customer's financial state from 18 hours ago. The automation worked. The outcome was wrong. Event-driven integration without process intelligence. Transaction data flows in real time across systems. An agentic AI system detects an anomaly and flags a potential fraud case. But there is no process intelligence layer defining what happens next. There is no approval gate, no escalation path, no audit trail. The agent acts, or it does not, and nobody can explain which or why. Trusted agentic AI without the other two. The agent is aligned, tested, and governed at the model level. But it receives context from a batch pipeline, so its reasoning is grounded in outdated information. And no process intelligence layer enforces boundaries on what it can do next. The agent behaves well in the lab. It causes problems in production. The Trinity in Action: Process Intelligence and Agentic AI Across Three Industries The following three scenarios show this architectural model working across industries. Each one is different. The pattern is the same: an event triggers a process, an agentic AI system acts within it, and process intelligence defines the boundary between automation and human control. Financial services. A transaction event triggers an agentic AI fraud risk assessment in real time. The risk score flows into a case management workflow. Below a defined threshold, the process is automated. Above it, the process intelligence layer routes the case to a human analyst before any account action is taken. The guardrail is not inside the model. It is inside the process. Healthcare. A patient monitoring system emits a deterioration signal. The event reaches a care pathway engine, which initiates the appropriate clinical workflow. An agentic AI system recommends an intervention. The process intelligence layer requires clinician confirmation before that recommendation becomes an order. The agent informs. The human decides. The process enforces that boundary every time. Supply chain. A supplier sends a disruption signal. The event reaches the process engine before the procurement team opens their inbox. An agentic AI system analyzes inventory, evaluates alternative suppliers, and proposes rerouting options. The process intelligence layer defines which decisions the agent can execute autonomously and which require sign-off. Speed comes from the event-driven layer. Governance comes from process intelligence. Trust comes from both working together. Build the Trinity, Not the Parts This Trinity is not a new product category. It is a way of thinking about a converged architecture that most enterprises have not yet adopted. Event-driven integration ensures that every process and every agentic AI system works on current reality. Process intelligence ensures that automation stays within governed, auditable boundaries. Trusted agentic AI ensures that agents behave reliably within the context they are given, and that the process intelligence layer catches what the agent cannot. The following architecture maps the complete picture across all three layers: Organizations that invest in all three separately will keep getting the results they are getting today. Organizations that design them to converge will build something qualitatively different: infrastructure that moves fast, governs well, and earns the trust of the business. The technology exists. The architectural commitment is what is missing.

By Kai Wähner CORE

Just a few months back, I observed a test suite with a self-healing feature “fixing” a failed selector three different times during the same sprint cycle. In each instance, the fix performed its function perfectly well; however, it didn’t address the real problem of a shipped UI regression, as its sole focus was on keeping the test green. No one on the team became aware of the situation until one of the customers discovered it. That was the point when my perception of AI in Quality Assurance changed dramatically more than any keynote or LinkedIn post. Today, if you are a QA Engineer, chances are high that you have come across similar headlines as well: manual testing is dying, autonomous agents create and fix test scripts in seconds, and your career is going to be at risk soon. The fear is justified, and I do not find it irrational as a Senior QA Manager who sees new testing tools appearing in our workflows every quarter, or even faster than we can develop any governing principles. What I see happening is quite different: not only is the position not going away, but it is becoming increasingly difficult to fake. AI excels at internalizing all the mechanical, low-context tasks that previously made up the bulk of a QA Engineer’s workload, leaving only those tasks that have never been mechanical or low-context and involve judgment, risk assessment, and determining what quality looks like for a particular product. This significantly narrows the number of people qualified for the position. 1. Stop Writing Tests. Start Auditing Them. For decades, a significant proportion of QA time was spent on the technicalities: automation scripts, manual click-through of UI workflows, and broken selectors caused by someone renaming a div tag. AI is really good at these types of jobs, and pretending otherwise is just a waste of time. You should approach AI-generated tests as you would a junior engineer’s pull request: they are useful and efficient, but require your review before implementation. Learn to feed it real context, not vibes. The difference between a useless AI-generated test and a genuinely good one almost always comes down to whether you gave it the actual acceptance criteria, edge cases, and business rules, or just a vague prompt. This is a real skill, and most QA teams haven’t invested in it yet.Get comfortable with self-healing tools, and stay suspicious of them. Self-healing automation is very valuable for handling cosmetic churn in your user interface. In addition, as shown by my story above about selectors, it can silently hide the very bugs it's supposed to detect.Your value moves from writing to verifying. That’s no downgrade. The ability to check that 100 automatically created tests are useful, as opposed to just being syntactically correct, is more difficult than having written 50 by hand. 2. Learn to Test the Thing That Doesn’t Give the Same Answer Twice Every product I am working on is trying to add AI, and none of the QA processes I have seen have been designed with the requirements of such a task in mind. In conventional software, the deterministic factor is the key component; in other words, whatever the input, the output will always be the same. But with the use of AI, there is nothing like that, since the same query asked twice yields two different answers. This opens up an actual underserved field of skills, known as AI Trust, Risk, and Security Management, and, to be honest, what you call it is less important than the brawn behind it. Areas where one could actually develop some skills: Bias and fairness testing. Learning to actually evaluate whether a model’s outputs skew unfairly across groups, not just whether the demo looks fine.Hallucination detection. Building repeatable ways to check whether an LLM’s output is grounded in real data or confidently making things up. This is genuinely hard and genuinely valuable; most teams are doing it on an ad hoc basis right now.Adversarial and prompt-injection testing. Deliberately trying to break an AI system’s guardrails before someone outside your company does it for you. I’ll be honest about the caveat here: this field is young enough that best practices are still being written in real time, including by people learning on the job. Nobody has fifteen years of AI-TRiSM experience, because it didn’t exist fifteen years ago. That’s exactly why it’s a good place to plant a flag now rather than waiting for it to mature. 3. Protect the Part of Quality AI Genuinely Can’t Do AI is just a statistical machine. It doesn’t have any firsthand knowledge of being frustrated with a difficult checkout process, any cultural knowledge to know why something that works well in one place doesn’t feel right somewhere else, and no sense of that hard-to-pin-down friction that you can’t specify. AI is responsible for functional testing; however, when it comes to the people-oriented aspect of quality, that’s where I would focus my efforts: Exploratory testing following a hunch. The best bugs I’ve ever discovered came from getting a slight feeling that something was amiss and investigating it, rather than through a written test case. The hunch does not stem from any particular model.Accessibility and usability should be top priorities rather than something ticked off a box prior to deployment. Is the product actually good to use? The algorithm will tell you whether the button meets the contrast ratio requirement. It cannot tell you whether the user flow around the button is confusing.Being there when risk is being discussed. AI will tell you whether the feature complies with the specification. AI has no way of knowing if the specification itself is incorrect for your market, your users, and the regulations. This discussion has to involve a human who knows the business, not the ticket. 4. Let Production Data Tell You Where to Look AI is based on data, and therefore, your testing approach should be too. The QA engineers who actually derive useful insights from AI do not test everything equally; instead, they let the data drive them. Analyze the real usage pattern of your application and prioritize automation accordingly; focus on testing those paths that users actually use, not those that were expected according to the initial requirements specification. Close the loop with your DevOps team about what is really breaking in production. If you find out that there are constantly recurring errors of one type or another, this information is directly relevant to the testing priorities of your AI solution, not something that you talk about separately.Understand what the data pipeline looks like, at least at a high level. A significant amount of “AI testing” in the future will involve testing the data pipeline that feeds into the AI algorithm, not just the outputs. A 90-Day Plan, If You Want One If you’d rather have a concrete starting point than a philosophy, here’s roughly how I’d sequence it: A 90 Day qa plan Days 1–30 Learn the tools Bring an AI coding assistant into your actual daily automation work, not a sandbox exercise, and pay attention to where it’s confidently wrong. Days 31–60 Expand the domain Take a real course on ML fundamentals or AI testing methodology, not just a vendor’s product training. Days 61–90+ Make it visible Propose one concrete AI-driven improvement on your current team, whether that’s AI-assisted test data generation or a pilot of self-healing UI tests with a defined review process attached. Regarding tooling: this landscape moves fast enough that my suggestions for product names will be obsolete by the end of the first year, but remember that it is the categories which are meaningful, not individual product names. Natural language test generation, self-healing test execution, visual testing, and AI security testing are just a few of the currently meaningful categories. Tools in any of those categories worth considering are those that let you see and adjust the AI's decisions. The Bottom Line Anxiety about AI in testing often arises from conflating two distinct concepts. Testing is a technical task, while QA is a mindset focused on protecting the user experience. AI excels at technical tasks but cannot replicate the QA mindset. In practice, automation is removing repetitive tasks, leaving the core responsibility of defining quality for each product and user group, and identifying issues beyond a model’s reach. This results in a more meaningful, though more demanding, role.

Enterprise AI is moving beyond isolated prompt-response calls and toward systems that observe events, preserve state, invoke tools, and publish decisions back into operational workflows. In that setting, event streaming is not simply middleware. It becomes the record of how intelligent behavior unfolds over time. Kafka is designed to read, write, store, and process streams of events across distributed systems, while Kafka Streams adds joins, aggregations, windowing, event-time processing, and exactly once support for stateful stream applications. At the same time, modern agent runtimes have shifted toward durable execution, persistence, and human-governed control flows rather than single-turn prompting alone. That convergence makes Kafka a strong coordination layer for autonomous agents that need to react continuously instead of responding once and disappearing. That architectural change also alters the role of the model. In an API-centric design, the model is often treated as a synchronous dependency behind a request. In an event-driven design, the model becomes one participant in a larger decision pipeline. Observations arrive as events, context is assembled from topics and state stores, agent steps are logged, and decisions are emitted as new events for downstream systems. Because Kafka topics can be replayed and reprocessed, the same stream can feed planners, validators, enrichment services, audit consumers, and human-review workflows without creating hard coupling between those components. The resulting system is easier to inspect, easier to recover, and easier to evolve than a chain of tightly bound remote calls. Turning Kafka Into the Coordination Layer The most important benefit is not only scale. It is the replacement of brittle request chains with an append-only coordination layer. A payment event, support ticket update, equipment alarm, or fraud signal can be published once and then consumed independently by retrieval components, compliance checks, planners, and execution agents. Kafka consumer groups divide partitions across consumers in the same group, and each partition is consumed by a single consumer within that group, which preserves ordering at the partition level while still allowing horizontal scale. For agentic systems, that detail is central. If all events for the same case, customer, or device are keyed consistently, one partition becomes the serialized timeline for that entity, and the agent no longer has to reconstruct order from racing HTTP callbacks. The event log also becomes a durable memory boundary. Kafka log compaction retains the latest value for each key, which makes compacted topics useful for task state, policy snapshots, approval status, or tool metadata that must survive restarts and recover quickly. On the runtime side, agent frameworks persist checkpoints and thread-scoped state so interrupted flows can resume from a saved step instead of starting over. Used together, those layers create a pragmatic split of responsibilities, such as Kafka preserves externally visible state transitions, and the agent runtime preserves internal execution context between steps, pauses, and failures. That is exactly the kind of separation needed when autonomous behavior must remain observable without being reduced to stateless prompt calls. Designing Agent Loops Around Events Once Kafka becomes the backbone, the agent loop changes shape. The entry point is no longer a prompt alone. It becomes a domain event that is enriched, correlated, and converted into a bounded task. Research on ReAct showed the value of interleaving reasoning and acting, and current agent frameworks translate that idea into practical workflows with durable execution, interrupts, and resumable state. The production version of an autonomous agent is therefore less like a chat session and more like a state machine that reasons, uses tools, emits intermediate facts, and pauses when a policy boundary requires approval. A concise stream processor can prepare that task before the model loop begins: Java builder.stream("order-events", Consumed.with(Serdes.String(), orderSerde)) .selectKey((key, event) -> event.customerId()) .join(customerTable, this::mergeContext) .mapValues(this::toAgentTask) .to("agent-tasks"); This pattern keeps context assembly close to the log instead of scattering it across synchronous service calls. Records are keyed by stable business identity, joined with the latest customer state, and emitted as small agent-tasks messages that the runtime can consume directly. Kafka Streams is explicitly intended for stateful processing with joins, event-time semantics, and exactly-once guarantees, so the enrichment stage remains deterministic, replayable, and independent from the model-serving layer. The execution boundary can remain equally narrow: Java @KafkaListener(topics = "agent-tasks", groupId = "claims-agent") @Transactional public void handle(AgentTask task) { AgentDecision decision = agentRuntime.run(task); kafkaTemplate.send("agent-decisions", task.taskId(), decision); } A compact runtime method can express the control flow without hiding it: Java public AgentDecision run(AgentTask task) { AgentState state = stateStore.load(task.taskId()); PlanStep step = planner.next(state, task); if (step.requiresApproval()) return AgentDecision.pause(task.taskId(), "manual-review"); ToolResult result = toolExecutor.execute(step.tool(), step.arguments()); return planner.complete(task, state, result); } This arrangement matters because the runtime receives a prepared task and emits an explicit decision event instead of mutating external systems invisibly. When transactions are enabled, Spring for Apache Kafka supports exactly-once semantics for the read-process-write sequence, and Kafka itself uses idempotent producers plus transactions so retries do not create duplicate log entries. External side effects still need idempotent design when they happen outside Kafka, but the event pipeline itself becomes much more predictable and auditable. Reliability and Control in Production Reliability in event-driven AI systems is usually lost at the edges rather than inside the model call. Kafka’s exactly-once features matter because an autonomous agent often emits decisions that trigger downstream actions, compensations, or audits. Kafka Streams supports exactly-once v2, and exactly-once flows configure consumers with read_committed isolation so aborted transactions do not leak into downstream processing. The event contract matters just as much as the delivery contract. Schema Registry centralizes schemas, validates them, and enforces compatibility modes so producers and consumers can evolve independently. In practice, a stable AgentDecision schema with explicit action type, confidence, explanation reference, and approval status is usually more valuable than a loosely structured JSON envelope because it can be consumed safely by analytics jobs, rule engines, operational systems, and auditors maintained by different teams. Operational control also has to assume malformed input, tool failure, and policy limits. Kafka Connect supports dead letter queues for records that cannot be processed successfully, and Spring Kafka supports dead-letter handling for repeated listener failures. Kafka also supports SASL-based authentication and ACL-driven authorization, which matters when planners, tool executors, and audit services must have different permissions over topics and consumer groups. Combined with interrupt-driven approval workflows from modern agent runtimes, those controls allow autonomous agents to operate inside explicit safety and governance boundaries instead of as opaque background processes. Where This Architecture Fits Best This architecture is strongest when work is asynchronous, stateful, and externally observable. Fraud triage, claims handling, supply chain exception management, field-service coordination, and security operations are better fits than chat-only assistance because the hard problem is not generating a sentence. The hard problem is reacting to a changing stream of facts, correlating them by entity and time, and making bounded decisions with replayable outcomes. Event-driven AI systems with Kafka and autonomous agents are compelling because they treat intelligence as part of an operational stream rather than as an isolated endpoint. The most effective implementations keep the log authoritative, keep schemas explicit, keep agent state durable, and keep irreversible actions observable and governable. That combination produces systems that are not only responsive, but also replayable, auditable, and resilient enough for enterprise use, which is ultimately the threshold that separates a convincing demo from a production architecture.

By Uthej Mopathi CORE

Most enterprise AI post-mortems do not blame the model. They blame the storage tier that starved the accelerators, the identity policy that over-granted access, the cost model that ignored egress, the forecast that leaked future data, or the region that failed and took a business process with it. The hard part of production AI was never intelligence. It was the engineering discipline around it. This article distills the architectural patterns that decide whether a cloud AI system is trustworthy at scale, spanning infrastructure, identity, cost, operations, the applied domains, low-code assembly, platform selection, and multi-cloud resilience. It is written for engineers who have to keep these systems running, not for a keynote. Infrastructure: The Interconnect Is the Bottleneck Distributed training is a systems problem before it is a machine learning problem. When a job spans many graphics processing units (GPUs), the fabric connecting them (e.g., NVLink within a node, InfiniBand, or a vendor fabric across nodes) frequently caps throughput more than raw compute does. Accelerators wired through an ordinary network idle while they wait to synchronize gradients. Storage is the symmetric constraint. If the file system cannot deliver data at the rate the accelerators consume it, utilization collapses. The pattern is a tiered design: Hot tier: parallel or block storage feeding active training at high input/output operations per second (IOPS).Warm tier: recent data staged for quick promotion.Durable lake: object storage providing petabyte-scale durability, partitioned and lifecycle-managed underneath. Two cost drivers hide from the pricing page: data egress (moving data across regions or out of a provider) and idle warm capacity. Optimizing only the advertised compute line item guarantees a surprise on the invoice. Identity Is the Perimeter In a service-to-service AI architecture, the network perimeter is gone; identity is the boundary. A zero-trust posture, where every request authenticates and receives least privilege, contains the blast radius when a component is compromised. Across providers, identity federation is the load-bearing pattern: a principal authenticates once and is recognized everywhere, so access is granted and revoked centrally instead of reconciled across three identity systems. Policy must travel with the workload; a rule enforced on one cloud and forgotten on another is not a policy. Model authorization is the emerging frontier. As models call tools and take actions, the question moves from who can query this model to what may this model do on a user's behalf. Least privilege applied to an autonomous agent is the boundary between useful and unbounded. Cost and Operations Are a Control Loop Cost management is not a spreadsheet; it is automation. Consistent resource tagging across every cloud is the prerequisite for attribution. On top sit budgets, alerts, and automated remediation that throttles runaway spend before it escalates. Site reliability engineering (SRE) supplies measurable targets. For AI workloads, the golden signals extend beyond latency and errors to accelerator utilization, queue depth, and prediction quality. A model can be fully available and quietly wrong, so define a service level objective (SLO) for output quality, not just uptime. Three techniques earn their complexity: Spot or preemptible capacity plus checkpointing cuts training cost sharply when jobs resume cleanly after reclamation.Predictive scaling anticipates load instead of reacting to it.LLM inference optimization becomes architectural: batch requests, cache frequent responses, route easy queries to smaller models, reserve the expensive model for queries that need it. The Applied Domains Share a Spine, Differ in Physics Vision is byte-heavy. High-resolution images and video streams make the data and network layers dominant. For real-time video, decouple frame capture from analysis and sample frames rather than processing every one. Critically, a business-rule layer, never the model alone, owns consequential decisions. Every extraction should carry a confidence score used as a routing gate: Python def route_extraction(field, threshold=0.90): if field["confidence"] >= threshold: return "auto_process" return "human_review" Language is byte-light but semantically treacherous, and because it replies directly to users, errors are visible. The defining risk of generative systems is hallucination. The strongest architectural defense is retrieval grounding, forcing answers from verified sources with citations: Python def answer(question, knowledge_base): passages = knowledge_base.search(question, top_k=3) context = "\n".join(p.text for p in passages) prompt = f"Answer using ONLY this context.\n{context}\n\nQ: {question}" return model.generate(prompt), [p.source for p in passages] Forecasting is defined by time order. You cannot shuffle a time series into random splits, and the most common failure is data leakage, using information unavailable at prediction time. Test on a fair, time-ordered holdout, and always emit a prediction interval; a point forecast that hides its uncertainty invites overconfident decisions. No-Code and Low-Code: Governed or Ungoverned No-code and low-code platforms collapse build cost from a scoped project to an afternoon, which is why adoption is exploding. The symmetric risk is sprawl: hundreds of ungoverned flows handling sensitive data, owned by no one. Govern with guardrails, not gates. Restrict which connectors and data sources are permitted, assign an owner and an SLO to every production flow, then let builders move freely inside the boundary. The goal is to make the safe path the easy path. Platform Selection Without Self-Deception Vendors all claim to be fastest, cheapest, and most reliable. Benchmark to replace claims with evidence: Latency: report percentiles (p95, p99), never averages that hide the slow tail.Quality: measure on your own representative data, not a public leaderboard.Cost: model total cost of ownership, including transfer, storage, idle capacity, operations, and migration, not the headline compute rate.Reliability: verify the platform meets your recovery time objective (RTO) and recovery point objective (RPO). Combine dimensions in a weighted scorecard whose weights are fixed before scores are seen. Adjusting weights afterward to crown a favorite converts analysis into rationalization. Multi-Cloud Resilience: Design for the Day a Cloud Fails For systems a business cannot lose, a single provider is a gamble. Multi-cloud resilience deliberately places critical workloads so no single provider failure takes the business down, applied only where the cost of failure exceeds the cost of prevention. Predict rather than react. Combine leading signals into a health score and fail over proactively: Python def health_score(latency_ms, error_rate, saturation): latency_factor = max(0, 1 - (latency_ms / 1000)) error_factor = max(0, 1 - (error_rate / 0.05)) saturation_factor = max(0, 1 - saturation) return round(0.4*latency_factor + 0.4*error_factor + 0.2*saturation_factor, 3) Kubernetes makes workloads portable; data replication (with the consistency-versus-availability trade-off decided per workload) keeps data ready on the other side; and a portable foundation of federated identity, uniform policy, and centralized monitoring makes failover routine rather than heroic. The discipline that separates real resilience from a slide deck is rehearsing failure on purpose. An untested failover path is a promise, not a capability. The Judgment Layer Across every layer, value came not from the most powerful component but from the judgment applied to it: matching effort to problem difficulty, keeping humans on consequential decisions, measuring before deciding, building governance in early, and designing for change. Tools will churn; foundation models will make today's designs look quaint. That is precisely why principles outlast product knowledge. The scarce resource in enterprise AI was never intelligence. It was judgment, and judgment does not ship from the cloud.

TL;DR: A Déjà-Vu? AI adoption seems to be scaling: 37% of respondents in McKinsey’s 2026 survey report an EBIT effect from AI, and Gartner finds that 22% of organizations have scaled it across business units. Now, Agile practitioners have seen this combination before, as AI transformations and Agile transformations rhyme. Five classic failure patterns from Agile transformation adventures are back under new names: mandates from above, licenses mistaken for training, greenfield showcases, parachuted consultants, and promised payroll savings dressed up as strategy. They share one condition: organizations make AI decisions at organizational scale without leaving inspectable evidence at the workflow level in the trenches. And for good measure, let us throw in ignoring culture and excluding most of the organization’s people in the process. History Does Not Repeat Itself, but AI Transformations and Agile Transformations Do Rhyme AI transformations in large organizations are scaling, individual productivity is up, leaders still plan to increase spending, and yet enterprise financial impact remains limited: McKinsey's 2026 State of AI survey (1,719 respondents, fieldwork May 4 to June 8, 2026) puts numbers on three of the four: 44 percent of respondents say AI is scaling across their enterprise, up from 38 percent a year earlier; 80 percent of those who use AI report improved individual productivity; and 37 percent attribute any EBIT impact to AI at all, with the "AI high performer" group flat at about 6 percent.Gartner's September 2026 survey of 1,303 respondents from organizations with at least $50 million in annual revenue supplies the spending picture: 85 percent of functional leaders plan to increase AI spending in 2026, 22 percent of organizations have scaled AI across multiple business units or adopted an AI-first approach, and 11 percent do not know what their function spent on AI in 2025. Something is happening, and something is also not translating. Agile practitioners have seen that combination before. "History does not repeat itself, but it rhymes," a line widely attributed to Mark Twain despite no evidence that he said it; the attribution to Twain dates back to 1970. The attribution is shaky; nevertheless, the observation holds. I wrote the Scrum Anti-Patterns Guide about what organizations do to Agile when they adopt it from the top down. The same organizations are now doing the same things to AI, with a new generation of leaders who consider the Agile years ancient history, and five rhymes stand out. The Five Rhymes of AI Transformations Rhyme 1: The Mandate From Above IBM's June 2026 study of 2,000 C-level technology executives found that 80% reported CEO-driven AI transformation mandates, and 77% said adoption is already outpacing their governance capabilities. The most public example is Shopify. In a late-March 2025 memo that he later posted on X, CEO Tobi Lütke told the company that "reflexive AI usage is now a baseline expectation at Shopify" and that, before asking for more headcount, teams "must demonstrate why they cannot get what they want done using AI," as Tom's Hardware and TechCrunch reported. Whether that works at Shopify, I cannot judge from the outside. What I can judge is the predictable risk when that kind of memo lands in an organization where governance is already falling behind: visible compliance and invisible workarounds. Agile practitioners remember the memo announcing "we are now an agile organization" and the Sprint Reviews that followed, which were ignored by everyone who could change a decision. Rhyme 2: The Belief That This Time Training Is Optional The Agile version bought a two-day certification class and called it a transformation. Often, AI transformations skip even that: buy Copilot or ChatGPT Enterprise licenses, send an email, done. The tool is "intuitive," so the reasoning goes; it is sold as the classic example of learning by applying. Lütke's own memo contradicts this, noting that "using AI well is a skill that needs to be carefully learned." The McKinsey gap between 80% reporting individual productivity gains and 37% reporting any EBIT impact shows why individual productivity is a poor proxy for organizational change. Individuals may get more productive, whatever that means in this context, which does not imply that the organization has changed at the same time. The same survey shows where the difference lies: nearly three-quarters of high performers report fundamentally redesigning workflows because of AI, against one-quarter of everyone else. Deloitte's June 2026 pulse check of nearly 3,700 professionals found that 48% were adding AI without redesigning workflows or roles, and only 12% were redesigning workflows or roles at scale. The divide runs between organizations that change the nature of work and those that bolt AI onto whatever structure they have. Rhyme 3: The Greenfield Showcase Every transformation needs a success story for the board, so a team with no dependencies on the legacy systems, no regulatory exposure, and no operational duty builds something impressive. The Agile version was the "pilot team" in the innovation lab with the fancy toys. The AI version is the internal chatbot that answers HR policy questions and was presented at the town hall as evidence of AI's great potential. Exploration detached from production constraints is useful. The anti-pattern is mistaking evidence that something can be built for evidence that the organization has created value. BCG's 2025 survey of 1,250 senior executives found that 70% of AI's potential value sits in core business functions such as sales and marketing, manufacturing, supply chain, and pricing, which is where the showcase usually never goes, due to the "unsexiness" of the use cases. Rhyme 4: The Consultancy That Sets It Up for You In come the slide decks, the "AI transformation office," and the currently fashionable forward-deployed engineers. The role name dates back to Palantir in the early 2010s; the practice is far older. Thomas Otter, who spent years at SAP, notes that "early chunks of SAP R/1 were built at ICI and John Deere," decades before anyone called the practice forward deployment. I do not consider the practice an anti-pattern. Engineers who join the organization, learn its culture, and stay long enough to hand over applications built on understanding are legitimate. The anti-pattern is the parachute version: the engineers arrive, do the tactical technical work, and leave, and the organization is now running systems it cannot explain. Agile had the consultancy-staffed transformation office that left when the budget line ended. Rhyme 5: The Cost Story Ask most leadership teams why the organization adopts AI, and you get a story about new business, better products, and faster learning. Ask what the business case they signed off actually contains, and you find payroll. Consultancies, in my observation, sell AI as they sold offshoring: a way to remove people who do repetitive work. Cost reduction, as such, is not the anti-pattern; however, turning it into the transformation objective is. About 80% of McKinsey's high performers, and everyone else, pursue efficiency, but most high performers also pursue growth or innovation, thereby distinguishing the two approaches. Klarna ran the other experiment in public. After claiming its AI assistant did the work of 700 customer service agents, CEO Sebastian Siemiatkowski told Bloomberg in May 2025, as CX Dive reported, that "cost unfortunately seems to have been a too predominant evaluation factor when organizing this; what you end up having is lower quality," and started hiring humans again. McKinsey's respondents have noticed which story their leadership actually believes: 39% now expect AI-related job cuts, up from 32% a year earlier. Agile had the same split. The board heard "faster and cheaper"; the teams heard "better products"; and when the two stories collided, the teams lost. What the Five Rhymes of AI Transformations Share Each AI transformation rhyme has a visibility problem. Leadership can see the headcount numbers perfectly well and still optimize them; a consultancy dependency is a capability-transfer problem, while a mandate is an authority and incentive problem. What the five have in common sits one level down. In each case, the organization makes its AI decisions at organizational scale (a mandate, a license contract, a showcase budget, a vendor engagement, a business case) without leaving inspectable evidence at the workflow scale. Too often, nobody can show, for a specific workflow, who decided that AI would do this work, on what terms, under what cost constraints, who checked it, and with what result. Visibility is the symptom, and missing evidence is the condition. Scrum already had low-tech answers to similar problems: an ordered Product Backlog, an explicit Definition of Done, and a recurring Retrospective. None of them needed a platform, and none of them made leadership act. What they did was let a team generate evidence about the system it worked inside. The A3 Delegation System borrows that design principle: make consequential decisions visible before buying another layer of tooling to manage them. Six stages (Decide, Route, Hand Over, Define Done, Inspect, Roll Up), seven artifacts, and no software beyond the AI the team already uses. It is an operating discipline for AI delegation, one workflow at a time, and the evidence is a byproduct of doing the work. Where Each Rhyme Meets a Countermeasure Let us come back to the five "rhymes" and how the A3 Delegation System can mitigate these AI transformation issues: The AI Workflow Inventory makes the license fallacy and the showcase inspectable: Before anything else, the team lists the workflows it already hands to AI, each with an owner. It takes an hour, and the assumption that "people will figure it out" collapses once the list shows what they figured out. You may discover personal AI habits that were never treated as organizational workflows at all, including some touching sensitive data. The inventory also refuses the greenfield showcase by construction. Only existing workflows with a named owner enter it. The A3 Framework decision and the Routing Policy put a countermeasure against the mandate: For each inventory entry, the team decides Assist (AI drafts, you decide), Automate (delegate execution, not responsibility), or Avoid (the cost of failure is trust). Then it routes the work to a model tier by stakes and cost. Leadership can set the boundaries: approved tools, prohibited data, risk limits, or spending constraints. It cannot make the workflow-specific delegation decision from a company-wide memo; the people who know the work can do so in minutes per entry, and the decision is then on paper for leadership to read. Routing is also where the token bill becomes a decision, and precision matters here: while the price per token keeps falling, the cost of operating AI keeps rising, because cheaper tokens invite longer, more autonomous workflows that consume far more of them. Gartner predicted in August 2026 that inference costs per agentic workflow will rise more than fivefold through 2028; its analyst, Will Sommer, said, "Product leaders cannot rely on more efficient token economics to rationalize AI costs." That is the economic problem the Routing Policy addresses at the workflow level: expensive intelligence is a deliberate choice, never a default. The A3 Handoff Canvas and the AI Definition of Done make the parachute inspectable: Six fields (task split, inputs, outputs, validation, failure response, records) and a one-page quality standard per task class. Here is the ownership test for anything a consultancy or a forward-deployed engineer built: can the team fill in these two documents for the system without calling the vendor? If yes, the team owns the delegation, whoever set it up. If no, the organization is renting understanding, and the rent comes due when the engineers leave. The Delegation Audit asks one question, and it is not the cost question: Monthly or every other Sprint, 45 to 60 minutes, four checks: output and source drift, model fit, reversibility, and category creep (Assist work that quietly became unreviewed Automate). Each finding gets an owner and a decision: change the A3 category, change the tier, update the AI Definition of Done, fix the stop rule, or retire the delegation. The Audit asks whether this delegation is still sound. It does not ask what the freed capacity produced. Roll Up, the last stage, compiles what the audits show for those who ask. Whether what they show is worth paying for is a leadership decision, and it sits outside the A3 Delegation System. The system produces evidence for the value conversation, but it does not own the value decision. The AI Working Agreement wraps the other six: It records the team's rules on data, disclosure, responsibility, and review, and it is the document the team hands upward when leadership asks what "AI adoption" looks like here. It is a page that beats a slide on every occasion. Where the A3 Delegation System Stops A skeptical reader, and my readers have watched frameworks overclaim for twenty years, will ask the obvious question: am I criticizing consultancies for selling transformation frameworks and then selling my own? A3 is not an AI-transformation methodology. It is an evidence-generating delegation discipline. It cannot make leadership respond to the evidence. It can make ignoring the evidence harder. It does not determine why your organization adopts AI, nor does it replace a strategy, a portfolio decision, or the conversation about what happens to the people whose repetitive work disappears. What it does is make the absence of those decisions visible within weeks, team by team, in writing, for the price of a few hours. There is a second limit: A3 can tell you whether AI should do a piece of work, which model, what it needs, what acceptable output means, whether the delegation has drifted, who owns it, and what it is allowed to cost. It does not tell you whether the workflow should exist. Suppose the system reduces a weekly reporting workflow from 4 hours to 40 minutes, with excellent output and impeccable governance. The question that remains is why the organization produces that report at all. The A3 Delegation system can prevent undisciplined delegation. It cannot prevent an organization from competently automating work that should have disappeared. The likely next development step of the A3 system is a single field on the AI Workflow Inventory, not another canvas: what changes if this workflow works? I have not added it yet. The team should expect the visibility A3 produces to be unwelcome. A team that runs the inventory, the decisions, and the Audit inside a mandate-driven transformation produces evidence the organization may refuse to absorb. I have watched organizations refuse the evidence their Retrospectives produced for years, and the refusal told the teams more about the transformation than any all-hands did. If your leadership will not read a one-page working agreement and a monthly audit log, you have learned what the AI transformation is for. Conclusion AI transformations may repeat many of the mistakes of Agile transformations. The A3 Delegation System does not prevent organizations from making them. However, it gives teams a simple way to make some of them visible before they become expensive: Count how many of the five rhymes are playing in your organization right now. Respect yourself and be honest while aggregating those. Then put a document against one of them.

By Stefan Wolpers CORE

Over the last two years, the industry has moved from simple chatbots and retrieval-augmented generation pipelines to something fundamentally more powerful: agentic systems. These systems do not just answer questions; they plan, act, observe the consequences of their actions, and keep iterating until a goal is achieved. Whether it is Cursor writing and debugging code, Perplexity performing multi-step research, Manus executing complex tasks through code, or Gemini Deep Research producing long-form investigative reports, the underlying architecture is agentic. At the heart of every agentic system lie 2 critical design decisions. The first is the overall topology: should the system be a single agent that owns the entire problem, or a multi-agent system where specialized agents collaborate under an orchestrator? The second is the choice of internal design patterns that govern how the agent reasons, selects tools, handles errors, and improves its own output. Get these decisions right, and the system becomes reliable, scalable, and genuinely useful. Get them wrong, and you end up with brittle loops, runaway costs, or agents that hallucinate tool calls. This article provides a clear, production-oriented map of both layers. Let us examine single-agent versus multi-agent architectures, then walk through the six design patterns that dominate real-world systems today. Each pattern is illustrated with a diagram and explained with concrete examples drawn from products already in production. 1. Agentic Systems Topology Every agentic system falls into one of two broad categories. Single Agent System In a single-agent architecture, one agent owns the complete loop. It receives the user query, maintains both short-term context and long-term memory, decides which tools or MCP servers to call, observes the results, and eventually produces the final output. This design is simple to implement, easy to debug, and has lower latency for focused tasks. It is the natural starting point for most teams. The main limitations appear when the problem becomes long-horizon or requires genuinely different skills. Context windows fill up, specialized knowledge is hard to isolate, and a single failure mode can bring the entire system down. Multi-Agent System A multi-agent system introduces a meta-agent (or orchestrator) that decomposes the high-level goal and delegates work to specialized agents, for example, a data-retrieval agent, a search agent, a coding agent, or a critic. Each specialist may have its own tools and memory. Results flow back to an aggregator LLM that synthesizes the final answer. Shared or private memory and MCP servers support the collaboration. The advantages are specialization, parallelism, and higher reliability through division of labor. The costs are coordination overhead, higher token consumption, and more complex failure modes (handoff errors, inconsistent state, cascading retries). Most production systems begin as single-agent and only move to multi-agent when the benefits clearly outweigh the complexity. 2. Core Design Patterns Topology decides how agents are organized. Design patterns decide how each agent thinks and acts. The six patterns below appear, often in combination, in virtually every serious agentic product shipping today. ReACT Agent (Reason + Act) The foundational pattern used by the majority of tool-using agents. The agent interleaves three explicit steps in a tight loop: Thought: verbalized reasoning about the current state, what is known, what is still missing, and which single action would close the biggest gap.Action: a concrete tool call with specific arguments.Observation: the tool result is injected back into the context so the next thought is grounded in reality. This loop continues until the agent decides the task is complete. Pure chain-of-thought reasoning is blind to the external world; pure tool calling is planless and reactive. ReAct fuses the two and remains the default architecture in LangGraph, LlamaIndex, the OpenAI Agents SDK, Claude’s tool-use agents, and most coding assistants. The main engineering concerns are infinite loops (always set a hard iteration limit) and context growth (every thought and observation consumes tokens). CodeAct Agent Used by Manus, OpenHands, and an increasing number of advanced coding and automation systems. Instead of emitting one discrete tool call per turn, the agent writes executable code (usually Python) as its primary action space. That code can contain control flow, multiple tool invocations, data transformations, filtering, and error handling — all in a single execution step. The sandbox runs the code and returns stdout, stderr, or artifacts. If something fails, the agent can observe the error and revise the code, achieving a form of self-debugging. CodeAct collapses what would have been many ReAct steps into one more expressive action. It also leverages the full power of existing software libraries. The trade-off is the need for a secure, well-instrumented execution environment. Self-Reflection Critical for reliability and high-stakes outputs. After generating a first draft (a plan, a piece of code, or an answer), the agent or a dedicated critic LLM evaluates the output against explicit criteria: correctness, completeness, safety, style, and grounding. If the critique fails, the agent revises using the feedback. The loop continues until the critique passes. Lessons can optionally be written into long-term memory, so the same mistake is less likely in the future. Self-reflection is the agentic analog of System-2 deliberative thinking. Andrew Ng has repeatedly listed it as one of the four fundamental building blocks of agentic systems (alongside planning, tool use, and multi-agent collaboration). It is especially valuable when the cost of a wrong answer is high. Tool Use and Agentic RAG Tool use is the substrate of nearly every modern agent. Cursor is a canonical example: the agent is given a rich set of tools (file system, terminal, search, browser, cloud APIs, MCP servers) and decides at each step which tools to call and with what arguments. The quality of the tool schemas and the system prompt that teaches the model when and how to use them is often as important as the underlying model itself. Agentic RAG elevates classic retrieval-augmented generation. In a traditional RAG pipeline, the retrieval step is fixed: embed the query, fetch top-k chunks, generate. In agentic RAG, retrieval itself becomes a tool the agent can invoke repeatedly. The agent can rewrite queries, decide which sources to consult (vector database, web search, structured APIs), evaluate intermediate results, and continue retrieving until it judges the context sufficient. Only then does it generate the final grounded answer, usually with citations. Perplexity’s more advanced research modes and modern enterprise agentic RAG frameworks follow this pattern. Multi-Agent Workflow A planner or meta-agent receives the high-level goal and decomposes it into sub-tasks. Specialized agents execute those sub-tasks, often in parallel, reading from and writing to shared memory and using tools as needed. An aggregator or synthesizer LLM collects the partial results and produces the final coherent output. Optional critique or reflection loops can be added on top. This pattern shines on long-horizon, multi-skill problems such as deep research, complex software engineering workflows, or multi-document analysis. Gemini Deep Research is a prominent production example. 3. Choosing and Combining Patterns PatternBest suited forLatency / CostReliabilityComplexityReActAdaptive multi-step tool useMediumHighLow–MedCodeActComplex logic & data pipelinesLower (fewer turns)HighMediumSelf-ReflectionHigh-stakes accuracyHigherVery HighMediumTool UseAny grounded actionLowBaselineLowAgentic RAGKnowledge-intensive questionsMedium–HighHighMediumMulti-AgentSpecialized + parallel long-horizon workHigherHighHigh In practice, these patterns are almost never used in isolation. Cursor combines heavy tool use with a ReAct-style loop and test-driven reflection. Manus leans on CodeAct. Deep research systems blend multi-agent orchestration, agentic RAG, and reflection. The patterns are composable building blocks rather than mutually exclusive choices. 4. Engineering Principles That Matter Most Regardless of which patterns you choose, several engineering practices determine whether the system is production-ready: Explicit termination conditions and hard iteration limits prevent runaway loops and cost explosions.High-quality tool schemas and descriptions are often more important than the choice of model. Clear parameter types, realistic examples, and guidance on when to use each tool dramatically improve reliability.Careful memory management decides what stays in the context window versus what is externalized to a vector store or key-value memory.Full observability: Every thought, action, and observation must be logged and inspectable. Without traces, debugging agentic systems is nearly impossible.Cost, safety, and permission guardrails must be enforced at the harness level, not left to the model.End-to-end evaluation that measures real task success (not just next-token accuracy) is essential for continuous improvement. Closing Thoughts Agentic systems represent a genuine architectural shift. We are moving from models that merely generate text to systems that can plan, act in the world, observe the consequences, and improve their own behavior. The difference between a fragile demo and a reliable product usually comes down to the quality of the topology and the design patterns chosen. We have discussed in this article, the following: Single-agent versus multi-agent structure, ReAct as the workhorse reasoning-and-acting loop, CodeAct for expressive actions, self-reflection for reliability, tool use as the universal substrate, agentic RAG for dynamic knowledge retrieval, and multi-agent workflows for complex collaboration give practitioners a practical decision framework. Start simple. Begin with a single agent using the ReAct pattern and solid tools. Add self-reflection when accuracy matters. Introduce CodeAct when the logic becomes complex. Move to multi-agent architectures only when specialization and parallelism clearly pay off. Instrument everything. Measure real task success. Iterate. The systems that follow these principles are already delivering meaningful value in coding assistants, research tools, enterprise automation, and personal AI agents. The patterns themselves will continue to evolve, but the underlying ideas — interleaving reasoning with action, grounding decisions in observation, and composing specialized capabilities — are likely to remain foundational for years to come.

By Ram Ghadiyaram CORE