AI/ML Resources

DZone's Featured AI/ML Resources

This $5,000 Berkeley Humanoid Can Be Built With a Desktop 3D Printer

By Aminu Abdullahi

A humanoid robot you can build with a desktop 3D printer is lowering the barrier to experimenting with machines that usually cost far more. Researchers at the University of California, Berkeley, have developed the Berkeley Humanoid Lite, an open-source humanoid robot designed to give students, hobbyists and researchers a cheaper way to experiment with robotics. The roughly 1-meter robot weighs about 16 kilograms and costs less than $5,000 in hardware, according to UC Berkeley Engineering. That is still a serious expense, but far below the cost of many commercially built humanoid platforms. More importantly, Berkeley is not selling it as a finished robot. The project is meant to be a starting point that people can build, modify, and learn from. The robot uses a modular actuator built around a brushless DC motor, magnetic encoder and 3D-printed cycloidal gearbox. The largest printed components fit within a standard 200-by-200-by-200-millimeter desktop 3D printer, while the rest can be sourced from common online suppliers. The hardware is only half the story The Berkeley team has also released the robot's hardware design, embedded code, training and deployment frameworks as open source. That makes the project potentially useful beyond humanoid robotics. The modular actuators can be used individually and adapted to different configurations, including bipedal and quadruped designs, according to Interesting Engineering. The researchers tested the actuators for efficiency and durability. Interesting Engineering reports that the gearbox reached about 90% mechanical efficiency under most conditions, while a 60-hour endurance test showed gradually increasing backlash as the printed components wore. The robot has also demonstrated basic walking and object manipulation. Researchers used reinforcement learning for locomotion and a VR-based teleoperation system for tasks including moving objects and solving a Rubik's Cube. The Berkeley team, however, acknowledges that its walking remains imperfect, while the long-term effects of heat and wear on the 3D-printed structure require more study. What eWeek found: The real opportunity is the actuator The most interesting part of Berkeley Humanoid Lite may not be the humanoid at all. Its modular actuator could become the project's most useful research and educational building block because developers can experiment with a single robotic joint before committing to an entire machine. That lowers both the financial and technical risk of getting started. The open-source design removes another barrier, but hardware ecosystems do not grow from design files alone. Berkeley still needs a community that builds the robot, documents failures, improves components, and makes those changes useful to the next person. If that happens, Berkeley Humanoid Lite could become more valuable as a platform than as a single robot. Its biggest contribution may ultimately be creating a repeatable way for students and smaller robotics teams to learn how humanoids are built from the joint up. Editor’s note: This article originally appeared on our sister publication, eWeek. More

Prompting AI for Analytics: The Missing Optimization Layer Between Your Question and the Model

By Dinesh Pamcheti

The answer was right. The question cost four times what it needed to. Every analytics team using AI models runs into the same quiet cost: wasted tokens from messy prompts. An analyst types a vague question, the model gives back a long, hedge-y answer, the analyst rewrites the question, asks again, and the loop repeats. Multiply that across hundreds of analysts asking questions every day, and it's not just money burned on tokens — it's time lost and answers that don't line up with each other. The usual fix is to train people to write better prompts. That doesn't really work in an analytics team, where people range from SQL experts to product managers who've never heard the term "system prompt." You can't expect everyone to become a prompting expert, and you shouldn't have to. What's actually missing is a layer that sits between the person and the model — something that takes a rough, real-world question, cleans it up into a tight, clear prompt, and only then sends it to the AI. Not a smarter model. A cleanup step in front of the one you already have. The Idea, in Practice Think of it as a quick check before takeoff. Before a question ever reaches the main AI model, it passes through a smaller, cheaper step that does four things: it strips out repeated context and filler words that burn tokens without adding anything useful; it clears up what's actually being asked, tightening a vague question like "show me the numbers" automatically or flagging it with one quick clarifying question; it reshapes the request into a consistent structure — goal, scope, filters, output format — that the model can act on faster; and it pulls in only the slice of a dashboard, schema, or past query history that's actually relevant, instead of attaching everything on hand. The result is that the expensive model only ever sees a short, clear prompt — not the rough, rambling way people actually type when they're thinking out loud. And because this happens in a small, cheap step before the real model call, the overhead of doing it is trivial next to what it saves downstream. This matters more in analytics than almost anywhere else AI gets used. Analysts tend to ask the same kind of question over and over — "compare Q2 vs Q3 revenue by region," "why did churn spike in March" — which means there's rarely a reason to reinvent the prompt from scratch each time. Volume is high and prompting skill is intentionally not the point: analysts want the answer, not a lesson in context engineering. And because business reporting depends on repeatable numbers, a standard prompt shape produces more consistent answers than everyone phrasing the same question their own way. Here's roughly what that looks like end to end — a request comes in, gets cleaned up by a small router model, gets grounded in real schema and glossary data, gets routed to the right model, and the outcome gets logged so the system can improve next time: A Heavier Example: The Quarterly Business Review The savings are easiest to see with a request that's genuinely heavy — the kind that piles up context fast in a normal BI workflow. Picture an analyst asking: "can you pull together everything on how we did this quarter — revenue, churn, top segments, regional breakdown, how we compare to the last few quarters, and check if the new pricing tier is helping or hurting. need the full picture for the board deck" Handled the way most people actually work, that request gets answered by attaching whatever's on hand — a few dashboard exports, several quarters of raw table data, the full schema documentation, maybe last quarter's board deck for reference. To be clear, the numbers below are an illustrative estimate, not a benchmark run against a real system — but if you've ever watched someone paste four exports and a glossary into a chat window before asking a question, the shape of it should feel familiar: that's easily thousands of tokens of attached context before the model has done any actual reasoning, covering five different metrics with no explicit scope. A cleanup layer instead decomposes the request into five clear sub-questions, resolves the comparison window (this quarter vs. the trailing four), pulls only the specific tables those five metrics actually touch, keeps aggregates instead of raw rows, and fixes the output shape up front — something like: Objective: quarterly business review, current quarter vs. trailing four. Metrics: revenue, churn, segment mix, regional mix, pricing-tier impact. Data: five named tables, summary level only. Output: a five-part board-ready summary. Same request, same five topics, but a small fraction of the original context — in a setup like this, you'd plausibly see a reduction in the range of 80–90%, since the naive version is mostly redundant attachments rather than information the model actually needs. The exact number depends entirely on how much got over-attached in the first place, which is precisely the point: the waste is rarely in the question; it's in what gets bolted onto it. The mechanism that makes this affordable is using a small, cheap model to do the cleanup, and reserving the expensive model for the actual reasoning. The cleanup pass costs a sliver of the token budget it saves. Memory, Guardrails, and the Cold-Start Problem A cleanup layer that only ever looks at one prompt at a time is leaving something on the table. Most analysts across a company end up asking structurally similar questions, just worded differently — and if the system re-derives the same structure from scratch every time, it never gets any smarter from what the organization has already asked. The natural extension is a shared memory of good prompt patterns: when a pattern is used, and the outcome is accepted without correction, it's a candidate for reuse. If a second analyst — on a different team, weeks later — asks for essentially the same kind of summary with different filters, the system can retrieve that pattern and adapt it instead of starting cold. This is also where the mechanism keeps learning: every outcome (accepted as-is, corrected, or manually edited before sending) gets logged, and that signal is what tells the system whether a pattern is worth keeping, needs review, or was actually a miss in the cleanup logic itself. That memory only works if it doesn't turn into a liability, so it needs three guardrails from day one, not bolted on later. First, access has to be inherited, not invented — a stored pattern only surfaces for someone who could already see the underlying data, and sensitive domains (HR, legal, anything under investigation) stay out of shared memory by default. Second, nothing gets trusted just because it was used once — a new pattern sits in a probationary state until it's been reused successfully by more than one person with no corrections, and only then graduates to something the system will actively suggest. Third, patterns expire — each one is tagged to the schema and glossary version it was built against, so when a business definition changes (what counts as an "active user," say), the pattern gets flagged for re-validation instead of quietly giving a stale answer with total confidence. This also answers the obvious objection: what happens on a genuinely new question, with no pattern to draw on? Nothing breaks — the system still has the business glossary, the schema, and a generic template for the type of question being asked (root-cause, comparison, trend, forecast), so a first-of-its-kind question still gets meaningfully cleaned up. It just doesn't get the extra head start that a repeat question gets once a pattern exists. Cold or warm, the request ends up in the same place; only the source of the structure differs. Where This Breaks Down None of this is free, and it's worth being upfront about the costs. There's a small amount of added latency from the cleanup step itself — for most analytics workloads that's a fair trade, but it's a real one, and it's not the right call for anything latency-sensitive. There's also a real risk of over-simplifying a question: an aggressive cleanup pass can strip out something the analyst actually meant, which is why the rewritten prompt should always be visible and editable before it's sent, not applied silently. And the system needs a clear owner — someone reviewing what gets promoted into shared memory, someone keeping the schema and access rules in sync — because a black box that nobody can inspect is worse than the problem it's solving. Worth noting too: this idea isn't limited to analytics questions. The same pattern — cleaning up an underspecified request before it becomes expensive — shows up in "vibe coding," where a developer describes a data pipeline in plain English and lets a model write it. The failure mode is nearly identical: the model guesses at table names, error handling, and whether a job is safe to re-run, because nobody specified it. A cleanup layer that injects the real schema and fills in the unstated technical spec solves the same problem there that it solves for an analyst's question here. The core claim is a modest one: most of what AI costs in an analytics team isn't the reasoning; it's the raw material we hand the model before it starts reasoning. Clean that up first, and the model you already have gets meaningfully cheaper and more consistent — no upgrade required. Further Reading Jiang et al., LLMLingua (Microsoft Research, EMNLP 2023) — prompt compression by pruning low-information tokens.Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (NeurIPS 2020) — the original RAG paper, the root idea behind grounding a prompt in real schema and glossary data before answering (RAG means the model's answer is grounded in retrieved data rather than pulled from its own training memory).Dekoninck et al., A Unified Approach to Routing and Cascading for LLMs — routing simple requests to a cheap model and reserving an expensive one for harder cases.AI Automation Essentials, Tuhin Chattopadhyay, DZone Refcard — broader background on AI automation architecture and governance. More

Multilingual Conversational Payments Chatbot Architecture: Enterprise RAG With Safety Guardrails, Human Handoff, and Multi-Modal Support

By Sriram Ramakrishnan

Building Agentic RAG, Step by Step: From Static Retrieval to Reasoning Pipelines

By Balaji Venkatasubramaniyar

CORE

Handling Large API Responses Without Freezing the Client: A Practical Architecture With Temporal, Kafka, and RAG

By Uthej Mopathi

CORE

MCP for Enterprise Tasks: Making the Rare Frequent Enough to Master

Enterprise software increasingly requires users to wear many hats. The security engineer who manages access controls one day handles incident response the next. The operations team that automates routine deployments must also handle one-off infrastructure exceptions. These occasional, cognitively complex tasks fall into an awkward gap: too infrequent for muscle memory, too important to leave to guesswork. Traditional enterprise UX has two unsatisfying answers: build heavyweight forms that users must re-learn each time, or route the work through approval chains that turn minutes into days. Model Context Protocol (MCP) paired with agentic LLM interfaces offers a third path — one that's particularly compelling for access control management and similar domains where tasks are rare but high-stakes. This article explores when and how agentic MCP front-ends make sense for enterprise applications, and why access control is the ideal beachhead for this pattern. Chapter 1: The Muscle Memory Problem The Invisible Boundary in User Experience Enterprise users develop deep muscle memory around frequent tasks. A Kubernetes operator who uses kubectl daily doesn't need documentation. A database administrator executing routine backups can do it blindfolded. The UI — whether CLI, web form, or API — recedes into the background. Decision-making becomes automatic. But this advantage exists only for tasks that happen regularly enough to build that muscle. Once a task drops below a certain frequency threshold — monthly, quarterly, or annual — muscle memory evaporates. The Cost of Infrequent Tasks When a user encounters an infrequent task, the friction is immediate: Discoverability: Where is the feature in the UI? The documentation is outdated. The form has moved.Decision anxiety: "Did I fill this out correctly last time?" Users second-guess themselves, leading to errors or unnecessary escalations.Cognitive load: The user must reconstruct mental models of how the system works, often while under pressure (access needed urgently, exception must be approved today).Documentation debt: The system requires extensive, constantly updated docs to compensate for the lack of muscle memory. Access control is textbook infrequent. A typical enterprise user might: Request a new team membership once per quarterGrant access to a departing colleague's replacement once a yearHandle an access exception during an incident — rarelyBulk-manage permissions after an org restructure — once every few years Each of these is cognitively complex (understanding role hierarchies, permission dependencies, compliance constraints) and high-stakes (granting too much access is a security incident; denying access blocks business). Yet the user has no muscle memory to fall back on. Why Traditional UIs Fail Rare Tasks Traditional enterprise interfaces optimize for either: Forms and wizards: Comprehensive, visually documented, but inflexible. They guide users through a fixed happy path. When reality is messier (conditional requirements, edge cases, cross-system coordination), the form either breaks down or forces escalation.Powerful but complex CLIs: Low friction for experts, high friction for newcomers. A user comfortable with ldapsearch might balk at learning another tool's syntax. Access control often requires touching multiple systems (LDAP, IAM, Git, Jira, cloud providers), each with its own CLI grammar.Delegated workflows: "Submit a ticket, someone will process it in 2 days." Safe, auditable, but slow. Turns infrequent tasks into day-long ordeals. All three approaches assume the user will invest time in mastery. For genuinely rare tasks, that assumption breaks down. Chapter 2: The Agentic LLM Opportunity Natural Language as the Rare-Task Interface An agentic LLM removes the requirement to learn the system's UX grammar. Instead of asking "where is the form?", the user simply describes what they need: "Grant Alice access to the backend services in production, but not the payment processing service. Make her a read-only user for the first week, then escalate to write access if the team signs off." The LLM understands intent, decomposes it into steps, and asks clarifying questions: "I notice Alice is joining the payment team. Should I assume she'll eventually need full access to payment services? And do you want me to set up a reminder to review her access after 30 days?" This is radically different from filling out a form. The user doesn't need to know the system's structure; they describe the outcome, and the system figures out how to achieve it. Why MCP Is the Natural Fit Model Context Protocol is purpose-built for this use case. It provides: Structured tool definitions: The LLM knows exactly what operations are available, their parameters, constraints, and side effects.Bidirectional communication: Tools can return results, ask for clarification, or report errors—creating a closed-loop decision-making process.Composability: Complex workflows are just sequences of MCP calls. The LLM orchestrates them.Auditability: Every operation is traceable back to the original intent and the reasoning chain that led to it. Access control exemplifies why MCP shines. A well-designed MCP server for access management might expose tools like: add_user_to_group (with parameter validation, compliance checks)grant_permission (with role hierarchy resolution)audit_permissions (to verify the user's current access before making changes)check_compliance (to ensure changes don't violate policies)request_approval (to route high-risk changes through approval chains) An agentic LLM chat interface orchestrates these tools based on user intent, handling nuance that no form could capture. The Chat Interface Advantage The chat interface solves the muscle memory problem at its root: No need to rediscover the UI: The user talks, not clicks.Ambient intelligence: The LLM can infer context, suggest next steps, and warn about edge cases.Bidirectional conversation: If the user is ambiguous, the LLM asks for clarification before taking action.Onboarding for free: A first-time user and a power user have the same entry point. The system adapts to their familiarity level. Chapter 3: Access Control Management as the Ideal Use Case Why Access Control Is Perfect for Agentic MCP Access control sits at the intersection of three factors that make MCP agentic interfaces shine: 1. Rare but recurrent. No user does access management daily, but nearly every user does it occasionally. The infrequency is high enough that muscle memory never forms. 2. High cognitive complexity. Access decisions aren't simple. They depend on: Role hierarchies and inheritanceCompliance and regulatory constraints (SOC2, GDPR, ISO 27001)Time-bound access (onboarding ramps, offboarding holds)Cross-system coordination (a user needs LDAP groups and Kubernetes RBAC and cloud IAM roles)Conditional logic ("access to staging unless it's production, unless it's an emergency") No UI can capture this without becoming baroque. 3. High stakes, high audit burden. Incorrect access decisions are security incidents. Every grant and revocation must be auditable, justified, and reversible. An agentic LLM shines precisely because it can: Handle complexity through natural languageReason about constraints and trade-offsGenerate clear audit trails (intent → reasoning → actions)Ask for approval before high-risk operations Real-World Access Control Scenarios Scenario 1: Onboarding (low complexity, high clarity) User: "Add Sarah to the backend team. She starts Monday." LLM reasoning: Infers role from team membershipChecks standard access package for backend teamDetects that Sarah needs database credentials (missing from form)Confirms: "Sarah will also need dev database access. Should I create credentials and email her securely?"Provisions all systems in parallelCreates a checklist for the hiring manager This is where agentic MCP is an unambiguous value-add. No form could anticipate the cross-system dance; a human coordinator would take days. Scenario 2: Offboarding with exceptions (medium complexity, high uncertainty) User: "John is leaving. Revoke most access, but leave him read-only access to the documentation wiki for 30 days so he can clean up his work." LLM reasoning: Audits John's current access (broader than expected)Asks: "John has admin access to staging. Remove that immediately, or revoke gradually?"Asks: "He also has database backups. Should those be revoked now or after 30 days?"Proposes: "I'll revoke write access everywhere except the wiki, set a calendar reminder to revoke the wiki access in 30 days, and flag this for security review."Generates an audit report before executing Scenario 3: Exception handling during incidents (high complexity, time pressure) User: "Production is down. Grant the on-call engineer temporary admin access to debug. We'll review it afterward." LLM reasoning: Verifies this is actually the on-call engineer (prevents social engineering)Creates time-bound access (auto-revoke in 2 hours)Escalates the request through the approval workflow (if configured)Logs the incident contextReminds the team to review what was done within 24 hours In each case, natural language captures intent and context that structured forms cannot. The agentic system handles the complex orchestration. Chapter 4: The Trust and Guardrails Problem Why Agentic Execution Requires Caution The power of agentic MCP comes with a serious caveat: users can't see what they're authorizing the same way they can click a button and see the result. When a user fills out a form and submits it, they have a mental model: "I clicked send; the form was processed; the database was updated." The causality is clear and immediate. With agentic execution, the user delegates to the LLM, which reasons, orchestrates, and executes. This introduces several risks: Hallucination or misinterpretation: The LLM might misunderstand the intent, or make an incorrect assumption about what's desired.Silent failure: The LLM might silently choose a path that violates unstated constraints. "Grant read access" might be interpreted as "grant access to production data" when the user meant "grant access to development documentation."Over-reaching scope: The LLM might infer that since it has permission to grant access, it should grant more than requested. "Add to the team" becomes "add to the team and the team's security group and the team's repository." For infrequent tasks, users are particularly vulnerable to these errors because they lack the domain knowledge to spot them. Guardrails: The Approval Layer The solution is explicit approval before write operations. This doesn't mean clicking OK on a dialog; it means: Clear step-through: Before executing any write operation, the LLM presents: What it intends to do (user-friendly summary)Why (the reasoning chain)What it will change (specific systems, permissions, values) Example: Plain Text I'm about to: 1. Add [email protected] to the 'backend-team' LDAP group 2. Grant her read-write access to the 'backend-repo' Git repository 3. Create database credentials for staging (prod will require separate approval) This is because: She's joining the backend team on Monday and needs the standard backend engineer access package. Proceed? (yes/no/modify) Conditional escalation: High-risk operations (production access, bulk changes, compliance-sensitive access) automatically escalate to a human approver before execution, not after.Audit logging with reasoning: Every operation is logged with: Original user intent The reasoning chain the LLM followed Approvals (who approved it, when) Actual operations executed Results and side effects This creates accountability and makes post-incident analysis tractable. Reversibility and undo: Design operations so they can be undone. A user grants access by mistake; they should be able to say "undo the last operation" without needing an operator.Time-bound access by default: Access should expire unless explicitly renewed. "Grant temporary admin access" (default: 2 hours) is safer than "grant admin access" (default: forever). The Two-Tier System In practice, access control implementations benefit from a two-tier approach: Tier 1: Agentic LLM + MCP for: Infrequent, ambiguous, high-context tasksUsers who lack deep domain knowledgeOnboarding and exception handlingTasks that require cross-system coordination Tier 2: Direct API/CLI for: Frequent, predictable tasksPower users and automationBulk operations that are scripted and testedCases where audit trail needs are served by code review, not LLM reasoning This allows organizations to optimize each path without forcing everyone through the same gate. Chapter 5: Implementation Considerations Designing the MCP Server A well-designed access control MCP server should expose: Core operations: add_user_to_group(user, group, [approval_required])remove_user_from_group(user, group)grant_permission(user, resource, level, [duration])revoke_permission(user, resource, [duration]) Query operations (for reasoning): get_user_access(user) – audit what access a user currently hasget_group_members(group) – understand group membershipcheck_compliance(operation) – verify that a proposed change doesn't violate policiesget_access_requirements(role) – understand what access a role typically needs Administrative operations: request_approval(operation, [approvers]) – route high-risk operations through approval chainslist_pending_approvals() – let users track requestsaudit_log(filter) – retrieve audit history Each operation should include: Clear parameter validation: The server, not the LLM, should enforce that parameters are valid.Side-effect reporting: Operations should return what actually changed, so the LLM can confirm with the user.Error clarity: When an operation fails (e.g., user doesn't exist), return a clear message, not a cryptic error code. Designing the LLM Prompts The system prompt should: Define the scope: "You are an access control assistant. You can grant access, revoke access, and audit permissions. You cannot delete accounts or change billing."Specify the approval flow: "Before any write operation, present the changes to the user for approval. Always escalate production access changes. Always escalate bulk changes (>5 users or >10 permissions)."Emphasize caution: "If you're uncertain about the user's intent, ask clarifying questions before taking action. Better to be verbose than to grant incorrect access."Define the reasoning style: "Explain your reasoning in plain English. Don't assume technical knowledge. Anticipate edge cases and ask about them."Set time defaults: "When access duration isn't specified, default to temporary access. For onboarding, default to 30 days or until manual review. For incidents, default to 2 hours." Integrating with Existing Systems Real access control is multi-system. The MCP server needs to orchestrate: LDAP/Active Directory: For user and group managementKubernetes RBAC: For cluster accessCloud IAM (AWS, GCP, Azure): For cloud resource accessGit/GitHub: For repository accessCustom applications: For app-specific roles and permissions This is complex, but it's exactly where agentic MCP shines. The LLM can say "grant access to all backend services," and the MCP server translates that into the right LDAP groups, Kubernetes roles, and cloud policies. Chapter 6: When NOT to Use Agentic MCP Domains Where Traditional UX Still Wins Agentic MCP isn't universally better. Some domains still belong in traditional UIs: Frequent, simple tasks: If users do something daily and it's straightforward (toggle a flag, submit a form), a button is faster than a chat. Muscle memory applies. Zero-tolerance for error: In domains where a single mistake is catastrophic (power grid control, surgical robotics, financial settlements), agentic systems might not be trustworthy enough, regardless of guardrails. Human-in-the-loop is necessary but not sufficient. Compliance-mandated workflows: Some regulations require specific approval chains, audit trails, or human sign-offs. An agentic system must be architected to satisfy these legally. Not all can. Completely novel or bespoke tasks: If users are doing something they've never done before and it isn't in the LLM's training data, the agentic system might confidently execute something incorrect. Custom, one-off work still benefits from domain expert guidance. Chapter 7: The Future of Enterprise Access Control Why This Matters Now The convergence of three trends makes agentic MCP access control practical today: LLMs are reliable enough: GPT-4, Claude 3, and similar models can follow complex instructions, reason about constraints, and avoid common errors. Not perfectly, but reliably enough for access control with guardrails.MCP standards are emerging: Protocol maturity means vendors can invest in MCP servers without betting their company on a proprietary API.Organizations are tired of access management friction: Security teams spend enormous effort on access reviews and exception handling. If an LLM can automate 80% of the cognitive work, that's valuable. Organizational Impact Access control is also a strategic inflection point for enterprises. It affects: Security posture: Faster, clearer access decisions reduce the window for misconfiguration.Compliance efficiency: Automated reasoning and audit trails make compliance reviews faster.Developer productivity: Developers spend less time waiting for access and more time shipping.Onboarding time: New hires get access faster and more correctly. An agentic access control system that works reliably becomes a competitive advantage. Conclusion The muscle memory boundary — between tasks users do frequently enough to master and tasks they do rarely enough to stumble through — is a real problem in enterprise software. Traditional UI patterns fail on the rare side. They demand learning, form-hunting, and approval escalations. Agentic LLM + MCP inverts this. By accepting natural language intent and orchestrating complex operations through MCP, the system can make the rare frequent enough to handle without friction. Access control management is the ideal proving ground. It's infrequent, cognitively complex, and high-stakes. It involves cross-system coordination that no single UI can capture. And it's complex enough that an LLM's reasoning ability genuinely adds value. But agentic execution is powerful precisely because it's invisible. The solution is not to hide the reasoning but to expose it: present clear step-through approvals, log reasoning alongside results, and make rollback and undo first-class operations. With these guardrails in place, agentic MCP access control isn't just convenient. It's safer, faster, and more auditable than the status quo. That's a compelling case for the enterprise. Further Reading MCP specification and examples: https://modelcontextprotocol.io/Access control best practices (NIST SP 800-162): https://csrc.nist.gov/publications/detail/sp/800-162/final

By Peter Verhas

CORE

Why I Don't Want an LLM Generating Java Business Logic

A pull request arrives. A few hundred lines of Java implementing the new discount rule: tiered thresholds, a regional exception, something about loyalty tiers that nobody can quite explain. It compiles. The tests pass. An LLM wrote it in about forty seconds. Now: who reviews it? The person who owns that rule is in commercial operations. She knows exactly which customers should get the discount and why the regional exception exists, and she cannot read Java. The person who can read Java has no idea whether the thresholds are right. He will check that the code looks reasonable, because that is the only thing he is equipped to check. So the review that happens is not the review that matters. That is the problem I keep coming back to, and it has nothing to do with how good the model is. This Is Not an Argument About Whether the Model Is Good Enough Most objections to generated code are about competence. The model hallucinates an API. It gets an edge case wrong. It writes something that works on the happy path and falls over in production. I find these arguments unconvincing because they expire. Models get better. Any position resting on today's error rate is a position with a shelf life, and people who staked one out three years ago have mostly had to retreat from it. The durable question is different. It is not how well the model writes. It is what the thing it writes is permitted to say. A model that never makes a mistake, handed Java, can still emit Runtime.getRuntime().exec(...). Not because it is malicious or confused — because that sentence is available in the language it was asked to write. Competence and authority are separate axes, and improving the first does nothing to the second. "Write it in Java" Is a Much Bigger Grant Than Anyone Means Consider what you actually authorize when you ask for a discount rule in Java. You authorize file system access. Network sockets. Reflection. Thread creation. Process execution. Every class on the classpath, including the ones that talk to your database, your payment provider, and your secrets manager. You authorize the loading of new code at runtime. Nobody intends to grant any of this. It arrives free with the language, the way a house key also opens the shed. The task needed perhaps six operations — look up an order, total it, check a customer's tier, apply a discount, log the decision, approve or refuse — and the language you handed over contains everything Java contains. That gap, between the authority the task requires and the authority the language confers, is the whole of it. It exists whether or not the model is trustworthy. It exists whether or not anyone acts on it. It is just very large, and it is not visible in the pull request. The Usual Guardrails Are Denial Lists The standard responses all share a shape. Tell the model in the prompt not to touch the file system. Review the generated code. Run static analysis and flag dangerous calls. Run it in a sandbox with a restricted security policy. Every one of these asks you to enumerate what must not happen, over a space of things that can happen which is effectively unbounded. You are writing a deny-list against a general-purpose language. You have to think of exec. Then of reflection reaching exec. Then of the dependency that shells out on your behalf. Then of the next one. We learned this lesson in security a long time ago and reached a settled answer: allow-lists beat deny-lists, because the allow-list is finite and you wrote it. Somehow, when the subject is generated code, we reach for the deny-list again. Shrink the Language, Not the Model The alternative is to stop constraining a powerful language and instead supply a small one. Give the model a vocabulary that contains exactly the operations the domain has — the six from earlier, say — and nothing else. Not a restricted Java. A different, much smaller language, whose entire vocabulary is a list your team wrote in advance, in Java, on purpose. Generated business logic then looks like this: Python PROGRAM ApproveOrder(orderId INTEGER, limit DECIMAL) RETURNS BOOLEAN DECLARE purchase Order DECLARE total DECIMAL purchase = LOAD_ORDER(orderId) total = ORDER_TOTAL(purchase) IF total > limit THEN REJECT purchase, "over limit" RETURN FALSE END IF APPROVE purchase RETURN TRUE END. LOAD_ORDER, ORDER_TOTAL, REJECT and APPROVE are not part of the language. They are Java classes somebody decided to expose. Order is a Java object the program can hold and pass and never look inside — there is no purchase.customer.account.balance here, only the operations the domain chose to have. Two things change, and the second matters more than the first. The obvious one: dangerous programs are no longer forbidden, they are inexpressible. If the model emits DELETE_ALL_ORDERS, nothing rejects it on policy grounds. The name means nothing. The program does not compile, for the same reason a typo does not compile. There is no deny-list because there is nothing to deny. The less obvious one: the commercial operations manager can read the program above. She can tell you whether the threshold is right, whether the rejection reason is the one the contract requires, whether an approval should have been logged. The review moves to the person who owns the rule. That is the review that was missing at the start of this article, and no amount of static analysis over generated Java produces it. A small language buys something else, quietly. With no data structures, one global scope, no null, and a compiler that refuses to run a program that reads a variable before it is set, entire families of subtle wrongness have nowhere to live. Not caught — absent. What It Costs, and What It Does Not Buy I would not trust this argument from someone who only listed the advantages, so here are the bills. You have to design the vocabulary. Somebody sits down and decides that the domain has ORDER_TOTAL and CUSTOMER_RISK and not forty other things. That is real work, done before the first generated line, by someone who understands the domain. And if nobody on your team can write that list, this approach will not help you. It will only show you that the list does not exist. That is worth finding out, but it is not a pleasant morning. Complex algorithms stay in Java. Business rules are algorithms too, and they belong in the small language; that is the point. But route optimization, a scoring model, anything with real computational substance belongs behind a function the small language calls. The signal is usually that you want to build up a data structure, or that you want a helper you can call from three places. Both mean you have wandered out of business logic and should walk back. The boundary bounds naming, not doing. This is the limit people miss, and overstating it is how the idea gets dismissed. A function you expose can do anything Java can do. RUN_SHELL_COMMAND is a perfectly registrable operation. The vocabulary is only as narrow as the operations you chose, and choosing them badly gets you exactly the exposure you were avoiding. There are no resource limits yet. A generated program can still loop forever. This one is a gap rather than a decision: the interpreter walks the program one statement at a time, so a step budget or a deadline is a small addition rather than a redesign, and it will go in when somebody needs it. Until then, untrusted input needs the same containment any untrusted workload needs. What you get is narrower than "safe" and more useful than it sounds: the set of things a generated program can name is finite, written down, and reviewable by a human before anything is generated at all. When I Would Still Write Java If the thing is genuinely computational, write Java. If it is a one-off that will be deleted next week, use whatever is nearest — Java, Python, a shell script — and let the model write it; do not build a vocabulary for something with a life expectancy of days. If the rules change so fast that the vocabulary would be obsolete before it settled, the overhead will not pay for itself. And if your business logic is already reviewed by people who can read it, understand it, and are accountable for it being right — you may not have the problem this solves. Plenty of teams do not. But if you are about to let a model write business rules in Java, ask the question I started with, because the answer is usually uncomfortable. Somebody is going to approve that pull request. Are they the person who knows whether the rule is correct? If not, the language is too big. I have been building a small language along these lines: BUBAS, an orchestration language for subject-matter experts, embedded in Java. The example above is real BUBAS. The idea does not require my implementation, though — the argument is about the size of the language you hand over, and you can shrink yours however you like.

By Peter Verhas

CORE

Video and Audio as Knowledge Sources: Content Understanding in Microsoft Foundry IQ

Your largest corpus is the one nobody indexed. Here's how to turn recordings into grounding data an agent can cite. The Corpus You Already Have and Never Indexed Every organization is sitting on years of recorded meetings, support calls, training sessions, conference talks, and screen recordings. Almost none of it is retrievable. When someone asks "what did we decide about the vendor migration," the answer exists — in a 47-minute recording nobody will ever scrub through. The reflex fix is to bolt a transcription service onto your RAG pipeline: run Whisper, dump the text into a blob container, index it. It works, sort of, and then you discover what you lost. The transcript has no speakers, no timestamps you can link back to, no slide content, no distinction between the presenter reading a bullet and someone in the room disagreeing with it. Your agent can now quote the meeting but can't tell you when it happened or who said it. Azure Content Understanding is Microsoft's answer to that gap: it ingests documents, audio, images, and video and extracts the most critical information to power well-grounded generative and agentic solutions, combining Document Intelligence's traditional AI with LLM-based content reasoning. And as of Build 2026, it is integrated with Foundry IQ standard mode for built-in content extraction inside Microsoft's retrieval and agent workflows. That integration is the subject of this article. Specifically, the part everyone gets wrong: there are two ways to get media into a knowledge base; they are not equivalent, and the one people assume exists is the one you should verify before you plan a sprint around it. This is a companion to my earlier piece on connecting a Foundry IQ knowledge base to LangGraph over MCP. That one covered retrieval. This one covers what you feed it. Architecture Figure 1 — Two paths into the same knowledge base. The native path is a flag on the knowledge source; the explicit path runs analyzers yourself and lands the output as text. The Two Paths, Stated Plainly Path A — native extraction inside the ingestion pipeline. Setting the contentExtractionMode property to standard on file-based indexed knowledge sources (Azure Blob, SharePoint, OneLake) enables Content Understanding functionality within the ingestion pipeline. One property. No orchestration code. This shipped in the Foundry IQ 2026-05-01-preview release, which focused on richer Content Understanding extraction and image serving for multimodal agentic retrieval. Path B — explicit analysis, then index the output. You run Content Understanding yourself, write the resulting Markdown and fields to blob storage, and point a normal knowledge source at that. More moving parts, complete control. Here is the honest caveat, and I would rather you hear it from me than discover it in week three: Microsoft's Foundry IQ extraction announcements emphasize document understanding — layout, tables, figures, and document-embedded images — rather than audio and video ingestion. Content Understanding standard mode itself is documented for documents, images, audio, and video. Whether your blob knowledge source will accept an .mp4 today, in your region, at your API version, is a question you should answer with a five-minute test rather than an assumption. So: test Path A first, build on Path B if it doesn't cover your media types. Path B is what this tutorial walks through in detail, because it works regardless, and because understanding it makes Path A trivial to adopt when it covers you. Prerequisites A Microsoft Foundry resource and a Content Understanding resource.An LLM deployment for analyzers. Analyzers are powered by LLM and embedding models you deploy in Foundry, and GPT-5.2 improves custom field extraction enough to avoid prompt-engineering gymnastics on mixed layouts, domain-specific language, and multilingual content. Analyzers built on GPT-4.1 continue to run unchanged.An Azure AI Search service for the knowledge base. Shell pip install 'markitdown[az-content-understanding]' azure-search-documents azure-identity Pin Your API Version Before Writing a Line of Code Content Understanding's GA API is 2025-11-01. The preview versions 2024-12-01-preview and 2025-05-01-preview were slated for retirement on July 15, 2026 — a date that has now passed, so if you inherited a codebase targeting either, it is already broken or about to be. Note the asymmetry with Foundry IQ, which is on 2026-05-01-preview for the content-extraction features. You will be running a GA content service against a preview retrieval service. Plan your support expectations accordingly. Step 1: Choose the Analyzer, and Know What Each Modality Costs You Content Understanding ships prebuilt analyzers per modality, and MarkItDown auto-selects among them: documents route to prebuilt-documentSearch, video to prebuilt-videoSearch, and audio to prebuilt-audioSearch. They are not interchangeable, and the differences determine what your agent can cite: DocumentsAudioVideoPrebuilt analyzerprebuilt-documentSearchprebuilt-audioSearchprebuilt-videoSearchPrimary outputLayout-aware MarkdownTranscript with structureTranscript plus visual contextStructure preservedHeadings, tables, figure descriptionsUtterance boundariesScene and segment boundariesNatural citation anchorPage and figure IDTimestampTimestamp and frameGrounding you gainTable cells stay in their tableWho spoke, and whenWhat was on screen, not just saidWhat you still loseNothing much, this is the mature pathVisual aids referenced verballyFine detail in dense slidesCustom field schemaYesYesYesBest forContracts, reports, formsSupport calls, interviewsMeetings, demos, training The row that matters most is natural citation anchor. A document chunk cites a page; an audio chunk cites a timestamp. If you flatten audio into plain text before indexing, you throw away the only anchor that makes a recording navigable — and no amount of clever chunking downstream will recover it. Step 2: The Fastest Path From a Recording to Indexable Text MarkItDown with the Content Understanding backend is the shortest route, and it is genuinely a few lines. Zero configuration auto-selects the analyzer per file type: Python from markitdown import MarkItDown md = MarkItDown(cu_endpoint="<content_understanding_endpoint>") doc = md.convert("report.pdf") # → prebuilt-documentSearch video = md.convert("meeting.mp4") # → prebuilt-videoSearch audio = md.convert("call.wav") # → prebuilt-audioSearch print(video.markdown) The output is Markdown with headings, tables, and figure descriptions inline — exactly the shape downstream chunkers and embedding models prefer. That last clause is the whole argument for this approach: you are not inventing a format; you are producing the format the rest of the stack already wants. With a custom analyzer, the output carries extracted fields as YAML front matter above the body: Python md = MarkItDown( cu_endpoint="<content_understanding_endpoint>", cu_analyzer_id="my-meeting-analyzer", ) result = md.convert("standup-2026-08-14.mp4") --- contentType: video fields: MeetingTitle: Vendor migration review Decision: Proceed with phased cutover Owner: A. Rivera ---  ... That front matter is not decoration. It is your metadata filter, your citation payload, and the difference between "the agent found a relevant meeting" and "the agent told me we decided to proceed, and here is the timestamp." Step 3: Design a Field Schema Worth Extracting Prebuilt analyzers give you good transcripts. Custom analyzers give you answers to questions you ask repeatedly — and those are what make retrieval feel like it understands your business rather than your file formats. Two constraints to plan around: Custom analyzers are built in Content Understanding Studio, not the Foundry portal. The Foundry portal surfaces prebuilt analyzers and a playground, with a deep link into CU Studio that preserves your project context, but custom analyzer creation lives in Studio. Expect to move between two tools. Standard mode is per-file. Standard mode handles single files with straightforward field extraction — documents, images, audio, or video without cross-file analysis or complex reasoning. Pro mode is for multi-step reasoning and cross-file analysis. Since Foundry IQ's integration is with standard mode, per-file extraction is your ingestion contract. Questions like "which meetings contradicted the Q2 plan" are retrieval-time work for the agentic retrieval engine, not ingestion-time work for the analyzer. A schema that earns its keep on meeting recordings: FieldWhy it earns its placeDecisionThe single most-asked question of any meeting corpusOwnerTurns retrieval into accountabilityDueDateEnables recency and deadline filtersSystemsMentionedLets an agent scope to "anything about the billing service"UnresolvedQuestionsSurfaces what a summary would smooth over Resist the urge to extract a summary field. The retrieval engine synthesizes at query time against the actual question; a pre-baked summary just adds a lossy paraphrase your agent might cite instead of the source. One operational note from Microsoft that applies whenever you change models: run side-by-side against your existing eval set before flipping production traffic, since confidence scores, latency, and output accuracy can all shift with a new model. Step 4: Land the Output Where a Knowledge Source Can Reach It Write the Markdown to blob storage, one file per recording, with the front matter intact: Python import pathlib from azure.identity import DefaultAzureCredential from azure.storage.blob import BlobServiceClient from markitdown import MarkItDown md = MarkItDown(cu_endpoint=CU_ENDPOINT, cu_analyzer_id="my-meeting-analyzer") blobs = BlobServiceClient( account_url="https://stfoundryiqprod.blob.core.windows.net", credential=DefaultAzureCredential(), ).get_container_client("meeting-transcripts") def ingest(path: str) -> str: """Analyze one recording and land the result as indexable Markdown.""" result = md.convert(path) name = pathlib.Path(path).stem + ".md" blobs.upload_blob(name, result.markdown.encode("utf-8"), overwrite=True) return name Keep the source recording and the derived Markdown in separate containers. You want the knowledge source pointed at text only, and you want the original media addressable for playback when a citation resolves. Mixing them means either indexing binaries you can't use or losing the link back to the thing a user actually wants to watch. Step 5: Create the Knowledge Source and the Knowledge Base Now it is an ordinary Foundry IQ ingestion. Foundry IQ automates document chunking, vector embedding generation, and metadata extraction for indexed knowledge sources, and schedules recurring indexer runs for incremental refresh. Python from azure.identity import DefaultAzureCredential from azure.search.documents.indexes import SearchIndexClient from azure.search.documents.indexes.models import ( AzureBlobKnowledgeSource, AzureBlobKnowledgeSourceParameters, KnowledgeBase, KnowledgeSourceReference, ) client = SearchIndexClient(endpoint=SEARCH_ENDPOINT, credential=DefaultAzureCredential()) source = AzureBlobKnowledgeSource( name="meeting-recordings-ks", description=( "Transcribed and structured meeting recordings: decisions, owners, " "due dates and unresolved questions, with timestamps. " "Use for questions about what was decided, by whom, and when." ), azure_blob_parameters=AzureBlobKnowledgeSourceParameters( connection_string=BLOB_CONNECTION, container_name="meeting-transcripts", # Path A: enable Content Understanding inside the ingestion pipeline. # Test this against your actual media types before relying on it. content_extraction_mode="standard", ), ) client.create_or_update_knowledge_source(knowledge_source=source) client.create_or_update_knowledge_base(knowledge_base=KnowledgeBase( name="meetings-kb", knowledge_sources=[KnowledgeSourceReference(name="meeting-recordings-ks")], retrieval_instructions=( "Prefer the most recent meeting when decisions conflict. " "Always surface the timestamp and speaker with any quoted decision." ), )) Two things to be deliberate about. The description is load-bearing. The agentic retrieval engine uses it to plan queries and select sources, so write it the way you would brief a new colleague: what is in here, and what kinds of questions it answers. "Meeting transcripts" is a wasted field. The content_extraction_mode="standard" flag is your Path A test. Point it at a container holding one .mp4 and one .md, run the indexer, and look at what got indexed. If the media file produced chunks, you can skip Steps 2 through 4 for that file type. If it didn't, your pipeline is already correct and you have lost five minutes. What Happens at Query Time Figure 2 — Ingestion runs once per recording; retrieval runs per question. The timestamp survives both, which is the point. Nothing about retrieval changes because the source was a video. That is the entire payoff: the agentic retrieval engine plans queries, selects sources, runs parallel searches, and aggregates results, returning extractive data with citations so agents can reason over raw content and trace answers to source documents. Your recordings are now just documents that happen to have timestamps. The 2026-05-01-preview release also added image serving — surfacing document-embedded images during agentic retrieval. For a slide-heavy deck or a demo recording, that is the difference between "the presenter showed an architecture diagram" and actually returning the frame. Combined with figure extraction from Office files, where each figure is retrievable by ID via GET /contentunderstanding/analyzerResults/{operationId}/files/figures/{figureId}, you can build citations that resolve to a picture rather than a paraphrase of one. Making Citations Survive the Pipeline This is where most media-RAG implementations quietly fail. The transcript is indexed, retrieval works, the answer is correct — and the citation says "meeting-2026-08-14.md," which is useless to someone who wants the 12 seconds where the decision was made. Three rules: Never strip the front matter during chunking. If your chunker treats YAML as noise, the extracted fields never reach the index and your metadata filters silently match nothing.Keep timestamp markers inside the chunk text, not only in metadata. Extractive retrieval returns content; if the timestamp lives only in a sidecar field, the model has nothing to quote.Store a deterministic link back to the media. A blob URL plus a timestamp fragment costs you one field at ingestion and turns every citation into a playable link. The test: ask your agent a question, take the citation it returns, and try to get to the exact moment in the recording. If you can't do it in one click, the pipeline isn't finished, however good the answer text looks. Troubleshooting SymptomLikely causeWhat to doMedia files in the container produce no chunksThe knowledge source didn't ingest that file type nativelyFall back to Path B: analyze with CU, index the Markdown output404 or version errors from the CU endpointCode still targeting a retired preview APIMove to the GA API version 2025-11-01Custom analyzer missing in the Foundry portalCustom analyzers are created in CU StudioUse the deep link from Foundry; project context is preservedExtraction quality dropped after a model changeConfidence scores, latency, and accuracy shift between modelsRe-run your eval set side by side before shifting production trafficFields extracted but never filterableFront matter stripped during chunkingPreserve YAML through the chunker; verify fields landed in the indexCitations resolve to a file, not a momentTimestamps kept only in metadataKeep markers in chunk text and store a media URL per chunkCross-file questions return thin answersStandard mode is per-file by designLet the agentic retrieval engine handle cross-document reasoning at query timeOffice figures not retrievableFigures need fetching by IDUse the figures/{figureId} endpoint referenced from the Markdown Where to Go Next A few things landed or were slated for July 2026 that change the calculus here, and are worth confirming against current docs before you design around them: a synchronous API for Read and Layout, an agentic understanding mode for complex documents, data zone and global zone processing for residency, improved custom analyzer training from your own examples, and labeled training data no longer being stored in CU so training inputs stay in your own storage. That last one matters most if compliance review is what's blocking you. The natural follow-on build is a routing layer: send short call recordings straight through prebuilt-audioSearch, send slide-heavy sessions through prebuilt-videoSearch so you keep the visual channel, and reserve custom analyzers for the recording types you query weekly. Extraction quality and cost both track how well that routing matches your corpus. References What is Foundry IQ? — knowledge sources, indexing automation, agentic retrieval and ACL enforcement.Content Understanding standard and pro modes — per-file vs. cross-file analysis, and the preview API retirement notice.Content Understanding models and deploymentsFoundry vs. Content Understanding Studio — which tool does what.Supported document formats and input file limitsBuild a RAG solution with Content UnderstandingContent Understanding in LangChainWhat's new in Azure Content Understanding at Build 2026 — the Foundry IQ standard-mode integration, MarkItDown backend, GPT-5.2 analyzers, and the July roadmap.Improved data processing features in Foundry IQ: richer content extraction and data enrichment — contentExtractionMode, SharePoint indexing expansion, and image serving.Foundry IQ: improve recall by up to 54% with knowledge basesBuild 2026 Session BRK242 — "Turn your agents into action" — agentic understanding mode and the Foundry IQ integration, demoed end to end.MarkItDown on GitHub — the CU-backed converter used throughout this article.Microsoft Agent Framework — for registering CU as an agent tool instead of a pipeline stage.

By Jubin Soni, FBCS

CORE

Designing Safe Agent Permissions: Why Least Privilege Must Exist Outside the Model

Nine seconds. That's how long it took an AI coding agent to delete a production database and its backups at PocketOS, a car-rental software vendor, in April 2026. By founder Jer Crane's account, an AI coding agent hit a credential mismatch during a routine staging task, searched the codebase, and found an API token in an unrelated file. That token carried blanket permissions across the Railway infrastructure API. The agent used the token. Database and backups were gone before anyone noticed. Nobody attacked PocketOS. No credentials were stolen, no prompt injection ran, no malware executed. The agent pursued a goal, hit an obstacle, and used the authority it had been handed to clear it. The problem isn't that the agent could delete a production database. The problem is that nothing in the authorization architecture prevented it from doing so — for a task that never should have touched production at all. That's the distinction this entire discipline turns on, and it's why the fix has to live in the architecture, not in a system prompt telling the agent to be careful. Permission Is Authority Over an Operation, Not Access to a System Most agent failures trace back to the same habit: authorizing at the integration level rather than at the operation level. "This agent can call the Railway API." "This agent has a Salesforce connection." Those describe access to a system, not authority over specific actions inside it — and that gap is exactly where PocketOS lost its database. A token scoped for "the Railway API" turned out to include the ability to delete production volumes, a capability nobody making the staging-fix decision actually wanted the agent to have, but nobody had explicitly excluded either. The fix is to define permission as a tuple: agent → action → resource, evaluated against live context by a policy engine at call time. A billing agent doesn't get billing:write; it gets something closer to billing.refund.issue, gated by conditions like amount <= $1,000 AND customer.region == agent.allowedRegions. That check has to happen when the agent tries to act, against an external policy engine — OpenFGA, Cedar, Open Policy Agent — not get baked into a prompt where the model is trusted to enforce its own boundaries. Layered onto that tuple is a risk hierarchy, because not all operations carry equal weight. Observe (read-only queries) can generally run autonomously, subject to row/column filtering. Retrieve/Modify (reversible writes — updating a record, sending a templated email) needs short-lived scoped tokens and before/after logging. Act (external side effects — calling a partner API, triggering a workflow) warrants just-in-time authorization and rate limits. Escalate (irreversible or sensitive — wire transfers, deleting production data, changing security config) requires explicit human approval, full stop. PocketOS had legitimate Tier 2 authority to modify staging configuration. What it actually exercised was Tier 4 authority to destroy production infrastructure, because the credential it stumbled into didn't distinguish between the two. Give Agents Identities, Not Borrowed Authority The corollary is that every agent needs its own identity — not a shared service account, not a borrowed human token. Shared credentials mean a compromised or overreaching agent inherits every privilege that account carries, and per-agent auditability disappears entirely. The mechanics for fixing this already exist in enterprise IAM; they just need to extend to non-human identities. Provision each agent as a distinct workload identity — an OIDC client or service principal — and manage its lifecycle through SCIM, so a retired workflow doesn't leave a live credential behind for someone to stumble across later. Use OAuth 2.0 client credentials or token exchange to obtain short-lived, task-scoped access tokens rather than static keys sitting in a config file. A credential that expires in minutes and is scoped to one task can't become the thing an agent finds three months later while fixing an unrelated bug — which is precisely the mechanism that sank PocketOS. Least Privilege Applies to Data as Much as to Actions Data exposure doesn't require a destructive action at all — just a retrieval system that doesn't enforce the boundaries a human user would respect. The reference case is Microsoft Copilot's "EchoLeak" vulnerability, tracked as CVE-2025-32711 and disclosed in mid-2025: a malicious email reached Copilot through normal retrieval, and the agent autonomously pulled internal Word documents, PowerPoint files, and Outlook content and transmitted them to an attacker-controlled server — zero-click, no user interaction required, evading Microsoft's own cross-prompt injection classifier in the process. Simon Willison, who coined the term "prompt injection" in 2022, named the underlying structural risk in June 2025: the "lethal trifecta." Any agent that simultaneously has access to private data, processes untrusted content, and can communicate externally is set up so that, in his words, "an attacker can easily trick it into accessing your private data and sending it to that attacker." Remove any one leg and the exploit collapses. The practical answer is filtering at the query layer — a WHERE tenant_id = :agentTenant clause, a database view that already excludes restricted columns — not a hope that the model will voluntarily ignore fields it technically received. Once sensitive data reaches an LLM's context window, downstream controls cannot reliably undo that exposure. A useful discipline is treating what an agent actually sees as the intersection of two permission sets — what the agent identity may touch, and what the human it's acting for may touch — rather than either alone, since agents routinely have broader technical reach than the specific person they're currently helping. Delegation Must Reduce Authority, Never Inherit It Multi-agent architectures introduce the same failure one layer up. When a primary agent spins up a specialized sub-agent, the instinct is to let it inherit whatever the parent was authorized to do — the shared-service-account mistake, moved up a layer of abstraction. Security architects have a decades-old name for this: the confused deputy problem. A sub-agent summarizing public documentation doesn't need write access to the ticketing system just because its orchestrator has it. There's a standard mechanism for avoiding it: OAuth2 token exchange (RFC 8693) lets a parent hand a sub-agent a narrower, short-lived on-behalf-of token — scoped to the sub-task, carrying the original human requester's identity as context rather than the parent's identity as a substitute. Every hop should independently hit the policy engine again; nobody skips the check because an upstream step already passed one. This is also the sharpest lens for reading Anthropic's November 2025 disclosure of GTG-1002, a Chinese state-linked group that manipulated Claude Code into functioning as a largely autonomous intrusion framework against roughly thirty organizations. Anthropic says the operators didn't write novel exploits — they convinced the agent it was performing a legitimate, authorized security assessment, breaking the operation into innocuous-looking steps, and let it run reconnaissance and credential harvesting with 80–90% of tactical work executed without a human in the loop. Independent researchers have challenged parts of Anthropic's framing, but the architectural lesson remains: an agent granted broad authority can turn seemingly benign subtasks into a larger chain of actions unless each delegation is independently authorized. Audit Trails Have to Answer More Than "What Did the Model Say" For every meaningful action, the record needs to capture who initiated the request, which agent identity acted, which tool or API was invoked and with what parameters, what operation was requested against what resource, which policy allowed or denied it, whether a human approved it, and a correlation ID tying the action to the broader workflow. For reversible writes, log the before/after diff, not just the fact a write occurred. And the logs need to be tamper-evident — append-only or cryptographically signed — because a log an agent (or an attacker) can edit isn't an audit trail; it's a suggestion. This is the layer that's usually missing. One research effort that reviewed 7,246 publicly reported AI incidents from September 2023 through May 2026 verified 344 as enterprise-relevant, and found that in 188 of them, an autonomous system caused harm directly in production with no attacker anywhere in the chain. Separately, a Cloud Security Alliance survey of 418 security and IT professionals, published in April 2026, found that 65% of respondents reported at least one AI-agent-related incident in the preceding twelve months. Neither number requires an adversary to make the risk alarming. The first shows that autonomous systems can cause production harm without an attacker; the second shows how widespread agent-related incidents have already become. A Worked Example Take a common agentic task: cancel a customer's order and issue the refund. Done properly, this is a sequence of independently checked steps, not one blanket-authorized action. The agent authenticates via OIDC and receives a JWT carrying its own agent ID, with the customer's request attached as context. Before touching the order, it asks the policy engine: can this agent cancel this order, given it hasn't shipped? If yes, it gets a token scoped to exactly that action. Only after cancellation succeeds does it request a second, separate authorization for the refund. Under a set threshold — say $50 — it's granted a fresh, narrowly scoped token for the payment call; above it, the same check routes to a human approval queue and the agent waits. Every step, allowed or denied, is logged with the agent ID, the policy version, and a correlation ID linking cancel-then-refund as one traceable workflow. None of this requires exotic infrastructure — it's OAuth2, a policy engine, and the discipline to check twice instead of once. Before deploying an agent, ask: Does it have its own identity, distinct from any shared account or human token?Is every tool permission scoped to a specific operation and resource, not a whole system?Is sensitive data filtered before it reaches the model, not trusted to the model afterward?Does every delegation to a sub-agent reduce authority rather than pass it along whole?Can you reconstruct every consequential action — and who authorized it — from the audit trail alone? Architecture, Not Prompt Engineering None of the incidents above happened because a model wasn't smart enough. They happened when a credential, role, or inherited permission gave the agent more authority than the task required — or when the surrounding architecture failed to constrain what that authority could reach — and the agent, reasoning correctly within the authority it had, found and used the edges of it. Least privilege for agents is an architectural boundary enforced outside the model, at the point where the agent actually touches a tool, a record, or another agent's output. Treat permissions as a system-identity problem, and you can reason about, log, and revoke them. Treat them as an instruction, and you get PocketOS: nine seconds, no attacker required. Sources "AI Agent Destroys Production Database in 9 Seconds," Zenity, April 2026 — https://zenity.io/blog/ai-agent-database-deletion-pocketos"Claude-Powered Cursor AI Agent Deletes an Entire Company Database in 9 Seconds," CX Today, April 29, 2026 — https://www.cxtoday.com/security-privacy-compliance/claude-powered-cursor-ai-agent-deletes-an-entire-company-database-in-9-seconds-is-your-customer-data-secure/Halamish, E. and Tokarev, V., "Agent-Inflicted Damage: Inside the Real-World Failures of Enterprise AI Systems," Cyera Research, May 28, 2026 — https://www.cyera.com/research/agent-inflicted-damage-inside-the-real-world-failures-of-enterprise-ai-systems"Autonomous but Not Controlled: AI Agent Incidents Now Common in Enterprises," Cloud Security Alliance / Token Security, April 2026 — https://cloudsecurityalliance.org/artifacts/autonomous-but-not-controlled-ai-agent-incidents-now-common-in-enterprisesWillison, S., "The lethal trifecta for AI agents," June 16, 2025 — https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/Anthropic, "Disrupting the first reported AI-orchestrated cyber espionage campaign," November 13, 2025 — https://www.anthropic.com/news/disrupting-AI-espionage

By Igboanugo David Ugochukwu

CORE

Enterprises Should Assume AI Agents Will Delete Their Production Base

Traditional identity mechanisms weren’t built to account for the autonomous and unpredictable nature of AI agents. So enterprises shouldn’t be surprised when they make unpredictable – or even destructive – decisions, like deleting a company’s entire production base. Without the right guardrails in place, these new actors will continue to wreak havoc across organizations. One doesn’t have to look far to find examples of this playing out in real time. PocketOS recently shared that a Cursor agent deleted its production database and backups while working on a routine task in what the company thought was a secure staging environment. The entire scenario unfolded in just 9 seconds, but took days to resolve. When the company asked the agent why it took that action, it admitted that it had “guessed instead of verifying.” This incident serves as an important cautionary tale: even seasoned developers working within the parameters of a staging environment aren’t immune to agents going rogue. Agents represent an entirely new paradigm. What makes them valuable (i.e., the autonomy to make decisions and take actions) is also what makes them risky. If a rogue agent can cause this much chaos in a sandbox, one can only imagine how harmful targeted, malicious prompt injection could be in production systems. Organizations need to strike a balance between giving agents enough freedom to provide value, but not so much that they accidentally blow up their operations. Simply prompting agents not to take harmful actions isn’t enough – you need infrastructure-level mitigations. If an action is possible, you can bet an agent will take it. Enterprises should expect agents to uncover exposed credentials or other loopholes they didn’t even know existed – it’s built into their design. Here are 5 ways every organization should be strengthening its agentic identity strategy: 1. Give Agents Their Own First-Class Identities Agents aren’t humans, machine identities, or your other typical NHIs. They’re an entirely new class of entities that warrant their own first-class identity. Agents need identities that make them traceable, observable, and bound to specific users so organizations can continuously monitor their actions, scoped permissions, and who they’re acting on behalf of. This way, if an agent makes a mistake, you can pull the plug quickly without impacting other agents and systems. 2. Reinforce Least-Privileged Access Enterprises need a robust approach to enforcing least-privileged access. Static credentials were suited to the machine identities of the past — not the highly dynamic, agentic identities running amok in code bases today. Ephemeral credentials that limit what agents can do and for how long are crucial. Agent credentials should be measured in minutes — not months. Additionally, progressive scoping ensures that agents can access only the minimum scope required to complete a task. Should they need additional permissions as the task progresses, they can ask for them, but a human must vet that request. 3. Authorize Read-Only Access for Sensitive Data Not all data should be treated equally, and agent permissions need to reflect that. Read-only access is lower risk than enabling write access, so it should be the standard for agents interacting with sensitive information. That way, if an agent veers off course, it won’t be able to alter systems or records that house sensitive data. 4. Set Up Sandbox Environments Sandbox environments are critical for letting agents learn, experiment, and fail, without impacting production systems. The PocketOS agent mishap illustrated just how crucial it is that these environments are truly secure. When configured correctly, sandboxes can help organizations identify unexpected behaviors, permission issues, or security risks before they have the chance to impact customers and/or operations. 5. Strong UX Makes for Better Human-in-the-Loop (HITL) Checkpoints HITL controls are essential for keeping agents in check. But many people experience consent fatigue after trying to decipher and approve dozens (or hundreds) of agent requests every day. UX matters here. Agent requests need to be transparent and human-readable, so users quickly know exactly what permissions they’re granting and why. Poor UX makes it harder for humans to identify sensitive actions and can lead to over-permissioning. Until enterprises modernize their approach to agentic identity, they can expect agents to occasionally take wildly unpredictable actions — like deleting their entire codebase. We’ll undoubtedly see more stories like this emerge as organizations rush to deploy agents without first understanding the new set of rules and implications surrounding their identity. It’s vital to establish these guardrails now — before your agents discover the ones you forgot to build.

By Meir Wahnon

Making Running Optional: Scaling AI Agents on Kubernetes With Agent Substrate

What if you could multiplex roughly 250 stateful agent sessions across eight Kubernetes worker Pods, then reactivate any one without losing its in-memory or filesystem state? The repository's demo reports 30x+ actor-to-worker oversubscription for that sample workload, with sub-second activation. It is a demonstration, not a production capacity guarantee. Agent Substrate is interesting not because it makes Kubernetes faster, but because it challenges a common deployment pattern around Kubernetes: coupling a workload's logical lifecycle to the compute allocated to run it. For AI agents that spend most of their time idle while retaining valuable state, separating those lifecycles could become an important building block for operating agents at significantly higher density. Kubernetes gave us an exceptionally durable abstraction for running workloads: the Pod. But emerging agent workloads expose places where that abstraction may become inefficient. They can be stateful, sandboxed, and overwhelmingly idle. A one-agent-per-Pod deployment model, while simple, couples each session's logical lifecycle to a Pod's runtime lifecycle. When sessions spend much of their time idle, that coupling can leave compute capacity allocated to workloads that are not actively executing. This is not a claim that Kubernetes is obsolete. It is an exploration of where Kubernetes remains the right substrate--provisioning capacity, managing worker Pods, and enforcing infrastructure policy--and where an agent-specific control plane may need a faster path for high-frequency lifecycle operations. The Thesis: Make Running Optional The central idea is surprisingly simple: an agent does not need to occupy compute merely because it exists. Agent Substrate calls an instance of a managed workload an actor. The deliberately broader term matters: an actor does not have to be an AI agent; it can be any OCI workload that benefits from being bursty, checkpointable, and independently suspendable. The system provides an agent-oriented workload runtime and control plane; it is not an agent framework or SDK. An actor can be suspended into a snapshot containing process memory, filesystem state, or both. The worker Pod is then freed. When another request arrives, the system restores the actor to a ready worker and routes the request to it. In this model, the actor becomes the logical workload, while the Pod becomes temporary compute capacity. That gives the architecture three defining properties: Warm capacity replaces per-session capacity. A smaller pool of ready workers serves a much larger population of actors over time.State survives worker reassignment. The next activation need not use the worker that ran the actor previously.Requests can initiate activation. The router can hold a request while the control plane brings a suspended actor back. The project's architecture document defines north-star targets including 100 ms p95 activation, one billion active and idle actors per cluster, and 1,000 wakeup events per second. These are architectural targets, not production benchmarks or guarantees. That distinction is important because the repository itself is candid that large parts of the architecture are still evolving. Architecture at a Glance The control flow becomes easier to follow once the logical workload is separated from the physical capacity: Why the Pod Becomes an Awkward Unit for Agents A conventional one-Pod-per-session deployment model can become inefficient when sessions spend most of their lifetime idle but retain valuable state. Agent-like workloads have a different shape: They wait much more than they compute.They may execute untrusted code, so multi-tenancy often means one sandbox per session.They keep useful state in memory and local filesystem changes.Their active periods can be short enough that creating and initializing a dedicated Pod for each session becomes noticeable user-facing latency. In a one-Pod-per-session design, the logical unit a user cares about — a coding session, sandboxed tool, or stateful agent — does not align cleanly with the physical unit Kubernetes schedules: a Pod. One straightforward approach is to keep each session's Pod alive. Agent Substrate asks whether that physical allocation can be temporary instead. Its answer is a pool of pre-provisioned worker Pods plus a separate actor record that tracks identity, lifecycle state, placement, and snapshots. This is an important architectural departure from Kubernetes' conventional control-plane model. Kubernetes intentionally optimizes for declarative desired state and asynchronous reconciliation. Agent Substrate moves high-churn actor state out of the Kubernetes API machinery so that wakeup, placement, and snapshot transitions can occur without making each actor a Kubernetes object. The point is not that Valkey or Redis is universally "faster than etcd." The interesting boundary is low-frequency desired state versus high-frequency runtime state: the two have different frequency and latency profiles, so the system gives them different control paths. Three Planes of State The resource model separates declarative configuration from dynamic runtime records. Operationally, snapshot contents form a useful third plane: STATE TYPEWHERE IT LIVESWHYActorTemplate, WorkerPool, and SandboxConfigKubernetes CRDsLow-frequency infrastructure configuration benefits from Kubernetes RBAC, auditability, and reconciliation.Actors, workers, assignments, lifecycle state, and snapshot referencesControl-plane store (Redis/Valkey by default; experimental PostgreSQL support is also available)These records change on lifecycle transitions and need low-latency reads and writes.Snapshot contentsNode-local storage for Pause; object storage for snapshots committed during SuspendSnapshot scopes trade off locality, durability, and transfer cost. An ActorTemplate defines an actor class: its container image, snapshot behavior, and compatible worker selection. A WorkerPool declares warm Pods. An Actor is a specific instance that moves between workers through its lifetime. Here is a trimmed ActorTemplate from the repository's multi-template demo: YAML apiVersion: ate.dev/v1alpha1 kind: ActorTemplate metadata: name: counter spec: containers: - name: counter image: ko://github.com/agent-substrate/substrate/demos/counter workerSelector: matchLabels: workload: multi-template-shared The important omission is a dedicated Pod. The template describes the workload and selects compatible reusable capacity; a separate WorkerPool provides the warm Pods, while the actor's identity and lifecycle remain independent of whichever worker hosts it. The deeper architectural pattern is a separation of three related lifecycles: infrastructure, workload, and execution. Kubernetes manages infrastructure capacity; the actor control plane manages logical workload identity, placement, and lifecycle; snapshot and sandbox machinery preserve and reconstitute execution state. Once those lifecycles are separated, a worker Pod becomes a reusable execution slot rather than the identity of the workload itself. An actor is addressed by (atespace, name), not by name alone. That is more than a naming detail: the glossary defines an atespace as a logical actor isolation boundary, not a replacement for Kubernetes namespaces or a sandbox security boundary. The same actor name can exist in different atespaces. The atespace also appears in the actor's routable DNS name: Plain Text <actor-name>.<atespace>.actors.resources.substrate.ate.dev This is the first place the project begins to look less like a set of Kubernetes objects and more like a runtime: stable logical identity remains while the physical worker assignment changes. The Request Path: Routing Becomes Placement The most consequential design choice is that ingress is part of activation. The networking architecture provides the actor DNS model and an Envoy-based router. The router's ext_proc handler reads the actor reference from the request authority, calls the control plane to ensure that actor is running, and then selects the assigned worker as the upstream. Plain Text Error: Parse error on line 22: ...atunnel Ateom->>Actor: Forward over ----------------------^ Expecting '+', '-', '()', 'ACTOR', got 'participant_actor' The ingress detail matters. The router does not forward directly to the actor's application endpoint. It opens an mTLS connection to the worker's atunnel listener. atunnel validates the router and forwards only to the actor currently assigned to that worker. That makes routing part of the security boundary, not merely service discovery. There is also a practical admission-control insight here. A saturated worker pool should not turn a burst of requests into an unbounded queue. The router can park a bounded number of requests while it retries transient capacity and control-plane conditions; once the parking limit is reached, it sheds new work. Activation latency is therefore not just a restore-time problem. It is also a backpressure problem. What Happens During Suspend and Resume The control plane coordinates a distributed workflow rather than pretending this is a single atomic operation. For a resume, it locks the actor, reads its state and template, selects an eligible idle worker, asks the node-level supervisor to restore a snapshot or cold boot, and marks the actor running only after the worker is ready. For a suspend, it checkpoints state, persists the requested snapshot scope, clears the worker assignment, and returns the worker to the pool. The details reveal deliberate distributed-systems trade-offs. In the default Redis backend, actor and worker records are separate keys that may occupy different cluster slots, so they cannot be updated in one cross-slot action. The implementation uses per-record version checks, actor locking, ordering, retries, and idempotent workflow steps to coordinate these transitions. The architectural implication is that lifecycle operations are treated as recoverable workflows rather than atomic infrastructure mutations. A repeated lifecycle call can discover completed steps and move forward instead of blindly redoing them. The Worker Is a Reusable Sandbox, Not the Actor Below the control plane, atelet runs as a DaemonSet and manages the node-side work: image preparation, OCI bundle assembly, snapshot transfer, and communication with the worker. ateom runs inside the worker Pod and drives the sandbox runtime. The repository currently defines gVisor and microVM sandbox classes. In the gVisor path, ateom drives runsc checkpoint and restore. Depending on the configured scope, snapshots can preserve process and filesystem state, allowing an actor to resume later on another worker. This is why the demo can show an in-memory counter continuing after a suspend/resume cycle: the application was restored, rather than restarted from scratch. The security model should be described with care. Sandboxing and mTLS are real implementation elements, and the project has a detailed threat model. But that document explicitly says security hardening remains early. A fair reading is that Agent Substrate is making the right boundaries visible--sandbox, worker reuse, actor identity, snapshot access, and router-to-worker authentication--rather than claiming those boundaries are already production complete. What the Demos Prove--and What They Do Not The most accessible proof is the counter demo. A tiny HTTP service increments an in-memory counter. Create an actor in an atespace, send requests through atenet-router, suspend it, and resume it. The counter continues. The demo makes the abstract claim concrete: memory and filesystem state can outlive a worker assignment. The README's published density demonstration goes further: about 250 stateful actors multiplexed across eight physical worker Pods. The repository also includes examples for Claude Code multiplexing, request parking, autoscaled worker pools, and different templates sharing a worker pool. These examples validate the model and its developer experience. They do not prove the project's one-billion-actor target, production reliability, or a universal cost model. Treating that distinction honestly makes the architecture more interesting, not less: the open questions are precisely where the difficult engineering begins. From Traffic Locality to Compute Locality My previous DZone article, "Zone-Aware Routing in Kubernetes", examined a related infrastructure question: how should a platform place traffic so requests stay local when that improves latency, resilience, or cost? That work led me to a broader question: if locality matters for packets, what happens when locality also matters for stateful compute? Zone-aware routing asks where traffic should go. Agent Substrate raises a harder question: where should the compute state itself live when a workload can disappear from one worker and reappear on another? This turns locality from a networking concern into a workload-lifecycle concern. That change has consequences: Scheduling cannot be evaluated only by where free CPU exists; snapshot location and resume cost matter too.Routing cannot be evaluated only by endpoint availability; it can trigger a state transition.Security cannot stop at the Pod boundary; worker reuse and snapshot access become first-class concerns.Autoscaling cannot only count replica demand; it must account for how long actors remain active, parked, or suspended.Storage cannot be treated as an afterthought; snapshot placement, transfer time, durability, and locality become part of the activation path. This suggests a broader infrastructure question for agent workloads. The challenge is not simply running more containers; it is hosting large populations of mostly-idle, stateful, potentially untrusted processes without allocating dedicated compute to each one. Where the Hard Work Remains The project is unusually direct about its unfinished work: control-plane performance and reliability, worker autoscaling, identity and policy, actor network isolation, storage design, observability, and support for different sandbox runtimes all remain active areas of development. Those concerns are not peripheral; they determine whether the architecture can operate reliably at the scale it targets. A system that makes activation fast must still decide how to shard state, restore safely, apply policy before execution, isolate one actor from the state left by another, and reason about locality without turning every wakeup into a storage bottleneck. Four questions are especially important: Snapshot locality. If an actor's state is remote, resume latency becomes partly a storage and network-transfer problem.Snapshot correctness. Checkpointing a live process is not equivalent to serializing application state. Open connections, timers, external leases, credentials, and dependencies can make a restored process semantically different from a freshly initialized one.Activation bursts. Multiplexing improves average utilization, but a correlated wake-up event can turn many inexpensive idle actors into a sudden demand spike. The system therefore needs admission control and worker autoscaling that respond to activation pressure, not only steady-state utilization.Fairness. A small number of highly active actors can monopolize workers unless scheduling and admission control account for competing demand. Agent Substrate is therefore more compelling as an emerging architectural pattern than as a product claim. Its value is in making these trade-offs explicit and providing a runnable implementation that exposes where the abstractions are strong and where they remain unfinished. Conclusion The Pod is unlikely to disappear. But it may stop being the only unit we think about when we build infrastructure for agents. Kubernetes remains a powerful system for provisioning and operating compute. Agent Substrate is exploring what happens when the logical lifecycle of an agent is separated from the lifecycle of the Pod that temporarily runs it. If agents become ubiquitous--long-lived, intermittently active, stateful, and capable of executing untrusted code--the infrastructure challenge will not simply be running more Pods. It will be deciding where an agent should exist when it is inactive, how quickly it can become active, and how efficiently thousands or millions of them can share the same underlying compute. That is the problem Agent Substrate is attempting to solve. At a billion actors, the central question is no longer how to run more agents. It is how to make running optional. Further Reading Agent Substrate repositoryArchitectureThreat modelRequest parkingCounter demo Agent Substrate is Apache-2.0 licensed, explicitly not an officially supported Google product, and in active early development. The architectural analysis and opinions in this article are my own.

By Mayowa Fajobi

Extracting Entities and Relationships From Engineering Documents With spaCy

Engineering teams generate a lot of useful knowledge, but most of it is locked inside text. A service ownership note may tell you who owns an application programming interface (API), while a runbook may tell you which database a service relies on. An incident review may detail how one fault impacted the other systems. Each of these is individually useful. It’s when we are able to link together all of these facts that we get our greatest value. As an example, think about a developer who wants to know: Plain Text Which team owns the API that Checkout Service depends on? To answer this question, the developer would need to make several connections: Plain Text Checkout Service depends on Payment API. Platform Team owns Payment API. With a keyword search, you could locate documents that include “Checkout Service,” “Payment API.” With a retrieval-augmented generation(RAG) pipeline, you could locate relevant chunks and forward them to your LLM. If there isn’t some understanding of the relationships between services, APIs, databases, etc., then your RAG may still fail to identify a connection. This is where entity and relationship extraction can help. We want to move beyond viewing engineering documents as plain text; instead, we’d like to see the text as a source of structured facts, such as: Plain Text Checkout Service --DEPENDS_ON--> Payment API Platform Team --OWNS--> Payment API Payment API --STORES_IN--> PostgreSQL These facts can then provide the basis for knowledge graphs, dependency analysis, impact analysis, and GraphRAG pipelines. Here we will define a simple Python pipeline using spaCy to extract specific types of engineering entities and relationship triplets from text. Our objective is to produce a working prototype, rather than creating the ultimate extraction tool. We also hope that this provides a practical starting point for developers to test, view, and modify their own documentation. Why This Pattern Matters in Real Systems This pattern applies to multiple aspects of real-world engineering systems. Service Ownership Lookup A developer can query a service owner by looking at a service they are unaware of (as opposed to manual searches of a service catalog). Analyzing Dependencies Using a graph to find all other services impacted when a system (API, Queue, Database) goes down. Response During Incidents Relationship extraction can transform incident notes and runbooks into navigationally easier-to-use dependency maps during outages. GraphRAG Pipelines Entity/relationship extraction is required for GraphRAGs prior to being able to get the appropriate graph contextual information. Poor entity/relationship extraction will result in poor graph contextual information regardless of how good the LLM is. Discovering Architecture Many large organizations have their architectural knowledge dispersed across various documents, diagrams, and repositories. Entity/relationship extraction provides a method to make this knowledge searchable and reusable. We'll use a simple dataset in this tutorial, but you could easily expand on it using your organization's service catalog(s), repository metadata, cloud inventory, or documentation from real engineering systems. What We Are Building We will create a small Python script that reads engineering notes and produces two files: Plain Text entities.json triples.json The first file contains detected entities: JSON [ { "text": "Checkout Service", "label": "SERVICE" }, { "text": "Payment API", "label": "API" } ] The second file contains relationships: JSON [ { "subject": "Checkout Service", "relation": "DEPENDS_ON", "object": "Payment API", "source_text": "Checkout Service depends on Payment API during payment authorization." } ] This output can later be loaded into NetworkX, Neo4j, a vector database, or a GraphRAG pipeline. Project Setup Create a new folder: Shell mkdir spacy-entity-relationship-extraction cd spacy-entity-relationship-extraction Install a virtual environment, spaCy, and the small English model: Shell python3.12 -m venv .venv source .venv/bin/activate pip install --upgrade pip pip install spacy # installs spacy python -m spacy download en_core_web_sm # installs english model The small English model is enough for this tutorial. For production work, you may need a larger model, domain-specific rules, or custom training data. Step 1: Define Sample Engineering Documents Create a file named extract_entities.py. Plain Text documents = [ "Checkout Service depends on Payment API during payment authorization.", "Payment API stores transaction metadata in PostgreSQL.", "Platform Team owns Payment API and manages its deployment pipeline.", "Recommendation Service calls Catalog API to retrieve product details.", "Catalog API indexes product data in Elasticsearch.", "Search Team owns Catalog API.", ] Examples like these are simple; however, they illustrate typical engineering documentation patterns: one System Depends Upon Another System,a service stores information in a database,a team is responsible for a service,one application calls an application programming interface (API). Step 2: Add Custom Entity Rules Most of the time, general-purpose named entity recognition (NER) models are trained to identify persons, organizations, locations, and other types of information such as dates. The engineering documentation our NLP tools will process contains a range of domain-specific entities, including services, APIs, databases, queues, teams, repositories, and cloud-based services. In order for these entities to be identified in the pipeline, it would make sense to incorporate domain-specific rules into our code. Python import spacy def create_pipeline(): nlp = spacy.load("en_core_web_sm") # Load English Model ruler = nlp.add_pipe("entity_ruler", before="ner") # Create Ruler patterns = [ {"label": "SERVICE", "pattern": "Checkout Service"}, {"label": "SERVICE", "pattern": "Recommendation Service"}, {"label": "API", "pattern": "Payment API"}, {"label": "API", "pattern": "Catalog API"}, {"label": "DATABASE", "pattern": "PostgreSQL"}, {"label": "SEARCH_INDEX", "pattern": "Elasticsearch"}, {"label": "TEAM", "pattern": "Platform Team"}, {"label": "TEAM", "pattern": "Search Team"}, ] ruler.add_patterns(patterns) # Add Patterns to Ruler return nlp EntityRuler enables you to include common domain-specific named entities from your company’s service catalog, team listing, repository listing, and database inventory, etc., without training a custom model. Step 3: Extract Entities Now add a function to extract entities from each document. Python def extract_entities(nlp, documents): results = [] for text in documents: doc = nlp(text) for entity in doc.ents: results.append( { "text": entity.text, "label": entity.label_, "source_text": text, } ) return results Run it: Python if __name__ == "__main__": nlp = create_pipeline() entities = extract_entities(nlp, documents) for entity in entities: print(entity) Expected output: Plain Text {'text': 'Checkout Service', 'label': 'SERVICE', 'source_text': 'Checkout Service depends on Payment API during payment authorization.'} {'text': 'Payment API', 'label': 'API', 'source_text': 'Checkout Service depends on Payment API during payment authorization.'} {'text': 'Payment API', 'label': 'API', 'source_text': 'Payment API stores transaction metadata in PostgreSQL.'} {'text': 'PostgreSQL', 'label': 'DATABASE', 'source_text': 'Payment API stores transaction metadata in PostgreSQL.'} The above output is much better than plain text. The output indicates which tokens within the document refer to engineering entities. Step 4: Normalize Duplicate Entities Because entities are almost always referred to by different names within the same document (eg., payment-api, Payment-API, payment service, etc.), you need an approach that uses aliases to link all of these references back into a single “concept” or node. The first approach to doing this would be creating an alias map. This could look something like this: Plain Text ALIASES = { "payment-api": "Payment API", "Payments API": "Payment API", "payment service": "Payment API", } The second part of this solution involves defining an additional helper function that will take each found entity reference and “normalize” it by replacing it with its corresponding concept label. The normalized value is then stored along with the original extracted data in order to be returned as output. Here's how we can do that: Python def normalize_entity_name(name): return ALIASES.get(name, name) Then update the entity extraction function: Python def extract_entities(nlp, documents): results = [] for text in documents: doc = nlp(text) for entity in doc.ents: results.append( { "text": normalize_entity_name(entity.text), "label": entity.label_, "source_text": text, } ) return results As previously stated, while this process appears to be relatively minor, normalizing entities is probably one of the most critical steps in developing a knowledge graph because if done poorly, what may appear to be a single system in the world may end up being represented as many separate nodes. Step 5: Extract Relationship Triples Now that we have identified all of our entities, we want to find the relationships. We will begin with the simplest way to do this using verb phrases. The simplicity of this method does not allow for much flexibility or learning from the data, as a trained relation extraction model would, but it has the advantage of being completely transparent and very easy to debug. Plain Text RELATION_VERBS = { "depends on": "DEPENDS_ON", "stores": "STORES_IN", "owns": "OWNS", "calls": "CALLS", "indexes": "INDEXES_IN", } Create an additional helper function that identifies entity locations within the text. Python def extract_relationships(nlp, documents): triples = [] for text in documents: doc = nlp(text) entities = list(doc.ents) if len(entities) < 2: continue for verb_phrase, relation in RELATION_VERBS.items(): if verb_phrase in text.lower(): subject = normalize_entity_name(entities[0].text) object_ = normalize_entity_name(entities[1].text) triples.append( { "subject": subject, "relation": relation, "object": object_, "source_text": text, } ) return triples The above code has no universal way to determine which entity is the subject or object; however, when you know your input data is structured like controlled engineering documentation, this method provides a good working example for a baseline. Run it: Python if __name__ == "__main__": nlp = create_pipeline() triples = extract_relationships(nlp, documents) for triple in triples: print(triple) Expected output: Plain Text {'subject': 'Checkout Service', 'relation': 'DEPENDS_ON', 'object': 'Payment API', 'source_text': 'Checkout Service depends on Payment API during payment authorization.'} {'subject': 'Payment API', 'relation': 'STORES_IN', 'object': 'PostgreSQL', 'source_text': 'Payment API stores transaction metadata in PostgreSQL.'} {'subject': 'Platform Team', 'relation': 'OWNS', 'object': 'Payment API', 'source_text': 'Platform Team owns Payment API and manages its deployment pipeline.'} For many internal engineering knowledge bases, this kind of controlled extraction is a good first step before adding more complex models. Step 6: Export JSON The final step is to save entities and triples. Python import json def write_json(path, data): with open(path, "w", encoding="utf-8") as file: json.dump(data, file, indent=2) Use it: Python if __name__ == "__main__": nlp = create_pipeline() entities = extract_entities(nlp, documents) triples = extract_relationships(nlp, documents) write_json("entities.json", entities) write_json("triples.json", triples) print(f"Wrote {len(entities)} entities to entities.json") print(f"Wrote {len(triples)} triples to triples.json") Complete Example Here is the full script: Python import json import spacy documents = [ "Checkout Service depends on Payment API during payment authorization.", "Payment API stores transaction metadata in PostgreSQL.", "Platform Team owns Payment API and manages its deployment pipeline.", "Recommendation Service calls Catalog API to retrieve product details.", "Catalog API indexes product data in Elasticsearch.", "Search Team owns Catalog API.", ] ALIASES = { "payment-api": "Payment API", "Payments API": "Payment API", "payment service": "Payment API", } RELATION_VERBS = { "depends on": "DEPENDS_ON", "stores": "STORES_IN", "owns": "OWNS", "calls": "CALLS", "indexes": "INDEXES_IN", } def create_pipeline(): nlp = spacy.load("en_core_web_sm") ruler = nlp.add_pipe("entity_ruler", before="ner") patterns = [ {"label": "SERVICE", "pattern": "Checkout Service"}, {"label": "SERVICE", "pattern": "Recommendation Service"}, {"label": "API", "pattern": "Payment API"}, {"label": "API", "pattern": "Catalog API"}, {"label": "DATABASE", "pattern": "PostgreSQL"}, {"label": "SEARCH_INDEX", "pattern": "Elasticsearch"}, {"label": "TEAM", "pattern": "Platform Team"}, {"label": "TEAM", "pattern": "Search Team"}, ] ruler.add_patterns(patterns) return nlp def normalize_entity_name(name): return ALIASES.get(name, name) def extract_entities(nlp, documents): results = [] for text in documents: doc = nlp(text) for entity in doc.ents: results.append( { "text": normalize_entity_name(entity.text), "label": entity.label_, "source_text": text, } ) return results def extract_relationships(nlp, documents): triples = [] for text in documents: doc = nlp(text) entities = list(doc.ents) if len(entities) < 2: continue for verb_phrase, relation in RELATION_VERBS.items(): if verb_phrase in text.lower(): subject = normalize_entity_name(entities[0].text) object_ = normalize_entity_name(entities[1].text) triples.append( { "subject": subject, "relation": relation, "object": object_, "source_text": text, } ) return triples def write_json(path, data): with open(path, "w", encoding="utf-8") as file: json.dump(data, file, indent=2) if __name__ == "__main__": nlp = create_pipeline() entities = extract_entities(nlp, documents) triples = extract_relationships(nlp, documents) write_json("entities.json", entities) write_json("triples.json", triples) print(f"Wrote {len(entities)} entities to entities.json") print(f"Wrote {len(triples)} triples to triples.json") Run it: Shell python extract_entities.py You should now see two output files: Plain Text entities.json triples.json Why This Matters to GraphRAG Systems GraphRAG systems rely on accurate entity and relationship information in order to perform well. Poor extraction means poor graph retrieval. The Large Language Model (LLM), while capable of reasoning about the context provided, cannot correct the underlying data problems. Therefore, entity extraction, entity normalization, relationship extraction, and source tracking are to be considered first-class engineering issues. Production Considerations While this tutorial uses a small set of rules to extract entities and relationships, larger-scale production systems require additional robustness through "guardrails". Use a service catalog when possible. If your organization has an existing internal listing of services, teams, repositories, and owners, then these items should be used as the sources of truth for extracting patterned entities. Track confidence. Do not treat all extracted relationships with equal weight. Inferences from unstructured documentation, such as a free-form document, likely have less confidence than those from structured documentation, such as a service catalog. Keep the source sentence. Each triple includes the original sentence text. This makes reviewing, debugging, and referencing answers much simpler. Review low-confidence edges. Graph errors grow rapidly. An error in one dependency can cause downstream traversal results to be misleading. Start simple before using an LLM. While LLMs can be helpful for extracting entities/relationships from unstructured data, they increase cost, latency, and variability. Rules + Domain Dictionary may be sufficient for predictable documentation like engineering documentation. Key Takeaways SpaCy is effective in both general NLP and domain-specific extraction of entities via custom entity rules. Domain documents like engineering documentation will have custom entity names that include SERVICE, API, DATABASE, TEAM, etc. The relationship triples represent an efficient path from the unstructured document to the knowledge graph. Entity Normalization is NOT optional. If you do not perform Entity Normalization on your extracted entities, then the single "Real-world" system will be represented by multiple isolated/disconnected nodes in your graph. In terms of performance with GraphRAG systems, quality of extraction is equal in importance as retrieval and prompting. Try It Yourself Add the following documents: Plain Text Billing Worker publishes events to Kafka. Data Platform Team owns Kafka. Then add the following to the entity patterns: Plain Text Kafka Billing Worker Data Platform Team Finally, add a new relationship type: Plain Text PUBLISHES_TO Your goal is to produce the following triple: Plain Text Billing Worker --PUBLISHES_TO--> Kafka This small exercise demonstrates the work that is needed to adapt entity and relationship extraction to your own engineering domain.

By Sriharsha Makineni

Beyond Agent-Washing: The Engineering Principles Behind Production-Ready AI Agents

An AI agent is not defined by how intelligently it talks. It's defined by what it's trusted to do. Give a language model a chat window, and you have an interface. Give it access to production APIs, identity, business logic, memory, and the authority to execute actions on your behalf, and you have something categorically different: a new kind of software actor, one that can read your data, write to your systems, and make decisions faster than any human reviewer can watch in real time. That distinction is where most of the industry's "agent" marketing quietly falls apart. And in 2026, the gap between marketing and engineering has become measurable. AvePoint's third annual State of AI report, based on 750 global IT leaders across financial services, healthcare, and government, found that 88.4% of organizations experienced at least one AI agent–related security breach in the past twelve months, with data leakage and manipulation by untrusted input as the two leading causes (AvePoint, State of AI 2026). Separately, McKinsey's 2026 AI Trust Maturity Survey — fielded across roughly 500 organizations between December 2025 and January 2026 — found that only about 30% of organizations have reached a mature level of governance and agentic controls (McKinsey, State of AI Trust 2026). Adoption has outrun governance by a wide margin, and the bill is starting to come due. I spent several hours this year with Ted Kornish, CTO of Gravity, a platform built for corporate sustainability and emissions reporting, to press on a narrower question than "does your product have an agent?" The question was: what has to be true, architecturally, before a system earns the word at all? Kornish's answers are the evidence in this piece, not the subject. Gravity is a useful case study — a production system operating in a regulated domain where a wrong answer can end up in front of a regulator — but the principles below are my own synthesis of what separates a genuine enterprise agent from a chatbot wearing a lanyard. The Five-Question Agent Test Before accepting any vendor's "AI agent" claim — including Gravity's — I now run it through five questions: Can it execute actions, not just generate recommendations?Can it compose multiple capabilities to accomplish a goal it wasn't explicitly scripted for?Does it operate through the same business logic and APIs your human users already use?Can every action it takes be attributed to a specific identity and audited after the fact?Can a potentially destructive action be previewed, validated, and approved before it commits? If the answer to the first three is no, you're looking at an AI feature — useful, maybe, but bounded and predictable. If the answer to four and five is no, you're looking at an AI feature that probably shouldn't have production access yet, regardless of how capable its model is. Kornish's version of this test, from our conversation, is the cleanest I've heard a vendor articulate unprompted: "If you can list every task it handles, it is a feature. If you can't because it can compose a wide variety of platform capabilities, it's an agent." Ask a vendor to enumerate every task their agent handles on a sales call. Most stall out after three or four items, and the stall itself is diagnostic — a genuine agent's capability surface resists a clean list because it's compositional, not because nobody documented it. Why API-First Architecture Is the Real Moat Here's the engineering claim underneath the marketing claim, and it's the one I'd stake the rest of this article on. Most enterprise platforms were built UI-first, with an API bolted on afterward as a partial mirror of what the interface already does. That ordering was fine for a decade of humans clicking buttons through a screen. It becomes a structural liability the instant an agent needs to act with the same range a human employee has — because if the API was never built to be complete, an engineering team is stuck choosing between two bad options: let the agent do less than a human could, or build a second, parallel execution path just to give it a fighting chance. Neither ages well. The first caps the agent's usefulness permanently. The second means maintaining two versions of every business rule indefinitely, with near-certainty that they drift apart in some edge case nobody thought to test. Gravity sidestepped that fork by never building a second path: "We never built a separate 'agent version' of Gravity with its own private set of functions. On the read path, the agent calls the exact same HTTP API our product UI calls... On the write path, there's a service layer that provides one execution path for any given action, one place validation and business rules live, and one codebase to test and maintain instead of two." The architectural implication here is bigger than agent convenience. Once an agent becomes just another API client, the API itself becomes part of the agent's safety boundary. Authentication, authorization, idempotency, validation, rate limiting, transaction handling, and auditability stop being secondary infrastructure and become prerequisites for autonomy. That reframes the first question an engineering team should ask when evaluating agent readiness. It isn't "which model should we use." It's closer to: can an untrusted software actor safely exercise the application capabilities we already have? Kornish's line on this is worth sitting with: "An agent is really just a new kind of caller on an API that was already capable; if that API doesn't exist, there's nothing for the agent to stand on." That's not a race to fine-tune the flashiest model. It's a race to have quietly built a serious, typed, permissioned API years before anyone had a reason to think an LLM would ever call it — and most incumbent platforms already lost that race without knowing it, because the decision was made a decade ago by a team with no reason to think about agents at all. The hardest part, in his account, isn't reads. Reads are comparatively low-stakes; nothing breaks if a summary is slightly stale. Writes are a different category of problem entirely, because most enterprise write paths aren't atomic — you're editing individual records one at a time, and the only thing worse than writing the wrong data is writing partial data, which leaves the system in a state nobody designed for and nobody can cleanly roll back. That's the real, unglamorous reason so many "enterprise AI agents" on the market today are read-only interfaces wearing an agent's branding: atomic writes are hard, and most legacy write paths were never built for it. What This Looks Like In Production Abstract architecture arguments are easy to nod along to and hard to actually picture. Here's a concrete workflow, reconstructed from how Gravity describes its own execution path for a regulatory reporting task: Plain Text User: "Prepare this quarter's emissions report and flag anything that needs my attention." ↓ Agent — parses the objective, checks its ~85 versioned skills for the relevant reporting operations (progressive disclosure: hand the model what's relevant to this step, not everything) ↓ APIs — retrieves source data through the same HTTP API the product UI uses, governed by the same permissions ↓ Deterministic Engine — runs the actual emissions math; the model never touches the calculation itself ↓ Validation — checks for anomalies, missing fields, and inconsistencies against prior filings ↓ Agent — drafts the report narrative, cites every source and every calculation it pulled from ↓ Human — reviews the staged preview, approves or rejects ↓ Production — the report commits; the full trace is retained The model's job in that chain isn't the arithmetic — it's everything around the arithmetic. Sourcing the right document, filling a gap, matching the tone of last year's filing so the report reads as one continuous document rather than two authors stitched together. The ground truth stays boring and deterministic. The model's intelligence gets spent on the parts that used to consume a compliance team's entire week. Agent vs. Copilot vs. Automation The three categories get flattened together constantly, and the flattening is exactly what lets the word "agent" get retrofitted onto anything with a chat box. CapabilityCopilotAutomationAI AgentGenerates contentYesSometimesYesFollows a predefined workflowYesYesYesHandles goals it wasn't explicitly scripted forLimitedNoYesComposes tools/capabilities dynamicallyLimitedNoYesExecutes production actionsLimitedYesYesRequires human approval on risky actionsUsuallySometimesShould, by designMaintains task state across sessionsLimitedYesYesAdapts execution mid-taskLimitedNoYes These categories overlap in practice more than the table suggests, and that's worth saying plainly: the meaningful distinction isn't the marketing label a vendor chose; it's the system's actual degree of autonomy, compositionality, and execution authority. The Five Layers, and Why They Aren't Interchangeable Looking at Gravity's stack alongside broader enterprise agent design patterns, a consistent structure emerges — five layers, each dependent on the one below it, each a distinct point of failure if it's missing or built poorly. Plain Text Layer 5 — Governance Identity · audit · human approval ↑ Layer 4 — Execution Layer The API business logic actually runs through ↑ Layer 3 — Domain Skills Versioned, tested, maintained like production content ↑ Layer 2 — Planning & Reasoning Goal decomposition, task checklists ↑ Layer 1 — Foundation Model Replaceable, increasingly commoditized My interpretation of this stack is that the layers are not equally substitutable. The foundation model is replaceable — most serious vendors have access to roughly the same handful of frontier models, and Layer 1 is where competitive advantage is evaporating fastest. The planning harness can evolve; skills can be versioned and rolled back like any other content. But the execution and governance layers are much harder to swap out, because they encode an organization's actual operating rules — its permission model, its validation logic, its regulatory obligations. That suggests the durable competitive advantage in enterprise agents sits lower in the stack than most of the AI discourse right now assumes. Everyone is fighting over Layer 1. Most of the actual failures live in Layer 4 and Layer 5 — the parts that don't show up in a demo. Kornish's own framing on where the industry is spending its engineering effort was more candid than I expected from a CTO on the record: "we're very bitter-lesson-pilled over here: increasingly, the agent is just a model and a thin harness... the system prompt is getting shorter over time as the models internalize more capabilities." The elaborate custom orchestration graphs that dominated agent architecture discussions for the last two years are being displaced — thinner harnesses, base models absorbing more of that responsibility natively, and open protocols like Model Context Protocol doing at the ecosystem level what bespoke orchestration used to do inside one vendor's codebase. My Technical Take: The Agent Is Only As Strong As Its Execution Boundary Here's the architectural lesson I actually take from this, stated plainly and separately from anything Kornish said: an enterprise agent shouldn't be designed as an intelligent application sitting on top of an existing platform. It should be designed as an execution client operating inside a security and transaction boundary that already exists independently of it. That distinction sounds pedantic until you trace what happens when it's missing. If a model can reason but can't safely execute, it's a copilot — useful, bounded, not what we're talking about in this piece. If it can execute but bypasses the platform's own authorization and validation logic to do so, it's not an agent, it's a security liability wearing an agent's branding. If it can execute safely but nothing about that execution is observable after the fact, it's operationally untrustworthy regardless of how well it performed in the room. And if it clears all three of those bars but is never re-evaluated as the model, the skills, and the surrounding data shift, its reliability will decay quietly, without anyone noticing until an incident forces the question. That chain leads to a fairly simple principle, and it's the one I'd want any engineering team to write on a whiteboard before they start: the model should decide what to attempt. The platform should decide what is allowed to happen. Collapse that distinction — let the model's judgment double as the authorization check — and you don't have an agent. You have a very articulate way of bypassing your own access controls. I'd formalize this as the agent execution boundary — the layer, architecturally distinct from the model itself, where a proposed action gets checked against identity, authorization, validation, deterministic logic, and (ideally) a dry run, before it's ever allowed near production data: Plain Text USER INTENT │ ▼ ┌─────────────────────┐ │ AI MODEL │ │ Reason/Plan │ └──────────┬──────────┘ │ ▼ ┌─────────────────────┐ │ AGENT SKILLS │ └──────────┬──────────┘ │ ▼ ┌─────────────────────────┐ │ EXECUTION BOUNDARY │ │ │ │ Identity │ │ Authorization │ │ Validation │ │ Deterministic Logic │ │ Dry Run │ │ Audit │ └────────────┬────────────┘ │ Human Approval │ ▼ PRODUCTION DATA The model should never be the final authority on whether an action is valid. It proposes; the execution boundary determines whether the proposal is permitted, reversible, and safe to commit. Everything Gravity described to me — the shared API surface, the dry-run engine, the staged preview, the permission-inherited approval gate — is one implementation of that boundary. It isn't the only valid implementation. But any architecture that skips having one, in favor of trusting the model's own judgment about what's safe, is building on a foundation that will eventually fail in a way nobody can cleanly attribute afterward. Deterministic vs. Probabilistic Is the Real Design Boundary The design decision that matters most isn't "AI versus not AI." It's probabilistic versus deterministic execution, and a serious architecture puts uncertainty exactly where judgment is valuable and removes it everywhere correctness is mandatory. Kornish didn't hedge when I asked him where Gravity draws that line: "Emissions math always runs through our deterministic calculation engine, no exceptions, because a regulator does not accept 'the model recalculated it slightly differently this time' as an answer." His broader rule generalizes well past sustainability reporting into any regulated domain — healthcare, finance, insurance: where correctness counts, use a calculator; where you need judgment, use a person; where approximate correctness is fine, AI saves time and money. The model can decide which document to inspect, which skill to invoke, and how to explain an anomaly in plain language. It should not silently become the source of truth for a number that could have been deterministically reproduced. Any output that could become evidence in an audit, a regulatory filing, or a legal dispute should trace back to a deterministic calculation — never a model's best guess, no matter how good that guess usually is. "Usually" is not a standard a regulator will accept, and it shouldn't be a standard engineering teams accept for themselves either. What Gravity Gets Right — And Where I'd Push Back To be specific about what Gravity's architecture actually demonstrates, rather than just asserting it's good: API-first execution with no parallel agent path, a deterministic calculation engine that the model cannot override, permission inheritance instead of a separate AI policy layer, staged writes with mandatory human approval, persistent task state across sessions, a visible reasoning trace and tool-invocation history, and continuous evaluation fed by production telemetry rather than a one-time launch gate. Gravity isn't interesting because it has an AI agent. What's interesting is the infrastructure surrounding the agent — the model is one component of the system; the API, the identity model, the deterministic engine, and the governance layer are what determine whether that model can be trusted to act in the real world at all. That said, don't mistake this for an endorsement without edges. A few things I'd want to see stress-tested before calling any system like this "solved," Gravity included: how the skill-versioning process holds up under a genuinely adversarial red-team exercise, not just replay against known scenarios; whether the "agent inherits operator permissions" model actually closes the gap on authorized-but-unintended actions (more on that below); and how the observability stack performs under a long-running, multi-day task where context has had real time to drift. None of these are unique to Gravity. They're unresolved for the industry broadly, which is exactly why they're worth naming instead of glossing over. Identity Is the Security Boundary — But It Isn't the Whole Boundary This is where I'd ask security-minded readers to slow down, because the industry's threat model for agents is still catching up to what's actually being deployed in production right now. Kornish's framing — treat the agent as a new employee with its own login, not a master key that opens every door in the building — maps directly onto zero trust: never assume implicit trust from network location or system role, verify explicitly on every call, grant only the minimum privilege the task requires. "Every action carries the operating user's identity through the entire call chain," he told me, "so when the agent hits our API, it's authorized exactly the way that user would be, not under some elevated service account." Permissions get recalculated on every request rather than cached at login — cached permissions are exactly the kind of stale-state bug that turns into an incident report months after the access was actually revoked. From a security engineering perspective, this changes the traditional question of "what can the agent access" into a more useful one: under whose authority is this specific action being performed? That distinction matters because it should follow the action through the entire call chain, not just the login event. But identity and least privilege, however well implemented, are necessary and not sufficient — and 2026 has produced hard evidence of exactly where that gap sits. In June 2025, researchers at Aim Security disclosed a zero-click prompt injection against Microsoft 365 Copilot, later assigned CVE-2025-32711 with a CVSS score of 9.3 and nicknamed EchoLeak. A single crafted email, no user interaction required, and Copilot followed hidden instructions embedded in the email to pull data out of OneDrive, SharePoint, and Teams (Beam AI, 5 Real AI Agent Security Breaches in 2026). The exploit didn't need a broken permission model. Copilot was, in a narrow technical sense, doing exactly what it was authorized to do — retrieve and summarize data the user could already access. It was the intent behind the action that was compromised, not the authorization. That's the distinction the OWASP GenAI Security Project formalized when it published the Top 10 for Agentic Applications in December 2025, ranking Agent Goal Hijack as ASI01 — an attacker redirecting an agent's objective through content it reads, rather than code it runs, so the agent pursues the attacker's goal while believing it still serves the user's (Cycode, OWASP Top 10 for Agentic Applications 2026). There's an important line between authorization failure and intent failure. Traditional security controls are built to prevent unauthorized actions. Agent security has to address a class of failure traditional AppSec never had to model: an authorized action performed for an unauthorized purpose. A compromised document can convince an agent to delete a record the operating user genuinely has permission to delete. The API correctly authorizes the request. The system is still compromised. Simon Willison's "lethal trifecta" is the cleanest mental model I've seen for when this actually turns dangerous: access to private data, exposure to untrusted content, and the ability to communicate externally. None of the three legs is a vulnerability in isolation. It's the combination that creates the exposure — an attacker slips an instruction into content the agent will process, the agent executes it, private data leaves the perimeter (Getia Consulting, AI Agent Security 2026). This isn't a theoretical framing anymore. Check Point Research documented a single operator combining Claude Code and GPT-4.1 to breach nine Mexican government agencies between late December 2025 and mid-February 2026, converting roughly 1,088 typed prompts into more than 5,300 AI-executed commands and exposing on the order of 400 million records — tax filings, civil registry, patient, vehicle, and electoral data (awesome-ai-agent-attacks, GitHub). That incident wasn't a prompt injection against a victim's agent — it was an attacker using agentic tooling as their own offensive platform — but it's a preview of the asymmetry defenders are up against: the same architecture that makes an agent useful for legitimate multi-step work makes it useful for an attacker's multi-step work too. The supply chain is its own exposure. In March 2026, a malicious package sat live on PyPI for roughly three hours — the compromised LiteLLM release, which serves as the model gateway for CrewAI, DSPy, Microsoft GraphRAG, and a long list of other agent frameworks — during which roughly 47,000 downloads occurred, pulling a backdoored autonomous attack tool in alongside the update (Help Net Security, June 2026). That maps to ASI04 in OWASP's taxonomy — agentic supply chain vulnerabilities — a category that barely existed as a named risk eighteen months ago. This is why identity propagation, least privilege, and audit trails are necessary but not sufficient on their own. Agent security also needs constrained tool semantics, confirmation boundaries on irreversible actions, data provenance the agent can actually answer for, and evaluation against adversarial instructions as a standing practice, not a pre-launch checklist item. Observability as an Accountability Mechanism, Not a Debugging Aid I'd push this argument one step further than the industry currently takes it: an agent's execution trace should be treated as a first-class security artifact, not an engineering convenience. A traditional distributed trace tells you where a request traveled. An agent trace needs to answer a different set of questions — why the system selected a given tool, which identity authorized the action, what data informed the decision, what parameters were supplied, what the tool returned, and whether a human actually approved the resulting write. Gravity's implementation — a visible execution checklist that doubles as a timeline, a reasoning trace per step, full tool-invocation history, and a staged preview of every proposed change before commit — is the same instinct that produced distributed tracing and OpenTelemetry in cloud-native infrastructure a decade ago: a system too complex to reason about from the outside has to narrate its own behavior from the inside, continuously, or nobody downstream trusts it under real load. That reframing — trace as accountability mechanism rather than debugging tool — is, in my view, the more consequential shift, because it's what turns an incident review from forensics into something closer to a routine audit. The Hard Problems That Remain None of the above should read as "solved." The honest list of what's still genuinely unresolved, industry-wide, in mid-2026: Prompt injection, both direct and indirect, remains structurally difficult to fully close — OWASP's own 2026 LLM Security research put the year-over-year surge in injection attempts at 340% (AI Magicx, April 2026).Memory poisoning — a persisted context corrupted so a later, unrelated task inherits false assumptions — is now its own OWASP category (ASI06) and doesn't map cleanly onto any pre-LLM threat model.Authorized-but-unintended actions, the EchoLeak pattern, aren't fixed by permission inheritance alone.Long-running task failures and context drift over multi-day agent sessions are still mostly evaluated in demos, not adversarial production conditions.Skill and harness regressions can silently degrade a capability customers were actively relying on, which is why versioning and rollback matter as much for skills as for any other production code.Evaluation at scale — proving an agent's judgment holds up across the long tail of messy, real scenarios rather than a fixed benchmark set — remains closer to an open research problem than a solved engineering one. The goal with each of these isn't to eliminate the risk outright. It's to make it observable, bounded, testable, and recoverable — which is a materially different, more honest bar than "safe." My Framework for Evaluating Enterprise Agents After going through Gravity's architecture in this level of detail and cross-referencing it against the broader 2026 threat and adoption data, I'd reduce production readiness to seven questions: Agency – Can the system independently execute multi-step objectives, not just suggest them?Compositionality – Can it combine capabilities it wasn't explicitly wired into a fixed workflow to perform?Execution – Does every action pass through the same validated business logic a human user would hit?Identity – Can every action be attributed to a specific principal, recalculated per call rather than cached?Determinism – Are correctness-critical operations owned by a deterministic system, not a model's best guess?Governance – Can risky mutations be previewed, validated, approved, and audited before they commit?Evaluation – Can the system demonstrate its behavior stays reliable as models, skills, and scenarios keep shifting underneath it? If an agent fails several of these, improving the model is very rarely the first engineering problem worth solving. Where This Is Headed Kornish's predictions for the next three to five years weren't flashy, which is exactly why I'd take them seriously. Scope stays the whole story — agents confined to what they were demoed on keep losing ground to agents that can operate across an entire platform, the same pattern that played out when narrow point solutions gave way to platforms in every other software category. Vendors without an API-first foundation fall further behind every year, because that kind of scope isn't something you retrofit after the fact. Gartner's own projection points in the same direction: 40% of enterprise applications are expected to embed task-specific AI agents by the end of 2026, up from under 5% in 2025 (Paul Okhrem, Enterprise AI Agents Adoption Statistics 2026). And observability finishes its transition from differentiator to baseline expectation, tracing the same arc logging and monitoring took in cloud infrastructure roughly a decade ago. The industry's conversation about enterprise AI has spent the last few years almost entirely on model capability. The next phase is being decided somewhere else — in the API surface, the identity model, the deterministic core, and the governance wrapped around every write. That's not a glamorous place for the conversation to move. It's also the only place it was ever going to survive contact with a regulator, an auditor, or an attacker. I conducted this interview with Ted Kornish, CTO of Gravity, specifically to understand how these architectural questions are being resolved inside a production system operating in a regulated domain. His platform is the case study here; the analysis and framework above are my own.

By Igboanugo David Ugochukwu

CORE

Golden Prompts: Turning AI Prompting into an Engineering Practice

AI-assisted software engineering is becoming part of software development on a daily basis. AI tools are used now for code creation, writing tests, reviewing changes, troubleshooting, documentation creation, and architecture and design decisions. But most teams still use AI largely in an ad hoc fashion. Every developer writes prompts differently, gives different levels of context, and expects different outputs. As a result, the quality of AI-generated results can be very different even for the same engineering task. Consider a simple code-review request: Review this code. One developer might expect security issues to be identified, another might focus on performance issues, while someone else may focus on maintainability and coding-standard violations. When all engineers create prompts independently, teams will end up with inconsistent results, repeated experimentation, and missing important engineering requirements. This is where golden prompts become useful. What Is a Golden Prompt? A golden prompt is a reusable and carefully designed prompt for a specific engineering task. It captures the right context, instructions, constraints, and expected output so that different engineers can achieve more consistent results. For example, instead of simply asking: Review this Terraform code. A golden prompt could ask the AI to review the code for security, IAM permissions, networking exposure, hard-coded secrets, maintainability, and potentially destructive changes, and return the findings in a predefined format. Golden Prompts vs. Ad Hoc Prompts The key shift is simple: Instead of treating prompts as disposable instructions, golden prompts treat them as reusable engineering assets. That shift moves organizations from ad hoc AI use to a more structured prompt-engineering practice. Why Do We Need Golden Prompts? As AI tools become part of everyday engineering, prompt quality directly affects the quality and consistency of the output. Golden prompts move teams from individual experimentation to a common and repeatable approach to working with AI. Inconsistent AI output across engineers: Engineers can describe the same task in very different ways, and the results are different in terms of detail, accuracy, and quality. Golden prompts provide a common starting point and make results more consistent.Repeated prompt engineering effort: Without shared prompts, engineers spend time refining similar instructions for code reviews, test generation, troubleshooting, documentation, or architecture analysis. Golden prompts avoid this duplication by recording prompts that already work well.Missing organizational context: Generic prompts do not reflect architecture standards, coding guidelines, technology choices, or operational practices in an organization. Golden prompts should be written with this context in mind so that AI-generated responses are more closely aligned to how the organization builds software.Security and compliance risks: A general-purpose prompt can neglect security, privacy, compliance, and governance requirements. Golden prompts should include these expectations explicitly and make them part of the AI-driven workflow rather than the engineering team having to remember them.Variability across AI models and tools: Different AI models and coding assistants can interpret the same instruction differently. Golden prompts cannot eliminate model differences, but well-defined context, constraints, and expected outputs can reduce unnecessary variation across tools.Capturing engineering best practices: Golden prompts are a simple way of taking engineering knowledge and sharing it with the rest of the team. A well-designed prompt can include the practices of code quality, security, testing, reliability, and infrastructure for the wider team. In short, golden prompts turn individual prompting knowledge into reusable team knowledge, improving consistency while reducing repeated effort. Business Benefits of Golden Prompts While golden prompts improve day-to-day engineering productivity, their value extends beyond individual users. Organizations can use golden prompts to improve consistency, accelerate AI adoption, and reduce operational risk. Reduced rework: Teams spend less time repeatedly refining prompts for common tasks.Faster onboarding: New engineers can leverage proven prompts instead of learning through trial and error.Improved consistency: Similar tasks are evaluated using common criteria and output formats.Better compliance and governance: Security, privacy, and organizational policies can be embedded into reusable prompts.Knowledge reuse: Engineering best practices become reusable assets that can be shared across teams.Higher AI ROI: Teams achieve more predictable and repeatable outcomes from AI-assisted workflows. In essence, golden prompts help transform AI usage from an individual productivity technique into a scalable organizational capability. The Impact of a Golden Prompt: A Before-and-After Example The value of a golden prompt becomes clearer when we compare it with a generic prompt for the same engineering task. Consider a simple code review scenario. Suppose the code being reviewed is: Python def get_user(id): query = "SELECT * FROM users WHERE id = " + id result = database.execute(query) return result Generic Prompt Review this code. A typical AI response can be — The function retrieves a user from the database. Potential improvements: Add error handling.Validate user input.Consider using parameterized queries.Improve readability. The response is technically correct, but it is broad and unstructured. There is no prioritization of issues, no severity assessment, and no clear guidance on what should be fixed first. Golden Prompt Act as a senior software engineer. Review the code for: Correctness, Security issues, Performance concerns and Maintainability. For each finding: Explain the issue, Assign a severity (High, Medium, Low), and Recommend a fix. If no issues are found, explain why the code is acceptable. A typical AI response can be — Code Review Findings Finding 1: SQL Injection Vulnerability Severity: High Issue: The query is built using direct string concatenation. Risk: An attacker could manipulate the input and execute unauthorized SQL commands. Recommendation: Use parameterized queries. ------------------------------------------------ Finding 2: Missing Input Validation Severity: Medium Issue: The function assumes the input is valid. Recommendation: Validate the ID before executing the query. ------------------------------------------------ Finding 3: Missing Error Handling Severity: Medium Issue: Database failures are not handled. Recommendation: Add exception handling and logging. ------------------------------------------------ Summary The code contains one High-severity security issue and two Medium-severity reliability concerns. The SQL Injection vulnerability should be addressed before deployment. The important difference is not that the golden prompt makes the AI smarter. It provides a clearer framework for evaluation by defining the review criteria, expected output structure, and severity classification. As a result, the response becomes more consistent, actionable, and reusable across different engineers and teams. Anatomy of a Golden Prompt A golden prompt is more than a detailed instruction. It is a structured prompt that gives the AI enough context, boundaries, and expectations to perform a task consistently. A simple way to visualize the anatomy of a golden prompt is: Role + Context + Input + Constraints + Standards + Evaluation Criteria + Output + Validation Not every golden prompt needs every element. The goal is to include enough structure to make the AI response reliable and repeatable without making the prompt unnecessarily complex. Golden Prompt Use Cases Across Engineering and Service Teams Engineering / Service Team Example Golden Prompt Use Cases Product Management Requirement refinement, user-story generation, acceptance criteria, feature prioritization, release-note drafting Development Code generation, refactoring, code explanation, unit-test generation Architecture Architecture review, technology evaluation, trade-off analysis, ADR generation DevOps & Platform Engineering CI/CD troubleshooting, Kubernetes diagnostics, IaC generation, runbook creation Security Threat modeling, secure code review, vulnerability analysis, security recommendations SRE & Operations Incident investigation, root-cause analysis, log and metric analysis, post-incident reviews Cloud Service Management Service health analysis, SLA/SLO review, capacity planning, change-impact assessment, operational readiness QA & Testing Test-case generation, negative testing, API testing, regression-test identification Designing Golden Prompts for Different AI Tools The same golden prompt should not always be used unchanged across AI tools. Different tools have different strengths, context mechanisms, integrations, and interaction models. The core intent can remain the same, but the prompt should be adapted to the tool. AI Tool / Tool Type Golden Prompt Considerations ChatGPT / General-Purpose LLMs Provide clear context, role, constraints, expected output, and relevant reference information. These tools work well for analysis, explanation, troubleshooting, and structured reasoning. GitHub Copilot / IDE Assistants Keep prompts closer to the code and development task. Include language, framework, coding conventions, testing expectations, and files or components that should be considered. Claude / Long-Context Assistants Take advantage of larger context by providing architecture documents, requirements, policies, or larger code sections when relevant. Clearly identify which information should drive the response. AI Code Review Tools Define the review criteria explicitly, such as correctness, security, performance, maintainability, error handling, and test coverage. Specify how findings should be prioritized. Security AI Tools Include the application context, threat model, security standards, trust boundaries, and expected severity classification. Avoid relying only on generic vulnerability identification. Cloud and Operations Assistants Provide environment details, logs, metrics, alerts, deployment information, and operational constraints. Ask the AI to base recommendations on evidence rather than assumptions. Infrastructure-as-Code Assistants Include the cloud platform, IaC framework, organizational standards, security requirements, naming conventions, and restrictions on destructive changes. Enterprise AI Platforms Include organizational policies, approved technologies, architecture standards, compliance requirements, and data-handling restrictions within the Golden Prompt or its supporting context. Golden Prompts Should Be Context-Aware A golden prompt should not be very general. Its effectiveness depends on whether the golden prompt is representative of the environment in which the task is being performed. The same task may need to be handled differently depending on the team, application, technology stack, repository, security requirements, and so on. Team context: Include the responsibilities and working practices of the team using the prompt. A developer, SRE, security engineer, or product manager might evaluate the same problem differently.Application context: Provide relevant information about the application, its purpose, architecture, critical components, dependencies, and operational needs.Technology-stack context: Specify the languages, frameworks, cloud platforms, databases, Kubernetes distributions, CI/CD tools, or other technologies that should influence the response.Repository context: Where applicable, include repository-specific information such as project structure, existing coding patterns, dependencies, configuration conventions, and testing practices.Security context: Define relevant security expectations, such as authentication, authorization, secrets handling, network exposure, data protection, and vulnerability requirements.Organizational policies: Golden Prompts can also incorporate internal standards, architecture principles, compliance requirements, approved technologies, and operational policies. This can be represented simply as: Golden Prompt = Base Prompt + Team Context + Tool Context + Task Context + Guardrails The base prompt defines the common task and expected outcome, while the additional context adapts it to a specific team, tool, environment, and situation. For example, a common Kubernetes troubleshooting prompt may be reused across teams, but the surrounding context can specify the cloud platform, monitoring tools, production constraints, security requirements, and operational procedures. The goal is not to create a completely different prompt for every situation. Instead, teams can maintain a stable base prompt and enrich it with the context required for the task at hand. Building a Golden Prompt Library As golden prompts grow across teams, they should be managed as shared engineering assets rather than scattered across personal notes or chat histories. A golden prompt library helps teams discover, use, and improve high-quality prompts consistently. Central prompt repository – Store all approved Golden Prompts in one centralized location that is easy to access and trust.Prompt categories – Organize prompts by domain, team, use case, or tool so users can quickly find the right prompt.Naming conventions – Use a clear and consistent naming pattern to make prompts easy to search, identify, and reference. Metadata and ownership – Capture details such as purpose, team, owner, use case, tags, and last updated date to ensure accountability. Version control – Track changes, maintain prompt versions, and support rollbacks to keep prompts accurate and reliable. Examples and usage guidance – Provide examples, expected inputs, sample outputs, and usage tips to help users apply prompts correctly.Reusable prompt templates – Provide templates for common scenarios that teams can customize to create new prompts quickly. A well-managed Golden Prompt Library makes prompts easier to discover, reuse, improve, and govern across teams. Golden Prompt Library Architecture The library can sit at the center of the AI-assisted engineering workflow, connecting users, engineering tools, version control, and governance. Users → Access Layer → Golden Prompt Repository → Tool Integrations with version control and governance applied across the library. The goal is simple: one trusted source for reusable prompts, with clear ownership, controlled evolution, and consistent usage across engineering teams. Golden Prompt Lifecycle A golden prompt should evolve rather than remain static. A simple lifecycle helps teams create, validate, publish, monitor, and continuously improve prompts as engineering needs, tools, and standards change. Golden Prompt Quality and Evaluation Criteria A golden prompt should be evaluated before it is published to the shared library. The goal is to ensure that it is clear, reliable, reusable, and produces useful results across different users and scenarios. Evaluation Criterion What to Check Weight Clarity Is the task and expected outcome clearly defined? 10% Completeness Does the prompt include the required context, inputs, constraints, and output expectations? 12% Accuracy Does the prompt guide the AI toward technically and factually correct responses? 15% Relevance Does it stay focused on the intended task? 5% Consistency Does it produce reasonably consistent results across repeated use? 10% Context Awareness Does it include the required team, application, technology, and organizational context? 10% Security & Safety Are appropriate security, privacy, and operational guardrails included? 12% Standards Alignment Does it reflect applicable engineering standards and organizational policies? 8% Actionability Does the output provide useful findings, recommendations, or next steps? 7% Output Quality Is the expected response structured and usable? 4% Tool / Model Compatibility Does it work effectively with the intended AI tool or model? 2% Reusability Can it be reused across similar scenarios with minimal changes? 3% Maintainability Can it be easily updated as requirements evolve? 2% Total 100% Note: The weights shown are illustrative and should be adjusted based on organizational priorities and risk appetite. You could classify prompts as: Final Score Quality Decision 90–100 Golden / Approved 80–89 Good – minor improvements 70–79 Needs refinement Below 70 Not ready for Golden Prompt library Accuracy, completeness, security and safety, clarity, consistency, and context awareness the highest-priority factors. Security and safety and accuracy can be treated as mandatory gates, not just weighted criteria. For example, a prompt should not be approved even with an overall score above 90 if it fails a critical security or accuracy check. Golden Prompt Governance Model A structured governance model ensures quality, security, consistency, and responsible use of golden prompts. Golden Prompt Anti-Patterns and Risks Golden prompts improve consistency, but poorly designed or poorly governed prompts can introduce their own risks. Recognizing common anti-patterns helps teams avoid prompts that are rigid, outdated, insecure, or unreliable. Anti-Pattern / Risk What It Looks Like Why It Is a Problem Overly Generic Prompt Uses broad instructions with little context Produces shallow or inconsistent results Overloaded Prompt Tries to cover too many tasks, rules, and scenarios in one prompt Makes the prompt difficult to understand, maintain, and execute reliably Hard-Coded Context Embeds environment-, team-, or tool-specific details directly into the base prompt Reduces reuse and quickly makes the prompt outdated Missing Guardrails Does not define security, operational, or compliance boundaries Can lead to unsafe or inappropriate recommendations Ambiguous Instructions Uses unclear objectives or vague success criteria Different users or models may interpret the task differently Unvalidated Prompt Is published without testing against representative scenarios Errors and poor outputs become reusable at scale Model-Specific Dependency Relies heavily on behavior unique to one model or tool May perform poorly when the underlying AI model changes Prompt Duplication Multiple teams maintain slightly different versions of the same prompt Creates inconsistency and unnecessary maintenance No Ownership No team or individual is responsible for maintaining the prompt Outdated or incorrect prompts remain in use No Version Control Changes are made without tracking or review Makes rollback, comparison, and auditing difficult Stale Prompt Continues referencing old technologies, policies, or architectures Produces recommendations that no longer match the environment Excessive Trust in Output AI-generated results are accepted without validation Can propagate incorrect assumptions, insecure configurations, or poor decisions Sensitive Data Exposure Prompts encourage users to paste secrets, credentials, or confidential data Creates security, privacy, and compliance risks Too Rigid Forces the same workflow and output for every scenario Prevents adaptation when task context differs A useful principle is: Golden prompts should standardize good engineering practices without becoming static, overly restrictive, or blindly trusted. From Golden Prompts to Golden AI Workflows Golden prompts are a strong starting point, but enterprise AI usage does not stop at a single prompt. As AI adoption matures, prompts become part of broader workflows that combine context, tools, policies, and automation. Evolve a single prompt into intelligent, policy-aware, tool-augmented AI workflows that deliver consistent and high-quality outcomes at scale. When combined with context, tools, and policies, golden prompts power intelligent workflows and agentic systems that drive productivity, consistency, and trust in enterprise AI engineering. The broader idea is that golden prompts provide the standardized intent and guidance, while workflows, tools, and agents provide the execution around them. Conclusion As AI becomes increasingly embedded in software engineering, prompts should no longer be viewed as disposable instructions. Well-designed prompts capture engineering knowledge, standards, context, and guardrails, making them valuable reusable engineering assets. Golden prompts bring standardization to AI-assisted engineering without removing developer flexibility. Teams can start from a trusted prompt and adapt the necessary context for a particular application, tool, or task while maintaining common quality and security expectations. Ultimately, golden prompts provide a foundation for consistent, scalable, and governed AI-assisted software engineering. As organizations move toward AI workflows, reusable skills, and agentic systems, these prompts can become the building blocks that connect engineering intent with context, tools, policies, and automation.

By Josephine Eskaline Joyce, Ph.D

CORE

Enterprise Architecture in the AI Era: Tools, Capabilities, and the Road to Autonomy

An enterprise architecture (EA) tool is a software platform that enterprises use to capture, connect, and continuously maintain a structured picture of the enterprise covering strategies, business capabilities, processes, applications, data, technologies, and the relationships between all these elements. An EA tool acts as a Central Enterprise Repository (a “single source of truth”) that architects and other stakeholders use to model both the current state of the enterprise and the desired future state. At its core, an EA tool is built on the following three foundational layers Metamodel: the taxonomy and rules that define what kinds of elements (applications, capabilities, goals, processes, risks, etc.) can exist and how they relate to one another.Modeling and repository environment: where those elements are created, stored, versioned, and analyzed.Collaboration layer: that connects a wide range of business and IT stakeholders so that EA insight isn’t locked inside the architecture team. EA tools are used across many architecture and IT disciplines covering business, information, security, application, and infrastructure architecture. They typically integrate with adjacent systems such as CMDBs, financial planning tools, project and portfolio management (PPM) systems, and process mining tools to pull in the data that keeps the model accurate and useful. Limitations of Traditional EA Tools and Their Usage Enterprises still struggle to communicate the quantifiable business value of EA tools to business stakeholders. The most prominent challenges faced by these business users and non-IT stakeholders are, Managing data qualityManual data managementComplexity of the tool’s usageLack of automation of EA workNo out-of-the-box AI capabilities in EA tools Despite steady progress, users of EA tools and heads of EA continue to encounter a consistent set of challenges: AI-enabled EA tool: The majority of EA tools are not fully automated to generate end-to-end target architecture. Current EA Tools are not mature enough to handle sophisticated tasks such as a comprehensive roadmap. Data quality: Current EA tools struggle with data issues, including poor data quality, data accuracy, and real-time data. This is because the architects, application owners, and EA tool administrators manually follow up with stakeholders for accurate and up-to-date data entry. Resistance to adoption: In most cases, EA tool use by owners across enterprises is centered on IT. The stakeholders outside IT are not clear on the value proposition of EA tools. This makes the enterprise-wide adoption of EA tools difficult and slows the demonstration of quantifiable business value to senior leadership. Siloed data sources: In most cases, the data lives in disconnected source systems across the enterprise. This results in inconsistent data and a lack of integrated pictures across the entire enterprise. Usage of multiple tools: Many enterprises use multiple tools for capturing the artifacts covering all the architecture domains, leading to the implementation of complex governance frameworks to standardize the processes and keep data consistent across tools. Embedding of AI governance: Most enterprises are still in the process of establishing AI Governance across the enterprise. Implementing data privacy, ethical usage, risk management, and regulatory compliance are the big challenges. Skills gap: Getting real value from increasingly AI-augmented EA tools requires continuous upskilling, both within the EA team and across the enterprise, which not all enterprises have invested in. Importance of EA Tools in Digital Era The fundamental purpose of an enterprise architecture (EA) tool is to turn a complex, ever-changing enterprise into something that can be modeled, reasoned about, and deliberately steered as business and technology conditions evolve. To achieve this, an EA tool establishes a common language and structured repository that links business strategy directly to technology decisions, enables leaders to analyze trends and risks to plan realistic future scenarios, and supports the complete arc of strategic execution, from defining initial goals to governing solution design and tracking benefits. Ultimately, by illuminating costs and risks across the enterprise landscape, it optimizes technology investments, reduces technical debt, and strengthens operational resilience to ensure continuous, alignment-driven transformation. Enterprise architecture tools in the digital era are required to, Establish business and IT collaboration to achieve enterprise strategic objectives and measurable business outcomes in terms of reduced downtime or faster project deliveryBusiness capability modelingIntegrate with enterprise data sources and repositories to automate data ingestionMinimize manual involvement and automate the enterprise architecture processesEvaluate assets, returns, and risks in the IT landscapeApplication portfolio rationalization: estimate interdependencies between portfolios for applications, technologies, projects, services, and APIsSupport business innovation, a new market segment or the back office, and determine how fast systems need to changeRoadmap planning, executive reporting, and dashboards Core Capabilities of EA Tools In general, EA tools enable enterprises to map their business architecture, business capabilities, business processes, application architecture, data architecture, integration architecture, security, and infrastructure by centralizing data. The most common capabilities of Next Generation EA Tool are, EA repository: It acts as the single source of truth. It supports business, information, technology, and solution viewpoints and versioning of all these objects and their relationships that support business direction, vision, strategy, etc. EA modeling: Support the viewpoints and relationships across strategies and goals of business, information, solutions, and technology. Modeling of As-Is and Target state, Impact Analysis and RoadmapsDecision analysis: Capabilities such as gap analysis, traceability, impact analysis, scenario planning, system thinking, and opportunities across portfolios of capabilities, investments, applications, and technologies.Multiple views: Support multiple views for different types of audience/users such as Executives, Architects/Designers, Business Planners, and Suppliers, etc. Support customization and extensions of meta-model, diagrams, menus, matrices, and reportsCollaboration and sharing: Provides good collaboration-oriented features, which include simultaneous model editing, a shared remote repository, version management including model comparison and merge, easy publishing, and review capabilitiesCompatibility: Support for multiple frameworks and standards, and should enable integration of these models into a single repository that enables interoperability in a tool chain and data migration between toolsAdministration: Enable security, user management, and other tasks. Ease of administration of various day-to-day operations. Configurability: Support for configuration of the tool to reflect the uniqueness of the enterprise. It helps in administering security, role-based access, and persona-specific user experiences across the tool.Integration: Usage of Open APIs for integration with other enterprise tools such as CMDB, PPM, BPM, JIRA, etc such that it can serve as an Enterprise Data Hub.Frameworks and standards: Supports standard EA frameworks (e.g., TOGAF, Zachman Framework) and industry-standard notations/conventions for business and IT architecture/design modelingPresentation: Provides the capabilities that are visual or interactive to meet the demands of a myriad of stakeholders. To present the content to various types of users, including web, thick clients, and reports.Publication: Distributes repository content broadly across the enterprise and captures feedback, comments, and scoring from consumers of that content. AI-Enabled Capabilities of Next-Generation EA Tools AI capabilities in EA tools help business stakeholders make informed strategic and operational decisions. They also support EA governance and streamline content creation across architecture layers and integrations. The next-generation AI capabilities that are being embedded in EA tools/platforms are, EA Copilot: It acts as an intelligent architecture assistant that enables architects and stakeholders to interact with enterprise architecture repositories using natural language. It provides contextual architecture recommendations, answers architecture-related questions, and performs impact analysis through conversational interfaces. By leveraging enterprise knowledge, standards, and architecture artifacts, it helps the architects make faster, more informed design and governance decisions. Innovation and sustainability: Used by innovation teams and business stakeholders to track ideas from inception to commercialization and connect them to business outcomes.AI portfolio rationalization: It uses machine learning to analyze application portfolios, identify redundancies, and assess business and technical value. It generates data-driven TIME (Tolerate, Invest, Migrate, Eliminate) recommendations, uncovers consolidation opportunities, and prioritizes modernization initiatives. The capability helps enterprises optimize technology investments, reduce operational costs, and simplify complex application landscapes.Automated architecture documentation: It leverages Generative AI to create and maintain architecture artifacts such as HLDs, LLDs, ADRs, and TOGAF deliverables. It can generate architecture diagrams and documentation directly from requirements, models, or existing system information. This significantly reduces manual effort, improves consistency, and accelerates architecture delivery while ensuring documentation remains current and reusable.EA governance: It is used by architecture review boards, risk/compliance teams, and delivery teams to apply control-based, outcome-based, agility-based, and autonomous governance styles as appropriate to context. AI-powered review bots continuously assess architecture artifacts, identify risks, detect architectural drift, and recommend corrective actions.EA linkages: It creates a connected, searchable view of enterprise architecture data by linking applications, technologies, business capabilities, processes, and infrastructure. AI-driven semantic search and relationship discovery enable architects to quickly identify dependencies, impacts, and hidden connections across the enterprise. This provides deeper architectural intelligence and supports faster decision-making for transformation initiatives.Technology radar AI: It continuously monitors technology trends, vendor ecosystems, and innovation signals to provide strategic technology insights. It assists architects with build-versus-buy decisions, evaluates emerging technologies, and analyzes product lifecycle risks and opportunities. By combining market intelligence with enterprise context, it helps organizations make informed technology investment and modernization decisions. Users of EA Tools The audience for EA information now extends well beyond the EA team. EA tools are designed for EA specialists who create and maintain models, as well as a broader group of non-EA stakeholders who primarily consume architecture content. Most EA tools offer self-service access, enabling non-technical users to engage with relevant information without learning the full tool. The various types of users of the EA tool are, CIOs, business leaders and executives, IT strategists Enterprise and business architects Solution and data architects PMO and project managers Architecture review boards Agile and DevOps delivery teamsRisk, security and compliance teamsBusiness users and external partners Users, purpose of usage, and typical outputs User / Personas Why They Use the Tool Typical Outputs Generated Enterprise & Business Architects Model the enterprise's current and future state; connect strategy, capabilities, processes, and technology. Advise leadership on trends and transformation options Business capability maps, value streams, operating-model diagrams, trend/disruption analyses, strategic recommendations Solution & Data Architects Translate approved concepts into detailed, standard-aligned designs. Identify information flows and processing needs Context diagrams, C4/UML models, solution architecture documents, data flow diagrams, decision logs Business Leaders & Executives Monitor progress toward strategic objectives. Make investment and prioritization decisions Executive dashboards, scorecards, heat maps, portfolio health summaries, cost-benefit views PMO & Project Managers Manage dependencies across initiatives, mitigate delivery risk. Track transformation progress Roadmaps, dependency maps, risk registers, status/progress reports Architecture Review Boards Approve or reject solution proposals. Enforce standards and principles, oversee architecture health Review decisions, compliance/audit findings, standards catalogs, governance dashboards Agile & DevOps Delivery Teams Consume approved designs to plan and execute delivery. Assess infrastructure and cloud implications Backlogs, epics, story points, release plans, cloud resource/deployment views Risk, Security & Compliance Teams Assess exposure, track regulatory alignment (e.g., GDPR, DORA, SOX), manage audit cycles Risk catalogs, compliance dashboards, audit reports, control-exception logs Business Users & External Partners Provide input without needing architecture expertise. Respond to surveys, confirm application usage, submit ideas, access relevant data securely Survey responses, application-fitness ratings, submitted ideas/comments, shared partner reports Benefits of Usage of EA Tools The following are the high-level benefits of the usage of Enterprise Architecture tools, Single source of truth: It reduces reliance on scattered spreadsheets and tribal knowledge for understanding applications, capabilities, and dependencies.Faster informed decisions: Stakeholders can see the cost, risk, and business impact of a change before committing to it.Business and IT alignment: Shared models and dashboards give both IT and business stakeholders a common frame of reference.Improved portfolio management: Helps in portfolio and application rationalization analysis that results in identifying aging or redundant applications and technologiesEstablishing governance: Automated workflows, standards catalogs, and review board support make governance more consistent without becoming a bottleneckTransformation roadmap: Helps in defining enterprise-wide roadmaps that connect strategy, investment, and delivery so that transformation programs stay on track.Architecture insight: Role-based views, surveys, and self-service dashboards allow business stakeholders to consume and contribute to EA content without needing deep technical expertise.AI-driven productivity gains: Automated data ingestion, diagram generation, and natural-language assistants reduce manual effort.Support for innovation and sustainability: Built-in idea management and tracking connect emerging trends and sustainability goals directly to the enterprise roadmap. Summary Enterprise Architecture is moving towards Agentic EA where AI agents autonomously assist architects with, Autonomous solution designArchitecture review automationGovernance enforcementSelf-updating architecture repositoriesMulti-agent collaborationContinuous architecture compliance monitoring AI-enabled EA tools are evolving from architecture repositories into intelligent architecture copilots that can automatically discover applications, generate architecture artifacts, rationalize portfolios, enforce governance, assess technical debt, and provide real-time strategic recommendations. It helps to identify redundant applications across the enterprise and retire them. This helps in improving cost savings. The tool also helps in integrating the EA across the enterprise. Acknowledgements The authors would like to thank Tricon Solutions LLC for giving the required time and support in many ways in bringing up this article. Disclaimer The views expressed in this article/presentation are those of the authors, and Tricon Solutions LLC does not subscribe to the substance, veracity, or truthfulness of the said opinion.

By Dr Gopala Krishna Behara

CORE

AI/ML

DZone's Featured AI/ML Resources

Monthly Top AI/ML Experts

The Latest AI/ML Topics