DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Newsletter
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

DZone Spotlight

Tuesday, October 27 View All Articles »
How to Build Your Own AI SRE From Scratch

How to Build Your Own AI SRE From Scratch

By Severin Neumann
If you are responsible for keeping a production system alive, you have probably been pitched an "AI SRE" in the last few months. Maybe more than one. Maybe more than one this week. When we sat down at Bronto to map that space, we ended up with 64 tools across seven camps: pure-play investigation agents, observability platforms with AI built in, incident management tools with AI on top, AIOps and alert correlation, hyperscaler agents like AWS DevOps Agent and Azure SRE Agent, Kubernetes copilots and open source projects, and a long tail of adjacent tools. All of them use the same two words on their landing page, and they solve very different problems. So the first question most teams ask is the classic one: do we buy, or do we build? In our landscape article, we argue that the choice isn't binary. You can go full DIY, build on open source foundations like HolmesGPT or K8sGPT, buy a product and extend it, or buy it as-is. Most teams land somewhere in the middle. In this article, I want to make a different argument for building one, at least once: build to understand what you buy. Even if you never run your own DIY agent in production, putting one together by hand shows you where its answers come from, what it can't see, and why it goes wrong when it does. After building one, you know which questions to ask a vendor, and you know how to get better results from whatever you end up running. That's the profit you should take away from this post. The fun is the building and seeing your agent try to investigate an incident: it might get it wrong, it might get it right, but you'll see how it interacts with your tools and your data. This post is based on a hands-on lab we ran at an AWS and Bronto evening, and everything is available through bronto-community/ai-sre-lab. By the end, you will have a small agent with the following capabilities: reads logs, traces, and metrics through an MCP server,reads the application's source code and commit history through GitHub's MCP server,writes its hypothesis, evidence, and confidence into a GitHub issue. Note that in the lab we use Bronto as the observability backend. I work there, so I am biased to think it is your best option, but you can of course use any other solution. What an AI SRE Is AI SRE is mostly a marketing term. In short, it's an agentic loop pointed at the production system that can answer questions about it and help you troubleshoot. Like any agentic loop, it has inputs on one side (an alert, a question from a human), a model in the middle, tools underneath (mostly MCP servers these days, for telemetry, code, cloud APIs, runbooks), and outputs on the other side (a Slack message, a ticket, a pull request). So, it's an agent like any other. It's what you connect to it that makes it an agent for SRE. The Patient: A Small Online Shop Our AI SRE needs something to investigate. The lab comes with a tiny storefront: four Python services instrumented with OpenTelemetry, plus a payment processor mock standing in for a third party. You can, of course, point the agent directly at your own data. It's read-only, so it is not able to harm your system. What You Need To run this lab, you need the following: A container engine, like Docker DesktopAn AWS account with access to a model on Amazon Bedrock.A Bronto API key (there is a free trial), or another observability backend with an MCP server.A GitHub account and a fine-grained personal access token. It needs to read the contents of your copy of the storefront repo and write issues to it.Optional, for nicer output: jq and glow. You can watch this video to follow along as we walk through each step: Start the Shop Clone the lab and generate the storefront repository. The script builds a small Git history with a handful of commits and tags two releases, v3.1.0 (healthy) and v3.2.0: Shell git clone https://github.com/bronto-community/ai-sre-lab cd ai-sre-lab ./build-storefront.sh Your agent will later read this code through GitHub, so push it to a repository of your own, tags included: Shell cd storefront gh repo create storefront --public --source . --push git push --tags cd .. Then point the Collector at your telemetry backend and bring everything up: Shell cd demo echo "BRONTO_API_KEY=<your key>" > .env make up Note, that demo/collector.yaml ships to Bronto's EU ingestion endpoint. If your account lives in another region, or you want a different backend, that file is the only thing you need to change. Next, run make up, which deploys v3.1.0 and starts the traffic. It prints a message that you should leave it about ten minutes before you do anything else. An investigator with no baseline has nothing to compare against. So, take your time going through the first few next steps so it can build up! Step 1: The Loop All agents start with a basic file that receives prompts, sends them to the model, and returns the results. It's a simple first agent, so here is more or less the code in full: Python import os from bedrock_agentcore import BedrockAgentCoreApp from strands import Agent from strands.models import BedrockModel model = BedrockModel( model_id=os.environ.get("BEDROCK_MODEL_ID", "us.anthropic.claude-haiku-4-5-20251001-v1:0"), region_name=os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION", "us-west-2"), ) app = BedrockAgentCoreApp() @app.entrypoint def invoke(payload: dict) -> dict: agent = Agent(model=model) return {"result": str(agent(payload.get("prompt", "Hello!")))} if __name__ == "__main__": app.run() # POST /invocations, GET /ping on :8080 Two libraries do the work. Strands Agents runs the agent loop: send the prompt to the model, run any tool the model asks for, feed the result back, repeat until the model has an answer. The Bedrock AgentCore SDK wraps that in an HTTP server with the /invocations and /ping contract AgentCore Runtime expects. Today that runs on your laptop. Later, the same container can run on AgentCore Runtime without changes. Every step of the lab is a prebuilt image, so you don't need to build anything. Create a .env next to where you run the commands: Shell AWS_DEFAULT_REGION=us-west-2 AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... AWS_SESSION_TOKEN=... BEDROCK_MODEL_ID=us.anthropic.claude-haiku-4-5-20251001-v1:0 With this setup, you can start the agent for the first time: Shell docker run --rm -p 8080:8080 --env-file .env ghcr.io/bronto-community/ai-sre-agent:step-1 Open a second terminal and ask it something: Shell curl -s localhost:8080/invocations -d '{"prompt": "Who are you?"}' | jq -r .result It will say, "Claude, an AI assistant made by Anthropic." Correct, and completely useless for incident response. Right now this is a model behind an endpoint, with no identity and no tools. Step 2: Identity The second step changes only one line of code. Every Markdown file in prompts/ becomes part of the system prompt: Python SYSTEM_PROMPT = "\n\n".join(p.read_text() for p in sorted(Path("prompts").glob("*.md"))) @app.entrypoint def invoke(payload: dict) -> dict: agent = Agent(model=model, system_prompt=SYSTEM_PROMPT) ... The image ships with two files. role.md: Markdown You are the on-call SRE for Storefront, an online shop. You investigate production incidents: what broke, where, and why. And rules.md: Markdown - Only state what you can back with evidence from your tools. - If you have no tool that can answer, say so and name what you would need. - Be brief. You also get to add your own. Create a my.md: Markdown - Answer in at most three bullet points. - End every answer with your confidence: low, medium or high. - Sign off as "your friendly on-call dino". Start step 2 as follows. The command contains your custom instructions as an additional volume: Shell docker run --rm -p 8080:8080 --env-file .env \ -v "$PWD/my.md:/app/prompts/my.md" ghcr.io/bronto-community/ai-sre-agent:step-2 Shell curl -s localhost:8080/invocations -d '{"prompt": "Checkout is slow. What is wrong?"}' | jq -r .result You should get a short answer that says it has no access to telemetry, rates its own confidence as low, and signs off as a dino. That's a good result. The rule "say so and name what you would need" did its job: an agent that admits it can't see anything is far more useful than one that guesses. Step 3: Eyes Now the agent gets something to look at. Step 3 adds one tool source, Bronto's MCP server: Python bronto = MCPClient(lambda: streamable_http_client( os.environ.get("BRONTO_MCP_URL", "https://mcp.eu.bronto.io/mcp"), http_client=httpx2.AsyncClient( headers={"X-BRONTO-API-KEY": os.environ["BRONTO_API_KEY"]}, timeout=60, ), )) @app.entrypoint def invoke(payload: dict) -> dict: agent = Agent(model=model, system_prompt=SYSTEM_PROMPT, tools=[bronto]) ... That's the whole change. The MCP server's tools (search logs, list datasets, and so on) become tools the agent can call, and Strands handles the plumbing. This is also the point where MCP pays off: if your telemetry lives somewhere else, you swap this client for that vendor's MCP server and keep everything else. Add BRONTO_API_KEY to your .env (and BRONTO_MCP_URL if your account isn't in the EU region), then: Shell docker run --rm -p 8080:8080 --env-file .env \ -v "$PWD/my.md:/app/prompts/my.md" ghcr.io/bronto-community/ai-sre-agent:step-3 Start with a question that tells you what it can see: Shell curl -s localhost:8080/invocations -d '{"prompt": "What services do you know about?"}' | jq -r .result It should list the storefront services. Go ahead and ask it some more questions about the state of those services: what HTTP endpoints are available? What is the average error rate right now? With this, you have a first working version. Next we want to investigate an incident. To do so, ship the next release of the storefront: Shell make release According to the Makefile, symptoms take about 60 seconds to show up in p95. So wait a minute or two and then ask it: Shell curl -s localhost:8080/invocations -d '{"prompt": "Anything wrong with checkout?"}' | jq -r .result It will come back with a hypothesis about what is wrong and what you might be able to do about it. Take a look at the telemetry yourself and check: is the analysis correct, or did the agent pick up some false information? Step 4: Mouth An answer in your terminal is nice for a demo, but in production you would want to have it sent to a Slack channel, a pager app, or an issue tracker. In our example, we send the answer as a GitHub issue. This would let you tag your coding agent to review the issue and open a pull request that fixes it. You now have two options for how this gets filed. Either you give your agent another tool, and it can decide for itself where to send the hypothesis, or you make the code file the issue, so every analysis gets pushed into a dedicated issue: Shell def file_issue(question: str, hypothesis: str) -> str: r = httpx2.post( f"https://api.github.com/repos/{os.environ.get('GITHUB_REPO', 'ai-sre-lab/storefront')}/issues", headers={"Authorization": f"Bearer {os.environ['GITHUB_TOKEN']}"}, json={"title": f"[investigation] {question}"[:120], "body": hypothesis}, timeout=30, ) r.raise_for_status() return r.json()["html_url"] @app.entrypoint def invoke(payload: dict) -> dict: question = payload.get("prompt", "Hello!") agent = Agent(model=model, system_prompt=SYSTEM_PROMPT, tools=[bronto]) hypothesis = agent(question) return {"result": file_issue(question, hypothesis)} To make sure that what the agent provides fits the output, an additional new prompt file tells the agent how to write it: Markdown Your answer is filed as a GitHub issue for the on-call team. Write it as one: a one-line hypothesis, the evidence for it, your confidence, and what you could not check. Before running the next version of your agent, add your GitHub token and your repository to .env: Shell GITHUB_TOKEN=github_pat_... GITHUB_REPO=<your-user>/storefront Run step 4: Shell docker run --rm -p 8080:8080 --env-file .env \ -v "$PWD/my.md:/app/prompts/my.md" ghcr.io/bronto-community/ai-sre-agent:step-4 Ask it again to investigate the issue: Shell curl -s localhost:8080/invocations -d '{"prompt": "Anything wrong with checkout?"}' | jq -r .result The response is now a link. Open it and read the hypothesis. Is it right? How confident is it? There's a good chance it's confidently wrong. While testing the lab, one run concluded that checkout was down because a query returned nothing: an empty result is not proof of an outage, but nothing told the agent that. Step 5: Better Instructions What you do next doesn't involve code. You do what you'd do with a new human on the rotation: you tell them how you investigate. Add these lines to your my.md: Markdown - Slow is broken too: compare latency (p50, p95) before and after, not only errors. - Compare against a quiet window earlier the same day. - Look for `deploy` spans near the onset: they name the commit that went live. Each line encodes a habit experienced SREs have, and models don't bring by default. Checkout can be very broken without throwing a single error. A number means little without a baseline. And the first question in most incidents is "what changed?" The prompt files are read when the container starts, so restart step 4 with the same command and ask a more realistic question: Shell curl -s localhost:8080/invocations \ -d '{"prompt": "Customers say checkout got slow in the last 30 minutes. What is going on?"}' | jq -r .result Compare the new issue with the old one. Is it better? Worse? Did it fail outright? Step 6: Better Model As you might now, a different model can create a different output, so if you are not satisfied with the results yet, change one line in .env: Shell BEDROCK_MODEL_ID=us.amazon.nova-lite-v1:0 Restart your agent, ask the question again, and compare the outputs again: better or worse? Step 7: Code So far the agent can see that something changed and maybe even when. If your instructions worked, it found the deploy. What it can't do is look at what was deployed. Step 7 adds the second MCP server, GitHub's, restricted to read-only repository tools: Python github = MCPClient(lambda: streamable_http_client( os.environ.get("GITHUB_MCP_URL", "https://api.githubcopilot.com/mcp/x/repos/readonly"), http_client=httpx2.AsyncClient( headers={"Authorization": f"Bearer {os.environ['GITHUB_TOKEN']}"}, timeout=60, ), )) @app.entrypoint def invoke(payload: dict) -> dict: question = payload.get("prompt", "Hello!") agent = Agent(model=model, system_prompt=SYSTEM_PROMPT, tools=[bronto, github]) ... To make it useful to the agent, add one more prompt file, code.md. It connects the telemetry to the code: Markdown The shop's source code is in the GitHub repository `ai-sre-lab/storefront`. Every service reports the commit it runs as `deployment.commit`. Replace the repository path with your own, then mount it as an additional volume: Shell docker run --rm -p 8080:8080 --env-file .env \ -v "$PWD/my.md:/app/prompts/my.md" \ -v "$PWD/code.md:/app/prompts/code.md" \ ghcr.io/bronto-community/ai-sre-agent:step-7 Ask the same question again: Shell curl -s localhost:8080/invocations \ -d '{"prompt": "Customers say checkout got slow in the last 30 minutes. What is going on?"}' | jq -r .result Does it find the deploy? Does it open the release and pick the right commit? In our testing, it found the deploy marker and the commit, and then stopped. The release contains several commits, and the last hop, reading every one of them and figuring out which change explains the symptom, didn't happen on its own. If you experience the same, try to fix it yourself by refining your system prompts or by trying yet another model. When you're done, make rollback puts the healthy version back, and make down cleans everything up. If you want to give your agent a harder case, make psp-slow degrades the third-party payment processor. So, Buy or Build? Now you know what an AI SRE is and how it works, from the inside. That's a much better position from which to answer the question this post started with. If you buy, a lot of what hurt along the way becomes someone else's problem: confidently wrong answers that need endless prompt tuning, context windows overflowing with search results, wiring up every integration yourself. AWS DevOps Agent, for example, starts investigating when an alert fires and connects to the tools you likely already run, and you can still extend it with custom skills and your own tools. Take the four questions from the landscape article with you into every demo: Where does it get its data? Does it act, or only advise? Can you see its reasoning? What happens when it's wrong? After this lab, you'll know what good and bad answers to those look like. If you build, what you have now is a skeleton, and you can build more features towards a complete picture on top of it: Let alerts trigger it instead of curl.Run it on AgentCore Runtime instead of your laptop (agentcore deploy).Give it more tools: your cluster, your cloud APIs, your past incidents and postmortems.Observe the agent itself. Its own traces tell you which tools it called, how many tokens it burned, and where it went wrong, and that's the input you need to improve it. Either way, the data underneath matters more than the agent on top. Every improvement in this lab depended on the telemetry being there: a baseline from ten quiet minutes, latency and not just errors, a deployment.commit attribute that connects a span to a line of code. An agent can only reason about what your system tells it. That was true for humans long before agents showed up. So: what would your agent get wrong? The full lab, including the slides, the agent images for every step, and the storefront harness, is at github.com/bronto-community/ai-sre-lab. Content is CC BY 4.0; code is MIT. If you build something on top of it, I'd love to hear about it. I've shown how easy it is to build a simple AI SRE agent for fun. If you want to see what a production-ready one looks like, read our post on the Bronto blog, Introducing Bronto's Open-Source Alert Investigation Agent. It's an open-source agent that replicates the Alert Investigation feature in Bronto, and it's essentially our production version of this lab: a Bronto monitor triggers it instead of curl, it sticks to a constrained set of tools and reasoning paths instead of improvising, and it hands you a report with likely root causes and suggested next steps. The code is at bronto-community/bronto-alert-investigation-agent. More
Why AI Agents Need a Shared Enterprise Ontology

Why AI Agents Need a Shared Enterprise Ontology

By Josephine Eskaline Joyce, Ph.D DZone Core CORE
Agents Can Communicate, But Do They Understand Each Other? Two agents. Same customer ID. Different decisions. That is the semantic gap hiding inside most multi-agent enterprise deployments today — and protocols like MCP and A2A alone cannot close it (nor will they ever). AI agents are getting better at interacting through APIs, events, messages, and shared protocols. They can exchange data, invoke services, and coordinate tasks across enterprise systems. But communication doesn’t guarantee understanding. Imagine three agents dealing with the same customer request. One sees a customer as an account holder, another as a billing entity, and a third as a CRM contact. Data is exchanged, but each agent interprets it differently. This exposes a fundamental challenge in multi-agent enterprises: agents can exchange information without sharing the same meaning. Protocols define how information is exchanged. APIs define what data is exchanged. Neither defines what enterprise concepts such as customer, product, entitlement, incident, or approval actually mean. As organizations deploy specialized agents in sales, finance, ops, security, and engineering, semantic consistency is paramount. Agents need more than connectivity — they need a shared understanding of business concepts, relationships, and constraints. “Agent interoperability is not only a protocol problem; it is a semantic problem.” The Semantic Gap in Multi-Agent Enterprises In a multi-agent enterprise, different agents are built by different teams, trained on different data sets, connected to different systems, and designed around different business contexts. They may all exchange valid JSON messages, call the same APIs, or participate in the same workflow but still interpret the underlying business concepts differently. This is the semantic gap: what’s different between exchanging data and agreeing on what that data means? For example, the term customer may refer to a: CRM contact for a sales agentBilling account for a finance agentService subscriber for a support agentLegal entity for a compliance agent Similarly, priorities, incidents, owners, entitlements, or approvals may have different meanings in systems and domains. These differences are manageable when humans are involved because people infer context, ask clarifying questions, and reconcile ambiguity. Autonomous agents cannot do this reliably unless the semantics are explicitly represented. Without shared semantics, multi-agent systems can suffer from: Inconsistent interpretations of the same enterprise dataIncorrect agent-to-agent handoffsDuplicated or conflicting decisionsBrittle mappings between systemsGovernance rules that are applied differently across agents The challenge therefore moves beyond technical interoperability. Enterprises need semantic interoperability — a common model of concepts, relationships, constraints, and vocabulary that gives every participating agent the same frame of reference. That semantic foundation is provided by a shared enterprise ontology. What Is a Shared Enterprise Ontology? A shared enterprise ontology is a common semantic model that describes the concepts an enterprise uses (and how they are related to each other), as well as rules that give meaning to those concepts. It gives business knowledge a machine-readable format so that people, applications, and AI agents interpret enterprise information consistently. An ontology may define concepts such as: CustomerProductOrderServiceIncidentEntitlementEmployeeApproval More importantly, it defines how these concepts relate. For example: Customer → purchases → ProductCustomer → owns → AccountProduct → provides → ServiceIncident → affects → ServiceEmployee → approves → Request This is different from simply maintaining a common vocabulary. An ontology captures meaning, relationships, hierarchy, and constraints. A shared enterprise ontology thus becomes a semantic reference layer that agents can use to interpret data from different systems in the same way. Instead of each agent deciding what a customer, incident, or approval is, the enterprise defines those concepts once and reuses them across the enterprise. For multi-agent systems, this shared semantic model can become the common language that enables agents not only to exchange information but also to reason about it consistently. Why APIs, MCP, and A2A Are Not Enough on Their Own APIs, MCP, and A2A are examples of how interoperability problems are solved. Agents discover capabilities; they exchange messages; invoke tools; coordinate actions across systems. But they are more about connectivity and interaction than shared meaning. An API can have a field called customer_id. MCP can expose a tool to get customer data. An A2A interaction can send that information from one agent to another, but none of these mechanisms by themselves guarantees that every participating agent will interpret customer, account, service, or entitlement in the same way. This distinction matters. APIs specify how applications exchange data and invoke functions.MCP standardizes how models and agents access tools, resources, and context.A2A allows agents to communicate, delegate, and collaborate. These mechanisms provide technical interoperability, but semantic interoperability requires an additional layer: a common understanding of enterprise concepts and relationships. For example, two agents may exchange an incident object through an API or agent protocol, but one interprets an incident as a customer support ticket and the other an operational outage. Technically, they succeed in communication, but the decision made may still be wrong. An ontology for shared enterprise complements APIs, MCP, and A2A by specifying the semantic contract behind the information exchanged. In that model: APIs provide access.MCP provides tool and context interoperability.A2A provides agent collaboration.Ontology provides shared meaning. Together, these layers create a stronger foundation for reliable multi-agent enterprise systems. What Should an Agent-Aware Enterprise Ontology Describe? A traditional enterprise ontology describes business concepts (customers, products, services, orders, assets, and incidents). An agent-aware enterprise ontology should also go further by describing the agents that operate on those concepts. At minimum, it should capture five areas: Enterprise entities – customer, product, service, account, incident, policy, asset, employee.Relationships – owns, depends on, affects, approves, belongs to, consumes, manages.Agent capabilities – what an agent can search, analyze, create, modify, approve, or execute.Permissions and constraints – what an agent is allowed to access or change, under which conditions.Operational context – which system owns the data, its sensitivity, lifecycle state, provenance, and applicable governance rules. For example, the ontology could say that a Support Agent can read customer entitlement data; create an incident; recommend a resolution — but cannot approve a refund above some threshold. A Finance Agent can access billing records and authorize certain financial actions. This is what makes the ontology useful not only for understanding enterprise data but also for understanding who can act on that data, how they can act, and under what constraints. An agent-aware ontology can therefore answer questions like: What does this concept mean?Which agent owns this capability?What data can the agent access?Which actions are permitted?What policies apply before execution? This turns the ontology from a passive knowledge model into a semantic foundation for agent discovery, coordination, authorization, and governance. From Skill Discovery to Semantic Capability Discovery Most agent platforms can already discover skills or tools: search a catalog, inspect a description, and invoke a capability. The limitation is that skill discovery is usually done based on names, keywords, or interface descriptions. An agent may find a tool called create_ticket, but still not understand: What type of ticket it createsWhich business entity the ticket relates toWhat prerequisites applyWhat permissions are requiredWhether another capability is semantically equivalent This is where semantic capability discovery becomes important. Instead of asking only: “Which skill matches this request?” an agent can ask:“Which capability can perform this business action on this enterprise entity under these constraints?” For example, an ontology could describe that: RefundProcessingAgent → canPerform → IssueRefundIssueRefund → appliesTo → CustomerOrderIssueRefund → requires → ApprovedReturnIssueRefund → governedBy → RefundPolicy Now the agent isn’t finding a skill by name alone. It is finding a capability based on meaning, relationships, prerequisites, and policy context. This is especially useful when multiple agents expose overlapping capabilities (one agent may support refunds on retail orders, another on subscriptions, and another only for low-value transactions). Semantic metadata lets the orchestrating agent choose the right capability based on business context rather than just string matching. In this model, the ontology becomes a semantic capability graph connecting: Agent → Capability → Business Entity → Preconditions → Policies → Constraints This enables richer agent discovery, better routing, safer composition, and more reliable delegation across the enterprise. Ontology-Driven Agent Selection and Governance As enterprises deploy more specialized agents, choosing the right agent gets harder. A request may match several agents, but not every agent will have the same capability, authority, data access, or policy constraints. An enterprise ontology can make this selection more intelligent. Instead of routing a request based only on keywords or static configuration, an orchestrator can use semantic relationships such as: Agent → hasCapability → ApproveRefundApproveRefund → appliesTo → CustomerOrderAgent → authorizedFor → RegionCapability → constrainedBy → RefundPolicy When a task arrives, the system can evaluate not only which agent can perform it, but also: Whether the agent has the required capabilityWhether it can access the relevant enterprise dataWhether it is authorized for the requested actionWhich policies and constraints applyWhether human approval is required For example, two agents can both process refunds - one may be limited to transactions below a defined threshold, while another may handle high-value refunds but require managerial approval. The ontology provides the semantic context needed to route the request correctly. The same model can also support governance. Policies can be directly associated with business entities, capabilities, roles, and actions, so agents can reason about constraints before execution. This creates an ontology-driven decision flow: Request → Business Intent → Required Capability → Eligible Agents → Policy Evaluation → Selected Agent → Execution The result is more than smarter routing. It enables policy-aware, context-aware, and explainable agent selection across the enterprise. Building a Minimal Agent-Aware Enterprise Ontology AgentsCapabilitiesEnterprise entitiesPolicies and constraints Consider a simple refund-processing scenario. An enterprise has a support agent that can process refunds for customer orders, but that capability is governed by a refund policy. A minimal semantic model could look like this: This can be represented using RDF/OWL-style semantics. For example, using Turtle: Turtle @prefix ex: <http://example.com/enterprise#> ex:SupportAgent a ex:Agent ; ex:hasCapability ex:ProcessRefund ; ex:accesses ex:CustomerAccount ex:ProcessRefund a ex:Capability ; ex:appliesTo ex:CustomerOrder ; ex:governedBy ex:RefundPolicy . ex:RefundPolicy a ex:Policy ; ex:maxAutomaticRefund "500" Even this small ontology gives an orchestrating agent more information than a conventional tool catalog. Instead of discovering only process_refund(order_id), the agent can understand that the capability: Operates on a CustomerOrderIs provided by the SupportAgentDepends on access to customer informationIs governed by a specific refund policy The next step is to query the ontology. For example, an orchestrator could ask: SQL SELECT ?agent WHERE { ?agent ex:hasCapability ex:ProcessRefund . } The result identifies agents that semantically provide the required capability. A richer query could also include policy and business context: SQL SELECT ?agent ?policy WHERE { ?agent ex:hasCapability ?capability . ?capability ex:appliesTo ex:CustomerOrder . ?capability ex:governedBy ?policy . } This creates the foundation for semantic agent selection. The important point is that the ontology does not replace APIs, MCP tools, or agent protocols. It sits above them as a semantic decision layer. The ontology determines: What does the request mean?Which capability is required?Which agent provides it?Which enterprise entity is affected?Which policy applies? The execution layer can then use an API, MCP tool, or A2A interaction to perform the actual action. For a production environment, this minimal model can gradually expand to include concepts such as roles, data sensitivity, regions, approval thresholds, provenance, dependencies, risk levels, and human-approval requirements. The key is to start small. A handful of well-defined enterprise concepts and relationships can already move agent orchestration from simple tool matching toward semantic, policy-aware capability discovery. Where This Architecture Fits in the Emerging Agent Stack The ontology-driven architecture fits between the agent reasoning layer and the execution layer of the emerging enterprise agent stack. A typical enterprise agent stack may include: User or application layer – where requests originateAgent/orchestration layer – interprets intent and coordinates workSemantic layer – maps intent to enterprise concepts, capabilities, policies, and eligible agentsInteraction layer – MCP, APIs, A2A, events, and messagingEnterprise systems layer – CRM, ERP, ITSM, data platforms, and internal services The shared enterprise ontology belongs primarily in the semantic layer. Its role is not to replace the LLM, agent framework, MCP server, API gateway, or A2A protocol. Instead, it provides the semantic context needed before those components are used. This separation is important because each layer solves a different problem. The reasoning layer determines what the user is asking.The ontology layer determines what that request means within the enterprise.The orchestration layer determines which agent should handle it.The interaction layer determines how the selected agent or tool is invoked.The enterprise systems layer performs or records the actual business action. As agent ecosystems become more distributed, this semantic layer can act as connective tissue across otherwise independent agent platforms. Different agents may use different models, frameworks, protocols, or runtime environments while still relying on the same enterprise definitions of capabilities, entities, and policies. The architecture therefore adds a missing layer to the emerging agent stack: semantic interoperability between reasoning and execution. Challenges: Ontology Evolution, Ownership, Federation, and Governance A shared enterprise ontology improves semantic interoperability but also brings its own operational and governance problems. Enterprise concepts are not fixed: products change, organizational structures evolve, regulations shift, and new agents bring new capabilities; the ontology must therefore evolve without breaking existing consumers. Ontology Evolution In a federated ontology landscape, a change in one domain propagates to the entire semantic layer. Changing CustomerAccount in the Customer/CRM ontology, tightening the Entitlement definition in the Product ontology, or adding a new relationship between Incident and Risk in the IT and Operations domain can affect every agent that reasons over those concepts (sometimes not immediately visible). A database schema change usually only affects one system, whereas an ontology change can impact how a sales agent qualifies a lead, how a support agent resolves a ticket, how a finance agent authorizes a payment, and how a compliance agent evaluates a policy, all at the same time. The blast radius is wide because the semantic layer is shared. Ontology evolution therefore needs practices borrowed from API lifecycle management: Versioning – URI-based version identifiers on classes and relationships (for example, ex:v2/CustomerAccount) so that agents can declare which ontology version they depend on.Backward compatibility – additive changes are preferred; breaking changes require a deprecation period with the old and new definitions coexisting.Deprecation policies – a defined timeline for retiring concepts, with explicit successor mappings so consuming agents can migrate incrementally.Change impact analysis – before publishing a change, tooling should identify which agents, tools, and downstream applications reference the affected concepts.Validation before rollout – semantic regression tests that verify existing agent queries and capability graphs still resolve correctly against the new version. Agents should declare which ontology version they are using, and orchestrators should be able to route requests to agents that share a compatible semantic baseline. This prevents a situation where two agents with different ontology versions interpret the same concept differently — which is precisely the problem the ontology was introduced to solve. Ownership A central question is: who owns the meaning of enterprise concepts? The answer is rarely the team that builds the ontology infrastructure. A platform or architecture team may maintain tooling, graph store, versioning pipeline, and publishing process — but it shouldn’t define what Customer, Risk, Incident, Entitlement, or Approval mean independently of those concepts. Those definitions carry business, legal, and operational weight. Semantic ownership should be closer to the business domain. Customer/CRM ontology should belong to teams that run CRM processes and understand what a customer relationship means commercially; Finance ontology should be owned by finance and accounting because they know the difference between an Invoice, Payment, Cost Center, and Financial Account; Security ontology (Identity, Role, Permission, Policy, and Risk) should belong to security teams who understand the compliance implications of each concept. This model (central semantic governance with distributed domain ownership) is how platform engineering works at scale. The platform team sets standards, tooling, and alignment rules across domains. Domain teams set meaning within their area. Neither can override the other unilaterally. In practice, this means having ownership metadata on every ontology concept: who owns it (team), who is the subject-matter expert, and which review process applies when that definition needs to change. If not, the ontology becomes either a bottleneck controlled by one central team or an inconsistent patchwork where every team modifies concepts without coordination. Federation Trying to force all of this into one monolithic ontology isn’t practical. Governance bottleneck; slows down domain teams; model is too abstract to be useful for any specific domain. The diagram above shows a scalable alternative: federated ontology design. Each domain has its own ontology (Customer/CRM, Finance, Product, IT and Operations, Security and HR) — they have full autonomy over the concepts, relationships, and terminology in that domain. Ontologies for each domain link to an Enterprise Ontology Core shared by all domains via explicit semantic mappings and alignment relationships. The core contains cross-domain concepts, shared vocabularies, and governance and versioning rules that make federation work. This architecture also extends outward in two directions. On one side, the enterprise core aligns with external industry ontologies — schema.org for general web semantics, GS1 for product and supply chain standards, FIBO for financial industry terminology, SNOMED CT for healthcare — allowing the enterprise to adopt established standards rather than reinventing definitions. On the other side, partner and ecosystem ontologies for suppliers, regulatory bodies, and industry-specific frameworks can federate with the core through controlled extension points. The goal is not to eliminate domain-specific meaning. It is to make the relationships between those meanings explicit and machine-readable, so that an agent operating in the Finance domain and an agent operating in the Customer domain can each reason within their own semantic context while still communicating consistently when their work intersects. Federation also means that new domains can be onboarded incrementally. A new AI agent introduced for HR or procurement does not require a redesign of the entire ontology. It connects through the same alignment and mapping pattern, inheriting cross-domain concepts while adding its own domain vocabulary. Governance Because AI agents use ontology definitions to make autonomous operational decisions, semantic changes are no longer just documentation updates — they are configuration changes with real business consequences. Changing the definition of Entitlement may cause a support agent to deny services it should approve. Altering the Policy concept may cause a finance agent to bypass a control it previously enforced. The ontology is live operational infrastructure. Governance must therefore be treated with the same rigor applied to production software releases. At minimum, it should define: Who can propose changes – any domain team can propose additions or modifications within their domain, but cross-domain changes require joint review.Who approves semantic changes – changes to core enterprise concepts should require approval from the relevant domain owners and a central semantic governance board.How conflicts between domains are resolved – when two domain teams define the same concept differently (for example, both Finance and CRM define Account), there must be a defined arbitration process that results in either a canonical definition or an explicit mapping.How provenance is recorded – every concept should carry metadata about who introduced it, when, why, and which version it appeared in, so that an audit trail is always available.How policy-related concepts are reviewed – concepts that directly govern agent behavior, such as approval thresholds, permission levels, and compliance rules, should require review by legal, compliance, or risk teams before they are published.How changes are tested against existing agent workflows – before a semantic change is released, it should be validated against a test suite of representative agent queries and orchestration flows to confirm that existing behavior is not unintentionally altered. For high-impact concepts, ontology updates should require the same level of review as a production policy change or a regulated configuration update. The bar is not arbitrary: if an agent can take a financial action, restrict access, or escalate an incident based on an ontology definition, then that definition carries the same operational weight as the code or policy that governs the same action. Ultimately, the challenge is not just building a shared ontology. It is maintaining a trusted semantic contract across teams, systems, and agents as the enterprise continuously changes. The federated architecture distributes that responsibility appropriately — each domain team owns what they understand, the core team maintains coherence, and governance ensures that changes are deliberate, reversible, and traceable. “A shared ontology is only valuable if its meaning remains governed, traceable, and trustworthy as the enterprise evolves.” Conclusion In the multi-agent enterprise, communication is only the first step. True collaboration requires a shared understanding of business meaning. Protocols connect agents; ontologies align them. As AI ecosystems grow, semantic interoperability will become as important as technical interoperability. The challenge will no longer be whether agents can communicate, but whether they understand enterprise concepts well enough to collaborate safely, consistently, and effectively. A practical starting point is simple: model three things, your key business entities, the agents that operate on them, and the policies that govern those interactions. The ontology can evolve incrementally as the agent ecosystem grows. The future of enterprise AI will depend not only on smarter agents, but on agents that share a common understanding of the enterprise they serve. More
Adding Graph Retrieval to a RAG Pipeline With Python
Adding Graph Retrieval to a RAG Pipeline With Python
By Sriharsha Makineni
The Three Tiers of Agentic Incident Response: When to Trust AI Autonomy
The Three Tiers of Agentic Incident Response: When to Trust AI Autonomy
By Neel Shah
AI Agents Need an Identity Plane: Rethinking Enterprise Authorization for Autonomous Software
AI Agents Need an Identity Plane: Rethinking Enterprise Authorization for Autonomous Software
By Igboanugo David Ugochukwu DZone Core CORE
Genkit Flows as senro Steps: Extremely Powerful AI Pipelines in Go
Genkit Flows as senro Steps: Extremely Powerful AI Pipelines in Go
By Xavier Portilla Edo DZone Core CORE

More Articles

Member Spotlight: Shai Almong
Member Spotlight: Shai Almong

What does it take to build and sustain an open-source project for more than a decade? For Shai Almog, creator of Codename One, it takes a willingness to challenge conventional approaches to software development and a continued commitment to the developers who rely on the work. I recently caught up with Shai to discuss the inspiration behind Codename One, the realities of maintaining an open-source project, and his perspective on how the Java ecosystem has evolved over the years. Check it out below! 1. What is Codename One, and what originally inspired you to create it? Codename One is what we always wanted Java to be: write once, run anywhere, without accepting the lowest common denominator. It’s what you would get if Spring Boot, Flutter, Go, and Node.js had a baby that inherited the best qualities of each. You get Java’s mature ecosystem, Flutter-style control over the UI/portability, Go-like native binaries and footprint, and the full-stack client/server connection people associate with Node.js. With Codename One, one Java codebase can target iOS, Android, macOS, Windows on x64 and ARM64, Linux on x64 and ARM64 with either glibc or musl, and even the browser. A small self-contained desktop GUI application can be around 4–5 MB, with no JVM to install. Underneath, Codename One uses different pipelines for different targets. On our ahead-of-time compiled platforms, ParparVM analyzes the reachable Java bytecode, removes unused code, translates what remains to C, and hands the result to the platform’s native compiler. Codename One also provides the portability libraries, UI toolkit, tests, theming, and build system needed to handle the ugly platform-specific details. We don’t pretend every platform is identical. Our builder architecture assembles the native pieces each application needs, so developers can use platform-specific capabilities without fragmenting the entire codebase. The original inspiration was frustration with Java’s unfulfilled promise. Java gave us portability, but conventional Java application deployment became increasingly heavy. We wanted the language and ecosystem without requiring a large runtime or giving up native performance, platform access, or control of the UI. Now we’re taking that architecture to the backend. The goal is to bring the startup time, footprint, and native compilation work we developed for constrained devices into serverless and microservice workloads. I want Java developers to get the full-stack productivity they expect from Spring Boot, with deployment economics closer to Go and Node.js. 2. Codename One has been around for more than a decade. What has kept you motivated? The community is the biggest reason. People have built businesses and products on Codename One. When someone has trusted your work with something that important, walking away isn’t an option. For a long time, competing in this market was brutal. Our competitors were not normal startups. They were Google, Facebook, Microsoft, and companies with teams larger than our entire community. There was a limit to how much a small team could build, test, document, and support. Agentic development changed that equation. It lets us scale engineering work in a way that simply wasn’t possible a few years ago. We now maintain an enormous cross-platform test matrix and can move at a pace that is competitive with much larger organizations. On most feature dimensions, I would argue that we are already ahead of the big-name alternatives. We still have gaps. We’re a small team, and pretending otherwise would be ridiculous. But our size is also an advantage. The person reading a bug report can be the same person who changes the compiler. We can respond to developers in hours where a large company may need three teams, a quarterly roadmap, and permission from a product manager. The other reason is simpler: we love Java, and we use the competing tools. Every time we do, we run into one of the compromises Codename One was created to remove. 3. What’s one of the biggest misconceptions developers have about maintaining an open source project? That maintaining open source is mostly about accepting code contributions. A pull request looks like free work from the outside. For a maintainer, it can be the beginning of a large amount of work. Is the code correct? Does it meet the project’s standards? Does it work on every target? Who maintains it five years later? How do you tell a volunteer who spent a week on something that the design is wrong? How many rounds of changes can you reasonably ask them to make? We recently stopped accepting outside pull requests. AI wasn’t the reason. If anything, AI improved the baseline quality of many contributions. It also made some of them enormous, because generating another thousand lines became almost free while reviewing those thousand lines did not. The real problem was validation. Codename One currently has more than 60 GitHub Actions workflow files. A small framework change can affect Android, iOS, JavaScript, desktop, bytecode translation, native compilation, and screenshot baselines. That matrix represents roughly $15,000 worth of monthly CI capacity, which GitHub currently provides to us. At that point, validating and repairing an outside patch often took longer than implementing the confirmed change ourselves. So today we ask people to contribute through reproducible issues, benchmark counterexamples, design discussions, testing, and documentation feedback. Those contributions are often more valuable than code because they expose problems we cannot reproduce in our own environments. Also sorry for all the GitHub Actions downtime ;-) 4. You’ve watched Java and the broader development ecosystem evolve considerably. What changes have surprised you the most? The biggest surprise is that Java effectively surrendered the idea of being small. GraalVM is a remarkable piece of engineering, so we aren’t literally alone (although it's a very different project that hasn't really taken the same direction and hasn't taken it as far). But mainstream Java spent years treating bigness as evidence of seriousness: larger runtimes, larger frameworks, more dynamic machinery, longer warmups, and more memory. The industry began treating that weight as inevitable. I'm not against comprehensive/large; Codename One itself is HUGE... But not in runtime, just in API and scope. Most successful newer platforms moved in the opposite direction. Go made self-contained native binaries normal. Node.js made application deployment almost trivial. Mobile and serverless made startup time and memory consumption economically important again. Java largely responded by optimizing large systems rather than questioning why every system needed to be large. Modules are a good example. They help organize and encapsulate a large dependency graph. They do not eliminate that graph or turn it into a small, closed-world native program. Codename One puts a real Java GUI application or full server into a roughly 5 MB binary. What surprises me is not that this works. We have spent years proving that it works. What surprises me is how lonely that position still is. Java could be one of the best languages for small native applications and low-cost serverless workloads. Most of the ecosystem simply stopped trying to make that happen and let Go eat our lunch. 5. After spending so much of your career in technology, what do you enjoy doing when you completely step away from a screen? Completely step away from a screen? I’m probably the wrong person to ask. I just got back from Thailand with my kids. We had a wonderful time. I also had my laptop with me and was doing pull requests on the beach. This is what I enjoy doing. I don’t have an unwind button. Vacations often leave me more exhausted than work because sitting still doesn’t recharge me. If I have an idea and I can build it, fix it, or argue about it with another developer, that is how I unwind. Explore more of Shai's content here.

By Dominique Pugh
How an AI Software Factory Works: From Developer Intent to Production
How an AI Software Factory Works: From Developer Intent to Production

I got interested in the AI Software Factory concept because it asks a much bigger question than “How can I write code faster?” It asks how AI agents can help move an idea through the entire software development lifecycle, from planning and implementation to testing, deployment, monitoring, and learning. That difference matters. Coding assistants are already useful, but generating code is only one part of building software. I still need to understand the service, validate the change, get the right approvals, and keep production healthy. When I started researching how companies like Uber approach agent-driven development, the infrastructure behind the agents fascinated me more than the code generation itself. The real opportunity is not a collection of clever tools working independently. It is a coordinated system that gives those tools context, boundaries, and a reliable path to production. What I Mean by an AI Software Factory I think of an AI Software Factory as an assembly line for software, with specialized agents handling different jobs and humans controlling the decisions that carry meaningful risk. The factory starts with developer intent and ends with verified software running in production, with operational feedback returning to the beginning. The manufacturing analogy helps me explain it. A car factory does not rely on one machine to perform every task. Different stations have specific responsibilities, and the assembly line determines how work moves between them. Software needs the same kind of coordination. An AI Software Factory can include agents for planning, coding, review, testing, security, release verification, deployment, and monitoring. But the agents alone do not make it a factory. The operating layer needs to connect their work, enforce policies, and expose what happened at each stage. Uber was one of the examples that drew me into this topic. The research I explored described agents handling more than 70% of the company’s pull requests. What interested me most was the supporting infrastructure: agents need to understand systems, run validation, and help maintain software at scale, not simply produce changes. That is the distinction I keep coming back to. The factory is the system around the agents, not just the agents themselves. Faster Coding Does Not Automatically Mean Faster Delivery I am excited about tools like GitHub Copilot and Cursor, but I do not confuse faster syntax generation with faster software delivery. If coding gets dramatically faster while everything after it stays the same, the bottleneck simply moves downstream. More code arrives in testing. More pull requests land in review queues. Senior engineers spend more time evaluating changes, and deployment processes still have the same constraints. The team can feel incredibly productive while the amount of reliable software reaching production barely changes. This is the problem an AI Software Factory is supposed to address. Optimizing one stage is not enough when delivery depends on the whole pipeline. I find it useful to separate two questions: How quickly can I generate a change? Coding assistants can help significantly here.How reliably can I deliver that change? This depends on context, testing, review, approvals, deployment, and operations. The second question is where the broader architecture becomes important. A bigger pile of generated code is not the outcome I want. I want useful changes that pass validation, fit the system, and reach production safely. For me, the value of an AI Software Factory lies in improving that complete flow rather than celebrating raw code volume. Governance Is What Separates a Factory from Agentic Chaos There is a risky version of this story: I add several autonomous agents to the development lifecycle, give them access to tools, and assume the result will be efficient. Without a common operating layer, that can become agentic chaos. When something breaks, I need answers. Which agent changed the service? What requirements and business context did it use? Which checks passed? Who approved the release? If those answers are missing, faster execution has made the system harder to control. A governed AI Software Factory makes that information visible and makes its boundaries enforceable. I want explicit inputs, defined output destinations, security rules, and approval gates that agents cannot quietly bypass. Earlier approaches such as infrastructure as code and GitOps helped automate infrastructure and deployment workflows. Agent-driven development extends automation into planning and implementation, which makes governance across the lifecycle even more important. My preferred division of responsibility is straightforward: agents perform focused work, while humans define the rules and authorize consequential decisions. An agent should not decide what business risk is acceptable. That is why I do not see human governance as the opposite of automation. In an AI Software Factory, governance is what makes broader automation usable. I Start with Service Context, Then Route Work to Specialists Before generating a single line of code, I want the factory to fetch the service context. That means knowing who owns the service, which systems it depends on, what requirements apply, and which policies govern the change. Without that foundation, an agent can produce an implementation that looks reasonable in isolation but does not fit the organization’s actual system. The initial context is not administrative overhead. It is part of the engineering input. I also would not hand an entire feature request to one giant prompt and ask a single model to do everything. Planning, implementation, review, and operations are different responsibilities. Mixing them together makes it harder to isolate mistakes and understand the output. In an AI Software Factory, I prefer a constrained handoff: Gather context: Resolve ownership, dependencies, requirements, and applicable rules.Create a plan: A planning agent maps the architecture and implementation approach.Approve the plan: A human checks the proposed direction before implementation begins.Build the change: A coding agent executes the approved, bounded plan.Validate the output: Review, testing, and security responsibilities follow through their own stages. These boundaries help contain errors rather than allowing an unsupported assumption to travel unchecked through every phase. They do not make hallucinations impossible, but they create clearer places to detect and stop them. That narrow division of labor is one of the most useful design ideas in the AI Software Factory model. CI and Human Approval Must Be Real Gates Once generated code moves forward, I still want the traditional continuous integration machinery to do its job. AI does not remove the need for builds, automated tests, or security validation. My rule is simple: a failed CI build blocks progression toward deployment. It should not become an advisory message that an agent can ignore because it thinks the change is probably fine. An AI Software Factory needs those checks wired into workflow control. For background on the underlying practice, GitHub’s continuous integration documentation explains how automated builds and tests support validation of repository changes. Passing CI is necessary, but it is not the same as permission to release. After automated validation, the workflow reaches a human review gate and pauses until someone explicitly authorizes the next step. At that gate, I want to evaluate architectural integrity, release readiness, and acceptable risk. The emphasis shifts away from spending all my attention on routine syntax checks and toward the questions that require engineering judgment. That does not mean I stop examining code when necessary. It means the factory should handle repetitive validation so that my review can focus on whether the change belongs in the system and whether it is safe to proceed. For me, an AI Software Factory protects speed by putting human judgment exactly where the stakes justify it. The Factory Continues After Deployment I do not consider deployment the finish line. Once a human approves the release, a delivery agent can move the change into production, but the workflow needs to continue into operations. A monitoring agent then checks live service health and looks for degradation after deployment. If health drops, incident response should follow predefined rules rather than depend on an agent improvising a recovery strategy. In the AI Software Factory model I explored, those responses include creating incident tickets, notifying the responsible team in Slack, and triggering predefined rollbacks. Service ownership and incident policies matter here just as much as they did during planning. The final piece is the feedback loop. Production signals and deployment outcomes need to flow back into the service context used for future work. I find this especially important because a pipeline that only moves forward is still largely a static automation sequence. A pipeline that brings operational results back to its starting point can improve the information available for the next plan. When I describe an AI Software Factory as learning, this is what I mean: deployment results and live system behavior inform subsequent work. I am not suggesting that agents should rewrite their own policies or expand their authority. The feedback improves context. Humans still own the boundaries. A Practical AI Software Factory Playground With Port To make the architecture concrete, I used a playground built around a customer support API. The developer intent was to build an API that summarizes tickets, stores conversation history, exposes REST endpoints, and can be safely deployed to production. I like this example because the request is more than a coding task. It includes functionality, persistence, an interface, and a delivery expectation. An AI Software Factory has to carry all of that intent through the lifecycle. In the playground, Port sits above the factory floor as the agentic SDLC layer. It supplies organizational context and connects agent and skill registries, workflows, policies, guardrails, and human gates. The visible stages run from intent and context through planning, building, reviewing, testing, security, release verification, deployment, post-deployment verification, and monitoring. An agent console and activity log expose the sequence of actions. When I ran the demo, the workflow encountered a human gate after planning and another before deployment. After approval, it continued through post-deployment verification and monitoring. The completed run showed 10 agents completed, 13 checks passed, two human gates, and a low release-risk indicator. I treat those numbers as the outcome of that illustrative run, not evidence that every release will be low risk. The playground demonstrates the coordination pattern; it is not proof that a real customer support API has been built and validated in a production environment. What I find useful is the visibility. I can see which specialist acted, where approval happened, and how work progressed through the AI Software Factory. The Guardrails I Want Before Giving Agents More Authority The factory floor is interesting, but I pay just as much attention to its guardrails. A workflow can look beautifully organized and still be unsafe if its permissions and release conditions are too loose. The playground’s AI Software Factory includes a concrete set of controls: Least privilege access to toolsNo secrets committed to codeDependency and license scanningStatic application security testing and injection-related checksRequired tests before releaseHuman approval for productionAn audit trail for every agent actionRollback support and post-deployment verification I do not think of these as decorations around the workflow. They determine what an agent may do, what evidence is required, and when work must stop. Least privilege limits tool access. Secret and dependency checks constrain what can enter the codebase. Audit trails help explain actions later. Production approval keeps release authority with a human, while rollback and verification extend safety beyond the initial deployment. For an AI Software Factory to be useful, these controls need to participate in execution, not just appear in a checklist. I want the orchestrator to enforce them consistently across stages. That is also why centralized service context matters. The factory needs to know the service owner, its policies, and the expected incident response before it can route work responsibly. How I Would Start Without Automating Everything on Day One My instinct as a developer is to connect the entire pipeline immediately. Planning agents, coding agents, deployment agents, monitoring agents: the possibilities are exciting. But I would resist that instinct when introducing an AI Software Factory to a team. I would begin with one small, reliable path: requirements, coding, CI, and a human review gate. The initial workflow should stop there. Before adding deployment authority, I want confidence that the inputs are clear, the checks block failures, and the approval boundary behaves as expected. Only after that loop is stable would I expand into deployment and monitoring orchestration. The point is not to have the most agents on day one. It is to establish a trustworthy handoff and then extend it. This changes how I think about the developer’s role. I am not just writing implementation details. I am defining system rules, approval criteria, service context, and the structure through which automated work moves. That is what excites me most about an AI Software Factory. The goal is not to remove engineering judgment. It is to free more of my time for that judgment by coordinating repetitive work around it. I want a factory that ships reliable software, explains its actions, and keeps meaningful decisions accountable to people. Faster code generation is useful. A governed delivery system is the bigger opportunity.

By Pavan Belagatti DZone Core CORE
ScyllaDB in Enterprise Java: High-Performance NoSQL With Eclipse JNoSQL
ScyllaDB in Enterprise Java: High-Performance NoSQL With Eclipse JNoSQL

Modern enterprise applications must process increasing data volumes while guaranteeing low latency, high availability, and predictable performance. While relational databases are essential for transactional workloads and complex queries, some applications do not fit the relational model. Systems that manage millions of user engagements, financial events, or IoT data often need a persistence solution built for distributed environments and horizontal scalability. In these cases, ScyllaDB affords a compelling option for enterprise architects. ScyllaDB is a high-performance, distributed NoSQL database based on the wide-column data model. Inspired by Apache Cassandra and compatible with its query language (CQL), ScyllaDB uses a shard-per-core architecture to maximize hardware efficiency and preserve consistent performance under intensive workloads. When should ScyllaDB be chosen over traditional relational databases or other NoSQL options? This article examines its column-family model, compares it with Cassandra, and shows how to integrate ScyllaDB into a Java enterprise application. Modern enterprise applications require more than basic data storage. They need high throughput, low latency, and horizontal scalability to manage complex workloads. ScyllaDB is a distributed, high-performance NoSQL database based on the wide-column data model, designed for applications that require predictable performance at scale. Companies such as Discord, Expedia, and Comcast use ScyllaDB in real-world scenarios, including messaging platforms, real-time applications, and high-volume data processing. ScyllaDB is especially well-suited for event-driven architectures, IoT platforms, financial systems, and applications with demanding read-and-write workloads. ScyllaDB uses a schema-based approach with Cassandra Query Language (CQL), which features SQL-like syntax, including statements such as CREATE TABLE, INSERT, and SELECT. Beyond primitive types, ScyllaDB supports collections like List, Set, and Map, as well as User-Defined Types (UDTs) for modeling complex data. This flexibility involves trade-offs: unlike relational databases, ScyllaDB uses query-driven data modeling, limits queries on arbitrary fields, and requires careful use of secondary indexes. This article will examine ScyllaDB’s column-family architecture, compare it with Apache Cassandra, review its capabilities and limitations, and show how to integrate ScyllaDB into enterprise Java applications. Setting Up ScyllaDB With Eclipse JNoSQL ScyllaDB supports deployment in cloud environments, on-premises data centers, and local development setups. Teams can select either self-managed or managed cloud services based on their infrastructure needs. In this tutorial, we will use a single ScyllaDB instance with Docker to focus on the data model and Java integration, avoiding the complexity of a distributed cluster. To start ScyllaDB locally, execute the following command: Shell docker run -d --name scylladb-instance -p 9042:9042 scylladb/scylla This command launches a ScyllaDB container and exposes port 9042, the default for Cassandra Query Language (CQL) connections. After initialization, the database is ready to accept application requests. Integrating ScyllaDB With Eclipse JNoSQL Next, determine how the Java application will connect to ScyllaDB. Starting with Eclipse JNoSQL 1.1.19, a dedicated driver provides native ScyllaDB support, enabling standard SQL operations alongside ScyllaDB-specific features. In this example, we use a Java application with Eclipse MicroProfile Config, CDI, and Jakarta JSON Processing. These technologies are common in MicroProfile and Jakarta EE environments, which helps with integration. First, add the ScyllaDB dependency to your Maven project: XML <dependency> <groupId>org.eclipse.jnosql.databases</groupId> <artifactId>jnosql-scylladb</artifactId> <version>${jnosql.version}</version> </dependency> Set the ${jnosql.version} property to Eclipse JNoSQL 1.1.19 or a compatible newer version. Configuring the Database Schema ScyllaDB uses a schema-based approach similar to Apache Cassandra. Developers define keyspaces, tables, column types, and user-defined types explicitly with CQL. In this example, we create a keyspace named developers, two tables for people and movies, and several user-defined types. Eclipse JNoSQL supports providing initialization queries through MicroProfile Config. Add the following properties to META-INF/microprofile-config.properties: Properties files jnosql.column.database=developers jnosql.scylladb.query.1=CREATE KEYSPACE IF NOT EXISTS developers WITH replication = {'class': 'NetworkTopologyStrategy', 'replication_factor': 1}; jnosql.scylladb.query.2=CREATE COLUMNFAMILY IF NOT EXISTS developers.Person (id bigint PRIMARY KEY, name text, phones list<text>); jnosql.scylladb.query.3=CREATE TYPE IF NOT EXISTS developers.director (name text, movies set<text>); jnosql.scylladb.query.4=CREATE COLUMNFAMILY IF NOT EXISTS developers.Movie (name text PRIMARY KEY, age int, director FROZEN<director>); jnosql.scylladb.query.5=CREATE INDEX IF NOT EXISTS ageIndex ON developers.Movie(age); jnosql.scylladb.query.6=CREATE TYPE IF NOT EXISTS developers.money (currency text, amount decimal); jnosql.scylladb.query.7=CREATE TYPE IF NOT EXISTS developers.headquarter (city text, country text); This configuration highlights multiple key points. CQL is similar to SQL, using statements likeCREATE, explicit types, and primary keys. ScyllaDB supports collections such as list and set, allowing you to represent multiple values without extra tables. User-defined types (UDTs) group related attributes into structured objects, such as the director type. The Movie table uses FROZEN<director>. In CQL, FROZEN stores the UDT as a single value, preventing independent modification of individual fields. We also created a secondary index on the age column. Although ScyllaDB supports secondary indexes, their effectiveness depends on the workload, cardinality, and query types. Do not treat them as equivalent to relational database indexes or as a substitute for proper partition-key design. Defining database schemas through application configuration works for tutorials but isn't recommended for production. Manage schema evolution separately using version-controlled migration tools such as Liquibase, which supports ScyllaDB via its Cassandra extension. This approach separates application deployment from schema changes and improves control over database updates. Mapping Entities and User-Defined Types With the database schema defined, we can create the Java models. We start with Movie, representing a ScyllaDB entity, and Director, representing a user-defined type. Java @Entity("Movie") public class Movie { @Id("name") private String name; @Column private Integer age; @Column(udt = "director") private Director director; } public class Director { @Column private String name; @Column private Set<String> movies; } The Movie entity uses the standard Jakarta NoSQL mapping approach with @Entity, @Id, and @Column. The director attribute maps to a ScyllaDB user-defined type via @Column(udt = "director"). During serialization and deserialization, Eclipse JNoSQL uses the ScyllaDB driver to convert between the Java object and its CQL UDT representation. A UDT is similar to a structured JSON object that groups related fields, but it's a strongly typed CQL value defined in the schema. The Set<String> field in Director demonstrates ScyllaDB’s native support for collections, allowing you to model a director’s movie titles without a separate relational table. Next, we will define the Person entity: Java @Entity("Person") public class Person { @Id("id") private long id; @Column private String name; @Column private List<String> phones; } The phones attribute demonstrates another collection type ScyllaDB supports: list<text>. Eclipse JNoSQL maps Java List<String> directly to the corresponding CQL collection without extra converters or custom serialization. These examples show how ScyllaDB combines a schema-based model with support for structured types and collections. Eclipse JNoSQL offers familiar mapping annotations, permitting applications to use ScyllaDB’s native features. However, object mapping convenience does not replace the need for query-driven data modeling. In distributed databases like ScyllaDB, partition keys, access patterns, and data distribution remain essential architectural considerations. Running the Application: Jakarta NoSQL and Jakarta Data Once you configure the database and map entities, the next step is to interact with ScyllaDB from Java. Eclipse JNoSQL provides two complementary approaches: Jakarta NoSQL, a template-based API for NoSQL databases, and Jakarta Data, which uses the Repository pattern. Both integrate with CDI, letting developers inject persistence components into applications running on either Java SE with CDI or Jakarta EE application servers. Jakarta NoSQL: The Template Approach The first approach uses Jakarta NoSQL with the ColumnTemplate interface, offering a consistent API for wide-column databases. Developers can insert, update, delete, and retrieve entities without interacting directly with the database driver. Java @Inject ColumnTemplate template; For example, you can create a Person entity, insert it, and retrieve it using a fluent query API: Java Person person = Person.builder() .withId(1L) .withName("Ada Lovelace") .withPhones(List.of("234", "432")) .build(); template.insert(person); Optional<Person> result = template.select(Person.class) .where("id").eq(1L) .singleResult(); The entity mapping automatically handles the Java List<String> and its corresponding CQL collection, so you don't need extra serialization code to persist phone numbers. The Template approach is useful when developers need explicit control over persistence operations, want to build queries programmatically, or prefer a consistent API across NoSQL implementations. Jakarta Data: The Repository Approach The second approach uses Jakarta Data and applies the Repository pattern to simplify persistence. Developers define an interface, and Eclipse JNoSQL supplies the implementation. Consider our PersonRepository: Java @Repository public interface PersonRepository extends CrudRepository<Person, Long> { } This interface provides standard CRUD operations, such as save, findById, and deleteById. The repository can be injected with CDI and used directly in the application: Java @Inject PersonRepository repository; repository.save(person); Optional<Person> result = repository.findById(1L); Jakarta Data also supports query methods derived from method names, allowing developers to express queries without writing implementations. MovieRepository demonstrates this capability and exposes ScyllaDB-specific functionality through its dedicated interface: Java @Repository public interface MovieRepository extends ScyllaDBRepository<Movie, String> { List<Movie> findByAge(Integer age); @CQL("SELECT * FROM developers.Movie") List<Movie> findAllQuery(); } The findByAge method demonstrates query derivation, while the @CQL annotation lets you execute native Cassandra Query Language statements. In this example, the age column has a secondary index, so you can query movies by age without using the partition key. However, query derivation does not remove ScyllaDB’s querying restrictions. Repository methods must match queries supported by the schema and indexes. Retrieving all records is useful for demonstrations but should be used cautiously in production with large datasets. Beyond Standard CRUD Operations While Jakarta NoSQL and Jakarta Data simplify common persistence operations, ScyllaDB integration extends beyond standard repositories and templates. Eclipse JNoSQL offers ScyllaDBTemplate, a specialized implementation that exposes database-specific features while maintaining the familiar Java programming model. With this integration, developers can: Control consistency levels: Specify Cassandra-compatible consistency levels, such as ConsistencyLevel.ONE, to balance consistency, latency, and availability.Execute native CQL: Run database-specific queries directly when standard abstractions are insufficient, including those using ScyllaDB-specific features.Work with structured data: Map collections such as List and Set, as well as user-defined types, without manually converting Java objects to CQL values.Choose the right abstraction: Use the vendor-neutral ColumnTemplate for general persistence, or ScyllaDBTemplate for additional database control. For instance, the specialized template supports an operation such as: Java scyllaDBTemplate.save(person, ConsistencyLevel.ONE); A key architectural feature of Eclipse JNoSQL is that abstraction does not mean losing access to database-specific capabilities. Developers can use standardized APIs for common operations and still access native functionality when needed. Jakarta NoSQL and Jakarta Data complement each other, offering different ways to organize persistence logic. The Template approach provides explicit, flexible interaction, while the Repository approach emphasizes declarative data access and separation of concerns. Both benefit from CDI integration, allowing you to use the same persistence components in standalone Java applications, microservices, or Jakarta EE environments. The choice depends on the application’s architectural requirements, but the underlying ScyllaDB capabilities and data-modeling constraints remain the same. Conclusion ScyllaDB is a strong choice for enterprise architects, especially for applications that demand high throughput, consistent latency, and horizontal scalability. Its schema-based wide-column model, support for collections and user-defined types, and compatibility with Cassandra Query Language provide a familiar and robust solution for distributed data persistence. However, to make informed architectural decisions, it's important to understand its query limitations, secondary indexes, and data-modeling trade-offs. Source: https://github.com/JNOSQL/demos-se#scylla.

By Otavio Santana DZone Core CORE
Avoiding AI Debt as a Team
Avoiding AI Debt as a Team

TL; DR: Reusing Agile Artifacts to Avoid Accumulating AI Debt In this video from the 76th Hands-on Agile Meetup, I walk you through the A3 Delegation System and show how it helps avoid AI debt by borrowing artifacts and practices from Agile, such as the Definition of Done and Retrospectives. What the Video on the A3 Delegation System Addresses AI Debt and the Delegation Lifecycle: The delegation, documentation, and ownership failures you already know from Agile reappear the moment an AI workflow runs unattended and undocumented, and the six-stage Delegation Lifecycle gives you a place to catch each one before it reaches a customer.The A3 Framework, Assist, Automate, Avoid: Every task lands in one of three boxes, Assist (AI drafts, you own the outcome), Automate (AI executes on explicit rules with an audit cadence), or Avoid (work that stays human because failure damages trust), and the real risk is Assist quietly drifting into Automate because the output has looked fine for four weeks.Model Routing by Cost Tier: Good enough is a deliberate decision with an owner, not a default, so you match each task class to local, mid-tier hosted, or frontier models by cost and stakes, and you keep proprietary process data out of the big labs because that data is your actual moat.The AI Definition of Done per Task Class: One Definition of Done for everything works in Scrum, but breaks under AI, so you write a separate one for each task class across four levels: verification, provenance disclosure, data hygiene, and sufficiency tier.The Delegation Audit and Governance Trail: Drift happens slowly, and approving a draft in four seconds is rubber-stamping, not reviewing, so you run a monthly audit for output drift, sufficiency drift, reversibility, and review creep, and the artifacts you create along the way answer any CFO, CTO, or procurement due diligence question without extra reporting work.

By Stefan Wolpers DZone Core CORE
Running a Self-Hosted WAF in Production: Practical Lessons From SafeLine
Running a Self-Hosted WAF in Production: Practical Lessons From SafeLine

Why Put a WAF in Front of the Application? Web application firewalls are often introduced as a list of security features: SQL injection protection, cross-site scripting detection, bot mitigation, rate limiting, and access control. Those capabilities matter, but they are only part of the operational problem. The more difficult questions usually appear after deployment: Where should the WAF sit in the traffic path?How do we avoid exposing the origin server?How do we tune blocking rules without interrupting legitimate users?How should security events be investigated?What happens when the application has unusual request formats?How much control do we retain when using a self-hosted solution? This article describes how SafeLine can be used as a self-hosted Web Application Firewall and reverse proxy. It focuses on the deployment model, request-processing flow, practical configuration, and operational lessons that matter when protecting real web applications. The WAF's Position in the Architecture A WAF operates at the HTTP layer. It can inspect requests before they reach the application and evaluate information such as: HTTP methodsURLs and query parametersRequest bodiesHeaders and cookiesSource IP addressesRequest frequency and behavioral patterns This position gives the WAF visibility that a network firewall does not have. A network firewall can allow or deny traffic based on addresses and ports, but it usually cannot determine whether a request contains an SQL injection payload or whether a client is repeatedly probing an administrative endpoint. The WAF also provides a safety boundary around applications that may contain unknown vulnerabilities. It does not replace secure coding, dependency updates, authentication, or database protections. It adds another enforcement point between an untrusted client and the application. The origin application should accept traffic only from the SafeLine host or from a trusted upstream load balancer. If users can bypass the WAF and connect directly to the origin, an attacker can simply avoid the protection layer. Figure 1. SafeLine sits between internet clients and the protected application. The origin should accept traffic only from a trusted SafeLine host. SafeLine's Deployment Model SafeLine is designed to run in a self-hosted environment. It acts as a reverse proxy, receives traffic for configured applications, analyzes the requests, and forwards acceptable requests to the configured upstream service. This deployment model is useful for teams that need: Control over where traffic and logs are processedA WAF that can run on their own infrastructureA relatively simple installation processProtection for applications deployed on virtual machines, bare-metal servers, or private infrastructureA management interface for applications, policies, and security events SafeLine can be installed on a Linux server using Docker-based deployment. Installation can be completed with a single command: Shell bash -c "$(curl -fsSLk https://waf.chaitin.com/release/latest/manager.sh)" -- --en Full guide to install SafeLine: https://ly.cyberserval.tech/b3QLLSJ. After installation, the administrator opens the management console, completes the initial setup, and adds the application that needs protection. Figure 2. Add the application you want to protect in the SafeLine management console. For a production deployment, treat the installation host as a security-sensitive system. At minimum, the host should have: A supported Linux distributionDocker installed and maintainedPersistent storage for configuration and logsRestricted access to the management interfaceA backup strategy for configurationMonitoring for CPU, memory, disk, and network usage The management interface should not be exposed broadly to the public internet. Restricting it by firewall policy, VPN, or an administrative allowlist reduces the attack surface. How a Request Is Processed When a request reaches SafeLine, it passes through several conceptual stages. First, SafeLine identifies the application and routing configuration. It determines which upstream service should receive the request. Next, it inspects the request structure. Depending on the request, this may include the URL, parameters, headers, cookies, and body content. The detection engine then evaluates whether the request resembles an attack. Common categories include: SQL injectionCross-site scriptingCommand injectionPath traversalFile inclusionMalicious scannersAutomated abuseExcessive request rates Finally, SafeLine applies the configured action. The request may be allowed and forwarded, logged for observation, challenged, rate limited, or blocked. Figure 3. HTTP Request Processing Flow in SafeLine A request can be observed before enforcement is enabled, allowing the team to identify legitimate traffic that needs a narrow exception. A team should not begin by blocking every suspicious request in a production environment without understanding the application's normal traffic. A safer rollout sequence is: Put the application behind the WAF.Observe requests and security events.Identify legitimate traffic that may look unusual.Adjust application-specific rules.Enable blocking for high-confidence attack categories.Continue reviewing events after enforcement begins. Deployment Example Assume the application listens on port 8080 and the public domain is app.example.com. The basic workflow is: Install SafeLine on a Linux host.Point the DNS record for app.example.com to the SafeLine host.Add the domain as a protected application.Configure HTTPS and certificates.Configure the upstream as http://application-host:8080.Restrict direct access to the application host.Verify that normal requests reach the application.Send controlled test requests and confirm that events appear in the dashboard. The most important network change is often the origin restriction. The application server should no longer accept arbitrary internet traffic. For example, its firewall can permit the application port only from the SafeLine server's private IP address. This is easy to overlook because the WAF may appear to be working even while the origin remains publicly reachable. Tuning Without Creating an Availability Problem A WAF cannot understand every business rule automatically. Some applications intentionally accept content that resembles an attack payload. Examples include: Security testing platformsDeveloper tools that accept code as inputSearch features that process special charactersAPIs that receive structured expressionsCMS systems with complex administrative requests The correct response to a false positive is usually a narrow exception. Avoid disabling protection for the entire application when only one endpoint or parameter requires special handling. Figure 4. SafeLine’s Custom Allow Rule and Deny Rule Configuration Interface False-positive handling should move from an observed event to the narrowest possible exception. For example, an exception for /api/search and one request parameter is easier to review than a global allowlist for the source IP. Narrow rules also reduce the chance that an attacker can reuse the exception against other endpoints. The application team should be involved in this process. Security events are often difficult to interpret without knowing what the endpoint is supposed to accept. Rate Limiting and Anti-Bot Challenge Application-layer attacks are not limited to SQL injection or cross-site scripting. Automated traffic can also consume resources, abuse login forms, scrape content, and repeatedly call expensive API endpoints. SafeLine provides two complementary controls for this type of traffic: Rate Limiting and Anti-bot Challenge. Rate limiting restricts how frequently a client can access an endpoint. It is useful for protecting login pages, password reset flows, search APIs, file upload endpoints, report-generation APIs, public APIs, and administrative paths. Anti-bot challenges address a different problem. Instead of relying only on request frequency, they challenge clients that exhibit suspicious automated behavior. Legitimate users can usually complete the challenge, while basic scripts, crawlers, and other automated tools may be unable to proceed. These controls are most effective when applied according to the behavior and sensitivity of each endpoint. A global request limit may block legitimate users, especially when many users share the same corporate or mobile network address. A challenge may also be inappropriate for machine-to-machine APIs that cannot interact with a browser-based verification flow. When configuring rate limits and anti-bot protection, consider: Requests per IP addressRequests per account or sessionRequests per endpointBurst behaviorResponse codesLegitimate client retry behaviorWhether the endpoint is accessed by browsers, mobile clients, or APIsGeographic and network distributionWhether the client can complete an interactive challenge A useful operational metric is not simply the number of blocked requests or triggered challenges. It is the relationship between automated traffic and application health. If an increase in blocked requests and challenged sessions coincides with lower application latency, fewer authentication failures, and reduced load on expensive endpoints, the controls may be addressing a real operational problem. The recommended approach is to begin in observation mode, identify the endpoints receiving suspicious automated traffic, and then apply the narrowest suitable control. Rate limiting is usually appropriate for high-frequency access, while anti-bot challenges are more suitable when the goal is to distinguish normal users from automated clients. Investigating Security Events A WAF is most useful when its events can support an investigation. For each event, the operations team should be able to answer: Which application was targeted?Which endpoint received the request?What type of attack was detected?When did the activity begin?Was the request blocked, challenged, or only logged?Did the source continue after the first response?Did the application generate errors?Are other endpoints or applications seeing the same pattern? SafeLine's management interface provides a central place to review protection events and traffic activity. It can help teams distinguish isolated scans from persistent automated activity. Figure 5. SafeLine’s Security Posture Dashboard However, WAF logs should be correlated with application and infrastructure logs. A blocked request may be useful evidence, but it does not prove that the application was compromised. Likewise, an allowed request does not prove that the application is safe. For a complete incident review, correlate WAF events with web server logs, application logs, authentication logs, and host and network telemetry. SafeLine live demo: https://ly.cyberserval.tech/i61INy7. SafeLine Pro Annual License Promotion SafeLine has reached more than 700,000 installations. That number is useful because it reflects broad adoption across different environments, application types, and operational constraints. To mark this milestone, the SafeLine team is offering a customer feedback discount for a Pro annual license. Figure 6. SafeLine 700K Installs Promotion Campaign The promotion details, eligibility requirements, validity period, and redemption process should be confirmed on the campaign page: https://ly.cyberserval.tech/DZAGbHo. Final Takeaways A WAF is not valuable simply because it has a long feature list. It is valuable when it can be placed correctly in the request path, tuned for the application, monitored by the operations team, and maintained as part of the broader security architecture. SafeLine's self-hosted reverse-proxy model makes it practical to introduce application-layer protection without redesigning the application itself. The most important implementation lessons are: Ensure all public traffic passes through the WAF.Restrict direct access to the origin server.Start with observation before enabling aggressive blocking.Handle false positives with narrow, reviewable exceptions.Apply rate limits to specific high-risk endpoints.Correlate WAF events with application and infrastructure logs.Continue using secure development practices alongside the WAF.Evaluate commercial licensing based on operational requirements rather than feature count alone.

By Carrie Luo
Testing Java and Spring Applications With Kotlin: Low-Risk to Adoption
Testing Java and Spring Applications With Kotlin: Low-Risk to Adoption

Teams considering or committed to adopting Kotlin often plan a migration as a series of rewrites, swapping Java services for Kotlin counterparts one microservice at a time. In practice, that kind of plan tends to stall before it produces any shipped code. Kotlin's role in the JVM ecosystem is expanding because it doesn't require that kind of commitment. Companies such as Netflix, Adobe, and DoorDash have adopted it incrementally, alongside existing JVM systems, rather than through a full rewrite. The lowest-risk starting point is tests. They give you Kotlin's language benefits, including null-safety, data classes, and expressive lambdas, without touching anything that runs in production. This guide walks through adding Kotlin to an existing Java + Spring Boot project for test sources only. You'll configure Gradle, write JUnit 5 tests in Kotlin, use MockK for Kotlin-native mocking, and apply Kotest assertions to improve test readability. Your production services don't change, and your team starts experiencing Kotlin in a bounded, no-risk context. Why Testing Is the Safest Place to Introduce Kotlin Kotlin test sources are isolated from the production application artifact and don't require Kotlin in the production runtime. They can still affect CI and test infrastructure, but leave production services untouched. That separation from production makes tests the right entry point for a language your team hasn't yet committed to. Introducing Kotlin in test sources means that your existing Java services remain completely unchanged, with no refactoring, no migration risk, and no regressions in production paths. It also allows your team to experiment with real Kotlin syntax on tests they'd be writing anyway, rather than on contrived examples. Developer buy-in builds faster when engineers are writing tests they actually want to write. Kotlin tests tend to be shorter, more readable, and easier to scan at code review. Leadership and platform teams are skeptical of language changes for good reasons. The proposal "we want to use Kotlin for tests" carries almost no risk to evaluate, since it doesn't touch production or your deployment infrastructure. It's a clean, bounded experiment, and the pattern has been validated by engineering organizations across the industry. Adding Kotlin to an Existing Java + Spring Boot Project You don't need to migrate your entire build to use Kotlin in tests. If you're running a standard Spring Boot + Gradle project, you add the Kotlin JVM plugin, pull in kotlin("stdlib") and io.mockk:mockk at test scope, and IntelliJ IDEA recognizes src/test/kotlin as a source set automatically. Open the project in IntelliJ IDEA and update your build.gradle.kts, adding the Kotlin JVM plugin in the plugins block and the test-scoped dependencies: Kotlin plugins { java id("org.springframework.boot") version "4.1.0" // example version, check for latest id("io.spring.dependency-management") version "1.1.5" // example version, check for latest // Add the Kotlin JVM plugin alongside your existing Java plugin kotlin("jvm") version "2.4.10" // example version, check kotlinlang.org for latest stable } dependencies { // Your existing production dependencies remain untouched implementation("org.springframework.boot:spring-boot-starter-web") // Existing test dependency testImplementation("org.springframework.boot:spring-boot-starter-test") // Required for coroutines testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.11.0") // Kotlin stdlib (test scope only) testImplementation(kotlin("stdlib")) // MockK: Kotlin-native mocking library (check GitHub for latest stable) testImplementation("io.mockk:mockk:1.14.11") // SpringMockK: @MockkBean and @SpykBean for Spring context tests (check GitHub for latest stable) testImplementation("com.ninja-squad:springmockk:5.0.1") // Kotest assertions, no need for the full test runner (check kotest.io for latest stable) testImplementation("io.kotest:kotest-assertions-core:6.2.3") } tasks.withType<Test> { useJUnitPlatform() } Always check the Kotlin and library documentation for the latest stable releases before pinning versions in a new project. If you’re using Maven, you need to add the kotlin-maven-plugin: XML <properties> <kotlin.version>2.4.10</kotlin.version> </properties> <dependencies> <!-- Existing Spring Boot test dependencies --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <!-- Kotlin standard library, test scope --> <dependency> <groupId>org.jetbrains.kotlin</groupId> <artifactId>kotlin-stdlib</artifactId> <version>${kotlin.version}</version> <scope>test</scope> </dependency> <!-- MockK --> <dependency> <groupId>io.mockk</groupId> <artifactId>mockk</artifactId> <version>1.14.11</version> <scope>test</scope> </dependency> <!-- Kotest assertions --> <dependency> <groupId>io.kotest</groupId> <artifactId>kotest-assertions-core</artifactId> <version>6.2.3</version> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.jetbrains.kotlin</groupId> <artifactId>kotlin-maven-plugin</artifactId> <version>${kotlin.version}</version> <executions> <execution> <id>test-compile</id> <phase>test-compile</phase> <goals> <goal>test-compile</goal> </goals> <configuration> <sourceDirs> <source>src/test/kotlin</source> <source>src/test/java</source> </sourceDirs> </configuration> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> </plugin> </plugins> </build> After saving, IntelliJ IDEA displays a notification to reload the Gradle project. Use Shift+Cmd+I on macOS or Ctrl+Shift+O on Windows and Linux to trigger the sync. Once complete, IntelliJ IDEA automatically recognizes src/test/kotlin as a source set, allowing you to use Kotlin alongside the existing Java test infrastructure, like base classes, testcontainer setup, test data factories, etc. Writing JUnit 6 Tests in Kotlin Writing JUnit 6 tests in Kotlin removes a lot of the friction of test setup. The language's expressiveness lets you focus on the scenario and expected behavior instead. Consider a Spring service that manages orders. Test code often spends more time constructing test data than testing behavior. In Java, creating several variations of the same object can require repeatedly calling setters: Java Order order = new Order(); order.setId(1L); order.setCustomerId(42L); order.setTotal(new BigDecimal("99.99")); order.setStatus(OrderStatus.PENDING); Order cancelledOrder = new Order(); cancelledOrder.setId(1L); cancelledOrder.setCustomerId(42L); cancelledOrder.setTotal(new BigDecimal("99.99")); cancelledOrder.setStatus(OrderStatus.CANCELLED); The setup becomes even more repetitive as a test suite introduces additional scenarios. A test for a large order, for example, would need to repeat the same boilerplate just to change the total. Kotlin's named and default arguments can already reduce that repetition, but you can go further by creating a small test-data DSL that describes orders in terms of the scenarios your tests care about: Kotlin fun orderScenario(block: OrderScenario.() -> Unit): Order = OrderScenario().apply(block).build() class OrderScenario { var id: Long = 1L var customerId: Long = 42L var total: BigDecimal = BigDecimal("99.99") var status: OrderStatus = OrderStatus.PENDING fun pending() { status = OrderStatus.PENDING } fun confirmed() { status = OrderStatus.CONFIRMED } fun cancelled() { status = OrderStatus.CANCELLED } fun forCustomer(id: Long) { customerId = id } fun withTotal(amount: String) { total = BigDecimal(amount) } fun build() = Order(id, customerId, total, status) } Tests can now describe the data they need as scenarios: Kotlin val pendingOrder = orderScenario { forCustomer(42L) withTotal("99.99") pending() } val cancelledOrder = orderScenario { forCustomer(42L) cancelled() } val largeOrder = orderScenario { withTotal("999.99") pending() } The DSL uses Kotlin's lambdas with receivers to make OrderScenario's properties and functions available directly inside the orderScenario { ... } block. Default values provide sensible test data, while individual tests override only the attributes relevant to their scenario. Most importantly, the production Order class can remain completely unchanged in Java. This approach becomes particularly useful as a test suite grows. Instead of accumulating specialized factory methods such as cancelledOrder(), largePendingOrder(), or confirmedOrderForCustomer(), the DSL gives tests a small vocabulary for composing those scenarios themselves. The test describes *what state it needs* without repeating the mechanics of constructing the Java object. A complete JUnit 6 test can then combine the scenario DSL with MockK and Kotest assertions: Kotlin import io.kotest.matchers.shouldBe import io.mockk.every import io.mockk.mockk import io.mockk.verify import org.junit.jupiter.api.Test internal class OrderServiceTest { private val repository = mockk<OrderRepository>() private val orderService = OrderService(repository) @Test fun `returns order when found by id`() { val order = orderScenario { forCustomer(42L) withTotal("99.99") pending() } every { repository.findById(order.id) } returns order val result = orderService.getOrder(order.id) result shouldBe order verify(exactly = 1) { repository.findById(order.id) } } @Test fun `throws when order is not found`() { every { repository.findById(any()) } returns null shouldThrow<NoSuchElementException> { orderService.getOrder(99L) } } } Backtick-delimited test names (`returns order when found by id`) let you write descriptive test names directly in the function declaration, so the name appears naturally in test runner output. This can also save you from using JUnit's `@Named` annotation when you want a readable test name. When a Java repository method returns a nullable `Order?`, the Kotlin compiler forces the test code to account for the null case before using the result. That can surface mismatches between the test and the Java API contract at compile time rather than as unexpected `NullPointerException`s during test execution. A full guide is available in the docs. MockK for Kotlin-Idiomatic Mocking Mockito remains a viable choice when adding Kotlin tests to a Java application, but MockK provides an API designed around Kotlin's syntax. Its concise every { ... } returns ... and verify { ... } constructs make mocked interactions easier to read, while allowing Kotlin tests to mock existing Java classes without changing the production code. This can be particularly useful when testing Java Spring applications that use Reactor. The production application can remain entirely in Java and continue to return Mono or Flux. Kotlin is only introduced in the test suite. MockK handles the mocked dependencies, while Reactor's StepVerifier verifies the resulting publishers. Consider a Java service that returns an order reactively: Java public Mono<Order> findOrder(Long id) { return repository.findById(id); } The service can be tested from Kotlin without changing its implementation: Kotlin import io.kotest.matchers.shouldBe import io.mockk.every import io.mockk.mockk import io.mockk.verify import org.junit.jupiter.api.Test import reactor.core.publisher.Mono import reactor.test.StepVerifier internal class OrderServiceTest { private val repository = mockk<OrderRepository>() private val orderService = OrderService(repository) @Test fun `returns order when found`() { val order = Order( id = 1L, customerId = 42L, total = BigDecimal("99.99"), status = OrderStatus.CONFIRMED ) every { repository.findById(1L) } returns Mono.just(order) val result = orderService.findOrder(1L) StepVerifier.create(result) .expectNext(order) .verifyComplete() verify(exactly = 1) { repository.findById(1L) } } } The production service and repository can remain in Java. The Kotlin test uses MockK's Kotlin-friendly syntax to define the mock behavior, while StepVerifier checks that the returned Mono emits the expected value and completes successfully. The same pattern applies to Flux-returning APIs. For example, a test can verify multiple emitted values without manually subscribing to the publisher: Kotlin every { repository.findAll() } returns Flux.just(order1, order2) StepVerifier.create(orderService.findAll()) .expectNext(order1, order2) .verifyComplete() MockK doesn't replace Reactor's testing tools. Each library handles a different part of the test. MockK provides concise mocking, while StepVerifier provides assertions over Reactor publishers. Kotlin brings those APIs together in a compact test that's focused on the behavior being verified. In practice, that means you can adopt Kotlin for the test source set, use MockK to mock your existing Java dependencies, and test reactive Java code without introducing Kotlin or coroutines into the production application. Kotest Assertions for Cleaner Test Intent You don't need Kotest's full test runner to benefit from its assertion library. The kotest-assertions-core dependency adds infix matchers (shouldBe, shouldNotBe, shouldThrow, shouldContain) that layer on top of your standard JUnit 5 tests. Compare JUnit's assertion style with Kotest's: Kotlin // JUnit 5 assertions Assertions.assertEquals(OrderStatus.CONFIRMED, result.status) Assertions.assertNotNull(result.id) // Kotest matchers (same assertions, different shape) result.status shouldBe OrderStatus.CONFIRMED result.id shouldNotBe null The Kotest versions read in the same order as a sentence describing what you expect. That ordering makes the intent visible without parsing the argument order of assertEquals. shouldThrow is particularly useful for testing Spring service exception paths: Kotlin val exception = shouldThrow<NoSuchElementException> { orderService.getOrder(99L) } exception.message shouldBe "Order 99 not found" The matcher returns the thrown exception, so you can chain further assertions on its properties without additional try/catch blocks. Spring Boot Integration Tests With @MockkBean For Spring slice tests (@WebMvcTest, @DataJpaTest, or a partial @SpringBootTest), you swap Spring's @MockBean for @MockkBean from the springmockk library. It integrates with Spring's test context the same way @MockBean does, but creates MockK instances: Kotlin package com.example.demo.controller import com.example.demo.model.Order import com.example.demo.model.OrderStatus import com.example.demo.service.OrderService import io.mockk.every import org.junit.jupiter.api.Test import org.springframework.beans.factory.annotation.Autowired import org.springframework.boot.webmvc.test.autoconfigure.WebMvcTest import org.springframework.test.web.servlet.MockMvc import org.springframework.test.web.servlet.get import com.ninjasquad.springmockk.MockkBean import java.math.BigDecimal import java.util.NoSuchElementException @WebMvcTest(OrderController::class) internal class OrderControllerTest { @Autowired private lateinit var mockMvc: MockMvc @MockkBean private lateinit var orderService: OrderService @Test fun `GET order returns 200 with order data`() { val order = Order( 1L, 42L, BigDecimal("99.99"), OrderStatus.CONFIRMED ) every { orderService.getOrder(1L) } returns order mockMvc.get("/orders/1") .andExpect { status { isOk() } jsonPath("$.id") { value(1) } jsonPath("$.customerId") { value(42) } jsonPath("$.total") { value(99.99) } jsonPath("$.status") { value("CONFIRMED") } } } @Test fun `GET missing order returns 500`() { every { orderService.getOrder(99L) } throws NoSuchElementException("Order 99 not found") mockMvc.get("/orders/99") .andExpect { status { is5xxServerError() } } } } The @MockkBean annotation replaces the service in the Spring context for the duration of the test. IntelliJ IDEA resolves the Kotlin DSL for MockMvc (mockMvc.get(...)) with full autocomplete support, removing the friction of looking up chained method names. Getting Started With Kotlin in Tests Introducing Kotlin through test sources, before any production code changes, is a validated incremental strategy. The tooling is mature, the Gradle setup is a handful of lines, and the first Kotlin test can ship the same day you configure the dependency. Over time, writing Kotlin tests can change how your team thinks about the language. Developers who spend a few months writing Kotlin tests start asking why they can't use data classes in production. They miss null safety in Java service code. Gradual adoption happens organically, driven by genuine preference rather than a top-down mandate. JetBrains and the Spring team have an official strategic partnership focused on improving the developer experience for backend engineers. If you want to explore what Kotlin can do for your team beyond the test suite, kotlinlang.org is the right place to start.

By Aniket Bhattacharyea
Why Kotlin or Java Is the Wrong Question For Enterprises
Why Kotlin or Java Is the Wrong Question For Enterprises

If you've sat through more than one architecture review, you've probably sat through a protracted Kotlin-versus-Java debate that goes nowhere. The team spends weeks comparing Kotlin's concise syntax to Java's maturity, runs a poll on developer preference, and produces a decision matrix nobody fully trusts. Then the meeting ends, and nothing changes, because nobody agreed on what problem they were actually trying to solve. Choosing between infrastructure instruments, such as Kotlin and Java, is an investment decision, and like any investment decision, it needs a thesis. Four business pressures should drive the discussion: time-to-market, operational risk, talent economics, and cross-platform consistency. The Problem With Asking Kotlin or Java When the question of Kotlin or Java arises, syntax and feature comparisons dominate the discussion, because they're concrete and measurable, and a natural place to start. You can count lines of code, benchmark compile times, and run surveys on developer satisfaction, but none of that will tell you if those metrics connect to any business outcome you actually care about. A language can be a good technical fit and still be a poor investment for a particular system. A stable backend handling 50,000 transactions a day with a three-person maintenance team faces a different set of risks than a payments service changing weekly under regulatory pressure, and both look different again from a monorepo running the same business logic across Android, backend, and edge. The JVM language strategy question comes down to where complexity actually lives in the system. Ask where Kotlin meaningfully supports your modernization strategy, and where staying with Java makes better sense right now. Modern Java May Be the Right Answer For a stable, low-change system, modernizing the existing Java stack can address the underlying problem without introducing the costs of another language, toolchain, or learning curve. If the system's main constraints don't align with Kotlin's strengths, staying with Java may offer a better return. If you're considering Kotlin, first identify where its capabilities address a problem worth solving. Start With the Business Problem Drawing on Yuri Geronimus's experience as Senior Staff Engineer and Chief Architect for EMEA Enterprise Payments Platforms at Verifone, the Kotlin Enterprise Playbook frames Kotlin adoption as an enterprise investment decision rather than a language preference. It organizes the business case around four key levers: delivery throughput and time-to-market, defect-cost avoidance, legacy modernization, and talent economics. Time-to-Market If your delivery lead time affects revenue, product launches, or certification cycles, ask whether the team can deliver faster. Kotlin's concise syntax means less boilerplate per feature, and fewer null pointer exceptions reaching production translates to less time spent on hotfixes and incident response. Whether that translates to faster delivery depends on where your bottlenecks sit. If the constraint is product clarity or test coverage rather than boilerplate or type errors, language choice won't fix it. Operational Risk Where are your expensive defects concentrated? Kotlin's type system encodes nullability explicitly, allowing some null-related errors to be caught at compile time rather than in production. For systems where a single production bug triggers a compliance review or a customer-facing outage, that compile-time safety has measurable financial value. For a read-heavy reporting service where reliability is already strong, the business case is thinner. Talent Economics The cost of a JVM talent problem can show up in longer time-to-hire, higher onboarding overhead, attrition, or difficulty moving engineers between teams. Java developers can build on their existing JVM experience when learning Kotlin, making internal mobility a potential part of the business case. The strategic partnership between JetBrains and Spring, announced in May 2025, also means that 27% of Spring developers who've already used Kotlin are a potential internal resource pool you may not have fully mapped. Cross-Platform Consistency If your organization maintains duplicated business logic across an Android app, a backend service, and an edge deployment, that duplication is an operational risk disguised as a technical decision. Kotlin Multiplatform (KMP) can let teams share domain logic, models, and validation across platforms, potentially reducing duplicated implementation and the risk of logic drift. That also reduces the compliance and maintenance burden of finding and fixing the same bug in multiple places. Keep in mind that you should evaluate the case for KMP separately, with the scope of shared code kept deliberately narrow. Kotlin doesn't automatically improve time-to-market any more than a new CI tool does. The question is whether you've identified the constraint it's positioned to address. Measure the Outcome You Actually Care About To make a solid business case for Kotlin, you need a measurable hypothesis. Each business lever maps to an observable metric. Time-to-market connects to delivery lead time and deployment frequency (both DORA metrics your platform team likely already tracks). Operational risk connects to change failure rate, mean time to recovery, and incident cost. Talent economics connects to time-to-hire, time to first commit after onboarding, and attrition rate. Cross-platform consistency connects to duplicated-logic defect rate and the volume of bugs that required fixes in more than one codebase. Those metrics support the business case, but you still have to translate them into financial terms. For an enterprise investment decision, connect the operational improvement to a financial outcome. Faster delivery can reduce the cost of delay, fewer incidents can reduce remediation and customer-impact costs, and shorter onboarding can reduce hiring and training costs. Bear in mind that productivity gains don't translate directly into headcount savings. Faster delivery can instead create capacity for new product work or reduce the cost of delay on a revenue-generating feature. Conflating the two overstates the financial case and creates a credibility problem when someone on the finance team models it out. Also, don't count productivity gains as both lower operating costs and increased revenue in the same proposal. A hypothesis that says "we expect Kotlin's null safety to reduce null-related production incidents by X% over 12 months, based on our current incident log showing Y% of incidents trace to NPEs" is defensible. A hypothesis that says "Kotlin will improve developer productivity" gives you nothing to measure against. Interoperability Makes This a Scoped Decision The Kotlin-or-Java framing misleads because it implies a binary. Kotlin compiles to JVM bytecode and is fully interoperable with Java, so Kotlin and Java files can coexist in the same project without a rewrite. Your existing Spring Boot or Hibernate configurations stay in place. Your Java libraries continue working. A single new service, a shared utilities module, or even the test layer of an existing Java codebase are all valid starting points for introducing Kotlin. IntelliJ IDEA, JetBrains' own IDE, includes a Java-to-Kotlin converter and first-class mixed-project support, so the tooling overhead of running both is lower than teams typically assume. That shifts the question from "should we adopt Kotlin?" to "where should we introduce Kotlin first?" A new, high-change service with a measurable business outcome may be a reasonable candidate. A stable order-management system with a small team and clean Java code can probably wait on Kotlin until a clearer business case emerges. Account for the Iceberg and the J-Curve An honest transition plan accounts for costs that typically don't appear in the optimistic spreadsheet. The J-curve is real. Before a team reaches full Kotlin proficiency, velocity drops. Training takes time, mixed-language code reviews add overhead (reviewers need to understand both languages to catch idiomatic issues), and CI pipelines may need reconfiguration for multi-language builds. Compliance documentation may need updating if your audit trail references language versions or toolchain components. The eventual decommissioning of legacy Java code, if that's the direction, carries its own cost that's easy to leave out of a five-year model. Scope the initial experiment carefully, and build these costs into the transition plan upfront. A proposal that acknowledges the J-curve and plans for it presents a more credible case than one that doesn't. Three Questions for Your Next Architecture Review If your team is heading into an architecture review where Kotlin is on the agenda, these three questions cut through the syntax debate and turn it into a scoped, testable proposal. Where are the expensive defects or operational problems concentrated? Pull the last 12 months of production incidents and categorize the ones with the highest cost (in engineer time, customer impact, or compliance exposure). If a significant share traces to null-related errors or type-safety failures, Kotlin's type system is directly relevant. If the distribution looks different, the investment case needs a different justification. What does your JVM talent pipeline actually look like? Map your current team's Kotlin exposure, your open Java roles, and the average time-to-hire over the past year. If you're competing for senior Java engineers in a tight market and losing candidates to organizations with more modern stacks, that's a talent economics argument worth quantifying. If your team is stable and hiring is closed, the argument doesn't apply. Which service, module, or domain is the lowest-risk place to test whether Kotlin improves the target outcome? Look for something actively changing (so you'll have delivery-cycle data to measure), with a well-defined scope, and without deep compliance dependencies that would complicate a mid-stream language shift. New functionality is often the best candidate. Adding Kotlin tests to an existing Java service can also provide a relatively low-production-risk way to gain experience with the language and toolchain. These questions convert an abstract language debate into a scoped experiment with a falsifiable hypothesis. Run the experiment, measure the result, and expand where it works. Make the Strategic Decision Your CFO and CTO won't ever see a line item for Java versus Kotlin, but they will see the costs of delivery lead time, incidents, onboarding overhead, and duplicated-logic defects. The strategic decision is where to invest to improve those outcomes, domain by domain, with measurement in place. Evaluate Kotlin where you have a number to test against, start with the lowest-risk entry point, and leave stable systems alone when the case is weak. The goal is a better outcome where the investment is justified. If you want to see how other enterprise teams have structured this transition, JetBrains has a practical case for it at kotlinlang.org/lp/enterprise-playbook. How has your team navigated this decision? Share your experience in the comments.

By Aniket Bhattacharyea
The Self-Hosting Math Changed While We Were Still Arguing About Sovereignty
The Self-Hosting Math Changed While We Were Still Arguing About Sovereignty

Global AI spending is forecast to hit $2.59 trillion by the end of the year, up 47% from 2025, with infrastructure the single largest line in that number. Underneath most of it sits one decision, repeated per workload: rent the serving or own it. That was always a portfolio question rather than a doctrinal one, and the portfolio was drawn by economics. Those economics moved this year. Most enterprise architecture is still drawn against where they used to sit. What Gated Self-Hosting Actually Is It was never sovereignty. Sovereignty is legitimate. Data residency rules are real, and weights you control sit inside infrastructure you own. The gate was cost at frontier scale. Qwen3.8-Max is 2.4 trillion parameters. Usable throughput from something that size means a multi-node buildout, GPU procurement, cluster management, ongoing inference tuning, and round-the-clock support for a system that can't go down. Cross that threshold, and you haven't self-hosted anything. You've started a model-serving business. That hasn't changed. Frontier scale just stopped being the price of frontier-class capability. Intelligence Per Dollar Is Falling Off a Cliff DeepSeek-V4-Flash runs 284B total parameters with 13B active and a one-million-token context window, and it serves on a single four-GPU node. Qwen3.8-Flash-Next pairs a 125B model with a 51B N-gram embedding table and activates 6B parameters per token; quantized builds are already running on two 64GB cards. Both are open-weight. Both land near where the closed frontier sat a few months ago on the work most enterprises actually run at volume: extraction, triage, code review, document analysis, routine agentic loops. A year ago, hosting a model with frontier-class capability meant a data center. Today, it means a node. The line separating sensible from delusional moved a long way, and most enterprise architecture decisions haven't caught up to where it landed. Serverless Takes Out the Last Real Objection The honest objection to self-hosting was never only capital cost; it was utilization. Owned GPUs bill whether or not you're using them, so the economics only work past a steady consumption floor. Below that floor, you are paying to keep silicon warm. Serverless GPU platforms like Modal drop that floor dramatically. Per-second billing on your own weights, autoscaling to zero, no fleet you have to keep busy to justify. That covers the awkward middle, where the workload is large enough that per-token API pricing starts to hurt but too spiky to justify reserved capacity. Own the weights, rent the seconds, and revisit the buy decision when consumption stabilizes. Post-Training Is Where This Gets Genuinely Interesting The bigger shift isn't cheaper serving. It's what you can do to a model once you control the weights. Most enterprises pay for firm-specific context the same way every time: stuff it into the prompt, pay for it on the way in, pay again on the way out, on every transaction, forever. Engram's work with Harvey is the clearest picture I've seen of the alternative. They built a synthetic law firm, 266 matters and 9,286 files across 100 million tokens, then trained an agent to study the whole workspace before answering anything, holding what it learned in a mix of text notes and weights. It runs on Qwen3.8-27B and answers at $0.13 per query against Opus 4.8 at $1.32, while scoring higher. Probed closed-book, with no tools and no notes, the model's knowledge of the firm climbed from 4.7% to 72.6% over the course of studying. Their framing is the part worth stealing: studying shifts compute from inference time to training time. Where many queries hit the same corpus, paying once to internalize it beats re-reading it on every call. That's an amortization argument, and it strengthens the longer a workspace lives, which describes nearly all enterprise work. Knowledge is the easy half. Capability is the half worth watching, and it runs on the same machinery the open labs converged on this year. The student generates its own rollouts inside an environment you define, and the teacher model grades what it produces token by token. Engram's recipe uses both halves, distillation on model outputs alongside reinforcement learning on problems the model writes for itself. Pull that thread far enough, and the frontier model stops being a runtime dependency and becomes a training input. You pay frontier prices once, during post-training, rather than on every inference call for the life of the system. Where Governed Hosting Still Wins A governed arrangement through AWS Bedrock or Databricks remains the right call for a real set of situations: workloads that genuinely need the current frontier and nothing below it, organizations with no ML engineering capacity to speak of, and volumes low enough that per-token pricing never becomes the largest line on the bill. Those are real conditions, and plenty of enterprises meet them. Far fewer meet them across their whole estate, which is how most of them are buying. Capability was never a pure compute question. It's how infrastructure decisions get made, what security posture is in place, whether an evaluation framework exists to tell you the model is actually performing, and whether anyone is tuning it as usage evolves. Self-hosting neither creates that burden nor removes it. It makes the burden much cheaper to carry, which means the binding constraint is now your people rather than your GPUs. The Math Worth Running Can we afford to be a model-serving company, and do we actually need to be one? For a growing share of workloads, that answer has quietly turned to yes, and most budgets are still built against the old one. What fraction of my traffic genuinely requires frontier intelligence, and what fraction is running on it out of habit? What is my steady-state consumption, and where does it cross the line where owning beats renting? What would it be worth to bake my firm's context and my firm's tasks into weights I control, instead of re-paying for both on every single call? Access to a frontier model's weights is still not the same thing as access to frontier capability. But the weights got cheap enough to serve and malleable enough to shape, and capability turned into something you can build rather than only something you can buy.

By Sid Vivek
Beyond the Wall of Charts: How Advanced Observability Displaced the Telemetry Dashboard
Beyond the Wall of Charts: How Advanced Observability Displaced the Telemetry Dashboard

For the past decade, the gold standard of Site Reliability Engineering (SRE) was "all your telemetry in one place, beautifully rendered." Centralized platforms allowed teams to build massive grids of charts tracking CPU usage, memory allocation, request latencies, and error rates. Yet this approach masks a critical flaw: traditional dashboards show only symptoms, forcing humans to act as data archaeologists to find the root cause. With microservices handling highly distributed workloads, a simple user request might cross a dozen internal boundaries. When latency spikes, a static dashboard tells you what is broken, but it cannot tell you why. The modern engineering directive has fundamentally shifted from reactive visibility to deep system understanding. The conventional approach of isolating metrics, logs, and traces into discrete dashboard tabs or separate tools has broken down under cloud-native complexity: The Contextual Chasm: When an alert fires, engineers are forced to pivot between systems, manually trying to correlate a metric spike in one graph with a raw text error in a log file. This manual synchronization creates a high mean time to resolution (MTTR).Pre-Aggregation Destruction: Traditional monitoring relies on pre-aggregated data averages to control cloud storage costs. This destroys fine-grained details, making it impractical to debug highly specific "unknown-unknown" incidents affecting rare user cohorts.Anticipation Bias: Dashboards assume that a software engineering team can accurately predict every failure mode ahead of time to build a chart for it. Complex systems fail in entirely unpredictable ways that static, predefined views cannot capture. The Observability Revolution: Core Paradigms Replacing the Dashboard Rather than forcing engineers to stare at screens waiting for a line to turn red, modern observability treats system telemetry as a unified, exploratory analytics problem: High-Cardinality Unified Data Layers: Modern architectures store unaggregated, raw event streams. By preserving trace IDs, customer IDs, and container headers, engines can instantly pinpoint the precise thread or database index responsible for a system failure.Contextual Topology and Causality: Advanced systems continuously map software dependencies and traffic patterns. Instead of sending a dozen isolated alert notifications, the system assembles an incident narrative, proving how a configuration drift upstream actively broke a payment processing gate downstream.Agentic AI Remediation: The industry is moving from monitoring toward autonomous operations. Specialized AI agents consume telemetry streams directly to execute preventative actions like scaling production clusters or rolling back buggy code deployments. The traditional dashboard view: application health, load, response times, and error rates show that something is wrong, but not why. The engineer still has to hunt for the cause. The high-cardinality view: instead of scanning charts, the failing dimension is isolated directly — here, errors tied to a specific service version (v350.10) and HTTP 401 status — pointing straight at the root cause. Adapting to the Age of Non-Deterministic Infrastructure The massive expansion of large language model (LLM) agents and generative software layers has further accelerated the demise of legacy dashboards. Traditional systems check whether an API endpoint returns a 200 OK HTTP status code. An AI-driven microservice can operate perfectly within technical boundaries, response times, and system limits, while still delivering entirely hallucinated or factually incorrect output to an enterprise client. Observability has adapted by monitoring semantic tokens, prompt guardrails, and vector pipeline drift, complex factors that cannot be displayed on a standard bar chart. The metric of success for a modern technology infrastructure is no longer the number of operational dashboards under management. Success is defined by autonomous, rapid understanding and instantaneous path-to-resolution, which is why the static visual graph has been displaced as the primary debugging surface. The Strategic Dilemma Transitioning from static, pre-aggregated dashboards to high-cardinality observability (tracking raw, un-aggregated event data across highly granular dimensions like user_id, container_id, and ip_address) is essential for modern debugging and reducing MTTR. However, storing raw, multi-dimensional event streams exponentially inflates cloud infrastructure and data ingest costs. Without strict architectural guardrails, observability spend can grow to consume an unsustainable percentage of total cloud budgets. The True Drivers of High-Cardinality Cost Exploding Volume of Unique Dimensions: Cloud-native elements (ephemeral containers, serverless functions) spin up and down constantly, generating billions of unique metric combinations that saturate traditional relational structures.Data Ingestion Overhead: Commercial observability vendors charge based on data volume ingested and index size, and self-hosted open-source clusters (e.g., OpenSearch, or Prometheus with Cortex/Thanos) carry the same cost pressure through the infrastructure required to store and index that volume.Hot Storage Reliance: Keeping raw, unaggregated logs and traces instantly queryable requires expensive, high-throughput SSD cloud storage, driving up baseline cloud infrastructure costs. Architectural Framework for Cost Mitigation To balance the technical necessity of high-cardinality data with strict fiscal control, tech leaders should mandate a tiered storage approach: Implement Intelligent Data Sampling Not all telemetry is created equal. SRE teams should configure tail-based sampling rules. For instance, store 100% of traces that result in an HTTP 5xx error or high latency, but sample only 1% to 5% of healthy, predictable HTTP 200 OK traces. Adopt Tiered Storage Horizons Hot Tier (0–7 Days): Keep full, high-cardinality raw telemetry in expensive, high-performance storage for immediate active incident response.Warm Tier (8–30 Days): Move data to aggregated metrics blocks or lower-cost search-optimized tiers for historical trend analysis.Cold Tier (31+ Days): Compress and offload raw data into highly economical object storage (such as Amazon S3, Google Cloud Storage, or Azure Blob) strictly for long-term compliance and audit requirements. Tiered storage keeps high-cardinality observability affordable: recent data stays hot for fast incident response, then ages into progressively cheaper, slower storage for trend analysis and compliance. Enforce Strict Lifecycle Policies. Automate the purging or down-sampling of high-cardinality dimensions as data ages. A specific container ID from three weeks ago holds little operational value for daily SRE workflows. Verdict Do not scale back observability goals to save on infrastructure bills. Instead, optimize how and where that telemetry lives. By implementing data sampling at ingestion and automated lifecycle workflows, organizations can unlock the full power of high-cardinality root-cause analysis while keeping data storage costs linear and predictable.

By Prashant Maria
Delta Lake UniForm to Snowflake: Cross-Platform Table Access Without a Second Copy
Delta Lake UniForm to Snowflake: Cross-Platform Table Access Without a Second Copy

Making a Databricks-owned Delta table queryable from Snowflake without copying it is one of those things where every individual piece — Delta Lake UniForm, Unity Catalog, private connectivity — is documented somewhere on its own, but nobody writes up how they actually fit together end to end. If you've been asked to give Snowflake read access to a dataset that Databricks owns, without standing up a nightly export job, this is for you. Why the Two Platforms Don't Share Data Natively Delta Lake and Apache Iceberg are both open table formats, but they track a table's files, schema, and change history differently. Delta Lake is Databricks' native format. Snowflake has strong native support for Iceberg. A table written as Delta Lake isn't something Snowflake can open directly, and a table written as Iceberg by some other engine isn't something Databricks reads as Delta. Historically, the fix for this mismatch was a second copy: export the Databricks table, land it somewhere Snowflake could read, and run a job to keep the copy in sync. That's double the storage, a sync job that needs monitoring, and a copy that's stale the moment the source table changes. None of that gets billed as "vendor lock-in" on any invoice, but that's exactly what it is. What UniForm Changes Delta Lake UniForm, generally available since Delta Lake 3.2 (Databricks Runtime 14.3+), writes Iceberg metadata alongside the Delta transaction log on every commit. Same Parquet files on disk, one additional metadata layer. Enabling it is a table property: SQL ALTER TABLE my_table SET TBLPROPERTIES ( 'delta.universalFormat.enabledFormats' = 'iceberg' ); Once that's set, Databricks keeps writing Delta exactly as before. Snowflake reads the same table as Iceberg, through Unity Catalog's Iceberg REST Catalog endpoint. No export step, no second storage bill. The Setup, End to End Getting from "UniForm is on" to "a Snowflake analyst can query the table" is four pieces. Piece 1: Enable UniForm on the source table. Covered above. One property, per table, Runtime 14.3 or later. Piece 2: Scope a service principal at the Unity Catalog level, not the storage account. Register a service principal and grant it Unity Catalog privileges — EXTERNAL USE SCHEMA specifically, which isn't part of ALL PRIVILEGES and is easy to grant around by accident. There's no separate storage role to assign. Unity Catalog holds the storage credential itself and vends short-lived, scoped credentials to Snowflake at query time, so the service principal only ever authenticates against the catalog API, never against the storage account directly. Piece 3: Put the connection on private networking. This is the piece that involves the most moving parts, and it's worth walking through because the Snowflake side does the provisioning, not Databricks. Snowflake account needs to be on Business Critical edition or higher for this, and both accounts need to be in the same Azure region. Provision an outbound private endpoint from Snowflake toward the Databricks workspace: SQL SELECT SYSTEM$PROVISION_PRIVATELINK_ENDPOINT( '<databricks_resource_id>', 'databricks_ui_api' ); That endpoint sits in a Pending state until someone with access to the Azure resource approves it in the Azure Portal — a step that's easy to lose a day to if you don't know to go looking for it. Then register the catalog integration: SQL CREATE OR REPLACE CATALOG INTEGRATION uc_catalog_int CATALOG_SOURCE = ICEBERG_REST CATALOG_API_TYPE = PRIVATE CATALOG_URI = 'https://<databricks-workspace-url>' ... ENABLED = TRUE; The detail that trips people up: use the standard public hostname for the workspace URL, not the PrivateLink endpoint's own hostname. Snowflake routes traffic through the private endpoint automatically once it exists — pointing the catalog integration at the PrivateLink hostname directly just breaks it. A second private endpoint, and an external volume with USE_PRIVATELINK_ENDPOINT = TRUE, does the same thing for the storage account so file reads stay off the public internet too. Piece 4: Link the catalog — don't hand-create Iceberg tables. The last step is a catalog-linked database, not individual CREATE ICEBERG TABLE statements: SQL CREATE DATABASE uc_linked_db LINKED_CATALOG = ( CATALOG = 'uc_catalog_int' ) EXTERNAL_VOLUME = 'uc_external_vol'; Once this exists, whatever is in Unity Catalog — including every UniForm-generated Iceberg table — shows up in Snowflake automatically, and stays in sync on its own. New tables and schemas created later in Databricks appear without touching Snowflake again. It's a refresh on an interval under the hood, not a live push, so don't oversell the freshness to whoever's asking for "real-time." Writes Are Newer, and Direction Still Matters Everything above is Snowflake reading a Databricks-written table. Writing the other way is newer, and support isn't symmetric yet. Snowflake now supports writing to Databricks-managed Iceberg tables in Unity Catalog — GA on Azure as of earlier this year. Snowflake Horizon's federation is bidirectional by design, using vended short-lived credentials rather than long-lived IAM roles in either direction. Unity Catalog's federation is still mostly inbound: strong at letting other engines read and write into it, weaker when Databricks itself needs to reach out to a catalog it doesn't own. If a design depends on writes going both directions, check current support for the specific cloud and catalog combination before committing to it — this part is moving fast. Where It Still Falls Short Row- and column-level masking is GA for Spark and expanding to other Iceberg REST-compatible engines, but Snowflake's Scan Plan API — cross-engine, attribute-based access control for federated Iceberg data — was still in private preview as of mid-2026. Don't assume a masking policy enforces the same way on both sides until it's been tested. Iceberg's own spec keeps moving underneath this too. Iceberg v3 features like deletion vectors and row lineage are in public preview on Databricks now, which is good news for performance, but it means "Iceberg compatible" isn't a fixed target — check that both platforms agree on the table spec version before standardizing on it. And disaster recovery for federated catalog setups is still gated or preview-stage on the Unity Catalog side as of this writing. Don't lean on this as part of a DR plan without confirming that directly. One Thing I'd Tell Anyone Doing This Check the edition and region requirements before you design anything, not after. Business Critical or higher on the Snowflake side, same Azure region on both sides — if either of those isn't true for your accounts, the private connectivity piece doesn't work at all, and there's no workaround short of moving one of the accounts. Confirm that first, then start building.

By Jeevan Krishna Paruchuri

Culture and Methodologies

Agile

Career Development

Methodologies

Team Management

To Switch, or Not to Switch the AI Model

October 26, 2026 by Stefan Wolpers DZone Core CORE

How to Build Your Own AI SRE From Scratch

October 23, 2026 by Severin Neumann

Avoiding AI Debt as a Team

October 23, 2026 by Stefan Wolpers DZone Core CORE

Data Engineering

AI/ML

Big Data

Databases

IoT

Beyond the Endpoint: How Agent Cards Enable Discovery, Interoperability, and Trust

October 26, 2026 by Josephine Eskaline Joyce, Ph.D DZone Core CORE

LangChainGo vs Genkit Go: Where Genkit Shines

October 26, 2026 by Xavier Portilla Edo DZone Core CORE

To Switch, or Not to Switch the AI Model

October 26, 2026 by Stefan Wolpers DZone Core CORE

Software Design and Architecture

Cloud Architecture

Integration

Microservices

Performance

SSRF Prevention in C#: Securing File Imports From User-Supplied URLs

October 26, 2026 by Brian O'Neill DZone Core CORE

Beyond the Endpoint: How Agent Cards Enable Discovery, Interoperability, and Trust

October 26, 2026 by Josephine Eskaline Joyce, Ph.D DZone Core CORE

Vision-Language Models in 2026: From Research Breakthroughs to Cloud-Native Production Systems

October 26, 2026 by Srinivas Chippagiri DZone Core CORE

Coding

Frameworks

Java

JavaScript

Languages

Tools

PWA on iOS, Part 2: HTTPS, Manifest and Service Worker

October 26, 2026 by Konstantin Shkurko

LangChainGo vs Genkit Go: Where Genkit Shines

October 26, 2026 by Xavier Portilla Edo DZone Core CORE

AI Agents Need an Identity Plane: Rethinking Enterprise Authorization for Autonomous Software

October 26, 2026 by Igboanugo David Ugochukwu DZone Core CORE

Testing, Deployment, and Maintenance

Deployment

DevOps and CI/CD

Maintenance

Monitoring and Observability

Vision-Language Models in 2026: From Research Breakthroughs to Cloud-Native Production Systems

October 26, 2026 by Srinivas Chippagiri DZone Core CORE

Genkit Flows as senro Steps: Extremely Powerful AI Pipelines in Go

October 26, 2026 by Xavier Portilla Edo DZone Core CORE

How to Build Your Own AI SRE From Scratch

October 23, 2026 by Severin Neumann

Popular

AI/ML

Java

JavaScript

Open Source

PWA on iOS, Part 2: HTTPS, Manifest and Service Worker

October 26, 2026 by Konstantin Shkurko

Beyond the Endpoint: How Agent Cards Enable Discovery, Interoperability, and Trust

October 26, 2026 by Josephine Eskaline Joyce, Ph.D DZone Core CORE

To Switch, or Not to Switch the AI Model

October 26, 2026 by Stefan Wolpers DZone Core CORE

  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×