Integration

Every recorded meeting your organization has ever held is already a knowledge base. It just happens to be stored in the least queryable format imaginable, which is a wall of MP4 files sitting in a storage account that nobody opens twice. The good news is that the gap between that wall of files and a working question-answering agent is now much shorter than it used to be, because Microsoft Foundry ships the two halves you need in one place. Fast transcription turns the audio into diarized text in seconds rather than in real time, and Foundry IQ turns that text into a permission-aware knowledge base that any agent can query through a single endpoint. This walkthrough builds the whole thing end to end. By the end you will have a pipeline that watches a blob container for new recordings, transcribes them with speaker labels, chunks them into speaker turns with enough metadata to make citations useful, indexes them as a Foundry IQ knowledge source, and exposes a Foundry agent that answers questions like "what did we decide about the pricing migration in Q2 and who pushed back" with real references back to the moment in the recording. A quick naming note before we start, because the ground has moved. At Ignite 2025, Microsoft renamed Azure AI Foundry to Microsoft Foundry, and the rename was formalized in the January 2026 Product Terms. The platform is the same platform, but there are now two portal experiences and two generations of SDK. The 2.x preview of azure-ai-projects targets the new Foundry portal and API, and the 1.x GA line targets what the docs call Foundry classic. Everything in this article uses the 2.x line and the Responses-based agent surface. What We Are Building, and the Shape of the Data Flow The pipeline has two independent halves that meet at a blob container of curated transcripts. The ingestion half is batch and event-driven. It cares about throughput and about not losing files. The retrieval half is synchronous and user-facing. It cares about latency and about grounding quality. Keeping them decoupled through storage means you can reindex, re-chunk, or swap the retrieval strategy without touching a byte of audio again. The flow is worth reading left to right once. A recording lands in raw-recordings. Event Grid picks up the Blob Created event and drops a message on a queue, which gives you retry semantics and a dead letter path for free. A queue-triggered Function pulls the message, POSTs the audio to the Foundry Speech fast transcription endpoint, and gets back a synchronous response containing diarized phrases. A second stage groups those phrases into speaker turns, attaches timestamps and meeting metadata, and writes JSONL into curated-transcripts. Foundry IQ indexes that container on a schedule. Why a queue between Event Grid and the Function rather than a direct trigger? Because fast transcription is synchronous and the audio files are large. A direct blob trigger gives you very little control over concurrency, and the first time somebody bulk-uploads six months of archived recordings, you will saturate your Speech resource and start collecting 429s. The queue lets you cap batchSize in host.json and shape the load. Standing up the Foundry Project and the Speech Resource Create a Foundry project first. In the portal, make sure the New Foundry toggle is on, then create or select a project. The thing you need out of the portal is the project endpoint, which has the form https://<resource-name>.services.ai.azure.com/api/projects/<project-name>. Install the preview packages. Shell pip install "azure-ai-projects>=2.4.0" azure-identity openai azure-storage-blob requests az login Entra ID is the only authentication method the project client supports, so there is no key-based escape hatch here. Give yourself the Azure AI User role on the project resource for development work. For the pipeline itself, use a user-assigned managed identity and grant it Azure AI User plus Storage Blob Data Contributor. Two environment variables carry the rest of the article. Shell export FOUNDRY_PROJECT_ENDPOINT="https://your-account.services.ai.azure.com/api/projects/meetings" export SPEECH_RESOURCE_NAME="your-speech-resource" Confirm the project client talks to the service before you build anything on top of it. Python import os from azure.ai.projects import AIProjectClient from azure.identity import DefaultAzureCredential with ( DefaultAzureCredential() as credential, AIProjectClient( endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], credential=credential, ) as project, ): openai = project.get_openai_client() r = openai.responses.create( model="gpt-5-mini", input="Reply with the single word ready.", ) print(r.output_text) get_openai_client() returns an authenticated client from the openai package configured to run Responses operations against your Foundry project endpoint. That is the pattern to internalize. You use the project client for setup, configuration, agents, and evaluations, and the OpenAI-compatible client for the actual model calls. Turning an Hour of Audio Into Diarized Speaker Turns Fast transcription is the right tool for recorded meetings. It returns results synchronously and much faster than real time, which is exactly the tradeoff you want for a file that already exists. Batch transcription is the alternative, and it wins on very long archives and on advanced customization, but for a one-hour standard-format recording, fast transcription gets you a result in a small number of seconds with predictable latency. The endpoint is /speechtotext/transcriptions:transcribe and the current generally available API version is 2025-10-15. It takes multipart/form-data with the audio in one part and a JSON definition in another. Diarization is configured with a diarization object carrying maxSpeakers, and the service can separate up to 35 distinct speakers in a single channel before it errors out. Here is the worker in full, with the retry behavior that you will absolutely need. Python import json import os import time import requests from azure.identity import DefaultAzureCredential SPEECH_ENDPOINT = ( f"https://{os.environ['SPEECH_RESOURCE_NAME']}" ".cognitiveservices.azure.com/speechtotext/transcriptions:transcribe" "?api-version=2025-10-15" ) SCOPE = "https://cognitiveservices.azure.com/.default" RETRYABLE = {408, 429, 500, 502, 503, 504} def transcribe(audio_path, locales=("en-US",), max_speakers=8, max_attempts=5): """Fast transcription with diarization and bounded exponential backoff.""" credential = DefaultAzureCredential() definition = { "locales": list(locales), "diarization": {"enabled": True, "maxSpeakers": max_speakers}, "profanityFilterMode": "None", } for attempt in range(max_attempts): token = credential.get_token(SCOPE).token with open(audio_path, "rb") as fh: response = requests.post( SPEECH_ENDPOINT, headers={"Authorization": f"Bearer {token}"}, files={"audio": (os.path.basename(audio_path), fh)}, data={"definition": json.dumps(definition)}, timeout=600, ) if response.status_code == 200: return response.json() if response.status_code not in RETRYABLE: raise RuntimeError( f"Fast transcription failed {response.status_code} {response.text[:400]}" ) wait = float(response.headers.get("Retry-After", 2 ** attempt)) time.sleep(min(wait, 60)) raise RuntimeError(f"Giving up on {audio_path} after {max_attempts} attempts") A few things in there earn their place. The Retry-After header is honored when the service sends one, which matters a lot under throttling because blind exponential backoff on a shared Speech resource just means every worker retries in lockstep. Profanity filtering is set to None because the default is Masked and masked words in a transcript quietly damage retrieval, since the asterisks become tokens that match nothing. The 600-second timeout is generous on purpose, because a large file uploading over a constrained egress path can spend a long while before the service even starts work. The response contains a phrases array where each entry carries speaker, offsetMilliseconds, durationMilliseconds, and text. Phrases are the wrong chunk size for retrieval. They are usually a sentence or two, which means an embedding of a phrase carries almost no context, and a citation to a phrase drops the reader into the middle of a thought. Group them into speaker turns instead. Python from dataclasses import dataclass, asdict @dataclass class Turn: meeting_id: str meeting_title: str meeting_date: str speaker: str start_ms: int end_ms: int text: str @property def chunk_id(self): return f"{self.meeting_id}-{self.start_ms:09d}" def to_turns(result, meta, max_chars=2400, gap_ms=4000): """Collapse diarized phrases into speaker turns, splitting very long ones.""" turns, current = [], None for p in result.get("phrases", []): speaker = f"Speaker {p.get('speaker', 'unknown')}" start = p["offsetMilliseconds"] end = start + p["durationMilliseconds"] same_speaker = current and current.speaker == speaker contiguous = current and (start - current.end_ms) < gap_ms room = current and (len(current.text) + len(p["text"])) < max_chars if same_speaker and contiguous and room: current.text += " " + p["text"] current.end_ms = end continue if current: turns.append(current) current = Turn( meeting_id=meta["meeting_id"], meeting_title=meta["title"], meeting_date=meta["date"], speaker=speaker, start_ms=start, end_ms=end, text=p["text"], ) if current: turns.append(current) return turns The gap_ms guard is the part people leave out. Without it, a speaker who talks at minute three and again at minute forty gets merged into one chunk if nobody else spoke in between, which is rare but produces a chunk whose timestamp range is meaningless. Four seconds of silence is a reasonable turn boundary for meeting audio. Making Chunks That Are Worth Citing Retrieval quality on meeting transcripts lives or dies on what surrounds the raw text. A bare speaker turn like "yeah I think that's fine, let's go with option two" is nearly unretrievable, because it contains no nouns. The fix is to write a small amount of generated context into each record and let the hybrid search match on that. Python def contextualize(openai, turn, neighbors): """Prepend a one-line situating summary so short turns stay retrievable.""" window = "\n".join(f"{n.speaker}: {n.text}" for n in neighbors) r = openai.responses.create( model="gpt-4.1-mini", input=( "Write one sentence, under 25 words, situating the final utterance " "inside this meeting excerpt. Name the topic and any decision. " "Do not editorialize.\n\n" f"Meeting: {turn.meeting_title} ({turn.meeting_date})\n\n" f"{window}\n\nFinal utterance: {turn.speaker}: {turn.text}" ), ) return r.output_text.strip() def to_records(openai, turns): for i, turn in enumerate(turns): neighbors = turns[max(0, i - 3): i + 1] context = contextualize(openai, turn, neighbors) yield { **asdict(turn), "chunk_id": turn.chunk_id, "context": context, "content": f"{context}\n\n{turn.speaker}: {turn.text}", "timecode": f"{turn.start_ms // 60000:02d}:{(turn.start_ms // 1000) % 60:02d}", } This costs one small model call per turn, which, in a one-hour meeting, is a few hundred calls of a couple hundred tokens each. Run it concurrently with a semaphore rather than serially. The timecode field is what makes citations feel like a product feature rather than a footnote, because you can render it as a deep link into your video player. Write the records as JSONL to curated-transcripts, one file per meeting, and you are done with audio forever. Wiring the Transcripts Into a Foundry IQ Knowledge Base Foundry IQ is the knowledge and retrieval layer built on Azure AI Search. The mental model is two nested objects. A knowledge source points at searchable content, and a knowledge base wraps one or more knowledge sources behind a single endpoint that agents query. For indexed sources, Foundry IQ manages the whole indexing pipeline, so content gets ingested, chunked, vectorized, and prepared for hybrid retrieval without you standing up a skillset by hand. Agentic retrieval features are generally available in the 2026-04-01 REST API. The 2026-05-01-preview version exposes the fuller feature set, including preview knowledge source kinds and the ability to attach an LLM to non-web sources. Blob Storage is a generally available indexed source kind, which is exactly what we need. Point a knowledge source at the curated container. Python from azure.search.documents.indexes import SearchIndexClient from azure.search.documents.indexes.models import ( KnowledgeBase, KnowledgeSourceReference, AzureBlobKnowledgeSource, AzureBlobKnowledgeSourceParameters, ) from azure.identity import DefaultAzureCredential index_client = SearchIndexClient( endpoint=os.environ["SEARCH_ENDPOINT"], credential=DefaultAzureCredential(), ) source = AzureBlobKnowledgeSource( name="meeting-transcripts", description=( "Diarized speaker turns from recorded internal meetings, 2024 onward. " "Each chunk carries meeting title, date, speaker label, and timecode." ), azure_blob_parameters=AzureBlobKnowledgeSourceParameters( connection_string=os.environ["BLOB_CONNECTION"], container_name="curated-transcripts", embedding_model=..., # your deployed text embedding model chat_completion_model=..., # optional, enables verbalization ), ) index_client.create_or_update_knowledge_source(source) That description field is not decoration. When a knowledge base holds several sources, the retrieval engine plans which sources to query, and the description is the primary signal it uses to route. Write it like you are briefing a colleague who has never seen your data. Now the knowledge base. Python kb = KnowledgeBase( name="meetings-kb", knowledge_sources=[ KnowledgeSourceReference(name="meeting-transcripts", always_query_source=False), ], retrieval_instructions=( "Meeting transcripts. When the user asks who said or decided something, " "return the speaker turns that contain the statement plus the surrounding turns. " "Prefer recent meetings when the question is about current state." ), ) index_client.create_or_update_knowledge_base(kb) The retrieval engine plans which sources to query and performs iterative search when the first pass does not clear its relevance bar. Iterative search depends on setting a medium retrieval reasoning effort, either on the knowledge base or per request. That single knob is also the biggest lever on both latency and spend, so treat it as a tuning parameter rather than a set-and-forget value. Reasoning effortWhat the engine doesGood fit forMinimalSingle pass, extractive results, no query planningLookup-style questions where the user names the meetingLowLight query decomposition across sourcesMost interactive chat trafficMediumIterative search plus richer planning over sourcesAnalytical questions spanning many meetings Giving the Agent a Knowledge Base and a Personality With the knowledge base in place, the agent is short. Agent operations in the 2.x SDK are built on the Responses protocol, and agents are versioned objects created with create_version. Python from azure.ai.projects.models import PromptAgentDefinition INSTRUCTIONS = """You answer questions about internal meetings using only the meeting transcript knowledge base. Rules you follow without exception. 1. Every factual claim carries a citation naming the meeting title, date, and timecode. 2. When you cannot find support in the transcripts, say so plainly and stop. 3. Attribute statements to the speaker label exactly as it appears. Never guess a real name. 4. When speakers disagreed, surface the disagreement rather than flattening it into consensus. 5. Distinguish a decision from a suggestion. Quote the language that makes it one or the other. """ agent = project.agents.create_version( agent_name="meeting-analyst", definition=PromptAgentDefinition( model="gpt-5-mini", instructions=INSTRUCTIONS, tools=[{"type": "knowledge_base", "knowledge_base": {"name": "meetings-kb"}], ), ) print(agent.id, agent.version) Rule three is doing real work. Diarization gives you stable speaker identifiers within a recording, not identities, so you get generic labels rather than names. If the instructions do not forbid it, a capable model will cheerfully infer that Speaker 2 is the person whose name appears in the meeting title, and it will be wrong roughly as often as it is right. If you need real names, map them yourself in the chunking stage from calendar metadata or from multichannel capture, and write the resolved name into the record. Calling the agent looks like any Responses call. Python def ask(openai, agent_name, question, previous_response_id=None): return openai.responses.create( extra_body={"agent": {"name": agent_name, "type": "agent_reference"}, input=question, previous_response_id=previous_response_id, ) first = ask(openai, "meeting-analyst", "What did we decide about the pricing migration, and did anyone object?") print(first.output_text) follow_up = ask(openai, "meeting-analyst", "Which of those objections were ever resolved?", previous_response_id=first.id) print(follow_up.output_text) Threading through previous_response_id keeps the conversation server-side, which means you are not shipping a growing transcript of the chat on every turn and you are not writing your own history store. Failing Well When Retrieval or the Model Does Not Cooperate Two failure classes matter in production, and they want different handling. Transient service errors want retries. Empty or weak retrieval wants a different answer, not a retry, because running the same query again against the same index returns the same nothing. Python import random from openai import APIStatusError, APITimeoutError TRANSIENT = {408, 409, 429, 500, 502, 503, 504} def ask_resilient(openai, agent_name, question, attempts=4, **kwargs): last = None for i in range(attempts): try: return ask(openai, agent_name, question, **kwargs) except APITimeoutError as exc: last = exc except APIStatusError as exc: if exc.status_code not in TRANSIENT: raise retry_after = exc.response.headers.get("retry-after") last = exc if retry_after: time.sleep(min(float(retry_after), 30)) continue time.sleep(min(2 ** i + random.random(), 30)) raise last Full jitter on the backoff is not optional at any real concurrency. Without it, your retries synchronize into a thundering herd, and you turn a brief throttle into a sustained one. For the retrieval side, the answer is to make the agent's failure visible rather than silent. Instruction two above tells the model to say it found nothing, and you should assert on that in your evaluation set. A grounded system that admits ignorance is far more valuable than one that produces confident prose from three irrelevant chunks, and the second failure mode is much harder to notice in production because the output looks fine. Measuring Whether the Thing Actually Works Two separate quality questions live in this pipeline, and they need separate measurement. The transcription layer has an accuracy problem measured in word error rate. The retrieval and generation layer has a groundedness problem measured by a judge model. A regression in either one looks identical from the outside, which is a good argument for measuring them apart. Build a golden set first. A hundred or so questions written against meetings you have actually listened to is worth more than a thousand synthetic ones, because the value is in the expected answers and only a human who sat through the meeting can write those. Cover the awkward shapes deliberately. Include questions whose answer is genuinely absent so you can measure refusal behavior. Include questions that span two meetings. Include questions where two people disagreed. JSON {"question": "Who owned the migration rollback plan after the March review?", "expected": "Speaker 3 accepted ownership at 41:12 in Platform Review 2026-03-04.", "must_cite": "Platform Review 2026-03-04", "kind": "attribution"} {"question": "What was the agreed SLA for the batch job?", "expected": "Not discussed in any recorded meeting.", "must_cite": null, "kind": "refusal"} The evaluation operations live on the project client in the 2.x SDK, under properties such as evaluators, evaluation_rules, and schedules. For groundedness and relevance, you use built-in judge evaluators. For word error rate, you register a custom evaluator, because that one is arithmetic rather than judgment. Python import jiwer def transcript_wer(reference_text, hypothesis_text): transform = jiwer.Compose([ jiwer.ToLowerCase(), jiwer.RemovePunctuation(), jiwer.RemoveMultipleSpaces(), jiwer.Strip(), jiwer.ReduceToListOfListOfWords(), ]) return jiwer.wer(reference_text, hypothesis_text, truth_transform=transform, hypothesis_transform=transform) Hand-correct twenty minutes of audio across three or four recordings and keep it as your reference. Twenty minutes sounds thin, and it is, but it catches the failures that matter, which are domain vocabulary and acronyms coming back as phonetic mush. If your WER on product names is bad, the fix is a phrase list rather than a better model. Phrase lists let you hand the recognizer a set of words likely to appear, and they move the needle hard on proper nouns and internal jargon. The metrics worth gating a deploy on are these four. MetricWhat it catchesWhere it comes fromWord error rate on domain termsVocabulary drift, new product names, bad audioCustom evaluator against hand-corrected referenceGroundednessAnswers not supported by retrieved chunksBuilt-in judge evaluatorCitation validityFabricated meeting titles, timecodes outside the recordingDeterministic check against chunk metadataRefusal rate on absent answersConfident invention when nothing was retrievedGolden set questions with no supporting content Citation validity is the cheap one everyone skips. You already have the chunk metadata, so parsing the citations out of the answer and asserting that each meeting title exists and each timecode falls inside that recording's duration is maybe thirty lines of code. It catches a specific and embarrassing failure that judge models are surprisingly forgiving of. Getting This to Production Without Regrets Reindex on a schedule and expect churn. Foundry IQ triggers indexing and data synchronization automatically for indexed sources, but your curated container is the contract. If you change chunking strategy, you are rewriting every record, and a full reindex of a large corpus is not instant. Version your chunking logic and write the version into each record so you can tell mixed-generation content apart during a migration. Decide the permission model before you index anything. Meeting recordings are among the most sensitive content an organization has. Retrieval in Foundry IQ respects user permissions for supported knowledge source types, and for the remote SharePoint source, Purview sensitivity labels and data classifications flow through the indexing and retrieval pipeline. Blob-backed sources do not give you that for free. If access control per meeting matters, either enforce it with security filters at query time using a field on each chunk, or keep recordings in SharePoint and use the remote source, where content never leaves SharePoint, and SharePoint enforces permissions. Retrofitting this later means reindexing everything and auditing every conversation that already happened. Instrument with tracing from day one. The projects SDK ships GenAI tracing instrumentation, currently an experimental preview where spans and attributes may change between versions. Turn it on anyway. When a user says the agent gave a bad answer, you want the retrieved chunk IDs and the query plan from that exact response, and reconstructing them after the fact from logs you did not write is miserable. Watch the two meters. Retrieval bills token usage for subquery execution and semantic reranking, and the model you attach for query planning and answer synthesis bills separately on the model side. Reasoning effort, source count, and how much content you route into synthesis are the levers, in that order. Plan the migration if you are on the old pattern. If you are still using Azure OpenAI On Your Data, the "Add your data" flow in the classic chat playground, it is deprecated and retires on October 14, 2026. The official migration target is exactly the stack in this article, which is Foundry Agent Service plus Foundry IQ. How This Compares to Rolling the Pipeline Yourself The obvious alternative is a hand-built stack. Whisper for transcription behind your own GPU or an inference endpoint, pyannote for diarization, your own chunker, a vector database, and LangChain or a custom orchestrator on top. That stack is genuinely good, and it is genuinely more work. The honest comparison looks like this. ConcernFoundry with fast transcription and Foundry IQSelf-hosted Whisper plus pyannote plus a vector DBAmazon Transcribe plus Bedrock Knowledge BasesGoogle Speech-to-Text plus Vertex AI SearchDiarizationBuilt into the same call, up to 35 speakersSeparate model, separate tuning, best-in-class quality achievableBuilt into the transcription jobBuilt into the recognizerTime to first working answerHoursDays to weeksHoursHoursRetrieval planningAgentic, multi-query, iterative at higher effortWhatever you writeManaged retrieval, less query planningManaged retrieval with good semantic rankingPermission-aware retrievalNative for supported sources, Purview labels honored for remote SharePointYou build itIAM-scoped, coarser at the chunk levelIAM-scopedWhere the audio goesYour Azure regionWherever you run it, including fully on-premisesYour AWS regionYour GCP regionEscape hatchKnowledge bases callable from any app through the Search APIsTotal controlBedrock APIsVertex APIs The self-hosted path wins on two things, and they are not small. One is cost at very high volume, because at some point per-minute transcription pricing loses to a GPU you already own. The other is data residency in the strict sense, meaning audio that legally cannot leave your premises. If neither applies to you, the managed path buys back weeks of work you would otherwise spend on chunking heuristics and retry logic. Within Azure, there is also a smaller decision, which is fast transcription against batch transcription. Fast wins on latency and simplicity for files under the size limit. Batch wins when you need to process very large archives asynchronously, when you want webhook notifications on completion, or when you want to bring your own storage account for the outputs. Where to Take It Next The pipeline above is the spine. The interesting extensions hang off the chunking stage, because that is where you decide what the retrieval layer is even capable of answering. Extracting action items into a structured field lets you answer "what did I commit to last month" without any retrieval creativity. Writing a sentiment or disagreement flag onto each turn lets the agent find contested moments directly. Adding a second knowledge source pointed at your specs and design docs turns "what did we decide" into "what did we decide and does the shipped code match", and because a knowledge base fronts multiple sources behind one endpoint, that is a configuration change rather than an architecture change. The part worth protecting as you extend is the evaluation loop. Meeting corpora grow continuously and unevenly, and a retrieval strategy tuned on six months of transcripts behaves differently on three years. The golden set is what tells you when that has happened. References Use the fast transcription APISpeech-to-text REST API referenceWhat is Foundry IQCreate a knowledge base in Azure AI SearchConnect agents to Foundry IQ knowledge basesQuickstart: Get started with the Microsoft Foundry SDKAzure AI Projects client library for Python

By Jubin Soni, FBCS CORE

Originally, back-end and front-end Site Reliability Engineering (SRE) were owned by teams. They code the programs, set up databases and infrastructure, and quickly spring to action at the beep of any anomaly. The advent of code vs no-code infrastructure, SaaS, API dependencies, third parties, and other modern systems seems to be eroding this authority. Mainstream and underdog companies now often leverage the significant advantages of outsourcing, collaboration, or delegation, which are usually accompanied by a silent clause: no or partial control. Unlike in previous systems, modern production is largely assembled rather than built from scratch. For example, a conventional SaaS product is built on interdependencies among payment processors, outsourced data infrastructure such as Amazon Web Services (AWS), messaging services, web hosting, design, AI inference APIs, authentication providers like Google, and more. These useful platforms and products are essentially outside teams' control stations, even though they critically impact users' experience. When they function effectively, you share the glory with the platforms. But when there is a system blackout, your users put you on your toes, even though you have no direct access to resolve the problem on time. Therefore, we shall be exposing SRE practices in platform-SaaS and API-dependent systems and how reliability is getting beyond the control of engineering teams and companies. Why Classical SRE Practices May Fail One major downside of SaaS and dependency on external platforms is that reliability control is often assumed to be in a team's hands, whereas it has been bargained. However, teams must reckon with the fact that the case is reversing. For example, traditional SRE models once alleged that: Service Level Indicators (SLIs) focus on availability or internal uptime and latency.Error budgets arise from changes teams make or deploy.Runbooks still suggest that teams can immediately reconfigure or directly work on faulty components. All these are becoming past cases, especially in platform-SaaS systems. You can have a system indicating 99.99% or even 100% uptime on the back end, while new users are struggling to sign up, probably because an authenticator provider is not fully functional. Dashboards and control panels may indicate green, but in reality, third-party payment APIs have been degraded. A New Definition of Reliability in Operating SRE Practices To resolve the new problem in site reliability engineering (SRE), there needs to be a conceptual shift from component health to an integrated, continuous user experience. Therefore, teams need to undergo a paradigm shift away from questions such as "Is our CPU working maximally?" “Is our API up?” “What are the error rates?” Instead, we should inquire: “Are users checking out seamlessly?” “How fast can they authenticate?” “Can they use the SaaS product to perform its key function?” These types of outcome-based questions span interdependent platforms beyond your full control. The login SLI needs to work with the identity provider; otherwise, its output is meaningless. If the checkout SLO skips payment authorization, then it's both fishy and unreliable. True, there may be some internal errors in a reliable system, but what really matters is an integrated multiplatform experience that the user enjoys. Error Budgets? An SRE Practice to Revisit How many teams would love error budgets to disappear when they give up control? But that’s not so. Instead, they are molecularized. When components of your systems are outsourced, the error budget doesn’t just fade away; it is instead transferred to the interdependent platforms. So, it’s better to plan for the fact that SaaS and API providers will consume some of your reliability budget. Doing so keeps you a few steps ahead and protects your business in the long run. Reliable SRE teams make decisions such as allocating part of their error budget to certain dependencies, setting acceptable parameters for degradation, and defining specific steps to take when a dependency exceeds the stipulated budgets. Here’s an example you can adapt: “We will accept payment authorization failure of 0.0% to 0.2% if it is caused by dependency instability. If it goes above that, we will turn on delayed capture or turn off promotions.” This SRE approach keeps you ready for downtime, as your systems automatically switch to planned or budgeted actions rather than relying solely on integrated platforms. What to Do When Failures Beyond Your Control Arise Actually, some failures may seem beyond your control. The more you attempt to resolve them, the more amplified they become. At this point, your team must adapt to the savvy absorption of such situations. Instead of focusing solely on retrial in an SRE approach, your team needs to design its processes and platforms. This could include failing selectively through circuit breakers, failing fast with timeouts, or failing visibly by keeping users informed. Some core settings should always remain non-negotiable and on standby. These could include the following: Read-only modes/cachesBulkheads that prevent a failure avalanche.Automated circuit breakersDeferred processing These reliable practices ensure there is some form of controlled uptime even when operations seem interrupted. Laser Observability That Proves Reliability In traditional SRE observability, the service boundary is usually the ultimate, but in most modern integrated SaaS platforms, this could be insufficient or worse, dangerous. Operators need to be aware of the actual dependency that is failing, how it is failing (e.g., errors or throttling), and how the failure affects the user experience. Accurate observability for platform-SaaS and API-dependent systems requires these four provisions: Specific dashboard and internal metrics for each vendor.SLI monitoring at the dependency level.Parallel tracing of all outbound calls.Simulation of real-time user experience and workflows. Essentially, whenever there is an emergency, operators should be able to promptly identify whether the source is internal or external. Accuracy and clarity facilitate swift response. Responding to Incidents Without Ownership Another distinct characteristic of modern SRE practice in platform-SaaS is how incidents are responded to. Without ownership, you often cannot debug on your own, roll back a bad deploy, or directly manage other issues. However, you can choose how your system responds by identifying when certain features are disabled, when signals to activate degraded modes are sent, when high traffic is redirected or shed, or when to notify users. To maintain reliability, incident response relies on runbooks to inform decisions. The following questions could help convert the technicality of runbooks to practical solutions: What is the impact on the customer?In what ways can we respond harmlessly?What can we reverse?What should we communicate externally? These questions help resolve incidents, mitigate losses, and intertwine reliability with sound judgment. Is Safety an Illusion in SLAs? SLA providers often readily contract for financial compensation when losses arise, but seldom give absolute reliability guarantees. You may not always expect vendors to consistently meet your availability goals or resolve an avalanche of outages. Safety is a critical consideration when building systems, because when users lose trust in a brand, compensation may not be able to redeem it. Therefore, advanced teams do not consider SLAs as safety nets but as risk pricing. They understand that contractual credits cannot replace trust, brand image, and some almost irredeemable damages. Human Factors in Platform-SaaS and API-Dependent Systems Dependency failures often escalate when cognitive load increases. There could be degraded performance, timeouts without error indicators, partial success, or inconsistent system behavior. Operators may not only focus on machines when dashboards lag or seem to lie. They examine the logs, failure history, or commands. Teams have to design systems with overrides and predictable degradation paths, and observability tools are beyond the failure systems. Reliability goes beyond the correct function of software; it's also about human operations. How Your SaaS and API Platforms Can Imbibe “Good” SRE Practice Effective SRE practices are modern. The following attributes know saas products and API-dependent platforms: Acknowledgment of lack of control very early.Ensuring reliability is embedded in the design.Measuring the outcomes of each SRE criterion or target, instead of just the components.Giving priority to clarity instead of trying to model or control everything because you do not own all the components.Making engineering and operations decisions and products as an integrated whole.Preparing for degradations as inevitable procedures when things fail. Your systems can be reliable if you anticipate failure and accept the reality. Conclusion Modern platform-as-a-service (SaaS) operates in a reliability-without-control manner, leading solid SRE teams to accept that they need to adapt when failures occur. It's simple logic: if you don't absolutely own everything end-to-end, then prepare for the worst: each dependency might fail. It's all about keeping the trust of your users and protecting your brand image.

Let's begin with the definition of an AI agent. Agents are software entities that perform tasks autonomously on behalf of a user or another program. Another way to say it is that agents can perceive the environment, think, and act to achieve a specific goal with minimal human intervention. Action is the key here. For example, if I ask my agent to book a flight from Bengaluru to Delhi. The agent will perform the following tasks. Check the flight availabilityCompare priceAsk for confirmation (Human in the loop)Book the ticket (Action) Now, can we use the same agent for every kind of action? The answer is no. It will be akin to building a monolithic application. Rather, we will prefer an architecture similar to microservices or multiple APIs designed for different functionalities. We will create multiple agents specialized for acting on specific tasks. Let's extend our previous example and think about multiple agents to build a complete travel solution. We have agents such as: Travel Agent → books flightsHotel Agent → reserves hotelFinance Agent → checks budget Now, if we have to achieve a common business goal (booking a flight and hotel after comparing the price), there will be a need for agents' collaboration and interaction. This is where the A2A protocol comes in. A2A is an open protocol that complements Anthropic's Model Context Protocol (MCP). This means MCP standardizes how AI applications connect to data sources, databases, and APIs. A2A focuses on how specialized, autonomous agents (e.g., a "Sales Agent" and a "Finance Agent") "talk" and exchange information to achieve a goal, even if they are built by different providers (OpenAI, Anthropic, Google) and on different frameworks. Agent Card is one of the key capabilities that facilitates communication between Client Agent and Remote Agent. In other words, Agent Card makes A2A possible. Agents can advertise their capabilities using an “Agent Card” in JSON format, allowing the client agent to identify the best agent that can perform a task and leverage A2A to communicate with the remote agent. We can understand agent card with an analogy. You might have seen WSDL file when there is a soap web service is exposed or open api specification for RESTFul apis. WSDL or Open API Specification describes the operations, methods, input, output etc. Similar to this Agent Card make the Agent discoverable which means the agent can actively broadcast its presence, capabilities, and endpoints so that other AI agents or orchestrators can find it and use it automatically, without a human developer having to manually hardcode the connection. (This is analogy is completely from two different software architecture. I have used this for simplifying the visualisation of Agent Card). Agent Card defines the following: What does the agent do?When should this agent be used?What input does this agent expect?What output does it return?What security schemes are supported by the agent?What is the endpoint to call this agent? If we take the previous analogy of an API, each API has a contract that defines input, output, endpoints, methods, etc. Similarly, you can understand an Agent Card as a clear contract for an Agent. JSON { "url": "https://api.travelbot-ai.com/v1/a2a", "documentationUrl": "https://docs.travelbot-ai.com/guide", "capabilities": { "streaming": true, "pushNotifications": true, "stateTransitionHistory": false }, "authentication": { "type": "bearer", "description": "JWT token obtained via OAuth2 client credentials flow." }, "defaultInputModes": ["text"], "defaultOutputModes": ["text", "data"], "skills": [ { "id": "skill-find-flights", "name": "Search Flights", "description": "Finds available flights based on origin, destination, and dates.", "tags": ["travel", "flights", "search"], "InputModes": ["text", "data"], "OutputModes": ["data"], "examples": [ "Find me a one-way flight from JFK to LAX on October 12th." ] }, { "id": "skill-book-hotel", "name": "Reserve Hotel Room", "description": "Books a specific hotel room for given check-in/check-out dates.", "tags": ["travel", "hotels", "booking"], "InputModes": ["data"], "OutputModes": ["text", "data"], "examples": [ "Book the Deluxe King Room at The Grand Hotel from Nov 1 to Nov 5." ] } ] } To see exactly how an Agent Card operates, it helps to look at its structure. In an Agent-to-Agent (A2A) workflow, a client agent requests this card from a server agent before sending a task, establishing exactly how they will interact. The key fields of the agent card are: URL: Where to connect to the agentDocumentationUrl: The user manual/guideCapabilities: What special features it supports (like live streaming or notifications)Authentication: How to securely log in (e.g., passwords, tokens)DefaultInputModes / DefaultOutputModes: How it talks and listens by default (text, audio, data)Skills: A list of specific jobs the agent can do, including details on how each job works To demonstrate this, we can build an agent with an agent card. I will use MuleSoft A2A Task Listener to demonstrate this. Do remember, Agent Card makes Agent-to-agent communication seamless; however, it is not limited to a2a. Any client that we want to connect to an agent and use it will be utilizing the Agent Card to understand the capabilities and skills of the agent. Step 1: Create a project in MuleSoft using the A2A Task Listener. Step 2: Configure A2A. Step 3: Configure the HTTP Listener. Step 4: Deploy the server. Step 5: Retrieve the agent-card using the local URL (http://localhost:8081/support-agent/.well-known/agent-card.json). Step 6: Deploy the code to CloudHub and test it again. You will receive the response as provided below: JSON { "name": "Travel Agent", "description": "Handles flight and hotel booking task.", "url": "https://travel-agent-of3h9v.5sc6y6-3.usa-e2.cloudhub.io/support-agent", "provider": { "organization": "MuleSoft", "url": "https://www.mulesoft.com" }, "version": "1.0.0", "capabilities": { "streaming": false, "pushNotifications": false, "stateTransitionHistory": false }, "defaultInputModes": [ "application/json", "text/plain" ], "defaultOutputModes": [ "application/json", "text/plain" ], "skills": [ { "id": "skill-find-flights", "name": "Search Flights", "description": "Finds available flights based on origin, destination, and dates.", "tags": [ "Flight Booking" ] }, { "id": "skill-book-hotel", "name": "Reserve Hotel Room", "description": "Books a specific hotel room for given check-in/check-out dates.", "tags": [ "Hotel Booking" ] } ], "supportsAuthenticatedExtendedCard": false, "preferredTransport": "JSONRPC", "protocolVersion": "0.3.0" } This will be used by the Client Agent to discover the skills of other agents and send the task request. Please watch the video for step-by-step implementation: I hope this helps. Let me know if you liked it.

It started as a fleeting thought while I was heads-down building agentic AI systems: somewhere between "just call the API" and "let's train our own model," we've quietly ended up with three completely different ways to solve the same problem. Most teams treat that as a single decision, made once, early, and never revisited. It isn't. It's a portfolio you manage for the life of the product. Here's the framework, and why I think most teams have the sequencing backward. The Three Tiers 1. Model API reliance. You call the frontier model, Claude, GPT, Gemini, whichever lab is ahead this quarter, and let its R&D absorb the part of the problem you don't understand yet. This is the right default when you genuinely don't know the shape of the task: when "correct" is still being defined, when volume is low, when the fastest way to learn is to ship and watch what breaks. 2. Fine-tuning open-source models. Once a use case turns out to be repeatable, same shape of input, same shape of output, high enough volume that you're paying real money for it every month, you stop renting intelligence and start owning it. You fine-tune an open-weight model on your own data. You don't have to chase every new open-source release to stay current; you can do this on a slow, deliberate cadence while gradually weaning that specific use case off the frontier API. 3. Migrating to declarative software. Eventually, for the use cases you understand well enough, you don't need a model call at all; you need code. Once you've mapped the edge cases, you write the deterministic pipeline: rules, retrieval, control flow, maybe a small model bolted onto the one genuinely ambiguous step. This is the least glamorous option and the most durable one: reliable, cheap, testable, and not a black box. Why This Feels Backward (and Why It Isn't) Andrej Karpathy's "Software 3.0" framing has been everywhere in AI circles since his 2025 "Software Is Changing (Again)" talk: software moved from Software 1.0 (humans hand-write code) to Software 2.0 (humans train neural network weights) to Software 3.0 (humans write natural-language prompts, treating the model itself as a new kind of programmable computer, with everything in its context window acting as the program). At the frontier, that arc is real; natural language keeps unlocking categories of software that used to require a full engineering team. But zoom into any single feature inside an actual product, and the maturity curve runs the other way. You start at 3.0, a prompt against a frontier model, because that's the fastest way to find out if the idea works at all. Once it works and repeats, you climb down to 2.0: weights you own. Once you fully understand it, you climb down further to 1.0: code you can read. Both arcs are true at the same time. Karpathy's arc is about what becomes possible. This arc is about what becomes worth hardening, once you've learned the actual shape of the problem. The frontier keeps pushing the ceiling up. Underneath it, mature teams keep pushing their own floor down. The Receipts This isn't just a personal theory; it's showing up everywhere once you look for it. Stanford University's DSPy framework is this pattern turned into an actual engineering discipline. Instead of hand-tuning prompt strings forever, you write a declarative "signature" of what a step should do, and a compiler decides, and re-decides, every time the underlying model or data changes, whether that step should run as a prompt, a set of few-shot examples, or fine-tuned weights. The program is code. The model call becomes just one swappable implementation detail inside it. Token prices, meanwhile, keep collapsing. One 2026 analysis of pricing across hundreds of models estimated something like a 600x drop in token costs since 2020, with cheaper model tiers now halving in price faster than Moore's Law ever moved. That actually complicates a naive cost argument for fine-tuning low-stakes, high-volume tasks; the API might already be close to free. What fine-tuning and code increasingly buy you isn't just savings; it's control, latency, and moat. Specialization keeps beating generality on narrow, well-defined tasks. A recent study on structured contract extraction found domain-trained small models matching or beating frontier general-purpose LLMs, at a fraction of the cost and deployable entirely inside enterprise infrastructure. That's tier 2, working exactly as advertised. And not everyone agrees on the timing, which is worth holding onto rather than smoothing over. Some sharp voices in AI investing argue the opposite case: frontier labs will keep out-improving your custom fine-tune faster than you can maintain it, so unless you're sitting on genuinely proprietary data, the better bet is to keep riding the API and pour your effort into the product wrapped around it. That's a real, unresolved tension. It's exactly why this is a portfolio decision and not a fixed rule. The Part Nobody's Actually Managing Here's what I think most roadmaps get wrong: this isn't three sequential stages for your product. It's three tiers running simultaneously, for different capabilities, all the time. Your onboarding flow might already be sitting at tier 3 because you nailed it a year ago. Your newest agentic feature is at tier 1 because you shipped it three weeks ago and don't know its failure modes yet. Something in the middle just crossed the volume threshold where fine-tuning finally pays for itself. That's not a one-time build-vs-buy fork. That's a resource allocation problem, a live one, shifting every quarter as usage patterns, model prices, and your own understanding of the task all move independently of each other. Most AI roadmaps are still built like it's a single decision made once at kickoff. A few questions I've found useful for figuring out where a given capability actually belongs: How often does it run? Low volume, sporadic — stay on the API. The fixed cost of owning it isn't worth paying yet.Is "correct" still moving? If your own definition of a good output changed last month, don't freeze it into weights or code. You'll just have to redo the work.Could a competitor replicate this with the same API call you're making? If yes, it was never your moat. Don't over-invest in owning it.What's your tolerance for a black box? Audit, compliance, and debuggability needs can pull a capability toward code even before the economics demand it.Do you actually have the data? You can't responsibly fine-tune or hard-code what you can't yet describe with real, labeled examples. Where This Leaves Us Having three ways to solve a problem instead of one is genuine abundance. A few years ago, "write the code yourself" was the only option on the table. That's insane! But abundance isn't free; it converts every roadmap into a standing allocation problem: what stays on the frontier, what gets pulled in-house, what gets frozen into something boring and reliable. Decided over and over, forever, as the ground shifts under all three tiers at once. Which of your product's capabilities do you think is sitting at the wrong tier right now?

Google's transition from Manifest V2 to Manifest V3 has been one of the most significant architectural overhauls in the history of browser extension development. For developers building ad blockers, privacy shields, or developer tools, the biggest impact is the deprecation of the blocking capabilities of the chrome.webRequest API. In its place is the chrome.declarativeNetRequest (DNR) API. Instead of letting extensions intercept and inspect network traffic in real-time, the browser now executes filtering on behalf of the extension using declarative rules. Understanding how to design, register, and optimize these declarative rules is essential for building modern web-filtering software. Here is a technical breakdown of the DNR API architecture, rule structure, dynamic rule updates, and current platform constraints. The Architectural Shift: Interception vs. Declaration In Manifest V2, network filtering occurred within the extension's background page or service worker. The extension registered a listener that executed JavaScript on every request before it was sent: JavaScript // The MV2 blocking request pattern (deprecated) chrome.webRequest.onBeforeRequest.addListener( (details) => { if (shouldBlock(details.url)) { return { cancel: true }; } }, { urls: ["<all_urls>"] }, ["blocking"] ); While highly flexible, this design introduced two major problems: Performance Overhead: The browser had to pause network requests, spin up the extension's background process, serialize the request metadata, run the extension's custom JavaScript, and wait for a response.User Privacy: Extensions required the broad <all_urls> permission, giving them access to read every request header, URL query parameter, and POST payload. Manifest V3 solves this by moving the execution engine into the browser itself. The extension defines what needs to be blocked or redirected beforehand. The browser reads these rules and applies them natively during the network stack lifecycle. The extension’s code is never executed during the request, which reduces memory consumption and protects user privacy. The Anatomy of a Declarative Rule Under the DNR model, everything is defined using rules. Each rule is a JSON object that specifies an action and the conditions under which that action should execute. Here is the standard structure of a declarative rule: JSON { "id": 1, "priority": 1, "action": { "type": "block" }, "condition": { "urlFilter": "||doubleclick.net", "resourceTypes": ["script", "sub_frame"] } } Every rule requires four primary keys: id: A unique integer (1 or greater) that identifies the rule.priority: An integer indicating order of execution. Rules with higher priority numbers override lower priority rules.action: Specifies what the browser should do when a match occurs. Valid types include block, redirect, allow (bypasses other blocks), allowAllRequests (bypasses all rules on a page), and modifyHeaders.condition: The criteria that must be met to trigger the action. This can filter by domain, URL pattern, initiator origin, request method, or resource type (such as image, xmlhttprequest, or stylesheet). Implementing Static Rulesets Extensions can bundle pre-defined rule lists within their distribution package. These are defined as static JSON files and declared in the manifest.json: JSON { "name": "Custom Focus Blocker", "version": "1.0", "manifest_version": 3, "permissions": ["declarativeNetRequest"], "declarative_net_request": { "rule_resources": [{ "id": "ruleset_social", "enabled": true, "path": "rules/social.json" }] } } The referenced social.json file contains an array of rules: JSON [ { "id": 101, "priority": 1, "action": { "type": "block" }, "condition": { "urlFilter": "||facebook.com", "resourceTypes": ["main_frame"] } } ] Managing Dynamic Rules Programmatically Static rulesets are read-only once compiled into the extension package. To allow users to add custom blocked domains or configure personal schedules, you must update the extension's dynamic rules at runtime. Chrome provides chrome.declarativeNetRequest.updateDynamicRules to modify rules programmatically. This method accepts arrays of rules to remove and rules to add. Here is a JavaScript helper class to manage dynamic site blocking: JavaScript class BlocklistManager { // Add a domain to the dynamic blocklist static async addDomain(ruleId, domain) { const newRule = { id: ruleId, priority: 1, action: { type: 'block' }, condition: { urlFilter: `*://${domain}/*`, resourceTypes: ['main_frame', 'sub_frame'] } }; await chrome.declarativeNetRequest.updateDynamicRules({ removeRuleIds: [ruleId], // Remove old rule with same ID to prevent duplicates addRules: [newRule] }); } // Remove a rule from the active dynamic set static async removeRule(ruleId) { await chrome.declarativeNetRequest.updateDynamicRules({ removeRuleIds: [ruleId] }); } // Retrieve all currently active dynamic rules static async getActiveRules() { return await chrome.declarativeNetRequest.getDynamicRules(); } } Session Rules vs. Dynamic Rules In addition to dynamic rules, Manifest V3 introduces Session Rules via the chrome.declarativeNetRequest.updateSessionRules API. Dynamic Rules: Persist across browser restarts and extension updates. They are stored in Chrome's internal extension storage.Session Rules: Saved purely in memory. They are cleared when the browser session ends, or the extension is reloaded. Session rules are ideal for temporary focus sessions, one-time study blocks, or incognito mode rules that should not write data permanently to the disk. Modifying HTTP Headers The DNR API also supports modifying HTTP request and response headers natively using the modifyHeaders action. This is useful for removing tracking cookies, injecting authentication tokens, or overriding Referrer headers. Here is a rule structure that strips the Cookie header from requests sent to a third-party tracking domain: JSON { "id": 201, "priority": 2, "action": { "type": "modifyHeaders", "requestHeaders": [ { "header": "cookie", "operation": "remove" } ] }, "condition": { "urlFilter": "||tracker-domain.com", "resourceTypes": ["xmlhttprequest", "sub_frame"] } } Platform Constraints and Rule Limits Because the browser must parse and evaluate all active rules in linear time to avoid latency, Google enforces strict limits on the number of rules you can register: Static Rulesets: An extension can declare up to 100 static rulesets, but only a limited number can be enabled simultaneously (typically 50).Dynamic and Session Rules: Extensions are limited to 5,000 dynamic rules and 5,000 session rules.Regex Filter Performance: You can use regular expressions in the regexFilter key under conditions, but the regex patterns must conform to a restricted syntax. Lookaheads, lookbehinds, backreferences, and lazy quantifiers are disabled to guarantee that matching runs in linear time. If a regex pattern is too complex, the API will fail to register the rule. Conclusion and Best Practices When building extensions under Manifest V3: Use Priorities Wisely: Use higher priority values for user-defined whitelists to ensure they override system-level blocklists.Minimize Rule Count: Instead of creating separate rules for sub.domain.com and domain.com, use wildcard patterns or regex expressions to group matches into single rules.Optimize Storage: Clean up unused dynamic rule IDs periodically. Retrieve active rules using getDynamicRules() to prevent collisions. By moving execution to the browser engine, Manifest V3 requires developers to change their approach to web filtering. Designing within these declarative constraints ensures your extension runs efficiently without compromising user privacy.

Every B2B SaaS product team knows this moment. You're trying to close a deal, and the prospect says, "We just need you to sync with our CRM. And our HRIS. Oh, and these three other tools. You can do that, right?" Your roadmap takes a hit, and your engineering backlog doubles overnight. And eventually someone says, "What about a unified API?" It sounds like the answer — one normalized schema, one auth model, and one point of connection for a dozen or more apps in a vertical. You buy it, hook it up, and ship the integrations before the quarter ends, the integration checkbox gets checked, and you move on. For a while, it works. But there's a problem most teams don't see until they start moving upmarket. For many SaaS teams, a unified API is the right first move. It's rarely the right last one. Unified APIs Exist for a Reason, and They're Good at What They Do Most apps in a category share the same data objects. CRMs have contacts, accounts, opportunities, and activities. HRIS platforms store employee, department, and compensation data. Ticketing systems track tickets, users, and statuses. A unified API vendor abstracts the data models for an app category into a common schema so that, rather than learning a dozen APIs, your devs learn one. For startups under pressure to ship quickly, that abstraction is valuable. You can launch integrations faster, reduce engineering work, and simplify auth across the board. If your customers need common objects and standard workflows, a unified API can meaningfully accelerate your roadmap. That's all positive. The negative shows up down the road. The Lowest Common Denominator Problem A normalized data model (which is what a unified API is based on) is, by definition, a reduced or simplified data model. To present a single schema across N apps, a unified API must identify the fields they have in common. The result is a model built on the smallest shared dataset. Anything that's app-specific is abstracted away, and anything proprietary is dropped. Unified APIs work until your customers stop being generic. Enterprise customers have Salesforce custom objects built for their unique processes. They have Workday compensation structures that don't fit a normalized HRIS schema. They have vertical-specific fields that are critical to their business processes. And, it's increasingly common for them to be running systems that the unified API vendor has never heard of. The moment a prospect asks you to sync a custom object, access a proprietary field, or connect to an app outside your unified API vendor's supported list, the abstraction layer is no longer sufficient. You either tell your prospect "No" or you build a custom, one-off integration anyway, which largely defeats the point of a unified API. At first, these seem like edge cases. Then you realize enterprise customers are the edge cases. And that they are bringing the highest-value deals in your pipeline. The "Zero Maintenance" Promise Doesn't Hold Up The biggest marketing claim of a unified API is that upstream API changes are no longer your problem: "They update their API, we handle the change." In reality, you're trading one type of maintenance for another. With native APIs, you worry about endpoint deprecations, auth updates, and rate limits. With a unified API, you worry about data lost in translation or debugging through an abstraction layer. When that happens for an enterprise customer, you can't just look at the target system's logs. You have to work through the unified API provider's black box. If the root cause is a nuance in how they handle a specific app's rate-limiting rules, your engineering team is now waiting on someone else's support ticket queue. The maintenance didn't go away. It just moved down the street. Complexity Comes Later The full cost of a unified API strategy rarely appears during implementation. Instead, it waits until things have settled into a steady rhythm and then shows up as operational complexity. Dual integration architectures – Once you need custom integrations alongside your unified API (and you will), your team will maintain two separate integration layers with different auth flows, error handling, retry logic, and monitoring. Every integration request now needs to go through a decision tree to determine which of these patterns (or perhaps even a new one) you should use for development.Data model constraints – Your app connects with the unified API's schema rather than to the underlying apps. When customers ask for fields the schema doesn't expose, your team builds manual workarounds, relocating rather than reducing the complexity.Vendor roadmap dependency – If your unified API provider doesn't support a specific endpoint, a webhook behavior, an advanced API feature, or a vertical SaaS platform your customer uses, you wait (or you build around it). Either way, the original value proposition isn't holding up to the rigors of reality.Escalation cost – Enterprise prospects bring technical evaluators. When those evaluators discover that your integration can't provide the specific data they depend on, the deal may end right there. That's not good for your bottom line. What the Workaround Trap Looks Like Most teams respond the same way when they hit these limits. They start building custom integrations in addition to those handled through the unified API. What began as a simplification strategy is starting to look like this: a unified API for common integrations, direct API connections for exceptions, custom middleware for unsupported workflows, separate auth handling, multiple sync models, and one-off transformation logic wherever it's needed. In short, that neatly ordered integration layer is no longer. The abstraction created to reduce maintenance has, in fact, increased it. Teams find they're burning an appreciable portion of their integration budget maintaining low-value integrations and working around the things their unified API vendor can't support. That's engineering time that isn't being devoted to your core product. Vertical SaaS Is the Forcing Function The continued fragmentation of B2B software makes this worse every year. Beyond mainstream CRMs and HR platforms, companies increasingly rely on industry-specific applications: systems narrowly designed and built for healthcare, manufacturing, financial services, and a score of other verticals. These systems rarely conform to standardized schemas. Many of them don't appear in any unified API vendor's list of supported apps. A unified API might help you connect to ten generic CRMs. It won't help much when your largest prospect is running Epic, Procore, or a heavily customized NetSuite instance. Those are the integrations that determine whether enterprise deals close. What Happens at Scale Unified APIs are usually evaluated based on how fast they help teams launch. However, the more important question is: "What happens when integration requirements grow more complex?" Because they always do. Every single time. As SaaS products mature, integration requests shift from "Can you connect to this category?" to "Can you support this exact workflow?" That move exposes the architectural limits of a unified API. And the teams that hit the limit mid-deal (or mid-contract) feel the immediate pain. Why Embedded iPaaS Is the Durable Foundation This is where embedded iPaaS platforms fundamentally differ from unified APIs: they aren't constrained to a single simplified schema. An embedded iPaaS gives your team a flexible integration foundation that handles both ends of the spectrum: the common apps that benefit from productized integrations, and the complex, vertical-specific, niche apps that don't fit any standardized model. Some of your customers need a basic CRM sync. Others need multi-flow orchestration, conditional business logic, extensive data mapping, and more. A rigid abstraction model breaks under those requirements. An embedded iPaaS doesn't. This Isn't "Unified APIs vs. Embedded iPaaS" Unified APIs still have value. For early-stage validation or straightforward category integrations at scale, they can accelerate time-to-market. Many mature teams use them alongside a more flexible platform for the scenarios where standardization works. But for most B2B SaaS teams, they are a way-station, not the destination. The mistake teams make is assuming the abstraction can scale indefinitely as customer complexity increases. But that's not true. It can't, and it doesn't. The bigger and more complex your customers get, the more a lowest-common-denominator approach becomes an obstacle instead of a shortcut.

Artificial intelligence has rapidly become a core capability in modern software development. For Java developers, integrating these capabilities into existing enterprise applications no longer requires learning entirely new frameworks or interacting directly with complex AI APIs. Spring AI bridges this gap by providing a familiar Spring programming model for working with large language models (LLMs) from providers such as OpenAI, Google Gemini, and others. In this article, we will build a simple AI-powered REST API using Spring Boot and Spring AI while exploring practices that help move beyond proof-of-concept implementations toward production-ready enterprise applications. A Typical Enterprise Architecture Rather than allowing clients to communicate directly with an AI provider, enterprise applications usually introduce a service layer responsible for security, validation, business logic, and monitoring. Plain Text Client Application │ ▼ Spring Boot REST API │ Validation & Business Logic │ ▼ Spring AI ChatClient │ ▼ Large Language Model (OpenAI / Gemini / Azure) This architecture keeps AI interactions behind your own APIs, allowing you to enforce authentication, authorization, logging, rate limiting, and governance without exposing provider-specific details to consumers. Creating the Spring Boot Project Getting started with Spring AI is straightforward. The application requires Spring Web, Validation, and the Spring AI starter. XML <properties> <java.version>21</java.version> <spring-ai.version>1.0.0</spring-ai.version> </properties> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-bom</artifactId> <version>${spring-ai.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-validation</artifactId> </dependency> <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-starter-model-openai</artifactId> </dependency> </dependencies> Configuring the AI Model One security practice I strongly recommend is avoiding hard-coded API keys or model names inside the application. Instead, configure them using environment variables or an enterprise secrets manager. YAML spring: ai: openai: api-key: ${OPENAI_API_KEY} chat: options: model: ${OPENAI_MODEL} temperature: 0.2 The lower temperature value encourages more deterministic responses, which is generally preferable for technical or business APIs where consistency matters. Designing the API Contract Rather than exposing raw AI requests directly, I prefer defining explicit request and response models. This keeps the REST API independent of the underlying AI provider and makes future changes much easier. Java import jakarta.validation.constraints.NotBlank; import jakarta.validation.constraints.Size; public record AIQuestionRequest( @NotBlank @Size(max = 2000) String question, String audience ) {} Response model: Java public record AIAnswerResponse( String answer ) {} Configuring the ChatClient Spring AI's ChatClient is responsible for interacting with the configured language model. Rather than repeating the same instructions in every request, we can configure a default system prompt once. Java @Configuration public class AIConfiguration { @Bean ChatClient chatClient(ChatClient.Builder builder) { return builder .defaultSystem(""" You are an experienced Java architect. Provide concise, accurate, production-ready answers. Never invent APIs. If uncertain, clearly state your assumptions. """) .build(); } } The system prompt establishes the overall behavior of the assistant. It ensures that every request follows the same guidelines, resulting in more predictable responses. Implementing the AI Service One architectural decision I recommend is keeping AI interactions inside a dedicated service layer rather than calling the language model directly from a controller. This separation makes the code easier to test, improves maintainability, and keeps business logic independent of the web layer. Java @Service public class TechnicalAssistantService { private final ChatClient chatClient; public TechnicalAssistantService(ChatClient chatClient) { this.chatClient = chatClient; } public AIAnswerResponse answer(AIQuestionRequest request) { String audience = request.audience() == null ? "Java Developer" : request.audience(); String response = chatClient.prompt() .user(user -> user .text(""" Explain the following question. Audience: {audience} Question: {question} Keep the answer under 300 words. """) .param("audience", audience) .param("question", request.question())) .call() .content(); return new AIAnswerResponse(response); } } Creating the REST Controller With the service layer complete, exposing the AI functionality through a REST endpoint becomes straightforward. Java @RestController @RequestMapping("/api/ai") public class AIController { private final TechnicalAssistantService assistantService; public AIController(TechnicalAssistantService assistantService) { this.assistantService = assistantService; } @PostMapping("/ask") public ResponseEntity<AIAnswerResponse> ask( @Valid @RequestBody AIQuestionRequest request) { return ResponseEntity.ok( assistantService.answer(request)); } } The endpoint accepts a JSON request, validates the input, invokes the service layer, and returns a structured response. Returning Structured AI Responses Many AI examples simply return text. While that's useful for chat applications, enterprise APIs usually need predictable JSON responses. For example, suppose we want AI to review Java code. Instead of receiving one long paragraph, we can ask the model to return structured data. Java public record CodeReviewResponse( String summary, List<String> strengths, List<String>issues, List<String>recommendations, String riskLevel ){} Now Spring AI can map the model response directly into a Java object. Java public CodeReviewResponse review(String sourceCode){ return chatClient.prompt() .system(""" You are a Senior Java Architect. Review the code for correctness, performance, security and maintainability. """) .user(sourceCode) .call() .entity(CodeReviewResponse.class); } This approach is much cleaner than parsing raw JSON or trying to interpret free-form responses manually. It also keeps the rest of the application strongly typed. Streaming AI Responses Some AI responses can take several seconds to complete. Rather than waiting until the entire response has been generated, Spring AI allows responses to be streamed back to the client. Java @RestController @RequestMapping("/api/ai") public class StreamingController { private final ChatClient chatClient; public StreamingController(ChatClient chatClient) { this.chatClient = chatClient; } @GetMapping( value="/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE) public Flux<String> stream( @RequestParam String question){ return chatClient.prompt() .user(question) .stream() .content(); } } Streaming significantly improves the user experience because clients can begin displaying the answer immediately instead of waiting for the complete response. This is especially useful for chat applications and AI assistants. Cache Responses When Appropriate AI requests introduce additional latency and cost because every request communicates with an external model. If the same prompt is frequently submitted, consider caching the response. Spring Cache makes this simple. Java @Service public class TechnicalAssistantService { @Cacheable("aiResponses") public AIAnswerResponse answer(AIQuestionRequest request) { // AI Call } } Caching works particularly well for frequently asked questions, product descriptions, technical explanations, and internal knowledge articles. Dynamic or user-specific responses generally should not be cached unless the cache key includes the relevant context. Final Thoughts What stands out to me is that Spring AI allows AI capabilities to become a natural extension of an existing Spring Boot application rather than requiring an entirely new architecture. Whether the goal is building an internal knowledge assistant, generating summaries, reviewing code, or automating repetitive tasks, the development experience remains consistent with the rest of the Spring ecosystem. That said, building a production-ready AI application involves much more than calling an LLM. Prompt design, security, validation, observability, performance, and cost management all play a critical role in delivering reliable solutions.

This guide explains zone-aware routing from a Kubernetes-first point of view. It covers: why zones matter in cloud platformswhich topology labels Kubernetes places on nodeshow Kubernetes first tried to solve locality through Servicewhat gaps remained after those Service-based featureshow Gateway API implementations such as Envoy Gateway and kgateway built on top of that foundation Why Zones Matter In cloud platforms, a zone is a logical failure domain inside a region. Zones usually have low-latency networking within the zone, but crossing zones can increase both latency and cost. That cost is not theoretical. AWS documents that traffic within the same Availability Zone is free, while traffic that crosses Availability Zones typically incurs data transfer charges, and cross-zone transfer is generally billed in both directions, so a single round trip can be charged twice. See: AWS Architecture Blog: Overview of Data Transfer Costs for Common ArchitecturesAmazon EC2 pricing: Data Transfer This is one reason distributed systems try to keep traffic local when they can, while still preserving failover to other zones. The Topology Information Kubernetes Already Has Kubernetes did not start by inventing zone-aware traffic policies. It started by carrying topology information on nodes. The two most important well-known labels are: topology.kubernetes.io/regiontopology.kubernetes.io/zone According to the Kubernetes reference, these labels are populated on Node objects by the kubelet or the external cloud-controller-manager when the cluster is integrated with a cloud provider. In non-cloud environments, operators can set them manually if the topology model still makes sense. Reference: Kubernetes well-known labels: topology.kubernetes.io/zone In managed clusters, these labels are commonly present by default. Here is the kind of node data Kubernetes typically exposes: YAML apiVersion: v1 kind: Node metadata: name: ip-10-0-12-34.ec2.internal labels: kubernetes.io/hostname: ip-10-0-12-34.ec2.internal topology.kubernetes.io/region: us-east-1 topology.kubernetes.io/zone: us-east-1a That topology data is useful for scheduling, spreading replicas, volume placement, and eventually traffic routing. The Original Service Model The original Kubernetes Service abstraction solved a different problem first: stable discovery and virtual IPs for ephemeral Pods. At the beginning, the model was simple: a Service selected a set of Podskube-proxy programmed forwarding rulestraffic could be sent to any healthy endpoint behind the Service That was excellent for reachability and abstraction, but it had no built-in notion of zone locality. The gap was straightforward: the Service abstraction knew which endpoints existed, but not that a client in zone-a should usually prefer endpoints in zone-a. Kubernetes' First Attempts to Improve Locality Through Services Kubernetes gradually added locality-aware behavior on top of Service, mostly by improving how endpoint selection works. Internal Traffic Policy One early mechanism was internalTrafficPolicy: Local. This tells kube-proxy to use only node-local endpoints for cluster-internal traffic. Example: YAML apiVersion: v1 kind: Service metadata: name: my-service spec: selector: app: my-app ports: - port: 80 targetPort: 8080 internalTrafficPolicy: Local Reference: Kubernetes Service Internal Traffic Policy This helps with node locality, but it is not zone-aware routing. Its limitations are important: it is node-local, not zone-localif a node has no local endpoint, the Service behaves as if it has zero endpoints from that node's perspectiveit is too strict for many multi-zone workloads that want zonal preference, not node affinity So this was useful, but it did not really solve multi-zone locality. Topology Aware Routing With Services Kubernetes next introduced Topology Aware Hints, now called Topology Aware Routing. This works through two components: The EndpointSlice controller looks at endpoint and node topology.kube-proxy consumes hints from EndpointSlices and prefers endpoints closer to the client zone. Historically, the Service-side configuration was commonly exposed through the service.kubernetes.io/topology-mode: Auto annotation: YAML apiVersion: v1 kind: Service metadata: name: zone-aware-backend annotations: service.kubernetes.io/topology-mode: Auto spec: selector: app: backend ports: - port: 80 targetPort: 8080 Conceptually, the flow looks like this: This was Kubernetes' first real zone-aware answer at the Service layer. It is useful historical context, but it is no longer the clearest Service-level API to emphasize for new users. Traffic Distribution Preferences Kubernetes later added trafficDistribution as a clearer way to express routing preferences. In current Kubernetes documentation, the relevant zone-level preference is: PreferSameZone The older PreferClose name is documented as deprecated in favor of PreferSameZone, though you may still see PreferClose in some provider and implementation docs that have not yet caught up. Example: YAML apiVersion: v1 kind: Service metadata: name: zone-aware-backend spec: selector: app: backend ports: - port: 80 targetPort: 8080 trafficDistribution: PreferSameZone Reference: Kubernetes Service trafficDistribution This is a better API shape than older annotations because it is explicit in the Service spec and described as a preference rather than a strict guarantee. In practice, that means current Kubernetes guidance emphasizes trafficDistribution: PreferSameZone, while the older topology-mode: Auto path is best understood as part of the feature's evolution. What Gap Remained After Service-Based Locality Kubernetes Services improved a lot, but they still left several gaps. The Behavior Is Best Effort Topology-aware routing is not a hard guarantee. Kubernetes documents multiple safeguard cases where the system falls back to cluster-wide routing. Examples include: too few endpointsimpossible balanced allocationmissing topology labels on one or more nodesmissing hints for one or more endpointsno hinted endpoint for the local zone That is correct for safety, but it means the behavior is heuristic and conditional. It Assumes a Certain Traffic Shape Kubernetes explicitly documents that Topology Aware Routing works best when traffic is roughly evenly distributed and when there are enough endpoints per zone. If most traffic originates from one zone, local subsets can overload while the global service still looks healthy. It Is Scoped to the Service Datapath This is the most important architectural gap. Service-level topology features influence how kube-proxy chooses endpoints for Service traffic. They do not automatically solve every higher-level data plane. In particular, they do not by themselves define: how an L7 gateway proxy should understand its own zonehow an Envoy-based gateway should configure locality-aware upstream load balancinghow a gateway controller should express stricter local preference versus simple best-effort localityhow policy should attach to particular routes, gateways, or backends That left room for Gateway API implementations to expose richer locality controls. Why Gateway API Implementations Stepped In Gateway API is intentionally expressive and extensible. It standardizes core routing objects, but implementations often add policy CRDs to expose features that are specific to their data plane. That distinction matters here: Gateway API itself does not define one universal, cross-implementation zone-aware policy. Instead, it gives implementations room to expose locality behavior in a way that matches their proxy and control-plane design. Reference: Gateway API overview This is where zone-aware routing became more explicit at the gateway layer. Instead of relying only on kube-proxy's Service behavior, gateway implementations can: understand the proxy's own localityread backend endpoint localityconfigure the underlying proxy's load balancer directlyexpose locality policies as route or backend-attached configuration Example of How Envoy Gateway Addresses the Gap Envoy Gateway supports two paths: Reusing Kubernetes Service-level locality such as Topology Aware Routing or trafficDistributionConfiguring zone awareness directly through BackendTrafficPolicy Reference: Envoy Gateway zone-aware routingEnvoy zone-aware routing Example BackendTrafficPolicy: YAML apiVersion: gateway.envoyproxy.io/v1alpha1 kind: BackendTrafficPolicy metadata: name: zone-aware-routing spec: targetRefs: - group: gateway.networking.k8s.io kind: HTTPRoute name: zone-aware-routing loadBalancer: type: RoundRobin zoneAware: preferLocal: minEndpointsThreshold: 1 force: minEndpointsInZoneThreshold: 1 That is a meaningful step beyond plain Service because the gateway layer is now explicitly participating in locality-aware upstream balancing. Example of How kgateway Addresses the Gap kgateway takes a similar approach in spirit: proxy locality is made explicit, and backend load-balancing behavior is configured through policy rather than relying only on Service heuristics. At a high level, kgateway combines: Gateway proxy locality configurationBackend-attached load-balancing policyNative Envoy locality-aware upstream load balancingEndpoint locality metadata that Envoy can use directly Architectural Summary The progression looks like this: Kubernetes Service solved stable discovery and reachability.internalTrafficPolicy improved node-local routing, but not zonal routing.Topology Aware Routing and trafficDistribution added zone-aware preferences to the Service datapath.Gateway API implementations extended the model so L7 gateways and proxies could make explicit locality-aware decisions themselves. Practical Takeaways Kubernetes already provides the topology metadata needed for zone-aware decisions.Service-native locality is useful, but it is heuristic and scoped to the Service datapath.Zone-aware traffic for gateways usually needs the gateway implementation to understand locality too.Modern Gateway API implementations fill that gap by attaching locality-aware load-balancing policy closer to the L7 data plane. Where Zone-Aware Routing Matters in Practice Zone-aware routing usually becomes worth the added operational attention when one or both of these are true: The workload has a tight latency budget, especially at p95 or p99The system moves enough east-west traffic that even a small per-GB cross-zone charge becomes material Common examples include: Gaming platforms, where matchmaking, player session state, inventory, and real-time coordination are sensitive to a few extra milliseconds of network delayFinancial services, where payment, quote, fraud, or checkout paths care more about predictable tail latency than average latencyLarge SaaS and enterprise control planes, where a gateway fans out to many internal APIs and the aggregate cross-zone traffic becomes a real monthly costAI inference, media delivery, logging, and telemetry pipelines, where payload sizes are large enough that bandwidth cost matters even when latency is less critical Worked Example: Multiplayer Gaming Backend Suppose a regional game API runs gateway proxies and backend pods in three zones. Players connect to a gateway in zone-a, and that gateway calls a player-state service that is also deployed in zone-a, zone-b, and zone-c. Assume the following: 25,000 requests per second reach the player-state service from zone-athe combined request and response payload is about 40 KiB per callcross-zone traffic is billed at a representative $0.01 per GBwithout zone awareness, only about one third of those calls stay in zone-a, while the other two thirds go to zone-b or zone-c Actual billing varies by provider, region, and direction of transfer, but the point of the example is that a seemingly small per-GB rate compounds quickly on hot service paths. That means the traffic volume from zone-a to the player-state service is about: 25,000 x 40 KiB per second, or roughly 1 GB/s totalif two thirds of that traffic crosses zones, that is about 0.67 GB/s of cross-zone trafficover a 30-day month, that is about 1.7 million GBat $0.01 per GB, that is about $17,000 per month in cross-zone transfer for just that one service path That is the cost side. The latency side can matter even more for the player experience. If each cross-zone hop adds only 1-3 ms, a request path that fans out to several internal services can add multiple milliseconds of extra tail latency. For a gaming workload, that can affect: matchmaking responsivenesssession join timethe smoothness of player state or presence updateshow stable the system feels during traffic spikes and retries This is why zone-aware routing is not only a cost optimization. In some industries, it is a user-experience and SLO control. Worked Example: Large SaaS Control Plane The same logic applies outside gaming. Consider a large enterprise SaaS platform where each incoming API request hits a gateway and then fans out to an auth service, tenant metadata service, feature-flag service, and audit pipeline. Even if each individual backend call is small, the gateway can generate a large amount of aggregate east-west traffic. In that kind of system, zone-aware routing helps in two ways: it removes avoidable cross-zone traffic from the steady-state hot pathit reduces the chance that a multi-hop request burns several extra milliseconds just on internal network distance For that kind of platform, the business case is usually a combination of lower regional data-transfer cost, tighter latency distributions, and better failure-domain alignment. Conclusion Zone-aware routing is the story of a single idea moving down the stack. Kubernetes started with topology labels on nodes, then taught the Service datapath to prefer local endpoints through internalTrafficPolicy, Topology Aware Routing, and trafficDistribution. Those features are valuable, but they are best-effort and they stop at the Service boundary, which leaves L7 gateways unable to reason about their own locality. Gateway API implementations such as Envoy Gateway and kgateway pick the idea up from there, making proxy locality explicit and pushing locality-aware load balancing into Envoy where it can act on real endpoint metadata. The practical guidance is short. Start with the Service-native controls, because they are simple and often enough. Reach for gateway-level locality policy when you have a tight tail-latency budget, or enough east-west traffic that cross-zone transfer becomes a line item you can see. In both cases, the goal is the same: keep traffic local when you safely can, and fail across zones when you must. Further Reading Kubernetes ServiceKubernetes Topology Aware RoutingKubernetes Service Internal Traffic PolicyKubernetes well-known topology labelsGateway API overviewAWS Architecture Blog: Data transfer costs

The evolution from monolithic applications to microservices transformed enterprise software by decomposing business capabilities into independently deployable services. REST APIs, asynchronous messaging, and service discovery enabled systems that scaled both organizationally and technically. Although this model remains effective for deterministic business logic, the emergence of AI agents introduces a different execution paradigm. Instead of invoking predefined endpoints, an agent receives an objective, reasons about available capabilities, selects appropriate services, and dynamically composes a workflow. This shift changes service boundaries from business functionality to decision-making and capability orchestration. Why This Matters Traditional microservices assume that applications already know which services to invoke. An Order Service calls Inventory, Payment, and Shipping because the workflow is explicitly encoded during development. An AI agent, however, begins with an intent rather than an execution path. A request such as "purchase the least expensive laptop available and deliver it tomorrow" requires evaluating inventory, pricing, promotions, shipping constraints, and fraud policies before any API is called. The workflow is determined during execution instead of implementation. A conventional orchestration service typically resembles the following implementation. Java public OrderResponse checkout(OrderRequest request) { Inventory inventory = inventoryClient.reserve(request); Payment payment = paymentClient.authorize(request); Shipping shipment = shippingClient.schedule(request); return new OrderResponse(payment, shipment); } The implementation is deterministic because every dependency is known beforehand. Adding another payment gateway or shipping provider requires modifying orchestration logic, gradually increasing coupling between services. As enterprises integrate AI-driven workflows, continuously extending predefined execution paths becomes increasingly difficult. Agent Services replace hardcoded dependencies with capability discovery. Rather than directly invoking an Inventory Service, the runtime identifies which registered capability satisfies the current intent. Java public Tool resolve(Intent intent) { return toolRegistry.stream() .filter(tool -> tool.supports(intent)) .findFirst() .orElseThrow(() -> new ToolNotFoundException(intent.name())); } The registry enables services to advertise capabilities instead of exposing only procedural APIs. Existing microservices remain responsible for inventory reservation, payment authorization, or shipment scheduling, but the responsibility for deciding which capability should execute moves into an intelligent coordination layer. New business capabilities can therefore be introduced without rewriting orchestration code. This distinction fundamentally changes API design. Traditional REST endpoints expose operations such as /reserveInventory or /authorizePayment. Agent-oriented systems instead expose semantic capabilities like "find lowest cost supplier," "recommend shipping option," or "detect payment risk." These descriptions allow planning engines to reason about business objectives instead of matching endpoint names. Reasoning requires an additional architectural component capable of translating natural language into executable plans. This responsibility belongs to an Intent Router, which functions similarly to an API Gateway but routes requests based on semantic meaning rather than URLs. Java public ExecutionPlan plan(String goal) { Intent intent = classifier.classify(goal); Tool tool = registry.resolve(intent); return planner.create(tool, goal); } The classifier converts an objective into structured intent, the registry discovers an appropriate capability, and the planner generates an execution strategy. Once planning completes, downstream execution remains deterministic. Large language models participate only during reasoning, while conventional microservices continue enforcing validation rules, transactional consistency, and domain constraints. Separating planning from execution preserves enterprise reliability while introducing adaptive behavior. This separation also dispels a common misconception that AI agents replace microservices. Business logic continues to belong inside deterministic services because payment authorization, inventory consistency, pricing calculations, and compliance rules require predictable execution. Agent Services instead provide an intelligent layer responsible for selecting, coordinating, and sequencing those services according to business objectives. Rather than replacing existing architectures, they extend them with decision-making capabilities that previously existed only inside application code. Consequently, service boundaries begin shifting away from business entities toward reusable decision engines. Instead of embedding procurement, logistics, or fraud decisions inside multiple applications, organizations can expose these responsibilities as independent Agent Services that orchestrate existing microservices. The underlying APIs remain stable while reasoning evolves independently, enabling enterprise systems to become progressively more adaptive without sacrificing the deterministic foundations that made microservice architectures successful. Taking Memory Into Account Memory becomes the next architectural concern once planning is separated from execution. Stateless REST requests work well for isolated transactions, but agents frequently solve objectives through multiple reasoning cycles. Intermediate decisions, retrieved knowledge, user preferences, and execution history must persist beyond a single request. This context is operational rather than transactional. Business entities continue residing in relational databases, while the agent memory layer preserves reasoning state that enables future decisions to remain consistent. Java public AgentContext update(String sessionId, Observation observation) { AgentContext context = repository.load(sessionId); context.append(observation); repository.save(context); return context; } Rather than storing business records, the memory layer continuously enriches execution context with observations generated during planning. Future reasoning cycles consume this accumulated context instead of repeatedly querying downstream services, reducing redundant tool execution while maintaining continuity across long-running workflows. As objectives become more sophisticated, a single agent rarely owns every required capability. Instead of directly invoking multiple APIs, an agent can delegate specialized responsibilities to another agent while maintaining overall coordination. This interaction is based on expertise rather than ownership, allowing procurement, logistics, compliance, or fraud agents to evolve independently while sharing the same underlying microservices. Java AgentResponse response = logisticsAgent.execute( new AgentTask( "Optimize shipping route", context)); Delegation transfers structured objectives instead of procedural API calls. Each agent independently plans its assigned task before returning a deterministic result. Existing Inventory, Payment, and Shipping services remain unchanged, while the coordination layer becomes modular and extensible. Observability Implications Observability must also evolve because traditional distributed tracing explains service execution but not decision making. Understanding why an agent selected one capability over another is equally important as measuring latency or availability. Reasoning traces therefore become first-class telemetry alongside conventional application metrics. Java Span span = tracer.nextSpan() .name("agent.plan"); span.tag("goal", goal); span.tag("selectedTool", tool.name()); span.tag("confidence", score.toString()); span.end(); Capturing planning metadata allows engineering teams to correlate business outcomes with reasoning quality. An operation may succeed technically while producing an incorrect recommendation because the planner selected an unsuitable capability. Monitoring therefore expands beyond response times to include tool selection, planning confidence, execution cost, and reasoning latency. Autonomous planning also introduces governance challenges. Traditional services authorize callers before executing business logic, whereas Agent Services must additionally validate that planners invoke only approved capabilities. Every tool should expose explicit permissions and execution policies so that reasoning engines remain constrained by enterprise governance regardless of how plans are generated. Java public ToolResult execute(AgentTask task) { policyEngine.authorize(task.agent(), task.tool()); return toolExecutor.run(task); } Separating authorization from planning ensures deterministic policy enforcement around probabilistic reasoning. Existing identity providers, audit systems, and compliance frameworks remain applicable because execution ultimately flows through governed business capabilities rather than unrestricted model outputs. A Final Word The transition from microservices to Agent Services is therefore not a replacement of proven architectural principles but their natural evolution. Microservices continue delivering transactional consistency, persistence, and deterministic business logic, while Agent Services introduce planning, semantic routing, capability discovery, memory, and adaptive orchestration. The architectural boundary shifts from exposing operations to exposing decisions, allowing intelligent planners to compose existing services according to business objectives rather than predefined workflows. Enterprise platforms adopting this layered approach preserve the reliability of mature microservice ecosystems while gaining the flexibility required for AI-native applications, making Agent Services the next logical abstraction for software systems where reasoning becomes as important as execution.

Every on-call engineer understands this situation well. An alert fires at 2 a.m., engineers spend the first five minutes figuring out what it means, the next few minutes searching Confluence for the relevant runbook, and finally start doing something useful. By that point, an automated system that could have classified the alert and retrieved the right procedure, proposed a remediation plan, and opened a ticket in thirty seconds has saved you nothing because it didn’t exist. That’s the problem this article addresses. We are going to build a working incident triage agent using .NET 10 and .NET Aspire 9 that does exactly that chain of steps automatically. The agent receives an HTTP alert payload, which classifies it using a Groq-hosted LLM, retrieves the matching runbook section from a Qdrant vector store, asks the LLM to propose remediation steps, and escalates to PagerDuty (through a local stub). If the severity warrants it, it writes a full audit record. The system will automatically follow all these without human involvement. What makes this integration not have any single component? It’s the combination of MCP as the tool contract, Aspire as the wiring layer, and a small eval harness that prevents the agent from quietly drifting over time. Let's go through how it's built. What are we Actually Solving? Before we bring AI into this, let’s be honest about what the actual problem is because “AI for incident triage” sounds impressive but means nothing without a clear picture of what exactly the AI is doing. When an alert goes out, the on-call engineer has three jobs Is this serious? (figuring out the severity before doing anything else)What do I do about it? (find the right procedure and follow it)Who else needs to know? (escalate the right people and open a ticket) Here are the things: Job one is mostly pattern matching, Job two is where it gets interesting, and Job three is completely mechanical but needs to be well understood, including failure modes, database connection pool exhaustion, memory leaks, and disk pressure. So, the answer is already written down somewhere in your runbook. Because the engineer is not thinking; they are searching. An LLM is genuinely good at steps one and two when given the right context. It can classify alerts and turn a runbook excerpt into a clear list of actions. The tricky part is making sure it gets the right runbook excerpt in the first place. If you ask an LLM to fix a memory leak without giving it the memory runbook, you’ll get a generic answer. So, give the right context, and you get something genuinely useful. Solution Architecture The solution is split into six focused .NET Aspire projects. Each project has a single, well-defined responsibility. App Host is the entry point to run. It doesn’t serve HTTP traffic or business logic. Its only job is to tell Aspire what services exist, which one needs to start, and which configuration needs to be injected into each of them. Think of it as the framework that describes the whole system. Services Defaults is a shared library that all other projects reference. It sets up the things every service should have, including structured logging with Serilog, distributed tracing, health check endpoints, and service discovery. So, these are all wired up with a single builder.AddServiceDefaults() call and never think about it again. Agent Service is the front door. It exposes one endpoint POST/triage and drives the five-step pipeline from start to finish. It doesn’t classify alerts, talk to Qdrant, and doesn’t know what pager duty is. It just calls the right tools in the right order and assembles the final response. MCP Tool Server is where the actual work happens. It hosts four MCP tools (alert classification, runbook lookup, PagerDuty escalation, and audit writing) and exposes them over HTTP using the Model Context Protocol. The Agent Service calls these tools by name without knowing anything about their internal implementation. PagerDuty Stub is a throwaway stand-in for the real PagerDuty API. In development, you do not want to fire real pagers or need a PagerDuty account just to test the escalation step. The stub accepts the same payload, logs it, and returns a synthetic ticket. Swap it for the real endpoints in production by changing one config value. Evals Harness is a safety check. It fires six carefully chosen alerts at the live agent and checks that the responses match expectations. If fewer than five pass, the process exits with a non-zero code, and your continuous integration pipeline fails. It is the thing that tells you when a model update or a config change has quietly broken something. The data flow for a single alert looks like this. The AgentService and McpToolServer are deliberately separate processes. The agent knows nothing about embeddings, Qdrant, or PagerDuty. It only knows how to call MCP tools by name. This is the core benefit of MCP. In the future, if we update the MCP server, the agent doesn’t change at all. MCP Tool Server The McpToolServer is an ASP.NET Core minimal API that exposes four tools over the MCP streamable HTTP transport. Each tool is a static class annotated with `McpServerToolType` and `McpServerTool`. C# [McpServerToolType] public static class AlertClassifierTool { [McpServerTool, Description("Classify an alert and return severity, category, and confidence.")] public static async Task<AlertClassification> ClassifyAsync( [Description("The raw alert text to classify")] string alertText, IChatClient chatClient, ILogger<AlertClassifierTool> logger, CancellationToken ct) { var prompt = $""" You are an incident classifier. Classify the following alert: {alertText} Respond with JSON only: {{ "severity": "Critical|High|Medium|Low", "category": "short category label", "confidence": 0.0-1.0, "reasoning": "one sentence" } """; var response = await chatClient.GetResponseAsync(prompt, new ChatOptions { ResponseFormat = ChatResponseFormat.Json }, ct); return JsonSerializer.Deserialize<AlertClassification>(response.Text) ?? throw new InvalidOperationException("LLM returned empty classification"); } } The `IChatClient` and `ILogger` parameters are injected by the MCP framework via ASP.NET Core’s dependency injection container. The tool itself is stateless, a plain static method. This keeps unit testing straightforward and allows you to pass in a mock `IChatClient`, call the method, and assert on the result. The `RunbookLookupTool` follows the same pattern but takes an `IEmbeddingGenerator<string, Embedding<float>>` and a `QdrantClient` instead of a chat client. C# [McpServerTool, Description("Find the most relevant runbook excerpts for a given incident category.")] public static async Task<List<RunbookExcerpt>> LookupAsync( [Description("Incident category from classification")] string category, IEmbeddingGenerator<string, Embedding<float>> embedder, QdrantClient qdrant, IConfiguration config, CancellationToken ct) { var topK = int.Parse(config["Qdrant:TopK"] ?? "3"); var colName = config["Qdrant:CollectionName"] ?? "runbooks"; var embedResult = await embedder.GenerateAsync([category], cancellationToken: ct); var vector = embedResult[0].Vector.ToArray(); var hits = await qdrant.SearchAsync(colName, vector, limit: (ulong)topK, cancellationToken: ct); return hits.Select(h => new RunbookExcerpt( Title: h.Payload["title"].StringValue, Content: h.Payload["content"].StringValue, Score: (float)h.Score)).ToList(); } The vector query uses cosine similarity, so (high memory usage on API node) still finds the memory-pressure runbook even though the wording doesn’t match. The embeddings capture semantic meaning, not keyword overlap. Custom Embeddings with Nomic AI Nomic AI `nomic-embed-text-v1.5` model produces 768-dimensional vectors at very low cost. The only catch is that Nomic uses a non-standard API path (`POST/v1/embedding/text` rather than the OpenAI-compatible `/V1/embeddings`), so we can’t use the default OpenAI embedding adapter from `Microsoft.Extensions.AI`. Instead, we implement `IEmbeddingGenerator<string, Embedding<float>>` directly. C# internal sealed class NomicEmbeddingGenerator( IHttpClientFactory httpClientFactory, string model, ILogger<NomicEmbeddingGenerator> logger) : IEmbeddingGenerator<string, Embedding<float>> { public EmbeddingGeneratorMetadata Metadata { get; } = new("nomic", providerUri: null, defaultModelId: model); public async Task<GeneratedEmbeddings<Embedding<float>>> GenerateAsync( IEnumerable<string> values, EmbeddingGenerationOptions? options = null, CancellationToken cancellationToken = default) { var client = httpClientFactory.CreateClient("nomic"); var requestBody = new NomicEmbedRequest(model, values.ToList(), "search_document"); using var response = await client.PostAsJsonAsync( "embedding/text", requestBody, NomicJsonContext.Default.NomicEmbedRequest, cancellationToken); response.EnsureSuccessStatusCode(); var result = await response.Content.ReadFromJsonAsync( NomicJsonContext.Default.NomicEmbedResponse, cancellationToken) ?? throw new InvalidOperationException("Nomic returned an empty response body"); return new GeneratedEmbeddings<Embedding<float>>( result.Embeddings.Select(v => new Embedding<float>(v)).ToList()); } public object? GetService(Type serviceType, object? serviceKey = null) => null; public void Dispose() { } } This class implements the full `IEmbeddingGenerator<string, Embedding<float>>` contract from `Microsoft.Extensions.AI`, so the rest of the codebase, including the `RunbookLookupTool`, sees a standard interface and never needs to know it’s talking to Nomic rather than OpenAI. The `JsonSerializable` source generation at the bottom of the file `NomicJsonContext` is important for trimming-safe serialization and for performance in hot paths. Both the request and response records must be at namespace scope (not nested inside the generator class) for the source generator to work correctly. This is a common mistake that produces `SYSLIB1032` at compile time. The Agent Service The Agent Service is where the triage pipeline is assembled. It uses Semantic Kernel to handle the remediation step (where we need prompt rendering and the injection filter) and calls all other steps via `McpClient.CallToolAsync`. The pipeline in `DotNetAspireTriageAgentService.cs` looks like this. C# // Step 1 — Classify var classification = await _mcpClient.CallToolAsync<AlertClassification>( "ClassifyAsync", new { alertText = payload.AlertText }, ct); // Step 2 — Runbook lookup (skip for Medium/Low) List<RunbookExcerpt> runbooks = []; if (_lookupSeverities.Contains(classification.Severity)) { runbooks = await _mcpClient.CallToolAsync<List<RunbookExcerpt>>( "LookupAsync", new { category = classification.Category }, ct); } // Step 3 — Remediation (via Semantic Kernel for prompt filter support) var proposal = await _kernel.InvokePromptAsync<RemediationProposal>( RemediationPromptTemplate, new KernelArguments { ["alert"] = payload.AlertText, ["runbooks"] = JsonSerializer.Serialize(runbooks), ["severity"] = classification.Severity }, cancellationToken: ct); // Step 4 — Escalate var escalation = await _mcpClient.CallToolAsync<EscalationResult>( "EscalateAsync", new { classification, correlationId = payload.CorrelationId }, ct); // Step 5 — Audit await _mcpClient.CallToolAsync( "WriteAuditAsync", new { classification, proposal, escalation }, ct); Defending Against Prompt Injection Prompt injection is a real concern in agentic systems where user-supplied text ends up literally inside an LLM prompt. An attacker who controls the alert body could try to override the system prompt and redirect the agent’s behavior. Prevent here uses Semantic Kernel’s `IPromptRenderFilter`, which fires after the prompt template is rendered but before the rendered string is sent to the model. C# public sealed class PromptInjectionFilter( InjectionDetectionContext context, ILogger<PromptInjectionFilter> logger) : IPromptRenderFilter { // Matches common injection patterns: "ignore previous instructions", // "disregard your system prompt", role-switching attempts, etc. private static readonly Regex InjectionPattern = new( @"(?i)(ignore\s+(all\s+)?(previous|prior|above)\s+instructions?" + @"|disregard\s+(your\s+)?(system\s+prompt|instructions?)" + @"|you\s+are\s+now\s+(?:a\s+)?(?:an?\s+)?\w+" + @"|act\s+as\s+(if\s+you\s+are\s+)?(?:a\s+)?(?:an?\s+)?\w+)", RegexOptions.Compiled | RegexOptions.CultureInvariant); public async Task OnPromptRenderAsync( PromptRenderContext context, Func<PromptRenderContext, Task> next) { await next(context); // let the template render first if (context.RenderedPrompt is not null && InjectionPattern.IsMatch(context.RenderedPrompt)) { context.RenderedPrompt = InjectionPattern.Replace( context.RenderedPrompt, "[SANITISED]"); this.context.InjectionDetected = true; logger.LogWarning( "Prompt injection attempt detected and sanitised — correlationId={CorrelationId}", context.Arguments["correlationId"]); } } } The filter doesn’t abort the request. It sanitizes the offending text and sets a flag that the agent includes in the response. This is a deliberate choice where failing silently is worse than completing with a sanitized prompt, because a failed triage means a missed escalation. The response `injectionDetected` field lets downstream systems know that something suspicious happened without stopping the pipeline. Handle Everything Together with .NET Aspire The AppHost is where everything comes together. Every service, dependency, and API key is declared in one place. When we run this project, Aspire reads those declarations and automatically starts the entire system in the correct order. C# var builder = DistributedApplication.CreateBuilder(args); // API keys from user-secrets or appsettings.json var groqApiKey = builder.AddParameter("GroqApiKey", secret: true); var nomicApiKey = builder.AddParameter("NomicApiKey", secret: true); // Qdrant container — persisted between restarts var qdrant = builder.AddQdrant("vectorstore") .WithLifetime(ContainerLifetime.Persistent); // PagerDuty development stub var pagerDutyStub = builder.AddProject<Projects.DotNetAspireTriageAgent_PagerDutyStub>( "pagerduty-stub"); // MCP Tool Server — waits for Qdrant and the PagerDuty stub var pagerDutyStubEndpoint = pagerDutyStub.GetEndpoint("http"); var mcpServer = builder.AddProject<Projects.DotNetAspireTriageAgent_McpToolServer>("mcp-tools") .WithReference(qdrant) .WithReference(pagerDutyStub) .WaitFor(qdrant) .WaitFor(pagerDutyStub) .WithEnvironment("Groq__ApiKey", groqApiKey) .WithEnvironment("Nomic__ApiKey", nomicApiKey) .WithEnvironment("PagerDuty__StubEndpoint", ReferenceExpression.Create($"{pagerDutyStubEndpoint}/pagerduty-stub/incidents")); // Agent Service — waits for the MCP server builder.AddProject<Projects.DotNetAspireTriageAgent_AgentService>("agent-service") .WithReference(mcpServer) .WaitFor(mcpServer) .WithEnvironment("Groq__ApiKey", groqApiKey); builder.Build().Run(); Three things in this code are worth understanding properly before moving on. .WithReference() vs .WithEnvironment(): These two look similar but do various jobs. When you call .WithReference(Qdrant), you are telling Aspire to figure out Qdrant’s host, port, and credentials at runtime and automatically inject the full connection string into McpToolServer. We do not need to mention it hardcoded anywhere. ReferenceExpression.Create. This one trips people up the first time. When McpToolServer needs to call the PagerDuty stub, it needs the stub’s full URL including the path (like domain/pagerduty-stub/incidents). The problem is you do not know the port number at the time you write the code; in this case, Aspire assigns it dynamically at startup. So instead of hardcoding a URL that will break on someone else’s machine, for this we write ReferenceExpression.Create($"{pagerDutyStubEndpoint}/pagerduty-stub/incidents") and let Aspire fill in the real address when it starts up. WaitFor This tells Aspire not to start McpToolServer until Qdrant and the PagerDuty stub are fully up and ready. Without it, McpToolServer would try to connect before they are ready and crash on the very first run. Once everything is running, the Aspire dashboard gives you a live view of the whole system. The resources tab shows all four services with their current health status and the URLs Aspire assigned to each one. The graph tab is even more useful when you are onboarding someone new to the project. It draws the exact dependency map you declared in the codebase, which service depends on which, which API keys go where, and how everything connects. Note: if a service fails to start, this graph tells you immediately which dependency in the chain is the problem instead of you having to read through logs across four different console windows. PagerDuty Stub Rather than mocking PagerDuty calls in covers or requiring a real PagerDuty account, the solution includes a lightweight stub service. It is a genuine Aspire project registered in Apphost. C# app.MapPost("/pagerduty-stub/incidents", async (HttpRequest request) => { // ... read and log the body ... var response = new PagerDutyStubResponse( Incident: new StubIncident( Id: correlationId, Status: "triggered", Number: Random.Shared.Next(1000, 9999))); return Results.Created( $"/pagerduty-stub/incidents/{correlationId}", response); }); Because the stub is a real Aspire project, its URL is dynamically allocated by Aspire and injected into McpToolServer via `ReferenceExpression.Create`. This means there are no hardcoded ports that break when someone else is already using that port, and the stub starts and stops with the rest of the solution. Swapping it for the real PagerDuty events API in Production means changing a single config value, the URL injected via `WithEnvironment`. Runbook Seeding on Startup The McpToolServer seeds its Qdrant collection on startup using a hosted service. It checks whether the collection already exists before doing any work, which means subsequent restarts are near-instant. C# public sealed class RunbookSeeder( QdrantClient qdrant, IEmbeddingGenerator<string, Embedding<float>> embedder, IConfiguration config, ILogger<RunbookSeeder> logger) : IHostedService { public async Task StartAsync(CancellationToken ct) { var collectionName = config["Qdrant:CollectionName"] ?? "runbooks"; var exists = await qdrant.CollectionExistsAsync(collectionName, ct); if (exists) { logger.LogInformation("Runbook collection already exists — skipping seed"); return; } await qdrant.CreateCollectionAsync(collectionName, new VectorsConfig(new VectorParams(size: 768, distance: Distance.Cosine)), ct); } } The runbooks test the most common failure categories, including high CPU, memory pressure, database connection exhaustion, disk saturation, network timeout, and pod restart loops. Each is stored as a Qdrant point with title and content payload fields that `RunbookLookupTool` reads back on retrieval. Eval Harness AI systems have a subtle problem that unit tests don’t catch. So, the agent can quietly get worse over time. A model version bumps, someone tweaks a prompt, a config value changes, and suddenly your critical alerts are coming back as medium with no error thrown anywhere. You only find out when a real incident gets missed. The Evals project is the safety net for exactly this. It fires six alert payloads at the live agent and checks that each response matches the expected severity, category, and escalation behavior. If fewer than five pass, the build fails. It is the same idea as a unit test suite, except it is testing the intelligence of the agent, not just the correctness of the code. Key Takeaways .NET Aspire service coordination makes it practical to run a multi-service AI agent system, including a vector database, an MCPToolServer, and an LLM-backed agent, locally with a single `dotnet run` command.The Model Context Protocol (MCP) gives you clean, language-agnostic control for exposing agent tools over HTTP, so the agent and its capabilities can evolve independently without tight coupling.Combining Nomic AI embeddings with a Qdrant vector store lets you attach a runbook knowledge base to an AI agent without fine-tuning a model that will help semantic search retrieve the right production even when the alert wording doesn’t match the runbook text exactly.Groq’s OpenAI-compatible API with `llama-3.3-70v-versatile` provides sub-second structured JSON responses, which is fast enough to complete a full five-step triage pipeline including classify, retrieve, remediate, escalate, and audit in under three seconds on most workloads.Adding a Semantic Kernel `IPromptRenderfilter` to scan every prompt render before it reaches the LLM is a lightweight, zero-overhead way to defend against prompt injection in agentic pipelines. Prerequisites To follow along with the code in this article, you will need: Visual Studio 2026 (17 or later) with the .NET Aspire package installed, or the .NET 10 SDK (10.0.300 or later) if you prefer using a terminal.Docker Desktop (4.x or later) must be running before you start because .NET Aspire automatically starts a Qdrant container.Groq API key (free get from console.groq.com) used for the alert classification and remediation via `llama-3.3-70v-versatile`.Nomic AI API key (free get from atlas.nomic.ai) used for runbook text embeddings via `nomic-embed-text-v1.5`. Note: No cloud subscription is required. Both API keys have generous free quotas that comfortably cover development and testing. Conclusion What we have built is a working blueprint for an AI triage agent that respects software engineering discipline, clean boundaries between components, a tool contract that survives dependency changes, prevents misuse, and a regression harness that makes model-level drift a continuous integration failure rather than a surprise. The combination of .NET Aspires coordination, Mcp tool abstraction, Groq’s low-latency inference, and Nomic embeddings means you can stand up a full agentic pipeline locally, with realistic dependencies, in the time it takes to run `dotnet run`. The development experience matters because it determines how quickly you can experiment, iterate, and validate changes. The next natural extensions are a persistent audit store, a document ingestion pipeline for runbooks, and a feedback loop that uses closed incidents to refine the classification prompts. All three can be added as new MCP tools without changing the agent. Appendix The complete source code for this article, including all six projects, runbook seed data, eval harness cases, and configuration examples, is available in the GitHub repository. You can clone it, run it locally with a single command, and use it as a starting point for your own incident triage pipeline. Full source code is available at the GitHub Repository.