AI/ML

Have you ever asked an LLM for JSON and got back "Sure! Here is your JSON:" followed by the JSON? Your parser fails, your pipeline stops, and someone adds a retry loop. I have been there. For a long time, the fix was a polite prompt. "Reply with valid JSON only. No extra words." It works most days. On the other days, it breaks. There is a better fix, and it does not rely on the model behaving well. It is called constrained decoding. In this article, you will learn: What encoders and decoders are, and when to use eachHow constrained decoding works, in plain wordsWhich tools do it todayWhat research says about its strengths and limitsHow it compares with a plain LLM, and with new decision models like Jev and LayaHow to use it safely inside an agent A quick note on the name. In this article, CDM means constrained decoding model: a decoder LLM run together with a constraint engine. CDM is the label I use for this setup. It is not a product name. Encoder and Decoder in Plain Words Almost every modern language model is built from blocks called transformers. What changes is how the blocks are arranged. A survey of language models for software work describes three layouts: BERT as encoder-only, LLaMA as decoder-only, and CodeT5+ as encoder plus decoder (arXiv 2408.02479). Encoder and Decoder Here is the easy way to remember it. An encoder is a reader. It looks at the whole message at once, and every word can see the words on both sides. The output is understanding, a set of numbers that capture meaning. That is why encoders are good at jobs like classifying, routing, searching, and filtering. Researchers who built small safety checkers note that encoder models are efficient at narrow jobs such as classification, name tagging, and sentiment (arXiv 2410.08442). A decoder is a writer. It produces one small piece of text, called a token, at a time. Each new piece depends on what it already wrote. Chat models like Llama work this way. Decoders became popular because they are flexible: almost any task can be turned into "write the answer." An encoder plus decoder reads one thing and writes another, like a translation or a summary. T5 and CodeT5+ are examples. If you need toUseWhyLabel, filter or route textEncoder, or a decision model built on oneReads once, small, cheapSearch by meaningEncoderProduces meaning numbersWrite, explain, plan or codeDecoder (LLM)Generates new textMake a decoder answer in a fixed shapeDecoder plus constrained decodingKeeps the writer inside the rulesExact lookups, math or datesPlain codeFast and easy to check A CDM sits in the fourth row. Jev and Laya are closer to the encoder idea. Laya, for example, is built on a ModernBERT encoder with a decision layer on top (Laya model card). What Is Constrained Decoding? A decoder picks its next token by scoring every token it knows. Constrained decoding adds a step. Before the model picks, a rule engine blocks the tokens that would break your rules. The XGrammar paper describes this as restricting the available tokens at each step (arXiv 2411.15100). How CDM works The loop has five steps: Score: the model scores every possible next token.Check: the rule engine works out which tokens are legal at this spot.Mask: illegal tokens are blocked.Pick: the model chooses from the legal tokens only.Repeat: the token is added, and the loop runs again until the answer is complete. The values in the close-up part of the picture are made up to show the idea. What matters is the shape of the process: blocked tokens disappear, and the remaining choices share the probability. What Kinds of Rules Can You Use? The common ones are a JSON schema, a regular expression, a fixed list of choices, or a grammar. vLLM, a popular serving engine, lists all four (vLLM docs). The Outlines library shows how simple this can be. For a yes-or-no answer, you use a Literal["Yes", "No"] type. For a sentiment label, you use Literal["Positive", "Negative", "Neutral"] (Outlines on GitHub). The model can no longer return anything else. That means a choice constraint turns a general LLM into a classifier that can never invent a label. One Thing to Remember Masking controls what the model is allowed to write. It does not change what the model understands. Everything in the research section below comes back to this point. Tools You Can Use Today The tools fall into three groups: engines that compute the mask, servers that run them, and hosted APIs that hide them. ToolWhat it isStart hereOutlinesA Python library that turns types, schemas or regex into a state machine that masks tokensPaper, GitHubXGrammarAn engine that prepares most of the mask ahead of time to keep overhead low. XGrammar-2 targets rules that change at runtimePaper, XGrammar-2llguidance (Guidance)A fast engine written in Rust that handles full grammars and JSON schemasGitHubllama.cpp grammarsGrammar support for local models, using GBNFGrammar guidevLLMA serving engine that uses XGrammar or Guidance as backendsDocsNVIDIA NIMContainerized serving with JSON schema, regex, grammar, and choice supportDocsOpenAI Structured OutputsA hosted option that enforces a JSON schema in strict modeAnnouncementClaude structured outputsA hosted option that compiles your JSON schema into a grammarDocs Two things to watch for. Hosted APIs support a part of JSON Schema, not all of it. For example, Google's documentation for Claude on Vertex AI says some constraints, such as numeric and string-length rules, are not supported (Vertex AI docs). If a rule is not enforced by the mask, check it in your own code. Your backend can change under you. vLLM's configuration notes say the automatic backend choice can change between releases (vLLM config). If you need repeatable behavior, set the backend yourself and pin the version. What the Research Says I read the main papers so you do not have to. Several 2026 papers are preprints, which means they have not been peer-reviewed yet. Here are the five points that matter most. 1. A valid format becomes a guarantee. A 2026 preprint on small open models found that constrained decoding removed format failures such as wrong types and missing fields (arXiv 2609.23742). If a program must parse the output, this is the main reason to use it. 2. Valid is not the same as right. The same paper found that the content can still be wrong or thin. A model that skips a required step still passes the schema if the schema allows it. The authors describe rescued answers that look complete but carry little meaning. The short version: constraints fix form, not understanding. 3. Real schemas are harder than demos. JSONSchemaBench tested several engines, including Guidance, Outlines, llama.cpp, XGrammar, OpenAI, and Gemini, on a large set of real-world schemas. Engines differ in which schema features they support (arXiv 2501.10868). That benchmark is from early in this wave of tools, and engines have improved, so test your own schemas on the engine you plan to use. 4. Strict formats can hurt reasoning. The paper "Let Me Speak Freely?" found that format restrictions can weaken reasoning, and stricter formats can do more harm. It also found that constraints can help classification (arXiv 2408.02442). In reasoning tasks, JSON mode often did not keep the order of "think first, answer second." So if the model must think, let it think first. 5. Masking can nudge what the model prefers. Research on grammar-aligned decoding shows that blocking tokens can distort the model's own probabilities, so some outputs are valid but lower in quality (arXiv 2405.21047). A 2026 preprint on draft-conditioned decoding targets a related problem, where masking pushes the model down a path that is valid but wrong (arXiv 2603.03305). FindingWhat it meansWhat to doFormat failures disappearOutput always fits the rulesUse it wherever a program parses the outputValid can still be wrongMeaning is not checkedTest the content, not only the formatEngines support different schema features"Supports JSON schema" is not one yes or noTest your schemas on your engineStrict formats can hurt reasoningEarly constraints take away thinking roomLet the model think, then constrain the final answerMasking can bend probabilitiesOpen-ended content can lose qualityWatch quality on long or creative output CDM vs. LLM vs. Jev and Laya Now we can compare the designs. All three can give you a valid answer from a fixed list. They get there in different ways. LLMCDMDecision modelCore designDecoderDecoder plus maskEncoder plus decision layerHow it answersWrites token by tokenWrites token by token, legal tokens onlyScores every allowed answer in one passOutput always validNoYes, inside the rules the engine supportsYes, it picks from your listProbabilitiesNot by defaultToken level, not trained to be calibratedTrained to be calibratedCan write or explainYesYes, inside the shapeNoOpen shapes like SQL or nested JSONYesYesNo, typed answers onlyMain riskBroken formatValid but wrong, reasoning lossConfident wrong picks, forced answers What Are Jev and Laya? They are decision models, sometimes called System One models, after the fast, gut-level mode of thinking. You give them an input, a question, and the allowed answers. They return probabilities for each answer in one pass. They do not write text. Jev comes from TypeSafe AI and is offered as a hosted service. Its evaluation paper describes typed answers: a choice, a position on a rubric, or a true-or-false probability (arXiv 2609.37647). Laya comes from Convai Innovations and is open, so you can host and fine-tune it yourself. Its model card explains that it is trained so that its probabilities are honest, a method it calls RLCD (Laya model card). I covered both in Classification Never Left. It Just Got a New Home in LLMs. How Do They Differ From a CDM? Think of a quiz with four answers. A CDM is a writer who is only allowed to write one of the four answers. It still goes through the writing loop, and it can also handle open-ended formats like SQL. A decision model reads the question once and hands you a score for each of the four answers. It cannot write anything else, and it is built for calibrated scores. ScaleDown's CDMs, covered in the next section, are a third variant. They are small models trained for one task, so the limit on what they can return is built in during training instead of added by a mask at run time. What About a Head-to-Head Test? I looked for a direct comparison of a constrained LLM against Jev or Laya on the same data and did not find one. So I will not name a winner. If you are choosing between them, test both on a few hundred of your own inputs. One weakness is shared. A closed list forces an answer. If a message fits none of your categories, both a decision model and a constrained LLM will still pick one. This follows from how the list works, and it is why you should always add an "other" option and test it on purpose. When to Use What When to use what - LLM vs CDM vs Decision Model Walk the questions from left to right and stop at the first yes. JobBest fitWhyTool call arguments for an agentCDMStrict JSON that must parseSQL, regex or a small languageCDM with a grammarOpen shape with clear rulesPulling fields out of text into a schemaCDMType and structure errors disappearTicket routing at high volumeDecision model, or a trained encoder classifierOne pass, cheap, probabilitiesUrgency score or yes or no flagDecision modelTyped scoresReasoning, then a structured answerLLM thinks freely, then a constrained last stepAvoids the reasoning lossSummaries, replies and explanationsLLMYou need wordsExact rules, lookups, math and datesPlain codeFast and easy to check Two rules of thumb: If the model must think, let it think first. Give it a free reasoning step, then constrain only the final answer. If you must use one schema, put the reasoning field before the answer field.If the answer is a pick, constrain it tightly. Picks are where constraints help most. How ScaleDown Brought CDMs Into Agents A recent post from LangChain shows where this idea is heading. Long-running agents spend a lot of effort on context management: summarizing old history, shrinking large tool results, and pulling the right values out of long inputs. Each of those is a small decision, and making them with a large LLM on every turn is slow and costly. ScaleDown's answer is a set of CDMs. Instead of writing any string one token at a time, each model is trained for one job and only returns answers that job allows, such as a label from your list or pieces of the input text. ScaleDown offers models to summarize, compress, extract, and classify, and the langchain-scaledown package wraps them as middleware for LangChain and Deep Agents. The big LLM keeps the planning and writing, and the CDMs handle the high-volume steps around it (LangChain blog). Its classify model will feel familiar if you have read about Jev. Both return a typed answer with a probability, and the post says ScaleDown also adds a short reason, which helps when you debug why an agent stopped or routed a task. The post also notes that ScaleDown is now available through the LangSmith LLM Gateway, so its calls are traced like your other model calls. One caution. This is a vendor-written post, and the speed and cost claims come from ScaleDown's own benchmarks, so test them on your own workload before you rely on them. Using a CDM Inside an Agent The two designs work well together. A CDM keeps the agent's tool call valid. A decision model, or a second choice constraint, decides whether that call is safe to run. CDM inside an agent Here is how it flows: A user or system sends a request.The LLM agent plans the next step and proposes a tool call.Constrained decoding makes sure the call matches the schema, so it always parses.A decision gate scores the call as allow, review, or block.Allowed calls run, unsure calls go to a person, and risky calls are rejected and logged. Three habits keep this honest: Test two things. Check that the output is valid, and check that it is right. They are not the same.Log every step. Keep the proposed call, the schema it matched, the gate's scores, and who approved it.Treat the gate as one control. Give the agent only the access it needs, and require a human for high-impact actions. I wrote more about these controls in AI-Era Cybersecurity, Part 2, and about the tools that sit beside them in Part 1. A Simple Checklist StepAction1List every place your system needs a fixed shape or a pick from a list2Move anything plain code can decide out of the model3Choose a rule type: choice, regex, JSON schema or grammar4Stay inside what your engine supports, and check the rest in code5Add an "other" or "unknown" option to every closed list6Put reasoning before the answer, or reason in a separate step7Pin the engine, backend and model version8Test format and meaning on real inputs, including ones that fit no category9Cache compiled schemas, or use an engine built for rules that change per request10Log decisions and recheck on a schedule Limits of This Article Some of the newest evidence is in preprints and small studies. Treat it as a signal, not a final answer.Hosted APIs do not publish their full mask engines, so I rely on their documentation.I found no same-data comparison between a constrained LLM and a decision model, so the comparison here rests on design differences and separate sources.Tools in this area change quickly. Check each official page before you decide. Related Articles From Me ArticleHow it connectsClassification Never Left. It Just Got a New Home in LLMsThe decision model side of this comparisonShipping Production-Grade AI Agents (Refcard #403)Guardrails and evaluation, where a CDM fitsA Field Guide to AI Agent FrameworksWhere structured output plugs into frameworksGraph Engineering: The Layer After Loop EngineeringA constrained choice can pick the next node in a graphAI-Era Cybersecurity, Part 2Agent identity, approval gates and measuring riskAI-Era Cybersecurity, Part 1The bug-fixing tools these gates sit besideHow AI Is Actually Changing SRE Tools, Part 2Alert triage is a pick-from-a-list problemFrom Prompts to Harnesses: How AI Engineering Has Grown UpThe harness around the model is where constraints live Conclusion A CDM is a decoder with guardrails. It gives you a promise that is easy to state and easy to test: the output will always fit the rules. It does not promise that the answer is right, and strict rules can cost you reasoning if you apply them too early. Decision models like Jev and Laya make a different trade. They give up writing in return for fast, one-pass scoring with probabilities built to be honest. They have their own risks, such as confident wrong picks and forced answers. So match the tool to the shape of the job. Constrain tightly when the output must parse, or the answer is a pick. Leave the model free when it needs to think. And test both the format and the meaning on your own data before you trust either one. If you try this in your own system, I would like to hear which part you tried first. Share it in the comments. References Papers Efficient Guided Generation for Large Language Models (Outlines)XGrammar: Flexible and Efficient Structured Generation EngineXGrammar-2: Dynamic and Efficient Structured Generation Engine for Agentic LLMsJSONSchemaBenchLet Me Speak Freely? Format Restrictions and LLM PerformanceGrammar-Aligned DecodingThe Hidden Cost of Structured Generation: Draft-Conditioned Constrained DecodingConstrained Decoding Eliminates Structural Failures in Small LLMsFrom LLMs to LLM-based Agents for Software Engineering: A SurveyJurEE not Judges: Small, Specialized Encoder EnsemblesEvaluating and Benchmarking the System One Model Jev Tools and Documentation OutlinesXGrammarllguidancellama.cpp grammarsvLLM structured outputsNVIDIA NIM structured generationOpenAI Structured Outputs announcementClaude structured outputsClaude structured outputs on Vertex AI ScaleDown and LangChain Building Long-Running Agents with ScaleDown and Deep Agents, LangChain blogScaleDown Decision Models Laya model card, Hugging FaceJev: Decision Models on Trial, Cisco Blogs

By Vidyasagar (Sarath Chandra) Machupalli FBCS CORE

The previous article put Genkit flows in Kubernetes pods with senro and let jev, a decision model, route a support ticket. The decisions there were small: which team, how urgent, send or hold. This one hands jev something bigger to judge: software. A software factory takes a feature request and produces a Go module: a design, a plan, the code, a green build, passing tests, and a pull request description. Gemini writes every one of those. jev decides, at every gate, whether the work moves on, goes back, or stops for a human. Each stage runs as a pod, each stage is a Genkit flow, and the whole thing is a few hundred lines of Go. Everything below ran against the real Gemini API (gemini-3.8-flash) and the real TypeSafe API (jev-latest), on Genkit Go v1.14.0, senro v1.4.5 and a local kind cluster. The code is in genkit-k8s-flows-jev-software-factory. Why a Factory Needs a Judge, Not a Second Writer The usual way to review model-written code is to ask another chat model. It works, up to a point. The reviewer writes a paragraph, you parse the paragraph, the paragraph is sometimes wrong in the same way the code is, and a well-written pull request description can talk the reviewer into "looks good to me." A decision model does not write. It takes a state (the request, the plan, the code, the test log) and answers typed questions with probabilities, in a few hundred milliseconds. There is no prose to parse and nothing to be talked into: BuildGreen is a number, and a red test log drags it down whatever the pull request says. Chat model as reviewerDecision model as reviewerVerdictA paragraph you parseChoice[CodeVerdict]: ship, rework or human, with a probability eachEvidenceMentioned, maybeA Noul per question: covers the request, tests are meaningful, build is greenFeedback for the next roundWhatever it felt like writingEach question that scored "yes" is one line of feedbackLatency and costSeconds, thousands of output tokensMilliseconds, tokens for the state onlyCan be persuaded by the authorYesNo: it never reads instructions, only state The split that falls out of this is the whole design: Gemini builds, jev decides. And senro is what gives each build step a pod, a workspace, a log, a retry policy, and a secret, and each decision a place in the graph. What We Are Building A feature request goes in. A reviewed Go module comes out, or a brief for a human saying why not. The factory is three senro pipelines and a small state machine over them: RunStepWhereWhoWhat it leaves behinddesigncleancoordinatornonean empty workspace with request.md in itdesigndesignpodGemini and jev, in a loopdesign.json, DESIGN.md and one record per review rounddesignplanpodGeminiplan.json, PLAN.mdimplementimplementpodGeminithe module's filesimplementvet, testpodgo vet, go testvet.log, test.log and their exit codesimplementreviewpodjevreview.json: ship, rework or humanreworkfix, then vet, test, reviewpodGemini, Go, jevthe same, after a fixfinishpr then publishpod, coordinatorGeminiPR.md and the module under out/<request>/srcfinishbrief then notifypod, coordinatorGeminiBRIEF.md for a human jev decides three times. Inside the design step, it reviews each design Gemini writes, and the loop revises until jev says ready, says reject, or three rounds are up. After test, it reviews the code with the logs in hand and picks ship, rework or human. And the state machine in main turns each verdict into the next senro run, with rework bounded at two. What You Need Go 1.27 or newer, Docker, kind, and kubectlGEMINI_API_KEY and TYPESAFE_API_KEY in your environmentthe senro CLI if you want the terminal UI (optional) Shell go get github.com/firebase/genkit/[email protected] go get github.com/xavidop/[email protected] go get github.com/xavidop/mamori Step 1: A Cluster and an Image With a Go Toolchain The cluster setup is the one from the previous article: kind, a namespace, a ServiceAccount with the permissions senro's Kubernetes executor needs (including get on pods/exec), and three environment variables, since senro never reads your kubeconfig. The repo's scripts/cluster-up.sh does it and source scripts/cluster-env.sh exports them. The one difference is the image. A func step needs only sh and tar in the pod, but this factory also runs go vet and go test as plain command steps, so every step uses the official Go image, pinned to its digest as senro requires: Go const stepImage = "docker.io/library/golang:1.27-alpine@sha256:f2c872bcee949b7be9c490efd02b48a8a2fbf947cdf533f1c58525ec9d8170fd" func pods() senro.ExecutorTarget { return k8s.Pod(stepImage, k8s.Namespace("factory"), k8s.Resources( map[string]string{"cpu": "500m", "memory": "512Mi"}, map[string]string{"cpu": "2", "memory": "2Gi"}, )) } Your pipeline binary is still not in that image. senro cross-compiles it and sends it into each pod, exactly as before. Step 2: The Design Loop, With jev Inside the Flow The first gate is the one that saves the most money: a bad design produces bad code, a green build and a confident pull request. So the design is reviewed before anything is planned, and the review is a loop inside one Genkit flow. The questions are a Go type. Every yes/no question is one a revision can act on: a "yes" becomes a line of feedback for the next round. Go type DesignVerdict string func (DesignVerdict) Criteria() map[DesignVerdict]string { return map[DesignVerdict]string{ "ready": "Complete, unambiguous, scoped to the request, and testable as described", "revise": "The right direction, with gaps a revision can close", "reject": "The request is too vague or too large to design from; a human has to clarify it first", } } type Risk int func (Risk) Levels() []string { return []string{"Low", "Medium", "High"} } type DesignReview struct { Verdict systemonex.Choice[DesignVerdict] `json:"verdict" jsonschema_description:"Is this design ready to implement?"` RequestClear systemonex.Noul `json:"request_clear" jsonschema_description:"Does the request say what to build precisely enough that two engineers would build the same thing?"` ScopeCreep systemonex.Noul `json:"scope_creep" jsonschema_description:"Does the design add things the request did not ask for?"` Underspecified systemonex.Noul `json:"underspecified" jsonschema_description:"Is any component described too loosely to implement without guessing: missing fields, signatures, edge cases or error behaviour?"` MissingTests systemonex.Noul `json:"missing_tests" jsonschema_description:"Does the test plan leave any requirement of the request untested?"` Risk systemonex.Score[Risk] `json:"risk" jsonschema_description:"How risky is implementing this design as written?"` } And the loop. Gemini designs; jev reviews; if the verdict is revise, Gemini gets the previous design plus the feedback and tries again: Go f.Design = genkit.DefineFlow(g, "designFeature", func(ctx context.Context, request string) (DesignResult, error) { var res DesignResult feedback := "" for round := 1; round <= maxDesignRounds; round++ { prompt := "Request:\n" + request if round > 1 { prompt += "\n\nYour previous design:\n" + asJSON(res.Design) + "\n\nReviewer feedback:\n" + feedback } d, _, err := genkit.GenerateData[Design](ctx, g, ai.WithModelName("googleai/gemini-3.8-flash"), ai.WithSystem("You are a senior Go engineer writing a short design for exactly what the request asks. "+ "Stay inside the request: no extra features. Components are the Go types and functions you will create. "+ "The test plan names concrete, deterministic test cases. Be brief."), ai.WithPrompt("%s", prompt)) if err != nil { return res, err } res.Design = *d out, resp, err := genkit.GenerateData[DesignReview](ctx, g, ai.WithModelName("typesafe/jev-latest"), ai.WithSystem("The state has a feature request and a design written for it."), ai.WithPromptParts(ai.NewDataPart(map[string]any{"request": request, "design": res.Design}))) if err != nil { return res, err } // jev's verdict, then our rules on top. verdict := string(out.Verdict.Choice) var notes []string if out.RequestClear.Probability < 0.4 { verdict = "reject" // nobody should design from a request two engineers would read differently } if out.ScopeCreep.Probability >= 0.5 { notes = append(notes, "remove everything the request did not ask for") } if out.Underspecified.Probability >= 0.5 { notes = append(notes, "specify every component fully: fields, signatures, edge cases and error behaviour") } if out.MissingTests.Probability >= 0.5 { notes = append(notes, "add a test case for every requirement in the request") } if verdict == "ready" && len(notes) > 0 { verdict = "revise" } res.Rounds = append(res.Rounds, DesignRound{Round: round, Verdict: verdict, Ready: out.Verdict.Probabilities["ready"], Feedback: notes /* ... */}) res.Verdict = verdict if verdict != "revise" { break } feedback = fmt.Sprintf("The reviewer rated the design %q (probability of ready %.2f). Do the following:\n- %s", verdict, out.Verdict.Probabilities["ready"], strings.Join(notes, "\n- ")) } return res, nil }) Two things to notice. The verdict is jev's, but the rules around it are yours: a request that two engineers would read differently is rejected whatever the design looks like, and a design with a flagged problem is not ready even if jev leaned that way. And the feedback is built from the questions, not from prose: a decision model never writes a sentence, so the sentences live in your code, one per question, and the next round gets exactly the ones that applied. The whole loop is one flow, so it is one senro step, one pod, one log, with every round recorded in design.json. Step 3: Plan, Implement, and Let Go Have Its Say The plan and the implementation are ordinary Gemini calls with typed outputs: a Plan with files and steps, then a Code with every file in full. The step writes the files into the workspace, refusing anything that would escape it: Go type Code struct { Files []File `json:"files"` Notes string `json:"notes,omitempty"` } func writeCode(ctx senro.Ctx, code Code) error { root := wsPath(ctx, ".") for _, f := range code.Files { rel := filepath.Clean(f.Path) if rel == "." || strings.HasPrefix(rel, "..") || filepath.IsAbs(rel) || strings.HasPrefix(rel, factoryDir) { return fmt.Errorf("refusing to write outside the module: %q", f.Path) } path := filepath.Join(root, rel) if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { return err } if err := os.WriteFile(path, []byte(f.Content), 0o644); err != nil { return err } fmt.Fprintf(ctx.Stdout(), "wrote %s (%d bytes)\n", rel, len(f.Content)) } return nil } Then the compiler and the tests run, as command steps in the same pod image, with no model anywhere near them: Go func goStep(w *senro.WorkflowBuilder, id, after, cmd string, ws *senro.WorkspaceRef) *senro.StepBuilder { script := "mkdir -p .factory; " + cmd + " > .factory/" + id + ".log 2>&1; echo $? > .factory/" + id + ".exit; " + "cat .factory/" + id + ".log; exit $(cat .factory/" + id + ".exit)" return w.Step(id, exec.Command("sh", "-c", script)). Needs(after). Mount(ws.At("/repo", senro.RW)). WorkDir("/repo"). Env("HOME", "/tmp"). Env("GOCACHE", "/tmp/gocache"). Env("GOFLAGS", "-mod=mod"). Env("GOPROXY", "off"). Env("GOTOOLCHAIN", "local"). Env("CGO_ENABLED", "0"). Timeout(stepTimeout). Retry(3, retry.OnInfra()). ContinueOnError() } goStep(code, "vet", "implement", "go vet ./...", ws) goStep(code, "test", "vet", "go test -count=1 ./...", ws) Each one leaves its output and exit code under .factory/ in the workspace, so the reviewer can read them. GOPROXY=off and GOTOOLCHAIN=local keep the pod honest: the request said standard library only, and the pod cannot download anything to pretend otherwise. And ContinueOnError is the important one: a red test is a failed step, but the review still runs. jev gets to see why the build failed, which is exactly the evidence a rework needs. Step 4: The Code Review Gate The second gate reads everything a human reviewer would open: the request, the plan, every source file, and what go vet and go test printed. Go type CodeVerdict string func (CodeVerdict) Criteria() map[CodeVerdict]string { return map[CodeVerdict]string{ "ship": "Builds, tests pass, does what the request asked, no obvious defects", "rework": "Fixable problems: failing or missing tests, a missed requirement, sloppy code", "human": "Something a model should not decide: security, data loss, or the request was misunderstood", } } type Quality int func (Quality) Levels() []string { return []string{"Poor", "Acceptable", "Good", "Excellent"} } type CodeReview struct { Verdict systemonex.Choice[CodeVerdict] `json:"verdict" jsonschema_description:"Should this change ship?"` Quality systemonex.Score[Quality] `json:"quality" jsonschema_description:"How good is the code?"` CoversRequest systemonex.Noul `json:"covers_request" jsonschema_description:"Does the implementation do everything the request asked for?"` TestsMeaningful systemonex.Noul `json:"tests_meaningful" jsonschema_description:"Do the tests exercise the behaviour the request cares about, rather than just compile?"` BuildGreen systemonex.Noul `json:"build_green" jsonschema_description:"According to the logs and exit codes, did go vet and go test both pass?"` } The flow sends the whole ReviewInput as a data part, reads the answers, and applies the factory's rules on top. A red build never ships, whatever the overall verdict: Go f.Review = genkit.DefineFlow(g, "reviewChange", func(ctx context.Context, in ReviewInput) (Review, error) { out, resp, err := genkit.GenerateData[CodeReview](ctx, g, ai.WithModelName("typesafe/jev-latest"), ai.WithSystem("The state has the feature request, the implementation plan, every source file, "+ "and the output and exit codes of go vet and go test. Exit code 0 means success."), ai.WithPromptParts(ai.NewDataPart(in))) if err != nil { return Review{}, err } r := Review{ Verdict: string(out.Verdict.Choice), Confidence: out.Verdict.Confidence, Quality: out.Quality.Label(), CoversRequest: out.CoversRequest.Probability, TestsMeaningful: out.TestsMeaningful.Probability, BuildGreen: out.BuildGreen.Probability, Model: systemonex.ResponseInfo(resp).Model, } if in.VetExit != 0 { r.Reasons = append(r.Reasons, "go vet failed") } if in.TestExit != 0 { r.Reasons = append(r.Reasons, "go test failed") } if r.CoversRequest < 0.6 { r.Reasons = append(r.Reasons, "the implementation does not cover the request") } if r.TestsMeaningful < 0.5 { r.Reasons = append(r.Reasons, "the tests do not exercise the requested behaviour") } if out.Quality.Level() == 0 { r.Reasons = append(r.Reasons, "code quality is poor") } if r.Verdict == "ship" && len(r.Reasons) > 0 { r.Verdict = "rework" } return r, nil }) The Reasons are what the fix flow gets, together with the logs. Again: the decision model decides, your code says it in words. Step 5: Three Pipelines and a State Machine The module under construction lives in a persistent workspace, so every run of one request works on the same tree. senro insists a persistent workspace is bounded, and refuses one without both bounds: Go func repo() *senro.WorkspaceRef { return senro.Workspace("factory-repo", senro.Scope(senro.ScopePersistent), senro.MaxAge(24*time.Hour), senro.MaxSize(256<<20)) } The three pipelines are short. Every AI step gets the same shape through a helper, with only the secrets it needs: Go func aiStep(s *senro.StepBuilder, ws *senro.WorkspaceRef, secrets ...string) *senro.StepBuilder { s = s.Mount(ws.At("/repo", senro.RW)).Timeout(stepTimeout).Retry(3, retry.OnInfra()) for _, name := range secrets { s = s.SecretEnv("SENRO_"+name, name) } return s } func buildPipeline(outDir string, rework bool) *senro.Pipeline { name, fn := "implement", "factory/implement" if rework { name, fn = "rework", "factory/fix" } p := senro.New(name) ws := repo() code := p.Workflow("code", senro.On(pods())) aiStep(code.Step(name, senro.Func(fn, NoParams{})), ws, "Gemini") goStep(code, "vet", name, "go vet ./...", ws) goStep(code, "test", "vet", "go test -count=1 ./...", ws) aiStep(code.Step("review", senro.Func("factory/review", NoParams{})), ws, "TypeSafe").Needs("test") handoff := p.Workflow("handoff", senro.Needs("code"), senro.On(senro.Local())) handoff.Step("collect", senro.Func("factory/collect", DirParams{Dir: outDir})). Mount(ws.At("/repo", senro.RO)) return p } finish declares both endings and lets a run parameter pick one, the same senro.When pattern as the previous article: Go func finishPipeline(outDir, reason string) *senro.Pipeline { p := senro.New("finish") ws := repo() ship := p.Workflow("ship", senro.On(pods()), senro.When(senro.ParamIs("verdict", "ship"))) aiStep(ship.Step("pr", senro.Func("factory/pr", NoParams{})), ws, "Gemini") publish := p.Workflow("publish", senro.Needs("ship"), senro.On(senro.Local()), senro.When(senro.ParamIs("verdict", "ship"))) publish.Step("publish", senro.Func("factory/publish", DirParams{Dir: outDir})). Mount(ws.At("/repo", senro.RO)) escalate := p.Workflow("escalate", senro.On(pods()), senro.When(senro.ParamIs("verdict", "human"))) aiStep(escalate.Step("brief", senro.Func("factory/brief", BriefParams{Reason: reason})), ws, "Gemini") // ...and a notify workflow on the coordinator that collects BRIEF.md return p } And main is the state machine. Each run ends with a record jev produced, and the record picks the next run: Go state, reason, reworks := "design", "", 0 for { switch state { case "design": if err := run(designPipeline(request, out)); err != nil { exit(err) } var d DesignResult read(out, "design.json", &d) if d.Verdict == "ready" { state = "implement" } else { state, reason = "human", fmt.Sprintf("the design review ended %q after %d round(s)", d.Verdict, len(d.Rounds)) } case "implement", "rework": // A red vet or test is a failed step, so the run is partial; the // review still ran (ContinueOnError) and its verdict is what counts. if err := run(buildPipeline(out, state == "rework")); err != nil && !partial(err) { exit(err) } var r Review read(out, "review.json", &r) switch r.Verdict { case "ship": state = "ship" case "rework": if reworks >= *maxRework { state, reason = "human", fmt.Sprintf("jev asked for rework %d times: %v", reworks+1, r.Reasons) } else { reworks++ state = "rework" } default: state, reason = "human", fmt.Sprintf("the code review ended %q: %v", r.Verdict, r.Reasons) } case "ship", "human": if err := run(finishPipeline(out, reason), senro.WithParams(senro.Params{"verdict": state})); err != nil { exit(err) } return } } Every arrow in the state diagram is one of those cases, and every run is a directory under runs/ with its plan, its events and one log per step. Step 6: Run It The first request is a token-bucket rate limiter, the kind of package every service ends up with: Plain Text # Feature request: a token-bucket rate limiter package We need a small, dependency-free Go package `ratelimit` (module path `github.com/acme/ratelimit`) that other services can import. - `New(rate float64, burst int) *Limiter`: `rate` is tokens added per second, `burst` is the bucket capacity. - `(*Limiter) Allow() bool`: takes one token if available, otherwise returns false without blocking. - `(*Limiter) AllowN(n int) bool` for batches. - Safe for concurrent use from many goroutines. - Time must be injectable so tests are deterministic (no sleeping in tests). - Standard library only. Tests included, run with `go test ./...`. Shell go run . -request requests/token-bucket.md The first run is the design. Read back with senro attach --run <id> --ui=plain: Plain Text clean succeeded design stdout | round 1: jev-1.13.0 decided "revise" (p_ready 0.44) request_clear=0.62 scope_creep=0.36 underspecified=0.56 missing_tests=0.38 risk=Low design stdout | - specify every component fully: fields, signatures, edge cases and error behaviour design stdout | round 2: jev-1.13.0 decided "ready" (p_ready 0.57) request_clear=0.57 scope_creep=0.49 underspecified=0.28 missing_tests=0.34 risk=Low design succeeded plan stdout | module github.com/acme/ratelimit, 3 files, 11 steps plan succeeded collect succeeded run succeeded Two rounds. jev found the first design underspecified (0.56), the feedback said so in one sentence, and the second design named every field, signature, and edge case. The ready probability moved from 0.44 to 0.57, and the design that came out of it is the one Gemini implemented; this is its component list in DESIGN.md: Plain Text ## Components - type Limiter struct { mu sync.Mutex; rate float64; burst float64; tokens float64; last time.Time; now func() time.Time } - concurrency-safe state tracking ... - func New(rate float64, burst int) *Limiter - public constructor delegating to newWithClock using time.Now; clamps negative rate to 0.0 and negative burst to 0 ... - func newWithClock(rate float64, burst int, now func() time.Time) *Limiter - internal constructor enabling time injection for tests; ... panics if now is nil. - func (l *Limiter) Allow() bool - convenience method returning l.AllowN(1) ... - func (l *Limiter) AllowN(n int) bool - thread-safe token deduction; edge cases: returns true immediately if n <= 0 without modifying state; returns false immediately if n > int(l.burst); advances tokens by elapsed.Seconds() * l.rate capped at l.burst ... The plan turns that into three files and eleven steps, and the second run executes it. Plain Text implement stdout | wrote go.mod (42 bytes) implement stdout | wrote ratelimit.go (1371 bytes) implement stdout | wrote ratelimit_test.go (3172 bytes) implement succeeded vet succeeded test stdout | ok github.com/acme/ratelimit 0.009s test succeeded review stdout | jev-1.13.0 decided "ship" (confidence 0.66) quality=Excellent covers_request=0.87 tests_meaningful=0.95 build_green=0.98 review succeeded collect succeeded run succeeded Gemini wrote the module, go vet and go test ran in the pod against it, and jev read the three files and the logs and said ship, with the build at 0.98 and the tests at 0.95. The module it reviewed: Go // AllowN reports whether n tokens can be consumed immediately. func (l *Limiter) AllowN(n int) bool { l.mu.Lock() defer l.mu.Unlock() if n <= 0 { return true } if float64(n) > l.burst { return false } curr := l.now() elapsed := curr.Sub(l.last) if elapsed > 0 { l.tokens += elapsed.Seconds() * l.rate if l.tokens > l.burst { l.tokens = l.burst } l.last = curr } if l.tokens >= float64(n) { l.tokens -= float64(n) return true } return false } with six tests beside it: clamping, a nil clock panic, burst capacity, AllowN edge cases, deterministic replenishment against an injected clock, and a race test over concurrent goroutines. No sleeping anywhere, as the request demanded. The third run has both endings declared and runs one: Plain Text pr stdout | # Implement token-bucket rate limiter package pr stdout | ## Summary pr stdout | Introduces `github.com/acme/ratelimit`, a thread-safe, zero-dependency token-bucket rate limiter. ... pr stdout | ## Tests pr stdout | ok github.com/acme/ratelimit 0.009s pr stdout | ## Risk pr stdout | Low. Mutex contention under high concurrency is mitigated by keeping critical sections limited to basic in-memory arithmetic. ... pr succeeded publish stdout | published 3 files to .../out/token-bucket/src publish succeeded brief skipped_condition notify skipped_condition run succeeded Three senro runs, seven pods, two Gemini designs, two jev design reviews, one implementation, one vet, one test, one jev code review and one pull request, in a little over a minute and a half. out/token-bucket/ holds every record and the module under src/. Step 7: When jev Says No The second request is the kind every team gets: Plain Text # Request The app feels slow lately, especially in the afternoon. Can you make it faster? Nothing specific, just whatever you think is best. Thanks. Shell go run . -request requests/make-it-faster.md Plain Text clean succeeded design stdout | round 1: jev-1.13.0 decided "reject" (p_ready 0.13) request_clear=0.05 scope_creep=0.87 underspecified=0.70 missing_tests=0.79 risk=Low design stdout | - remove everything the request did not ask for design stdout | - specify every component fully: fields, signatures, edge cases and error behaviour design stdout | - add a test case for every requirement in the request design succeeded plan stdout | design ended "reject", nothing to plan plan succeeded collect succeeded run succeeded Gemini did what a chat model does with a vague request: it invented a feature, in this case pprof routes and a latency middleware. jev put the probability that the request is clear at 0.05, and the factory's rule turned that into reject in the first round, before a plan, a line of code or a second model call. The finish run then takes the other ending: Plain Text pr skipped_condition publish skipped_condition brief stdout | ### What was asked brief stdout | A user submitted a vague performance complaint stating the app "feels slow lately, especially in the afternoon" and asked to "make it faster" without providing specific endpoints, metrics, or reproduction steps. brief stdout | ### Why it stopped brief stdout | The design was rejected on Round 1 (Readiness: 13%, Clarity: 5%). The factory attempted to introduce Go `net/http/pprof` diagnostic routes and custom in-memory latency middleware, triggering a rejection for high scope creep (87%) ... brief stdout | ### What a human should decide brief stdout | 1. **Clarify the issue:** Contact the user to identify affected routes, user workflows, and target latency SLOs. brief stdout | 2. **Authorize observability work:** Determine if the team should prioritize a dedicated ticket for production profiling/metrics ... brief succeeded notify succeeded run succeeded The brief is written by Gemini, from the record jev produced. The two models keep their jobs even in the failure path. A Request jev Cannot Be Talked Into The third request is a glob matcher: *, **, ?, character classes with ranges and negation, backslash escapes, and an error for an unterminated class. It is precise, it is testable, and the factory never got to write it: Plain Text clean succeeded design stdout | round 1: jev-1.13.0 decided "revise" (p_ready 0.39) request_clear=0.68 scope_creep=0.35 underspecified=0.49 missing_tests=0.32 risk=Medium design stdout | round 2: jev-1.13.0 decided "revise" (p_ready 0.39) request_clear=0.65 scope_creep=0.40 underspecified=0.34 missing_tests=0.22 risk=Medium design stdout | round 3: jev-1.13.0 decided "revise" (p_ready 0.47) request_clear=0.59 scope_creep=0.66 underspecified=0.33 missing_tests=0.20 risk=Medium design stdout | - remove everything the request did not ask for design succeeded Gemini's design transpiles the glob into a Go regular expression. jev never called it ready, the ready probability never crossed 0.5 in three rounds, and in two of those rounds none of the specific questions crossed the line either, so the feedback Gemini got was the generic "tighten the approach". The loop cannot converge on a complaint it has no question for, and after three rounds the factory stops and writes the brief: Plain Text ## What was asked Implement a standard-library Go package (`github.com/acme/glob`) providing `Compile(pattern string) (*Glob, error)` and `(*Glob) Match(path string) bool`. ... ## Why it stopped The design review failed after 3 rounds, ending in a "revise" verdict (`p_ready` plateaued at 0.46). Automated reviewers flagged persistent specification risks and scope creep regarding the chosen approach (transpiling globs into Go `regexp`), specifically around complex regex metacharacter escaping and separator handling in negated character classes. ## What a human should decide 1. **Architecture:** Approve regex transpilation (accepting edge-case escaping risks) or mandate a dedicated AST/rune-based glob parser. 2. **Path boundaries:** Explicitly define exact semantics for `**` edge cases and `/` exclusions inside negated classes. That brief is the factory working as designed. A decision model only answers the questions you ask, and when it keeps saying no without any of them firing, the honest move is to stop and ask a person, with the evidence attached. The fix for the next version of the factory is one more field on DesignReview: a question about the approach itself, so that "this should be a parser, not a regex" becomes a sentence Gemini can act on. What the Combination Gives You ConcernWho handles itDesign, plan, code, fix, pull request textGemini, through genkit.GenerateData and GenerateTextIs the design ready, should the code shipjev, through the same GenerateData call with a decision typeFeedback for the next roundYour code, one sentence per question jev answered "yes"Compiling and testinggo vet and go test, as command steps in the pod, with no model involvedA red build reaching the reviewerContinueOnError on the Go stepsThe module between runsA bounded persistent workspaceWhich ending runssenro.When(senro.ParamIs("verdict", ...))Credentialsmamori resolves, senro delivers a file per pod, per stepEvidenceruns/<id>/ per run, plus .factory/ records that travel with the code Conclusion A software factory is not one big agent. It is a handful of small, typed calls with a judge between them, and the judge is the part most people leave out. jev makes that part cheap: a design review is one call that answers six typed questions in a few hundred milliseconds, and the questions are Go fields you choose. Put the loop inside a Genkit flow, the flow inside a senro step, the step in a pod, and the verdict in a run parameter, and the factory is a state machine you can read in one screen. The code: genkit-k8s-flows-jev-software-factoryThe previous article: Genkit flows on Kubernetes, routed by a decision modelDecision models in Genkit Go and Flowssenro's Kubernetes executor and persistent workspaces

By Xavier Portilla Edo CORE

AI-generated pull requests are moving from experimental workflow to routine software delivery. Coding agents can now plan changes, edit multiple files, run tests, and submit pull requests with limited human intervention. A 2025 study of 567 Claude Code pull requests across 157 open-source projects found that 83.8% were eventually merged, and about 55% of merged changes required no further modification. A broader 2026 study analyzed more than 302,000 verified AI-authored commits from 6,299 repositories. The practical question is therefore no longer whether generated code can enter production. The harder question is what happens after the merge, when readability problems, architectural drift, latent defects, and future maintenance costs become part of the codebase. Faster Generation Changes the Review Equation AI assistance does not automatically imply lower quality code. A GitHub randomized controlled study involving 202 experienced developers found that Copilot-assisted implementations performed better on the study's functional tests and received modestly higher ratings for readability, reliability, maintainability, and conciseness. That evidence matters because technical debt should not be treated as an unavoidable property of machine-generated code. Production repositories, however, create conditions that controlled coding tasks cannot fully reproduce. The 2026 large-scale study of AI-authored commits identified 484,366 introduced issues where 89.3% were code smells, and more than 15% of commits attributed to every assistant studied introduced at least one detectable issue. Most importantly, 22.7% of tracked AI-introduced issues were still present at the repository's latest analyzed revision. The same research also found that AI-authored commits can remove existing smells, showing that generated code can both create and repay debt. That distinction changes the engineering problem. The central risk is not generation itself but generation speed combined with weak validation. DORA's 2025 research describes AI as an amplifier that magnifies existing organizational strengths and weaknesses. Earlier DORA analysis also associated increased AI adoption with larger change batches, slower review, and weaker delivery stability. Faster code production therefore raises the value of disciplined review rather than reducing it. Debt Begins at the Merge Boundary Technical debt becomes an organizational liability when a change is accepted into a maintained branch. The model that produced the code cannot own an incident, negotiate a migration, preserve a service level objective, or explain an architectural compromise six months later. Accountability must therefore follow the repository and the component, not the mechanism that generated the diff. This makes the pull request author field an increasingly weak representation of ownership. A generated change may carry an agent identity, a bot account, or a human account that delegated most implementation work. None of those identities answers the operational question of which engineering group is responsible for the behavior after deployment. Repository ownership rules provide a stronger boundary. GitHub CODEOWNERS can automatically request reviewers for designated paths, and protected branch settings can require approval from a code owner before merge. Required status checks can additionally prevent merging until defined CI checks pass. A policy can remain simple: Plain Text /src/main/java/com/acme/payments/ @payments-platform /src/main/java/com/acme/identity/ @identity-platform The important property is not the syntax. The important property is that an AI-generated payment change still reaches the team accountable for payment correctness, while an identity change still reaches the team accountable for authentication and authorization. Provenance may influence review depth, but provenance should never replace domain ownership. Human Review Becomes More Valuable, Not Less Higher generation throughput can create pressure to automate review with another model. AI review is useful for summarization, repetitive checks, suspicious patterns, and localized suggestions, but available evidence does not support treating it as equivalent to accountable human review in every context. A 2026 study covering 278,790 review conversations across 300 mature open source projects found that human reviewers exchanged 11.8% more review rounds when handling AI-generated code than human-written code. Human reviewer suggestions were adopted at a much higher rate than AI-agent suggestions, 56.5% versus 16.6%, and more than half of unadopted AI suggestions were either incorrect or addressed through alternative fixes. Another study analyzing more than 22,000 comments from AI-based GitHub review actions found substantial variation in effectiveness and reported that concise, contextual, hunk-level feedback was more likely to produce code changes. Current GitHub documentation explicitly states that Copilot code review does not replace human code review. GitHub also supports configurations in which Copilot approvals can count toward merge requirements, making repository policy an important governance decision rather than a tooling default. For critical components, AI review is better positioned as an additional signal before accountable approval, not as the entity carrying final ownership. Quality Gates Must Follow Risk A generated pull request should not receive weaker controls because tests pass. Passing tests establish only the behavior covered by those tests. Technical debt often appears elsewhere as duplicated logic, boundary violations, unnecessary abstractions, missing failure handling, insecure defaults, excessive coupling, or code that satisfies the immediate task while conflicting with local design conventions. CI can convert some of those conventions from reviewer memory into executable policy. An AI-assisted label, for example, can trigger additional checks without requiring a separate delivery pipeline: YAML ai-change-gate: if: contains(github.event.pull_request.labels.*.name, 'ai-assisted') steps: - run: ./gradlew test integrationTest - run: ./gradlew check - run: ./scripts/verify-architecture.sh The architecture check can encode repository-specific constraints such as forbidden dependencies, package boundaries, or service-layer access rules. The AI label is only a routing signal where the underlying controls should be based on change risk. A generated documentation correction does not require the same scrutiny as an authorization change, while a human-written authorization change should receive the same high-risk checks. GitHub rulesets support required checks and required reviews as merge conditions, allowing these policies to be enforced rather than documented as optional guidance. The Useful Metric Is Rework Counting generated lines, completed pull requests, or acceptance rate measures production volume, not maintenance quality. A more meaningful evaluation connects AI-assisted changes to downstream engineering cost. Review rounds, escaped defects, rollbacks, post-merge rework, issue survival, architecture violations, and time spent modifying recently merged code reveal whether higher implementation throughput is creating future work. This framing also avoids a false binary between "AI code" and "human code." The large-scale debt study found both issue introduction and issue removal in AI-authored commits. The relevant objective is therefore not minimizing AI involvement, but minimizing unmanaged debt while preserving useful productivity gains. Ownership should remain attached to the team that accepts and operates the change, while automated gates provide evidence that acceptance criteria were actually satisfied. Conclusion AI-generated pull requests change who produces code, but they do not change who must maintain software after deployment. Technical debt cannot be delegated to a model, hidden behind a bot identity, or treated as the future responsibility of an unspecified reviewer. The durable ownership boundary is the engineering team responsible for the affected component, supported by required human approval, executable quality gates, repository ownership rules, and measurements that expose downstream rework. AI can accelerate implementation and can even remove existing debt, but merge authority must remain tied to accountable engineering ownership. When generation becomes nearly instantaneous, disciplined acceptance becomes the scarce resource, and that acceptance is where responsibility for technical debt belongs.

By Uthej Mopathi CORE

Last month, a nightly regression came back with 60 failures out of roughly a thousand tests. I spent the first half of the next morning fixing none of them. I sorted them: some were the same failure under different names, some were environment noise, and some were the product actually breaking. By lunch it was three things. A dependent service had returned errors for about a two-hour window, a data seeding job hadn't finished before the suite started, and one genuine defect. Sixty red rows, three problems. That sorting is most of my week. It's also the part nobody automates, because it isn't a fixed sequence of steps. It's judgment applied to evidence scattered around a CI dashboard, a log group, a ticket history, and whatever got merged Friday afternoon. So I built a triage step that does the mechanical sorting for me. Not an agent, not self-healing tests. A post-suite job that clusters failures, pulls surrounding evidence, and hands me a summary I can verify. This is what's in it, what it cost, and the two ways it went wrong before it went right. What Automation Is Good At, and What It Isn't Automation answers one question extremely well: did this pass or did it fail? It answers it faster than I can and the same way every time, which is the whole point. What it doesn't tell me: Why did this fail?Are these eight failures actually one failure?Which of them do I care about before the release call at 11?Have we seen this before? (We usually have.)What should I look at next? Every one of those is a reading problem, not an execution problem. The data already exists. It's just scattered around, and reassembling it by hand is slow and boring. The split I settled on: automation executes, a model reads, and I decide. The third part isn't a formality, and most of the engineering below exists to protect it. Step One: Cluster Without the Model My first mistake was handing raw failures to a model and asking what happened. Sixty stack traces is a lot of tokens; the answer took thirty seconds and was vague in a way that's hard to argue with. The fix was to do the deterministic part deterministically. Most failures in a big suite are duplicates. You can find that out with string normalization and a hash — no inference involved. Python # triage/cluster.py import hashlib import re from collections import defaultdict from xml.etree import ElementTree as ET # Order matters: strip the most specific patterns first. NOISE = [ (re.compile(r"\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-" r"[0-9a-f]{4}-[0-9a-f]{12}\b"), "<UUID>"), (re.compile(r"\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}\S*"), "<TS>"), (re.compile(r"0x[0-9a-fA-F]+"), "<ADDR>"), (re.compile(r"https?://[^\s'\"]+"), "<URL>"), (re.compile(r"'[^']{0,80}'"), "<STR>"), (re.compile(r"\b\d+\b"), "<N>"), ] def signature(message: str) -> str: """Collapse a failure message to a stable shape.""" if not message.strip(): return "" first_line = message.strip().splitlines()[0][:300] for pattern, replacement in NOISE: first_line = pattern.sub(replacement, first_line) return first_line def cluster(junit_path: str) -> dict: groups = defaultdict(list) for case in ET.parse(junit_path).getroot().iter("testcase"): problem = case.find("failure") if problem is None: problem = case.find("error") if problem is None: continue shape = signature(problem.get("message", "")) key = hashlib.sha1(shape.encode()).hexdigest()[:10] groups[key].append({ "test": f"{case.get('classname')}.{case.get('name')}", "shape": shape, "time": float(case.get("time", 0)), }) return groups On that 60-failure run, this collapsed to six clusters. Two of them held 30 and 15 tests. That single number — 60 down to 6 — is most of the value, and no AI was involved in producing it. It also changes the economics of everything downstream. The model never sees 60 stack traces. It sees six normalized shapes plus a slice of evidence for each shape, which is roughly 4K tokens instead of 60K. Cheaper, faster, and a much smaller surface for the thing to make something up on. If you build one piece of this, build this piece. Step Two: Attach Evidence, With Hard Limits Each cluster then gets enriched from sources that are already there: the log group for the window the suite ran in, the deploys that landed since the last green run, and any tickets whose text matches the normalized shape. Python # triage/evidence.py import boto3 logs = boto3.client("logs") PLACEHOLDERS = ("<UUID>", "<TS>", "<ADDR>", "<URL>", "<STR>", "<N>") def log_hits(shape: str, start_ms: int, end_ms: int, group: str, limit: int = 5): """Pull a bounded sample of matching log lines for one cluster.""" # The normalized shape has placeholders in it; search on the # longest literal run instead. stripped = shape for token in PLACEHOLDERS: stripped = stripped.replace(token, "|") literal = max(stripped.split("|"), key=len).strip() if len(literal) < 12: return [] # too generic to search on; don't guess response = logs.filter_log_events( logGroupName=group, startTime=start_ms, endTime=end_ms, filterPattern=f'"{literal}"', limit=limit, ) return [ {"ts": e["timestamp"], "stream": e["logStreamName"], "message": e["message"][:400]} for e in response.get("events", []) ] Two constraints in there are deliberate, and both came from things going wrong. The limit=5 is because the second version of this pulled every matching line and blew past the context window on a noisy cluster. Five representative lines with timestamps answer "is this the same error, and when did it start" just as well as five hundred. The len(literal) < 12 guard is because a cluster whose shape normalizes down to something like Expected but was will match half the log group. Returning nothing is correct there. An empty evidence list is a fact, and the prompt below is written so the model has to say so rather than fill the gap. Step Three: Ask for a Reading, Not a Verdict The prompt is the part I rewrote the most. What finally worked was forcing every claim to point at a piece of evidence by ID, and constraining the service field to a list of things that actually exist. Python SYSTEM = """You are triaging failures from one CI run. Rules: - Every claim must cite evidence by its id. No id, no claim. - If the evidence does not support a cause, set cause to null. Do not guess. "Insufficient evidence" is a correct answer. - suspected_service must be one of KNOWN_SERVICES or null. - Do not restate the stack trace back to me. Return only JSON matching this schema: {"clusters":[{"cluster_id":str,"test_count":int, "cause":str|null,"suspected_service":str|null, "confidence":"low"|"medium"|"high", "evidence_ids":[str],"suggested_next_step":str}]} """ Before that enum existed, the model named a service we don't run. It sounded completely reasonable — right naming convention, plausible responsibility, adjacent to the real one. That's the failure mode that actually costs you: not obvious nonsense, but a confident answer that reads as if it came from someone who knows your architecture. An engineer who didn't know the service list would have gone looking for it. KNOWN_SERVICES gets injected from the same dependency manifest the deployment uses, so it can't drift from reality. Step Four: Check Its Work Before a Human Sees It This is the piece I'd argue is non-optional, and it's about forty lines. Python # triage/validate.py def validate(model_output: dict, clusters: dict, evidence: dict, window: tuple) -> dict: start_ms, end_ms = window kept, dropped = [], [] for item in model_output.get("clusters", []): cid = item.get("cluster_id") if cid not in clusters: dropped.append((cid, "unknown cluster id")) continue if item.get("test_count") != len(clusters[cid]): item["test_count"] = len(clusters[cid]) # we own this number cited = set(item.get("evidence_ids", [])) real = {e["id"] for e in evidence.get(cid, [])} if item.get("cause") and not cited: dropped.append((cid, "cause with no evidence")) continue if not cited <= real: dropped.append((cid, f"cited missing evidence: {cited - real}")) continue for e in evidence[cid]: if e["id"] in cited and not (start_ms <= e["ts"] <= end_ms): dropped.append((cid, "evidence outside run window")) break else: kept.append(item) return {"clusters": kept, "dropped": dropped} Counts come from the parser, never from the model — that's the one class of error I refuse to ship, because a wrong test count makes every other number in the summary suspect. Anything citing evidence that doesn't exist gets dropped rather than shown with a caveat, since a caveat still puts the idea in the reader's head. The dropped list is not thrown away. It's the closest thing I have to a quality signal on the triage step itself, and I look at it weekly. When one cluster type keeps landing there, the problem is usually my evidence collection, not the model. What It Actually Produces Before, for one cluster: Plain Text 30 tests failed. After: Plain Text cluster 4a1c9f2e — 30 tests cause: dependent service returned 503 on order lookups service: order-api confidence: high evidence: log lines at 02:14:06, 02:31:44, 03:02:10 (same message, same stream); no matching lines in the prior night's run next: check order-api health for the 02:10–03:20 window The second one I can disprove in about ninety seconds. That's the actual bar. Not "is it right" — "can I check it quickly." Every design decision above serves that. Numbers Triage time on a large regression run: roughly 90 minutes to about 20.Clusters per run: 5–8, from 40–70 raw failures.Model cost: a few cents per run. Not a factor.Wall clock for the triage step: under a minute, running after the suite.Claims dropped by the validator: around 1 in 8 in the first month, closer to 1 in 20 once the service enum and evidence limits were in. That last number is the one I'd watch if you build this. A rising drop rate means something upstream changed. Where I Stopped, On Purpose The obvious next step is letting it act. Reopen the ticket, rerun the cluster, patch the test. I did the first two. I won't do the third. The pitch for self-healing tests goes: the agent notices a failing test, decides the locator is stale, updates it, the test goes green, nobody gets paged. Run that on a test that was failing because the product broke. The agent didn't fix anything. It deleted the evidence, faster than a human would have noticed. The suite is green, the dashboard is calm, the defect ships. Green isn't the goal. Green with the check still intact is the goal, and nothing I've built or used reliably tells the difference between "this assertion is stale" and "this assertion just caught something." So I tier by blast radius: Tier Examples Gate Read-only clustering, summarizing, log search, explaining a trace none — worst case I ignore it Suggest root cause, test selection, defect priority, proposed edits engineer reviews before it moves Change config, closing defects, disabling tests, anything release-related explicit approval, always Most of the anxiety about AI in QA comes from treating it as one permission level. It isn't one. Testing the Thing That Helps You Test There's a part of this the industry hasn't sorted out, and I'm including it because pretending otherwise would be dishonest. The output isn't deterministic, so pass/fail assertions don't map onto it cleanly. What I do instead is keep a fixture set of about twenty past runs where I already know the real root cause, and score new prompt or model versions against them on two things: did it name the right cause, and did it cite evidence that actually supports it. It's regression testing with a rubric instead of an assert, and it's the only reason I noticed when a model version change made the summaries longer and vaguer without making them wrong. That's the other thing to plan for. Behavior shifts underneath you on someone else's release schedule. A prompt-based workflow that's been stable for months can quietly degrade after an upgrade nobody on your team asked for. Pin your model version, and treat a version bump as a change that needs a test run. And you are feeding logs — possibly source, possibly customer data — into a service. Whatever your org's answer is on that, get it in writing before you build this, not after. It's a security conversation more than a QA one, but it lands on our desk anyway. The Point The most valuable thing here isn't generating tests. It's reading everything we already collect and can't get through fast enough. Speed comes from the model. Repeatability comes from automation. Evidence comes from systems we already run. The call stays with me, mostly because I have to answer for the release.

Beyond Manual Mapping: AI-Orchestrated Governance for Hybrid Cloud It's Tuesday morning, and Slack is already on fire. A Salesforce admin renamed a custom field over the weekend, and your Spring Boot microservice has been throwing 400 errors since 6 a.m. You spend the next four hours tracing logs, patching a DTO, and redeploying. By Wednesday, you're in an "alignment meeting," manually cross-referencing field mappings between two systems that are supposed to talk to each other without your help. Every architect running a hybrid Salesforce and Spring Boot stack has lived this week. It's not a one-time incident; it's a recurring tax on the org. It grows with every schema change, every new field, every integration you bolt on. Most teams accept it as the cost of doing business. It doesn't have to be. Where the Wheels Come Off The friction comes from two architectures that were built with opposite priorities. Salesforce is optimized for speed. A business user creates a custom field in Setup, and it's live in minutes, no code review, no pipeline, no waiting. Spring Boot is optimized for rigor. Fields live in strongly typed DTOs. Changing one means a commit, a PR review, a test run, and a deployment through CI/CD. That's a feature, not a bug; it's what keeps your backend from falling over. When the two connect over REST, that mismatch turns into outages. Here's the kind of DTO that breaks the moment Phone__c becomes PhoneNumber__c on the Salesforce side: Java public class CustomerDTO { private String customerId; private String phone; // maps to Salesforce field Phone__c // getters/setters omitted } Nothing in this class tells you the field it depends on just got renamed. The Salesforce admin has no idea a Java class is bound to Phone__c. The developer has no idea the picklist they're parsing was just restructured with two new values. Information doesn't flow; it fragments. Teams patch around it, and the patches pile into a Frankenstein integration made of XSLT transforms, one-off converters, and tribal knowledge that lives in one person's head. That tax is measurable. Developers routinely lose 30 to 40 percent of their time to keeping brittle connections alive instead of shipping features. Every schema change becomes a small bet against production. A Different Approach: Zero-Touch Governance What if the system watched itself and fixed what it could, with a human still holding the approval button? That's the idea behind AI-orchestrated API governance: an orchestration layer that watches both Salesforce and Spring Boot, understands what changed and why it matters, and proposes (or, once trusted, implements) the fix. The important word is "understands." A naive diff tool sees Cust_Phone__c disappear and CustomerPhone__c appear, and flags it as a breaking deletion plus a breaking addition. An AI layer doing semantic analysis recognizes it as a rename, maps it to the existing customerPhoneNumber field in your DTO, and drafts the OpenAPI and Java changes before anyone gets paged. Building the Intelligence Layer Three components have to work together for this to be trustworthy instead of terrifying. 1. The Observation Engine This layer taps Salesforce Change Data Capture (or Platform Events) on one side and your Spring Boot Actuator endpoints on the other, then asks an LLM to classify what actually happened. A simplified CDC listener might look like this: Java @Component public class SalesforceChangeListener { @EmpConnectorSubscription(channel = "/data/Contact_ChangeEvent") public void onFieldChange(ChangeEvent event) { FieldChange change = FieldChange.builder() .objectName(event.getEntityName()) .oldFieldApiName(event.getChangedFieldOldName()) .newFieldApiName(event.getChangedFieldNewName()) .changeType(event.getChangeType()) .build(); semanticClassifier.classify(change); } } The classifier itself is a targeted prompt, not a general chatbot call: Java String prompt = """ A Salesforce field changed on object %s. Old API name: %s New API name: %s Change type: %s Classify this as one of: RENAME, TYPE_CHANGE, DEPRECATION, NEW_FIELD, BREAKING_REMOVAL. If RENAME, return the confidence score and the likely corresponding Spring Boot DTO field name. Respond only in JSON. """.formatted(objectName, oldName, newName, changeType); That last constraint matters more than it looks. Constraining the model's output to a fixed schema is what makes the downstream Policy Engine able to act on the result programmatically instead of parsing prose. 2. The Policy Engine Detection without guardrails is just a faster way to break things. The Policy Engine enforces the rules your architecture team already agreed on, things like "no breaking changes to public APIs without a major version bump" or "PII fields must stay encrypted in transit." Java public class GovernancePolicy { public PolicyResult evaluate(ProposedChange change) { if (change.isBreaking() && !change.hasMajorVersionBump()) { return PolicyResult.blocked( "Breaking change requires API major version bump"); } if (change.touchesPiiField() && !change.isEncrypted()) { return PolicyResult.blocked( "PII fields must remain encrypted end to end"); } if (change.isCustomerFacing() && change.deprecationWindowHours() < 48) { return PolicyResult.blocked( "Customer-facing changes need a 48 hour deprecation window"); } return PolicyResult.approved(); } } If a proposed change trips one of these rules, automation stops and an architect gets looped in. That's the whole point: the system earns trust by refusing to act outside its lane, not by acting fast. 3. The Execution Layer Once a change clears policy, this layer turns the proposal into an actual pull request. It can regenerate the OpenAPI contract: YAML CustomerDTO: type: object properties: customerId: type: string customerPhoneNumber: # renamed from "phone" type: string description: "Renamed from Salesforce Cust_Phone__c to CustomerPhone__c" and the matching Java class: Java public class CustomerDTO { private String customerId; private String customerPhoneNumber; // was: phone, mapped from CustomerPhone__c // getters/setters omitted } then open a PR, run the existing test suite in staging, and post the results back to the team. The API contract becomes the single source of truth both systems reconcile against, instead of a document that drifts out of date the week after it's written. A Second Example: Picklist Drift Renames aren't the only failure mode. Picklists shift constantly, and Spring Boot enums don't forgive silently. Say a Salesforce admin adds "Pending Review" to an Order_Status__c picklist that a Java enum is validating against: Java public enum OrderStatus { NEW, IN_PROGRESS, SHIPPED, CANCELLED // "Pending Review" doesn't exist here yet } Any order that lands in that new status now fails deserialization. The Observation Engine catches the new picklist value through the Metadata API, the Policy Engine checks whether enum changes count as breaking under your versioning rules, and the Execution Layer proposes the enum update alongside a migration note for any records currently in that state. That's a five-minute PR instead of a production incident discovered by a customer. What This Means for Your Team The obvious worry is that this replaces developers. It doesn't; it changes what they spend their day on. Right now, a good chunk of your time goes to translation work: mapping fields, writing converters, chasing down what changed and why. That work is necessary, but it isn't strategic. Once an orchestration layer is handling the translation, your role shifts from translator to curator. You define the data architecture, you write the policies the system enforces, and you spend your attention on the problems that actually need a human's judgment. Making the Transition Nobody should flip this on in production on day one. Trust gets earned in stages. Start with AI-Suggested Governance. The system proposes changes and opens PRs; a human reviews and merges. You're delegating the tedious analysis, not the decision. Once accuracy holds up across enough changes, typically approaching 99 percent, move to AI-Automated Testing. The system applies changes in staging, runs your regression suite, and reports results without a human triggering each step. Only after it's proven itself across hundreds of real changes should you consider Zero-Touch in Production, where the system can act autonomously within policy boundaries you've already defined and tested. Wrapping Up The hybrid enterprise, Salesforce for the customer relationship, Spring Boot for the backend logic, isn't going away. But manually bridging the two with mapping layers and tribal knowledge is a relic you can retire. AI-orchestrated API governance doesn't remove developers from the loop. It removes the busywork from their day and gives back the four hours you used to spend in a war room every time a field got renamed.

Go has quietly become the language of AI infrastructure. MCP servers, agent runtimes, model gateways, Kubernetes operators that talk to LLMs: a large share of that plumbing is written in Go, because Go is what you reach for when something has to be a small static binary that handles thousands of concurrent connections without drama. What was missing for a long time was the framework layer on top of the raw provider SDKs. In 2026, that gap is closed, and the question is no longer "Go or Python?" but "which Go framework?" This article covers the five frameworks a Go developer will actually run into in 2026: Genkit Go, Eino, Google ADK Go, tRPC-Agent-Go, and LangChainGo. Four of them are alive and moving fast. The fifth, LangChainGo, is the one most Go developers heard of first, and it is now effectively unmaintained. It stays in the list because you will find it in existing codebases, and you need to know what that means for you. Same rules as the Java and JavaScript/TypeScript editions: this is not a benchmark post. Tokens per second go stale in weeks. This is a developer experience and architecture comparison, with working code and the real output of that code for every framework, all against the same model, gemini-3.8-flash, on Go 1.27. Versions used: Genkit Go v1.13.1, Eino v0.9.20, Google ADK Go v2.4.0, tRPC-Agent-Go v1.11.2, LangChainGo v0.1.14. APIs as of September 2026. Genkit Go History and Direction Genkit was announced by Google at I/O 2024 as a TypeScript-first framework. The Go SDK arrived only three months later, with v0.1.0 in August 2024, and unlike the Java SDK, which is a community effort, Genkit Go is an official, first-party SDK maintained by the same Google team that ships the TypeScript runtime. It reached 1.0 in September 2025 and sits at v1.13.1 as of September 2026, on a cadence of roughly one minor release per month. The direction is parity with TypeScript plus Go-native ergonomics: generics instead of Zod schemas, context.Context everywhere, plain net/http handlers for flows, and an agents package, still marked experimental, with session stores, multi-agent delegation, artifacts, and a Skills middleware. The monorepo (genkit-ai/genkit) has 6.4k stars and 80 commits in the last 30 days across all SDKs. If you want the long-form introduction, start with Stop Using Python for Your Gen AI Apps, Use Go and Genkit Instead. What Makes Genkit Go Stand Out Genkit Go gives you three levels of abstraction in a single SDK: direct model calls, typed flows, and agents, all traced by the same OpenTelemetry pipeline and all visible in the same Dev UI. On top of that sits a middleware layer (retry, fallback, tool approval, filesystem, skills) that wraps any of the three. No other Go framework gives you that combination. Supported languages: Go 1.25+ for this SDK. The same framework exists for TypeScript, Python, Dart/Flutter, and, unofficially, Java. Vanilla Generation Go import ( "github.com/genkit-ai/genkit/go/ai" "github.com/genkit-ai/genkit/go/genkit" "github.com/genkit-ai/genkit/go/plugins/googlegenai" ) g := genkit.Init(ctx, genkit.WithPlugins(&googlegenai.GoogleAI{}), genkit.WithDefaultModel("googleai/gemini-3.8-flash")) text, err := genkit.GenerateText(ctx, g, ai.WithPrompt("Explain the CAP theorem in two sentences.")) Plain Text The CAP theorem states that a distributed data system can simultaneously provide at most two of three guarantees: Consistency, Availability, and Partition tolerance. Because network partitions are inevitable in the real world, systems must ultimately choose between returning the most up-to-date data (Consistency) or ensuring every request receives a response (Availability). Typed Flows: Observable Pipelines Flows are the heart of Genkit. A flow is a named, typed, traceable unit that you can call in-process, expose as an HTTP endpoint with one line, and run from the Dev UI. The JSON schema for the output comes from the Go struct, and GenerateData returns the struct, not a string you have to parse. Go type TranslateRequest struct { Text string `json:"text"` TargetLanguage string `json:"target_language"` } type TranslateResponse struct { Translation string `json:"translation"` DetectedLanguage string `json:"detected_language"` } translate := genkit.DefineFlow(g, "translateText", func(ctx context.Context, req TranslateRequest) (*TranslateResponse, error) { out, _, err := genkit.GenerateData[TranslateResponse](ctx, g, ai.WithPrompt("Translate %q to %s and detect the source language.", req.Text, req.TargetLanguage)) return out, err }) // Call it in-process... out, err := translate.Run(ctx, TranslateRequest{Text: "Buenos días", TargetLanguage: "English"}) fmt.Printf("%+v\n", *out) // ...or expose it over HTTP (and to the Dev UI). mux := http.NewServeMux() mux.HandleFunc("POST /translateText", genkit.Handler(translate)) log.Fatal(server.Start(ctx, "127.0.0.1:3400", mux)) Plain Text {Translation:Good morning DetectedLanguage:Spanish} INFO server listening addr=127.0.0.1:3400 Shell curl -X POST http://127.0.0.1:3400/translateText \ -H 'Content-Type: application/json' \ -d '{"data":{"text":"Buenos días","target_language":"English"}' # {"result":{"translation":"Good morning","detected_language":"Spanish"} Tools and Agents Tools are typed functions. The input schema is inferred from the struct, and the tool loop (model asks, tool runs, model answers) happens inside Generate. Go type WeatherInput struct { City string `json:"city"` } weather := genkit.DefineTool(g, "getWeather", "Returns the current weather for a city.", func(ctx *ai.ToolContext, in WeatherInput) (string, error) { return fmt.Sprintf("Sunny, 24C in %s", in.City), nil }) resp, err := genkit.Generate(ctx, g, ai.WithPrompt("What's the weather like in Tokyo?"), ai.WithTools(weather)) fmt.Println(resp.Text()) Plain Text The weather in Tokyo is currently sunny and 24°C. When you need memory across turns, the experimental agents package adds sessions on top of the same tools. Enable it with genkit.WithExperimental(): Go import ( aix "github.com/genkit-ai/genkit/go/ai/exp" "github.com/genkit-ai/genkit/go/ai/exp/localstore" genkitx "github.com/genkit-ai/genkit/go/genkit/exp" ) advisor := genkitx.DefineAgent(g, "travelAdvisor", aix.InlinePrompt{ ai.WithSystem("You are a helpful travel advisor. Use tools to give accurate advice. Be brief."), ai.WithTools(weather), }, aix.WithSessionStore(localstore.NewInMemorySessionStore[any]())) first, err := advisor.RunText(ctx, "Should I pack a jacket for my trip to Lisbon?") second, err := advisor.RunText(ctx, "Which city did I ask about?", aix.WithSessionID[any](first.SessionID)) Plain Text It is currently sunny and 24°C (75°F) in Lisbon. You won't need a heavy coat, but packing a light jacket or sweater is recommended for cooler evenings and coastal breezes. You asked about Lisbon. Multi-agent delegation, cross-vendor fallback, and tool approval are covered in Building Agents in Go: Where Genkit Shines vs ADK 2.0. The Dev UI: Same Power as TypeScript The same Genkit Developer UI that TypeScript developers use works with Go, with no code changes. Install the CLI and start your Go program through it: Shell npm install -g genkit genkit start -- go run . Plain Text Genkit Developer UI: http://localhost:4000 The flow above shows up as /flow/translateText next to every Gemini model the plugin registered, and you get: Flow runner: execute any flow with a custom input and inspect the typed output.Trace explorer: OpenTelemetry traces for every Generate and flow call, with latency, token counts, and the exact prompts.Model playground: call any registered model directly.Tool testing: run tools in isolation.Dotprompt editor: edit .prompt files live. This is the single biggest advantage Genkit has over every other framework in this list: a zero-config, local trace explorer that replaces Langfuse or LangSmith during development. Provider Support Official plugins in the Go module: Google AI (Gemini), Vertex AI, Anthropic, Ollama, and an OpenAI-compatible family (compat_oai) with ready-made packages for OpenAI, xAI, DeepSeek, OpenRouter, Kimi, DashScope, and Z.ai. Vector stores: localvec (local, no service needed), PostgreSQL/pgvector, AlloyDB, Pinecone, Weaviate, and Firebase Firestore. Plus MCP (client and server), evaluators, and an A2UI plugin for streaming UI components. Pros and Cons ✅ Pros❌ ConsBest-in-class Dev UI with local trace explorerAgents package still marked experimentalMulti-level abstractions: vanilla, flows, agentsFewer vector stores than LangChainGoMiddleware: retry, fallback, tool approvalGo 1.25+ requiredOfficial Google SDK, monthly releasesTyped output from Go structs, no parsingSame framework in TS, Python, Dart, Java Eino History and Direction Eino (pronounced "aino") comes from ByteDance's CloudWeGo team, the group behind Kitex and Hertz, the RPC and HTTP frameworks that run a large part of ByteDance's Go services. It was open-sourced in December 2024 after internal use, and it is now the most-starred LLM framework in Go at 13.1k stars, ahead of LangChainGo. Releases land every few days: v0.9.20 on September 20, 2026, with v0.10.0 already at alpha 35. Eino's own description is that it draws from LangChain and Google ADK but follows Go conventions. The framework splits into two: compose, for orchestration (Chain, Graph, Workflow), and adk, an Agent Development Kit that borrows Google ADK's vocabulary of agents, runners, and events. Components (models, tools, retrievers, indexers, loaders) live in a separate repo, eino-ext, together with callback handlers for Langfuse, LangSmith, CozeLoop, and ByteDance's APMPlus. The 2026 roadmap is all agents: DeepAgent (planning plus sub-agents), agent middleware, interrupt and resume for human-in-the-loop, skills, A2UI streaming components, and a turn loop with preemption. The English documentation on cloudwego.io is now a full eleven-chapter course, a big change from 2025 when most of it was Chinese-first. What Makes Eino Stand Out Orchestration with automatic stream handling. Every node in an Eino graph can be invoked or streamed, and the framework concatenates, boxes, merges, and copies streams as data moves between nodes. You build the graph once and get both Invoke and Stream for free. Graph inputs and outputs are generic types, so the compiler checks that the nodes fit together. Cross-cutting concerns go through callbacks at fixed points (OnStart, OnEnd, OnError, plus the streaming variants) that apply to components, graphs, and agents alike. Supported languages: Go only. The module declares go 1.18, the lowest bar in this list. One thing to know: Eino depends on ByteDance's sonic JSON library, and on Go 1.27 it prints a warning at startup and falls back to encoding/json. Harmless, but noisy. ChatModel Eino does not own the provider client. You create the genai client yourself and hand it to the component. Go import ( "github.com/cloudwego/eino-ext/components/model/gemini" "github.com/cloudwego/eino/schema" "google.golang.org/genai" ) client, err := genai.NewClient(ctx, &genai.ClientConfig{APIKey: os.Getenv("GEMINI_API_KEY")}) cm, err := gemini.NewChatModel(ctx, &gemini.Config{Client: client, Model: "gemini-3.8-flash"}) msg, err := cm.Generate(ctx, []*schema.Message{ schema.UserMessage("Explain the CAP theorem in two sentences."), }) fmt.Println(msg.Content) Plain Text The CAP theorem states that a distributed data store can simultaneously provide at most two of three guarantees: Consistency, Availability, and Partition tolerance. Because network partitions are practically inevitable in distributed systems, architects must choose between keeping data strictly up-to-date (Consistency) or ensuring the system remains responsive (Availability) during a network failure. Chains, Graphs, and Callbacks A chain is a linear pipeline. Here the callback handler is attached at invocation time and fires for the chain, the template, and the model. Go import ( "github.com/cloudwego/eino/callbacks" "github.com/cloudwego/eino/components/prompt" "github.com/cloudwego/eino/compose" ) template := prompt.FromMessages(schema.FString, schema.SystemMessage("You are a translator. Reply with the translation only."), schema.UserMessage("Translate {text} to {language}.")) chain := compose.NewChain[map[string]any, *schema.Message]() chain.AppendChatTemplate(template).AppendChatModel(cm) runnable, err := chain.Compile(ctx) tracer := callbacks.NewHandlerBuilder(). OnStartFn(func(ctx context.Context, info *callbacks.RunInfo, in callbacks.CallbackInput) context.Context { log.Printf("start %s %s", info.Component, info.Type) return ctx }). OnEndFn(func(ctx context.Context, info *callbacks.RunInfo, out callbacks.CallbackOutput) context.Context { log.Printf("end %s %s", info.Component, info.Type) return ctx }). Build() out, err := runnable.Invoke(ctx, map[string]any{"text": "Buenos días", "language": "English"}, compose.WithCallbacks(tracer)) fmt.Println(out.Content) Plain Text start Chain start ChatTemplate Default end ChatTemplate Default start ChatModel Gemini end ChatModel Gemini end Chain Good morning Graphs are where Eino earns its name. Nodes are typed, edges are explicit, and plain Go functions slot in as lambda nodes: Go graph := compose.NewGraph[map[string]any, string]() _ = graph.AddChatTemplateNode("template", template) _ = graph.AddChatModelNode("generate", cm) _ = graph.AddLambdaNode("shout", compose.InvokableLambda( func(ctx context.Context, msg *schema.Message) (string, error) { return strings.ToUpper(strings.TrimSpace(msg.Content)), nil })) _ = graph.AddEdge(compose.START, "template") _ = graph.AddEdge("template", "generate") _ = graph.AddEdge("generate", "shout") _ = graph.AddEdge("shout", compose.END) runnable, err := graph.Compile(ctx) out, err := runnable.Invoke(ctx, map[string]any{"product": "a Go framework for LLM apps"}) Plain Text BUILD PRODUCTION-GRADE LLM APPS AT GO SPEED. Branches, parallel fan-out, and sub-graphs use the same API, and a compiled graph can be wrapped as a tool and handed to an agent. Eino ADK: Agents and Workflows Tools are inferred from a Go struct with jsonschema tags. The agent runs the ReAct loop and emits one event per step, so you see the tool call, the tool result, and the final answer as separate messages. Go import ( "github.com/cloudwego/eino/adk" "github.com/cloudwego/eino/components/tool" "github.com/cloudwego/eino/components/tool/utils" ) type WeatherInput struct { City string `json:"city" jsonschema:"description=The city to look up"` } weather, err := utils.InferTool("get_weather", "Returns the current weather for a city.", func(ctx context.Context, in WeatherInput) (string, error) { return fmt.Sprintf("Sunny, 24C in %s", in.City), nil }) advisor, err := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{ Name: "travel_advisor", Description: "Helps with trip planning and weather-based advice.", Instruction: "You are a helpful travel advisor. Use tools to give accurate advice. Be brief.", Model: cm, ToolsConfig: adk.ToolsConfig{ ToolsNodeConfig: compose.ToolsNodeConfig{Tools: []tool.BaseTool{weather}, }, }) runner := adk.NewRunner(ctx, adk.RunnerConfig{Agent: advisor}) iter := runner.Query(ctx, "Should I pack a jacket for my trip to Lisbon?") for { event, ok := iter.Next() if !ok { break } if event.Err != nil { log.Fatal(event.Err) } if event.Output == nil || event.Output.MessageOutput == nil { continue } msg, _ := event.Output.MessageOutput.GetMessage() fmt.Printf("[%s] %s\n", msg.Role, msg.Content) } Plain Text [assistant] [tool] Sunny, 24C in Lisbon [assistant] It is currently sunny and 24°C (75°F) in Lisbon. You won't need a heavy coat, but packing a light jacket or cardigan is a good idea for breezy coastal evenings. The first assistant event is the tool call itself, which is why its content is empty. Multi-agent workflows are one constructor away: Go pipeline, err := adk.NewSequentialAgent(ctx, &adk.SequentialAgentConfig{ Name: "content_pipeline", Description: "Research, then write.", SubAgents: []adk.Agent{researcher, writer}, }) Plain Text [researcher] 1. Created at Google by Robert Griesemer, Rob Pike, and Ken Thompson; released in 2009. 2. Statically typed, compiled language with built-in garbage collection. 3. Features native, lightweight concurrency through goroutines and channels. [writer] Created at Google by Robert Griesemer, Rob Pike, and Ken Thompson, the Go programming language was released in 2009. It is a statically typed, compiled language that features built-in garbage collection and native, lightweight concurrency through goroutines and channels. NewParallelAgent and NewLoopAgent complete the set, and DeepAgent adds planning and delegation on top. Developer Tooling Eino's answer to observability during development is the Eino Dev plugin for GoLand and VS Code, backed by the devops module in eino-ext. It renders your chains and graphs visually inside the editor and lets you start execution from any node with mock input. It is an IDE feature rather than a browser Dev UI, so there is no trace explorer for a running service. For that you attach one of the callback handlers (Langfuse, LangSmith, CozeLoop, APMPlus). Provider Support Model components in eino-ext: OpenAI, Claude, Gemini, DeepSeek, Qwen, Ollama, OpenRouter, Ark (Volcano Engine), and Qianfan (Baidu), plus "agentic" variants of the main ones for the newer agentic message format. Retrievers and indexers: Elasticsearch (7, 8, 9), Milvus, Qdrant, Redis, OpenSearch, Dify, and VikingDB. Tools: MCP, Google Search, Bing, DuckDuckGo, SearXNG, Wikipedia, HTTP requests, command line, browser use, and sequential thinking. Pros and Cons ✅ Pros❌ ConsMost active independent Go LLM framework (13.1k stars)Still 0.x with frequent alphas; expect API churnTyped graph orchestration with automatic streamingNo browser Dev UI; visual debugging lives in the IDE pluginCallback aspects for logging, tracing, and metricsVerbose: config structs and explicit error checks everywhereADK-style agents plus DeepAgent and interrupt/resumeByteDance-flavoured corners (Ark, VikingDB, CozeLoop)Broad component ecosystem in eino-extsonic dependency warns on the newest Go toolchainsRuns on Go 1.18+Chinese-first history still shows in some docs and examples Google ADK Go History and Direction ADK, the Agent Development Kit, was announced at Google Cloud Next 2025 with Python as its first language. The Go SDK made its first tagged release, v0.1.0, in November 2025, went 1.0 on March 23, 2026, and shipped a new major module, google.golang.org/adk/v2, on June 30, 2026. It is at v2.4.0 (September 11, 2026), has 8.8k stars, and logged over 100 commits in the last 30 days: the highest commit volume in this list. It is an official Google product with the Python, Java, Kotlin, and TypeScript SDKs as siblings. The design is the same as its siblings: everything is an agent, a workflow agent, or a tool. Gemini is the native model, with openaimodel for anything speaking the OpenAI wire format and an Apigee model for enterprise gateways. Deployment gravity points at Vertex AI Agent Engine, Cloud Run, and GKE, with Pub/Sub and Eventarc triggers built into the launcher. The minimum Go version is 1.26.6, the highest bar of the five. A deeper look at the low-level differences with Genkit is in Genkit Go vs ADK Go 2.0. ADK Go's Position: Agent-Only, Enterprise-Grade ADK Go is an agent framework. There is no vanilla generation primitive and no flow abstraction: you always build an agent, give it to a runner, and iterate the events. In exchange, you get sessions, memory, artifacts, workflow agents, agent-to-agent (A2A) protocol support, MCP toolsets, tool confirmation, and an embedded web UI, all first-party. Supported languages: Go 1.26.6+. Same framework in Python (primary), Java, Kotlin, and TypeScript. Go import ( "google.golang.org/adk/v2/agent" "google.golang.org/adk/v2/agent/llmagent" "google.golang.org/adk/v2/model/gemini" "google.golang.org/adk/v2/runner" "google.golang.org/adk/v2/tool" "google.golang.org/adk/v2/tool/functiontool" "google.golang.org/genai" ) model, err := gemini.NewModel(ctx, "gemini-3.8-flash", &genai.ClientConfig{ APIKey: os.Getenv("GOOGLE_API_KEY"), }) weather, err := functiontool.New(functiontool.Config{ Name: "get_weather", Description: "Returns the current weather for a city.", }, func(_ agent.Context, in WeatherInput) (WeatherOutput, error) { return WeatherOutput{Report: fmt.Sprintf("Sunny, 24C in %s", in.City)}, nil }) advisor, err := llmagent.New(llmagent.Config{ Name: "travel_advisor", Model: model, Instruction: "You are a helpful travel advisor. Use tools to give accurate advice. Be brief.", Tools: []tool.Tool{weather}, }) r, err := runner.NewInMemory("travel-app", advisor) msg := genai.NewContentFromText("Should I pack a jacket for my trip to Lisbon?", genai.RoleUser) for event, err := range r.Run(ctx, "user-1", "session-1", msg, agent.RunConfig{}) { if err != nil { log.Fatal(err) } if event.LLMResponse.Content == nil || !event.IsFinalResponse() { continue } for _, part := range event.LLMResponse.Content.Parts { fmt.Print(part.Text) } } Plain Text It is currently sunny and warm at 24°C (75°F) in Lisbon. You won't need a heavy jacket during the day, but packing a light jacket or sweater is recommended for cooler evenings and coastal breezes. Two things to notice. The runner returns a Go 1.23 iterator, so you range over events with the error as the second value, which is the most idiomatic event loop of the five. And the function tool is generic: input and output structs give the schema; nothing is spelled out twice. Multi-Agent Orchestration Workflow agents compose LLM agents. The OutputKey writes an agent's answer into session state, and the next agent reads it back through a {placeholder} in its instruction. Go import "google.golang.org/adk/v2/agent/workflowagents/sequentialagent" researcher, err := llmagent.New(llmagent.Config{ Name: "researcher", Model: model, Instruction: "List three key facts about the user's topic. Be terse.", OutputKey: "facts", // saved into session state }) writer, err := llmagent.New(llmagent.Config{ Name: "writer", Model: model, Instruction: "Write a two-sentence brief using these facts:\n{facts}", }) pipeline, err := sequentialagent.New(sequentialagent.Config{ AgentConfig: agent.Config{ Name: "content_pipeline", SubAgents: []agent.Agent{researcher, writer}, }, }) Plain Text [researcher] * **Origin:** Developed at Google by Robert Griesemer, Rob Pike, and Ken Thompson; released as open source in 2009. * **Concurrency:** Features built-in, lightweight concurrency primitives known as goroutines and channels, managed by its own runtime. * **Ecosystem Role:** Statically typed and compiled, it is the primary language behind core cloud-native technologies, including Docker and Kubernetes. [writer] Developed at Google by Robert Griesemer, Rob Pike, and Ken Thompson and released as open source in 2009, Go is a statically typed, compiled language featuring built-in, lightweight concurrency primitives known as goroutines and channels that are managed by its own runtime. Owing to these strengths, it has become the primary language powering core cloud-native technologies, including Docker and Kubernetes. parallelagent and loopagent follow the same pattern. The Dev UI: ADK Web, Embedded in Your Binary The full launcher turns any agent into a CLI with console and web modes. The web mode takes sub-launchers, and webui serves the Angular ADK Web app straight out of your Go binary via go:embed, so there is nothing to install: Go import ( "google.golang.org/adk/v2/cmd/launcher" "google.golang.org/adk/v2/cmd/launcher/full" ) l := full.NewLauncher() config := &launcher.Config{AgentLoader: agent.NewSingleLoader(advisor)} if err := l.Execute(ctx, config, os.Args[1:]); err != nil { log.Fatalf("%v\n\n%s", err, l.CommandLineSyntax()) } Shell go run . web -port 8080 api webui Plain Text Web servers starts on http://localhost:8080 api: you can access API using http://localhost:8080/api The root URL redirects to /ui/, the Agent Development Kit Dev UI, and /api/list-apps answers ["assistant"]. The launcher also wires a2a, pubsub, and eventarc sub-servers, and an -otel_to_cloud flag exports traces to Google Cloud. Flags go before the sub-launcher names, or the sub-launcher rejects them. Vertex AI Gravity You can run ADK Go anywhere. But the session and memory services with persistence are the in-memory ones, a database-backed session store, and Vertex AI; the evaluation tooling, the managed scaling, and the telemetry defaults all assume Google Cloud. Same trade-off as ADK Java and ADK Python: the best experience is on GCP. Pros and Cons ✅ Pros❌ ConsOfficial Google support, highest commit velocityAgent-only: no vanilla generation, no flowsIdiomatic Go: generics for tools, iterators for eventsGo 1.26.6+ requiredEmbedded ADK Web UI, no extra installStrong pull toward Vertex AI and GCPSequential, parallel, and loop workflow agentsGemini-first; other providers only through OpenAI-compatible endpointsA2A, MCP toolsets, tool confirmation, artifactsNo vector store components; RAG goes through tools or VertexSame mental model as ADK Python/Java/TSConfig structs get long for real agents tRPC-Agent-Go History and Direction tRPC-Agent-Go is Tencent's entry, built by the team behind tRPC, the RPC framework that underpins much of Tencent's backend. Development started in May 2025; the first public release (v0.0.1) landed in August 2025, and the project reached 1.x at the end of December 2025. It is now at v1.11.2 (August 20, 2026), with 1.8k stars and 56 commits in the last 30 days. The README credits Tencent Yuanbao, Tencent Video, Tencent News, IMA, and QQ Music with real-world validation, and I have no reason to doubt it. The framework is unapologetically ADK-shaped: llmagent, runner, chainagent, parallelagent, cycleagent, sessions, memory, OutputKey, {placeholder} instructions. If you know ADK, you already know most of it. What it adds on top is where it gets interesting: a GraphAgent that is a faithful LangGraph in Go (state schemas, conditional edges, checkpoints, time travel), an AG-UI server for streaming agent events to frontends, A2A, MCP, Anthropic-style Agent Skills with sandboxed execution, agent self-evolution, an evaluation package, model failover and hedging, prompt caching, and OpenTelemetry traces and metrics with Langfuse examples. RAG is covered by a knowledge module with embedders for OpenAI, Gemini, Ollama, and Hugging Face, and stores from in-memory and SQLite-vec up to pgvector, Elasticsearch, Milvus, Qdrant, and TcVector. What Makes tRPC-Agent-Go Stand Out The GraphAgent. Nobody else in Go ships a LangGraph-equivalent state graph as a first-class agent type that plugs into the same runner, sessions, and telemetry as the LLM agents. Supported languages: Go only, go 1.21+. Model providers: OpenAI and anything OpenAI-compatible (DeepSeek, Qwen, and friends through variants), Anthropic, Gemini, Bedrock, Ollama, Hugging Face, and Hunyuan. Go import ( "trpc.group/trpc-go/trpc-agent-go/agent/llmagent" "trpc.group/trpc-go/trpc-agent-go/model" "trpc.group/trpc-go/trpc-agent-go/model/gemini" "trpc.group/trpc-go/trpc-agent-go/runner" "trpc.group/trpc-go/trpc-agent-go/tool" "trpc.group/trpc-go/trpc-agent-go/tool/function" ) type weatherReq struct { City string `json:"city" jsonschema:"description=The city to look up,required"` } type weatherRsp struct { Report string `json:"report"` } func getWeather(ctx context.Context, req weatherReq) (weatherRsp, error) { return weatherRsp{Report: fmt.Sprintf("Sunny, 24C in %s", req.City)}, nil } m, err := gemini.New(ctx, "gemini-3.8-flash") // reads GEMINI_API_KEY / GOOGLE_API_KEY weather := function.NewFunctionTool(getWeather, function.WithName("get_weather"), function.WithDescription("Returns the current weather for a city.")) advisor := llmagent.New("travel_advisor", llmagent.WithModel(m), llmagent.WithInstruction("You are a helpful travel advisor. Use tools to give accurate advice. Be brief."), llmagent.WithTools([]tool.Tool{weather})) r := runner.NewRunner("travel-app", advisor) events, err := r.Run(ctx, "user-1", "session-1", model.NewUserMessage("Should I pack a jacket for my trip to Lisbon?")) for evt := range events { if evt.Error != nil { log.Fatalf("%s", evt.Error.Message) } if evt.Response == nil || len(evt.Response.Choices) == 0 { continue } if evt.IsFinalResponse() { fmt.Println(evt.Response.Choices[0].Message.Content) } } Plain Text It is currently sunny and warm in Lisbon at 24°C (75°F), so you will not need a heavy coat. However, packing a light jacket or sweater is recommended for cooler, breezy evenings near the coast. Events arrive on a channel and use OpenAI's response shape (Choices[0].Message or Choices[0].Delta when streaming), which is familiar but means digging through nested structs for every message. Chains The chain agent is ADK's sequential agent under another name, OutputKey and {facts} included: Go import ( "trpc.group/trpc-go/trpc-agent-go/agent" "trpc.group/trpc-go/trpc-agent-go/agent/chainagent" ) researcher := llmagent.New("researcher", llmagent.WithModel(m), llmagent.WithInstruction("List three key facts about the user's topic. Be terse."), llmagent.WithOutputKey("facts")) writer := llmagent.New("writer", llmagent.WithModel(m), llmagent.WithInstruction("Write a two-sentence brief using these facts:\n{facts}")) pipeline := chainagent.New("content_pipeline", chainagent.WithSubAgents([]agent.Agent{researcher, writer})) Plain Text [researcher] 1. Developed at Google by Robert Griesemer, Rob Pike, and Ken Thompson; released in 2009. 2. Statically typed, compiled language featuring automatic garbage collection. 3. Native concurrency support using lightweight threads called goroutines and channels. [writer] Developed at Google by Robert Griesemer, Rob Pike, and Ken Thompson and released in 2009, Go is a statically typed, compiled language featuring automatic garbage collection. It is designed with native concurrency support, utilizing channels and lightweight threads called goroutines. GraphAgent: LangGraph in Go A state graph with a routing node, a conditional edge, and two LLM nodes. Short questions go to the quick node, long ones to the deep node. Go import ( "trpc.group/trpc-go/trpc-agent-go/agent/graphagent" "trpc.group/trpc-go/trpc-agent-go/graph" ) sg := graph.NewStateGraph(graph.MessagesStateSchema()) sg.AddNode("route", func(ctx context.Context, state graph.State) (any, error) { return nil, nil // pure routing node, no state change }) sg.AddLLMNode("quick", m, "Answer in one short sentence.", nil) sg.AddLLMNode("deep", m, "Answer thoroughly. Follow the length the user asks for.", nil) sg.SetEntryPoint("route") sg.AddConditionalEdges("route", func(ctx context.Context, state graph.State) (string, error) { input, _ := state[graph.StateKeyUserInput].(string) if len(strings.Fields(input)) > 8 { return "deep", nil } return "quick", nil }, map[string]string{"quick": "quick", "deep": "deep"}) sg.SetFinishPoint("quick") sg.SetFinishPoint("deep") g, err := sg.Compile() router, err := graphagent.New("router", g) r := runner.NewRunner("graph-app", router) Asked "What is Go?" and then a request for two paragraphs on goroutines and channels: Plain Text Go is an ancient strategy board game, as well as an open-source programming language developed by Google. Unlike operating system threads that require megabytes of memory and expensive kernel-level context switching, goroutines are extremely lightweight user-space constructs that start with just a few kilobytes of stack space. [...] The same graph package gives you checkpoints, interrupts for human input, sub-graphs, map-reduce fan-out, per-node callbacks, and a visualization export. It is the most complete workflow engine of the five, and it is the reason to look at this framework even if you never use its LLM agents. Pros and Cons ✅ Pros❌ ConsGraphAgent: a real LangGraph for GoSmallest community of the five (1.8k stars)ADK mental model without the GCP gravityNo Dev UI; observability means OpenTelemetry plus LangfuseAG-UI, A2A, MCP, Agent Skills, evaluationHuge option surface (llmagent alone has dozens)Model failover and hedging built inOpenAI-shaped event structs are verbose to consumeOpenTelemetry traces and metrics first-partyTencent-flavored corners (Hunyuan, tRPC integration)Runs on Go 1.21+Documentation partly Chinese-first LangChainGo History and Direction LangChainGo was the first. Travis Cline started it in February 2023, weeks after LangChain Python went viral, and for two years it was the default answer to "how do I call an LLM from Go". It has 9.7k stars, 17 LLM providers, and 15 vector stores, and the LangChain names (chains, prompts, agents, memory, outputparser) make it instantly readable to anyone coming from Python. And it has stalled. The last release, v0.1.14, is from October 20, 2025. The last commit to main is from January 11, 2026. It is still 0.1.x after three and a half years. There are 243 open issues and 174 open pull requests. The repository is not archived, but nobody is merging. That would be survivable if the code kept working. It does not: The default model is retired. googleai.New(ctx) with no options targets gemini-2.0-flash, and Google turned that model off: Plain Text default model: gemini-2.0-flash err: googleapi: Error 404: This model models/gemini-2.0-flash is no longer available. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. The Gemini provider sits on a dead SDK. llms/googleai is built on google/generative-ai-go, whose support ended on November 30, 2025. Streaming is broken on Gemini. The chunks arrive, and then the call returns an error, so every caller that checks errors treats the request as failed: Plain Text One, two, three, four, five. err: error in stream mode: invalid character ']' looking for beginning of value No middleware, retry, fallback, or OpenTelemetry anywhere. The callback handler is seventeen observe-only methods with no way to change the request. Tools are Call(ctx, string) (string, error): no schema, no typed arguments. The full teardown, with a capstone app built in both frameworks, is in LangChainGo vs Genkit Go: Where Genkit Shines. LangChainGo's Position: Legacy None of this means the basics stopped working. Pin a current model and the classic API still runs: Go import ( "github.com/tmc/langchaingo/llms" "github.com/tmc/langchaingo/llms/googleai" ) llm, err := googleai.New(ctx, googleai.WithDefaultModel("gemini-3.8-flash")) out, err := llms.GenerateFromSinglePrompt(ctx, llm, "Explain the CAP theorem in two sentences.") Plain Text The CAP theorem states that a distributed data store can simultaneously provide at most two of three guarantees: Consistency, Availability, and Partition tolerance. Because network partitions are inevitable in real-world systems, architects must fundamentally choose between serving the most up-to-date data (Consistency) or keeping the system operational (Availability) when a network failure occurs. Go import ( "github.com/tmc/langchaingo/chains" "github.com/tmc/langchaingo/prompts" ) prompt := prompts.NewPromptTemplate( "Translate {{.text} to {{.language}. Reply with the translation only.", []string{"text", "language"}) chain := chains.NewLLMChain(llm, prompt) out, err := chains.Predict(ctx, chain, map[string]any{ "text": "Buenos días", "language": "English", }) Plain Text Good morning Tools are strings in, strings out. The agent is a ReAct loop driven by prompt text: Go import ( "github.com/tmc/langchaingo/agents" "github.com/tmc/langchaingo/tools" ) type WeatherTool struct{} func (WeatherTool) Name() string { return "get_weather" } func (WeatherTool) Description() string { return "Returns the current weather. Input: a city name." } func (WeatherTool) Call(ctx context.Context, city string) (string, error) { return fmt.Sprintf("Sunny, 24C in %s", city), nil } agent := agents.NewOneShotAgent(llm, []tools.Tool{WeatherTool{}, agents.WithMaxIterations(3)) out, err := chains.Run(ctx, agents.NewExecutor(agent), "What's the weather like in Tokyo?") Plain Text The weather in Tokyo is currently sunny with a temperature of 24°C. If you are on LangChainGo today, the move is not urgent, but it is inevitable: pin your models, avoid streaming on Gemini, and plan the migration. The concept-by-concept guide is in Migrating from LangChainGo to Genkit Go. Pros and Cons ✅ Pros❌ ConsLargest catalog: 17 providers, 15 vector storesUnmaintained since January 2026; still 0.1.xLangChain vocabulary, easy for Python teamsGemini provider on a retired SDK; streaming brokenConversation memory is ergonomicNo middleware, retry, fallback, or OpenTelemetryRuns on Go 1.24+Tools are untyped stringsMany existing tutorials and examplesEvery vector store needs an external service Head-to-Head Comparison Developer Experience FrameworkDX HighlightsShortcomingsGenkit GoDev UI with trace explorer. Typed output from structs. Flows are one line from HTTP.Agents still experimentalEinoTyped graphs, automatic streaming, callbacks everywhere.Verbose config structs; 0.x churnADK GoIterator-based event loop, generic tools, embedded web UI.Agent-only; Go 1.26.6tRPC-Agent-GoGraphAgent, ADK mental model, resilience built in.OpenAI-shaped events; option overloadLangChainGoFamiliar names, biggest catalog.Unmaintained; broken Gemini streaming Abstraction Levels Genkit Go is the only framework here with all three levels in one SDK: vanilla generation, typed flows, and agents. Eino covers generation and orchestration, plus agents through its ADK module. ADK Go and tRPC-Agent-Go are agent frameworks: every call goes through an agent and a runner. LangChainGo has chains and agents, but no typed pipeline and no session-aware agent. Observability FrameworkLocal DevProductionGenkit GoDev UI with trace explorerOpenTelemetry export, Google Cloud plugin, any OTEL collectorEinoEino Dev plugin for GoLand and VS Code (graph rendering, node debugging)Callback handlers: Langfuse, LangSmith, CozeLoop, APMPlusADK GoEmbedded ADK Web UIOpenTelemetry to Google Cloud (-otel_to_cloud)tRPC-Agent-GoEvents on a channel, no UIOpenTelemetry traces and metrics, LangfuseLangChainGoLogging callbacks onlyNothing built in Provider Support FrameworkModel ProvidersVector StoresGenkit GoGemini (Google AI, Vertex AI), Anthropic, Ollama, OpenAI-compatible family (OpenAI, xAI, DeepSeek, OpenRouter, Kimi, DashScope, Z.ai)localvec, pgvector, AlloyDB, Pinecone, Weaviate, FirestoreEinoOpenAI, Claude, Gemini, DeepSeek, Qwen, Ollama, OpenRouter, Ark, QianfanElasticsearch, Milvus, Qdrant, Redis, OpenSearch, Dify, VikingDBADK GoGemini, OpenAI-compatible, ApigeeNone as components (Vertex AI Search via tools)tRPC-Agent-GoOpenAI-compatible, Anthropic, Gemini, Bedrock, Ollama, Hugging Face, HunyuanIn-memory, SQLite-vec, pgvector, Elasticsearch, Milvus, Qdrant, TcVectorLangChainGo17, including OpenAI, Anthropic, Google AI, Vertex, Bedrock, Cohere, Mistral, Ollama15, all external services How Go Does It Feel? FrameworkMin GoToolsStreamingStyleGenkit Go1.25Generic, schema from structCallback per chunkFunctional options (ai.With...)Eino1.18Generic (utils.InferTool)StreamReader, automatic across nodesConfig structs plus builder chainsADK Go1.26.6Generic (functiontool.New)iter.Seq2 over eventsConfig structstRPC-Agent-Go1.21Generic plus jsonschema tagsChannel of eventsFunctional optionsLangChainGo1.24Untyped stringsCallback per chunk (fails on Gemini)Interfaces plus option funcs Only ADK Go uses range-over-func iterators for its event loop. Genkit and Eino use generics the most naturally; LangChainGo predates all of it and shows its age. Maintenance Snapshot Numbers as of September 21, 2026: FrameworkStarsLatest ReleaseLast CommitCommits, Last 30 DaysGenkit Go6.4k (monorepo)go/v1.13.1, Sep 3, 2026Sep 21, 202680 (all SDKs)Eino13.1kv0.9.20, Sep 20, 2026Sep 20, 20267 on main, plus five v0.10 alphasADK Go8.8kv2.4.0, Sep 11, 2026Sep 21, 2026100+tRPC-Agent-Go1.8kv1.11.2, Aug 20, 2026Sep 21, 202656LangChainGo9.7kv0.1.14, Oct 20, 2025Jan 11, 20260 Stars measure the past. The last two columns measure the present, and they tell the story. Which Framework Should You Choose? Choose Genkit Go if: You want to iterate on your AI fast and get feedback with less back-and-forth. The Dev UI is the best local development loop in Go, by a distance.You need vanilla calls, typed flows, and agents in the same service.Provider neutrality matters: mixing Gemini, Claude, OpenAI, and Ollama should be a one-line change.Your team also writes TypeScript, Dart, or Java and wants one framework story. Choose Eino if: Your application is a real graph: branches, parallel fan-out, sub-graphs, and streaming across all of it.You want the most active independent community in Go and are fine living on 0.x.Callback-based instrumentation into Langfuse, LangSmith, or your own APM is what you already do.You need ByteDance-scale orchestration patterns, or you run on Volcano Engine. Choose Google ADK Go if: You are building multi-agent systems and Google Cloud is your runtime.You want official Google support, the embedded ADK Web UI, and A2A interoperability.Your organization already uses ADK in Python or Java and wants Go services with the same shape.The newest Go toolchain is not a problem for you. Choose tRPC-Agent-Go if: You want LangGraph-style state graphs with checkpoints and interrupts, in Go, plugged into agents.You like the ADK mental model but do not want the GCP gravity.AG-UI frontends, Agent Skills, evaluation, and OpenTelemetry out of the box matter more than a Dev UI.You can live with a smaller community and a large API surface. Stay on LangChainGo only if: You already run it in production, your models are pinned, and you do not stream through Gemini.Even then, plan the migration. Nothing is being merged. Conclusion Go's Gen AI framework landscape in 2026 is richer than most Go developers realize, and it has a clear shape. Genkit Go is the most versatile choice and the best developer experience, with the Dev UI as a category of its own. Eino is the ambitious independent, the framework to pick when orchestration is the hard part of your problem. Google ADK Go is the enterprise agent runtime, at its best on Google Cloud. tRPC-Agent-Go is the dark horse, with the best workflow engine of the five hiding behind a modest star count. And LangChainGo is the framework that got everyone started and now needs a farewell plan. If I had to pick one for a new Go service today with no cloud commitment, it would be Genkit Go, for the same reason as in TypeScript and Java: the combination of multiple abstraction levels, provider neutrality, and local observability shortens the loop between an idea and a working feature more than anything else on this list. Whatever you pick, pick it deliberately. Go finally has the tooling. Use it. Last updated: September 2026. Framework versions referenced: Genkit Go v1.13.1, Eino v0.9.20 (with eino-ext gemini v0.1.36), Google ADK Go v2.4.0, tRPC-Agent-Go v1.11.2, LangChainGo v0.1.14. Go 1.27.1, model gemini-3.8-flash.

By Xavier Portilla Edo CORE

Every Abstraction Leaks. Generative AI Is No Exception. Joel Spolsky gave us the Law of Leaky Abstractions years ago, and it lands hard on the newest layer in software development: generative AI. When you prompt a model to produce code, it builds an abstraction over the implementation. You get working functionality without touching the underlying logic. Great, until the day it breaks. So what do you do when the abstraction leaks? The conventional answer is simple: go back and read the code. Tudor Girba, creator of Glamorous Toolkit and the voice behind moldable development, argues that answer is a trap. When AI-generated code fails or needs to be understood, the right move is not heroic manual reading. It is reaching for deterministic, contextual micro-tools that make the system explainable on demand. Here is the uncomfortable part. AI code can follow unfamiliar patterns, skip the mental model a human author would have embedded, and ignore your team's conventions. That often makes it harder to read than human-written code, not easier. And the evidence that the abstraction is already leaking is piling up. The Numbers Are Not Subtle GitClear analyzed millions of changed lines of code from 2020 through 2023. Their finding: code churn, the share of lines reverted or rewritten within two weeks of being authored, is projected to double in 2024 against its 2021 pre-AI baseline. The mix of changes shifted too. "Added" and "copy and pasted" code grew relative to "updated," "deleted," and "moved" code. In plain terms, AI-assisted code tends to break the DRY principle and reads like the output of a short-term contractor rather than a senior engineer. Then there is plausible incorrectness. A peer-reviewed study by Kabir and colleagues looked at 517 ChatGPT answers to Stack Overflow questions. Fifty-two percent contained incorrect information, and users missed that misinformation 39 percent of the time. Why? Because 77 percent of the answers were verbose, and the polished prose masked the errors underneath. People still preferred the ChatGPT answers 35 percent of the time, high error rate and all. Capability is climbing, to be fair. On SWE-bench, Princeton's benchmark of 2,294 real GitHub issues, the best model at launch (Claude 2) solved just 1.96 percent. By mid-2025, purpose-built agent scaffolding reached about 65 percent on a curated 500-instance subset. That is roughly a 33x jump. It is also still one-third of real-world issues left unsolved, because real engineering means coordinating changes across many functions, classes, and files at once. The Verification Bottleneck Is the Leak in Its Purest Form Here is the asymmetry that should keep engineering leaders up at night. Generating 500 lines of code takes seconds. Verifying those lines for security holes or logic errors can take an hour. The volume of AI output is becoming too large for humans to review. That creates pressure to hand review over to the AI itself, which introduces a recursive trust problem: you are now trusting one non-deterministic system to check another. The very speed that makes AI attractive is exactly what makes its failures dangerous. A Taxonomy of How AI Code Fails Failure Mode What Happens Evidence Code churn amplification Lines reverted within two weeks at double the pre-AI rate GitClear, 153M lines DRY violation More copy-paste at the expense of refactoring GitClear Plausible incorrectness 52 percent error rate masked by confident language Kabir et al. Verification asymmetry Generation in seconds, verification in hours GitClear community Compositional failure Models stumble on multi-file coordination SWE-bench Review overload AI output exceeds human review capacity GitClear The Quiet Risk: Skill Atrophy There is a second failure happening in slow motion, and it is about people, not code. Automation bias is the human tendency to favor automated suggestions and overlook contradicting information, even when we are right. In coding, it shows up in two ways. Errors of commission: you accept an AI suggestion without checking it. Errors of omission: you miss a bug because you trusted the output. When the AI eventually fails, and you have lost the skill to diagnose it, you either blindly trust the next suggestion or abandon the tool entirely. Neither is good. GitHub says developers write code 55 percent faster with Copilot. GitClear's data suggests that speed arrives with a maintainability tax, a productivity-versus-quality paradox. The Nielsen Norman Group found generative AI lifts employee performance by an average of 66 percent, with the biggest gains going to the least-skilled workers. Helpful for output, worrying for growth: junior developers may lean on AI before building foundational skills, while seniors use it as a genuine multiplier. This is not a new story. Lisanne Bainbridge named it in 1983 with the "ironies of automation." Automation removes the routine practice that keeps skills sharp, yet the human is needed most precisely when the automation fails. Air France Flight 447 is the hard lesson: when the autopilot disconnected over iced sensors, pilots with degraded manual flying skills stalled the aircraft. Research on GPS navigation shows the same erosion of spatial memory. Passively accepting AI code risks the same slow loss of algorithmic reasoning. Why "Read the Code" Cannot Be the Fallback Anymore Girba's challenge rests on four points, and each one holds up. Scale defeats comprehension. Modern systems are too large and interconnected to understand by reading. AI generates code faster than any team can review it. The reading fallback simply cannot keep pace. AI code lacks authorial context. An LLM may use unexpected idioms and carry none of the design intent a human would embed. Two developers reading the same generated function can reach different conclusions about what it is for. Reading is ad hoc and unrepeatable. It does not scale across a team or across time. Traditional debugging assumes determinism. Breakpoints, step-through, and stack traces all assume the same input yields the same output. LLM output is non-deterministic. The same prompt can produce structurally different code each time, so reproducing a bug becomes unreliable. Multi-step agent workflows branch in ways a conventional stack trace never captures. Semantic equivalence hides errors. Two outputs can be semantically identical but syntactically different, which defeats diff-based and line-by-line debugging. The real question has shifted from "what does this line do?" to "what did this system actually do at runtime?" Reading alone cannot answer that. So Girba reframes the core skill. It moves from "the ability to read and write code" to "the ability to make systems explainable." Moldable Development and the Deterministic Toolkit Moldable development starts from one observation: developers spend most of their time understanding systems, not writing code. Every system is unique, so the tools used to understand it should be moldable, meaning customizable to the system in front of you. Instead of leaning only on generic IDE features, you build many small custom tools: views, inspectors, and queries tuned to your domain. The idea is that any question about a system can be answered through narratives made of contextual tools. Think data science, but pointed at your software. Glamorous Toolkit is the concrete implementation, an open-source environment for building custom inspectors and views for every object in your system. A query result renders as a table, a graph structure renders visually, a business rule shows its decision tree. All deterministic, all contextual. It now ships a built-in moldable agent harness with programmable agents and explainable agent actions, so you can understand what the AI did deterministically rather than taking its word for it. The Broader Ecosystem Is Already Here Time-travel debuggers. Tools like rr record an execution once and replay it deterministically, with identical address spaces, registers, and syscall data every run. Pernosco, built on rr, delivers omniscient debugging with instant navigation and dataflow analysis, and users report roughly 5x faster bug resolution, sometimes 100x. Replay QA explores builds, reproduces failures, and provides deterministic recordings with root cause analysis.Causal tracing. OpenTelemetry lets you understand a system from the outside by tracing the full path of a request. Jaeger implements compatible trace collection and visualization for distributed systems.AI-era observability. Langfuse offers open-source trace logging, cost tracking, and prompt management. LangSmith gives full trace visibility from a single trace up to production-wide metrics. Braintrust surfaces critical patterns in agent traces automatically. OpenTelemetry has standardized GenAI span conventions across OpenAI, Anthropic, and AWS Bedrock.Compiled AI frameworks. Stanford's DSPy compiles programs into optimized, deterministic pipelines, pulling the LLM out of runtime control flow entirely and replacing prompt-response guesswork with reproducible execution. The Behavioral Harness: Deterministic Tools in Production The most mature pattern for wiring deterministic tools into AI coding is the behavioral harness, a two-loop design. The inner loop gives coding agents deterministic feedback during development. The outer loop runs independent automated acceptance tests against the generated service. The inner loop combines guides (instructions that shape the agent) with sensors (deterministic checks that validate output). Contract-driven mocks wire OpenAPI and AsyncAPI specs into the app so downstream interactions are executable, letting the agent verify its work against defined responses, failures, and recovery paths. Thoughtworks draws a useful line between computational controls, run by deterministic tools, and inferential controls, which rely on model judgment. The strongest harnesses combine both, with behavioral checks producing feedback without an LLM judging correctness. Schema tightening is the first line of defense. An "examples validate" step checks payloads against the API spec before any test runs. Sensors then confirm contract conformance, verify HTTP values, and assert observable side effects. Keep guides and sensors aligned, and you get allowlist-style gating: the agent passes only if its output matches the spec. The Result That Changes the Economics In October 2026, Naresh Jain ran a controlled experiment with Specmatic and Thoughtworks: coding agents with and without a behavioral harness, 10 runs each, against 178 automated acceptance tests. Metric Sol (No Harness) Sol (Harness) Luna (No Harness) Luna (Harness) Accepted runs 10/10 10/10 3/10 10/10 Est. cost per run ~$0.41 ~$2.68 N/A ~$0.27 Avg time 6 min 51 sec 19 min 53 sec N/A N/A Read that Luna column again. The frontier model, GPT-6 Sol, passed everything with or without the harness. The cost-optimized GPT-6 Luna succeeded in only 3 of 10 unharnessed runs, then hit 10 of 10 with the harness, at about 33 percent lower cost per run than the frontier model ($0.273 versus $0.406). The takeaway is sharp. Deterministic validation layers can make a cheaper model match frontier reliability. Note the flip side too: the harness added roughly 7x cost to Sol with no quality gain, so harness value depends on the underlying model. Use it where it pays. Humans on the Loop, Not in It Kief Morris of Thoughtworks frames three postures: Humans outside the loop: vibe coding, no oversight.Humans in the loop: line-by-line inspection of every change.Humans on the loop: engineering the harness that validates the work. The "on the loop" model fixes the debugging bottleneck. When output is unsatisfactory, an in-the-loop human patches the artifact directly. An on-the-loop human changes the harness so future runs come out correct. The harness becomes a form of "executable attention," pointing you at the exact contract, response value, or interaction that broke, instead of asking you to read everything. Separate the "why loop" (outcomes) from the "how loop" (code artifacts), and human focus moves to outcome validation rather than implementation detail. The New Job: Harness Engineer The chain from abstraction to skill atrophy to leakage is not a prediction. It is measurable today in churn rates, error frequencies, and benchmark gaps. Girba's reframing reorients the whole problem. The question is not how to preserve our ability to read code. It is how to build systems that stay explainable without anyone having to read the code at all. There is a convergence happening. Moldable development says every question about a system should be answerable through contextual tools. The behavioral harness says deterministic sensors can provide targeted feedback without human inspection. The Specmatic and Thoughtworks experiment shows this is real, not theoretical. So the developer's role is shifting. Less code reader and writer, more harness engineer and system explainer: the person who designs the deterministic tools, specifications, and validation layers that keep AI-generated systems trustworthy. That is not a smaller job. It demands deep system understanding, just expressed through tooling instead of line-by-line reading. The investment priority follows directly. Teams adopting GenAI coding tools should invest proportionally in deterministic contextual tooling: time-travel debuggers, causal tracing, moldable inspectors, behavioral harnesses, and AI-era observability. When the abstraction leaks, and it will, the fallback should be a tool that answers the question, not a developer staring at unfamiliar code. References Glamorous ToolkitGitClear: Coding on CopilotKabir et al., ChatGPT Stack Overflow studyAre Harnesses Worth the Cost? (martinfowler.com)rr projectPernoscoOpenTelemetry observability primerLangfuse docsLangSmith docsDSPyGitClear: Durable Change Velocity 2026SWE-bench paperSWE-benchAutomation biasNielsen Norman Group: AI tools productivity gainsDeskillingAir France Flight 447Glamorous Toolkit siteReplay.ioLangfuse tracingSemantic equivalence studyMoldable DevelopmentJaeger docsBraintrust docsOpenTelemetry GenAI semantic conventionsOpenTelemetry GenAI conventions repoHumans and Agents (martinfowler.com)

By Viquar Khan CORE

"A new disease of which some readers may not yet have heard the name." — John Maynard Keynes (Economic Possibilities for our Grandchildren, 1930) In 1930, John Maynard Keynes told the world it was being afflicted by something new. The disease was technological unemployment: machines saving labor faster than we find new uses for it. Keynes said readers would hear a great deal about it. He was right. Ninety-six years later, you're hearing it again, this time about AI. I've spent more than 20 years as a Chief Technical Architect and Fractional CTO, helping companies modernize the platforms they run on. I started my career in 2003 optimizing a physics engine for games. Today I design AI-first, event-driven platforms that write product content and translate it into 29 languages. Apache Kafka, the event streaming system behind those platforms, didn't exist when I graduated. Neither did anything like ChatGPT. So I have a personal stake in one question: does AI end careers like mine, or change them? Here's my answer, built from history and current research. Machines take tasks, and people invent new work. That pattern has held for well over a century. But this time has one real catch, and it lands first on people just starting out. The Stable Hand Could Not Have Predicted the Mechanic Around 1880, New York and Brooklyn together had between 150,000 and 175,000 horses. Researcher Eric Morris estimates their manure added up to three to four million pounds a day, as he explains in From Horse Power to Horsepower. Whole careers circled those animals: stable hands, blacksmiths, carriage makers, and feed suppliers. Then cars arrived, and many of those jobs shrank for good. That's the half of the story people remember. The other half is mechanics, gas station attendants, road crews, motel owners, and traffic engineers. In 1900, nobody could post a job ad for a mechanic, because there were no cars to fix. The same script ran again with electricity, the personal computer, the internet, and the smartphone. Each wave erased some work, like typing pools and travel agencies, and created work no one had a name for yet. Economists have now measured how much. In a landmark study of US census data, MIT economist David Autor and his coauthors found that: "The majority of current employment is in new job specialties introduced since 1940." — Autor et al., (New Frontiers: The Origins and Content of New Work, 1940-2018, Quarterly Journal of Economics, 2024) In plain terms, roughly six in ten jobs Americans do today did not exist in 1940. The Bank Teller Plot Twist ATMs are the textbook case. They arrived in the 1970s and took over the tasks everyone linked to tellers: handing out cash and taking deposits. The forecast seemed clear. Tellers were finished. Economist James Bessen checked what happened. ATMs did cut the tellers needed at an average urban branch from 20 to 13 between 1988 and 2004. But cheaper branches meant banks opened more of them, and urban branches grew 43 percent. Teller jobs did not fall as ATMs spread. The work itself changed too, as Bessen explained in the IMF's Finance & Development magazine: "Cash handling became less important and human interaction more important" — James Bessen (Toil and Technology, March 2015) Counting cash faded. Relationship building grew. The story has a sequel, and it matters. Mobile banking removed the need for the branch itself. The Bureau of Labor Statistics now projects teller jobs to fall 13 percent between 2024 and 2034. So the lesson isn't that jobs are immortal. It's that outcomes depend on whether demand grows and what work remains after the machine takes its share. Why Cheaper Work Often Means More Work In 1865, economist William Stanley Jevons studied coal. Steam engines were getting more efficient, so people expected coal use to fall. It rose instead. In his book The Coal Question, Jevons wrote: "It is the very economy of its use which leads to its extensive consumption." — William Stanley Jevons (The Coal Question, 1865) Economists now call this the Jevons paradox. When something gets cheaper to use, people use much more of it. Weaving shows the same effect. Bessen notes that power looms automated 98 percent of the labor needed to weave a yard of cloth, yet weaving jobs grew through the 19th century because cheaper cloth sparked far more demand. Now apply the idea to software. If AI makes software faster and cheaper to build, businesses will want more of it: the internal tool nobody could justify, the custom app for a ten-person team. The effect isn't guaranteed, because demand has to grow when prices fall. But it explains why "less labor per unit" so often turns into "more units." A Job Is a Bundle, and Bundles Get Repackaged Here's the mental model that clears up most of the panic. A job isn't one task. It's a bundle of tasks. Automation pulls some tasks out of the bundle, and the rest get repackaged. Autor explains why the leftovers gain value: "Tasks that cannot be substituted by automation are generally complemented by it," — David Autor (Why Are There Still So Many Jobs?, Journal of Economic Perspectives, 2015) Take the teller again. Less cash counting made the human conversation more valuable. I see the same repackaging in AI work. When generative AI drafts a product description or a translation, the scarce skill is no longer typing it. It's defining what "correct" means, building test sets, and measuring precision and recall until quality is a number the business can trust. Precision asks how often the system is right. Recall asks how much it misses. That work wasn't on any job board when I started. The same research team found something every system designer should hear. New work tends to grow from technology that complements what people do, while innovations that only automate tasks or shrink demand for an occupation slow new work down. That's a design brief. Build AI that augments people, not just AI that removes steps. Where the Pattern Gets Uncomfortable So what do the numbers say about AI today? Generative AI went mainstream in November 2022. The Yale Budget Lab tracks the US labor market and updates its findings regularly. Its September 2026 update, which includes August data, says its analysis "does not provide clear evidence of labor market disruption associated with AI." The broad job market looks ordinary so far. The lab itself warns that this may change as new data arrives. But averages hide pockets. A Stanford Digital Economy Lab team of Erik Brynjolfsson, Bharat Chandar, and Ruyu Chen studied payroll data and looked at workers aged 22 to 25 in the most AI-exposed jobs, such as software development and customer service. They found a 16 percent relative decline in employment. More experienced workers in the same jobs stayed stable. The authors call this early evidence consistent with AI hitting entry-level workers hardest, so treat it as a warning light, not a verdict. That's the catch. One plausible reason is that entry-level tasks lean on textbook knowledge, which AI copies well, while experience builds judgment that is harder to copy. Autor says, "the real concern is for whom the new work has been created." That's the right question for 2026. Not "will work exist?" but "who gets the first foothold in it?" Three Moves for Ground That Keeps Shifting First, learn the tools until they feel ordinary. Every wave rewarded the people who got comfortable early, whether it was spreadsheets, the internet, or an AI assistant. Use the tools on real work, not demos. Second, own what the machine hands back. Judgment, quality, and accountability tend to survive automation. Someone must decide what good looks like, catch what the model gets wrong, and answer for the result. If you're early in your career, start now. Check AI output, write down what you changed and why, and build a record of good calls. Third, practice inventing. Roughly six in ten of today's jobs did not exist in 1940, and someone invented every one of them. Build a small tool. Automate a boring process at your workplace. Publish what you learn. Job titles follow evidence. Is this time different? Maybe in speed. Maybe in scope. The data so far says the old pattern is holding, and the uncomfortable parts are real and specific. Both things can be true. Curious people who keep learning, measuring, and building have historically been the ones who found the next job on the far side of the shift. There's little reason to expect that to change. Faisal Feroz is a Chief Technical Architect and Fractional CTO with more than 20 years of experience in enterprise platform modernization, AI-first architecture, and event-driven systems. Readers who want to plan an AI strategy or a career move with a practical eye can connect with Faisal on LinkedIn, explore more of his writing on his blog, or reach out about fractional and advisory engagements. If this article helped you think more clearly about AI and the future of work, pass it to someone who's worried about their next step.

The most consequential security problem in enterprise AI may not be a model giving a wrong answer. It may be what happens when that model is wired to an identity with enough authority to turn the answer into an action. An agent that can query a database, call an infrastructure API, update a ticket, and trigger a workflow is no longer an interface to a model. It is an executing software principal. Most enterprises are still securing it like a web app with a clever text box, and I think that mismatch is where the next wave of incidents will come from. The Question Nobody Logs Picture an incident response agent. A pager fires, the agent pulls telemetry, queries a database, opens a ticket, calls a cloud API, and proposes a fix. No human clicked anything between steps two and five. Now ask the forensic question. Which principal performed each of those operations? Not who typed the prompt, but whose credential hit the database, under what scope, and who approved it. Microsoft's own security guidance names this problem plainly. Its least privilege pattern for agents lists "identity ambiguity" as a core challenge: agents running under shared secrets, service principals, or mixed "on behalf of user" modes, which makes it unclear which principal is responsible for an action. It also notes that logs often capture the chat response rather than the underlying tool actions and downstream authorization decisions. Source: Least privilege for AI agents with Microsoft Entra Agent ID (page updated July 15, 2026). That is the gap. Authentication tells you the agent is who it claims to be. It says nothing about whether each action should be allowed. From Application to Principal In a traditional application, the logic is fixed at design time. A user acts, the app calls an API, the API touches data. You can enumerate the paths. An agent chooses its path at runtime. It selects tools, chains calls, reads results that influence the next call, and may cross system boundaries. Microsoft describes this directly: agents plan and execute multistep workflows, often chain actions across multiple systems, and can take delegated actions with minimal or no human intervention. Once software can decide and act, it needs what every other actor in the enterprise already has: its own identity, an accountable owner, scoped permissions, a lifecycle and a way to be shut off. Microsoft's Entra Agent ID work reflects the same direction, modeling agents as a dedicated identity type rather than reusing ordinary application identities. See the overview of agent identities in Microsoft Entra. My position is blunt. The LLM must never be the security boundary. A model can be persuaded, confused, or hijacked by text inside a retrieved document. The boundary has to sit around the identity, the tools, and the downstream systems, where it can be enforced deterministically. Authentication Is Not Authorization An agent can authenticate perfectly and still be dangerously overprivileged. Take an incident agent that legitimately needs to read logs, metrics, tickets, and configuration, and to write one thing: an incident ticket. It should not be able to delete infrastructure, modify IAM roles, create administrators, export customer records, or change security policy. Microsoft's pattern turns this into concrete guidance. Separate the read role used for evidence gathering from the write role used for ticket creation. Allowlist only the "create or update ticket" actions. Block delete and admin actions. Require human approval for bulk updates. Aggregate Privilege: The Number Nobody Computes Permission creep is old news. Agents give it a new shape. Microsoft calls out "permission creep and excessive aggregate privileges": teams grant broad roles to unblock a pilot and never narrow them, or they layer many narrow roles without task separation, and the combined roles create overly broad effective access. Without aggregate analysis, the agent's true capability across systems is easy to underestimate. Each grant looks defensible in isolation. CRM read, mailbox access, ticketing, a cloud role, a deployment pipeline. Together they describe a high-privilege operator that never took a day of onboarding. So the review question changes from "is this permission safe?" to "what can this agent do end to end when every role and tool is combined?" Microsoft's first implementation step is exactly that: inventory deployed agents and tool integrations and identify effective aggregate permissions. The Tool Surface Every tool an agent can call is attack surface, and the dangerous failures are boring ones. Microsoft's pattern warns that if an agent can invoke any available tool, a single prompt injection or workflow bug can trigger high-impact operations such as export, delete, or privilege changes, or chain actions across services. The industry has started to formalize this. The OWASP GenAI Security Project announced its Top 10 for Agentic Applications on December 9, 2025. The list covers ten risk classes, including goal hijack, tool misuse, and identity and privilege abuse. Its ASI03 entry describes agents operating with borrowed, over-scoped, or long-lived credentials, so a compromised agent inherits far more access than its task needs. A readable summary is in Teleport's breakdown of the OWASP list. The design rule that follows: a tool is not trusted because an agent asked for it. Microsoft's guidance is to deny unreviewed tools, plugins, integrations, and cross-tenant or guest paths by default, and to verify authorization at the downstream system on each call rather than relying on the orchestrator alone. It describes revalidating identity, roles, and scope at every hop, from orchestrator to tool to downstream service, so one weak integration cannot bypass the rest. The trade-off is real, and Microsoft states it: this depends on downstream systems correctly re-checking authorization, and gaps there reintroduce the risk. Delegation Is a Security Boundary When an agent acts as itself, the audit trail is clean. When it acts on behalf of Alice, a harder question appears: does it inherit everything Alice can do? It should not. Alice may hold broad rights that a single task never needs. The fix is explicit "on behalf of" context, narrowed scopes, and a named owner for the agent. Microsoft recommends logging the "on behalf of" user wherever it applies. OWASP's ASI03 framing is the mirror image: delegated credentials, cached sessions, and implicit identity let an attacker use an agent to do what the original owner never intended. Authorization Cannot End at Login Traditional IAM answers "may this identity reach this resource?" Agentic systems need a second question: "should this specific action happen right now?" Microsoft's recommended pattern is a stable, lifecycle-managed agent identity combined with time-limited privileges through just-in-time entitlements, meaning temporary role activation, short-lived tokens, or approvals, so higher privilege exists only for the duration of a specific workflow. For remediation agents, it suggests JIT elevation or approval gates, scoping to specific resource groups, denying subscription-wide rights, keeping rollback procedures, and requiring step-up controls for destructive actions. Here is the decision shape I would want in front of every tool call: Python def authorize(agent_id, on_behalf_of, tool, action, resource, context): scope = effective_scope(agent_id, on_behalf_of) # computed, never assumed if tool not in allowlist(agent_id): return deny("tool not allowlisted") if action not in scope.allowed_actions(resource): return deny("action outside effective scope") if is_high_impact(action): # delete, export, privilege change return require_approval_or_jit(agent_id, action, resource) return allow(ttl_seconds=300, correlation_id=context.correlation_id) The point is not the code. The decision happens outside the model, per action, with a short lifetime on the grant. Log the Action Chain, Not the Conversation A transcript is not an audit trail. Microsoft's guidance lists what a useful record contains: agent identity, role, effective scope, action, resource, correlation ID, and the "on behalf of" user where applicable. Its success metrics include the percentage of agent actions emitting those required fields and the percentage of workflows traceable end to end across orchestrator, tools, and downstream systems. If your SIEM can show that an agent "answered a question" but cannot show which API it called under which scope, you have chat history, not telemetry. Revocation Is a Metric Most agent programs I read about spend pages on capability and a sentence on shutdown. Microsoft treats the stop path as a first-class concern: test revocation by disabling the agent, rotating credentials, invalidating tokens, and removing stale permissions. Its listed success factors include mean time to revoke or disable an agent identity, including token invalidation, and the percentage of environments validated with a tested kill switch and recovery procedure. I would put mean time to revoke on the same dashboard as mean time to detect. If disabling an agent takes a ticket, an approval, and two days, then that agent's autonomy was never bounded. It was only unobserved. Why Legacy Estates Make This Harder Agents cannot safely automate what an enterprise cannot understand or control. Legacy environments tend to hide undocumented business logic, old APIs, fragmented data, inconsistent identity models, and weak observability. Point an agent at that, and you hand a fast, tireless actor a map nobody has drawn. That is why identity, integration, observability and governance now sit next to model selection in modernization work. It is also why vendors in the modernization space are repositioning around agents. On March 3, 2026, Sonata Software said it had been recognized as a Microsoft Frontier Partner, describing an approach that combines AI agents with human engineering across cloud and AI platforms, AI business solutions and security. On October 7, 2026, Sonata Software announced that it had earned the 2026–2027 Microsoft AI Business Solutions Inner Circle award, marking the company’s sixth Inner Circle recognition. Microsoft's Niels Jensen, AI Business Solutions ERP Lead in Enterprise Partner Solutions, was quoted in the release saying, "Inner Circle partners represent our top-selling partners worldwide." (Sonata's press release) Both items are vendor announcements, so read them as signals about where the services market is heading, not as proof of any specific control. What matters for this argument is the shift itself: modernization is being reframed as preparation for governed autonomy, and governance starts with the identity layer. What to Do on Monday Inventory every agent and tool integration, including guest and cross-tenant paths, and compute each agent's aggregate effective permissions.Give each production agent a unique identity and a named owner. Retire shared credentials.Replace broad roles with task-scoped ones. Split read from write.Allowlist tools and actions. Gate delete, export, and privilege changes behind approval or JIT elevation.Enforce authorization at the downstream system, not only in the orchestrator.Emit identity, role, scope, action, resource, and correlation ID for every action, and ship it to your SIEM.Run a revocation drill and record the time it takes. The Argument in One Line The goal is not to stop enterprises from giving agents autonomy. It is to make that autonomy bounded, attributable, and revocable. If an agent can act like an employee, an application, and an automation system at once, the question worth asking is why we still secure it as if it were only an application.

By Igboanugo David Ugochukwu CORE

I was working on a content extraction pipeline for a RAG implementation that ingests regulatory filings and legal documents to create a knowledge repository. The pipeline processes ingested files, runs them through an LLM, and produces structured summaries and extracts key insights. The output looked well-formatted and presented in a professional tone. Everything looked great to pass the unit test or integration test. Then I decided to measure the output. The median F1 score across our initial test set was 0.611, which is well below our 0.80 target. The summaries read correctly. They just weren't complete. The system was confidently presenting a polished version of half the truth. This article is about what I learned building the evaluation approach that uncovered this, which metrics actually matter, what they miss, and the single biggest insight that changed how I think about RAG pipelines. The Failure Mode Here's what our initial numbers looked like: Precision: 0.844 (median) — high. What the model said was almost always correct.Recall: 0.481 (median) — low. It was missing more than half the important content.F1: 0.611 — the harmonic mean told the real story. A system with high Precision but low Recall is the most dangerous failure mode in regulated domains. It never said anything wrong; it just doesn't say enough. Stakeholders see a well-written summary and assume completeness. They don't know what's missing because they didn't read the 160-page original. In my experience, this pattern is common in summarization pipelines that haven't been properly evaluated. If your team hasn't measured Recall independently from Precision, you're probably in the same boat. Picking the Right Metrics I tried every metric in the toolkit. Here's what actually proved useful: What Worked Metric Threshold I Used What It Actually Told Us Token-level F1 > 0.80 target Best single signal for "is this summary complete and accurate?" ROUGE-1 > 0.40 = acceptable Caught cases where the summary used completely different vocabulary BERTScore > 0.85 = acceptable Caught the semantic drift when the summary paraphrased into something different What Disappointed Us Metric Why It Underperformed ROUGE-2 (bigram) Punished valid paraphrases heavily. A perfect summary written in different words scored 0.06. METEOR Theoretically handles synonyms. In practice, scores were so low across the board (0.01–0.04) that they provided no discriminative signal. TF-IDF Cosine Similarity Fooled by topical similarity. Two documents about the same regulation scored high even when the facts differed. Automated Metrics Pitfall None of them catch: Logical errors: Correct facts in the wrong relationshipAttribution/Citation mistakes: "Article 7 states..." when it was actually Article 12Temporal confusion: Mixing up current provisions with repealed onesImportant omissions that aren't in the reference either What I recommend now: Use F1 + ROUGE-1 + BERTScore as automated gates. Then human evaluation on 20–30 samples before any production decision. There's no shortcut past that second step. How Document Length Impacts Scores This was the finding that made everything else make sense. We ran evaluation across documents ranging from 10 to 162 pages. Here's what the data showed: Document Pages Precision Recall F1 Short regulation 10 0.85 0.62 0.71 Medium directive 28 0.83 0.53 0.65 Long regulation 58 0.80 0.42 0.55 Very long regulation 162 0.89 0.04 0.07 That last row surprised me the most. Precision of 0.89 with Recall of 0.04. The system produced an accurate summary of pages 1–5 and completely ignored the other 157 pages. When I ran correlation analysis: Recall vs. document length: r = -0.71 (p < 0.05)F1 vs. document length: r = -0.78 (p < 0.05)Precision vs. document length: r = +0.05 (not significant) The pattern was consistent: as documents got longer, the system's coverage collapsed while its confidence stayed high. It became a confident partial summarizer, which is never wrong, just incomplete. Why This Happens My initial design was to process the first ~5 pages of each document. This was done to reduce token count and keep the cost in control. Usually, you will find the abstract, purpose, and high-level detail of a regulation in most of the regulatory content for any domain. So I thought this would work well, and all stakeholders agreed on this approach. This worked well during demos and the initial testing phase. We tested for a 10-page document; the first 5 pages here mean 50% coverage, which is very good. However, for a 162-page document, that's just 3%. The system literally could not see 97% of the content. If you only test on short documents (which most teams do during development), this never shows up. The evaluation looks great, and we implement it. Then a compliance team relies on a summary that silently drops the critical clause on page 47. How I Fixed It What I Tried Hierarchical summarization: We split the full document into segments or chunks, summarize each one, then produce a final summary from the segment summaries. This guarantees every section gets seen. This improved Recall from 0.48 to ~0.65 on our test set. Not perfect, but a massive improvement for one architectural change.Contextual segmentation: Instead of fixed-page splits, we explored semantic segmentation. Embedding sentences and splitting where topic similarity drops. Better preservation of logical sections, but more complex to implement and tune.Adaptive chunk sizing: Make the chunk size proportional to document length so the entire document always gets processed. Simple, and ensures nothing is skipped. What I'd Recommend Starting With Hierarchical summarization or Summary or Summaries. It's the fastest fix with the highest impact. You can implement it in a day, split into 10-page segments, summarize each, store it in your index, then summarize the summaries. It won't solve everything, but it eliminates the catastrophic "invisible 97%" failure mode on long documents. Building an Evaluation Approach That Actually Works Here's the process I'd set up again if starting from scratch: 1. Get a Stratified Golden Dataset Include documents at 2x, 5x, and 10x your median document length. This is where we failed initially. We tested on short documents and missed the length problem entirely. Minimum 30 document summary pairs. Stratify by length, domain, and complexity. Here's what a single entry looked like in ours: JSON { "doc_id": "REG_2019_1156", "source_file": "regulation_2019_1156.pdf", "pages": 12, "domain": "environmental_compliance", "reference_summary": "This regulation establishes emission standards for...", "key_provisions": [ "Article 4: Emission limits for Category 3 machinery", "Article 7: Penalties up to $10M for non-compliance", "Article 12: Transitional period ending 2024-01-01" ], "difficulty": "medium", "notes": "Critical penalty clause is on page 9 - good test for coverage" } If you notice the field key_provisions, this field was the most valuable addition. It lets us measure not just vocabulary overlap but whether specific critical facts/insights were captured. If Article 7's penalty clause doesn't appear in the summary, that's a material failure regardless of what ROUGE says. I implemented a scoring pipeline that ran nightly: a Lambda function pulled new summaries, computed F1 and BERTScore against the golden references, and pushed results to CloudWatch with alerts on threshold breaches. 2. Score at Multiple Levels Don't just compute one number. Break it down: By document length: This is where you'll find the cliffBy document type: Some content is inherently harderBy section position: Are conclusions (typically at the end) being missed? 3. Set Honest Thresholds Metric Minimum (don't ship below) Target Action if below minimum F1 0.60 0.80 Revisit your chunking/coverage approach Recall 0.50 0.70 You're missing content — increase document coverage Precision 0.75 0.85 Check for hallucination 4. Don't Skip Human Evaluation I've seen cases where ROUGE was 0.45 (acceptable) and the summary was completely wrong because it used the right vocabulary to say the wrong things. Automated metrics are necessary but not sufficient without human supervision. I know it's expensive, and it might slow things down, but don’t skip human evaluation. For faithfulness specifically, checking whether the summary only says things that are actually in the source, we used an LLM-as-judge with this prompt pattern: Markdown “You are evaluating a summary for faithfulness to its source document. SOURCE DOCUMENT (or relevant excerpt): {source_text} GENERATED SUMMARY: {generated_summary} For each factual claim in the summary: 1. State the claim 2. Is it SUPPORTED, PARTIALLY SUPPORTED, or NOT SUPPORTED by the source? 3. If supported, quote the relevant passage. Then give an overall faithfulness score (0.0 - 1.0): - 1.0 = every claim is directly supported - 0.5 = some claims lack source backing - 0.0 = the summary fabricates or contradicts the source Output your assessment as structured JSON.” This caught things ROUGE never could. Cases where the summary stated the penalty was "$5M" when the regulation actually said "$10M." The words overlapped; the meaning was wrong. Sample 20–30 summaries. Have domain experts rate completeness and accuracy on a 1–5 scale. This is your production readiness gate. Common Mistakes to Avoid Testing only on short documents: Our development set was mostly 10–20 page documents. Everything looked great. The length problem was invisible until we deliberately tested on 100+ page documents.Trusting Precision alone: When we found "Precision: 0.84", I thought we were in good shape. I should have immediately asked, "and what's the Recall?" High Precision with unknown Recall means nothing.No baseline comparison: We didn't know if our pipeline was better than simply taking the first 3 paragraphs. Turns out, for short documents, "first 3 paragraphs" was a surprisingly strong baseline.Evaluating once, not continuously: The first evaluation revealed the length problem. We fixed it, but we didn't set up ongoing monitoring. Six months later, new document types crept in that the pipeline struggled with, and nobody noticed until a stakeholder complained. Key Takeaways Measure Recall independently. If your Precision is high but you haven't checked Recall, you might have a confident liar. Test on long documents: Your pipeline's real weakness only appears at 5x your median document length. If you're not testing there, you don't know your system.The length/recall correlation is real: r = -0.71 in our case. If you're using any fixed-size processing approach, expect the same pattern.Fix coverage first: If Precision is already 0.80+, your problem is almost certainly Recall. Hierarchical summarization is the fastest fix.Automated metrics get you 80%. The last 20% requires human eyes. Evaluate continuously: A continuous automated evaluation pipeline ensures the system remains reliable over time. The goal isn't a perfect score, it's knowing exactly where your pipeline fails so you can make informed decisions about what to deploy, and what to hold back. Build the evaluation pipeline before you need it, because by the time a compliance team finds the gap, you've already shipped a confident liar.