Methodologies Resources

DZone's Featured Methodologies Resources

How We Cut Our LLM Summarizer's Cost And Latency While Protecting Quality

We run an LLM pipeline that summarizes support tickets for a large network operations team: it reads long, noisy ticket logs and produces short summaries, customer status notifications, technician ticket summaries, and regional executive rollups. It works. But once it was running steadily, a quieter problem showed up. On an ordinary day, summarizing every ticket with a top-tier model and a full log was more expensive and slower than it needed to be. Four levers — incremental summarization, trimming, right-sizing, and parallel execution — are compounding to cut blended LLM cost by roughly 60%. Keeping the pipeline alive during traffic spikes is a separate story. This one is about the bill and latency on a normal Tuesday, and the four changes that, so far, are cutting our LLM cost by roughly 60% and making summaries faster while protecting quality. Where the Cost and the Milliseconds Actually Go Before optimizing anything, it helps to look at the anatomy of a single summarization call. It costs you two things — tokens and time — and both come from the same handful of places: Input tokens: The ticket log, plus the context we retrieve from our vector store, plus the prompt. For a summarizer, this is the giant. Logs are long; the summary is a few sentences.Output tokens: The summary itself is small, but output tokens are the most expensive per token, and generation time scales with how many you produce.The model: Price per token varies enormously between a flagship model and a smaller one.Round-trips: Doing calls one at a time serializes latency that doesn't need to be serial. That anatomy hands you four levers, roughly in order of leverage: stop re-reading what you already summarized (incremental summarization), trim what's left (dedupe and truncate), use a cheaper model where quality holds (right-size), and stop waiting on calls one at a time (concurrency). Lever 1: Incremental Summarization — Stop Rereading the Whole Log A ticket log is not static. A busy incident ticket accumulates thousands of log lines over its life, and every status update needs a fresh summary. The naive approach re-sends the entire log to the model on every regeneration, so the same thousands of lines are paid for again and again, and the token bill grows with the ticket. Instead, we cache each ticket's latest summary in Redis and treat it as running state. When the ticket updates, we send the model the cached prior summary plus only the new log lines since we last ran, and ask it to fold that delta into an updated summary. The model reads a short summary and a small delta rather than the full history — and because logs can run to thousands of lines, this was where the biggest token savings came from. The guardrails: we periodically force a full re-summarization so small errors don't compound across many incremental updates, and the cached summary is always refreshed against new log lines rather than served stale. This is the classic rolling summarization pattern — cheap to run, and it turns an ever-growing log into a bounded, near-constant cost per update. Incremental summarization: instead of re-sending the entire growing log on every update, we send the cached summary plus only the new lines - so cost stays flat as the ticket grows. Lever 2: Token and Context Trimming — Make Each Call Smaller Incremental summarization stops us re-sending the whole history; trimming shrinks what's left on the calls we do make. Because input tokens dominate a summarizer's cost, we squeezed the rest: Deduplicate the log: Machine-generated logs repeat the same alarm and heartbeat lines over and over. Collapsing those duplicates removed large redundant blocks before they ever hit the model.Truncate to what matters: Instead of shipping a ticket's entire history, we cap the window to the most recent and most relevant events.Cap the retrieved context: More retrieved chunks from the vector store is not automatically better - each one is pure input cost. We limited how many we pull.Cap max output tokens: Summaries are short by design, so a hard output ceiling prevents runaway generations and, because generation time tracks output length, directly trims latency. Together, these cut a large share of the remaining input on every call. Fewer tokens in and out is the rare optimization that reduces cost and latency at the same time. Lever 3: Model Right-Sizing — Pay for Quality Only Where It Matters Not every ticket deserves the flagship model. We select the model per ticket from business signals — severity, number of customers impacted, and revenue impact — so premium-model tokens go to the tickets that actually warrant them (major outages, high-value customers) while routine tickets get a smaller, cheaper, and faster model. The lowest-severity tickets skip the LLM altogether. A non-LLM extractive summary (key fields, error counts, first and last events) costs zero tokens. It's the same severity tiering we use to keep the pipeline resilient — here it doubles as the cheapest tier of all. The guardrail is a quality bar: before trusting a cheaper tier for a class of tickets, we validate its summaries against the flagship's, and when the signal is ambiguous, we route up, not down. Right-sizing is a double win: smaller models cost less and respond faster, so it reduces both spend and typical response time. Lever 4: Parallel Execution — Stop Waiting in Line Summaries are independent of one another, so processing them one at a time was leaving latency on the table. Running calls concurrently (async) collapses the wall-clock time to clear a burst of tickets and sharply improves perceived latency when several land at once. The one caveat is that unbounded concurrency will happily blow past your provider's tokens-per-minute limit. So concurrency runs under a shared token budget: as many calls in flight as the budget allows, and no more. You get the throughput without tripping the ceiling. Results No single trick did it; the levers compounded. Together they're bringing our blended LLM cost down by roughly 60% so far, while making summaries faster and steadier under load: leverwhat changed Incremental summarization Full log on every update → prior summary + new lines only Trimming Deduplicated and truncated input, capped output Right-sizing Flagship-for-everything → cheaper model by severity; lowest tier skips the LLM (extractive) Concurrency Serial calls → parallel, under the token budget Blended LLM cost ~60% lower (early results) The biggest surprise wasn't an exotic trick — it was simply not paying to re-read tokens we had already summarized. The other three levers compounded on top. What We Watch For Summary drift: Because each update builds on the previous summary rather than the raw log, a small error can propagate across updates. A periodic full re-summarization from the original log resets any accumulated drift.Quality regressions: Over-truncation can drop the one log line that mattered, and a too-cheap model can miss nuance. We keep a periodic eval/spot-check, so cost cuts don't quietly erode quality.Don't optimize noise: Measure first; a low-volume path rarely repays the complexity. Takeaways Measure where the tokens go – For a summarizer, input dominates - optimize that first.Biggest wins first – Reuse prior summaries instead of re-reading the whole log, trim what you do send, and right-size the model. Exotic tricks come later.Concurrency is a latency lever, not a cost lever – and it must respect your rate limits.Guard quality whenever you trim or downshift – pair every cost cut with an eval. More

Spring Security JWT Validation Best Practices

By Alka Nimje

A few months into a large-scale legacy-to-cloud migration for a Fortune 50 insurance carrier, I found something that had been sitting quietly in our codebase for a while: application code that was manually re-checking JWT claims — signature, issuer, audience, expiry — that Spring Security's OAuth2 Resource Server had already validated at the framework level, before the request ever reached that code. Nobody had done this carelessly. Each addition, on its own, looked like good security hygiene. That's exactly why it survived review for as long as it did. Why This Happens So Easily A JWT is a self-contained, signed structure carrying its own claims. Validating untrusted input by hand is a deeply ingrained instinct for engineers, and for good reason — it usually is the right instinct. The problem is visibility. When a developer writes if (claims.getExpiration().before(now)), a reviewer can point at that line and confirm the check exists. What's much harder to see is the framework machinery quietly doing the exact same job, correctly, one layer up. Spring Security's OAuth2 Resource Server support validates signature, expiry, issuer, and audience (when configured) in line with RFC 7519 and RFC 8725. Its JwtAuthenticationProvider runs these checks on every incoming token, and as the identity provider rotates its signing keys, it automatically re-fetches and re-validates against the new keys — no application code involved. Almost none of this is visible unless you've actually read the framework internals, which is exactly why teams tend to reimplement it instead of trusting it. The audience claim was the clearest example in our case. It identifies which resource server a token is meant for, and the framework exposes it as a single config property. We'd been treating it as something that needed hand-written comparison logic — when really, the value had just never been wired into configuration in the first place. The fix wasn't more code. It was one line of YAML. What We Changed We removed the manual validation entirely and let Spring Security's Resource Server own both authentication and authority mapping: YAML spring: security: oauth2: resourceserver: jwt: issuer-uri: https://login.microsoftonline.com/{tenant-id}/v2.0 audiences: api://{application-id} authorities-claim-name: roles authority-prefix: GRANT_ That block replaced manual signature verification, issuer checking, expiry checking, audience checking, and — on Spring Boot 3.3+ — the hand-written role-to-authority converter too. No JwtAuthenticationConverter, no custom Bean. The framework builds the authority mapping straight from the roles claim and applies whatever prefix you configure. (GRANT_ isn't a framework requirement, by the way — pick anything, as long as it matches what your @PreAuthorize expressions expect downstream.) The Objection We Had to Take Seriously This didn't get waved through. It took a few weeks of real back-and-forth, because the pushback was a legitimate security position, not just resistance to change: don't trust any single layer — verify independently, everywhere. Concretely: if issuer-uri got misconfigured — pointed at a stale tenant, or worse, something attacker-influenced — the Resource Server would faithfully validate every token against the wrong authority, and nothing in a minimal-code setup would catch that. A hardcoded, independent check plausibly would. Nobody disputed that the risk was real. The actual question was which layer should own catching it. Our answer: a hardcoded application-layer check and a misconfigured deployment value are different failure modes, touched by different people, on different release cadences. Duplicating validation in code doesn't actually protect against a bad issuer-uri — it just adds a second place that can also drift out of sync. Instead, we added a CI/CD gate that asserts the deployed issuer-uri and audience resolve to the expected tenant, and fails the pipeline on mismatch, on every deploy. Is that still "a second check"? Sure. But it fails differently, and that difference is the whole argument: a broken pipeline gate is a loud, visible build failure someone has to explicitly override. A stale hardcoded value in application code can sit silently wrong for months, because nothing forces anyone to look at it again. What I Can Honestly Say Held Up I want to be straight about this: since the change shipped, we haven't had a key rotation or a deployment misconfiguration that actually tested either mechanism in anger. So I can't tell you the CI/CD gate has caught a real incident, because it hasn't had to yet. What I can say: the manual validation code is gone, no duplicate signature/issuer/audience/expiry logic remains anywhere in the codebase, and in the months since, nothing attributable to the change has come up in review. That's evidence the change didn't introduce a regression — not proof it's been battle-tested under a live failure. Worth being honest about the difference. The Takeaway Treat hand-rolled token validation as a prompt to investigate, not an automatic red flag. In review, ask what the framework's resource-server config already checks before accepting new validation code as necessary.If a function exists purely to re-derive something the framework already validates, the fix is usually a config line, not a code change.Defense-in-depth concerns about misconfiguration are legitimate — answer them directly. Often the better answer is verifying config at deploy time (CI/CD gating), not re-validating tokens at runtime.The two approaches fail differently, and that's the actual argument for preferring the config-time check: a broken gate is loud, a stale hardcoded check is silent. This example is Spring Security and Entra ID, but the underlying review question — what does the framework already check before I add code that checks it again — travels well to any OAuth2/OIDC resource-server setup. More

Solving Comprehension Debt in AI-Generated Codebases

By Pushkar Gopalakrishna

How to Build Your Own AI SRE From Scratch

By Severin Neumann

To Switch, or Not to Switch the AI Model

By Stefan Wolpers

CORE

Beyond the Wall of Charts: How Advanced Observability Displaced the Telemetry Dashboard

For the past decade, the gold standard of Site Reliability Engineering (SRE) was "all your telemetry in one place, beautifully rendered." Centralized platforms allowed teams to build massive grids of charts tracking CPU usage, memory allocation, request latencies, and error rates. Yet this approach masks a critical flaw: traditional dashboards show only symptoms, forcing humans to act as data archaeologists to find the root cause. With microservices handling highly distributed workloads, a simple user request might cross a dozen internal boundaries. When latency spikes, a static dashboard tells you what is broken, but it cannot tell you why. The modern engineering directive has fundamentally shifted from reactive visibility to deep system understanding. The conventional approach of isolating metrics, logs, and traces into discrete dashboard tabs or separate tools has broken down under cloud-native complexity: The Contextual Chasm: When an alert fires, engineers are forced to pivot between systems, manually trying to correlate a metric spike in one graph with a raw text error in a log file. This manual synchronization creates a high mean time to resolution (MTTR).Pre-Aggregation Destruction: Traditional monitoring relies on pre-aggregated data averages to control cloud storage costs. This destroys fine-grained details, making it impractical to debug highly specific "unknown-unknown" incidents affecting rare user cohorts.Anticipation Bias: Dashboards assume that a software engineering team can accurately predict every failure mode ahead of time to build a chart for it. Complex systems fail in entirely unpredictable ways that static, predefined views cannot capture. The Observability Revolution: Core Paradigms Replacing the Dashboard Rather than forcing engineers to stare at screens waiting for a line to turn red, modern observability treats system telemetry as a unified, exploratory analytics problem: High-Cardinality Unified Data Layers: Modern architectures store unaggregated, raw event streams. By preserving trace IDs, customer IDs, and container headers, engines can instantly pinpoint the precise thread or database index responsible for a system failure.Contextual Topology and Causality: Advanced systems continuously map software dependencies and traffic patterns. Instead of sending a dozen isolated alert notifications, the system assembles an incident narrative, proving how a configuration drift upstream actively broke a payment processing gate downstream.Agentic AI Remediation: The industry is moving from monitoring toward autonomous operations. Specialized AI agents consume telemetry streams directly to execute preventative actions like scaling production clusters or rolling back buggy code deployments. The traditional dashboard view: application health, load, response times, and error rates show that something is wrong, but not why. The engineer still has to hunt for the cause. The high-cardinality view: instead of scanning charts, the failing dimension is isolated directly — here, errors tied to a specific service version (v350.10) and HTTP 401 status — pointing straight at the root cause. Adapting to the Age of Non-Deterministic Infrastructure The massive expansion of large language model (LLM) agents and generative software layers has further accelerated the demise of legacy dashboards. Traditional systems check whether an API endpoint returns a 200 OK HTTP status code. An AI-driven microservice can operate perfectly within technical boundaries, response times, and system limits, while still delivering entirely hallucinated or factually incorrect output to an enterprise client. Observability has adapted by monitoring semantic tokens, prompt guardrails, and vector pipeline drift, complex factors that cannot be displayed on a standard bar chart. The metric of success for a modern technology infrastructure is no longer the number of operational dashboards under management. Success is defined by autonomous, rapid understanding and instantaneous path-to-resolution, which is why the static visual graph has been displaced as the primary debugging surface. The Strategic Dilemma Transitioning from static, pre-aggregated dashboards to high-cardinality observability (tracking raw, un-aggregated event data across highly granular dimensions like user_id, container_id, and ip_address) is essential for modern debugging and reducing MTTR. However, storing raw, multi-dimensional event streams exponentially inflates cloud infrastructure and data ingest costs. Without strict architectural guardrails, observability spend can grow to consume an unsustainable percentage of total cloud budgets. The True Drivers of High-Cardinality Cost Exploding Volume of Unique Dimensions: Cloud-native elements (ephemeral containers, serverless functions) spin up and down constantly, generating billions of unique metric combinations that saturate traditional relational structures.Data Ingestion Overhead: Commercial observability vendors charge based on data volume ingested and index size, and self-hosted open-source clusters (e.g., OpenSearch, or Prometheus with Cortex/Thanos) carry the same cost pressure through the infrastructure required to store and index that volume.Hot Storage Reliance: Keeping raw, unaggregated logs and traces instantly queryable requires expensive, high-throughput SSD cloud storage, driving up baseline cloud infrastructure costs. Architectural Framework for Cost Mitigation To balance the technical necessity of high-cardinality data with strict fiscal control, tech leaders should mandate a tiered storage approach: Implement Intelligent Data Sampling Not all telemetry is created equal. SRE teams should configure tail-based sampling rules. For instance, store 100% of traces that result in an HTTP 5xx error or high latency, but sample only 1% to 5% of healthy, predictable HTTP 200 OK traces. Adopt Tiered Storage Horizons Hot Tier (0–7 Days): Keep full, high-cardinality raw telemetry in expensive, high-performance storage for immediate active incident response.Warm Tier (8–30 Days): Move data to aggregated metrics blocks or lower-cost search-optimized tiers for historical trend analysis.Cold Tier (31+ Days): Compress and offload raw data into highly economical object storage (such as Amazon S3, Google Cloud Storage, or Azure Blob) strictly for long-term compliance and audit requirements. Tiered storage keeps high-cardinality observability affordable: recent data stays hot for fast incident response, then ages into progressively cheaper, slower storage for trend analysis and compliance. Enforce Strict Lifecycle Policies. Automate the purging or down-sampling of high-cardinality dimensions as data ages. A specific container ID from three weeks ago holds little operational value for daily SRE workflows. Verdict Do not scale back observability goals to save on infrastructure bills. Instead, optimize how and where that telemetry lives. By implementing data sampling at ingestion and automated lifecycle workflows, organizations can unlock the full power of high-cardinality root-cause analysis while keeping data storage costs linear and predictable.

By Prashant Maria

Why Your GPU Fleet Is Both Full and Idle: Building a Capacity Orchestration Layer

Most machine learning (ML) platforms still hand out accelerators through static quotas, spreadsheet-driven approvals, and hallway negotiation. That model was tolerable when capacity was cheap, and demand was steady. Neither holds for modern AI workloads. This article walks through why static allocation breaks down, the five components of a capacity orchestration layer (intent capture, policy, hardware brokering, continuous reallocation, and goodput accounting), and a 90-day rollout plan that won't destroy your users' trust in the scheduler. The Contradiction You Have Probably Already Measured If you run infrastructure for a large ML organization, you have likely seen a dashboard that says something uncomfortable. The fleet is 90-plus percent allocated. Teams are filing tickets about blocked capacity. And a sampled profile of the same fleet shows a meaningful fraction of devices doing no useful work. Buying more accelerators does not resolve this. It scales the inefficiency. The contradiction comes from a specific architectural choice: treating an accelerator reservation as ownership rather than as a lease with conditions. The industry has spent years optimizing acquisition: supply agreements, multi-year commitments, reserved cloud capacity. Comparatively little engineering has gone into what happens after the hardware lands in the fleet. Why Static Quotas Fail for ML Specifically Conventional service capacity planning works because service demand is autocorrelated. Today's traffic predicts tomorrow's. ML workload demand does not behave this way, for four reasons. Demand is event-driven. An eval result at 2 p.m. can turn a 16-device research project into a 512-device training run by the next morning. No quarterly quota cycle accommodates that. Jobs differ in shape, not just size. A gang-scheduled training job with all-reduce dependencies across 64 devices and a stateless inference replica are both "GPU requests" to a naive scheduler, but they have incompatible placement, topology, and preemption semantics. A fleet fragmented without topology awareness ends up with allocatable devices that no large job can actually use. Reservation and consumption diverge. A job holding 32 devices while blocked on an input pipeline is occupying capacity and producing nothing. Device-level utilization metrics will not reliably catch this, because a device spinning on a synchronization barrier can still report activity. Returning capacity is asymmetrically risky. This one dominates. If reacquiring capacity takes two weeks of negotiation, the rational move for every team is to hold everything they might need at peak, indefinitely. Static allocation rewards hoarding. Each over-request is a locally correct decision inside a badly designed system, which is why exhortations to "be good citizens" never fix it. The Architecture A capacity orchestration layer sits between your workload submission API and the underlying scheduler, whether that is Kubernetes with Kueue or Volcano, Slurm, Ray, or an internal system. It does not replace the scheduler. It feeds the scheduler better information and revisits its decisions after admission. 1. Intent Capture: A Device Count Is an Answer, Not a Requirement When a user requests "32 A100s," they have already collapsed their actual requirement into one number. The scheduler needs the requirement itself: what the workload can substitute, what it can tolerate, and what it cannot give up. In practice that looks like a spec along these lines: YAML apiVersion: capacity.internal/v1 kind: WorkloadIntent metadata: name: ranking-hparam-sweep spec: class: research-batch # drives policy, not placement preferred: accelerator: a100-80gb count: 32 substitutable: - accelerator: h100-80gb count: 20 # based on measured step-time on this model - accelerator: tpu-v5e count: 64 requiresRecompile: true # XLA path validated in CI topology: gangScheduled: true minInterconnect: nvlink-domain elasticity: minReplicas: 8 maxReplicas: 32 resizeCost: 90s # checkpoint + reshard preemption: tolerated: true gracePeriod: 120s checkpointInterval: 600s deadline: "2026-08-14T00:00:00Z" Two fields carry most of the value. substitutable is what makes cross-architecture brokering possible. preemption.tolerated, paired with a real checkpointInterval, is what makes a job safely reclaimable. Interruption tolerance should be verified, not self-declared. Run a preemption drill in CI: kill the job, confirm it resumes from checkpoint within the stated grace period. A self-reported tolerance that has never been exercised will eventually be exercised in production, on a bad day. Be honest about the engineering cost. Portable workload definitions, container images built for multiple backends, and checkpoint formats that survive a device-count change are all real work. Elastic resharding in particular is hard for models with complex parallelism strategies. Treat the full intent spec as something teams grow into. A job that declares only preferred still schedules; it just gets fewer options. 2. Policy Engine: Encode the Argument Once The point of a policy engine is not to remove human judgment. It moves judgment out of per-request negotiation and into rules that can be reviewed, versioned, and revised. On Kubernetes, Kueue's cohort model already expresses the core mechanics through nominal quota, borrowing, and lending: YAML apiVersion: kueue.x-k8s.io/v1beta1 kind: ClusterQueue metadata: name: research-shared spec: cohort: fleet-accelerators preemption: reclaimWithinCohort: Any withinClusterQueue: LowerPriority resourceGroups: - coveredResources: ["nvidia.com/gpu"] flavors: - name: h100-80gb resources: - name: "nvidia.com/gpu" nominalQuota: 256 borrowingLimit: 512 # may burst into idle cohort capacity lendingLimit: 192 # keeps a protected floor of 64 devices Read those preemption lines together with the limits, because the pair is what changes team behavior. lendingLimit guarantees a floor that is never lent out, and reclaimWithinCohort: Any guarantees that lent capacity comes back the moment the owner needs it, by preempting borrowers if necessary. Once returning idle capacity carries no risk of losing it, the economic case for hoarding mostly evaporates. Beyond quota mechanics, useful policy inputs include production criticality, launch dates, queue age, the historical efficiency of the requesting workload, and per-organization fairness floors. Dominant Resource Fairness (Ghodsi et al., NSDI 2011) is a sensible default for multi-resource sharing, with queue aging layered on top so low-priority work eventually runs instead of starving. One requirement is non-negotiable: explainability. Every scheduling decision needs a retrievable answer to "why is my job still queued?" naming the binding policy, the preempting workload, and a projected start time. Without that, engineers conclude the system is arbitrary and route around it. An orchestration layer people distrust is worse than static quotas, because now you have a scheduler plus a shadow economy of private allocations. 3. Hardware Broker: One Portfolio, Not Several Fiefdoms Many organizations run accelerator families as separate silos, each with its own team, toolchain, and queue. The fragmentation hides substitution opportunities and strands capacity on whichever side happens to be idle. A broker matches substitutable declarations against live supply and makes the tradeoff explicit: 32 A100s in four hours, versus 20 H100s now, versus 64 TPU v5e chips now plus a 12-minute recompile. Comparing those options requires measured throughput per accelerator class per model family, which in turn means maintaining a benchmark corpus and refreshing it as software evolves. Compiler-level changes alone can shift the relative ranking of hardware classes over time; our work on compiler autotuning for TPU fleets (CATWILD, MLSys 2026) found that tuning decisions which were optimal at one point degrade as workloads and toolchains drift, and the same staleness problem applies to any performance model a broker relies on. A stale model will confidently route work to the wrong hardware. Scope this realistically. Workloads with hand-tuned CUDA kernels, custom collectives, or architecture-specific memory layouts are not portable, and forcing portability on them wastes effort. The goal is narrower: preserve flexibility where it is technically sound, and make sure the jobs that genuinely are portable are visible to the scheduler as such. 4. Continuous Reallocation: Placement Does Not End at Admission Traditional schedulers decide placement once, at admission, and then stop thinking. AI infrastructure needs a control loop that keeps asking whether each device is still attached to the highest-value feasible work. Conditions the loop should detect and act on: Idle reservations. Allocated devices with sustained near-zero productive compute past a grace threshold get reclaimed.Over-provisioned elastic jobs. When scaling efficiency degrades past a threshold, the marginal device is buying almost nothing. Resize down and return the difference.Blocked jobs. A job stalled on data loading, or an external dependency for longer than its checkpoint interval, gets checkpointed and requeued rather than left squatting.Priority inversions. A launch-critical workload arriving after lower-priority jobs have claimed capacity triggers graceful preemption, per declared policy.Substitution openings. A portable queued job starts now on a different accelerator class instead of waiting for its preferred one. The safety constraints matter as much as the actions. Rate-limit preemptions per team per hour. Never preempt a job whose checkpoint drill has not passed. Cap how many times a single job can be preempted before it gets promoted, or some unlucky workload will never finish. And write every reallocation event into the job's own log with a reason attached. Unexplained preemption is the fastest way to lose organizational support for the whole system. 5. Goodput Accounting: Allocated Is Not Productive You cannot manage what your metrics obscure, and device-level utilization obscures a lot. A layered view separates the questions: LayerQuestionRough definitionAllocation rateIs the fleet handed out?Allocated device-seconds / total device-secondsOccupancyAre allocated devices active?Active device-seconds / allocated device-secondsGoodputIs the activity producing committed progress?Device-seconds contributing to non-discarded work / allocated device-secondsMFUHow close is the work to the hardware roofline?Achieved model FLOPs / peak FLOPs Goodput is the layer that changes behavior. Computing it means joining hardware telemetry with job-level progress signals such as steps committed, checkpoints written, and requests served, so that restart replay, discarded partial work, and stalled collectives count as loss rather than as utilization. A training job that crashes and replays six hours from a stale checkpoint looks fine by occupancy and terrible by goodput, and goodput is telling the truth. Model FLOPs utilization, introduced in the PaLM paper (Chowdhery et al., 2022), is worth tracking but answers a different question. MFU is a compiler and kernel efficiency signal, not an allocation signal. A job can be perfectly scheduled and still leave performance on the floor; that is a different team's bug. The Fairness Constraint A system that optimizes purely for near-term commercial value fails predictably. Exploratory research never runs, because its value is uncertain by definition, and small teams lose every contest against large ones. The safeguards are concrete: a protected research pool that revenue workloads cannot preempt, queue aging that raises effective priority over time, per-team caps within each cohort, and reservation expiry so held capacity gets re-justified periodically. All of these cost measured efficiency, and that cost should be budgeted deliberately. The alternative is an infrastructure layer that quietly defunds the products you have not built yet. A 90-Day Rollout Days 0 to 15: instrument before changing anything. Inventory accelerator types, ownership boundaries, queue mechanisms, and approval paths. Separate reservation time from productive execution time in telemetry, and publish the gap. If you cannot produce a defensible goodput number, everything downstream is guesswork. Expect the first honest measurement to be politically uncomfortable. Days 16 to 45: write the allocation contract. Define priority classes, fairness floors, preemption rules, reservation expiry, and substitution policies, then circulate them with engineering, infrastructure, and product leadership before automating anything. Decide which workloads carry hard guarantees and which are explicitly flexible. This phase is mostly negotiation rather than implementation, and skipping it is the most common way these projects die. Days 46 to 90: automate one bounded pool. Choose a cohort with a contained blast radius. Batch research capacity is usually right, since it tolerates interruption. Enable reclamation, resizing, and substitution there only, and track queue time, end-to-end completion time, goodput, failed migrations, and policy exceptions. Then hold. Do not expand scope until teams trust both the decisions and the explanations behind them. Trust, not scheduler throughput, is the binding constraint on adoption. Closing Accelerator demand is not going to flatten. Models grow, inference spreads into more product surfaces, and experiment volume rises with headcount. Newer hardware improves per-device throughput while doing nothing about contention; if anything, it raises the stakes, since a stranded H100 is a more expensive mistake than the stranded V100 it replaced. Organizations that scale well will stop treating accelerators as assets assigned to teams and start managing them as a continuously rebalanced portfolio governed by explicit, explainable policy. Procurement still matters. Orchestration determines how much of what you procured turns into shipped product. The most valuable accelerator in your fleet is not the next one you buy. It is the one you already own, currently attached to the wrong job.

By Ankit Sinha

AI on Top of a Dysfunctional System

TL;DR: Polished Artifacts, Unchanged Decisions, Or Ten Backlog Anti-Patterns AI Makes Worse Add AI to a Product Backlog process that already struggles, and everything seems to improve within an afternoon. The problem is that polishing artifacts with AI doesn’t fix the root cause: the basis for the team’s decisions doesn’t change; AI only removes the visible discomfort that used to signal something was broken, along with some of the pressure to fix it. AI applied to a dysfunctional system – here, the Product Backlog process – makes the dysfunction look like progress. This is the first article of a new series that explains the uselessness of bolting AI onto a dysfunctional system. It explains why generative AI worsens ten Product Backlog anti-patterns, how polished AI artifacts can hide missing evidence and authority, and how teams can check whether their process is fit for AI. Thesis: Adding AI onto a dysfunctional system, for example, the Product Backlog, makes ten typical backlog anti-patterns worse, because polished AI artifacts hide missing customer evidence, decision authority, and feedback; the article explains these mechanisms, their costs, and how teams test whether their process is ready for AI. Disclaimer: I read Charniak/McDermott’s book on “Artificial Intelligence” decades ago; of course, I use AI for research, translations, proofreading, challenging story arcs and article structures, and summarization. It is a production tool, not a substitute for thinking. A Scenario You May Recognize Consider the following scenario: A Product Owner under pressure feeds stakeholders' requirement documents into an LLM, adds context such as the product roadmap or access to Jira. Soon, the Product Backlog holds 40 new items, each with a user story, five acceptance criteria, and an "estimated" size. Next Monday, Sprint Planning runs smoothly, and the stakeholders are delighted: finally, the team seems "ready." Now ask the four questions nobody asked: Which customer problem does this work address?Whose evidence says it matters?Which alternative did the team consider and reject?And who could have stopped it? If the answers are "whatever the stakeholders requested," "none," "none," and "nobody," the process didn't improve; it just stopped showing its flaws. (You may recall that Agile is really good at exposing problems, issues, and flaws.) What Must Work Before You Accelerate the Process Product Backlog management runs in a loop. What the team knows and assumes informs its product choices, and refinement exposes the uncertainty in those choices. Experiments, research, and delivered Increments then put those choices in front of customers and produce new evidence, which changes the next choices. That is why the Product Backlog is emergent, and every item comes with an implicit "based on what we know now." Work may legitimately enter the Product Backlog to investigate an open question; what matters is that people distinguish what they know from what they still need to learn and pick a suitable next step. If a team fails to feed valuable work into this loop, however, Scrum will be just as effective at building things no one needs: garbage in, garbage out. (Keep in mind that Scrum can be an effective delivery framework, but it completely lacks the product discovery part.) Refinement is a continuous activity within that loop, and it produces a shared understanding sufficient to make the next investment decision, including the decision not to build something at all. Selecting work for a Sprint happens later, in Sprint Planning, once the team has decided on a Sprint Goal: what is most valuable to create? A dysfunctional Product Backlog process usually announces itself: thin items, awkward Sprint Planning sessions, arguments about scope in the middle of a Sprint. That discomfort is useful, as it is the evidence a Scrum Master brings to the Retrospective and a Product Owner brings to stakeholders. AI can remove it without removing its cause. The Scrum Guide 2020 warning then applies literally: "Transparency enables inspection. Inspection without transparency is misleading and wasteful." Google's announcement of the 2025 DORA report, based on survey responses from nearly 5,000 technology professionals, describes the general pattern: "AI doesn't fix a team; it amplifies what's already there." The same announcement reports positive associations between AI adoption and both delivery throughput and product performance, alongside a continued negative association with delivery stability (Announcing the 2025 DORA Report). The findings cited do not establish the ten mechanisms described below; applying the amplification argument to them is my interpretation. AI can help repair a dysfunctional process if you deliberately use it to expose gaps: list the assumptions behind an item, find contradictions between a stakeholder request and the Product Goal (or the equivalent planning objective), or organize customer evidence the team already has. However, that only works while everyone stays aware of what the tool is doing. Polished artifacts fool people quickly into believing that AI is doing the real job, and teams need to be very careful not to fall into that trap. The useful question is whether an AI output makes evidence, assumptions, and alternatives easier to examine, or easier to skip. The dangerous intervention is the second: generating more apparently implementation-ready work before addressing the gaps. (AI, given the right context, is really good at papering over flaws and polishing its output.) What "Functional" Means Compared to a Dysfunctional System A functional Product Backlog process does not need to be flawless, fully documented, or unchanging. It needs to detect and correct poor decisions, which depends on three things: Direction: A customer problem or Product Goal makes a piece of work worth considering in the first place.Authority: Someone can challenge, change, or reject proposed work, and the organization respects that decision.Feedback: When delivery shows an assumption was wrong, later decisions change. The ten anti-patterns resulting from putting AI on top of a dysfunctional system below fail on one or more of these, and they fall into three groups: AI on Top of a Dysfunctional System, Group 1: Choosing Work Without Evidence 1. Prioritization by Proxy Stakeholders decide what goes into the Product Backlog, and the Product Owner passes their decisions on. With an LLM, a stakeholder now arrives with a ten-page requirements document, including personas and success metrics, produced in an afternoon. The polish does not create the power imbalance; it makes the existing one harder to confront, because rejecting a document that looks finished feels like obstruction and puts your career at risk. A better refinement session will not restore product management here, since the problem lies in who holds decision authority. The team becomes an internal development agency, only faster, and the question of whether something else would serve customers better never gets asked, because the polished document seems to have answered it already. 2. The Oracle Product Owner The Product Owner involves neither stakeholders nor subject matter experts, and the Product Backlog contains no research tasks, such as building prototypes or spikes. The AI version consults a model instead: synthetic personas and a generated market analysis. Source-backed desk research is legitimate work; the failure occurs when generated hypotheses acquire the status of observed customer demand. Product discovery gets skipped while looking complete, and the team then builds the wrong thing with considerable confidence, which may well be the most expensive way of building it, even if agentic coding makes building simpler. Without relevant customer evidence, generated personas and market narratives remain hypotheses, no matter how convincing they sound. 3. The Copy and Paste Product Owner The Product Owner breaks stakeholder requirement documents into smaller chunks. A model does this in seconds and returns neatly sliced items. Smaller items, however, do not make the underlying solution appropriate, nor does decomposing a document turn it into incremental learning. The Product Backlog commits prematurely to one solution, and nobody on the team has to understand the problem well enough to propose a cheaper or better-fitting one. The Product Backlog thereby fills with the stakeholder's solution, cut into Sprint-sized portions, and the later slices quietly turn into assumed commitments, which discourages the team from learning anything from the first slice before building the next one. 4. 100% in Advance The organization asks the team to rebuild a legacy application one-for-one, so the team prepares the complete Product Backlog upfront. With AI, a model reads the old codebase and produces an apparently complete specification with hundreds of items. The danger lies in treating that extracted description of the existing system as the approved specification for its replacement, including every workaround users have complained about for years. A rebuild is a rare opportunity to improve processes and usability; treating a generated inventory as fixed scope quietly closes that opportunity. I worked with a Scrum Team on such a rebuild for a large utility, and the team delivered two weeks early and under budget while streamlining processes and improving usability; all gains came from questioning inherited requirements. An extracted specification can even make existing behavior easier to question. AI on Top of a Dysfunctional System, Group 2: Turning Assumptions Into Apparent Readiness 5. Refinement Without the Team The Product Owner refines with the "lead engineer" and a designer, while the other Developers prefer creating code by herding agents. Asynchronous comments and preparation by a subset of the team are not dysfunctional as such; the problem arises when consequential disagreement never gets resolved. Estimation shows it best. The number was never the point; comparing estimates reveals whether Developers have the same work in mind before they start, as I argued in Estimates Are Useful, Just Ditch the Numbers. Accept an AI-suggested size without independent examination, and no one ever surfaces the split in opinion; the different ideas still exist, but nobody learns about them until the Sprint is underway. 6. Cosmetic Readiness All items look thoroughly detailed and estimated, yet the team has not applied INVEST in substance. AI makes every item look ready: correct template, complete fields, a checklist passed. Proper formatting, however, cannot prove what readiness depends on: credible evidence, understood uncertainty, workable dependencies, and shared understanding. A model can propose alternatives and analyze potential value; it cannot assert customer value or make an organization willing to negotiate. Sprint Planning then selects work that looks transparent but isn't, and the Developers discover missing understanding mid-Sprint, after implementation has started, and correcting it may require rework. 7. Acceptance Criteria Inflation The Product Owner covers every conceivable edge case without negotiating with the Developers. Ask a model for acceptance criteria, and you will typically receive a long list of plausible conditions. My heuristic is that three to five acceptance criteria usually suffice, and needing more often indicates the item needs splitting. The count, as such, is not the problem; the problem is criteria that expand scope without support or add constraints nobody examined, which the team's agents then build without evidence that real users need them. A separate risk arises when criteria start prescribing the solution instead of describing the required behavior: then the Product Owner drifts from the why and the what into the how, which belongs to the Developers, and the Developers end up negotiating with a document instead of a colleague. This is a particular challenge nowadays, when many organizations expect their product managers or product owners to be product builders, vibe-coding the prototype themselves. 8. The Last-Minute Product Backlog The team does not invest in Product Backlog management and rushes to fill the Product Backlog right before Sprint Planning. Before AI, the scramble was visible: thin items, a random assortment of stuff to fill the Sprint and appear busy, consequently resulting in a "Sprint Goal" nobody could explain. Now, an hour of prompting produces items that look as if the team had refined them for weeks. The preparation gap becomes much harder to see, although rework, confusion, and missed goals may still expose it later, by which point the cost has already been incurred. AI on Top of a Dysfunctional System, Group 3: Expanding Work Beyond the Capacity to Inspect and Sustain It 9. The Infinite Product Backlog Oversized Product Backlogs, Product Backlogs used as storage for ideas, and items nobody has touched for months predate AI. The economics change, though: generating candidates becomes nearly free, while assessing, ordering, and maintaining them still consumes the same human attention. If nobody moves anything to a separate list of permanently or temporarily discarded ideas, what I call the "Anti-Product Backlog," the most valuable items get buried in noise, and the ordering stops informing any decision. At that point, the Product Owner administers a database, and the team, facing hundreds of plausible items, stops reading the Product Backlog altogether. (Of course, you can instruct an agent to go through that list and flag all entries the agent believes offer no value. But this is a cleanup operation, not a proper process.) 10. What Technical Debt? The team ships feature after feature, and the Product Backlog reserves no capacity for bugs, refactoring, or platform maintenance. AI-assisted development increases the volume of change, and DORA's findings on delivery stability point to the risk when control systems, such as automated testing and fast feedback loops, are weak. This is, above all, a product decision: if the organization rewards feature output and never negotiates quality expectations or capacity trade-offs, AI will produce more features faster, and customers will experience the result as instability, not to mention that they are unlikely to pay for the flood of new features they never asked for. The Cost of Apparent Efficiency "We created the Product Backlog faster" is an incomplete business case, but typical for bolting AI onto a dysfunctional system. Consider an illustration, not a measured result: saving six hours of preparation has little value if unsupported scope then consumes three Sprints of delivery capacity, plus the review effort for AI output nobody inspected closely, the rework once users react, and the maintenance of features nobody needed. Meanwhile, the more valuable problem waits, and your competitors move ahead. When reporting after an AI rollout focuses on preparation time, the number of refined items, time spent on creating shared understanding, and the duration of Sprint Planning, none of these costs show up. The dashboard looks like a success while the expensive part of the path, building and maintaining the wrong things, sits in a different budget line and appears months later. Therefore, evaluate the whole path, from identifying a customer problem to learning whether the delivered change helped, including the effort it takes to inspect what the model produced. The costs also compound: every unexamined decision that "worked" teaches the organization that examination is optional. You can apply the A3 Delegation System to the problem to get a better understanding of where AI may support the process, where you should avoid using AI, what "good quality of AI outputs" looks like, and how to bring transparency to your team’s use of AI. Conclusion: Can Your Process Reject the Wrong Work? Before you accelerate product work by employing AI on top of a dysfunctional system, establish that your organization can question the work's justification, reject it, and learn whether it helped. Inspect a recent, important Product Backlog item and ask yourself which of the following criteria are true for your team: Grounding work in a problem: The team can name the evidence supporting the item and distinguish it from assumptions.Making choices: Someone can explain why the item took precedence and name a plausible request that was rejected or deferred.Exposing uncertainty: The people involved can name open questions and explain how they affect the next step.Exercising authority: The accountable person can change or stop the work when its justification fails.Learning from results: Feedback from delivered work has changed a later product decision. A first round producing convincing answers is rarely enough, so follow up and ask for recent examples: Which proposed item did the team stop or materially change because its evidence was weak?Which discovery or delivery result changed the next product decision?When did someone last exercise the authority the team claims to have? If nobody can point to such examples, generating more apparently ready work will only make the gap harder to see, although AI may still help you expose it. Key Questions This Article on the Effects of Employing AI on Top of a Dysfunctional System Answers How Does AI Make Product Backlog Anti-Patterns Worse? AI makes a dysfunctional Product Backlog process look functional, a typical first impression of bolting AI onto a dysfunctional system. It produces polished items, acceptance criteria, and estimates within minutes, while the basis for the decisions stays the same: missing customer evidence, unclear decision authority, and no working feedback loop. The visible discomfort that used to expose the problem, such as thin items or chaotic Sprint Planning sessions, disappears, and with it some of the pressure to fix the process. Should a Scrum Team Use AI to Write User Stories and Acceptance Criteria? Only if the team's Product Backlog process can already detect and correct poor decisions. AI generates user stories and long lists of acceptance criteria quickly, yet formatting cannot prove credible evidence, understood uncertainty, or shared understanding. Generated criteria also tend to expand scope without support; as a heuristic, three to five acceptance criteria usually suffice, and needing more often signals that the item needs splitting. How Can a Team Tell Whether Its Product Backlog Process Is Ready for AI? A process is ready when the organization can question the justification of work, reject it, and learn whether it helped. Test it with recent behavior instead of convincing answers: Which proposed item did the team stop or materially change because its evidence was weak? Which delivery result changed the next product decision? And when did someone last exercise the authority to stop work? Can AI Help Fix a Dysfunctional Scrum Process? Yes, when the team deliberately uses AI to expose gaps and stays aware of what the tool is doing. Useful applications include listing the assumptions behind a Product Backlog item, finding contradictions between a stakeholder request and the Product Goal, and organizing customer evidence the team already has. The dangerous use is generating more apparently implementation-ready work before those gaps are addressed. Why Does Creating a Product Backlog Faster With AI Not Save Money? Hours saved in preparation lose their value when unsupported scope consumes weeks of delivery capacity. The full cost includes reviewing AI output, reworking after users react, maintaining features nobody needed, and delaying work on more valuable problems. Reports that focus on preparation time, refined item counts, or Sprint Planning duration miss these costs because they appear months later in different budget lines.

By Stefan Wolpers

CORE

From Wild West to Context-Driven Engineering: How Culture Shapes Software Decisions

Software organizations require rules to ensure predictability, knowledge sharing, and sound technical decisions. However, excessive rules, processes, and standards can obstruct progress if they eclipse desired outcomes. This longstanding tension in software engineering is now more pronounced as AI accelerates code generation, solution exploration, and implementation of change. The main challenge is not the number of rules, but how they fit with context, autonomy, and responsibility. Some organizations have minimal constraints, others tightly control decisions, and a few adapt principles to specific situations. Noticing these models helps engineering leaders assess their organization, understand associated risks, and move toward more effective technical governance. Software Architecture Is Also an Organizational Problem Software architecture is often defined by technical elements such as components, APIs, databases, infrastructure, quality attributes, and architectural styles. While these matter, they exist within a wider context. People design and maintain software, influenced by business constraints, communication structures, incentives, policies, and authority levels. Architecture is not only a technical artifact; it also reflects the environment in which teams make technical decisions. This broader perspective reflects the socio-technical nature of software engineering and is consistent with standards such as ISO/IEC/IEEE 42010, which recognize that architectural concerns encompass organizational, economic, regulatory, and community factors. Conway's Law shows that organizational communication structures often mirror system design. If organizational structure shapes software, then the distribution of authority, management of disagreement, response to failure, and governance of technical decisions likewise influence architecture. Therefore, engineering culture and governance are architectural concerns, not just management issues. Culture Shapes How Engineering Decisions Are Made If architecture is formed by its organizational context, organizational culture also directly influences engineers’ daily decisions. Sociologist Ron Westrum offers a system for understanding how organizations respond to problems and opportunities, particularly in high-risk environments. His model focuses on information flow, including cooperation, safe communication of bad news, responsibility management, failure investigation, and acceptance of new ideas. Westrum identified three main cultural patterns: pathological (power-oriented), bureaucratic (rule-oriented), and generative (performance-oriented) organizations. This distinction matters in software engineering, where architectural decisions depend on information moving across teams, technical boundaries, and leadership levels. Organizations that conceal problems, discourage disagreement, or fragment responsibility make different technical choices than those that discuss risks openly, encourage collaboration, and treat failures as chances to learn. DORA adopted Westrum's model in its research and consistently links generative, high-trust cultures to stronger software delivery and organizational performance. Culture isn't just the environment around engineering work; it shapes how teams interpret technical information, who can challenge decisions, how rules are enforced, and how architecture evolves. PathologicalBureaucraticGenerativePower orientedRule orientedPerformance orientedLow cooperationModest cooperationHigh cooperationMessengers “shot”Messengers neglectedMessengers trainedResponsibilities shirkedNarrow responsibilitiesRisks are sharedBridging discouragedBridging toleratedBridging encouragedFailure leads to scapegoatingFailure leads to justiceFailure leads to inquiryNovelty crushedNovelty leads to problemsNovelty implemented Mode 1: Wild West Engineering Wild West Engineering refers to environments with weak governance, unclear ownership, and few shared engineering principles. Teams may appear highly autonomous, but such autonomy lacks architectural direction, reliable feedback, and consistent accountability. Technical decisions are frequently reactive and localized, with tools and approaches selected for immediate needs or personal preference rather than the wider system context. The problem is not simply a lack of rules. Experienced teams can operate effectively with minimal rules if they share strong principles and understand the impact of their decisions. Wild West Engineering occurs when autonomy exists without sufficient shared context, accountability, or coordination. Over time, architectural knowledge becomes siloed, similar problems are solved inconsistently, and changes become riskier as no one fully understands the system. Westrum’s pathological culture is a relevant comparison: cooperation is limited, responsibilities are avoided, bad news is suppressed, and failures result in blame rather than inquiry. In software organizations, this often creates a hero culture where a few engineers become indispensable for managing disorder. AI can worsen this problem; when teams generate code, add dependencies, or make architectural decisions without shared guardrails, inconsistency and technical debt can spread quickly. Indicators Typical signs involve unclear ownership, multiple solutions for similar problems, undocumented architectural decisions, reliance on tribal knowledge, fear of changing legacy components, recurring blame after incidents, and dependence on a few key engineers. Another important sign is resignation: engineers stop proposing improvements because changing the system seems riskier than leaving existing problems unresolved. How to Move Forward Imposing heavy bureaucracy is not the solution to chaos. The first step is to establish minimum shared guardrails: clarify ownership, document key architectural decisions, define core engineering principles, set baseline expectations for security and operability, and create safer ways to discuss failures and technical debt. The goal is to support autonomy with enough structure to ensure local decisions conform to a coherent system. Mode 2: Bureaucratic Engineering Bureaucratic Engineering represents the opposite extreme, where governance is pervasive. Standards, approval processes, architectural reviews, mandatory technologies, and detailed procedures dictate how teams operate. While this structure creates predictability and consistency, problems arise when adherence to process outweighs knowing its purpose. Teams may focus on compliance rather than evaluating whether rules remain relevant. Westrum’s bureaucratic culture reflects similar traits: limited cooperation, narrow responsibilities, minimal cross-boundary collaboration, and a tendency to view novelty as problematic. In engineering, this occurs when architectural decisions are centralized, outdated standards persist, and exceptions are hard to secure. While these mechanisms may guarantee a baseline of quality, they may also limit performance. The core issue is not the presence of rules, but the erosion of judgment. Governance becomes rigid when organizations treat principles as permanent mandates and compliance as a substitute for engineering quality. AI can increase this tension, as organizations may respond to new risks with broad restrictions, approved-tool lists, or complex authorization processes. Although these measures may reduce risk, they can also hinder teams from pursuing valuable opportunities, making the organization safer but less nimble and flexible. Indicators Typical signs include rules missing explicit justification, standards defended only by “we have always done it this way,” centralized approval for minor technical decisions, difficulty obtaining exceptions, and architectural decisions that ignore local context. Status quo bias helps explain this persistence. Samuelson and Zeckhauser found that people often prefer existing or default options. In engineering organizations, this reinforces architectural inertia: once a technology, methodology, or rule becomes standard, replacing it requires far more justification than maintaining it. As a result, rules persist not because they are optimal, but because they are easier to retain. Organizational silence is another key warning sign. Morrison and Milliken define this as a collective tendency to withhold concerns when employees believe speaking up is unwise or ineffective. In engineering, this occurs when developers privately disagree with decisions but remain silent in meetings, believing that contesting established processes will not lead to change. Persistent silence should not be mistaken for agreement; it may signal that the organization discourages dissent. Another warning sign is when success is measured mainly by process compliance rather than software outcomes. Teams may meet all requirements yet become slower, less innovative, and disconnected from the initial intent of these controls. How to Move Forward To move beyond Bureaucratic Engineering, organizations should redefine governance rather than eliminate it. Leaders must frequently review the purpose of rules, distinguish primary constraints from preferences, and replace unnecessary approvals with clear principles. Teams need defined boundaries, authority to make decisions within them, and a clear process for exceptions when justified by context. A practical test is to ask whether a rule is still justified by its intended outcome. If no one can explain the risk it addresses, the value it protects, or evidence of its need, the organization may be maintaining the process for its own sake. The aim is to shift from prescriptive control to outcomes-based control, with accountability and feedback. Mode 3: Context-Driven Engineering Context-Driven Engineering integrates governance with local judgment. Rules are tools for managing risk and ensuring consistency, not ends in themselves. Teams work within defined principles, constraints, and responsibilities, while retaining the autonomy to adapt decisions to their particular technical and commercial context. The core assumption is that the same rule may yield different outcomes in different environments. This approach corresponds with Westrum’s generative culture, defined by high cooperation, shared risk, cross-boundary collaboration, inquiry following failures, and openness to innovation. In engineering, teams are expected to follow standards, understand their purpose, and recognize when exceptions are warranted. Architectural decisions are treated as explicit trade-offs, not automatic pattern applications. Teams evaluate technologies such as microservices, hexagonal architecture, event-driven systems, or specific cloud platforms based on the problem, constraints, and desired outcomes, rather than assuming they are universally correct. Applying a rule in the wrong context can lead to poor decisions. The same applies to architectural patterns: they address recurring problems in specific contexts, and applying them without considering context can create anti-patterns. For example, a lifeboat is essential in the ocean but irrelevant in a desert. Similarly, providing water is life-saving for someone dehydrated, but ineffective for someone drowning. A tool or rule's effectiveness depends on the context in which it is used. This environment requires greater maturity, not reduced governance. Autonomy must be balanced with accountability, feedback, and a readiness to revisit decisions as new information arises. AI fits well with this model, as its use can be governed based on context and risk: low-risk tasks may permit broad experimentation, while sensitive data, production changes, or high-impact decisions demand stricter controls. The objective is responsible judgment within clear boundaries, not unrestricted freedom. Indicators Key indicators include clear engineering principles, documented decision rationales, explicit ownership, and constructive disagreement. Teams should be able to explain both the rule and its rationale. Exceptions are allowed but must be justified. Failures prompt learning rather than blame, and architecture teams periodically review standards instead of letting them become outdated. Another strong indicator is that teams can contest established practices without equating disagreement with disloyalty or process violations. A context-driven organization distinguishes between non-negotiable constraints and situation-dependent choices. Security, legal requirements, data protection, and regulatory obligations remain strict, while implementation details such as architectural style, framework selection, or deployment strategy are evaluated locally. This distinction ensures autonomy does not devolve into unstructured or uncontrolled practices. How to Sustain This Model The main risk is regression. Insufficient discipline can lead to disorder, whereas excessive concern for consistency can create bureaucracy. Upkeeping this model requires ongoing review of principles, active feedback loops, documented architectural decisions, clear exception processes, and leaders willing to reexamine their assumptions. A practical test is whether the organization can answer four questions: What problem are we solving? Which constraints are truly non-negotiable? What trade-off are we accepting? What evidence would prompt us to revisit the decision? When teams can answer these questions consistently, governance functions as a learning system rather than a control mechanism. Conclusion Three Engineering Governance models inspired by Ron Westrum Engineering organizations improve not by simply adding or removing rules, but by ensuring teams understand their purpose, have the required context, and are trusted to exercise judgment within clear boundaries. Wild West Engineering lacks the structure for long-term growth, while Bureaucratic Engineering frequently leads to rigidity and limits learning. Context-Driven Engineering seeks balance through prioritizing principles over prescriptions, fostering autonomy with accountability, and encouraging decisions based on trade-offs rather than routine. As AI accelerates code generation and change, maintaining this harmony is increasingly important. The most adaptable organizations will foster responsible, context-aware decision-making.

By Otavio Santana

CORE

Can Your Team Name the Work It Already Runs With AI?

TL;DR: The AI Workflow Inventory Finalizes the A3 Delegation System You probably know your own AI shortcuts: the colleague who drafts the stakeholder update, the nightly job somebody set up before they left, the interview notes that go through a model on demand. However, I want to challenge you: that perceived knowledge can create false confidence, because it feels like knowing the team’s way of working with AI, when it is still only a diffuse understanding of the practice. Ask the team to combine those individual accounts into one list, and you may discover how little of the whole anyone can see. The AI Workflow Inventory is the artifact for that list: one row per recurring task, refined into provisional task classes to prepare the team’s next AI delegation decisions. Disclaimer: I read Charniak/McDermott’s book on “Artificial Intelligence” decades ago; of course, I use AI for research, translation, proofreading, challenging story arcs and article structures, and summarization. It is a production tool, not a substitute for thinking. Your Own Shortcuts Are the Easy Part When I published the AI Delegation Lifecycle in June 2026, the point before stage 1 was deliberately left without an artifact: the team was supposed to know which work it does with a model, at what frequency, and at what stakes, which I call a forensic analysis of your own workflow. However, as I quickly discovered, this assumption that a team could produce the analysis from memory was wrong, and updating the student reference for the A3 Delegation System earlier this month showed me its design flaw: The Routing Policy assigns an AI model tier per task class,The AI Definition of Done is written per task class (never per task), andThe Delegation Audit opens with a walkthrough of what the team has handed over. All three artifacts consume task classes, but none of the six stages of the A3 Delegation System produces them. The AI Working Agreement is the exception on the other side of the lifecycle: it records the team’s defaults and boundaries and can exist before the inventory is complete. The gap is easy to miss from the inside, because every individual on the team can answer for their own use, and nobody can answer for the sum: who else drafts with a model, which outputs leave the team that way, and which jobs run unattended. That sum, useful, undocumented, and unowned, is what I call “AI Debt,” and it works right up to the moment someone asks who is responsible for it. What One Row Looks Like in the AI Workflow Inventory The AI Workflow Inventory is a one-page canvas with an identity strip (team, owner, date, review date), a panel on where to look, a panel with five refinement rules, and the inventory table itself. Let me show you an example from a Scrum team: Number: I-01Task: Transcribe photos of Retrospective sticky notesProcess: Retrospective facilitationTask class: Transcriptions for internal useHow often: Per SprintWho runs it: Scrum MasterTool: GPT-5.6 SolOutput goes to: Stays in the teamData that enters the model: Photos of handwritten notes and names. The task is written as verb and object, the class is named by output and audience, and “output goes to” has three values: stays in the team, leaves the team, or customer-facing. The number travels to the Workflow Card later, so the row and the respective decision can be matched months apart. Two design choices carry most of the weight: The person and the tool sit in separate columns because people change and tools change while the task stays; a field that says “Anna, ChatGPT” makes it harder to update either one when Anna moves teams or the license changes.The last two fields ask different questions that teams routinely conflate: where the output goes tells you about the stakes on the way out, whereas what enters the model tells you about the exposure on the way in, and an output that “stays in the team” can still have been produced from photographs with names on them. The entry in the data field is a fact to review; it is not permission to keep uploading the names. What the canvas does not have is a life cycle stage or status column, and the omission is deliberate. The inventory records what happens today; the poster and the Workflow Cards show where a workflow stands in the A3 Delegation Life Cycle, and a retired workflow is a row in the Re-classification Log, identified by the same inventory number. A stage column would turn the pre-decision list into a second, competing status board, and agile practitioners already know how that ends from the Product Backlog in Jira and the “real” one in a spreadsheet debate. The AI Workflow Inventory takes work to maintain, but it reduces repeated documentation and decision-making: The team captures each recurring task once.Compatible tasks share one review standard and one routing decision instead of getting one each.The inventory number connects a task to every later decision about it.The life cycle status stays on the poster and in the records that already exist. Update the inventory when the work changes or when the team discovers, corrects, or refines what it has recorded. (I know, it sounds like a database schema, and I started preliminary work on how to design an application for the A3 Delegation System.) Where to Look The canvas names three sources for AI workflows, and the 30-day window matters because memory is a poor inventory tool, while last month’s outputs are evidence: Your meetings come first: which outputs of the events that repeat (planning, pipeline or campaign reviews, stand-ups, monthly reporting, Retrospectives, etc.) did a model draft, summarize, transcribe, or translate?Then your tools: each colleague looks at their own chat history from the last 30 days and brings the prompts, applied Skills, or agent tasks that come back every week or every Sprint, who wrote them, and how often they were rewritten; this is a self-report, and it has to be, since nobody should be reading anyone else’s conversations.Then your outbox: the emails, reports, tickets, and documents that left the team, and the question of which of them started as a model draft. Thirty days is the starting window. Once the obvious rows are on the page, ask about the work no chat history shows: the jobs that run unattended, the quarterly report, the thing that only happens at release time. A nightly test run, such as the one in the example below, may leave no trace in anyone’s recent chat history. The line that decides the session is the last one in that panel: list the shortcuts nobody approved as well, because that is where the AI debt sits. If people read the inventory as a check on whether they followed the rules, those shortcuts stay invisible, and the team ends up with a list that looks tidy and describes a team that does not exist. Therefore, going first with a shortcut of your own helps, as does explaining, before the first row, that recording a task does not approve its use: the session starts with understanding what happens today, and concerns that need immediate attention under the team’s existing boundaries will still be addressed. Provisional Task Classes and the Grouping Test The grouping rule creates a puzzle that a careful reader will spot: it asks whether two tasks share the same AI Definition of Done, the same reviewer, and the same model tier, and those are decisions for later stages of the AI Delegation lifecycle, which the team has not reached. The resolution is that classes in the first session are provisional. Name them by output and audience, test the grouping against whatever review standards and routing decisions the team already has, and where those are missing, leave the grouping provisional and revisit it as the team works through the stages. An AI workflow inventory can carry uncertainty; what it must not carry is a settled-looking decision nobody made. The five rules on the AI Workflow Inventory canvas are short: Capture first, judge later: The A3 decision comes in stage 1, after the row exists, and a row approves nothing.One row per recurring task: Four prompt revisions used for the same weekly update are one task, and the tool goes in its own column.Name the class by output and audience: “Transcriptions for internal use,” “status communication leaving the company.” A class name that fits any task (“AI support”) is too wide to tell anyone which standard applies. (That naming approach is not different from coding.)Same standard, same reviewer, same tier, one class: When one of the three differs, split the class.Every class goes to stage 1; every audit refines the list: A class without an A3 decision is AI debt with a date on it. Three failure patterns are worth watching for: the sanctioned-use list, where only approved use gets written down, and your existing AI debt stays invisible; the prompt or Skill catalog, with a row per prompt/Skill instead of per task, so the classes never form; and the one-time census, filled once and never refined, so that six months later the audit walks a stale list and everyone nods at rows that no longer run. Example: Four Rows From a Scrum Team Here is a full first pass from a fictitious Scrum team of seven: Transcribe photos of Retrospective sticky notes (Scrum Master, GPT-5.6 Sol, stays in the team; input: handwritten notes with names): Internal output can still involve sensitive input.Rewrite customer feedback tickets into Product Backlog item drafts (class: requirement drafts for internal use): A concrete task inside a reusable class.Draft acceptance criteria for new items (same class): A second task that may share that class, as long as reviewer, standard, and tier stay the same.Rewrite interview notes into the hiring feedback template (on demand, Scrum Master, leaves the team; input: candidate names): Informal use that an approved-use-only list would have missed. The last row is on the list only because capture came first; under a rule that lists only approved use, the person running it would have left it out, and the team would have gone into A3’s stage 1 without knowing candidate names were entering a model on demand. From Row to Workflow Card Each decided row of the AI Workflow Inventory becomes a Workflow Card within the A3 Delegation System: workflow, task class, the A3 category, and later the tier, the owner, and the audit dates; the card goes on the poster where the workflow stands today. The word “decided” matters: the row exists before the A3 “Assist-Automate-Avoid” decision; the card exists after it. A row without a card means one of two things, and the remedy differs: either nobody has decided, in which case the class goes to the A3 Framework next, or somebody decided and never recorded it, in which case the team confirms the decision and writes it down. Ask which it is before doing either. For the transcription row, that means: the AI Workflow Inventory gives it a number, a task, and a provisional class; A3’s stage 1 decides whether it is Assist, Automate, or Avoid, and knowing that GPT-5.6 Sol runs it today settles nothing; stage 3 settles the owner, and the Scrum Master who runs it this Sprint does not become the owner by default. Both decisions go on the card. Your First 60 Minutes with the AI Workflow Inventory Book the session, aim for eight rows, and treat it as a first pass. One way to split the time: State the purpose and the boundary (capture, not approval): 10 minutesWalk the three sources, including unapproved shortcuts and unattended jobs: 20 minutesForm provisional task classes with the grouping test: 20 minutesName the inventory owner and the review date, and agree on the next step: 10 minutes The next step is A3’s stage 1: take every class to the A3 decision with the people who run the tasks in the room. If a row needs attention under the team’s existing boundaries before that (candidate names in a model, say), address it; “capture first” gives the decision its own step and does not postpone it. At each Delegation Audit, the list gets two kinds of maintenance. Add the rows that appeared since the last audit, and reconcile the existing ones: does the task still run, does the same person run it, with the same tool, on the same inputs, for the same audience? When you retire a task, record that decision in the Re-classification Log under the same inventory number, and keep the link to its original inventory record. Conclusion Which recurring use would your team’s current list miss? If your team cannot describe where AI already contributes to recurring work, start with a 60-minute AI Workflow Inventory session. The list gives you a shared basis for A3 decisions and helps you find the workflows that individual recollection might miss; then choose where in the A3 Delegation System to apply the method first.

By Stefan Wolpers

CORE

Beyond Token Intelligence: Why AI Code Review Needs Cognitive Architectures

A few months ago, I watched a senior engineer spend forty-five minutes reviewing a single pull request — a PR that an AI assistant had generated in under two minutes. The code looked clean. The tests passed. But she kept cross-referencing an incident postmortem from eight months earlier, muttering something about retry amplification. She caught a real production risk. The AI reviewer had flagged nothing. That moment stuck with me. We've spent years optimizing how fast we can write code. But we haven't seriously reckoned with what happens when review can't keep up. The Bottleneck Has Shifted A single engineer with AI assistance can now produce hundreds of lines of code, large refactors, infrastructure changes, and test suites — all within minutes. Review complexity, however, grows exponentially with change size and system interdependency. The core problem is no longer "Can AI write code?" It's "Can humans reliably validate what AI wrote?" Code generation speed increases. Human cognitive review capacity stays flat. That imbalance is quietly accumulating risk in engineering organizations everywhere. Why Current AI Reviewers Fall Short Most AI PR review systems today operate on static diffs, syntax-level reasoning, and shallow best-practice detection. They produce comments like: "Potential null pointer.""Consider renaming this variable.""Possible optimization opportunity." Occasionally useful. Rarely sufficient for production-critical systems. The structural problem is that these tools treat PR review as a language problem instead of a systems reasoning problem. They assume software correctness is inferable from local code semantics alone. In reality, production safety emerges from interactions between architecture, runtime behavior, operational history, and organizational context. The Shallow Review Problem in Practice Here's a concrete example. An AI assistant generates this database query optimization: Python # AI-optimized version def get_user_orders(user_id): return db.query(""" SELECT o.*, p.*, i.* FROM orders o JOIN payments p ON o.id = p.order_id JOIN items i ON o.id = i.order_id WHERE o.user_id = ? """, user_id) Typical AI reviewer comment: "Query optimized with JOIN to reduce round trips." What a senior engineer sees: "This will cause a Cartesian explosion. The orders table has 50M rows, items averages 8 per order. This returns 400M+ rows for power users. We had a nearly identical incident (INC-287) that took down the read replica. Needs pagination and selective columns." The difference isn't token count or model size. It's operational memory and causal reasoning. The Real Challenge Is Not Context Windows Many people assume the fix is larger context windows. Feed the model the whole repo, and it'll review like a senior engineer. But experienced engineers don't review code by loading entire systems into working memory. They use abstraction, selective attention, and compressed mental models. A senior engineer reviewing a Kafka retry change doesn't reread the entire messaging subsystem — they remember prior incidents, retry amplification risks, and historical outages. That's cognitive compression, not token recall. Modern LLMs are exceptional at syntax fluency, pattern completion, and probabilistic association — what you might call token intelligence. But effective PR review requires something deeper: causal reasoning, architectural abstraction, operational memory, risk forecasting. Call it cognitive intelligence — persistent contextual reasoning grounded in operational history and causality. The distinction matters because it changes what we need to build. What a Cognitive Review Architecture Looks Like Instead of: Plain Text Large Prompt + Large LLM → Review We need: Plain Text Structured Memory + Semantic Retrieval + Runtime Context + Specialized Review Agents + Reasoning Layer + LLM → Review The LLM should not be the memory. It should be the reasoning interface over structured engineering knowledge. Intent Reconstruction Before reviewing code, the system needs to understand why the change exists. Business intent, bug root cause, architectural motivation. Inputs include Jira tickets, PR descriptions, ADRs, incident reports, and commit timelines. Without intent, review quality stays shallow regardless of model size. Engineering Knowledge Graphs Human reviewers carry organizational memory: fragile services, latency-sensitive paths, scaling bottlenecks, previous outages, dangerous dependencies. AI reviewers need persistent semantic memory systems encoding the same — service relationships, API contracts, operational metadata, incident history, ownership boundaries. This creates an engineering cognition layer far richer than raw repository context. Multi-Agent Review Systems A single reviewer model is insufficient. Future systems will consist of specialized agents working together: Architecture Reviewer – dependency boundaries, coupling risk, architectural driftReliability Reviewer – retries, backpressure, idempotency, failover behaviorSecurity Reviewer – injection risks, auth issues, secret exposurePerformance Reviewer – memory growth, query amplification, scaling regressionsHistorical Regression Reviewer – correlation with past outages, postmortems, incident fingerprints This begins to approximate how experienced engineering organizations actually review software. Runtime-Aware Review Static analysis alone misses emergent runtime behavior. Future cognitive review systems will integrate observability telemetry, tracing data, production metrics, and traffic patterns. Compare these two responses to a retry configuration change: Traditional AI reviewer: "Code follows retry best practices." Cognitive AI reviewer with operational memory: "HIGH RISK: Similar retry configuration caused incident on 2023-09-15. This service processes 2M messages/hour at peak. 10 retries with exponential backoff = up to 17 minutes per message. Previous incident resulted in 8M message consumer lag and cascading downstream failures. Recommend: max 3 retries, circuit breaker, dead letter queue, idempotency check before db.save(). See ADR-089." That is a fundamentally different class of intelligence — and a fundamentally different class of safety. Engineering Memory Is the Missing Piece One of the biggest gaps in current AI systems is durable operational memory. Experienced engineers develop intuition through outages, failed deployments, debugging sessions, and production emergencies. These experiences become compressed heuristics: "This retry increase feels dangerous" — not because of syntax, but because of remembered causal relationships. Replicating this requires episodic memory systems, incident-aware reasoning, and causal knowledge graphs. Much of this mirrors practices long established in Site Reliability Engineering, where institutional learning from incidents is treated as critical infrastructure. Incident postmortems aren't just documentation — they're organizational immune system responses. Getting AI systems to genuinely learn from incidents rather than just pattern-match against them remains one of the harder open problems in this space. What Teams Can Do Today Fully cognitive review systems don't exist yet. But organizations can meaningfully improve AI-assisted review quality right now: Capture architectural knowledge in machine-readable form. Service boundaries, retry policies, timeout configurations, scaling assumptions — not just in wikis, but in structured formats AI systems can query.Link PRs explicitly to incident history. Build connections between code changes and the incidents they caused or prevented. This is organizational memory that AI systems can leverage today.Tag services with operational metadata. Criticality tier, traffic patterns, known failure modes, blast radius. Treat repositories as systems, not just files.Integrate observability into review pipelines. Connect production metrics and tracing data to code review. Runtime context dramatically improves review quality.Prioritize high-signal AI feedback. Review fatigue from noisy, low-signal comments is a real trust problem. Focus AI comments on incident-correlated patterns, architectural violations, and operational risks. The Trust Calibration Problem One concern I keep coming back to: bad AI reviewers are dangerous not because they miss things, but because they sound confident while missing things. They reduce human vigilance through automation bias. They generate fatigue through noise. They normalize shallow approval. Future cognitive review systems need to be not just more accurate, but properly calibrated — knowing when they lack sufficient context and escalating accordingly. An AI reviewer should be able to say: "I may not have enough confidence to validate this safely." That self-awareness may matter more than raw capability. The Road Ahead The next era of AI software engineering will not be defined by who generates the most code. It will be defined by trust, reasoning quality, and operational awareness. The future belongs to systems capable of understanding not just what changed — but why it changed, what it affects, and whether it's safe. That's the difference between code generation and engineering intelligence. And honestly, solving it seems harder and more interesting than anything we've built so far. Key Takeaways The bottleneck has shifted from code generation to code review and validation.Larger context windows alone won't bridge token intelligence and cognitive intelligence.Human-like review requires structured memory, causal reasoning, and operational awareness.Multi-agent architectures with specialized reviewers mirror how engineering teams actually work.Runtime-aware systems integrating production telemetry represent the next frontier.Engineering memory — learning from incidents — is critical for trust and safety.Teams can start today by capturing architectural knowledge and linking incidents to code changes. References Vaswani, A., et al. (2017). "Attention Is All You Need." NeurIPS.Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.Lewis, P., et al. (2020). "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks." NeurIPS.Shinn, N., et al. (2023). "Reflexion: Language Agents with Verbal Reinforcement Learning." arXiv.Beyer, B., et al. (2016). Site Reliability Engineering: How Google Runs Production Systems. O'Reilly Media.Allspaw, J. (2012). "Blameless PostMortems and a Just Culture." Etsy Engineering.

By Sayan Chatterjee

Three Hidden Traps That Shape Software Engineering Decisions

Software engineering is often seen as a technical field focused on designing systems, writing code, selecting architectures, defining APIs, and optimizing for performance, scalability, and maintainability. However, many of the most challenging problems are not technical, but human. As engineers advance into leadership roles such as senior engineer, Staff Engineer, architect, or engineering manager, the challenge extends beyond technical solutions. Leaders must also understand how people make decisions, how teams respond to change, and how individual behavior can influence software quality. Humans are not always rational. We depend on habits, shortcuts, intuition, experience, and assumptions. While these can improve efficiency, they may also lead to decisions that don't align with requirements, constraints, or long-term goals. A team may retain a design simply because “we have always done it this way.” Engineers might accept poor code quality if the existing codebase is already disorganized. Teams may also select a complex architecture because complexity appears more robust, even when a simpler solution would suffice. While these decisions may seem minor individually, over time they can impact more than a single implementation. They may influence team behavior, increase technical debt, reduce maintainability, and shape the system’s architecture. This article examines three common behavioral traps that affect software engineering decisions: status quo bias, the broken windows effect, and complexity bias. We will also discuss how engineers and technical leaders can identify these patterns and foster more deliberate, evidence-based decisions. What Are Cognitive Biases, and How Do They Affect Software? Cognitive biases are systematic patterns in how people interpret information, evaluate alternatives, and make decisions. Rather than random mistakes, they are mental shortcuts that help us act quickly but can also influence our judgment without our awareness. This is especially relevant in software engineering, where decisions are often made under uncertainty, time constraints, incomplete information, and the influence of prior experience. Engineers rarely evaluate every alternative from scratch. Instead, we reuse patterns, rely on intuition, trust familiar technologies, and simplify complex situations to maintain progress. Most of the time, this is useful. Software development would be impossibly slow if every decision required exhaustive analysis. Problems arise when these shortcuts replace thoughtful reasoning instead of supporting it. For example, a team may retain an implementation simply because it exists, not because it meets current requirements. Developers might accept poor practices if the codebase already appears neglected. Architects may select more complex solutions because complexity seems more robust or professional. These effects can extend far beyond the initial decision. A cognitive bias that starts as an individual preference can influence code reviews, become a team convention, shape architectural decisions, and eventually contribute to technical debt. The impact therefore goes beyond a single decision. Biases can affect: Code quality and maintainabilityArchitectural consistencyTechnical debtTechnology selectionResistance to changeTeam standards and engineering culture The most concerning aspect is that biased decisions rarely seem irrational when made. They often appear reasonable: “this is how we normally do it,” “the code is already messy,” or “we need something more robust.” Recognizing these patterns does not eliminate them, but it enables engineers and technical leaders to challenge assumptions before they become embedded in the architecture. This understanding also provides a natural transition to the following sections: status quo bias, the broken windows effect, and complexity bias. Status Quo Bias: “We’ve Always Done It This Way” Status quo bias is the tendency to favor the current situation solely because it is established. Samuelson and Zeckhauser showed that people often favor existing options, even when alternatives are available: Status Quo Bias in Decision Making. In software engineering, this bias can be more harmful than simple resistance to change, as it may keep teams from evaluating decisions against current circumstances. Software evolves continuously. Requirements shift, languages gain new features, frameworks improve, and teams learn from experience. A decision that was appropriate years ago may no longer be suitable today. Problems arise when teams assume the current solution is correct by default. “We’ve always done it this way.” How Status Quo Bias Impacts Software Engineering The greater risk is that teams may stop analyzing whether existing decisions still fit the current context, not just that they avoid change. For example, a Java team may continue using XML configuration simply because it was the original convention, even when annotations or programmatic configuration would better suit parts of the system today. The problem is not XML itself. XML may still be the right choice. The issue arises when the reasoning is: we used XML before, so we use XML now. This differs from evaluating current requirements, available language and framework capabilities, trade-offs, and then selecting the most appropriate configuration strategy. The same pattern can appear almost anywhere: “We always store this in the session.”“We always use inheritance here.”“We always create a microservice for this.”“We don’t use that newer Java feature.”“This framework has always worked for us.” Each of these statements may reflect a valid decision, but none alone provides sufficient technical justification. The broader risk is that architecture may reflect outdated constraints rather than current needs. Teams may overlook new language features, framework improvements, better APIs, production lessons, infrastructure changes, or architectural alternatives unavailable when the original decision was made. Over time, the codebase may remain internally consistent but become increasingly disconnected from its surrounding ecosystem. Research on software architecture decision-making shows these choices are not purely rational and can be influenced by bounded rationality and cognitive biases: Decision Making in Software Architecture. How to Handle Status Quo Bias in a Team The goal is not to challenge every existing decision or adopt every new technology. Consistency, migration cost, operational risk, and familiarity are all valid engineering concerns. The key is to ensure that history informs the decision, but does not determine it. A simple question can help: If we were making this decision today, with our current requirements and tools, would we make the same choice? This separates two different questions: What is the best design for the current context?Is changing the existing design worth the cost? Those are not the same problem. Another useful technique is to make evaluation criteria explicit. Compare the current approach and alternatives using the same dimensions: maintainability, complexity, operational risk, migration cost, framework support, and long-term evolution. This approach creates a fairer discussion, as the current solution no longer receives an automatic advantage simply because it exists. Naming the Bias Can Move the Discussion Forward One effective leadership technique is to name what may be happening. Instead of saying: “You are resisting change.” A staff engineer, architect, or manager can ask: “Could status quo bias be influencing this decision?” This shifts the discussion from evaluating individuals to evaluating the decision-making process. Someone may then realize: “I prefer this solution because I know it well, not because we compared the alternatives.” That distinction matters. Experience says: “We tried this before, and here are the trade-offs we learned.” Inertia says: “We do this because we have always done it.” Effective technical leadership involves helping teams recognize this distinction. The goal is not modernization for its own sake, nor is newer automatically better. The aim is to ensure architecture evolves with the context, rather than remaining unchanged simply because the current solution is familiar. Complexity Bias: “It Can’t Be That Simple” Complexity bias is the tendency to trust a complicated solution more because it appears more complete, sophisticated, or robust. Simple solutions can feel uncomfortable, especially for important problems, leading us to expect equally complex architectures. Research on causal reasoning shows that people often view complexity as a sign that an explanation fits the evidence, even though they prefer simplicity in other contexts. See Simplicity and Complexity Preferences in Causal Explanation (ScienceDirect). In software engineering, this bias often leads to overengineering. How Complexity Bias Impacts Software Engineering The main problem is not complexity itself. Some systems are legitimately complex. The real issue is unnecessary complexity, as each abstraction, service, dependency, process, and architectural layer adds risk. Additional components increase the effort required to understand, integrate, test, deploy, monitor, and modify the system. Solutions meant to improve safety can instead expand the potential for failure and raise the cost of future changes. Simple requirements can accumulate queues, events, retries, services, orchestration, and frameworks, making the architecture seem more robust even when these additions are not justified. Forbes identifies over-engineering solutions as one of the bad habits technology leaders want developers to avoid, noting that developers sometimes solve problems that don't exist or add complexity before fully understanding the problem (Forbes). Complexity bias also influences organizational practices. Teams may add approval stages, meetings, documents, and processes because increased control feels safer. Eventually, even agility suffers. Teams may become attached to processes without evaluating whether they still support effective software delivery. Just Enough Architecture George Fairbanks’ concept of Just Enough Software Architecture offers a useful counterpoint. His risk-driven approach suggests that architectural effort should match the actual risk of failure. When risk is low, extensive design can be wasteful. When risk is high, more deliberate design is warranted (George Fairbanks). This gives us a better question than: “How can we make this architecture more robust?” Ask instead: “What risks are we trying to mitigate, and how much architecture do those risks justify?” That changes the discussion significantly. An extra service may be justified by isolation requirements. Asynchronous communication may be justified by availability or throughput. Regulation may require additional governance. However, you should always justify complexity. Fairbanks summarizes the principle well: architectural effort should match the risk of failure (George Fairbanks). How to Handle Complexity Bias in a Team A useful starting question is: What is the simplest solution that satisfies the requirements and mitigates the risks we actually have? Technical leaders can then challenge complexity with concrete questions: What risk does this abstraction reduce? What requirement needs another service? What would happen if we did not introduce this complexity yet? Naming the bias can also help: “Are we adding this because the problem requires it, or because the simpler solution feels too simple?” The goal is not simplicity at any cost. Underengineering poses risks similar to overengineering. The goal is just enough architecture: introduce the complexity needed to manage real risks, but no more than the problem justifies. Broken Windows: When the Environment Lowers the Standard Unlike status quo or complexity bias, Broken Windows Theory is not a cognitive bias. Instead, it represents a risk: visible disorder can influence future behavior and gradually redefine what is considered acceptable. Although the concept originated outside software engineering, researchers have applied it to code quality for decades. Recently, researchers examined whether this effect occurs in software systems. A 2024 study in Empirical Software Engineering found that developers working in systems with higher technical debt were more likely to introduce additional debt, such as poor variable names, duplicated behavior, and other issues detected by static analysis (Springer Link). How Broken Windows Impact Software Engineering A neglected codebase sends signals. When engineers repeatedly encounter duplicated code, weak tests, inconsistent naming, abandoned abstractions, or ignored warnings, the implicit message can become: “This level of quality is acceptable here.” That can change behavior. Rather than considering how the code should ideally be structured, developers start adapting to their environment: “This module is already messy.” “Nobody tests this part anyway.” “There are already dozens of warnings.” “One more shortcut will not make a difference.” The risk is cumulative. While a single compromise may seem minor, repeated compromises can gradually lower the entire system's engineering standards. The 2024 controlled experiment is notable because it provides direct evidence of this effect. Developers working on systems with higher technical debt were more likely to introduce new debt (Springer Link). A second 2024 study examining code history also found that existing code quality can influence the quality of subsequent changes, though the effect varies by quality characteristic (arXiv). The Risk Goes Beyond Code Broken windows can also appear in engineering culture. If architectural rules are routinely ignored, pull requests are poorly reviewed, failing tests are tolerated, or technical debt is continually postponed, developers learn what the organization truly values. Written standards may say one thing, while the environment says another. That distinction matters because engineering culture is shaped not only by what leaders say, but also by what teams repeatedly observe. A codebase can therefore create its own feedback loop: Poor quality normalizes lower standards. These changes create additional visible disorder, which further reduces expectations. How to Handle It in the Team The goal isn't to create a perfectly clean system, as that can lead to overengineering. Instead, teams should prevent visible problems from becoming the accepted baseline. Small actions matter: Fix obvious problems when touching nearby codeAddress warnings and failing tests promptly, preventing them from becoming permanent background noiseSet a minimum quality standard for new changesMake technical debt visible rather than silently normalizing it Technical leaders can also name the pattern directly: “Are we accepting this because it is genuinely the right trade-off, or because the surrounding code has already lowered our expectations?” This question helps distinguish between pragmatic technical debt and simple degradation. The objective isn't perfection, but preventing the environment from silently teaching the team that quality no longer matters. A useful principle is: Every codebase teaches developers how it expects to be treated. Once poor quality becomes the norm, reversing that expectation becomes much more difficult.

By Otavio Santana

CORE

Architecting Production AI Across Clouds: Patterns That Decide System Survival

Most enterprise AI post-mortems do not blame the model. They blame the storage tier that starved the accelerators, the identity policy that over-granted access, the cost model that ignored egress, the forecast that leaked future data, or the region that failed and took a business process with it. The hard part of production AI was never intelligence. It was the engineering discipline around it. This article distills the architectural patterns that decide whether a cloud AI system is trustworthy at scale, spanning infrastructure, identity, cost, operations, the applied domains, low-code assembly, platform selection, and multi-cloud resilience. It is written for engineers who have to keep these systems running, not for a keynote. Infrastructure: The Interconnect Is the Bottleneck Distributed training is a systems problem before it is a machine learning problem. When a job spans many graphics processing units (GPUs), the fabric connecting them (e.g., NVLink within a node, InfiniBand, or a vendor fabric across nodes) frequently caps throughput more than raw compute does. Accelerators wired through an ordinary network idle while they wait to synchronize gradients. Storage is the symmetric constraint. If the file system cannot deliver data at the rate the accelerators consume it, utilization collapses. The pattern is a tiered design: Hot tier: parallel or block storage feeding active training at high input/output operations per second (IOPS).Warm tier: recent data staged for quick promotion.Durable lake: object storage providing petabyte-scale durability, partitioned and lifecycle-managed underneath. Two cost drivers hide from the pricing page: data egress (moving data across regions or out of a provider) and idle warm capacity. Optimizing only the advertised compute line item guarantees a surprise on the invoice. Identity Is the Perimeter In a service-to-service AI architecture, the network perimeter is gone; identity is the boundary. A zero-trust posture, where every request authenticates and receives least privilege, contains the blast radius when a component is compromised. Across providers, identity federation is the load-bearing pattern: a principal authenticates once and is recognized everywhere, so access is granted and revoked centrally instead of reconciled across three identity systems. Policy must travel with the workload; a rule enforced on one cloud and forgotten on another is not a policy. Model authorization is the emerging frontier. As models call tools and take actions, the question moves from who can query this model to what may this model do on a user's behalf. Least privilege applied to an autonomous agent is the boundary between useful and unbounded. Cost and Operations Are a Control Loop Cost management is not a spreadsheet; it is automation. Consistent resource tagging across every cloud is the prerequisite for attribution. On top sit budgets, alerts, and automated remediation that throttles runaway spend before it escalates. Site reliability engineering (SRE) supplies measurable targets. For AI workloads, the golden signals extend beyond latency and errors to accelerator utilization, queue depth, and prediction quality. A model can be fully available and quietly wrong, so define a service level objective (SLO) for output quality, not just uptime. Three techniques earn their complexity: Spot or preemptible capacity plus checkpointing cuts training cost sharply when jobs resume cleanly after reclamation.Predictive scaling anticipates load instead of reacting to it.LLM inference optimization becomes architectural: batch requests, cache frequent responses, route easy queries to smaller models, reserve the expensive model for queries that need it. The Applied Domains Share a Spine, Differ in Physics Vision is byte-heavy. High-resolution images and video streams make the data and network layers dominant. For real-time video, decouple frame capture from analysis and sample frames rather than processing every one. Critically, a business-rule layer, never the model alone, owns consequential decisions. Every extraction should carry a confidence score used as a routing gate: Python def route_extraction(field, threshold=0.90): if field["confidence"] >= threshold: return "auto_process" return "human_review" Language is byte-light but semantically treacherous, and because it replies directly to users, errors are visible. The defining risk of generative systems is hallucination. The strongest architectural defense is retrieval grounding, forcing answers from verified sources with citations: Python def answer(question, knowledge_base): passages = knowledge_base.search(question, top_k=3) context = "\n".join(p.text for p in passages) prompt = f"Answer using ONLY this context.\n{context}\n\nQ: {question}" return model.generate(prompt), [p.source for p in passages] Forecasting is defined by time order. You cannot shuffle a time series into random splits, and the most common failure is data leakage, using information unavailable at prediction time. Test on a fair, time-ordered holdout, and always emit a prediction interval; a point forecast that hides its uncertainty invites overconfident decisions. No-Code and Low-Code: Governed or Ungoverned No-code and low-code platforms collapse build cost from a scoped project to an afternoon, which is why adoption is exploding. The symmetric risk is sprawl: hundreds of ungoverned flows handling sensitive data, owned by no one. Govern with guardrails, not gates. Restrict which connectors and data sources are permitted, assign an owner and an SLO to every production flow, then let builders move freely inside the boundary. The goal is to make the safe path the easy path. Platform Selection Without Self-Deception Vendors all claim to be fastest, cheapest, and most reliable. Benchmark to replace claims with evidence: Latency: report percentiles (p95, p99), never averages that hide the slow tail.Quality: measure on your own representative data, not a public leaderboard.Cost: model total cost of ownership, including transfer, storage, idle capacity, operations, and migration, not the headline compute rate.Reliability: verify the platform meets your recovery time objective (RTO) and recovery point objective (RPO). Combine dimensions in a weighted scorecard whose weights are fixed before scores are seen. Adjusting weights afterward to crown a favorite converts analysis into rationalization. Multi-Cloud Resilience: Design for the Day a Cloud Fails For systems a business cannot lose, a single provider is a gamble. Multi-cloud resilience deliberately places critical workloads so no single provider failure takes the business down, applied only where the cost of failure exceeds the cost of prevention. Predict rather than react. Combine leading signals into a health score and fail over proactively: Python def health_score(latency_ms, error_rate, saturation): latency_factor = max(0, 1 - (latency_ms / 1000)) error_factor = max(0, 1 - (error_rate / 0.05)) saturation_factor = max(0, 1 - saturation) return round(0.4*latency_factor + 0.4*error_factor + 0.2*saturation_factor, 3) Kubernetes makes workloads portable; data replication (with the consistency-versus-availability trade-off decided per workload) keeps data ready on the other side; and a portable foundation of federated identity, uniform policy, and centralized monitoring makes failover routine rather than heroic. The discipline that separates real resilience from a slide deck is rehearsing failure on purpose. An untested failover path is a promise, not a capability. The Judgment Layer Across every layer, value came not from the most powerful component but from the judgment applied to it: matching effort to problem difficulty, keeping humans on consequential decisions, measuring before deciding, building governance in early, and designing for change. Tools will churn; foundation models will make today's designs look quaint. That is precisely why principles outlast product knowledge. The scarce resource in enterprise AI was never intelligence. It was judgment, and judgment does not ship from the cloud.

By VenkataSrinivas Kantamneni

Reliability Without Control: Operating SRE Practices in Platform–SaaS and API-Dependent Systems

Originally, back-end and front-end Site Reliability Engineering (SRE) were owned by teams. They code the programs, set up databases and infrastructure, and quickly spring to action at the beep of any anomaly. The advent of code vs no-code infrastructure, SaaS, API dependencies, third parties, and other modern systems seems to be eroding this authority. Mainstream and underdog companies now often leverage the significant advantages of outsourcing, collaboration, or delegation, which are usually accompanied by a silent clause: no or partial control. Unlike in previous systems, modern production is largely assembled rather than built from scratch. For example, a conventional SaaS product is built on interdependencies among payment processors, outsourced data infrastructure such as Amazon Web Services (AWS), messaging services, web hosting, design, AI inference APIs, authentication providers like Google, and more. These useful platforms and products are essentially outside teams' control stations, even though they critically impact users' experience. When they function effectively, you share the glory with the platforms. But when there is a system blackout, your users put you on your toes, even though you have no direct access to resolve the problem on time. Therefore, we shall be exposing SRE practices in platform-SaaS and API-dependent systems and how reliability is getting beyond the control of engineering teams and companies. Why Classical SRE Practices May Fail One major downside of SaaS and dependency on external platforms is that reliability control is often assumed to be in a team's hands, whereas it has been bargained. However, teams must reckon with the fact that the case is reversing. For example, traditional SRE models once alleged that: Service Level Indicators (SLIs) focus on availability or internal uptime and latency.Error budgets arise from changes teams make or deploy.Runbooks still suggest that teams can immediately reconfigure or directly work on faulty components. All these are becoming past cases, especially in platform-SaaS systems. You can have a system indicating 99.99% or even 100% uptime on the back end, while new users are struggling to sign up, probably because an authenticator provider is not fully functional. Dashboards and control panels may indicate green, but in reality, third-party payment APIs have been degraded. A New Definition of Reliability in Operating SRE Practices To resolve the new problem in site reliability engineering (SRE), there needs to be a conceptual shift from component health to an integrated, continuous user experience. Therefore, teams need to undergo a paradigm shift away from questions such as "Is our CPU working maximally?" “Is our API up?” “What are the error rates?” Instead, we should inquire: “Are users checking out seamlessly?” “How fast can they authenticate?” “Can they use the SaaS product to perform its key function?” These types of outcome-based questions span interdependent platforms beyond your full control. The login SLI needs to work with the identity provider; otherwise, its output is meaningless. If the checkout SLO skips payment authorization, then it's both fishy and unreliable. True, there may be some internal errors in a reliable system, but what really matters is an integrated multiplatform experience that the user enjoys. Error Budgets? An SRE Practice to Revisit How many teams would love error budgets to disappear when they give up control? But that’s not so. Instead, they are molecularized. When components of your systems are outsourced, the error budget doesn’t just fade away; it is instead transferred to the interdependent platforms. So, it’s better to plan for the fact that SaaS and API providers will consume some of your reliability budget. Doing so keeps you a few steps ahead and protects your business in the long run. Reliable SRE teams make decisions such as allocating part of their error budget to certain dependencies, setting acceptable parameters for degradation, and defining specific steps to take when a dependency exceeds the stipulated budgets. Here’s an example you can adapt: “We will accept payment authorization failure of 0.0% to 0.2% if it is caused by dependency instability. If it goes above that, we will turn on delayed capture or turn off promotions.” This SRE approach keeps you ready for downtime, as your systems automatically switch to planned or budgeted actions rather than relying solely on integrated platforms. What to Do When Failures Beyond Your Control Arise Actually, some failures may seem beyond your control. The more you attempt to resolve them, the more amplified they become. At this point, your team must adapt to the savvy absorption of such situations. Instead of focusing solely on retrial in an SRE approach, your team needs to design its processes and platforms. This could include failing selectively through circuit breakers, failing fast with timeouts, or failing visibly by keeping users informed. Some core settings should always remain non-negotiable and on standby. These could include the following: Read-only modes/cachesBulkheads that prevent a failure avalanche.Automated circuit breakersDeferred processing These reliable practices ensure there is some form of controlled uptime even when operations seem interrupted. Laser Observability That Proves Reliability In traditional SRE observability, the service boundary is usually the ultimate, but in most modern integrated SaaS platforms, this could be insufficient or worse, dangerous. Operators need to be aware of the actual dependency that is failing, how it is failing (e.g., errors or throttling), and how the failure affects the user experience. Accurate observability for platform-SaaS and API-dependent systems requires these four provisions: Specific dashboard and internal metrics for each vendor.SLI monitoring at the dependency level.Parallel tracing of all outbound calls.Simulation of real-time user experience and workflows. Essentially, whenever there is an emergency, operators should be able to promptly identify whether the source is internal or external. Accuracy and clarity facilitate swift response. Responding to Incidents Without Ownership Another distinct characteristic of modern SRE practice in platform-SaaS is how incidents are responded to. Without ownership, you often cannot debug on your own, roll back a bad deploy, or directly manage other issues. However, you can choose how your system responds by identifying when certain features are disabled, when signals to activate degraded modes are sent, when high traffic is redirected or shed, or when to notify users. To maintain reliability, incident response relies on runbooks to inform decisions. The following questions could help convert the technicality of runbooks to practical solutions: What is the impact on the customer?In what ways can we respond harmlessly?What can we reverse?What should we communicate externally? These questions help resolve incidents, mitigate losses, and intertwine reliability with sound judgment. Is Safety an Illusion in SLAs? SLA providers often readily contract for financial compensation when losses arise, but seldom give absolute reliability guarantees. You may not always expect vendors to consistently meet your availability goals or resolve an avalanche of outages. Safety is a critical consideration when building systems, because when users lose trust in a brand, compensation may not be able to redeem it. Therefore, advanced teams do not consider SLAs as safety nets but as risk pricing. They understand that contractual credits cannot replace trust, brand image, and some almost irredeemable damages. Human Factors in Platform-SaaS and API-Dependent Systems Dependency failures often escalate when cognitive load increases. There could be degraded performance, timeouts without error indicators, partial success, or inconsistent system behavior. Operators may not only focus on machines when dashboards lag or seem to lie. They examine the logs, failure history, or commands. Teams have to design systems with overrides and predictable degradation paths, and observability tools are beyond the failure systems. Reliability goes beyond the correct function of software; it's also about human operations. How Your SaaS and API Platforms Can Imbibe “Good” SRE Practice Effective SRE practices are modern. The following attributes know saas products and API-dependent platforms: Acknowledgment of lack of control very early.Ensuring reliability is embedded in the design.Measuring the outcomes of each SRE criterion or target, instead of just the components.Giving priority to clarity instead of trying to model or control everything because you do not own all the components.Making engineering and operations decisions and products as an integrated whole.Preparing for degradations as inevitable procedures when things fail. Your systems can be reliable if you anticipate failure and accept the reality. Conclusion Modern platform-as-a-service (SaaS) operates in a reliability-without-control manner, leading solid SRE teams to accept that they need to adapt when failures occur. It's simple logic: if you don't absolutely own everything end-to-end, then prepare for the worst: each dependency might fail. It's all about keeping the trust of your users and protecting your brand image.

By Oreoluwa Omoike

From Agile to the Product Operating Model

TL; DR: The “Agile to the Product Operating Model” Survey Results Between August 2 and August 10, 2026, 48 practitioners participated in my Agile to Product Operating Model (POM) survey, which tries to shed light on what is actually changing. Let me summarize the answers for you: the reported transformations change decision-making less than the Cagan framework suggests. Where respondents report improvements, they appear in delivery and collaboration rather than in business results. Unfortunately, the human side of the transition is the least encouraging part of the answers. Thesis: Product operating model transformations mostly change vocabulary and organizational structure while leaving the decision system — who decides what gets built, on what evidence, at what speed- largely untouched. AI is changing product decisions independently of POM transformations. Who Answered, and Why I Report Counts, Not Percentages Of the 48 respondents of the Agile to Product Operating Model, 26 work in organizations that have adopted a product operating model or are moving toward it (we refer to them as the “movers” from here on). The other 22 work in organizations that are not moving; these organizations are still discussing the issue, have decided against it, or have never considered it. So you get absolute counts, and every finding below is directional, not definitive. Additional limitations are: 24 of the 48 respondents are Scrum Masters or Agile Coaches, the roles under the most pressure from this shift.38 of 48 work in organizations with more than 250 people; the sample contains 2 respondents from startups and 2 from scale-ups. Apparently, the people who have participated in the survey are those who still care enough about agile practice to read a newsletter about it; people who left “Agile” entirely are structurally missing. Also, as I did not ask respondents to identify employers, the 48 participants report on their organizations, not 48 distinct verified organizations. Putting it all together, this is a report from inside large organizations, where the product operating model makes its boldest promises. One Fundamental Change in 26 Answers Asked which statement best reflects their experience so far, the 26 movers split like this: 9 say mostly relabeling, the same way of working with new vocabulary9 say real change in some areas, relabeling in others7 say it is too early to tell. Exactly one participant reports a fundamental change in how the organization decides what to build, which is the most interesting learning from the responses: changing the vocabulary, changing the organizational structure, and changing how decisions actually get made are three different things. Better Delivery, Inconclusive Business Results, Worse Morale The Agile to Product Operating Model survey asked movers to compare six outcomes against their previous way of working. Aggregating “somewhat” and “clearly” in each direction: Speed of delivery: Better (9), No change (10), Worse (1), Too early / cannot tell (6)Value delivered to customers: Better (8), No change (8), Worse (2), Too early / cannot tell (8)Business results: Better (3), No change (8), Worse (4), Too early / cannot tell (11)Team motivation and morale: Better (7), No change (4), Worse (9), Too early / cannot tell (6)Developers’ satisfaction: Better (4), No change (8), Worse (7), Too early / cannot tell (7)Collaboration with stakeholders: Better (9), No change (6), Worse (4), Too early / cannot tell (7) The table does not say that the transitions are failing: Delivery speed leans positive: 9 better against 1 worse.Customer value leans positive: 8 against 2.Collaboration with stakeholders leans positive: 9 against 4.Business results are inconclusive, with 11 of 26 unable to say and the rest split. The people dimensions are the only ones that lean consistently negative: morale at 9 orgs is worse than at 7, where it improved; developers’ satisfaction is similar: worse at 7 orgs and better at 4 orgs. Two ways to interpret this information, and the survey does not help to choose: (1) The improvements concentrate on the things enterprises already know how to optimize (flow, coordination, delivery), while the transformations may be falling short of the things the model claims to change most fundamentally. (2) Operational effects precede commercial effects, because business outcomes have longer feedback cycles, and 11 of 26 respondents explicitly cannot yet say. Consider that flow, coordination, or delivery are easier to measure and attribute than commercial effects, which is a significantly fuzzier area. Therefore, the interesting question is how this changes over time, not how it looks at one moment. The range of individual experience behind those aggregates is wide. One respondent, 12 months into adoption at a large financial-services organization, described the hardest part as “surviving the political game that the POM shift has been” and reports worse answers on five of the six dimensions. The other two respondents who have been at 12 or more months report the opposite: better value, better morale, and better collaboration. Three long-tenured accounts cannot agree on whether the practical transition mechanics, culture, affected products and services, or the organization itself makes the difference. (That is the curse of a small sample size.) Agile to Product Operating Model: The Empowerment Gap, and What Settles Uncertainty Cagan’s own test separates empowered product teams from feature teams: in a product team, the team is tasked with solving a problem and owns the solution. In a feature team, “the value and business viability are the responsibility of the stakeholder or executive that requested the feature.” The survey asked movers which pattern operates today, in the middle of their transitions: 9 of 26 report the empowered pattern (leadership sets goals and problems to solve, teams decide what to build)8 of 26 report that leadership decides which features get built and teams implement them.4 of 26 report stakeholder requests are filling the product backlog reactively.4 of 26 say it is genuinely unclear or contested right nowA single one reports teams setting their own direction. So even among respondents whose organizations are actively adopting a model whose entire premise is empowered teams, the “feature-list or reactive-backlog” pattern (12) outnumbers the “empowered” pattern (9). A related question asked what settles it when the organization is uncertain whether something is worth building: 6 said debate and prioritization before anything gets built5 said whoever has the most authority decides5 said things just get built and shipped, and an explicit decision rarely follows. 4 said research evidence5 said it varies too much to say. In total, only five respondents fall into the categories I pre-registered as evidence-led: research evidence, disposable prototypes, or ship-and-measure. There is one case among all the answers: a respondent reports that a disposable AI-assisted prototype reduces build uncertainty: build to learn, decide, and throw it away. That same respondent, 12 months into adoption, also reports better responses across all six outcome dimensions listed above. Of course, one case is not evidence of a relationship. It is a question worth asking in a larger sample, not a claim I am entitled to make here. In Little Code, Big Waste, I argued that cheap AI code removes the cost gate that used to force a should-we-build-this decision, and that “when generating plausible code becomes cheap, every hour spent building the wrong thing becomes waste that can now be produced at scale”. This survey says the prototype-to-decide pattern barely exists yet in this sample. Agile to Product Operating Model Survey Shows: The Two Transitions Run Separately If you believe the keynote version of events, AI is forcing organizations to redesign their operating model around it. The 26 movers report something different. Only one respondent says AI is a core reason for the change; 6 of 26 call it one factor among several; 16 say AI adoption runs in parallel, but separately from, the operating model change; and 3 report little or no role. Now the other side. Among the 22 respondents in non-moving organizations, 15 report that AI is changing how product decisions are made anyway: 6 noticeably, 9 in pockets, without any formal model change. Despite the small sample size, the data support a narrow claim: for most movers in this sample, AI and operating-model transformation are separate initiatives, while for most respondents in non-moving organizations, AI is changing product decisions without any model change at all. Formal operating-model change is neither a prerequisite for AI-driven changes to product decision-making nor, so far, organized around them. The survey did not measure how AI enters these organizations or who authorizes it, so I will not claim more than that. The disconnect between the two is already the finding. My Pre-Survey Hypothesis Scoreboard Here are my verdicts on my four hypotheses, H1-H4, that led to the creation of the Agile to Product Operating Model survey: H1, consultant-driven transitions produce more relabeling than leadership-driven ones: Relatively consistent, but no verdict: 3 of the 4 respondents whose transition is driven by consultants or a transformation office report “mostly relabeling,” against 4 of 16 in product-led or executive-led transitions. But those are just four participants. H2, the empowerment gap: The feature-list plus reactive-backlog patterns (12) outnumber the empowered pattern (9). That is observed in this sample, however, not established beyond it. H3, where AI drives the move, product judgment is the scarcest capability: incorrect on my side: Among the 7 respondents whose organizations treat AI as a core reason or a factor in the move, the most frequent scarcity answer is “delivery capacity is still the bottleneck” (3), ahead of product judgment (2). Across all 26 movers, the scarcity question splits three ways: “stakeholder alignment and decision speed” (7), delivery capacity (6), and product judgment (6). One distinction keeps H3’s rejection from closing the question. The survey measures perceived constraints, and perception rarely reflects reality accurately. AI may have changed the economics of implementation faster than organizations have updated their sense of where the workflow bottleneck sits. Whether delivery capacity objectively remains the constraint is a different question, and this survey cannot answer it. What it can say: practitioners today experience decision speed, delivery, and judgment as roughly competing constraints, and the judgment-scarcity era I anticipated is not the world they report living in. H4, organizations that build without explicit decisions report worse customer value than evidence-led ones: rejected as stated: 1 of 5 build-without-deciding respondents reports better customer value, against 2 of 5 evidence-led ones. Again, the number of replies is too small. Changing the Structure Without Changing the Decision System Three of the findings above belong side by side. Only 1 of 26 movers reports a fundamental change in how build decisions are made. Only 9 of 26 report the empowered decision pattern operating today. Only 5 of 26 fall into the evidence-led categories for resolving build uncertainty. Together, they suggest a more precise diagnosis than “the transformation is theater.” Transformations change three different layers, and the layers move at different speeds: Vocabulary changes first: Product operating model, empowered teams, outcomes over outputs; you get the idea.Structure changes second, and often really does change roles, reporting lines, team topologies, or artifacts.The decision system changes last, if at all: Who decides, based on what evidence, under what uncertainty, with what authority, at what speed. This survey reads like a snapshot of organizations renaming the first layer, reorganizing the second, and leaving the third largely untouched. That is why the 1-of-26 count deserves the weight I put on it earlier. One of the product operating model’s defining promises is a different way of deciding what to build. If only one of the 26 respondents experiences a fundamental change in that mechanism, the question is no longer whether the transformation is proceeding fast enough, but what is being transformed. That mental model offers one possible explanation for the outcome table: structural change could improve coordination and flow before it alters the quality of product decisions. Whether that explains the pattern here is impossible to establish from 26 responses. A long-term study of organizations and involved practitioners would need to test whether decision-system change predicts eventual business outcomes. (Consider that a hypothesis this survey generated, not a result it delivered.) Agile to Product Operating Model: Why Product Washing Is Easier Than Empowerment The survey cannot tell us why the decision system resists change. What follows is my hypothesis, argued from two decades of watching transformations, not a survey result. The tempting explanation is that these organizations are implementing the product operating model badly, and that a proper implementation would deliver. I spent those two decades watching the Agile community run exactly that defense. Every failed adoption was “not real Scrum.” The argument is unfalsifiable, and it taught an entire industry to blame practitioners instead of examining incentives. I will not run the same defense for the product operating model. In November 2024, I described Product Washing: the hollow adoption of product practices that “leaves companies stuck in the same old dynamics but with a new vocabulary,” transformation by reprinting business cards. My hypothesis for the mechanism: product washing is not an implementation failure but what enterprise incentives produce when you ask powerful people to redistribute their own power. The model demands that stakeholders with budget authority hand problem-selection to product leadership and solution-selection to teams. Budget authority is power, and in many large organizations, “product leadership” has become a new title for the same stakeholders who control the money. Meanwhile, middle layers face asymmetric career payoffs: a visible failure damages a career far more than a shared success advances it. Under those payoffs, routing decisions through committees and sign-offs is rational self-protection, and it does not vanish because the org chart was redrawn. Two honest caveats bound this hypothesis: First, in regulated industries, some of that scrutiny is very relevant: when a named person must answer to a regulator, a sign-off chain is accountability, and the respondents from banking and the public sector live with constraints no product operating model erases. The skill worth having is telling required governance apart from accountability theater; most organizations run both and label neither. Second, this survey is a cross-section, not a time series, and two competing explanations fit the same data: The transition hypothesis: decision authority changes slowest of all the layers, and these organizations are simply not there yet.The attractor hypothesis: enterprise incentives pull transformations toward renamed feature factories and hold them there. My incentive argument predicts the attractor. With only 3 respondents at 12 or more months, this survey cannot distinguish between them. That is a testable question for a future survey. The Non-Movers’ Catch-22 The 22 respondents in non-moving organizations deserve more attention than transformation literature usually grants them. Their top reasons for staying put: Leadership sees no need or has other priorities (15),Lacking the product-management maturity to build on (13), andThe cost and fatigue of yet another transformation (10). Only 6 claim their current way of working performs well enough. The dominant non-mover position is not a confident endorsement of the status quo. And inside the reasons sits a genuine Catch-22. Organizations supposedly need the product operating model because their product capabilities are weak, while 13 of 22 respondents say weak product capabilities are precisely why their organization cannot adopt it. A transformation that requires the maturity it promises to create is a hard sell to people who have already survived several that made the same offer. Conclusion: Three Questions for Monday Morning Three questions locate your own organization on this map: First: who decided the last thing your team built, and would your CPO name the same person? If the answers differ, you have found the gap between the model on the slides and the model in operation. Second: what settled your organization’s last genuinely contested build decision: evidence, debate, or seniority? “Whoever has the most authority decides” got 5 votes out of 26 in this survey. Be honest about whether your organization would add a sixth. Third: what would have to be true for a disposable prototype to settle the next contested decision instead? That is a political question, not a technical one: who would have to accept evidence as a tiebreaker, and what would it cost them? Forty-eight answers later, the better question is no longer which operating model organizations adopt. It is how they make decisions when the cost of trying something has collapsed while the cost of deciding has not: Who decides on what evidence, and how quickly can the organization act on what it learns? That is where my work is heading next.

By Stefan Wolpers

CORE

Methodologies

DZone's Featured Methodologies Resources

Monthly Top Methodologies Experts

The Latest Methodologies Topics