DZone Spotlight

Many assume that leadership in software engineering starts only when you stop coding and become a manager. I once shared this belief, thinking technology would be simpler than working with people. That was an early misconception. While it is possible to focus your career on code, architecture, databases, and other technical areas, the challenges that shape your impact become less technical over time. Even the best architectural decision has little value if others do not trust, understand, or support it. This does not mean every experienced software engineer should become a manager. Leadership is equally important on the technical track. Senior individual contributors, such as staff engineers, architects, and principal engineers, are expected to influence decisions beyond their own code. As Will Larson discusses in Staff Engineer, advancing beyond senior engineering focuses on technical leadership rather than people management. To increase your technical impact, others must listen to your ideas, trust your judgment, include you in key discussions, and act on your recommendations. You may choose not to manage people, but avoiding leadership will eventually limit your growth as a software engineer. What Do We Mean by Leadership? Leadership predates corporations, job titles, and management frameworks. For example, the Roman military’s success relied not only on superior weapons or armor but also on effective organization. Legions were divided into smaller units, each with defined responsibilities and led by centurions. Leadership was distributed throughout the ranks, enabling coordinated efforts toward larger objectives. A similar concept appears in the term architect, commonly used by software engineers. Derived from the Greek arkhitekton — arkhi meaning chief and tekton meaning builder — an architect was the master builder, responsible for both understanding the craft and directing others. This role closely resembles that of an effective software architect today. Another example comes from the nineteenth-century Prussian military, which developed its General Staff as a professional body focused on planning, coordination, and operational readiness. This approach recognized that complex organizations require skilled individuals to address critical challenges without making each one the commander. The model became influential and was adopted by other militaries. Software engineering faced a similar challenge: how can experienced engineers expand their organizational impact without moving into people management? An early solution appeared in the British Royal Navy, where managing large fleets required separating command authority from technical expertise. Naval operations relied on both captains and skilled officers responsible for navigation, planning, logistics, and coordination. This staff function supported fleet-level decision-making without direct command and influenced how organizations approach distributed expertise and coordination. Modern software organizations independently adopted a similar approach. Titles such as Staff Engineer, Principal Engineer, and Distinguished Engineer now represent technical leadership roles. Will Larson highlights this distinction in his 2021 book, Staff Engineer: Leadership Beyond the Management Track, which explores Staff-plus engineering as leadership outside the traditional management ladder. This distinction is essential: management is a role, while leadership is an activity. Managers have formal responsibilities for people, performance, hiring, priorities, and processes. Technical leaders may lack formal authority, but their influence comes from expertise, judgment, communication, trust, and guiding better technical decisions. In software engineering, leadership does not require direct reports; it often means being the trusted engineer who provides direction in complex situations. Why Should a Software Engineer Care About Leadership? While understanding leadership is valuable, it is even more important to consider why a Software Engineer who does not plan to become a manager should invest time in developing these skills. As your career advances, your impact depends not only on your technical skills but also on your ability to influence decisions, collaborate effectively, and guide the organization toward better technical outcomes. 1. Software Development Is About People Software development is inherently a social activity. Software is built collaboratively with engineers, product managers, designers, architects, clients, managers, and other stakeholders. Even highly technical decisions must be explained, discussed, challenged, negotiated, or approved by others. A database migration may seem purely technical until it impacts another team. An architectural decision becomes a communication challenge when multiple teams must adopt it. Even an elegant solution can fail if it does not address the client’s real needs. While much of your day may involve working with machines, software exists for people, is created by people, and ultimately serves people. Choosing not to pursue management does not eliminate the human aspect of software engineering. 2. Technical Expertise Needs Trust, Influence, and Access Technical correctness alone is not sufficient. You might understand why a particular architecture will not scale, recognize that a technology introduces unnecessary complexity, identify an important security risk, or propose a significantly better design. However, your expertise has limited impact if others do not listen. Technical expertise leads to organizational impact only when you can influence the organization’s actions. Influence rarely stems from expertise alone. People must trust your judgment, see that you understand the context, listen to opposing views, explain trade-offs clearly, and adapt your position when evidence changes. This is also why relationships matter in a technical career. As your responsibilities increase, many key decisions occur outside the codebase, such as during architecture reviews, design discussions, planning sessions, incident reviews, roadmap meetings, and cross-team or stakeholder meetings. If you want to influence those decisions, you need to be part of those conversations. Leadership helps you build the credibility and trust needed to participate in these discussions and ensures your voice is heard. A strong technical leader does more than provide correct answers. They create conditions where good technical decisions can be understood, challenged, accepted, and implemented. 3. If You Do Not Lead, Someone Else Will Make the Decision When experienced engineers avoid leadership, an uncomfortable consequence arises. The decisions do not disappear. Someone else will make them. And that person may have considerably less technical understanding of the consequences. This can lead organizations to measure engineering productivity using questionable proxies such as lines of code, number of commits, tickets closed, or tokens consumed by AI tools. When engineers encounter such decisions, their natural reaction is often: “Who thought this was a good idea?” A better question might occasionally be: “Which experienced engineers were involved when this decision was made?” Leadership ensures that technical knowledge is represented in decisions affecting engineering. You do not need to control every decision, but you should be willing to participate in the important ones. 4. Leadership Multiplies Your Technical Impact There is a natural limit to how much software one person can build individually. Even exceptional engineers have limited time each day. Leadership enables your expertise to extend beyond those limits. By helping others make better design decisions, establishing reusable architectural approaches, mentoring engineers, improving practices, or preventing costly mistakes, your impact exceeds your individual contributions. This marks an important transition in senior technical careers. Early in your career, your value is largely based on your individual contributions. Later, your value increasingly comes from enabling other engineers and teams to succeed. Your code remains important, but it is no longer the sole measure of your contribution. 5. Leadership Becomes Part of Technical Career Progression Leadership becomes increasingly integral to career progression beyond the Senior Software Engineer role. Staff Engineers, Principal Engineers, Distinguished Engineers, and Software Architects may remain Individual Contributors, yet their responsibilities typically extend beyond implementing individual features. They are expected to provide technical direction, navigate ambiguity, resolve difficult trade-offs, connect teams, mentor engineers, challenge assumptions, and influence decisions whose consequences may extend across an organization. None of those responsibilities inherently requires becoming a people manager. But almost all of them require leadership. Treating leadership as exclusive to management can eventually limit the career growth of experienced Software Engineers. You can choose not to manage people. You can choose to remain deeply technical. As your scope and impact grow, leadership increasingly becomes integral to technical work. Conclusion Leadership in software engineering does not mean leaving the technical path or becoming a manager. It means understanding that software is built by people, and that technical expertise must be paired with trust and influence to drive change. Key decisions need experienced engineers involved, and leadership enables your knowledge to reach beyond your own code. As you advance to roles like Staff Engineer, Principal Engineer, or Software Architect, leadership becomes essential for greater impact. You can remain an Individual Contributor and stay deeply technical, but to increase your influence, you must also guide, communicate, build trust, and help others make better decisions.

By Otavio Santana CORE

Once upon a time, site reliability engineering rested on a linear assumption: monitor more, detect early, and you’ll recover faster. The rise of alert fatigue makes modern SRE teams realize otherwise: Ramadass's (2025) paper, Building an AI-Powered Observability Pipeline for Modern System Reliability, cited research that discovered that: More than two-thirds (82%, actually) of institutions experience alert spikes constantly.Most traditional monitoring tools generate approximately 2,100 alerts daily, with about 70% of them unnecessary and safe to ignore.66% of SRE professionals stated that increased false alerts lead to fatigue, potentially causing them to miss serious issues. How Should We Describe This Situation? Vigilance or Noise? Collaborative systems such as SaaS, third-party APIs, and microservices enhance the degree of observability and notification within systems. Everything is monitored, and occasionally these dependencies may duplicate alerts. When systems request superhuman attention, on-call engineers become fatigued rather than lazy or sloppy. Instead of swift action, alerts are responded to with mistrust. Reliability vs. Experience vs. Metrics Traditional alerting metrics follow traditional reliability practices, that is, error rates, uptime percentages, latency, etc. Although these are essential, they are not actual mirrors of how operators or users experience reliability. Operators may expect reliable alerting to inform decisions, while users may simply define reliability as how well a system enables them to fulfill their intentions. If alerts do not clearly connect to the user experience, there is a gap between detection and action. Over time, the gaps lead to fatigue. On-call engineers begin to “reasonably” ignore these alerts. Why worry over alerts that are not logically related to user outcomes? They may assume. Over time, organizations may end up paying dearly for real issues because alerts were missed or delayed. An On-Call Engineer Experience Here is a typical example of a system design problem an on-call engineer or SRE team may face: 01:15 AM Alert: Latency spikes on a third-party API.01:16 AM Alert: Retry queues are filled.01:16 AM Alert: Timeout alert storms on three dependencies.01:17 AM Alert: Error-rate notification on unrelated endpoints.01:18 AM Alert: Memory and update alerts. And this sequence of alert storms continues, with the on-call engineer receiving more than 20 alerts in just four minutes. The system seems to pass standard observability SRE practice. But what about the long-run reliability suspicions that the bugging signals may create? In this case, the teams are not just grappling with response speed but also with the amplification of confusion when critical alerts are mixed with non-actionable ones. When Detection Outpaces Interpretation We can’t rule out the fact that monitoring in the past decades has taken an advanced leap. And we might be at its cloying stage, where system detection software is outpacing on-call engineers’ interpretation. Systems are “wonder-full” when it comes to identifying when something seems “off.” However, they rarely give explicit descriptions to aid SRE teams’ understanding. An alert can indicate that a queue has exceeded its depth, but may not categorically state whether the issue is temporary or actionable, or whether users are affected. This occurrence spans dozens of dependencies, each with its own signal. The on-call engineer is kept puzzled about the best action to take at the right time. Hence, a reliable response could be excessive caution or delay as the engineer seeks to clarify the situation. The users are negatively impacted. Although the system met technical observability SRE standards, it failed operationally due to its opacity. The Hidden Cost of Alert Overload We rarely see the outcome of alert fatigue overnight. Its effects build up. Delayed response time accumulates. The aftermath incident review loses credibility. Engineers are skeptical of alerts and hesitate to decide first whether they are real or false. The cultural cost of alert fatigue is that on-call roles become a burden SRE teams endure rather than enjoy with a sense of responsibility. In the long run, engineers may feel they have no control over issues due to the confusion that multiple alerts create. Ironically, the same reliability problems that alerts were designed to solve are what they quietly create. Are Alerts Creating a False Sense of Safety? Lots of alerts may seem like a good thing or a sign of strong monitoring at first glance. But here is the truth: alerts could be hiding actual risk. As every deviation is notified, critical and minor alerts blend in. Teams begin to feel alert fatigue and delay response. Then, real problems begin to breed behind the scenes. Remember how SLAs could paint an illusory picture of safety? Similarly, alert volume could do so. Therefore, your SRE team should bind these caveats as the core of their modus operandi. Alerts shouldn’t replace action.Alerts shouldn’t be unsorted (by machines or humans).Alerts shouldn't be discarded. Alerts are signs that our systems need attention, and we should never be tired of listening. SRE Teams Designing Systems that Alert Smartly High-quality systems respond efficiently when dependencies fail. Instead of creating panic, they automatically degrade. SRE teams could design circuit breakers that could inhibit alert storms before they explode. They could also install bulkheads to prevent a single failure from spreading. There could be alert limits and a summary of conditions that resolve the problem of spamming. Instead of relying on metrics, system engineers could set up composite alerts that describe system states. For instance, it’s clearer if a system alert indicates, “Checkout degraded because of latency in payment dependency.” This composite alert is better than 7 alerts that say “Checkout Timeout.” The former shows impact, cause, scope, and urgency. Clarity clears fatigue. Noise does the opposite. Redesigning SRE: Human Reliability That Quells Alert Fatigue We have seen that technical designs may be great, yet other aspects of SRE remain wanting. One such area that could resolve a system design problem is humaneness. To avoid alert fatigue, our design choices must acknowledge human limitations. Therefore, we should accept that some alerts may not require immediate response. Conversely, not every anomaly should trigger an alarm. Understood silence could sometimes be a golden sign that nothing critical is wrong. Advanced SRE teams do not focus on events (or every deviation) but on the states of the system or infrastructure. They are guided by the question: What conditions really impact users, business objectives, or the system's overall health? To achieve this, engineers need to balance product understanding with technical operations. Then they can give a human touch to their designs. Designing systems for human reliability requires a high level of discipline. Site reliability engineers have to continually review, refine, and repair alerts and their trigger commands. Systems are like living organisms that need constant feeding of updates. The evolving nature of alerts could make a helpful one-time alert redundant or harmful in six months. On-Call as a Reliability Interface of SRE No doubt, humans have a role to play in ensuring reliability, but system designs that depend on heroic actions are built not with resilience but with fragility. Reliability is truly achieved when on-call engineers are guided by predefined scripts, models, runbooks, signals, and interfaces. These reduce the tendency to resort to fallible improvisations when issues arise. On-call engineers often take the appellation of “last point of call.” A careful look at their roles shows that they are intermediaries among complex systems, user experience, and consequences. We can thus see that the role of on-call engineers extends beyond problem resolution to stewardship. Conclusion Alert fatigue is a design problem. It often arises when on-call engineers prioritize detection over interpretation, or technical workability over user experience. The dependencies of modern SRE teams make it necessary to align technical alerts with human capability. Alert storms could wear out hardworking engineers who need to take a break. So, system designs need to account for human limitations, recognize that runbooks are better than on-the-spot improvisation, and prioritize clarity over opacity. Designs that account for these factors reduce or eliminate fatigue and preserve the very essence of alerts. In summary, reliability goes beyond resolving many problems to responding to what matters most. When teams can always trust their alerts, they will be more likely to follow up on new cases.

Originally, back-end and front-end Site Reliability Engineering (SRE) were owned by teams. They code the programs, set up databases and infrastructure, and quickly spring to action at the beep of any anomaly. The advent of code vs no-code infrastructure, SaaS, API dependencies, third parties, and other modern systems seems to be eroding this authority. Mainstream and underdog companies now often leverage the significant advantages of outsourcing, collaboration, or delegation, which are usually accompanied by a silent clause: no or partial control. Unlike in previous systems, modern production is largely assembled rather than built from scratch. For example, a conventional SaaS product is built on interdependencies among payment processors, outsourced data infrastructure such as Amazon Web Services (AWS), messaging services, web hosting, design, AI inference APIs, authentication providers like Google, and more. These useful platforms and products are essentially outside teams' control stations, even though they critically impact users' experience. When they function effectively, you share the glory with the platforms. But when there is a system blackout, your users put you on your toes, even though you have no direct access to resolve the problem on time. Therefore, we shall be exposing SRE practices in platform-SaaS and API-dependent systems and how reliability is getting beyond the control of engineering teams and companies. Why Classical SRE Practices May Fail One major downside of SaaS and dependency on external platforms is that reliability control is often assumed to be in a team's hands, whereas it has been bargained. However, teams must reckon with the fact that the case is reversing. For example, traditional SRE models once alleged that: Service Level Indicators (SLIs) focus on availability or internal uptime and latency.Error budgets arise from changes teams make or deploy.Runbooks still suggest that teams can immediately reconfigure or directly work on faulty components. All these are becoming past cases, especially in platform-SaaS systems. You can have a system indicating 99.99% or even 100% uptime on the back end, while new users are struggling to sign up, probably because an authenticator provider is not fully functional. Dashboards and control panels may indicate green, but in reality, third-party payment APIs have been degraded. A New Definition of Reliability in Operating SRE Practices To resolve the new problem in site reliability engineering (SRE), there needs to be a conceptual shift from component health to an integrated, continuous user experience. Therefore, teams need to undergo a paradigm shift away from questions such as "Is our CPU working maximally?" “Is our API up?” “What are the error rates?” Instead, we should inquire: “Are users checking out seamlessly?” “How fast can they authenticate?” “Can they use the SaaS product to perform its key function?” These types of outcome-based questions span interdependent platforms beyond your full control. The login SLI needs to work with the identity provider; otherwise, its output is meaningless. If the checkout SLO skips payment authorization, then it's both fishy and unreliable. True, there may be some internal errors in a reliable system, but what really matters is an integrated multiplatform experience that the user enjoys. Error Budgets? An SRE Practice to Revisit How many teams would love error budgets to disappear when they give up control? But that’s not so. Instead, they are molecularized. When components of your systems are outsourced, the error budget doesn’t just fade away; it is instead transferred to the interdependent platforms. So, it’s better to plan for the fact that SaaS and API providers will consume some of your reliability budget. Doing so keeps you a few steps ahead and protects your business in the long run. Reliable SRE teams make decisions such as allocating part of their error budget to certain dependencies, setting acceptable parameters for degradation, and defining specific steps to take when a dependency exceeds the stipulated budgets. Here’s an example you can adapt: “We will accept payment authorization failure of 0.0% to 0.2% if it is caused by dependency instability. If it goes above that, we will turn on delayed capture or turn off promotions.” This SRE approach keeps you ready for downtime, as your systems automatically switch to planned or budgeted actions rather than relying solely on integrated platforms. What to Do When Failures Beyond Your Control Arise Actually, some failures may seem beyond your control. The more you attempt to resolve them, the more amplified they become. At this point, your team must adapt to the savvy absorption of such situations. Instead of focusing solely on retrial in an SRE approach, your team needs to design its processes and platforms. This could include failing selectively through circuit breakers, failing fast with timeouts, or failing visibly by keeping users informed. Some core settings should always remain non-negotiable and on standby. These could include the following: Read-only modes/cachesBulkheads that prevent a failure avalanche.Automated circuit breakersDeferred processing These reliable practices ensure there is some form of controlled uptime even when operations seem interrupted. Laser Observability That Proves Reliability In traditional SRE observability, the service boundary is usually the ultimate, but in most modern integrated SaaS platforms, this could be insufficient or worse, dangerous. Operators need to be aware of the actual dependency that is failing, how it is failing (e.g., errors or throttling), and how the failure affects the user experience. Accurate observability for platform-SaaS and API-dependent systems requires these four provisions: Specific dashboard and internal metrics for each vendor.SLI monitoring at the dependency level.Parallel tracing of all outbound calls.Simulation of real-time user experience and workflows. Essentially, whenever there is an emergency, operators should be able to promptly identify whether the source is internal or external. Accuracy and clarity facilitate swift response. Responding to Incidents Without Ownership Another distinct characteristic of modern SRE practice in platform-SaaS is how incidents are responded to. Without ownership, you often cannot debug on your own, roll back a bad deploy, or directly manage other issues. However, you can choose how your system responds by identifying when certain features are disabled, when signals to activate degraded modes are sent, when high traffic is redirected or shed, or when to notify users. To maintain reliability, incident response relies on runbooks to inform decisions. The following questions could help convert the technicality of runbooks to practical solutions: What is the impact on the customer?In what ways can we respond harmlessly?What can we reverse?What should we communicate externally? These questions help resolve incidents, mitigate losses, and intertwine reliability with sound judgment. Is Safety an Illusion in SLAs? SLA providers often readily contract for financial compensation when losses arise, but seldom give absolute reliability guarantees. You may not always expect vendors to consistently meet your availability goals or resolve an avalanche of outages. Safety is a critical consideration when building systems, because when users lose trust in a brand, compensation may not be able to redeem it. Therefore, advanced teams do not consider SLAs as safety nets but as risk pricing. They understand that contractual credits cannot replace trust, brand image, and some almost irredeemable damages. Human Factors in Platform-SaaS and API-Dependent Systems Dependency failures often escalate when cognitive load increases. There could be degraded performance, timeouts without error indicators, partial success, or inconsistent system behavior. Operators may not only focus on machines when dashboards lag or seem to lie. They examine the logs, failure history, or commands. Teams have to design systems with overrides and predictable degradation paths, and observability tools are beyond the failure systems. Reliability goes beyond the correct function of software; it's also about human operations. How Your SaaS and API Platforms Can Imbibe “Good” SRE Practice Effective SRE practices are modern. The following attributes know saas products and API-dependent platforms: Acknowledgment of lack of control very early.Ensuring reliability is embedded in the design.Measuring the outcomes of each SRE criterion or target, instead of just the components.Giving priority to clarity instead of trying to model or control everything because you do not own all the components.Making engineering and operations decisions and products as an integrated whole.Preparing for degradations as inevitable procedures when things fail. Your systems can be reliable if you anticipate failure and accept the reality. Conclusion Modern platform-as-a-service (SaaS) operates in a reliability-without-control manner, leading solid SRE teams to accept that they need to adapt when failures occur. It's simple logic: if you don't absolutely own everything end-to-end, then prepare for the worst: each dependency might fail. It's all about keeping the trust of your users and protecting your brand image.

A chat screen looks like a weekend project: a list of bubbles and a text input pinned to the bottom. In React Native, it is one of the hardest things to ship well, because it sits on top of the two most hostile surfaces in mobile development: the software keyboard and a scrolling list that changes size while you're looking at it. We're putting LLMs into everything now, and there is still no good drop-in chat view for React Native. You glue together an aging library with strong opinions, or you hand-roll it. I hand-rolled it. Then I made the LLM stream its replies token by token, and the whole thing fell apart in a way that took a week to understand. This is the story of that break, and the fix, which arrived with suspicious good timing as a library release three months ago. The App I work on an app built around an LLM chat: characters that remember you and reply as an open-ended story unfolds. The messages between the reader and the characters are rendered in a chat-like view: an inverted list, the newest message at the bottom, and a composer pinned above the keyboard. Standard chat anatomy. The twist that makes it hard: the character replies are generated by an LLM, and they stream. Tokens arrive in bursts, a few every hundred milliseconds, with a full reply landing over two or three seconds. Each batch makes the last bubble taller. The list isn't just appending a finished message. It's growing on every frame, while the user might be typing, scrolling, or dismissing the keyboard. That single fact is what turns "I'll just use a FlatList" into weeks of work. Why There's Nothing Good to Reach For The first thing I did was look for a library. The honest state of the art: react-native-gifted-chat is the default answer and it's showing its age. It's opinionated about your data shape, its rendering, and its layout, and fighting those opinions costs more than writing your own.Most "chat UI" packages are really just a styled `FlatList` plus a text input. They solve the easy half and hand you the two genuinely hard problems: keyboard choreography and a live-resizing list.The keyboard utilities that _do_ exist (`KeyboardAvoidingView` and friends) were built for forms, not for an inverted list whose last row is growing while the keyboard animates. So I wrote my own keyboard-and-scroll layer. It was close to 500 lines of KeyboardAvoidingView overrides, manual scrollToOffset calls, listeners on keyboard show/hide events, and offset math to keep the composer glued to the keyboard. It worked, demos looked clean, and I shipped it. The Break: Streaming Meets the Keyboard The bug reports were all variations on "the chat is jumpy." No crashes, just jank. I couldn't reproduce it at first because each of the two features behaved perfectly on its own. The keyboard animation was smooth. The streaming was smooth. The problem only showed up at their intersection. That's the kind of bug that costs a week, because nothing is actually broken. Two correct things are simply disagreeing. Here's what was actually happening. While a character reply streams in: Every batch of tokens makes the last bubble taller.On an inverted list, growing the bottom row shifts the content offset.React Native re-runs the layout to absorb the new height.If the keyboard is open, or worse, mid-animation, my keyboard layer is _also_ adjusting offsets at the same time. Two systems are writing to the scroll position on the same frames. The result: the content jumps, the composer twitches, and if the user has scrolled up to re-read an earlier message, the stream yanks them around. Layout thrash. A steady 60fps collapsed into the low teens precisely when the app is supposed to feel most alive, and on a mid-range Android phone, it was worse. TypeScript // The naive streaming append: looks innocent, thrashes layout. // Every chunk triggers a re-measure of the growing bubble, // which fights whatever the keyboard handler is doing this frame. for await (const chunk of stream) { setMessages((prev) => { const next = [...prev]; next[0] = { ...next[0], text: next[0].text + chunk }; // index 0 = newest, inverted list return next; }); } The streaming itself has its own sharp edges, and they compound the layout problem. Two worth calling out before the fix: React Native's fetch can't stream a response body. There's no response.body.getReader() in stock RN. You reach for an SSE polyfill like react-native-sse or if you're on Expo like me, the streaming-capable fetch from expo/fetch. Pick deliberately. This is the single most common thing people get wrong on day one. TypeScript import { fetch } from "expo/fetch"; const res = await fetch(url, { method: "POST", body, signal: controller.signal, }); const reader = res.body.getReader(); const decoder = new TextDecoder(); // ...read loop, parse SSE frames, dispatch tokens Partial markdown will bite you. Tokens arrive mid-syntax. At some frame, your buffer is literally The dragon turned and **stared with the bold marker opened and not yet closed. A naive markdown renderer will either render the asterisks as literal text or flip half the conversation bold. You need a renderer that tolerates unterminated syntax, or you sanitize the buffer before each render. Cancellation has to be real. The user closes the chat, switches characters, or fires off a new message mid-reply. You need an AbortController whose signal actually reaches the fetch. Skip it and you're billed for tokens nobody will read, streamed into a view that already unmounted. The Fix I was about to rewrite my keyboard layer for the fourth time when react-native-keyboard-controller shipped KeyboardChatScrollView in v1.21.0, on March 16, 2026. It is, as far as I can tell, the first component built specifically for the chat-plus-keyboard problem rather than the form-plus-keyboard one, and it happens to solve the streaming case directly. The piece that matters for an LLM app is built on a ClippingScrollView that provides cross-platform contentInset behavior by extending the scrollable geometry rather than recomputing the layout. That one design choice is why the thrash disappears. The keyboard no longer fights the list because absorbing keyboard height is no longer a layout operation. The props read like a tour of every chat app you've used: keyboardLiftBehavior picks how the content reacts to the keyboard. "always" keeps the latest messages visible no matter where you've scrolled (Telegram, WhatsApp). "whenAtEnd" lifts only when you're already at the bottom, and leaves you alone if you've scrolled up to read history (ChatGPT). "persistent" lifts when the keyboard opens and, unlike the rest, stays put when it closes instead of snapping back down (Claude). "never" lets the keyboard cover the content and moves nothing (Perplexity).blankSpace reserves room for an incoming response while absorbing keyboard height. This is the direct antidote to streaming jank. Instead of the list growing reactively frame by frame and fighting the keyboard, you reserve the space up front and let the tokens fill it.extraContentPadding handles a composer that grows as the user types a long message, without jumping the content.freeze locks the layout during emoji and attachment-picker transitions, the other place chat UIs jump. TypeScript import { KeyboardChatScrollView } from "react-native-keyboard-controller"; <KeyboardChatScrollView keyboardLiftBehavior="persistent" // the Claude pattern: lifts on open, stays put on close blankSpace={pendingReply ? estimatedReplyHeight : 0} > {messages.map(renderBubble)} </KeyboardChatScrollView>; On paper whenAtEnd is the tidy answer for a reading-heavy app: don't move the content out from under someone studying an old exchange. I shipped persistent anyway. So many of my users live in assistant apps that Claude's settle-and-stay behavior is just what their hands expect, and familiarity beat theory. Nobody had to relearn how the chat feels. My streaming loop didn't change. What changed is that the loop is now the only thing touching layout while a reply comes in. The keyboard handler stepped out of the fight. The composer stopped twitching. The user who scrolls up to re-read an old exchange stays put while the character keeps talking below the fold. What I'd Keep, and What I'd Throw Away If I were starting Y/N's chat today, I'd delete my hand-rolled keyboard layer without ceremony and start from KeyboardChatScrollView. The custom code I'd keep is the part that was always mine to own: the streaming reader, the partial-markdown guard, and the cancellation plumbing. Those aren't keyboard problems, and no layout library will solve them for you. The general lesson applies well beyond chat. The expensive bug is almost never one broken feature. It's two correct features interacting on the same frame. My keyboard handler was right. My streaming was right. The week disappeared into the seam between them. When something janks and every part tests clean in isolation, stop testing the parts and go look at what they're both writing to. And the smaller, practical one: the chat box is never the easy part of the app. Budget for it like it's a feature, because it is one. For the first time in a while, you don't have to build all of it yourself. If you've solved the Android side of this, or made partial-markdown rendering feel good while streaming, I'd be glad to compare notes in the comments.

For years, Arm64 was the platform people talked about as a future bet. It was useful in embedded systems, interesting in research, and easy to dismiss as “not the main thing.” That era is over. In a conversation between Dave Neary, Director of Developer Relations at Ampere Computing, and Greg Kroah-Hartman, Linux stable kernel maintainer and long-time kernel developer, the message is clear: Arm64 has become mainstream. It is no longer a special-case architecture. It is a first-class platform in Linux development, deployment, and maintenance. Arm64 Has Become a First-Class Platform in Linux Development Kroah-Hartman’s history with Linux goes back to the late 1990s, when his work in embedded systems led him into kernel development. He started by solving practical device problems, such as getting USB hardware working across many systems. That hands-on work turned into a career built around making Linux more reliable, more portable, and more useful across different hardware. One of the biggest changes he describes is how the Linux community matured. Early on, Linux developers often borrowed ideas from Unix, BSD, and Windows. The goal was to make things function. Over time, Linux moved from catching up to leading. Once that happened, the work became harder. Developers were no longer copying proven models; they were building new infrastructure, new interfaces, and new processes that had to work at scale. That shift also explains why the stable kernel process matters so much. In 2005, Linux moved toward time-based releases and created a stable kernel series focused only on bug fixes. That decision made it possible to keep improving Linux without breaking user space or workloads. For developers, that means a reliable update path. For users, it means confidence that the system will continue to work. Arm64’s growth has made that stability even more important. Today, Arm64 is everywhere: phones, laptops, embedded systems, cloud servers, appliances, and high-performance computing. Linux now runs across all of it. That breadth has changed the ecosystem. When Arm64 breaks, the impact is no longer small. It affects real products and real users across the industry. Upstream Development Improves Arm64 Linux Reliability and Maintainability Kroah-Hartman also highlighted the role of upstream development. The Linux community has long encouraged vendors to work directly on the mainline kernel rather than maintain private patches. That approach saves time, reduces long-term cost, and improves quality. Some vendors learned this the hard way. Others embraced it early and benefited from tighter collaboration with the community. Native Arm64 Testing Gives Kernel Developers Faster Feedback A major practical change for Kroah-Hartman came from using a native Arm64 build server from Ampere. Before that, he mostly tested on x86 and only discovered Arm64 issues later. Now he can build and test Arm64 kernels locally before sending patches out for review. That means fewer mistakes, faster feedback, and less wasted time for everyone involved. The value of that setup is simple: it matches the reality of modern development. Arm64 is no longer a side project. It is part of the core infrastructure of Linux. Native Arm64 tools help developers build better software for the platforms where Linux actually runs. For the Arm64 community, the lesson is direct. Mainstream status brings responsibility. It also brings leverage. The more Arm64 developers work upstream, test locally, and focus on reliability, the stronger the ecosystem becomes. View the full video here: To learn more about Ampere’s developer efforts and find best practices, visit Ampere’s Developer Center and join the conversation in the Ampere Developer Community. Check out the full Ampere article collection here.

Anyone who has carried a pager long enough develops a professional numbness. A queue backs up, a p99 drifts past budget, a deploy does something stupid at 40% rollout. You fix it, you write it up, you go back to sleep. The stakes are real but abstract: revenue per minute, an SLA credit, a churn number on somebody's spreadsheet. That numbness doesn't survive contact with a product people rely on for safety. Picture a regional connectivity degradation after midnight, and an impact estimate on a screen. Normal night, ordinary graph. Then consider what the number is actually counting. For some slice of households in that region, a camera at a front door has gone dark. A parent who checks whether their kid got home. Someone who installed cameras after a break-in and now sleeps better. I want to be careful here, because "our uptime saves lives" is the sort of thing that makes engineers roll their eyes, rightly. Most minutes of most outages harm nobody. But the distribution has a tail, and when a product is somebody's sense of safety, the tail is where the meaning of the work lives. Taking that seriously changes what you measure, what you alert on, and what you're willing to drop under load. Edge Reliability Is a Different Animal Much of my earlier career was conventional infrastructure work with data-center consolidation, phased cloud migrations of several thousand workloads, where most of the energy goes into sequencing risk rather than into any particular technology. Good training in systems thinking. Almost no preparation for how consumer hardware behaves in the wild. Three properties make this class of system unlike a web service. The edge is hostile, and you own none of it. Devices sit on consumer Wi-Fi, behind a cheap router, on an oversubscribed ISP, drawing power that browns out during exactly the storms when people most want their cameras working. You own none of the last mile and all of the customer's expectation of it.Demand is correlated, which breaks naive capacity planning. Web traffic averages out; event traffic from a physical fleet doesn't. A delivery wave or a thunderstorm crossing three states produces millions of events inside the same few minutes. Independent load is easy. Synchronized load is what pages you.Failure costs are wildly uneven. A dropped analytics event is a rounding error. A live view that spins for eight seconds while somebody stands on a porch is arguably worse than an outright failure, because the customer sat there and watched the product not work. If not all errors cost the same, they shouldn't page the same, yet nearly every monitoring setup I've encountered treats them identically. Together, those produce the failure mode that should drive the whole design: you can be green on CPU, memory, and 5xx rate while customers are having a red night. Define "Up" in the Customer's Language The first fight in this kind of environment is about vocabulary. "The service is up" tends to mean "the servers are up," and those are not the same claim. The alternative is SLIs built on journeys a person can perceive. For a home-security-shaped product, that's notification latency from edge event to push landing on the phone, live-view time-to-first-frame, and clip availability and retention. The second one is where I'd push hardest, because there's a tempting shortcut. "Session established" is easy to measure and usually already instrumented. It's also a lie, since a session can establish and then deliver nothing watchable for another three seconds. Move to first rendered frame, and your numbers get noticeably worse — which is how you learn the old metric was flattering you. An objective in that shape reads roughly: 1SLI: proportion of device events whose push notification is2delivered in <= 4s, measured end to end.3SLO: 99.5% of events over a rolling 28-day window. Two choices there matter more than the threshold. Use rolling windows rather than calendar months, because calendar boundaries teach teams to hold their breath until the first and then ship anyway. And give the error budget real authority: burn it, the deploy freeze happens, no case-by-case negotiation with whoever has the loudest roadmap. That second one is where program-management discipline earns its keep more than any architectural decision — governance, entry and exit criteria, named decision rights. It sounds like bureaucracy right until the first freeze holds without anyone having to win a political argument. One test worth applying to every proposed SLO: if breaching it wouldn't change what anybody does next week, it isn't an SLO. It's a dashboard. Measuring Is Harder Than Target-Setting Picking four seconds is easy. Knowing whether you hit it is hard, because the clock starts on a device you don't control and stops on a phone you don't either. The approach that works is correlation across the full path. The device stamps an event ID at capture, and it rides every hop — ingest, media pipeline, notification service, push provider, client ACK. Each hop logs the shared ID with a timestamp, and you reassemble the journey afterward. In Splunk, that looks something like: SQL index=device_events (stage=capture OR stage=push_ack) | stats earliest(_time) as t_capture, latest(_time) as t_ack, values(region) as region by event_id | eval e2e_latency_s = t_ack - t_capture | where isnotnull(t_ack) | eval met_slo = if(e2e_latency_s <= 4, 1, 0) | stats count as total, sum(met_slo) as ok, perc99(e2e_latency_s) as p99_s by region | eval attainment = round(100 * ok / total, 3) | sort - p99_s Unglamorous, and it answers the only question that matters four minutes after waking up: are real people getting notifications on time, and if not, where. Sorting by regional p99 turns "something feels slow" into a location. Now look at the isnotnull(t_ack) filter, because it's the most instructive line in the query. It quietly excludes every journey that never completed, which is the worst outcome for a customer. That exclusion is how an entire class of failure hides in plain sight. The Failure Class That Never Pages You The scenario I'd most want a team to design against is the one that generates no alert at all. A subset of devices with one hardware revision, one firmware version, one specific reconnect path stops delivering notifications while continuing to report healthy. Capture succeeds. The event enters the pipeline. It simply never produces an ACK. Every aggregate metric absorbs it. The affected population is small enough that regional p99 doesn't budge, attainment stays inside budget, and no error rate moves, because nothing errored. Detection ends up coming from a cluster of support tickets, which is the most expensive monitoring you can buy: it means your customers are doing it for you. Two defenses follow directly. Alert on capture events with no matching ACK inside a window, as its own signal rather than folded into latency with slow and failed have different runbooks and different customer meanings. And break tier-1 attainment out by device model and firmware version, accepting the cardinality cost, because a well-behaved aggregate is very good at hiding a badly-behaved cohort. Averages don't just lie about the tail. They lie about who's in it. Running the Room Detection without a fast, calm response is just expensive telemetry. Three things I'd insist on anywhere. Separate the Incident Commander from the person fixing it. Those are genuinely different jobs, and you can watch the debugging degrade in real time when one person does both while fielding stakeholder pings. It's the cheapest reliability improvement available and costs zero engineering hours. Define severity by customer impact, never by component. An internal dashboard degrading is a SEV-3 no matter how loudly its owner complains. Notifications delayed across a region is a SEV-1 immediately, because the product's core promise is broken. Writing that down ends a lot of arguments before they start. Run game days, and expect the first one to be humbling. In my experience, the runbook is wrong in several places, at least one dashboard fails to load under load, and finding the person who can trigger a manual failover takes longer than anyone predicted. By the fifth game day it's boring, which is the entire point. What I'd Tell My Earlier Self Write SLOs in the customer's language and give the budget teeth. Instrument the journey with a correlation key, because component metrics tell you what broke while journey metrics tell you what the customer felt. Decide in advance what you shed under load, and that judgment is too important to make at 3 a.m. Break tier-1 metrics out by cohort, because aggregates hide the people you're failing. And separate the commander from the fixer. The stack will keep moving: more inference at the edge, new codecs, whatever replaces today's push mechanics. The operating philosophy doesn't change. Measure what people actually experience, protect the moments that matter, fail gently, respond as it counts. The best feedback this work gets is silence.

Every few weeks, someone on my team, or in a client meeting, asks me the same question: "Which cloud should we use for our AI workloads?" I have been building enterprise integrations for over fourteen years now, and lately most of my time goes into RAG pipelines, vector databases, and agentic orchestration on top of these platforms. So I get this question a lot, and honestly, there is no single right answer. The right cloud depends on where your data already lives, what your compliance team will accept, and which models your architecture actually needs. In this article, I want to walk through the three big players, AWS Bedrock, Google Vertex AI, and Microsoft Azure AI Foundry, and share what I have learned working with these platforms in real enterprise settings, not just from reading marketing pages. AWS Bedrock Bedrock started as a model marketplace back in 2023, and it has grown into a full platform with Guardrails for content filtering, Knowledge Bases for RAG, and AgentCore for building agentic workflows. What I like most about Bedrock is the sheer breadth of models available behind a single API. You get Claude from Anthropic, Llama from Meta, Mistral, Cohere's Command models, and Amazon's own Nova family, all through one consistent interface. If your architecture needs to swap models without rewriting your integration layer, Bedrock makes that easier than the other two. Pros: Broadest model catalog of the three, so you are not locked into one vendor's models.Strong identity and governance story if you are already running on AWS, since it plugs directly into IAM, CloudTrail, and Macie.Bedrock is one of the few places where you get Claude with enterprise indemnification, which matters a lot when legal teams get involved.Provisioned throughput options give you predictable latency for production workloads that cannot tolerate spikes. Cons: If your organization is not already AWS-native, the onboarding curve is steeper than it looks.Cross-cloud portability is basically nonexistent. A model you fine-tune on Bedrock does not export cleanly to Vertex AI or Foundry. That is a real switching cost you should plan for on day one, not something to figure out later.Some of the newer agentic tooling is still maturing, so documentation gaps show up more than I would like. Google Vertex AI Vertex AI feels different from the other two because Google's DNA here is research first. If your team cares about multimodal capability, or you want access to Gemini models the moment they ship, Vertex AI tends to be ahead. It is also the strongest option if your data already lives in BigQuery, because the integration between Vertex and BigQuery for feature engineering and MLOps pipelines is genuinely smooth. Pros: Best fit for teams doing custom model training, not just calling a hosted API. AutoML and the broader MLOps tooling cut training time noticeably compared to the other two.Tight coupling with BigQuery is a huge advantage if your organization already runs its analytics there. You avoid a lot of data movement overhead.Gemini-first multimodal workflows, plus Google Search grounding for agents, which is something neither Bedrock nor Foundry offers natively.TPU support gives real throughput advantages for heavy batch processing. Cons: If your organization is not GCP-centric already, the value proposition weakens fast. You end up paying a data-gravity tax to move information into Google's ecosystem.Governance and compliance tooling, while solid, is not as battle-tested across regulated industries as AWS's certifications.The agent ecosystem, while improving, still trails Bedrock's AgentCore and Foundry's Azure AI Agents in terms of enterprise adoption stories I have personally seen. Azure AI Foundry Foundry, formerly Azure AI Services, is Microsoft's rebranded and expanded platform, and it is the one I have written about before because it is what my own recent client work has centered on. If your enterprise already lives inside Microsoft 365, Entra ID, and Azure infrastructure, Foundry removes almost all of the identity and governance friction you would otherwise deal with. That matters more than people expect once you are past the proof of concept stage and into actual production rollout with security review. Pros: Deep Microsoft 365 and Entra ID integration means your existing enterprise approvals and identity workflows extend naturally into your AI layer.Strong OpenAI-led model access, since Microsoft's partnership with OpenAI gives Foundry early and deep access to GPT-family models.Hybrid deployment options are genuinely better here than on the other two platforms, which matters if you have on-prem systems you are not ready to fully cloud-migrate.Roughly three-quarters of Fortune 500 companies already run on Microsoft's stack, so for a lot of enterprises Foundry is simply the path of least resistance. Cons: Model breadth is narrower than Bedrock's catalog, so if you need a specific non-OpenAI model family, you may find yourself stitching together a secondary platform anyway.Because it is tied so closely to Azure compute pricing, cost predictability requires more upfront modeling than teams expect.Some newer agentic and orchestration features are still catching up to what AWS has shipped with AgentCore. So Which One Should You Actually Pick? Here is the honest answer I give in client meetings: do not choose based on a benchmark screenshot or a features table. Choose based on where your data already lives and where your governance and compliance story already works. If you are AWS-first and want maximum model flexibility, go with Bedrock. If you are Microsoft-heavy and need your AI layer to inherit existing Entra ID and 365 approvals without a fight, Foundry is the path of least resistance. If your analytics already lives in BigQuery and multimodal Gemini capability is core to your roadmap, Vertex AI earns its place. What I am increasingly seeing among the teams I work with is a hybrid pattern. A primary cloud handles the bulk of regulated workloads, and a secondary cloud gets called in only when a specific model family is not well supported on the primary platform. It is not the cleanest architecture on paper, but it reflects how fast this space is still moving. None of these three platforms is standing still, and the leader on any given feature this quarter is not guaranteed to hold that spot by next year. My suggestion, whichever cloud you land on: build your RAG and orchestration layer with enough abstraction that swapping the underlying model provider is a configuration change, not a rewrite. That single decision will save you more pain than picking the "right" cloud ever will.

It started as a fleeting thought while I was heads-down building agentic AI systems: somewhere between "just call the API" and "let's train our own model," we've quietly ended up with three completely different ways to solve the same problem. Most teams treat that as a single decision, made once, early, and never revisited. It isn't. It's a portfolio you manage for the life of the product. Here's the framework, and why I think most teams have the sequencing backward. The Three Tiers 1. Model API reliance. You call the frontier model, Claude, GPT, Gemini, whichever lab is ahead this quarter, and let its R&D absorb the part of the problem you don't understand yet. This is the right default when you genuinely don't know the shape of the task: when "correct" is still being defined, when volume is low, when the fastest way to learn is to ship and watch what breaks. 2. Fine-tuning open-source models. Once a use case turns out to be repeatable, same shape of input, same shape of output, high enough volume that you're paying real money for it every month, you stop renting intelligence and start owning it. You fine-tune an open-weight model on your own data. You don't have to chase every new open-source release to stay current; you can do this on a slow, deliberate cadence while gradually weaning that specific use case off the frontier API. 3. Migrating to declarative software. Eventually, for the use cases you understand well enough, you don't need a model call at all; you need code. Once you've mapped the edge cases, you write the deterministic pipeline: rules, retrieval, control flow, maybe a small model bolted onto the one genuinely ambiguous step. This is the least glamorous option and the most durable one: reliable, cheap, testable, and not a black box. Why This Feels Backward (and Why It Isn't) Andrej Karpathy's "Software 3.0" framing has been everywhere in AI circles since his 2025 "Software Is Changing (Again)" talk: software moved from Software 1.0 (humans hand-write code) to Software 2.0 (humans train neural network weights) to Software 3.0 (humans write natural-language prompts, treating the model itself as a new kind of programmable computer, with everything in its context window acting as the program). At the frontier, that arc is real; natural language keeps unlocking categories of software that used to require a full engineering team. But zoom into any single feature inside an actual product, and the maturity curve runs the other way. You start at 3.0, a prompt against a frontier model, because that's the fastest way to find out if the idea works at all. Once it works and repeats, you climb down to 2.0: weights you own. Once you fully understand it, you climb down further to 1.0: code you can read. Both arcs are true at the same time. Karpathy's arc is about what becomes possible. This arc is about what becomes worth hardening, once you've learned the actual shape of the problem. The frontier keeps pushing the ceiling up. Underneath it, mature teams keep pushing their own floor down. The Receipts This isn't just a personal theory; it's showing up everywhere once you look for it. Stanford University's DSPy framework is this pattern turned into an actual engineering discipline. Instead of hand-tuning prompt strings forever, you write a declarative "signature" of what a step should do, and a compiler decides, and re-decides, every time the underlying model or data changes, whether that step should run as a prompt, a set of few-shot examples, or fine-tuned weights. The program is code. The model call becomes just one swappable implementation detail inside it. Token prices, meanwhile, keep collapsing. One 2026 analysis of pricing across hundreds of models estimated something like a 600x drop in token costs since 2020, with cheaper model tiers now halving in price faster than Moore's Law ever moved. That actually complicates a naive cost argument for fine-tuning low-stakes, high-volume tasks; the API might already be close to free. What fine-tuning and code increasingly buy you isn't just savings; it's control, latency, and moat. Specialization keeps beating generality on narrow, well-defined tasks. A recent study on structured contract extraction found domain-trained small models matching or beating frontier general-purpose LLMs, at a fraction of the cost and deployable entirely inside enterprise infrastructure. That's tier 2, working exactly as advertised. And not everyone agrees on the timing, which is worth holding onto rather than smoothing over. Some sharp voices in AI investing argue the opposite case: frontier labs will keep out-improving your custom fine-tune faster than you can maintain it, so unless you're sitting on genuinely proprietary data, the better bet is to keep riding the API and pour your effort into the product wrapped around it. That's a real, unresolved tension. It's exactly why this is a portfolio decision and not a fixed rule. The Part Nobody's Actually Managing Here's what I think most roadmaps get wrong: this isn't three sequential stages for your product. It's three tiers running simultaneously, for different capabilities, all the time. Your onboarding flow might already be sitting at tier 3 because you nailed it a year ago. Your newest agentic feature is at tier 1 because you shipped it three weeks ago and don't know its failure modes yet. Something in the middle just crossed the volume threshold where fine-tuning finally pays for itself. That's not a one-time build-vs-buy fork. That's a resource allocation problem, a live one, shifting every quarter as usage patterns, model prices, and your own understanding of the task all move independently of each other. Most AI roadmaps are still built like it's a single decision made once at kickoff. A few questions I've found useful for figuring out where a given capability actually belongs: How often does it run? Low volume, sporadic — stay on the API. The fixed cost of owning it isn't worth paying yet.Is "correct" still moving? If your own definition of a good output changed last month, don't freeze it into weights or code. You'll just have to redo the work.Could a competitor replicate this with the same API call you're making? If yes, it was never your moat. Don't over-invest in owning it.What's your tolerance for a black box? Audit, compliance, and debuggability needs can pull a capability toward code even before the economics demand it.Do you actually have the data? You can't responsibly fine-tune or hard-code what you can't yet describe with real, labeled examples. Where This Leaves Us Having three ways to solve a problem instead of one is genuine abundance. A few years ago, "write the code yourself" was the only option on the table. That's insane! But abundance isn't free; it converts every roadmap into a standing allocation problem: what stays on the frontier, what gets pulled in-house, what gets frozen into something boring and reliable. Decided over and over, forever, as the ground shifts under all three tiers at once. Which of your product's capabilities do you think is sitting at the wrong tier right now?

Dynamic testing is essential because it uncovers vulnerabilities in running applications. But while SAST gets the attention because it’s shift-left and relatively straightforward to fix, DAST often gets stuck in the backlog. Application security testing generally splits into two approaches. SAST (static analysis) scans source code before it ever runs, catching issues while a developer is still in the file, which is why fixes tend to happen fast. You're editing code you just wrote, with full context on what it does and why. DAST (dynamic analysis) works differently. It tests an application while it's running, sending real requests at live endpoints to see what breaks, the same way an attacker would probe it from outside. That's what makes DAST so valuable. It catches vulnerabilities that only show up in production behavior, not in the code itself. But it's also what makes DAST findings harder to act on. A SAST finding points to a file and a line. A DAST finding might simply point to a URL that returned something it shouldn't have, with no direct link back to the code that caused it. That gap is why DAST findings so often stall in the backlog while SAST findings get resolved first. Let’s break down how developers can make DAST findings behave less like alerts and more like bug reports they can actually work on. A Finding Needs Repro Evidence, Not Just a Vulnerability Name A vulnerability name alone is not very useful. What matters are the details and the ability to quickly reproduce the issue. A developer needs the request, the payload, the vulnerable parameter, and the auth context it ran under. It’s important to hand the finding over as a request the developer can run, not just a description they have to read. A HAR file captures the full request and response cycle. A working curl command lets someone fire off the exact same request from their terminal and watch it fail the same way. Either format turns a mere alert into a bug report worth acting on. A Finding Doesn't Know Who Owns It A DAST finding lives at the network layer. It knows the endpoint that responded and the payload that broke it. On the other hand, DAST is unaware of which repository owns that endpoint, or which team gets paged if it breaks. It’s important to remember here that DAST works by hitting an application from the outside, in the same way that an attacker might, so it was never going to have visibility into the underlying codebase. If you can’t bridge that gap, then a finding just sits there, because nobody can confirm it's theirs to fix. The solution comes from correlating the runtime finding with a repository and a code path. Pairing DAST with SAST data helps, since SAST already has the codebase mapped, though the correlation is rarely perfect. A monorepo or a shared service layer can leave 'this endpoint belongs to this team' genuinely unclear, no matter how good the integration is. API inventory data helps too, tying live endpoints back to the services behind them, but it depends on that inventory being kept current, which not every team manages well. The finding still won’t be actionable on its own. But if you manage to turn "this URL is vulnerable" into "this line, in this repo, probably owned by this team," then you’ve effectively given a developer a starting point from which they can actually work. A Severity Score Doesn't Tell You If Anyone Can Reach It Knowing where a finding lives and who owns it still doesn't guarantee anyone acts on it. A developer with a backlog full of feature work needs a reason to bump a security fix ahead of everything else, and a severity label alone rarely makes that case. CVSS scores describe how bad a vulnerability might be theoretically, based on the vulnerability itself, but they do so without any context into the environment in which the vulnerability sits. A 9.8 score on an endpoint that isn't internet-facing and requires authentication hardly deserves the same attention as a 9.8 that's wide open. Indeed, the score alone can't tell a developer which type of situation they're facing. The catch is that this context isn't always so easy to attach. Someone has to actually know the app's architecture well enough to know what’s actually reachable — parameters that are often under-documented, especially as apps undergo so many dynamic changes. When reachability context is missing, the honest move is to flag the finding as an unverified exposure rather than allow an outdated assumption to drive a prioritization decision. Getting this right matters more than getting to it fast, since a developer who acts on bad exposure data once will start ignoring the context field altogether. Findings Die in Dashboards Developers Never Open A finding can have perfect reproduction steps, a clear owner, and full exploitability context, and still go nowhere if it's sitting in a security dashboard the developer never logs into. Not to imply that this is a discipline problem. Developers simply prefer to work out of their own backlog – Jira, Linear, whatever the team uses – and a separate security tool is one more login, one more context switch, one more thing to remember to check. Ideally, findings should land automatically in the same place developers already work, whether that's a ticket created through the platform's API or a message in the team's Slack channel. Prioritizing these pushes can be challenging to nail down as well, because auto-creating a ticket for every low-severity finding just adds noise to a backlog, effectively training developers to ignore the security label entirely. Opening a dev ticket works best when it's reserved for critical and high-severity findings. Lower-severity findings are often better left in a queue that gets triaged in batches. A Fix That's Never Retested Is Just a Guess The final step in remediation is confirming that the fix actually worked. Ironically, it's the step most likely to get skipped. All too often, a developer closes the ticket, moves on, and nobody circles back to check whether the underlying request still fails the same way it did before. The fastest way to check is to simply rerun the exact request that triggered the finding in the first place. This can take place either as a one-click retest button or as an automated check upon the next deploy. No matter which method you use, the developer shouldn't have to manually rebuild the original request from memory or dig back through the ticket to reconstruct what to test. Retesting is also where a false sense of progress might creep in. A finding that stops firing isn't necessarily a finding that's been fixed – the endpoint could have moved, a WAF rule could be masking it, the auth flow could have changed in a way that makes the original payload irrelevant without addressing the underlying flaw. A clean retest is useful as a signal, not proof, and treating it as automatic closure is how vulnerabilities quietly resurface months later under a different path. Conclusion DAST finds real vulnerabilities; they’re just harder to act on. But that doesn’t make it acceptable to ignore them. Reproduction evidence turns an alert into something the developer can run and test themselves. Mapping a finding to its repo turns it into an assigned task. Exploitability context gives it urgency. Routing it into Jira or Slack gets it seen. Retesting proves that the fix has taken hold. With a few tweaks in how findings are created and reach developers, critical vulnerabilities that surface during dynamic testing can finally get the attention they deserve.

By Philip Piletic CORE

In Part 1, I walked through how AI is changing incident response, from correlation engines like BigPanda and PagerDuty's AIOps features to a newer category of dedicated AI SRE agents like Traversal, Resolve.ai, and Cleric that investigate incidents autonomously instead of just clustering alerts you already collected. Incident response gets the spotlight because it's the loudest, most visible part of the job. But if you actually track where an SRE's week goes, a good chunk of it isn't firefighting at all. It's ITOps tickets, chaos testing, SLO math, on-call scheduling, and the slow grind of writing and maintaining runbooks nobody reads until 3 a.m. This second part covers where AI is showing up in all of that, with the same rule I applied in Part 1: vendor-reported numbers get flagged as vendor-reported, and I say plainly where adoption is still low regardless of how good the tooling has gotten. ITOps: A Slower But Real Shift ITOps has been slower to change than incident management, partly because the data is messier. CMDB entries are stale, ticket categories are inconsistent, and a lot of ITOps work still runs through change advisory boards that move at the speed of a Tuesday meeting. Even so, a few areas have real AI traction: Predictive capacity planning. Rather than static thresholds ("alert at 80% CPU"), some platforms now model usage trends and flag capacity issues days before they'd trip a traditional threshold.Automated ticket triage and routing. Classifying a ticket and routing it to the right queue used to be a rules engine with hundreds of brittle conditions. Language models handle the free-text classification part noticeably better.Change risk scoring. A few platforms now score proposed changes against historical incident data to flag "this type of change caused an outage 3 of the last 20 times." Useful as a second opinion, not a replacement for review. I covered the automation side of this shift, provisioning and managing the infrastructure these ITOps tools sit on top of, in more depth in Infrastructure as Code: How Automation Evolved to Power AI Workloads and Cloud Automation Excellence: Terraform, Ansible, and Nomad for Enterprise Architecture. ITOps AI features are only as good as the infrastructure state they're reasoning about, and that state is usually managed by exactly these kinds of tools. Beyond Incidents and Tickets: The Rest of the SRE Job Incident management The point of this diagram is that the reactive stuff on the left only gets easier if the proactive and human-layer work on the right actually happens. A great incident agent bolted onto a team with no SLOs and a runbook wiki nobody's touched in two years will still struggle. Chaos Engineering Gets a Reasoning Layer Chaos tools like Gremlin, Steadybit, and Harness's chaos engineering module used to require someone to manually design experiments: pick a service, pick a failure mode, guess a reasonable blast radius. That design step is where AI is actually helping now. Harness added generative capabilities that analyze your architecture and operational data to suggest which experiments would teach you the most, instead of you guessing. Steadybit went further and shipped what it calls the first MCP server built for chaos engineering, letting LLM agents query past experiment results directly. That's the same protocol-level pattern I wrote about in MCP vs Skills vs Agents With Scripts: giving an agent a standardized way to query a tool's data instead of scraping a dashboard. In practice, it means an incident investigation agent could eventually ask "have we ever tested this failure mode before" and get a real, structured answer instead of nothing. Worth saying plainly: adoption here is still low industry-wide. Independent research from LogicMonitor's 2026 SRE Report found that resilience engineering is widely valued on paper, but production chaos testing remains uncommon, and many organizations still have low tolerance for deliberate failure injection. AI lowers the design cost of running an experiment, but it doesn't fix the organizational nervousness about deliberately breaking things, and that's a culture problem no agent solves for you. SLOs and Error Budgets Get Easier to Set Up, Not Easier to Enforce Tools like Nobl9 have leaned into AI mostly at the setup stage: pointing at your existing observability data and proposing a reasonable SLO instead of making you guess a number out of thin air, and flagging when a service's error budget burn rate suggests you should stop shipping features and go fix things instead. That second part, "should we stop shipping," is still an organizational decision no tool makes for you. What AI changes here is the friction of getting from zero SLOs to a defensible first draft, which used to take a workshop and a spreadsheet and increasingly takes an afternoon. On-Call Scheduling and Toil Reduction This is the least flashy category and probably the most immediately felt by individual engineers. PagerDuty, Opsgenie, and similar tools have had smart scheduling for years, balancing load and skipping people near PTO. What's newer is toil-specific analysis: some platforms now scan a team's ticket and page history to flag which recurring alerts are pure noise versus which ones represent real, fixable problems, and rank them by engineer-hours wasted. It's a small feature compared to an AI SRE agent doing live root cause analysis, but for a burned-out on-call rotation, "here are your top five noisiest alerts by wasted hours" is sometimes the most useful report in the whole stack. Runbook and Knowledge Management The unglamorous truth about most incidents is that the fix was already documented somewhere, if anyone could find it. This is turning into one of the more genuinely useful applications of retrieval-augmented generation in the SRE space: instead of an engineer grepping a wiki during an active incident, a chat interface pulls the relevant runbook section, the last few times this alert fired, and who fixed it, in one query. Datadog's Bits AI, ServiceNow's Now Assist, and most of the dedicated AI SRE agents from Part 1 all lean on this pattern. The quality ceiling here is entirely set by how good your existing documentation is, which brings us back to a point worth repeating: these tools reward teams that already write things down. If your team is still deciding how to organize that knowledge layer for agents to query safely, that's exactly the ground covered in Trust No Agent: How to Secure Autonomous Tools on Your Machine. Capacity and Cost Optimization I touched on predictive capacity planning under ITOps, but it deserves a wider frame. A lot of what used to be manual FinOps work — right-sizing instances, catching orphaned resources, forecasting when a service will outgrow its current tier — is now a background AI process in platforms like Datadog, Dynatrace, and the major cloud providers' own cost tools. For SREs, the payoff isn't glamorous, but it's real: fewer capacity-related pages, because the system flagged the trend three weeks before it became an incident instead of after. What All of This Adds Up To for the SRE Persona Put the incident agents from Part 1 together with the chaos assistants, SLO copilots, and runbook retrieval from this part, and the actual shift in the job looks less like "AI does SRE work" and more like this: TaskBeforeNowWhat the SRE Still OwnsInvestigating an incidentManual dashboard hoppingAgent proposes root cause with evidenceValidating the evidence, deciding the fixDesigning a chaos experimentManual guesswork on blast radiusAI suggests high-value experimentsDeciding organizational risk toleranceSetting an SLOWorkshop, spreadsheet mathAI proposes a data-backed draftDeciding what the business actually needsFinding the right runbookWiki search during a live pageChat interface surfaces it in secondsJudging if it still appliesScheduling on-callManual rotation and swapsAI balances load, flags toil hotspotsDeciding if the rotation itself is sustainableForecasting capacityManual trend-watchingAI flags the trend earlyApproving the spend The pattern repeating across every row: AI is good at surfacing options and drafts, and still bad at owning the judgment call that has actual consequences. That's not a limitation to apologize for. It's the correct division of labor for now, and probably for a while. What I'd Actually Recommend If you're evaluating tools for your team across either part of this series, a few things I've learned the hard way: Fix your data before buying a tool. Correlation and retrieval are only as good as your alert taxonomy and postmortem history. A brilliant model over garbage data still gives you garbage.Don't let AI auto-remediate anything you haven't tested extensively. Suggestion is fine. Auto-restart-the-production-database is not, unless you've earned that trust over months. This applies doubly to the AI SRE agents from Part 1, most of which default to read-only for exactly this reason.Budget time for the writing habit, not just the tool. Auto-drafted postmortems only help if someone still reviews and improves them. Teams that treat the draft as final start losing institutional knowledge fast.Pilot on one team first. ITOps and chaos engineering rollouts especially tend to get sold org-wide before anyone's tested them against your actual ticket mess or your actual appetite for deliberate failure.Ask what the agent actually queries, not just what it outputs. For the AI SRE agent category specifically, the evidence trail is the product. If a tool can't show you exactly what it checked before proposing a root cause, treat the proposal as a guess with good formatting. Where This Is Headed I keep going back and forth on how much further this goes. The correlation and drafting gains from Part 1 are real, and I use them daily now, and the newer agent-based investigation tools are the first thing in a while that's actually changed how fast I can get from "page fires" to "I know what broke." The proactive side covered here — chaos experiment design, SLO drafting, toil analysis — is quieter progress, but it's the kind that compounds: every noisy alert an AI flags and a team actually fixes is one less 2 a.m. page for good, not just a faster resolution of the next one. But there's a gap between "explains what probably happened" and "understands the system well enough to fix novel failures," and I don't think that gap closes with a bigger model. It closes with better telemetry, better documentation, and engineers who still know how to read a stack trace without an assistant summarizing it for them. If you want a broader look at the open-source side of that telemetry and tooling layer, I put together a rundown in Open-Source LLM Tools Worth Your Time and Developer Tools That Actually Matter in 2026. If your team is evaluating AI features anywhere in your reliability stack this year — incident response, ITOps, chaos testing, or SLO management — start by asking what data problem it's solving, not what model it's built on. The model is rarely the bottleneck. Your alert hygiene, your documentation, and your SLO coverage almost always are.

By Vidyasagar (Sarath Chandra) Machupalli FBCS CORE