DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library
  1. DZone
  2. Events
  3. Video Library
  4. Fix the Underlying Issue: One Agent, Live, Through a Full Incident Lifecycle

Fix the Underlying Issue: One Agent, Live, Through a Full Incident Lifecycle

Details

Expert-Led Takeaways

  • Shifting From Reactive to Preventive Operations: While typical production monitoring tools focus solely on quick alert routing and reactive handling, the platform leverages scheduled agent tasks to catch silent regressions—such as backend schema mismatches or expiring certificates—before they trigger a critical user-facing incident or middle-of-the-night alert queue flood.

  • Traceable Causal Chains Over Log Guesswork: Instead of dumping an engineer into raw, uncontextualized system log fragments during a live failure, the agent programmatically links external service expectations (e.g., HubSpot backend payloads) to frontend application code deficiencies. It maps out the exact dependencies to form a clear root cause analysis.

  • Continuous Off-Peak Optimization: Beyond isolating and staging fixes for live software defects, the operations agent runs ongoing background routines like cloud cost optimizations. This ensures infrastructure efficiency is evaluated consistently between active downtime incidents rather than being treated as an intermittent engineering project.


Key Insights & Practical Implications

  • Proactively Scheduled Agent Tasks

    • Key Insight: The production ops agent relies on background tasks that run on a customizable schedule to scan your application. It is preconfigured to handle items like security certificate expirations, recent code change risk assessments, and ongoing cost optimizations.

    • Practical Implication: Engineering teams can transition away from high-fatigue, reactive alert loops. By catching system regressions (like an upcoming certificate expiry) before they cause a breakdown, teams prevent cascade failures from ever reaching a critical user-facing status.

  • Automated Jira-to-GitHub Remediation Workflows

    • Key Insight: When an automated investigation flags an active failure, it triggers a designated workflow. For example, the system can automatically generate a detailed Jira tracking ticket and immediately pass the requirements to a GitHub Copilot to stage code modifications.

    • Practical Implication: This drastically cuts down manual troubleshooting and ticket drafting overhead. Engineers gain a documented audit trail of the root cause alongside a pre-written pull request; resolving the underlying bug is reduced to a quick, singular human-gated code review and merge.

  • Centralized Cross-Platform Connections

    • Key Insight: The platform supports unified data connections across public cloud providers (AWS, Azure, GCP), logging and observability utilities (Splunk), and engineering hubs (GitHub, Jira, Slack).

    • Practical Implication: The agent can reason across disjointed technology stacks to form an accurate diagnosis. Instead of forcing engineers to manually scan fragmented logs during an outage, the platform synthesizes live metrics, traces, and historical code changes to isolate exactly why a specific pathway is failing.

  • Direct Infrastructure Mitigation Action

    • Key Insight: Beyond staging code patches, the agent tasks and workflows can be configured to interact directly with infrastructure alerts, such as reacting to Amazon CloudWatch metrics.

    • Practical Implication: It allows the platform to perform immediate operational scaling or self-healing corrections, such as automatically spinning up more pods in a Kubernetes cluster or executing a node restart. This minimizes total mitigation time without requiring manual intervention from a paged engineer.


Relevant FAQs

Q: What types of systems and tools can be connected to NeuBird's platform?

A: The platform supports integrations with standard cloud infrastructure like AWS, Azure, and GCP; observability and logging utilities like Splunk; code source controls like GitHub; project management backlogs like Jira; collaborative messaging tools like Slack; and custom MCB servers that track live website logs, traces, and metrics.

Q: How does the Production Ops Agent detect issues before they escalate into high-priority alerts?

A: The agent relies on scheduled "agent tasks" configured via custom prompts to scan environments on a variable timeline, ranging from minute-by-minute cron jobs to daily or weekly intervals. This lets the system catch silent problems, such as a HubSpot form missing an expected backend data field or a security certificate nearing expiration.

Q: Can the platform automatically execute direct infrastructure changes to mitigate cloud service issues?

A: Yes. If integrated with tools like AWS and CloudWatch, the agent can be assigned automated workflow policies that scale up pods inside a Kubernetes cluster or execute direct server node restarts to immediately drop system mitigation times.


Complete Transcript

Hey everyone, my name is Andrew, technical marketing engineer at NeuBird AI. If you're watching this, you've probably been woken up at 3:00 a.m. from a pager to fix a production issue or you oversee your company infrastructure and you want to reduce the level of fatigue your engineers face as well as operational costs of downtime. So in this session, I'm going to show you how you can reduce the noise and the toil of manually responding to incidents using NeuBird's production ops agent. So let me share my screen now and show you an example that anyone can understand and that is a web page.

So this is our production website neubird.ai, and I want to show you a failure on here that NeuBird is going to solve automatically. A web page issue can be as small as some kind of a HTTP 400 path not found to maybe a 500 issue with a database or it could be a variety of things. Right? So, what I'm going to show you is a form submission that does not quite work as intended. So, I'm going to fill out this form and request access. And then it says, "Something went wrong." So from an end-user perspective, this is like, "Well, I guess I won't try again." I mean I could keep trying but it'll keep failing. So now the marketing team on the back end is thinking, "Well, we're not getting any leads from this form. What happened?" So the engineering team then looks at the logs and, you know, if they have logs for it they might see the errors and what's wrong with it, and they might be getting paged for this issue as well. So there's a variety of things involved with this particular failure.

So let's see how this can be remediated automatically from NeuBird's side. So if we take a look at the NeuBird platform that we call the production ops agent, we're going to start at this connections page. So let me show you what kind of connections we support into the platform so that you have an idea of what you can connect and the kinds of data that you can investigate. So we have our usual suspects on the cloud infrastructure like AWS, Azure, and GCP, as well as the observability tools, monitoring tools, logging like Splunk, as well as incident management if you have that, and source control like GitHub. And lastly, you know, if you have custom servers, MCB servers, you can also connect those as well. So, we support a variety of different tools.

Now, once you get it all connected like you see here, we organize them into what we call projects. Now, you can see that we have a project called NeuBird AI website. And then here we have the GitHub connected, Jira, and an MCB connection which looks at the logs, traces, and metrics of the live website, as well as Slack. So we can interact with it from Slack.

So when we go into the project, we have this notion of an agent task. These are tasks that run on a schedule and they can check for a variety of things. In fact, they can check kind of whatever you want on the website. We have preconfigured a number of things by default like a certificate expiry, maybe some kind of a recent code change risk assessment, as well as cost optimization. So these can be scheduled to run so that you periodically get a refresh of what status your current application is at, right? Are there any upcoming certificate expirations? Because, well, if you do have one and they expire, then that'll lead to a cascade into a lot of different problems.

Now, one of these I've created myself is the HubSpot form submission issue. So maybe this form is an important campaign that you're running and you want to make sure that this is running as intended, and you want the signups to continue flowing in. And so in this particular prompt, I've asked it to scan for any HubSpot issues on that particular path to check for any kind of errors or failures. And what this will do is run this investigation periodically. You can see from the frequency, it could be every hour, every minute if you wanted on the cron job, daily or weekly, what have you.

So once that investigation runs, if it finds any issue, it's going to run an associated workflow. And what this workflow does in this case is create a Jira ticket and then hand it off to a co-pilot, a GitHub Copilot, to make the code fixes. And we can see from a recent run about 11 minutes ago, it ran an investigation. So if we look at the result of the investigation, it has identified that the form on this path has been failing 100% of the time and it's been resulting in an HTTP 500 error. And the root cause is that there's a missing field in the front end. So HubSpot's back end expects this field, but it was never created in the website code. The remediation steps are outlined, which is to simply add the field.

So let's see how this was handled. If we look at the artifacts of the investigation, notice that there's a Jira ticket. So because our workflow is Jira to Copilot, it's first created this ticket outlining what went wrong so that, you know, engineering teams have a backlog that they can audit to see what happened, and it gives a detailed root cause analysis and then the code fix that is required. And then as part of the workflow, we also hand it off to GitHub. So in GitHub, we have a pull request as well as an issue that mirrors the Jira ticket. So if we take a look at the pull request, you can see it's requested for my review and then it's made some code changes in the form of adding in that missing field. So let's go back and press that "Ready for review," and then I'm going to merge this into the live website.

Right? So, what's going to happen now? It's going to kick off the deployment process. Once the deployment is completed, then it's going to be reflected on the live website. And when it's done in about a couple minutes, we'll come back and refresh the page and we'll see that newly added field. Now that the deployment has completed, we're going to refresh our live web page and see the results. So, as you can see, we now have this company field that's been added thanks to our automatically created PR. So, we're going to attempt to submit another one here. And this time, we've succeeded in submitting our form.

And you know, this is a simple website scenario, but you can apply this to just about any kind of tool that you use. And as you've seen from our connections, since we support AWS, for example, let's say you have an application running on AWS monitored by CloudWatch. Well, we can look at the CloudWatch alerts, react to that, do an automated investigation, and maybe the mitigation step is to scale up the number of pods in the Kubernetes cluster. Maybe the remediation is to restart a particular node, and that will dramatically reduce your mitigation time because we can apply those steps automatically through our agent tasks which are scheduled to run those investigations, and then the workflows which are triggered by the investigation to fix the issue that gives you the pull request. So that, you know, when there's an issue and you log in in the morning, all you have to do is kind of review the PR, and if it looks good you merge it and that'll remediate your problem.

So I appreciate you taking your time to watch this demo and I hope you find it useful. If you'd like to learn more about what we do and apply the same kind of demo to your environment and your data, please visit us at neubird.ai and you can go to the schedule demo page which is located on the top right and fill out the form for us and we'll reach out to you. Thanks for watching and we'll see you later.

Presenters:

Presenter Avatar

Andrew Lee

Technical Marketing Engineer at NeuBird AI

Join Now for More Content & Events

For event and sponsorship inquiries, please email: [email protected]

  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook