DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Telemetry Pipelines Workshop: Installing Fluent Bit From Source
  • 11 Observability Tools You Should Know
  • How Observability Is Redefining Developer Roles
  • A Deep Dive Into Distributed Tracing

Trending

  • Java's Quiet Revolution: Thriving in the Serverless Kubernetes Era
  • Comparing SaaS vs. PaaS for Kafka and Flink Data Streaming
  • Stateless vs Stateful Stream Processing With Kafka Streams and Apache Flink
  • Immutable Secrets Management: A Zero-Trust Approach to Sensitive Data in Containers
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Monitoring and Observability
  4. O11y Guide: Without Observability, It's Just Code

O11y Guide: Without Observability, It's Just Code

Learn what makes cloud-native observability so crucial to your organization's success and find out why without observability, it's just code that you are deploying.

By 
Eric D.  Schabell user avatar
Eric D. Schabell
DZone Core CORE ·
Jun. 28, 23 · Opinion
Likes (4)
Comment
Save
Tweet
Share
6.6K Views

Join the DZone community and get the full member experience.

Join For Free

Welcome to another chapter in the ongoing series I started covering my journey into the world of cloud-native observability. If you missed any of the previous articles, head on back to the introduction for a quick update.Without observability, it's just code

After laying out the groundwork for this series in the initial article, I spent time sharing who the observability players are, looked at the ongoing discussion around monitoring pillars versus phases, shared thoughts on architectural level choices being made, and shared the open standards available within the open source landscape. I continued onwards with a few of the architectural challenges you might encounter when older monolithic applications and monitoring tools are still part of an organization's infrastructure landscape. Finally, I walked you through the open-source visualization and dashboard project called Perses by introducing my hands-on workshop.

Being a developer from my early days in IT, it's been very interesting to explore the complexities of cloud-native o11y. Monitoring applications goes way beyond just writing and deploying code, especially in the cloud-native world. One thing remains the same: maintaining your organization's architecture always requires that you understand that without observability, it's just code that you are deploying.

Let's look at what makes cloud-native observability so crucial to your organization's success.

The phrase "Without observability, it's just code" originated back 10 years ago when I first designed a t-shirt for some event with the theme being around DevOps with a similar phrase, "Without Ops, it's just code." Today, it's not hard to see the transition to cloud-native observability has become essential for many organizations as they explore digital transformation paths to modernize their architectures.

Cloud-Native Complexity

Initially, when developers and operations teams were moving into the cloud, they were confronted with Infrastructure-as-a-Service (IaaS) processes and often they were VM-based environments. These led to environments they could both understand and monitor without too much drama. This was due to the one-to-one relationship often found between the VM-based infrastructure, hosting a monolithic application, that delivered to the business a product or service that their customers were able to use.

As the evolution to cloud-native became a reality, these same developers and operations teams were exposed to auto-scaling, descriptive environments where containers and microservices became the new normal. The complexity skyrocketed overnight with relationships becoming many-to-many as infrastructure became a sea of containers running on container platforms, applications split out from single monoliths into multiple microservices, and businesses are now presenting use cases, clients, and geo-located interactions with the underlying microservices.

This complexity brings more data with it: both application-generated data, as customers interact, and monitoring data. This growth in data means that costs are higher to operate in these environments, especially when you are applying the same observability technologies to this new cloud-native environment.

What happens when your organization is successful in a cloud-native environment and all of the above issues are applied at a massive scale?

Cloud-Native at Scale

When developers and operations teams are working in a cloud-native environment at scale, other problems arise. The time with which teams must act and react is drastically shortened, with the delivery of code being done in smaller increments and at a much faster pace. With hundreds or thousands of microservices being developed, updated, and deployed multiple times a day or even within an hour, the processes that bring together the DevOps culture become a necessity.

Along with the processes comes a flood of new data, from applications, microservices, containers, observability, and other aspects of the cloud-native environment. This also results in tooling sprawl, where many new tools enter the domains of the DevOps teams. All of this increases the cognitive overload and stress for these teams.

If you look around the internet searching for how much time developers are actually spending on the job they signed up for (coding and building new things), it is shocking. Here are just a few samples of the results you might find:

  • Developers are spending 40% of their time on testing and quality assurance. (Source: TechRepublic)
  • Developers spend 32% of their time writing or improving new code. (Source: TheNewStack)
  • 83% of developers are burning out, and 20% of engineers suffering from burnout attributed the cause to "unreliable software." (Source: IT Pro)

In the following diagram, you can see the [maybe a bit optimistic] split on how developers in DevOps teams are buried in everything but the new development for organizations that hired them.

Developers in DevOps teams time split

Over 50% of their time is spent on deployment and related issues, 18% on other overhead like meetings and administration, with just over 30% of their time spent on new code. That's just a little more than one day a week.

Are you wondering how organizations are dealing with these challenges?

Facing the Challenges

In a recent article I read, The Growth of Observability Data is out of Control, two important statements were made that describe the situation and how organizations are facing these challenges.

"It's remarkable how common this situation is, where an organization is paying more for their observability data (typically metrics, logs, traces, and sometimes events) than they do for their production infrastructure." The research backs up this statement: ". . . 71% of companies are concerned with the rate of growth of their observability data."

Furthermore, when a simple "Hello,World!" application was deployed on a four-node cloud cluster and basic metrics are collected, the results were shocking. Thirty days of data were collected and it ended up generating +450 GB of data, that's almost half a terabyte of data you have to store, query, and pay for. Can you imagine if you run hundreds of thousands of applications?

Let's be honest: the cost would not be a problem if the outcomes were better - if your customers were happy, if you could fix issues quickly, or if you are able to retain your DevOps resources without burning them out on call. Short of collecting less data on purpose, the problem will remain as your data increases.

So what does observability look like that elevates your code beyond just code?

O11y Happiness

The key to understanding observability happiness is to no longer approach this as solvable with the mantra of metrics, logs, and tracing. Putting a solid strategy in place, with a business outcome focus will lay the foundations for a successful cloud-native existence. Focus on the three phases of observability: know, triage, and understand.

You need the ability for your on-call engineers to approach cloud-native observability through the three phases:

  • First, you need to know about issues, as quickly as possible. You need to know what happened and when and what systems were impacted - ideally, finding out BEFORE a customer is impacted.
  • Then you need to triage and begin understanding the impact and who can help. From there you can determine which teams need to get involved and what priority level it is.
  • Lastly, you need to be able to understand the root cause of the problem.

A large part of this approach is being able to get control of the data being generated, collected, and stored in your cloud-native environments. Having the ability to gain insights into how much data is actually useful and what data is not being used at all. Imagine finding out that you are ingesting +130k data points per second and paying for that storage, while you find out later that with 180 data points, you are drawing the same graphs on your dashboard that is observing this system.

Comparison of data generated

This all leads to better outcomes, from cost savings on data storage to lower cognitive loads on your teams. Making sure you are aware of the complexities and challenges of cloud-native environments gives you the chance to tackle these challenges before they become obstacles to your organization's cloud-native success. Take the time to understand your observability needs, because in a modern cloud-native world, without observability, all you are doing is generating a lot of code.

Observability Open source microservice Cloud native computing

Published at DZone with permission of Eric D. Schabell, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Telemetry Pipelines Workshop: Installing Fluent Bit From Source
  • 11 Observability Tools You Should Know
  • How Observability Is Redefining Developer Roles
  • A Deep Dive Into Distributed Tracing

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!