DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Data
  4. Towards Turnkey Distributed Tracing

Towards Turnkey Distributed Tracing

OpenTracing is a new, open distributed tracing standard for applications and OSS packages. Find out more about it here.

Ben Sigelman user avatar by
Ben Sigelman
·
Jun. 21, 16 · Opinion
Like (3)
Save
Tweet
Share
2.65K Views

Join the DZone community and get the full member experience.

Join For Free

Antique hardware, pre-standardization: where we are with software instrumentation today.

OpenTracing is a new, open distributed tracing standard for applications and OSS packages. Developers with experience building microservices at scale understand the role and importance of distributed tracing: per-process logging and metric monitoring have their place, but neither can reconstruct the elaborate journeys that transactions take as they propagate across a distributed system. Distributed traces are these journeys.

It’s been decades since the first academic papers about distributed tracing were published. It’s been 12 years since Google started using Dapper internally. It’s been 6 years since the Dapper paper was put online. Zipkin was open-sourced over 4 years ago. This stuff is not new! Yet if you operate a complex services architecture, deploying a distributed tracing system requires person-years of engineer effort, monkey-patched communication packages, and countless inconsistencies across platforms.

If distributed tracing is so valuable, why doesn’t everyone do it already?

Because tracing instrumentation has been broken until now.

Distributed tracing is challenging because the instrumentation must propagate the tracing context both within and between processes. Accomplishing this touches almost every part of an application. In particular, tracing context must be passed through:

  • Self-contained OSS services (e.g., NGINX, Cassandra, Redis, etc)
  • OSS packages linked into custom services (e.g., grpc, ORMs, etc)
  • Arbitrary application glue and business logic built around the above

And there’s the rub:It is not reasonable to ask all OSS services and all OSS packages and all application-specific code to use a single tracing vendor; yet, if they don’t share a mechanism for trace description and propagation, the causal chain is broken and the traces are truncated, often severely.We need a single, standard mechanism to describe the behavior of our systems.

Enter OpenTracing

OpenTracing is that “single, standard mechanism.” OpenTracing allows developers of application code, OSS packages, and OSS services to instrument their own code without binding to any particular tracing vendor. Every component of a distributed system can be instrumented in isolation, and the distributed application maintainer can choose (or switch, or multiplex) a downstream tracing technology with a configuration change.

So we know we need standardization: but standardization where? All of the following are possibilities:

  1. Standardized span management: programmatic APIs to start, finish, and decorate timed operations (“spans” in Dapper and Zipkin’s terminology).
  2. Standardized inter-process propagation: programmatic APIs to aid in the transfer of tracing context across process boundaries.
  3. Standardized active span management: programmatic APIs to store and retrieve the active span across package boundaries in a single process.
  4. Standardized in-band context encoding: specification of the precise wire-encoding format for tracing context passed alongside application data between processes.
  5. Standardized out-of-band trace data encoding: specification of how decorated trace and span data is encoded as it heads towards the distributed tracing vendor.

Previous efforts to standardize distributed tracing have focused on the encoding and representation of trace and context data, both in-band and out-of-band (numbers 4 and 5 above). Ironically, per the table below, standardization of encoding formats has few benefits for instrumentation API consistency, tracing vendor lock-in, or the tidiness of dependencies for OSS projects: the very things that stand in the way of turnkey tracing today.

What’s truly needed — and what OpenTracing provides — is standardization of span management APIs, inter-process propagation APIs, and ideally active span management APIs. The table below shows the five key benefits of standardization around distributed tracing, and the importance of each of the five kinds of standardization for each of those benefits.

OpenTracing picks its battles: by focusing on APIs rather than implementation encodings, it provides the standardization benefits that developers actually need.

OpenTracing gives us the standards we need to make accurate, turnkey distributed tracing a reality across modern software systems, including OSS packages and other third-party code. It does this while allowing OpenTracing implementations to take control over both in-band and out-of-band encoding formats; this in turn gives the application owner the flexibility to switch (or add) tracing vendors with an O(1) declarative configuration change.

The OpenTracing software architecture. Application code and OSS packages program against the abstract OpenTracing APIs, describing the path that requests take within each process as well as the propagation between processes. OpenTracing implementations control the buffering and encoding of trace span data, and they also control the semantics of process-to-process trace context information. As a result, application code can describe and propagate traces without making any assumptions about the OpenTracing implementation.

This summer we will be announcing integrations with important pieces of the application-level microservices stack. As OpenTracing proliferates into other popular packages and subsystems, we will be able to have our cake and eat it, too: high-quality distributed traces with little to no instrumentation effort by the application programmer.

Thanks for reading, feel free to recommend this post if you found it valuable. If you’d like to learn more about OpenTracing,

  • Check out opentracing.io
  • Join our Gitter
  • Sign up for the OpenTracing newsletter
application microservice Data (computing) Instrumentation (computer programming) IT Implementation Business logic dev

Published at DZone with permission of Ben Sigelman, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Exploring the Benefits of Cloud Computing: From IaaS, PaaS, SaaS to Google Cloud, AWS, and Microsoft
  • A Beginner's Guide to Back-End Development
  • Debugging Threads and Asynchronous Code
  • Mr. Over, the Engineer [Comic]

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: