DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Because the DevOps movement has redefined engineering responsibilities, SREs now have to become stewards of observability strategy.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Related

  • Enhancing Business Decision-Making Through Advanced Data Visualization Techniques
  • Data Privacy and Security: A Developer's Guide to Handling Sensitive Data With DuckDB
  • How to Simplify Complex Conditions With Python's Match Statement
  • Unleashing the Power of Gemini With LlamaIndex

Trending

  • Introducing Graph Concepts in Java With Eclipse JNoSQL
  • The Evolution of Scalable and Resilient Container Infrastructure
  • Supervised Fine-Tuning (SFT) on VLMs: From Pre-trained Checkpoints To Tuned Models
  • SaaS in an Enterprise - An Implementation Roadmap
  1. DZone
  2. Data Engineering
  3. Data
  4. How-to: Python Data into Graphite for Monitoring Bliss

How-to: Python Data into Graphite for Monitoring Bliss

By 
Corey Goldberg user avatar
Corey Goldberg
·
Apr. 20, 12 · Interview
Likes (0)
Comment
Save
Tweet
Share
25.0K Views

Join the DZone community and get the full member experience.

Join For Free

This post shows code examples in Python (2.7) for sending data to Graphite.

Once you have a Graphite server setup, with Carbon running/collecting, you need to send it data for graphing.

Basically, you write a program to collect numeric values and send them to Graphite's backend aggregator (Carbon).

To send data, you create a socket connection to the graphite/carbon server and send a message (string) in the format:

"metric_path value timestamp\n"

  • `metric_path`: arbitrary namespace containing substrings delimited by dots. The most general name is at the left and the most specific is at the right.
  • `value`: numeric value to store.
  • `timestamp`: epoch time.
  • messages must end with a trailing newline.
  • multiple messages maybe be batched and sent in a single socket operation. each message is delimited by a newline, with a trailing newline at the end of the message batch.

Example message:

"foo.bar.baz 42 74857843\n" 

Let's look at some (Python 2.7) code for sending data to graphite...


Here is a simple client that sends a single message to graphite.

Code:

#!/usr/bin/env python

import socket
import time


CARBON_SERVER = '0.0.0.0'
CARBON_PORT = 2003

message = 'foo.bar.baz 42 %d\n' % int(time.time())

print 'sending message:\n%s' % message
sock = socket.socket()
sock.connect((CARBON_SERVER, CARBON_PORT))
sock.sendall(message)
sock.close()

Here is a command line client that sends a single message to graphite:

Usage:

$ python client-cli.py metric_path value

Code: 

#!/usr/bin/env python

import argparse
import socket
import time


CARBON_SERVER = '0.0.0.0'
CARBON_PORT = 2003


parser = argparse.ArgumentParser()
parser.add_argument('metric_path')
parser.add_argument('value')
args = parser.parse_args()


if __name__ == '__main__':
    timestamp = int(time.time())
    message = '%s %s %d\n' % (args.metric_path, args.value, timestamp)
    
    print 'sending message:\n%s' % message
    sock = socket.socket()
    sock.connect((CARBON_SERVER, CARBON_PORT))
    sock.sendall(message)
    sock.close()

Here is a client that collects load average (Linux-only) and sends a batch of 3 messages (1min/5min/15min loadavg) to graphite. It will run continuously in a loop until killed. (adjust the delay for faster/slower collection interval): 

#!/usr/bin/env python
 
import platform
import socket
import time


CARBON_SERVER = '0.0.0.0'
CARBON_PORT = 2003
DELAY = 15  # secs


def get_loadavgs():
    with open('/proc/loadavg') as f:
        return f.read().strip().split()[:3]


def send_msg(message):
    print 'sending message:\n%s' % message
    sock = socket.socket()
    sock.connect((CARBON_SERVER, CARBON_PORT))
    sock.sendall(message)
    sock.close()


if __name__ == '__main__':
    node = platform.node().replace('.', '-')
    while True:
        timestamp = int(time.time())
        loadavgs = get_loadavgs()
        lines = [
            'system.%s.loadavg_1min %s %d' % (node, loadavgs[0], timestamp),
            'system.%s.loadavg_5min %s %d' % (node, loadavgs[1], timestamp),
            'system.%s.loadavg_15min %s %d' % (node, loadavgs[2], timestamp)
        ]
        message = '\n'.join(lines) + '\n'
        send_msg(message)
        time.sleep(DELAY)

Resources:

  • Graphite Docs
  • Graphite Docs - Getting Your Data Into Graphite
  • Installing Graphite 0.9.9 on Ubuntu 12.04 LTS
  • Installing and configuring Graphite

 

 

END

Graphite (software) Data (computing) Python (language)

Opinions expressed by DZone contributors are their own.

Related

  • Enhancing Business Decision-Making Through Advanced Data Visualization Techniques
  • Data Privacy and Security: A Developer's Guide to Handling Sensitive Data With DuckDB
  • How to Simplify Complex Conditions With Python's Match Statement
  • Unleashing the Power of Gemini With LlamaIndex

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!