DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Related

  • Top React Libraries for Data-Driven Dashboard App Development
  • Reporting in Microservices: How To Optimize Performance
  • How to Build a Pokedex React App with a Slash GraphQL Backend
  • 10 Node.js Security Practices

Trending

  • Agile and Quality Engineering: A Holistic Perspective
  • Endpoint Security Controls: Designing a Secure Endpoint Architecture, Part 1
  • Unmasking Entity-Based Data Masking: Best Practices 2025
  • Beyond Simple Responses: Building Truly Conversational LLM Chatbots
  1. DZone
  2. Data Engineering
  3. Data
  4. How to Use Grafana for Technical Monitoring in Software Products

How to Use Grafana for Technical Monitoring in Software Products

Learn the basics of Grafana for visualizing metrics and other data on app performance.

By 
Daryna Tkachenko user avatar
Daryna Tkachenko
·
Oct. 06, 18 · Tutorial
Likes (4)
Comment
Save
Tweet
Share
59.0K Views

Join the DZone community and get the full member experience.

Join For Free

Definition of Grafana

Grafana is an open-source platform for data visualization, monitoring and analysis. Our company uses this tool, paired up with Graylog, to monitor technical state of software systems we use internally or build for our customers. Grafana allows users to create dashboards with panels, each representing specific metrics over a set time-frame. Every dashboard is versatile, so it could be custom-tailored for a specific project or any development and/or business needs.

We mostly use Grafana with Elasticsearch and InfluxDB, but there is a variety of other supported data sources (Prometheus, MySQL, Postgres to name just a few) for this software. For each data source, Grafana has a customized query editor and specific syntax.

Grafana Notions

  • A Panel is the basic visualization building block presented per the metrics selected. Grafana supports graph, singlestat, table, heatmap, and freetext panels, as well as integration with official and community-built plugins (like world map or clock) and apps that could be visualized, too. Each panel can be customized in terms of style and format; all panels could be dragged, dropped, resized, and rearranged.
  • A Dashboard is a set of individual panels arranged on a grid with a set of variables (like server, application and sensor name). By changing variables, you can switch the data being displayed in a dashboard (for instance, data from two separate servers). All dashboards could be customized and sliced and diced depending on the user needs. Grafana has a large community of contributors and users, so there is a large ecosystem of ready-made dashboards for different data types and sources.
  • Dashboards can utilize annotations to display certain events across panels. An annotation is added by custom requests to Elasticsearch; it shows as a vertical red line on the graph. When hovering over an annotation, you can get event description and tags, for instance, to track when server responds with 5xx error code or when the system restarts. This way, it is easy to correlate with a time, specific event and its consequences in an application and investigate system behaviour.

Our Best Practices With Grafana

Grafana in Internal Projects

For our internal IoT solution for office weather monitoring, we connected Grafana to InfluxDB, a time series database, to visualize the changes in office weather parameters and react to them accordingly. A set of sensors measure temperature, humidity, atmospheric pressure and CO2 level in every zone of our office; these parameters are collected and visualized with Grafana graphs on a large kitchen monitor and online.

Grafana Dashboard with Logicify Office ZonesGrafana Dashboard with Logicify Office Zones.

This way, we keep constant track of air quality parameters, and our Office Manager reacts to the changes: opens windows if the level of CO2 is too high, turns AC and humidifiers on and off.

Grafana Dashboard with Office Weather ParametersGrafana Dashboard with Office Weather Parameters.

Through Grafana-displayed time series graphs and annotations, we analysed trends in office weather changes over months and seasons. We also used the tool to visualize some useful widgets and pieces of information (weather forecast, currency exchange rates, internal calendars) on a large kitchen monitor.

How to Use Grafana in Custom Web Apps

Grafana + Graylog

We use Graylog to store and manage the logs of web applications and monitor their performance both in the development and production phases. Grafana is the tool that “translates” the logs stored in Graylog into visual forms for analytical and system monitoring purposes. For one of our ongoing projects, Grafana can figuratively be called a UI for a web application load and performance as well as customer flow. Graylog and Grafana exist independently from each other, and we made no custom and complex integration to connect them. Since Graylog stores all log data in Elasticsearch, one of Grafana’s data sources, we simply use certain Elasticsearch index where the logs are stored to connect Grafana to Graylog.

Visualizing Metrics in Grafana for Web Applications

Pure text logs or error notifications are not “interesting” to Grafana as its main purpose is to visualize the data in graphs, charts, and tables. We wrote a custom module for Django to collect the data we’d like to track for every web/worker request and response processed. It is not just the success/failure status but a set of structured fields (both general and project-specific), such as

  • App version
  • Uunique ID of every request
  • Response time and status
  • Error code (if any)
  • IP address from where the request was sent
  • User info (e-mail, username for registered users, role, permissions)
  • Device
  • Etc.

Django pushes custom structured analytical record into Graylog, which stores them in a separate stream. Though these data could be visualized by native Graylog dashboards, they are not that good-looking as Grafana’s. So we make Grafana read these analytical data and visualize them. This way, we keep track of the application performance and load both in real-time and in retrospective.

Graylog-Grafana_Architecture_Diagram

Grafana as a Debugging Tool

Primarily, Grafana dashboards help us in debugging the application. If the end customer reports a problem, Grafana gives us a way to distinguish between errors on customer/server side and real bugs or loopholes in application logic. We track all web requests initiated by the customer (using e-mail address), app admins and application itself within a given time-slot and find whom to blame by elimination.

We also do debugging and bug fixing if we notice an anomaly on the dashboard with application load and performance graphs. The following example of a Grafana graph shows the response time to web requests during a certain time-frame. For every web request, we could track a max, min and average response time. If we see a request that took us too long to process, we could scale a certain part of the graph and investigate the issue.

Example of Grafana Graph with Response Time to Web Requests

Example of Grafana Graph with Response Time to Web Requests.

Another graph shows system load over a set time-frame and is useful for traffic tracking. If we see an unusual spike in activity on the graph, e.g. in non-business hours or on weekends, we investigate it. It could be caused, for instance, by Google crawlers who index the website content or evil bots scanning our system for vulnerability. Again, each case is investigated and addressed accordingly.

Example of Grafana Graph with Application LoadExample of Grafana Graph with Application Load.

Grafana has a built-in alerting engine (e.g. email or Slack notifications) per some conditional rules. We do not use this option of Grafana as we have all notifications configured on Graylog side. However, some issues in system performance could be seen only after a runtime, e.g. unusually long response time to a web request. We would not receive a Graylog notification about this, yet the anomaly would be clearly seen in Grafana graph. So, both tools go hand-in-hand when we get to know about an issue: we check Grafana to understand what happened and why on the high level, then dig deeper in Graylog using a specific ID of a request.

Unlike Graylog, used both for apps under development and in production, Grafana is used only for the apps in production. The only exception when it is used for an app still underway is performance testing. We emulate system load with JMeter, then check Grafana dashboards to see how it responds.

Grafana for Business Analytics

Apart from performance tracking and debugging purposes, Grafana dashboards are a powerful tool for informed business decisions. When set up properly (preferably, in tandem with Google Analytics), Grafana can visualize custom analytics on user behavior in the system in the form of pie charts, time bar graphs, and other graphics. Based on these, product stakeholders could make the decisions on further scaling the application, adding or removing some functionality and improving customer journey.

Example of Grafana Dashboard with User Behavior in an eCommerce AppExample of Grafana Dashboard with User Behavior in an eCommerce App.

Since the above dashboard is more business-oriented, developers use it internally, more like a collateral tool to keep abreast of the customers flow in e-commerce applications: signups, logins, orders placed within a set time interval, etc.

Here are a couple of real project cases where Grafana helped to improve the usability of a web app.

  • Via Grafana, we regularly monitor the status of recurring orders in the system and filter failed ones. These orders are subscription-based, which means they are generated in the system each month, and money is automatically withdrawn from the customers’ bank accounts. Sometimes, payments fail (not enough money / financial institution refuse), so system admins check the case and contact the clients to re-generate the order manually. This way, no order is left behind, so both clients and vendors are satisfied.
  • Using Grafana-generated reports for an eCommerce app, we figured out that a large percent of new clients jump off the Checkout page, though they already have products in their carts. This finding was backed up by Google Analytics reports, so the checkout procedure was analyzed step-by-step and improved; users are now able to complete the order in just a couple mouse-clicks. This increased the conversion rate, and consequently, the vendor’s profit.

Grafana is an important component of Logicify monitoring system both for internal and external projects. It is an open-source software with a large and active community of contributors, but what we like most about this software is its flexibility — it supports multiple data sources and allows easy customizations of dashboards and panels.

Grafana Web application Software Dashboard (Mac OS) app Data (computing) Database Requests Open source

Opinions expressed by DZone contributors are their own.

Related

  • Top React Libraries for Data-Driven Dashboard App Development
  • Reporting in Microservices: How To Optimize Performance
  • How to Build a Pokedex React App with a Slash GraphQL Backend
  • 10 Node.js Security Practices

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!