DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • How to Build a URL Shortener Web App With Flask Framework
  • Android Cloud Apps with Azure
  • How to Build a Full-Stack App With Next.js, Prisma, Postgres, and Fastify
  • Unified Observability: Metrics, Logs, and Tracing of App and Database Tiers in a Single Grafana Console

Trending

  • Developers Beware: Slopsquatting and Vibe Coding Can Increase Risk of AI-Powered Attacks
  • My LLM Journey as a Software Engineer Exploring a New Domain
  • Scalable, Resilient Data Orchestration: The Power of Intelligent Systems
  • Medallion Architecture: Efficient Batch and Stream Processing Data Pipelines With Azure Databricks and Delta Lake
  1. DZone
  2. Data Engineering
  3. Databases
  4. Error Tracking in Python

Error Tracking in Python

By 
Sonu Kumar user avatar
Sonu Kumar
·
Dec. 04, 19 · Tutorial
Likes (2)
Comment
Save
Tweet
Share
12.7K Views

Join the DZone community and get the full member experience.

Join For Free

404-the-strand

Error tracking is essential for any application when many people are relying on the app. If your application is not working in production for whatever reason, it can be a nightmare experience for users depending on correct functionality.  

There are many exceptions tracking plugins and modules available. Some of them are open source whereas others are paid, like HoneyBadger,  NewRelic,  ELK, etc. 

What about having a custom exception handler that can track errors in different contexts with additional features like:

  •  Send emails or notify developers (interested entities).

  • Create or update tickets in Bugzilla or Jira.

  • Push exceptions to the ELK stack or any other system for analytics and store detailed exceptions in DB.

  • While capturing exceptions, provide local and global variables details (failure context).

You may also like: The Top Four Exception Tracking Services

Error Tracker

Error tracker is a python library. It provides all required interfaces and supports all the above features. One interesting thing about this library is that it can mask variables that are sensitive (i.e should not be exposed, like password, credit card number, etc.). It's well documented at Error Tracker doc and available via PyPI  

A typical setup of Error Tracker can be visualized as:

Error Tracking

Error Tracking


Error Tracker can be used with any type of Python application. We'll start with Flask.

Tools we need:

  •  Editor any IDE.

  • Python.

  • Pip.

Installation

 pip install error-tracker 

We'll use settings file for app configuration you might call it config.py or some other format like YAML etc, we'll use python file for simplicity. 

settings.py

APP_ERROR_SEND_NOTIFICATION = True
APP_ERROR_RECIPIENT_EMAIL = ('dev@example.com',)
APP_ERROR_EMAIL_SENDER="server@example.com"
APP_ERROR_SUBJECT_PREFIX = ""
APP_ERROR_MASK_WITH = "**************"
APP_ERROR_MASKED_KEY_HAS = ("password", "secret")
APP_ERROR_URL_PREFIX = "/dev/error"


 There're seven configurations in this file:

 APP_ERROR_SEND_NOTIFICATION  — This configures whether a notification has to be sent or not. If it's set, then it will try to send notification using the notification interface. 

 APP_ERROR_RECIPIENT_EMAIL  — This refers to whom the email has to be sent to. It is required to build email content. It can be a list of emails or a single email address. 

 APP_ERROR_EMAIL_SENDER  — Who is sending email to the recipient(s).

 APP_ERROR_SUBJECT_PREFIX  — Any email subject prefix. By default, it will be [IndexError] [GET] http://127.0.0.1:5000/go.

 APP_ERROR_MASK_WITH — What should be used to mask the sensitive information.

 APP_ERROR_MASKED_KEY_HAS — What variables should be masked. It can mask all local and global variables. Masking is done based on the variable name. If a data type is a dictionary, or it's subclass then their keys are examined. 

 APP_ERROR_URL_PREFIX — All unique exceptions are stored in the database and can be browsed at this endpoint.

Create another python file 

For email-sender, we'll implement NotificationMixin 

from flask_mail import Mail, Message
class Notifier(Mail, NotificationMixin):
    def notify(self, request, exception,
               email_subject=None,
               email_body=None,
               from_email=None,
               recipient_list=None):
        message = Message(email_subject, recipient_list, email_body, sender=from_email)
        self.send(message)


For issue tracking, we'll implement TicketingMixin 

class Ticketing(TicketingMixin):
    def raise_ticket(self, exception, request=None):
        # implement this method to communicate with Jira or Bugzilla etc
        pass


Let's create a Flask app to serve requests. We'll use SQLite for simplicity.

app = Flask(__name__)
app.config.from_object(settings)
app.config['SQLALCHEMY_DATABASE_URI'] = "sqlite:///tmp.sqlite"
db = SQLAlchemy(app)


The errors can be tracked by creating an instance of AppErrorTracker and initializing this with proper details. This application can store all the exceptions in the database. 

app_error = AppErrorTracker(app=app, db=db,
                            notifier=Notifier(app=app), ticketing=Ticketing())
db.create_all()


For error tracking, we are going to use the decorator  track_exception provided by AppErrorTracker  class. We'll attach this decorator whenever HTTP 500 occurs. 

@app.errorhandler(500)
@app_error.track_exception
def error_500(e):
    return render_template('500.html'), 500


Now, it's ready to be used. We can fire some requests and see how it works. For testing, let's add one endpoint that will throw 500 errors with different types of exceptions.

@app.route('/go')
def die():
    import random
    password = "test"
    foo = {}
    foo['password'] = "Oh! My Gosh"
    foo['secret'] = "NO ONE KNOWS"
    exceptions = [KeyError, ArithmeticError, BaseException, IndentationError, IndexError, MemoryError,
                  NameError, NotImplementedError, ImportError, FloatingPointError, EOFError,
                  OSError, AssertionError, AttributeError, GeneratorExit, Exception,
                  EnvironmentError, ImportError,
                  NotImplementedError, RuntimeError]
    raise random.choice(exceptions)


Now, run the app in debug=False mode and browse, as it will fail. Now, go to http://127.0.0.1:5000/dev/error/. This will display the current exceptions in a table like: 

Exceptions caught

Exceptions caught

Any of the specific exceptions can be browsed by clicking the corresponding link, in the Exception detail  section, all frame details that have been captured can be seen. 

Exceptions available for analysis

Exceptions available for analysis

The complete Flask application can be found at https://github.com/sonus21/error-tracker/tree/master/examples/flask-sample.

Implementation With Django

Using Exception Tracker with Django is also very simple. Install the application using pip install error-tracker. Once we've installed it, we need to configure some part of the applications using the settings.py file.

For the Django application, we need to add this app in the INSTALLED_APPS  list and add Middleware  to catch exceptions. Middleware should be added at the end of the list so that it will be called first whenever an exception occurs. 

APP_ERROR_RECIPIENT_EMAIL = ('example@example.com',)
APP_ERROR_SUBJECT_PREFIX = "Server Error"
APP_ERROR_EMAIL_SENDER = 'user@example.com'

INSTALLED_APPS = [
    ...
    'error_tracker.DjangoErrorTracker'
]
MIDDLEWARE = [
    ...
    'error_tracker.django.middleware.ExceptionTrackerMiddleWare'
]


Error Tracker provides a few default pages for listing exception, deleting exception, and browsing a specific exception. These endpoints can be plugged in with Django's other URLs.

 from error_tracker.django import urls

urlpatterns = [
    ...
    url("dev/", include(urls)),
]


These are all the code changes we need for the Django app. The complete code is available at https://github.com/sonus21/error-tracker/tree/master/examples/DjangoSample.

Conclusion

Apps using Error Tracker can be further customized, like what models to use. Currently, it records only one instance of failure, while all failures can be tracked and dumped to some database. Read more about them at https://error-tracker.readthedocs.io/en/latest/index.html#.

The complete code is available on GitHub.

If you found this post helpful, please share and give a thumbs up.


Further Reading

  • Error Handling Strategies.
  • Create a Simple API Using Django REST Framework in Python.
Python (language) application Database app Tracker (business software)

Opinions expressed by DZone contributors are their own.

Related

  • How to Build a URL Shortener Web App With Flask Framework
  • Android Cloud Apps with Azure
  • How to Build a Full-Stack App With Next.js, Prisma, Postgres, and Fastify
  • Unified Observability: Metrics, Logs, and Tracing of App and Database Tiers in a Single Grafana Console

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!