DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

SBOMs are essential to circumventing software supply chain attacks, and they provide visibility into various software components.

Related

  • A Beginner’s Guide to Playwright: End-to-End Testing Made Easy
  • Automating E2E Tests With MFA: Streamline Your Testing Workflow
  • Exploring Cloud-Based Testing With the Elastic Execution Grid
  • Integrating Selenium With Amazon S3 for Test Artifact Management

Trending

  • The Agile Paradox
  • The Rise of the Intelligent AI Agent: Revolutionizing Database Management With Agentic DBA
  • Continuous Quality Engineering: The Convergence of CI/CD, Chaos Testing, and AI-Powered Test Orchestration
  • Build Real-Time Analytics Applications With AWS Kinesis and Amazon Redshift
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. Canary Tests

Canary Tests

By 
Cyrille Martraire user avatar
Cyrille Martraire
·
Feb. 07, 14 · Interview
Likes (9)
Comment
Save
Tweet
Share
40.1K Views

Join the DZone community and get the full member experience.

Join For Free

canary tests are minimal tests to quickly and automatically verify that the everything you depend on is ready. you run canary tests before other time-consuming tests, and before wasting time investigating in your code when the other tests are red. if the canary test fails, you know you have to fix something on the environments first.

this idea of canary test is different from the canary deployment. in canary deployment you deploy to a small fraction of your users to check everything’s fine before rolling out to more users.

save time by checking what should be always ok

canary tests check for the obvious and frequent sources of issues, such as:

  • connectivity to network : firewall rules ok, ports open, proxy working fine, nat, ping below a good threshold

  • databases and middleware are up
  • disk quota for logs not almost full
  • every needed login and password is valid
  • installed software available in the right version: dll installed, registry set-up, environment variables set, user directories all exist, the frameworks and os versions are fit, timezone and locale are as expected
  • reference data integrity and consistency (dates, valuations…) are ok
  • database schema and audit of applied scripts are as expected
  • licences are not expired (there is usually a way to check that automatically)

canary tests should run regularly, ideally before any expensive tests like end-to-end tests. of course you want to run them whenever there is a trouble somewhere, before wasting time on manual investigations in your code when the expected environment is not fully available.

even at the code level, a canary test is just a trivial test to verify that the testing framework works correctly, as mentioned by marcus on his blog :

	asserttrue(true)

don’t forget to verify that your tests can fail too!

simple and low-maintenance

the canary test tools should not assume much from the application. they must be independent from new developments to be as stable as possible. they should require little to no maintenance at all.

one way to do that in practice is to simply scan configuration files for every url, password and just ping them one by one against a predefined time threshold. any log path mentioned in the configuration files can be scanned and checked for the required write permissions and available disk space. any login and password can be checked, even though this may be more complicated.

canary tests are documentation too

doing canary tests may require explicit declarations of expectations, e.g. an annotation assumedpermission(’777′) to declare the permissions required on the files referenced in the configuration files. alternatively you may rely on a convention over configuration principle. for example every

 log.*.path

variable is assumed  to be a log path to check against some predefined expectations like being writable and being ok with disk quota.

when you add canary tests, this automation itself is a form of documentation that makes assumption more explicit.

you could export a report of every canary test that has been ran into a readable form that can become part of your living documentation .

Testing

Published at DZone with permission of Cyrille Martraire, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • A Beginner’s Guide to Playwright: End-to-End Testing Made Easy
  • Automating E2E Tests With MFA: Streamline Your Testing Workflow
  • Exploring Cloud-Based Testing With the Elastic Execution Grid
  • Integrating Selenium With Amazon S3 for Test Artifact Management

Partner Resources

×

Comments

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • [email protected]

Let's be friends: