Containerizing and Testing a Python Backtesting System With Docker and GitHub Actions

Not long ago, I broke a backtest without changing a single line of code. I moved the script to a different machine—same OS, supposedly the same Python version — and the equity curve suddenly told a completely different story.

Nothing in the logic had changed. The environment was the only obvious difference. That was the day I stopped treating the runtime environment as an afterthought and started treating reproducibility as part of the experiment itself.

What I learned is this: a backtest isn’t truly reproducible just because it ran once on your laptop. A result you can’t regenerate reliably isn’t a research finding — it’s a coincidence. And when that coincidence eventually meets real money, it can get expensive fast.

What follows is a minimal but complete pipeline for containerizing and automatically testing a Python backtesting system. No over-engineering — just Docker, GitHub Actions, and a few habits that make your results trustworthy.

The Real Goal

We aren’t building a live trading platform. We’re building a workflow that ensures every change is verified in a controlled environment:

Code change → automated tests → versioned Docker image build

The system must do four things:

1. Produce deterministic results, within an acceptable numerical tolerance, from the same versioned inputs in the same containerized environment.
2. Automatically test every code change before it can be merged.
3. Halt the pipeline when a test fails, with no exceptions.
4. Tag every image build so we can trace it back to the exact source-code version.

If a workflow can’t do that, it’s just a script living on someone’s laptop.

Project Structure

Before containerizing the application, let’s organize the repository clearly:

backtesting-system/
├── app/
│   ├── engine.py
│   └── main.py
├── tests/
│   └── test_engine.py
├── data/
│   └── sample.csv
├── requirements.txt
├── requirements-dev.txt
├── Dockerfile
└── .github/
    └── workflows/
        └── ci.yml

The app/ directory holds the strategy logic, while tests/ remains separate. The data/ directory contains a small, frozen sample dataset that never changes — our gold standard.

There are no absolute paths or machine-specific assumptions. Everything that might vary, including the data path, starting capital, and fee rate, comes from environment variables or a configuration file.

Containerizing With Docker

The classic “works on my machine” problem usually indicates an environment mismatch. Docker reduces this drift by packaging the application and its runtime dependencies into a versioned image.

Here is the Dockerfile:

FROM python:3.12-slim 
WORKDIR /app 
COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt 
COPY app ./app COPY data ./data 
CMD ["python", "-m", "app.main"]

A few decisions matter here.

We pin the Python 3.12 image series and can use an image digest when stricter reproducibility is required. Dependencies are pinned to exact versions in requirements.txt, since version ranges can silently introduce changes. We also use slim to keep the image small.

Crucially, we never copy local keys, cached results, or temporary files into the image. The container doesn’t guarantee that the logic is correct. It helps ensure that sound logic runs in a controlled and substantially more consistent environment.

Adding Tests That Matter

Automation without tests simply automates mistakes. Our tests don’t attempt to prove that a strategy is profitable. They prove that the program behaves consistently.

We check for:

• Clear errors when input files are empty or missing.
• Deterministic results when the same data and seed are used.
• Correct fee calculations.
• Rejection of malformed rows rather than silent processing.
• Required fields in every output.

Here is one example, including the necessary imports:

import pytest
from app.engine import run_backtest


def test_backtest_is_reproducible():
    first = run_backtest("data/sample.csv", seed=42)
    second = run_backtest("data/sample.csv", seed=42)

    assert first["trades"] == second["trades"]
    assert first["final_equity"] == pytest.approx(
        second["final_equity"], rel=1e-9
    )

This test establishes a simple contract: given the same starting conditions, the system will not drift beyond an acceptable margin.

The direct comparison of trades works here because we assume that every trade entry uses standardized types such as integers and strings. If the trade records contain floating-point prices, those values should be checked individually with an appropriate tolerance.

Building the GitHub Actions Pipeline

Now we automate the workflow. It runs on pushes, pull requests, and manual triggers through workflow_dispatch:

name: Backtest CI

on:
  push:
  pull_request:
  workflow_dispatch:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - run: pip install -r requirements-dev.txt

      - run: pytest

      - run: docker build -t backtest:${{ github.sha }} .

The steps are straightforward: check out the code, set up Python, install the development dependencies, run the tests, and build the Docker image.

The requirements-dev.txt file includes both the production dependencies and a pinned version of pytest:

-r requirements.txt
pytest==8.3.4

If pytest detects a failure, the job stops immediately. The broken change never reaches the image-build step.

The resulting image is tagged with the Git commit hash, creating a clear link between the source code and the image built during that workflow run.

Managing Configuration and Secrets

Environment-specific configuration and secrets should never be baked into the image. Environment variables can control data paths and run modes.

If the system is later connected to a live data source, API credentials should be stored in GitHub Secrets or a cloud secrets manager—never in the source code or Dockerfile. Logs must not expose keys or sensitive headers.

Even if today’s “production” environment is only a scheduled test run, development and production should use separate configuration sets. Treat every secret as sensitive, and keep environment-specific configuration outside the image.

Lightweight Monitoring and a Path to Rollback

Once the pipeline runs regularly, monitoring must go beyond asking whether the process is still alive.

Useful questions include:

• Did the latest job complete, or did it hang?
• Has execution time increased dramatically?
• Is the input data intact?
• Were the output files generated, and are they non-empty?
• Which image version produced the latest results?

If images are later pushed to a container registry, retaining the last few stable versions provides a straightforward rollback path.

For scheduled backtest runs, we should also archive the data snapshot, parameters, and results. That allows us to return a month later and answer a very specific question: “What exactly did we test on July 24?”

These practices aren’t unique to systems we build ourselves. Commercial grid-trading interfaces make automated execution accessible without revealing every part of their internal deployment pipelines.

BYDFi is one example I encounter in my work. Because I work with the platform, this is a disclosed reference rather than an independent recommendation.

The comparison is conceptual: understanding reproducibility, automated checks, and configuration management helps developers evaluate any automated tool more thoughtfully.

The Experiment Isn’t Finished Until It’s Verified

We started with a broken backtest and a frustrating realization. Now we have a different mindset.

Docker reduces environment drift. Automated tests guard program behavior. GitHub Actions ensures every change passes through the same gate. Monitoring and versioning give us a clear path to detect problems and support rollback as the pipeline evolves.

In a reliable backtesting system, reproducibility and verification are not tasks that come after the experiment. They are part of the experiment itself.

The moment we treat them that way, our results stop being anecdotes and start becoming evidence. And when the decisions involved can carry real financial weight, evidence is the only thing worth building.