How We Know What We Know

In 1936, a mathematician named Alan Turing asked one of the most consequential questions in the history of computing: Can a machine think? The question has become especially popular nowadays with the development of AI. But before Turing would answer it, he had to answer a harder question: how would we know? What would count as evidence? His answer — the Turing test — was not a definition of intelligence. It was a framework for generating evidence. The question was epistemological before it was technical. It concerned not the question itself, but how we come to know an answer.

Software testing is the same kind of problem.

Before we can ask whether our software is reliable or safe, we must ask something more fundamental: what would it mean to know that? What counts as evidence? How do we weigh it? When is it sufficient? And when are we fooling ourselves into believing we know something we do not?

These are not questions for philosophy seminars. They have direct, measurable consequences for the engineering decisions made in your organization every day. Decisions about when software ships, what risks are accepted, how incidents are explained, and where investment in quality is placed. The organizations that reason well about these questions can build systems that fail less, recover faster, and cost less to maintain over time. The ones that do not tend to confuse the absence of evidence for evidence of absence.

We Learn By Testing

Consider how a child learns that a surface is hot. Not by being told. Not by reading a warning. By touching it — and observing the result. The test is the touch. The evidence is the sensation. The knowledge is the updated belief about surfaces of that kind. This is not a metaphor. It is the literal structure of empirical learning.

Now consider how an engineer learns that a login form works. They enter valid credentials and observe whether they are admitted. They enter invalid credentials and observe whether they are rejected. They enter a password of 10,000 characters and observe whether the system handles it gracefully. Each of these is a test. Each result is a piece of evidence. The accumulation of that evidence is what the engineer means when they say the login form "works" — not a metaphysical certainty, but a body of evidence sufficient to support the belief.

The last point is the one that causes the most organizational damage. Uncertain knowledge is manageable — you know what you don't know, and perhaps you can plan accordingly. Confident ignorance is dangerous because the team wrongly believes that the software is shippable. They have the test results to prove it. They ship. And then something breaks that their tests were incapable of detecting. Their tests were not designed to generate evidence about that kind of behavior.

What Does A Passing Test Tell Us?

One way that confident ignorance occurs is when a test passes when it should fail. A test can pass while the code it tests does not work as expected. Let’s see a few examples.

The test may be making a wrong assertion — it checks that a value is returned, but not that the value is right. The test may be structured in a way that never actually exercises the code path it was written to probe. A reason for this could be a mocked dependency that silently absorbs a call that should have triggered real behavior. As a third example, a test may be correct today but pass tomorrow only because the environment has changed. It has changed in a way that masks a genuine defect: a database that was seeded with accommodating data, a clock that was frozen at a convenient moment, a network call that was stubbed before it could fail.

Such tests fail to catch bugs, but more importantly, they create a false sense of quality. False evidence results in confident ignorance and, at the end of the day, unhappy customers because we’ve introduced more bugs in production.

Confident ignorance also occurs when a test suite passes, and we think that we’re done. We think that we know everything. Ideally, a passing test tells us that the software did what it was supposed to do. What else is there to say?

A great deal, as it turns out.

Consider the following test, written for an e-commerce checkout system:

def test_order_total_with_discount():
    cart = Cart()
    cart.add_item(Product(name='Laptop', price=1000.00), quantity=1)
    cart.apply_discount(code='SAVE10', percent=10)
    assert cart.total() == 900.00

A passing test: the discount is applied, and the total is correct.

But notice what it does not tell you:

• What happens when two discounts are applied simultaneously.
• What happens when the discount code expires.
• What happens when the product price changes between adding to cart and checkout.
• What happens when the quantity is zero, or negative, or a floating-point number.
• What happens when the discount calculation produces a result with more than two decimal places.
• What happens at the boundary: does a 100% discount produce 0.00 or raise an exception?

The test passed. But the space of untested behavior vastly exceeds the space of tested behavior. The passing test is not evidence that the system is correct. It is evidence that this system, with this input, at this moment, produced this output. The difference between those two things is a fundamental problem in software testing.

We trust passing tests if they provide evidence. But we commit a category error when we treat the presence of passing tests as evidence of correctness in general. A passing test tells you that one path through the system behaved as expected. It may not say much about other paths.

There is another reason why passing tests give us weaker evidence than we intuitively feel they do. When a test passes, it confirms something we already expected to be true. We wrote the test because we believed the system should behave that way. Confirmation of an existing belief is the weakest form of evidence, because it does not probe the limits of the belief. It merely restates it in executable form.

This is why some experienced testers may feel a mild, persistent unease even in the face of thousands of passing tests. It is epistemological hygiene built through experience at work.

Why Does a Failed Test Reveal More?

If passing tests give us weaker evidence than we expect, failing tests give us stronger evidence than we typically acknowledge.

A failing test is a learning opportunity. It tells you that you need to investigate. Does the world match your model of it, or is there something else going wrong? Failing tests in any empirical discipline are the engine of understanding. Physics advances when experiments produce results inconsistent with the prevailing theory. Medicine advances when patients respond unexpectedly to treatment. Software understanding advances when a test fails.

There is a philosophical tradition — associated with the Austrian-British thinker Karl Popper — which holds that no amount of confirming evidence can prove a general statement true, but a single disconfirming observation can prove it false. A thousand white swans do not prove that all swans are white. One black swan, however, does prove that not all swans are white.

The same asymmetry applies in testing. Consider this modification to the earlier example:

def test_expired_discount_is_rejected():

    cart = Cart()

    cart.add_item(Product(name='Laptop', price=1000.00), quantity=1)

    cart.apply_discount(code='EXPIRED99', percent=10)

    # We expect the discount to be silently ignored

    assert cart.total() == 1000.00

Suppose this test fails — suppose cart.total() returns 900.00. What do we now know?

We know something precise and actionable: the system is applying expired discount codes. We know the specific condition under which it happens. We know the exact incorrect output. We can trace the code path. We can fix it. The failing test has given us a map to the problem.

More importantly, the failing test has revealed something about the gap between our model of the system and its actual behavior. That gap is where risk lives. Every untested gap is a place where your mental model of the software could potentially diverge from what the software actually does. Failing tests can close those gaps. Passing tests, at best, could confirm that a particular gap does not exist.

There is a practical implication here that is often resisted: when a test fails, the correct initial response is not to fix the code. It is to understand what the failure is telling you about the system. Sometimes the code is wrong. Sometimes the test is wrong. Sometimes the requirement was ambiguous, and the failure has surfaced a genuine disagreement about what the system should do. Each of these is a different kind of finding, and treating them all as "bugs to fix" discards important distinctions. Let’s have a closer look:

If a failure is caused by our code, we should go beyond just fixing the implementation. The investigation should identify why the code violated the expected behavior. Was it a faulty algorithm? An overlooked edge case? A misunderstanding of an API? A race condition? A hidden dependency? Each answer reveals something about the system’s weaknesses. It often suggests broader improvements, such as simplifying the design or adding similar tests in related areas. If the investigation stops at changing a few lines of code until the test turns green, the defect disappears, but our feedback about the system is lost.

A failure caused by an incorrect test requires a very different response. The test might contain an invalid assumption or outdated expectations. It might contain unrealistic test data, an incorrect oracle, or excessive coupling to implementation details. The goal is not just to make the test green. It is to understand why the test reached the wrong conclusion, if the conclusion was actually wrong. Was the behavior intentionally changed? Did the test encode a misunderstanding of the requirement? Was it checking something that should never have been specified in the first place? Correcting the test improves not only the quality of the test suite but also the credibility of every future failure. A trustworthy test suite is built as much by removing incorrect tests as by adding new ones.

Perhaps the most valuable case is when the failure exposes an ambiguous requirement. Here, neither the code nor the test is necessarily wrong. Instead, the failure reveals that different people disagree on what the system is supposed to do. Developers may have implemented one interpretation, testers another, and product owners may discover that neither matches the intended behavior. The correct action is not to modify the code or the test immediately, but to resolve the ambiguity by making the requirement explicit. Once the intended behavior is agreed upon, both the implementation and the test can be aligned with that shared understanding. In this sense, the failure has uncovered a gap in what the system does.

These three outcomes illustrate why treating every failed test simply as “a bug to fix” is a missed learning opportunity. A failed test could be evidence that two views of the system disagree: the implementation, the test, and the specification can each represent a different view. Understanding which disagreement has occurred determines the appropriate action and produces knowledge that survives long after the immediate failure has been resolved.

What Counts as Evidence That Software Is "Correct"?

The word "correct" appears constantly in discussions of software quality. We want the software to be correct. We test to verify correctness. We define done as correct behavior. But what does correctness actually mean, and what kind of evidence is capable of establishing it?

Correctness is always relative to a specification. Software is correct if it behaves in accordance with what it is specified to do. This sounds straightforward until you examine it closely, at which point three problems emerge immediately.

The Specification Problem

More often than not, software does not have a complete, unambiguous specification. It has a collection of requirements documents in varying states of currency. It could be user stories in a backlog or tribal knowledge in engineers' heads. It could be a legacy codebase that implicitly encodes decisions no one can fully reconstruct. The specification against which correctness would be measured is itself a moving, contested, partially-documented target.

This means that when a test passes, it is agreeing to someone's interpretation of what the software should do, at the time they wrote the test, based on the information they had available. This is not nothing. But it is considerably less than "the software is correct."

The Oracle Problem

For a test to establish correctness, you need to know what the correct output is. This is called the test oracle — the mechanism by which you determine whether the observed output is the expected output. For simple, deterministic systems, the oracle is straightforward: you calculate the expected output and assert against it.

But consider testing a machine learning model that recommends products to users. What is the correct recommendation? There is no single right answer. The oracle is probabilistic, contextual, and partially subjective. Or consider testing a distributed system under network partition conditions. The system may behave differently on different runs due to timing effects. What does "correct" mean when the system is inherently non-deterministic? The concept of correctness assumes the existence of a reliable oracle, and in many real-world systems, that oracle is unavailable, unreliable, or only partially defined.

The Completeness Problem

Even if you have a complete specification and a reliable oracle, you cannot test all possible inputs, states, and conditions. The input space for any non-trivial system is effectively infinite. You must select a finite subset of tests from that infinite space, and the quality of that selection determines the quality of your evidence.

Consider a function that accepts a date string and returns whether it falls within a valid booking window:

def test_date_validation():
    assert is_valid_booking_date('2029-06-15') == True   # valid future date
    assert is_valid_booking_date('2020-01-01') == False  # past date
    assert is_valid_booking_date('2024-02-30') == False  # impossible date

Three tests. Passing. But the input space contains infinitely many date strings — and many interesting subsets remain untested.

What about '2024-02-29' in a non-leap year? What about dates beyond the year 9999? What about the string 'yesterday'? What about an empty string? What about a date that is valid today but becomes invalid by the time the booking is processed? Each of these is a region of the input space that the three tests above say nothing about.

The implication is important: correctness, as established by testing, is always partial. We can accumulate evidence for correctness in tested regions. We cannot establish correctness in untested regions. The most honest statement any team can make about their software is not "it is correct" but rather "it has behaved correctly under the conditions we have tested, and we have made a considered judgment that the untested regions pose acceptable risk." At the end of the day, it's all about confidence.

Why Do Experienced Testers Remain Skeptical?

If you have spent time with genuinely skilled software testers, you have probably noticed something that can seem like pessimism or obstruction: they are never quite satisfied. A thousand passing tests does not produce for them the confidence it produces in someone less experienced. They keep asking questions. They keep looking for the untested edge. They keep saying "but what happens when..."

This is not a personality trait. It is a trained posture, and it is entirely rational.

The experienced tester has seen, repeatedly, the pattern by which software that appeared thoroughly tested failed in production. And it failed in ways that, in retrospect, were discoverable. They have learned, through accumulated evidence of their own, that the gap between "what we tested" and "what is possible" is always larger than it appears. Their skepticism is not a belief that the software is bad. It is a calibrated resistance to the cognitive bias that afflicts everyone who works closely with a codebase: the tendency to mistake familiarity for correctness.

The Familiarity Trap

You have written a piece of code and tested it many times. You’ve watched it pass in CI for months, and you develop an intuitive sense that you understand it. This sense of understanding feels like evidence of correctness. It is not. It is evidence of familiarity. The two are easily confused and devastatingly different.

Familiarity narrows the space of inputs you consider. You test the paths you have already thought about. You write assertions for the outcomes you already expect. Your tests become a reflection of your mental model of the system rather than a probe of the system's actual behavior. But your mental model, however detailed, is always an approximation.

This is why the most effective testers often come to a system with a degree of productive ignorance. They do not know what the system is "supposed" to do, so they are free to observe what it actually does.

The Regress of Confidence

There is a second reason for experienced skepticism. Adding more tests does not necessarily increase confidence. The relationship is more complex, and experienced testers have an intuitive grasp of it even if they have never formalized it.

Consider a payment processing system with the following test coverage:

The 2,400 passing tests produced genuine confidence about what they tested. But the incident did not occur in what they tested. It occurred in the gap. The experienced tester would have looked at this suite and asked: what is not here? What conditions does this suite not probe? Where is the coverage thin?

The question is never "how many tests do we have?" The question is "what is the shape of our evidence, and where are the edges?"

When Does Adding More Tests Stop Increasing Confidence?

This is one of the most practically important questions in testing strategy, and it has no single answer — which is itself important to understand.

Confidence in software quality does not increase linearly with the number of tests. It increases with the coverage of meaningful risk. These are not the same thing, and the difference between them determines whether additional testing is an investment or a cost.

The Diminishing Returns of Redundant Tests

Consider a login function. You write a test for a valid username and password. You write another test with a different valid username and password. And another. What confidence does each additional test produce since you are probing the same region of behavior? You are adding volume to an already-sampled area of the input space.

# These three tests cover the same behavioral region.

# Passing all three provides only marginally more confidence than passing one.


def test_login_alice():

    assert login('[email protected]', 'correct_password') == SUCCESS


def test_login_bob():

    assert login('[email protected]', 'correct_password') == SUCCESS


def test_login_carol():

    assert login('[email protected]', 'correct_password') == SUCCESS

Redundant coverage: three tests, one behavioral region.

Contrast this with the following:

# These tests each probe a distinct behavioral region.

# Each one produces a meaningful increment of confidence.


def test_login_valid_credentials():

    assert login('[email protected]', 'correct_password') == SUCCESS


def test_login_wrong_password():

    assert login('[email protected]', 'wrong_password') == FAILURE


def test_login_nonexistent_user():

    assert login('[email protected]', 'any_password') == FAILURE


def test_login_empty_password():

    assert login('[email protected]', '') == FAILURE


def test_login_sql_injection_attempt():

    assert login("' OR '1'='1", 'any') == FAILURE


def test_login_after_account_lockout():

    trigger_lockout('[email protected]')

    assert login('[email protected]', 'correct_password') == LOCKED

Six tests, six distinct behavioral regions. Each adds genuine information.

The second suite produces far more confidence. This is because confidence is a function of the diversity and relevance of evidence, not its volume.

The Point of Saturation

There is a point in any test suite's development where the next test you could write would probe behavior so unlikely, or so low-consequence, that the effort of writing and maintaining it exceeds the risk it mitigates. This is the point of saturation for that area of the system.

Identifying that point requires judgment, not just measurement. It requires an understanding of:

• The risk profile of the system: what failure modes matter most, and how much?
• The probability distribution of actual usage: what conditions will real users encounter?
• The cost of a failure in each region: a failure in payment processing has a different cost than a failure in a help text tooltip.
• The detectability of failures in production: if a failure would be caught immediately by monitoring and easily remediated, the pre-production testing threshold can be lower.

This is why testing is a risk management and confidence discipline, not a completeness and correctness discipline. The goal is not to test everything. The goal is to accumulate sufficient evidence about the right things to make an informed decision to ship — or not to ship.

The Evidence Model: What Do We Actually Know?

We can formulate a practical model that applies equally to a startup's first test suite and to a regulated enterprise system's quality assurance program. It applies to unit tests and to system tests, to manual exploratory testing and to AI-generated test scripts. We can call it the Evidence Model of software testing.

These five concepts replace the naive vocabulary of "passing tests" and "coverage percentages" with a richer language for talking about what a test suite actually knows about a system. 

We can use this model to evaluate specific testing decisions — what kind of evidence they generate, what gaps they leave, and whether the residual risk those gaps represent is acceptable given the context. But the model itself is the foundation. Before you can reason about testing strategy, you need to be clear about what testing is actually doing: it is generating evidence, and evidence has properties that can be evaluated.

What This Means for Leadership

What does this have to do with shipping velocity, team structure, or technology investment?

It has everything to do with all of those things, for the following reason.

Many common and costly quality failures in software organizations are not technical failures. They are situations where the team believed they knew something about their software that they did not in fact know. Testing was done, and the tests passed. The confidence was high. And then the system failed in a way that the tests were incapable of detecting.

When organizations respond to these failures by demanding more tests, faster testing, or automated testing at scale, they are not addressing the cause.

The questions that produce better outcomes are harder, and they require that leaders understand what testing is actually doing:

• Are we testing the right things, or are we testing what is easy to test?
• Does our test suite generate evidence about the risks that would cause the most damage if realised?
• When our tests pass, are we confident because we have strong evidence, or because we have a large volume of evidence in a narrow region?
• When our tests fail, do we learn from the failure — do we ask what it reveals about the gap between our model and reality — or do we simply fix it and move on?
• When we add AI-generated tests to our suite, what evidence are those tests generating, and what regions of risk do they leave unprobed?

The answers to these questions require clarity. And clarity is necessary in a world where AI is reshaping how code is written and what testing looks like. However, it cannot reshape what testing is for.

Wrapping Up

In software, a great deal of what we know comes from testing. The CTO knows whether the platform can absorb next quarter's growth because load tests and failure simulations produced evidence about its limits. The product owner knows whether a feature is ready for customers because acceptance criteria were checked against observed behavior, not assumed behavior. The developer knows whether a refactor preserved existing functionality because a regression suite ran and reported back its feedback. The tester knows where the system is fragile because exploratory sessions and structured test design surfaced behavior no one had anticipated. The DevOps engineer knows whether a deployment is safe to proceed because monitoring, canary analysis, and synthetic checks are themselves a form of testing in production. They generate evidence in real time about how the system behaves under real conditions.

Each of these roles is asking a question, at a different altitude, with different consequences attached to the answer. But the underlying mechanism is the same at every level of the organization: testing produces evidence, evidence becomes feedback, and feedback is what we use to decide what to do next. The CTO's decision to invest in a new architecture. The product owner's decision to ship or hold. The developer's decision to merge. The tester's decision to escalate a risk. The DevOps engineer's decision to roll back. None of these are made in a vacuum. They are made on the strength of the evidence available at the moment the decision is required. Testing, understood this broadly, is not a phase or a department. It is the organization's backbone for converting uncertainty into decisions it can support confidently.