Categories
Allgemein

Try to Falsify Your Theories Before You Act On Them

Edsger Dijkstra once stated that “[…] program testing can be used very effectively to show the presence of bugs but never to show their absence.” A passing test does not prove you are right. A failing test, however, gives you something solid: an assumption that did not survive contact with reality.

This asymmetry shapes how I think about writing software. Every line of code is based on a theory—about what a function does, how hardware behaves, what the user actually needs. The goal is not to prove yourself correct. It is to find out where you are wrong, as early and as cheaply as possible, before that wrongness gets baked into software you will ship.

Why formulation matters

Most programmers test their assumptions from time to time. You have a hunch something might be slow, so you measure it. You suspect a race condition, so you write a stress test. That is already good practice. But there is a difference between having an assumption in the back of your mind and stating it as an explicit theory. The difference matters in three ways.

First, a properly stated theory opens itself to critique. “This interrupt handler might miss events under load” is a vague worry. You cannot easily disprove it because it does not commit to anything specific. “If the interrupt fires slower than every 200 microseconds, the handler will still process every event” is a theory. It makes a concrete claim. You can write a test that generates interrupts at increasing rates and checks whether the dropout pattern matches the prediction. The precision invites falsification. The vagueness does not.

Second, unstated assumptions feed decision paralysis. When you have not laid out what you believe and how strongly you believe it, every option looks equally risky. You end up stuck because you cannot weigh one concern against another. Formulating “I am 80% confident this message queue blocked because of excessive load in the other process” gives you something to act on. It also tells you exactly where to invest more investigation if that 20% doubt turns out to matter. You move forward with strengthened confidence instead of standing still with diffuse anxiety.

Third, explicit theories make conversations with coworkers more productive. “I chose to apply a moving average before processing this sensor data; I found it to differ by 200 mV between cycles.” That sentence explains the reasoning. A colleague can now engage with the theory directly. Maybe they know something about the sensor or measurement that weakens it. Maybe they can suggest a simpler approach that still satisfies the constraint. What they cannot do is stare at the code and wonder what you were thinking. The theory documents the intent.

The day-to-day cycle

Once the theory is stated, the next step is finding the cheapest way to challenge it. The cost of being wrong rises sharply once code has been integrated, tested on hardware, and deployed. So you want the falsification attempt to happen as early and as lightly as possible.

I ran into this with a popular theory about performance among my coworkers. They have been inlining logic by hand because the theory was: “Function call overhead in this hot path will eat into the timing budget enough to matter.” That is a statement you can check. I collected some timing measurements comparing the inlined version against one with proper function calls. Every other aspect of the code had larger impact than the manual inlining. The measured difference was noise. The theory was false. Thirty minutes saved me from continuing to write harder-to-read code for no reason.

The check does not need to be elaborate. A unit test that exercises a boundary condition is cheap. A small code experiment that runs on the bench is cheap. Asking the engineer next to you “does this match your understanding?” is cheap. The key is that the check must be capable of returning a clear “no” if the theory is wrong. If the test cannot fail, it cannot teach you anything.

I acknowledge that cheap might be a relative term if it comes to safety-critical applications or the implications of a false “no” are otherwise large. Adjust your efforts accordingly.

Testing assumptions about the world

Not all theories are about code behaviour. Some are about the environment the code operates in. These domain assumptions cause just as much waste when they are wrong, but they can be harder to spot because you question them less frequently.

I recently needed to decide whether to keep supporting a particular hardware configuration. The configuration complicated the code with extra branches and special cases. My theory was: “Nobody is using this configuration in the field anymore.” I could have acted on that hunch, or I could have carried the complexity forward out of caution. Instead, I asked the team that manages customer configurations. Ten minutes of conversation confirmed the configuration had been absent from all deployments for years. The theory held, and we removed the code.

That is the same falsification logic applied to the problem domain rather than the implementation. The check was cheap—a conversation instead of a test script—but it served the same purpose. It let reality push back before I committed.

The distinction between essential and accidental complexity runs through this. A system that carries accidental complexity becomes harder to test, harder to modify, and harder to reason about. But you cannot tell which complexity is essential without testing your beliefs about the problem. Expert interviews, searching through field logs, checking with the team that handles customer reports—these are all ways to surface wrong assumptions about the world your code lives in.

Misguided effort

Wrong theories cost you in two ways. The first is direct: you build something based on a false belief, and it breaks. The second is more subtle: you build something based on a belief that is not false, but also not worth spending time on. You optimize a code path that runs once at boot. You guard against a timing condition that cannot occur with your scheduler. You design around a constraint that does not actually constrain anything.

This second category is harder to notice because the code works correctly. It just does not address anything relevant. The only defence is to state the belief clearly enough that you can ask: is this worth my attention right now? “Register spilling is the performance bottleneck” is a theory you can check. But the prior theory — “performance in this code path is currently a problem we need to solve” — also needs checking.

Summing up

The argument is not that you need certainty before you act. It is that the steps should be small, the theories should be written down or spoken aloud, and the checks should be cheap. When each piece of a system rests on a stated and examined belief, the whole holds up better. When something does go wrong, you can trace it back to a specific theory and re-examine it.

The habit builds over time. You start reaching for performance logs, the integration test, or the colleague with domain knowledge before you reach for the editor. You get better at phrasing your hunches in ways that can be shot down. A ten-minute check that saves a week of rework is not a distraction from the real work. It is the real work, done in the right order.