The test suite nobody trusted

A team has 1,400 tests. The suite takes 40 minutes and fails about one run in three, always on something unrelated.

So people re-run it. Then they re-run it twice. Then somebody adds a rule that you may merge if it fails on a "known flaky" test, and within a month nobody reads the results at all.

The suite still exists. It is no longer a test suite; it is a delay.

The failure was trust, not coverage

A test that fails at random teaches people to ignore failures. Once that habit forms, the real failure it eventually catches is ignored too, which is the whole point of having tests.

So an intermittent test is not a minor annoyance to be retried. It is either a real race in the code, or a timing assumption in the test, and both are worth a day to find.

What to test

Test the behaviour, not the implementation. A test asserting that a repository method was called with certain arguments breaks whenever the method is renamed and passes while the result is wrong. A test asserting that the order total is 180 survives refactoring and fails when the total is wrong, which is the point.

A rough shape most teams end up with:

Not because the pyramid is sacred, but because the wide base is cheap and fast and the narrow top is slow and brittle.

What not to test

Getters. Framework behaviour. Third-party libraries. A mock returning what you told it to return.

Each of those costs maintenance and catches nothing, and a suite full of them is slow for no benefit, which is how you end up at forty minutes.