All writing

When it breaks

The test that was loosened to make a release

Every mature estate contains assertions that were weakened under deadline and never restored. Finding them is straightforward; the hard part is the conversation.

1 minute read

It happens on a Thursday. A release is blocked by a uniqueness test that fails on four rows out of nine million, the four rows are a known data-quality issue at source, the fix is somebody else's team, and the release has to go. So the test gets a `where` clause, or a severity of warn, or a threshold of five. It ships. The source issue is never fixed, because nothing fails any more.

Why these are worth hunting

A loosened test is worse than no test. No test is a known gap. A loosened test reads as coverage on every dashboard and in every audit, while guarding nothing — and the number it was loosened to is almost always a number from the day it was loosened, not a number anybody would choose today.

What to look for

  • Tests with a row threshold, especially a small arbitrary one, and especially where the current failing count is close to it.
  • Tests downgraded to warn where nothing consumes warnings.
  • Filters inside a test that exclude a named source, region or date range.
  • Assertions disabled by configuration rather than removed, which keeps them in the count.

The conversation, not the commit

Tightening one of these is a two-line change that can break a nightly run for a team that does not know you exist. The useful output is not a pull request — it is a list, with the current state of each: how many rows fail today against the threshold, when the threshold was introduced, and what would happen tonight if it were restored. That list is something an owner can work through. A surprise failing build is not.

Run this against your own estate

Gecko reads metadata and query history — never the rows in your tables — and tells you what it found.

Get started