Proving a fix before proposing it

Anybody can write a plausible change to a failing model. The hard and valuable part is the sentence that follows it: here is why this is correct, and here is what I ran to establish that. Without it, a reviewer has to redo the whole investigation to approve a diff, which costs more than writing the fix themselves.

Three levels of evidence, in increasing order of worth

• It parses and compiles. Establishes that the change is syntactically real, nothing more.
• The affected tests pass. Establishes that the change does not break what was already guarded.
• The output is what it should be, demonstrated against the data. Establishes the change is right.

Most tooling stops at the first level and reports it as success. The second is table stakes and still relatively rare. The third is where review time actually goes, and it is the one that requires reading the data rather than the code: the row count before and after, the null rate on the column in question, the specific rows that changed and why they should have.

The failure mode to design against

There is a category of fix that makes the symptom disappear without addressing the cause: deleting the test, loosening the assertion until it cannot fail, filtering out the rows that expose the problem. Each produces a green build and leaves the estate worse. Any automated system that proposes changes needs an explicit rule against this — and, more importantly, needs to be required to say in writing why the rows it excluded are legitimately out of scope, if it excludes any.

A precise diagnosis with no patch is more useful than a plausible patch that hides the problem.

Evidence has to travel with the change

The queries that were run, their results, the test output, the diff: all of it on the same screen as the decision. A link to a build log in another system is not evidence, it is a second errand, and reviewers who are busy will approve without clicking it.