Resources
Engineering notes.
The hard part is being right about what is broken and why. Three arguments about that, written as the standard the Fixer and the Verifier had to meet, before either shipped.
01The notes
Each grounded in a file you can name.
- Engineering noteVerification is not testingA test written after a patch, by the process that wrote the patch, proves the test agrees with the patch. It does not prove the defect existed, or that it is gone.packages/engine/src/verify.ts · packages/shared/src/evidence.tsRead
- Engineering noteThe false modification problemAn agent that edits working code is worse than an agent that fixes less. Abstention has to be a first-class outcome, with its own success criteria, and it has to be measured.credda-bench cases · issue-already-resolved · working-as-intendedRead
- Engineering noteFailure signatures: comparing failures, not logsDiffing raw command output answers the wrong question. The question is whether this is the failure that was reported. The dangerous answer is a real failure that is the wrong one.packages/evidence/src/signature.tsRead
02Benchmark
The numbers moved to their own page.
Four corpora, kept apart, on the benchmark page.