Mutation-testing foundation
exhibit_a.verdict.mutation_testing measures how strongly a generated test
constrains suspect code. It is deliberately a scoring layer, not an evidence gate:
flip_check.py remains the only module that can admit a Case.
Process
discover_mutationstokenizes caller-selected Python source files and emits a deterministic ordered set of allowlisted operator replacements. Optional suspect line sets narrow the surface; input paths are repository-relative and cannot traverse outside the checkout.score_mutationsreruns the candidate on the unmodified pass-state baseline. If that baseline is not deterministically green, no kill rate is reported.- Each mutant is independently applied to a fresh disposable executor copy. Docker execution retains the no-network, dropped-capability, read-only-container policy.
- A mutant is
killedonly when every run fails with the same extractable signature,survivedonly when every run passes, andinvalidwhen unsupported, flaky, timed out, or broken at the harness/environment layer.
The score reports killed / (killed + survived). Invalid mutants remain visible but
are excluded from the denominator. No score can upgrade, downgrade, or fabricate the
raw verdict. The current bug-repro judge emits VERIFIED, PARTIAL, or
UNCERTAIN; FAILED is reserved for a future claim judge that can deterministically
disprove its stated goal.
Initial operator set
- equality and ordering boundaries:
==,!=,<,<=,>,>= - arithmetic operators:
+,-,*,//,% - Boolean literals:
True,False
These are syntax-preserving token edits rather than model-generated patches. The
executor independently rejects any replacement outside the allowlist, even if a
caller constructs a SourceMutation directly. The current foundation targets UTF-8
Python source and requires callers to identify suspect files; broader language and
mutation-operator studies belong in later research commits.