Shipmoor 0.7.0 shipped the first cut of Claim Check: an acceptance set, a deterministic floor, a bring-your-own judge for the judgment calls the floor can’t reach mechanically. It worked, and it was honest about what it couldn’t do yet. Underneath the verdict, only about a third of the original seven-stage design was actually wired. Every acceptance criterion the floor couldn’t check fell straight to the judge, every run, with no way to close that gap for good.
Shipmoor 0.8.0 wires the rest of it. The acceptance set now has a real approval step. A run that isn’t fully READY tells you exactly what to do about it. And a companion capability, Test Evidence, joins the fabric to answer a narrower, sharper question: did your tests actually pass, or did something just make the exit code say so.
The acceptance set now has an approval step
In 0.7.0, you either committed an acceptance set by hand or the run had nothing to check against. 0.8.0 adds a middle path: if none exists, Claim Check can auto-derive one from your stated intent and write it as a draft, .shipmoor/acceptance.yaml.draft, rather than the live acceptance file. A draft is disclosed as unapproved. An otherwise-clean run against an unapproved draft reads INCONCLUSIVE, not READY: the obligations exist, but nobody has agreed they’re the right ones yet. Approval is committing the file. There’s no flag that skips this.
Intent itself also has a clearer home now. --intent-ticket and --intent-prompt replace the old --intent/--prompt/--session trio on the gate command itself (that older three-flag surface still exists, just moved to the claim-check resolve preview utility). A ticket outranks a prompt, a prompt outranks the acceptance file’s own stated goal, and if your sources don’t roughly agree with each other, confidence drops rather than picking one arbitrarily.
The fix packet now tells you what to write, and checks it itself
This is the headline change. In 0.7.0, a judgment obligation the floor couldn’t decide just sat open, run after run, with a rationale attached and nowhere to go. In 0.8.0, that obligation comes with an exact tag string, and writing a test that carries it is how you close the gap for good.
Here’s what that looks like on a real Kubernetes admission-controller change, the same repo the 0.7.0 post used:
Claim Check BLOCKED · exit 1
✓ The webhook rejects a pod with hostNetwork: true.
✓ The change compiles and the existing suite stays green.
⊘ Only cluster admins can grant the privileged-pod exemption.
requirement: stage3.byo_judge · required · result: blocked
rationale: the exemption check reads a namespace label, not the caller's RBAC role,
so a namespace owner without admin rights can still grant it
evidence: judge confirmed via 3/3 sample agreement
∅ Denied requests are logged with a reason, not just an HTTP code.
requirement: stage1.floor.deny_reason_logged · required · result: unbound
tag: ac_04_9f2ac4d1Alongside that report is a fix packet naming precisely what to do next: fix the RBAC check the judge flagged, and write a test carrying the tag ac_04_9f2ac4d1 in its id or filename. Not a fuzzy description, an exact substring. Claim Check doesn’t read your test’s assertions to decide if you did it right; your test does that. What it checks is narrower and mechanical: does a test carrying that tag exist, does it fail on the base commit, and does it pass at the head commit. A test that already passed before your change proves nothing and won’t bind.
Claim Check READY · exit 0
✓ The webhook rejects a pod with hostNetwork: true.
✓ The change compiles and the existing suite stays green.
✓ Only cluster admins can grant the privileged-pod exemption.
✓ Denied requests are logged with a reason, not just an HTTP code.
requirement: stage1.floor.deny_reason_logged · required · result: satisfied
evidence: tests/webhook/test_deny_reason.py::test_ac_04_9f2ac4d1 (fail→pass)Once that obligation binds to a real, executed test, it never goes back to the judge. It’s the floor’s for good, and the second run needed no judge call at all to confirm it.
The gate still doesn’t loosen
One thing didn’t change in spirit, and got more explicit in practice: BLOCKED and INCONCLUSIVE map to a non-zero exit code, unconditionally. There’s no policy flag that softens this. If you want the judge’s opinion visible in CI without giving it the power to fail a merge, run it as a separate, deliberately non-blocking step; there’s no first-party advisory mode on the gate itself. The floor’s own two cross-cutting toggles (a scan-severity threshold, artifact validation) are the only real dial, and every core requirement stays enforced regardless of how you set them.
The judge’s own calibration also moved: three samples instead of five, two-of-three agreement instead of four-of-five, recalibrated down to bound cost while most obligations are still large. A confirmed block that cites a specific test or command is now executed hermetically at the reviewed head before it counts; a cited failure that doesn’t actually reproduce doesn’t survive as a block. The judge still only ever blocks or abstains. A READY verdict always rides the deterministic floor.
Test Evidence: a green checkmark now has to earn it
A coding agent under pressure to turn a suite green has a few shortcuts available: delete the failing test, gut an assertion into something that can never fail, or pipe the run through something that swallows a red exit code. Test Evidence, GA in this release, answers three narrow questions instead of trusting a bare “tests passed” claim:
passed: did the tests genuinely run and pass, harvested from a hook wired into your coding agent’s own tool calls rather than re-run from scratch, bound to the exact working-tree state so a single uncommitted edit makes a stale result show its age.reduced: did a test that existed at the base commit silently disappear by the head commit. The highest-value cheat, caught by a simple set difference, with a disclosed, reasoned escape hatch for a genuine removal.weakened: was a kept test gutted rather than deleted: an assertion emptied, an early return added before the real checks, an expected failure loosened. There’s no deterministic rule for this, so it rides the same judge machinery Claim Check uses, with one deliberate constraint: the schema has nopassfield. It can report a weakening or say nothing. It can never certify a test as fine.
We measured this against five scenarios in our own evaluation harness before shipping it: a real passing run, a deleted test, the same deletion properly disclosed, a gutted assertion, and a legitimate rename that keeps its real assertions. Zero false positives, zero false negatives, across all five. Harvesting a real result runs at roughly a quarter of the cost of a naive re-run for a representative suite, and the advantage widens as the suite gets bigger.
Test Evidence is a fully standalone command, shipmoor test-evidence, with its own report. It’s also automatically composed into shipmoor claim-check for any entitled caller: passed decides the gate’s own tests cell with real counts, reduced adds its findings alongside the gate’s existing anti-cheat checks, weakened rides the judge channel exactly like Claim Check’s own judge, subtractive only. You don’t have to run it separately for Claim Check to benefit; you do want the hook wired upstream in your agent’s own settings for the strongest signal.
The CLI surface
shipmoor claim-check [target] # the floor, plus the judge in full mode
shipmoor claim-check init # scaffold a starter acceptance set
shipmoor claim-check resolve # resolve intent without a diff
shipmoor claim-check probes {gaps,scaffold,coverage}
shipmoor test-evidence [target] # standalone: passed, reduced, weakened
shipmoor test-evidence hook # wired as a PostToolUse hook, never gatedThe flags that matter on the gate: --intent-ticket / --intent-prompt for frozen intent, --floor-only for the deterministic-only path, --agent for the judge (claude is the only bare preset with a built-in invocation today; anything else needs its full command), --vsa-out for the attestation. Full detail on both commands lives in the docs.
Install and upgrade
curl -fsSL https://dl.shipmoor.dev/install.sh | bash
shipmoor version
# shipmoor 0.8.0
shipmoor claim-check . --agent claudeIf you pinned Shipmoor in CI on the exit-code contract, that contract is unchanged: READY/READY WITH GAPS still exit 0, BLOCKED/INCONCLUSIVE still exit 1. What’s new is everything upstream of that verdict: a real approval step, a fix packet that closes its own gaps, and a second producer that stops taking a green checkmark at its word.
- Claim Check docs: the full reference, updated for 0.8.0.
- Test Evidence docs: the three signals, in depth.
- How Claim Check works: the design walkthrough.
The reviewer’s last hard question was never “does it compile.” It was “did this change do what it claimed, and did the tests that say so actually mean it.” 0.8.0 is our answer to both halves at once.
- From the
shipmoor.devEngineering Team