test-run-resource-claims.sh cannot pass on a fresh clone — one scenario reads a gitignored .devwork/ journal #484

New issue

Open

opened 2026-08-31 12:46:13 +00:00 by jbr870 · 0 comments

jbr870 commented

2026-08-31 12:46:13 +00:00

Owner

Spawned from finding F-PO-47-6-3 (pre-existing) on issue #47 during decision D-PO-47-6-3.

The defect

scripts/test-run-resource-claims.sh — #43's gate over the run-resource-claims contract — contains one scenario that cannot pass in a fresh checkout:

scenario_cr70_cr57_comment_misstates_journal_spelling

It reads:

REAL_JOURNAL = .devwork/feature-qa-intra-run-lane-ownership/dispatch-journal.md

and returns 1 when that file is absent:

CR-70: REAL_JOURNAL (…) not present — cannot check the comment's factual claim
against the real correction entry it is about on this checkout

Under this project's tea-cli adapter .devwork/ is gitignored (forge-contract: the issue is the durable record, .devwork/ is worktree-local staging). So that journal was never committed. It exists only in the worktree that produced it.

Why it matters more than one red line

The suite cannot go green on a fresh clone, in CI, or for any operator who is not #43. Its green is reachable on exactly one machine.
A suite that is red-by-default trains readers to skip it. That is the damage that outlives the fixture — and this suite is #43's own gate over its contract, so its failure is precisely where a future regression in that contract would have to surface.
It is also a coverage claim that is not true: the scenario reports itself as covering "the comment's factual claim against the real correction entry", and in every environment but one it covers nothing.

Evidence that it is pre-existing, not #47's doing

Verified rather than assumed during #47's integration, because a red suite at the merge gate has to be attributed before it can be waived:

git archive origin/main | tar -x -C <scratch>   # clean origin/main, no #47 commits
cd <scratch> && git init -q && git add -A && git commit -qm snapshot
bash scripts/test-run-resource-claims.sh
→ rc=1
→ FAIL: scenario_cr70_cr57_comment_misstates_journal_spelling
→ test-run-resource-claims: 1 FAIL(s) (fixture_fallback_count=1)

Identical single failure, same scenario, same fallback count. #47's merge did not cause it and did not make it worse.

Directions (not yet decided)

The scenario's own header explains what it is checking and why it is scoped the way it is, so the intent is recoverable. Options, roughly in increasing order of effort:

Commit a minimal fixture carrying just the DISCOVERY correction block the scenario greps for, and point REAL_JOURNAL at it — the suite already has a fixture_fallback_count, suggesting fixture fallback is an established pattern here.
Skip-with-a-reason rather than fail when the journal is absent, so the suite is green by default and the scenario reports honestly that it did not run. Weaker: a skip that is permanent everywhere is barely better than no scenario.
Drop the scenario if the thing it guards is adequately covered by its neighbours (scenario_cr84_cr74_blind_to_wrapped_reintroduction looks adjacent).

Whichever is chosen, the general rule worth extracting: a test may not depend on a gitignored artifact. If the artifact is the contract, commit a fixture of it; if it is incidental, do not assert on it.

Filed by /dev:integrate on issue #47. Component: component:shared (the harness). Surface belongs to #43 (qa-intra-run-lane-ownership).

Spawned from finding `F-PO-47-6-3` (pre-existing) on issue #47 during decision `D-PO-47-6-3`. ## The defect `scripts/test-run-resource-claims.sh` — #43's gate over the run-resource-claims contract — contains one scenario that **cannot pass in a fresh checkout**: ``` scenario_cr70_cr57_comment_misstates_journal_spelling ``` It reads: ``` REAL_JOURNAL = .devwork/feature-qa-intra-run-lane-ownership/dispatch-journal.md ``` and returns 1 when that file is absent: ``` CR-70: REAL_JOURNAL (…) not present — cannot check the comment's factual claim against the real correction entry it is about on this checkout ``` Under this project's `tea-cli` adapter **`.devwork/` is gitignored** (forge-contract: the issue is the durable record, `.devwork/` is worktree-local staging). So that journal was never committed. It exists only in the worktree that produced it. ## Why it matters more than one red line - The suite **cannot go green on a fresh clone, in CI, or for any operator who is not #43**. Its green is reachable on exactly one machine. - A suite that is red-by-default trains readers to skip it. That is the damage that outlives the fixture — and this suite is #43's own gate over its contract, so its failure is precisely where a future regression in that contract would have to surface. - It is also a **coverage claim that is not true**: the scenario reports itself as covering "the comment's factual claim against the real correction entry", and in every environment but one it covers nothing. ## Evidence that it is pre-existing, not #47's doing Verified rather than assumed during #47's integration, because a red suite at the merge gate has to be attributed before it can be waived: ``` git archive origin/main | tar -x -C <scratch> # clean origin/main, no #47 commits cd <scratch> && git init -q && git add -A && git commit -qm snapshot bash scripts/test-run-resource-claims.sh → rc=1 → FAIL: scenario_cr70_cr57_comment_misstates_journal_spelling → test-run-resource-claims: 1 FAIL(s) (fixture_fallback_count=1) ``` Identical single failure, same scenario, same fallback count. #47's merge did not cause it and did not make it worse. ## Directions (not yet decided) The scenario's own header explains what it is checking and why it is scoped the way it is, so the intent is recoverable. Options, roughly in increasing order of effort: 1. **Commit a minimal fixture** carrying just the DISCOVERY correction block the scenario greps for, and point `REAL_JOURNAL` at it — the suite already has a `fixture_fallback_count`, suggesting fixture fallback is an established pattern here. 2. **Skip-with-a-reason** rather than fail when the journal is absent, so the suite is green by default and the scenario reports honestly that it did not run. Weaker: a skip that is permanent everywhere is barely better than no scenario. 3. **Drop the scenario** if the thing it guards is adequately covered by its neighbours (`scenario_cr84_cr74_blind_to_wrapped_reintroduction` looks adjacent). Whichever is chosen, the general rule worth extracting: **a test may not depend on a gitignored artifact.** If the artifact is the contract, commit a fixture of it; if it is incidental, do not assert on it. --- *Filed by `/dev:integrate` on issue #47. Component: `component:shared` (the harness). Surface belongs to #43 (qa-intra-run-lane-ownership).*

jbr870 added the

labels

2026-08-31 12:46:13 +00:00

jbr870 referenced this issue

2026-08-31 12:46:18 +00:00

release-pipeline-status.sh --ref returns a stale pipeline — a false green immediately before a production tag #47