jbr870/devwork-skills
1
0
Fork 0
Code Issues 64 Pull requests Projects Releases Packages Wiki Activity Actions

Baseline rule: a gate's output goes to a file and its exit code comes from the command, never a pipeline #49

New issue
Closed
opened 2026-08-14 13:01:56 +00:00 by jbr870 · 1 comment
jbr870 commented 2026-08-14 13:01:56 +00:00
Owner
Copy link

Rank 13 of 13 in #37 · est. saving ~6m directly; larger as a false-signal class

Evidence

Three occurrences in this feature, all the same shape — a gate's real result hidden by the shell
plumbing around it:

  1. Version-stamp chore, 2026-08-13 10:24. pnpm test 2>&1 | tail -25 truncated past the
    summary, so the entire 5m 40s suite was run a second time with a different filter.
  2. QA, 16:52 UTC. A grep pipeline made the harness record exit code 0 while the suite had
    actually failed; the orchestrator noted "the harness recorded 'exit code 0' because my grep
    succeeded"    . This fed the 18-minute false alarm in #43.
  3. QA, 16:14. A head -40 pipe sent SIGPIPE to a server the model had just started, killing it
    mid-request and producing an error boundary it briefly read as a product defect.

There is precedent for the fix shape: commit e3a5369 (#33) already requires rc-checked stdout
for every helper invocation in phase-outcome. This generalises the same rule to gate commands.

Proposed change

A rule in the portability/verification baseline: a gate command writes its full output to a file
and its exit code is read from the command, never from a pipeline. Filter the file afterwards.
Concretely — cmd > "$OUT" 2>&1; rc=$? then grep $OUT, never cmd | tail.

Two corollaries worth stating: a pipeline's exit status is the last stage's, so | grep and
| tail silently rewrite a red gate green; and a truncating reader can SIGPIPE a long-running
producer it is reading from.

Acceptance

  • The rule is in the verification/portability baseline where skill authors will meet it.
  • scripts/lint-conventions.sh flags pnpm test|lint|typecheck|build piped directly into a filter
    in skill-emitted fences.
  • Existing skill text that pipes a gate into tail/grep is corrected.
**Rank 13 of 13** in #37 · est. saving **~6m directly; larger as a false-signal class** ## Evidence Three occurrences in this feature, all the same shape — a gate's real result hidden by the shell plumbing around it: 1. **Version-stamp chore, 2026-08-13 10:24.** `pnpm test 2>&1 | tail -25` truncated past the summary, so the entire **5m 40s** suite was run a second time with a different filter. 2. **QA, 16:52 UTC.** A `grep` pipeline made the harness record `exit code 0` while the suite had actually failed; the orchestrator noted *"the harness recorded 'exit code 0' because my grep succeeded"*. This fed the 18-minute false alarm in #43. 3. **QA, 16:14.** A `head -40` pipe sent SIGPIPE to a server the model had just started, killing it mid-request and producing an error boundary it briefly read as a product defect. There is precedent for the fix shape: commit `e3a5369` (#33) already requires *rc-checked stdout* for every helper invocation in `phase-outcome`. This generalises the same rule to gate commands. ## Proposed change A rule in the portability/verification baseline: **a gate command writes its full output to a file and its exit code is read from the command, never from a pipeline.** Filter the file afterwards. Concretely — `cmd > "$OUT" 2>&1; rc=$?` then grep `$OUT`, never `cmd | tail`. Two corollaries worth stating: a pipeline's exit status is the *last* stage's, so `| grep` and `| tail` silently rewrite a red gate green; and a truncating reader can SIGPIPE a long-running producer it is reading from. ## Acceptance - The rule is in the verification/portability baseline where skill authors will meet it. - `scripts/lint-conventions.sh` flags `pnpm test|lint|typecheck|build` piped directly into a filter in skill-emitted fences. - Existing skill text that pipes a gate into `tail`/`grep` is corrected.
jbr870 commented 2026-08-25 10:22:27 +00:00
Author
Owner
Copy link

/meta-amend outcome — amended

Commit 6174683 on main.

Verified at HEAD (2e0e43e)

  • The rule was absent from the portability baseline. The nearest thing was the rc-checked stdout requirement (phase-outcome.md, from #33/e3a5369), which governs helper invocations only — the issue's own read of the precedent is correct, and it had not been generalised to gate commands.
  • AC3 was already satisfied. Grepping the shipped tree for a gate piped into a filter returns nothing: the only | head in skill markdown is git worktree list | head -1 inside a [ … ] test in integrate/SKILL.md, which is not a gate and whose verdict is the test's, not the pipeline's. Nothing needed correcting — recorded here rather than presented as work done.

The amendment

The rule lands in three places, none restating another:

  1. CLAUDE.md → Portability baseline (AC1). "A gate's output goes to a file; its exit code comes from the command, never a pipeline." cmd > "$OUT" 2>&1; rc=$?, then filter $OUT. All three recorded failure modes named, and tied back to #33: an output is trustworthy only once you have checked the status of the thing that produced it.
  2. _shared/procedures/prerequisites.md Step 1 — the runtime half. This is where every skill resolves the project's test/lint/typecheck commands, so it is where a skill actually about to run one will meet the rule. Carries the shape as a fence plus the three failure modes, because "don't pipe a gate" is only obviously right once you have seen each way it goes wrong.
  3. scripts/lint-conventions.sh — the gate (AC2). New tier-1 fence check.

(Note on where the authoring rule went: CLAUDE.md is the operator's instructions file and this queue came from a peer session, so I did not edit it on a peer's say-so — I asked Jochem directly and he approved both edits, including keeping the Verification section's description of the linter accurate.)

The linter check, and how it was verified

Two gate shapes are matched — a package manager plus a gate verb (pnpm test, npm run lint, make build, go test, cargo test), and a test runner invoked directly or via npx/bunx/pnpm exec (pytest, vitest, jest, tsc, …) — piped into tail|head|grep|awk|sed|less|more.

The gate must sit at a command position (start of the fence line, or after &&/||/;). Without that anchor, a runner merely named mid-line reads as a gate — grep -r vitest . | head -5 was a real false positive during development, and it is exactly the kind of check that gets disabled once it cries wolf.

Verified by executing, not by inspection — 18 cases:

  • flags all 10 pipes: pnpm test 2>&1 | tail -25, pnpm test | grep -E 'FAIL', npm run lint | head -40, npx vitest run 2>&1 | tail -5, go test ./... | grep FAIL, pytest tests/ -v | tail -20, make build 2>&1 | sed -n '1,20p', cargo test | less, cd apps/web && pnpm test | tail -20, npx tsc --noEmit 2>&1 | grep error
  • stays quiet on all 8 non-gates: grep -r vitest . | head -5, pnpm install | tail -5, git worktree list | head -1, jq -r .body <<<"$got1" | head -1, echo "run pytest first" | grep pytest, and the redirect-then-filter form the rule prescribes
  • fires in situ: a planted pnpm test 2>&1 | tail -25 fence in develop/SKILL.md produced plugin/skills/develop/SKILL.md:692: gate command piped into a filter and exit 1; reverted, and the tree lints clean at 693 fence lines.

Desk-check against the originating scenario

Occurrence Under the rule
Version-stamp chore, 10:24 — pnpm test 2>&1 | tail -25 truncated past the summary, 5m 40s suite re-run Linter flags the fence at authoring time; the full output is in $OUT and re-readable with a different filter, so there is nothing to re-run
QA 16:52 — grep pipeline made the harness record exit 0 while the suite had failed rc is the suite's, so the gate reads red. This is the one that mattered: it fed the 18-minute false alarm in #43
QA 16:14 — head -40 SIGPIPEd a just-started server No pipe from the producer, so no SIGPIPE; the error boundary that looked like a product defect never appears

All three acceptance criteria met: the rule is in the baseline and in the runtime procedure; the linter flags the shape in skill-emitted fences; and the shipped tree carried nothing to correct.

What validates it next

The linter is the validation — it runs on every meta-lane commit and the check is proven to fire. The residual risk is coverage, not correctness: the runner list is finite, so a project on a runner outside it (a bare ./gradlew check alias, a shell wrapper) gets the prose rule without the gate. If that shows up in a dogfood run, extend the alternation rather than loosening the command-position anchor — the anchor is what keeps the check trustworthy.

## /meta-amend outcome — amended Commit `6174683` on `main`. ### Verified at HEAD (`2e0e43e`) - The rule was absent from the portability baseline. The nearest thing was the **rc-checked stdout** requirement (`phase-outcome.md`, from #33/`e3a5369`), which governs *helper invocations* only — the issue's own read of the precedent is correct, and it had not been generalised to gate commands. - **AC3 was already satisfied.** Grepping the shipped tree for a gate piped into a filter returns nothing: the only `| head` in skill markdown is `git worktree list | head -1` inside a `[ … ]` test in `integrate/SKILL.md`, which is not a gate and whose verdict is the test's, not the pipeline's. Nothing needed correcting — recorded here rather than presented as work done. ### The amendment The rule lands in three places, none restating another: 1. **`CLAUDE.md` → Portability baseline (AC1).** *"A gate's output goes to a file; its exit code comes from the command, never a pipeline."* `cmd > "$OUT" 2>&1; rc=$?`, then filter `$OUT`. All three recorded failure modes named, and tied back to #33: *an output is trustworthy only once you have checked the status of the thing that produced it.* 2. **`_shared/procedures/prerequisites.md` Step 1 — the runtime half.** This is where every skill resolves the project's test/lint/typecheck commands, so it is where a skill actually about to run one will meet the rule. Carries the shape as a fence plus the three failure modes, because "don't pipe a gate" is only obviously right once you have seen each way it goes wrong. 3. **`scripts/lint-conventions.sh` — the gate (AC2).** New tier-1 fence check. *(Note on where the authoring rule went: `CLAUDE.md` is the operator's instructions file and this queue came from a peer session, so I did not edit it on a peer's say-so — I asked Jochem directly and he approved both edits, including keeping the Verification section's description of the linter accurate.)* ### The linter check, and how it was verified Two gate shapes are matched — a package manager plus a gate verb (`pnpm test`, `npm run lint`, `make build`, `go test`, `cargo test`), and a test runner invoked directly or via `npx`/`bunx`/`pnpm exec` (`pytest`, `vitest`, `jest`, `tsc`, …) — piped into `tail|head|grep|awk|sed|less|more`. **The gate must sit at a command position** (start of the fence line, or after `&&`/`||`/`;`). Without that anchor, a runner merely *named* mid-line reads as a gate — `grep -r vitest . | head -5` was a real false positive during development, and it is exactly the kind of check that gets disabled once it cries wolf. Verified by executing, not by inspection — 18 cases: - **flags all 10 pipes:** `pnpm test 2>&1 | tail -25`, `pnpm test | grep -E 'FAIL'`, `npm run lint | head -40`, `npx vitest run 2>&1 | tail -5`, `go test ./... | grep FAIL`, `pytest tests/ -v | tail -20`, `make build 2>&1 | sed -n '1,20p'`, `cargo test | less`, `cd apps/web && pnpm test | tail -20`, `npx tsc --noEmit 2>&1 | grep error` - **stays quiet on all 8 non-gates:** `grep -r vitest . | head -5`, `pnpm install | tail -5`, `git worktree list | head -1`, `jq -r .body <<<"$got1" | head -1`, `echo "run pytest first" | grep pytest`, and the redirect-then-filter form the rule prescribes - **fires in situ:** a planted `pnpm test 2>&1 | tail -25` fence in `develop/SKILL.md` produced `plugin/skills/develop/SKILL.md:692: gate command piped into a filter` and exit 1; reverted, and the tree lints clean at 693 fence lines. ### Desk-check against the originating scenario | Occurrence | Under the rule | |---|---| | Version-stamp chore, 10:24 — `pnpm test 2>&1 \| tail -25` truncated past the summary, 5m 40s suite re-run | Linter flags the fence at authoring time; the full output is in `$OUT` and re-readable with a different filter, so there is nothing to re-run | | QA 16:52 — grep pipeline made the harness record exit 0 while the suite had failed | `rc` is the suite's, so the gate reads red. This is the one that mattered: it fed the 18-minute false alarm in #43 | | QA 16:14 — `head -40` SIGPIPEd a just-started server | No pipe from the producer, so no SIGPIPE; the error boundary that looked like a product defect never appears | All three acceptance criteria met: the rule is in the baseline **and** in the runtime procedure; the linter flags the shape in skill-emitted fences; and the shipped tree carried nothing to correct. ### What validates it next The linter is the validation — it runs on every meta-lane commit and the check is proven to fire. The residual risk is coverage, not correctness: the runner list is finite, so a project on a runner outside it (a bare `./gradlew check` alias, a shell wrapper) gets the prose rule without the gate. If that shows up in a dogfood run, extend the alternation rather than loosening the command-position anchor — the anchor is what keeps the check trustworthy.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
interactive-skills
No results found.
Labels
Clear labels   
component:adapters

   
component:lifecycle

   
component:qa

   
component:setup

   
component:shared

   
component:worktrees

   
phase/accepted

   
phase/backlog

   
phase/deployed

   
phase/developing

   
phase/integrating

   
phase/planning

   
phase/qa

   
phase/requirements

   
phase/uat

   
priority:critical

   
priority:critical

   
priority:high

   
priority:high

   
priority:low

   
priority:low

   
priority:medium

   
priority:medium

   
type:bug

   
type:chore

   
type:docs

   
type:feature

   
type:infra

   
type:tech-debt
No labels
component:adapters
component:lifecycle
component:qa
component:setup
component:shared
component:worktrees
phase/accepted
phase/backlog
phase/deployed
phase/developing
phase/integrating
phase/planning
phase/qa
phase/requirements
phase/uat
priority:critical
priority:critical
priority:high
priority:high
priority:low
priority:low
priority:medium
priority:medium
type:bug
type:chore
type:docs
type:feature
type:infra
type:tech-debt
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jbr870/devwork-skills#49
No description provided.