jbr870/devwork-skills

Fork 0

qa domains: briefs point subagents at unexpandable ~/.claude paths — 12 failed reads, silent-coverage-loss shape #35

New issue

Closed

opened 2026-08-13 13:05:29 +00:00 by jbr870 · 1 comment

jbr870 commented

2026-08-13 13:05:29 +00:00

Owner

Observation

QA domain briefs instruct subagents to Read ~/.claude/skills/_shared/procedures/playwright-conventions.md — verified at HEAD: domains/e2e.md:110,240,289 and domains/a11y.md:376,411. The Read tool does not expand ~, and the path also hard-assumes a user-level install. In the slot3/#191 run this cost 12 failed reads across 3 QA agents, each guessing a different wrong home (/home/jochem/, worktree-local .claude/) before finding the devcontainer's $HOME=/home/node. Sharp edge per the report: an agent that gave up would run its domain with no conventions checklist and produce an identical-looking report — silent coverage loss.

Recurrence

Mechanical — verified at HEAD (5 occurrences across 2 domain docs). One observed run, but the defect is in the shipped text, not the run.

Hypothesis

The lead resolves the conventions doc's absolute path once (it knows where it loaded skills from) and passes it in the brief — or inlines the few rules the brief actually needs. Any ~/… literal in a brief handed to a subagent is the same bug; worth a lint check for ~/.claude inside domain/procedure text.

Suggested lane

meta-amend (text + lint check).

Filed by /meta-retrospective 2026-08-13 (slot3 deep-dive).

## Observation QA domain briefs instruct subagents to Read `~/.claude/skills/_shared/procedures/playwright-conventions.md` — verified at HEAD: `domains/e2e.md:110,240,289` and `domains/a11y.md:376,411`. The Read tool does not expand `~`, and the path also hard-assumes a user-level install. In the slot3/#191 run this cost 12 failed reads across 3 QA agents, each guessing a different wrong home (`/home/jochem/`, worktree-local `.claude/`) before finding the devcontainer's `$HOME=/home/node`. Sharp edge per the report: an agent that gave up would run its domain **with no conventions checklist and produce an identical-looking report** — silent coverage loss. ## Recurrence Mechanical — verified at HEAD (5 occurrences across 2 domain docs). One observed run, but the defect is in the shipped text, not the run. ## Hypothesis The lead resolves the conventions doc's absolute path once (it knows where it loaded skills from) and passes it in the brief — or inlines the few rules the brief actually needs. Any `~/…` literal in a brief handed to a subagent is the same bug; worth a lint check for `~/.claude` inside domain/procedure text. ## Suggested lane meta-amend (text + lint check). *Filed by /meta-retrospective 2026-08-13 (slot3 deep-dive).*

jbr870 added the

component:qa

type:bug

labels

2026-08-13 13:05:29 +00:00

jbr870 referenced this issue

2026-08-22 06:44:45 +00:00

Ship the suite as a Claude Code marketplace plugin (repo move, plugin root, CLAUDE_PLUGIN_ROOT paths, namespacing) #50

jbr870 referenced this issue from a commit

2026-08-25 10:24:39 +00:00

phase-runner: resolve plugin paths before they go into a subagent brief

jbr870 commented

2026-08-25 10:25:05 +00:00

Author

Owner

/meta-amend outcome — original finding FIXED at HEAD; the residual shape amended

Commit 07d9ad8 on main.

Verified at HEAD (`6174683`) — the finding as written no longer holds

All five cited occurrences are gone. domains/e2e.md:110,240,289 and domains/a11y.md:376,411 now read ${CLAUDE_PLUGIN_ROOT}/skills/_shared/procedures/playwright-conventions.md; a tree-wide grep finds zero ~/.claude paths anywhere under plugin/. Fixed by the plugin move (#50), not by this issue.

The issue's own suggestion — "worth a lint check for ~/.claude inside domain/procedure text" — also already exists, and is stronger than proposed: lint-conventions.sh class 1 greps the whole shipped tree (-I, every file type, not just markdown — a path in a shipped .json is just as unresolvable) for (~|${HOME})/.claude/skills, with a deliberate allow: user-scope-path anchor for the one line that must document adapter discovery order. So the regression guard is in place.

What survives, one substitution later

The ~ is gone; the shape is not. A ${CLAUDE_PLUGIN_ROOT}/… literal copied into a subagent brief is a path only its writer can resolve — the writer located the file the literal came from, and a freshly spawned agent has located nothing. That is the same silent-coverage-loss failure the issue describes, and nothing in the suite said to resolve it at the writing end.

I did not verify whether a spawned agent inherits the substitution — testing it needs a real spawn, which this lane doesn't do — so the rule is written to be correct either way: resolve the path before it goes in the brief; resolving costs nothing if the spawned agent could have resolved it anyway.

The amendment

phase-runner.md §6 — canonical home, chosen because that section already enumerates what is and isn't true inside a spawned agent. New sub-section Resolve plugin paths before they go into a brief: substitute the absolute path, or inline the few rules the brief needs; applies to every path crossing a spawn boundary and to every brief-writing site.
Why it earns a rule rather than a note, stated in the text: the failure is silent. An agent that cannot read the doc it was pointed at doesn't stop — it runs its domain without the checklist and emits a report indistinguishable from a complete one. This issue's own evidence is the argument: 12 failed reads across three agents, each guessing a different wrong home, and the agents that gave up would have been invisible.
qa-playbook.md §2 (the observed site — QA domain briefs) and develop/SKILL.md 3.2 (both teammate prompts) reference it in one clause each rather than restating it.

No new lint check — and why

The issue suggested "text + lint check". The text half is done; the lint half has no target. Briefs are assembled at runtime from domain files and run state — there is no brief in the source tree for a static gate to match, and flagging every ${CLAUDE_PLUGIN_ROOT} in a procedure doc would flag the correct usage as well as the incorrect one, since the same literal is right in a doc and wrong in a brief. The existing class-1 check still covers the ~/.claude regression, which is the part a static gate can see.

Gate

scripts/lint-conventions.sh — clean.

Desk-check against the originating scenario (slot3/#191)

Three QA agents were handed a conventions-doc path they could not resolve, burned 12 reads guessing (/home/jochem/, worktree-local .claude/) before finding $HOME=/home/node. Under the amended text the lead resolves the path once — it already knows where it loaded the domain file from — and each brief carries an absolute path. Zero failed reads, and the sharp edge (an agent that gives up and reports as if complete) has no occasion to appear. The devcontainer detail that made this expensive — the lead's $HOME differing from the agents' — is exactly what resolving at the writing end removes, since the resolved path is a fact rather than a per-process variable.

What validates it next

A QA run in a container whose $HOME differs from the driver's — the configuration that produced the original 12 failed reads. Absent that, the change is unfalsifiable by observation: its success looks like nothing happening. The honest bar is that no future run reports a failed read of a suite doc from inside a spawned agent.

## /meta-amend outcome — original finding FIXED at HEAD; the residual shape amended Commit `07d9ad8` on `main`. ### Verified at HEAD (`6174683`) — the finding as written no longer holds All five cited occurrences are gone. `domains/e2e.md:110,240,289` and `domains/a11y.md:376,411` now read `${CLAUDE_PLUGIN_ROOT}/skills/_shared/procedures/playwright-conventions.md`; a tree-wide grep finds **zero** `~/.claude` paths anywhere under `plugin/`. Fixed by the plugin move (#50), not by this issue. The issue's own suggestion — *"worth a lint check for `~/.claude` inside domain/procedure text"* — also already exists, and is stronger than proposed: `lint-conventions.sh` **class 1** greps the whole shipped tree (`-I`, every file type, not just markdown — a path in a shipped `.json` is just as unresolvable) for `(~|${HOME})/.claude/skills`, with a deliberate `allow: user-scope-path` anchor for the one line that must document adapter discovery order. So the regression guard is in place. ### What survives, one substitution later The `~` is gone; the **shape** is not. A `${CLAUDE_PLUGIN_ROOT}/…` literal copied into a subagent brief is a path only its **writer** can resolve — the writer located the file the literal came from, and a freshly spawned agent has located nothing. That is the same silent-coverage-loss failure the issue describes, and nothing in the suite said to resolve it at the writing end. **I did not verify whether a spawned agent inherits the substitution** — testing it needs a real spawn, which this lane doesn't do — so the rule is written to be correct either way: *resolve the path before it goes in the brief; resolving costs nothing if the spawned agent could have resolved it anyway.* ### The amendment - **`phase-runner.md` §6 — canonical home**, chosen because that section already enumerates what is and isn't true inside a spawned agent. New sub-section *Resolve plugin paths before they go into a brief*: substitute the absolute path, or inline the few rules the brief needs; applies to every path crossing a spawn boundary and to every brief-writing site. - **Why it earns a rule rather than a note**, stated in the text: the failure is silent. An agent that cannot read the doc it was pointed at doesn't stop — it runs its domain without the checklist and emits a report indistinguishable from a complete one. This issue's own evidence is the argument: 12 failed reads across three agents, each guessing a different wrong home, and *the agents that gave up would have been invisible*. - **`qa-playbook.md` §2** (the observed site — QA domain briefs) and **`develop/SKILL.md` 3.2** (both teammate prompts) reference it in one clause each rather than restating it. ### No new lint check — and why The issue suggested "text + lint check". The text half is done; the lint half has no target. Briefs are **assembled at runtime** from domain files and run state — there is no brief in the source tree for a static gate to match, and flagging every `${CLAUDE_PLUGIN_ROOT}` in a procedure doc would flag the correct usage as well as the incorrect one, since the same literal is right in a doc and wrong in a brief. The existing class-1 check still covers the `~/.claude` regression, which is the part a static gate *can* see. ### Gate `scripts/lint-conventions.sh` — clean. ### Desk-check against the originating scenario (slot3/#191) Three QA agents were handed a conventions-doc path they could not resolve, burned 12 reads guessing (`/home/jochem/`, worktree-local `.claude/`) before finding `$HOME=/home/node`. Under the amended text the lead resolves the path once — it already knows where it loaded the domain file from — and each brief carries an absolute path. Zero failed reads, and the sharp edge (an agent that gives up and reports as if complete) has no occasion to appear. The devcontainer detail that made this expensive — the lead's `$HOME` differing from the agents' — is exactly what resolving at the writing end removes, since the resolved path is a fact rather than a per-process variable. ### What validates it next A QA run in a container whose `$HOME` differs from the driver's — the configuration that produced the original 12 failed reads. Absent that, the change is unfalsifiable by observation: its success looks like nothing happening. The honest bar is that no future run reports a failed read of a suite doc from inside a spawned agent.

jbr870 closed this issue

2026-08-25 10:25:05 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

jbr870/devwork-skills#35

No description provided.

Rows
Columns