jbr870/devwork-skills
1
0
Fork 0
Code Issues 64 Pull requests Projects Releases Packages Wiki Activity Actions

provenance: record (and eventually pin) the suite SHA a run executes — skill text moved 20+ times under a live run #34

New issue
Closed
opened 2026-08-13 13:05:28 +00:00 by jbr870 · 3 comments
jbr870 commented 2026-08-13 13:05:28 +00:00
Owner
Copy link

Observation

Nothing records — let alone pins — which version of the suite a run executed. The slot3/#191 deep-dive (report verity-slot3-191-20260812.md, finding 1) established that ~/.claude/skills/_shared on the desktop is a symlink into a live git working tree that received 20+ commits, including a merge, during the run — one mid-QA commit changed the QA playbook's preconditions between stage launches. The model noticed exactly once and re-read one file. Consequences: a run's behavior cannot be attributed to any suite SHA (my own earlier attribution of the 2026-08-12 runs to ae802e0 was wrong); cross-slot comparison on a parallel day is confounded; and a "resolved at HEAD" verification in the process review says nothing about what text a given run actually followed.

Recurrence

Mechanical/systemic — verified by construction: the symlink-farm install (README option A) makes every concurrently-running session read whatever the checkout currently holds. Every multi-session day has this property.

Hypothesis

Two independently useful halves: (1) record — Phase Outcomes gain a suite_sha (or the PO post helper stamps it) so attribution is possible after the fact; (2) pin — a run resolves its skill text once (e.g. a per-run worktree/checkout of the suite at a SHA) so text cannot move mid-run. Recording is cheap and should come first; pinning may fall out of the plugin work (#22, cross-referenced).

Suggested lane

Full pipeline (helper + schema change) for recording; #22 for pinning.

Filed by /meta-retrospective 2026-08-13 (slot3 deep-dive).

Update 2026-08-22 — UAT of #50 confirms the second half: installed-release attribution (UAT-F33)

The plugin migration (#50) reached UAT and its Phase 7 walk produced UAT-F33, which is this
issue seen from the installed side rather than the checkout side:

Run records carry no version attribution at all. PO-239-1 on verity#239 — the one real
consumer-machine record — is headed <!-- phase-outcome:v1 id=PO-239-1 skill=requirements -->
and its JSON body has no version field. Nothing in phase-outcome.md's schema or in
phase-outcome-post.sh stamps the installed release. #50's AC-12 asks that "desktop run records
are attributable to the installed release version"; today that depends on the operator
remembering the README consumer-runbook step and pasting the version by hand — and in the one
real run we have, it was not remembered. An instruction the tooling does not enforce is not an
attribution mechanism.

So the "record" half of this issue has two identity sources, and which one applies is decided
by the machine, not by the caller.

Ground truth, measured 2026-08-22 (not inferred)

Probed on both machines rather than assumed — this is the input the technical plan needs:

dev machine (live checkout) desktop jochems-forge (installed plugin)
$CLAUDE_PLUGIN_ROOT inside the repo work tree ~/.claude/plugins/cache/devwork-skills/dev/0.1.5
is a git work tree? yes no — fatal: not a git repository; it is an extracted copy with only .claude-plugin/, skills/, .in_use
.claude-plugin/plugin.json .version 0.1.5 — but the tree may be arbitrarily far ahead of that tag 0.1.5, and the version is also literally in the install path
commit identity git rev-parse HEAD (+ dirty flag) none locally

The two cases are cleanly separable by one probe — is $CLAUDE_PLUGIN_ROOT inside a git work
tree — so a single helper can stamp SHA-on-checkout / version-on-install without a mode flag and
without the operator declaring anything. Version alone would not have been safe (a dev checkout
ahead of the tag still reads 0.1.5); the git probe is what makes it honest.

Also observed, and worth exploiting rather than duplicating: Claude Code's own
~/.claude/plugins/installed_plugins.json already records both for each install —
"version": "0.1.5" and "gitCommitSha": "bc6214d…". Whether to read that file (its path and
multi-scope array shape are undocumented and not ours) or stay with the $CLAUDE_PLUGIN_ROOT
probe is a plan-time call; the probe is the portable one, that file is the corroborating one.

Separately: the desktop's marketplace snapshot at ~/.claude/plugins/marketplaces/devwork-skills
is a git clone (currently main @ bc6214d, no longer shallow after Scenario 21's
fetch --unshallow, carrying only tags dev--v0.1.0/dev--v0.1.1). It is not the plugin root and
must not be mistaken for one — the run executes out of cache/, not out of the snapshot.

Why this is not #50's to fix — and what #50's own decisions actually said

Checked against the resolutions rather than assumed. #50 split this subject in two at plan time and
handled the halves differently:

  • Dev-machine half — accepted as a non-goal. D-PO-50-2-9 and D-PO-50-2-14 both resolved
    accept, on the reasoning "git is the dev machine's version identity; release attribution is
    desktop-only (AC-12)". Worth noting for this issue's own framing: UAT partially undercuts that
    reasoning. Git is only an identity if something records the SHA — and nothing does, which is
    precisely the original observation at the top of this issue. The disposition was right that the
    dev machine needs no release version; it was wrong that no recording was needed at all.
  • Desktop half — in scope via AC-12, but only ever had a manual mechanism. #50's Out of
    Scope     bars "any skill logic change: this feature is layout + references + packaging + tooling
    only", so the attribution AC-12 asked for could only be delivered as a README consumer-runbook
    step telling the operator to record the version by hand. UAT-F33 is the finding that this is not
    an attribution mechanism: in the one real consumer-machine run, it was not done.

So the tooling-enforced capability was never inside #50's scope on either half, and lands here.
AC-12 has been amended on #50 to claim only the manual mechanism it actually scoped, with the
enforced version pointed at this issue.

Lane

Unchanged: full pipeline (/dev:requirements → …). It is a helper behaviour change plus a
change to a versioned contract schema (phase-outcome:v1 gains a field — whether that is a
backward-compatible optional field or a :v2 is itself a plan-time question, because
phase-outcome-post.sh's retry-guard normalisation and phase-outcome-read-all.sh's parser both
read these records, and every already-posted PO lacks the field).

Adjacent defect found while confirming this (2026-08-22) — filed separately

scan_comments --header 'phase-outcome:v1' matches the token as a substring of line 1 with no
shape validation, so any comment whose header merely contains it is absorbed into the Phase Outcome
ledger. A UAT closure note posted on #50 with the header <!-- phase-outcome:v1 scenario=13 -->
(no id=, no skill=, no JSON fence) made phase-outcome-read-all.sh --issue 50 die with a bare
jq error — null (null) only strings can be parsed — naming no comment and no cause, and would
have made the next phase-outcome-post.sh mint PO-50-7, skipping PO-50-6 and breaking ordinal
density permanently. Repaired by hand (header stripped from comment 1067). Cross-referenced here
because it is the same subject: the integrity of the run record.

## Observation Nothing records — let alone pins — which version of the suite a run executed. The slot3/#191 deep-dive (report `verity-slot3-191-20260812.md`, finding 1) established that `~/.claude/skills/_shared` on the desktop is a symlink into a **live git working tree that received 20+ commits, including a merge, during the run** — one mid-QA commit changed the QA playbook's preconditions between stage launches. The model noticed exactly once and re-read one file. Consequences: a run's behavior cannot be attributed to any suite SHA (my own earlier attribution of the 2026-08-12 runs to `ae802e0` was wrong); cross-slot comparison on a parallel day is confounded; and a "resolved at HEAD" verification in the process review says nothing about what text a given run actually followed. ## Recurrence Mechanical/systemic — verified by construction: the symlink-farm install (README option A) makes every concurrently-running session read whatever the checkout currently holds. Every multi-session day has this property. ## Hypothesis Two independently useful halves: (1) **record** — Phase Outcomes gain a `suite_sha` (or the PO post helper stamps it) so attribution is possible after the fact; (2) **pin** — a run resolves its skill text once (e.g. a per-run worktree/checkout of the suite at a SHA) so text cannot move mid-run. Recording is cheap and should come first; pinning may fall out of the plugin work (#22, cross-referenced). ## Suggested lane Full pipeline (helper + schema change) for recording; #22 for pinning. *Filed by /meta-retrospective 2026-08-13 (slot3 deep-dive).* --- ## Update 2026-08-22 — UAT of #50 confirms the second half: installed-release attribution (UAT-F33) The plugin migration (#50) reached UAT and its Phase 7 walk produced **UAT-F33**, which is this issue seen from the *installed* side rather than the checkout side: > Run records carry no version attribution at all. `PO-239-1` on verity#239 — the one real > consumer-machine record — is headed `<!-- phase-outcome:v1 id=PO-239-1 skill=requirements -->` > and its JSON body has no version field. Nothing in `phase-outcome.md`'s schema or in > `phase-outcome-post.sh` stamps the installed release. #50's AC-12 asks that "desktop run records > are attributable to the installed release version"; today that depends on the operator > remembering the README consumer-runbook step and pasting the version by hand — and in the one > real run we have, it was not remembered. **An instruction the tooling does not enforce is not an > attribution mechanism.** So the "record" half of this issue has **two** identity sources, and which one applies is decided by the machine, not by the caller. ### Ground truth, measured 2026-08-22 (not inferred) Probed on both machines rather than assumed — this is the input the technical plan needs: | | dev machine (live checkout) | desktop `jochems-forge` (installed plugin) | |---|---|---| | `$CLAUDE_PLUGIN_ROOT` | inside the repo work tree | `~/.claude/plugins/cache/devwork-skills/dev/0.1.5` | | is a git work tree? | **yes** | **no** — `fatal: not a git repository`; it is an extracted copy with only `.claude-plugin/`, `skills/`, `.in_use` | | `.claude-plugin/plugin.json` `.version` | `0.1.5` — but the tree may be arbitrarily far ahead of that tag | `0.1.5`, and the **version is also literally in the install path** | | commit identity | `git rev-parse HEAD` (+ dirty flag) | none locally | **The two cases are cleanly separable by one probe** — is `$CLAUDE_PLUGIN_ROOT` inside a git work tree — so a single helper can stamp SHA-on-checkout / version-on-install without a mode flag and without the operator declaring anything. Version alone would *not* have been safe (a dev checkout ahead of the tag still reads `0.1.5`); the git probe is what makes it honest. Also observed, and worth exploiting rather than duplicating: Claude Code's own `~/.claude/plugins/installed_plugins.json` already records **both** for each install — `"version": "0.1.5"` and `"gitCommitSha": "bc6214d…"`. Whether to read that file (its path and multi-scope array shape are undocumented and not ours) or stay with the `$CLAUDE_PLUGIN_ROOT` probe is a plan-time call; the probe is the portable one, that file is the corroborating one. Separately: the desktop's marketplace snapshot at `~/.claude/plugins/marketplaces/devwork-skills` **is** a git clone (currently `main` @ `bc6214d`, no longer shallow after Scenario 21's `fetch --unshallow`, carrying only tags `dev--v0.1.0`/`dev--v0.1.1`). It is not the plugin root and must not be mistaken for one — the run executes out of `cache/`, not out of the snapshot. ### Why this is not #50's to fix — and what #50's own decisions actually said Checked against the resolutions rather than assumed. #50 split this subject in two at plan time and handled the halves differently: - **Dev-machine half — accepted as a non-goal.** `D-PO-50-2-9` and `D-PO-50-2-14` both resolved `accept`, on the reasoning "git is the dev machine's version identity; release attribution is desktop-only (AC-12)". Worth noting for this issue's own framing: UAT partially undercuts that reasoning. Git is only an *identity* if something records the SHA — and nothing does, which is precisely the original observation at the top of this issue. The disposition was right that the dev machine needs no *release version*; it was wrong that no recording was needed at all. - **Desktop half — in scope via AC-12, but only ever had a manual mechanism.** #50's **Out of Scope** bars "any skill *logic* change: this feature is layout + references + packaging + tooling only", so the attribution AC-12 asked for could only be delivered as a README consumer-runbook step telling the operator to record the version by hand. UAT-F33 is the finding that this is not an attribution mechanism: in the one real consumer-machine run, it was not done. So the *tooling-enforced* capability was never inside #50's scope on either half, and lands here. AC-12 has been amended on #50 to claim only the manual mechanism it actually scoped, with the enforced version pointed at this issue. ### Lane Unchanged: **full pipeline** (`/dev:requirements` → …). It is a helper *behaviour* change plus a change to a versioned contract schema (`phase-outcome:v1` gains a field — whether that is a backward-compatible optional field or a `:v2` is itself a plan-time question, because `phase-outcome-post.sh`'s retry-guard normalisation and `phase-outcome-read-all.sh`'s parser both read these records, and every already-posted PO lacks the field). ### Adjacent defect found while confirming this (2026-08-22) — filed separately `scan_comments --header 'phase-outcome:v1'` matches the token as a **substring of line 1** with no shape validation, so any comment whose header merely contains it is absorbed into the Phase Outcome ledger. A UAT closure note posted on #50 with the header `<!-- phase-outcome:v1 scenario=13 -->` (no `id=`, no `skill=`, no JSON fence) made `phase-outcome-read-all.sh --issue 50` die with a bare jq error — `null (null) only strings can be parsed` — naming no comment and no cause, and would have made the next `phase-outcome-post.sh` mint `PO-50-7`, skipping `PO-50-6` and breaking ordinal density permanently. Repaired by hand (header stripped from comment 1067). Cross-referenced here because it is the same subject: the integrity of the run record.
jbr870 commented 2026-08-14 14:31:57 +00:00
Author
Owner
Copy link

Live evidence from a full-pipeline run, recovered after the fact.

Feature verity #230 (in-app release notes) ran the whole pipeline — /requirements through
/promote to a verified production deploy — on ae802e0 (2026-08-12), 15 commits behind
origin/main. The run itself recorded nothing about which suite it executed. The SHA was only
recovered on 2026-08-14, two days later, by SSHing to the deployed checkout at
jochems-forge:~/.claude/repos/devwork-skills and reading its HEAD.

Why this is the sharp version of the problem rather than a mild one:

  • The gap was not visible from inside the run. Nothing in any Phase Outcome, deliverable or
    journal names the suite version, so a reader of the forge trail cannot tell which skill text
    produced it.
  • The run produced 13 process findings (now #37 and its children). Every one had to be
    re-verified against origin/main before filing, because a finding against two-day-old text may
    already be fixed. All 13 survived — but that verification pass was only possible because the
    deployed checkout happened to be clean and reachable. Had it been dirty, or had it been pulled
    in the meantime, the evidence would have been unreconstructable.
  • One finding overlapped a commit the run did not have (e3a5369, #33 — rc-checked stdout for
    helper invocations). It was reframed as a generalisation to gate commands rather than filed as a
    rediscovery. Without knowing the SHA, that call could not have been made.

Concrete shape this argues for: the first Phase Outcome of a run records the suite SHA and
whether the checkout was dirty, and every subsequent one carries it forward — cheap to write,
and it makes "is this finding still real?" answerable without archaeology.

Recorded in sdlc-dogfood-run-reports/verity-slot4-230.md §1.

Live evidence from a full-pipeline run, recovered after the fact. Feature **verity #230** (in-app release notes) ran the whole pipeline — `/requirements` through `/promote` to a verified production deploy — on **`ae802e0` (2026-08-12), 15 commits behind `origin/main`**. The run itself recorded nothing about which suite it executed. The SHA was only recovered on 2026-08-14, two days later, by SSHing to the deployed checkout at `jochems-forge:~/.claude/repos/devwork-skills` and reading its HEAD. Why this is the sharp version of the problem rather than a mild one: - The gap was **not visible from inside the run**. Nothing in any Phase Outcome, deliverable or journal names the suite version, so a reader of the forge trail cannot tell which skill text produced it. - The run produced **13 process findings** (now #37 and its children). Every one had to be re-verified against `origin/main` before filing, because a finding against two-day-old text may already be fixed. All 13 survived — but that verification pass was only possible because the deployed checkout happened to be clean and reachable. Had it been dirty, or had it been pulled in the meantime, the evidence would have been unreconstructable. - One finding overlapped a commit the run did not have (`e3a5369`, #33 — rc-checked stdout for helper invocations). It was reframed as a generalisation to gate commands rather than filed as a rediscovery. Without knowing the SHA, that call could not have been made. Concrete shape this argues for: the **first** Phase Outcome of a run records the suite SHA and whether the checkout was dirty, and every subsequent one carries it forward — cheap to write, and it makes "is this finding still real?" answerable without archaeology. Recorded in `sdlc-dogfood-run-reports/verity-slot4-230.md` §1.
jbr870 commented 2026-08-22 12:54:45 +00:00
Author
Owner
Copy link

Linked: this issue is relates_to #54 (recorded by the devwork pipeline).

Linked: this issue is **relates_to** #54 (recorded by the devwork pipeline).
jbr870 commented 2026-08-24 09:55:01 +00:00
Author
Owner
Copy link
{"status":"active","into_issue":"57","source_kind":"sibling-stub","absorbed_finding_refs":[],"rationale":"the RECORD half (provenance stamp via the worktree probe) lands in #57; the PIN half was split to #56 before promotion, so closing this issue strands no scope"}
<!-- promoted-into:v1 issue=34 into_issue=57 --> ```json {"status":"active","into_issue":"57","source_kind":"sibling-stub","absorbed_finding_refs":[],"rationale":"the RECORD half (provenance stamp via the worktree probe) lands in #57; the PIN half was split to #56 before promotion, so closing this issue strands no scope"} ```
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
interactive-skills
No results found.
Labels
Clear labels   
component:adapters

   
component:lifecycle

   
component:qa

   
component:setup

   
component:shared

   
component:worktrees

   
phase/accepted

   
phase/backlog

   
phase/deployed

   
phase/developing

   
phase/integrating

   
phase/planning

   
phase/qa

   
phase/requirements

   
phase/uat

   
priority:critical

   
priority:critical

   
priority:high

   
priority:high

   
priority:low

   
priority:low

   
priority:medium

   
priority:medium

   
type:bug

   
type:chore

   
type:docs

   
type:feature

   
type:infra

   
type:tech-debt
No labels
component:adapters
component:lifecycle
component:qa
component:setup
component:shared
component:worktrees
phase/accepted
phase/backlog
phase/deployed
phase/developing
phase/integrating
phase/planning
phase/qa
phase/requirements
phase/uat
priority:critical
priority:critical
priority:high
priority:high
priority:low
priority:low
priority:medium
priority:medium
type:bug
type:chore
type:docs
type:feature
type:infra
type:tech-debt
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jbr870/devwork-skills#34
No description provided.