jbr870/devwork-skills
1
0
Fork 0
Code Issues 64 Pull requests Projects Releases Packages Wiki Activity Actions

/develop + QA: run the full browser suite at phase boundaries, not after every unit and every fix round #38

New issue
Closed
opened 2026-08-14 13:01:44 +00:00 by jbr870 · 1 comment
jbr870 commented 2026-08-14 13:01:44 +00:00
Owner
Copy link

Rank 1 of 13 in #37 · est. saving ~2h per feature of this size

Evidence

The full Playwright suite (~21 min, 469 tests) ran seven times across feature #230:

When Result
/develop WU-6 469 · 21.1m
QA Stage A baseline 469 · 20.8m
QA after fix round 1 469 · 20.9m
QA after fix round 3 469 · 21.0m
QA after fix round 4 468 · flake, passed on re-run
QA final sweep 469 · 20.2m
/integrate gate 469 · 23.1m

Six of seven returned a byte-identical 469 passed, 3 skipped, 0 failed. The seventh was an
unrelated load-sensitive spec (verity#222) that passed in isolation. Total: 2h 29m — 19% of the
feature's active time — to establish seven times that no sibling spec had broken.

What happens now

/develop runs the full browser suite as part of its per-unit and cross-unit verification, and the
QA playbook re-runs it after every fix round. Neither rule distinguishes "the feature's own specs"
from "the whole suite".

Proposed change

Split the browser gate in two, in develop/SKILL.md and the QA playbook:

  • Per unit / per fix round: run only the feature's own spec files.
  • Per phase boundary: run the full suite once — end of /develop, end of QA, and /integrate.

Three full runs instead of seven, with the regression signal preserved at every point where work
leaves a phase.

Acceptance

  • Skill text names the two tiers explicitly and says which runs at which boundary.
  • A run of this shape executes the full browser suite no more than three times.
  • The narrow run still fails the unit if the feature's own specs regress.
**Rank 1 of 13** in #37 · est. saving **~2h per feature of this size** ## Evidence The full Playwright suite (~21 min, 469 tests) ran **seven times** across feature #230: | When | Result | |---|---| | `/develop` WU-6 | 469 · 21.1m | | QA Stage A baseline | 469 · 20.8m | | QA after fix round 1 | 469 · 20.9m | | QA after fix round 3 | 469 · 21.0m | | QA after fix round 4 | **468 · flake**, passed on re-run | | QA final sweep | 469 · 20.2m | | `/integrate` gate | 469 · 23.1m | Six of seven returned a byte-identical `469 passed, 3 skipped, 0 failed`. The seventh was an unrelated load-sensitive spec (verity#222) that passed in isolation. Total: **2h 29m — 19% of the feature's active time** — to establish seven times that no sibling spec had broken. ## What happens now `/develop` runs the full browser suite as part of its per-unit and cross-unit verification, and the QA playbook re-runs it after every fix round. Neither rule distinguishes "the feature's own specs" from "the whole suite". ## Proposed change Split the browser gate in two, in `develop/SKILL.md` and the QA playbook: - **Per unit / per fix round:** run only the feature's own spec files. - **Per phase boundary:** run the full suite once — end of `/develop`, end of QA, and `/integrate`. Three full runs instead of seven, with the regression signal preserved at every point where work leaves a phase. ## Acceptance - Skill text names the two tiers explicitly and says which runs at which boundary. - A run of this shape executes the full browser suite no more than three times. - The narrow run still fails the unit if the feature's own specs regress.
jbr870 commented 2026-08-25 09:26:05 +00:00
Author
Owner
Copy link

/meta-amend outcome — amended

Addressed together with #41 as one two-tier gate policy (as #41 asked), commit 7f75154 on main.

Verified at HEAD

Both halves still held at 7453ac5:

  • develop/SKILL.md Implementer prompt: "4. Run the full test suite to check for regressions" + checklist "Full test suite passes (no regressions)" — no distinction between the feature's specs and the whole suite.
  • fix-workflow.md §4 "Run Full Test Suite … run the complete test suite" and test-workflow.md §5 "Run the project's full test suite" — fired after every fix round.
  • domains/code.md was worse than the issue recorded: its fixer ran the full suite after every individual issue (step 5), plus again after all issues.
  • qa-playbook.md §2 already carried the principle for domains ("Re-validate the stage, not the world") but said nothing about the deterministic suites, so the workflows above overrode it in practice.

The amendment

Two tiers, named explicitly, with the boundary each fires at:

  • Narrow — per work unit / per fix round. Project verification commands scoped to the declared file boundaries: the package(s)/area(s) touched, plus this feature's own end-to-end/browser specs. A unit spanning two packages runs both. Explicitly still a gate — the feature's own specs going red fails the unit.
  • Wide — per phase boundary. Whole workspace suite + whole browser suite, once each, at /dev:develop 3.7 cross-unit, the QA sweep, and /dev:integrate Step 3.

Plus one rule that makes three runs actually reachable: QA does not re-prove the previous phase's exit — it inherits a HEAD /dev:develop's cross-unit gate already ran wide against, checked the way §7 checks the sweep (against the recorded HEAD; if HEAD moved, the wide run is owed again).

How a run is scoped stays a project declaration (workspace filter, path/pattern argument, tag) — no scoping mechanism is hardcoded, and "run whole" is written as the explicit fallback, never the default.

Files: develop/SKILL.md (tier definition at the top of Phase 3; Implementer process step 4; Implementer verification checklist; 3.7 named as the wide tier), _shared/procedures/qa-playbook.md §2, _shared/procedures/fix-workflow.md §4 + report template, _shared/procedures/test-workflow.md §5, _shared/domains/code.md fixer.

Gate

scripts/lint-conventions.sh — clean (117 helper scripts, 690 fence lines, 202 shipped-tree files).

Desk-check against the originating scenario (verity #230)

Replaying the recorded run against the amended text:

Full browser suite Recorded Amended
/develop WU-6 per-unit 1 0 (narrow: feature specs)
/develop cross-unit (3.7) — 1
QA Stage A baseline 1 0 (inherited HEAD, wide run not owed)
QA after fix rounds 1, 3, 4 3 0 (narrow)
QA sweep 1 1
/integrate gate 1 1
Total 7 (2h 29m) 3

And the vitest half: the full workspace suite ran 7× across six work units; under the narrow per-unit gate it runs once, at 3.7. Both issues' numeric acceptance criteria are met, and the two qualitative ones ("a unit that touches two packages runs both"; "the narrow run still fails the unit if the feature's own specs regress") are stated in the text rather than implied.

What validates it next

A desk-check, not a run. The next dogfooded feature that decomposes into more than two work units is what proves the per-unit narrow gate actually catches a regression it is supposed to catch — watch for a WU that closes green under the narrow gate and then fails at 3.7.

## /meta-amend outcome — amended Addressed together with **#41** as one two-tier gate policy (as #41 asked), commit `7f75154` on `main`. ### Verified at HEAD Both halves still held at `7453ac5`: - `develop/SKILL.md` Implementer prompt: *"4. Run the full test suite to check for regressions"* + checklist *"Full test suite passes (no regressions)"* — no distinction between the feature's specs and the whole suite. - `fix-workflow.md` §4 *"Run Full Test Suite … run the complete test suite"* and `test-workflow.md` §5 *"Run the project's full test suite"* — fired after every fix round. - `domains/code.md` was worse than the issue recorded: its fixer ran the full suite **after every individual issue** (step 5), plus again after all issues. - `qa-playbook.md` §2 already carried the principle for *domains* (*"Re-validate the stage, not the world"*) but said nothing about the deterministic suites, so the workflows above overrode it in practice. ### The amendment Two tiers, named explicitly, with the boundary each fires at: - **Narrow — per work unit / per fix round.** Project verification commands **scoped to the declared file boundaries**: the package(s)/area(s) touched, plus this feature's own end-to-end/browser specs. A unit spanning two packages runs both. Explicitly still a gate — the feature's own specs going red fails the unit. - **Wide — per phase boundary.** Whole workspace suite + whole browser suite, once each, at `/dev:develop` 3.7 cross-unit, the QA sweep, and `/dev:integrate` Step 3. Plus one rule that makes three runs actually reachable: **QA does not re-prove the previous phase's exit** — it inherits a HEAD `/dev:develop`'s cross-unit gate already ran wide against, checked the way §7 checks the sweep (against the recorded HEAD; if HEAD moved, the wide run is owed again). *How* a run is scoped stays a project declaration (workspace filter, path/pattern argument, tag) — no scoping mechanism is hardcoded, and "run whole" is written as the explicit fallback, never the default. **Files:** `develop/SKILL.md` (tier definition at the top of Phase 3; Implementer process step 4; Implementer verification checklist; 3.7 named as the wide tier), `_shared/procedures/qa-playbook.md` §2, `_shared/procedures/fix-workflow.md` §4 + report template, `_shared/procedures/test-workflow.md` §5, `_shared/domains/code.md` fixer. ### Gate `scripts/lint-conventions.sh` — clean (117 helper scripts, 690 fence lines, 202 shipped-tree files). ### Desk-check against the originating scenario (verity #230) Replaying the recorded run against the amended text: | Full browser suite | Recorded | Amended | |---|---|---| | `/develop` WU-6 per-unit | 1 | 0 (narrow: feature specs) | | `/develop` cross-unit (3.7) | — | 1 | | QA Stage A baseline | 1 | 0 (inherited HEAD, wide run not owed) | | QA after fix rounds 1, 3, 4 | 3 | 0 (narrow) | | QA sweep | 1 | 1 | | `/integrate` gate | 1 | 1 | | **Total** | **7 (2h 29m)** | **3** | And the vitest half: the full workspace suite ran 7× across six work units; under the narrow per-unit gate it runs **once**, at 3.7. Both issues' numeric acceptance criteria are met, and the two qualitative ones ("a unit that touches two packages runs both"; "the narrow run still fails the unit if the feature's own specs regress") are stated in the text rather than implied. ### What validates it next A desk-check, not a run. The next dogfooded feature that decomposes into more than two work units is what proves the per-unit narrow gate actually catches a regression it is supposed to catch — watch for a WU that closes green under the narrow gate and then fails at 3.7.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
interactive-skills
No results found.
Labels
Clear labels   
component:adapters

   
component:lifecycle

   
component:qa

   
component:setup

   
component:shared

   
component:worktrees

   
phase/accepted

   
phase/backlog

   
phase/deployed

   
phase/developing

   
phase/integrating

   
phase/planning

   
phase/qa

   
phase/requirements

   
phase/uat

   
priority:critical

   
priority:critical

   
priority:high

   
priority:high

   
priority:low

   
priority:low

   
priority:medium

   
priority:medium

   
type:bug

   
type:chore

   
type:docs

   
type:feature

   
type:infra

   
type:tech-debt
No labels
component:adapters
component:lifecycle
component:qa
component:setup
component:shared
component:worktrees
phase/accepted
phase/backlog
phase/deployed
phase/developing
phase/integrating
phase/planning
phase/qa
phase/requirements
phase/uat
priority:critical
priority:critical
priority:high
priority:high
priority:low
priority:low
priority:medium
priority:medium
type:bug
type:chore
type:docs
type:feature
type:infra
type:tech-debt
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jbr870/devwork-skills#38
No description provided.