jbr870/devwork-skills
1
0
Fork 0
Code Issues 64 Pull requests Projects Releases Packages Wiki Activity Actions

QA fix loop: stop patching after a second same-class re-validation finding #40

New issue
Closed
opened 2026-08-14 13:01:46 +00:00 by jbr870 · 1 comment
jbr870 commented 2026-08-14 13:01:46 +00:00
Owner
Copy link

Rank 3 of 13 in #37 · est. saving ~45m

Evidence

QA on #230 spent 2h 21m on a single defect class — the release-notes markdown parser
mishandling continuation lines — across four fix rounds, and still did not converge:

Round Fix What the next re-validation found
1 reflow continuation lines into the preceding block over-reached; nested bullets merged into their parent
2 confine reflow to indented non-construct lines only list markers counted as constructs; 1) slipped
3 invert the test — enumerate what may reflow broke three cases round 2 had right
4 list-context rule (the actual root) two more cases — 5th recurrence → deferred to verity#235

Each round cost a red-test write, a fixer agent, an assertion-constraint check, four gates and a
fresh-context re-validation. The orchestrator itself named the pattern at round 3: it had accepted
a fix that traded correctness elsewhere, and said so.

What happens now

fix-workflow.md has no stopping rule. Each re-validation finding is treated as a new finding and
enters a new round, so a mis-scoped rule can be patched indefinitely.

Proposed change

Add a convergence rule: when a second re-validation finds a defect of the same class as the one
just fixed, the loop stops patching. The next round must either be explicitly aimed at the root
(stated as a class, with the red tests written against the class rather than the reported cases),
or the class is filed as an issue and the run advances with the known residue recorded.

Acceptance

  • fix-workflow.md defines "same class" and the two permitted exits.
  • A third same-class round cannot start without an explicit root-cause framing in the brief.
  • The residue, if deferred, appears as a finding on the QA Phase Outcome.
**Rank 3 of 13** in #37 · est. saving **~45m** ## Evidence QA on #230 spent **2h 21m** on a single defect class — the release-notes markdown parser mishandling continuation lines — across four fix rounds, and still did not converge: | Round | Fix | What the next re-validation found | |---|---|---| | 1 | reflow continuation lines into the preceding block | over-reached; nested bullets merged into their parent | | 2 | confine reflow to indented non-construct lines | only list markers counted as constructs; `1)` slipped | | 3 | invert the test — enumerate what may reflow | **broke three cases round 2 had right** | | 4 | list-context rule (the actual root) | two more cases — 5th recurrence → deferred to verity#235 | Each round cost a red-test write, a fixer agent, an assertion-constraint check, four gates and a fresh-context re-validation. The orchestrator itself named the pattern at round 3: it had accepted a fix that traded correctness elsewhere, and said so. ## What happens now `fix-workflow.md` has no stopping rule. Each re-validation finding is treated as a new finding and enters a new round, so a mis-scoped rule can be patched indefinitely. ## Proposed change Add a convergence rule: **when a second re-validation finds a defect of the same class as the one just fixed, the loop stops patching.** The next round must either be explicitly aimed at the root (stated as a class, with the red tests written against the class rather than the reported cases), or the class is filed as an issue and the run advances with the known residue recorded. ## Acceptance - `fix-workflow.md` defines "same class" and the two permitted exits. - A third same-class round cannot start without an explicit root-cause framing in the brief. - The residue, if deferred, appears as a finding on the QA Phase Outcome.
jbr870 commented 2026-08-25 09:28:43 +00:00
Author
Owner
Copy link

/meta-amend outcome — amended

Commit 6aeae6b on main.

Verified at HEAD (7f75154)

The finding held. fix-workflow.md Step 2 built the fix plan from the current report and prior-round decisions (escalation retry-with-guidance, fix-now promotions) but never compared this round's findings against what the previous round fixed. Step 3 carried "Fix the family, not just the instance" — a within-round rule about sibling occurrences, not a cross-round convergence rule — and qa-playbook.md §3's stopping guidance was entirely §7's exit invariant, which stops the loop when it is done and says nothing about a loop that is stuck.

The amendment

New subsection in fix-workflow.md §2, The convergence rule — a second same-class finding stops the patching:

  • "Same class" is defined by cause, not symptom: one correct change would have prevented both — same code path, same abstraction (a parser rule, a schema-field family, a lifecycle hook, a permission check), or the new case is one the previous fix's condition was meant to cover and didn't. Differing symptoms are written up as the normal appearance of a mis-scoped rule, not as evidence of two classes — that misreading is what let #230's loop run to four rounds.
  • First same-class finding: ordinary, the fix was incomplete, continue. Second: stop patching; a third round of the same shape is not permitted.
  • Exit A — the root-framed round. Available only if the brief can name the class in one sentence and state the rule governing it. The red tests are written against the class including the cases earlier rounds already got right. If the best framing available is still a list of cases, the root has not been found and Exit A is closed — written explicitly, because writing a vague class name to unlock a third patch is the exact failure mode.
  • Exit B — defer the class. Disposition the class (not the cases) defer-to-issue through the normal settling path, and record the residue as a Finding on the round's Phase Outcome. The already-written regression tests stay red-marked against the class.
  • Mechanical foothold: the fix-plan template gains a Same-class: [class, Nth recurrence — which exit] line and the summary gains **Class aimed at:**.
  • qa-playbook.md §3 references it rather than restating it: "§7 stops the loop when it is done; the convergence rule stops it when it is stuck."

Gate

scripts/lint-conventions.sh — clean.

Desk-check against the originating scenario (verity #230, the release-notes continuation-line parser)

Replaying the four recorded rounds against the amended text:

Round Recorded Under the rule
1 — reflow continuation lines re-validation: nested bullets merged into parent first same-class finding → ordinary, continue
2 — confine to indented non-construct lines re-validation: 1) slipped through second same-class finding → stop patching, take an exit
3 — invert the test broke three cases round 2 had right Exit A's brief is "which lines may reflow, in list context" — and its red tests must include round 2's passing cases, so this fix fails its own tests in-round instead of surfacing at round 4. If the class couldn't be named (round 3's actual framing was a mechanism, not a class), Exit B ends the run here with the residue recorded.
4 — list-context rule, 2 cases left, deferred at 5th recurrence 2h 21m total run advances at round 3 either way

All three acceptance criteria are met: the definition and the two exits are in fix-workflow.md; a third same-class round requires the class named in the brief (and reported); the deferred residue lands as a Finding on the Phase Outcome.

What validates it next

The next QA run whose re-validation returns a same-class finding twice. Two things to watch: whether "same class" is legible enough to be applied without hindsight (the risk is under-calling it, since differing symptoms are persuasive in the moment), and whether Exit A's "include the cases earlier rounds got right" actually gets written into the red tests — that clause is the one carrying the round-3 regression guard.

## /meta-amend outcome — amended Commit `6aeae6b` on `main`. ### Verified at HEAD (`7f75154`) The finding held. `fix-workflow.md` Step 2 built the fix plan from the current report and prior-round *decisions* (escalation `retry-with-guidance`, `fix-now` promotions) but never compared this round's findings against **what the previous round fixed**. Step 3 carried "Fix the family, not just the instance" — a within-round rule about sibling occurrences, not a cross-round convergence rule — and `qa-playbook.md` §3's stopping guidance was entirely §7's exit invariant, which stops the loop when it is *done* and says nothing about a loop that is *stuck*. ### The amendment New subsection in `fix-workflow.md` §2, **The convergence rule — a second same-class finding stops the patching**: - **"Same class" is defined by cause, not symptom:** one correct change would have prevented both — same code path, same abstraction (a parser rule, a schema-field family, a lifecycle hook, a permission check), or the new case is one the previous fix's condition was meant to cover and didn't. Differing symptoms are written up as *the normal appearance of a mis-scoped rule*, not as evidence of two classes — that misreading is what let #230's loop run to four rounds. - **First** same-class finding: ordinary, the fix was incomplete, continue. **Second:** stop patching; a third round of the same shape is not permitted. - **Exit A — the root-framed round.** Available only if the brief can name the class in one sentence and state the rule governing it. The red tests are written against the class **including the cases earlier rounds already got right**. If the best framing available is still a list of cases, the root has not been found and Exit A is closed — written explicitly, because writing a vague class name to unlock a third patch is the exact failure mode. - **Exit B — defer the class.** Disposition the *class* (not the cases) `defer-to-issue` through the normal settling path, and record the residue as a Finding on the round's Phase Outcome. The already-written regression tests stay red-marked against the class. - Mechanical foothold: the fix-plan template gains a `Same-class: [class, Nth recurrence — which exit]` line and the summary gains `**Class aimed at:**`. - `qa-playbook.md` §3 references it rather than restating it: *"§7 stops the loop when it is done; the convergence rule stops it when it is stuck."* ### Gate `scripts/lint-conventions.sh` — clean. ### Desk-check against the originating scenario (verity #230, the release-notes continuation-line parser) Replaying the four recorded rounds against the amended text: | Round | Recorded | Under the rule | |---|---|---| | 1 — reflow continuation lines | re-validation: nested bullets merged into parent | **first** same-class finding → ordinary, continue | | 2 — confine to indented non-construct lines | re-validation: `1)` slipped through | **second** same-class finding → **stop patching**, take an exit | | 3 — invert the test | **broke three cases round 2 had right** | Exit A's brief is "which lines may reflow, in list context" — and its red tests must include round 2's passing cases, so this fix fails its own tests in-round instead of surfacing at round 4. If the class couldn't be named (round 3's actual framing was a mechanism, not a class), Exit B ends the run here with the residue recorded. | | 4 — list-context rule, 2 cases left, deferred at 5th recurrence | 2h 21m total | run advances at round 3 either way | All three acceptance criteria are met: the definition and the two exits are in `fix-workflow.md`; a third same-class round requires the class named in the brief (and reported); the deferred residue lands as a Finding on the Phase Outcome. ### What validates it next The next QA run whose re-validation returns a same-class finding twice. Two things to watch: whether "same class" is legible enough to be applied without hindsight (the risk is under-calling it, since differing symptoms are persuasive in the moment), and whether Exit A's "include the cases earlier rounds got right" actually gets written into the red tests — that clause is the one carrying the round-3 regression guard.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
interactive-skills
No results found.
Labels
Clear labels   
component:adapters

   
component:lifecycle

   
component:qa

   
component:setup

   
component:shared

   
component:worktrees

   
phase/accepted

   
phase/backlog

   
phase/deployed

   
phase/developing

   
phase/integrating

   
phase/planning

   
phase/qa

   
phase/requirements

   
phase/uat

   
priority:critical

   
priority:critical

   
priority:high

   
priority:high

   
priority:low

   
priority:low

   
priority:medium

   
priority:medium

   
type:bug

   
type:chore

   
type:docs

   
type:feature

   
type:infra

   
type:tech-debt
No labels
component:adapters
component:lifecycle
component:qa
component:setup
component:shared
component:worktrees
phase/accepted
phase/backlog
phase/deployed
phase/developing
phase/integrating
phase/planning
phase/qa
phase/requirements
phase/uat
priority:critical
priority:critical
priority:high
priority:high
priority:low
priority:low
priority:medium
priority:medium
type:bug
type:chore
type:docs
type:feature
type:infra
type:tech-debt
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jbr870/devwork-skills#40
No description provided.