Enrich auto-rules.toml with provenance metadata #3

New issue

Open

opened 2026-04-09 13:37:57 +00:00 by jbr870 · 0 comments

jbr870 commented

2026-04-09 13:37:57 +00:00

Owner

Summary

When rules are auto-promoted to auto-rules.toml, the LLM reasoning, session context, and decision tier are discarded. Only a timestamp comment survives. Users reviewing their auto-rules can't answer "why was this rule created?" without digging through the audit log.

User Stories Served

US-11: Auto-rule provenance — see which auto-rules exist and how they got there, with the original LLM reasoning

Current State

# Auto-promoted 2026-02-26T12:34:56Z: Bash: npm test
[[allow]]
tool = 'Bash'
[allow.fields]
command = '^npm\s+test'

Proposed Enhancement

Add metadata fields that the rule engine ignores but humans and tooling can read:

# Auto-promoted 2026-02-26T12:34:56Z: Bash: npm test
[[allow]]
tool = 'Bash'
promoted_at = '2026-02-26T12:34:56Z'
promoted_tier = 'GREEN'
promoted_reasoning = 'npm test is a read-only test runner, safe to auto-approve'
promoted_session = 'abc-123'
[allow.fields]
command = '^npm\s+test'

The rule engine already ignores unknown fields (serde deny_unknown_fields is not set), so these are backwards-compatible.

Context

See wiki: Data Capture Analysis — "Structural Issues §2: Auto-rules.toml loses provenance"

## Summary When rules are auto-promoted to auto-rules.toml, the LLM reasoning, session context, and decision tier are discarded. Only a timestamp comment survives. Users reviewing their auto-rules can't answer "why was this rule created?" without digging through the audit log. ## User Stories Served - **US-11:** Auto-rule provenance — see which auto-rules exist and how they got there, with the original LLM reasoning ## Current State ```toml # Auto-promoted 2026-02-26T12:34:56Z: Bash: npm test [[allow]] tool = 'Bash' [allow.fields] command = '^npm\s+test' ``` ## Proposed Enhancement Add metadata fields that the rule engine ignores but humans and tooling can read: ```toml # Auto-promoted 2026-02-26T12:34:56Z: Bash: npm test [[allow]] tool = 'Bash' promoted_at = '2026-02-26T12:34:56Z' promoted_tier = 'GREEN' promoted_reasoning = 'npm test is a read-only test runner, safe to auto-approve' promoted_session = 'abc-123' [allow.fields] command = '^npm\s+test' ``` The rule engine already ignores unknown fields (serde `deny_unknown_fields` is not set), so these are backwards-compatible. ## Context See wiki: [Data Capture Analysis](https://git.wihslon.com/jbr870/claude-permit/wiki/product%2Fdata-capture-analysis) — "Structural Issues §2: Auto-rules.toml loses provenance"

jbr870 added the

observability

enhancement

labels

2026-04-09 13:37:57 +00:00

jbr870 referenced this issue

2026-04-27 12:59:39 +00:00

Friction insights in HTML report: surface repeat-prompt patterns + suggest fixes #20