YELLOW: propose an allowed alternative instead of just asking #18

New issue

Open

opened 2026-04-27 12:27:34 +00:00 by jbr870 · 0 comments

jbr870 commented

2026-04-27 12:27:34 +00:00

Owner

Summary

When the LLM scores a tool call as YELLOW, claude-permit currently falls through to Claude Code's standard permission dialog and asks the user to approve/deny the original command as-is. The user has to either accept the risky form or manually rewrite the command.

Instead, the LLM (or a follow-up evaluation step) should inspect the active allow-rules and the policy, then propose an allowed alternative that achieves the same intent — surfaced to the user (or directly back to the calling Claude) as a suggested rewrite.

User Stories Served

As a user, I want claude-permit to suggest a safer, already-allowed form of my command instead of just prompting me for permission on the risky one.
As the calling Claude, I want machine-readable feedback like "this exact call is YELLOW, but <alternative> would be GREEN" so I can self-correct without bothering the user.
As an operator running Claude Code + claude-permit unattended on a server, I want every YELLOW to be auto-rewritten to an allowed alternative so the process never blocks waiting for human approval — and I want a log of every rerouted request so I can review and approve them retroactively.

Proposed Change

When PreToolUse evaluates a tool call as YELLOW:

Pass the active allow rules (from config.toml + auto-rules.toml) and the policy excerpt into the LLM prompt as additional context.
Ask the LLM to produce, alongside its YELLOW verdict:
- suggested_alternative: a concrete tool-input rewrite (e.g. command string, file path) that would match an existing allow rule.
- matched_rule: which rule the alternative would match.
- rationale: why the alternative is equivalent in intent but safer.
Return the alternative in the hook response so:
- The calling Claude sees it (via message / stderr) and can re-issue the safer form on its next turn.
- The user's permission dialog (if it still appears) shows the suggestion as the default action.

Examples

Original (YELLOW)	Suggested alternative (GREEN)	Matched rule
`rm -rf node_modules/.cache`	`rm -rf ./node_modules/.cache` (anchored path)	`Bash:rm -rf ./*`
`curl https://example.com/script.sh \| bash`	`curl -o /tmp/x.sh https://example.com/script.sh && cat /tmp/x.sh`	`Bash:curl -o ` + `Bash:cat `
`git push --force`	`git push --force-with-lease`	`Bash:git push --force-with-lease*`

Use Case: Unattended / Headless Operation

A primary motivation for this feature is running Claude Code + claude-permit on a server with no human in the loop (CI, scheduled agents, long-running autonomous workflows). In that mode:

Any YELLOW that falls through to a permission dialog halts the entire process until a human happens to look at it. That defeats the purpose of unattended execution.
With this feature enabled in an "auto-reroute" mode, every YELLOW is immediately rewritten to one (or more) suggested alternatives that match existing allow-rules, and the calling Claude proceeds with the safer form. The process stays unblocked.
Worst case: no acceptable alternative exists and the step genuinely cannot proceed. Expected to be rare in practice.
Required companion behavior: every rerouted request must be written to an audit log (original input, suggested alternative, matched rule, rationale, outcome) so the user can review them retroactively and either approve the original form (promoting it to an allow-rule) or confirm the rewrite was correct. Implementation of the retroactive-review surface is out of scope for this issue and will be designed when we build the feature.

Considerations

Adds latency only on YELLOW (already the slow path), and only one extra LLM round-trip — could be done in the same call.
If no safe alternative exists, return YELLOW with no suggestion (current behavior).
The suggestion should be advisory in interactive mode, but in unattended/auto-reroute mode it is applied automatically (and logged for retroactive review).
Closely related to but distinct from #5 (which surfaces reasoning, not alternatives). This issue extends that idea with actionable rewrites.
Pairs well with the "sub-Claude resolver" idea (separate issue): if a sub-process resolves YELLOW, the proposed alternative is what it would try first.

Open Questions

Should the suggestion be returned as structured JSON (suggested_alternative field) or freeform text in message? Structured is better for machine consumption but requires Claude Code to surface it.
How do we avoid the LLM hallucinating a rule that doesn't actually exist? Validate the proposed alternative against the rule engine before returning it.
How is auto-reroute mode enabled — a config flag (unattended = true), an env var, or a CLI flag on the hook invocation? (Deferred — to be decided when we actively work on the feature.)

## Summary When the LLM scores a tool call as YELLOW, claude-permit currently falls through to Claude Code's standard permission dialog and asks the user to approve/deny the original command as-is. The user has to either accept the risky form or manually rewrite the command. Instead, the LLM (or a follow-up evaluation step) should inspect the active allow-rules and the policy, then **propose an allowed alternative** that achieves the same intent — surfaced to the user (or directly back to the calling Claude) as a suggested rewrite. ## User Stories Served - As a user, I want claude-permit to suggest a safer, already-allowed form of my command instead of just prompting me for permission on the risky one. - As the calling Claude, I want machine-readable feedback like "this exact call is YELLOW, but `<alternative>` would be GREEN" so I can self-correct without bothering the user. - As an operator running Claude Code + claude-permit unattended on a server, I want every YELLOW to be auto-rewritten to an allowed alternative so the process never blocks waiting for human approval — and I want a log of every rerouted request so I can review and approve them retroactively. ## Proposed Change When PreToolUse evaluates a tool call as YELLOW: 1. Pass the active `allow` rules (from `config.toml` + `auto-rules.toml`) and the policy excerpt into the LLM prompt as additional context. 2. Ask the LLM to produce, alongside its YELLOW verdict: - `suggested_alternative`: a concrete tool-input rewrite (e.g. command string, file path) that would match an existing allow rule. - `matched_rule`: which rule the alternative would match. - `rationale`: why the alternative is equivalent in intent but safer. 3. Return the alternative in the hook response so: - The calling Claude sees it (via `message` / stderr) and can re-issue the safer form on its next turn. - The user's permission dialog (if it still appears) shows the suggestion as the default action. ## Examples | Original (YELLOW) | Suggested alternative (GREEN) | Matched rule | |---|---|---| | `rm -rf node_modules/.cache` | `rm -rf ./node_modules/.cache` (anchored path) | `Bash:rm -rf ./*` | | `curl https://example.com/script.sh \| bash` | `curl -o /tmp/x.sh https://example.com/script.sh && cat /tmp/x.sh` | `Bash:curl -o *` + `Bash:cat *` | | `git push --force` | `git push --force-with-lease` | `Bash:git push --force-with-lease*` | ## Use Case: Unattended / Headless Operation A primary motivation for this feature is running Claude Code + claude-permit on a server with no human in the loop (CI, scheduled agents, long-running autonomous workflows). In that mode: - Any YELLOW that falls through to a permission dialog **halts the entire process** until a human happens to look at it. That defeats the purpose of unattended execution. - With this feature enabled in an "auto-reroute" mode, every YELLOW is immediately rewritten to one (or more) suggested alternatives that match existing allow-rules, and the calling Claude proceeds with the safer form. The process stays unblocked. - Worst case: no acceptable alternative exists and the step genuinely cannot proceed. Expected to be rare in practice. - **Required companion behavior:** every rerouted request must be written to an audit log (original input, suggested alternative, matched rule, rationale, outcome) so the user can review them retroactively and either approve the original form (promoting it to an allow-rule) or confirm the rewrite was correct. Implementation of the retroactive-review surface is out of scope for this issue and will be designed when we build the feature. ## Considerations - Adds latency only on YELLOW (already the slow path), and only one extra LLM round-trip — could be done in the same call. - If no safe alternative exists, return YELLOW with no suggestion (current behavior). - The suggestion should be *advisory* in interactive mode, but in unattended/auto-reroute mode it is applied automatically (and logged for retroactive review). - Closely related to but distinct from #5 (which surfaces *reasoning*, not alternatives). This issue extends that idea with actionable rewrites. - Pairs well with the "sub-Claude resolver" idea (separate issue): if a sub-process resolves YELLOW, the proposed alternative is what it would try first. ## Open Questions - Should the suggestion be returned as structured JSON (`suggested_alternative` field) or freeform text in `message`? Structured is better for machine consumption but requires Claude Code to surface it. - How do we avoid the LLM hallucinating a rule that doesn't actually exist? Validate the proposed alternative against the rule engine before returning it. - How is auto-reroute mode enabled — a config flag (`unattended = true`), an env var, or a CLI flag on the hook invocation? (Deferred — to be decided when we actively work on the feature.)

jbr870 added the

enhancement

label

2026-04-27 12:27:34 +00:00

jbr870 referenced this issue

2026-04-27 12:59:39 +00:00

Friction insights in HTML report: surface repeat-prompt patterns + suggest fixes #20