Surface LLM reasoning in hook output for YELLOW/RED decisions #5

New issue

Open

opened 2026-04-09 13:38:16 +00:00 by jbr870 · 0 comments

jbr870 commented

2026-04-09 13:38:16 +00:00

Owner

Summary

When the LLM scores a tool call as YELLOW or RED, the reasoning is logged to audit.jsonl but not shown to the user. The user sees only a generic permission dialog (YELLOW) or silent denial (RED) with no explanation.

User Stories Served

US-5: See LLM reasoning at decision time, not buried in a log file

Proposed Change

The Claude Code hook protocol supports a message field in the JSON response. Currently claude-permit only returns decision and optionally reason. Add the LLM reasoning to the response:

For RED (deny):

{
  "decision": "deny",
  "reason": "LLM safety evaluation: RED — This command attempts to delete system files. Risk: data loss, system instability."
}

For YELLOW (ask user):
The PermissionRequest hook could include the LLM reasoning in its passthrough, so Claude Code's permission dialog shows context.

Considerations

Keep messages concise — the LLM reasoning can be verbose
RED messages should be actionable: explain what triggered it and hint at how to override (e.g., "add an allow rule for this pattern")
YELLOW is trickier — the user sees Claude Code's standard permission dialog, and we can't inject text into it. May need to print to stderr as a notification.

## Summary When the LLM scores a tool call as YELLOW or RED, the reasoning is logged to audit.jsonl but not shown to the user. The user sees only a generic permission dialog (YELLOW) or silent denial (RED) with no explanation. ## User Stories Served - **US-5:** See LLM reasoning at decision time, not buried in a log file ## Proposed Change The Claude Code hook protocol supports a `message` field in the JSON response. Currently claude-permit only returns `decision` and optionally `reason`. Add the LLM reasoning to the response: **For RED (deny):** ```json { "decision": "deny", "reason": "LLM safety evaluation: RED — This command attempts to delete system files. Risk: data loss, system instability." } ``` **For YELLOW (ask user):** The PermissionRequest hook could include the LLM reasoning in its passthrough, so Claude Code's permission dialog shows context. ## Considerations - Keep messages concise — the LLM reasoning can be verbose - RED messages should be actionable: explain what triggered it and hint at how to override (e.g., "add an allow rule for this pattern") - YELLOW is trickier — the user sees Claude Code's standard permission dialog, and we can't inject text into it. May need to print to stderr as a notification.

jbr870 added the

observability

enhancement

labels

2026-04-09 13:38:16 +00:00

jbr870 referenced this issue

2026-04-27 12:27:34 +00:00

YELLOW: propose an allowed alternative instead of just asking #18

jbr870 referenced this issue

2026-04-27 12:27:41 +00:00

YELLOW: spawn sub-Claude to resolve instead of falling through to dialog #19

jbr870 referenced this issue