Skip to content

Review every PR with Claude, including from authors without write access - #2043

Merged
sbryngelson merged 2 commits into
MFlowCode:masterfrom
sbryngelson:ci/claude-review-all-prs
Oct 13, 2026
Merged

sbryngelson merged 2 commits into
MFlowCode:masterfrom
sbryngelson:ci/claude-review-all-prs

Conversation

@sbryngelson

Copy link
Copy Markdown
Member

Outside contributors get no Claude review today. claude-code-action stops with

Action failed with error: Actor does not have write permissions to the repository

and because the step is continue-on-error, the job goes green and the fallback posts "The review produced no output (Claude step: failure)" — which reads like a Claude bug rather than a policy. #2038 (BCKim55) and #1926 (thierrydaoud) are both in that state; both authors have read.

The check is on the triggering actor, not the PR author, so it only bites when the contributor themselves opens the PR. A maintainer commenting @claude full review already works — but that is manual, and every new PR from a contributor still gets the confusing comment first.

Changes

1. Review PRs without a shell. The prompt only reads prepared inputs (the diff, the changed-file list, the bounded context directory, three docs) and writes .claude-review/output.md for the existing publish step — it already says "Do not post GitHub comments yourself." So --allowedTools "Bash" and --dangerously-skip-permissions become --allowedTools "Read,Write,Glob,Grep", and the two prompt steps that said cat now say Read. Worth doing regardless of the gate.

2. allowed_non_write_users: "*". The action's supported way to lift the check. It takes effect only because github_token is already passed; allowed_bots stays at its default, so bots still cannot trigger a review.

Why this is safe to open up

The action's docs call this bypass risky and ask that it be paired with a limited permission scope. With the shell gone:

  • Claude cannot run commands, reach the OAuth token, or call the GitHub API. It reads prepared files and writes one markdown file.
  • Nothing executes that file; the publish step is defined in the workflow and only reads it.
  • The checkout stays on the base ref (no ref: override), so a PR's code is never executed in this privileged context — the thing pull_request_target actually gets wrong when it goes wrong.
  • No untrusted github.event.* field is interpolated into any run: block, so there is no script-injection surface.
  • Default subprocess secret scrubbing stays on.

The residual risk is prompt injection steering the text of a review comment. That is content, not privilege.

Tradeoff, stated plainly

"*" means anyone who opens a PR spends the OAuth token's budget. That is the intended effect — every PR gets a review — but it is a real cost/abuse surface on a public repo. An explicit username list is the conservative alternative if that becomes a problem.

Testing

pull_request_target runs the workflow from the base branch, so this cannot exercise itself on this PR — it takes effect once merged. Verified by inspection instead: YAML parses, the claude step's inputs are allowed_non_write_users, claude_args, claude_code_oauth_token, github_token, prompt, and claude_args is --max-turns 40 --allowedTools "Read,Write,Glob,Grep". ./mfc.sh precheck passes.

First PR from a read-access contributor after merge is the real check. If the review comes back thin, the tool allowlist is the first thing to look at — Read,Write,Glob,Grep should cover the prompt, but it is the one behavioural change here.

Follow-up, deliberately not in this PR

The job still holds pull-requests: write / issues: write while Claude runs. Splitting it into an unprivileged review job and a privileged publish job would be tidier, but with the shell removed Claude cannot reach those permissions, so the marginal gain is small next to the risk of breaking publishing in a workflow that cannot be tested before merge.

The review prompt only reads prepared inputs -- the diff, the changed-file
list, the bounded context directory and three docs -- and writes one file,
.claude-review/output.md, which a later step publishes. It never needed to
run commands, so grant Read/Write/Glob/Grep instead of Bash and drop
--dangerously-skip-permissions. The two prompt steps that said "cat" now
say "Read".

This is worth having on its own, and it is a precondition for opening the
review to authors without write access: shell access in a job holding the
OAuth token is the payload a prompt injection in a PR diff would want.
claude-code-action refuses to run when the triggering actor lacks write
access ("Actor does not have write permissions to the repository"), so
outside contributors' PRs got no review -- just the fallback comment
saying the review produced no output. MFlowCode#2038 and MFlowCode#1926 are both in that
state.

Set allowed_non_write_users: "*", the action's supported way to lift
that check. It only takes effect because github_token is already passed
below it. allowed_bots is left at its default, so bots still cannot
trigger a review.

The action's docs call this bypass risky and ask that it be paired with
a limited permission scope, which is why the preceding commit took the
shell away: Claude now has Read/Write/Glob/Grep, cannot reach the token
or the GitHub API, and only writes .claude-review/output.md for a later
step to publish. The checkout stays on the base ref, so a PR's code is
never executed here.

Note this opens the OAuth token's budget to anyone who opens a PR.
Copilot AI balanced review requested due to automatic review settings October 13, 2026 02:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T02:47:46.828423Z bb9621f PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@sbryngelson
sbryngelson merged commit 1466113 into MFlowCode:master Oct 13, 2026
20 checks passed
@sbryngelson
sbryngelson deleted the ci/claude-review-all-prs branch October 13, 2026 02:43

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb9621fe0e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

--max-turns 40
--allowedTools
"Bash"
"Read,Write,Glob,Grep"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Stop persisting the token before accepting untrusted prompts

When a non-write user opens a fork PR, this workflow feeds their diff and context into Claude; the CLI reference describes --allowedTools as allowing tools “without prompting for permission” (Claude Code CLI reference), so the bare Read permission is not confined to the prepared inputs. The checkout at .github/workflows/claude-code-review.yml:67-70 leaves persist-credentials at its default, which this repository itself documents as storing GITHUB_TOKEN in an http.extraheader (coverage-refresh.yml:29-35); a prompt-injected diff can therefore have Claude read .git/config and copy this job's pull-request/issues-write token into the subsequently published output.md. Disable checkout credential persistence or isolate Claude in a tokenless job before accepting these prompts.

AGENTS.md reference: AGENTS.md:L4-L6

Useful? React with 👍 / 👎.

# otherwise stops on "Actor does not have write permissions"; it needs
# github_token above to honour this. allowed_bots stays empty, so bots still
# cannot trigger a review.
allowed_non_write_users: "*"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep arbitrary commenters from spending review budget

The existing issue_comment arm at .github/workflows/claude-code-review.yml:24-28 accepts @claude full review from any commenter, and this wildcard removes the action's non-write-user gate for that event as well as for PR creation. On any public PR, a non-bot account that neither authored nor maintains the PR can now post the phrase repeatedly and start a separate 40-turn OAuth-backed review each time, with no association, label, or rate check. Restrict the manual trigger to maintainers or authorized author associations, or separate its authorization policy from automatic PR reviews.

Useful? React with 👍 / 👎.

@github-actions

Copy link
Copy Markdown

Claude Code Review

Head SHA: bb9621f

Files changed:

  • 1
  • .github/workflows/claude-code-review.yml

Findings:

  • medium .github/workflows/claude-code-review.yml: allowed_non_write_users: "*" on a pull_request_target workflow lets any outside author trigger a run that has CLAUDE_CODE_OAUTH_TOKEN and a pull-requests: write / issues: write token. Fork-controlled diff and file contents are fed to the model, so prompt injection is now reachable by anyone. The change does narrow the risk. --dangerously-skip-permissions and unrestricted Bash are gone, and the allowlist is now Read,Write,Glob,Grep. Two gaps remain: Write and Read are not path-scoped, so the model can read any runner file (including the checkout and .git/config) and write anywhere, and the final step publishes output.md verbatim as a PR comment, so injected content could be exfiltrated or posted. Consider scoping the tools (for example Write(.claude-review/output.md) and Read limited to .claude-review/**, the docs and .claude/CLAUDE.md), and confirm the checkout step does not persist credentials. Confidence: medium. This is a trust-model concern, not a functional bug.
  • medium .github/workflows/claude-code-review.yml: the step 1 and 3 wording changed from cat to Read, and the allowlist now omits Bash. Nothing in the prompt's step 5 requires a shell, so the workflow should still work. Confirm that the allowedTools comma-separated format is accepted when combined with the multi-line folded claude_args (it is quoted as a single argument, which should be fine). Confidence: low.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants