Skip to content

feat(ci): orchestrate automatic and interactive Claude reviews#34810

Merged
sfreudenthaler merged 3 commits intomainfrom
codex/34809-claude-auto-pr-review
Mar 3, 2026
Merged

feat(ci): orchestrate automatic and interactive Claude reviews#34810
sfreudenthaler merged 3 commits intomainfrom
codex/34809-claude-auto-pr-review

Conversation

@sfreudenthaler
Copy link
Member

@sfreudenthaler sfreudenthaler commented Feb 28, 2026

Summary

  • rename workflow to ai_claude-orchestrator.yml to reflect multi-trigger orchestration
  • add pull_request triggers (opened, synchronize) for automatic Claude reviews
  • add claude-automatic-review job using trigger_mode automatic
  • skip automatic runs when PR title/body already contains @claude to avoid double-triggering (but we have concurrently control anyways)
  • set direct prompt to a senior/staff review style (no praise, flag risk/issues, one-line clean verdict)

Non-functional considerations

  • to make sure that a bad actor can't FDoS us by opening a slew of PRs on junk to devour our tokens runs we are gated by public org membership (downside is that it must be set by our devs manually). Should that fail or be removed...
    • limits set on the anthropic side
    • we have reporting and telemetry by key
    • the key isn't hard-coded to protect against forks and such

Validation

  • reviewed workflow YAML diff locally

Closes #34809

@github-actions github-actions bot added the Area : CI/CD PR changes GitHub Actions/workflows label Feb 28, 2026
@sfreudenthaler sfreudenthaler marked this pull request as ready for review February 28, 2026 16:29
@sfreudenthaler sfreudenthaler requested a review from a team as a code owner February 28, 2026 16:29
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1cd8b89b4e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Contributor

@wezell wezell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see this work

@sfreudenthaler sfreudenthaler self-assigned this Feb 28, 2026
@sfreudenthaler
Copy link
Member Author

Let's see this work

just needs someone from core workflow devs group. I should actually just widen that net or just make it any two ppl

@spbolton
Copy link
Contributor

spbolton commented Mar 2, 2026

@sfreudenthaler Note that due to our current security model on the Repo generally external users cannot spam us with PRs. The primary reason for this is we only allow any interaction to the repo by "collaborators" so anyone needing write access to the repository needs to be approved including creating PRs. https://github.com/dotCMS/core/settings/interaction_limits.

Interestingly there is a new configuration option just added that would not have impact with our current restriction, you you can now separately prevent access to creating PRs but allow access to the repo otherwise. This could mean you can allow access for external users to create issues, but not PRs. https://github.blog/changelog/2026-02-13-new-repository-settings-for-configuring-pull-request-access/

@spbolton
Copy link
Contributor

spbolton commented Mar 2, 2026

Generally fine. Just a couple of points.

  1. Without cancel-in-progress as true if there are changes pushed to the PR in quick succession then each will independently run in parallel to completion and create their own reviews, whereas the last commit pushed should be able to cover the current state of the code. With this set to false we just use more tokens.

  2. Do we want even draft PRs to trigger review or only when it is made non-draft.

  3. I am not sure if the allowed_tools is too restrictive and it may not allow claude to read files, check history or undersand context. git diff provides the full diff but

@sfreudenthaler
Copy link
Member Author

sfreudenthaler commented Mar 2, 2026

Generally fine. Just a couple of points.

  1. Without cancel-in-progress as true if there are changes pushed to the PR in quick succession then each will independently run in parallel to completion and create their own reviews, whereas the last commit pushed should be able to cover the current state of the code. With this set to false we just use more tokens.

Let me double check. I thought the orchestrator in ai-workflows handled this but I might have mistaken.

  1. Do we want even draft PRs to trigger review or only when it is made non-draft.

Yes :) early feedback for devs is worth the offset in token waste

  1. I am not sure if the allowed_tools is too restrictive and it may not allow claude to read files, check history or undersand context. git diff provides the full diff but

Probably but lets see

Move concurrency control from workflow level to job level so that
automatic review cancellations on new pushes never interrupt an
in-progress user @claude session.

- claude-interactive: cancel-in-progress: false (protect user sessions)
- claude-automatic-review: cancel-in-progress: true (always review latest)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Member Author

@sfreudenthaler sfreudenthaler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ready for review again @spbolton

@sfreudenthaler sfreudenthaler requested a review from spbolton March 2, 2026 15:12
@sfreudenthaler sfreudenthaler added this pull request to the merge queue Mar 3, 2026
Merged via the queue into main with commit 5f6da07 Mar 3, 2026
25 checks passed
@sfreudenthaler sfreudenthaler deleted the codex/34809-claude-auto-pr-review branch March 3, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area : CI/CD PR changes GitHub Actions/workflows

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Enable automatic Claude PR reviews in dotcms/core

4 participants