Check Run and merge gating

How Qodex reports PR review status in GitHub and how to opt in to merge blocking. Learn how statuses map to merge decisions.

Check Run and merge gating

Qodex posts a GitHub Check Run on every review so the PR’s checks panel shows the review status.

By default, the check is advisory. You can opt in to merge blocking when you want verified findings above a chosen severity to stop a PR from merging.

How the Check Run works

Each review has a Check Run lifecycle: posted as in_progress when the review starts, updated to completed with a conclusion when the review finishes. A re-review on the same head SHA updates the same Check Run rather than stacking new ones.

There are three possible conclusions: neutral, success, and failure. Only verified findings at or above the block_on_severity threshold flip the conclusion to failure, and only when the repo opts in via .qodex.yaml.

Lifecycle

1. Start: the Check Run is posted against the head SHA as Qodex review, in_progress, with a placeholder summary while the review runs.
2. Finish: the same Check Run is updated to completed, with the computed conclusion, a link to the walkthrough comment, a severity-count summary, and the list of blocking findings if any.
3. Re-review: a subsequent push to a new head SHA creates a fresh Check Run. A re-review on the same head SHA (via @qodex-ai review) updates the existing one.

Conclusion policy

The conclusion is decided per review:

• If .qodex.yaml has pr_review.block_pr_merge: false (the default), the conclusion is always neutral. This is advisory mode.
• If block_pr_merge: true and any verified finding meets block_on_severity, the conclusion is failure. The merge is gated.
• Otherwise the conclusion is success.

Two non-obvious rules apply.

• Only verified findings count toward the gate. An unverified finding never flips the conclusion to failure. Unverified means “the diff suggested the bug but Qodex could not confirm it”; blocking on those would destroy trust faster than missing a bug.
• Skipped or failed reviews are always neutral. Draft PRs, .qodex.yaml opt-out, repo unlinked, LLM errors, GitHub errors. Qodex never gates a merge because its own infrastructure had a problem.

Conclusion meanings

Conclusion	When it fires
neutral	Advisory mode (block_pr_merge: false). Or the review was skipped (draft, opt-out, unlinked). Or it failed (LLM error, GitHub error).
success	Gated mode and zero verified findings at or above block_on_severity.
failure	Gated mode and at least one verified finding at or above block_on_severity.

Title and summary

The Check Run title and summary describe the outcome.

• failure: title Blocking: N findings. Summary names the count, the lowest blocking severity, and the path to allow the merge.
• success: title Clean review (zero findings) or No blocking findings (some findings, none verified at threshold).
• neutral advisory mode: title Advisory: N findings or Clean review (advisory).

The Check Run details panel renders a severity-count table, a blocking findings list when any, a link to the walkthrough comment, and a one-line gate policy footer such as Gate policy: blocking - threshold: critical.

How to opt in to merge blocking

Two settings on the repo opt you in.

1. Set block_pr_merge: true in .qodex.yaml

pr_review:
  block_pr_merge: true
  block_on_severity: critical

2. Require the Qodex review check in branch protection

In GitHub, open Settings > Branches > Branch protection rules > Edit. Under “Require status checks to pass before merging”, add Qodex review. Without this, the Check Run still posts but GitHub will not block the merge button.

Tune the threshold

block_on_severity defaults to critical. Raise the bar by setting it higher (no severity is above critical, so critical is the strictest). Lower the bar by setting major, minor, nitpick, or info. Only verified findings at or above this severity flip the gate.

pr_review:
  block_pr_merge: true
  block_on_severity: major   # block on verified critical OR major findings

Use @qodex-ai resolve on a PR to mark its open Qodex findings resolved. The command does not alter the merge-gate threshold; it closes the PR’s Qodex findings and review threads.

Related

Verification probes

.qodex.yaml reference

Walkthrough anatomy

Troubleshooting