Skip to main content

Limits and caveats

Qodex’s PR review is designed for high-signal feedback on normal pull requests. These limits explain what it reviews today, what it skips, and how to work around common boundaries.

Quick summary

  • Large diffs are truncated at 200 KB.
  • Findings below 0.7 confidence are dropped.
  • Reviews are advisory by default unless you opt in to merge blocking.
  • Draft PRs and disabled repos are skipped with a neutral Check Run.
  • Verification probes are GET-only and require a discoverable preview deployment.
  • The project plan can limit both the number of pull requests reviewed in a rolling window and automatic re-reviews of one pull request.

PR-review plan limits

The Settings > PR review panel shows the effective allowance for that project, including the number of pull requests reviewed in the current window and any automatic-review limit per pull request. On the default Free PR-review tier, Qodex reviews one distinct pull request per rolling 7-day window and runs up to two automatic reviews for that pull request. A review someone explicitly requests with @qodex-ai review does not consume the automatic re-review allowance, although the first review of a new pull request still uses its project allowance. Paid or custom plans can have different limits. Use the project panel rather than a plan name as the source of truth for the active allowance.

Diff truncation at 200 KB

The unified diff is capped at 200 KB before it is sent to the model. A PR that touches a large generated file, a vendored library, or a refactor across hundreds of files will be truncated. The agent is told the diff is truncated and is instructed to focus on what it can see. The walkthrough footer shows diff truncated at 200KB when this fires. What to do: split large PRs when possible. Qodex reviews each PR independently, so two 100 KB PRs get full review coverage where one 400 KB PR loses part of the diff.

Confidence floor at 0.7

Every finding has a confidence score between 0.7 and 1.0. Anything below 0.7 is dropped before it ever reaches your PR. The floor is not tunable per project today. What to know: this is intentional. Qodex optimizes for precision over recall so the PR thread stays useful.

Review depth is risk-based

Every review starts with a high-precision pass across correctness, security, performance, maintainability, style, and conventions. On a non-trivial diff that appears clean, Qodex can run a deeper follow-up pass before reporting a clean result. The walkthrough’s What Qodex checked block records the passes and filters that actually ran. What to know: no review is a proof that a change has no defects. Findings are still limited by the available diff, repository context, and configured exclusions.

No scheduled re-review

A review fires on a GitHub event, not on a timer. If you change .qodex.yaml and want the new policy applied to an open PR, push a new commit or run @qodex-ai review. The updated policy does not retroactively re-score past findings.

Draft PRs are skipped

A draft PR is skipped with a neutral Check Run and the reason “PR is a draft; Qodex will review on ready_for_review.” Until ready-for-review handling lands, push a new commit after marking the PR ready so the next event triggers a review.

Inline comments must land on lines in the diff

GitHub rejects inline comments on lines that are not part of the PR diff. Findings that reference a line the PR did not touch move into the walkthrough body under “Findings outside the diff” and are tagged advisory. The author still sees them.

Misanchor guard can strip a useful suggestion

When a finding is anchored on a comment, blank line, import, or brace-only line, Qodex strips the suggestion block and adds an Anchor uncertain badge. The check is conservative because a wrong suggestion can replace the wrong line. The description always still posts.

Check Run is advisory by default

.qodex.yaml ships with pr_review.block_pr_merge: false. The Check Run conclusion is always neutral until the repo opts in. Even when opted in, only verified findings at or above block_on_severity flip to failure; unverified findings never block. What to know: a fresh repo with no .qodex.yaml will never gate a merge. This is the safe default.

.qodex.yaml is capped at 16 KB

The config file is fetched from the PR head SHA and rejected if larger than 16 KB. A larger file falls back to defaults silently with a logged warning.

Repo size cap at 500 MB

A repo over 500 MB cannot be linked. The link POST returns 413 with the message “Repo is MB, exceeds 500MB cap. Subdirectory selection for monorepos is planned for v2.”

10 repos per project

A Qodex project can link at most 10 GitHub repos. Hitting the cap returns 409 with “Project already has 10 linked repos (max). Unlink one before adding another.” This cap is per project, not per install.

Probe is GET-only

Verification probes are GET-only today. Every other verb is rejected before any request is made. Non-idempotent probes are gated behind an explicit .qodex.yaml opt-in (probes.allow_non_get: true) and are on the roadmap.

Probe is one request

A probe is one GET. There is no multi-step probe, no login-then-probe flow, no probe chaining, and no probe retries. A flaky preview environment marks probes as failed. The finding still posts, just without the verification badge.

Preview deployment must come from GitHub Deployments

Qodex discovers the preview URL through the GitHub Deployments API. Vercel, Netlify, Cloudflare Pages, Render, and Railway all set Deployments by default. Custom CI that does not write to the Deployments API will not be discovered, and probes land as skipped.

On the roadmap

Non-GET probes (POST, PUT, PATCH, and DELETE) remain unavailable. They are reserved for preview environments where mutation can be explicitly controlled.

Troubleshooting

.qodex.yaml reference

Verification probes

Check Run and merge gating