Skip to main content

​
The steps

  1. GitHub notifies CodeWarden. A pull request is opened, reopened, marked ready for review, or pushed to. Draft pull requests are not reviewed.
  2. Checks before anything is read. The review is not run if the repository is switched off in the console, the installation is suspended, or the organization has no subscription or has used its allowance. Nothing is posted on the pull request, and except for a switched-off repository, the review is recorded in the console with the reason.
  3. Read through the GitHub API. The list of changed files, the pull request’s title and description, and .codewarden.yml from the base branch. If the file skips this pull request, the review stops here, recorded as skipped. Otherwise CodeWarden reads the current content of each changed file worth reviewing. Nothing is cloned and nothing is executed.
  4. Extra context, when there is some. A Jira ticket or Sentry issue linked in the pull request, and up to three files the pull request does not touch but that use code it changes, found with GitHub code search inside the repository and read from the base branch.
  5. One model call. Everything above goes to the model in one request.
  6. Filter. Findings below the confidence threshold, over the comment cap, already decided by someone, already posted on an earlier push, or outside the lines the pull request added are dropped.
  7. Post one review. All inline comments go out together in a single GitHub review, which never approves or requests changes.

​
Which files are read

These are skipped without being asked: You can add paths with ignore and rescue built-in ones with unignore in .codewarden.yml. A file with no diff, or with more than 1,500 changed lines, cannot be rescued. The files that were skipped are named to the model, so it knows they are part of the pull request and does not report them as missing.

​
How much of each file

Small files (under about 20,000 characters) are sent whole, so the model can see a guard clause ten lines above the change. Larger files are sent as windows of 40 lines around each change. When a pull request is very large, the windows shrink, and only after that are the last files left out, by name.

​
Incremental review

The first review of a pull request reads all of it. After that, each push is reviewed only for what it changed. “The last reviewed commit” means the last review whose comments you could have seen: one that was published or that found nothing. A review that failed or was skipped does not count, so the next push is compared against code someone actually reviewed. Incremental review cannot be switched off. Repeating a comment the author already answered is how people learn to stop reading a review bot.

​
What gets posted

  • A first review posts a summary of the change, even with zero comments.
  • A follow-up review with nothing to say posts nothing at all.
  • Inline comments go only on lines the pull request added. GitHub rejects a whole review if a single comment falls outside the diff, so any such comment is dropped before posting.
  • The summary ends with how many comments were published and how many were filtered out. When a Jira ticket was read, it adds Checked against PROJ-123. When a Sentry issue was read, it says how many files of the stack trace the change touches.

​
What decides which comments are published

​
When a review fails

If the model does not answer in time, the review is retried with less of each file, up to three attempts in all. If the model twice answers with something that is not the JSON it was asked for, the review stops rather than guess. A review that fails posts nothing, uses no AI turn, and appears in the console with the status failed. A Jira, Sentry or .codewarden.yml problem never fails a review: the review goes on without that source.