Scoring Guidelines

Overview

Scoring guidelines define the criteria the AI uses to evaluate pull requests. GitVelocity ships with default guidelines that are calibrated across thousands of scored PRs and work well for most teams.

Default Guidelines

The default guidelines instruct the AI to evaluate PRs across the six standard dimensions: Scope, Architecture, Implementation, Risk, Quality, and Performance/Security. These defaults have been refined over time to produce consistent, meaningful scores across different languages, frameworks, and team sizes.

For most teams, the defaults are the right choice. They provide a shared baseline that makes scores comparable across repositories and contributors.

Customizing Guidelines

If your team has domain-specific evaluation criteria that the defaults don't capture, you can customize the scoring prompt on the repository settings page.

Examples of when customization makes sense:

  • Your team values test coverage more heavily than the defaults reflect
  • You work in a domain where security considerations should carry extra weight (e.g., fintech, healthcare)
  • You want to de-emphasize certain dimensions that aren't relevant to your codebase

Edit the guidelines text to adjust how the AI weighs different factors. The AI will follow your custom instructions when scoring PRs for that repository.

Guideline Versioning

Every change to the scoring guidelines creates a new version. The version history is visible on the settings page, showing what changed and when.

Versioning ensures you can:

  • Track how scoring criteria have evolved over time
  • Understand if a shift in scores corresponds to a guideline change rather than a change in team output
  • Revert to a previous version if a change doesn't produce the results you expected

Recommendations

Don't change the guidelines unless you're confident the defaults aren't capturing what matters for your team. Customizing guidelines means your scores may not be directly comparable to teams using the defaults. If you do customize, make incremental changes and observe the impact before making further adjustments.