AI Artifact Scoring

Overview

Documentation traditionally scores 1-2 in GitVelocity -- READMEs, changelogs, and the like get stripped from effective-line counting because they're prose, not code. AI artifacts are different. A SKILL.md file changes how an agent behaves at runtime, the same way a config file or a prompt template does. Adding one to a team's .claude/skills/ directory can reshape how every PR review, every commit message, every release note is produced.

GitVelocity recognizes AI artifact files as behavioral code, not documentation. This works for skills built for Claude Code (Anthropic), Codex (OpenAI), and Gemini CLI (Google) -- all three engines ship skills that follow the open agentskills.io standard. Artifacts count toward effective lines, contribute to ESF tier, and get a skill-aware sub-rubric instead of being lumped in with READMEs.

What counts as an AI artifact

By default, GitVelocity counts these file types as AI artifacts:

  • Open-standard skills (interoperable) -- **/.agents/skills/**/SKILL.md (require valid YAML frontmatter). The .agents/skills/ path is the interoperable location that works across Claude Code, Codex, and Gemini CLI.
  • Claude Code skills -- **/.claude/skills/**/SKILL.md (require valid YAML frontmatter)
  • Claude Code slash commands -- **/.claude/commands/*.md (require valid YAML frontmatter)
  • Claude Code sub-agents -- **/.claude/agents/**/*.md (require valid YAML frontmatter)
  • Gemini CLI skills -- **/.gemini/skills/**/SKILL.md (require valid YAML frontmatter)
  • Open-standard skills (alt layout) -- **/skills/*/SKILL.md (require valid YAML frontmatter, follows agentskills.io)
  • Memory files -- **/CLAUDE.md (Claude Code), **/AGENTS.md (Codex + open standard), **/GEMINI.md (Gemini CLI). No frontmatter required by spec.

Files matching the spec-grounded patterns above must declare valid YAML frontmatter (a block starting with ---, containing at least one key: value line, and ending with ---). Memory files are exempt because they're structured prose by spec.

How the score differs

When the AI scoring assistant sees an AI artifact in a PR, it switches to the AI Artifact Sub-Rubric. The same six dimensions (S/A/I/R/Q/P) still apply, but with skill-grading semantics:

  • Scope -- How broadly does this skill change agent behavior? A new dispatcher entry is system-wide; a one-off helper is localized.
  • Architecture -- Does the skill introduce a new pattern (governance layer, sub-agent dispatch, reviewer-of-reviewers)?
  • Implementation -- Prompt engineering rigor: complete frontmatter, principled tool/permission scopes, anchored examples, structured outputs, decision trees.
  • Risk -- Does the skill modify destructive behavior, expand tool permissions, or loosen safety rails?
  • Quality -- Worked examples, clear "when to use / when NOT to use" triggers, frontmatter that is actually referenced in the body.
  • Performance/Security -- Permission tightness, secret-handling guidance, prompt-injection defenses.

Inclusion overrides exclusion

If a file matches both an Exclusion Filter (e.g., a blanket *.md rule) and an AI artifact pattern, the AI artifact rule wins. This means you can leave *.md in your exclusion list to keep generic READMEs out of scoring while still getting full credit for skills, commands, and agents.

Configuring custom patterns

The default patterns cover the standard layouts for Claude Code, Codex, and Gemini CLI. If your monorepo puts skills in a non-standard location, add patterns to the AI & Skill Files field on the AI Scoring Settings page.

Examples:

# Per-app skills in a monorepo
apps/*/.claude/skills/**/SKILL.md

# Internal prompt library
src/agents/**/SKILL.md

# Custom skill ecosystem with a different filename convention
prompts/**/index.md

Lines starting with # are comments. Patterns use glob syntax via minimatch.

Patterns added here are additive to the defaults -- they do not replace them. You only need to add patterns if your layout differs from the standards above.

Anti-gaming controls

To prevent score inflation from prompt-only PRs, GitVelocity enforces:

  • Frontmatter rigor -- A SKILL.md with frontmatter that declares tools or dependencies but never references them in the body is treated as gaming and reduces the Quality score.
  • AI-only ESF cap -- A PR consisting entirely of AI artifact files (no production code) caps at the Medium ESF tier (0.6x), preventing volume-based score inflation.
  • Memory-file structure -- Stream-of-consciousness CLAUDE.md brain dumps score the same as a sloppy README. Structured, durable instructions earn full credit.
  • Glob patterns only -- Customer-supplied patterns are matched via minimatch, never compiled as raw regexes. Pattern syntax errors are rejected on save.