Weighted Averages Lie About AI Readiness - The Case for Bottleneck Scoring

Most self-assessment tools score you the same way: answer questions, multiply by weights, sum it up. Higher total = more ready. That model has a structural flaw.

A team that doesn't use Git - but aces documentation, AI policy, and project fit - can score high. Yet without version control, every large AI-generated change is an irreversible overwrite. No amount of strength elsewhere compensates. Call it the honor student problem: additive scoring rewards averages when what actually matters is the weakest link.

This article walks through how I designed a scoring model that caps the total score when a fatal precondition is missing - and why the whole mechanism collapses into a single Math.min().

The Honor Student Problem

How typical readiness assessments work: weighted average across categories. The failure mode: catastrophic weakness in one area is diluted by strength in others.

Concrete example: no Git + perfect everything else = high score under additive scoring. Why this matters more for AI-driven development: AI multiplies change volume, so missing safety rails are amplified, not averaged away.

Bottleneck Scoring: Average for Progress, Cap for Preconditions

Keep the weighted average - it's good at expressing continuous improvement. Add a second layer: fatal preconditions that cap the total score when absent. The entire mechanism reduces to Math.min(baseScore, cap).

Analogy: Liebig's law of the minimum (the barrel with one short stave).

The Six Preconditions and Their Caps

• No version control (Git) → capped at 49
• Almost no written specs → capped at 49
• No task/ticket management → capped at 59
• No human review / production approval → capped at 59
• No automated tests and no change checklist → capped at 59
• No rules on what must not be fed to AI → capped at 69

When multiple fire, the lowest cap wins (rate-limited by the worst bottleneck).

Why 49, 59, 69 - Caps as Level Ceilings

100-point scale, 5 axes: Documentation 25 / Process 25 / Quality Assurance 20 / AI Usage 15 / Project Fit 15. 5 levels (Lv1: 0-29 … Lv5: 85-100). Caps sit deliberately just below level boundaries (50, 70): a cap encodes "the highest level you can reach while carrying this gap."

Proving It Works: Test Cases as Design Documentation

• All answers max → 100, Lv5
• No Git, everything else perfect → forced to 49 (Lv2)
• Multiple caps at once → the minimum (49) is applied

Deterministic scoring (same input, same output) guaranteed by Vitest unit tests.

Design Decisions That Followed From the Same Principle

• "I don't know" scores 0.2, not 0 - not knowing is different from not having
• Solo developers: team-only questions (e.g., PR review practice) get 0.5× weight
• Recommendations prioritized by impact (axis points / questions per axis × unmet degree), organized into a now / 1-month / 3-month roadmap
• Strength selection enforces axis diversity
• Of 8 development phases evaluated for AI fitness, documentation is the only one that can never be "not recommended"

The Tool Itself (Briefly)

Fully client-side static app: TypeScript, React 19, Vite 8, Tailwind CSS 4, shadcn-ui; GitHub Pages. Zero external transmission - answers stay in LocalStorage/IndexedDB, verified automatically with Playwright E2E tests. Scoring logic versioned (SCHEMA_VERSION); old results get an "outdated version" badge. 5 languages, MIT license, just shipped - no usage numbers to brag about, and don't pretend otherwise.

What bottleneck is your own team averaging away?