DEV Community 2h ago

Your AI makes CI green by cheating. I built three GitHub Actions to stop it.

The ratchet idea

A ratchet is the mechanism that lets a wrench turn one way and never back. All three actions share one loop:

Set a baseline: the number of escape hatches you have today.
If a PR increases the count → the gate fails.
If it decreases → the gate prints IMPROVED, and you lower the baseline to lock it in.
The number can only go down.

The important part: you don't start from zero. Real codebases have Anys and # noqas that exist for defensible reasons. "Keep today's mess, forbid tomorrow's" is an on-ramp a team will actually take - and every cleanup tightens the ratchet by one click.

The three gates

1. type-ratchet - stops type-checker silencing

Counts: TypeScript any (in type position), as any, @ts-ignore, @ts-expect-error; Python Any, # type: ignore.

Why existing tools don't cover it: tsc and mypy --strict don't count their own escape hatches - that's what the hatches are for. Write as any and the type checker goes green without telling anyone it was silenced.

# .github/workflows/type-ratchet.yml
name: Type Ratchet Gate
on:
  pull_request:
    branches: [main]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: motchalini-llc/type-ratchet@v1
        with:
          language: typescript  # python | typescript | auto

Notes: test files (*.test.ts, *.spec.ts) are excluded from the count - mocks legitimately use as any - but you can still pass a typecheck-command (e.g. pnpm exec tsc --noEmit) to type-check everything alongside the count. Baselines are split (baseline-any / baseline-suppress) so you can tighten dynamic types and type-suppressions independently.

One honest caveat: detection is grep-based, so an "any" inside a comment or string can false-positive. That's the price of transparency (you can read exactly what's counted); an opt-in ESLint no-explicit-any backing is on the roadmap for people who want strictness.

2. test-ratchet - stops test dodging

Counts two different sins:

Skipped tests (ratcheted via baseline-skip): Python @pytest.mark.skip/skipif/xfail, pytest.skip(), unittest @skip variants; TypeScript .skip / .todo / .fails on it/test/describe/bench, plus xit / xdescribe.
.only (hard-forbidden at zero, TypeScript): a single it.only makes the runner execute that block and nothing else - the rest of your suite silently stops running while CI stays green. This one isn't baselined; one occurrence is red (disable with forbid-only: false).

Why existing tools don't cover it: eslint-plugin-no-only-tests is the known answer for .only, but it assumes TypeScript and an ESLint setup. I couldn't find a cross-language, zero-config PR gate - and nothing at all that ratchets pytest skips.

# .github/workflows/test-ratchet.yml
name: Test Ratchet Gate
on:
  pull_request:
    branches: [main]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: motchalini-llc/test-ratchet@v1
        with:
          language: typescript  # python | typescript | auto

3. suppress-ratchet - stops linter silencing

Counts: TypeScript eslint-disable (line / next-line / block) and biome-ignore; Python # noqa, # ruff: noqa, # pylint: disable.

Why existing tools don't cover it: linters are designed to respect suppression comments. The more you write, the quieter the linter gets, and nothing in the standard toolchain pushes back on the trend.

# .github/workflows/suppress-ratchet.yml
name: Suppress Ratchet Gate
on:
  pull_request:
    branches: [main]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: motchalini-llc/suppress-ratchet@v1
        with:
          language: typescript  # python | typescript | auto
          baseline-suppress: '12'  # suppressions already in the codebase

Notes: type suppressions (@ts-ignore, # type: ignore) are deliberately not counted here - they belong to type-ratchet, and the same comment shouldn't be double-counted by two gates. Vendor and generated trees (node_modules, dist, .venv, .next, …) are excluded.

Design decisions shared by all three

Zero-dependency bash. Each action is a composite action plus one gate.sh; the machinery is essentially grep and wc. You can read the script and know exactly what's counted - the opposite of a black-box AI reviewer. Your code never leaves your CI.
Zero config. One uses: line. Language is auto-detected from pyproject.toml / tsconfig.json and friends.
One-way only. Increase fails, decrease tightens. Baseline lives in an input or a baseline-file in the repo.
The gates test themselves. Each repo ships clean/dirty fixtures and a self-test workflow asserting clean passes, dirty fails (test- and suppress-ratchet run that matrix for both languages), so the gate can't silently break.
Light version first, heavy version on demand. v1 is a free grep gate. Autofix (mode: fix) and AST-precise detection are on the roadmap, opt-in, with your own LLM API key and a hard cost cap - I'll build them when someone actually asks.
Global count, honestly. The judgment is a repo-wide total; per-file baselines (monorepos) are future work. Coarse, but it's what makes zero-config possible.

Dogfooding numbers

Before publishing I ran all three on my own two repos. Real numbers:

crypto-trading-system (Python, mypy --strict): baseline any=5, type-suppressions 2, skips 0, linter suppressions 4.
meguri (Next.js 15 / React 19, strict TS): any=0 at adoption - so there the same gate runs as keep-it-at-zero insurance rather than a cleanup driver.

The baseline-suppress: 4 was the instructive one. Those four were pre-existing # noqas in test code - F401 unused imports, PLC0415 function-level imports - each left there on purpose. If the gate had demanded zero on day one, I'd have ripped it out by lunch. "Today's 4 are fine; the 5th is not" is what made it adoptable, and that exact number now sits in the README example.

I also opened a demo PR that deliberately added any / as any to watch the gate go red with inline annotations on the offending lines, then closed it. (Deliberate demo - not a caught-the-AI-in-the-act story.)

How they were built

The bash is honestly simple. The hard part was deciding what counts and what doesn't: excluding test files from the any count, keeping type suppressions out of suppress-ratchet, hard-forbidding .only instead of baselining it. Those line-drawing decisions are the product.

Mechanically: solo dev plus Claude Code, two to three days per action, the same playbook three times - build → dogfood on my own repos → publish to the GitHub Marketplace → announce. The skeleton (composite action + gate.sh + clean/dirty self-test + README + roadmap) carries over; only the patterns change. Hence: three siblings.