CI Watchdog - The Autonomous CI Minutes & Spend Sentinel with Open AI Failure Triage

CI Watchdog - an autonomous GitHub App and SaaS platform that stops wasted CI spend before it hits your invoice, and automatically diagnoses broken builds using open-weight AI.

What I Built

Every developer has that one friend or team lead who dreads the 20th of the month: the day GitHub sends the notification: "You have used 100% of your included GitHub Actions minutes." I built CI Watchdog for my friend and engineering teammate, who was constantly battling ballooning CI bills and jammed pipeline queues on open-source and startup projects.

The problems:

• Developers push 4 quick commits to a PR, and GitHub blindly spawns 4 redundant 15-minute test suites in parallel on expensive runners (Ubuntu, macOS, Windows).
• When a build fails after 20 minutes, developers have to dig through 30,000 lines of terminal logs just to discover a blocked database container port or a missing env variable.

The solution:

• Autonomous Waste Prevention: Detects and cancels superseded runs, runaway jobs exceeding historical p95 execution baselines, and PR workflows left running on closed branches.
• Proactive Workflow Hygiene: Audits repository workflows for missing concurrency blocks and generates 1-click YAML fixes.
• AI Failure Investigation: The moment a build fails, an integrated AI agent pulls the raw runner logs, redacts secrets, diagnoses the exact root cause, pinpoints the failed step, categorizes the bug, and provides copy-paste solutions.
• Human-in-the-Loop Control Plane: Includes a kill switch, dry-run observation safety rails, team approvals for cancellations, and interactive ROI spend simulators.

Demo

• 🌐 Live Dashboard: ci-watchdog.vercel.app
• 🧪 Live Demo Pull Request: Live Demo

Key Features to Explore

Route	Feature	Description
/app	Control Room Overview	Live and dry-run minutes-saved counters, real-time system status indicators, and weekly savings charts.
/app/investigations	AI Investigations	Instant root-cause diagnostic cards generated by Gemma / Gemini 3.8 Flash, with failure hypotheses, log evidence snippets, and recommended next steps.
/app/decisions	Decision Engine & CSV Export	Full searchable decision history with rule and status filters, plus 1-click CSV download for engineering management reports.
/app/approvals	Human Approval Queue	First-N quota approval flow with 1-click Approve and Deny buttons.
/app/digest	Interactive ROI Calculator	Real-time savings simulator modeling monthly minute volume and runner tiers (see pricing below).

Runner pricing used in the ROI calculator:

Runner	Cost per minute
Ubuntu	$0.008
Windows	$0.016
macOS	$0.080
Apple Silicon (M-series)	$0.160

Code

The entire codebase is open-source and structured as a high-performance TypeScript monorepo using Turborepo and pnpm:

Path	Description
apps/watchdog	Event-driven Node.js background service deployed on Render. Connects to the GitHub App webhook stream, processes workflow states, runs the pure rule engine, and coordinates AI triage.
apps/dashboard	Next.js 16 (Turbopack) dashboard deployed on Vercel with Shadcn UI, Auth.js (OAuth with GitHub & Google), and real-time state synchronization.
config/watchdog.yml	Declarative team policies with per-repository overrides.

How I Built It

Architecture Overview

1. Pure, Deterministic Rule Engine

The rule engine (apps/watchdog/src/rules/) was designed to be 100% pure with zero side effects. Given the current run, sibling runs, historical durations, and workflow YAML, it evaluates four rules:

Rule	Behavior
SUPERSEDED	Cancels older runs queued on the same ref when a newer commit arrives.
RUNAWAY_DURATION	Compares running jobs against historical p95 execution times and alerts/cancels if a job hangs.
MISSING_CONCURRENCY	Scans workflow ASTs and alerts if jobs lack top-level concurrency cancellation blocks.
STALE_ON_CLOSE	Immediately terminates active pipelines when a PR is merged or closed.

2. Intelligent AI Failure Triage

When a workflow concludes with a failure, CI Watchdog activates the AI investigator:

• Fetches the raw runner logs via the GitHub API.
• Runs pattern redaction to scrub tokens and sensitive credentials.
• Dispatches the condensed failure excerpt to open-weight models (Google Gemma / Gemini 3.8 Flash via OpenAI-compatible endpoints).
• Validates the output with Zod, enforcing structured JSON containing:

Field	Description
rootCause	Explicit technical diagnosis (e.g., missing Docker service container, database port refusal).
failedStep	Exact bash command or step name that exited non-zero.
category	One of: Infrastructure, Configuration, Test, or Code.
nextSteps	Concrete remediation actions.

3. Safety Rails & Human-in-the-Loop

Automated cancellation can be intimidating for developers, so I implemented strict safety rails:

• Default Dry-Run Mode: Repositories begin in observation mode so teams can review what would have been canceled without impacting active builds.
• 48-Hour Observation Window: Go-live can only be unlocked after a repository has been observed.
• First-N Live Approval Quota: The first cancellations require explicit human confirmation via dashboard or Slack buttons.
• Protected Branch Immunity: Never cancels main, master, or release/* branches.
• Global Kill Switch: Instantly pauses all active cancellations across all repositories.

Why Does Open Innovation Matter?

Open innovation is what makes a tool like CI Watchdog possible without compromising developer trust:

• Data Sovereignty & Privacy: Build logs often contain internal paths, dependencies, proprietary package names, and sensitive environment details. Relying on closed, proprietary cloud AI black boxes creates data-leakage risks for enterprise codebases. Open-weight models like Gemma enable teams to run AI failure analysis locally or within their own private infrastructure.
• Deterministic Customization: Open tooling and open protocols allow developers to customize rule thresholds, inspect prompts, tune temperature, and swap backend models based on compute constraints.
• No Vendor Lock-In: Developers shouldn't need a six-figure enterprise contract just to know why their CI pipeline broke or to prevent redundant job execution. Open innovation democratizes cost control for individual creators, open-source maintainers, and startups alike.

Prize Categories

• Build for a Friend
• Open Innovation & Open-Weight AI

Built with ❤️ during Hacktoberfest.