DEV Community

Let AI Triage Static-Analysis Alerts Without Letting It Hide Bugs

Keep Detection and Disposition Separate

โ€œUse AI to remove static-analysis false positivesโ€ sounds like a classification feature. In production it is an authorization feature: who may hide a warning, for how long, with what evidence, and what happens when the code changes?

Keep detection and disposition separate:

scanner -> immutable finding -> context builder -> AI proposal
                                      |                    |
                                      v                    v
                               source snapshot      evidence bundle
                                                            |
                                                            v
                                              deterministic policy / reviewer
                                                            |
                                                            v
                                              time-bounded disposition

The model should never delete a finding. It proposes one of needs_review, likely_false_positive, or likely_true_positive, plus cited code ranges and reasoning.

Store a Durable Contract

type TriageProposal = {
  findingId: string;
  scannerRule: string;
  sourceRevision: string;
  verdict: "needs_review" | "likely_false_positive" | "likely_true_positive";
  citedRanges: Array<{ path: string; start: number; end: number }>;
  modelId: string;
  promptVersion: string;
};

type Disposition = {
  proposalId: string;
  action: "keep_open" | "suppress_until_change";
  decidedBy: string;
  decidedAt: string;
  expiresAt: string;
};

Bind every proposal to the scanner rule and source revision. If a cited file, data flow, dependency, or rule version changes, reopen the finding. โ€œSuppressed foreverโ€ is not a recovery policy.

Put Deterministic Gates After the Model

  • Never auto-suppress findings involving credentials, authorization, injection sinks, unsafe deserialization, cryptography, or public network exposure.
  • Require cited ranges to exist in the scanned revision. Reject proposals with missing evidence.
  • Sample accepted suppressions for review and track reopen rate by rule and repository.

Evaluation and Measurement

Your evaluation set needs:

  • Confirmed true positives
  • Confirmed false positives
  • Ambiguous cases
  • Code changes that invalidate an earlier disposition

Measure missed true positives separately from analyst time saved; averaging them into one accuracy number hides the expensive error.

Context and Disclosure

The public MonkeyCode repository describes automated PR/MR review, AI tasks, and managed development environments. The contract above is applicable to AI-assisted review generally, but it is not a claim about MonkeyCode's scanner integrations or current suppression behavior.

Disclosure: I contribute to the MonkeyCode project. Product context comes from public documentation; this workflow is an independent design pattern.

AI can reduce review effort by assembling context. Authority to hide security evidence should remain explicit, reversible, and auditable.

Comments

No comments yet. Start the discussion.