A Field Taxonomy of Bugs in AI-Generated Python

Researchers sort bugs in AI-generated code into about eight classes, and only some of them have a structural marker a scanner can match on. BrassCoders, the scanner that catches what AI assistants structurally miss, catches those deterministically and flags the rest for a human or an LLM to judge. Knowing which class is which tells you what to automate and what to read by hand. This is a map, not a ranking. The point is matching each bug class to the layer that can actually catch it.

The Eight Bug Classes

BrassCoders' coverage maps onto a published taxonomy: a 2025 survey sorts bugs in AI-generated code into eight classes, from functional and reliability bugs to a category the authors call hallucination, where the code looks plausible and runs but is factually wrong. The survey, A Survey of Bugs in AI-Generated Code by Gao and colleagues, names the eight:

• functional bugs
• reliability bugs
• syntax bugs
• code style and standards issues
• hallucination
• system bugs
• test bugs
• an unspecified catch-all

A separate empirical study, Bugs in Large Language Models Generated Code, drills into ten recurring patterns from 333 real bugs, including misinterpretations, missing corner cases, and hallucinated objects. The two frameworks are distinct, and together they describe the terrain.

What A Deterministic Scanner Catches

BrassCoders catches the classes with a structural fingerprint: hallucinated imports, performance anti-patterns, hardcoded secrets, syntax errors, and the cross-file taint that single-file review skips. Its 12 scanners match patterns, not intent. These map to the survey's syntax, reliability, and parts of the hallucination classes.

An import that doesn't resolve, a csv += loop that rebuilds a string every iteration, a subprocess call with shell=True, a high-entropy string shaped like an AWS key: each has a marker that a rule fires on, the same way every time. No judgment about what the code is for is needed to flag them. That is precisely the work a deterministic gate should own.

What Needs A Human Or An LLM

BrassCoders flags but does not triage the classes that turn on intent: functional logic errors, missing corner cases, and confident-but-wrong code that runs cleanly while computing the wrong answer. A rule has nothing to match on when the syntax is fine and the logic is subtly off. These are the survey's functional bugs and the empirical study's misinterpretations and missing-corner-case patterns.

Catching them means knowing what the code was supposed to do, which is reasoning a scanner can't do without inferring context, and inferring context is where a scanner starts being wrong. BrassCoders' design draws the line there on purpose: it reports the pattern and the surrounding context as YAML, and a human or an LLM like Claude Code decides whether the logic holds.

The Hardest Class: Hallucination

BrassCoders turns one slice of the hardest class into a clean check: hallucinated imports either resolve or they don't. A 2025 study of developer-LLM conversations found undefined variables in 83.4% of Python snippets, and a USENIX study found 19.7% of AI-recommended packages don't exist.

Hallucination is hard in general because plausible-looking wrong code has no universal marker. But the import sub-case is deterministic: a package is on the index or it isn't; a name is defined in scope or it isn't. The developer-LLM study puts undefined-variable rates above 83% for Python, and the package-hallucination study puts non-existent imports near one in five. BrassCoders checks both before pip install runs, which removes the supply-chain risk from a class that otherwise needs a human.

Mapping The Taxonomy To Your Scan

BrassCoders runs the structural classes as 12 scanners and hands the intent-dependent ones to your reviewer as YAML, so the taxonomy becomes a division of labor rather than a reading list. The scan output names the class for each finding.

The takeaway from the research is that AI-code bugs split cleanly into "has a marker" and "needs reasoning." Automate the first set with a deterministic gate; route the second to a human or an LLM with the context attached. Run the scan, read the YAML, and spend your attention on the bugs that actually need a brain.

pip install brasscoders
brasscoders --offline scan /path/to/your/project