We ran Pinaka on an open, unresolved Prisma bug. No ground truth. Here's what it found.

The Bug

When using the PrismaPg driver adapter, filtering nullable String? fields for null values throws a PrismaClientValidationError:

// These all throw:
prisma.user.findMany({ where: { name: null } })
prisma.user.findMany({ where: { name: { equals: null } } })
prisma.user.findMany({ where: { name: { not: null } } })

// This works fine (same syntax, different field type):
prisma.user.findMany({ where: { createdAt: null } }) // DateTime? - no error

The error:

PrismaClientValidationError: Invalid `prisma.user.findMany()` invocation:

{
  where: {
    name: null
    ~~~~
  }
}

Argument `name`: Invalid value provided. Expected StringNullableFilter or String, provided Null.

The bug is reproducible, consistent, and type-specific - it only fires on String? fields, not DateTime? or integer fields with identical null filter syntax. It has been open and unresolved across four Prisma minor releases.

What Pinaka was working with

This run had live SDK runtime context - real logs, real stack trace, real request context captured at crash time via @pinaka/sdk. The logs included something important:

[DEBUG] Executing prisma.user.findMany with where: { name: null }
[ERROR] PrismaClientValidationError thrown
[DEBUG] Retried with { name: { equals: null } } - same error
[DEBUG] Tried { createdAt: null } on DateTime? field - works fine

Those last two lines - the retry attempt and the DateTime contrast - are what made this diagnosis possible. Without them, a static-code-only RCA would have seen the error message and stopped at "user passed wrong type." The logs proved it wasn't user error by documenting what was already tried.

Pipeline stats for this run

• Vector search: 25 candidates
• Direct stack trace file fetch: 3 files force-included
• Total reranker candidates: 45 (25 vector + 20 direct fetch)
• Final context passed to LLM: 8 chunks
• Reranker: Voyage rerank-2, no fallback triggered
• Time to RCA from job pickup: ~3 seconds

What Pinaka diagnosed

Failure class: Logic error in the PrismaPg adapter's SQL generation layer.

Root cause: The PrismaPg adapter incorrectly generates WHERE "name" = NULL instead of WHERE "name" IS NULL for findMany operations on nullable String? fields. In PostgreSQL, = NULL always evaluates to false - IS NULL is the correct predicate. This explains why validation fails before the query even reaches the database.

How Pinaka isolated it: By contrasting three cases found in retrieved test evidence:

The intersection of those three data points isolates the bug to findMany + String? specifically - not a general null handling problem, not a schema problem, not a DMMF problem.

Fault ownership: Framework bug in the PrismaPg adapter, not user error. Any application-side workaround is exactly that - a workaround. The correct fix belongs upstream in Prisma.

Key hypotheses eliminated:

• "The field isn't nullable" - ruled out: null records can be created successfully, and the error is client-side validation, not a DB constraint
• "Prisma doesn't support null filtering at all" - ruled out by the DateTime? working case with identical syntax
• "The fix lives in the client runtime" - ruled out: the error originates in @prisma/client/runtime/library.js, but behavioral evidence points to adapter-level SQL generation as the deeper cause

Directional fix: Change = NULL to IS NULL in the adapter's SQL generation path for findMany + String? cases. Pinaka did not invent API syntax or a specific line-level fix it couldn't confirm from retrieved code - it stated this directionally without overreaching.

Where Pinaka was appropriately uncertain

The adapter's actual source (packages/adapter-pg/) was not retrieved. The exact file and line of the SQL generation fault was not pinpointed. Confidence was flagged as HIGH in the output, but the caveat correctly noted the adapter source was missing - this is a known limitation of how retrieval works: the ticket framing pulled vector search toward client runtime files rather than adapter internals. An adapter-focused ticket would likely have surfaced the SQL generation code directly. We're saying this in the body of the post, not buried in footnotes.

What the SDK actually contributed here

The SDK-captured log lines did more diagnostic work than the entire codebase read. Specifically:

The line "Tried { createdAt: null } on DateTime? field - works fine" - captured automatically because the engineer's retry attempt happened in the same request context - is what enabled asymmetric behavior diagnosis. Without that log line, the investigation would have stayed focused on the error message alone, which points to client-side validation, not adapter-level SQL generation.

Static code reading tells you what the code says it should do. Runtime context tells you what actually happened when someone tried it three different ways and got three different results. Those aren't redundant - they're complementary, and in this case the runtime evidence was the lever.

What this proves - and what it doesn't

Unlike the BullMQ benchmark, there is no merged PR to score against. We cannot tell you "Pinaka's diagnosis matches the correct fix" because no correct fix exists yet.

What we can tell you: the diagnosis is consistent with all behavioral evidence available - the asymmetric behavior across field types, the error message origin, the test file contrast cases - and the reasoning chain is transparent enough to evaluate independently.

The harder demonstration here isn't scoring accuracy. It's that Pinaka can reason from behavioral contrast evidence across test files, not just trace a known stack to a known answer. Those are different capabilities, and the second one matters more in the cases where no one has found the answer yet.

If Prisma ships a fix for #29480 that differs from what Pinaka diagnosed, we'll update this post.

Honest limitations

• No ground truth - diagnosis validated against behavioral evidence only, not a merged fix
• Adapter source not retrieved - exact fault line not pinpointed
• Single bug, single repo - not statistically powered
• Ticket framing affects retrieval - an adapter-focused ticket would likely have surfaced the SQL generation code directly

We're continuing to run this across more repositories. We'll publish what we find - including results that don't look as clean as this one.

Try it

If you want to see what Pinaka finds on your own codebase, join waitlist at getpinaka.com. We're building this in public - benchmarks like this one are how we're deciding what to build next, not just what to put in a launch post.

Previously: Can AI actually find the root cause of a bug, or does it just sound confident? (BullMQ benchmark)