Nebula Agent: An AI That Surfaces Contradictions in Structured Content
DEV Community

Nebula Agent: An AI That Surfaces Contradictions in Structured Content

What I Built

A Sanity-powered AI agent that answers questions about a fictional board game called Nebula and, crucially, surfaces contradictions between sources. The agent doesn't just search for keywords. It writes its own GROQ (Graph Relational Object Queries) against a Sanity Content Lake in real time, reads the structured articles it retrieves, and when two sources disagree presents both claims side by side with their source URLs.

Core question: "According to the Nebula rulebook and errata, how many energy tokens do players start with? Are there any contradictions?"

Agent's answer:

The sources provide conflicting information:

  • Nebula Core Rulebook - Setup states players start with 5 energy tokens (source)
  • Nebula Official Errata v2.1 corrects this to 8 energy tokens (source)

A keyword search would have returned both articles. It would not have known which was authoritative, nor that the errata corrects the rulebook. My agent does this because the content was structured.

Demo

Since the agent runs in the terminal, here's a live transcript from two consecutive runs.

Question 1 - Energy tokens

๐Ÿ‘ค User: According to the Nebula rulebook and errata, how many energy tokens do players start with? Are there any contradictions?

๐Ÿค– Agent using tool: query_documents
Args: {"query":"*[_type match \"*rule*\" || _type match \"*errata*\" || lower(title) match \"*rule*\" || lower(title) match \"*errata*\"]"}

Result preview: [{"_type":"article","body":"To win Nebula, you must collect 10 stars..."}]

๐Ÿค– Agent: According to the Nebula rulebook and errata, there is a contradiction regarding how many energy tokens players start with:
* Nebula Core Rulebook - Setup: States that each player starts with 5 energy tokens. Source: https://example.com/nebula-rulebook
* Nebula Official Errata v2.1: States that the core rulebook is incorrect and that the correct starting amount is 8 energy tokens. Source: https://example.com/nebula-errata

Question 2 - Win condition

๐Ÿ‘ค User: What is the win condition for Nebula?

๐Ÿค– Agent using tool: query_documents
Args: {"query":"*[_type == \"article\" && (title match \"*Nebula*\" || body match \"*Nebula*\")]{title, body, source}"}

Result preview: [{"body":"Many players struggle with energy management..."}]

๐Ÿค– Agent: The sources provide conflicting information regarding the win condition for Nebula:
* According to the Nebula Core Rulebook and the strategy guide, you must collect 10 stars to win (Rulebook, Strategy Guide).
* According to the Nebula Official Errata v2.1, the win condition was updated to collecting 12 stars (Errata).

Code

Repository: Vicarioy / sanity-nebula-agent
Description: Nebula Sanity Agent - A command-line agent that uses Gemini to answer questions from article content stored in Sanity.

How It Works

The agent checks its Sanity connection, sends a built-in question about Nebula's win condition to Gemini, and answers using Sanity query results. It can query documents with GROQ or list the dataset's document types and fields. Its instructions require source URLs for claims and ask it to show contradictory sources side by side. The command prints the question, connection status, tool name, a result preview, and the final answer. Query arguments are not printed. Errors are written to stderr.

Stack

  • Node.js with ES modules
  • Gemini model gemini-3.5-flash-lite
  • Sanity project uyvc8si1, dataset production
  • GROQ for Sanity queries

Setup

Run these commands from the agent directory:

npm install

Create an .env file in the agent directory with both keys:

GEMINI_API_KEY=your-gemini-api-key
SANITY_API_TOKEN=your-sanity-api-token

The Sanity token must have… (details omitted in source)

How I Used Sanity

Structured Content Model

I defined a single document type in Sanity Studio using TypeScript:

export const article = defineType({
  name: 'article',
  title: 'Article',
  type: 'document',
  fields: [
    defineField({ name: 'title', title: 'Title', type: 'string' }),
    defineField({
      name: 'slug',
      title: 'Slug',
      type: 'slug',
      options: { source: 'title' }
    }),
    defineField({ name: 'body', title: 'Body', type: 'text' }),
    defineField({ name: 'source', title: 'Source', type: 'url' }),
  ],
})

I populated it with three documents that set up a deliberate contradiction:

Title Source
Nebula Core Rulebook; The Setup https://example.com/nebula-rulebook
Nebula Official Errata https://example.com/nebula-errata
How to Win at Nebula https://example.com/nebula-strategy

The source URLs are placeholder examples. In a real deployment, this field would point to the actual publisher page.

How the Agent Reads and Queries Content

The agent uses function calling with Gemini. It has one primary tool, coded in JavaScript:

{
  name: "query_documents",
  description: "Run a GROQ query against the Sanity dataset. Returns JSON.",
  parameters: {
    type: "object",
    properties: {
      query: { type: "string", description: "A GROQ query string." }
    },
    required: ["query"]
  }
}

Flow when the user asks a question:

  1. The LLM writes a GROQ query itself (example: *[_type == "article"]{title, body, source})
  2. My code executes it via @sanity/client against project uyvc8sil, dataset production
  3. The JSON result is fed back into the LLM's context
  4. The LLM reads the content and answers with citations

System prompt instructions:

"When two sources contradict each other, show both claims side by side with their sources. Cite the source URL for every claim. Never invent information."

What the Agent Does with Retrieved Content

Because the content is structured, the agent can:

  • Compare specific fields across documents - it reads body from the rulebook and body from the errata and notices the numeric values differ.
  • Attribute each claim to its source URL - the citations aren't guessed, they're data.
  • Distinguish types of documents - the errata is authoritative over the rulebook because its title says so.

A plain-text search would have surfaced both documents but had no way to reason about which was newer, more authoritative, or even that they contradicted each other.

Design Decisions That Mattered

Schema-Informed Prompting

The LLM initially guessed type names like card and document, returning empty results. I injected a schema description into the system prompt so it knows to query *[_type == "article"]. This small detail dramatically improved reliability.

Sanity as the Source of Truth

No content is hardcoded in the agent. Every fact the agent reports comes from a live GROQ query against the Content Lake. If I update an article in the Studio, the next question gets the updated answer.

useCdn(false)

I disabled the CDN for the agent so responses are always fresh from the Content Lake and always 100% real-time. This matters for an agent where "the latest errata" is the whole point.

Sanity Project Details

  • Project ID: uyvc8sil
  • Dataset: production
  • Document type: article (fields: title, slug, body, source)

What I Learned

I learned a lot because this is my first time hearing about Sanity. I learned how to make the case for structured content viscerally clear. This provided me an opportunity that an LLM can reason about provenance when we have fields like title, body, and source defined in a schema. This led me to understand the difference between these statements:

  • "Here are two documents"
  • "These two sources disagree, and here's the URL for each claim"

The hardest part wasn't the LLM logic - it was teaching the agent the schema. Once I stopped trying to make the model guess type names and instead told it what was in the Content Lake, everything clicked.

Thanks

Thanks to Sanity and DEV for the challenge. It pushed me to build something I'd been wanting to try: an agent that treats structured content as the foundation, not an afterthought.

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.