LLM Wiki Maintenance: Drift, Contradictions and Review

Introduction

An LLM Wiki fails when old facts remain plausible, contradictions become polished, and generated summaries drift from their sources. Maintenance is the real product of any compiled knowledge system. Creating wiki pages is straightforward compared with keeping them trustworthy across months of ingest, edits, rewrites, and new sources.

This article covers the operational side of LLM Wiki systems: drift detection, contradiction checks, citation discipline, linting, Git review, and maintenance workflows. It assumes you already understand the basic pattern described in LLM Wiki - Compiled Knowledge That RAG Cannot Replace: raw sources are compiled into durable Markdown pages that humans and agents can query later.

The calm but opinionated view is that an LLM Wiki without maintenance is just a nicer-looking knowledge graveyard.

What Makes LLM Wiki Maintenance Different

Traditional wikis rot because people stop updating them. RAG systems drift because the corpus changes, chunks get stale, metadata is weak, and retrieval keeps finding plausible but outdated fragments. The deeper reason is that retrieval and representation solve different problems: retrieval can be re-run against fresh data, but a representation you already compiled has to be actively kept honest.

An LLM Wiki has a different failure mode. It can look clean even when it is wrong. The pages may be well formatted. The links may work. The summaries may sound balanced. But underneath that neat surface, the system may have dropped critical facts, merged incompatible concepts, cited summaries instead of sources, or preserved an old decision as if it still applied. That is why LLM Wiki maintenance must check both structure and meaning.

The Maintenance Goal

The goal is not to make every page perfect. That is too expensive, and it usually leads to abandoned systems. The goal is to keep the wiki useful, inspectable, and recoverable. A maintained LLM Wiki should make it easy to answer these questions:

• What sources support this claim?
• When was this page last reviewed?
• Has this concept changed?
• Are there conflicting pages?
• Is this summary still current?
• Did the agent rewrite more than it should?
• Can we roll back a bad update?
• Can a human understand why the page says what it says?

If the answer is no, the problem is not only content quality. It is system design.

A Simple Maintenance Loop

A useful LLM Wiki needs a repeatable loop. The loop should be simple enough to run often and strict enough to catch drift before it becomes invisible.

flowchart TD
A[Add or update source] --> B[Compile into wiki pages]
B --> C[Update links and indexes]
C --> D[Run structural lint checks]
D --> E[Run semantic review checks]
E --> F[Human review of risky changes]
F --> G[Commit approved changes]
G --> H[Schedule stale-page review]
H --> A

This loop is not glamorous. That is the point. A knowledge system becomes durable through boring maintenance: source preservation, explicit review, predictable structure, and small safe updates. The same instinct drives evergreen notes: a note or a wiki page only compounds in value if someone keeps refining it instead of leaving it to rot after the first draft.

Types of Drift in an LLM Wiki

Knowledge drift is not one thing. Different drift types require different checks. A good maintenance system should name them clearly.

Source Drift

Source drift happens when the underlying source material changes. For example, a tool releases a new version, a policy is updated, an API changes, or a vendor deprecates an old feature. The old wiki page may still be accurate for the previous version, but wrong for current use. Source drift is dangerous because the old claim may still be true in historical context. The problem is not that the claim is fake. The problem is that it no longer answers the current question.

Maintenance response:

• Record source dates
• Record last reviewed dates
• Mark version-specific pages clearly
• Link old pages to superseding pages
• Avoid mixing old and new versions without labels

Concept Drift

Concept drift happens when the meaning of a term changes over time. This is common in AI and software architecture. Terms like “agent”, “memory”, “RAG”, “workflow”, “tool use”, “structured output”, and “context engineering” can shift meaning quickly. A wiki can accidentally preserve several meanings of the same term without explaining the difference. That creates confusing pages that sound coherent but combine incompatible ideas.

Maintenance response:

• Maintain glossary pages
• Add “meaning in this wiki” sections
• Separate overloaded concepts into distinct pages
• Link related meanings explicitly
• Avoid letting the model silently merge terms

Terminology Drift

Terminology drift is smaller than concept drift but still harmful. It happens when the wiki uses multiple names for the same thing: “LLM Wiki”, “compiled knowledge base”, “AI-maintained wiki”, “agent-maintained wiki”, and “Markdown knowledge base”. Some variation is fine. Too much variation breaks search, linking, and review.

Maintenance response:

• Define canonical page names
• Keep aliases in front matter or page metadata
• Redirect duplicate pages
• Lint for near-duplicate titles
• Use consistent anchor text in internal links

Decision Drift

Decision drift happens when a past decision remains documented but no longer reflects current practice. For example, a page might say that the project uses vector RAG for all document search, while newer pages describe an LLM Wiki workflow. Both statements may be historically true, but the wiki must show which one is current. This matters for architecture notes, engineering processes, content strategy, and internal tooling.

The same discipline applies to decision records in AI-driven development: a decision page is only trustworthy if superseded choices are marked as superseded rather than silently overwritten.

Maintenance response:

• Mark decisions as proposed, accepted, superseded, or rejected
• Keep decision dates
• Link superseded decisions to replacements
• Preserve historical context
• Avoid deleting old decisions without trace

Citation Drift

Citation drift happens when a page cites a source, but the claim no longer matches what the source says. This can happen after a rewrite. The citation remains in place, but the sentence around it changes. The page still looks sourced, yet the citation no longer supports the claim. This is one of the most serious LLM Wiki failure modes because it creates false confidence.

Maintenance response:

• Check claim-level citation support
• Avoid citing only summaries
• Keep raw sources
• Require citations for important claims
• Flag paragraphs with sources but no direct support
• Review citation changes in Git diffs

Structure Drift

Structure drift happens when the wiki slowly loses navigability. New pages are created instead of updating old ones. Index pages fall behind. Duplicate pages appear. Orphan pages accumulate. Related pages stop linking to each other. The wiki still contains useful knowledge, but finding and trusting it becomes harder.

Maintenance response:

• Lint for orphan pages
• Maintain index pages
• Detect duplicate topics
• Require backlinks for canonical pages
• Archive rather than scatter
• Keep folder roles clear

The Core Maintenance Files

An LLM Wiki should not rely on memory, vibes, or repeated prompting. It needs operating files that tell humans and agents how to maintain it. A practical structure can look like this:

llm-wiki/
  raw/
    sources/
    transcripts/
    documents/
  wiki/
    index.md
    concepts/
    entities/
    projects/
    decisions/
  maintenance/
    review-log.md
    lint-report.md
    stale-pages.md
    contradiction-report.md
  rules/
    AGENTS.md
    schema.md
    style-guide.md
    citation-policy.md
    source-policy.md

The exact names do not matter. The roles do. The wiki needs source storage, compiled pages, review artifacts, and rules that survive across sessions.

AGENTS.md

AGENTS.md should explain how the agent is allowed to work. It should answer questions like:

• Should the agent update existing pages before creating new ones?
• When should it ask for review?
• How should it cite sources?
• Can it reorganize folders?
• Can it rewrite old pages?
• What should it do with contradictions?
• What should it never delete?

A useful rule is: Prefer small, source-backed updates over broad rewrites. That one sentence prevents many bad maintenance habits. It also draws a useful boundary relative to agent memory: memory shapes how an agent behaves in the moment, while the wiki’s rules file shapes what the agent is allowed to change in the shared knowledge base.

schema.md

schema.md should define the page structure. For example:

# Page Title

## Summary
Short current summary.

## Key Claims
- Claim with source reference.
- Claim with source reference.

## Current Status
Current, historical, superseded, draft, or uncertain.

## Details
Main compiled explanation.

## Related Pages
Internal links.

## Sources
Raw source references.

## Review Notes
Last reviewed date and reviewer.

A schema does not need to be rigid. But without a schema, the wiki becomes a pile of inconsistent essays. The schema gives the agent something to preserve.

citation-policy.md

The citation policy should say what requires a source. At minimum, require citations for:

• Technical claims
• Comparisons
• Product behavior
• Version-specific statements
• Benchmark results
• Pricing
• Legal or compliance claims
• Current status claims
• Claims copied or derived from a source

A good citation policy also says what not to do:

• Do not cite a compiled page as if it were the original source.
• Do not attach citations to paragraphs they do not support.
• Do not keep a citation after rewriting a claim unless the source still supports it.

Citation discipline is the difference between a useful LLM Wiki and decorative confidence.

source-policy.md

The source policy should define how raw sources are stored. For example:

• Keep original files when possible
• Preserve URLs and access dates
• Store copied notes separately from generated summaries
• Record source type
• Mark low-confidence sources
• Avoid overwriting raw material
• Keep superseded sources when they explain historical decisions

This policy protects the wiki from a common mistake: summarizing a source, discarding the original, and then treating the summary as evidence. That is not knowledge management. That is lossy compression.

What an LLM Wiki Linter Should Check

A linter should check structure first. Structural problems are easier to automate and often reveal deeper semantic problems. Useful structural checks include:

• Broken internal links
• Orphan pages
• Duplicate titles
• Missing source sections
• Missing review dates
• Pages without backlinks
• Empty or placeholder sections
• Very long pages without section structure
• Pages with no incoming links from an index
• Inconsistent naming conventions
• Invalid front matter
• Stale “current” pages older than a review threshold

These checks do not prove the knowledge is true. They prove the wiki is still maintainable. That is a necessary starting point.

Semantic Checks

Semantic checks are harder but more valuable. Useful semantic checks include:

• Claims without direct source support
• Two pages making incompatible claims
• Old decisions presented as current
• Duplicate concepts with different names
• One concept page mixing several meanings
• Summaries that omit known constraints
• “Best” or “recommended” claims without criteria
• Version-specific claims without version labels
• Pages that contradict newer sources

These checks should not automatically rewrite the wiki. They should usually produce a report for review. The safer pattern is: Detect automatically. Explain clearly. Update deliberately. Review risky changes.

Contradiction Detection

Contradiction detection is not just asking an LLM whether two pages contradict each other. That can help, but it is too vague. A better approach is to compare claims. A contradiction workflow can look like this:

flowchart TD
A[Extract claims from page] --> B[Find related pages and sources]
B --> C[Extract claims from related material]
C --> D[Group claims by subject]
D --> E[Compare status, date, version, and scope]
E --> F{Conflict found?}
F -->|No| G[No action]
F -->|Yes| H[Classify conflict]
H --> I[Create contradiction report]
I --> J[Human or agent-assisted resolution]

This works better because many apparent contradictions are not real contradictions. One page may describe version 1.0 and another version 2.0. One page may describe personal use and another enterprise use. One page may describe the design goal and another the implementation reality.

The contradiction report should include:

• Conflicting claims
• Pages involved
• Source references
• Date or version context
• Likely explanation
• Suggested resolution
• Whether human review is required

Do not let the agent silently resolve contradictions by blending both claims into a vague compromise. That creates smooth nonsense.

How to Resolve Contradictions

There are several valid ways to resolve a contradiction.

• If one claim is outdated, mark it as superseded and link to the newer page.
• If both claims are true in different contexts, split the context clearly.
• If the sources disagree, preserve the disagreement and explain it.
• If the wiki page invented or overgeneralized a claim, remove or narrow the claim.
• If the contradiction reflects a real unresolved decision, create or update a decision page.

The goal is not to remove all tension. Some tension is useful. A good wiki shows where knowledge is settled and where it is still uncertain.

Stale Page Review

Every compiled page should have a review status. A simple metadata block can be enough:

status: current
last_reviewed: 2026-07-10
review_after: 2026-10-10
source_confidence: medium

For fast-moving topics, use short review windows. For stable concepts, use longer windows.