How to Verify a 'Trained-From-Scratch' LLM in 2026: A Provenance and Fingerprinting Guide

TL;DR

Can an outsider verify that an LLM was "trained from scratch"? In 2026, largely yes. Using only public Hugging Face artifacts - config.json, tokenizer.json, and embedding weights - you can fingerprint a model's lineage and place it on a spectrum from fully original to a straight port of a foreign base. This guide explains the method, what it can and cannot prove, and the open-weight debate that made it a 2026 talking point.

Key takeaways

• Three public signals - architecture config, tokenizer overlap, and embedding-space similarity (Linear CKA) - are enough to estimate whether a model was built from scratch or derived from an open-weight base.
• One matching field is noise; five at once is a fingerprint. Provenance is a preponderance-of-evidence judgment, not a single test.
• Fingerprinting reveals lineage, not intent. Building on an open-weight base is a legitimate, industry-standard practice - the interesting output is the label, not an accusation.
• The topic went mainstream in 2026 because several "self-developed, DeepSeek-class" claims from non-US/China labs were publicly stress-tested and, in places, found more derivative than advertised.

Why this matters in 2026

Every few weeks a lab announces a "from-scratch, self-developed" foundation model. In mid-2026 that claim stopped being taken on faith. A Zhihu roundtable on the summer model wave and a widely-cited Zhihu thread with millions of views became the venue where the Chinese-language community tore into "self-developed" claims from Korean and Indian labs - concluding, in several cases, that the models leaned harder on Chinese open-weight bases than the marketing implied (coverage).

The useful development is that the argument moved from vibes to a reproducible procedure. Public tools now let anyone run the check. This is a refreshed 2026 walkthrough of that procedure - updating the original method write-up with the debate context and the links you need.

The three fingerprint signals

Model provenance rests on three axes you can read straight from a model's public files. No leaked internals required.

1. Architecture fingerprint (config.json)

Compare the structural fields a lab chooses when it designs a model: model_type, vocab_size, hidden_size, intermediate_size, num_hidden_layers, and the attention head counts. Independently designed models rarely line up on all of these. A single coincidental field means nothing; five simultaneously is a fingerprint of a shared skeleton.

2. Tokenizer overlap (tokenizer.json)

Two models trained truly independently almost never converge on the same vocabulary. Measure the ratio of shared tokens against the smaller vocabulary. High overlap is a strong hint of shared ancestry - but, as the traps below explain, not a verdict on its own.

3. Embedding similarity via Linear CKA

Centered Kernel Alignment (CKA) compares representation geometry in a way that is robust to rotations and rescalings - the superficial transformations that fool cruder weight comparisons. Run on embedding layers, it reliably flags from-scratch training. Its honest limitation: it struggles to cleanly separate continued-pretraining derivatives, where a model keeps a base's weights but trains heavily on top.

The genotype framework

One public framework, Model Genome Korea, collapses the three signals into four labels - a clean mental model for reading any provenance result:

Two traps that produce confident wrong answers

Trap 1 - Row-wise cosine similarity looks rigorous but isn't.

Comparing weight matrices row by row with cosine similarity feels quantitative, yet it is fooled by rotation invariance: a genuinely derived model can be rotated to look "different," and a naive check clears it. This is exactly why the method leans on rotation-robust CKA and on structural config signals rather than raw weight-row comparisons.

Trap 2 - A shared tokenizer proves nothing by itself.

Reusing a tokenizer is frequently a licensing or convenience decision, not evidence of weight-level derivation. Treat tokenizer overlap as a supporting signal, never a conclusion.

What fingerprinting cannot tell you

• Lineage, not intent. It can show that model B shares structure with model A. It cannot tell you whether that was disclosed, licensed, or hidden - that is a question of ethics and paperwork, not linear algebra.
• "From scratch" is a spectrum, not a boolean. Data, initialization, architecture, and post-training sit on a continuum of originality; a single label flattens a gradient.
• Building on open weights is legitimate. The point of a provenance tool is transparency and accurate labeling, not accusation. Most of the industry stands on open-weight shoulders - the honest move is to say so.

FAQ

Can you tell if an LLM was really trained from scratch?

In most cases, yes - to a strong probability. Cross-check architecture config, tokenizer overlap, and embedding CKA against candidate base models; agreement across all three indicates derivation, while divergence supports a from-scratch claim.

What is LLM fingerprinting / model provenance?

It is the practice of estimating a model's origin - original vs. derived from an existing open-weight base - using only its public artifacts (config, tokenizer, weights), without access to training data or internal logs.

Is building a model on Llama, Qwen, or DeepSeek legitimate?

Yes. Fine-tuning or continued-pretraining an open-weight base is standard, licensed (subject to each license), industry practice. Provenance tools report lineage; they do not allege wrongdoing.

How do you distinguish a fine-tuned model from a from-scratch one?

From-scratch models diverge on architecture and tokenizer and show low embedding CKA to any base. Fine-tuned or continued-pretraining derivatives keep the base's skeleton and tokenizer and retain high embedding similarity - the hardest case, where CKA alone is weakest and config/tokenizer evidence matters most.

Does a shared tokenizer mean a model is derived?

No. It is a supporting signal only. Confirm with architecture and weight evidence before drawing a conclusion.

Why Linear CKA instead of cosine similarity?

Because CKA is robust to rotation and scaling, the transformations that let a derived model masquerade as "different" under naive cosine comparisons.

Resources & related links

• Reproducible method write-up: How to Tell If an LLM Was Really Trained From Scratch
• Deep-dive article: Model DNA - architecture & weight lineage
• Live tool: Model Genome Korea
• Announcement post: Architecture lineage of Korea's sovereign-AI models
• The debate: Zhihu roundtable "大模型卷一夏" · Zhihu thread · media coverage (coverage)