Structural Admission: verify a sequential task’s claimed dependency structure before interpreting learning [R]

When we train agents on staged or multi-phase environments, it is tempting to interpret learning curves, transfer, or apparent “emergence” as evidence for a particular causal or informational structure. But has that structure actually been verified under the same observation and action interface seen by the learner?

Structural Admission is a small, standard-library-only Python harness for testing that question before training begins. Researchers implement a task adapter and a separate scripted oracle; the core calibration, rollout, validation, reporting, and reproduction logic remains unchanged.

Enforced Properties

• Calibration seeds disjoint from task-rollout seeds
• A CMI threshold fixed from synthetic calibration before candidate evaluation
• Evaluation under both uniform-random and scripted-oracle policies
• Oracle access limited by the learner-facing observation contract
• CI measurement restricted to preregistered phases
• Paired environment noise and random-policy draws across conditions
• Pre-disclosure leakage checks on declared observable field groups
• Raw trajectory storage before aggregation
• Balanced policies, conditions, seeds, and sample cardinalities
• Immutable formal output directories
• Content-hashed reports and byte-level reproduction of deterministic artifacts

Output and Example

The tool reports Admitted, Rejected, or Inconclusive. Failures are preserved rather than tuned away. One motivating case involved a relation intended to be non-operative. Its measured conditional mutual information was 0.07181 bits, above a previously calibrated threshold of 0.05902 bits. The task was rejected before learning experiments were interpreted, and the residual dependency had to be diagnosed structurally.

Meaning of Admission

Admission has deliberately narrow meaning: the configured task passed its preregistered operational checks. It does not prove a theory, guarantee learnability under a particular optimizer, establish causal identification outside the task model, or imply external validity.

Installation and Usage

git clone https://github.com/btisler-DS/structural-admission.git
cd structural-admission
git checkout v0.1.0
python -m pip install .

Create and run an external task:

structural-admission init my-task
cd my-task
structural-admission run config.json --output runs/reference
structural-admission reproduce runs/reference --output reproduction/reference

The adapter contract is intentionally small, and task implementations do not modify package internals.

Repository and Feedback

Repository: https://github.com/btisler-DS/structural-admission

I’d especially welcome criticism of the statistical procedure, leakage model, adapter boundary, and what should-or should not-count as structural admission.