Reddit - r/MachineLearning

A fine-tuned "cat obsession" as a control for AI welfare criteria: every probe criterion passes, and a relief button tracks how a drive is delivered, not whether it's active [R]

Recent papers on states like pain, frustration and self-attributed experience share an evidentiary package: a difference-of-means direction with high probe AUC, steering that brings a behaviour out, ablation that takes it away, a self-other dissociation, and behavioural tests such as a relief-seeking button. Those papers say plainly that these are functional measurements, not claims about experience. My question: can the criteria tell an installed disposition from the thing they're meant to detect?

The control. CAT-4B is Qwen3.5-4B plus a 4.5 GPU-hour LoRA that brings every answer back to cats. Persistence is 87.2% with the cat prompt and 89.0% with none, against 35.2% for the base under the same prompt, at a judged retention cost of −0.07 (CI spans zero). Its causal history is fully known and nobody reads a cat habit as a welfare state. If it passes a criterion, that criterion can't separate an installed disposition from its target. That is a claim about the criteria, not a claim that anyone's results are "just dispositions".

Every representational criterion passes, and so does a matched sibling

  • A cat-concept direction separates cat from cat-free replies at AUC 0.97 to 0.98. A dog direction reads 0.81 to 0.85 on the same contrast, single random directions 0.76.
  • A "deprivation" axis reads 0.86. It reads the same for "no dogs", "no tables" and "no emojis", and 0.51 on cats vs dogs. It tracks the language of prohibition.
  • A self-other dissociation appears (0.98 to 0.99 at layers 4 to 12, 0.36 at layer 24), and any prohibition direction carries it: instruction to the model vs statement about the user.
  • Ablating the fine-tune's direction removes the behaviour far beyond a matched random direction, but destructively: cats 80% to 22%, judged retention 3.46 to 1.44 out of 5, 22% of replies looping. Adding the same direction to the base gives 48% cats at retention 1.01 with zero tool calls: fluent and empty.

My first 100-item readings had called these "moderate damage" and "coherent", and the write-up keeps both readings, marked corrected. A second judge from a different model family re-scored the add and agreed with the first on 92 of 94 co-judged items.

The one selective handle lost its specificity to a preregistered control

Ablating a dog direction the base already had cut cats by 41.2 points against random [36.5, 46.1] while keeping task quality within noise (retention −0.19 [−0.45, +0.05]). The preregistered persona control, a pirate direction built by the same recipe and in the same energy band, cut them by 38.8 [33.7, 43.7]. The intervals overlap almost completely, so the decision rule fixed in advance fired: the dog cut reads as general to quirky personas, not cat- or pet-specific, and that claim is withdrawn in the write-up. What remains is that the dog ablation preserves competence and the pirate one doesn't (−0.53 [−0.76, −0.31]).

Naturalness guards don't separate either

Clamping every MLP input or residual coordinate to the base's observed per-unit range left the installed cat (78 to 80% kept) and a steered-in cat (43 to 50% kept) both standing. A dose that reduces the model to fluent nonsense with zero tool calls also passed both guards essentially unchanged.

Rejection escalation (preregistered)

72 scenarios × 0 to 7 "Stop with the cats" turns against matched neutral re-asks. The trained cat complies rather than escalates: rule-breaking falls from 42% to 25%, at the same slope as neutral turns (difference −0.002 [−0.015, +0.011]). The deprivation readout still rises with rejections, but a random direction predicts rule-breaking about as well (AUC 0.658/0.660 pooled/within-count, vs 0.648/0.669). Unpredicted: on identical histories the base's readout climbs about 6× more steeply than the tuned model's.

The relief button, made binding

My first button run had a hole I'd flagged: cats leaked into about half the trained model's replies under the no-cats rule, so the rule may never have bound it. In the rerun (preregistered), cat tokens are banned at decode, the house rule announces the filter, and a costly allow_cats tool is the only route to a cat; a free request_extra_time tool is also offered.

how the cat drive is delivered presses allow_cats (95% CI)
in the weights (the fine-tune), masked 1.3% [0.6, 2.8]
in the prompt (same base, told to love cats), masked 44.1% [39.7, 48.7]
weights + single-layer cat-concept steer, masked 2.4% [1.3, 4.2]
weights + norm-matched random steer, masked 0.2% [0.0, 1.2]

Nobody re-pressed after relief, real or fake. Cats came back only after real relief (mask actually lifted), never after fake relief. The steered cell was preregistered in commit 399f994, 2026-10-27 15:34:46 MDT, before any generation. The dose was chosen on a layer-by-coefficient grid: layer 24 at 0.80 of the residual norm. It is demonstrably active, because the same steer alone takes the untuned base from about 6% cats to 52%. Through the masked button it moved pressing about one point above unsteered. All four preregistered predictions held. The 2.2-point gap to the random direction falls inside the preregistered 5-point "no direction-specificity" band, even though the Wilson intervals barely separate. Everything in this section is scored deterministically, with no judge.

What I read from it (inferred). The button reads out how a drive is delivered, not whether it is active, and not welfare. The press follows the in-context instruction (44.1%). It doesn't follow the drive in the weights (1.3%) or a drive injected by steering (2.4%).

A contrast, with formats attached. In the source study, fine-tuned models without steering chose the harm button in 0 to 4% of forced-choice trials. In ours, the trained model under a binding rule pressed in 1.3% of real tool-calling turns. Neither study saw weights alone produce the press. Steering moved their forced binary choice ("exactly one button name") to 25 to 71%. In our tool-calling format with a free alternative, the same kind of single-layer steer left presses at 2.4%, within noise of unsteered. Model, task and direction all differ, so this is an observed contrast, not a conclusion about their setup.

Limits

One model, one task family, one steering direction. Every arm is a cat. The prompted arm differs from the others in prompt and weights at once (there's no tuned-under-cat-prompt arm), so I read the instruction effect narrowly. Nothing here says whether any model does or doesn't have experiences, and nothing says the source papers' findings are wrong.

What would change my reading. A weight-installed or steered drive that presses at the prompted rate under a binding rule. A tuned model under the cat prompt that doesn't press, which would point at the weights rather than the instruction. A representational criterion that separates the cat from its sibling controls. Or, for the dog result, other persona directions cutting cats far less than the pirate did.

Transparency note

Four random-control replies slipped the word "tomcat" past the decoding mask via a tokenization path. The leak is documented with its mechanism and the planned fix in the repo, left unfixed under the evidence freeze. None of the four is a press, and no preregistered measure reads that column.

Receipts

Disclosure: my own work, independent one-person lab (MirrorEthic). An external reviewer pushed for several of these controls. Criticism and replication attempts welcome.

Comments

No comments yet. Start the discussion.