If the Model Can Write a Better Sentence, Whose Sentence Is It?

One might argue that the most disquieting thing about language models is not that they are fluent. It is that fluency was supposed to be evidence of an interior.

For two centuries, in the Western philosophical tradition, good writing functioned as a proxy for a rich inner life. Style was thought to be the residue of a self. You could tell there was someone in there because the sentences had a particularity to them, a signature.

We built an entire theory of education on this. We teach people to write well because we believe the process of doing so is what makes them into someone.

Now a machine produces a well-formed paragraph on request. Prompt it badly and it gives you something a competent undergraduate could have written. Prompt it well and it gives you something better.

The proxy is broken. Whatever fluency was evidence of, it is no longer that.

The question is not whether it thinks

This is where most of the public conversation gets stuck, and it is an unproductive place to be stuck. The question "is it conscious" is unresolvable in practice, and the way it is usually framed invites both an overconfident yes and an overconfident no.

The more tractable question is this: what did we think writing was, and what do we do now that we were probably wrong?

If fluency can be generated without interiority, then the vice versa may also hold: there may be interiority that is not fluent. And in fact we know this to be true. We knew it before the models. The person who cannot articulate their grief is not therefore grieving less.

We simply had no instrument for measuring it and so we inferred from the articulate ones and called the inference wisdom. A machine has now told us, somewhat rudely, that our wisdom was a heuristic.

Meaning is not a property of the text

One might put it this way. Meaning is not in the sentence. Meaning is in the relationship between the sentence and a life.

Read the line "she did not turn around" in a novel and it can be devastating, because you have spent two hundred pages with someone who wanted her to turn around. Read the same line with no context and it is a description of a person facing forward.

The model produces lines. It has read so much that it can produce lines that, in the right context, would devastate. But it is not producing the context. The context is the reader's history, and the writer's, and the shared assumption that the withholding matters.

This is why AI-generated fiction frequently feels like a beautifully furnished apartment with nobody home. The furniture is real. All the objects are where they should be. The absence is not a defect in the objects.

What is left for us

I do not have a tidy answer to what human value is worth in the machine age, and I distrust any essay that claims to. But I can say what the models have clarified by taking away.

They have clarified that a great deal of what we called thinking was retrieval. That is uncomfortable but it is also liberating: if most of the intellectual labour we were proud of was pattern completion, then the parts that remain are the parts that were always harder to name.

• Judgment.
• The decision of what matters.
• The willingness to be wrong in public.

They have clarified that originality is not novelty of surface. A model can produce a surface no human has produced before. What it cannot do is care that the surface should be that one and not another.

And they have clarified, at least for me, that the human thing we need most now is not more production. It is selection. There is already too much to read. The scarce role is the person who can say this one, not that one, and here is why.

The uncomfortable part

The uncomfortable part is that this may not be enough. Selection is a real skill and it may not support the number of people currently employed to produce. I am not going to pretend otherwise.

There is a version of the future in which the machine is a good writer, most people are not needed to write, and the transition is ugly for everyone who trained for it. That version is plausible. Anyone who tells you it definitely will not happen is selling something.

But there is another version, and I think it is at least as likely. In it, the collapse of the cost of production makes the cost of attention the only thing that matters, and attention cannot be manufactured at scale, only earned.

In that world the people who matter are the ones who can be trusted, which was always the thing we actually valued and mistook for fluency.

The road was always paved.