We put an AI helper in our course and spent weeks teaching it to say I don't know

The easy part was making it talk

We wanted a helper inside our course that could answer student questions about the material. Getting it to answer was a weekend. You wire up a model, feed it the lesson content, students ask things, it responds. Demos great. Everyone claps.

Then a student asked it something we had never covered, and it made up an answer. Confident, fluent, wrong. It invented a feature that does not exist and told the student to go use it.

That was the moment the real project started, and the real project had almost nothing to do with generating text. It was about the opposite: teaching the thing to stop.

Most of the weeks after launch went into three problems:

• Scoping what it was allowed to know.
• Getting it to refuse instead of invent.
• Knowing when to get out of the way and hand a student to a person.

I want to walk through each, because the honest version of building an AI helper is mostly this, not the demo.

Scoping what it knows

Our first instinct was to give the model everything. All the lessons, all the notes, every old thread. More context, better answers, right. Wrong, mostly.

When the knowledge was one big blob, the model treated a throwaway comment in an old draft with the same confidence as a core lesson. It could not tell canon from noise. So we cut it down. The helper only sees a defined set of source material, each chunk tagged with where it came from and how authoritative it is. A published lesson outranks a forum reply. A forum reply outranks nothing, because we stopped feeding it random forum replies.

We also made the boundary explicit in the system prompt. The helper is told, in plain terms, what subjects it covers and that everything else is out of scope. That sounds obvious, but before we wrote it down the model assumed its scope was "anything a helpful assistant could discuss," which is the whole internet. Naming the fence mattered more than any retrieval trick we tried.

The other thing scoping bought us was traceability. Because every answer is built from tagged chunks, we can look at a bad answer and see exactly which source led it astray. Half of our fixes were not prompt changes at all. They were us finding one stale document and deleting it.

Teaching it to refuse

Getting a model to refuse is weirdly hard, because the whole training gradient pushes it to be helpful. Ask it something outside its knowledge and its instinct is to help anyway, which means guess. We attacked this from a few sides.

In the prompt we gave it permission to not know, in strong language, with examples of good refusals. A refusal in our system is not a dead end. It says what it cannot answer, and it points the student somewhere real, usually a human. Giving the model a concrete "I do not have that, here is what to do next" template moved the needle more than telling it "do not hallucinate," which models mostly ignore.

Then we tested it like software, which is the part I would repeat on any project like this. We built a set of questions we know are out of scope, plus a set of near misses, questions that sound in scope but are not. Every change to the prompt or the sources runs against that set. If the refusal rate on the out of scope questions drops, the change does not ship. Before we had that test, every