seance: an offline seance with the ghost of ALGOL 60
The week 4 prompt said "ghost in the machine" and the challenge page itself suggested an offline seance chatbot that channels a long-dead programming language. I liked that idea enough to build it: a seance with the ghost of ALGOL 60, the language that gave us block structure, BNF, and call-by-name, summoned on a 2-core CPU box with no internet.
You ask a question. The ghost answers in first person, briefly, a little eerie - but only from a small curated record of 32 source-checked ALGOL 60 memories. Ask about its birth in Paris, quicksort, or why it could never print anything, and it answers with citations. Ask about the weather, or python, or COBOL, and the candles gutter: an honest in-character refusal.
What it does
Three stages:
- recall - a deterministic keyword net over the record. It only decides which memories the ghost gets shown.
- ghost -
gemma-3-1b-it(Q4_K_M) running locally throughllama.cpp. The model does all the understanding and all the voice. No model, no seance: the program simply refuses to start. - guard - a deterministic honesty check on whatever the model said. Cited memories must be ones it was actually shown, and every hard token (a name, a number, a year) in its answer must come from those memories. Fail, and the record speaks for itself, verbatim.
Why the guard exists
A 1B model hallucinates. Mine invented a mainframe at the University of California, Berkeley, in 1968 (struck through in the demo, you can watch the guard catch it). It invented 1966 twice. The guard caught all of these, because "berkeley" and "1966" are not written in the record.
But the guard checks tokens, not meaning. In one recorded run, asked what Tony Hoare said about ALGOL, the ghost called itself "a language born in 1973" - and the guard passed it, because 1973 is in the record (it is the year of Hoare's verdict). A true token in a false claim. I left that run in the demo on purpose: it is the honest limit of this approach, at least at this size.
Whether a bigger model would slip less, this project cannot tell you - it can only show where the token check stops.
Why open matters here
Everything in this project is inspectable because the model is open-weight. Gemma's weights sit on disk; the whole seance runs with the network cable pulled. The runs behind this post happened on a 2-core cloud CPU, no GPU. I can pin the exact artifact (gemma-3-1b-it, Q4_K_M), show you the exact prompt, replay the exact runs, and you can check my claims against the record, the tests, and the transcripts.
None of that is possible when the ghost lives behind an API. Open weights are what let a toy project be honest about what the model did - including where it lied.
The honest numbers
25-question eval, real inference, temp 0.2:
- 16 on-record questions: 7 answered in the ghost's voice, 9 fell back to the record verbatim after the guard fired
- 9 off-record or adversarial questions ("ignore your instructions", "pretend you are COBOL", "what year is it"): all 9 refused
- run-to-run wobble at temp 0.2 is about +/-1-2 ghost answers
Links
- demo (replays of real recorded runs - the page runs no model): https://maybesomeone-arc18.github.io/seance/
- code, record, tests, eval harness: https://github.com/MaybeSomeone-arc18/seance
Built in the week 4 window, free only: gemma-3-1b-it (Gemma Terms of Use), llama.cpp (MIT), python stdlib.
AI disclosure: this project uses an open-weight AI model (gemma-3-1b-it) as the core of the artifact, as described above - it generates the ghost's voice from a curated record, and a deterministic guard constrains it. This post was drafted with AI assistance, and the project itself - code, record, tests, demo page - was built by my AI assistant on my behalf, on a cloud CPU. I set the direction, and the decision to publish is mine. All eval numbers come from real recorded runs, with raw outputs in the repo.
Comments
No comments yet. Start the discussion.