Show HN: Lumabri - What if LLMs worked like Napster?

Run huge mixture-of-experts models from a swarm of peers, with the colibri engine. Pure C, no dependencies. One machine shares a model. Any other machine chats with it: nothing is downloaded up front, the bytes an inference actually touches arrive from the peer on first use and stay in a local mirror. The second question is served from local disk at full speed. The engine binary is unmodified. The founding principle: any machine may join, GPU or not. The engine was built for CPU and SSD first; a GPU only makes it faster, never different, and the output is byte-identical either way. A swarm with no GPU at all is a working swarm. Networks that pool GPUs recruit from the few; lumabri recruits from everyone. Build: make On the machine that has a model (any colibri model directory): ./lumabri serve --model /path/to/model On the machine that wants to chat (needs a colibri build for the engine): ./lumabri chat --tracker :7300 --engines-dir /path/to/colibri/c That is all. The first answer is slower while the working set crosses the network; afterwards the mirror in ~/.lumabri keeps serving even if the server goes offline. In the chat: /swarm shows the network live and anonymous (peers are numbered, never named: model held, GB, bytes served, heartbeat), /model lists the models on the swarm and switches between them, restarting the engine on the fly. lumabri No arguments. It asks for the swarm's address and, once, for the operator's public key; it finds the engines itself; and it remembers all of it in ~/.lumabri/config , so the second time it is Enter, Enter, and you are in. That matters more than convenience. Everything it asks for used to be a flag, and getting one wrong does not produce "invalid argument" - a missing --engines-dir produces a 299 GB download, and a missing key produces a model nobody verified. Flags still win when given, so a script never inherits somebody's saved answers. lumabri chat asks once, on the way in, and Enter means "just chat" so the impatient path is one key: come entri nello sciame? 1 solo chattare non condividi niente 2 chatti e doni disco tieni un pezzo di glm per lo sciame 3 chatti e doni calcolo esegui esperti per gli altri 4 tutti e due invio = solo chattare Pick 2 and it asks how many GB (Enter takes a quarter of the free space, capped), then starts a maintainer with that budget: the tracker assigns it the least-replicated files first, it pulls them verifying every byte against the operator's signature, and serves them. Pick 3 and it starts the expert node for that model's engine. Both run as children of the chat and stop when you close it - which is the honest lifetime for something offered from a terminal you have open. A donor that should outlive the session is lumabri serve --join . Donating compute needs the model on your disk (an expert node reads the weights from there), so option 3 is offered only with --model-dir DIR . Donating disk needs nothing: you start empty and the swarm fills you. Scripts skip the question: --role chat|disk|compute|all , with --donate GB and --model-dir DIR . One tracker is an index, not a model server, so it holds as many models as you point at it - from one machine or many: lumabri serve --model /models/glm --port 7300 lumabri serve --model /models/olmoe --join 127.0.0.1:7300 --port 7310 lumabri serve --model /models/deepseek --join 127.0.0.1:7300 --port 7320 Each serve brings its own maintainer (the bytes) and its own expert node (the compute), and each registers under its model's name. A chatter sees 3 modelli sullo sciame and switches with /model : the engine is restarted against the new model, and since the engine binary is chosen from that model's model_type , switching between different architectures works too - GLM to OLMoE to DeepSeek, one client, one tracker. Donors are per model as well: a machine can hold a slice of one model and execute experts for another. The tracker keeps them apart, and a chatter only ever discovers the peers for the model it is talking to. Donating space, the server decides: a machine with an empty directory can offer a byte budget and the tracker assigns it the slice to hold, rarest-first, so every donated gigabyte lands where the swarm is thinnest. The donor pulls its slice from the swarm, then serves it. ./lumabri serve --model ./slice --join TRACKER:7300 \ --model-name tiny_olmoe --donate 5 NAT floor: maintainers keep one outbound control connection to the tracker and, when a direct dial fails, bytes are relayed through it. A peer behind any home NAT serves with zero router configuration; direct peer-to-peer stays the first choice. The selftest proves byte identity on the relay path too. Private swarms: set LUMABRI_TOKEN=S on every machine - serve passes it to its tracker, and both the tracker and every maintainer refuse unauthenticated connections, so the token guards the bytes, not just the index. A maintainer advertising localhost from another machine gets its address corrected by the tracker to what the connection shows. No model at hand? Generate the tiny synthetic one (python3 + numpy) and serve it - every step below is the real thing, just small. make && make fixture # builds everything + tiny_olmoe/ ./lumabri serve --model ./tiny_olmoe In a second terminal, the chatter. It needs the engine binaries from a colibri build: ./lumabri chat --engines-dir /path/to/colibri/c What you should see, in order: - The engine boots against a directory that does not exist on the chatter's disk: the config and tokenizer arrive from the swarm on first touch. - Ask something. The first answer is the slow one - watch the net MB counter climb while the working set crosses the wire. - Ask again. mirror caldo, zero rete : the second answer is served from~/.lumabri/ /cache at local speed. /swarm shows the network, anonymous;/model lists what else it holds.- Kill the serve terminal and keep chatting: the warm mirror answers with every peer dead. That is pass 3 of make test , lived instead of read. To grow it, a friend on another machine chats with --tracker :7300 , or donates disk so the model survives you turning your machine off: ./lumabri serve --model ./slice --join :7300 \ --model-name tiny_olmoe --donate 2 The tracker assigns the donor the least-replicated files first; the donor pulls them from the swarm, then serves them. /swarm in your chat now shows two peers. Two stages: everything on one machine first, then more machines. Nothing below is a simulation - the single-machine version runs the same binaries over the same sockets. On the machine with the model. Build both halves of phase 2 (chatters is the patched engines, engines is the expert nodes) and check the model runs at all before any network is involved: make phase2-all ENGINE=/path/to/colibri/c && sudo make install lumabri chat --local /path/to/model --engines-dir /path/to/colibri/c Then the swarm, still on one machine: lumabri key --out swarm # once, keep swarm.key safe lumabri serve --model /path/to/model --key swarm.key --exec-cache 256 For a swarm anyone else can reach, add --advertise : peers publish the address they are given, and without it they publish 127.0.0.1 - right for this machine, useless for everyone else. serve says so loudly if you forget, because the failure mode is a remote chatter that falls back to the relay and never turns phase 2 on, which reads as "slow" rather than "misconfigured". Expect, in order: the hashing progress (first start only - minutes on a big model), ORIGIN: signed the truth of N files , and serving EXEC on :7302 … registered with tracker . In a second terminal: LUMABRI_PUBKEY=$(cat swarm.pub) lumabri chat --tracker 127.0.0.1:7300 --engines-dir /path/to/colibri/c The line to look for is [lumabri] phase 2 active . Without it the chatter is running the stock engine and will download expert weights instead of asking peers to run them - du -sh ~/.lumabri is the other tell: with phase 2 on it grows by the dense part and stops. Adding machines. Open 7300-7302 (and +10 per extra model) in the firewall - on a cloud host, in the provider's console as well as in ufw . Every other machine needs lumabri and a colibri checkout, then make phase2-all ENGINE=… . From there the three roles: | they want to | command | |---|---| | chat | LUMABRI_PUBKEY= lumabri chat --tracker IP:7300 --engines-dir … | | chat + donate disk | the same, then pick 2 (or --role disk --donate 50 ) | | donate compute | needs the model locally: --role compute --model-dir DIR , or expert_node_ as a service | /swarm in any chat shows who arrived. The proof that the swarm is really carrying the work: while a reply is generating, kill a donor - you get one failover line and the tokens continue, identical. concurrency_test.sh runs the same generation from N chatters simultaneously and reports the spread between the fastest and the slowest, because "does it answer" is the easy question and "does anyone get starved" is the real one. On one 6-core box, tiny_olmoe, everything (server, peers, clients) sharing those cores: | chatters | fastest | slowest | spread | |---|---|---|---| | 1 | 1.0 s | 1.0 s | 0.0 s | | 2 | 1.7 s | 1.8 s | 0.0 s | | 4 | 10.4 s | 10.7 s | 0.3 s | Nobody is starved - the spread stays flat while the absolute time grows, which is what CPU contention looks like and not what a lock convoy looks like. Four clients on six cores that are already running the server and its expert node is oversubscription, and on separate machines the clients bring their own cores. Where contention actually lives, so those numbers can be read honestly: - bytes scale. The maintainer answers reads with positional pread on shared fds - no lock on the read path - and the page cache serves every client the same hot bytes. - hot experts scale, cold ones queue. The expert node runs a thread per connection, but a cache miss holds a single loader lock, because the engine loaders are engine-internal state and not re-entrant. Size --cache so the working set fits and misses are rare; tha