Outside Quest: I asked a local Gemma to name birds. It couldn't, so it sends you on a scavenger hunt instead

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass What I Built Outside Quest is a photo scavenger hunt that runs on Google's open-weight Gemma 4, entirely on a laptop with Ollama. It's designed so the screen is the shortest part of the experience: - Before the walk (about 2 minutes): type where you're going. Gemma writes a six-item quest card that fits the real place and season. Print it, or glance at it once. Then a full-screen "Phone away 🌳" card takes over. - On the walk (an hour): the phone stays in your pocket. You only take it out to photograph a find. - Back home (a few minutes): drop in your photos. Gemma checks which quests each photo completes and says what it saw. You get points, badges and a walk journal: a timeline from the photos' timestamps and a small route map drawn from their GPS tags, with no map tiles and no internet. It's for anyone who says "I should go outside more" and then doesn't: families with kids, friends who need an excuse for a walk, or me after a day of staring at code. The quest card adapts to where you are. For Ambazari lake garden, Nagpur, India in October it asked for "a seed pod or fruit on the ground" and "moss or damp earth" (post-monsoon tropical). For Central Park, New York it asked for "crimson leaf under a sturdy oak" and "a squirrel busy gathering nuts". That only happened after I told the prompt to think about the real local climate; the first version cheerfully asked Nagpur for "autumn colour change". Demo The results above use six Wikimedia Commons photos (credited in the repo) with made-up timestamps and GPS points, so I could test the journal and map. Code JayPokale / outside-quest Two minutes on a screen, an hour outside: an offline photo scavenger hunt powered by Gemma 4 + Ollama. Hacktoberfest 2026 Touch Grass. Outside Quest 🌿 Two minutes on a screen, an hour outside. An offline photo scavenger hunt powered by Google's open-weight Gemma 4, running on your own computer with Ollama. - Before you go: tell it where you're walking. Gemma writes a 6-item quest card that fits the real local season ("a seed pod on the ground" in tropical Nagpur in October, "crimson leaves under an oak" in New York) Print it or glance at it once. - On the walk: phone in your pocket. You only take it out to snap a photo of each find. - Back home: drop in your photos. Gemma checks which quests each photo completes, with the visible evidence and builds a walk journal: a timeline from the photos' timestamps and a little route map from their GPS tags. No account, no upload, no internet needed. Your photos (and their location data) never leave the computer. … Try it ollama pull gemma4:e2b git clone https://github.com/JayPokale/outside-quest && cd outside-quest pip install -r requirements.txt && python3 server.py # http://127.0.0.1:8777 No account, no API key. There's also a Quick card button that works with no model at all. How I Built It Stack: gemma4:e2b through Ollama on a 6 GB GTX 1660 Ti laptop GPU · a Python standard-library server plus Pillow (for EXIF) · one HTML file, no framework. A quest card takes ~30 seconds, checking a photo ~30 seconds. Plan A failed, and that's the most useful thing I learned My first idea was the obvious one: a bird and plant identifier that works on the trail with no signal. Before building the UI, I tested the model on six photos: a Common Myna, a neem tree, a Monarch butterfly, a fly agaric mushroom, a peacock and a storm cloud. | Photo | gemma4:e2b said | gemma4:e4b said | |---|---|---| | Common Myna | "Jackdaw", confidence high | "Weaver Bird", confidence high | | Monarch butterfly | "butterfly, undetermined" | "Heliconius" | | Neem leaves | "deciduous trees, undetermined" | "trees / forest canopy" | | Peacock | Peacock ✓ | Peacock ✓ | Asking for a top-3 list didn't rescue it: the Myna never appeared. Gemma 3 4B did worse and broke its JSON. A tool that names the wrong bird with high confidence is worse than no tool, especially outdoors, where people make decisions about mushrooms and berries. But in every run, the coarse answer was right: there's a bird, a butterfly on a flower, red-capped mushrooms, a storm cloud, a canopy of leaves. So I flipped the design: the model only answers the question small open models are good at, "does this photo show a bird?", and the naming is left to you and a field guide. A scavenger hunt turned out to be a better way to get people outside anyway. Making the judge strict The first judge was too generous: it gave "a leaf" points for the butterfly photo because there were leaves in the background. Word-matching heuristics made it worse (it then rejected the peacock for "a bird"). What worked was asking Gemma, through a JSON schema, two explicit yes/no questions for every quest on every photo: { "id": "q2", "evidence": "A butterfly (an insect) is clearly resting on the flower.", "is_main_subject": true, "completed": true } A quest only counts if both are true. On a test card (a bird, an insect on a flower, a mushroom, a stormy cloud, leaves up close, something red) and the six test photos, that gave 5 correct matches and no false ones. The miss was the storm cloud, which it read as a seascape. It isn't perfect: in the run shown above it gave "a leaf bigger than your hand" to a canopy photo, where nobody can judge leaf size. But it errs strict far more often than generous, and for a game that's the right way round. Safety lives in code, not in the prompt Quests the model writes pass through plain-code filters before anyone sees them. Anything about climbing, entering water, touching, picking, eating, feeding or chasing animals, roads, railway tracks, private land or night walks is dropped and replaced from a safe built-in pool. Mushrooms, berries, nests and eggs always get "(photo only, don't touch)". If the model is down or returns junk, the card fills from the same pool, and there's an instant "Quick card" button that skips the model entirely. Why Does Open Innovation Matter? - Your photos are a map of where you were. Walk photos carry GPS coordinates and timestamps. Sending them to a cloud API would hand someone a log of when and where you walk. Here the model and the photos stay on your laptop. - It works where the walk is. No API key, no signal needed, no per-photo cost, so a family can check a hundred photos after a holiday without a bill. - I could test it honestly and change course. Because the model runs locally and is free to call, I could benchmark e2b against e4b and Gemma 3 on the same photos in an evening, see exactly where it fails, and design around that instead of trusting a demo. - Where closed would win: a frontier cloud model would almost certainly name the Myna. For this app that's the wrong trade: I'd rather have a private, free, offline game that's honest about what it knows. My Agent Session Built with heavy help from Claude Code (an AI coding agent): it ran the model benchmarks above, wrote most of the code, and drove the UI in a browser to test it. The design decisions (dropping species ID, strict judging, code-level safety filters) came out of those test runs. Prize Categories - Best Use of Gemma Top comments (0)