Project Log #21: The Grand Finale. We Shipped. (Plus: Full Setup Guide)

21 build logs. Months of work. One shipped project. Here's the full journey-and how to set up the Phone Agent on your own device. The first build log was published months ago. "I'm building an AI agent that controls a phone." No code. No repo. Just an idea and a cracked phone. Today, after 21 build logs spread across months, the project is shipped. This wasn't a straight line. There were gaps. Weeks where the log went silent-not because the work stopped, but because life doesn't pause for build logs. I took breaks to survive exam season. I paused to ship 9 portfolio websites. I stepped away when the code refused to cooperate and my brain needed rest. But every time I came back, the agent was still there. Waiting. And every log picked up where the last one left off. What We Built An autonomous AI agent that controls an Android phone using natural language commands. It can parse your words into actions, read the screen, tap buttons, type text, switch between apps, verify financial data, and serve a web interface-all offline. The Real Timeline | Phase | What Happened | |---|---| | Days 1-4 | Foundation. Gemma 4 + ADB. First working pipeline. | | Days 5-8 | Vision overhaul. UI tree. OCR. Template matching. | | Days 9-12 | Accessibility audit. 30 apps scored. | | Days 13-16 | Multi-app workflows. Task memory. Home reset. | | Days 17-19 | Financial verification. Accuracy from 80% to 94%. | | Breaks | Exams. Portfolio sites. Life. | | Days 20-21 | Web interface. Flask backend. Shipped. | 📖 FULL SETUP GUIDE: How to Install and Use the Phone Agent Follow these steps to get the agent running on your own Android phone. Prerequisites - An Android phone (Android 7 or later) - At least 6GB of free storage space - A WiFi connection for the initial download Step 1: Install Termux Do NOT install Termux from the Google Play Store-that version is outdated. Install it from F-Droid instead. - Open your phone's browser - Go to f-droid.org - Download and install the F-Droid app - Open F-Droid and search for "Termux" - Install Termux from F-Droid Step 2: Set Up Termux Open Termux and run these commands one by one. Wait for each to finish before typing the next. bash pkg update pkg upgrade pkg install python git curl wget pkg install tesseract pkg install android-tools Step 3: Install Ollama Ollama is the tool that runs Gemma 4 locally on your phone. bash curl -fsSL https://ollama.com/install.sh | sh After installation, start Ollama: bash ollama serve Open a new Termux session (swipe from left, tap "New session") and pull the Gemma 4 model: bash ollama pull gemma4:4b This downloads about 2-3GB. Use WiFi. Be patient. Step 4: Clone the Phone Agent Repository bash git clone https://github.com/Dexter2344/phone-agent.git cd phone-agent Step 5: Install Python Dependencies bash pip install flask flask-cors requests numpy PyPDF2 lancedb Step 6: Enable Developer Options and ADB Go to Settings → About Phone Tap "Build Number" 7 times until you see "You are now a developer" Go back to Settings → Developer Options Enable "USB Debugging" Connect your phone to itself via ADB: bash adb devices You may need to approve a prompt on your phone. Once approved, you'll see your device listed. Step 7: Start the Phone Agent Using the Web Interface (Recommended): bash python server.py Then open your phone's browser and go to: http://localhost:5000 You'll see a chat interface. Type commands like: "Open WhatsApp and send a message to Mom saying I'll call later" "Copy my bank balance and send it to Mom on WhatsApp" Using the Terminal (Alternative): bash python agent.py Then type commands directly into the terminal. Troubleshooting Problem Fix "Ollama not found" Make sure you ran ollama serve in a separate Termux session "ADB device not found" Check USB Debugging is enabled. Run adb kill-server then adb devices again "Module not found" Run pip install for the missing module "Tesseract not found" Run pkg install tesseract Phone gets hot Close other apps. The agent is CPU-intensive. Take breaks. Agent misreads numbers The verification layer will catch most errors. For financial data, always double-check. Using the Web Interface Once server.py is running and you've opened http://localhost:5000: Status bar: Green dots mean Ollama and ADB are connected. Red dots mean something is wrong. Command input: Type what you want the agent to do. Be specific. Example commands: Click any example to auto-fill and send. History: Your last 20 commands and responses are stored. The Repo 👉 github.com/Dexter2344/phone-agent 21 build logs. Months of work. One shipped project. Full setup guide included. Thank You To everyone who followed this log-through the daily posts and the weeks of silence-thank you. This was never about shipping fast. It was about shipping honestly. This was Log #1: "I'm starting a new project. It's the most ambitious thing I've attempted from a phone." This is Log #21. We shipped. Top comments (1) Interesting project. The verification layer for financial data caught my attention, especially the jump from 80% to 94% accuracy. I’ve been working on a somewhat similar problem in AI agent tooling (MSCodeBase): an agent can execute the correct-looking action, and the verification can also succeed, while the underlying target or source was actually wrong. In our case, we had a case where symbol resolution selected a test fixture instead of the intended source - the execution itself was valid, but the conclusion was wrong. That made me curious about your financial-data verification: when you double/triple-read an OCR result and the readings agree, how do you distinguish correctness from a consistent OCR error? For example, if OCR reads the same wrong value twice, the verification layer would see strong agreement - but not necessarily truth. Have you tested the verification layer with deliberately introduced but repeatable OCR errors? Do you keep the evidence behind each verification (what was observed, when, and by which method)? I ask because a previously verified state can become stale when the UI changes, and that evidence helps distinguish “still valid” from “needs re-verification.” I think there’s an interesting distinction here between verification of an observation and verification of the conclusion drawn from that observation.