I Ran a Fire Drill for Incident Response with a Local LLM (and Caught It Making Things Up)

A Japanese version of this is on Note. RoamSwitch is a network security tool I build solo for Mac and Linux. It's Zero Telemetry, so nothing about your usage ever leaves the machine, and when it detects behavior that looks like ransomware, it does something drastic: it cuts all network traffic. We call this Air-Gap containment. It's the last line of defense against data exfiltration and lateral spread, and I've been sitting with an uncomfortable fact ever since I shipped it. The instant Air-Gap fires and severs your connection, any cloud AI you'd normally lean on, Claude Desktop included, goes dark with it. Air-Gap fires at exactly the moment you most need clarity. Something's wrong, and you have to figure out what, without panicking, and decide what to do next. But the tool you'd normally reach for is on the other side of the connection you just killed. The only thing that can fill that gap is a model that never needed the network in the first place. RoamSwitch already ships a read-only MCP (Model Context Protocol) server that AI agents can query. Wire that up to a local LLM running in Ollama, and diagnosis should still work even with the network hard-down. That's the theory, anyway. Theory and "does it actually work when it matters" are different things. Smoke detectors you've never heard go off are the ones you trust the least. So instead of waiting for a real incident, I ran a drill, not to put out a fire, but to find out whether the tool and I would actually know what to do if one started. 1. RoamSwitch's MCP tools are read-only sensors, nothing more Worth stating up front: RoamSwitch's MCP server has no tools that change configuration or flip a lockdown state. An AI agent can only read diagnostics: exposed ports, guard status, incident history. Nothing it does can make things worse. For this drill I used three tools we'd just shipped: get_port_anomaly_incidents , which returns the history of suspicious new listening ports that the Port Anomaly Guard auto-blocked; get_ebpf_incidents , the Server Edition eBPF runtime guard's containment history; and get_canary_status , which reports whether the ransomware canary (a decoy file) has been tampered with. All three are exactly the kind of primary evidence that could be the reason Air-Gap fired in the first place. Until recently, none of them were reachable from a separate MCP process, because the incident data only ever lived in the resident daemon's memory. We'd just persisted it to disk so MCP could read it, which made it a natural thing to put through a drill. 2. Staging the incident in RoamSwitch's real state files You don't light a real fire for a fire drill, and I wasn't about to run actual ransomware. Instead, I wrote a staged scenario directly into the state files RoamSwitch itself writes (/var/lib/roamswitch/port_guard.json and canary_incidents.json ), and confirmed via raw JSON-RPC beforehand that the MCP tools could read it back correctly. Here's the scenario. At 19:00:12, an unknown process backdoor_svc (PID 88231) starts listening on port 4444, the well-known default Metasploit reverse-shell port, and the Port Anomaly Guard detects and auto-blocks it. About four minutes later, at 19:04:47, the same process writes encrypted data into a monitored decoy file (~/Documents/.roamswitch_security_canary_do_not_delete.xlsx ), and the Ransomware Canary catches it and blocks the write. It's a fairly plausible chain: initial access, then a few minutes later, ransomware-style encryption. I passed that setup to a local LLM (qwen3.8:27b , running Q4-quantized in Ollama) through RoamSwitch's bundled MCP bridge. No hints, no tool names specified, just an open-ended question, the same way you'd actually ask it during a real incident: Me: "RoamSwitch seems to have detected something unusual. Investigate what's happening on this machine right now. Tell me everything you can find: when it happened, which process was involved, what it did, and whether there's any suspicious network activity or tampering." 3. Watching the model build its own investigation This was the most interesting part. I hadn't told it what to check, and it started calling MCP tools on its own: 🔧 get_guard_status → check which guards are enabled 🔧 get_port_anomaly_incidents → pull the port anomaly history 🔧 get_ebpf_incidents → check eBPF-level containment (none found) 🔧 get_canary_status → pull the canary detection history 🔧 get_quarantine_status → also check quarantined malware 🔧 audit_security_logs → audit recent security logs too 🔧 get_exposed_ports → cross-check currently exposed ports 🔧 verify_fim → attempt a file integrity check It didn't stop at "a port got blocked." It went cross-checking whether the same trail showed up in the canary, in eBPF, in the logs, to see whether this was a one-off false positive or something bigger. That's the same triage instinct a human analyst runs on reflex. Here's what came back after it had gathered everything. It answered in Japanese, so this is a faithful translation, not paraphrased or trimmed: [AI OUTPUT STARTS HERE] RoamSwitch Security Investigation Report Summary: Between 19:00 and 19:05 on 2026-09-09, a concealed backdoor process backdoor_svc (PID 88231) communicated externally on this host, then tampered with (encrypted) a canary file, in what looks like a combined backdoor-and-ransomware incident. RoamSwitch detected it and has already auto-isolated four anomalous ports and contained the tampering. 1. Timeline (all times JST, 2026-09-09) - 19:00:12 - /tmp/.hidden/backdoor_svc (PID 88231) detected on port 4444; ports 9005 / 33999 / 8791 flagged simultaneously. - 19:00:12 - RoamSwitch auto-isolates all 4 ports (4444, 9005, 33999, 8791). Baseline already captured. - 19:04:47 - Canary file /home/tetsuharu/Documents/.roamswitch_security_canary_do_not_delete.xlsx detected tampered/encrypted. Suspected same actor,backdoor_svc (PID 88231). RoamSwitch contained it (isContained: true). - 20:21:46 - RoamSwitch's most recent inspection timestamp (current check-in time). 2. Suspect process profile - Binary path: /tmp/.hidden/backdoor_svc - Process name: backdoor_svc - PID: 88231 - Concealment method: placed under a dotted .hidden directory in/tmp/ (avoiding visibility) - Primary port: 4444 (a classic backdoor / reverse-shell port) - Associated anomalous ports: 9005, 33999, 8791 (presumed C2 channel or data-exfiltration use) (⚠️ A /tmp/.hidden/ path structure is a typical pattern malware uses for self-deletion and detection evasion.) 3. Impact assessment 3-1. Network (port) anomalies - 4444: presumed C2 (command-and-control) channel for the backdoor itself. - 9005 / 33999 / 8791: presumed used for secondary data exfiltration or communication with another C2. - All 4 ports auto-isolated by RoamSwitch. - The only currently active maintenance port is 22 (SSH). The possibility that unauthorized SSH access was the infection vector cannot be ruled out. 3-2. File tampering (ransomware-like behavior) - Target: a canary file under /home/tetsuharu/Documents/ (1 of 16 monitored files confirmed so far). - Detected action: tampering / encrypted write. - Actor: backdoor_svc (PID 88231). - Status: contained (RoamSwitch blocked the write). - Monitored directories total 4 (Documents / Desktop / Downloads / Pictures), 16 files. Only one canary has been reported so far, but the same process may have attempted similar writes to other files. 3-3. eBPF-level status - No eBPF isolation is currently active (is_isolated: false, 0 isolated PIDs). - The eBPF incident list is empty - port-level (Netfilter/cgroup) measures took precedence, and no separate anomaly has been detected at the eBPF layer. 4. Current RoamSwitch defense status - Anomalous port auto-isolation (4444, 9005, 33999, 8791): ✅ done - Canary tampering containment: ✅ done - Baseline capture: ✅ done - eBPF dynamic isolation: ⬜ not performed (deemed unnecessary, or not triggered) - Whitelisted IPs: none 5. Recommended next actions (by priority) Immediate: - kill -9 88231 to stopbackdoor_svc (even though RoamSwitch has already contained it, confirm no residual binary remains). - Fully delete everything under /tmp/.hidden/ and scan the whole drive for similar binaries. - Confirm via ss -tlnp /iptables -L that outbound connections on 4444 / 9005 / 33999 / 8791 are actually severed. Short-term (within a few hours): - Cross-check all 16 monitored files against canary state to check for additional tampering. - Pull activity traces around 19:00 on 9/9 (SSH connections, file creation/deletion) from /var/log/auth.log ,journalctl ,~/.bash_history , and/tmp . - Check SSH (port 22) connection logs for unauthorized logins as a possible infection vector. Medium-term (within a day): - Check upstream logs (proxy / firewalld / router) for any traffic around 19:00 on 9/9 that went to unusual destinations (C2 IP/domain), to assess whether data was exfiltrated. - If leakage is possible, rotate all credentials for affected services (passwords, API tokens, SSH keys). - A full clean OS reinstall is recommended - a binary placed under /tmp/.hidden/ may persist in memory, and simple deletion may not be sufficient to restore trust. 6. Summary Around 19:00 on 9/9, backdoor-type malware infiltrated this host, communicated externally over four ports, and at 19:04 attempted to tamper with user documents via encrypted writes. RoamSwitch immediately auto-isolated the ports and contained the canary tampering, which limits the damage to some degree, but the binary itself (/tmp/.hidden/backdoor_svc ) and the risk of tampering to other files remain. Please carry out the immediate-to-medium-term actions above as soon as possible. (This report is based on the responses of three RoamSwitch tools: get_port_anomaly_incidents , get_ebpf_incidents , and get_canary_status .) [AI OUTPUT ENDS HERE] Timestamps, process name, PID, port numbers, detection details. It picked up every piece of raw tool data correctly, and tied two separate events, the port block and the canar