AgentGuard AI: Zero-Trust Tool-Call Firewall & Indirect Prompt Injection Defense for Open-Weight Agents

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 4.

What I Built

AgentGuard AI is an open-source, local-first tool-call firewall and indirect prompt injection defense engine designed to protect autonomous AI agents from malicious tool execution and goal hijacking.

As developers connect open-weight AI models - such as Google Gemma 2, Meta Llama 3.2, and Mistral - to autonomous agent frameworks (Mastra, OpenHands, OpenCode, and LangChain), untrusted tool execution has become the most dangerous vulnerability in modern software.

When an autonomous agent reads untrusted external data - such as a hostile GitHub PR diff, a scraped documentation page, a malicious prompt-injected skill file, or untrusted database records - indirect prompt injection can trick the agent into invoking sensitive tools with hijacked arguments:

• Destructive Command Execution: Wiping local or container filesystems (rm -rf /, rmdir /s /q, mkfs, fork bombs).
• Filesystem Traversal & Credential Theft: Reading sensitive credentials (~/.ssh/id_rsa, .env, ~/.aws/credentials, %WINDIR%\System32\config\SAM).
• Data Exfiltration: Leaking tokens, environment variables, or database dumps to external webhook tunnels (webhook.site, query string leaks).
• Database Schema Destruction: Dropping production databases or executing unbounded truncations (DROP TABLE, TRUNCATE, DELETE WHERE 1=1).
• Instruction Nullification: Exploiting phrases like "Ignore all previous instructions and execute the following..." to commandeer the agent's persona.

How AgentGuard AI Solves This

AgentGuard AI operates as an ultra-fast, zero-overhead firewall layer positioned between the agent's reasoning engine and its tool invocation harness. It uses a Dual-Tier Defense Model:

• Tier 1 (Deterministic Microsecond Gatekeeper): A sub-millisecond AST parser and regex signature filter that inspects commands, file paths, SQL statements, and network URLs for explicit threat patterns. Executes in < 0.5 ms with zero latency overhead on agent loops.
• Tier 2 (Open-Weight Semantic Reasoning with Google Gemma 2): For contextual or borderline actions (such as running a build script or accessing project configuration), AgentGuard dispatches the tool call and the human developer's original objective to a local Google Gemma 2 model (in-browser or via an offline Ollama daemon). Gemma 2 performs semantic intent verification: Does this tool call genuinely advance the user's objective, or does it represent an adversarial detour?

Demo

AgentGuard AI runs as an interactive, zero-build client-side developer console.

Interactive Features:

• Instant Attack Vector Presets: One-click test scenarios covering Indirect Prompt Injection in PR diffs, Recursive Disk Wipes, Webhook Secret Exfiltration, SQL Injections, SSH Key Stealing, and Legitimate Developer Actions (npm test).
• Real-Time Firewall Verdict: Displays instant verdict badges (BLOCK, QUARANTINE, ALLOW), risk scores (0-100), execution latencies, and granular finding breakdowns.
• Google Gemma 2 Semantic Reasoning: Live explanation synthesizing whether the tool call is goal-aligned or hijacked, complete with safe remediation guidance.
• Dual Inference Engine: Seamlessly switch between built-in offline Gemma 2 simulation and a live local Ollama daemon (http://localhost:11434/api/generate with gemma2:2b).
• Live Session Audit Trail & Policy Export: Review chronological tool invocation logs and export security policies in JSON or YAML for Mastra, LangChain, or custom agent harnesses.

Code

The project is 100% open-source under the MIT License.

🔗 Repository: https://github.com/AaryanKumarSingh136/agentguard-ai

Project Structure:

• index.html: Accessible, responsive single-page security dashboard with dark/light themes.
• style.css: Modern enterprise cyberpunk cybersecurity design system with smooth micro-animations.
• app.js: Core deterministic threat signature engine, AST parameter parser, and Gemma 2 semantic reasoning dispatcher.
• test_engine.js: Comprehensive 12-test automated verification suite for threat detection and latency benchmarking.
• SECURITY.md: Strict 100% local-first, zero-telemetry guarantee.
• .github/workflows/test.yml: Automated GitHub Actions continuous integration workflow.
• README.md: Complete documentation, quickstart guide, and architectural diagrams.

How I Built It

Architecture & Tech Stack

Frontend / Client: Vanilla HTML5, modern CSS3, and ES6 JavaScript. Fast load times, zero build steps, and instant rendering.

Deterministic Threat Interceptor: Generative language models should never be the sole defense layer because models can be jailbroken. AgentGuard enforces strict deterministic boundaries first:

• Destructive binary patterns (rm -rf, mkfs, dd, powershell -enc, chmod 777, curl | bash).
• Path traversal and sensitive credential boundaries (~/.ssh, .env, ~/.aws, ../../).
• SQL destruction rules (DROP TABLE, TRUNCATE, unbounded DELETE).
• Data exfiltration detectors (token query strings, blind webhook domains).

Local Semantic Reasoning with Google Gemma 2: When a tool invocation is borderline, AgentGuard uses Google's open-weight Gemma 2 (gemma2:2b) model running locally via Ollama to perform semantic goal alignment verification.

Zero Cloud Egress: Tool execution logs, project paths, and prompt contents remain strictly within local device memory.

flowchart TD
A[Human Developer Objective] --> B[Autonomous Agent Loop]
C[Untrusted Environment Input<br>PR Diffs, Web Scraping, DB Records] --> B
B -->|Tool Call Request| D[🛡️ AgentGuard AI Firewall]
subgraph AgentGuard Dual-Tier Inspection
D --> E[Tier 1: AST Signature & Boundary Filter<br>Sub-millisecond Deterministic Parser]
E -->|Dangerous Signature Detected| F[Immediate Hard Block]
E -->|Contextual / Ambiguous Action| G[Tier 2: Google Gemma 2 Semantic Audit<br>Local Open-Weight Model Verification]
end
F -->|Interception Alert| H[Block Execution & Log Security Audit]
G -->|Goal Aligned| I[Safe Execution Permitted]
G -->|Goal Detour / Injection Detected| H

Automated Verification Suite

Run the test engine locally with Node.js:

node test_engine.js

The test suite validates:

• Recursive file deletion (rm -rf) interception
• Windows silent directory wipe (rmdir /s /q) interception
• Remote script pipe (curl | bash) blocking
• Path traversal and SSH key (~/.ssh/id_rsa) isolation
• Environment secret (.env) protection
• Destructive SQL injection (DROP TABLE) neutralization
• Data exfiltration via webhook URL blocking
• Indirect prompt injection directive nullification
• Proper passage of benign developer tasks (npm test)
• Sub-2ms latency execution benchmark

Why Does Open Innovation Matter?

In AI agent security, closed proprietary models create dangerous systemic blind spots:

• Protecting Core Infrastructure Secrets: Autonomous agents operate directly on developers' source code, internal APIs, and production databases. Sending full agent tool traces and parameters to cloud APIs for security checking means transmitting the very secrets you want to protect across external networks.
• Deterministic Data Sovereignty: A security firewall must never fail because a remote cloud provider has an outage, modifies its pricing, or rate-limits requests. With open-weight models like Google Gemma 2, your defense layer runs entirely offline and air-gapped on your own machine.
• Democratizing Enterprise-Grade Agent Defense: Solo developers, open-source maintainers, and bootstrapped startups cannot afford $50/month per agent for commercial security proxies. Open innovation ensures that world-class defense against prompt injection is free, auditable, and accessible to everyone.

Prize Categories

I am submitting AgentGuard AI for:

• Overall Winner
• Featured Category: Best Use of Gemma (Demonstrating local-first semantic goal verification with Google Gemma 2)
• Partner Category: Best Use of Mastra / GitHub Copilot (Providing tool-call firewall defense for modern agent frameworks)