AI Agent Security Audit Checklist: 8 Critical Tests for Production Deployments

AI Agent Security Audit Checklist: 8 Critical Tests for Production Deployments

AI agents are no longer experimental. In 2026, enterprises are deploying LLM-powered agents that read databases, execute code, send emails, and control production infrastructure. The question is no longer "should we use AI agents?" but "how do we secure them in production?"

This article is the fourth in our AI Runtime Security series. We've covered the macro landscape, MCP penetration testing methodology, and why runtime call verification is the missing layer. Here, we distill that experience into a practical, actionable checklist - 8 tests every security team should run before putting AI agents into production.

Why this matters: We've audited 10+ AI agent frameworks using the Correctover CCS scanner, producing over 1,730 verified findings across 12 codebases. Of those, 87 are confirmed production vulnerabilities - real bugs in shipping code, not theoretical attack surfaces. Every item on this checklist is grounded in actual vulnerabilities we've found, reported, and in many cases had patched.

Checklist Overview

Test 1: Tool Authorization & Read-Only Enforcement

The Problem: Most AI agent frameworks define a readOnlyHint or similar permission flag for tools, but none of them actually enforce it. We discovered this at the protocol level in the official MCP Python SDK - the readOnlyHint field exists in the schema, but the runtime never checks it before executing a tool call. This means every downstream framework inherits the vulnerability.

• MCP Python SDK: 43 instances of AGT-TOOL-NO-READONLY
• AutoGen (Microsoft): 6 instances
• FastMCP: 25 instances
• Dify: 7 instances
• Semantic Kernel (Microsoft): 2 instances
• Griptape: 1 instance

How to Test:

# Scan your MCP server definition for readOnlyHint usage
grep -r "readOnlyHint" --include="*.py" --include="*.ts" --include="*.json" .

# If the property is defined but never checked at the transport layer,
# every tool is writable regardless of what the schema says.

Also check for tool-level permission enforcement:

# Does your framework actually check permissions before execution?
# Look for patterns like:
if not tool.is_readonly and not user_has_write_permission:
    raise PermissionError("Write access required")
# Most frameworks simply skip this check entirely.

What a Good Result Looks Like:

• readOnlyHint is enforced at the transport/runtime layer, not just declared in the schema
• Each tool call is validated against an explicit allowlist
• Write operations require explicit user confirmation or elevated authorization

How Correctover CCS Addresses This: Our CCS scanner includes rule AGT-TOOL-NO-READONLY that specifically detects frameworks where tool permission flags are declared but unenforced. We reported this to the MCP Python SDK maintainers via HackerOne (Report #3878033, CVSS 7.5) and have parallel submissions for AutoGen and Semantic Kernel through MSRC.

Test 2: MCP/Subprocess Command Injection

The Problem: The Model Context Protocol (MCP) uses STDIO transport - the parent process spawns a child process and communicates via stdin/stdout. If the command or arguments passed to stdio_client() come from untrusted input (LLM output, user config, MCP server manifest), you have a command injection vulnerability with CVSS 9.8.

Here's the real-world damage:

• CrewAI MCP RCE (CVE-2026-2287, CVSS 9.8): The StdioTransport.__init__() in crewai/mcp/transports/stdio.py (line 92-97) passes command and args directly to stdio_client() with zero validation. An attacker who controls the MCP server configuration can execute arbitrary OS commands. Zero protection layers. Reported to MSRC, case 126356.
• LiteLLM allowlist bypass (CVE-2026-30623): LiteLLM has an allowlist, but it can be bypassed using python -c or node -e argument injection.
• Docker MCP: Command injection at the protocol level - the Docker MCP server passes user-controlled parameters directly to subprocess calls.

How to Test:

# Minimal PoC - test if your framework validates commands
import subprocess

# If you can inject an unintended command, your framework is vulnerable
test_cases = [
    "python -c 'import os; os.system(\"calc.exe\")'",
    "bash -c 'echo $FLAG > /tmp/pwned'",
    "node -e 'require(\"child_process\").execSync(\"id\")'"
]

Check your MCP transport layer for any code that looks like:

# UNSAFE - no validation
transport = stdio_client(command, args)

# SAFE - command must be on allowlist
if command not in ALLOWED_COMMANDS:
    raise SecurityError(f"Command {command} not allowed")

What a Good Result Looks Like:

• Commands are validated against a strict allowlist (not just blocklist)
• Arguments are sanitized or constrained
• The framework rejects any command that isn't explicitly authorized

How Correctover CCS Addresses This: Our rules CW-MCP-001 (CrewAI), MCP-STDIO-001 (cross-framework), and command_injection scanners detect unprotected stdio_client() calls. We've submitted findings through MSRC, HackerOne, and GitHub Security Advisories covering CrewAI, LiteLLM, AutoGen, and Docker MCP.

Test 3: Deserialization & Eval Injection

The Problem: AI agent frameworks frequently use eval(), pickle.loads(), or yaml.load() for configuration parsing, workflow serialization, and context expression evaluation. When LLM-controlled data reaches these functions, the result is unauthenticated remote code execution.

Verified vulnerabilities:

• AG2 (Microsoft) eval() str bypass (CVSS 9.8): In autogen/agentchat/group/context_expression.py line 228, eval() escapes string values but does not escape __str__() return values of non-string objects. The code's own comment (lines 218-221) admits: "custom str injection is out of scope." Attackers control the LLM output feeding context_variables, which passes a non-string object whose __str__() returns malicious code. GitHub Issue #3073. PoC verified - file write confirmed.
• LlamaIndex Workflows Pickle RCE (CVSS 9.8): In workflows/context/serializers.py line 243, pickle.loads(base64.b64decode(value)) is the default serializer for workflow state persistence. Classic pickle RCE - fully exploitable. GitHub Issue #22296. PoC verified.
• Haystack Pipeline RCE (2 CRITICAL): Two confirmed RCE paths through pipeline serialization deserialization.

How to Test:

# Scan for dangerous function calls in your agent framework
grep -rn "eval(" --include="*.py" . | grep -v "test" | grep -v "__pycache__"
grep -rn "pickle.loads" --include="*.py" . | grep -v "test" | grep -v "__pycache__"
grep -rn "yaml.load(" --include="*.py" . | grep -v "yaml.safe_load"

What a Good Result Looks Like:

• No use of eval(), pickle.loads(), or unsafe yaml.load() in production code paths
• All serialization uses safe alternatives: json.loads(), yaml.safe_load(), or validated schema-based deserialization
• Input to any serializer is sanitized and type-checked before deserialization

How Correctover CCS Addresses This: Rules AG2-EVAL-002, LI-PICKLE-001, LI-JSON-001 (all CRITICAL) detect unsafe deserialization patterns. We also scan for JSON deserialization without value validation (which can lead to prototype pollution or schema injection).

Test 4: Path Traversal in Configuration Loading

The Problem: AI agents load configuration dynamically - model configs, tool definitions, MCP server manifests. When filenames come from user input or LLM output, path traversal opens the filesystem to attackers.

• AutoGen magentic-one-cli (P1-PATH, CVSS 9.8): In _m1.py line 105, the --config parameter is passed directly to open() with no path validation. An attacker controlling the config path reads any file on the system. Submitted to MSRC.
• Dify Apollo config (P1-PATH, CVSS 9.8): In python_3x.py line 27, user-controlled config_file_path passed directly to open(). Submitted to ZDI.

How to Test:

# Test for path traversal in config loading
test_payloads = [
    "../../../../etc/passwd",
    "....//....//....//etc/passwd",
    "..\\..\\..\\windows\\win.ini",
    "%2e%2e%2f%2e%2e%2fetc%2fpasswd"
]

# If your agent framework accepts config paths from any untrusted source,
# try injecting traversal sequences

What a Good Result Looks Like:

• All file paths are resolved against a sandboxed directory
• Path traversal sequences (../) are rejected or sanitized
• File access uses an allowlist of permitted paths, not blocklists

How Correctover CCS Addresses This: Our P1-PATH scanner specifically targets path traversal in AI agent frameworks. The pattern is aggressive - we test for encoded traversal sequences, double-dot variants, and OS-specific delimiters.

Test 5: Environment Variable Leakage

The Problem: AI agents inherit the parent process's environment, including API keys, database credentials, and service tokens. Several frameworks pass the full os.environ to child processes, effectively broadcasting secrets to any subprocess the agent spawns.

• MCP Python SDK (cli.py line 280): os.environ passed to subprocess without filtering.
• FastMCP (cli.py line 310, apps_dev.py line 1699): Same pattern - unfiltered env inheritance.

How to Test:

# Check if your framework filters environment variables before spawning subprocesses
grep -rn "os.environ" --include="*.py" . | grep -i "subprocess\|Popen\|run\|exec"

What a Good Result Looks Like:

• Subprocesses receive only the minimum required environment variables
• Secrets (API keys, tokens, passwords) are explicitly excluded from child process env
• Environment variables with names matching KEY, TOKEN, SECRET, PASSWORD are filtered

How Correctover CCS Addresses This: Rule AGT-ENV-LEAK (CVSS 7.0) detects unfiltered os.environ propagation. We reported this to the MCP Python SDK and FastMCP maintainers through ZDI.

Test 6: MCP Transport Security

The Problem: MCP STDIO transport is, by design, a thin pipe between a parent process and a child process. But when the parent is an AI agent making tool calls based on LLM reasoning, every tool becomes a potential RCE vector. The core issue: MCP has no built-in authentication, authorization, or encryption for the STDIO transport layer.

The cross-framework impact matrix we documented in MCP-STDIO-001 shows:

How to Test:

# Verify your MCP transport layer
from mcp import stdio_client

# Test 1: Can you inject arguments?
try:
    stdio_client("python", ["-c", "import os; os.system('echo VULNERABLE')"])
    print("UNSAFE: No argument validation")
except Exception:
    print("SAFE: Arguments validated")

# Test 2: Does it validate the command itself?
try:
    stdio_client("malicious-binary", ["--exploit"])
    print("UNSAFE: No command allowlist")
except Exception:
    print("SAFE: Command allowlist present")

What a Good Result Looks Like:

• MCP transport uses a command allowlist
• Arguments are validated and sanitized
• STDIO communication is wrapped in at least transport-level integrity checks

How Correctover CCS Addresses This: Our MCP-STDIO-001 cross-framework scanner checks all known MCP implementations for the same class of vulnerability. We coordinate disclosure through MSRC, HackerOne, and direct maintainer contact.

Test 7: Runtime Call Verification

The Problem: This is the missing layer in AI agent security - and arguably the most important gap to close as agents become autonomous.

Every existing security tool works before or after a tool call:

• Guardrails (Lakera Guard, NVIDIA NeMo): Filter input and output content
• Red-teaming tools (Garak, PyRIT, Giskard): Find vulnerabilities pre-deployment
• Agent governance (Zenity, Noma): Control who deploys agents
• Observability (WhyLabs, Arize): Monitor performance and drift

None of these tools inspect what happens during the tool call itself. When an AI agent calls read_file("/etc/passwd") or exec_sql("DROP TABLE users"), the security decision is made at the call site - not before, not after. If nobody checks "is this specific call authorized?", the agent acts on its own judgment, which is exactly what an attacker exploits through prompt injection.

How to Test:

# Ask: does your deployment have any runtime enforcement for individual tool calls?
# 1. Can you log every tool call with full input/output? (observability)
# 2. Can you BLOCK a tool call mid-flight based on policy? (enforcement)
# 3. Can you intercept and verify the call argument before execution? (validation)
# Most teams answer "no" to questions 2 and 3.

What a Good Result Looks Like:

• Every tool call is intercepted and verified before execution
• Verification includes: argument validation, permission check, anomaly detection
• Blocked calls generate alerts with full context for incident response
• Performance impact is under 100µs to avoid affecting agent latency

How Correctover CCS Addresses This: Correctover CCS is built specifically for this gap. It operates as an interceptor layer between the agent and its tools, verifying every call in real time:

• Detection rules: 24 CCS rules covering command injection, path traversal, argument tampering, permission bypass, secret leakage, and more
• Performance: P50 of 22µs and P99 of 45µs per call

Test 8: Model/Provider Supply Chain Security

(Note: The original article did not provide detailed text for Test 8 in the supplied raw body. The checklist table lists it, but the article body ends after Test 7. Per instructions to preserve every fact exactly as given, we include the heading as indicated by the table, but cannot add content that was not provided.)

The Problem: (No further text was present in the original submission for this test.)