What a Vibe Coding Security Scanner Can (and Cannot) Tell You

AI-assisted builders can take an idea from prompt to production in a weekend. That speed is useful, but it also compresses the part of the process where someone normally reviews deployment settings, browser-visible secrets, authorization boundaries, and recovery plans.

A public security scanner is a good first pass for that problem. It is also easy to misunderstand. A clean public scan does not mean an application is secure, and a warning does not always mean a vulnerability is exploitable. The useful question is not “Did the scanner pass my app?” It is “What evidence could this scanner actually observe?”

Layer 1: The Public Deployment Surface

A passive scanner can request the same resources that a normal visitor can reach. Depending on its scope, it may inspect:

• HTTP security headers such as Content-Security-Policy and Strict-Transport-Security
• HTTPS behavior and redirect consistency
• Public JavaScript bundles for credential-shaped strings
• Public source maps that expose original source structure
• Common sensitive paths such as environment files or repository metadata
• Cookie attributes and other response-level deployment signals

These checks are valuable because they test the deployed result, not the configuration you intended to ship. For example, a repository may contain a CSP configuration while the CDN response does not. A source map may be disabled in one build configuration but still appear in production. A key may be stored safely on the server in most code paths while one client bundle accidentally contains a privileged token. The deployed surface is where those mistakes become observable.

Layer 2: Source-Code Review

A public URL cannot reveal every control behind an application. Source review or SAST can inspect code paths, configuration, data flow, and dangerous implementation patterns that never appear in a normal response. This is where you can answer questions such as:

• Is authorization enforced on the server?
• Can a user change an object ID and read another tenant’s data?
• Are database queries parameterized?
• Are uploaded files validated before storage?
• Do administrative actions check roles consistently?

Public scanning and source review are complementary. One checks what actually shipped; the other checks logic that may not be externally visible.

Layer 3: Dependencies and Supply Chain

Dependency scanners answer a different question again. They compare package manifests, lockfiles, containers, or software bills of materials against known vulnerability data. That can identify a vulnerable library version, but it cannot prove that your custom authorization is correct or that your CDN is serving the headers you expected.

For AI-built applications, it is worth combining dependency scanning with a manual review of newly introduced packages. Generated code can add unnecessary libraries, outdated examples, or packages whose names were never verified. GitHub’s documentation on secret scanning and the OWASP Secure Coding with AI Cheat Sheet are useful starting points.

Layer 4: Authenticated Assessment

The deepest application risks often require test accounts and an agreed scope. An authenticated assessment can compare user roles, tenant boundaries, protected workflows, payment states, and business-logic behavior. A public scanner should not attempt to log in, submit forms, bypass access controls, or simulate exploits without explicit authorization. If your application handles payments, health information, private customer data, or privileged workflows, this layer is not optional.

Three Findings That Need Careful Interpretation

“Content-Security-Policy was not observed”

This means an important browser-side mitigation was missing from the checked response. It does not prove that cross-site scripting is present. The next step is to inventory required scripts, styles, frames, and connections, then introduce a restrictive policy in report-only mode before enforcement. Mozilla’s HTTP Observatory is also useful for reviewing header posture.

“A source map is publicly reachable”

A production source map can reveal original file names, source structure, and implementation details. That is an exposure signal, not automatically a security vulnerability. If the map is not required publicly, disable it or upload it privately to your error-monitoring service.

“A credential-shaped string was found”

Some browser keys are intentionally public. Others, including service-role tokens and private credentials, are not. A scanner can identify and redact the pattern, but a human still needs to confirm the provider and privilege level. If the credential is private, rotate it and move it behind a server-side boundary.

A Practical Pre-Launch Sequence

For a small AI-built application, a reasonable minimum process is:

1. Run a passive check against the deployed URL.
2. Review the repository for secrets, authorization, input validation, and unsafe defaults.
3. Scan manifests and lockfiles for known vulnerable dependencies.
4. Verify database policies with separate low-privilege test accounts.
5. Test the production build from a clean browser and mobile device.
6. Record what was checked and what remains outside scope.

I built Check My Vibe as a free, no-account implementation of the first step. It checks a deliberately limited public surface and keeps its automated result separate from a 36-point manual checklist. It does not claim to replace source review, authenticated testing, or a professional assessment. That boundary is the main thing I want builders to take away: security tools are most useful when their evidence and limitations are explicit. A scanner is a starting signal. A secure release still requires human judgment across the layers the scanner cannot see.