Back to Blog

    Is Vibe Coding a National Security Risk? 57,947-App Security Audit

    •Cahlen Humphreys•
    AI Strategy
    AI Security
    Application-Layer Governance
    Enterprise AI
    Vibe Coding
    Developer Tools
    Production AI

    DGX Spark audit of 57,947 vibe-coded apps from Lovable, Replit, Bolt, and Create.xyz: 5,924 leak Supabase JWTs, 345 leak live API keys, 7 expose service-role keys. Inside the new shadow IT crisis.

    Is Vibe Coding a National Security Risk? 57,947-App Security Audit

    Picture a healthcare coordinator on a Thursday afternoon. She needs a patient intake form — nothing fancy, just names, contact details, and insurance information before appointments. IT's backlog runs six weeks. So she opens Lovable, types what she wants in plain English, and twenty minutes later she has a working web app with a live database, a public URL, and an AI-generated interface that looks genuinely professional.

    She deployed it before her next meeting.

    Download the underlying research: Surface-Only Research Memo — 57,947 vibe-coded apps audited (PDF)
    A passive, surface-only audit of every public homepage we could render across Lovable, Replit Agent, Bolt.new, and Create.xyz. Counts and percentages only — no hostnames, no JWTs, no excerpts.

    This is vibe coding: generating, scaffolding, and deploying functional software through natural language prompts, with platforms like Bolt, Lovable, Replit, Cursor, and Windsurf doing the technical heavy lifting. For individual productivity, it's remarkable. For enterprise security, it may be the most structurally dangerous development since employees started expensing Dropbox accounts to route around IT storage limits.

    The difference is that shadow IT 1.0 was a procurement problem. Vibe coding is an infrastructure proliferation crisis — and it's running on autopilot.

    The Audit That Should Be Keeping CISOs Awake

    To put real numbers on the problem, we ran our own. Over the course of a few days in early May 2026, we pointed an in-house cyber-agent system — running entirely on a single DGX Spark (Blackwell GB10) sitting on our own network, with nvidia/Llama-3.1-8B-Instruct-FP4 doing the validation pass under vLLM on native NVFP4 tensor cores — at the public surface of every vibe-coded app we could discover. The agent rendered each candidate through a self-hosted Firecrawl instance, ran deterministic regex and JWT decoding over the rendered HTML and every linked JS bundle, and then re-classified every Supabase-using host with the local LLM at roughly 28 hosts per second. No authenticated requests. No POSTs. No private records ever touched. Everything below is what an unauthenticated browser already receives on a single page load.

    The corpus: 57,947 successfully rendered apps across Lovable (19,015), Replit Agent (10,907), Bolt.new (650), and Create.xyz (27,375). The full memo, including the per-platform breakdown and severity distribution, is linked at the top of this post and again at the bottom.

    57,947 vibe-coded apps audited — 5,924 anon JWTs, 386 concrete secrets, 345 third-party API keys, 7 service_role JWTs

    The headline numbers from that scan:

    • 10.22% of fetched hosts (5,924 apps) ship a Supabase anon JWT directly to the browser.
    • 0.67% (386 apps) ship a concrete secret of some kind in client code — third-party API keys, service-role JWTs, the lot.
    • 345 apps leak a live third-party API key (OpenAI, Anthropic, Stripe, Google, AWS, GitHub, SendGrid, Resend) directly in their JavaScript bundle.
    • 41 apps were LLM-validated as shipping the anon JWT alongside client code that concretely names PII-bearing tables — the exact pattern that turned CVE-2025-48757 into a breach class. That's a deliberately strict count; the deterministic regex match before LLM validation flagged 3,380.
    • 7 apps ship a Supabase service_role JWT to every visitor. A service-role key bypasses Row Level Security entirely and grants unrestricted database access with no policy enforcement. If one of those resolves to a live database containing patient records or financial data, the exposure isn't theoretical — it's a direct, exploitable compromise available to anyone who views the page source.

    The exposure is wildly uneven across platforms. Replit Agent's defaults keep credentials out of the client bundle in nearly every case. Lovable, which hands users a Supabase project by default, has the highest anon-JWT exposure rate by an order of magnitude — and the most concrete leaks in absolute terms.

    Per-platform anon JWT exposure: Lovable 30.43%, Bolt.new 18.31%, Create.xyz 1.5%, Replit Agent 0.17%

    The headline number to remember isn't 5,924, or 386, or even 7. It's the funnel itself — what falls out of 57,947 apps once you filter for the patterns that actually resolve to a breach.

    Exposure funnel: 57,947 apps audited, 5,924 ship anon JWT, 3,380 anon JWT plus PII tables, 386 concrete secrets, 41 LLM-validated CVE patterns, 7 service_role JWTs

    To understand why these numbers matter, one technical distinction separates genuine alarm from credible alarm. Supabase anon keys are intended to be public-facing; the architecture assumes the key will appear in client JavaScript, with Row Level Security policies enforcing what any given user can actually access. An anon key by itself isn't a catastrophe — but an anon key deployed by someone who has never heard of Row Level Security, who generated the app in twenty minutes on a Thursday afternoon, is a different situation entirely. The 41 LLM-validated cases and the 7 service-role leaks are the ones that resolve straight into compromise.

    This maps cleanly to three categories in OWASP's API Security Top 10: broken authentication (API2:2023), broken object property level authorization (API3:2023), and security misconfiguration (API8:2023). Security teams have playbooks for all three. The problem is that none of those playbooks were written for infrastructure that was never committed to a repository they control.

    Why This Is Structurally Different From Any Shadow IT You've Seen Before

    The original shadow IT crisis — Dropbox, Slack, unauthorized SaaS subscriptions — was a visibility and procurement problem. Security teams learned to instrument network traffic, enforce SSO, and run software audits. Slow, imperfect, but tractable.

    Vibe coding is something different. These platforms implement what AI researchers call an agentic architecture: the user provides an intent, the model generates code, a sandboxed executor runs it, the model observes errors and iterates. What makes this agentic rather than merely generative is everything that happens in the scaffolding layer — the autonomous decisions the platform makes in service of the user's high-level goal. Database schema creation. Environment variable injection. Public URL assignment. Third-party API wiring.

    The user never issues those sub-instructions explicitly. She said "make me a patient intake form." The platform decided, on its own, to create a Postgres database, expose it at a public endpoint, and inject connection credentials into the client bundle.

    Anthropic's published guidance on responsible agentic deployment recommends that autonomous systems should prefer reversible over irreversible actions and err toward doing less when the intended scope is uncertain. A deployed public URL with a live database connection is, by any definition, an irreversible side effect. The platforms are structurally incentivized to do exactly the opposite — because minimizing friction is the entire product proposition, and friction is where security decisions live.

    The compression of timelines makes this qualitatively harder to govern. A traditional software development lifecycle runs weeks or months and touches version control, code review, security scanning, and deployment approval. The agentic scaffold compresses that into minutes. No governance model built for the SDLC can run at that speed.

    Traditional AppSec tooling compounds the problem. SAST scanners and dependency auditors were designed for human-authored code in version-controlled repositories. Infrastructure that was generated, deployed, and is serving live traffic without ever touching an enterprise-controlled repo is effectively invisible to the entire security stack.

    The Compliance Exposure Is Not Hypothetical

    The legal mechanism here is cleaner and more severe than most practitioners have internalized.

    Under HIPAA's Security Rule, a covered entity is liable for ePHI exposure regardless of how the software causing it was created. "The model generated it" isn't a weak legal defense — it doesn't engage the statutory question at all. HIPAA asks whether the covered entity implemented appropriate administrative, physical, and technical safeguards. An app generated by a coordinator in twenty minutes and deployed publicly is definitional evidence that it didn't. The healthcare coordinator from the opening scenario isn't a hypothetical edge case. She's a compliance incident waiting to be discovered.

    The EU exposure arrives via Article 25 of GDPR — Data Protection by Design and by Default. A platform that emits database credentials to client-side code by default is arguably incompatible with Article 25 for any application touching EU resident data. The EU AI Act adds a second vector for healthcare and education deployments specifically, where organizational accountability provisions for high-risk AI systems create additional obligations.

    The FTC angle is perhaps the most underappreciated in the US context. The Commission's 2024 report on AI and its existing Section 5 unfairness authority create real exposure for companies that deployed AI-generated applications handling consumer data without disclosure, security review, or appropriate oversight.

    The Platforms Are Patching. It's a Treadmill.

    To their credit, the platforms aren't ignoring the problem. Lovable added an explicit Security Audit prompt feature in early 2025 specifically because misconfigured deployments were becoming a documented issue. Replit's Deployments product moved toward secrets management that keeps environment variables out of client bundles. Bolt has iterated on its default security posture.

    But these are reactive patches, addressing known failure modes one at a time, while the user base generating apps grows exponentially. Every patch closes a visible hole. The unknown ones are still being discovered.

    One that belongs in every CISO briefing right now: prompt injection in the generation pipeline. As vibe-coded apps increasingly wire in AI features of their own — chatbots, summarizers, data extractors — an attacker who can influence the input to those embedded LLM calls may be able to manipulate the underlying model's behavior entirely. OWASP's LLM Top 10 categorizes this as LLM01. It's a second-order agentic risk that most vibe coding security conversations haven't reached yet.

    What Enterprise Governance Actually Looks Like

    The counterargument from the productivity camp holds that AI-assisted code is actually more auditable than shadow IT because it's structured and reviewable. This is true, but only in a narrow domain — when a security-aware engineer reviews AI-generated code inside a proper development workflow. It does not apply when a healthcare coordinator generates and deploys a patient form in Lovable on a Thursday afternoon. These are categorically different populations running categorically different workflows, and conflating them is the rhetorical move the productivity argument requires to function.

    Enterprises that want the productivity benefits without the structural exposure need what the developer platform community calls an Internal Developer Platform extended to cover AI-generated artifacts. The controls aren't exotic — they're standard AppSec tooling applied to a new artifact class.

    Secrets management via Vault or AWS Secrets Manager, with policies that explicitly block credential injection into client bundles. Enforced deployment targets that route all generated apps through an internal gateway before any public exposure. Row Level Security policy validation as a mandatory CI gate — Supabase's own supabase inspect tooling can serve this function. DAST scanning of deployed endpoints before traffic reaches them. Software catalog tooling like Backstage, now a CNCF graduated project, gives every AI-generated app an owner, a risk tier, and an expiry date.

    The goal is a governed AI app factory — a path that makes the right thing as easy as the dangerous thing, rather than simply prohibiting the capability that workers are already using.

    The Question Every CTO Should Be Asking

    The deeper issue isn't any specific platform or vulnerability class. It's that a consumer-grade agentic product, optimized for user delight and deployment speed, is making consequential infrastructure decisions on behalf of users who don't know those decisions are being made.

    The twenty-minute patient intake form already exists. It's already deployed. The question isn't whether vibe coding will enter the enterprise — it already has. The question is whether organizations will build governance infrastructure before the first breach, or whether it will take a seven-figure regulatory action and a patient data exposure to concentrate minds.

    The shadow IT crisis of the 2010s taught enterprises that humans will route around friction to get work done — and that the answer was guardrails that moved with the work rather than against it. That lesson is arriving again, faster, with higher stakes, and with infrastructure that can be created and compromised before the security team finishes their morning standup.


    The full research memo
    57,947 vibe-coded apps audited surface-only. Method, per-platform breakdown, severity distribution, and the exact indicator counts behind every figure in this article. Generated locally on a single DGX Spark — no customer hostnames, JWTs, or excerpts in the published artifact.

    Download the PDF →