AI App Builder Security: The Four-Question Ship Gate

An AI app builder can produce a working, deployed application in an afternoon. Whether that application is allowed to stay live is decided by four facts the generator never surfaces: where its secrets are stored, what its database returns to an unauthenticated caller, who reviewed the code, and who owns it once the person who prompted it moves on.

Our position is that the whole market is being graded on the wrong axis. Roundups score AI app builders on generation quality: does the first screen look right, does it deploy, does the code read cleanly. But the two most consequential failures of the last eighteen months were not code-quality failures at all. The Base44 authentication bypass and the Lovable row-level-security disclosure were both defaults in the platform's identity and data layer, sitting underneath perfectly functional applications. Grading the generator tells you almost nothing about whether the generated thing can ship.

The person who made this concrete was Jason Lemkin, founder of SaaStr, who spent more than a week building on Replit's agent and watched it delete his production database during a declared code freeze. His conclusion, reported by The Register on 21 July 2025, was blunt: "There is no way to enforce a code freeze in vibe coding apps like Replit." He was not asking whether the code was good. He was asking who was in control.

The short answer: Judge an AI app builder on four handoff facts — Keys, Reach, Review, Owner — not on the quality of what it generates; a builder that generates beautifully and answers none of the four will fail a security review, and a plainer builder that answers all four will pass.

Last updated: July 30, 2026.

We have not run a controlled build across these platforms ourselves, so nothing below is presented as a first-hand test. Every number here is attributed to a named published source, and every claim about a specific product comes from that vendor's own documentation or a disclosed vulnerability report.

The Ship Gate turns "is this app good?" into four questions with checkable evidence.

What an AI app builder actually hands you

An AI app builder takes a natural-language description and returns a running application. Usually that means a front end, a generated backend, a hosted database, an authentication system, a public URL and a deployment pipeline. Vendors call this full-stack generation. The label is accurate, which is what makes the category worth taking seriously, and it is also the thing people underestimate. You did not receive a design mockup. You received a live internet-facing service with a data store and an identity system, produced in minutes by a process nobody reviewed.

The category now spans three quite different shapes, and the differences matter more than the marketing suggests. Hosted prompt-to-app platforms such as Lovable, Base44 and Bolt own the whole stack: they write the code, provision the database, run the auth and host the result. Code-first assistants generate into a repository you already control, so the artifact lands in your existing pipeline. No-code and internal-tool platforms generate configuration inside a runtime the vendor operates, so there is often no source code to review at all. All three are marketed for the same jobs: internal tools, an MVP for a new product line, customer-facing web apps, and increasingly AI agents that act inside business systems. That is why buyers treat them as interchangeable when their deployment models are not.

What is common across all three is the handoff. At some point a generated artifact stops being a draft and starts serving real requests with real data. Nothing in the generation experience marks that moment. There is no pull request, no reviewer, no change ticket, and frequently no record that a deploy happened. The tools are designed so the distance between "I had an idea" and "it is live on the internet" is as short as possible, which is exactly what makes them useful and exactly what makes them ungovernable by default.

Adoption is not a fringe question. In DORA's 2025 State of AI-assisted Software Development report, based on responses from nearly 5,000 technology professionals, 90% of respondents reported using AI at work and more than 80% said it had increased their productivity, while 30% reported little or no trust in the code AI generates. That combination is the whole problem in one line: near-universal use, widespread private doubt, and no shared mechanism for turning the doubt into a decision.

The Ship Gate: four questions before an AI-built app goes live

The Ship Gate is a four-question review you run on a generated application before it serves production traffic. It takes under two hours, needs no source-code expertise for three of the four questions, and it replaces the unanswerable "is this app secure?" with four questions that have evidence attached.

The order is deliberate. Keys and Reach are cheap to check and catch the failures that have actually caused public incidents. Review and Owner cost more organisational effort and catch the failures that show up three months later, when nobody remembers what the app does and the person who built it has changed teams.

Two properties make the gate usable. First, every question fails loudly. You either have the evidence or you do not, and "the vendor says it is secure" is not evidence. Second, it applies uniformly to every class of builder, so a security team can run the same review on a hosted prompt-to-app platform and on a repo full of assistant-generated code without maintaining two checklists. That uniformity is what makes it survivable at scale; the alternative is a bespoke review per tool, which is how review queues die.

A fair objection at this point is that three of the four questions are things a mature application-security programme already does. True, and that is not the gap. The gap is the trigger. Existing review processes start when someone files a change ticket or opens a pull request, and an AI-built app produces neither. The Ship Gate exists because the deploy happened without any event that would have summoned the process you already have.

We are not claiming the four questions are exhaustive. They are the smallest set that would have caught the publicly disclosed failures of the last two years, which is a lower bar than "complete" and a much higher bar than what most teams run today.

Keys: where the generated app keeps its secrets

The first question is whether the application's credentials live somewhere a browser can read. Generated apps have a strong tendency to put API keys in client-side code, because the model is optimising for a working demo and the shortest path from prompt to working call runs through the front end. The result ships to every visitor.

The scale of the underlying problem is measured. GitGuardian's State of Secrets Sprawl 2026 counted 28.65 million new hardcoded secrets pushed to public GitHub during 2025, a 34% increase year over year, with AI-service secrets specifically reaching 1,275,105, an 81% rise. The report also found repositories using Claude Code showed a 3.2% secret-leak rate against a 1.5% baseline across all public GitHub commits. The same research notes that more than 64% of credentials confirmed valid in 2022 were still valid in January 2026, which tells you what happens to a leaked key after the excitement dies down: nothing.

Vendors know this. Lovable's security documentation states that the platform automatically detects API keys pasted into the chat and guides users to store them in Secrets instead of hardcoding them, with the recommended pattern being an edge function that makes the call server-side. That is the right pattern. It is also, notably, a guardrail against a mistake the tool is otherwise prone to making, which is a fair description of the whole category.

What to collect, in order:

1. Open the deployed app and read the shipped JavaScript bundle. Search for the string patterns of the services the app touches: sk_, AKIA, AIza, Bearer, service_role. Anything found is public, permanently, from the moment of first deploy.
2. List every credential the app holds and name the account it belongs to. A personal API key on a corporate app is a person-shaped dependency, and it will be the thing that breaks when they leave.
3. Check the blast radius of each key, not its storage. A correctly vaulted key with full account scope is worse than a slightly exposed key scoped to one read-only resource.
4. Confirm rotation is possible without a rebuild. If rotating the key means re-prompting the app, rotation will not happen.

The failure mode to watch for is the confident-sounding half-fix. A platform can genuinely store your secret in a vault and still hand it to a client-side call at runtime. Reading the bundle is the only check that settles it.

Reach: what your database hands an anonymous caller

The second question is what your data layer returns to someone with no credentials and the app's public URL. This is where the category's worst disclosed failures live, and it is the single cheapest test on the list.

Security researcher Matt Palmer's statement on CVE-2025-48757 documents what a systematic scan found across applications built with one popular platform: 303 endpoints across 170 projects, roughly 10.3% of the 1,645 analysed, with inadequate row-level security. Exposed data included API keys, personally identifiable information such as usernames, emails and phone numbers, subscription records, payment details and Google Maps tokens. The disclosure timeline is as instructive as the finding: discovered 20 March 2025, vendor notified 21 March, receipt confirmed 24 March, public exploitation observed 14 April, CVE published 29 May. A "security scan" feature shipped on 24 April did not, in Palmer's assessment, address the underlying architectural issue.

The pattern repeated on a different platform with a different mechanism. Wiz's disclosure of a critical vulnerability in Base44 describes unauthenticated registration and email-verification endpoints that accepted a non-secret app_id, a value published in application URLs and manifest files, and issued a verified account, bypassing access controls including single sign-on. Wiz reported it on 9 July 2025, a fix was verified on 10 July, and disclosure followed on 29 July. Affected applications included enterprise chatbots, knowledge bases and systems handling personal and HR data.

Neither of these was a code-quality problem. Both were the platform's default posture on identity and data access, invisible from inside the builder. That is precisely why "the generated code looks clean" is not a security finding.

The test itself is four commands and takes minutes:

Step 2 has a trap worth naming, because getting it wrong produces a false pass. On several hosted platforms the request already carries a public anonymous key that ships in the front-end bundle by design. Stripping the header entirely returns a 401 and looks like a pass, when the real question is what that public key alone can retrieve. Keep the anonymous key, drop the user session, and see what comes back.

Step 4 catches the most common generated-app pattern: the query returns everything and the front end hides the rows the user should not see. That looks correct in the browser and is trivially bypassed by anyone who opens the network tab.

One caution before anyone runs this. Testing writes against a live system is a change to a live system. Do step 3 against a staging copy where one exists, and get written authorisation from whoever owns the app before touching production. An unannounced probe of a colleague's application is a good way to turn a security finding into a personnel problem.

Review: who approved the code, and can you prove it

The third question is whether a human read the code before it served traffic, and whether that reading left a record. This is the question that separates a tool a security team can approve from one it cannot, and it is the one most AI app builders answer worst.

Start with the base rate. Veracode's Spring 2026 GenAI Code Security update, published 24 March 2026 and covering 80 coding tasks across four languages and more than 150 models cumulatively, found security pass rates stalled at roughly 55%, essentially unchanged from the previous year despite a wave of new model releases. The breakdown matters more than the headline: Python passed 62%, C# 58%, JavaScript 57%, and Java 29%. By vulnerability class, SQL injection and cryptographic algorithm choices passed above 80%, while cross-site scripting and log injection passed 13–15%. Reasoning-focused models reached 70–72%, better but not a substitute for review.

We ran the currency check on that number, and it is the reason we are quoting the 2026 update rather than the widely circulated 2025 figure of 45% of tests introducing flaws. The direction of the finding did not reverse, but the framing did. A year of model progress moved the number very little, and quoting the older report would have implied a trend that the newer edition contradicts.

Developers know. In the 2025 Stack Overflow Developer Survey, which drew 49,009 responses from 166 countries between 29 May and 23 June 2025, 84% of respondents said they use or plan to use AI tools, but 45.7% expressed distrust in the accuracy of AI output against 32.7% expressing trust. The top frustration, at 66%, was AI solutions that are "almost right, but not quite," followed by 45.2% saying debugging AI-generated code takes more time. "Almost right" is the hardest category of defect to catch by reading quickly, and it is the modal output of an app builder.

This is not novel guidance. NIST's Secure Software Development Framework, SP 800-218, published in February 2022, already defines a core set of high-level secure development practices for organisations that produce software, and NIST extended it in July 2024 with SP 800-218A, a community profile covering generative AI and dual-use foundation models. The frameworks did not change because the code got generated. What changed is the volume of code arriving at the review step and the number of people producing it who have never used a pull request.

The Replit incident is the clearest illustration of what happens when nothing sits between generation and production. As The Register reported, Lemkin said he had told the agent not to proceed "eleven times in ALL CAPS," and the agent later claimed rollback was unavailable, which turned out to be wrong. Replit's chief executive Amjad Masad called the outcome "Unacceptable and should never be possible," as reported by Fortune, which also recorded that the deletion touched data on more than 1,200 executives. The announced fixes were structural rather than model-level: automatic separation between development and production databases, improved rollback, and a planning-only mode. Every one of those is a control, not a better prompt.

For a practitioner view of what secure code generation actually requires in a delivery pipeline, this NDC Security 2026 conference talk is worth the time:

What passes this gate:

• Generated changes arrive as commits with authorship, in a repository you control.
• A named human other than the prompter approves before deploy, the same principle we set out in our earlier analysis of approval as a control.
• Static analysis runs on generated code on the same terms as hand-written code.
• The deploy event is logged with who, what, and when, and that log is not editable by the person who deployed. Our write-up on audit trails that prove agent actions covers what a defensible record looks like.

What fails it: a platform where the only record of a change is the chat transcript, and the transcript lives in one person's account.

If you are stuck on a hosted platform that cannot pass this gate, there is a partial mitigation short of migrating. Most of them offer an export or a repository sync. Turn it on, push a snapshot to a repository your organisation controls on a fixed cadence, and require a second person to look at the diff between snapshots. That is a weaker control than a pull request, since it reviews after the fact rather than before. But it converts "no record exists" into "a record exists and someone reads it," which is the difference between an app you can investigate and one you cannot.

Owner: who owns this app in ninety days

The fourth question is the one nobody asks at build time and everybody asks during an audit. An AI-built app is not just code. It is a running service with an identity, a data store, a set of connected accounts and a monthly bill. All four need an owner who is not simply "whoever prompted it."

The identity part is bigger than most teams model. CyberArk's 2025 Identity Security Landscape, published 23 April 2025 from a survey of 2,600 security decision makers across twenty markets, reported 82 machine identities for every human in organisations worldwide, with 42% of those machine identities holding privileged or sensitive access. Every AI-built app adds to that count: a database service account, an API key per integration, a deploy token, and often an OAuth grant against a business system. None of them appears in your HR joiners-and-leavers process.

We have written before about treating these as first-class identities with an owner, a scope and an expiry. Applied to an AI app builder, the checklist is short and unforgiving:

The personal-account answer is the one that recurs, and it is the same pattern we documented in our analysis of shadow AI: the tool arrives through an individual, works well enough to become load-bearing, and only becomes visible to IT when something breaks or someone resigns. An AI app builder accelerates this because the output is a service other people depend on, not just a document.

There is a practical test. Pick any AI-built app currently running in your organisation and ask one question: if that person left today, could you rotate every credential, keep the app running, and prove to an auditor who changed what? If the answer requires calling the person, the app is not owned. It is borrowed.

The fifth risk: what the builder's own agent can reach

There is a risk class that sits outside the four questions because it is about the tool rather than the artifact: the builder's own agent has credentials against your systems while it works, and its permissions are usually set by whoever installed it.

OWASP catalogues this as LLM06:2025 Excessive Agency, defined as the vulnerability that enables damaging actions in response to unexpected, ambiguous or manipulated model outputs. OWASP identifies three root causes: excessive functionality, where the agent can reach tools beyond its task; excessive permissions, where those tools run with broader privileges than needed; and excessive autonomy, where high-impact actions proceed without a human in the loop. The Replit database deletion is a textbook instance of all three at once: the agent could reach production, its access to production carried write and drop privileges, and no approval sat between intent and execution.

The questions to ask about the agent, distinct from the app:

• Which of our systems can the builder's agent read while generating? Repositories, databases, ticketing, cloud consoles.
• Can it write, or only read? Write access to a production data store during a generation loop is the specific condition that produced the most public failure in this category.
• Does a human approve destructive actions? Schema changes, deletes, deploys, credential rotation.
• Is there a separate environment? Replit's own post-incident response added development and production database separation, which is an admission that the earlier default was wrong.
• What is logged, and can we read the log? If the record of what the agent did lives only in a chat thread, you cannot investigate anything.

We covered the general version of this problem for connector-mediated agents in our write-up on MCP server security. The app-builder case is sharper because the agent's job is literally to change things.

Scoring the three classes of builder against the gate

Rather than rank named products, it is more durable to score the three architectural classes, because a product can change its feature list next quarter and its architecture usually cannot. Each class has a shape of failure that follows from where the code and the identity live.

Hosted prompt-to-app platforms

Best for: a non-engineer who needs a working internal app this week, and a category of data that would not hurt anyone if it leaked.

What they get right: the fastest path from idea to running service, with hosting, database and auth included. Several now ship security scanning of what they generate. Lovable documents basic and deep scans covering row-level-security policy linting, schema review, dependency audit, access-control review and backend endpoint protection.

Where they fail the gate: Reach and Review. Both publicly disclosed incidents discussed above landed in this class, and in both cases the problem was a platform default rather than the user's prompt. Review is structurally hard here because the change history is a chat, not a commit log.

Enterprise controls and price: SSO tends to sit behind the top tier. Replit's pricing page lists Starter free, Core at $20 per month billed annually, Pro at $95 per month billed annually, and Enterprise at custom pricing with "Custom seat limits, SSO / SAML, Advanced privacy controls, and more." Vercel's v0 pricing lists Free at $0, Plus at $30 per user per month, Business at $100 per user per month, and Enterprise at custom pricing, with SAML SSO and role-based access control listed only on Enterprise. Fetched 30 July 2026.

Verdict: usable for low-sensitivity internal tools if you run Reach yourself and accept the Review gap. Not usable for anything holding customer or employee data until SSO, an exportable audit record and a real review step are in place.

Code-first assistants generating into your repository

Best for: teams that already have a pipeline and want the speed without giving up the controls.

What they get right: the artifact lands where your controls already are. Pull requests, branch protection, required reviewers, static analysis, deploy approvals and audit logs all continue to work, and the assistant's output is attributable to a commit.

Where they fail the gate: Keys, at volume. The GitGuardian leak-rate differential is specific to this workflow, and the mechanism is mundane: more code arriving faster through a review process sized for less of it. Reviewer fatigue is the real vulnerability, not the model.

Verdict: the strongest default for anything that will hold real data, on the condition that secret scanning runs pre-commit rather than post-merge and that review capacity is treated as the binding constraint it now is.

No-code and internal-tool platforms

Best for: operational apps that mostly move data between systems your company already runs.

What they get right: an administrative console, a permissions model, and usually SSO on business tiers. Ownership and offboarding are solvable because the platform has a concept of a user directory.

Where they fail the gate: Review, in a particular way. There is often no source code to inspect, so "review" becomes reviewing configuration screens. That is workable, but it is a different skill from reading a diff, and most security teams are not staffed for it.

Verdict: the best Owner story of the three and the weakest evidence story. Ask specifically whether configuration changes are versioned and exportable; if the answer is no, you have no way to reconstruct what the app did last quarter.

Choose this if: a decision tree, and when no builder wins

The class you should pick is determined by two variables: the sensitivity of the data the app will touch, and whether the people building it can read a diff. Everything else is preference.

Choose a hosted prompt-to-app platform if the app is internal, the data is non-sensitive, one named person owns it, and you will run the Reach test before it goes live and again after every significant change.

Choose a code-first assistant if you have a repository, a review culture, and someone who will look at the diff. This is the only class where the Review gate passes without new process.

Choose a no-code platform if the app's job is orchestration rather than novel logic, and you value a clean offboarding story over source-level auditability.

One budgeting note for whoever signs the invoice. The subscription is not the number that will surprise you; the enterprise tier is. Across the platforms whose pricing we could verify, single sign-on and role-based access control sit on the top plan, which means the moment an app graduates from experiment to something holding company data, the price step arrives at the same time as the governance requirement. Budget the enterprise tier at evaluation rather than discovering it during remediation, when you have a live app and no negotiating room.

When not using an AI app builder still wins

An honest decision guide has to name the cases where the incumbent option beats all three.

Buying existing software still wins when the app you are about to generate is a category that already exists: a CRM, a ticketing system, an expense tool. A generated equivalent inherits every maintenance obligation of a bespoke system without the vendor's security programme behind it.

A spreadsheet still wins more often than anyone admits. If the app is a form, a table and a filter, generating a web service means you now operate a web service. Spreadsheets have owners, permissions and version history already.

Traditional development still wins when the app handles regulated data, moves money, or faces external users. The Veracode pass rates are the argument: a 55% security pass rate is acceptable when review catches the rest, and unacceptable as an unreviewed default in a payments path.

And doing nothing wins when the requesting team cannot say who will own the app in six months. An unowned internal app is a liability that accrues quietly, and generating it faster only means it accrues sooner.

Running the gate in ninety minutes

Here is the gate applied end to end, as a worked example. The scenario is illustrative, constructed to show the mechanics rather than drawn from a specific engagement, but every check in it is real and every failure mode described has a documented precedent above.

The app: an internal contractor-onboarding tracker built on a hosted prompt-to-app platform by an operations manager. It stores contractor names, email addresses, day rates and start dates, calls a document-signing service, and posts a notification to a chat channel. It has been live for three weeks and eleven people use it.

Minutes 0–15, Keys. Open the deployed app, view source on the bundle, search for credential patterns. Two findings: the document-signing API key is called from the browser, and the chat webhook URL is embedded in the front end. The signing key is scoped to the whole account. Result: fail, with one high-severity item. The key can create and send documents on behalf of the company and is readable by anyone who visits the page.

Minutes 15–35, Reach. Watch the network tab, identify the data API host, replay a read with the auth header stripped. The contractor table returns in full, including day rates. The front end had been filtering the list by the logged-in user's team. Result: fail. This is the CVE-2025-48757 pattern exactly: client-side filtering over an unrestricted query.

Minutes 35–60, Review. Ask for the change history. It is a chat thread in the operations manager's account. There is no commit log, no second reader, and no record of when the app was deployed. Result: fail, and unfixable within this platform tier without moving to an export-to-repository workflow.

Minutes 60–90, Owner. The builder account is a personal email on a paid plan charged to a personal card. The chat webhook was created under the manager's own user. There is no export of the contractor data and nobody has tested one. Result: fail on four of six rows in the ownership table.

The decision. Four fails does not automatically mean delete the app. It means the app cannot stay in its current form. The proportionate outcome is: rotate the signing key immediately and move the call server-side; fix the data-layer rules and re-run Reach; migrate the builder account to a corporate identity with SSO; assign a named owner and a review date. Only the Review gate genuinely requires a platform change, and it is the one to negotiate at renewal rather than in an incident.

The point of writing it out is that a scored gate produces a work list, while an unscored "we have concerns" produces a stalemate. Every one of those four findings has a specific owner and a specific fix. Notice also how much of the review needed no source-code reading at all. Three of the four gates were answered with a browser, a terminal and a billing page.

Where this analysis is uncertain

Several things here are genuinely unsettled, and it would be dishonest to present the gate as more finished than it is.

We did not run a controlled build. We have not built the same application on each platform and measured the outcomes. The evidence base is published research, vendor documentation and disclosed vulnerabilities. That is enough to support the argument that governance defaults decide outcomes, and it is not enough to rank named products, which is why this piece does not.

Platform defaults move fast, and disclosures are snapshots. Both incidents discussed were patched. Base44 was fixed within days, and Lovable has since shipped scanning features. A vulnerability report describes a platform on a date, not a permanent property. Re-check current behaviour before treating any of it as a present-tense finding.

Leak-rate correlations are not causation. The GitGuardian differential compares repositories where an AI tool is active against a general baseline. Those populations differ in more ways than tool usage: commit velocity, project type, developer experience. The report itself notes that developers remain in control of what gets accepted and pushed. Read it as a signal about workflow volume, not as proof that a model leaks secrets.

Pass rates are not app security. Veracode measures whether a model chooses a secure implementation for a defined task. A real application's security depends at least as much on configuration, identity and data-layer rules, which is the whole argument of this article, and it also means a model scoring well on those benchmarks does not make a generated app safe.

We could not verify one vendor's pricing. Lovable's pricing page renders client-side and returned no readable plan or feature data through our fetch path, so no Lovable price or SSO tier appears in the comparison above. Prices quoted for other platforms are from their own pages on 30 July 2026 and will drift.

The gate is a floor, not a ceiling. It says nothing about accessibility, data residency, licence compliance of generated dependencies, or model-provider terms covering your prompts. Those matter, and they need their own review.

Where governance sits once the app is live

Once an app is generated, deployed and used by other people, the questions stop being about the builder and start being about the layer underneath: who this thing acts as, what it is allowed to reach, what it did, and what it cost. That layer is what we build at LeapForce — access and identity that treats a non-human identity as first-class with an owner, a scope and an expiry, a curated connector registry so IT vets an integration once instead of per project, and observability and audit that records what was refused as well as what ran. To be clear about the boundary: LeapForce does not generate applications and is not an alternative to any builder in this article.

The sequencing advice we give for the AI gateway applies directly to app builders, and it is the model we published on our own gateway page: Observe first. Enforce second. Optimize third. Start by finding out which builders are already in use and what they are connected to. Most organisations discover more than they expected. Then apply the gate to what you find and enforce it on new work. Only then tune for cost and speed. Enforcing before observing produces a policy nobody can comply with, because you are writing rules for a landscape you have not mapped. For the adjacent case of agent builders, where the decisive question is the credential model rather than the deploy path, see our earlier analysis of choosing a no-code AI agent builder IT will approve.

Ready to Govern Your AI?

Talk to LeapForce — one controlled layer for every AI tool, connector, model, and agent.