It Works in the Demo: The Checklist Between an AI-Built App and One You Can Put Customers On
Ten checks before customers use an app an AI built: access rules, keys, input handling, dependencies, errors, rate limits, backups, tests and staging.

An app that works in a demo has only proved the happy path: one person, clean data, nobody hostile. Before customers use it, check ten things, and start with the one that does most of the damage: who can read and write each table in your database. This guide is for a founder or small team that built a web app with an AI coding tool or app builder (a front end plus a hosted back end such as Supabase) and now wants real customers on it.
For each check you get what goes wrong, a few-minute test and the fix. Most of it applies to apps built outside WordPress. If an AI wrote a WordPress plugin or snippet, jump to the WordPress section, and read our 10 checks for AI-written WordPress snippets.
We checked every product detail against vendor documentation on 15 November 2026. Defaults, plan limits and even command flags change, so re-check the linked pages. The SQL and commands come from the official docs; we did not run them against a live project. This is engineering guidance, not legal or compliance advice.
What does an AI tool give you, and what does it leave to you?
It gives you speed: screens, sign-in, forms that save data and a hosted database, often from one conversation. It does not make the decisions production asks for: which rows each person may see or change, which keys may be public, which packages are acceptable, what happens when something fails, how you get data back after a mistake, and how a change reaches the live database. Defaults make those decisions, or nobody does.
The first item matters most. In this kind of app the browser often talks to the database directly, through an API the provider generates. No server of your own stands in between to say no, so the only thing deciding who can read or write a row is the set of access rules inside the database. If they are missing, the demo still works, because the demo user is allowed everything. A stranger may be allowed everything too.
The US National Vulnerability Database records CVE-2025-48757 (published 30 May 2025): “An insufficient database Row-Level Security policy in Lovable through 2025-04-15 allows remote unauthenticated attackers to read or write to arbitrary database tables of generated sites.” The supplier disputes it, saying each customer is responsible for protecting their application’s data. NVD shows a CVSS 3.1 score of 9.3 from a secondary source and the status “Deferred” (read 15 November 2026). We do not judge the dispute. Either way, the owner of the customer data finds out first when the rules are wrong.
Check 1: who can read and write each table?
Short answer: only the people you name, and the database has to enforce it. Turn on Row Level Security (RLS) for every table the API can reach, and write a policy for each action you allow.
What goes wrong
Supabase is a hosted Postgres database with an automatic API in front of it. Row Level Security (RLS) is a Postgres feature: rules, per table, that decide which rows a person may read, add, change or delete. The Supabase RLS documentation is direct: “A table in an exposed schema without RLS is readable and writable by any role with a grant on it.” An exposed schema is one the API publishes, usually public.
Two roles matter: anon for visitors who are not signed in, authenticated for signed-in users. On existing projects, a new table in public starts with every privilege already granted to them, though not every project does, so check yours. Postgres runs two gates: grants (may this role touch this table at all) and policies (which rows). The docs warn that “adding policies doesn’t remove” grants, so a table protected only by policies can still hand anon an insert path. Set both. With RLS on and no policy the API returns nothing, which fails safe and is why an app can look broken right after this fix. Write policies; do not switch RLS off.
With RLS on and no policy, the API returns nothing, which fails safe. It is also why an app can look broken right after this fix. Write policies; do not switch RLS off.
How to check in minutes
Open the Security Advisor in the dashboard. The Database Advisors include 0013_rls_disabled_in_public (“Anyone with your project URL can read, edit, and delete all data in this table”) and 0024_permissive_rls_policy, which catches always-true conditions such as USING (true). In the current CLI reference, supabase db advisors inspects the local database by default, so add --remote for your linked project:
# checked on 15 Nov 2026 against the Supabase CLI reference
supabase link
supabase db advisors --remote --type security --level warnThen list every table, including those the tool made for sign-in, files, orders and invites, and answer “who may read it, who may write it”. Test as people: signed out in a private window, then with two test accounts, confirming the second cannot see or change the first one’s data, including by guessing an ID in a URL. OWASP ranks broken access control first in its Top 10:2025.
For each table, enable RLS, remove the default grants and give back only what signed-in users need. These statements follow the pattern in the Supabase docs (replace reports with your table):
alter table public.reports enable row level security;
revoke all on table public.reports from anon, authenticated;
grant select, insert, update, delete on table public.reports to authenticated;Then write one policy per action you allow. The docs’ pattern for “see only your own rows” compares an owner column with the ID of whoever is asking:
create policy "Users can view their own todos."
on todos for select to authenticated
using ( (select auth.uid()) = user_id );Two traps. A policy that relies on the user_metadata claim is unsafe, because users can edit that data about themselves (Advisor lint 0015). And do not add using ( true ) to silence the Advisor: it removes the warning and keeps the data open.
Check 2: what stops a signed-in user from doing things they should not?
Short answer: an explicit permission check on every action that changes money, roles or other people’s data, made on the server or in the database, never in the browser.
What goes wrong
RLS answers “which rows”. It does not answer “may this person approve a refund, invite an admin or export every customer”. AI tools often hide an admin button in the interface and leave the call behind it open to anyone who can type the request.
Database functions need a special look. A function declared SECURITY DEFINER runs with its owner’s rights and bypasses RLS, and a function created in public is callable through the API at /rest/v1/rpc/<name> by default. The Advisors docs say a helper written for an admin script can become “an exfiltration endpoint for any signed-up user”. Lints 0028 and 0029 flag these, and 0010_security_definer_view covers views.
How to check in minutes
List every action only some people should do and try each as an ordinary signed-in account, using the network tab or a command-line request, not the interface. Read every Advisor finding that mentions a function or view. Check Auth settings against the production security checklist (updated 2 November 2026): email confirmations on, test phone numbers removed, one-time-code expiry at 3600 seconds or lower, a custom SMTP server for auth email.
The fix
Move privileged actions into a server function that reads the caller’s verified identity, checks their role and only then acts. Revoke execute on functions from roles that should not call them, prefer SECURITY INVOKER, and keep roles in a table your policies read, not in a claim the user can edit.
Check 3: which keys can live in the browser, and which must never?
Short answer: only the publishable key belongs in browser code. Secret keys live on the server and nowhere else.
What goes wrong
The Supabase API keys documentation separates two kinds. A publishable key (sb_publishable_...) is for client code: “Anyone can read it, so it only reaches what Row Level Security allows”. A secret key (sb_secret_...) is for servers and “bypasses Row Level Security, so it must never leave your control”. Older projects use the legacy anon (browser-safe) and service_role (secret) keys, which the docs say are being deprecated by the end of 2026.
A secret key skips every rule from Check 1, because it maps to the service_role role, which has the BYPASSRLS attribute. If it reaches the browser, a public repository, a screenshot or a chat with an assistant, Check 1 protects nothing. The new keys are short strings; the docs note that a long key beginning with eyJ is a legacy key.
How to check in minutes
Search the repository, its history and the built site that visitors download:
# Current files
grep -rIn -E "sb_secret_|service_role" .
# Anything ever committed, even if deleted later
git log -p --all -S"sb_secret_" --oneline
# The built output (dist, build or .next)
grep -rIn -E "sb_secret_|service_role" dist/Add patterns for your other providers (payments, email, AI services) and search the live app’s Sources tab for the same strings. Any variable your build tool exposes to the browser (names starting NEXT_PUBLIC_ or VITE_) ends up in the bundle, so a secret must never carry that prefix.
The fix
Keep secrets server-side: host environment variables, or supabase secrets set NAME=value for edge functions (docs; names cannot start with SUPABASE_). Browser code gets the publishable key only.
If a secret was exposed, rotate it in the docs’ order: create a new key, replace the old one everywhere, confirm every component uses it, then delete the old one, after fixing the cause of the leak. Assume the old value was copied and read the logs for activity you do not recognise. Deleting it from code is not enough, because git history keeps it.
How to keep it fixed
Turn on secret scanning. GitHub’s docs say it runs free on public repositories, while private ones need GitHub Secret Protection on a Team or Enterprise Cloud plan. Push protection “blocks pushes that contain secrets before they reach your repository”; at repository level it is off by default and a writer can bypass it with a reason. Plans change, so check the page. Keep a written list of every key and where it lives.
Check 4: what happens to the data people type into the app?
Short answer: validate it where it is stored, and escape it where it is shown. A form that checks input in the browser only is a suggestion.
What goes wrong
Anyone can skip your form and call the API with whatever they like: a negative price, a 5 MB name, a role they were never given, a script tag in a comment. If the browser did the only checking, the database accepts all of it. The second failure is output: text one user saved is shown to another, and if the page treats it as HTML, it runs in the other person’s browser.
How to check in minutes
Take your three most important forms and send requests that break each rule on purpose (empty required field, text where a number belongs, a very long string, an extra field such as is_admin). If the database accepts it, the rule lives only in the interface. Then search for innerHTML and dangerouslySetInnerHTML: React’s documentation says it “should be used with extreme caution” and that untrusted HTML, such as anything based on user data, risks an XSS vulnerability.
The fix
Put rules in the database too: NOT NULL, CHECK constraints, foreign keys and proper column types (see the Postgres constraints docs). In a server function, validate the body against a schema, accept only the fields you expect, and never build SQL by joining user text; use parameterised calls. Let the framework escape output, and use a maintained sanitiser where you must render user HTML.
Check 5: are your dependencies real, maintained and pinned?
Short answer: know every package you ship, audit them, and install from a lockfile. Never install a package only because a generated file imports it.
What goes wrong
Your app is mostly other people’s code from a registry such as npm, and an assistant chose much of it. A package can have a known vulnerability, be abandoned, or not exist. A paper presented at USENIX Security 2025, “We Have a Package for You!” (Spracklen and colleagues), studied code-generating models that suggest package names that were never published, a form of package confusion attack: if someone later publishes a real package under the invented name, anyone who installs it runs that person’s code. We quote none of its rates, which depend on the models tested. OWASP lists Software Supply Chain Failures as A03 in its 2025 Top 10.
How to check in minutes
Read package.json. For each dependency ask: do we recognise it, does its registry page show a real maintainer, a repository and recent releases, and do we use it? Remove what fails. Run npm audit: per the npm docs it “submits a description of the dependencies configured in your project to your default registry and asks for a report of known vulnerabilities”, exits non-zero on findings, and --audit-level sets the threshold. Confirm package-lock.json is committed.
The fix
Remove unused or unknown packages and apply npm audit fix, reading the changes. Deploy with npm ci: per the npm docs it needs a lockfile and never writes to package.json or the lockfile, so your live build matches what you tested.
How to keep it fixed
Turn on Dependabot alerts, which “notify you about vulnerable dependencies so you can upgrade to secure versions”, though GitHub warns “Alerts can’t catch every security issue” (docs). Dependency review shows what each pull request adds or updates. Which repositories get which feature depends on your plan.
Check 6: what happens when something breaks, and will you hear about it?
Short answer: a failure should stop the action safely, show the user a plain message, and leave a record only your team can read.
What goes wrong
Generated code often swallows errors (an empty catch that carries on as if the payment worked) or prints them in full to the user, with a stack trace or a table name. OWASP added Mishandling of Exceptional Conditions as a new 2025 category (A10), covering failing open and error messages that contain sensitive information. Security Logging and Alerting Failures (A09) covers the other half: failed logins not logged, sensitive data written into logs, and no alert that reaches anyone.
How to check in minutes
In a copy of the app, break things on purpose: stop a network call, send an empty body, use a wrong password, submit the same order twice. Read what the user sees and what gets recorded. The Supabase logs documentation covers logs for the API gateway, Postgres, Auth, Storage and Edge Functions, with retention that depends on your plan. Also find where front-end and server errors go, and who reads them.
The fix
Fail closed: if a permission check or payment call errors, the action does not happen. Show users a short message with a reference code and write the detail to the log. Log sign-ins, failed sign-ins, role changes, exports and payments, with who and when, and never log passwords, keys or card data. Give each alert a named owner and a destination a person reads, and set an external uptime check on the home page and one database-backed page.
Check 7: what stops one visitor from using up your limits or your budget?
Short answer: limit sign-up, sign-in and password reset, cap anything that sends email or calls a paid service, and decide in advance what your spending cap should do.
What goes wrong
The first signs in a young app are quiet: a sign-up form hit by a script, a table being scraped, an email quota gone by lunchtime, a bill that doubles. Every public form and costly endpoint needs an answer to “what stops one visitor calling this a million times?”.
How to check in minutes
The Auth rate limits page lists the limits on sign-ups, sign-ins, one-time codes and token refresh, adjustable under Authentication, Rate Limits. The production checklist notes CAPTCHA protection on sign-up, sign-in and password reset, and that with custom SMTP the default limit for auth emails is 30 new users per hour, which a launch can exceed. Also read how the Spend Cap works: on, further use of a covered item is disallowed until the next billing cycle and the overage is not charged; off, projects keep running and extra use is billed. Compute is not covered by the cap.
The fix
Turn on CAPTCHA for the forms an attacker hits first, set custom SMTP and raise its limit before a launch, and put your own per-user limit in front of any endpoint that costs money. Choose the Spend Cap setting on purpose: a paused feature or an invoice, whichever hurts less.
Check 8: can you get your data back, and have you proved it?
Short answer: a backup you have never restored is a hope. Restore one into a separate project, time it and write down what was missing.
What goes wrong
The plan lacks the backup you assumed, the backup lacks something you assumed, or nobody knows the steps on the day.
How to check in minutes
Read your plan’s terms on the backups page. As written on 15 November 2026: Pro keeps 7 days of daily backups, Team 14, Enterprise up to 30. Point in Time Recovery (PITR) restores to a chosen moment, is a paid add-on, and replaces the daily backups when enabled. Free projects are told to export with supabase db dump and keep off-site copies. Backups do not include files stored through the Storage API, so uploads need their own copy plan.
The fix
State a recovery target in plain words: “we can lose at most a day” or “a few minutes”. Then take an export you control. In the current CLI, dump commands read the local database unless you add --remote, so an old command from a tutorial may quietly dump the wrong thing:
# checked on 15 Nov 2026 against the Supabase CLI reference
supabase db dump --remote -f supabase/schema.sql
supabase db dump --remote --data-only -f data.sql
supabase db dump --remote --role-only -f roles.sqlThe dump excludes Supabase-managed schemas such as auth and storage, so it is not everything. Next, restore into a separate project, never production. The Restore to a New Project feature (a beta, updated 29 October 2026) needs a paid plan with physical backups. It carries across schema, data, roles and auth users with hashed passwords; storage objects, Edge Functions, auth settings and API keys, Realtime settings, extensions and read replicas must be set up again. Point a copy of the app at it, rerun the Check 1 tests, and record how long it took and what was missing. Repeat after large schema changes and at least every quarter.
Check 9: do tests fail when a rule breaks?
Short answer: write a small test for each rule that matters, starting with access rules and money, and run them before every release.
What goes wrong
AI tools produce screens quickly and tests rarely, so the next generated change can remove a policy or loosen a check and nothing tells you.
How to check in minutes
Look for a test folder and a command that runs it. In Supabase, database tests live in supabase/tests and run with supabase test db (the testing guide uses pgTAP). Ask whether tests exist for each table’s read and write rules, each privileged action from Check 2, each rule from Check 4, and the payment path.
The fix
Create a file with supabase test new <table>_rls.test. Inside a test, switch role with set local role anon; or authenticated, set the caller with set local request.jwt.claim.sub = '<UUID>';, and test all four operations for both roles. Assert on the result: the docs warn never to prove an allowed write with lives_ok, because it passes when the write matched zero rows. Add a few end-to-end tests of the main journeys (sign up, create a record, pay), run in CI on every pull request, and write a test for every production bug before its fix.
Check 10: how do changes reach the live database, and is there a staging copy?
Short answer: through reviewed migration files, tried first on a staging project. Nobody edits the live database by hand.
What goes wrong
Dashboards make a table change one click. Nobody records it, and a month later nobody can say why a column exists or who removed a policy. Staging and production drift apart.
How to check in minutes
Ask whether you could recreate the database from files in the repository. Look for a supabase/migrations folder of dated SQL files. If it is empty and the app has tables, the schema lives only in the dashboard. supabase migration list reads the local database unless you pass --remote:
# checked on 15 Nov 2026 against the Supabase CLI reference
supabase migration list --remoteAdopt migrations. The migrations guide gives the workflow: supabase link, supabase db pull --remote to capture the existing schema, then per change supabase migration new <name>, SQL in the file, and supabase db reset to apply it to the local stack (the same command with --remote drops everything in the remote database). Preview, then deploy:
supabase db push --remote --dry-run
supabase db push --remoteThen add a second project. The managing environments guide describes separate staging and production projects, with staging receiving changes first. One rule: nothing reaches production that has not run somewhere that is not production, and staging holds fake or anonymised data, not customer records.
How to keep it fixed
Make the repository the source of truth, and have a person review every migration that touches a policy, grant or function. Only one person (or the CI pipeline) should run db push at a time.
What if an AI wrote a WordPress plugin or site instead?
Short answer: the same ten ideas apply, and WordPress has a named function for most of them. Use the core function, and do not accept a hand-rolled replacement.
These are the functions we most often find missing or misused in AI-written plugin code, each confirmed on developer.wordpress.org on 15 November 2026. For a full review of a pasted snippet, use our snippet review guide.
Job | Use | What the official reference says |
|---|---|---|
Permission check |
| Returns whether the current user has the capability. The nonce guide adds that nonces “should never be relied on for authentication or authorization”. |
Request intent (CSRF) |
| Verify a security token; they check intent, not who the user is, so pair them with a capability check. |
REST route |
| Required; since WordPress 5.5 a missing one triggers a |
Clean input |
| Sanitising cleans input; the handbook says validation is preferred because it is more specific. |
Escape output |
| Escape late, “as late as possible, ideally as data is being outputted”. |
Database query |
| Prepares a query for safe execution; placeholders include |
For logging, the debugging handbook documents WP_DEBUG_LOG set to true with WP_DEBUG_DISPLAY set to false, so errors go to a file and not to visitors (our debug.log guide covers the cases where nothing is written). Return failures with WP_Error, and use wp_get_environment_type() to keep debug output off production.
Turn debugging off again once you have what you need. For staging, test the plugin on a copy before it touches the live site, as in our PHP upgrade staging check, and keep the five-minute WordPress security audit handy after you deploy. If the site itself was built by AI and is misbehaving, the Wbcom guide Your AI-Built WordPress Site Broke: What to Check, and When to Hire Someone to Fix It goes through what to check first.
Should you harden the generated app, or rebuild it?
Most apps can be hardened, and the ten checks are hardening. Harden when the schema is readable (clear table names, an owner column on each row), you can state the access rules in a few sentences per table, and a build from a clean checkout works. Rebuild the part that fails when nobody can say who is meant to see what, rules lean on data users can edit, each fix breaks something else with no tests to say what, or the builder overwrites your edits. Often the answer is a split: harden the signed-in product and move content (marketing pages, a blog, documentation) to something its writers can run without touching the application database, such as WordPress, a static site generator or a hosted CMS.
Who should own this after launch?
One named person, with a deputy. Not “the team” and not “the AI”. They hold the checklist, the key list, the restore record and the alert inbox, and approve changes to policies and grants. Budget under an hour a month for logs, usage and dependency alerts, a few hours a quarter for a restore test and key review, and a rerun of Checks 1 and 2 after any release that adds a table or an action.
Common mistakes people report
We checked these on 15 November 2026. They are paraphrased from public sources, and each one maps to a check above.
- A table with no access rules. Security firm Wiz reported on 2 February 2026 that a Supabase key in the client-side JavaScript of Moltbook, a vibe-coded app, opened the whole database because RLS policies were missing, exposing about 1.5 million API tokens and 35,000 email addresses. The key was the public kind; the missing rules were the problem (Check 1).
- Silencing the warning. The Advisors docs say an always-true policy such as
USING (true)“defeats the purpose” of enabling RLS (Check 1). - A helper function that became a public endpoint. A
SECURITY DEFINERfunction inpublicis callable through the API by default and skips RLS, hence lints 0028 and 0029 (Check 2). - A leaked secret treated as fixed once deleted. Git history keeps it; Supabase’s guidance is to rotate (Check 3).
- An invented package name. The USENIX Security 2025 paper, which tested 16 code-generating models, found models can recommend packages that do not exist (Check 5).
- A deploy that fails because the live database drifted. Dashboard edits bypass migration history and make
db pushfail; a Supabase CLI issue opened 3 August 2026 shows another route to the same error, mixed 8-digit and 14-digit migration timestamps (Check 10).
Questions people ask
Is my AI-built app unsafe by default?
It is unverified by default. Per the Supabase docs, a table in an exposed schema without RLS is readable and writable by any role with a grant on it. Run the Security Advisor and the two-account test from Check 1.
We pushed a secret key to GitHub. What now?
Close the path that leaked it, then rotate: new key, replace everywhere, confirm, delete the old one. Deleting the file is not enough, because git history keeps the value.
If you would like a second pair of eyes on an app an AI built, our AI-built sites page explains how we review and harden them, and the contact page is the place to describe what you have built and what you need before customers arrive.
Plugins we build and run
We maintain these ourselves, which is why we can support them on your site.
- BuddyNextThe self-hosted community engine for WordPress
- BuddyXFree community theme, Pro when you outgrow it
- ReignPremium community theme. Add pieces, build anything
- JetonomyCommunity forum and Q&A for WordPress
- EventonomyEvents and RSVPs for WordPress communities
- LearnomyCourses and learning communities on WordPress
- ListoraDirectories and listings your members can browse
- SnipShareCode sharing built for developer communities
- WB Ad ManagerAd placements that turn traffic into revenue
- WP Career BoardA job board your community actually uses
- WP Sell ServicesSell services with offers, orders, and payouts
- MediaVersePhotos, video, and media albums for members
- WB GamificationPoints, badges, and rewards that keep members active
- Product RoadmapPublic roadmaps with voting your users trust
Related Posts

LiteSpeed Cache Vulnerability CVE-2026-93389: Check Your Version and What Was Exposed

Freemius SDK Stored XSS: How to Find Every Bundled Copy on a Client Site (Anything Under 2.13.5)

