The Vibe Coding Audit Checklist Before Your App Ships to Real Users

Some weeks the work we do most is a vibe coding audit.

A founder ships us a repository. A product leader sends over what their internal team built with Cursor, Lovable, or Claude Code. The ask is almost always the same. Their app works. The demo goes well. They want a second read from an engineering firm before they put it in front of real customers, real money, or real customer data.

The gap between “the app works” and “the app can carry real customers” is where our audit team spends most of its hours. That gap sits underneath the interface. State that lives in six places and disagrees with itself. Permission logic that assumed only one user would ever sign in. Tests that prove the AI did what the AI already did, and check nothing else. Calculations that are correct for the ten example inputs the AI generated against and quietly wrong for the eleventh.

One recent audit is representative.

An investor spent three weekends prompting an AI assistant and built himself a working trading dashboard. Charts, a real time price feed, a login screen, a portfolio view. He was not a software engineer. He understood markets, positions, and portfolio construction. He was ready to pilot the app with a small group of retail traders and asked us to give the code a Friday morning review before Monday’s soft launch.

Then we opened the repo.

Portfolio state lived in six different places, two of them disagreeing by design. The data fetching logic had been rewritten four times across three files. Error handling changed dialect from module to module. The permissions logic assumed a single user forever. On paper the profit and loss math was correct. It was correct for the sample tickers the AI had used during generation. It broke quietly on any symbol with a dividend event, a stock split, or a partial fill. The tests, when there were tests, proved only that the functions returned values. Nothing in the suite anchored a check against a real business requirement.

The application worked. It also carried real risk. If it had gone live with real money on Monday, the first corporate action would have produced numbers that neither the investor nor his customers could reconcile.

That audit is one of many our team has run this year. The applications differ. The failure modes repeat. AI has made it easier for anyone to produce software that looks finished before the code is ready to carry real users. The gap only becomes visible when an experienced engineer traces the plumbing.

Before we get to the ten checks, one framing note. How strictly to apply this list depends on what the app is for. An internal throwaway tool built to solve one operations problem for one team can carry more debt safely. A product that handles customers, money, private data, or an operational workflow the business depends on needs a much stricter launch bar. The checks below are written for the second case, and worth reading for the first too.

If you built the app yourself, these are the same ten questions to put to your codebase. If you commissioned the app from a freelancer or an agency, these are the questions to put to them before you sign off on launch.

Fig 1 – Modular Code Cards

1. The app has to be broken into pieces small enough to audit

The most common problem in a vibe coded app is that it is one enormous blob. Fifty files generated in one long AI session, all of it landing at once, none of it looked at piece by piece as it was produced.

If nobody looked at the app in small pieces while it was being built, nobody can look at the app in small pieces to fix it either. Every bug becomes a whole app problem. Every fix has to reason about everything at once. The economics of running the app collapse before the first hundred customers arrive.

Reading every line by hand does not scale with AI code output either. The shape that does scale is smaller units in the first place, automated gates that check every unit as it lands, risk based routing that sends the important parts to a human, and one accountable owner for each area.

For teams, this means reviewing the plan first, then reviewing stacked increments as the AI generates them. Waiting for one final giant change at the end is how review debt accumulates. For a solo builder, the same principle applies at the file level. Commit small, look at each file as it lands, keep understanding proportional to code volume.

If your vibe coded app arrived as one large landing, the first job is to break it into pieces, understand each one, and hold each piece to the same standard as if you had just built it fresh.

Fig 2 – Owner Understanding

2. Someone has to actually understand what each part does

This is the ownership question. When you or your team built the app, who understood what was being written as it was being written?

We saw the failure most clearly in one audit last quarter. The founder had prompted an AI assistant into a database migration for his customer records. Our reviewer asked why the migration used a particular locking strategy. He answered that the AI had chosen it. The reviewer asked what would happen if the lock timed out under real load. He did not know.

The migration ran fine in staging. It would have caused a two hour outage on his production database the first Tuesday after launch.

Judgment is the multiplier here. AI produces the best output when the operator can evaluate architecture, tradeoffs, and failure modes. Weak direction produces weak software faster than any human could. An AI writing code nobody on your side understands is a debt with no owner. The reviewer or the auditor cannot be the first human to try to understand what the app is doing. Someone on your side must be able to explain, in a paragraph each, what any given file in the app does and why the AI chose that approach.

We ask, on every audit, that whoever hands us the app can answer four questions per major module. What does this do. Why was it built this way. How was it verified. Where could it fail. If those answers do not exist yet, the priority is building that understanding first. An audit only produces value after that step.

Fig 3 – Plan, Prompt Log, App

3. There should be a plan, and the reasoning trail should be recorded

AI assistants drift when the goal is broad and nobody wrote it down. They produce noticeably better work when the task is narrow, the architecture is named, and the failure paths are called out ahead of time.

For any app that will touch real customers, we look for evidence that someone thought about the shape of the app before prompting the AI. A one page description of what it does, what it does not do, what data flows where, what could go wrong, and how it would be rolled back if launch day went badly.

If that one page exists, the app tends to be coherent. If it does not, the app is usually a shape the AI settled into by accident, and the accidents accumulate.

The plan on its own is the start. What holds up in a real audit is the whole reasoning trail. Along with the plan, keep the key assumptions, the approaches you rejected and why, and the significant prompts that produced the code. When a future maintainer opens the repo six months from now, the code alone will not explain why the app was shaped this way. The reasoning trail will. Future you counts as a future maintainer.

The plan is also the cheapest lever a founder has. Twenty minutes writing the goal, the non-goals, and the failure paths before opening Cursor saves days of debugging later. It also produces better AI output. Same tool, sharper input, smaller and more coherent app.

If the app you are auditing came out of an AI session with no plan behind it, write the plan now and compare it against what the AI actually built. The gap between the two is your first fix list.

Fig 4 – Automated Safety Nets

4. Automatic safety nets should be running, and someone should have authority to enforce them

There is a class of software problem that a computer catches without any human looking. Undefined variables. Type mismatches. Known bad dependencies. Common security missteps like a secret pushed to a public repository.

These get caught by tools that sit in the background of any well built project. A linter. A type checker. A dependency scanner. A basic security scan on every save. They are free, they are fast, and they add nothing to the price of the app.

A vibe coded app built without them still runs, and it also carries a long tail of problems that never had to exist. When we audit, we run these tools first. The report they produce is a floor. Everything they catch is a problem your app has that a machine could have caught on day one.

Guardrails only work when someone has the authority to enforce them. An automated check that fires and gets waved through has the enforcement weight of a suggestion. Assign one person who can hold the launch when a check fails, and back that authority when it is used. For a solo builder, that person is you. For a team, the escalation path has to be explicit, or the checks quietly stop mattering the first time they are inconvenient.

If the app you are looking at did not run any of these while it was being built, the fix is short. Turn them on. Address what they find. Assign an owner. Then move on to the harder questions in the rest of this list.

Fig 5 – Fresh Context Second Reader

5. A second reader should have looked at the code before it went live

The AI that wrote the code has already convinced itself the code is correct. That is what “the demo works” means.

The audit question is simple. Did anyone else look at the code before it shipped. Another engineer is best. A second AI session, run cold on the same code with no memory of the writing conversation, is the next best.

The technique is straightforward. Open a fresh chat. Paste the file you are least sure about. Ask the model to assume the code is unfinished and to list the three most likely ways it fails in production, with a file and line reference for each. Read the answer. The specificity matters. A vague “this code could have security issues” adds noise. A concrete “line 84 in checkout.py runs the tenant check after the write, so a race condition can create a record for the wrong customer” is a lead worth chasing. If any of the three land, you have found something the original session missed.

At team scale, this same technique works well as a standing pass. One team we know ran an AI review across every code change they shipped, at more than a hundred changes a week. Most of what it flagged was routine, easy to ignore. Then it caught a single missing resource ownership check on an authenticated endpoint. Any signed-in user could pull records belonging to any other user. That one catch, in a stream of routine noise, was the return on the whole review pipeline. The economics work when even one high-severity defect is caught before it becomes a customer-visible incident. Comment volume is not the metric. Avoided incidents are.

One line to hold. AI review is a screening layer. It surfaces risks, summarizes changes, and gives the human reviewer a sharper input to work from. The formal signoff still belongs to a human, because a human is the one who wears the incident if the code fails in production.

Fig 6 – Blast Radius Map

6. The high risk parts of the app need the closest look

Not every part of a vibe coded app carries the same weight. A typo on an internal admin screen and a broken permission on a customer record are not the same kind of problem, and they do not deserve the same amount of scrutiny.

When we audit, the deepest reading goes to the parts of the app that carry blast radius. Authentication and login. Authorization and permission boundaries. Anything the app does with money. Any place one customer’s data could touch another customer’s data. Database migrations. Background jobs that run at scale, unattended, on production data. Anything that decides who can do what.

These are the surfaces where a single defect turns into a very expensive incident. A logic bug on a status page is a small annoyance. A logic bug on the permissions check that decides whose portfolio the app shows is an incident that can end a business.

Diff size is a weak proxy for risk here. A three line change to an authorization check often deserves more scrutiny than a three hundred line refactor of a cosmetic component. Route review by blast radius, and hold the small high risk changes to the same bar as the big architectural ones.

If you built the app yourself, this is the section to have a professional look at even if you skip the rest. If you commissioned it from a freelancer, this is where you insist on a second opinion before launch.

Fig 7 – Customer Focused Tests

7. The tests should check what customers care about

AI can write a lot of tests quickly. Test coverage as a number gets high fast. The number can also mislead.

The trading dashboard we opened this piece with had ninety two percent test coverage. It also had zero tests for corporate actions, which is the exact scenario that would have broken it for real customers. Coverage was a floor. It said nothing about whether the tests checked what the business actually needed.

When we audit tests on a vibe coded app, the first thing we ask is what each test is proving. A test that checks a function returns the value the function was written to return has almost no audit value. The valuable tests are the ones tied to specific business promises. If the app is supposed to keep customer records private, one test fails when an unauthorized user tries to open another user’s records. If the app is supposed to reject any order below fifty dollars, one test fails when someone tries to place a forty nine dollar order. If the app is supposed to send a receipt within thirty seconds of a purchase, one test fails when the send takes forty seconds.

For every bug fix, the new test has to fail against the previous implementation. Write the test first, watch it fail against the broken code, then apply the fix and watch it pass. If the test passes on the old code, it is not testing the bug, and the fix is unverified.

Our audits also check what has been removed or weakened. Look for deleted assertions. Skipped test suites. Tests rewritten to match whatever the AI happened to generate, which no longer enforces the original requirement. Loosened expectations that used to be strict. A test that passes because the assertion was quietly relaxed is a defect the app is treating as a safety guarantee.

For workflows with combinatorial states (a trading dashboard, a shopping cart, a permissions matrix, a scheduling engine, any asynchronous system), consider property based and state machine tests. These generate large numbers of input combinations and orderings, and they surface the edge cases where AI generated code most often breaks. Property based tests catch ordering and edge case bugs that traditional example tests routinely walk past.

If the tests do not describe the requirement, the reviewer has to verify the behavior by hand. That is possible, and it is expensive, and it is the reason many vibe coded apps ship with unknown defects.

Fig 8 – Polish and Foundation

8. Polish is a neutral signal

Experienced engineers used to read code quality partly from how the code looked. Awkward code, messy formatting, unusual structures were signals that something might be off underneath. Well organized, well named, well commented code was a signal that a professional had touched it.

AI has broken that signal. Generated code often looks polished, with naming conventions in place, comments in place, and sometimes tests in place. Underneath the polish, the reasoning can be wrong in ways the surface never advertises. AI launders away the traditional code smell without necessarily fixing the thing underneath.

There is a second, sharper version of this failure. AI can solve the wrong problem cleanly. The implementation runs, the tests pass, the code reviews well on visual inspection, and the app still does not do what the business needed. A generated function that computes weighted returns using the wrong weighting scheme will look identical, at the file level, to one that uses the right scheme. The mistake lives upstream in the requirement translation, and the code craft carries it downstream in polished form.

The trading dashboard was the cleanest looking codebase we audited that quarter. It was also the one carrying the most business risk. The correlation between clean appearance and correctness was near zero.

When we audit now, we treat polish as a neutral signal. Trust in a vibe coded app comes from the plan, the tests that describe the requirements, and the ability of someone on your side to explain the failure paths in plain language. Anything below that bar goes back for another pass, however polished the code looks.

Fig 9 – File Paragraph Explanation

9. Every part of the app should have a paragraph explanation

This is a simple test, and it catches most of what the earlier checks miss.

Pick any file in the app. Ask whoever is responsible for the app what it does and why it exists. If the answer is a fluent paragraph (this file handles X, we chose this approach because Y, it can fail at Z), the app has a maintainer. If the answer is “the AI wrote that, I am not sure exactly what it does,” the app has no maintainer.

An app with no maintainer is a business risk waiting to be discovered. The first serious bug will not have a fix path. The first serious feature request will not have a home. The first security question from a customer will not have an answer.

There is a second reason to keep these paragraphs. A codebase that is hard for a human to reason about is also hard for a future AI agent to modify safely. Vibe coded apps often grow with more AI assisted changes over time. Every one of those changes needs the AI to fit new work into the existing structure. If the existing structure is opaque, the AI has no anchor. The next round of generation makes the mess worse. Clear structure and clear per-file descriptions do double duty. They help humans maintain the app, and they help the next AI session keep its output coherent.

Our audits routinely find the failure. The founder built the app in one long session, moved on, and cannot now say what a given file does. The fix has to happen before launch. Read the app module by module and write a one paragraph description of each. If you cannot write the paragraph, that is the module that needs another pass with the AI or a human engineer sitting with you.

The paragraphs become the app’s map. Without a map, no one can maintain the app after you, human or otherwise.

Fig 10 – Observability Dashboard

10. There has to be a way to know when something is breaking for a real customer

Vibe coded apps often ship with no observability. No logs. No alerts. No error reporting. The founder finds out something is broken when a customer emails to complain, or when the app has quietly been failing for a week.

For a demo, no logging is acceptable. For an app with paying customers, no logging becomes the reason problems stay hidden until they are already expensive.

“No disaster so far” is not evidence the app is stable. Infrastructure failures, data corruption bugs, and security issues tend to stay quiet until scale, an edge case, or a specific customer workflow exposes them. A month of clean uptime on a demo scale app tells you almost nothing about the first Monday morning when real load hits.

When we audit, we look for four things. Logs that record what the app is doing. Alerts that fire when something goes wrong. Error tracking that surfaces exceptions to whoever runs the app. A simple health check the founder can look at once a day to know the app is alive.

None of these are expensive. All of them can be added to a vibe coded app in an afternoon. The reason to add them before launch is that after the first incident, the founder is already dealing with the incident and does not have the afternoon to set up the tooling that would have caught it earlier.

If your app has no logging and no error tracking, it is not ready for real customers. This is the simplest of the ten checks, and the one most vibe coded apps fail.

Fig 11 – Definition of Done Board

A working definition of done for AI generated code

The ten checks collapse into a short launch bar. Before a vibe coded app meets real users, seven things should be true.

The plan exists in writing. The owner can explain each module in a paragraph. The automated checks pass, and someone has authority to hold the launch if they do not. The tests describe real business requirements, and the tests for the highest risk paths were reviewed by a second reader. The high risk areas (auth, payments, migrations, customer data) have been read carefully by a person, in addition to any AI review pass. Observability is in place. A rollback plan exists.

That is the working Definition of Done we use on every audit. Everything above the line is nuance. Below the line is the app not being ready.

Fig 12 – Coding Time vs Review Attention

How to plan the next AI assisted build

One point for the founder or the team about to build the next thing.

The instinctive way to estimate an AI assisted project is by coding time. The AI writes fast, so the project must be fast. That estimate is wrong. Human review attention is what the project actually spends. The code the AI wrote in an hour may need a day of a senior person’s careful reading before it can carry real users.

Estimate by attention. How many hours of a senior person’s careful reading does the app need. How many hours of writing design notes. How many hours of test review. How many hours of walking through the high risk areas. Those hours are the real cost of building responsibly with AI. Adding coding hours to those attention hours produces a project timeline that lands close to reality. Skipping the attention hours and estimating from coding time alone produces a project that lands in cleanup. Building that way on purpose, with the review attention costed in from the start, is what our vibe coding development engagements are.

What we tell founders after the audit

AI has changed the economics of building an app. The economics of running one, with real customers and real risk, still belong to the founder.

An app that carries real customers, real money, or real data has to be understood, planned, tested against real requirements, watched for failure, and maintained by someone who can explain what it does. The AI can write the code. The founder is still responsible for what happens after the code ships.

For teams, one further discipline is worth naming. Every change should ship with a short note that says what changed, why it changed, where the risk sits, and how it would roll back. On a team, that note protects the reviewer from spending expensive time reconstructing intent. For a solo builder, that same note is the message to your own future self, six months from now, when the memory of what you were thinking has faded.

Our audit team runs these ten checks every week on vibe coded apps that founders bring to us before launch. Most of the apps we look at pass some checks and fail others. Where they fail, we either fix the gaps or hand back a short list of what to fix before the app goes live. That is the service we describe as vibe coding cleanup, and it exists for exactly this situation.

If you want to see the failure modes we see most often in the wild, our companion piece on seven ways vibe coded apps break after launch is the sibling to this list.