How should I test an app built with AI coding agents?

Test an AI-built app around the work users need to complete and the conditions under which that work can fail. Cover the important permissions, data changes, integrations, and error paths. Tests generated by the same agent that wrote a feature still need review against independently defined requirements.

Playwright's testing best practices emphasize testing user-visible behaviour and keeping tests resilient. For a business owner, that translates into asking whether the test proves a useful outcome rather than merely matching the implementation.

Where should the test plan begin?

List the workflows that would cause the greatest disruption if they failed. For a booking business, these might be creating an appointment, preventing a double booking, rescheduling it, and notifying the right person.

Define the expected result in business terms. "Clicking the button succeeds" is incomplete. "The appointment is saved once for the right customer and slot" gives the reviewer something meaningful to inspect.

Include an owner for each expected result. The business should confirm the rules, while the engineer chooses and implements the appropriate tests.

Which layers need checking?

Some checks belong close to a calculation or rule. Others need to exercise the API, database, or complete user journey. Ask the engineer to explain why the chosen mix covers the important boundaries without repeating the same check everywhere.

Use representative test data and isolated accounts. Tests that rely on a particular production record can become fragile or expose customer information.

How do I avoid circular AI-generated tests?

Give the agent requirements and examples that are separate from its own implementation. Ask it to identify cases that would make a superficially plausible solution fail.

Review the assertions. A test that mocks every important component and then confirms the mock was called may say little about whether the actual workflow works. Conversely, a single large end-to-end test can be difficult to diagnose when several things go wrong.

For a pricing feature, independently prepare a few expected examples, including discounts, boundary quantities, and missing values. Those examples help detect an implementation and generated test that share the same mistaken assumption.

What should happen before a release?

Run the relevant automated checks and verify the critical journey in an environment representative of the intended deployment. Record the version, conditions, and unresolved issues so the result is reviewable later.

Check basic usability too: keyboard navigation, mobile layout, labels, readable errors, and a reliable route to support. A functionally correct form that customers cannot use has not achieved the business goal.

Keep the test plan alive when the product changes. Each important incident or misunderstood requirement can reveal a missing case. Add a useful regression check instead of expanding the suite with tests that only restate the code.

Build a release checklist around your app

Share the workflows that matter to your customers and staff. Discuss production-readiness support to define acceptance criteria, useful tests, and the evidence needed for release.

Frequently asked questions

How much test coverage should we require?

A coverage percentage alone cannot establish quality. Ask which important behaviours and failure paths are covered, what remains untested, and why.

Can AI write our tests?

Yes, it can help draft and maintain them. Review the expected results and assertions independently so the tests can detect mistakes in the generated feature.

Do we still need manual testing?

Targeted manual checks are useful for usability, unexpected interactions, and areas not well covered by automation. Agree which checks are needed for the release instead of treating manual and automated testing as substitutes.