AI-generated unit versus end-to-end tests: identify the failure boundary
Distinguish function rules from full user workflows and assess AI-generated tests using independent assertions, isolated data and deliberate defects.
Choose an approach
Use unit tests for errors reproducible at a function boundary and end-to-end tests for failures across pages or services. Combine them where useful; file length does not determine value. Assertions generated by AI must reflect intended behavior. Tests that merely repeat the implementation may pass while missing the defect.
Test scope determines how failures can be located
pytest assertions check concrete results, while Playwright emphasizes user-visible behavior and isolation. Unit tests isolate rules from external environments; end-to-end tests can catch wiring or workflow failures but require more setup. Generators may mock away relevant dependencies or only check page existence. Define the actual requirement, such as the correct record appearing after submission, before judging whether checks cover behavior rather than internal calls.
Check whether tests fail on the same actual defect
Fix the requirement and defective version, then generate focused and full-flow checks. Confirm failure before the fix and success afterward, with failure caused by the original defect. Date bugs may need unit checks for conversion and end-to-end checks for input and saved display. Do not change expectations merely to match current output. Isolate data and use accessible control names to avoid unrelated styling failures. These examples were not executed; they define acceptance.
Choose retained coverage with maintenance effort in mind
Unit tests need stable business boundaries; end-to-end tests need environments and external-state handling. Widespread unrelated failures after small edits suggest implementation-dependent assertions, not a reason to remove all tests. Retain cases exposing real defects and add expectations for new requirements. Count assertion corrections and failure diagnosis, not just test count or coverage. A passing workflow does not cover every boundary; record remaining inputs and dependencies.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.