The page says Playwright tests "breaks on selector/layout changes" but that this is more robust because it "reads the page like a human, via ARIA semantics". Any well-written e2e test should be using ARIA semantics to drive page object selectors, precisely because using other selector strategies are fragile. Playwright also has a test healer AI agent to solve this problem.
Fair point, getByRole is the right way to write Playwright tests and I worded that comparison badly. Thanks for calling it out.
What I actually care about is who maintains the thing. Even a well-written getByRole test is still code that someone has to own: page objects, assertions, a scripted path. In qpilot the only stored artifact is the plain text test case. Whatever maps steps to elements gets derived from the live page at runtime on every run, so there's no stored selector layer that can go stale.
Healer agents solve "my test code broke, fix it". I'm aiming at manual QA teams that never had test code in the first place. For them the realistic alternative isn't nicely written Playwright, it's clicking through the cases by hand every release.
The page says Playwright tests "breaks on selector/layout changes" but that this is more robust because it "reads the page like a human, via ARIA semantics". Any well-written e2e test should be using ARIA semantics to drive page object selectors, precisely because using other selector strategies are fragile. Playwright also has a test healer AI agent to solve this problem.
Fair point, getByRole is the right way to write Playwright tests and I worded that comparison badly. Thanks for calling it out.
What I actually care about is who maintains the thing. Even a well-written getByRole test is still code that someone has to own: page objects, assertions, a scripted path. In qpilot the only stored artifact is the plain text test case. Whatever maps steps to elements gets derived from the live page at runtime on every run, so there's no stored selector layer that can go stale.
Healer agents solve "my test code broke, fix it". I'm aiming at manual QA teams that never had test code in the first place. For them the realistic alternative isn't nicely written Playwright, it's clicking through the cases by hand every release.