The Quality Trail: October 2025 QA News
From the Desk of the Editor
Welcome to the October edition of the Quality Trail, your one stop shop for the latest news and information in the QA space. There’s a lot going on and we’re excited to jump right into it!
As always, let us know if you think we’ve missed something, or share the link with your colleagues or partners who may benefit from some or all of this information. You can also sign up to receive these testing updates via email.
– The QualityLogic Editorial Team
What’s Inside
Conferences & Events
- Pacific NW Software Quality Conference (PNWSQC) 2025: October 13 – 15, 2025 in Portland, at the University Place Hotel
- CypressConf 2025: October 21 – 24, 2025 (virtual)
- TestCon Europe: October 21 – 24, 2025 in the workshop venue at M.K. Čiurlionio St. 84 (Vilnius, Lithuania)
- STARCANADA: October 27 – 30, 2025 in Toronto at the Hyatt Regency
AI in QA
ROI of AI + Experts vs. Legacy QA
A recent analysis by Jason Arbon highlights just how dramatically AI is reshaping software quality assurance. Traditional QA (with layers of manual testing, SDETs, and slow test-case development) has long struggled to balance speed, cost, and coverage. In a 90-day head-to-head comparison, Arbon found that an AI + expert hybrid approach delivered equivalent or better coverage at roughly one-tenth the cost: about $19K compared to $121K for a conventional QA process.
The methodology goes something like this. Legacy QA teams start with weeks of preparation: read specs, write test cases, fix errors, plan execution, and only then is it time to run a first manual pass. Turning manual procedures into test cases as close as possible to 100% coverage takes a long time, and even then, they usually fall apart pretty quickly.
When we strategically add GenAI into the flow, it now goes something like this: AI starts by running thousands of checks and generating hundreds of tests, a QA expert reviews the AI’s tests and findings to validate flows and filter false positives, then explores edge cases, adds new prompts, and continues refining. Sometimes this is all in a day’s work.
The key is leveraging AI to generate rapid, broad test coverage while experienced QA professionals validate and refine the outputs. This “Four-Shot Flow” model reduces startup time-to-value from weeks to hours and scales effortlessly across releases. The result isn’t just higher ROI, but a redefinition of how quality teams can deliver more value, faster, by combining machine efficiency with human insight.
Playwright Announces Agents
Recently, Playwright dropped support for three agents:
- Planner explores your app and automatically produces a Markdown test plan
- Generator transforms the Markdown plan into runnable Playwright tests
- Healer executes the test suite and automatically debugs and fixes failing tests
This has the potential to speed up traditional workflows by several orders of magnitude. To try it out:
- Update to Playwright version 1.56
- Then every time you update Playwright, run one of the following commands, depending on your preferred IDE/editor:
- npx playwright init-agents –loop=claude
- npx playwright init-agents –loop=vscode
- npx playwright init-agents –loop=opencode
- From here, instruct the planner agent to generate the plan for a new feature, the generator to spit out code from the plan, and the healer to run and fix your tests
From an internal conversation we had at QualityLogic about the new functionality and the result it will have on Quality as a whole:
“The main thing is going to be about trust. The fact is tests need to be deterministic, and a lot of trust is put on the agent to do the lifting and be comprehensive. One of the biggest issues with a lot of AI generated content is that it is “broadly” right. In QA that could well result in a site being “broadly functional”.
On the bright side, more projects get more tests in less time. That’s fundamentally a good thing.
On the other hand, only time will tell whether it will lead to people reviewing test activities, overlooking issues, and then ultimately having higher bug rates. As with everything AI, the barrier to entry is now much lower, but the barrier to excellence is higher. Effectively, it has to be as good and thorough as what a person can do. It’ll be fascinating to see what it does when it runs into the QA buzz-saws that we’ve had to try to solve for.”
Articles and Resources
- Google launched a Chrome DevTools MCP server, which gives AI agents access to all of the features available in Chrome DevTools, thereby enabling them to debug webpages in real-time from within the browser.
- Playwright Agentic Coding Tips – Awesome Testing: This piece is an extremely well-rounded deep dive on agentic AI architecture, agents, billing (premium request vs token based pricing), and actionable tips for test generation using AI and Playwright.
- When AI chatbots leak and how it happens – Malwarebytes
- Hacking with AI SASTs: An overview of ‘AI Security Engineers’ / ‘LLM Security Scanners’ for Penetration Testers and Security Teams – Joshua Rogers:
- Testing AI features: from 0 to Test Strategy – Thiago Werner (Medium)
- Announcing OpenTest.AI: A New Community for Testers in the Age of AI – Jason Arbon (Medium)
What We’ve Been Reading
- 99 essential resources to help software testers – Ministry of Testing: Literally as it sounds, 99 different links to read and reference, irrespective of where you stand in your quality journey. There are a good number of links to MOT here, but some GitHub repositories, Medium articles, and YouTube videos as well.
- The Automation Maturity Pyramid. How effective is your automation test – David Ingraham (Medium)
- How modern browsers work – Addy Osmani (Elevate): Web developers often treat the browser as a black box that magically transforms HTML, CSS, and JavaScript into interactive web applications. This is hands-down the best developer-friendly explanation we’ve ever read on the topic, and it’s not even close.
- The Debug Trap: Why Smart Engineers Waste Hours on Trivial Problems – CNCF: Makes the point that when stuff fails, “your first instinct shouldn’t be to understand the problem. It should be to undo whatever caused it”, which goes against the compulsion engineers have to jump to the root cause.
- Stop Confusing Test Cases with Test Scenarios — You’re Wasting Everyone’s Time – Nureko Kustiarno Wibowo (Medium)
- The QA Document That Could Save Your Job – Nureko Kustiarno Wibowo (Medium)
Interested in More Information About QualityLogic?
Let us know how we can help out – we love to share ideas! (Or click here to subscribe to our monthly newsletter email, free from spam.)