Our developers ship far more code with AI now. Can QA keep up without becoming the release bottleneck?

Only if the engagement is priced against your release cadence rather than a test count. DORA's State of AI-assisted Software Development 2025 found AI adoption positively related to throughput and product performance but persistently negatively related to delivery stability, which is the bottleneck moving downstream rather than disappearing. In practice that means fast automated gates on every pull request, test impact analysis and sharding so CI cost does not scale with commit volume, quarantine for flaky tests, and human hours spent on exploratory and risk-based testing rather than script execution.

Could your AI-generated or self-healing tests silently hide a real regression?

They can, which is why the healer runs under a written policy rather than on trust. Self-healing locators cope well with incremental UI change but fail completely on redesigns that rebuild the information architecture. Locator and DOM change is also only one cause of flakiness among several, with async and synchronisation issues, race conditions, test-data drift and environment variability making up much of the rest. We define up front what a healer may change without review, what is escalated to a human, and how each healed test is audited in the pull request that carries it.

Who owns the test suite after the engagement ends?

You do, and it is standard code rather than a cloud artefact. We deliver Playwright and Appium projects that live in your repository and run in your CI, so nothing depends on our account or a vendor subscription staying alive. That matters more in 2026 than it used to, given that Promptfoo has been acquired by OpenAI and LambdaTest rebranded to TestMu AI in January 2026. We also state the per-release maintenance hours and name who maintains the suite after handover.

How do you test an LLM or agent feature that has no single correct answer?

With an eval suite that has score thresholds wired into CI as a gate, not with string assertions or golden-output snapshots. That typically means DeepEval or RAGAS for quality and groundedness thresholds, promptfoo or DeepTeam for adversarial red-teaming mapped to the OWASP Top 10 for LLM Applications 2025 and the OWASP Top 10 for Agentic Applications 2026, and a tracing platform for production monitoring. An automated judge is a regression floor rather than a replacement for people: a 2026 study of a deployed multi-turn ordering agent found the judge caught 2 of 9 problem patterns in one batch and none in a batch where humans confirmed 23 defects. We run judges at temperature zero and name a human review sampling rate in the scope.

What accessibility standard do you test against, and is an axe scan enough?

We test to WCAG 2.2 AA. The European Accessibility Act has been enforceable since 28 June 2025, and the EN 301 549 revision reached final draft as v4.1.0 in June 2026. Once it is cited in the EU Official Journal it moves the baseline from WCAG 2.1 AA to 2.2, adding six A and AA success criteria including Target Size Minimum and Accessible Authentication. An automated scan is not enough on its own: Deque's study across more than 13,000 pages and nearly 300,000 issues puts axe-core at 57.38% of issues detected by volume, so the remainder needs manual keyboard and screen-reader testing. We deliver a conformance report, and we do not install overlay widgets, which do not satisfy EN 301 549.

QA and QC Testing Services for Web, Mobile and AI Features

QA and QC is the work of deciding what could go wrong, proving whether it does, and reporting that in a form somebody can act on before release. Mixcore Studio has been doing that work for more than eight years across 320+ delivered projects, with a team of around twenty. This section covers two connected practices, manual and exploratory testing and automation testing, plus the newer work of testing AI features that have no single correct output.

Testing has changed more in the past two years than in the decade before it, and a QA page that still reads the way ours did in 2021 would be hiding that. AI assistants raised the volume of code arriving at the release gate, agentic test authoring moved inside the open-source frameworks, accessibility became enforceable law across the European Union, and non-deterministic features arrived without deterministic assertions to check them. What follows is where that leaves an engagement, including the parts we tell clients not to buy.

The commercial case for QA now rests on measurement rather than a slogan. DORA's State of AI-assisted Software Development 2025, published by Google Cloud, reports that 90% of respondents use AI at work and over 80% say it increased their productivity, while AI adoption shows a positive relationship with delivery throughput and product performance and a persisting negative relationship with delivery stability. Stack Overflow's Developer Survey 2025 found only 3.1% of respondents highly trust the accuracy of AI output against 45.7% who distrust it (n = 33,244), with "AI solutions that are almost right, but not quite" the top frustration at 66%. A METR randomised controlled trial published on 10 July 2025 found 16 experienced open-source developers working on 246 real issues took 19% longer with AI tools, after expecting a 24% speed-up and still believing afterwards they had been sped up by 20%. Perceived speed and actual speed came apart, which is exactly the condition verification exists for. AI does not reduce the need for testing; it moves the bottleneck downstream and shifts the human hours from script execution towards exploratory, risk-based and domain testing.

Playwright is the default runtime, and the gap is not close

npm registry download figures for the week of 3 to 9 August 2026 put playwright at 80,091,572 weekly downloads and @playwright/test at 52,792,524, against puppeteer at 11,502,491, cypress at 7,360,936, webdriverio at 3,142,779, selenium-webdriver at 2,045,892 and testcafe at 215,701. That is an order of magnitude, not a contest. We build browser suites on Playwright by default and treat anything else as a decision that needs a stated reason.

Playwright 1.62.1 — released 30 July 2026. The 1.62 line adds a component-testing stories and galleries model with fixtures.mount(), AbortSignal support, lossless WebP screenshots, an isolated retry strategy and reporter.preprocess(), and it bundles the Playwright MCP server and CLI. Recent releases also brought WebAuthn passkey virtual authenticators and the Web Storage API (1.61), locator.drop(), tracing.startHar() and ARIA snapshots with bounding boxes (1.60).
Cypress 15.20.1 — released 10 August 2026 and still actively shipping, with per-axis scrollBehavior and HTTP QUERY support in cy.request() and cy.intercept(). It is a developer-experience preference in 2026 rather than a default.
Selenium 4.47 — released 10 August 2026. Its release notes record that CDP access with Firefox is now blocked in the .NET, Python and Ruby bindings, reinforcing WebDriver BiDi as the standard cross-browser protocol. A Selenium 5 release charter is also published: its stated purpose is cross-binding convergence, with a full classic-over-BiDi migration and any deprecation of CDP support explicitly out of scope.
WebdriverIO 9.30.1 and Appium 3.6.0 — released 3 August and 25 July 2026. WebdriverIO ships @wdio/mcp as a single integration point for web and mobile, and Appium remains the base for native mobile automation, with Maestro CLI 2.8.0 (31 July 2026) for lighter UI flows.

Four things here are migration debt rather than live choices. The Chrome DevTools Protocol is no longer a safe cross-browser automation transport, with Selenium 4.47 blocking CDP access against Firefox in three of its bindings and WebDriver BiDi the standards-track replacement. Protractor has been dead as Angular's end-to-end tool for years, so any surviving suite is a port waiting to happen. TestCafe is still maintained at 3.7.6 but has no meaningful ecosystem left at its current download volume. And selector suites built on XPath or CSS strings with no test IDs are worth replacing: role- and accessibility-tree locators such as getByRole and ARIA snapshots are more stable under refactoring and far easier for an AI agent to reason about.

Agentic test authoring moved into the frameworks

The clearest structural change of 2026 is that AI test generation stopped being a product category and became a framework feature. Playwright ships first-party Test Agents, installed with npx playwright init-agents and a loop flag naming the coding client, supporting VS Code 1.105 and above, Claude Code, Codex and OpenCode. There are three. A planner explores the application and produces a Markdown test plan. A generator turns that plan into Playwright test files while verifying selectors against the live application. A healer runs the suite, repairs failing tests, and can skip a test when it judges the functionality genuinely broken. Chrome DevTools MCP has done the equivalent for debugging since its public preview on 23 September 2025, and WebdriverIO's @wdio/mcp covers web and mobile. Buying AI test generation as a separate subscription is a harder case to make than it was a year ago.

The healer is also the sharpest objection buyers raise, and they are right to raise it. A healer that quietly rewrites a broken selector can bury a real regression. Self-healing locators cope well with incremental UI change, and they fail outright on redesigns that rebuild information architecture and component hierarchy, because there is no fingerprint left to match. More to the point, locator and DOM change is only one cause of flakiness among several. Async and synchronisation issues, concurrency and race conditions, test-data drift and environment variability account for much of the rest, and no amount of selector healing touches any of it.

So the policy is written down before the first agent runs: what the healer may change without review, what is escalated to a human, and how every healed test is audited in the pull request that carries it. Agent definitions must be regenerated on each Playwright upgrade, which makes this a standing maintenance item rather than a one-off setup.

Testing AI features is a separate discipline

If a feature calls a model, deterministic string assertions and golden-output snapshots do not test it, they only fail intermittently. The 2026 approach is an eval suite with score thresholds wired into CI as a gate, plus adversarial red-teaming, and the tooling has consolidated into a recognisable split: promptfoo 0.122.0 and DeepTeam for whether an attacker can break it, DeepEval 4.1.8 and RAGAS for whether it meets a quality threshold, and Braintrust, LangSmith, Langfuse or Arize Phoenix for trace-level production monitoring and human annotation. OpenAI announced its acquisition of Promptfoo on 9 March 2026, with the open-source project committed to continuing under its new owner, which settled the question of whether evals and red-teaming are strategic infrastructure or a niche.

Red-teaming maps to a published taxonomy rather than improvisation. The OWASP Top 10 for LLM Applications 2025 covers the model layer, from LLM01 Prompt Injection to LLM10 Unbounded Consumption. The OWASP Top 10 for Agentic Applications 2026, published on 9 December 2025 with more than 100 contributors, covers agents: goal hijack, tool misuse, identity and privilege abuse, agentic supply chain compromise, unexpected code execution, memory and context poisoning, insecure inter-agent communication, cascading agent failures, human-agent trust exploitation and rogue agents. The companion Solutions Landscape red-teaming taxonomy followed on 28 June 2026.

On LLM-as-a-judge the honest position is that it is a regression floor, not a replacement for human review, and the evidence for that is now specific. Norman, Rivera and Hughes (arXiv:2606.19544, June 2026) evaluated 21 judges from nine providers across 118 runs and roughly 541,000 individual judgments, and found that exact-match agreement overstates discriminative ability by 33 to 41 percentage points on MT-Bench against chance-corrected Cohen's kappa, that two production-deployed judges combined test-retest reliability above 0.95 with position bias above 0.10, and that judge rankings shift by up to 14 positions across benchmarks. A separate 2026 study of a deployed food and beverage ordering agent (arXiv:2606.10315) found the automated judge caught 2 of 9 problem patterns in one batch and flagged nothing at all in a batch where human reviewers confirmed 23 distinct defects, missing cross-turn state failures such as cart hallucination and confirmation-gate lockout entirely. Its conclusion is worth quoting directly: "For production multi-turn agents, automated judging is a regression floor, not a substitute for human review." We run judges at temperature zero, because same-verdict rates fall materially as temperature rises, report chance-corrected agreement rather than raw agreement, and name a human review sampling rate in the statement of work.

Accessibility is a procurement gate, and the bar rises this year

The European Accessibility Act (Directive (EU) 2019/882) has had enforcement powers active since 28 June 2025, which makes 2026 the first full year in which national authorities supervise against it. It applies to consumer-facing digital services sold into any member state regardless of where the business is headquartered, penalties are set by national law rather than EU-wide, and most authorities issue a remediation order before financial penalties. The cited technical baseline today is EN 301 549 v3.2.1, which incorporates WCAG 2.1 AA, and the revision went to public comment as draft v4.1.0 in November 2025 and reached final draft in June 2026, now in the ETSI vote phase. It updates clauses 9, 10 and 11 to align with WCAG 2.2, and confers presumption of conformity once cited in the EU Official Journal. Six A and AA success criteria enter scope: Consistent Help (A), Redundant Entry (A), Focus Not Obscured Minimum (AA), Dragging Movements (AA), Target Size Minimum at 24 by 24 CSS pixels (AA) and Accessible Authentication Minimum (AA). WCAG 4.1.1 Parsing drops out. Testing to WCAG 2.1 AA today books a re-audit, so we test to 2.2 AA by default and name the target in the scope.

Meanwhile the web itself got worse. WebAIM Million 2026 found 95.9% of the top one million home pages had detected WCAG failures, up from 94.8% in 2025, with an average of 56.1 errors per home page, a rise of 10.1% on the 51 recorded in 2025. Six error types account for 96% of all detected issues: low-contrast text on 83.9% of pages, missing image alt text on 53.1%, unlabelled form inputs on 51%, empty links on 46.3%, empty buttons on 30.6% and missing document language on 13.5%. Pages using ARIA averaged 59.1 errors against 42 on pages without it, which is a useful reminder that ARIA added without testing makes things worse rather than better.

Automation alone will not get you there. Deque's coverage study, measured across more than 13,000 pages and nearly 300,000 issues from first-time audits, puts axe-core at 57.38% of accessibility issues detected by volume. That is higher than the conventional 20 to 30% figure, which counts success criteria rather than actual issue volume, but it still leaves close to half the work to manual keyboard and screen-reader testing. We embed @axe-core/playwright 4.13.0 assertions in the suite so regressions fail the build, test the remainder by hand, and issue a conformance report rather than a scan output. We do not install accessibility overlay widgets. They do not satisfy EN 301 549 conformance and they do not survive a complaint-triggered investigation.

Performance, flake rate and the CI bill

Performance testing has consolidated on code-first tools that measure protocol load and front-end experience in one run. Grafana k6 reached v2.0.0 on 11 May 2026 and v2.2.0 on 10 August 2026, with a browser module capturing Web Vitals alongside HTTP virtual users. If you are upgrading from v1, note what v2.0.0 removed: the externally-controlled executor, k6/experimental/redis, the login, pause, resume, scale and status commands, and options.ext.loadimpact.

Single-page applications finally became measurable. Chrome is enabling the Soft Navigations API by default from Chrome 151, which reached stable in July 2026 after a period behind a flag and an earlier origin trial, so LCP, INP, CLS and FCP can be observed per soft navigation through PerformanceObserver and the web-vitals library from v6.0.0. How soft navigations will be reported in CrUX is explicitly still to be determined, so lab and field numbers will disagree for a while and we always state which one a figure came from. Core Web Vitals remain LCP, INP and CLS, with the Long Animation Frames API as the standard tool for INP attribution. Any performance report still quoting First Input Delay is stale; INP replaced it in March 2024.

Flake management became a product category in its own right, with Trunk Flaky Tests, Datadog Test Optimization and BuildPulse competing on automatic detection, quarantine out of the release-blocking path, and accounting for CI time wasted per test. Running a full regression suite on every commit with no sharding, no test impact analysis and no quarantine is no longer defensible on either cost or trust. An engagement in 2026 should hand you a measured flake rate, a written quarantine policy and a cost per release, not a test count and a coverage percentage. The outcome metrics we report against are escaped-defect rate, change failure rate, mean time to restore, and field Core Web Vitals at the 75th percentile.

What we will tell you not to buy

A codeless tool that does not emit portable code — if the output cannot be exported as standard Playwright or Appium and run in your own CI, the suite dies with the contract. Ownership and portability are the first things we put in writing.
An AI judge as the only quality signal — the published evidence above is clear enough. A judge with no human review layer is a defensible-practice failure, not a saving.
Full end-to-end automation of a UI that is still changing weekly — before the interaction design settles, exploratory testing plus a thin smoke suite finds more defects per hour and costs far less to maintain. Automating later and deeper is usually the cheaper path.
A separate AI test-generation subscription — Playwright's Test Agents, the bundled Playwright MCP server and Chrome DevTools MCP now cover most of what that category sold in 2025. Managed services and device clouds still have real reasons to exist; AI generating the tests is no longer one of them on its own.

Two further points worth stating plainly. Vendor churn in this market is real: Promptfoo has been acquired by OpenAI, and LambdaTest rebranded to TestMu AI on 12 January 2026 with platform, infrastructure, accounts and integrations unchanged. We prefer stacks where the artefact you own outlives the vendor. And data exposure through testing tooling deserves a decision rather than a default. Production-like fixtures, screenshots, session recordings and prompts flowing to third-party AI testing services and hosted LLM judges need a written answer on where traces are stored, how long they are retained, and whether anything is used for training.

What this section covers

Manual and exploratory testing
Playwright and Appium automation
LLM eval suites and CI gates
Security and adversarial red-teaming
WCAG 2.2 AA accessibility testing
Performance and Core Web Vitals

How a QA engagement runs

Three phases, each ending in something measurable rather than a status slide. The flake rate, the accessibility target and the healer policy are agreed at the start, not discovered at handover.

01

Risk model and baseline

We map what actually breaks in your product and what it costs when it does, then measure the starting position, which means the current flake rate, CI minutes wasted per release, an accessibility baseline against WCAG 2.2 AA, and field Core Web Vitals at the 75th percentile. Scope is written against your release cadence, not a target test count.
02

A suite you own

Playwright for browser, Appium or Maestro for native mobile, role- and accessibility-tree locators rather than XPath strings, @axe-core/playwright assertions embedded in the run, and eval thresholds through promptfoo or DeepEval where a feature calls a model. Everything is standard code in your repository that runs in your CI. Where Playwright Test Agents author or heal a test, the written policy says what they may change and what escalates to a human.
03

Gates, evidence and handover

Sharding and test impact analysis so CI cost stays proportionate, automatic quarantine to keep flaky tests out of the release-blocking path, red-teaming mapped to the OWASP LLM and Agentic Top 10s, and an accessibility conformance report covering manual keyboard and screen-reader testing. We name who maintains the suite afterwards and what the per-release maintenance hours look like before the engagement ends.

Frequently asked questions

Our developers ship far more code with AI now. Can QA keep up without becoming the release bottleneck?

Only if the engagement is priced against your release cadence rather than a test count. DORA's State of AI-assisted Software Development 2025 found AI adoption positively related to throughput and product performance but persistently negatively related to delivery stability, which is the bottleneck moving downstream rather than disappearing. In practice that means fast automated gates on every pull request, test impact analysis and sharding so CI cost does not scale with commit volume, quarantine for flaky tests, and human hours spent on exploratory and risk-based testing rather than script execution.
Could your AI-generated or self-healing tests silently hide a real regression?

They can, which is why the healer runs under a written policy rather than on trust. Self-healing locators cope well with incremental UI change but fail completely on redesigns that rebuild the information architecture. Locator and DOM change is also only one cause of flakiness among several, with async and synchronisation issues, race conditions, test-data drift and environment variability making up much of the rest. We define up front what a healer may change without review, what is escalated to a human, and how each healed test is audited in the pull request that carries it.
Who owns the test suite after the engagement ends?

You do, and it is standard code rather than a cloud artefact. We deliver Playwright and Appium projects that live in your repository and run in your CI, so nothing depends on our account or a vendor subscription staying alive. That matters more in 2026 than it used to, given that Promptfoo has been acquired by OpenAI and LambdaTest rebranded to TestMu AI in January 2026. We also state the per-release maintenance hours and name who maintains the suite after handover.
How do you test an LLM or agent feature that has no single correct answer?

With an eval suite that has score thresholds wired into CI as a gate, not with string assertions or golden-output snapshots. That typically means DeepEval or RAGAS for quality and groundedness thresholds, promptfoo or DeepTeam for adversarial red-teaming mapped to the OWASP Top 10 for LLM Applications 2025 and the OWASP Top 10 for Agentic Applications 2026, and a tracing platform for production monitoring. An automated judge is a regression floor rather than a replacement for people: a 2026 study of a deployed multi-turn ordering agent found the judge caught 2 of 9 problem patterns in one batch and none in a batch where humans confirmed 23 defects. We run judges at temperature zero and name a human review sampling rate in the scope.
What accessibility standard do you test against, and is an axe scan enough?

We test to WCAG 2.2 AA. The European Accessibility Act has been enforceable since 28 June 2025, and the EN 301 549 revision reached final draft as v4.1.0 in June 2026. Once it is cited in the EU Official Journal it moves the baseline from WCAG 2.1 AA to 2.2, adding six A and AA success criteria including Target Size Minimum and Accessible Authentication. An automated scan is not enough on its own: Deque's study across more than 13,000 pages and nearly 300,000 issues puts axe-core at 57.38% of issues detected by volume, so the remainder needs manual keyboard and screen-reader testing. We deliver a conformance report, and we do not install overlay widgets, which do not satisfy EN 301 549.