Every release breaks something on our website.

Product teams that ship often and find out about broken pages, forms or checkouts from their customers.

Usually seen inTeams shipping weekly or faster

First testWhat broke in your last releases

PracticeAI

What you're probably seeing

If this sounds familiar, you're in the right place.

Customers report broken pages before anyone on the team notices.
Scripted end-to-end tests break whenever the interface changes, so people stop trusting them.
Manual testing before a release takes a day, and still misses things.
Nobody knows which flows matter most if something breaks.
Fixes for one bug break something else.

The usual ways out

What teams usually try. Each works somewhere. None works everywhere.

The way	Works when	The catch
Hire a manual tester	Releases are infrequent	Slow, and doesn't scale with release speed
Write more scripted tests	The interface is stable	They break on every redesign and miss what a person would notice
Record-and-replay tools	You need quick coverage of a few flows	Brittle when the page changes
An agent that uses the site like a person	Flows change often and you need judgement, not exact clicks	Needs clear pass and fail rules, or it guesses
Monitoring and error tracking in production	You can tolerate finding bugs after release	Tells you after customers are hit
Slow down releases	Stability matters more than speed this quarter	Bugs still ship, just less often
Move checks into unit and component tests	Most bugs sit inside one service or component	Misses what breaks only when everything runs together

What other companies did. Their published results, not ours.

We read each company's own write-up and asked one question: would our diagnostic's tests have led to the same call? We judge the approach, not the result.

Company	What they did	What they published	Source
SlackHundreds of internal services; by mid-2023, 73% of customer-facing incidents were triggered by Slack's own changes, mostly code deploys.	Tracked one metric, hours of customer impact from change-caused incidents, and moved backend deploys step by step from metric monitoring to alerts with manual rollback, then automatic deploys and rollback.	Customer impact hours fell about 90% from the peak quarter (February–April 2024) to January 2025.They noted: Improving manual fixes did much less than automation, and results took 3–6 months to show in the incident data.	Deploy Safety: Reducing customer impact from changeOct 2025
FundaA large property website moving from a monolith with many external dependencies to microservices.	After a year of fighting flaky Selenium tests, ran a month-long experiment with them switched off, moved checks into unit and component tests, and adopted Playwright tests against a local server with external calls mocked.	Over that year, 30–40% of the Selenium tests failed due to flakiness; in the month-long experiment Selenium caught 4 failures in team environments, and switching it off on acceptance caused no production failures.They noted: Their Playwright tests are closer to component tests than true end-to-end tests, and they accepted that trade-off.	Game changer: why we ditched Selenium for PlaywrightOct 2022
UberThousands of developers and over 3,000 simultaneous experiments; engineers working on test scripts spent 30–40% of their time on maintenance, as small UI changes kept breaking them.	Built DragonCrawl, a small language model that reads the screen and picks actions to reach a test goal, such as completing a trip, with guardrails that re-prompt, backtrack and detect loops.	99%+ stability in November and December 2023; it completed trips in 85 of 89 top cities and blocked ten high-priority bugs in its first three months.They noted: In some cities it chose scheduled rides instead of the intended product, and new pop-ups sent it down long paths.	DragonCrawl: Generative AI for High-Quality Mobile TestingApr 2024
PocusAn 11–50 person sales-intelligence company whose frontend tests were all in Playwright.	Replaced Playwright with Momentic, whose tests are written in natural language and find page elements from context, and runs them in CI before every production release.	Vendor-reported: 100% of production deployments validated before release, and 20 times less test flakiness than with Playwright.They noted: Before the switch, UI changes broke locators, so tests failed even when the flows worked.	How Pocus validates every deployment with MomenticApr 2026 · vendor story

Our read

Same call. Each team first looked at what was actually breaking and how it was found, then chose a fix to match. That's the order our tests follow.

Slack started from incident data showing most customer-facing incidents came from its own changes, and chose monitoring with automatic rollback. Funda counted what its Selenium suite really caught before retiring it. Uber and Pocus found scripted tests failing on UI changes rather than real bugs, and moved to goal-based tests. We would still check that an agent can reach and judge each money-making flow, since Uber's post doesn't state clear pass and fail rules.

What we would test first: List what broke in your last ten releases, and how each problem was found.

What the diagnostic would test

The tests that rule the wrong ways out.

On your own systems and data, read-only first. The number of tests sets the diagnostic's price, quoted before it starts.

1
The flows that make money or support customers, ranked.
2
What broke in your last ten releases, and how it was found.
3
How your current tests fail: real bugs, or changed buttons.
4
Whether an agent can reach and judge each flow, including login and payment test modes.
5
Where results should land: the pull request, Slack, or a release checklist.

This week, without us

Three things you can do yourself.

1
List your five most important user flows, and click through them yourself before the next release.
2
Go through the last ten bug reports and note how each one was found.
3
Mark which of your existing tests failed because of a real bug, and which because the page changed.

Sound like you? Bring it to a 30-minute call. We'll tell you which tests your case needs, and what they cost.

Back to every case

This page describes what teams in this situation commonly face. It is not a client story.