What we asked and what we found
Every entry answers the same eight questions, in this order.
Question
Why did Sales Desk pass its automated tests and its build, and still fail in a rep's hand?
Why it matters
The tests measured the code. The rep measured the product. Layout, tap targets and how a page fits a phone screen are not in an automated test.
State
In use Running in our own work and in client work. Delivered through Lehvel engagements, not as self-serve software.
What we built or tested
- A check that opens the pages as a phone would, at real phone widths (360 to 430 pixels), run before release beside the tests.
Evidence
- Finding
- Defects surfaced only in rendered mobile QA, after green unit tests and a green build
What we learned
- Name the check that would catch the failure you care about, then run that one.
- Our team judges on an iPhone. So does the customer.
Limitations
- A QA script proves what it looked at. It does not prove the pages it did not open.
What changes next
Run the same phone-width check on every public page in this redesign.
Failed tests are published as results. A state on this page describes today, not plans.
Next in Labs
- Next: An automated caller asked a building-products supplier for a window price
One reported call. The questions that followed were not about the technology. They were about who the caller worked for and whether anything it asked for should ever turn into an order.
Field notes
Get Upstream
One email a week: what we asked the AI, what it got wrong, and how we checked.
Our research reaches 17,000+ followers across our channels. Upstream is the weekly version.
Weekly. Unsubscribe anytime. Privacy