Answer
Sessions where a real founder or marketer uses the product while we watch; none has ever been run, so this page holds a proposed first round for Param to accept, and lists the expert reviews done instead.
Today
Sessions run
- Usability sessions with founders or marketers: 0 brand-directions/book/content/d2.js d-12 what-we-dont handbook research-and-insights/interview-and-call-library.md
What we did instead: reviews by our own team
These are QA walks and code reads by our team and AI auditors. They found real problems, but none watched a founder use the product.
| Date | Review | What it found | What was fixed | Source |
|---|---|---|---|---|
| 15 to 16 Oct | Full audit | Outsiders could read a brand's read; Slack actions and GA4 ownership open; Cozy written for India though it sells in the US | Leaks closed, live 00196 | memory brew-audit-2026-10-15 |
| 19 to 20 Oct | Sai's sheet, The Feed QA Master | 49 bugs | All fixed and live 00405, about 35 half fixes closed after a compare pass | memory brew-qa-sweep-2026-10-19 |
| 20 Oct | Four auditors on 11 brands | About 86 findings | P0s and most P1s live by 00447; 3 left open | memory brew-qa-audit-2026-10-20 |
| 23 to 25 Oct | Round 2 and 4 on Arata | Sections A, B and C approved, 3.6 dropped; round 4: 32 pass | Live from 00655; 25 Oct recheck: 25 pass, 4 fixed, 3 pass with a note, 1 dropped | memory brew-qa-round2-2026-10-24 |
| 24 Oct | Design partner audit of the five | 9 findings across read, numbers, PDFs and logos | Each fixed with a guard | brew/docs/audit-2026-10-24-design-partners.md:1-20 |
| 25 Oct | Code read of origin/main b68aa9f | 89 surfaces; 16 ways into a jam; about 11 screens and up to 14 decisions before the first card; six skip labels | Fed the redesign brief | brand-directions/_research/product-audit.md:7 brand-directions/_research/product-audit.md:177 brand-directions/_research/product-audit.md:255 |
Ten places a first time founder may get lost, from code only
These are guesses to test, not findings about people brand-directions/_research/product-audit.md:322-335.
- Signing in before anything happens. brand-directions/_research/product-audit.md:324
- A long onboarding that asks twice. brand-directions/_research/product-audit.md:325
- The check page under a clock. brand-directions/_research/product-audit.md:326
- A crowded first feed screen. brand-directions/_research/product-audit.md:327
- Three numbers for today. brand-directions/_research/product-audit.md:328
- Where things live. brand-directions/_research/product-audit.md:329
- Three different making spaces. brand-directions/_research/product-audit.md:330
- What Send brief does. brand-directions/_research/product-audit.md:331
- Seats and Viewing as. brand-directions/_research/product-audit.md:332
- Making costs and outside names before a result. brand-directions/_research/product-audit.md:333
Tomorrow
The proposed first round, from the old book's test plan. The scripts were written for the redesign's screens, so the task words stay and the screen names change to the live build.
| Test | What it answers | Pass | Version | Tag |
|---|---|---|---|---|
| Instant understanding: Today shown for five seconds | Do they get what it does? | Says ideas or marketing, drafted, for my store | v1 | New |
| Script 1, the first ten minutes | Where does paste to first hand off stall? | First hand off without help | v1 | New |
| Script 2, a Monday return | Do five picks feel finishable? | Says the week is done | v1 | New |
| Script 3, make it and hand it off | Is it clear who got the brief? | Says who has it and that nothing was posted | v1 | New |
| Script 4, the person who ships it | Can they work from the brief page with no login? | Knows what to make by when | v1 | New |
| Script 5, what it knows about you | Can they find and fix a wrong fact? | Fixes one fact and sees it saved | v1 | New |
| Script 6, every way in | Do Shopify, invite and return land on Today? | Each lands without help | v1 | New |
How we do it
- Recruit five people for a round. Five is a team choice for speed, not a sourced number.
- Use their own phone and their own store where possible.
- Run 30 minutes:
| Minutes | Part |
|---|---|
| 0 to 2 | Consent and warm up |
| 2 to 4 | The instant understanding test |
| 4 to 24 | One script |
| 24 to 29 | Debrief, three questions |
| 29 to 30 | Thanks |
- Read the consent line: "We are testing the product, not you. You can stop at any time. We record the screen and your voice only to take notes, and delete the recording after the notes are written. Is that all right?"
- Moderator rules: never explain the screen; answer a question with "What would you expect?"; note the first tap and every pause over five seconds; never fix a mistake for them.
- Score pass or fail per signal the same day, and write one line per finding with a clip or a quote given with permission.
- Log the session in the interview and call library, findings in the insight repository, and fixes as product for every brand.
Numbers
| Metric | Definition | Where measured | Status |
|---|---|---|---|
| Sessions run | Usability sessions with a dated row | Interview and call library | 0 today |
| Instant pass rate | Share of people who pass all four instant signals | Session scores | Proposed; no target in any source |
| Minutes to first hand off | Median from paste to first hand off in script 1 | Session timer | Proposed |
| Long pauses | Pauses over five seconds per screen | Session notes | Proposed |
Open
- Do we run the first round? Proposal: yes, one person from each of Arata, Nutrova, Mana, Puma India and Scrubroom, in the two weeks from 2 Nov, inside the weekly call slot.
- Live build or the redesign screens? Proposal: the live build first, so the redesign has a baseline to beat.
- Which scripts first? Proposal: the instant test plus script 1 for every person, script 2 a week later.
- What is the pass bar? Proposal: four of five people pass all four instant signals.
- Who moderates? Proposal: Sai moderates, Param watches with the camera off and says nothing.