{
 "id": "5.5",
 "title": "The read",
 "department": "Engineering",
 "department_slug": "engineering",
 "direction": null,
 "branded": false,
 "status": "draft",
 "owner": "Param",
 "last_verified": "Mon Oct 26",
 "holds": "Stages in order, what each reads, healing, versions",
 "urls": {
  "html": "https://handbook.joinsquareup.com/engineering/the-read",
  "markdown": "https://handbook.joinsquareup.com/engineering/the-read.md",
  "json": "https://handbook.joinsquareup.com/engineering/the-read.json"
 },
 "other_directions": [],
 "tldr": {
  "headline": "A pasted store address becomes a sourced profile in seven stages",
  "points": [
   {
    "icon": "eye",
    "text": "Stages run in order: open, brand, catalog, reviews, ads, site, answers."
   },
   {
    "icon": "clock",
    "text": "Each stage has a deadline and one retry; paid answers come from cache."
   },
   {
    "icon": "check",
    "text": "Check what we read waits up to 15 minutes for the team's corrections."
   },
   {
    "icon": "lock",
    "text": "A brand's first read writes Plays only after someone opens the feed."
   },
   {
    "icon": "book",
    "text": "Bump a stage version whenever its output shape changes, so saved reads heal."
   }
  ],
  "stat": {
   "value": "15",
   "label": "minutes the check waits"
  },
  "decide": "What does a read cost on Gemini today? Proposal: measure the next Cozy re-read.",
  "mood": "building"
 },
 "sources": [
  "brew origin/main: reader/index.js:1-40, 115-140, 181-310; reader/stages/*.js (version constants); reader/adlib.js; reader/products.js",
  "brew origin/main: build.js:302-345 (run: read, check, brief, feed gate), 779-860 (heal, checkData), 1600",
  "brew origin/main: treg.js:8-28; SPEC.md (profile, ledger, onboarding stream); CLAUDE.md 'Every brand, every platform'",
  "brew origin/main: docs/handover/engineering/02-feature-audit.md section 1, 05-ai-and-external-data.md (15 Oct)",
  "brew origin/main: docs/audit-2026-10-24-design-partners.md",
  "memory brew-checkout-walk, brew-read-stream-backfill, brew-feed-open-moves, brew-apify-watch, brew-shopify-app-review, brew-external-data-skill, feed-for-all-brands"
 ],
 "open_questions": [
  "**What does a read cost on Gemini today?** Proposal: measure the next Cozy re-read and record it here.",
  "**Should a signed-out visitor still start a read?** Today non-official emails cannot start a new read. Proposal: keep it so; the demo is the place for prospects. [src: memory brew-access-rules]"
 ],
 "parts": {
  "Answer": "The read turns a pasted store address into a sourced profile and ledger through seven visible stages with deadlines and one retry each, and this page settles the stage order, what each one reads, how saved reads heal and when a stage version must be bumped.",
  "Today": "### The flow around the read\n\n1. `POST /api/brands {url}` starts a build. A first read writes the profile, ledger and findings. [src: brew/build.js:310-322]\n2. If the store did not open, the build stops before spending more and says so (or says the store is behind Shopify's password page). [src: brew/build.js:316-318]\n3. \"Check what we read\" waits for the team's corrections. A signed-out read waits with no time limit; otherwise Brew goes on at `writeAt` (`BUILD_CONFIRM_WAIT_MS`, 15 minutes on staging and production). [src: brew/build.js:327-340] [src: brew/deploy.sh:39]\n4. The brief is written, then a brand's first read waits until someone opens the feed before writing Plays (`feedGate`, `BUILD_FEED_GATE=0` turns it off). Reminders go out at 3 hours and 24 hours. [src: brew/build.js:356-362] [src: memory brew-feed-open-moves]\n\n### Stages in order\n\n| Step | Shown as | Deadline | What it reads | Version | Source |\n|---|---|---|---|---|---|\n| open | Open the store | 45 s, one try | platform, market, pages, sitemaps, headless Shopify | | [src: brew/reader/index.js:27-36] [src: brew/reader/stages/open.js] |\n| brand | Read the brand | 40 s | logo, colours, fonts, one-liner | | [src: brew/reader/stages/brand.js] |\n| catalog | Pull the catalog | 60 s | products on any platform, assortment (own label or resold) | `CATALOG_V` 2, `ASSORT_V` 2 | [src: brew/reader/stages/catalog.js:11] [src: brew/assortment.js:6] |\n| reviews | Read what customers say | 75 s | review apps on the site, marketplace ratings | `REVIEWS_V` 2 | [src: brew/reader/stages/reviews.js:32] |\n| ads | Find live ads, yours and rivals | 240 s (rivals 110 s, 30 s per rival price) | Meta Ad Library via treg, free headless fallback; five rivals | `ADS_V` 5 | [src: brew/reader/stages/ads.js:18] [src: brew/reader/adlib.js] |\n| site | Walk the path to checkout | 200 s | headless checkout walk, tags that fire, PageSpeed | `SITE_V` 4 | [src: brew/reader/stages/site.js:75] |\n| answers | Ask the AI tools | 170 s | ChatGPT, Gemini, Perplexity answers via treg; category; named check (100 s) | | [src: brew/reader/stages/answers.js] [src: brew/reader/stages/named.js] |\n| (no step) socials, company | Films card, founders and press | 120 s, 150 s | Instagram, YouTube (found by search when the site links none), about page | `SOCIALS_V` 2, `SOCIAL_SEARCH_V` 1, `COMPANY_V` 3 | [src: brew/reader/stages/socials.js:6-8] [src: brew/reader/stages/company.js:18] |\n\nDeadlines are `LIMIT` in reader/index.js. A stage that fails or times out gets one more try; paid calls that already answered come back from cache. A timed-out stage keeps running but can no longer emit. Then the step says plainly what happened and the read goes on. [src: brew/reader/index.js:35, 121-140]\n\nA store behind Shopify's password page is read from the Admin API (`reader/stages/admin.js`); the stages that need the public site say \"not checked\". [src: brew/reader/index.js:115-117] [src: memory brew-shopify-app-review]\n\nUnder Gemini the \"Claude\" AI-visibility ask is skipped, so Gemini is never labelled Claude. [src: memory brew-gemini-provider]\n\n### Outside data rules\n\n- Every treg call goes through `treg.js`, cached by signal: company data 30 days; reviews, search, AI answers and profiles 7 days; ads, news and prices 1 day; generation never cached; failures never cached. [src: brew/treg.js:12-27]\n- Paid sources fall back to free ones (Ad Library: `adCall`). [src: brew/CLAUDE.md]\n- An error is never shown as \"none\" or \"0\"; it keeps `error` and says the check did not go through. [src: brew/CLAUDE.md]\n- treg first, Apify only where treg has nothing or is weak; Apify capped at $10 a brand a month. [src: brew/SCOPE.md] [src: memory brew-apify-watch]\n- Every change here starts by loading the `brew-external-data` skill. [src: brew/CLAUDE.md]\n\n### Healing saved reads\n\n- `heal()` reruns `reviews`, `socials` and `company` from a saved profile when they failed, and refreshes a stage whose version is older than the code's (ads, assortment, catalog, reviews, company, socials search, site). [src: brew/reader/index.js:38, 181-241]\n- Heal runs from the brand's feed open (a builder nudge the worker runs) or a signed-in `/check`; brands mid-build or mid-setup are skipped. [src: brew/server.js:469] [src: memory brew-checkout-walk] [src: memory brew-read-stream-backfill]\n- The reading page fills in cards and step lines it missed when a deploy cut its stream. [src: memory brew-read-stream-backfill]\n\n### Every brand, every platform\n\nKinds that broke before and must keep working: Shopify and headless Shopify, WooCommerce, Magento, BigCommerce, Salesforce Commerce, custom; stores that sell by enquiry; one or many markets; \"PUMA India\" also as \"PUMA\"; bot walls, script-drawn pages, nested sitemaps, size pickers, tags via GTM, reviews off; background runs across deploys. Offline cases live in `test/non-shopify.test.js`. [src: brew/CLAUDE.md]\n\n### Cost and time\n\nThe last measured full read: about 3.5 minutes and about $6 for Arata on 15 Oct, on Claude. No measurement on Gemini is recorded. [src: brew/docs/handover/engineering/02-feature-audit.md]",
  "Tomorrow": "| Item | Version | Tag |\n|---|---|---|\n| Seven visible stages, deadlines, one retry, plain errors | v1 | Stays |\n| Read only: the read never changes the store | v1 | Stays |\n| A cancel button for a read in progress | v2 | New |\n| Unit tests for `company.js` and the Ad Library HTML parser | v2 | New |\n| Reading reviews from a page the brand points to on the check page | v2 | New |\n| Marketplaces as a real read per platform | v3 | New |",
  "How we do it": "1. Load the `brew-external-data` skill before touching a stage, a treg call or a check-page field.\n2. When a stage starts reading differently, bump its `_V` constant so saved reads heal once. [src: brew/CLAUDE.md]\n3. Add a case to `test/non-shopify.test.js` for every new kind of brand that breaks.\n4. Never run a full read just to test (Plays included it is about $80 at Opus prices). Use mocks, cheap partial runs or saved packs. [src: memory feed-for-all-brands]\n5. To force a heal of one brand, call `/api/brands/<slug>/check` signed in as staff. [src: memory brew-checkout-walk]",
  "Numbers": "| Metric | Definition | Where measured |\n|---|---|---|\n| Read time | Start to `read-done` | `status.json` stages (`read`) |\n| Read cost | AI plus treg dollars for the read | `cost.json` byTag; treg `spend/<month>.json` |\n| Stage failure rate | Share of reads where a stage ended \"could not finish\" | `profile.meta.errors` (Proposed to aggregate) |\n| Source health | ok, empty, error per treg and Apify source per month | `_brew/ops/sources-YYYY-MM.json`, `GET /api/admin/sources` [src: memory brew-apify-watch] |",
  "Open": "1. **What does a read cost on Gemini today?** Proposal: measure the next Cozy re-read and record it here.\n2. **Should a signed-out visitor still start a read?** Today non-official emails cannot start a new read. Proposal: keep it so; the demo is the place for prospects. [src: memory brew-access-rules]"
 }
}