Snapshots

A pasted store address becomes a sourced profile in seven stages

Being built
  • Stages run in order: open, brand, catalog, reviews, ads, site, answers.
  • Each stage has a deadline and one retry; paid answers come from cache.
  • Check what we read waits up to 15 minutes for the team's corrections.
  • A brand's first read writes Plays only after someone opens the feed.
  • Bump a stage version whenever its output shape changes, so saved reads heal.
15minutes the check waits
Your call

What does a read cost on Gemini today? Proposal: measure the next Cozy re-read.

Engineering · 5.5

The read

DraftLast checked Mon Oct 26Owner Param

Answer

The read turns a pasted store address into a sourced profile and ledger through seven visible stages with deadlines and one retry each, and this page settles the stage order, what each one reads, how saved reads heal and when a stage version must be bumped.

Today

The flow around the read

  1. POST /api/brands {url} starts a build. A first read writes the profile, ledger and findings. brew/build.js:310-322
  2. If the store did not open, the build stops before spending more and says so (or says the store is behind Shopify's password page). brew/build.js:316-318
  3. "Check what we read" waits for the team's corrections. A signed-out read waits with no time limit; otherwise Brew goes on at writeAt (BUILD_CONFIRM_WAIT_MS, 15 minutes on staging and production). brew/build.js:327-340 brew/deploy.sh:39
  4. The brief is written, then a brand's first read waits until someone opens the feed before writing Plays (feedGate, BUILD_FEED_GATE=0 turns it off). Reminders go out at 3 hours and 24 hours. brew/build.js:356-362 memory brew-feed-open-moves

Stages in order

Step Shown as Deadline What it reads Version Source
open Open the store 45 s, one try platform, market, pages, sitemaps, headless Shopify brew/reader/index.js:27-36 brew/reader/stages/open.js
brand Read the brand 40 s logo, colours, fonts, one-liner brew/reader/stages/brand.js
catalog Pull the catalog 60 s products on any platform, assortment (own label or resold) CATALOG_V 2, ASSORT_V 2 brew/reader/stages/catalog.js:11 brew/assortment.js:6
reviews Read what customers say 75 s review apps on the site, marketplace ratings REVIEWS_V 2 brew/reader/stages/reviews.js:32
ads Find live ads, yours and rivals 240 s (rivals 110 s, 30 s per rival price) Meta Ad Library via treg, free headless fallback; five rivals ADS_V 5 brew/reader/stages/ads.js:18 brew/reader/adlib.js
site Walk the path to checkout 200 s headless checkout walk, tags that fire, PageSpeed SITE_V 4 brew/reader/stages/site.js:75
answers Ask the AI tools 170 s ChatGPT, Gemini, Perplexity answers via treg; category; named check (100 s) brew/reader/stages/answers.js brew/reader/stages/named.js
(no step) socials, company Films card, founders and press 120 s, 150 s Instagram, YouTube (found by search when the site links none), about page SOCIALS_V 2, SOCIAL_SEARCH_V 1, COMPANY_V 3 brew/reader/stages/socials.js:6-8 brew/reader/stages/company.js:18

Deadlines are LIMIT in reader/index.js. A stage that fails or times out gets one more try; paid calls that already answered come back from cache. A timed-out stage keeps running but can no longer emit. Then the step says plainly what happened and the read goes on. brew/reader/index.js:35, 121-140

A store behind Shopify's password page is read from the Admin API (reader/stages/admin.js); the stages that need the public site say "not checked". brew/reader/index.js:115-117 memory brew-shopify-app-review

Under Gemini the "Claude" AI-visibility ask is skipped, so Gemini is never labelled Claude. memory brew-gemini-provider

Outside data rules

  • Every treg call goes through treg.js, cached by signal: company data 30 days; reviews, search, AI answers and profiles 7 days; ads, news and prices 1 day; generation never cached; failures never cached. brew/treg.js:12-27
  • Paid sources fall back to free ones (Ad Library: adCall). brew/CLAUDE.md
  • An error is never shown as "none" or "0"; it keeps error and says the check did not go through. brew/CLAUDE.md
  • treg first, Apify only where treg has nothing or is weak; Apify capped at $10 a brand a month. brew/SCOPE.md memory brew-apify-watch
  • Every change here starts by loading the brew-external-data skill. brew/CLAUDE.md

Healing saved reads

  • heal() reruns reviews, socials and company from a saved profile when they failed, and refreshes a stage whose version is older than the code's (ads, assortment, catalog, reviews, company, socials search, site). brew/reader/index.js:38, 181-241
  • Heal runs from the brand's feed open (a builder nudge the worker runs) or a signed-in /check; brands mid-build or mid-setup are skipped. brew/server.js:469 memory brew-checkout-walk memory brew-read-stream-backfill
  • The reading page fills in cards and step lines it missed when a deploy cut its stream. memory brew-read-stream-backfill

Every brand, every platform

Kinds that broke before and must keep working: Shopify and headless Shopify, WooCommerce, Magento, BigCommerce, Salesforce Commerce, custom; stores that sell by enquiry; one or many markets; "PUMA India" also as "PUMA"; bot walls, script-drawn pages, nested sitemaps, size pickers, tags via GTM, reviews off; background runs across deploys. Offline cases live in test/non-shopify.test.js. brew/CLAUDE.md

Cost and time

The last measured full read: about 3.5 minutes and about $6 for Arata on 15 Oct, on Claude. No measurement on Gemini is recorded. brew/docs/handover/engineering/02-feature-audit.md

Tomorrow

Item Version Tag
Seven visible stages, deadlines, one retry, plain errors v1 Stays
Read only: the read never changes the store v1 Stays
A cancel button for a read in progress v2 New
Unit tests for company.js and the Ad Library HTML parser v2 New
Reading reviews from a page the brand points to on the check page v2 New
Marketplaces as a real read per platform v3 New

How we do it

  1. Load the brew-external-data skill before touching a stage, a treg call or a check-page field.
  2. When a stage starts reading differently, bump its _V constant so saved reads heal once. brew/CLAUDE.md
  3. Add a case to test/non-shopify.test.js for every new kind of brand that breaks.
  4. Never run a full read just to test (Plays included it is about $80 at Opus prices). Use mocks, cheap partial runs or saved packs. memory feed-for-all-brands
  5. To force a heal of one brand, call /api/brands/<slug>/check signed in as staff. memory brew-checkout-walk

Numbers

Metric Definition Where measured
Read time Start to read-done status.json stages (read)
Read cost AI plus treg dollars for the read cost.json byTag; treg spend/<month>.json
Stage failure rate Share of reads where a stage ended "could not finish" profile.meta.errors (Proposed to aggregate)
Source health ok, empty, error per treg and Apify source per month _brew/ops/sources-YYYY-MM.json, GET /api/admin/sources memory brew-apify-watch

Open

  1. What does a read cost on Gemini today? Proposal: measure the next Cozy re-read and record it here.
  2. Should a signed-out visitor still start a read? Today non-official emails cannot start a new read. Proposal: keep it so; the demo is the place for prospects. memory brew-access-rules
Sources
  • brew origin/main: reader/index.js:1-40, 115-140, 181-310; reader/stages/*.js (version constants); reader/adlib.js; reader/products.js
  • brew origin/main: build.js:302-345 (run: read, check, brief, feed gate), 779-860 (heal, checkData), 1600
  • brew origin/main: treg.js:8-28; SPEC.md (profile, ledger, onboarding stream); CLAUDE.md 'Every brand, every platform'
  • brew origin/main: docs/handover/engineering/02-feature-audit.md section 1, 05-ai-and-external-data.md (15 Oct)
  • brew origin/main: docs/audit-2026-10-24-design-partners.md
  • memory brew-checkout-walk, brew-read-stream-backfill, brew-feed-open-moves, brew-apify-watch, brew-shopify-app-review, brew-external-data-skill, feed-for-all-brands

Source file content/engineering/the-read.md · For agents: Markdown · JSON