--- id: "7.2" title: "Measurement engine" dept: data-and-analytics status: draft owner: Param branded: false last_verified: 2026-10-26 sources: - "brew/SCOPE.md:39" - "brew/DUMMY.md:37" - "brew/measure-stats.js:1-23" - "brew/measure-families.js:1-3, 11-40, 41-48, 68, 93-127" - "brew/data/measure-families.json" - "brew/measure-spec.js:1-30, 64-97" - "brew/impact.js:1-12" - "brew/jam-brief.js:14" - "brew/moves-engine.js:14" - "brew/remix.js:38, 1949" - "brew/feed-service.js:315" - "memory feed-measurement-engine" - "handbook content/product/v3-prd.md, content/company-and-strategy/open-questions.md:37-40, content/engineering/codebase-map.md:306" --- # Measurement engine ## Answer Measurement is attribution by design: every card belongs to a family that names its nearest number, its fingerprint and the strongest fair comparison, and a result earns a grade from A, a holdout, down to D, before and after; v1 keeps only the families and the plan, and the engine that reads results returns in v3. ## Today ### What is cut - Measuring results is cut: no engine, launch records, report back, grades or result columns. [src: brew/SCOPE.md:39] [src: brew/DUMMY.md:37] - `measure-engine.js` is not on origin/main; one comment in `feed-service.js` still names it. [src: brew/feed-service.js:315] - The old engine ran on the archived feed for one brand from 14 Oct, behind `MEASURE_ENGINE`. [src: memory feed-measurement-engine] ### What stays and is used | Piece | What it does | Used by | Source | |---|---|---|---| | Families | 13 in the data file plus 6 from the listening loop: each names the nearest number, a fingerprint, the comparisons possible and a grade ceiling | Plays, the Studio, impact | [src: brew/data/measure-families.json] [src: brew/measure-families.js:41-48] [src: brew/moves-engine.js:14] [src: brew/remix.js:38, 1949] | | Timings | Per family: check days, the one decide day, the minimum sample, early stop rules | Timing lines on Plays and cards | [src: brew/measure-families.js:11-40] | | Grades | A holdout, B launched against untouched, C tagged sales only, D before and after | Words and ceilings | [src: brew/measure-stats.js:1-23] | | How we measure it | A section of each brief built from blocks; every number must already be in the brief or it is dropped | Briefs | [src: brew/measure-spec.js:1-30, 64-97] [src: brew/jam-brief.js:14] | | Projected impact | A range for one change, labelled a projection, not a promise | The Studio | [src: brew/impact.js:1-12] | | Other proposals | Three repeats of the same Other card draft a new family for an admin | Stored; no route approves them on main | [src: brew/measure-families.js:68, 93-127] | The brief's watch block says the team runs the checks; Brew does not watch or stop anything. [src: brew/measure-spec.js:23] ### The families | Family | Nearest number | Comparisons | Ceiling | Decide day | Source | |---|---|---|---|---|---| | Paid ads | Cost per first order | holdout, launched against untouched, tagged, before and after | A | 14 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Site and checkout | Visits that become an order | holdout, launched against untouched, before and after | A | 14 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Customer messages | Revenue per person messaged | holdout, tagged, before and after | A | 7 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Price and offers | Revenue per visitor | launched against untouched, tagged, before and after | B | 14 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Marketplace listings | Orders from the listing per week | launched against untouched, before and after | B | 14 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Search and AI visibility | Search clicks to the touched pages | launched against untouched, before and after | B | 90 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Content, social and creators | First orders from tagged links and codes | tagged, before and after | C | 7 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Partners, PR and referral | Orders from the partner or referral code | tagged, before and after | C | 28 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Offline and B2B | Orders or accounts from that channel | launched against untouched, tagged, before and after | B | 28 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Foundations | Orders we can trace to a source | before and after | D | 14 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Strategy | What the work it led to achieved | none | D | 90 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Research | Launches it led to | none | D | 60 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | | Other | Picked in the jam | all four | A | 14 | [src: brew/data/measure-families.json] [src: brew/measure-families.js:11-40] | ## Tomorrow | Item | Version | Tag | |---|---|---| | Families and the How we measure it plan in every brief | v1 | Stays, as the team's plan | | Delete unused engine code; keep what impact and copy use | v2 | Changes | | Launch record when a Play ships: date, scope, fingerprint | v3 | New | | Reads from the brand's own connected accounts on the family's check days | v3 | New | | How it went: effect, range, grade, method, source | v3 | New | | Only A and B results move ranking; C and D show to the team only | v3 | New, proposed | | Holdouts sent through the brand's own Klaviyo or WhatsApp tool | v3 | New, proposed | | Ad rules stop a run on the family's early stop rule | v4 | New | ## How we do it 1. A new kind of Play lands in a family, or in Other; never new engine code. [src: brew/measure-families.js:1-3] 2. Only a declared holdout earns grade A; a comparison series alone is B. [src: memory feed-measurement-engine] 3. The fact gate drops a measured number unless it is passed as a live number. [src: memory feed-measurement-engine] 4. Never show an early read as a result; only the decide day gives a grade. ## Numbers | Metric | Definition | Where measured | Status | |---|---|---|---| | Measured share | Share of shipped Plays with a How it went result | Launch records | v3, Proposed | | Grade mix | Results by grade A to D | Launch records | v3, Proposed | | Other share | Share of cards that land in Other | `measure-families.extra.json` others | Proposed | ## Open 1. **Which grades teach the feed?** Proposal: A and B only; C and D show to the team. [src: handbook content/company-and-strategy/open-questions.md:37] 2. **Keep How we measure it in v1 briefs?** Proposal: keep it, renamed How your team measures it. [src: handbook content/company-and-strategy/open-questions.md:40] 3. **Approve Other proposals without a route?** Proposal: leave them stored until v3 brings the engine back.