Answer
Brew is guarded by one free test suite that every deploy runs, a set of honesty and brand-matrix tests, audit tools that check saved data for every brand, and QA rounds run after each phase; this page settles what each catches and when it must run.
Today
The suite
- Run:
AI_MOCK=1 node --test test/*.test.js. Free: no AI, no network. brew/README.md - 215 test files and about 1,355
test()calls on origin/main d6154b3 (counted from source). The jam audit closed on 25 Oct with 1,173 tests passing. brew/test/ memory brew-v1-creative-stage deploy.shruns it and refuses to deploy on a failure; a failed run is retried once after 20 s because two sessions may run suites side by side. brew/deploy.sh:13-16tools/deploy-demo.shruns no tests. brew/docs/handover/engineering/07-deploy-and-ops.md- Postgres tests (
*-pg,qv1-*,qv2-*,phase3-pg) run against a local Postgres throughtest/pg-helper.js, which starts a throwaway Postgres 17 cluster per file. brew/test/pg-helper.js brew/.github/workflows/test.yml:22-23
CI
- GitHub Actions workflow
testruns the same gate asdeploy.shon every push and every pull request: Ubuntu 24.04, Node 22 as the Dockerfile,npm ci, Postgres 17 binaries, thennode --test test/*.test.jswithAI_MOCK=1. A newer push cancels the older run; a run stops after 30 minutes. brew/.github/workflows/test.yml:1-37 - It deploys nothing. brew/.github/workflows/test.yml:9-37
- First green run: https://github.com/wulfor24/brew/actions/runs/40286754218. GitHub Actions run 40286754218
- CI found that
timed()inlisten-watch.jsandwithTimeout()inreader/util.jslet a job or a test file exit mid run when a source never settled; the timer is now held and cleared when the work settles. brew commit 4799f78 brew/listen-watch.js:81 brew/reader/util.js:179 - Paid probes, run only on purpose with Param's yes:
cadence-probe.mjs,ideas-probe.mjs,listen-probe.mjs,memory-probe.mjs. brew/test/ memory no-spend-while-building
Tests that hold a rule
| Test | Rule it holds | Source |
|---|---|---|
honesty.test.js |
A dummy never passes as real: simulated rows never evidence, no canned sign-offs, healed old wording | brew/DUMMY.md |
non-shopify.test.js |
The brand matrix: platforms, enquiry stores, markets, names | brew/CLAUDE.md |
brand-book-guards.test.js |
The 23 Oct Brand Book issue classes never return | brew/CLAUDE.md |
no-undef-consts.test.js |
Fails on any undeclared ALL_CAPS name (a missing import once shipped) | memory brew-one-version |
qv1-*, qv2-*, qv3-guards |
Storage, roles, queue, cache, HTTP and promote findings from the 25 Oct QV rounds | memory brew-prod-backend |
one-stage.test.js, setup-tab.test.js |
The jam stage contract and Set-up | memory brew-one-stage memory brew-setup-v2 |
private-leaks.test.js, access.test.js, qa-final-access.test.js |
Team-only data never reaches visitors | brew/test/ |
make-claim.test.js |
One paid make per board across instances | git b91e364 |
product-name.test.js |
Customer templates never say ShopOS or moves, name the product only from product-name.js, and every served page has a language and a description |
brew/test/product-name.test.js:51-84 |
brief-staff.test.js |
Only our team moves a brief on; the client's steps match the server's; the controls never show to a brand member, guest or the demo | brew/test/brief-staff.test.js:31-94 |
brand-wipe-pg.test.js |
Reset and delete: dry run removes nothing, partners refused, connections unchanged, audit row written, bucket scripts refuse on Postgres | brew/test/brand-wipe-pg.test.js:46-142 |
Audit tools (saved data, every brand)
| Tool | Checks | Source |
|---|---|---|
tools/audit-card-fit.mjs DIR [--before] |
Wrong medium, empty asset lines, button and asset fit, old vs new code | memory brew-card-fit-and-counts |
tools/audit-brand-book.mjs DIR |
Brand Book issue classes in every scope | brew/CLAUDE.md |
tools/audit-layout.mjs <links> |
The jam rendered at 1000×600, 1280×800 and 390×844, every tab | memory brew-jam-rca-2026-10-26 |
tools/meta-audit.mjs, tools/shopify-audit.mjs |
What Brew shows against the platform, read only | memory brew-meta-connector-qa |
tools/probe-sources.mjs |
treg and Apify sources on real brands | memory brew-apify-watch |
.claude/skills/brew-external-data/audit.py |
Errors shown as none, outdated stage versions | brew/CLAUDE.md |
db/prod-clean.mjs |
Production settings and data are clean | brew/db/prod-clean.mjs:1-6 |
db/parity.mjs |
Bucket and database agree | memory brew-prod-backend |
QA rounds so far
| Date | Round | Found and fixed | Source |
|---|---|---|---|
| 15 Oct | Handover audits A to F | 96 feature rows; P0 honesty leaks fixed the same day | brew/docs/handover/engineering/02-feature-audit.md |
| 19 to 20 Oct | Sai's QA sheet | 49 bugs, then about 35 half-done fixes | memory brew-qa-sweep-2026-10-19 |
| 20 Oct | Four auditors, 11 brands | about 86 issues | memory brew-qa-audit-2026-10-20 |
| 22 to 25 Oct | Jam audits | about 60, then about 50, then the close-out | memory brew-v1-creative-stage |
| 24 Oct | Round 2 report and design-partner audit | sections A, B, C approved and fixed | memory brew-qa-round2-2026-10-24 brew/docs/audit-2026-10-24-design-partners.md |
| 25 Oct | Production backend QV, phases 1 to 6 plus 3 final rounds | about 45, about 18, then about 35 | memory brew-prod-backend |
| 26 Oct | Jam RCA from Arata screenshots | five root causes | memory brew-jam-rca-2026-10-26 |
What still has no test
Route gating has no single route table to test against; ranking (score, rank, feed) has no fixtures from several brands; few client files have a test, brief-staff.test.js among them; there is no lint or type check, and CI runs only the suite. brew/docs/handover/engineering/08-testing-and-qa.md brew/.github/workflows/test.yml:35-37
Tomorrow
| Item | Version | Tag |
|---|---|---|
| The free suite as the deploy gate | v1 | Stays |
A GitHub Action running npm ci and the suite on every push and pull request, live since 26 Oct |
v1 | Stays |
node --check and a lint step in the same Action |
v1 | New, proposed |
| A route table plus one test of every route's auth level | v2 | New |
| Ranking fixtures from the five partners | v2 | New |
| Tests run before the demo deploy | v1 | Changes |
| Headless smoke of the feed and jam on staging after each deploy | v2 | New |
How we do it
- Every bug: fix the shared cause, add a test or audit that would have caught it, heal saved data, deploy. memory fix-as-product
- Before saying something works, check it on several brands and card types, and the feed view (renders, scrolls, opens, no console errors). memory feed-for-all-brands
- After each phase of phased work: an independent review of the diff, live checks on staging, fixes with tests, redeploy, verify. Three more full rounds before closing. memory brew-qv-process
- A local
.brands/left in a worktree breaksfeed-engagement.test.js; remove it or symlink the real one before deploying. memory brew-v1-creative-stage - After a deploy, reload the page before judging; hash navigation keeps the old app.js. memory brew-one-stage
Numbers
| Metric | Definition | Where measured |
|---|---|---|
| Suite size | Test files and passing tests | node --test output at deploy |
| Suite time | Wall time of the suite | deploy log, and each GitHub Actions run |
| CI pass rate | Share of Actions runs on main that pass | GitHub Actions, workflow test, Proposed |
| Audit findings | Findings per audit tool run on the five partners | tool output (Proposed to log weekly) |
| Escaped bugs | Bugs found on production that a test should have caught | Proposed |
Open
- Should a red CI run block a deploy? Proposal: yes,
deploy.shchecks the commit's Actions status first and stops on red. - Who owns QA rounds once Sai and Arvind work in the code? Proposal: whoever ships a phase runs its QV round; Param signs off the three final rounds.