Snapshots

Every runbook in one list, so nobody runs a script from memory

Waiting on you
  • Runbooks are grouped by releasing, restoring and checking, and wiping and resetting.
  • Check which site a script names before running it; staging and production differ.
  • tools/reset-jams.sh still edits the old bucket, though staging keeps records in Postgres.
  • New: a brew-prod-ops job for deletions on production, and retire three bucket scripts.
1Stale runbook today
Your call

Who may run production runbooks, and should the bucket scripts be deleted now?

Ops and infrastructure · 8.1

Runbook index

DraftLast checked 2026-10-26Owner Param

Answer

This is the list of every operational runbook we have, what each one is for and when to reach for it, so nobody runs a script from memory or picks the wrong one for the site in front of them.

Today

Releasing

Runbook Use it when Site Where it is written
./deploy.sh A change is tested and committed; deploy right away Staging Deploy and promote brew/deploy.sh:1-7
tools/promote.sh Staging runs what production should run Production Deploy and promote brew/tools/promote.sh:1-9
tools/deploy-demo.sh The demo needs new code Demo brew/tools/deploy-demo.sh:1-6
tools/demo-sync.sh <slug> A prepared brand's public pack goes to the demo Demo brew/tools/demo-sync.sh:1-7
tools/partner-copy.sh <slug> apply A partner's data is copied once from staging to production Staging to production brew/tools/partner-copy.sh:1-7
python3 brand-directions/_tools/deploy-handbook.py The handbook changed and check.mjs passes Handbook Hosting map brand-directions/_tools/deploy-handbook.py:1-10
python3 brand-directions/_tools/deploy-docs.py The shared docs changed Docs brand-directions/_tools/deploy-docs.py:1-4

Restoring and checking

Runbook Use it when Site Where it is written
Clone to a moment A bad migration or mass damage on production Production Restore brew/.wt-prod/docs/production.md:47-50
One brand back One brand deleted or damaged by mistake Production Restore brew/.wt-prod/docs/production.md:52-55
Restore test Monthly Production into staging Restore brew/.wt-prod/docs/production.md:59-64
db/status.mjs through job brew-ops Is Arata still connected, how big are the tables, are jobs stuck Staging brew/db/status.mjs:1-2 memory brew-prod-backend
db/connector-ping.mjs through jobs brew-prod-ping and brew-staging-ping Do a brand's stored sign-ins still read, after a restore or a copy Either Protected connections brew/db/connector-ping.mjs:1-4 memory brew-prod-backend
db/parity.mjs Does Postgres hold everything the bucket holds Staging brew/db/parity.mjs:1-3
db/import-bucket.mjs through job brew-import After any staging deploy that ran on the bucket Staging brew/db/import-bucket.mjs:1-6 memory brew-storage
Three DNS checks Someone says the site is down Any DNS and domains memory brew-dns-local

Wiping and resetting

Runbook Use it when Site Where it is written
db/reset-jams.mjs through job brew-ops Param asks to start the jams fresh Staging, on Postgres Wipe data and reset jams brew/db/reset-jams.mjs:1-7
"Wipe data" Param says "wipe data" for a test brand Staging Wipe data and reset jams memory brew-wipe-data
db/reset-brand.mjs through job brew-ops One brand back to a fresh first run, connections kept; also the way to flush every brand, one slug at a time Staging Wipe data and reset jams brew/db/reset-brand.mjs:1-11
db/delete-brand.mjs through job brew-ops, or brew-prod-ops once made A brand asks us to delete its data Staging; production after the next promote Data requests brew/db/delete-brand.mjs:1-10 brew/docs/production.md:95-106

Scripts that no longer fit the site they name

  • tools/reset-jams.sh edits gs://brew-prototype directly. Staging has kept its records in Postgres since 25 Oct, so it edits the mirror, not the truth. brew/tools/reset-jams.sh:6 memory brew-jam-reset
  • tools/reset-brand.sh and tools/flush-live.sh are bucket era too, but since 26 Oct they exit 3 unless STORAGE=gcs and print the brew-ops command to run instead. brew/tools/reset-brand.sh:11-19 brew/tools/flush-live.sh:17-26

Checking

Runbook Use it when Site Where it is written
GitHub Actions workflow test Every push and pull request runs the deploy test gate on Node 22 with Postgres 17 GitHub Testing and QA brew/.github/workflows/test.yml:1-37

Tomorrow

Item Version Tag
One index of runbooks, each linked to its page v1 Stays
A Postgres brand reset and deletion, run as brew-ops jobs, on origin/main v1 Stays
A brew-prod-ops job on production for deletions v1 New
Retire the three bucket scripts; two already refuse on Postgres v1 Goes
Every runbook printed from code into this index on each build v2 New

How we do it

  1. Find the job in the tables above, then open its page. Never run a script from memory.
  2. Check which site the script names before running it. Staging scripts hard-code shopos-feed and brew.joinsquareup.com; production scripts source tools/prod-env.sh.
  3. Run cloud scripts from a clean, detached worktree of origin/main, never the shared checkout:
git -C /Users/param/Documents/ChatGPT/brew fetch -q origin
git -C /Users/param/Documents/ChatGPT/brew worktree add --detach "$TMPDIR/brew-ops" origin/main
  1. Job arguments with flags use gcloud's "--args=^:^a:b:c" form. memory brew-prod-backend
  2. When a runbook changes, update its page and this index in the same session.

Numbers

Metric Definition Where measured
Runbooks without a page Scripts in tools/ or db/ that this index does not list This page against git ls-tree origin/main tools/ db/, Proposed
Stale runbooks Scripts that target storage the site no longer uses and still run This page, 1 today, tools/reset-jams.sh

Open

  1. Who may run production runbooks? Today the scripts run with whoever's gcloud sign-in is on the Mac. Proposal: only Param runs promote.sh and the restore runbooks until a second person has done one restore test with him.
  2. Should the bucket scripts be deleted now? tools/reset-brand.sh and tools/flush-live.sh already stop on STORAGE=pg. Proposal: give tools/reset-jams.sh the same first lines, and delete all three once production has run on Postgres for a month.
Sources
  • brew origin/main a559de6: deploy.sh:1-53, tools/promote.sh:1-119, tools/prod-env.sh:1-13, tools/flush-live.sh:1-26, tools/reset-brand.sh:1-20, db/reset-brand.mjs:1-13, db/delete-brand.mjs:1-12, tools/reset-jams.sh:1-30, tools/partner-copy.sh:1-22, tools/deploy-demo.sh:1-26, tools/demo-sync.sh:1-30, db/reset-jams.mjs:1-30, db/status.mjs:1-27, db/connector-ping.mjs:1-24, db/parity.mjs:1-30, db/import-bucket.mjs:1-30, db/migrate.js:1-27
  • brew/.wt-prod/docs/production.md:23-81
  • brew origin/main d6154b3: docs/production.md:66-110, .github/workflows/test.yml:1-37
  • brand-directions/_tools/deploy-handbook.py:1-115, brand-directions/_tools/deploy-docs.py:1-60
  • memory brew-prod-backend, brew-wipe-data, brew-jam-reset, brew-dns-local, brew-parallel-sessions-deploy, brew-storage

Source file content/ops-and-infrastructure/runbook-index.md · For agents: Markdown · JSON