--- id: "8.2" title: "Deploy and promote" dept: ops-and-infrastructure status: draft owner: Param branded: false last_verified: 2026-10-26 sources: - "brew origin/main a559de6: deploy.sh:5-53, tools/promote.sh:1-119, tools/prod-env.sh:1-13, tools/deploy-demo.sh:7-26, db/prod-clean.mjs:1-6, CLAUDE.md 'Release rhythm'" - "brew/.wt-prod/docs/production.md:23-33" - "memory always-deploy-to-test, prod-clean-only, brew-parallel-sessions-deploy, brew-qa-sweep-2026-10-19, brew-prod-backend, brew-pro-switch-and-flush, brew-storage" - "handbook content/engineering/release-and-promote.md, content/engineering/parallel-sessions.md" --- # Deploy and promote ## Answer Code goes to staging with `./deploy.sh` the moment it is tested, and to production only with `tools/promote.sh`, which ships staging's exact image after a backup and a clean gate and puts production back by itself if anything fails; this page is the operator's checklist for both. ## Today | | Staging deploy | Production promote | Demo deploy | |---|---|---|---| | Command | `./deploy.sh` | `tools/promote.sh` | `tools/deploy-demo.sh` [src: brew/deploy.sh:7-53] [src: brew/tools/promote.sh:16-119] [src: brew/tools/deploy-demo.sh:9-26] | | Builds an image | yes, `--source .` | no, reuses staging's image by digest | yes [src: brew/deploy.sh:7-53] [src: brew/tools/promote.sh:16-119] [src: brew/tools/deploy-demo.sh:9-26] | | Refuses a dirty tree | yes | no tree check; refuses a staging sha not on origin/main | yes [src: brew/deploy.sh:7-53] [src: brew/tools/promote.sh:16-119] [src: brew/tools/deploy-demo.sh:9-26] | | Runs the test suite | yes, one retry after 20 s | no, it relies on staging | no [src: brew/deploy.sh:7-53] [src: brew/tools/promote.sh:16-119] [src: brew/tools/deploy-demo.sh:9-26] | | Backup first | no | on-demand Cloud SQL backup, waited for | no [src: brew/deploy.sh:7-53] [src: brew/tools/promote.sh:16-119] [src: brew/tools/deploy-demo.sh:9-26] | | Gate | none | `db/prod-clean.mjs` on the candidate and the worker | none [src: brew/deploy.sh:7-53] [src: brew/tools/promote.sh:16-119] [src: brew/tools/deploy-demo.sh:9-26] | | Rollback | by hand, newest 3 revisions kept | by itself on any failure after a change | by hand, newest 3 kept [src: brew/deploy.sh:7-53] [src: brew/tools/promote.sh:16-119] [src: brew/tools/deploy-demo.sh:9-26] | | Smoke checks | curls 4 staging pages, never fails the deploy | 6 pages through the load balancer, 5 tries each | none [src: brew/deploy.sh:7-53] [src: brew/tools/promote.sh:16-119] [src: brew/tools/deploy-demo.sh:9-26] | Sources: [src: brew/deploy.sh:7-16] [src: brew/deploy.sh:39-53] [src: brew/tools/promote.sh:16-22] [src: brew/tools/promote.sh:54-119] [src: brew/tools/deploy-demo.sh:9-26] ### Rules in force - Deploy after every tested change without asking; Param tests on the live staging URL. Still ask before anything that spends money or touches real ads or stores. [src: memory always-deploy-to-test] - A deploy never waits for background work. The old instance stops starting Plays and saves; the new one resumes. Never add a wait back. [src: brew/CLAUDE.md] [src: memory brew-pro-switch-and-flush] - Nothing dummy or from our testing reaches production; the clean gate refuses a promote on any finding. [src: memory prod-clean-only] [src: brew/db/prod-clean.mjs:1-6] - One promote at a time, through the lock `gs://brew-prod-backups/locks/promote.lock`; a lock older than 2 hours is taken over. [src: brew/tools/promote.sh:25-36] - Never edit `promote.sh` while it runs; bash reads it as it goes and that once caused a rollback. [src: brew/.wt-prod/docs/production.md:33] [src: memory brew-prod-backend] - After a staging deploy that ran on the bucket, run the `brew-import` job. [src: brew/deploy.sh:30-32] [src: memory brew-storage] - Model choice is set by environment at deploy time: staging defaults to Gemini, Pro for Plays and Flash for the rest. [src: brew/deploy.sh:23-29] [src: brew/deploy.sh:41] ## Tomorrow | Item | Version | Tag | |---|---|---| | Staging on every tested change, production only by `promote.sh` | v1 | Stays | | Staging smoke checks fail the deploy when a page is not 200 | v1 | Changes | | A short release note per promote, written into the handbook changelog | v1 | New | | A second person able to promote | v2 | New, after onboarding | ## How we do it ### Staging 1. Commit only your lines, push, then deploy from a clean worktree of origin/main. [src: memory brew-parallel-sessions-deploy] ```bash cd /Users/param/Documents/ChatGPT/brew git pull --rebase && git push origin HEAD:main git worktree add --detach "$TMPDIR/brew-deploy" origin/main ln -s /Users/param/Documents/ChatGPT/brew/node_modules "$TMPDIR/brew-deploy/node_modules" cd "$TMPDIR/brew-deploy" && BRANDS_DIR=/Users/param/Documents/ChatGPT/brew/.brands ./deploy.sh ``` 2. Confirm the live site serves your commit: grep the served `app.js` for a new identifier and check the revision's `git-sha` label descends from your commit. [src: memory brew-qa-sweep-2026-10-19] 3. After a change on the pack or card path, time the seed, not just the status code: ```bash curl -s -o /dev/null -w '%{time_total}\n' https://brew.joinsquareup.com/b/cozysnugglet-shop/seed.js ``` 4. Roll back by hand if needed: ```bash gcloud run services update-traffic shopos-feed --to-revisions==100 --region=asia-south1 ``` ### Production 1. Staging must run the commit you want, built from origin/main, and have been tested there. 2. From a clean checkout of origin/main: ```bash tools/promote.sh ``` 3. Read the eight steps as they print. Any `STOP:` line means production traffic did not move or was put back. `ROLLBACK FAILED` means put the named revision back by hand with `update-traffic`. [src: brew/tools/promote.sh:49-52] 4. If the lock is held and no promote is running: `gcloud storage rm gs://brew-prod-backups/locks/promote.lock`. [src: brew/tools/promote.sh:32] 5. After the promote, open feed.joinsquareup.com and one partner brand signed in. Full step detail lives in [Release and promote](/engineering/release-and-promote). ## Numbers | Metric | Definition | Where measured | |---|---|---| | Promotes per week | Runs of `promote.sh` that reach 8/8 | promote output; Proposed: log each in the changelog | | Promote rollbacks | Runs that print "rolling back" | same, Proposed | | Staging deploys per day | New `shopos-feed` revisions | `gcloud run revisions list`, Proposed | | Time from staging to production | Hours between a commit's staging deploy and its promote | Proposed | ## Open 1. **How often should production be promoted?** No rhythm is written down; Next carries one pending promote of the 26 Oct jam fixes. Proposal: promote at most once a day, after Param has walked Arata and one other partner on staging. 2. **Should staging smoke checks fail the deploy?** Today they only print codes. Proposal: yes, fail on any non-200 from `/` or `/api/config`.