Snapshots

Pro writes the Plays, Flash does the rest, Claude waits for credit

Waiting on you
  • One switch, AI_PROVIDER, sends every text call to Claude or to Gemini.
  • Plays alone run on Gemini Pro; brief, ideas, cards and jam stay on Flash.
  • Anthropic credit ran out on 20 Oct, so live moved to Gemini that day.
  • Failed calls fall back down a chain and billing failures pause callers for 15 minutes.
  • Opus Plays were clearly richer than Pro, but the test was confounded by an audit upload.
54Arata Plays compared
Your call

Switch Plays back to Opus once Anthropic is topped up, after a fair 5-Play bake-off?

AI and models · 6.1

Model policy

DraftLast checked Mon Oct 26Owner Param

Answer

Which model writes, judges and draws each thing in Brew today, why Plays alone run on Gemini Pro while everything else runs on Gemini Flash, what takes over when a model fails, and the one deploy that switches back to Claude.

Today

The switch

Setting Staging, brew.joinsquareup.com Production, feed.joinsquareup.com What it does
AI_PROVIDER gemini by default gemini anthropic sends every text call to Claude; anything else goes to Gemini on Vertex brew/deploy.sh:41 brew/tools/prod-env.sh:9 brew/gemini-client.js:11
GEMINI_MOVES_MODEL gemini-3.1-pro-preview gemini-3.1-pro-preview The model that writes Plays when the provider is Gemini brew/moves-engine.js:17
GEMINI_DEEP_MODEL gemini-3.5-flash gemini-3.5-flash Every call that asks for Opus: brief, ideas, other deep jobs brew/gemini-client.js:13-14
GEMINI_FAST_MODEL gemini-3.5-flash gemini-3.5-flash Every other call: Sonnet, Haiku and jam calls brew/gemini-client.js:13-14
GEMINI_IMAGE_MODEL gemini-2.5-flash-image gemini-2.5-flash-image Draft pictures brew/server.js:118 brew/tools/prod-env.sh:9
MOVES_EFFORT unset, so medium unset, so medium Thinking depth for Plays brew/build.js:402
  • The code asks for Claude names everywhere; geminiModel() maps any name with "opus" to the deep model and every other name to the fast one brew/gemini-client.js:14.
  • Claude effort maps to Gemini thinking level: low to LOW, medium and high to MEDIUM, xhigh and max to HIGH brew/gemini-client.js:27.
  • Gemini thinking counts against its output limit, so the shim adds 16,000 tokens of headroom to thinking jobs and 2,000 to the rest, capped at 65,536 brew/gemini-client.js:68.
  • Search plus schema calls on Gemini took 8 to 95 seconds when measured memory brew-gemini-provider.
  • Google startup credits pay for Vertex: $20,763 of $25,000 left on the 2024 GFS Cloud Program, ending 2028-05-19, as of 16 Oct memory brew-gemini-provider.
  • The Anthropic key ran out of credit on 20 Oct at 07:50 IST, and live moved to Gemini that day memory brew-pro-switch-and-flush brew/deploy.sh:25.

Who does what today

Job Model the code asks for Model that runs today Source
Plays, all of them MOVE_MODEL(), Opus on Claude gemini-3.1-pro-preview, effort medium brew/moves-engine.js:17 brew/moves-engine.js:473
Play checker and fixer Sonnet low, Sonnet medium gemini-3.5-flash brew/moves-engine.js:553 brew/moves-engine.js:568
Brand brief Opus high gemini-3.5-flash brew/brief.js:108
Ideas slate and writing Opus medium, Opus high gemini-3.5-flash brew/ideas.js:382 brew/ideas.js:400
Reader stages, cards, plain words, learnings, memory, listening Sonnet gemini-3.5-flash see the Prompts registry
Jam and Studio words, chat, briefs claude-sonnet-5-5, effort low, thinking off gemini-3.5-flash brew/server.js:116 brew/server.js:139-142
Product check and look judge on pictures claude-haiku-5-5 through IMAGE_CHECK_MODEL gemini-3.5-flash brew/server.js:223 brew/server.js:233
Video reading VIDEO_MODEL, default gemini-3.5-flash, then gemini-2.5-flash same brew/video-read.js:5 brew/video-read.js:97
Draft pictures gemini-2.5-flash-image, then gemini-3-pro-image, then OpenAI gpt-image-1 same brew/server.js:118 brew/server.js:191 brew/media.js:4
Draft video veo-3.1-fast-generate-001, then Veo 3.1 same brew/vertex-media.js:70 brew/media.js:5
Voice gemini-2.5-flash-tts, then OpenAI TTS same brew/vertex-media.js:53 brew/media.js:6
Music lyria-002 same brew/vertex-media.js:62
Studio shots and films Higgsfield first where routed, Vertex second depends on Higgsfield keys brew/data/engines.json
Finals Sloosh over MCP, Gemini or Veo as fallback same brew/finals.js:8-13

Fallbacks

  • Claude calls opt into the server-side refusal fallback: betas: ['server-side-fallback-2026-07-01'], fallbacks: 'default' brew/ai.js:71.
  • A billing failure pauses every caller for 15 minutes and emails ops once a day, instead of retrying every few seconds brew/ai.js:34-45.
  • Gemini retries a 429 or 5xx three times with backoff of 4, 8 and 16 seconds brew/gemini-client.js:110.
  • A Play rejected by structured output is asked again as plain JSON brew/moves-engine.js:483.
  • Media chains hand each failure to the next provider and record who made the result brew/media.js:1-8.
  • A Studio fallback partner never costs more than the price shown, except up to 1.5 times after a refunded refusal brew/jam2/engines.js:131-139.

Honesty rule tied to the model

  • Under Gemini, the Claude question in the AI answers check sits out, because Gemini's answer would be labelled Claude brew/reader/stages/answers.js:110 brew/live-watch.js:159.

Why Pro only for Plays

  • Param, 20 Oct: only the Plays run on a higher model; the brief, ideas, cards, jam and the rest stay on Flash brew/deploy.sh:23 memory brew-pro-switch-and-flush.
  • Arata, 54 matched Plays, 20 Oct: Pro median $0.18, 2.4 minutes, about 2,250 words, 10 facts, 20 sources; Opus median $0.83, 5.7 minutes, about 5,450 words, 54 facts, 62 sources memory brew-model-compare-2026-10-20.
  • Pro reused one stat on 8 Plays and wrote templated titles; Opus wrote brand-specific stats and derived money memory brew-model-compare-2026-10-20.
  • The comparison is confounded: Opus had the team's 19 Oct audit upload and Pro did not memory brew-model-compare-2026-10-20.

Where sources disagree

  • deploy.sh comments still say production runs every text model on Claude, while its own default and prod-env.sh both say Gemini brew/deploy.sh:27 brew/deploy.sh:41 brew/tools/prod-env.sh:9. The code is newer and wins.
  • live-watch.js header says the weekly check asks from India; the code asks from the brand's market brew/live-watch.js:4 brew/live-watch.js:37.

Tomorrow

Item Version Tag
One provider switch, set per deploy, with no code change v1 Stays
Plays on the best writing model we can afford; everything short on a fast model v1 Stays
Plays back on Claude Opus at effort medium once the Anthropic key has credit and a fair bake-off confirms it v1 Changes
A model registry file that names model, tier, effort and fallback per tag, read by ai.js, so the Prompts registry and the code cannot drift v2 New
Model choice per brand tier, for example Pro Plays for paying brands v3 New
Cheap open models for gates and classifiers, chosen by the eval set v3 New
Comments in deploy.sh that contradict the defaults v1 Goes
Vendor and model names on founder screens v1 Goes, per the book draft brand-directions/book/content/c2.js

How we do it

  1. Read today's settings before changing anything:
    git -C /Users/param/Documents/ChatGPT/brew show origin/main:deploy.sh | grep -o 'AI_PROVIDER=[^|]*\|GEMINI_[A-Z_]*=[^|]*'
git -C /Users/param/Documents/ChatGPT/brew show origin/main:tools/prod-env.sh | grep -o 'AI_PROVIDER=[^|]*\|GEMINI_[A-Z_]*=[^|]*'
  2. Switch staging to Claude, once there is Anthropic credit:
    AI_PROVIDER=anthropic ./deploy.sh
  3. Run staging all on Flash for cheap development:
    AI_PROVIDER=gemini GEMINI_MOVES_MODEL=gemini-3.5-flash ./deploy.sh
  4. Production changes go in tools/prod-env.sh, then tools/promote.sh brew/tools/prod-env.sh:1-2.
  5. Before a switch, raise the caps that follow the model: the build cap with BUILD_CAP_USD, the weekly rewrite pot with BRAND_WEEK_CAP_USD memory brew-pro-switch-and-flush brew/rewrite-pot.js:71.
  6. Tests must not read GEMINI_DEEP_MODEL or GEMINI_MOVES_MODEL from the shell; rewrite-pot.test.js deletes them memory brew-pro-switch-and-flush.
  7. Never call a model by a name Vertex does not serve. Served on squareup-492617 when probed on 16 Oct: gemini-3.5-flash, gemini-3-flash-preview, gemini-2.5-flash, gemini-3.1-pro-preview. 404 then: gemini-3-pro, gemini-3.5-pro, gemini-3.5-flash-preview memory brew-gemini-provider.
  8. After a switch, watch spend in Google Billing, Reports, Vertex AI, and the [ai] log lines that print model, seconds and dollars per call memory brew-gemini-provider brew/ai.js:95.

Numbers

Metric Definition Where measured Status
Cost per Play Dollars of the Play's own call, from meta.usd; a run costs about 1.5 times that with search, check and fix Play meta in the brand pack, rewrite-pot.js OVERHEAD brew/rewrite-pot.js:24 Live
Model mix Share of spend by model and tag meter.byTag in ai.js brew/ai.js:51 Live, not reported anywhere
Fallback rate Share of calls answered by a fallback model or provider media.js tried, finals provider Proposed
Billing pauses Count of 15-minute AI pauses a day ops alert ai-pause brew/ai.js:44 Live

Open

  1. Back to Claude for Plays? Proposal: top up Anthropic, run the fair bake-off from the Evals document on 5 Arata Plays, about $5 to $10, and switch Plays back to Opus medium only if Opus wins most blind pairs.
  2. Is Claude on Vertex covered by the Google credits? It has never been used memory brew-gemini-provider. Proposal: one small paid probe on Vertex with your yes, then decide whether Claude runs on the credits.
  3. Brief and ideas on Flash. They asked for Opus high and get Flash today. Proposal: move the brief to Pro alongside Plays, since every Play reads it.
  4. Jam spend has no dollar meter. Jam calls count only as daily units brew/server.js:146-171. Proposal: meter them through ai.js so cost per brand includes the jam.
Sources
  • brew origin/main b91e364: ai.js:7-82, gemini-client.js:11-110, moves-engine.js:15-17 and 471-473, server.js:116-118, 137-142, 166, 191, 220-240
  • brew origin/main: deploy.sh:21-41, tools/prod-env.sh:9, build.js:113-115 and 402, media.js:1-8, vertex-media.js:53-85, video-read.js:5 and 97, jam2/engines.js, data/engines.json, data/higgsfield-routes.json
  • brew origin/main: reader/stages/answers.js:106-110, live-watch.js:159, rewrite-pot.js:17-21
  • memory brew-gemini-provider, brew-pro-switch-and-flush, brew-model-compare-2026-10-20, brew-lean-pipeline, brew-sloosh-finals, brew-jam-v2, brew-draft-quality, brand-directions-2026-10-25

Source file content/ai-and-models/model-policy.md · For agents: Markdown · JSON