{
 "id": "6.1",
 "title": "Model policy",
 "department": "AI and models",
 "department_slug": "ai-and-models",
 "direction": null,
 "branded": false,
 "status": "draft",
 "owner": "Param",
 "last_verified": "Mon Oct 26",
 "holds": "Which model does what, Pro or Flash, the fallback, when to switch",
 "urls": {
  "html": "https://handbook.joinsquareup.com/ai-and-models/model-policy",
  "markdown": "https://handbook.joinsquareup.com/ai-and-models/model-policy.md",
  "json": "https://handbook.joinsquareup.com/ai-and-models/model-policy.json"
 },
 "other_directions": [],
 "tldr": {
  "headline": "Pro writes the Plays, Flash does the rest, Claude waits for credit",
  "points": [
   {
    "icon": "gear",
    "text": "One switch, AI_PROVIDER, sends every text call to Claude or to Gemini."
   },
   {
    "icon": "rocket",
    "text": "Plays alone run on Gemini Pro; brief, ideas, cards and jam stay on Flash."
   },
   {
    "icon": "money",
    "text": "Anthropic credit ran out on 20 Oct, so live moved to Gemini that day."
   },
   {
    "icon": "check",
    "text": "Failed calls fall back down a chain and billing failures pause callers for 15 minutes."
   },
   {
    "icon": "chart",
    "text": "Opus Plays were clearly richer than Pro, but the test was confounded by an audit upload."
   }
  ],
  "stat": {
   "value": "54",
   "label": "Arata Plays compared"
  },
  "decide": "Switch Plays back to Opus once Anthropic is topped up, after a fair 5-Play bake-off?",
  "mood": "waiting"
 },
 "sources": [
  "brew origin/main b91e364: ai.js:7-82, gemini-client.js:11-110, moves-engine.js:15-17 and 471-473, server.js:116-118, 137-142, 166, 191, 220-240",
  "brew origin/main: deploy.sh:21-41, tools/prod-env.sh:9, build.js:113-115 and 402, media.js:1-8, vertex-media.js:53-85, video-read.js:5 and 97, jam2/engines.js, data/engines.json, data/higgsfield-routes.json",
  "brew origin/main: reader/stages/answers.js:106-110, live-watch.js:159, rewrite-pot.js:17-21",
  "memory brew-gemini-provider, brew-pro-switch-and-flush, brew-model-compare-2026-10-20, brew-lean-pipeline, brew-sloosh-finals, brew-jam-v2, brew-draft-quality, brand-directions-2026-10-25"
 ],
 "open_questions": [
  "**Back to Claude for Plays?** Proposal: top up Anthropic, run the fair bake-off from the Evals document on 5 Arata Plays, about $5 to $10, and switch Plays back to Opus medium only if Opus wins most blind pairs.",
  "**Is Claude on Vertex covered by the Google credits?** It has never been used [src: memory brew-gemini-provider]. Proposal: one small paid probe on Vertex with your yes, then decide whether Claude runs on the credits.",
  "**Brief and ideas on Flash.** They asked for Opus high and get Flash today. Proposal: move the brief to Pro alongside Plays, since every Play reads it.",
  "**Jam spend has no dollar meter.** Jam calls count only as daily units [src: brew/server.js:146-171]. Proposal: meter them through ai.js so cost per brand includes the jam."
 ],
 "parts": {
  "Answer": "Which model writes, judges and draws each thing in Brew today, why Plays alone run on Gemini Pro while everything else runs on Gemini Flash, what takes over when a model fails, and the one deploy that switches back to Claude.",
  "Today": "### The switch\n\n| Setting | Staging, brew.joinsquareup.com | Production, feed.joinsquareup.com | What it does |\n|---|---|---|---|\n| `AI_PROVIDER` | `gemini` by default | `gemini` | `anthropic` sends every text call to Claude; anything else goes to Gemini on Vertex [src: brew/deploy.sh:41] [src: brew/tools/prod-env.sh:9] [src: brew/gemini-client.js:11] |\n| `GEMINI_MOVES_MODEL` | `gemini-3.1-pro-preview` | `gemini-3.1-pro-preview` | The model that writes Plays when the provider is Gemini [src: brew/moves-engine.js:17] |\n| `GEMINI_DEEP_MODEL` | `gemini-3.5-flash` | `gemini-3.5-flash` | Every call that asks for Opus: brief, ideas, other deep jobs [src: brew/gemini-client.js:13-14] |\n| `GEMINI_FAST_MODEL` | `gemini-3.5-flash` | `gemini-3.5-flash` | Every other call: Sonnet, Haiku and jam calls [src: brew/gemini-client.js:13-14] |\n| `GEMINI_IMAGE_MODEL` | `gemini-2.5-flash-image` | `gemini-2.5-flash-image` | Draft pictures [src: brew/server.js:118] [src: brew/tools/prod-env.sh:9] |\n| `MOVES_EFFORT` | unset, so `medium` | unset, so `medium` | Thinking depth for Plays [src: brew/build.js:402] |\n\n- The code asks for Claude names everywhere; `geminiModel()` maps any name with \"opus\" to the deep model and every other name to the fast one [src: brew/gemini-client.js:14].\n- Claude effort maps to Gemini thinking level: low to LOW, medium and high to MEDIUM, xhigh and max to HIGH [src: brew/gemini-client.js:27].\n- Gemini thinking counts against its output limit, so the shim adds 16,000 tokens of headroom to thinking jobs and 2,000 to the rest, capped at 65,536 [src: brew/gemini-client.js:68].\n- Search plus schema calls on Gemini took 8 to 95 seconds when measured [src: memory brew-gemini-provider].\n- Google startup credits pay for Vertex: $20,763 of $25,000 left on the 2024 GFS Cloud Program, ending 2028-05-19, as of 16 Oct [src: memory brew-gemini-provider].\n- The Anthropic key ran out of credit on 20 Oct at 07:50 IST, and live moved to Gemini that day [src: memory brew-pro-switch-and-flush] [src: brew/deploy.sh:25].\n\n### Who does what today\n\n| Job | Model the code asks for | Model that runs today | Source |\n|---|---|---|---|\n| Plays, all of them | `MOVE_MODEL()`, Opus on Claude | `gemini-3.1-pro-preview`, effort medium | [src: brew/moves-engine.js:17] [src: brew/moves-engine.js:473] |\n| Play checker and fixer | Sonnet low, Sonnet medium | `gemini-3.5-flash` | [src: brew/moves-engine.js:553] [src: brew/moves-engine.js:568] |\n| Brand brief | Opus high | `gemini-3.5-flash` | [src: brew/brief.js:108] |\n| Ideas slate and writing | Opus medium, Opus high | `gemini-3.5-flash` | [src: brew/ideas.js:382] [src: brew/ideas.js:400] |\n| Reader stages, cards, plain words, learnings, memory, listening | Sonnet | `gemini-3.5-flash` | see the Prompts registry |\n| Jam and Studio words, chat, briefs | `claude-sonnet-5-5`, effort low, thinking off | `gemini-3.5-flash` | [src: brew/server.js:116] [src: brew/server.js:139-142] |\n| Product check and look judge on pictures | `claude-haiku-5-5` through `IMAGE_CHECK_MODEL` | `gemini-3.5-flash` | [src: brew/server.js:223] [src: brew/server.js:233] |\n| Video reading | `VIDEO_MODEL`, default `gemini-3.5-flash`, then `gemini-2.5-flash` | same | [src: brew/video-read.js:5] [src: brew/video-read.js:97] |\n| Draft pictures | `gemini-2.5-flash-image`, then `gemini-3-pro-image`, then OpenAI `gpt-image-1` | same | [src: brew/server.js:118] [src: brew/server.js:191] [src: brew/media.js:4] |\n| Draft video | `veo-3.1-fast-generate-001`, then Veo 3.1 | same | [src: brew/vertex-media.js:70] [src: brew/media.js:5] |\n| Voice | `gemini-2.5-flash-tts`, then OpenAI TTS | same | [src: brew/vertex-media.js:53] [src: brew/media.js:6] |\n| Music | `lyria-002` | same | [src: brew/vertex-media.js:62] |\n| Studio shots and films | Higgsfield first where routed, Vertex second | depends on Higgsfield keys | [src: brew/data/engines.json] |\n| Finals | Sloosh over MCP, Gemini or Veo as fallback | same | [src: brew/finals.js:8-13] |\n\n### Fallbacks\n\n- Claude calls opt into the server-side refusal fallback: `betas: ['server-side-fallback-2026-07-01'], fallbacks: 'default'` [src: brew/ai.js:71].\n- A billing failure pauses every caller for 15 minutes and emails ops once a day, instead of retrying every few seconds [src: brew/ai.js:34-45].\n- Gemini retries a 429 or 5xx three times with backoff of 4, 8 and 16 seconds [src: brew/gemini-client.js:110].\n- A Play rejected by structured output is asked again as plain JSON [src: brew/moves-engine.js:483].\n- Media chains hand each failure to the next provider and record who made the result [src: brew/media.js:1-8].\n- A Studio fallback partner never costs more than the price shown, except up to 1.5 times after a refunded refusal [src: brew/jam2/engines.js:131-139].\n\n### Honesty rule tied to the model\n\n- Under Gemini, the Claude question in the AI answers check sits out, because Gemini's answer would be labelled Claude [src: brew/reader/stages/answers.js:110] [src: brew/live-watch.js:159].\n\n### Why Pro only for Plays\n\n- Param, 20 Oct: only the Plays run on a higher model; the brief, ideas, cards, jam and the rest stay on Flash [src: brew/deploy.sh:23] [src: memory brew-pro-switch-and-flush].\n- Arata, 54 matched Plays, 20 Oct: Pro median $0.18, 2.4 minutes, about 2,250 words, 10 facts, 20 sources; Opus median $0.83, 5.7 minutes, about 5,450 words, 54 facts, 62 sources [src: memory brew-model-compare-2026-10-20].\n- Pro reused one stat on 8 Plays and wrote templated titles; Opus wrote brand-specific stats and derived money [src: memory brew-model-compare-2026-10-20].\n- The comparison is confounded: Opus had the team's 19 Oct audit upload and Pro did not [src: memory brew-model-compare-2026-10-20].\n\n### Where sources disagree\n\n- deploy.sh comments still say production runs every text model on Claude, while its own default and prod-env.sh both say Gemini [src: brew/deploy.sh:27] [src: brew/deploy.sh:41] [src: brew/tools/prod-env.sh:9]. The code is newer and wins.\n- live-watch.js header says the weekly check asks from India; the code asks from the brand's market [src: brew/live-watch.js:4] [src: brew/live-watch.js:37].",
  "Tomorrow": "| Item | Version | Tag |\n|---|---|---|\n| One provider switch, set per deploy, with no code change | v1 | Stays |\n| Plays on the best writing model we can afford; everything short on a fast model | v1 | Stays |\n| Plays back on Claude Opus at effort medium once the Anthropic key has credit and a fair bake-off confirms it | v1 | Changes |\n| A model registry file that names model, tier, effort and fallback per tag, read by ai.js, so the Prompts registry and the code cannot drift | v2 | New |\n| Model choice per brand tier, for example Pro Plays for paying brands | v3 | New |\n| Cheap open models for gates and classifiers, chosen by the eval set | v3 | New |\n| Comments in deploy.sh that contradict the defaults | v1 | Goes |\n| Vendor and model names on founder screens | v1 | Goes, per the book draft [src: brand-directions/book/content/c2.js] |",
  "How we do it": "1. Read today's settings before changing anything:\n   ```bash\n   git -C /Users/param/Documents/ChatGPT/brew show origin/main:deploy.sh | grep -o 'AI_PROVIDER=[^|]*\\|GEMINI_[A-Z_]*=[^|]*'\n   git -C /Users/param/Documents/ChatGPT/brew show origin/main:tools/prod-env.sh | grep -o 'AI_PROVIDER=[^|]*\\|GEMINI_[A-Z_]*=[^|]*'\n   ```\n2. Switch staging to Claude, once there is Anthropic credit:\n   ```bash\n   AI_PROVIDER=anthropic ./deploy.sh\n   ```\n3. Run staging all on Flash for cheap development:\n   ```bash\n   AI_PROVIDER=gemini GEMINI_MOVES_MODEL=gemini-3.5-flash ./deploy.sh\n   ```\n4. Production changes go in tools/prod-env.sh, then tools/promote.sh [src: brew/tools/prod-env.sh:1-2].\n5. Before a switch, raise the caps that follow the model: the build cap with `BUILD_CAP_USD`, the weekly rewrite pot with `BRAND_WEEK_CAP_USD` [src: memory brew-pro-switch-and-flush] [src: brew/rewrite-pot.js:71].\n6. Tests must not read `GEMINI_DEEP_MODEL` or `GEMINI_MOVES_MODEL` from the shell; rewrite-pot.test.js deletes them [src: memory brew-pro-switch-and-flush].\n7. Never call a model by a name Vertex does not serve. Served on squareup-492617 when probed on 16 Oct: gemini-3.5-flash, gemini-3-flash-preview, gemini-2.5-flash, gemini-3.1-pro-preview. 404 then: gemini-3-pro, gemini-3.5-pro, gemini-3.5-flash-preview [src: memory brew-gemini-provider].\n8. After a switch, watch spend in Google Billing, Reports, Vertex AI, and the `[ai]` log lines that print model, seconds and dollars per call [src: memory brew-gemini-provider] [src: brew/ai.js:95].",
  "Numbers": "| Metric | Definition | Where measured | Status |\n|---|---|---|---|\n| Cost per Play | Dollars of the Play's own call, from `meta.usd`; a run costs about 1.5 times that with search, check and fix | Play meta in the brand pack, rewrite-pot.js `OVERHEAD` [src: brew/rewrite-pot.js:24] | Live |\n| Model mix | Share of spend by model and tag | `meter.byTag` in ai.js [src: brew/ai.js:51] | Live, not reported anywhere |\n| Fallback rate | Share of calls answered by a fallback model or provider | media.js `tried`, finals `provider` | Proposed |\n| Billing pauses | Count of 15-minute AI pauses a day | ops alert `ai-pause` [src: brew/ai.js:44] | Live |",
  "Open": "1. **Back to Claude for Plays?** Proposal: top up Anthropic, run the fair bake-off from the Evals document on 5 Arata Plays, about $5 to $10, and switch Plays back to Opus medium only if Opus wins most blind pairs.\n2. **Is Claude on Vertex covered by the Google credits?** It has never been used [src: memory brew-gemini-provider]. Proposal: one small paid probe on Vertex with your yes, then decide whether Claude runs on the credits.\n3. **Brief and ideas on Flash.** They asked for Opus high and get Flash today. Proposal: move the brief to Pro alongside Plays, since every Play reads it.\n4. **Jam spend has no dollar meter.** Jam calls count only as daily units [src: brew/server.js:146-171]. Proposal: meter them through ai.js so cost per brand includes the jam."
 }
}