Outside data rules

Answer

How Brew reads data it does not own, through treg, Apify, the public Meta Ad Library and model web search: treg first, a free fallback wherever one exists, every paid answer cached and shared so it is paid for once, a cap on every source, and a failure never shown as a fact.

Today

The sources

The rules in code

1. treg first, Apify only where treg has nothing or is weak memory brew-apify-watch.
2. Data only through treg; assets never. Images, video and voice run on our own Vertex and OpenAI keys memory treg-usage-rules.
3. Every call goes through the shared client. Never fetch('https://treg.to/...') directly; every Ad Library call goes through adCall(treg, id, o) brew/reader/adlib.js:76 memory brew-external-data-skill.
4. Paid once per window. The same endpoint and parameters are paid for once per signal window, by any caller on any instance; identical calls in flight share one answer brew/treg.js:11-15 brew/treg.js:57.

5. No cache: false for read-only data. A caller that needs fresher data passes maxAgeH instead brew/treg.js:64-65 memory treg-usage-rules.
6. Re-reads reuse what we have: the read keeps the category and its search terms, and reuses the last rivals, terms and questions brew/reader/stages/answers.js:39-47 memory brew-lean-pipeline.
7. An error is never an empty result. Three branches every time: found, truly none, could not check and retries memory brew-external-data-skill. Apify rows say a platform could not be read, never "not sold there" brew/shelf.js:5.
8. A fallback must match the paid source's numbers, not just its shape. The Ad Library fallback reads the page itself for its full count; keyword results once gave Snitch 23 ads instead of 516 memory brew-external-data-skill brew/reader/adlib.js:62-75.
9. Every live call is measured for source health: ok, empty or error, time and cost; the router sends work to the healthiest source first brew/treg.js:76-79 memory brew-apify-watch.
10. Saved reads heal on open and on the check page, once a day or hourly while errors remain; a stage that reads differently bumps its version, such as ADS_V memory brew-external-data-skill.
11. Brand-agnostic. Never hardcode a brand; a bucket patch is a stopgap memory brew-external-data-skill.
12. No captcha bypass. The Ad Library reader stops when the page asks for a captcha brew/reader/adlib.js:37 brew/reader/adlib.js:54.

Caps

Preferred endpoints

The full map is treg-map.js on main; memory keeps the prices of 14 Oct memory treg-catalog-for-brew.

Tomorrow

How we do it

1. Load the brew-external-data skill before any change that reads or shows outside data memory brew-external-data-skill.
2. Check the live balance before saying treg is empty; never trust old logs memory treg-usage-rules:

   treg balance
   

3. Find stragglers that skip the shared client:

   git -C /Users/param/Documents/ChatGPT/brew grep -n "treg.to/call\|adlibrary" origin/main -- '*.js'
   

   Only treg.js, treg-map.js, adlib.js and guarded local tools should show memory brew-external-data-skill.
4. Audit every brand, then test with treg forced to fail:

   python3 /Users/param/Documents/ChatGPT/.claude/skills/brew-external-data/audit.py brew-prototype
   node /Users/param/Documents/ChatGPT/.claude/skills/brew-external-data/heal-test.mjs <slug>
   

5. Probe a new Apify actor on real brands before using it: node tools/probe-sources.mjs memory brew-apify-watch.
6. Tag every call <what>-<slug>, so spend is booked per brand in spend/<YYYY-MM>.json brew/treg.js:3-5 brew/treg.js:121.

Numbers

Open

1. The skill is not in the repo. It lives in the parent folder, not on brew origin/main. Proposal: move it into brew/.claude/skills so every Brew session loads it.
2. One cap per brand. treg is capped per day for all brands, Apify per brand per month. Proposal: add a per-brand monthly treg cap of $15 beside Apify's $10, both shown on the watch page.
3. The Apify token was pasted in chat on 21 Oct. Proposal: rotate it memory brew-viral-remakes.