Media Buyer turned Token Shark

AI on a budget

I use AI wherever it pays for itself, and I can show you the math on where it doesn’t.

A global subscription e-commerce companyCRM · Cost accounting

AI-personalized CRM offers, run as a cost line

I redesigned CRM offers so send timing and discount level were personalized per customer by AI. Every campaign also carried its own AI bill, so we could see whether the lift paid for the tokens.

We already had 50+ offer copy variants in rotation, with manual segmentation. What I built replaced that with "personalization on demand" — copy and offer level generated per customer instead of pre-written and hand-sorted into segments. That 100x'd our voice.

What I built

  1. 1
    Inputs
    • Inputs from the data warehouse
    • PII handling under GDPR
    Data warehouseGDPRDPA
    Details

    Inputs from the data warehouse. Order history, skips, pauses, cancellations, plan changes, tenure and discount history.

    PII handling under GDPR. Customer IDs were pseudonymized, and no names or emails went to the LLM. Data processing stayed in the EU, with a DPA in place.

  2. 2
    Agent decision
    • Model routing as a cost lever
    • Offers from a fixed menu
    • Orchestration in self-hosted n8n
    Claude HaikuClaude Sonnetn8n
    Haiku enrich / classify
    Sonnet offer copy
    Offer menuIllustration
    A cash discount today
    A free kitchen utensilExample pick
    A free CPG product
    A credit toward a future item
    Menu only — no invented offers
    Details

    Model routing as a cost lever. Claude Haiku handled enrichment and classification. Claude Sonnet wrote the offer copy. Each task went to the cheapest model that could do it.

    Offers from a fixed menu. The agent personalized offers within hard constraints. It could only choose from a fixed menu of approved discounts and incentives, so it could not invent an offer. Its job was to pick which offer and which message would resonate most with each customer. The menu included a cash discount today, a free kitchen utensil, a free CPG product and a credit toward a future item.

    Orchestration in self-hosted n8n. LLM calls ran in batches, with retries and execution logs per workflow.

  3. 3
    Execution
    • CRM execution in Braze
    Braze
    Details

    CRM execution in Braze. Canvas flows and Liquid personalization, with Connected Content and catalogs injecting each customer's send timing and discount tier.

  4. 4
    Measurement
    • Product analytics in Amplitude
    • Experiment design
    • Graded on revenue given up
    • Discount accounting
    AmplitudeHoldoutMDESeasonalityMulti-region
    What each agent was graded on
    • Revenue given up
    • Net revenue
    • Conversion lift
    • AI spend
    Details

    Product analytics in Amplitude. Cohorts for targeting, and funnels from send to open to reactivation or order.

    Experiment design. A randomized holdout, with the minimum detectable effect set in advance. Seasonality was controlled for, and the lift was verified in multiple regions before it was called a win. Guardrail metrics were unsubscribes, cancellations and discount cost per order.

    Graded on revenue given up. Agents were graded on revenue given up, meaning the revenue lost to the incentive. We always knew the net result per campaign.

    Discount accounting. Discounts counted as a revenue deduction, so personalization had to win on net revenue, not gross.

  5. 5
    Cost & audit
    • Cost reporting and pipeline
    • Dashboards
    • Financial framing
    • Audit practices
    Anthropic Admin APIData warehouseLookerTableau
    Details

    Cost reporting and pipeline. The Anthropic Usage & Cost Admin API, with one API key per workflow, so AI spend could be attributed to each project. A daily pull brought that spend into the warehouse, joined to campaign IDs.

    Dashboards. Looker and Tableau showed AI spend next to conversion rate, revenue and discount cost.

    Financial framing. AI spend was booked as a variable cost per campaign. We tracked cost per incremental conversion, incremental contribution margin and break-even lift, meaning the minimum conversion gain that pays for the tokens.

    Audit practices. Spend was allocated by project tag and shown back to team leads. The invoice was reconciled against metered usage every month, and we tracked variance against the token budget. Anomaly alerts flagged spend spikes.

Break-even lift
+2–4 pp
Conversion rate lift from the MVP
Per campaign
AI spend reported next to conversion rate and revenue

The MVP lifted conversion rate by 2 to 4 percentage points. AI spend now sits next to conversion rate and revenue, so every campaign answers whether the lift paid for the tokens.

Next phase · roadmap

The next step is token efficiency. The incremental lift stays fixed while spend comes down. The levers are prompt caching, batch API pricing, trimmed context and structured outputs, plus a cheaper model tier wherever an eval set shows no loss in quality.

Illustration: lift held flat, spend going down
45 growth marketers, analysts and PMs · project pods of 2–15Team systems · Agents

Multiplayer AI for project pods

Everyone on the team used AI, but each person used it alone. Specialists drafted their sections of a plan in separate chats, from separate sources. Each person got faster. The plans got worse, because nothing checked whether the sections agreed with each other.

Strategy vs. highlights. The strategy section was built on a set of experiments that called for efficiency. The monthly highlight in the same plan celebrated a new milestone with an optimistic read. SLT asked which one to believe, and whether we were cutting back or pushing forward.

Roadmap drift. Product changed its feature roadmap, and the marketing team whose test roadmap depended on it found out late. Changes got made and absorbed in private DMs instead of going through the pod.

The cost landed at review. Drafts piled up 50+ saved revisions, review calls turned into conflict resolution, and contradictions surfaced right before the plan went to SLT, when they were most expensive to fix. People also lost hours hunting for a dashboard or experiment a coworker had already found.

Multiplayer AI put the pod and one shared agent in the same Slack channel, working from the same sources, with the agent reading the whole plan before anyone reviewed it.

Before
Strategy chat
Growth chat
Technical chat
DM: “where’s that dashboard?”
Call to reconcile the two sections
Conflict found at review
Project plan draft · 50+ revisions
Where the time lands · 1 square = 40 min
one person, one doc40 min
pod of 21h 20m
pod of 1510h
every plan, every week×

40 minutes per person in draft review, multiplied by the pod. This repeats in every pod, on every doc, across a 45-person team.

After
# pod channel · team + agent
Agent: conflict between two sections, tagging both owners
Rubric check, then a person reviews
Project plan draft · under 10 revisions
Where the time goes instead
one person, one doc40 min saved
pod of 1510h back
token spendunchanged

Same token spend as single-player AI, more time back. People who were 6+ weeks into the system took on more clients and revenue streams.

What I built

  1. 1
    Shared sources
    • Shared drives
    • Meeting transcripts
    • Tool connections per project
    Google DriveGemini
    Ad platforms
    MetaTikTok
    Product analytics
    Amplitude
    App intelligence
    AppfiguresSensor TowerAppTweak
    Creative analytics
    MotionAtria
    Web/SEO
    SEMrushSimilarWeb
    Design
    Figma
    Details

    Shared drives. Google Drive shared drives replaced personal folders: PRDs, roadmaps, SOWs, experiment reports, UA strategies, creative and video.

    Meeting transcripts. Gemini meeting transcripts went into the pod's shared sources.

    Tool connections per project. Ad platforms (Meta, TikTok), product analytics (Amplitude), app intelligence (Appfigures, Sensor Tower, AppTweak), creative analytics (Motion, Atria), web/SEO (SEMrush, SimilarWeb) and design (Figma). Agents could see design systems and components in Figma, so creative and UX recommendations matched what already existed. The stack varied by project.

  2. 2
    Shared workspace
    • Agent per pod
    • Slack replaced Asana
    • Drafting from shared sources
    ClaudeSlackAsana
    Details

    Agent per pod. One Claude agent per pod, living in the pod's shared Slack channel instead of private chats.

    Slack replaced Asana. Slack channels replaced the Asana hub for tasks, scope questions and experiment readouts.

    Drafting from shared sources. The agent drafted sections from the shared sources. Doc version history tracked saved revisions.

  3. 3
    Checks before review
    • Consistency check
    • Rubric gate
    What the agent checks before a plan goes to review
    • It reads the whole draft plus the pod's shared sources, not one section at a time.
    • It checks that every experiment is cited with the same numbers and the same conclusion in every section.
    • When two experiments point different ways, it states each one's relative impact against the overall plan and which should take priority, so strategy and highlights don't send SLT mixed signals.
    • It checks that each section's tone and recommendation, efficiency-first or growth-first, matches the evidence the section cites.
    • It checks recommendations against the agreed scope and budget.
    • It checks test-roadmap dates against Product's latest feature roadmap.
    • It posts conflicts in the pod's Slack thread, tagging both section owners with the conflicting excerpts and the source. The owners resolve it in the thread, and the decision stays in the channel for the next plan.
    AgentIllustration

    Strategy (efficiency-first) and Monthly highlights (optimistic read on a new milestone) disagree. Relative impact on the overall plan: … @strategy-owner @highlights-owner — resolve here.

    Details

    Consistency check. The agent reads the whole plan against the pod's shared sources before review. The full checklist is on the card.

    Rubric gate. Each agent output passed a checklist before a person saw it: sources cited, numbers matching the dashboards, no conflict with other sections.

  4. 4
    Learn and adjust
    • Skills library
    • Retros
    • Roles and responsibilities
    • Time tracking
    Details

    Skills library. Overlapping skills merged into a shared library used across projects.

    Retros. Regular retros, with outcomes written back into the agent instructions.

    Roles and responsibilities. Roles and responsibilities rewritten over several rounds as the AI got stronger: who owns, reviews and approves each section.

    Time tracking. Hours per document measured from time logs.

↺ Agents improved together across projects · back to 1
−40 min
Draft-review call time per person, project pods of 2–15
50+ → under 10
Saved revisions per draft
Same token spend
As single-player AI, with more time saved
6+ weeks
People on the system this long took on more clients and revenue streams

Most contradictions were caught at the draft stage instead of at review. Fewer people felt interrupted by coworkers hunting for a dashboard, report or experiment.

Back to the workSay hello on LinkedIn