Leapfrog Labs
Leapfrog · Companion Labs

The book is the why.This is where you build.

Every chapter of Leapfrog ends with something small and shippable. This is where those experiments live — with the working tools, starters, and current pointers the book points to.

Browse the labs Open the tools
How it works

A map, and a place to walk it

The book teaches the durable layer beneath the tools. The labs are where you get your hands dirty — and where the fast-moving specifics stay current.

Read

The book is the shape

Why AI is a must, where it belongs, and the architecture that survives the next model release.

Build

The labs are the reps

Each chapter's experiments, made concrete — small, real, shippable. Start with the one that fits your afternoon.

Stay current

The pointers move

Prices, models, and regulation change in weeks. The tools compute; the pointers link to primary sources.

The Labs

Experiments to run

Forty-six small, real tasks — one set per chapter — drawn straight from the book. No task needs more than an afternoon.

Part I

Why & Where

Why AI is a must, and where it actually belongs.

Chapter 1

The Convergence

  1. Spend $5 and a lunch break. Get an API key from any one provider your organization already touches, and make a single call from your laptop that sends a real internal question and gets an answer back. The goal is not the answer — it is to feel, viscerally, how low the barrier to entry actually is compared to what the "yes, but" culture told you.
  2. Find your smallest real task. Identify one genuinely small, genuinely real piece of work in your world — a report nobody enjoys writing, a triage step, a lookup — that a model could plausibly help with. Do not build it yet. Just write down what "delivered" would mean: who would use it, and how you'd know it worked. You are practicing the scarce skill — *knowing what to build* — before the easy one.
  3. Break something on purpose. Ask a model the same question five times and keep the answers side by side. Watch it be non-deterministic. Sit with the discomfort. This is the new muscle; better to meet it in a throwaway experiment than in production.
  4. Write your own "yes, but," then answer it. Take the objection you personally expect to hear first when you propose building something with AI at work, and draft the two-sentence rebuttal. You will use it sooner than you think.
Chapter 2

Where AI Belongs

  1. Run the spellchecker test. Take an AI idea someone in your organization is excited about and ask: would this end up with a Chief Something Officer and a budget line, or would it just disappear into how the work gets done? If the former, you're probably staring at a chatbot. Go find its embedded version.
  2. Map a workflow to the jagged frontier. Pick one real workflow, break it into its tasks, and label each: AI clearly helps, AI clearly hurts, or human-only. The labels will not match your first instinct — and that surprise is the entire point.
  3. Find the redesign, not the add-on. For that same workflow, write how it would run if it were *rebuilt* around AI — different steps, different handoffs — rather than AI stapled onto today's version. That description is the co-invention. It's the actual work, and most teams never write it down.
  4. Name the rung. For a feature you're considering, decide honestly which rung it should live on — assistant, embedded, or agentic — and whether what you're about to build is aimed there or quietly stuck on rung one.
Chapter 3

The Landscape Without the Hype

  1. Same prompt, two providers, one interface. Put a gateway (hosted or a lightweight self-hosted one) in front of two different providers and send the same real internal prompt to each. Compare the answers, the latency, and the cost side by side. The goal is not to crown a winner — it's to feel how little your code cares which one answered.
  2. Route by difficulty. Send an easy, high-volume task (say, classifying or summarizing) to a cheap small model, and a genuinely hard one to a frontier model. Measure the cost difference. That gap, multiplied by your real volume, is the money a multi-model strategy leaves on the table when you hard-code one model.
  3. Run the exit test. Ask honestly: if your main provider doubled its price or pulled your model tomorrow, is switching a config change or a multi-sprint project? Then actually try to reroute one workload and time it. Discover the answer now, on your terms, not during an incident.
  4. Break a provider on purpose. Wire up a fallback chain, then pull the credentials for your primary provider and watch traffic fail over to the backup. Resilience you haven't tested is a resilience you don't have.
Chapter 4

Cloud Foundations for the AI Engineer

  1. Run a model on your own machine tonight. Install a local runtime, pull a small open-weight model, and call its local endpoint from a script. Total cost: zero. The point is to feel, physically, how low the on-ramp is.
  2. Turn "local" into "hosted." Take that same little app and deploy it as a container behind a stable endpoint somewhere it stays up — however small. Notice the exact moment it stops being local, and notice that your application code barely changed. That's the "there is always a host" lesson, felt rather than read.
  3. Version a prompt like code. Put a prompt in a repository, change it in a pull request, and require one automated check to pass before it can merge. That's governance-as-code in its smallest possible form — build the habit at toy scale.
  4. Destroy it and bring it back. Define your environment entirely in infrastructure-as-code, tear the whole thing down, and recreate it identically from the code. The confidence that gives you *is* your license to experiment freely.
Chapter 5

How Models Are Served and Consumed

  1. Read your own meter. Run one real task and log the input tokens, the output tokens, and the cost separately. Then ask the model for a concise or structured answer and run it again. Watch the bill fall from the output side. You'll never again forget which direction is expensive.
  2. Pull all three levers. Take a workload you run repeatedly. Cache the system prompt, route the easy cases to a cheaper model, and move the non-urgent ones to batch. Measure cost per unit of work before and after — that delta is the difference between a pilot and a product.
  3. Pay the reasoning tax on purpose. Run the same hard task with reasoning off, low, and high. Line up quality, tokens, latency, and cost side by side, and find the point where more thinking stops earning its keep. That point is a design parameter, not a mystery.
  4. Find your real ceiling. Take a model with a big advertised window, bury one specific fact in the middle of a long input, and ask for it back. Whether it comes back tells you your *effective* context — the number the datasheet won't.
  5. Meter an image. Send the same picture to a vision model at high detail and at low detail, and log the token cost of each. Then compare a high-volume image workload against its text-only equivalent. That gap is why resolution is a cost lever, not a formatting choice.
Part II

Build the Thing

Patterns that hold up, and grounding a model in your data.

Chapter 6

Patterns That Hold Up

  1. Assemble the context on purpose. Take one task and deliberately design what goes into the window — the instructions, the few facts it actually needs, the output format — then strip half of it out and watch the quality move. That swing is context engineering, felt rather than read.
  2. Enforce a contract. Make a model call return strict, schema-valid JSON that your code parses with no defensive gymnastics. You've just converted a text generator into a typed component you can build on.
  3. Give it one tool — then sabotage it. Connect a single tool through the open protocol and watch the model call it. Then vague out the prompt until the model *has* the tool but stops using it well. That's the difference between integration and intelligence, in your hands.
  4. Resist the agent. Take the thing you're itching to build as an autonomous agent and build it as a fixed workflow instead. Ship that first. Reach for agency only when the workflow provably can't handle the variance — and notice how rarely that turns out to be true.
Chapter 7

Data, Retrieval, and Grounding

  1. Watch naive RAG fail. Build the simplest embed → top-k → stuff → generate pipeline on your own documents, then hunt for a query it gets wrong — one with an exact term, a part number, or a name, usually does it. That failure is the whole reason the rest of the stack exists.
  2. Add hybrid search and a reranker. Put keyword search alongside your vector search, fuse the results, and rerank the top candidates. Measure retrieval quality before and after on a set of real questions. Feel where the twenty-to-thirty-five percent comes from.
  3. Break the permission boundary — on purpose. Put two different users' documents in one index and check, honestly, whether user A can retrieve user B's data. Then fix it at retrieval, not at the interface. Far better you find this than an auditor does.
  4. Ground it, then check faithfulness. Make the model cite its sources, and then verify that the citations actually support the answer. The gap between "cited" and "supported" is the gap between genuinely grounded and merely confident.
  5. Run the fine-tuning decision honestly. Take a case where someone wants to fine-tune and sort the goal into *form* or *facts*. If it's facts, prove prompting and retrieval can't do it first. If it's form, check whether a stricter prompt and structured outputs (Chapter 6) already get you there. You'll usually talk yourself out of a training pipeline — which is the point.
Interlude

Ship Something Tiny This Week

Stop reading. Close one small loop end to end, before the deep chapters explain what you just did.

The Tiny Build

One workflow, one afternoon, training wheels on

  1. Pick the workflow & define "better." One real task you're allowed to touch and can measure. Write down the baseline and the target first.
  2. Make one real call. Send a real input to a hosted model and read your meter — tokens in, tokens out, cost.
  3. Ground it in a little real data. Feed it a handful of your own documents with the simplest retrieval. Watch where it helps and where it fails.
  4. Make it do one real thing. A structured output your code acts on, or one tool call. Now it's embedded, not a chat.
  5. Add a tiny check & a tiny bound. One eval on a few cases; least privilege, a cap, a human on anything irreversible.
  6. Show one real user. The person who does this workflow. Their reaction is your truest eval.

That's the whole loop this book is about — access, cost, grounding, action, evaluation, control — small enough to hold in your head. The gap between the 5% and the 95% was never knowledge. It was having shipped.

Part III

Survive Monday

Deploy it, keep it in control, and know it works.

Chapter 8

Deploying and Operating AI Workloads

  1. Put your build behind a deployed endpoint. Take the thing you built in Chapters 6–7, containerize it, deploy it somewhere it stays up unattended, and call it from outside. Cross the local-to-hosted line from Chapter 4 for real, and notice everything that "worked in the notebook" now needs a second look.
  2. Add an eval gate. Build a tiny golden dataset and wire it into your pipeline so a prompt change can't merge unless it passes. Then break a prompt on purpose and watch the gate stop you. That gate is what stands between you and a silent regression.
  3. Shadow a model swap. Route a slice of traffic to a different model or version, compare its outputs against the current one without showing them to users, and decide with data instead of a hunch. You've just rehearsed your defense against prompt drift.
  4. Give an agent a kill switch and a leash. Take a simple agent, scope its tools to least privilege, cap its steps and its spend, and add a stop you can hit. Then try to make it misbehave — and confirm the blast radius is exactly as small as you designed it. That confirmation is what "in control" feels like.
Chapter 9

Observability, Evaluation, and Debugging

  1. Trace one real request end to end. Instrument a single request and look at the whole tree — generation, retrieval, tool calls, each with its inputs, outputs, and cost. Notice how much your old logs never showed you.
  2. Score a golden set. Build a small set of representative inputs, pick one metric — faithfulness is a good first — and score your outputs to get a real number. Then curate five genuine failures back into the set. You've just started the feedback loop.
  3. Alert on quality, not latency. Wire an alert that fires when an evaluation score drops rather than when a request is slow. Break something on purpose and watch it catch what your infrastructure dashboards never would.
  4. Root-cause a bad answer. Take one wrong output and walk its trace to name the actual culprit. Bet, before you look, that it's the model — and notice how often it isn't.
Part IV

At Scale & Forward

Cost, performance, security, and the road ahead.

Chapter 10

Scaling, Performance, and Cost Control

  1. Attribute one feature's spend. Pass team-and-feature metadata through your gateway, join it to the bill, and produce a single number: the cost per transaction of one real feature. Then say that number out loud in your CFO's vocabulary, not in tokens.
  2. Find your value per token. For one workload, compute the cost per unit of value it produces — per resolved ticket, per booking, per document. Decide, with a straight face, whether it's worth it. That decision is the whole discipline in miniature.
  3. Cache, then verify. Add semantic caching to a repeated workload and measure the cost drop — then run the Chapter 9 eval to confirm quality held. A saving that degrades the product isn't a saving.
  4. Do the edge math honestly. Pick one latency-sensitive, high-volume workload and estimate centralized versus edge: latency, egress cost, residency — and the operational overhead of running a fleet. Let the full picture, not the latency number alone, make the call.
Chapter 11

Security, Governance, and the Road Ahead

  1. Red-team your own thing. Take something you built and attack it. Type a direct injection; then hide an instruction inside a document it will retrieve and watch whether it obeys. Fifteen minutes here teaches more than any threat report.
  2. Write the one-page risk note. For one system, on a single page: what it can do (its blast radius), what could go wrong (the two or three OWASP risks that actually apply), what controls bound it (Chapter 8), and who holds stop authority. That page is governance made real.
  3. Inventory your AI. List every AI system and feature running in your corner of the organization and classify each by risk tier. You will find shadow AI you didn't know about — and that list is where governance actually begins.
  4. Place yourself on the roadmap. Rate honestly where you and your organization are — one workflow, or attribution, or guardrails, or agents — then name the single next rung and go ship it. The roadmap only works if you take the first step on it.
The Tools

A companion that computes

Three working tools for the decisions the book keeps asking you to make. Everything runs in your browser — nothing is sent anywhere.

Token & Cost Calculator

Ch 5 · 10

Read your meter before the invoice does. Enter the shape of one call and your volume; see cost per call, per thousand, and per month. Output tokens are the expensive side — watch what happens when you cut them.

—
per call
—
per 1,000 calls
—
per day
—
per month (×30)
—

Workflow → Jagged-Frontier Mapper

Ch 2

Pick a workflow, not a chatbot. Break it into tasks and tag each against the frontier — where AI helps, where it hurts, where a human must stay. Then choose how the human and machine share the work, and the rung you're aiming for.

One-Page Risk Note

Ch 8 · 11

The whole of AI governance a busy engineer needs on a single page: what it can do, what could go wrong, what bounds it, and who can stop it. Fill it in, generate the note, hand it to whoever owns the risk.

Staying Current

Follow primary sources, not the hype

The book taught the durable layer on purpose. For the surface that moves monthly, go to the people who actually measure it.

Research · Economy

Stanford HAI — AI Index

The annual, independent state-of-AI report behind many of the book's numbers.
Research · Enterprise

MIT Sloan — Ideas Made to Matter

Enterprise AI, workflow redesign, and the co-invention that separates the 5%.
Security

OWASP — LLM & Agentic Top 10

The shared coverage map for prompt injection and the rest of the threat surface.
Governance

NIST — AI Risk Management Framework

What good practice looks like: govern, map, measure, manage — plus the GenAI profile.
Benchmarks · Pricing

Artificial Analysis

Independent model price, latency, and quality tracking — read the market, don't endorse it.
Regulation

EU AI Act — official

The risk tiers and the moving timeline. Always check the current status before you rely on a date.

Prices, model names, and regulatory dates change in weeks. Treat any specific number as current-as-of-reading, and verify it here before you build on it.

The Book

Leapfrog

Leapfrog — A Corporate Engineer's Field Guide to Delivering AI on the Cloud. The corporate trap is deferral dressed as prudence, and AI has revoked the patience that made it survivable. You were always qualified for this: delivering AI at scale is a cloud-engineering problem, and that's your ground. The book is the why and the shape; these labs are where you build.

These labs and the book are the author's personal opinions and experience — not official BMW guidance. The credit for the ideas belongs to the researchers and builders in the book's references; any mistakes are the author's own.