Cloud Foundations for the AI Engineer

You can run a capable model on your laptop tonight. You cannot ship it from your laptop. Everything between those two facts is this chapter.

The last two chapters were about why and what. This one is about the ground you build on — and it starts by dissolving a false choice that traps a lot of engineers before they begin: the idea that “doing AI” means either a toy on your machine or a giant cloud programme, with nothing in between.

The truth is simpler and more useful. You start local because it’s the fastest on-ramp in the history of the field. Then you discover that nothing you actually deliver stays local — there is always a host — and the machinery that runs whatever you ship, wherever you ship it, is the cloud control plane and software lifecycle you already know. Learn to see AI through that lens and the whole thing stops being foreign.

---

Start local — it’s the best on-ramp

The barrier to your first real experiment is one command.

Local model runtimes have matured into what is fairly described as Docker for language models: you pull a model by name, and the tool handles the download, the memory management, the hardware acceleration, and — critically — exposes the model behind a small local web API that mirrors the format the big cloud providers use. A capable open-weight model in the single-digit-billion-parameter range, quantized down to a few gigabytes, runs comfortably on an ordinary laptop, silently, on battery, offline, for free. Apple’s unified-memory machines punch well above their price here; a modest used GPU can replace a monthly API bill within months.

This matters for exactly the reason the whole book keeps returning to. Starting local removes every institutional obstacle at once. No cloud account. No procurement. No security review before you’re allowed to send a single prompt. The distance between “I wonder if a model could help with this” and “I just tried it on real data” collapses to an afternoon. This is where experiment or die literally begins, because it’s the one place you can build with zero permission.

It’s also genuinely useful, not just a sandbox. Local is the right tool for prototyping, for iterating on prompts a hundred times without a bill, for putting sensitive data through a model without it ever leaving your machine, and for running two or three models side by side to feel the differences from Chapter 3 with your own hands. And because the local runtime speaks the same API dialect as the hosted providers, the mature architecture is often hybrid — route routine work to a small on-device model, fall back to a cloud model for the hard cases — with the switch between them being a line of configuration rather than a rewrite.

Hold on to that last point, because it’s the hinge of the entire chapter.

But nothing real stays local

The moment anyone other than you needs the thing, local ends.

A teammate wants to use it. A scheduled job has to call it at 3 a.m. A product feature depends on it being there. The instant that happens, the requirements change from “runs on my machine when I’m looking at it” to “reachable, available, authenticated, maintained, and up when I’m asleep.” Your laptop is none of those things. So the model, or the app around it, gets wrapped in a container, placed behind a stable endpoint, and deployed somewhere that stays running — with authentication in front of it, scaling behind it, and a way to update it without taking it down.

That somewhere is a host. And here is the point worth saying plainly, because it quietly settles a lot of confused debates: there is always a hosting. “Local versus cloud” is the wrong axis. Local is where you develop; a host is where you deliver; and the same OpenAI-shaped API that let you swap a local model for a cloud one also means your application code barely notices the move. The app is portable. The hosting is not optional. Something, somewhere, always runs it — with an address, an owner, a bill, and a policy.

So the durable skill this chapter teaches is not a model runtime. Runtimes come and go. It’s the thing that runs whatever you deliver, wherever it runs: the control plane, and the lifecycle around it. That’s the rest of the chapter, and it’s almost entirely made of things you already know.

The control plane you already know

Strip away the AI vocabulary and the control plane for an AI system is the same set of primitives you’ve used for years. Four of them here; the fifth — how you define and reproduce all of it — gets its own section because it’s what makes experimentation safe.

Compute, plus one new primitive. Virtual machines, containers, and serverless functions behave exactly as you expect. The only genuinely new element is the accelerator — the GPU or specialized chip that runs the model. And the good news for a corporate engineer is that you mostly don’t touch it directly. Most of the time you consume inference: you call a managed endpoint and someone else runs the accelerator. You only meet the GPU face-to-face when you choose to self-host, and that’s a deliberate trade, not a default. It’s worth knowing that these accelerators are a genuinely scarce resource — the current generation was reportedly sold out well into 2026 against an enormous backlog, and inference is now the majority of all accelerator spending — because scarcity is what makes “just run it yourself” more expensive and more constrained than it sounds. When you do self-host, the industry has consolidated on a recognizable pattern: containers and Kubernetes underneath, a specialized inference engine to run the model efficiently, and a serving layer on top for autoscaling and routing — with the managed ML platforms used for experimentation and Kubernetes-based stacks taking over at production scale. You don’t need that machinery to start. You need to know it exists and when to reach for it, which is Chapter 8’s job.

Networking, with a meter running. Virtual private networks and private endpoints do here what they’ve always done: keep traffic — including your prompts and your data — off the public internet and inside a boundary you control. The AI-specific trap is the egress meter. Moving data costs money, and AI systems move a lot of it; a design that shuttles large volumes across regions or out of the cloud is a bill waiting to surprise someone. The old instinct applies with new force: data has gravity. Keep the compute near the data, not the other way around. If you came from the edge or CDN world, this is your native language — locality, latency, and caching were always the game.

Storage. Object storage is the backbone: it’s where your documents live for retrieval, where model weights sit, where logs accumulate. Nothing exotic; the same durable, cheap bucket you already use, now holding the raw material an AI system reads from.

Identity. Least-privilege access to models and to data, exactly as for any other resource. The one habit to internalize immediately: an API key to a model provider is a secret, with all that implies — stored in a secret manager, scoped tightly, rotated, never in a repo. A model endpoint is just another privileged resource, and it gets governed like one.

None of this is new to you. That’s the entire point. The control plane for AI is the control plane you already operate, with one new primitive bolted on.

The software lifecycle is the same lifecycle

AI does not get its own special software development lifecycle. It rides the one you already run — and recognizing that is what turns “we need an AI initiative” back into “we ship software, and some of it now calls a model.”

Source control is still the spine. What changes is how much of the system is now text you version. Prompts are text. The choice of model is configuration. Guardrails, retrieval settings, evaluation sets — all text, all belonging in the repository, all subject to review, diff, and rollback. The mental shift that saves teams the most pain is this: a prompt change is a code change. It can break production as thoroughly as a bad function, so it goes through the same door — a commit, a review, a history you can revert to.

CI/CD is still the pipeline. The same build-test-deploy machinery now also assembles and checks the AI parts of the system. You are not inventing a new delivery mechanism; you are extending the one you have.

Infrastructure-as-code is what makes the recklessness safe. Defining your environments in code — so any of them can be created, destroyed, and recreated identically — is the quiet hero of this whole book. It is precisely what lets experiment or die not become experiment and leave a smoking crater. You can stand up a full environment to try something, tear it down when you’re done, and know you left no orphaned resources leaking cost and no snowflake configuration nobody can reproduce. Reproducibility is the safety net stretched under the experimentation. Without it, moving fast is genuinely dangerous; with it, moving fast is just moving fast.

There’s a role shift hiding in here, and it’s the same one Chapter 1 promised. As more of the system becomes text that a model can help you produce, the highest-value work moves up the stack — away from typing lines and toward framing the problem, designing the system, validating the output, and governing the whole thing. Which brings us to the part you specifically shouldn’t leave for last.

Shift governance left

The corporate default is to treat governance as a gate at the end: build the thing, then submit it to a review board that says yes or no weeks later. That gate is exactly the “yes, but” culture from Chapter 1 in institutional form, and it’s where velocity goes to die.

The alternative is to shift governance left — to move the controls earlier, into the commit, the pull request, and the build, where problems are cheap to catch and cheap to fix. This isn’t a philosophy; it’s arithmetic. A governance violation can enter a codebase at four distinct moments — commit time, review time, build time, and release time — and each earlier catch is dramatically cheaper than the one after it. A secret caught in a pull request never reaches the main branch. A vulnerable dependency caught at build never ships. A bad prompt caught by a test never reaches a user.

Made concrete, shift-left governance is a set of automated checks distributed across the pipeline, expressed as policy-as-code so they’re enforced by the machine rather than remembered by a person: secret scanning and dependency review on every change; a software bill of materials so you always know what’s inside a release; provenance and attestation so you can prove how an artifact was built; and — the AI-specific one that ties into Chapter 9 — evaluation gates, where a change to a prompt or a model must pass a quality check before it’s allowed to merge. You bind the whole set together with a single required status check: nothing reaches the main branch unless every applicable governance check is green. The rules live in the repository, run automatically, and fail loudly. Modern governance frameworks even derive their default gates from a declared compliance posture, so the connection to formal standards is a configuration choice rather than a manual audit — but the mechanism is what matters here; the specific regulations are Chapter 11.

And this is where “there is always a host” comes back to bite. Every host has data policies, and those policies change. In 2026, a widely used AI coding assistant quietly shifted its lower tiers so that user interactions became training data by default unless you opted out, while its enterprise tier was exempt — a change that turned “do you train on our code?” into a question every regulated team suddenly had to ask of every host in its stack. The lesson isn’t about one vendor. It’s that the answer to that question is a governance fact you encode from the first commit, not a surprise you discover in a contract review after you’ve already shipped. You govern the hosting because there is always a hosting.

Done this way, governance stops being the thing that blocks delivery and becomes the thing that enables it: you move fast precisely because you can prove, at every step, that what you’re moving is defensible. That is the opposite of the review board, and it is how you leapfrog the “yes, but” without being reckless.

The prerequisite short list

Here is the payoff, and it’s deliberately short, because the whole argument of this book is that you do not need to master all of cloud before you deliver AI. You need this subset — and you can start acquiring it today, in parallel, without waiting for any migration to finish.

Need now, to deliver your first real thing: containers; a source repository with CI/CD; infrastructure-as-code for one reproducible environment; a way to call a model (a managed endpoint or a gateway from Chapter 3); secrets management for your keys; and one minimal governance gate on the pipeline.

Learn when you hit it, not before: Kubernetes and the self-hosting serving stack (only when you self-host at scale); GPU scheduling and the internals of accelerators; advanced multi-region networking; and the deeper reaches of model operations, which the next chapters open up as you need them.

That’s it. That’s the ground floor. The migration your organization keeps deferring is not a prerequisite for standing on it. Start local tonight, and know that the path from there to something real runs through machinery you already understand.

---