Build-Not-Run Architecture: Keep the Language Model Out of the Live Execution Path

Build-Not-Run Architecture: Keep the Language Model Out of the Live Execution Path

The most reliable production AI we see does something counterintuitive: it keeps the language model out of the live path of most requests. You use a frontier model up front to design the queries, pipelines, and procedures. Then cheap deterministic infrastructure runs them, and you reserve expensive inference only for genuinely novel work. Design with the model; run without it.

THE WRONG DEFAULT

Why does putting a model in the live path feel right?

Putting the model in front of every request feels right because it is the fastest thing to ship. The model is flexible, it answers anything, and a demo comes together in an afternoon. That same flexibility becomes the failure mode the moment real volume arrives.

A common pattern for operators is to wire a model directly into the live flow and call it done. Every lookup, every classification, every routine transformation now routes through inference. It works beautifully in the demo, because a demo runs the same happy path a handful of times.

In the engagements we run, this is the design that breaks first. The model is nondeterministic by nature, so the same input can produce different output on different days. When that variability sits in the hot path of work that should be boringly repeatable, you get flaky behavior, surprise costs, and a system nobody trusts.

The instinct is not wrong about the model's power. It is wrong about where to spend it.

THE TWO KINDS OF WORK

What work actually needs a model in the moment?

Almost all production work splits into two kinds, and only one of them needs live inference. The first kind is repeatable: the same shape of request, over and over, where a correct procedure exists and can be written down. The second kind is genuinely novel: a request you could not have anticipated, where reasoning has to happen fresh.

Repeatable work does not need a model at runtime. It needs a model once, up front, to figure out the right procedure. After that, the procedure is just code, a query, or a pipeline that runs the same way every time for a fraction of a cent.

Novel work is where live inference earns its cost. When a request is truly new, no precompiled path exists, and a model reasoning in the moment is exactly the right tool. The mistake is treating all work as novel because the model can technically handle all of it.

Sorting these two is the whole game. Most teams never draw the line, so they pay novel-work prices for repeatable-work volume.

THE FRAMEWORK

Build-Not-Run Architecture, defined

Build-Not-Run Architecture is a way of placing the model deliberately instead of by default. You let a frontier model design the system, then you let cheap deterministic infrastructure run it. The model is a designer and a fallback, not the engine of every request.

This is the Vista framework we use to make AI systems hold up in production. It rests on three moves.

Build-Not-Run Architecture. A production design that uses a frontier model up front to design the queries, pipelines, and procedures, then lets cheap deterministic infrastructure run them, reserving live inference only for genuinely novel requests. The model builds the system; it does not run the system. The principle in one line: design with the model, run without it.

The three moves are the load-bearing part of the design:

1. Design with the model, not in the model. Use the frontier model to author the query, the procedure, the decision tree. Capture its output as durable artifacts you can read, version, and run. The intelligence gets compiled into infrastructure, not re-purchased on every call.
2. Run on deterministic infrastructure. Once the procedure exists, the runtime is ordinary code, queries, and rules. It is fast, cheap, and produces the same result every time, which is exactly what repeatable work demands.
3. Reserve inference for the genuinely novel. Route only the requests that no precompiled path covers to a live model. This is the escape hatch that keeps the system flexible without putting the model in front of everything.

TWO LANES, SIDE BY SIDE

How do the two designs compare in production?

The difference is not how smart the system is. It is where the intelligence sits and what runs at request time. One design pays for a model on every call; the other pays once to design, then runs nearly free.

Dimension	Model in the live path	Build-Not-Run
Cost at volume	Scales with every request	Mostly fixed, paid once up front
Reliability	Nondeterministic, drifts	Same input, same output
Latency	Inference on every call	Fast deterministic runtime
Debuggability	Opaque, hard to reproduce	Readable, versioned artifacts
What breaks	Costs and behavior at scale	Only the genuinely novel edge

The right-hand column has a quieter advantage operators undervalue: you can read what the system does. A compiled procedure is an artifact you can inspect, test, and fix. A model in the hot path is a black box you can only coax.

Design with the model. Run without it. Pay for inference only when the work is truly new.

THE PATTERN WE SEE

Why do the model-in-everything systems run up shocking bills?

The bill is the symptom; the cause is the model running work it never needed to run. When a model sits in the path of every request, your token cost is multiplied by your volume, and volume is exactly what a successful system produces. Growth becomes the thing that hurts you.

In the engagements we run, the systems that hold up are the ones where a model designed the system but does not run every request. The ones that break, and post the surprising invoices, are the ones that put a model in the hot path of everything, including work a simple query could have handled.

The fix is rarely a cheaper model. It is moving the repeatable work off live inference entirely. Once that work runs on deterministic infrastructure, the bill stops tracking your volume and starts tracking only your genuinely novel requests, which is a far smaller number than most teams expect.

That is the quiet engine of the framework. You are not trying to use less intelligence. You are trying to stop re-buying the same answer.

BUILD IT THIS QUARTER

How to apply Build-Not-Run Architecture

Start by sorting your work, not by picking a model. Look at what actually flows through your system and split it into repeatable and novel. Here is the sequence we walk operators through.

1. Inventory your requests by repeatability. List the real request types your system handles. Mark each one repeatable or genuinely novel. Most teams are surprised how much of the volume is repeatable work in disguise.
2. Use the model to design the repeatable paths. For each repeatable type, have a frontier model author the query, procedure, or pipeline once. Treat its output as a durable artifact, not a throwaway answer.
3. Compile it into deterministic infrastructure. Move the designed procedure into ordinary code, queries, and rules that run the same way every time. This is the runtime for everything repeatable.
4. Route only the novel work to live inference. Build a clear path for requests no precompiled procedure covers. That is where a live model belongs, and where its cost is justified.
5. Watch the bill track novelty, not volume. Once the split is real, your inference cost should rise only when genuinely new work rises. If it tracks total volume, repeatable work is still leaking into the hot path.

A practical place to pressure-test this thinking is the free Vista AI Lab, where we work through where the model belongs in real operator systems. If you want this applied to your own stack with ongoing help, the Vista AI Collective is the membership where operators build production AI on this kind of architecture together.

THE BIGGER MOVE

The architecture is a skill, not a tool choice

Build-Not-Run Architecture is not about a specific model or vendor. It is a habit of asking, for every request, whether you are designing a system or running one. The durable skill is knowing which work is genuinely novel and worth live inference, and which is repeatable and worth compiling into infrastructure once.

That instinct is what separates AI systems that survive production from ones that look great in a demo and collapse under their own bill. When an operator's AI is fragile or expensive, the real constraint is usually not the model. It is a design that asked the model to run work it should only have designed.

Fixing that forces the better question: where does intelligence belong, and where does it just cost you? Answering it well is the difference between an AI system you trust at scale and one you quietly turn off.

Frequently asked questions

What is Build-Not-Run Architecture?

It is a Vista framework for production AI that uses a frontier model up front to design queries, pipelines, and procedures, then runs them on cheap deterministic infrastructure. Live inference is reserved only for genuinely novel requests. The principle is short: design with the model, run without it, so the system stays cheap and reliable.

Why not just put a model in front of every request?

Because the model is nondeterministic and priced per call. In the live path of every request, it makes repeatable work flaky and ties your cost directly to your volume. The systems that hold up in production keep the model out of that hot path and run repeatable work on deterministic infrastructure instead.

How do I know which work needs live inference?

Sort requests into repeatable and genuinely novel. Repeatable work follows a procedure you can write down once and run forever, so it does not need a model at runtime. Genuinely novel work has no precompiled path and benefits from fresh reasoning. Most volume is repeatable in disguise, which is where the cost savings hide.

Does this mean I use AI less?

No. You use the model more deliberately, often more deeply, but in design rather than in every runtime call. The intelligence still shapes the system; it just gets compiled into infrastructure once instead of being re-purchased on every request. You spend on inference where it earns its cost, not everywhere by default.

Will this make my AI system slower to build?

Slightly slower to design, much cheaper and steadier to run. Putting a model in the live path is the fastest demo and the most expensive production system. Designing the repeatable paths once costs an extra step up front, then pays back every time that work runs without an inference call behind it.