The gap between an agent demo and an agent in production is not intelligence — it's accountability. A demo has to work once, in front of a friendly audience. A production agent has to work on the ten-thousandth alert, at 3am, on an input nobody anticipated, and then explain itself when someone asks why. After a dozen agent deployments in regulated environments, we've concluded the differentiating work is almost entirely in the unglamorous layers.
1. Evals are the product spec
Before we write a system prompt, we build a golden set: a few hundred real cases with the answer a senior human would give. That set is negotiated with the client the way a spec used to be — and it becomes the contract. Every prompt change, model swap, or retrieval tweak runs against it in CI. When a stakeholder asks "did the update make it better?", the answer is a number, not a vibe.
If you can't measure the agent against cases you'd stake your name on, you don't have a product yet — you have a demo with latency.
2. Escalation is a feature, not a failure
The most trusted agents we've shipped are the ones that decline the most. Confidence thresholds, ambiguity detection, and "not my job" boundaries are designed explicitly: what the agent may decide alone, what it drafts for approval, and what it must hand to a human untouched. In our compliance deployments the split settles around:
3. The audit log is the sales deck
Every tool call, every retrieved document, every draft and revision — logged, timestamped, and replayable. This feels like overhead until the first audit, procurement review, or incident. Then it's the reason you keep the contract. We've watched a compliance officer approve an agent rollout specifically because she could reconstruct any decision after the fact. The model didn't win the deal; the log did.
4. Ship the smallest trustworthy loop
Our first production release is always narrower than the client asked for: one workflow, full accountability stack, humans approving everything. Trust compounds from there — thresholds loosen as eval scores hold, autonomy expands one decision class at a time. The teams that try to launch the full autonomous vision on day one spend the next two quarters rebuilding these boring parts under pressure. Build them first, and the exciting parts get to stay exciting.