Anyone can demo an AI agent now; that stopped being impressive a while ago. The interesting question is what the systems still running a year later have in common. Across the ones we operate, the answer is consistent — and none of it is glamorous.
An eval suite instead of a vibe check
Every production system we run has a set of real cases with known-good outputs, and every change — prompt, model, retrieval, schema — runs against it before it ships. Without that, "we improved the prompt" means "it felt better on the three examples we tried". Model upgrades are the sneaky one: a newer, smarter model can quietly break formatting or tone that downstream steps depend on. The eval suite catches it; vibes do not.
Guardrails, and a human on the edges
Structured outputs validated against schemas. Hard bounds on anything that touches money. And a designed lane for the cases the system should not decide: our document pipeline clears the backlog precisely because the odd malformed document goes to a person instead of jamming the machine or — worse — being guessed at. The human review lane is not a temporary crutch to remove later; it is a permanent load-bearing part of the design.
You cannot fix what you cannot replay
When someone says "the assistant gave a weird answer yesterday", the difference between a five-minute fix and a shrug is whether you can replay exactly what happened: the input, the retrieved context, the model call, the output, the validation result. Every step logged, every decision traceable. This is ordinary software engineering discipline applied to systems people keep treating as magic.
Inputs drift. Someone has to notice.
The world the system was built against changes: suppliers redesign their invoices, product catalogs grow, users start asking new kinds of questions. Production systems need the boring feedback loop — corrections from the human lane flowing back into evals, metrics that make drift visible before users complain, and an owner whose job includes looking at them.
The boring stack wins
Notice what is missing from this list: exotic frameworks, autonomous swarms, anything from this week’s launch videos. The systems that survive are narrow, observable, validated, and owned. That is also why estimating them is possible at all — infrastructure is predictable in a way that research is not.
