Reliable Agentic AI: The Patterns I Run in Production (Not the Hype)

I run two systems with AI at their core in production: Tamperlens, which analyzes documents, and an internal agent that coordinates the operations of one small business over WhatsApp — mine, today, with one client pilot provisioned. Neither is impressive to look at. Neither has a magic chat that does everything. What they have is a handful of engineering decisions that make them reliable — and reliable, in an AI system, means one specific thing: the worst case is safe, not that the average case is pretty.

This post is about those patterns. None of them is about a “magic prompt.” All of them are about what happens when the model is wrong, when the process dies mid-flight, when the reply is slow, when silence could be mistaken for a “yes.” Because in a demo the model gets it right; in production it gets it wrong, and the system has to stay safe when it does.

Pattern 1: the published number derives from a test, not an opinion

I’ll start with the cheapest to describe and the most expensive to maintain. No number I publish about a system’s behavior can be typed by hand. It has to be produced by a test that runs, and the test has to break when the number stops being true.

At Tamperlens this is literal. The matrix that describes which evasion techniques defeat which signals is not a table I wrote by reading the code — each cell came from running a file through a real tool (qpdf, Ghostscript, exiftool) and re-inspecting the result, with the clock pinned to a known instant so the test is deterministic. The raw result is recorded in a versioned JSON, and there’s a test that compares that JSON against the engine and against the published post. When the engine version moves past the version recorded in the file, the suite fails and names the command that needs to run again.

Why does this matter? Because the alternative — a number that was once true, written in a post nobody revalidates — is how documentation rots. And in an AI system the temptation is doubled, because the model is fluent: it produces a confident number effortlessly, and fluency is indistinguishable from correctness until someone checks. When I did that re-inspection, in fact, six of the cells I had deduced from the code were wrong. The deduction was plausible. The measurement was something else. That’s why the rule is: the published number derives from the test, and the test is the source of truth — not my reading of the code, not the model’s answer, the test.

Pattern 2: subagents that verify adversarially

The owner’s-agent architecture isn’t one brain. It’s a coordinator that routes to subagents per area — marketing, support, finance, ops, legal — each defined by files (a manifest, knowledge folders, skills), not by code. Deploying to another company is filling folders and pointing an inbox, not editing Python. This runs on deepagents on top of LangGraph, which provides planning, subagents, and a working file system out of the box.

But the part that makes it reliable isn’t the division of labor — it’s using one subagent to attack what another produced, before a human sees it. Before a meeting with a pilot client, I ran a subagent whose only job was to be adversarial: to criticize the product like a skeptic would, to look for what breaks. It found things the optimistic flow would never find — including that a planned database migration would lose records silently, no error, no log, just gone.

That’s the general point: an agent asked “is this good?” tends to agree with itself — it’s fluent and cooperative by construction. An agent whose task is “find what’s wrong here” runs on the inverted incentive, and the inverted incentive is what exposes the flaw. It’s the same logic as adversarial code review between humans, applied inside the AI loop: you don’t trust the first output; you trust the output that survived a second agent trying to knock it down.

Pattern 3: the human approval loop, and the capability firewall

The system’s most important rule is the simplest to state: every action that leaves the company is an explicit tool, and every tool that sends, publishes, or triggers something interrupts and waits for a human. Reading is free; acting on the world is not. The agent can fetch email, read a bank balance, draft a reply, assemble an invoice — no permission needed. Sending the email, issuing the invoice, triggering the automation, putting something on the calendar — each of those interrupts and becomes a question to the owner over WhatsApp.

This is called a capability firewall, and the design choice is deliberate: the boundary isn’t “how confident the model is,” it’s “is this action reversible?” Stripe access is read-only by design — not because the agent is dumb, but because there’s no reason for it to have the capability to charge anyone, so it doesn’t. Trust in the model is never the variable that decides whether an irreversible action happens. The capability simply doesn’t exist on the wrong side of the firewall.

And the approval is a real conversation: the owner’s reply in the thread is the decision. “Yes” approves; “no” rejects; any other text rejects with that text as the reason. The conversation itself, with the support system’s notes, is the audit trail — there’s no parallel approval dashboard someone forgets to check.

Pattern 4: the agent can’t approve itself by silence

This is the pattern I’d defend above all others, and it’s one line of code. When the system translates the human’s reply into a decision, the default is to reject. Empty reply? Reject, with the reason “no answer from the owner.” Timeout, lost message, ambiguity that matches neither “yes” nor “no”? Reject.

Silence is never consent. A system that reads “didn’t answer” as “go ahead” is a system that will execute its most dangerous action at exactly the moment the human was distracted, offline, or too confused to reply — the worst possible moment. By making silence mean “no,” the failure leans to the safe side: the worst case of a missed approval is an action that didn’t happen and needs to be repeated, never an irreversible action that happened with nobody deciding.

It’s the exact inverse of how most excited systems are built, where friction is the enemy and the happy path assumes the “yes.” In a system that touches money and client communication, friction in the right place is the feature.

Pattern 5: at-least-once, because the process will die mid-flight

The last pattern is the least glamorous and the one that most separates a demo from production. The webhooks that wake the agent arrive from a system that redelivers: if it doesn’t get its “200 OK” in time, it sends again. That’s an at-least-once guarantee, and it forces two obligations.

First, idempotency: each message carries an id, and the system records the ids it has already processed. The same message arriving twice is processed once. Without this, a redelivery becomes a duplicate invoice or an email sent twice — and the worst concurrency case here is exactly that: an irreversible action repeated. (This is also why there’s a per-conversation lock: two webhooks from the same conversation — the “yes” and the audio right behind it — don’t run in parallel and scramble the pending decision.)

Second, durability against the restart. The payload of every accepted webhook is written before the background task is scheduled, with status “received.” If the process dies mid-flight — deploy, crash, whatever — on restart the service sweeps the events still marked “received” (accepted with 200, never completed) and reprocesses them, oldest to newest. A message that was accepted is never silently lost because the container restarted.

Notice none of this is AI. It’s plain distributed-systems engineering — at-least-once, idempotency, durability, locks — applied around the model. And that’s exactly where the reliability lives. The model is the least predictable component in the system; reliability comes from surrounding it with components that are predictable, and from making the worst case of each of them safe.

The thread that stitches it together

If there’s a single principle behind the five patterns, it’s this: at every point where the system can fail, the failure leans to the safe side. The unmeasured number isn’t published. The unattacked output isn’t trusted. The irreversible action doesn’t happen without a human. Silence doesn’t become yes. The accepted message isn’t lost on restart.

None of this is about the model being smarter. It’s about designing the system so that the model’s smartness is optional for its safety. That’s the difference between an agent that demos well and an agent you’d let run over someone else’s money and clients — and it’s the only part of “agentic AI” worth taking seriously.

If you want an agent built this way for your business — not a chatbot, an agent with a capability firewall, an approval loop, and a designed worst case — that’s the kind of work I do.