Subject: The Smartest AI Architecture Pattern You Haven’t Tried
Here’s a story that I keep coming back to. Mendral — a CI/CD company nobody outside DevOps has heard of — upgraded from Sonnet 4.0 to Opus 4.6, and their LLM costs dropped. Not because the new model is cheaper. Because they changed how they thought about the problem.
Most organizations approach AI agents the same way: throw the biggest model at everything and hope the costs sort themselves out later. Mendral did something smarter. They asked a different question: what does each model actually need to see?
Three Tiers, One Principle
The architecture they built has three layers. On the surface it sounds obvious, but the details matter.
At the front door sits a cheap model — Haiku — with one job: “Has this failure happened before?” It checks exact error strings and uses semantic search for the fuzzy matches. “operator does not exist bigint character varying” and “migration type mismatch on installation_id” are different strings that mean the same thing. Semantic search catches that. When Haiku isn’t sure, it escalates — false positives cost money, but false negatives cost missed issues.
Eighty percent of CI failures never go past this layer.
The failures that do escalate hit the expensive model — Opus. Here’s the counterintuitive part: Opus never reads the raw logs. It forms a hypothesis about what went wrong, then spawns narrow sub-agents to find specific information. “Return the exact error messages from the pnpm install step” is a very different prompt from “analyze these logs.” Opus decides what to look for; Haiku finds it.
Those sub-agents are strictly limited — one level deep, no spawning their own children. They start fresh, do their narrow job, and their context is discarded immediately. Only structured summaries make it back to the orchestrator. This keeps the expensive model’s context window clean for actual reasoning instead of drowning in log noise.
What Makes This Actually Hard
I want to pause here because this sounds like a neat trick, but it only became viable in the last six months. The insight isn’t the architecture — it’s the timing. Opus 4.6 is finally good enough to plan investigations and write precise sub-agent prompts. Haiku 4.5 is finally good enough to execute narrow directed tasks reliably. Neither was good enough six months ago.
The tiered architecture works because both ends of the model spectrum improved simultaneously. That’s the hidden variable in every “AI is moving too fast” conversation — the ecosystem is maturing at multiple layers at once, and some patterns only emerge when all the pieces converge.
What I Think This Means
The Mendral case validates a principle I’ve been working toward in our PromptOps framework: expensive models should plan and decide. Cheap models should read and execute. Most organizations do the opposite — flooding their most capable model with raw data and wondering why reasoning degrades as costs balloon.
This pattern generalizes. Security log analysis. IoT telemetry. Financial transaction monitoring. Any high-volume event stream where most events are noise benefits from the same approach: triage cheap, escalate carefully, keep expensive context pristine.
There are three design decisions worth stealing from Mendral. Pull context instead of pushing it — give agents a query interface to their data rather than dumping logs into prompts. Cap sub-agent depth at one level to prevent runaway costs and recursion loops. Discard sub-agent context immediately after use and keep only structured summaries.
These three rules alone would save most organizations forty to sixty percent on their agent inference costs. More importantly, they’d improve reliability. An orchestrator that isn’t drowning in irrelevant data makes better decisions.
The Real Story
But the real story here isn’t about cost savings. It’s about architecture maturity. We’re past the phase where the right answer is “throw a better model at it.” The organizations that win won’t be the ones using the most capable models. They’ll be the ones that built the right scaffolding around them.
That’s what PromptOps looks like in practice. Not as theory. As a deployed system that makes the expensive stuff cheap and the impossible stuff routine.
