In April I published Why Your Claude Got 100x Dumber: thinking depth down 67% in independent measurements, file reads before code edits down from six to two, all of it shipped quietly. I ended with a line I meant: "we have the receipts."
People still find that post and email me the same question: so what did you do, switch to another model?
No. I did something that turned out to matter a lot more. I stopped treating any model as the product.
The mistake wasn't trusting Claude. It was trusting one of anything.
Here's what the April incident actually taught me, once I got past being angry about it: model quality is a supply input, and I had built my agency's operations on a single supplier with no monitoring, no fallback, and no contract on quality. When the supplier quietly changed the recipe, everything downstream of it changed too, and I found out from my own output getting worse.
No operator would run a business that way on any other input. You don't buy all your inventory from one vendor with no QC and no second source. But that's exactly how most people use AI right now: one chatbot, one subscription, blind trust that today's model is yesterday's model.
The fix isn't loyalty to a better vendor. The fix is architecture.
What the stack looks like now
Today, Claude is still at the center of my operation. But its job changed. It doesn't do most of the work anymore. It runs the work.
Claude is the harness. Concretely, that means Claude Code sits in the orchestrator seat and does five jobs: it plans the task and writes the spec with acceptance criteria; it routes each piece to whichever model owns that lane; it redlines what comes back the way a senior dev redlines a pull request, line by line, against the spec, with the authority to reject and reassign; it convenes the review board before anything client-facing ships; and it executes and verifies the deploy, down to catching the hosting webhook that once silently dropped a release. Think general contractor, not tradesman. The harness almost never writes the first draft of anything anymore. It decides, checks, and signs.
Other models are the workforce. Not hypothetically: the current bench is Codex, Kimi, DeepSeek, GLM, and Grok, each holding lanes they earned in testing, drafting, coding, transforming data, executing defined tasks. The lanes shift as the models shift, which is the point of having a table instead of a habit. Two things surprised me about running it this way. The cost: much of this work runs at a fraction of what I used to pay Claude or Codex to do the same jobs alone. And the quality: in some lanes, the bench's output beats what either of them produced solo. Different models are genuinely better at different jobs, and using several means no single vendor's quiet regression can sink the pipeline. When a model has a bad month, the harness notices, because the redlines pile up.
Local models are the floor. Part of my rig runs on a Mac Studio in my office. Local models are slower and dumber than frontier models, but they have one property no API can offer: they cannot be changed without my consent. The weights on my SSD in July are the weights on my SSD in December. For certain repeatable jobs, that stability is worth more than intelligence.
The routing table
The core of the system is embarrassingly unsexy: it's a table. Every agent in our operation has a defined primary model, a defined fallback, and rules about when the fallback is allowed.
A few real rows from ours:
- Otto (execution agent): primary Sonnet, fallback Haiku, and the fallback is permitted only for repetitive, low-complexity, non-client-facing subtasks.
- Atlas (SEO analysis): primary Sonnet, no fallback. Analysis on a cheaper model isn't cheaper analysis, it's wrong analysis with a lower invoice.
- Verifier (QA): primary Sonnet, no fallback, because the whole point of a verifier is that it never phones it in.
- Pulse (monitoring): Haiku only. Watching logs doesn't need a genius.
Those rows are the in-house agent side, all one vendor by original design. The workforce bench above sits under the same table with the same discipline: a lane, a fallback, and rules. That's what changed since April, the table stopped being a one-vendor document.
Two rules do most of the work. First: switching to a cheaper model mid-task must be logged with a reason, and sustained fallback use triggers an alert. Second: switching to a more expensive model requires human approval, because quality drift and cost drift are both drift.
If that sounds like bureaucracy, consider what it replaces: the silent version, where the vendor makes the routing decision for you, at their margin's convenience, and calls it adaptive.
The part nobody does: models checking models
The piece of this architecture that has produced the most value isn't routing. It's adversarial review.
Before anything client-facing ships, it goes through a review board: multiple passes, each with a different critical lens, each blind to the others' critiques so they can't converge into consensus theater. The chair sees all the critiques and rules on them. We run copy through this. We run code through this. We ran the very post you're reading through it.
Two details matter. The reviewers don't see each other's work before judging, because reviewers who see a prior critique just agree with it. And we deliberately rotate at least one seat to a different model family, because identical models share identical blind spots. A panel of five copies of the same model is one reviewer with a megaphone.
Does it work? An exhibit from this exact page: an earlier draft claimed our latest article went from keyword to "live, indexed page" in a day. The board struck "indexed," because indexing had been requested, not confirmed. A one-word overclaim, caught before you could read it. That's the system doing its job on its own author.
This is also the honest answer to the effort problem from the April post. When a vendor decides how hard the model thinks by default, you get their floor. When the harness sets effort explicitly per task, high for judgment calls, low for mechanical ones, you get your floor. The knob existed all along. The mistake was letting someone else hold it.
What this looks like in practice
A concrete example from this month. We published a ranked comparison of SEO companies in our own market. The pipeline: research and SERP data pulled by tools, first draft structured by the harness, the draft run through the adversarial board, every factual claim checked against a source before publication, and the deploy itself executed and verified by the same system, including catching a hosting webhook that silently dropped the release.
Total elapsed time from keyword decision to live page with indexing requested: about a day. The old way, one person prompting one chatbot and pasting the output into a CMS, would have been faster to first draft and dramatically worse at everything after: no verification layer, no claim checking, no one catching the failed deploy.
That's the trade the harness makes everywhere. Slightly slower to start. Massively harder to be wrong.
What you should take from this
If AI is a toy in your operation, none of this is worth the overhead and you should skip it. If AI is load-bearing, meaning client work or revenue actually flows through it, three things:
- Measure your dependency. If a vendor degraded your model by 30% tomorrow, would you find out from your metrics or from your customers? If the answer is customers, you have monitoring debt.
- Never let a vendor hold the effort knob. Defaults serve the vendor's costs, not your quality. Set reasoning effort explicitly for anything that matters.
- Make models check models. One model reviewing its own work is a student grading their own exam. Cross-model, blind, adversarial review is the cheapest quality system you can build, and almost no one builds it.
The April post ended with "we have the receipts." The receipts turned into requirements, and the requirements turned into a system that doesn't care whose model got quietly worse this month.
Want the structure without reverse-engineering it? I turned our routing table, routing rules, effort policy, and review protocol into a fill-in template: the AI Model Routing Table. Get the template here, or reach out if you'd rather talk through your setup first. If you want to see what the machinery produces when it's pointed at marketing, the case studies are the output.
That's the actual answer to model degradation. Not a better vendor. A structure where no single vendor can hurt you.