IWhat Broke When the Tools Got Better
Twice this month, a team handed an AI agent better tools and watched it get worse at the job. Not a weaker model — better tools, more raw capability, and a measurable drop in quality. GitHub saw it inside its own code-review agent. Independent developers saw it in how the newest Claude models handle a third-party editing tool. Same failure mode both times. And it's the exact one my production agent swarm is built to avoid.
Here's the lesson before any of the background, because it's the whole point: job-shaped instructions beat better tools. Hand an agent a powerful, general toolset and a vague brief, and it will use all that capability to do the wrong thing thoroughly — browsing, re-reading, piling material into its own context until the one fact that mattered is buried. Scope the job, name the evidence it's hunting for, tell it when to stop, and a cheap model with plain tools will out-perform an expensive one turned loose. Everything below is the receipts.
IIWhy Everyone's Suddenly Talking About Context
The vocabulary finally caught up to the problem. shadcn, the developer behind the shadcn/ui component library, posted a line that was half joke and half confession: "Sometimes I leave out context and just sit and watch the agents struggle. Reminds them who's still in charge." It pulled 1,706 likes and 95 reposts — because everyone building with agents recognized it.
A few days later, Box CEO Aaron Levie made the same point as strategy, arguing on X that the battle in AI is shaping up to be a battle for context. Agent effectiveness, he said, comes down to domain expertise, the right tools, and access to the right data — and "the applied AI layer has a lot more value than just being an LLM wrapper." Both are circling the same operational truth: what you put in front of an agent, and just as much what you leave out, decides whether it works. Context management stopped being a tuning detail and became the game.
IIICase One: GitHub Gave Copilot Better Tools — and Review Quality Dropped
GitHub's engineering team told this one on themselves, which is why it's worth trusting. They migrated Copilot's code review off its own custom code-exploration tools and onto the shared, general-purpose search-and-read tools that already powered Copilot's command-line agent — the same grep, glob, and view commands. On paper, an upgrade: one battle-tested toolset instead of two.
In their offline benchmarks it went the other way. Average review cost went up, and the number of genuinely useful comments the agent caught went down. The reason is the interesting part. With broad, general tools in hand, the agent started behaving as if it were learning the whole repository — browsing widely, pulling in files, carrying all that extra context forward — instead of doing the narrow job in front of it: reviewing one pull request's changes.
The fix wasn't a new model or a new tool. It was rewriting the tool instructions to match how a human reviewer actually works — start from the diff, form specific questions, narrow down with a search before opening any file, read a file in full only once you know it's the right one, and batch the cheap discovery steps before the expensive reads. That instruction-level change flipped the regression into roughly 20% lower average review cost while holding quality steady. Same tools. Better instructions.
IVCase Two: The Newest Models Are Worse at Following Your Tool's Schema
The second case comes from Simon Willison, writing up a finding from developer Armin Ronacher. Ronacher's coding tool, Pi, exposes an editing tool that expects a specific set of fields. Anthropic's newest models — Opus 4.8 and Sonnet 5 — kept calling that tool with extra, invented fields that weren't in the schema. Pi rejected the malformed calls, the model retried, and every retry burned time and tokens.
Older Claude models didn't do this. As Willison put it, the state-of-the-art models in the family are worse at this specific tool schema than their older siblings. The likely cause: the newer models were reinforcement-trained hard to excel at Claude Code's own built-in editing tool, and that sharpening came at the expense of generalizing to someone else's schema. "Better," measured on the benchmark everyone cares about, quietly meant "worse" at the thing your custom tool actually needs. If your agents depend on tools you built, a model upgrade is not automatically an upgrade — test it against your own schema before you trust it.
VThe Research That Explains Both Cases
Neither story is a fluke. The research that's piled up over the last year says this is simply how agents behave, and it's why context management is now its own discipline:
- Anthropic calls it context engineering — the practice of "curating and maintaining the optimal set of tokens" an agent sees during inference, with the guidance to keep that context "informative, yet tight" and to build tools that are token-efficient and unambiguous rather than bloated and multi-purpose. The broad, do-everything toolset that hurt Copilot is exactly that bloated-and-multi-purpose trap.
- Chroma measured the decay. Its Context Rot report tested 18 frontier models and found none of them process context uniformly — performance grows increasingly unreliable as the input gets longer, even on deliberately simple tasks, well before the context window is anywhere near full. More context is not more accuracy.
- Multi-agent systems fail a lot. The MAST failure taxonomy, a NeurIPS 2025 study, hand-annotated 1,642 execution traces across seven popular multi-agent frameworks and found failure rates from roughly 41% to 86.7%, spread across 14 distinct failure modes. That rhymes with my own lessons from a week of AI agent outages.
- Agents over-retrieve. ContextBench ran 1,136 issue-resolution tasks across 66 repositories and found models consistently favor recall over precision — they pull in far more than they use — and that fancier agent scaffolding barely improves it. The authors call it the "Bitter Lesson" of coding agents: the scaffolding you bolt on matters less than you'd hope.
Put it together and the pattern is plain. Agents don't just call tools — they internalize the workflow implied by the tool surface and its instructions. Every extra chunk of material an agent reads doesn't vanish; it sits in the context window, inflates cost, and can drown the one line of evidence that actually mattered.
VIWhat My Swarm Does Differently
So here's the counter-example I run every day. My production swarm is about a dozen named agents — a scout, a curator, an observer, a listener, an engineer, a synthesizer, and a handful of others — and each one has a single narrow job. There's no orchestration framework, no message bus, no state machine holding it together. Just cron schedules that decide when each agent wakes up, connectors that scope exactly what each agent is allowed to read and write, and failure alerts landing in one Slack channel. Claude Sonnet is the default brain; Haiku takes the high-volume, low-latency work. The agents are stateless between runs and hand off through shared data, not by chattering at each other. I've written the full build up separately — how I run the swarm without an orchestration platform — but the design principle is the part that matters here.
I'd love to give you a war story about the day context bloat took down a run. I don't have one, and that's the honest point. The design is preventive. Each agent gets a job-shaped brief and a connector that only exposes the data that job needs, so no agent has the room to wander off and start "learning the repo." It's the same move GitHub eventually made — define the job, define the evidence, define the stopping rule — baked into the architecture from the start instead of retrofitted after a regression.
VIIThe Cost Math Behind the Discipline
Tight context isn't only an accuracy fix. It's the single biggest cost lever you have, because the expensive thing about agents is re-reading. Here's the math I run against:
By my own tally of the wider research, re-sent and repeated context — agents re-reading the same material to each other — accounts for roughly 62% of total agent inference cost across the industry. And coordination gains plateau fast: past about four agents, the overhead of keeping them aligned grows faster than the value each new one adds. That's why my swarm stays cheap. The constant, high-frequency jobs run on cheap models and barely register on the bill; a single weekly Claude Opus synthesis run — one genuinely hard judgment call — costs more than a full month of the frequent workers combined. One expensive decision a week, a swarm of cheap workers the rest of the time. The full breakdown is in what the swarm actually costs to run each month.
VIIIThe Three Questions Before You Add a Tool or Agent
Before I bolt a new tool onto an agent — or add an agent at all — I make it pass three questions:
- What's the job? Not the capability, the job. "Review this diff," not "understand this codebase." A tool exists to serve the job; if it invites the agent to do more than the job, it's the wrong tool.
- What's the evidence? Name the specific thing the agent is looking for, so it knows when it's found it. An agent with no target just keeps retrieving.
- What's the stopping rule? The clearest sign a job isn't defined is an agent that never stops. A research or sourcing agent with generic instructions turns into an infinite browser — the same failure mode as Copilot "learning the repo" instead of reviewing the pull request in front of it. A stopping rule is what turns a browser back into a worker.
If a new tool or agent can't answer all three, it isn't making the system better. It's just giving it more room to get worse.
IXFrequently Asked Questions
Does adding more AI agents to a workflow always make it better?
No — and this is the mistake I see most. Coordination overhead beats the gains past roughly four agents, so a tenth agent usually costs more than it contributes. Black Matter VC — my AI studio and consultancy — scopes each agent to one narrow job for exactly this reason. A few agents doing defined work beats a crowd re-reading each other's output.
How do you keep agent costs predictable for a fund?
Three habits. Job-shaped instructions so agents don't wander and rack up tokens; cheap default models for the constant work, with a premium model reserved for the rare hard judgment call; and connectors that limit what each agent can pull in, so context stays tight. That's how a dozen working agents came in under $1,500 for June 2026 — not by capping usage, but by not wasting it.
Do I need a new orchestration platform to fix agent context problems?
No. Most of the reliability I get comes from the opposite of a platform — plain cron schedules, connectors that scope each agent's data access, stateless handoffs through shared data, and alerts in one Slack channel. That's the pattern I set up on a build-and-operate retainer: not a bigger framework, but tighter jobs. If you want the full toolset, it's the stack I build this on.
— Michael Rouveure
