Ask an LLM a question with 255 tools attached and it reads every tool's schema first. Not sometimes. Every turn. That reading is the overhead, it's measured in tokens, and on our benchmark it costs more than the conversation that triggered it, because unlike the conversation it never shrinks and it never ends, since every new turn opens with the same 255 schemas lined up and waiting to be read all over again.
A 50-server, 255-tool catalog. Native tools/list JSON schemas versus
mcptoon's names-only index. tiktoken cl100k_base; source of truth
assets/benchmark_tiktoken.json. −99.2%, every tool still callable.
Three layers, stacked.
First, the tool list itself: names, descriptions, and full JSON Schemas, injected into context before your first token of "real" prompt. Second, the selection step: the model reads all of it to decide which tool, if any, to call. Then the repetition. Context is re-read every turn. The overhead is per turn, not per session, and not per call. That's the layer nobody budgets for.
Function calling APIs charge the same way. Attach a hundred well-documented functions and you've spent tens of thousands of tokens before the model has thought about your question, because every one of those descriptions, parameter objects, and nested constraint blocks rides along into every request whether the model needs them this turn or not. Attach a thousand and the overhead is most of your window. The schemas don't know they're not needed this turn. The tokenizer doesn't care. You pay either way, and the invoice line never shows up labeled "tool overhead." It just makes every other line bigger.
Three steps. That's the whole audit. It's the same audit we ran before building anything, and it takes two minutes.
tools/list
payload, and if you've never looked at it raw, brace yourself. It's bigger than you think.cl100k_base, the tokenizer our own benchmark uses, and
write the number down. You'll want it again after you fix things, when the improvement needs to be a
number rather than a feeling.Or let the tool do the counting:
python -m pip install mcptoon mcptoon bench # measure your catalog, same method as our benchmark mcptoon demo # watch the math live, zero config
Three levers exist. Two of them are traps.
| Approach | Effect | Trade-off |
|---|---|---|
| Trim descriptions by hand | small | brittle, redo on every update |
| Route to per-task tool subsets | complex | you own the router now |
| Serve a names-only index, fetch schemas on demand | −99.2% | one lookup step |
The third row is mcptoon. It fronts your MCP servers, keeps full schemas on disk, and hands the agent one compact name index, with selection happening on names alone and the chosen tool's schema fetched fresh at call time. That's the entire trick. No router to maintain. Related reads: the protocol view in the tools/list token bloat guide, the client view in the Cursor guide, and the same disease in skill catalogs in the skills guide.
Does tool calling overhead affect latency too?
Yes, indirectly. More context tokens means more prefill work per request, and prefill is part of what you feel as "slow first token." The token math on this page is about cost and context pressure. Latency improves by the same mechanism.
Is this only an MCP problem?
No. OpenAI-style function calling, Anthropic tool use, and MCP all inject schemas into context. All of them re-inject on every request. MCP just makes it easy to accumulate hundreds of tools across dozens of servers without noticing. At that point the overhead stops being background noise and becomes the line item that ends your session early.
Can I just ask the model to ignore unused tools?
No. The schemas are in context before any instruction runs. Asking the model to ignore them doesn't refund the tokens. It can't. The only real fix is not putting them there, which is what a gateway is for.