Accurate tool choice in a large catalog
Tool-selection accuracy falls away once a client holds more than a few dozen definitions. Toolspoke hands the model an index and lets it search, so the shortlist stays small however large the catalog gets.
A workspace holding the whole first-party catalogue, 1,143 actions, counted as tokens the client carries before it does anything.
- Every schema, sent up front
- 73,000
- The name-only index Toolspoke sends instead
- 2,600
Both figures are our own estimates for a fully stocked workspace, not a benchmark.
Why accuracy falls off first
Token savings are real and increasingly commoditised, and clients are starting to do their own tool search. The reason to care is what happens to the model's judgement.
A catalog of three hundred actions contains a dozen plausible ways to do most things. Four of them list issues. Three of them send a message. Two of them look identical apart from which workspace they touch. The model is not confused because it lacks intelligence; it is confused because the distinctions live in prose it is skimming under budget pressure.
Searching first inverts that. The model states what it is trying to do, gets back a small ranked set with full schemas attached, and chooses between five candidates instead of three hundred. The choice it is making is one it can actually make well.
Ranking is a lexical match on toolkit name, action name, project and description, weighted so an exact name wins outright. Where embeddings are configured and available, a semantic score is folded in on top of that.
When they are not, or when the embedding call is slow, search falls back to words alone and says nothing about it. An agent should not be told it is operating in a degraded mode when the answer it gets is the same answer.
Ambiguity is never resolved by guessing. A tool name that exists in two projects comes back as a question naming both, not a coin flip.
What the client actually sees
Toolspoke advertises a small fixed set of tools regardless of how much is installed behind it. Calling an installed action directly by its own name still works, so a client that already knows what it wants is not forced through search.
- search_tools
- Ranks the catalog against a query and returns the full input schema for the handful that matched. An empty query returns the index instead, so an agent can orient before it commits.
- list_toolkits
- Every toolkit the caller can reach, with the names of the actions inside it. No schemas, so the whole map fits in a fraction of the budget one toolkit's schemas would take.
- use_tool
- Calls a read-only tool by name with an input object. The audit log records the underlying action, not this wrapper, so the record is still readable a year later.
- run_tool
- The same, for a tool that creates or updates something. Held apart from the read-only runner so a client can tell, before the call, whether anything can change.
- run_destructive_tool
- The same again, for a tool that deletes. A search result says which of the three a tool belongs to, so the agent never has to infer it from the name.
- read_guide
- Fetches a toolkit's usage guide when it has one. Search returns only a marker saying a guide exists and whether it is required; the text is fetched when it is wanted.
Searching, listing and reading a guide cost nothing. No credit is charged and no audit row is written, because none of them touch a provider. Only the call that leaves the gateway is charged, and it is charged once even if the client retries.
What a single request looks like
The agent is asked to file a bug
It has never seen this workspace before. It calls list_toolkits and gets every toolkit and action name it can reach, in about the space one large schema would have taken.
It searches for what it needs
A query of "create an issue" comes back ranked, with full input schemas on the handful that matched and nothing on the hundreds that did not.
It finds the toolkit has a guide
The result carries a marker saying so. If the guide is marked required, the agent reads it before calling rather than after failing.
It makes one call
The search result said the tool is a write, so it calls run_tool with the chosen name and an input object. The audit log records the real action name, the person the agent key belongs to, and the arguments that went out.
Install as much as your team needs
The index does not grow the way the schemas would, so adding the thirtieth toolkit does not make the first twenty-nine harder to find.