Benchmarks of MCP tool selection accuracy and latency. V2 evaluates 143 Woku tools with GPT-6 Luna API calls and Jev; includes Go/Rust cores, one TypeScript SDK and reproducible reports.
View on GitHub ↗Homepage ↗Toolgate: tool selection before model inference
An experimental comparison of direct MCP and three Toolgate designs, with Go/Rust cores and one TypeScript SDK. V1 studies latency with 22 read-only tools and Codex; V2 studies selection accuracy with all 143 active Woku tools and GPT-6 Luna API calls.
V2 accuracy results · Architecture · V1 single-Choice results · V1 parallel binary results · Reproduce the experiments · Connect your MCP
V2: selection accuracy with the complete catalog
143 active Woku tools, 60 English requirements per condition, 240 evaluated cases. The evaluated agent is gpt-6-luna through the Responses API with high reasoning, not the Codex app server. Go is fixed across the three Toolgate conditions. The corpus contains 30 reads and 30 mutations, executed by real Woku handlers on an isolated synthetic database restored before each case.
Condition First reference tool Expected handler succeeded Median task time Estimated USD per 60 cases ------:---:---:---: Direct MCP 43/60 60/60 10.48 s $0.071341 Design 1: agent prepares 60/60 60/60 9.53 s $0.048590 Design 2: host-first Choice 59/60 59/60 4.41 s $0.034459 Design 3: host-first binary 59/60 58/60 5.75 s $0.104033
All four conditions produced the requested persisted change in 30/30 mutations; no non-target mutation calls were observed. This is a limited synthetic test, not evidence of production safety. There were 239 final answers: design 1 executed the expected tool in one case but reached its eight-turn limit before answering.
From the project README.
Add the radar badge to your README — it shows your project was picked up by MCP Radar and links to this page:
[](https://mcp.liqiwa.com/s/diorrego--toolgate-experiment.html)
Local-first MCP plugin for continuous software-quality review by AI coding agents, powered by Jev.
bybit-exchange/kaasTurn scattered notes, docs and transcripts into a queryable Markdown wiki — an LLM knowledge-base compiler with MCP access, no embeddings, self-hosted.
nassim-arifette/jevgrepJev-powered semantic code search for coding agents — find behavior across repositories via CLI or MCP, with exact source excerpts and line numbers.
PerryLink/jevcoreTypeSafe Jev for DeepSeek Harness, the Model Context Protocol, and plain Node: typed judgments instead of prose, offline by default.
agent-chaperone/agent-chaperoneScreens an AI agent's tool calls before they run and tool results before the agent reads them. An MCP proxy plus a hooks adapter for a client's built-in tools.
ismaelsoilet/jev-harnessZero-dependency System One decision harness: 5 semantic gates saving frontier AI agent tokens on trivial errors & doom loops. Python + TypeScript + Rust. MCP-compatible.
The top new MCP servers of the week, every Monday. No spam, unsubscribe anytime.