A self-hosted token-completion server for local open models, with chat and completion APIs, SSE streaming, WebSockets, gRPC, MCP and verified setup.
Impossible Inference is a ready-made, self-hosted token-completion server for local open models. Version 0.1 provides one curated CPU inference path with automatic verified setup, offline operation after installation, OpenAI-compatible completion and chat endpoints, SSE token streaming, WebSocket cancellation, unary and streaming gRPC, and bounded MCP tools.
The repository is implemented in incremental, buildable commits. See the v0.1 product contract for the exact promise and explicit non-goals.
Install the pinned Rust 1.88 toolchain, clone the repository, then run exactly one setup command and one serve command. Setup downloads about 510 MB into the ignored runtime-artifacts/ directory.
The second command remains in the foreground. From another terminal, verify generation:
The HTTP/MCP server binds to 127.0.0.1:8080 and gRPC binds to 127.0.0.1:50051 by default; both refuse non-loopback addresses. Effective configuration precedence is command-line flag, environment variable, optional bounded JSON file, then safe default. Run cargo run -p impossible-inferences-server -- serve --help for the exact variables and limits. The current control plane exposes /health/live, /health/ready, /metrics, /version, /v1/capabilities, /v1/models, /status, /v1/completions, /v1/chat/completions, /v1/ws, and /mcp; readiness remains false until a verified runtime and model are installed and the private generation sidecar is healthy. See the HTTP, SSE, and WebSocket API guide and gRPC and MCP API guide for the bounded wire contracts.
From the project README.
Add the radar badge to your README — it shows your project was picked up by MCP Radar and links to this page:
[](https://mcp.liqiwa.com/s/impossibleG--impossible-inference.html)
A self-hosted speech server with streaming transcription, speech synthesis, verified local model setup, WebSockets, HTTP, gRPC and MCP APIs.
impossibleG/impossible-embeddingsA self-hosted embedding server with HTTP, gRPC and MCP APIs, verified model installation, batching, health checks, metrics and Windows and Linux releases.
mahingbun-dev/plugin-hubRust 插件中台：契约治理 · 注册发现 · 声明式编排 · 事件总线 · MCP 工具面 / Out-of-process plugin hub in Rust: contracts, discovery, DAG orchestration, event bus, MCP tool surface
impossibleG/impossible-ocrA self-hosted OCR server that turns PNG and JPEG images into structured text using local PP-OCRv5 models through HTTP, gRPC and MCP.
systempromptio/systemprompt-demoAI Governance Infrastructure — open gateway demo. Self-hosted systemprompt.io gateway in a single Rust binary: policy, audit trails, and cost attribution on every AI request, with the Systemprompt Bri
dhpradeep/aegisTurn your Claude subscription into a multi-tenant agent API with rate limits, cost tracking, autonomous objectives, and an admin dashboard. Self-hostable, no API key.
The top new MCP servers of the week, every Monday. No spam, unsubscribe anytime.