jayleaton/localrouter

Fast & opinionated local AI routing: one MCP server that loads local image and video models on demand on your own GPU box (DGX Spark first)

About localrouter

[!WARNING] Early preview. LocalRouter is new and under active development: expect bugs and breaking changes. Please open an issue if something doesn't work.

LocalRouter runs generative models on your own GPU box and hands them to your agents. One small daemon owns the machine: an agent asks for an image or a video over MCP (or an OpenAI-style HTTP API), LocalRouter loads the model that can do it, makes room for it if it has to, runs the job and unloads the model when it goes idle. Nothing is held in memory that nobody is using, and the models you care about can stay warm.

It starts on the NVIDIA DGX Spark with two hand-built engines, written in Zig with their own GPU kernels and checked bit for bit against reference implementations of the same models:

Model Can do On a DGX Spark (GB10) --- --- --- Qwen-Image 2.1 (FP8 by default, closest to the original model; NVFP4 is the faster option) text to image only; image edit not yet supported FP8: 1024x1024 in 16.7 s warm, about 22 s from cold; 576x576 in 5.0 s. NVFP4: 12.7 s warm, 16.1 s from cold MiniMax H3 (video with audio, Turbo 8 steps) text to video only; image to video not yet supported 768x448 with stereo audio, warm: a 2.3 s clip (56 frames) in 21.7 s, a 5 s clip (124 frames) in 54 s

The API accepts image-edit and image-to-video inputs, but both shipped model engines refuse those requests until their input-image ports land. Check each model's advertised capabilities before submitting a job.

From the project README.

Maintaining this server?

Add the radar badge to your README — it shows your project was picked up by MCP Radar and links to this page:

[![On MCP Radar](https://mcp.liqiwa.com/badge.svg)](https://mcp.liqiwa.com/s/jayleaton--localrouter.html)

Related MCP servers