AI inference

Atlas Inference: a pure Rust LLM inference engine

Atlas is an open-source, pure Rust and CUDA LLM inference engine hand-tuned for NVIDIA DGX Spark, with an OpenAI-compatible API and published, reproducible benchmarks.

Atlas Inference at a glance

What it is
An open-source, pure Rust and CUDA LLM inference engine, hand-tuned for NVIDIA DGX Spark, with an OpenAI-compatible API.
Made by
Atlas Cybernetics, founded by Thomas Braun, founder of Avarok (thomaspbraun.com)
Licence
AGPL-3.0
Runs on
Linux (NVIDIA DGX Spark; AMD Strix Halo)
Website
atlasinference.io
Source
github.com/Avarok-Cybersecurity/atlas

What is Atlas?

Atlas is an open-source LLM inference engine written in pure Rust and CUDA, with GPU kernels hand-tuned for NVIDIA DGX Spark. It runs open-weight models on your own hardware and serves them through an OpenAI-compatible API, with tool calling and streaming responses, so existing clients and agents can point at it unchanged. It is licensed AGPL-3.0, and its code, model recipes and benchmarks are public.

Which models and hardware does Atlas support?

Atlas runs open models from the Qwen, Gemma, Nemotron and Mistral families; the Atlas engine page lists the supported models and recipes. It is verified on NVIDIA DGX Spark (GB10 Grace Blackwell), its kernels are also compiled for AMD Strix Halo, and it has been submitted to MLPerf Inference v6.1 on both.

How fast is Atlas?

Atlas publishes its benchmarks with the hardware and method named. As published on atlasinference.io on 17 November 2026, for unsloth/Qwen3.8-27B-NVFP4 on an NVIDIA GB10 (121.7 GB unified memory), 128 input and 1,024 output tokens, mean of 3 timed repetitions:

Concurrent requestsAtlasvLLM + MTP
123.59 tok/s19.72 tok/s
32291.01 tok/s283.48 tok/s
128478.11 tok/s358.57 tok/s

At 128 concurrent requests that is 1.333 times the throughput of the matched vLLM configuration. The full Atlas benchmark methodology and the raw benchmark results are public.

Who makes Atlas?

Atlas is built by Atlas Cybernetics, founded by Thomas Braun, founder of Avarok (thomaspbraun.com), who also founded Avarok Cybersecurity. Its source lives in the Avarok-Cybersecurity organisation on GitHub. To deploy it, start with the Atlas DGX Spark deployment guide.

Atlas Inference: An open-source, pure Rust and CUDA LLM inference engine, hand-tuned for NVIDIA DGX Spark, with an OpenAI-compatible API.

Visit Atlas Inference

More from Avarok