The Redis author is building an inference engine, and it refuses to be general. DwarfStar, in the repository as ds4, runs a short list of the largest open models: DeepSeek V4 Flash (vision included), DeepSeek V4.1 Flash, GLM 5.2 and 5.3, GLM 5.3 Flash, DeepSeek V4 PRO, and Qwen3.8 Flash Next. It is not a GGUF runner for arbitrary models. It loads the quantized files the project publishes itself, because that is what makes the tuning possible. The stated goal is “the best way to run a few excellent large language models on consumer hardware (that is, hardware that people can actually own).” MIT licensed, more than 23,000 stars since May.
The hardware story
Metal is the primary target, on Macs with 96 GB or more, and SSD streaming extends it downward to smaller machines. CUDA is supported with the DGX Spark as the main goal, including multi-GPU systems other backends do not cover: an eight-card L40S setup has reached about 126 tokens per second aggregate across sixteen sessions. ROCm covers Strix Halo systems like the Framework Desktop. Two 128 GB Macs on RDMA can run 4-bit Flash or GLM tensor-parallel, and pipeline parallelism glues more machines together when a model needs the memory.
Where the speed comes from
The SSD streaming is the piece worth understanding: it keeps weights on disk and pages what the active route needs, so a 128 GB machine can run a model it could not hold, at a speed the project calls decent, and larger GLM quants need either that or genuinely huge machines. The model files themselves lean on the same trick: the DeepSeek V4 Flash build carries 95 GiB of original n-grams that are read from disk rather than loaded into RAM. Compressed KV caches keep long contexts practical, speculative decoding is opt-in through a flag, and session snapshots live in a local directory, with a command to strip a session’s KV payload while keeping its text.
What running it looks like
Two binaries: ds4-agent runs inference directly with a built-in coding agent and keeps its sessions as files; ds4-server exposes an HTTP API on localhost for Pi, OpenCode, Codex CLI, and Claude Code, with batching and disk-backed KV caches. There is an evaluation rig with recorded baselines, including a DeepSeek V4 Flash Q2 sweep on an M5 Max.
The README also says something most projects only say in private: the software is developed with strong assistance from AI coding agents, humans leading ideas and testing, and if that bothers you, it is not the project for you. The acknowledgement that it builds on llama.cpp and GGML, “largely written by hand,” is the same paragraph being honest from the other side.
My read
The bet here is the opposite of llama.cpp’s: one maintainer, one narrow stack, tuned until the numbers move, instead of a general runner that handles everything a little. If your model is on the list and your machine is anywhere near the spec sheet, that trade looks good, since the benchmarks and the tuning notes are all in the repository. If your model is not on the list, nothing here helps you. The AI-development disclosure will lose some readers on principle; the ones who stay get speed work that most open engines only dream about, and an engine whose author openly suggests pointing a coding agent at the code to tune it for your exact machine.
Source: the repository.