Local AI
Models on your own metal: engines, frontends, and what actually fits.
Running a model yourself is less about beating the big labs than about owning the whole loop: the weights, the prompts, and the privacy. The practical questions are boring ones. What fits on the hardware you have, what serves the model, and what you talk to it through.
The desk’s coverage runs from the engine out to the interface: inference stacks built for a handful of models, an all-in-one server for OpenAI-shaped APIs, chat frontends, and the smaller experiments at the edge.
DwarfStar is an inference engine for the few models you actually run. The engine piece: a stack built for a short model list rather than a catalog.
LocalAI 4.11.0 brings remembered speakers and failover chains. The all-in-one option, for people who want OpenAI-shaped APIs on local hardware.
LibreChat v0.8.8 adds steering, approvals and background tasks for agents. The frontend piece: a chat interface over whichever backend you chose.
Kolibri is Aleph Alpha’s open-weight bet on on-premise AI. Open weights from a European lab, aimed squarely at the on-premise crowd.
Needle 2 turns plain English into local actions on a Raspberry Pi 5. The fun end of the scale: a Pi that listens and acts, with nothing leaving the house.