oMLX
oMLX
Free
oMLX turns an Apple Silicon Mac into a local inference server that coding agents can actually wait on. The job is not another chat window. It is stopping Claude Code, Cursor, and Codex from recomputing a long prompt every time a tool call shifts the context. KV-cache blocks persist across RAM and SSD, so a prefix the agent already paid for comes back from disk. The server speaks OpenAI and Anthropic APIs on localhost:8000. A native Swift menu bar app starts it; the admin dashboard loads models and one-click configs for the agents. Apache 2.0. Install the DMG or Homebrew. macOS 15+, M-series only. 16 GB RAM is the floor; they recommend 64 GB+ for daily coding. It reads Hugging Face and LM Studio caches so you do not re-download. If you wanted Ollama for a chatbot, stay there. If your local agent dies on every tool call because the cache got blown away, this is the server that treats that as the bug.
Free
Keywords
local-llm, mlx, apple-silicon, claude-code, inference
- SSD KV cache: hot RAM, cold safetensors on disk. Prefixes survive context shifts and restarts — the difference between 90 seconds and about 5 on a long agent turn.
- Drop-in APIs: OpenAI and Anthropic endpoints. Dashboard generates the env command for Claude Code, Cursor, Codex, OpenClaw, OpenCode, Copilot.
- Menu bar, not Electron: signed Swift app, auto-update, crash restart. Web admin at
/adminfor models, chat, bench, pinning, TTL. - Multi-model: LLMs, VLMs, OCR, embeddings, rerankers in one process. LRU eviction; pin what you live on.
- Free, Apache 2.0: 21k GitHub stars as of this week. Apple Silicon and macOS 15+ only. No Windows story.
TMRW VERDICT
Ollama if the customer is the coding agent
by TMRW Team·Aug 31, 2026
Local models have been “fine” for chat for a year. They have been painful for agents, because an agent invalidates the prompt cache dozens of times a session and a naive server recomputes the whole prefix. oMLX is the first Mac server we opened this week that is designed around that failure, not around a pretty chat UI. The homepage says Claude Code and Cursor in 5 seconds, not 90. The README is more useful: paged KV, SSD offload, continuous batching, native /v1/messages.
That is a different job from Lizard, which hosts what the agent just wrote, and from BrowserOS neo, which gives the agent a browser. oMLX gives the agent a model that lives on the desk. Skip it if you do not have Apple Silicon, or if 64 GB is a fantasy, or if you are happy paying Anthropic for the tokens. Skip the multi-Mac cluster docs unless you like experimental pipeline ranks over Thunderbolt — that is not the listing.
Install it if you already keep MLX weights around and you are tired of watching Claude Code spin while the local backend pretends this is the first message. The 21,000 GitHub stars this week are the breakout signal. Stars are not a review of your M-series thermals.
No rating. We opened the live site, the GitHub README, and the dashboard chrome on the homepage. We have not installed the DMG on this machine or timed a Claude Code session against Ollama. Until we have, the card is a pointer to a specific primitive, not a score.