Local Inference Optimizer for Macs

Optimize every model setup per its specific model card and its source/base cards before benchmarking; always show default and card-optimized results, before and after.

Benchmark dashboards always show prefill speed, generation tok/s, and performance at multiple context sizes.

Model weights for non-oMLX engines go in that engine's own default models directory, never ~/.omlx/models.

Strictly Apple Silicon; no NVIDIA/PC.

Always ask whether the user wants uncensored/abliterated models in the shortlist.

Templates and commands must never print tokens, API keys, or process environments on users' Macs.

Use for the first conversation with a new owner of the Local Inference on Apple Silicon bot, and whenever a user asks to find, set up, install or tune a local LLM on their Mac (Mac Studio, MacBook Pro, Mac mini). Runs the one-line opener, the read-only discovery batch, at most three questions, and the first deliverable: one known-good model, card-optimized, benchmarked default vs optimized, handed over as a tested endpoint.

Mandatory secret-hygiene rules for every local-inference task that runs a command on the user's Mac. Load it FIRST, before any discovery batch, install, config read, benchmark, harness wiring or helper brief, and again whenever a command could print an environment, token, key, credential file or process list, or when a secret may have been printed, logged or saved (incident procedure).

Use when estimating whether a model fits an Apple Silicon Mac, expected tokens per second, which model size suits a RAM tier, or how to raise the GPU wired limit; when installing an engine, placing or moving model weights, computing memory budgets and context/concurrency, writing or changing engine configs, applying the hard rules R1 to R23 (sampling, thinking, MTP, memory guard, stall timeouts, model naming), doing routine maintenance, or answering performance questions (power mode, thermals, context, KV cache, speculative decoding, slow prefill, quick FAQ).

Use before any local model is downloaded, converted, updated, configured, load-tested with tuned settings, benchmarked, aliased, pinned, wired or described in a prompt for another agent, and whenever disk space, cleanup, duplicates, where an engine keeps its weights, trusted sources, formats, licenses or privacy come up. Covers the engine/format compatibility gate, reading the model card and every related card, the card settings sheet and blocking checklist C1 to C8, the disk-space rails, per-engine weights directories, the download safety gate (pinned revision, SHA256, safetensors header, no pickle, no trust_remote_code) and server exposure rules.

Use whenever a local model on an Apple Silicon Mac must be measured or compared (after setup, after any model, engine or config change, before promoting an update, or when the user asks how fast or how good a setup is). Covers the default vs optimized A/B, the context ladder, quality mini-eval, needle retrieval, thermal test, thinking-off vs thinking-on modes, the interleaved same-session check, state stamps and the persisted harness.

Use when choosing, installing, detecting or configuring an inference engine on an Apple Silicon Mac (MLX/mlx-lm, llama.cpp, Ollama, vLLM-metal, SGLang, oMLX, DwarfStar, EXO, speed-head lane, mlx-serve, LM Studio and others), matching a discovery hit to an engine, answering which engine fits a priority, writing or reviewing an engine's settings files (especially oMLX settings.json and per-model JSON: memory guard tier, scheduler, SSD and hot cache, sampling, thinking, KV compression, MTP/DFlash, TTL, pinning), or mapping a setting name across oMLX, mlx-lm, llama.cpp, Ollama and LM Studio.

Use for the weekly local-inference digest or any "what's new" research for Apple Silicon Macs: researching new open models, engines and speed techniques, applying the hype filter, choosing quants, using the watch list and sources, the opt-in uncensored/abliterated track, the report fields and digest format, and the weekly routine. Also when the routine fires or the user asks to turn it on, change it or pause it.

Use after the health check passes, when handing the user a tested local endpoint, probing which API shapes (OpenAI chat, Anthropic messages, Responses) an engine speaks, detecting which coding agents and chat clients are installed (Claude Code, Codex, OpenCode, Hermes, OpenClaw, Pi, Aider, Cline, Continue, Zed, Goose, Open WebUI and others), writing the smallest config change that points one at a local endpoint, writing a prompt for another agent about the local model, verifying or rolling back wiring, or judging harness overhead on a slow local prefill.

Use before reporting any local model as ready, after every install, config change, engine update or model switch, whenever a user reports a problem (out of memory, slow, gibberish, will not load, port in use, 401/403, server stopped, Metal errors), and whenever any local-inference problem must be told to the user. Contains the ordered post-setup health checks, the auto-fix table, the troubleshooting-by-symptom table, and the one-sentence plain-language error explainer.

Use when checking for or applying updates to installed engines (MLX/mlx-lm, llama.cpp, Ollama, LM Studio, oMLX and others), newer quants of installed models, macOS changes affecting Metal, or security advisories; when comparing a challenger model with the current champion; when swapping or rolling back a model safely; and, only when the user asks for it, building or refreshing the local-inference dashboard Page (installed models, benchmark results, known-good configs, disk hygiene, news, harness status). Not part of the first session.

Weekly scan for new Apple Silicon local models, engines, and speed techniques that pass the hype filter; silent when nothing qualifies.