Blog
A concrete walkthrough of what happens to one generated token: hidden state, router scores, expert selection, DeltaNet state, attention KV cache, and the final logits.
A clean glossary for the model internals behind inferedThoughts: tokens, layers, GatedDeltaNet, attention, KV cache, MoE, experts, tensors, GGUF and quantization.
The proxy showed 57 TPS, but llama-cli gave 36. Same hardware. This post answers why — then adds MTP to reach 64 TPS at 131k context.
A 600-line proxy bridges Anthropic's API and llama-server — with lifetime metrics tracking and a live dashboard.
Ollama leaves 2.2× performance on the table for MoE models — here's how to recover it.
A clear breakdown of what large language models are, how they work, and why they matter for local inference.
How to change the default directory for Ollama models on Windows using an environment variable.
No posts match your search.