Skip to content

Search stories, models, companies…Subscribe

Topic · Concept

Small models

Small models are language models designed to be cheap and fast enough to run at high volume or on local hardware such as laptops and phones. Labs ship them as "mini," "Flash," "Haiku" or "Lite" tiers, and they are often the best choice when a task does not need a frontier model.

24 stories · updated Oct 9

Follow via RSS Get Small models alerts

Key facts

Checked Oct 8

Example on-device model: phi-3-mini, 3.8B parameters, "small enough to be deployed on a phone" (Apr 22, 2024)
Knowledge distillation (canonical paper): Hinton, Vinyals, Dean, "Distilling the Knowledge in a Neural Network," Mar 9, 2015
Example commercial small tier: Claude Haiku 5.5: 1M context; API price "from $0.10 / from $0.50" per MTok
Example commercial small tier: Gemini 3.5 Flash-Lite and 3.1 Flash-Lite listed as stable

Latest on Small models

Hardware · ExplainerWhat an NPU actually does, and why the TOPS number on your laptop matters less than you thinkCopilot+ PCs need a 40-TOPS 'AI engine'. Here's what that chip accelerates, which apps really use it, and why tools like Ollama mostly ignore it.Oct 9·7 min read

Models · ExplainerHow to choose a small AI model for your laptop: the memory math and the current shortlistParameters, quantization and context length decide whether a model fits. Here's how to do the arithmetic, and which open models to try at 8, 16 and 32 GB.Oct 9·7 min read

Developers · ExplainerModel routing explained: how to mix cheap, mid and frontier modelsSending every request to a frontier model can cost 40 times more than it needs to. How routers, cascades and advisor patterns work, and what vendors actually offer.Oct 9·8 min read

Open source · GuideHow to run an open model locally with Ollama (October 2026)Install Ollama, pull a model, chat with it, call it from code, and keep the server off the public internet. Every command comes from Ollama's official documentation.Oct 9·8 min read

Nothing in this format on this page — try another filter or the next page.