Skip to content

Search stories, models, companies…Subscribe

Topic · Concept

Small models

Small models are language models designed to be cheap and fast enough to run at high volume or on local hardware such as laptops and phones. Labs ship them as "mini," "Flash," "Haiku" or "Lite" tiers, and they are often the best choice when a task does not need a frontier model.

24 stories · updated Oct 29

Follow via RSS Get Small models alerts

Key facts

Checked Oct 8

Example on-device model: phi-3-mini, 3.8B parameters, "small enough to be deployed on a phone" (Apr 22, 2024)
Knowledge distillation (canonical paper): Hinton, Vinyals, Dean, "Distilling the Knowledge in a Neural Network," Mar 9, 2015
Example commercial small tier: Claude Haiku 5.5: 1M context; API price "from $0.10 / from $0.50" per MTok
Example commercial small tier: Gemini 3.5 Flash-Lite and 3.1 Flash-Lite listed as stable

Start here

Developers · ExplainerModel routing explained: how to mix cheap, mid and frontier modelsSending every request to a frontier model can cost 40 times more than it needs to. How routers, cascades and advisor patterns work, and what vendors actually offer.Oct 9·8 min read

Models · ExplainerHow to choose a small AI model for your laptop: the memory math and the current shortlistParameters, quantization and context length decide whether a model fits. Here's how to do the arithmetic, and which open models to try at 8, 16 and 32 GB.Oct 9·7 min read

Hardware · ExplainerWhat an NPU actually does, and why the TOPS number on your laptop matters less than you thinkCopilot+ PCs need a 40-TOPS 'AI engine'. Here's what that chip accelerates, which apps really use it, and why tools like Ollama mostly ignore it.Oct 9·7 min read

Latest on Small models

Hardware · ExplainerMac Studio M5 Ultra 512GB: reported at $22,299, with 1.2TB/s of bandwidth, and which of the biggest open models actually fitThe Mac Studio M5 Ultra with 512GB of unified memory went on sale on October 28, 2026, at $22,299 according to MacRumors and Cult of Mac. What 512GB and 1.2TB/s mean for running the biggest open models locally: Reflection Beam fits at 4-bit, DeepSeek V4.1 Flash barely does, and Kimi K3.1 needs several machines.Oct 29·8 min read

Hardware · ExplainerM6 MacBook Pro for AI: what the Dual 16-core Neural Engine, 32GB memory ceiling and 170GB/s bandwidth mean for running local modelsIs the M6 MacBook Pro good for AI? Apple's October 27, 2026 M6 MacBook Pro and iMac top out at 32GB of unified memory and 170GB/s of bandwidth. What Apple claims for AI, what fits in that memory, and when the M5 Pro or M5 Max models still make more sense for local models.Oct 29·8 min read

Products · ExplainerOffline AI assistant: how Survival LLM pairs a local model with a 930,000-article emergency library, and how people run AI with no internetAn offline AI assistant runs the model and its reference library on your own computer, so it keeps working when the internet doesn't. Survival LLM, which drew wide attention on October 27, 2026, bundles one of four models with about 930,000 Wikipedia articles and medical and repair references. What it needs, how Kiwix, Ollama, llama.cpp and LM Studio work offline, and where offline AI falls short.Oct 29·9 min read

Open source · NewsLiquid AI LFM3: six 2B-class open-weight models for text, vision, audio, omni and decisions, free only below $10 million in revenueLiquid AI LFM3, released October 27, 2026, puts text, vision, audio, omni and two 'decision' models on one 1.7B backbone. What 'every modality' means, the sizes, the LFM Open License's $10 million revenue limit, and where the models run.Oct 29·6 min read

Hardware · NewsLenovo AI Host P7: what's confirmed about the 300-gram box said to run 125B-parameter models locallyLenovo unveiled the AI Host P7 in Beijing on May 19, 2026 and showed a preview edition in China on October 24: a palm-sized box with a Cixin P1 chip and a Houmo M50 accelerator that Chinese reports say runs models of up to 125 billion parameters locally. There's no price or release date, and the specs come from chip partners and reports, not a Lenovo spec page.Oct 27·6 min read

Models · NewsBlock's Dessa: a full-duplex conversation model trained only on speech, with no text, released as a closed research previewBlock's Dessa is a speech-to-speech conversation model that Block says was trained from scratch on human conversation audio, with no text and no pretrained models. What that claim means, what Block has disclosed, and what it hasn't: size, license, benchmarks or a release date.Oct 26·4 min read

Open source · NewsMistral Timestral: four open-weight time-series models, a $20M revenue cap, and a CERN statement of intentMistral Timestral is a family of four time-series forecasting models, from 5M to 209M parameters, with weights on Hugging Face under a revenue-capped 'Modified MIT' license. The same week, CERN and Mistral signed a statement of intent, not a contract, to build a Large Physics Model.Oct 20·5 min read

Models · ExplainerWhat is a diffusion language model? Inception's Mercury 3 explainedA diffusion language model drafts a whole block of text and refines it in parallel, instead of writing one token at a time. Inception's Mercury 3, launched October 19, 2026, claims 1,068 tokens per second, a 1-million-token context and $0.20/$0.70 pricing. Here's how diffusion LLMs work and what the trade-offs are.Oct 20·7 min read

Industry · AnalysisIs AI usage slowing? What OpenRouter's token data and Gartner's CIO survey actually showA 'token-maxxing is dead' thread and Gartner's finding that only 13% of CIOs see significant value from AI have revived the question: is AI usage slowing? The public data shows cost discipline and a shift to cheaper models, not a fall in overall use.Oct 20·7 min read

Models · ExplainerCheap, fast AI models compared (October 2026): DeepSeek V4.1 Flash 1017 vs MiniMax M3.1 Flash vs Claude Haiku 5.5 vs Gemini Flash vs GPT-6 LunaCheap, fast AI models compared as of October 20, 2026: MiniMax M3.1 Flash, DeepSeek V4.1 Flash 1017, Claude Haiku 5.5, Gemini 3.8 Flash and GPT-6 Luna on official prices, context, output limits, open weights, and what Artificial Analysis and Vals AI report.Oct 20·9 min read

Nothing in this format on this page — try another filter or the next page.