Small language model (SLM)
Also known as: SLM, small language models, small LLM
A small language model (SLM) is an AI language model with relatively few parameters, typically from under a billion to a few billion or so, designed to run quickly and cheaply, sometimes directly on a phone, laptop, or small server.
SLMs handle focused tasks like summarizing, classifying, or autocompleting text well. While they can't match the broad knowledge and reasoning of the biggest models, they keep data on the device, respond faster, and cost far less to run.
Small language model (SLM), explained at three levels
At home
Think of a pocket dictionary versus a full encyclopedia set. The pocket version fits in your bag and answers everyday questions instantly, but it can't cover everything. A small language model is the pocket version of an AI chatbot. It's compact enough to run on your phone or laptop, so it responds quickly, works offline, and keeps your data on the device.
At work
A small language model is an AI language model with relatively few parameters, usually under a billion to several billion, built to run cheaply on modest hardware such as a laptop, phone, or single server. Businesses use SLMs for focused, high-volume tasks like classifying tickets or extracting fields, where they cost a fraction of large models and can keep data in-house.
At home
On-device AI features such as suggested replies and notification summaries often run on a model of a few billion parameters, like the roughly 3-billion-parameter model in Apple Intelligence. To fit, it is compressed with quantization, storing each number in about 4 bits instead of 16, so it needs only a couple of gigabytes of memory. The phone's neural processing unit runs it, so these features work fast and without internet.
At work
SLMs reach useful quality through distillation, training on the outputs of larger 'teacher' models, and through carefully curated data, as Microsoft showed with Phi. Fine-tuned for one job, such as routing support tickets or extracting invoice fields, they can rival large models at a fraction of the cost. Common families include Microsoft Phi, Google Gemma, Meta's small Llama models, and Alibaba's Qwen. Many systems route easy requests to an SLM and escalate hard ones to a larger model.
At home
A model's parameters are its storage space for knowledge, so a small model simply holds fewer facts. Ask it about something obscure and it fills the gap with a plausible guess. Its language skills, though, stay strong: summarizing, rewording, and sorting the text you give it play to its strengths. While it can't be a reliable encyclopedia, it is a capable editor. In my opinion, small models are best for helping with your own content.
At work
On devices, SLMs face memory, heat, and battery limits, and aggressive quantization can quietly degrade accuracy on edge cases. Updating models across thousands of devices also needs a distribution plan. Their strength is control: an SLM on your own server or laptop keeps data in-house, which simplifies compliance with Canadian privacy laws such as PIPEDA and provincial health rules. I'd evaluate an SLM on your own tasks, with a fallback route to a larger model for low-confidence cases.
Real-world examples
At home
Aisha's phone summarized a long group chat while she was on a flight with no internet. The summary came from a small language model running on the phone itself.
At work
Jerome manages IT at a Quebec manufacturing company. He deployed a small model on a local server to classify maintenance requests, keeping data on-site and avoiding per-request cloud fees.
How it compares
| feature | Small language model (SLM) | Large Language Model |
|---|---|---|
| Size | Under a billion to a few billion parameters | Tens to hundreds of billions or more |
| Runs on | Phones, laptops, small servers | Data centre GPUs |
| Cost per task | Very low | Higher |
| Breadth of ability | Narrower | Broad |
Small models are fast, cheap, and private. Large models know and reason more.
For analysts
For high-volume, well-defined tasks, I'd test whether a small model can do the job before paying for a large one. The savings and privacy benefits add up quickly.
History & origin
As frontier models grew to hundreds of billions of parameters, researchers explored how capable smaller ones could become. Microsoft's Phi-1, described in a June 2023 paper titled 'Textbooks Are All You Need,' showed a 1.3-billion-parameter model trained on carefully chosen data could code surprisingly well, and Phi-3 followed in April 2024. Google released its open Gemma models in February 2024, and Meta released 1- and 3-billion-parameter Llama 3.2 models for phones in September 2024. Apple built Apple Intelligence around an on-device model of roughly 3 billion parameters, announced in June 2024. Techniques such as distillation from larger models and quantization kept improving small models. In my view, small models will quietly do most of the world's everyday AI work, while giant models get the headlines.