Mixture of experts (MoE)
Also known as: mixture-of-experts, sparse mixture of experts, MoE model
A mixture of experts (MoE) is an AI model design that divides part of the network into many smaller 'experts' and uses a router to send each token to only a few of them.
The model can hold a huge amount of knowledge in total while doing only a fraction of the work for each word it processes. While MoE has made very large models faster and cheaper to run, it still needs lots of memory to hold all the experts, and training them evenly is tricky.
Mixture of experts (MoE), explained at three levels
At home
Picture a hospital with hundreds of specialists. A receptionist sends each patient to the two or three doctors who fit best, so no one waits for every doctor in the building. A mixture of experts model works the same way: a small 'router' sends each word to a few specialist sub-networks, so the model can be huge without doing huge work for every word.
At work
A mixture of experts is a way of building AI models so that only part of the network runs for each word it processes. That lets providers offer very capable models with faster responses and lower computing cost than a traditional model of the same total size. The catch is memory: all the experts still need to be loaded on the servers.
At home
Inside a language model, each layer has a feed-forward block that does much of the processing. An MoE model replaces that block with many smaller copies, the experts, plus a router that picks a few for each token. So a model can hold hundreds of billions of parameters but use only a fraction per word. Meta's Llama 4 Maverick, for example, has about 400 billion parameters in total but only 17 billion active.
At work
For each token, the router scores every expert and sends the token to the top few, typically one to eight out of dozens or hundreds; DeepSeek-V3 picks eight of 256 routed experts, plus one shared expert. Compute per token depends on active parameters, while memory depends on total parameters. Experts don't map neatly to human subjects like law or biology; their specialization emerges during training and often tracks things like grammar or token types.
At home
The speed of an MoE model comes from skipping work, not from being small. Every expert must sit in memory, because the router might pick any of them for the next word, so a model with 17 billion active parameters can still need hundreds of gigabytes. While quantization and offloading to system memory help, offloading slows things down. In my opinion, the total size is the number that matters for home setups.
At work
MoE training needs a load-balancing loss or bias adjustments, or the router keeps favouring a few experts while others go unused. At scale, experts are spread across GPUs, called expert parallelism, so tokens must be shuffled between chips over fast interconnects, adding communication overhead. Serving suffers when uneven routing leaves some GPUs idle, and small batches waste capacity. Quantization and expert offloading cut memory needs, but I'd benchmark under real traffic before committing to hardware.
Real-world examples
At home
Thomas tried to run an open MoE model on his gaming PC. Although only a small part of the model ran per word, he still needed enough memory to load every expert.
At work
Aisha manages AI infrastructure at a Montreal startup. Switching to an MoE model cut response times and GPU costs for their chatbot, but required servers with more memory.
How it compares
| feature | Mixture of experts (MoE) | Dense model |
|---|---|---|
| Parameters used per token | A small fraction | All of them |
| Speed for its size | Faster | Slower |
| Memory needed | High, all experts loaded | Matches total size |
| Training complexity | Higher | Lower |
MoE models do less work per word but need more memory than their speed suggests.
For analysts
When comparing AI models for self-hosting, I'd look at both total and active parameter counts. Total size drives hardware cost, while active size drives speed.
History & origin
Robert Jacobs, Michael Jordan, Steven Nowlan, and Geoffrey Hinton published the original 'adaptive mixtures of local experts' paper in 1991. Noam Shazeer and colleagues at Google revived the idea at scale in 2017 with a 'sparsely-gated' mixture of experts layer containing thousands of experts, and Google's Switch Transformer in 2021 simplified routing to one expert per token. Mistral AI's open Mixtral 8x7B model, released in December 2023, brought MoE into the mainstream. China's DeepSeek used a fine-grained MoE design for DeepSeek-V3, released in December 2024, and many leading open models since then, including OpenAI's gpt-oss in 2025, have used MoE. Several closed frontier models are widely reported to use it as well. In my view, MoE is the main reason frontier-scale AI didn't become impossibly expensive to serve.