NPU (neural processing unit)
Also known as: NPU, neural processing unit, neural engine, AI accelerator, AI engine
A neural processing unit (NPU) is a chip, or part of a chip, designed specifically to run AI workloads efficiently, such as recognizing faces, blurring video backgrounds, or running small language models on a device.
While NPUs became a marketing headline with 'AI PCs,' they've quietly powered smartphone features for years. Their value is efficiency: they do AI maths with far less power than a CPU or GPU.
NPU (neural processing unit), explained at three levels
At home
Think of a busy kitchen with a head chef and a dedicated pastry station. The head chef can make dessert, but slowly; the pastry station is set up to make it fast and with less effort. Your phone's main processor is the head chef, and the NPU is the pastry station for AI. When your camera spots faces or your laptop blurs your video background, the NPU does that maths using little battery.
At work
A neural processing unit, or NPU, is a block of a processor built specifically to run AI models. It handles tasks such as live captions, background noise removal, and on-device assistants with much less power than the CPU or GPU. Businesses care because NPUs let AI features run locally on laptops and phones, which keeps sensitive data off the cloud and can lower cloud costs.
At home
AI models are mostly huge piles of multiplications and additions on grids of numbers called matrices. An NPU packs hundreds or thousands of small multiply-accumulate units that work in parallel, with local memory beside them so data doesn't keep travelling. Its speed is quoted in TOPS, trillions of operations per second, usually for 8-bit integer maths. A Copilot+ PC needs at least 40 TOPS. Apps must be written to use the NPU, or the work falls back to the CPU or GPU.
At work
In business laptops, the NPU sits on the same chip as the CPU and GPU, in processors such as Intel's Core Ultra, AMD's Ryzen AI, Qualcomm's Snapdragon X, and Apple's M-series. The operating system and each app decide which engine runs an AI task. Windows uses the NPU for features such as Studio Effects and Live Captions, but many third-party apps still run AI on the GPU or in the cloud. Which applications actually target the NPU determines whether the extra cost pays off.
At home
NPUs gain efficiency in two ways. They use low-precision numbers, such as 8-bit or 4-bit integers, which take far less energy and memory than 32-bit decimals, and they keep data in on-chip memory so it rarely travels to main RAM. But a TOPS figure only counts peak maths. While more TOPS can help, in my view the figure is often misleading, because real AI speed on a laptop usually depends on memory bandwidth and software support more than raw maths throughput.
At work
Developers reach NPUs through software layers: Windows ML and ONNX Runtime on Windows, Core ML on Apple devices, and vendor toolkits such as Intel's OpenVINO. Models usually need quantization, converting weights to lower precision, which can reduce accuracy. NPUs also support only certain operations, so unusual model layers fall back to the CPU or GPU. The emerging pattern is hybrid AI: small or private tasks on the device, larger ones in the cloud. Procurement should follow actual application roadmaps, not peak specifications.
Real-world examples
At home
Carlos noticed his new phone could translate a menu photo instantly, even in airplane mode. He learned the translation model ran on the phone's NPU, so it didn't need an internet connection and kept his data on the device.
At work
Lena manages IT for a law firm considering AI laptops. She tested whether the firm's transcription and document tools actually used the NPU. Only some did, so she bought AI laptops for staff who relied on those tools and standard models for everyone else.
How it compares
| feature | NPU (neural processing unit) | GPU | CPU |
|---|---|---|---|
| Designed for | Efficient AI inference | Graphics and parallel computing, including AI | General-purpose tasks |
| Power efficiency for AI | Highest | Moderate | Lowest |
| Flexibility | Limited to supported operations | High | Highest |
| Typical location | Phones, laptops, edge devices | PCs, servers | Every computer |
NPUs trade flexibility for efficiency on AI tasks, which matters most on battery-powered devices.
For analysts
Don't buy AI hardware on TOPS alone. I'd ask which of your actual applications use the NPU today, because unused AI capability is just a more expensive laptop.
History & origin
Smartphone makers led the way. Huawei announced the Kirin 970, with a dedicated NPU, in September 2017, and Apple introduced its Neural Engine in the A11 Bionic chip that same month. Qualcomm and Google followed with their own AI accelerators in mobile chips. NPUs came to laptops in the early 2020s, and in May 2024 Microsoft launched 'Copilot+ PCs,' requiring NPUs capable of at least 40 trillion operations per second (TOPS). Qualcomm, Intel, and AMD all shipped PC processors meeting that bar. In my opinion, the AI PC push was more marketing than demand at first, but on-device AI is genuinely useful for privacy and battery life.