Why run AI on the NPU?
Your laptop has three chips that can run a neural network. The NPU is the one built for it, and the one almost nothing on Linux was using.
- CPU
- Runs everything else you’re doing. AI on it means a hot lap, a loud fan and a slower browser while it works.
- GPU
- Fast at AI too, but it draws real power, and on Linux it needs a compute runtime most distros don’t install.
- NPU
- A small coprocessor that only runs neural networks, at a few watts. Idle until an app asks, and asleep again 100 ms after it answers.
- 10%of one battery charge to index 9,763 files with vinoSearch: 66 minutes at ~9.2 W, against 4.5 W idle
- 3%of one CPU core while it does. The NPU runs the model; the CPU only reads the files
- 3.3×faster live captions than the CPU: 12 seconds of speech in 0.70 s on the NPU, 2.30 s on the CPU
- 0 Wwhen idle. The NPU powers down 100 ms after its last job, and vinoWhisper’s server exits itself after five quiet minutes
Longer battery, cooler laptop
A model on the NPU costs a few watts; the same model on the CPU drags the whole chip up with it. vinoSearch indexes about 5× faster on the NPU than on the CPU at about a tenth of the energy per chunk. On the NPU, a video call with background blur or an hour of live captions is a background task, not a reason to find the charger.
Fast enough to feel live
Captions trail the audio by about 1.5 s on the NPU, against roughly 4.6 s on the CPU. The person segmentation behind background blur takes 1.4 ms a frame instead of 3.4. Face unlock is done in about two seconds, and vinoSpeak starts talking in about 1.5 s however long the text is.
Room for better models
Because the NPU is cheap to run, the apps pick the model that’s accurate, not the one that’s small. vinoAuthFace uses a 175 MB ResNet50 face model on NPU installs instead of the 13 MB MobileFaceNet that CPU installs use. vinoWhisper runs whisper-small.en rather than the tiny and base models, which are 2.6 to 3.8× faster but mishear words. Studio Light’s Auto light in vinoOpenEffects, a low-light model, only runs on the NPU: 10 ms a frame there, 850 ms on the CPU.
Your CPU and GPU stay yours
The NPU is a separate chip. Live captions, a blurred background and a file index running at once don’t slow your compiler, your game or your browser, and vinoWatch shows you the split in the tray. No NPU? Every app falls back to the CPU, most can use the Intel GPU too, and each says so rather than failing quietly.
| Workload | NPU | Intel iGPU | CPU |
|---|---|---|---|
| Live captions, 12 seconds of speech vinoWhisper | 0.70 s | 0.95 s | 2.30 s |
| Person segmentation for blur, one frame vinoOpenEffects | 1.4 ms | — | 3.4 ms |
| Face detection, one frame vinoOpenEffects | 4.9 ms | — | 8.1 ms |
| Auto light (Studio Light), one frame vinoOpenEffects | 10 ms | — | 850 ms |
| Indexing files, per chunk vinoSearch | 1× time, 1× energy | — | ~5× the time, ~10× the energy |
| App | With an Intel NPU | Without one |
|---|---|---|
| vinoAuthFace | ResNet50 face model, 175 MB, mean match score 0.875 | MobileFaceNet, 13 MB, mean match score 0.815 |
| vinoWhisper | whisper-small.en, about 1.5 s behind the audio, plus SmolLM3-3B for dictation cleanup and session summaries | Same model on the iGPU (~2 s behind) or CPU (~4.6 s) |
| vinoOpenEffects | Blur, replace, Studio Light with Auto light, Center Stage and reactions | Everything except Studio Light’s Auto light |
| vinoPop | Gemma 4 E2B (4 GB) or the more accurate E4B (5 GB) | Same models on the iGPU or CPU, slower |
| vinoSearch | EmbeddingGemma 2 for text and photos, plus OCR for screenshots and scans, indexing at 3% of a core | Same models on the CPU, picked automatically, ~5× slower at ~10× the energy |
| vinoSpeak | Kokoro-82M, speaking in about 1.5 s | Same voices on the iGPU or CPU, which needs no driver |
- The app hands its model to OpenVINOIntel’s open source inference runtime. Every vinoPilot app uses it, so they share one NPU without stepping on each other.
- OpenVINO compiles it for the NPUThrough Intel’s open source
intel_vpudriver, which ships in the mainline Linux kernel. The compiled copy is cached, so the next start is quick. - The NPU runs it, then sleepsInference happens on the coprocessor at a few watts. 100 ms after the last job it powers down on its own.
- No NPU? It tells youEach app falls back to the CPU, most can use the Intel iGPU too, and says which one it picked. Its doctor command names the missing driver and the command to install it, even on Bazzite or Silverblue.