Intel is supposedly putting an AI coprocessor into its latest Arrow Lake desktop CPUs, but these don’t the 40 trillion operations per second (TOPS) minimum performance to run Windows 11 Copilot+. Why is valuable chip real estate being taken up by this mental midget, relative to a standard graphics card?
“Intel’s Arrow Lake-S won’t be an AI powerhouse — 13 TOPS NPU is only slightly better than Meteor Lake, much less than Lunar Lake” (Tom’s Hardware, July 9, 2024):
Arrow Lake-S will be the first Intel desktop architecture with a neural processing unit (NPU), but it won’t be as fast as people might expect. @Jaykihn on X reports that Arrow Lake-S will include an NPU that is only slightly more powerful than Meteor Lake’s NPU, featuring just 13 TOPS of AI performance.
Having an NPU in a desktop environment is virtually useless; the main job of an NPU is to provide ultra-high AI performance with a low impact on laptop battery life. Desktops can also be used more often than laptops in conjunction with discrete GPUs, which provide substantially more AI performance than the best NPUs from Intel, AMD, or Qualcomm. For instance, Nvidia’s RTX 40 series graphics cards are capable of up to 1,300 TOPS of AI performance.
The bottom-of-the-line Nvidia RTX 4060 has a claimed performance of “242 AI TOPS” and is available on a card for less than 300 Bidies. Is the idea that a lot of desktop machines are sold without a GPU and that Microsoft and others will eventually find a way to “do AI” with however much NPU power is available within the Arrow Lake CPU? (Software that evolved to require less hardware would be a historic first!)
AMD already has a desktop CPU with distinct NPU and GPU sections, the Ryzen 8000G.
AMD Ryzen 8000G Series processors bring together some of the best, cutting-edge AMD technologies into one unique package; high-performance processing power, intense graphics capabilities, and the first neural processing unit (NPU) on a desktop PC processor.
Based on the powerful “Zen 4” architecture, these new processors offer up to eight cores and 16 threads, 24MB of total cache, and AMD Radeon™ 700M Series graphics. Combining all of this into one chip enables new possibilities for customers, in gaming, work, and much more; without the need to purchase a discrete processor and graphics card, customers can keep their budget lower, while enjoying outstanding performance.
“The Ryzen 7 8700G leads the pack …The processor has a combined AI throughput of 39 TOPS, with 16 TOPS from the NPU.” (source) If the 39 TOPS number is correct, it seems unfortunate given the Windows 11 Copilot+ demand for 40 TOPS.
Why not just build more GPU power and let it be used for graphics or AI depending on what programs are running? The big advantage of the NPU seems to be in power efficiency (source), but why does that matter for a desktop computer? Even at California or Maskachusetts electricity rates, the savings converted to dollars can’t be significant.
My laptop has an intel processor with integrated graphics and an nvidia GPU. Windows is clever enough to power down the GPU when the laptop is on battery, and when the integrated graphics are sufficient for the software in use. When the laptop was brand new, and the GPU was being used heavily, the battery lasted almost two hours. A year later I am lucky to get 30 minutes.
If a software feature requires a GPU, I can’t use it without a power cord. A feature that requires the NPU is not limited in that way.
CL: Thanks for this real-world report. The NPU makes sense to me for laptops, but I can’t figure out why it makes sense on desktops unless it is at least powerful enough to support Copilot+ (and even then I don’t see why the power savings are worth the additional complexity compared to just having a GPU).
They’re useful for non realtime tasks like a video editor that runs semantic segmentation offline. The industry has swung from the realtime self driving car blockbusters to non realtime tasks like stable diffusion & offline photo editing.
Good questions, but shouldn’t the inquiry include why Intel names all these things ____ Lake?
Very kind of you to consider Intel after what they did to customers who bought their crap “Raptor Lake” processors. https://www.reddit.com/r/starcitizen/s/LjCCmvQKwH
That Reddit post is a little strange. It talks about via corrosion on the PCB (printed circuit board), which isn’t part of a CPU. Professor Google says that the vias are inside the silicon and also that Intel is approving all warranty claims.
https://techreport.com/news/intel-cpu-issues-release-patch-mid-august/
https://www.pcmag.com/news/intel-reveals-which-13th-14th-gen-desktop-chips-get-extended-warranty
So it sounds bad (do they at least do advance replacement? Or are consumers left without a desktop computer for some period of time? I couldn’t find anything about that except for some special partners of Intel). The worst mistakes were in the microcode? It is always software people who ruin everything.
YouTuber Gamer Nexus has led the charge to get Intel to admit that there’s a problem, and even then Intel has been criticized on how they’ve handled it. Gamer Nexus says the issue is “oxidation or potential corrosion within the silicon, specifically relating to the vias, and other tips like excessive voltage, memory speeds, unstable frequency, and failure rates.” I don’t have a dog in the fight to watch all the Gamer Nexus vids (like here: https://m.youtube.com/watch?v=gTeubeCIwRw) but consumer advocates like Louis Rossmann also criticized Intel’s response, like limiting which batches of processors qualify for the recall…
Gino: Fair to say that this debacle proves the superior qualities of female leadership (AMD) and, therefore, is another reason to vote for Kamala Harris in November?
Phil – when you’re right, you’re right! Kamala’s record speaks for itself, just ask your nearest Venezuelan migrant paying for his groceries with a US gov’t issued gift card. Haters like RFK, Jr. need to get out of the way, and not spread their anti-Pfizer rhetoric. https://x.com/RobertKennedyJr/status/1825950913107775707
The question shouldn’t be “Why doesn’t RFK, Jr. sit down when a Woman of Color is speaking?” but “Why was RFK, Jr. able to stand up when a Woman of Color was available to speak?” (same question for any other white male)
Wise words indeed. Only thing that gives me pause, is… I was watching the annual Taliban victory over NATO parade, as one does, and I noticed a stunning lack of diversity and lack of female soldiers among the bearded freedom fighters (mujahideen). When on the other side, we had freshly elected Biden, diversity, Kamala, and LGBT troops. Something doesn’t compute… Source: https://youtu.be/rpYnUyUr8Ak