Quick answer: TOPS (Tera Operations per Second) is a term used to measure, simplify, and advertise Neural Processing Unit (NPU) performance in AI PCs. TOPS is not a perfect metric, and many variables contribute to how well a system can handle AI tasks. It nevertheless provides a quick reference to an NPU’s speed and how it compares to its competition.
As we wade into the world of AI PCs, plenty of new terms and acronyms are popping up that explain the new hardware and abilities.
You might have noticed that many modern laptops — and soon desktops, with AMD’s Ryzen PRO 8000 announcement — are shipping with processors (CPU) that include a Neural Processing Unit (NPU) for a boost in AI-specific performance.
With the NPU comes a need for a new metric to measure performance, and that is where the term “TOPS” comes into play. Here’s why you’ll be hearing a lot more about TOPS in the future as AI PCs continue to saturate the market.
What is TOPS?
TOPS is not a new term in the tech world, but it’s certainly getting a lot of mainstream attention recently with the rise of high-end AI PCs (the definition is still murky, but it boils down to a PC with modern CPU, GPU, and NPU). Windows Central Editor Rebecca Spear wrote a great article about how NPUs differ from GPUs, which you can reference for more information.
TOPS simply denotes how many trillions of operations a system can perform per second. TOPS can be used to measure overall system performance, but it can also be split up for specific hardware.
For example, NVIDIA’s RTX 4090 graphics card can offer more than 1,300 TOPS of performance, whether for gaming or to accelerate AI tasks. It’s one of the most powerful GPUs on the market today, yet that number is still slightly baffling.
Intel’s Core Ultra CPUs — the first from the company to offer an NPU — offer 34 TOPS overall (including the CPU, integrated GPU, and NPU), with about 10 TOPS carved out of the total for the NPU alone. AMD’s Ryzen 8040 mobile CPUs bump the NPU performance up to 16 TOPS, and the upcoming Qualcomm Snapdragon X Elite will launch with an NPU capable of 45 TOPS.
Intel, AMD, and Qualcomm are expected to continue buffing NPU performance in upcoming generations. We now know that Intel’s next-gen Lunar Lake CPUs will offer an NPU with 45 TOPS, and AMD has said it expects to triple NPU performance in its next run of mobile chips.
These chip makers are leaning into TOPS to advertise their products, and it’s mainly to simplify performance and help buyers understand what they’re getting.
Why is AI performance measured with TOPS?
TOPS certainly does not represent the whole picture when analyzing NPU performance. Like the TFLOPs craze in 2020 when next-gen consoles were still on the horizon, TOPS is being used to simplify and advertise system performance to more people. An NPU capable of two or three times as many TOPS as its competitor looks great on paper, but the metric ultimately lacks nuance.
It’s a “Big number good, small number bad” situation, at least at first glance, but there are many more factors at play that can influence an NPU’s performance. Theoretically speaking, a specific NPU with, say, 100 TOPS might be tuned for a specific task, in which it can outperform a separate NPU with 200 TOPS that is tuned for a different task.
Measuring a PC’s overall performance generally boils down to concrete benchmark numbers and abstract observations. In our laptop reviews, for example, we provide plenty of hard numbers using specialized testing software, but we also provide a more general “feel” of how the system performs in real-world scenarios.
Testing an NPU is also something that needs to be worked out as we move forward. There are benchmarks available that target the NPU alone, and they’re a good start on the way to measuring power in terms of specific tasks.
TOPS is also being used to set hardware benchmarks for on-board AI tools. As Tom’s Hardware first reported, Microsoft has set a 40 TOPS requirement for running Copilot locally on your system with no reliance on a connection to Microsoft’s servers.
So, even if TOPS isn’t the best metric to judge an NPU’s performance, it does give buyers a standard with which they can roughly compare AI PCs.
Should you care about TOPS?
Measuring TOPS is a quick and easy way to compare NPUs or to judge whether or not an NPU is capable of handling certain tools. However, TOPS should not be considered the end-all measurement when it comes to NPU power. There’s a lot of nuance involved, and you want to be sure that you’re getting the right hardware for the job ahead.
Should you care about TOPS? It depends. If you’re a casual PC user who handles email, web browsing, and productivity, you can no doubt continue without worrying about NPUs and the methods used to measure their performance. If you’re shopping for a new laptop, you might still want to pay attention to the requirements for certain AI tools that can help make your life easier.
If you are, on the other hand, interested in the emerging world of AI PCs, NPUs and their capabilities measured in TOPS are likely already familiar topics. Just be sure to consider that TOPS is not the perfect metric and is instead an easy way for chip makers to compare and advertise their products.
