// TOM'S HARDWARE US — HARDWARE & GADGET
Our first step at defining a CPU benchmark for agentic AI
The future of agentic AI lies in CPUs, or so the story goes. Here’s our first attempt to measure that.
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
Agentic AI has set off a firestorm in the world of CPUs over the past six months, yet it’s an area of performance that still doesn’t have solid, repeatable benchmarks to evaluate where the latest hardware falls. We’ve been working on an internal set of tests to evaluate at least one aspect of contentious agentic loops end-to-end, isolating CPU and memory performance from the accelerator to understand how these components contribute to the agentic loop.
This is still a work in progress, but we wanted to bring Tom’s Hardware Premium readers behind the scenes to understand how we’re going about tackling this problem, our methodology for testing, and detail what we’ve learned about agentic loops along the way. We've prototyped the test and settled the central design, but there's still work to do before the numbers it produces hold up to scrutiny.
Given that this area of computing is bleeding-edge and rapidly evolving, we undoubtedly have blind spots here. That’s OK. The testing we’re doing is built to look at how agentic performance scales across the CPU as sessions increase, providing an ideal throughput for each chip that provides as many agent sessions as possible while remaining interactive.
Toward the end of this article, we've included a section on some of the shortfalls of this approach, as well as some ideas we’re toying with to tackle things like multi-turn agentic flows. And keep in mind that these tests aren't meant to replace our other testing of local AI, but to supplement it. As this space continues to evolve and the hardware characteristics are better defined, we’ll be able to make more informed decisions about where this testing goes in the future.
"AI agents" imply some little workers that, strapped with an LLM, can go and handle whatever task you throw at them. It's a fine description, but we'll need a bit more a technical definition if we're going to benchmark the performance of agentic AI.
There are two parts here that are important. First are functions. With agentic AI, we give the LLM access to tools, which, in the case of popular agentic frameworks like LangGraph, are Python functions. The tool may be reading or writing a file, querying a database, running tests on code, or just about anything else you can define in a function. Here's a basic example.
This is how you write a tool call with LangGraph inside of Python, and it's an example of a tool call that the we've given the LLM access to through our LangGraph persona. We identify the function as a tool the LLM can call with "@tool," and then we define the function like normal Python with "def run_command," passing through the command as an argument. The extra "-> str" on the end notes that the argument will be a string, and the function should return a string.
That's followed up by a description of the tool that the LLM can read (triple quotes just allow you to space this across multiple lines in Python). We're prompting the LLM that it can run build or test commands using make or pytest. For this example, we wrap the function up with the anonymous lambda function and pass it back to an overarching trace function. That's not important right now, however. The goal is to illustrate what a tool call looks like.