In Search of Perfection: aci-cpp vs. llama.cpp v1.0.0

Justin Bennington Principal Engineer on November 18, 2026.

Heston Blumenthal is the head chef of The Fat Duck, a restaurant in the UK which for some amount of years has changed the underlying structure of fine dining. He had a show a few years back on the BBC network called “In Search of Perfection”. He’d use calipers to measure people’s jaws to find out the perfect size of a cheeseburger based on the clearance of the average person’s bite, spend months trying to figure out how to make his own processed American cheese from reverse engineering McDonalds quarter pounders, combining all sorts of different optimizations derived from studying the form of anything he was seeking to ultimately make perfect.

At Actual, we are similarly chasing perfection in our own domain, especially in how we develop our ex-nihilo flagship engine, e. Today, we’re sharing improvements we’ve made using the same development processes to llama.cpp: aci-cpp is our oldest engine, which we have continuously improved since we first gave users access to the platform. Here’s what we’ve accomplished, and where we're taking it in the future.

What's aci-cpp?

aci-cpp is our fork of llama.cpp. It's the engine the Actual client runs on Macs, on machines like the DGX Spark, on consumer Nvidia cards, and tons of other hardware. llama.cpp is one of the best pieces of open-source software ever created, and we build on it rather than around it. Where upstream ships a better kernel, we take it. Where our users' workloads need something upstream doesn't do yet, we write it.

The client now ships aci-cpp 7e2590782a. Below, we compare it with llama.cpp v1.0.0, the upstream release, on Qwen3.8-27B at Q4_K_M, a model many of you run.

How fast is the next turn?

Agents and long chats spend most of their life on the second turn and beyond: a long conversation is already in memory, a few hundred new tokens arrive, and you're waiting for the first word back.

On a DGX Spark, aci-cpp answers a follow-up turn 17% to 28% faster than llama.cpp v1.0.0, from 8K to 250K tokens of context. With 64K tokens in context, the first token comes back in 0.78 seconds instead of 0.97. With 250K, it's 1.79 seconds instead of 2.10.

Time to first token for a follow-up turn on DGX Spark, aci-cpp versus llama.cpp v1.0.0, from 8K to 250K tokens of context

Our previous build took 5.2 seconds at 250K. This one takes 1.79.

How much can one DGX Spark serve?

Serving many streams at once, aci-cpp delivers 6% to 25% more total throughput than llama.cpp v1.0.0 on 512-token prompts, and 19% to 37% more on 4,096-token prompts, from 2 to 16 concurrent streams.

Total output throughput on DGX Spark for 512-token prompts, aci-cpp versus llama.cpp v1.0.0, from 2 to 16 concurrent streams

We also ran the same grid against upstream's latest development build, which has its own multi-token prediction (MTP). With speculation off in both, aci-cpp led in every cell. With MTP on in both, our drafter beat upstream's MTP by 4% to 38% in every cell.

What about long context on a Mac?

Long context is where Macs used to struggle. Reading a quarter-million tokens of memory for every new word puts real pressure on the memory system.

On an M3 Ultra with 250K tokens in context, aci-cpp now decodes at 12.6 tokens per second, 46% faster than our previous client build, and slightly ahead of llama.cpp v1.0.0 at every depth we measured, from 8K to 250K.

Single-stream decode speed on M3 Ultra by context length, aci-cpp versus llama.cpp v1.0.0 and our previous build

Under many streams, the M3 Ultra serves 16% to 26% more tokens per second than v1.0.0 on 512-token prompts. On an M2 Ultra, batched decode runs up to 34% faster at 32 streams, and served throughput on 512-token prompts is 6% to 8% higher.

Where do we stand?

Here's every platform and workload we measured against llama.cpp v1.0.0, side by side.

aci-cpp speed relative to llama.cpp v1.0.0 by platform and workload, geometric mean with lowest and highest measured cell

aci-cpp is ahead of llama.cpp v1.0.0 on eight of the ten workloads, by up to 28%, and within 1% on the other two.

What did we change?

Sixty merged performance changes since we opened the platform. A few themes:

  • Prefill on NVIDIA. New int8 tensor-core matrix multiplies read GGUF weights in place. On the DGX Spark, prompt processing at 2,048 tokens got 27% faster on Q4_K_M and about 31% faster on Q8_0, each measured against our own previous build. RTX Blackwell cards got the same family of kernels.
  • Qwen3.8's linear-attention layers. Fused, chunked kernels for the gated DeltaNet on CUDA and Metal, for both prefill and decode.
  • Speculative decoding. A faster MTP drafter, and few-row matrix multiplies that verify several draft tokens for about the cost of one.
  • Long context. Flash attention that skips masked blocks, per-sequence KV blocks in shared caches, and a row-list read path for very long KV spans on the M3 Ultra.
  • Serving. Prefill scheduled by token budget across slots, parallel sampling, and faster prompt-cache saves.

How we measured

Every number here is measured in our lab on Qwen3.8-27B Q4_K_M, alternating builds on the same machine; each chart notes its conditions.

Our belief at Actual is that token efficiency should only be bounded by the physical limitations of the hardware, and not by trade-offs within the software. Ultimately, allowing you -- the Actual user -- to get more tokens of the same quality out of the hardware you already own is our idea of achieving this goal. We couldn't do it without you. Every email, bug report, or DM our employees receive has been critical for our ability to build this for you, and we can't thank you enough for your support.

For you, we will keep chasing perfection.

Install Actual

Run AI on your own computers. One command installs Actual; then authorize the device at the actual.inc/device link it prints.

macOS & Linux setup guide