- Neomind Labs says a patched ik_llama.cpp build let a 2013 Intel Xeon E5-2690 v2 with no GPU run Gemma 4 26B at roughly 5 tokens per second, though no one else has replicated that number.
- The author says the fix required adding AVX1-only build flags after diagnosing a compile failure with help from Claude, and the resulting PR #2138 was still awaiting maintainer review at publication.
- Gemma 4 26B-A4B's mixture-of-experts design activates only about 4 billion parameters per token, which multiple sources confirm helps explain why the model can run faster than its headline size suggests on memory-limited hardware.
What Folks Are Saying Down at the Feed Store
Word around the Hacker News barn is that a developer going by Neomind Labs published a blog post on June 8, 2026, claiming to have gotten Google's Gemma 4 26B mixture-of-experts model running on a 2013-era Intel Xeon E5-2690 v2 — that's Ivy Bridge generation silicon, older than most people's pickup trucks — with zero GPU help, according to the Neomind Labs post. The author says the setup used the ik_llama.cpp inference fork and a Q8_0 quantization of the model, and the claimed throughput, per the Neomind Labs post, came out to roughly five tokens per second. Now, before you go betting the hog farm on that number, understand that it comes from exactly one person's personal engineering diary, with no independent replication published as of the time you're reading this.
The Neomind Labs author is upfront — almost endearingly so — about not being a C++ programmer and about leaning on the Claude AI assistant to diagnose the build failure that nearly stopped the whole experiment cold. That kind of honesty is worth a biscuit, but it also means the technical depth of the patch work is harder to evaluate from the outside. The stated motivation, per the Neomind Labs post, is practical and modest: keep a local model around on retired enterprise server hardware as a free fallback when paid APIs go dark or for cheap batch jobs, not as any kind of production showpiece.
What We Actually Know for Certain
A few things here are as solid as a cast-iron skillet. The Gemma 4 26B-A4B model does use a mixture-of-experts architecture with 128 total experts, and multiple independent sources — including the n1n.ai local benchmarking writeup — confirm that only roughly 2 experts, amounting to about 3.8 to 4 billion active parameters, are engaged per token during inference. That architectural fact matters a lot for understanding why this model might behave faster on memory-bandwidth-constrained hardware than its 26-billion-parameter name would suggest, and it's not something the Neomind Labs author made up.
The patch the author submitted to fix AVX1 compatibility is publicly cited as ikawrakow/ik_llama.cpp PR #2138, which anyone can look up. The Neomind Labs post states clearly that the core technical obstacle was that ik_llama.cpp assumes the presence of AVX2 and FMA3 instruction sets — features Intel didn't ship until the Haswell generation in 2014 — while the Ivy Bridge E5-2690 v2 only supports AVX1. The author says the fix involved targeted build-flag changes and dropping the runtime-repack flag. As of the blog post's publication date, that PR was still open and had not been merged by the maintainer, according to the Neomind Labs post.
What Nobody's Verified Yet
The five-tokens-per-second figure itself is the big ol' catfish in the pond: plausible-looking but unconfirmed. A peer-reviewed paper in CEUR Workshop Proceedings independently benchmarked CPU-only llama.cpp inference on an Intel Xeon E5-2695 v2 — that's the same Ivy Bridge-EP family as the chip in this story — and recorded roughly 5 tokens per second on a 12-billion-parameter model and only about 2 tokens per second on a 24-billion-parameter model. That academic data gives us a useful ballpark for what Ivy Bridge Xeon silicon can do, but it does not cover Gemma 4 specifically, and a 26B model outpacing the academic benchmark's 24B result would need explaining.
Meanwhile, the hardware guide over at gemma4-ai.com — a specialist how-to site, not an independently peer-reviewed source — characterizes 26B CPU-only inference as producing 0.5 to 2 tokens per second and describes the experience as, and we're paraphrasing here because that site's words are their own, painfully slow but functional. That's notably below the figure Neomind Labs claims. The Neomind Labs author's explanation for the gap is the MoE active-parameter count and ik_llama.cpp's specific optimizations, which is a reasonable engineering argument — but an argument is not a replication.
Our Analysis: Plausible But Still a Tall Tale Till Someone Else Tests It
Here's where we put on our thinking overalls and speculate a little, so label this as analysis rather than reporting. The story Neomind Labs is telling is internally coherent. The MoE architecture's low active-parameter count is a legitimate reason a 26B-labeled model might behave more like a 4B model on a memory-bandwidth bottleneck, and ik_llama.cpp is a known performance-focused fork with real engineering behind it. The existence of a public, reviewable patch PR adds at least a thread of verifiable substance that a pure vaporware claim would lack.
That said, the academic benchmarks from the CEUR paper suggest that comparable Ivy Bridge Xeon hardware tops out around 2 tokens per second at 24B parameter scale under standard CPU inference conditions. Beating that by 2.5 times on a larger model requires the MoE explanation to be carrying a lot of weight. It might well be doing exactly that — but until a second engineer with a dusty old Xeon fires up the same setup and publishes numbers, this story is still a one-man hog call echoing across an empty field. The community interest it generated on Hacker News might be the best thing that happens to it: someone out there probably has a decommissioned Ivy Bridge box and an afternoon to kill.
Who is doing the hollering
These links show where the chatter came from. A link is attribution, not our endorsement or independent confirmation.
- Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPUNeomind Labs · primary
- Deploying LLMs on CPU-only Environments with llama.cppCEUR Workshop Proceedings (Vol-4164) · specialist
- Can Your PC Run Gemma 4? RAM & GPU Requirements 2026gemma4-ai.com · specialist
- Benchmarking Google Gemma 4 26B and 31B Locallyn1n.ai · specialist
Last checked Jul 16, 2026, 5:07 AM EDT. Talk Around Town: The 5 tokens/second figure for Gemma 4 26B on a 2013-era Ivy Bridge Xeon is claimed by a single author in a personal blog post. The patch enabling it had not yet been merged by the maintainer as of publication. No independent tester has published a replication. Treat performance numbers as one engineer's reported result, not a benchmarked baseline.