# Running Gemma 4 on a 13-Year-Old Xeon Without a GPU

The unconventional story of running a modern language model on an old server.

- **Source:** [Hacker News](https://news.ycombinator.com)
- **Published:** 2026-07-15
- **Topic:** [Data & AI](https://brifing.si/c/data-ai)
- **Original article:** <https://www.neomindlabs.com/2026/06/08/running-gemma-4-26b-at-5-tokens-sec-on-a-13-year-old-xeon-with-no-gpu/>
- **On Brifing:** https://brifing.si/s/6e9d057b-4d08-4426-957d-4cbcaeabb80b

## An Unusual Experiment

A 13-year-old HP StoreVirtual server wasn't designed to run modern language models. Yet, with dual Xeon E5-2690 v2 CPUs and DDR3 memory, it managed to run Google's Gemma 4 at 5 tokens per second. This is a significant step in making AI more accessible.

## Pushing Hardware Limits

An old Xeon with AVX1 but no AVX2 or FMA3 seemed inadequate for a modern mixture-of-experts model. However, the right optimizations overcame these limitations. This exemplifies maximizing hardware potential.

## The True Value of AI Agents

An AI agent like Claude quickly diagnosed why the code failed and suggested a solution. This shows AI can be used not just for ready-made solutions but also for solving complex technical problems.

## The Power of Optimization

The ik_llama.cpp fork includes optimizations needed for Gemma 4's MoE inference. Modifications to run without AVX2 enabled performance without loss on older hardware.

## A Real Engineering Feat

Techniques like speculative decoding, CPU-aware mixture-of-experts routing, and flash attention delivered effective results even on old hardware. This proves how optimized modern AI inference can be.

## Low-Cost AI Solutions

Running this model on hardware under $300 shows AI can be accessible to everyone. It offers a cost-effective solution, especially for batch jobs.

## AI and Hardware Evolution

A CPU from 2013 running a 2024 AI model demonstrates how far hardware and software optimizations have come. It shows older hardware can still be useful.

## Open-Source Contribution

Patch ikawrakow/ik_llama.cpp#2138 highlights the importance of community contributions. Open-source projects enable older hardware to run modern models.

## The Future of AI

This experiment shows AI isn't just for high-performance GPUs. Low-cost, accessible solutions can bring AI to broader audiences.

## Conclusion and Takeaways

Running Gemma 4 on an old Xeon is a significant step in democratizing AI technology. It underscores the importance of local models, especially when APIs are down.

## Sources covering this story

- https://www.neomindlabs.com/2026/06/08/running-gemma-4-26b-at-5-tokens-sec-on-a-13-year-old-xeon-with-no-gpu/ — 

---

_Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._