# The Real Cost of Running a Local LLM: Measured in Euros per Million Tokens

Energy costs of three models on RTX 3090 measured token by token.

- **Source:** [Towards Data Science](https://towardsdatascience.com)
- **Published:** 2026-07-14
- **Topic:** [Data & AI](https://brifing.si/c/data-ai)
- **Original article:** <https://towardsdatascience.com/how-much-does-it-actually-cost-to-run-a-local-llm-e-per-million-tokens-measured/>
- **On Brifing:** https://brifing.si/s/66c64f72-4064-4d2d-ba1b-8a55ab44a9b0

## Introduction

There's a common belief that running local language models is free. To test this claim, we measured the energy consumption of three different models using an RTX 3090 GPU. The actual costs calculated based on electricity tariffs were compared against cloud API pricing.

## Measurement Method

The experiment used a machine running openSUSE with an RTX 3090 GPU. Three models (gemma3:1b, gemma4:26b, gemma3:27b) were tested under identical workloads using Ollama. Power consumption was sampled every 10 seconds using nvidia-smi.

## Energy Cost Calculation

Energy consumption during each model's runtime was calculated using Bulgaria's current electricity rates (0.30 BGN daytime, 0.18 BGN nighttime). The data was converted to Euros to determine cost per million tokens.

## Tested Models

Small-scale gemma3:1b, medium-scale gemma4:26b, and large-scale gemma3:27b models were tested with Q4_K_M quantization. Each model processed a fixed workload of 256 tokens and ran for 4 minutes to reach thermal equilibrium.

## Monitoring Tool

An open-source software called HomeLab Monitor tracked GPU power consumption. This tool integrated data from nvidia-smi to calculate actual costs. The MIT-licensed software runs as a single container.

## Setup Process

The measurement system was deployed using Docker Compose. Configuration files from the homelab-monitor GitHub repo were downloaded and launched with a single command. The system monitors CPU, RAM, disk, and GPU usage in real-time.

## API Integration

The benchmark script used a special API key to send data to the monitoring system. A Python-based client, working with standard libraries, transmitted measurement data without requiring additional installations.

## Workload Scenario

Tests used ollama's /api/generate endpoint with fixed prompts in a loop. Each call recorded generated tokens (eval_count) and processing time (eval_duration) to capture actual performance metrics.

## Cold Start Effect

To eliminate cold start effects, each model was warmed up before measurements. Main tests only covered periods when the GPU reached stable operation.

## Interpreting Results

Five models proved cheaper than cloud APIs, while three unexpectedly cost more. Interestingly, these outcomes couldn't be predicted solely based on parameter counts.

## Cost Comparison

Measured costs were compared against 'Flash'-class cloud API pricing. Local execution wasn't universally cheaper, with results varying by model size and efficiency.

## Conclusion

This study shows local LLMs don't automatically offer cost advantages. Proper model selection and hardware efficiency can yield significant savings compared to cloud solutions. Developers should conduct such measurements for effective cost optimization.

## Sources covering this story

- https://towardsdatascience.com/how-much-does-it-actually-cost-to-run-a-local-llm-e-per-million-tokens-measured/ — 

---

_Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._