How Cloudflare Saved 100TB of Memory by Optimizing 1.1.1.1's DNS Cache
Five structural changes reduced memory usage by over 50% while improving performance.
1The Scale of DNS Caching
Cloudflare's Big Pineapple platform manages over 250 billion cache entries for 1.1.1.1 and other DNS services. Wasting just one byte per entry translates to over 250GB of memory loss across their fleet. This massive scale demonstrates how even tiny optimizations can have enormous impact when applied globally.
2Memory Reduction Goals
Engineers successfully reduced the memory footprint of each cache entry by over 50%. This saved approximately 100 terabytes of memory total - equivalent to the RAM capacity of 130 Gen 13 servers. The optimizations also improved cache performance metrics significantly.
3Performance Improvements
The changes resulted in 43% higher insert throughput and 19% lower lookup latency. Better memory locality and fewer allocations meant these memory savings didn't come at the cost of speed, eliminating the typical performance trade-off.
4Cache Operation Principles
Big Pineapple starts with an empty cache on cold start. As DNS queries arrive, the cache fills until reaching maximum capacity, when older or less popular items get evicted. With ECS enabled, different versions of the same query get cached, increasing memory usage substantially.
5Data Structure Optimizations
Cache entries consist of key-value pairs. The value stores DNS responses along with metadata like creation time, hit counter and TTL. Engineers achieved significant savings by restructuring these data containers without changing functionality.
6Switching from Vec to Box
Replacing Vec<T> with Box<[T]> eliminated capacity fields, saving 8 bytes per field. This also removed heap space reserved for future growth. The change saved 64 bytes per entry, totaling over 15TB across all cache entries.
7Simplifying List Structures
Instead of separate lists for answer, authority and additional sections, a single list with u16 offsets was implemented. This replaced 8-byte pointers and lengths with 2-byte offsets, saving 28 bytes per entry while maintaining functionality.
8Memory Alignment Benefits
Rust's memory alignment requirements meant removing small fields could eliminate additional padding. Converting boolean fields to bitflags reduced surrounding padding, shrinking structs more than just the size of the booleans themselves.
9Owner Field Optimization
The typically redundant owner field was removed from DNS records, only preserved when different from the queried domain (like with CNAME records). This optimization proved particularly impactful in ECS-heavy locations.
10Production Environment Testing
To measure impact, engineers generated random cache entries matching production traffic patterns. A custom allocator tracked memory usage while insert throughput and lookup latency were measured. Results were later verified in actual production environments.
11EDNS Client Subnet Impact
When ECS is enabled, different versions of the same query are cached for different networks. This increases both entry count and memory consumption. The optimizations proved especially valuable in locations with heavy ECS usage.
12Industry Significance
Cloudflare's optimizations set an important precedent for large-scale DNS providers. The techniques achieving 50% memory reduction while improving performance could become industry standards. As internet traffic grows, such optimizations will only become more critical for infrastructure efficiency.