Cloudflare Saves 100 Terabytes of Memory with DNS Cache Optimization
DNS cache optimization leads to performance boost and resource savings.
1DNS Cache and Scale
Cloudflare stores over 250 billion DNS cache entries for 1.1.1.1 and other DNS services. At this scale, wasting a single byte per entry results in over 250 gigabytes of memory loss across the fleet. This highlights the importance of optimization efforts.
2Memory Savings and Performance Boost
Five successive changes reduced memory usage per cache entry by over 50%. These changes freed up roughly 100 terabytes of memory across the fleet. Additionally, the cache became faster: insertion throughput increased by 43%, and lookup latency dropped by 19%.
3Cache Occupancy and Eviction of Old Entries
Cloudflare starts with an empty cache on cold start. As DNS queries arrive, the cache fills up, and when it reaches the maximum entry count, older or less popular items are evicted to make room. This process ensures efficient cache usage.
4Cache Size by Data Center
Cache size varies by data center. When EDNS Client Subnet (ECS) is used, authoritative servers return different responses based on the client’s network. This results in caching multiple versions of the same query, increasing the impact of optimizations.
5Structure of Cache Entries
Each item in the cache is a key-value pair. The key identifies what was queried, while the value stores the DNS response, including metadata like timestamp and hit count. This structure forms the basis for optimizations.
6Strategies to Reduce Memory Usage
Using Box[T] and Box str instead of Vec and String reduced memory usage. These changes saved 64 bytes per entry, freeing up over 15 terabytes of memory across 250 billion entries.
7Optimizing DNS Records
Storing the answer, authority, and additional sections in a single list with offsets reduced memory usage further. This method saved 28 bytes per entry and improved cache efficiency.
8Rust Memory Management and Alignment
Rust inserts padding for memory alignment and rounds a struct’s size up to a multiple of its alignment. Removing small fields can eliminate additional padding, further reducing memory usage. This technique enhanced memory savings.
9DNS Record Owner and Compression
The owner of a DNS record is the domain it belongs to. In many cases, this owner matches the queried domain. However, with CNAME, the owner can differ. DNS wire format compresses repeated owners, reducing memory usage.
10Measurements and Production Impact
The impact of optimizations was measured in production. Memory usage varied based on traffic mix, cache occupancy, and memory state. These measurements validated the real-world effects of the optimizations.
11Balancing Performance and Memory Savings
Optimizations achieved memory savings while boosting performance. Fewer allocations and better memory locality balanced speed and space, making Cloudflare’s DNS services more efficient.
12Conclusion and Future Steps
Cloudflare’s DNS cache optimization delivered both memory savings and performance improvements. This success sets a foundation for future optimization efforts. Similar techniques are expected to be applied to other services.