# Differential Privacy for Hugging Face Trainers Without Rewriting Loops Create privacy-preserving models with DPTrainer without rewriting training loops. - **Source:** [The JetBrains Blog](https://brifing.si/source/782eebd1-7be8-49cc-b27c-e552516691b2) - **Published:** 2026-08-27 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/b46a8f6a-f52f-4d56-ba09-5503b26ee598 ## What is Differential Privacy? Differential privacy is a mathematical framework that protects individual data points. It ensures a model behaves almost identically regardless of whether any single example was included in the training set. For large language models, this is the strongest defense against training data leakage, particularly effective against membership inference attacks. ## The DP-SGD Method Differential privacy is applied in neural network training via DP-SGD (Differentially Private Stochastic Gradient Descent). This method computes a gradient per sample, clips outliers, aggregates gradients in batches, and adds noise to obscure any single example's footprint. This guarantees the privacy of the training method. ## Opacus and Hugging Face Integration Opacus is a library for DP-SGD in PyTorch but requires a manual training loop and lacks good Hugging Face integration. Hugging Face Trainer automates many functions like distributed training, checkpoints, and evaluation but lacks differential privacy support out of the box. ## The Birth of DPTrainer JetBrains Research developed DPTrainer to bridge the gap between Opacus and Hugging Face Trainer. This library automates complex processes like model wrapping, optimizer creation, and data loading. Created by Evgeny Grigorenko and David Stanojevic, DPTrainer is now maintained by Mihajlo Linic. ## Privacy Budget A key concept in differential privacy, the privacy budget represents the maximum acceptable risk of information leakage. It limits how much any single data point can shift the output distribution. DPTrainer manages this budget via the PrivacyArguments dataclass. ## Code Example Using DPTrainer is straightforward. Set a target epsilon value with PrivacyArguments and initialize DPTrainer. An internal accountant tracks privacy budget expenditure, and the remaining budget is saved in checkpoints for easy resumption of training. ## Specialized Trainers DPTrainer's real power lies in its privatize_trainer function, which patches any Trainer-based class at runtime to inject DPTrainer into its inheritance chain. This allows specialized trainers like DPOTrainer to use differential privacy without rewriting their logic. ## Impact and Conclusion DPTrainer is a significant step in addressing data privacy concerns. It enables creating differentially private models without rewriting training loops, offering major advantages for organizations handling sensitive data. This ensures future models will be more secure by design. ## Sources covering this story - https://blog.jetbrains.com/research/2026/08/dp-trainer/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._