# Automating Prompt Engineering with JetBrains' Koog Optimization Library Optimize agent instructions automatically using feedback loops and iterative refinement in Kotlin. - **Source:** [The JetBrains Blog](https://brifing.si/source/782eebd1-7be8-49cc-b27c-e552516691b2) - **Published:** 2026-10-29 - **Topic:** [Software & Engineering](https://brifing.si/c/engineering) - **Original article:** - **On Brifing:** https://brifing.si/s/a5a6a3b1-1e47-4aab-96ea-4f52fe30de4b ## Automating Agent Instruction Tuning JetBrains Research introduced the Koog Agents Optimization Library to automate prompt refinement for Kotlin-based agents. Instead of manual prompt engineering, developers define optimizable subgraphs within their agent architecture. The system runs the agent against a training set, evaluates performance, and generates improved instructions as a JSON artifact without requiring code rewrites. ## Integrating the Optimization Workflow The library replaces standard subgraphs with optimizable versions, allowing developers to isolate specific instructions for tuning. By providing a scoring function and a feedback mechanism, the optimizer iterates through multiple candidates. This process keeps several instructions alive simultaneously, ensuring that the model learns from both failures and successes during training rounds. ## Benchmark Performance on PUPA Testing on the PUPA privacy benchmark demonstrated significant gains in agent efficacy. The baseline score of 81.19 improved to 94.22 after the optimization process. These results, calculated as the mean of five distinct runs, confirm that the automated feedback loop effectively enhances agent performance while maintaining strict privacy-preserving data handling. ## Mechanism of Instruction Evolution The optimizer transforms vague, two-line instructions into highly detailed, multi-character directives. For instance, the system autonomously generated specific rules for redaction, such as replacing professor names and course codes with neutral placeholders. This evolution occurs entirely through iterative testing, uncovering optimal strategies that developers might not have explicitly defined themselves. ## Balancing Privacy and Utility The generated instructions reflect a complex trade-off between zero data leakage and task utility. The system learned to preserve enough context for the LLM to execute the user's request accurately while ensuring no private information is exposed. This emergent strategy demonstrates the power of automated tuning in handling sensitive data pipelines. ## Standardized Optimizer Interface The library provides a consistent interface across different optimization algorithms, including GEPA, BootstrapFewShot, MIPROv2, and ACE. Developers can switch between these methods by changing a single constructor, as the train and loadOptimizedAgent methods remain uniform. This modular design allows for easy experimentation with various strategies without altering the core agent code. ## Cost-Effective Optimization Options For teams seeking efficiency, BootstrapFewShot serves as a cost-effective starting point by mining demonstrations from successful runs without additional LLM calls. Conversely, ACE allows agents to accumulate knowledge through a playbook of insights rather than just refining prompts. These diverse options cater to different performance requirements and computational budget constraints. ## Future of Agentic Development This research highlights a shift toward treating agent instructions as dynamic artifacts rather than static strings. By integrating optimization directly into the Kotlin development lifecycle, JetBrains provides a scalable path for improving complex agent behaviors. Future updates are expected to merge the optimizable-subgraph support into the main Koog repository for broader adoption. ## Sources covering this story - https://blog.jetbrains.com/research/2026/10/koog-agents-optimization-library-prompts/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._