Junie Local Achieves 70% Speed Boost on Mac M5 Hardware
New inference engine and custom draft models accelerate local LLM coding performance significantly.
1Performance Gains on M5
Junie Local has achieved a significant performance jump on Mac M5 hardware, increasing coding task speeds from 40 tokens per second to 70 tokens per second. Burst speeds now reach 107 tokens per second. This improvement stems from replacing the previous MLX-VLM engine with the specialized Splash inference engine architecture.
2Splash Engine Integration
The Splash engine, originally released by Inco AI, replaces MLX-VLM to provide superior decoding efficiency for Qwen models. By verifying eight drafted tokens simultaneously, it reduces context attention costs per position. The team ported their existing int8 operations from MLX-VLM to Splash, ensuring compatibility with M5 chip architecture while maintaining high performance.
3Speculative Decoding Strategy
Decode speed in speculative systems relies heavily on the model-drafter's acceptance rate. The team addressed the overthinking tendencies of Qwen 3.8 by integrating the DFlash2 draft model. This model is specifically tuned for the Qwen3.8-3.6-27B-blend, balancing model capability with token generation efficiency to prevent speed bottlenecks during complex coding tasks.
4Training the DFlash2 Drafter
To optimize the DFlash2 model, the team generated diverse agentic coding trajectories using their internal environment. They fine-tuned the model using SpecForge, requiring only one million generated tokens to match Inco’s acceptance rates. This process highlights that minimal data is needed to effectively train drafters for specific in-domain coding tasks.
5Hardware and Memory Efficiency
Beyond speed, the update optimizes memory consumption, requiring approximately 24 GB of RAM for a 100k context window. The software is now compatible with M4 Macs and newer, provided they have at least 64GB of RAM. However, users should note that prefill speeds on M4 chips are four times slower than M5.
6Comparative Performance Metrics
The combination of the Splash engine and DFlash2 drafters delivers a two-fold improvement over the previous MTP and MLX-VLM approach. Prefill performance has improved from 850 tokens per second to 1200 tokens per second. These metrics demonstrate the impact of community-driven inference optimizations on local LLM deployment workflows for professional developers.
7Upgrade and Installation Path
Existing users can upgrade by running the 'junie update' command and then executing '/local' with a ctrl+r command to refresh the engine. New users can install the system and local models via a provided curl script. The DFlash2 drafters are also publicly available for download on the HuggingFace platform.
8Future of Local Inference
This release underscores the rapid evolution of LLM inference, where community innovations can drastically alter performance within just two months. By focusing on model-specific engines and targeted fine-tuning for speculative decoding, Junie Local demonstrates how developers can maintain high-speed coding assistance on local hardware, effectively bridging the gap between cloud and edge.
Original article: The JetBrains Blog — Junie Local Achieves 70% Speed Boost on Mac M5 Hardware