Jeff: Compact Decision Models for Local Zero-Shot Classification
Small, fast models for calibrated probability outputs in local environments.
1Compact decision models
Jeff offers small, fast models (0.8B and 2B parameters) fine-tuned from Qwen3.5 and Gemma 4 for zero-shot classification. These models provide calibrated probability outputs for decision-making tasks, slotting easily into local code. They operate with the same request format as Jev, making them compatible with existing workflows. The models are optimized for speed, with decision times as low as 22 ms on an RTX PRO 6000 and 28 ms on an Apple M4 Max.
2Zero-shot flexibility
Jeff's models excel in zero-shot scenarios, where options don't need to appear in the training data. Users describe situations and options in plain words, and Jeff returns probabilities for each choice. This makes it versatile for applications like support queue routing, user intent classification, moderation labeling, and game move selection. The system supports three question types: choice (pick one option), noul (yes/no probability), and score (scale point).
3Performance benchmarks
In benchmarks, Jeff approaches or sometimes surpasses Jev's performance, particularly in classification and grounding tasks. However, its smaller size limits reasoning capabilities compared to larger models like Jev. On reasoning-heavy benchmarks (BBH, JudgeBench, JevBench), Jeff performs below larger models. The 0.8B model shows a sweet spot for fast option picking, while the 2B model is more cautious but scores higher on some benchmarks.
4Training and fine-tuning
Jeff models are trained entirely on local hardware, using synthetic data generated by open models. Training takes about 2 hours for the 0.8B model and 3.5 hours for the 2B model on an RTX PRO 6000. Fine-tuning can significantly improve accuracy; a voice-navigation fine-tune boosted held-out accuracy from 31.7% to 95.8% in under half an hour. The training pipeline includes synthetic data generation, leak filtering, and learning-rate sweeps.
5Game testing
Jeff was tested on three games (Doom, Frogger, Pac-Man) to evaluate zero-shot performance. The models describe game situations and legal moves in words, then pick an action. Jeff-0.8B decides in 29-49 ms per move on an M4 Max, compared to Jev's 212 ms per API call. Game performance doesn't always correlate with benchmark scores; the untrained Gemma 4 E2B scored well on benchmarks but performed worst in games.
6Practical considerations
Jeff is designed for fast, calibrated choices between described options, not multi-step reasoning. Wording matters significantly; consistent option descriptions improve performance. Short option keys and descriptive text are recommended. Independent questions should be asked together in one request. The system is English-only and text-only, with no multi-language or voice support.
7Technical implementation
Jeff builds on AutoJev's core design: one forward pass per decision, trained answer readout, and fitted temperature for calibration. It adds small student models, a local synthetic-data pipeline, prompt layouts for domain fine-tunes, and MLX serving for Apple silicon. The code is MIT-licensed, model weights are Apache 2.0, and training data sources maintain their original licenses. The project is independent of TypeSafe, Jev's maker.
8Use cases and limitations
Jeff excels in applications requiring fast, local decision-making between clearly described options. It's particularly useful for routing systems, intent classification, and simple game AI. Limitations include no multi-step reasoning, English-only support, and performance below larger models on complex tasks. The 0.8B model offers the best balance of speed and performance for most use cases, while the 2B model provides slightly better accuracy at a speed cost.
Original article: Hacker News — Jeff: Compact Decision Models for Local Zero-Shot Classification