Jeff: Compact Decision Models for Local Zero-Shot Classification
Small, fast models compatible with Jev format for instant decision-making
1Specialized decision models
Jeff offers fine-tuned versions of Qwen3.5 and Gemma4 models specifically optimized for zero-shot classification tasks. These compact models (0.8B-2B parameters) provide calibrated probabilities for decision options in 22-28ms per call, matching Jev's request format without requiring API calls. The system accepts natural language descriptions of situations and options, returning probabilities for each possible choice in a single forward pass.
2Local hardware efficiency
All training was done on local hardware, including RTX PRO 6000 workstation GPUs and DGX Sparks, with testing on MacBooks. The 0.8B model trains in about 2 hours, while the 2B version takes approximately 3.5 hours. This local-first approach eliminates cloud GPU dependencies and maintains complete control over the training pipeline, using only open models for synthetic data generation and quality checks.
3Zero-shot flexibility
Jeff's models handle arbitrary categories not present in training data, making them suitable for diverse applications like support queue routing, user intent classification, content moderation, and game decision-making. The system achieved 31.7% accuracy in initial voice navigation tests, which improved to 95.8% after a 30-minute fine-tuning session on domain-specific examples using just one GPU.
4Performance benchmarks
In tests across 4,599 questions from five public benchmarks plus JevBench's hard tier, Jeff models matched or exceeded Jev's performance in classification and grounding tasks. However, they lag behind in reasoning-heavy benchmarks like BBH and JudgeBench, as expected given their smaller size. The 0.8B model proved particularly effective for fast option selection despite its compact architecture.
5Game testing results
Jeff's decision-making capability was tested through three games (Doom, Frogger, Pac-Man) where it processed game states described in natural language. The 0.8B model achieved 29-49ms decision times on an M4 Max, significantly faster than Jev's published 212ms API response time. Performance varied by game, with the model demonstrating particular strength in Pac-Man after fine-tuning.
6Training methodology
The training pipeline uses synthetic data generated by open models (Qwen3.8-Flash-Next) with rigorous leak filtering. Models undergo full-weight fine-tuning for one epoch with 256-sized batches, using cross-entropy loss over option letters followed by temperature calibration. Checkpoints are selected based on development set performance, never on benchmark data.
7Practical implementation
Jeff works best when options clearly state their consequences rather than abstract descriptions. Short option keys with descriptive text improve performance, while long IDs degrade it without benefit. The system handles multiple independent questions in single requests efficiently. For domains where zero-shot performance is insufficient, the framework supports rapid fine-tuning with dramatic accuracy improvements.
8Open ecosystem approach
As an independent project building on AutoJev's open recipe, Jeff maintains MIT licensing for code and Apache 2.0 for model weights. The training data combines public sources with synthetic generation, with each component retaining its original license. This approach enables local, transparent AI decision-making without reliance on proprietary cloud services or closed models.