Training a Cost-Effective NER Model with Gemini's Labels
How $9 of LLM annotations replaced expensive API calls with a local model
1The high-cost NER problem
The author initially used Gemini 3.1 Pro's API for named-entity recognition on Reddit knife discussions, paying per comment. While accurate (identifying brands like Mazaki, models like Fibrox, and steel types like white #2), costs scaled linearly with comment volume. This became unsustainable for continuous scraping of dynamic forums where new content appears constantly. The solution needed to maintain accuracy while eliminating recurring API expenses.
2Zero-shot baseline shortcomings
GLiNER, an open NER model, offered a cost-free alternative but with significant accuracy trade-offs. In zero-shot mode (without training), it achieved only 0.65 F1 score compared to Gemini's outputs. This performance gap represented the challenge: could limited Gemini-labeled data train GLiNER to approach Gemini's accuracy? The 35-point difference made the case for supervised fine-tuning rather than relying on out-of-the-box capabilities.
3Annotation economics
Gemini labeled 4,290 Reddit comments for $9 total ($0.0021 per comment), creating the training dataset. This one-time cost would break even after processing approximately 4,291 future comments - assuming similar comment lengths and using existing GPU resources. The approach transformed variable operational costs (API calls) into fixed capital expenditure (training), favorable for long-term scraping projects with predictable volumes.
4Prompt engineering insights
Critical prompt design avoided asking Gemini for character offsets directly, as LLMs handle positional counting poorly. Instead, the prompt requested exact substrings with labels, letting TypeScript calculate precise offsets. This prevented span misalignments by 2-3 positions. When mentioned strings didn't appear in comments, entities were automatically dropped - a simple but effective validation mechanism against hallucinated outputs.
5Training data composition
The 4,290-comment dataset contained 1,575 positive examples (69.6%) across three entity types: 1,720 brand mentions, 1,345 product models, and 842 material specifications. Notably, 30% were deliberately empty (containing known false-positive triggers like 'gyuto' or 'carbon' but no actual products), teaching the model to avoid over-prediction. This balanced approach improved precision on real-world data where most text contains no entities.
6Debugging the training failures
Five initial training runs failed completely due to a subtle tensor configuration error. The 'words_mask' tensor required word indices (0 for special tokens, then 1,2,3...) but was incorrectly filled with ones. This made the model treat entire comments as single tokens, preventing span detection. The issue escaped normal checks because the loss curve appeared normal while learning nothing - a dangerous failure mode for custom training loops.
7Performance optimization
After fixing configuration, the 459M parameter model achieved 0.83 F1 on unseen comments - a 18-point gain over zero-shot. Key improvements included: per-class confidence thresholds (boosting material recall from 0.787 to 0.911), controlled negative example ratios, and early stopping at epoch 2 to prevent overfitting. The model learned nuanced distinctions like 'carbon steel' being a category rather than a specific steel type.
8Broader implications
This case demonstrates how LLMs can economically bootstrap smaller, specialized models through one-time annotation. The $11.50 total cost ($9 labels + $2.50 GPU) highlights accessible fine-tuning potential, though debugging dominated actual effort. It suggests a paradigm where expensive LLMs generate training data for cheaper, focused models - particularly valuable for domain-specific NER tasks with consistent entity patterns.
Original article: Hacker News: Front Page — Training a Cost-Effective NER Model with Gemini's Labels