Jev-like wrapper for LLMs with vision capabilities
A Python script enabling single-token queries to vision-capable LLMs
1Jev-inspired LLM wrapper
The article introduces a Python wrapper that applies Jev's single-token query approach to vision-capable LLMs. By forcing models to output just one token, it enables rapid decision-making from visual inputs. The implementation works with both local models like Gemma 4 12B and cloud services like OpenAI's GPT-6 Luna. This technique leverages token probabilities to determine the most likely answer from predefined options, avoiding lengthy model responses.
2Vision model adaptation
The wrapper extends Jev's text-only format to handle visual inputs through an attachments field. It captures webcam frames as base64-encoded JPEGs and sends them to vision-capable LLMs. The example implementation answers three questions per frame: person detection, indoor/outdoor classification, and scene brightness assessment. On an RTX 3090 with Gemma 4 12B, it achieves about 1 frame per second, while OpenAI's GPT-6 Luna processes at 0.2 FPS.
3Token probability technique
The system uses logprobs to analyze the model's confidence in different options. By limiting responses to single tokens (letters representing choices), it can precisely measure the model's preference among alternatives. The implementation normalizes these probabilities to produce final scores. This approach works for multiple question types: binary (noul), multiple choice, and ordinal scoring. The wrapper handles API differences between local llama.cpp and OpenAI endpoints.
4Technical implementation
The Python script uses OpenCV for webcam access but delegates vision processing entirely to LLMs. It encodes frames as base64 JPEGs and sends them with Jev-style queries. The implementation includes special handling for different API formats: llama.cpp uses Chat Completions while OpenAI uses Responses. The system maintains a background worker to process frames while continuously displaying the camera feed. Setup requires specific model files and projector weights for local execution.
5Performance considerations
The article notes significant performance differences between local and cloud-based models. Gemma 4 12B on an RTX 3090 achieves 1 FPS with three questions per frame, while GPT-6 Luna processes at 0.2 FPS. The author suggests this difference comes from connection overhead in OpenAI's system. The implementation doesn't use specialized computer vision models, trading efficiency for flexibility in changing conditions through natural language prompts.
6Flexibility advantage
The key benefit of this approach is the ability to modify vision tasks through natural language without retraining models. Users can change detection criteria by simply editing the text prompts. The system can handle various question types including binary detection, classification, and ordinal scoring. This makes it adaptable to different use cases while maintaining a consistent interface. The implementation demonstrates how LLMs can serve as general-purpose vision processors.
7Setup requirements
Local execution requires specific components: NVIDIA drivers, llama.cpp binary, and properly formatted model files. The Gemma 4 12B QAT model needs both main weights (7GB) and a multimodal projector (175MB). The Python environment requires OpenCV for camera access and basic web framework dependencies. For OpenAI usage, only the API key and endpoint URL are needed. The script automatically handles the differences between these execution environments.
8Question format versatility
The wrapper supports three question types through a standardized format. Binary (noul) questions return probability of true/false. Choice questions select among multiple options with confidence scores. Score questions assess ordinal levels (like brightness) with weighted averages. The system validates that each question has 2-20 possible responses. Questions share the same visual context but can have completely different evaluation criteria.
9Practical applications
This technique enables rapid prototyping of vision applications without specialized ML expertise. Potential uses include environmental monitoring, accessibility tools, and simple surveillance systems. The natural language interface allows non-technical users to modify detection criteria. While less efficient than dedicated vision models, this approach offers unparalleled flexibility for experimental or rapidly changing requirements. It demonstrates how LLMs can democratize computer vision applications.
Original article: Hacker News: Front Page — Jev-like wrapper for LLMs with vision capabilities