Full offline voice agent in 1.2GB RAM: Executes commands instantly
Your phone can now respond to voice commands in 0.9 seconds without internet
1On-Device Voice Technology
The Speech-Android project developed a fully offline voice assistant. Voice detection with Silero VAD, speech-to-text with Parakeet STT, command understanding with FunctionGemma, and voice response with Pocket TTS - all processes occur on-device. Network is only needed for initial model downloads.
2Impressive Performance
Tests on Galaxy S23 Ultra show the system transcribes speech in 217ms, understands commands in 294ms, and starts generating voice response in 179ms after speech ends. Total latency is just 908ms. The entire system uses 1.1GB memory.
3Technical Architecture
Built with Kotlin SDK, the system works through a 250-line JNI bridge to a C++17 pipeline engine. Uses ONNX Runtime for speech models and LiteRT-LM for command routing. Available as open source under Apache-2.0 license.
4Smart Command Management
The system only exposes tools valid for current device state, enabling reliable command routing even with a small 270M parameter model. This approach boosts performance while optimizing memory usage.
5User Experience
Currently running as a foreground service holding continuous audio focus. The developer is considering implementing transient audio focus per turn, allowing media apps to duck rather than pause during voice interactions.
6Future of Mobile AI
This project proves powerful AI models can run efficiently on mobile devices. Demonstrates the possibility of low-latency voice assistants without cloud dependency, crucial for privacy-focused users. Demo video and source code available on GitHub.