# From Cloud LLM to On-Device Whisper: Speech to Structured Actions on Android Processing speech and extracting actions with on-device Whisper instead of cloud solutions. - **Source:** [ProAndroidDev - Medium](https://brifing.si/source/d9553fa9-7b72-4b9b-bc73-c6bf9a40dd5f) - **Published:** 2026-07-23 - **Topic:** [Software & Engineering](https://brifing.si/c/engineering) - **Original article:** - **On Brifing:** https://brifing.si/s/3731e0c4-82c8-4e9e-aa53-ce550dad9dab ## Transition from Cloud to On-Device The article explains the shift from cloud-based LLMs to on-device Whisper for processing voice commands. Privacy, offline capability, and cost advantages motivated this transition. While Firebase with Gemini has limitations, the benefits of local solutions are emphasized. ## Speech Recognition with Whisper.cpp Whisper.cpp, a lightweight version of OpenAI's Whisper model, offers a C/C++ based solution. The 42MB quantized model runs efficiently on mobile devices. Integrated with Kotlin via JNI bridge, it represents true 'on-device AI' implementation. ## Deterministic Parser Design In the cloud version, LLM handled both transcription and structured data extraction. The new system separates these steps: Whisper only transcribes while a Kotlin-based deterministic parser processes the text. This ensures predictability and testability. ## Architecture Decisions and Interface Consistency The system is designed around SpeechToTransactionService interface, facilitating switching between cloud and on-device solutions. Hilt dependency injection isolates UI layer from implementation details. This approach provides architectural flexibility. ## Native Code Integration JNI bridge integrates Whisper.cpp with Kotlin. A dedicated coroutine dispatcher handles thread safety. CMake and Gradle configurations support only arm64-v8a architecture to optimize APK size. ## Performance and Constraints The small model was selected considering quality-performance balance. Thread-unsafe native code requires special management. The rule-based parser's lack of flexibility stands as the main trade-off. ## Security and Privacy Advantages Keeping voice data on-device is crucial for sensitive scenarios like financial transactions. Fully offline processing maximizes user privacy. Long-term costs are also reduced. ## Real-World Application Scenario In the sample app, a voice command like 'Send Maria 10 euros for the taxi' is converted into a structured transaction object. Action type, amount, receiver and currency fields are automatically parsed, significantly improving user experience. ## Learning Experience and Future The author describes this as a learning exercise. With limited NDK/C++ knowledge, AI assistance was utilized. While not a production blueprint, it presents a valuable case study demonstrating on-device AI potential. Future integration of more sophisticated models is envisioned. ## Sources covering this story - https://proandroiddev.com/from-cloud-llm-to-on-device-whisper-turning-speech-into-structured-actions-on-android-19791f383f4e?source=rss----c72404660798---4 — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._