LiteRT: Custom Machine Learning Models on Android
Run your custom models on Android devices with Google's LiteRT framework.
1What is LiteRT?
LiteRT is a runtime framework developed by Google for running machine learning and generative AI models on Android, iOS, Web, and desktop platforms. It powers MediaPipe Tasks and manages low-level model execution. LiteRT offers two main runtime interfaces: the CompiledModel API, which optimizes hardware acceleration for new applications, and the Interpreter API, maintained for backward compatibility.
2Which Models Can It Run?
LiteRT can execute both classical machine learning models (.tflite) and modern generative AI models. For generative AI, it supports open-weight large language models like Gemma, Llama, Phi, Qwen, and SmoLM. For classical ML, it natively runs architectures for vision (EfficientNet, MobileNet, ResNet), natural language (BERT, MobileBERT), and audio (YAMNet).
3Converting Custom Models
Custom models built in PyTorch, TensorFlow, or JAX can be converted into .tflite flatbuffers for deployment on LiteRT. This conversion process enables models to run on Android devices. However, model execution is limited by the physical capabilities of the target device.
4Memory Constraints
LiteRT demands significant memory requirements to run large-parameter models. For example, running a 3B or 7B parameter LLM requires several gigabytes of RAM. This makes it impractical on budget smartphones with less than 4GB-6GB of system memory.
5Hardware Acceleration
Modern generative AI models require dedicated NPU (Neural Processing Unit) or high-performance GPU acceleration to achieve acceptable tokens-per-second speeds. Running these models on a standard mobile CPU results in severe latency.
6Battery Consumption
High-capacity model files, even when compressed via INT4/INT8 quantization, demand significant disk space and cause rapid battery drain due to continuous high-power compute cycles during inference.
7Hands-on Implementation Samples
Every LiteRT workflow follows a predictable three-step pattern: model loading, tensor buffer preparation, and inference execution. This process demonstrates how custom models can be run on Android devices.
8Image Classification Example
The LiteRTImageClassifier class uses an 8-bit quantized model to classify a bitmap into ImageNet categories. Quantization parameters are extracted directly from embedded metadata, keeping the quantization pipeline robust against hardcoded configuration drift.
9Text Classification Example
Text models require manual tokenization before tensor ingestion. The BertClassifier example demonstrates complete buffer lifecycle management, input tensor shaping, and logit probability normalization.
10Conclusion and Importance
LiteRT provides a robust solution for running custom machine learning models on Android devices. However, the physical capabilities of the device can limit the successful deployment and execution of models. This requires developers to optimize their models and consider the capacities of target devices.