Google DeepMind launches AI that translates sign language to text
Breakthrough technology breaks communication barriers for 70 million Deaf and hard of hearing individuals
1Historic leap for the Deaf community
Google DeepMind introduced SL2T, a model capable of translating over 200 sign languages into text. Designed to meet daily communication needs of 70 million Deaf individuals, the technology initially supports American Sign Language (ASL) to English translation. Integrated into Pixel 11's Gboard and Live Transcribe apps, it enables users to communicate via signing instead of typing. This marks the first time sign language AI has moved from labs to consumer products.
2The innovation behind the technology
SL2T employs a novel approach to decode sign language complexity. It tracks 33 body points, converting movements into geometric coordinates while discarding raw video to preserve privacy. Trained on 100,000+ hours of data across 50+ sign languages, the multilingual design helps the system learn underlying structures. Unlike traditional methods, SL2T skips intermediate 'gloss' steps, directly translating from landmarks to text for more accurate non-linear sign language features.
3Unique challenges of sign languages
Sign language translation presents two core challenges distinct from speech recognition. First, sign languages have unique grammar and vocabulary. Second, the system must simultaneously interpret hand, arm, torso, head, and facial movements. SL2T overcomes these by translating directly from geometric coordinates, better capturing sign language's rich, non-linear nature. This approach eliminates artificial vocabulary limits seen in gloss-based methods.
4User experience enhancements
SL2T scored 70 BLEURT on FLEURS-ASL benchmark, becoming the most capable sign translation model to date. Beyond academic metrics, Google addressed practical issues: ensuring fairness for left-handed signers (10% of users), optimizing one-handed signing while holding phones, and minimizing streaming latency. Testers report ASL feels faster and more natural than typing in English. The model also handles rapid fingerspelling and complex classifier constructions.
5Social impact and inclusivity
Sign languages form the cornerstone of Deaf cultural identity, yet proficiency varies widely among Deaf individuals. SL2T supports this diversity, enhancing communication inclusivity. The technology bridges communication gaps between Deaf and hearing communities. Google collaborated closely with Deaf communities throughout development, ensuring the technology addresses real user needs rather than technological possibilities alone.
6Technical infrastructure and privacy
SL2T's architecture prioritizes user privacy. Using MediaPipe Holistic, it processes geometric body coordinates instead of raw camera feeds. Only these anonymized data points are sent to servers. This approach protects personal data while ensuring consistent performance across devices. The on-device component tracks signers' movements in real-time, enabling responsive translation without compromising visual privacy.
7Future roadmap and expansion
Google plans to expand SL2T to more devices and operating systems. Additional sign languages beyond ASL will roll out in coming months. Ongoing collaboration with Deaf communities aims to broaden the model's coverage and accuracy. Long-term, the technology could enable real-time translation in video calls and live events. This expansion represents Google's commitment to making technology universally accessible.
8A revolutionary step in communication
SL2T exemplifies AI's potential for social good, offering 70 million Deaf individuals full participation in digital life. Google's breakthrough may redefine accessibility standards across tech industries. More than a product, SL2T represents a significant stride toward communication equality. Its success demonstrates how technology, when developed inclusively, can transform lives and bridge societal divides.