Live captions make it easier for people with disabilities to understand videos and audio. They show you what is being said on your phone’s screen. Over time, the feature has evolved beyond accessibility, and now helps anyone who has trouble hearing sound in noisy environments or other situations.
To further improve on what exists, Google has introduced Expressive Captions on Android. The new feature adds emotional context to help you understand not just what is being said, but also how it is being said.
Now you can feel captions and not just read them
The recently announced Expressive Captions feature uses artificial intelligence to capture tone, volume, background sounds, and human noises, such as sighs or gasps. It will also capitalize words to show intensity. For instance, if someone excitedly shouts a Merry Christmas in a movie you’re watching, the caption could appear as “MERRY CHRISTMAS!!!”.
To build Expressive Captions, our Android and Google DeepMind teams worked to understand how we engage with content on our devices without sound. Using multiple AI models, Expressive Captions not only captures spoken words but also translates them into stylized captions, while providing labels for an even wider range of background sounds. This makes captions just as vibrant as listening to audio. It’s just one way we’re building for the real lived experiences of people with disabilities and using AI to build for everyone.
Here, the capitalization emphasizes the excitement, and you might also see captions like [cheering] or [laughter] to reflect the festive atmosphere. These subtle vocal cues add emotion so that your experience isn’t just bland. Afterall, a sigh can indicate frustration or boredom, while a grunt can mean discomfort or effort. So it’s not enough that someone simply says a word without you understanding the deeper meanings behind them.
Expressive Captions will be available on Android devices running Android 14 and above, with support for English in the USA. It’s possible a wider rollout will follow in the coming days. The feature is integrated directly into your phone’s system through Live Caption, so it’ll work across most apps from your Instagram reels, video messages from friends and family, and livestreams.
Communicate with more emotion and privacy
What’s most impressive about Expressive Captions is how it goes beyond just traditional media. Now, you can enhance real-life interactions with loved ones. So it’s not only for passive content, but also for active communication. Conversations can feel more natural and add human-like quality to captions wherever hearing is difficult.
To turn on the feature, press either of the volume buttons on the side of your phone. Then tap the Live Caption icon under the volume controls. Next, tap the Expressive Captions icon under Live Captions and tap Done to confirm your selection. The new icon has a star that’s hard to miss.
The captions appear in real time, and since they are processed on the device itself, they work even when you’re offline or when your phone is in airplane mode. More importantly, data on the content you’re viewing doesn’t get uploaded to some shady remote server. Or at least, that’s what Google implies. You don’t want your private conversations or sensitive data being monitored.
