The Google Speech team is now hard at work bringing Live Captions from mobile to Chrome, based on a recent string of commits spotted in the Chromium Gerrit code repository. The commit itself points to the creation of a Speech On-Device API (SODA) Service.
Now, that doesn’t explicitly suggest a Live Captions feature is incoming but the comments associated with the commit do. Comments say that Live Captions is just one feature that could stem from the service’s implementation. But Live Captions is also referenced in other commits surrounding the SODA Service and in bug reports tied to the same.
The implication of the addition of SODA is that it will be used to build out Live Captions found on mobile devices. At least one comment more directly indicates it will be called Live Captions in the browser, for parity with mobile. But, in this case, the team will be enabling the feature for Chromebooks, as well as on computers running Mac, Linux, or Windows.
The Google team expects SODA to be used for other voice and speech recognition features in the future as well. But it hasn’t detailed exactly what those features might be.
What exactly is this feature?
Live Captions has been well-reported but is a feature that’s currently only found on a few mobile devices. It’s predominantly found on Pixel-branded handsets. And as of Samsung’s Unpacked 2020 event, it’s only shipping on one non-Pixel gadget out of the box. That’s the Samsung Galaxy S20 series.
Summarily, Live Captions does exactly what its branding implies. It applies closed captions to just about any video or audio content. Those are AI-derived via three separate algorithms, on-device. The algorithms allow more accurate speech recognition even in noisy environments and even when speech isn’t clear. The captions are presented in near-real-time, just like standard captions. But they can be applied even in apps where captions aren’t typically available.
The benefit of all of that is that Live Captions effectively act as a system-level captioning system. That’s as opposed to being an app-level implementation.
The new SODA Service lays out a crucial framework for launching those types of experiences in Chrome’s renderer process. So it will be a part of Chrome itself and work outside of the Android ecosystem.
The timeline for Chrome Live Captions is not set
The commits in the Chromium Gerrit and associated comments show that the company is serious about bringing a Live Captions-like feature to Chrome. But it isn’t necessarily coming soon. In fact, a Chromium Bugs entry places the sandboxed SODA feature with a priority 3 rating. That’s the least urgent priority that can be assigned.
More telling, there don’t appear to be any experimental ways to turn the feature on in any of the Chrome channels.
That doesn’t necessarily mean this isn’t being worked on but it absolutely is not important for Google that it gets done quickly. The team behind the feature has marked its status as “Started” and open. So, although it won’t arrive in the near future, it shouldn’t be too far off. Some form of release to the test channels of Chrome could be expected within the next few updates.