YouTube’s automatic caption generation feature has been in the spotlight lately due to some serious improvements, and the newest improvement gives the service a basic ability to pick up and interpret non-vocal sounds. Right now, the range of sounds is quite limited, with basic things like audience applause, laughter, generic music tags and the like being covered for now. Like any other technology involving machine learning, of course, this will improve over time as Google’s engineers tackle issues like determining whether it’s a doorbell, cell phone, or alarm ringing, and as the machines behind the algorithms pick up more and more information from successfully deciphering videos.
The new ability, while still basic in scale and compatibility, is already going public on some videos, such as the one attached below. In the video below, a pair of dancers earn applause and Howard Stern’s coveted golden buzzer, and if you switch on captions, their music and the audience’s uproarious applause earn the recognition of YouTube’s automatic caption system. The stream of sounds comes right alongside the usual stream of auto-captioned speech. The captioning happens mostly automatically, but during this formative period for the new capability, Google’s engineers will be keeping a close eye on things to look for chances to improve the service. It should go without saying that this ability, for now, is only on a small amount of videos, all of which were already eligible for automatic captioning.
YouTube’s automatic caption system recently hit a huge milestone, with over 1 billion videos captioned. The system’s machine learning and neural networking backend is improving by the day, and this newest addition to its growing repertoire is a telltale sign that Google is buckling down and getting serious about automatic captioning on YouTube. The timing for ramping up YouTube’s features could not be better; the service is in the midst of a crisis of sorts surrounding its advertising, and both direct rival video services like Vimeo and more niche options like Twitch are beginning to gain ground on the internet video giant. For now, YouTube remains the de facto home of user-generated video content on the internet.