Google is now working on a new application called Lookout that is intended to aid users with visual impairments by utilizing a device’s camera and A.I. Of course, the application isn’t quite ready yet but the company has provided a substantial number of details about it prior to its launch. It works by using machine learning and the onboard camera to both examine objects and people in the surrounding environment and to gauge the contextual relevance of those. Feedback is provided to the user about that environment via audio. Although some things, such as restrooms or exits are more likely to be pointed out across more situations, there are also several different modes planned.
Those different modes will make it possible to provide additional context for the A.I. to work with. For example, Work and Play mode is designed to help in office spaces, shopping, or new environments. Home mode centers around the user’s household, with contexts provided which focus on cooking, chores, and other aspects of home life. There is also a text-specific mode called Scan and an Experimental mode for new features. The A.I. interacts with the user by reading text aloud or pointing out specific objects within the confines of what’s relevant based on those modes. Presumably, at some point, Lookout will also be able to utilize other audio output methods. So it should prove useful in a huge variety of circumstances. The addition of other output options would also mean it can also be used more privately if desired.
The move is hardly surprising given both the search giant’s recent push to deliver new accessibility options and its drive to deliver on the A.I. front. Those two categories of technology have in fact been a huge part of this year’s Google I/O 2018 event, as well. For now, Google hasn’t provided details about when, exactly, Lookout will become available to users. However, it is expected to arrive in the Play Store before the end of the year. In the meantime, the company has also shared a video primer to show what the app is capable of right now.
