Google has plans to create a system that allows users to interact with and control their smart home devices beyond voice commands using a UI designed for wearables, based on a patent spotted by Android Headlines at the World Intellectual Property Organization (WIPO). The design encompasses a method for controlling those IoT devices with a pair of smart glasses that follows the wearer’s gaze and registers their head motions. Smart home gadgets the user wants to interact with are identified using machine vision and AI that compares the objects within a field of view to those stored in images on-device. From there, a menu is shown with guides for how to interact with selections. If a user looks at a connected lamp the system would inform the user that tilting their head up or down will adjust the brightness of the light.
More intuitive actions are included as well. Looking at a smart television would result in the TV turning on while looking away or moving from the room would pause the media. The method also allows for IoT devices to interact based on location data stemming from the stored images. More images and information about the wearer’s surrounding environment can be added during a calibration process that orients the glasses to the home. For example, music started in one room on a set of smart speakers would be able to follow the user as they move from room to room. Presumably, other forms of media such as a streamed movie or TV show could similarly be moved across televisions, smartphones, or tablets, based on where the user is and what display they’re looking at. Other smart home alerts from security systems and other devices would also be shown on the glasses, with options to move the content to another display or appropriate medium.
Background: The concepts behind Google’s patent are ambitious but seem to mirror the visions shared for AI-empowered technology by nearly every involved company or organization. At last year’s IBC 2017 event, Nokia put forward a concept called ‘Any Vision‘ that it believed could bring television to any surface or display. Through that technology, media would follow users around, pausing the stream wherever necessary and providing access to smartphone-like information and features. Google’s patent takes a similar approach, with a vastly different result because of its dependence on smart glasses and a heads-up display. That adds a level of automation and interaction that wasn’t seen in Nokia’s proposed solution due to its dependence on touch inputs and hand-motion for directing which display or surface the media should move to.
The search giant also has its own pair of smart glasses already on the market with Google Glass. In its current form, the gadget is entirely geared toward business customers in the enterprise sector. Initially, the company had sought to sell the wearable to the general consumer market but pricing and blowback from the inclusion of a camera held those efforts back. That led to a relaunch of the display-enabled glasses as an augment for factory workers and other industry employees who could benefit from real-time information and hands-free IoT connectivity. A new variation on Google Glass has been spotted in recently-approved FCC filings too but the associated documentation designates the new device as ‘Glass Enterprise Edition 2‘. That means that those won’t likely be appearing outside of the market only for enterprise uses.
Impact: The device shown in the patent images is relatively generic in appearance, although it does show some touch-based controls apparently associated with device-specific menus. That seems to imply that the technology will not be based around any current or previously spotted iteration of Google Glass. The design chosen for the patent could also mean that the search giant plans for this UI to be used by multiple manufacturers or that Google hasn’t refined its own hardware design yet. In either case, Google will probably not use that branding at all since it is so closely associated with the enterprise segment now. With that said, the described UI and technology could be groundbreaking for the IoT industry if the company follows through to release a real-world product with those capabilities.
