Qualcomm has been talking about their deep learning machine intelligence platform, Zeroth, for a number of months now. When the company first started talking about including the technology into System-on-Chips, this might have meant that the Snapdragon 820 was going to include a Qualcomm Neural Processing Unit, or NPU, as part of the hardware. Instead, Qualcomm included the Zeroth software on the device rather than a core. Over the weekend, Qualcomm has announced that they are releasing the Zeroth SDK so that developers can start to utilize this specialist software on the device rather than relying on cloud computing to process this data.
Neural processing has been used for a number of years now. The technology is based around the premise of rather than programming a computer to do something, we teach it. Neural processing relies on the computer giving variables a weighting, or importance, and using the output of this to form a decision. Computers can put a situation into context and take a decision based on this. As an example, if you hear and see a car alarm going off, how do you determine what action to take? Firstly, is it your car alarm? Secondly, what time is it? Your reaction might be difficult if it is two in the morning rather than three in the afternoon. Did you hear the sound of breaking glass? Were there kids kicking a soccer ball immediately before the car alarm? Are the neighbors loading or unloading a car? Deep learning technology builds in a much higher capacity to consider factors and apply weights to each of these. Qualcomm currently uses the Zeroth deep learning technology to help determine if a device’s security has been breached using Qualcomm Smart Protect.
Most current examples of deep learning technology involve information collected by the local device, sent to the cloud where it is processed and a decision reached, then the information is sent back to the device. This includes applications such as Google Photos, where the search engine uses deep learning – and as regular users of Google Photos will almost certainly understand, the system is not yet perfect! For understanding the contents of existing photographs there is no problem if the decision takes a second or two. When it comes to taking a picture and using a deep learning process in order to give the camera the best set up, this sort of delay is much too much in a world where our cameras feature laser auto-focus systems. A second or two of delay is considered the best case but network conditions could mean it takes a lot longer. This is why Qualcomm leverage the Snapdragon 820’s Zeroth technology for camera scene selection purposes, as it’s very quick compared with cloud computing. Another good case study of deep learning is driverless car technology: driving a vehicle requires a great many factors being taken into account plus putting things into context. However, deep learning has traditionally required considerable computing power behind the decision-making process and self-driving cars contain many other specialist technologies designed to help this process.
It’s these two case studies where Qualcomm is either planning or already adding value with the Zeroth software. Qualcomm’s Snapdragon 820 features an intelligent way to set up the camera and to process the images as they are taken. Furthermore, the Qualcomm Snapdragon 820A – the automobile optimized version of the chipset – is designed for in-car use. Driverless cars may not use the Qualcomm Snapdragon 820A as their control chip but may be used for certain parts of vehicle operation such as pedestrian detection and avoidance, adaptive cruise control and spotting road signs. And this is the technology that the Zeroth software development kit is unlocking for developers.
Qualcomm’s new SDK is already being used by Nauto, which is a startup business developing smart dashboard cameras designed for vehicle fleet operators. Nauto uses its deep learning technology to analyze driving, provide evidence in the event of a collision and can even remind drivers to pay attention and stop fiddling with their ‘phone. Handset manufacturers have been looking at the potential for Zeroth but with their own applications and datasets. Going forward, there is a significant market for deep learning technology. Qualcomm’s SDK will help unlock the market with an off-the-shelf chipset. Currently, Zeroth is only available for the Qualcomm Snapdragon 820 family of processors but over time, it may be added to future chipsets. The Snapdragon 820 could compete with some of the more specialist chipsets available, such as the Movidius Myriad, which is used in Google’s Project Tango devices for spatial awareness. These more specialized chipsets could be more capable but require additional supporting chips for a given device. For our smartphones, deep learning enabled chipsets could be used to enhance how our smartphones and other devices interact with us as it will improve their understanding of human language – without reliance on transferring data to and from the cloud system.