HCI and tracking solutions developer uSens took advantage of this year’s Augmented World Expo 2018 (AWE) to take the wraps off of the new uSensAR Hand Tracking SDK for mobile platforms. The SDK, meant to help developers create better AR and VR experiences for Android and iOS, utilizes a smartphone’s camera and machine learning to generate a 3D hand skeleton. That, in turn, is coded into a game or app so that users can interact with the digital world using nothing more than their own hands. To be clear, other SDKs and tools are available that allow for fingertip tracking. Those allow for a similar result but tracking a full hand allows an end user to rely on their entire hand to interact with a VR or AR object or scent. What’s more, more immersive experiences can arguably be created because graphics and elements can be aligned more accurately.
For AWE 2018, uSens launched a beta version of its uSensAR Hand Tracking SDK. That’s currently being tested out and developed further alongside several “major companies,” including at least one unnamed organization boasting more than 270 million users. That’s said to be a social media app provider but the statement doesn’t narrow things down too much so it’s probably best not to speculate. However, it also released a couple of videos explaining the new tools and demonstrated the SDK in action live at the event. Those show how the SDK can be used to create truly immersive, controller-free AR and VR using nothing more than a smartphone. The obvious benefit here is that mobile users won’t need to buy expensive hardware or depend on more traditional means for input. That’s good news, but the technology could subsequently also be used along with headsets for a much more involved experience. Most of the popular headsets available are, after all, camera-enabled and the technology could be widely used in training or enterprise applications rather than just for gaming.
With the launch of this beta test, it’s likely only a matter of time before developers and hardware manufacturers begin making use of the new SDK. In the meantime, the company hasn’t outlined a timeline with regard to when it expects its solution to leave the beta phase of development. As a result, there’s no way to know for sure when that might happen until uSens makes some form of official announcement or the new VR tech starts showing up.