Google is working on a new deep learning algorithm which could help to take still shots and seamlessly weave them together to create a single piece of fluid animation. Google calls this new algorithm DeepStereo, and what it does is take still shots which have been captured by Google’s Street View service and stitches them together to create an animation which looks like a video. While it might be hard for some to envision what good this would do Google or anyone else, it’s already being looked at as having numerous applications from film to image stabilization, and with virtual reality technology on the rise it could have some use there too.
Where still images provide views from every possible angle if taken, they do so without presenting everything as one seamless motion. DeepStereo is able to analyze these images with different angles and look at multiple elements like the colors and depth in the images, and then compare them against each pixel in the image to get the final result like seen in the image below. According to the report DeepStereo currently does this by connecting the images of five different vantage points, but this limited number also sometimes causes the animations to show as partially distorted because the algorithm isn’t able to fit things together the way that it should. This is just a minor setback though, as a way around this would be to add more vantage points, which the developers on the project state is the plan once they’re able to do so as things are still a work in progress.
In all of Google’s examples of the imagery created using DeepStereo, they show you which frames (single frame shots) were used to get the fluid animation by displaying them in a small box in the top right corner. As the animation takes us through the entire collection of vantage points, you can see the frames which correspond to those points. The fact that things are as far along as they are could mean that it won’t be too far in the future before we start to see this technology applied to various things. Your future VR experiences or the future movies you see could be built with the help of DeepStereo.