Google has announced in a blog post published on Thursday that it developed a new algorithm meant to address the longstanding issue with Street View: image overlaps and misalignments. Google Street View is designed to produce street-level views of different parts of the world through the creation of panoramas captured using a multi-camera rig called a rosette and stitched into a 360-degree image with Google’s image blending software. However, Google’s Mike Krainin, a software engineer, and Ce Liu, research scientist for machine perception, explained in the blog article that an issue can arise out of creating these panoramas due to several factors such as wrong calibration of rosette’s camera geometry, parallax and inconsistency in the timing between adjacent cameras. These factors can interfere in the creation of seamless panoramas.
With the new algorithm for Street View, Google aims to eliminate visual flaws and make images align within areas of the overlap. The approach uses two steps in order to create seamless panoramas in a less complex manner and meet computational requirements. The first step is called optical flow, meant to align the pixels for each pair of overlapping photos by computing the optical flow between the two images. To ensure a near perfect result, the Google Street View team also removed correspondences once visual structures became insufficient. The second step is global optimization, and it is designed to adjust the corresponding pixel locations from the areas where overlapping occurs by distorting the rosette’s images. This results in the proper alignment of images once warping is performed, though Google admits to the challenge of making these adjustments due to the fact that the overlap regions are so small that the room for errors become extremely limited.
Krainin and Liu said Google added the new algorithm to the Street View stitching pipeline so that existing panoramas in the application are now being re-blended to fix issues with old Street View images. The Mountain View, California-based internet giant also recently trained an artificial intelligence (AI) technology to find beautiful panoramas from Google Maps, blend the images together and perform post-processing techniques on the panoramas to try to achieve a professional-looking result. More upgrades to Street View are likely to follow in the near future as Google continues to advance its AI footprint.
