Technologically pioneering internet firm, Google, has been hard at work, endeavoring to break new ground in the 3D video streaming field. Extended Reality (XR) has been a key area of focus for a few years now, but the firm is making headway. With YouTube acting as a showcase platform for Google’,s 3D streaming capabilities. Mass 3D video streaming may soon be a standard part of media consumption.
The Internet’s most famous ambassador, Google, has been hard at work pioneering pushes into new kinds of technology. Owing to the Corona Virus induced reality that humanity is still coming to terms with, communal events where we gather in our masses to generate a lot of noise while interacting, learning, and being entertained have gone digital. Google’s 3D offering seems geared to such a reality.
The internet’s go-to video content sharing platform, YouTube, first enabled 3D video streaming back in 2015. This time however, content creators will be able virtually put the audience in the action (imagine streaming 360 Degree views of a concert, or sporting event) complete with spatial audio, to enhance the viewing experience.
“…We’re also launching spatial audio for on-demand YouTube videos. Just as watching a concert in 360 degrees can give you an unmatched immersive experience, spatial audio allows you to listen along as you do in real life, where depth, distance and intensity all play a role.” – YouTube statement, 2016
It wasn’t until June of this year, however, that Google announced a method to bring live 360 degree video streaming to their over – 1 billion strong – user base. The firm revealed that it’s research team had figured out a new method of capturing and compressing the 3D footage down to a lighter format, making it possible to render the video on standalone VR/AR devices.
Volumetric video, as the technology is called, enables viewers to move their heads around naturally to get a full view of their virtual environment. This aspect surpasses current VR display tech, which allows for a limited, at times awkward, amount of head movement.
To achieve this, Google’s Research and Development squad fitted 46 time-synced cameras to a dome, 92cm in diameter. This delivers, to the viewer, an 80cm area of natural head movement in a 220+ degree field of vision, 10 pixels per degree of angular resolution, as well as 30 frames per second video capturing capability. All the better to virtually put you right in the action with, my dear.
Researchers claim their volumetric video system can reconstruct objects that are just 20cm away from the camera. This is achieved via an interpolation algorithm planted in the firm’s deep learning software, DeepView.
The team have switched the standard multiplane image scene representation with a set of sphere shaped shells, apparently they work better for capturing panoramic light field footage.
“We further process this data to reduce the large number of shell layers to a small, fixed number of RGBA+depth layers without significant loss in visual quality. The resulting RGB, alpha, and depth channels in these layers are then compressed using conventional texture atlasing and video compression techniques. The final, compressed representation is lightweight and can be rendered on mobile VR/AR platforms or in a web browser,” – Google’s research people.
It’s immediately apparent that Google may have, not only, improved 360 degree video capturing capability, but made it easier and cheaper for folks to produce and consume said content. It is still unclear how Google plans to deploy this discovery, but we may soon see it become a feature on YouTube. It would really make using Google maps a blast.