All Axis Content: Light-Fields Are The Future Of Sports VR
Computer technology continues to influence interactive entertainment, and no-more so than in sports broadcast. As 360 cameras make their way onto fields and into stadia, audiences are becoming more deeply immersed in the action. Despite these innovations, sports content lacks the interactivity that virtual reality can provide in video games. This is mainly due to sports content being firmly rooted in 2D video. Today’s 360 cameras stitch together what is essentially a panoramic view of a scene, limiting the viewer to the camera’s perspective to three degrees of motion. This greatly reduces the interactivity of the experience when compared to the polygon based rendering used in video games that provide six degrees of motion. Polygon rendering lacks the realism of video, and does not recreate live scenes, making it a poor choice for sports broadcast. Light-field technology promises to combine the interactivity of VR games with the realism of HD video to create deeper and more engaging fan experiences.
To improve the experience for video and live sports, viewers must have six degrees of freedom of movement. To solve the problem of 6DOF (degrees of freedom) video, companies are looking to volumetric video and light-field technology for solutions. Unlike 360 cameras, which use an array of cameras in combination with software to stitch together a scene, light-field cameras can absorb new information that allow computers to calculate depth. These computer-intensive depth calculations recreate a real-world environment that allow viewers to walk through an entire scene, rather than watching it from a fixed vantage point. For sports entertainment, this means a fan can walk across a field during live action, instead of being forced to watch from a single perspective.
Sounds amazing, but there are many technical challenges to recording, transmitting and viewing light-fields:
Data Collection: After collecting light-field data, the information needs to be parsed and reformatted to be useful for VR and AR applications. This requires an enormous amount of computing power to calculate, much more than consumer and mobile devices can provide.
Content Delivery: Once the light-field has been calculated, it must be sent to a compatible device to be useful. This presents compression challenges for light-fields and the accompanying video data.
Display Technology: Displaying light-fields is not as simple as displaying 2D video in VR. New technologies need to be developed to help solve the problem of near-object eye focus.
Lack of Consensus: There are many companies developing light-field cameras, but there is little agreement as to what a light-field means. Many companies claim to have light-field technology that are not analogous to each other. In addition, the architecture for an integrated pipeline from content creation to delivery has yet to be established.
Several companies are developing solutions to these unique problems, here are a few we are watching closely:
Visby - Working on pure light-field technology that can offer true 6DOF for video (six degrees of freedom).
Lytro - Using light-fields to enable new kinds of video editing for photographers and cinema.
Avegant - Producing technology to display light-fields that can be licenced to HMD producers.
Varjo - Bringing human-eye resolution to HMDs.
Apple - Their AR development kit, ARkit, coupled with their depth sensing iPhone camera, provides app developers with innovative solutions for light-field video recording, cloud based processing, and delivery.
Intel & HypeVR - Pushing light-fields into pro-sports arenas and delivering fully immersive 6DOF video experiences.
Nvidia - Developing HMDs that utilize two LCD screens per eye to provide accurate light information to eyes.
Otoy - Providing software and services to edit and deliver VR video, including light-fields.
Takeaway:
Any time professional sports content is captured and recorded, rights ownership and legal issues emerge that can slow down innovation. To test this new technology without getting mired in lengthy rights negotiations, companies should seek partners for non-league or off-season sporting events. Sports leagues that have shown a willingness to experiment, such as UFC and America’s Cup, are ripe for tech companies like Qualcomm, IBM, and Amazon to test new light-field VR devices and services for live sporting events without the friction of rights deals.