Intel has installed 38 5K cameras high above the field at NRG Stadium in Houston, bolted onto the building’s metal structure. Pointed downwards, these cameras operate more like sensors, feeding visual data back to a rack of servers elsewhere inside the stadium. Working together, those servers can digitally reconstruct the 3D world of the game, representing real objects using 3D pixels, known as voxels. To reduce the data processing load—around one terabyte for a 15 to 30 second clip—during a game many of the features inside the stadium, including the field itself, are pre-rendered by the servers. Only moving objects like the players need to be added in real time.
When FOX sends a request—perhaps for a quarterback’s view from the pocket during a crucial play, or a linebacker’s view from the other side of the ball—two Intel staffers will take over. The system’s pilot will operate a virtual camera, choosing where and when to position the viewpoint in the 3D reconstruction. The navigator will package the visual feed from the pilot into a clip that can be relayed back to FOX. The whole process will take a couple of minutes, so don’t expect instant replays yet.
This year the system will be used to show fans what players see, but ultimately any viewpoint is possible. “Any player or referee on the field, you can have that view,” Hopper says. The league is unlikely to ever want to let fans see what officials see, because that would only fuel controversies over questionable calls, but Intel 360 Replay could eventually be used to help get decisions right.
