Spatial random access enabled video system with a three-dimensional viewing volume
First Claim
1. A method for displaying an environment from a viewpoint, the method comprising:
- at an input device, receiving user input designating the viewpoint within a viewing volume;
at one or more processors, identifying, from among a plurality of vantages within the viewing volume, a subset of the vantages nearest to the viewpoint comprising at least two of the vantages, each of which has associated video data;
at a data store, retrieving the video data from the subset of the vantages;
at the one or more processors, combining the video data from the subset of the vantages to generate viewpoint video data depicting the environment from the viewpoint; and
at a display device, displaying the viewpoint video data, wherein identifying the subset comprises identifying four vantages of the plurality of vantages that define a tetrahedral shape around the viewpoint.
2 Assignments
0 Petitions
Accused Products
Abstract
An environment may be displayed from a viewpoint. According to one method, volumetric video data may be acquired depicting the environment, for example, using a tiled camera array. A plurality of vantages may be distributed throughout a viewing volume from which the environment is to be viewed. The volumetric video data may be used to generate video data for each vantage, representing the view of the environment from that vantage. User input may be received designating a viewpoint within the viewing volume. From among the plurality of vantages, a subset nearest to the viewpoint may be identified. The video data from the subset may be retrieved and combined to generate viewpoint video data depicting the environment from the viewpoint. The viewpoint video data may be displayed for the viewer to display a view of the environment from the viewpoint selected by the user.
-
Citations
41 Claims
-
1. A method for displaying an environment from a viewpoint, the method comprising:
-
at an input device, receiving user input designating the viewpoint within a viewing volume; at one or more processors, identifying, from among a plurality of vantages within the viewing volume, a subset of the vantages nearest to the viewpoint comprising at least two of the vantages, each of which has associated video data; at a data store, retrieving the video data from the subset of the vantages; at the one or more processors, combining the video data from the subset of the vantages to generate viewpoint video data depicting the environment from the viewpoint; and at a display device, displaying the viewpoint video data, wherein identifying the subset comprises identifying four vantages of the plurality of vantages that define a tetrahedral shape around the viewpoint. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A method for displaying an environment from a viewpoint, the method comprising:
-
at an input device, receiving user input designating the viewpoint within a viewing volume; at one or more processors, identifying, from among a plurality of vantages within the viewing volume, a subset of the vantages nearest to the viewpoint comprising at least two of the vantages, each of which has associated video data; at a data store, retrieving the video data from the subset of the vantages; at the one or more processors, combining the video data from the subset of the vantages to generate viewpoint video data depicting the environment from the viewpoint; and at a display device, displaying the viewpoint video data, wherein the video data for each of the vantages comprises at least a first layer having a first resolution, and a second layer having a second resolution higher than the first resolution and further comprising, at the one or more processors; identifying a shortage of at least one of bandwidth, storage space, and processing power; and in response to identification of the shortage, selecting the first layer; wherein, combining the video data from the subset of the vantages to generate the viewpoint video data comprises combining the first layers of the video data from the subset. - View Dependent Claims (11, 12, 13)
-
-
14. A method for displaying an environment from a viewpoint, the method comprising:
-
at an input device, receiving user input designating the viewpoint within a viewing volume; at one or more processors, identifying, from among a plurality of vantages within the viewing volume, a subset of the vantages nearest to the viewpoint comprising at least two of the vantages, each of which has associated video data; at a data store, retrieving the video data from the subset of the vantages; at the one or more processors, combining the video data from the subset of the vantages to generate viewpoint video data depicting the environment from the viewpoint; and at a display device, displaying the viewpoint video data, wherein; the user input further designates a view direction; the viewpoint video data further depicts the environment from along the view direction; and the method further comprises using the video data from the subset of the vantages to predict from the viewpoint at one of the view direction and an additional view direction different from the view direction.
-
-
15. A method for preparing viewpoint video data for display for a user, the method comprising:
-
with one or more cameras, capturing volumetric video data depicting an environment from within a viewing volume; at one or more processors, distributing a plurality of vantages throughout the viewing volume; and at the one or more processors, using the volumetric video data to generate video data for each of the vantages such that the video data for each vantage comprises a subset of the volumetric video data that depicts the environment as viewed from the vantage, wherein generating the video data for each of the vantages comprises generating, within the video data for each of the vantages, at least a first layer having a first resolution and a second layer having a second resolution higher than the first resolution, and further comprising transmitting the first layer, but not the second layer, of the video data for a subset of the vantages to a client computing device having a display device. - View Dependent Claims (16, 17, 18, 19)
-
-
20. A non-transitory computer-readable medium for displaying an environment from a viewpoint, comprising instructions stored thereon, that when executed by one or more processors, perform the steps of:
-
causing an input device to receive user input designating the viewpoint within a viewing volume; identifying, from among a plurality of vantages within the viewing volume, a subset of the vantages nearest to the viewpoint comprising at least two of the vantages, each of which has associated video data; causing a data store to retrieve the video data from the subset of the vantages; combining the video data from the subset of the vantages to generate viewpoint video data depicting the environment from the viewpoint; and causing a display device to display the viewpoint video data, wherein the video data for each of the vantages comprises at least a first layer having a first resolution, and a second layer having a second resolution higher than the first resolution and further comprising instructions stored thereon, that when executed by one or more processors, perform the steps of; identifying a shortage of at least one of bandwidth, storage space, and processing power; and in response to identification of the shortage, selecting the first layer; wherein, combining the video data from the subset of the vantages to generate the viewpoint video data comprises combining the first layers of the video data from the subset. - View Dependent Claims (21, 22, 23, 24, 25, 26)
-
-
27. A non-transitory computer-readable medium for preparing viewpoint video data for display for a user, comprising instructions stored thereon, that when executed by one or more processors, perform the steps of:
-
causing one or more cameras to capture volumetric video data depicting an environment from within a viewing volume; distributing a plurality of vantages throughout the viewing volume; and using the volumetric video data to generate video data for each of the vantages such that the video data for each vantage comprises a subset of the volumetric video data that depicts the environment as viewed from the vantage, wherein generating the video data for each of the vantages comprises generating, within the video data for each of the vantages, at least a first layer having a first resolution and a second layer having a second resolution higher than the first resolution; wherein the non-transitory computer-readable medium further comprises instructions stored thereon, that when executed by one or more processors, initiate transmission of the first layer, but not the second layer, of the video data for a subset of the vantages to a client computing device having a display device. - View Dependent Claims (28, 29, 30)
-
-
31. A system for displaying an environment from a viewpoint, the system comprising:
-
an input device configured to receive user input designating the viewpoint within a viewing volume; one or more processors, communicatively coupled to the input device configured to identify, from among a plurality of vantages within the viewing volume, a subset of the vantages nearest to the viewpoint comprising at least two of the vantages, each of which has associated video data; a data store, communicatively coupled to the processor, configured to retrieve the video data from the subset of the vantages; and a display device communicatively coupled to the processor; wherein; the one or more processors are further configured to combine the video data from the subset of the vantages to generate viewpoint video data depicting the environment from the viewpoint; and the display device is configured to display the viewpoint video data, wherein the video data for each of the vantages comprises at least a first layer having a first resolution, and a second layer having a second resolution higher than the first resolution and wherein the one or more processors are further configured to; identify a shortage of at least one of bandwidth, storage space, and processing power; and in response to identification of the shortage, select the first layer; wherein, the one or more processors are further configured to combine the video data from the subset of the vantages to generate the viewpoint video data by combining the first layers of the video data from the subset. - View Dependent Claims (32, 33, 34, 35, 36, 37)
-
-
38. A system for preparing viewpoint video data for display for a user, the system comprising:
-
one or more cameras configured to capture volumetric video data depicting an environment from within a viewing volume; and one or more processors, communicatively coupled to the one or more cameras, configured to; distribute a plurality of vantages throughout the viewing volume; and use the volumetric video data to generate video data for each of the vantages such that the video data for each vantage comprises a subset of the volumetric video data that depicts the environment as viewed from the vantage, wherein the one or more processors are further configured to generate the video data for each of the vantages by generating, within the video data for each of the vantages, at least a first layer having a first resolution and a second layer having a second resolution higher than the first resolution; wherein the one or more processors are further configured to initiate transmission of the first layer, but not the second layer, of the video data for a subset of the vantages to a client computing device having a display device. - View Dependent Claims (39, 40, 41)
-
Specification