Video compression with adaptive view-dependent lighting removal
First Claim
1. A method for compressing a video stream for a virtual reality or augmented reality experience, the method comprising:
- at one or more image capture devices, capturing a video stream of a scene;
at a data store, retrieving, from the video stream, base vantage data comprising base vantage color data depicting the scene from a base vantage location, and target vantage data comprising target vantage color data depicting the scene from a target vantage location;
at a processor, reprojecting the base vantage data to the target vantage location to obtain reprojected target vantage data;
at the processor, comparing the reprojected target vantage data with the target vantage data to obtain residual data;
at the processor, compressing the residual data to obtain compressed residual data by removing a subset of the residual data that is of a frequency that does not contribute to perceptual quality compared to a remainder of the residual data; and
in the data store, storing a compressed video stream comprising the base vantage data and the compressed residual data, wherein removing the subset of the residual data comprises applying entropy encoding to the residual data.
2 Assignments
0 Petitions
Accused Products
Abstract
A video stream of a scene for a virtual reality or augmented reality experience may be captured by one or more image capture devices. Data from the video stream may be retrieved, including base vantage data with base vantage color data depicting the scene from a base vantage location, and target vantage data with target vantage color data depicting the scene from a target vantage location. The base vantage data may be reprojected to the target vantage location to obtain reprojected target vantage data. The reprojected target vantage data may be compared with the target vantage data to obtain residual data. The residual data may be compressed by removing a subset of the residual data that is likely to be less viewer-discernable than a remainder of the residual data. A compressed video stream may be stored, including the base vantage data and the compressed residual data.
-
Citations
30 Claims
-
1. A method for compressing a video stream for a virtual reality or augmented reality experience, the method comprising:
-
at one or more image capture devices, capturing a video stream of a scene; at a data store, retrieving, from the video stream, base vantage data comprising base vantage color data depicting the scene from a base vantage location, and target vantage data comprising target vantage color data depicting the scene from a target vantage location; at a processor, reprojecting the base vantage data to the target vantage location to obtain reprojected target vantage data; at the processor, comparing the reprojected target vantage data with the target vantage data to obtain residual data; at the processor, compressing the residual data to obtain compressed residual data by removing a subset of the residual data that is of a frequency that does not contribute to perceptual quality compared to a remainder of the residual data; and in the data store, storing a compressed video stream comprising the base vantage data and the compressed residual data, wherein removing the subset of the residual data comprises applying entropy encoding to the residual data. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A non-transitory computer-readable medium for compressing a video stream for a virtual reality or augmented reality experience, comprising instructions stored thereon, that when executed by a processor, perform the steps of:
-
causing a data store to retrieve, from a video stream of a scene captured by one or more image capture devices, base vantage data comprising base vantage color data depicting the scene from a base vantage location, and target vantage data comprising target vantage color data depicting the scene from a target vantage location; reprojecting the base vantage data to the target vantage location to obtain reprojected target vantage data; comparing the reprojected target vantage data with the target vantage data to obtain residual data; compressing the residual data to obtain compressed residual data by removing a subset of the residual data that is of a frequency that does not contribute to perceptual quality compared to a remainder of the residual data; and causing the data store to store a compressed video stream comprising the base vantage data and the compressed residual data, wherein removing the subset of the residual data comprises applying entropy encoding to the residual data. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. A system for compressing a video stream for a virtual reality or augmented reality experience, the system comprising:
-
one or more image capture devices configured to capture a video stream of a scene; a data store configured to retrieve, from the video stream, base vantage data comprising base vantage color data depicting the scene from a base vantage location, and target vantage data comprising target vantage color data depicting the scene from a target vantage location; and a processor configured to; reproject the base vantage data to the target vantage location to obtain reprojected target vantage data; compare the reprojected target vantage data with the target vantage data to obtain residual data; and compress the residual data to obtain compressed residual data by removing a subset of the residual data that is of a frequency that does not contribute to perceptual quality compared to a remainder of the residual data; wherein the data store is further configured to store a compressed video stream comprising the base vantage data and the compressed residual data, wherein removing the subset of the residual data comprises applying entropy encoding to the residual data. - View Dependent Claims (23, 24, 25, 26, 27, 28, 29, 30)
-
Specification