Encoding and decoding virtual reality video
First Claim
1. A method for decoding a video stream of a scene for a virtual reality or augmented reality experience, the method comprising:
- from a data store, retrieving at least part of the video stream;
at an input device, receiving a first viewer position and/or orientation;
at a processor, partially decoding the video stream to generate a partially-decoded bitstream;
at a graphics processor, further decoding the partially-decoded bitstream to generate viewpoint video of the scene from a first virtual viewpoint corresponding to the first viewer position and/or orientation; and
at a display device, displaying the viewpoint video, wherein a handoff of the partially-decoded bitstream from the processor to the graphics processor is based upon a scanline iteration, the scanline iteration being indicative of a manner in which storage occurs in the graphics processor.
2 Assignments
0 Petitions
Accused Products
Abstract
A virtual reality or augmented reality experience of a scene may be decoded for playback for a viewer through a combination of CPU and GPU processing. A video stream may be retrieved from a data store. A first viewer position and/or orientation may be received from an input device, such as the sensor package on a head-mounted display (HMD). At a processor, the video stream may be partially decoded to generate a partially-decoded bitstream. At a graphics processor, the partially-decoded bitstream may be further decoded to generate viewpoint video of the scene from a first virtual viewpoint corresponding to the first viewer position and/or orientation. The viewpoint video may be displayed on a display device, such as screen of the HMD.
-
Citations
30 Claims
-
1. A method for decoding a video stream of a scene for a virtual reality or augmented reality experience, the method comprising:
-
from a data store, retrieving at least part of the video stream; at an input device, receiving a first viewer position and/or orientation; at a processor, partially decoding the video stream to generate a partially-decoded bitstream; at a graphics processor, further decoding the partially-decoded bitstream to generate viewpoint video of the scene from a first virtual viewpoint corresponding to the first viewer position and/or orientation; and at a display device, displaying the viewpoint video, wherein a handoff of the partially-decoded bitstream from the processor to the graphics processor is based upon a scanline iteration, the scanline iteration being indicative of a manner in which storage occurs in the graphics processor. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method for encoding a volumetric video data of a scene for a virtual reality or augmented reality experience, the method comprising:
-
from a data store, retrieving the volumetric video data; and at a processor, encoding the volumetric video data to generate a video stream configured to facilitate decoding via a decoding method comprising the steps of; at a processor, partially decoding the video stream to generate a partially-decoded bitstream; and at a graphics processor, further decoding the partially-decoded bitstream to generate viewpoint video of the scene from a virtual viewpoint corresponding to a viewer-designated viewer position and/or orientation, wherein a handoff of the partially-decoded bitstream from the processor to the graphics processor is based upon a scanline iteration, the scanline iteration being indicative of a manner in which storage occurs in the graphics processor. - View Dependent Claims (12)
-
-
13. A non-transitory computer-readable medium for decoding a video stream of a scene for a virtual reality or augmented reality experience, comprising instructions stored thereon, that when executed by one or more processors, perform the steps of:
-
causing a data store to retrieve at least part of the video stream; causing an input device to receive a first viewer position and/or orientation; partially decoding the video stream to generate a partially-decoded bitstream; causing a graphics processor to further decode the partially-decoded bitstream to generate viewpoint video of the scene from a first virtual viewpoint corresponding to the first viewer position and/or orientation; and causing a display device to display the viewpoint video, wherein a handoff of the partially-decoded bitstream from the processor to the graphics processor is based upon a scanline iteration, the scanline iteration being indicative of a manner in which storage occurs in the graphics processor. - View Dependent Claims (14, 15, 16, 17, 18, 19)
-
-
20. A non-transitory computer-readable medium for encoding a volumetric video data of a scene for a virtual reality or augmented reality experience, comprising instructions stored thereon, that when executed by one or more processors, perform the steps of:
-
causing a data store to retrieve the volumetric video data; and encoding the volumetric video data to generate a video stream configured to facilitate decoding via a decoding method comprising the steps of; at a processor, partially decoding the video stream to generate a partially-decoded bitstream; and at a graphics processor, further decoding the partially-decoded bitstream to generate viewpoint video of the scene from a virtual viewpoint corresponding to a viewer-designated viewer position and/or orientation, wherein a handoff of the partially-decoded bitstream from the processor to the graphics processor is based upon a scanline iteration, the scanline iteration being indicative of a manner in which storage occurs in the graphics processor. - View Dependent Claims (21)
-
-
22. A system for decoding a video stream of a scene for a virtual reality or augmented reality experience, the system comprising:
-
a data store configured to retrieve at least part of the video stream; an input device configured to receive a first viewer position and/or orientation; a processor configured to partially decode the video stream to generate a partially-decoded bitstream; a graphics processor configured to further decode the partially-decoded bitstream to generate viewpoint video of the scene from a first virtual viewpoint corresponding to the first viewer position and/or orientation; and a display device configured to display the viewpoint video, wherein a handoff of the partially-decoded bitstream from the processor to the graphics processor is based upon a scanline iteration, the scanline iteration being indicative of a manner in which storage occurs in the graphics processor. - View Dependent Claims (23, 24, 25, 26, 27, 28)
-
-
29. A system for encoding a volumetric video data of a scene for a virtual reality or augmented reality experience, the system comprising:
-
a data store configured to retrieve the volumetric video data; and a first processor configured to encode the volumetric video data to generate a video stream configured to facilitate decoding via a decoding method comprising the steps of; at a second processor, partially decoding the video stream to generate a partially-decoded bitstream; and at a graphics processor, further decoding the partially-decoded bitstream to generate viewpoint video of the scene from a virtual viewpoint corresponding to a viewer-designated viewer position and/or orientation, wherein a handoff of the partially-decoded bitstream from the processor to the graphics processor is based upon a scanline iteration, the scanline iteration being indicative of a manner in which storage occurs in the graphics processor. - View Dependent Claims (30)
-
Specification