TRANSMISSION OF THREE-DIMENSIONAL VIDEO

• US 20160088282A1
• Filed: 09/16/2015
• Published: 03/24/2016
• Est. Priority Date: 09/22/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Sadi, Sajid, Perdices-Gonzalez, Sergio, Budhiraja, Rahul, Lee, Brian Dongwoo, Khwaja, Ayesha Mudassir, Mistry, Pranav
• Granted Patent: US 10,547,825 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G03B 35/08 by simultaneous recording
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/265 Mixing

Abstract

In one embodiment, a method includes receiving, from a client device, a request corresponding to a particular view of a three-dimensional video comprising multiple views. The method also includes accessing a data stream corresponding to the video. The method further includes selecting, based on the request, one or more portions of the accessed data stream, at least one of the selected portions corresponding to the particular view. The method also includes transmitting the selected portion of the accessed data stream to the client device.

Claims

1. A method comprising:

receiving, from a client device, a request corresponding to a particular view of a three-dimensional video comprising a plurality of views;

accessing a data stream corresponding to the video;

selecting, based on the request, one or more portions of the accessed data stream, at least one of the selected portions corresponding to the particular view; and

transmitting the selected portion of the accessed data stream to the client device.

2. The method of claim 1, wherein the received request comprises instructions for transmitting one or more of the portions of the accessed data stream at a specified bit rate or omitting one or more portions of the accessed data stream.

3. The method of claim 1, wherein the received request comprises instructions for transforming the accessed data stream.

4. The method of claim 3, wherein the transformation comprises transcoding or resizing the accessed data stream.

5. The method of claim 3, further comprising storing the results of a transformed data stream.

6. The method of claim 1, further comprising:

decoding the accessed data stream; and

extracting each portion of the selected data stream from the decoded data stream.

7. The method of claim 6, wherein:

the accessed data stream comprises a plurality of data streams; and

each of the plurality of data streams comprises a particular one of the extracted portions.

8. The method of claim 1, wherein the video comprises a video stream with multiple resolutions, a video stream from a single camera, a manipulated video stream, or a video stream from a top camera.

9. The method of claim 1, wherein the accessed data stream comprises a 360°

stereoscopic three-dimensional (3-D) video.

10. The method of claim 1, wherein the accessed data stream comprises one or more audio streams that are time synchronized to the video.

11. The method of claim 1, wherein the accessed data stream comprises data to reconstruct the video on the client device.

12. The method of claim 11, wherein the reconstruction data comprises spatial filtering data, distortion data, correction information, or projection parameters of the video.

13. The method of claim 11, wherein the reconstruction data comprises audio spatial reconstruction data.

14. The method of claim 1, wherein the selected portion corresponds to a high-resolution video corresponding to the particular view.

15. The method of claim 14, wherein the selected portion further comprises a portion of the accessed data stream corresponding to a low-resolution video of one or more areas outside of the particular view.

16. The method of claim 1, wherein a resolution of the video increases proportionally to an amount of time the particular view is stationary.

17. The method of claim 1, further comprising

inferring a change in the particular view;

and

selecting, based on the inferred change, one or more portions of the accessed data stream corresponding to the inferred change.

18. The method of claim 1, wherein the transmitting comprises adjusting a bit rate based on a detected bandwidth of a connection with the client device.

19. A method comprising:

transmitting, from a client computing device to a server computing device, a request for a particular view of a three-dimensional video comprising a plurality of views;

receiving, from the server computing device, a data stream comprising;

a plurality of images corresponding to the particular view of the three-dimensional video; and

instructions for combining the plurality of images to create particular view of the three-dimensional video;

combining, by the client computing device, the plurality of images according to the instructions; and

presenting, by the client computing device, the particular view of the three-dimensional video.

20. The method of claim 19, wherein:

the data stream further comprises;

a plurality of images corresponding to at least one other view of the three-dimensional video; and

instructions for combining the plurality of images corresponding to the other view to create the other view of the three-dimensional video; and

combining, by the client computing device, the plurality of images comprises;

determining the particular plurality of images corresponding to the particular view of the three-dimensional video; and

combining, according to the instructions for the particular plurality of images, the particular plurality of images.

21. The method of claim 20, wherein:

the particular view of the three-dimensional video is a high-resolution video; and

the at least one other view of the three-dimensional video is a low-resolution video of one or more areas outside of the particular view.

22. The method of claim 19, wherein the request comprises instructions for transmitting one or more of the portions of the three-dimensional video at a specified bit rate.

23. One or more non-transitory computer-readable storage media embodying instructions that are operable when executed to:

receive, from a client device, a request corresponding to a particular view of a three-dimensional video comprising a plurality of views;

access a data stream corresponding to the video;

select, based on the request, one or more portions of the accessed data stream, at least one of the selected portions corresponding to the particular view; and

transmit the selected portion of the accessed data stream to the client device.

24. The media of claim 23, wherein the received request comprises instructions for transmitting one or more of the portions of the accessed data stream at a specified bit rate or omitting one or more portions of the accessed data stream.

25. The media of claim 23, wherein the received request comprises instructions for transforming the accessed data stream.

26. The media of claim 23, wherein the instructions are further operable when executed to:

decode the accessed data stream; and

extract each portion of the accessed data stream from the decoded data stream.

27. The media of claim 23, wherein the video comprises a video stream with multiple resolutions, a video stream from a single camera, a manipulated video stream, or a video stream from a top camera.

28. The media of claim 23, wherein the instructions are further operable when executed to:

infer a change in the particular view; and

select, based on the inferred change, one or more portions of the accessed data stream corresponding to the inferred change.

29. An apparatus comprising:

one or more non-transitory computer-readable storage media embodying instructions; and

one or more processors coupled to the storage media and configured to execute the instructions to;

receive, from a client device, a request corresponding to a particular view of a three-dimensional video comprising a plurality of views;

access a data stream corresponding to the video;

select, based on the request, one or more portions of the accessed data stream, at least one of the selected portions corresponding to the particular view; and

transmit the selected portion of the accessed data stream to the client device.

30. The apparatus of claim 29, wherein the received request comprises instructions for transmitting one or more of the portions of the accessed data stream at a specified bit rate or omitting one or more portions of the accessed data stream.

31. The apparatus of claim 29, wherein the received request comprises instructions for transforming the accessed data stream.

32. The apparatus of claim 29, wherein the processors are further configured to execute the instructions to:

decode the accessed data stream; and

extract each portion of the accessed data stream from the decoded data stream.

33. The apparatus of claim 29, wherein the video comprises a video stream with multiple resolutions, a video stream from a single camera, a manipulated video stream, or a video stream from a top camera.

34. The apparatus of claim 29, wherein the processors are further configured to execute the instructions to:

infer a change in the particular view; and

select, based on the inferred change, one or more portions of the accessed data stream corresponding to the inferred change.