Systems and Methods for Creating an Immersive Video Content Environment

• US 20170318283A1
• Filed: 04/27/2016
• Published: 11/02/2017
• Est. Priority Date: 04/27/2016
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Watson, Scott F., Smoot, Lanny S.
• Granted Patent: US 10,523,929 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 21/00 Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18

Abstract

There is provided a system including a non-transitory memory storing an executable code and a hardware processor executing the code to receive a video content including a plurality of frames showing a scene from a perspective of a real camera, create a three-dimensional (3D) model of the scene using the plurality of frames, store the 3D model of the scene in the non-transitory memory, construct a synthetic view of the scene showing additional perspectives from one or more virtual cameras at one or more locations in the scene, transmit the synthetic view of the scene for being displayed on a display, display a scene of the video content on the display, track a position of a viewer moving in a room, and adjust the display of the scene being displayed on the display based on the position of the viewer in the room relative to the display.

Claims

1. A system comprising:

a non-transitory memory storing an executable code;

a hardware processor executing the executable code to;

receive a video content including a plurality of frames showing a scene from a perspective of a real camera;

create a three-dimensional (3D) model of the scene using the plurality of frames;

store the 3D model of the scene in the non-transitory memory;

construct a synthetic view of the scene showing additional perspectives from one or more virtual cameras at one or more locations in the scene; and

transmit the synthetic view of the scene for being displayed on a display.

2. The system of claim 1, wherein a viewer chooses the one or more locations of the one or more virtual cameras in the scene.

3. The system of claim 1, wherein the synthetic view is a dynamic view showing the scene as it appears from the perspective of a dynamic virtual camera moving in the scene based on the 3D model of the scene.

4. The system of claim 1, wherein the video content is one of a live event and a pre-recorded event.

5. The system of claim 1, wherein the hardware processor executes the executable code to create the 3D model of the scene of the video content in real-time.

6. The system of claim 5, wherein the hardware processor executes the executable code to construct the synthetic view in real-time.

7. The system of claim 1. further comprising a display, wherein the processor further executes the executable code to:

display a scene of a video content on the display;

track a position of a viewer moving in the room; and

adjust the display of the scene being displayed on the display based on the position of the viewer in the room relative to the display.

8. The system of claim 7, wherein the processor further executes the executable code to continuously adjust the view of the scene of the video content being displayed on the display while the viewer moves relative to the display based on the position of the user relative to the display.

9. The system of claim 8, wherein the processor further executes the executable code to continuously adjust he view of the scene of the video content being displayed on the display to maintain an angularly correct display of the scene of the video content based on the position of the user relative to the display.

10. The system of claim 7, wherein the display shows an angularly accurate view of the video content for up to 360 degrees around a viewer based on the position of the user relative to the display.

11. The system of claim 7, wherein the display includes a spherical display displaying the video content above and below the viewer up to 4π

steradians.

13. The method of claim 11, wherein a viewer chooses the one or more locations of the one or more virtual cameras in the scene.

14. The method of claim 11, wherein the synthetic view is a dynamic view showing the scene as it appears from the perspective of a dynamic virtual camera moving in the scene based on the 3D model of the scene.

15. The method of claim 11, wherein the media content is one of a live event and a pre-recorded event.

16. The method of claim 11, wherein creating the 3D model of the scene of the video content and constructing the synthetic view are performed in real-time.

17. The method of claim 11, further comprising:

displaying, using the hardware processor, a scene of a video content on the display;

tracking, using the hardware processor, a position of a viewer moving in the room; and

adjusting, using the hardware processor, the display of the scene being displayed on the display based on the position of the viewer in the room relative to the display.

18. The method of claim 17, further comprising continuously adjusting the view of the scene being displayed on the display based on the position of the user relative to the display while the viewer moves relative to the display.

19. The method of claim 18, further comprising continuously adjusting the view of the scene being displayed on the display to maintain an angularly correct display of the scene of the video content based on the position of the user relative to the display.

20. The method of claim 17, wherein the display includes a spherical display displaying an angularly accurate view of the video content around, above, and below the viewer up to 4π

steradians.

12. A method for use with a system having a non-transitory memory and a hardware processor, the method comprising:

receiving, using the hardware processor, a video content including a plurality of frames showing a scene from a perspective of a real camera;

creating, using the hardware processor, a three-dimensional (3D) model of the scene using the plurality of frames;

storing the 3D model of the scene in the non-transitory memory;

constructing, using the hardware processor, a synthetic view of the scene showing additional perspectives from one or more virtual cameras at one or more locations in the scene; and

transmitting, using the hardware processor, the synthetic view of the scene for being displayed on a display.