Biometric feedback in production and playback of video content

• US 10,497,399 B2
• Filed: 05/26/2018
• Issued: 12/03/2019
• Est. Priority Date: 12/05/2014
• Status: Active Grant

• Current Assignee: Warner Bros Entertainment Incorporated (Warner Bros. Discovery, Inc.)
• Original Assignee: Warner Bros Entertainment Incorporated (Warner Bros. Discovery, Inc.)
• Inventors: Gewickey, Gregory I., Ostrover, Lewis S., Smith, Michael, Zink, Michael
• Primary Examiner(s): Adams, Eileen M
• Application Number:
  US15/990,589
• Publication Number:
  US 20190005983A1
• Time in Patent Office: 556 Days
• Field of Search: None
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/388 Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume

Abstract

Methods for digital content production and playback of an immersive stereographic video work provide or enhance interactivity of immersive entertainment using various different playback and production techniques. “Immersive stereographic” may refer to virtual reality, augmented reality, or both. The methods may be implemented using specialized equipment for immersive stereographic playback or production. Aspects of the methods may be encoded as instructions in a computer memory, executable by one or more processors of the equipment to perform the aspects.

Claims

1. A method for controlling a display of a video content player apparatus, the method comprising:

playing video content on a content player apparatus;

receiving, during the playing, user input indicating an interest in an object appearing in a display of the content player apparatus;

identifying at least one of a three-dimensional model or a rendered image of the three-dimensional model from a computer memory, based on an identity of the object;

combining, during the playing, a rendered image of the three-dimensional model with video data of the video content, wherein the rendered image replaces an image of the object; and

changing a position, an orientation, a position relative to user viewpoint, or an orientation relative to user viewpoint, of the three-dimensional model in response to user input, whereby the rendered image of the three-dimensional model reveals a view of the object that is not included in the video content without the rendered image, wherein the three-dimensional model is selected from one of a prop for replacing a static object or a figure movable by an armature for replacing a character.

2. The method of claim 1, wherein receiving the input comprises processing sensor data indicating at least one of a gaze direction or an intended view direction of a user of the content player apparatus.

3. The method of claim 1, wherein receiving the input comprises receiving input from a pointing device manually operated by the user.

4. The method of claim 1, wherein receiving the input comprises processing sensor data indicating a bodily orientation or movement of a user of the content player apparatus.

5. The method of claim 1, wherein the image of the object includes photographically captured data, and the rendered image is at least partly computer generated.

6. The method of claim 1, wherein the view of the object that is not included in the video content includes hidden information associated with the object that is not visible without user interaction.

7. The method of claim 1, wherein the prop may be manipulated in response to the user input.

8. A video content player apparatus, comprising a processor operatively coupled to a memory, the memory holding program instructions, that when executed by the processor cause the apparatus to perform:

playing video content;

receiving, during the playing, user input indicating an interest in an object appearing in a display of the content player apparatus;

identifying at least one of a three-dimensional model or a rendered image of the three-dimensional model from a computer memory, based on an identity of the object;

combining, during the playing, a rendered image of the three-dimensional model with video data of the video content, wherein the rendered image replaces an image of the object; and

changing a position, an orientation, a position relative to user viewpoint, or an orientation relative to user viewpoint, of the three-dimensional model in response to user input, whereby the rendered image of the three-dimensional model reveals a view of the object that is not included in the video content without the rendered image, wherein the three-dimensional model is selected from one of a prop for replacing a static object or a figure movable by an armature for replacing a character.

9. The apparatus of claim 8, wherein the program instructions for receiving the input further comprise instructions for processing sensor data indicating at least one of a gaze direction or an intended view direction of a user of the video content player apparatus.

10. The apparatus of claim 8, wherein the program instructions for receiving the input further comprise instructions for receiving input from a pointing device manually operated by the user.

11. The apparatus of claim 8, wherein receiving the input comprises processing sensor data indicating a bodily orientation or movement of a user of the content player apparatus.

12. The apparatus of claim 8, wherein the image of the object includes photographically captured data, and the rendered image is at least partly computer generated.

13. The apparatus of claim 8, wherein the view of the object that is not included in the video content includes hidden information associated with the object that is not visible without user interaction.

14. The apparatus of claim 8, wherein the prop may be manipulated in response to the user input.

15. A video content player apparatus comprising:

means for playing video content on a content player apparatus;

means for receiving, during the playing, user input indicating an interest in an object appearing in a display of the content player apparatus;

means for identifying at least one of a three-dimensional model or a rendered image of the three-dimensional model from a computer memory, based on an identity of the object;

means for combining, during the playing, a rendered image of the three-dimensional model with video data of the video content, wherein the rendered image replaces an image of the object; and

means for changing a position, an orientation, a position relative to user viewpoint, or an orientation relative to user viewpoint, of the three-dimensional model in response to user input, whereby the rendered image of the three-dimensional model reveals a view of the object that is not included in the video content without the rendered image, wherein the three-dimensional model is selected from one of a prop for replacing a static object or a figure movable by an armature for replacing a character.

16. The apparatus of claim 15, wherein the program instructions for receiving the input further comprise instructions for processing sensor data indicating at least one of a gaze direction or an intended view direction of a user of the video content player apparatus.

17. The apparatus of claim 15, wherein the program instructions for receiving the input further comprise instructions for receiving input from a pointing device manually operated by the user.

18. The apparatus of claim 15, wherein receiving the input comprises processing sensor data indicating a bodily orientation or movement of a user of the content player apparatus.

19. The apparatus of claim 15, wherein the view of the object that is not included in the video content includes hidden information associated with the object that is not visible without user interaction.

20. The apparatus of claim 15, wherein the prop may be manipulated in response to the user input.