AUGMENTATION OF MULTIMEDIA CONSUMPTION

• US 20140009476A1
• Filed: 07/06/2012
• Published: 01/09/2014
• Est. Priority Date: 07/06/2012
• Status: Active Grant

• Current Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Original Assignee: General Instrument Corporation (CommScope Holding Company, Inc.)
• Inventors: Venkitaraman, Narayanan, Chaysinh, Shirley A., Mandalia, Hiren M., Metcalf, Crysta J., Novak, Ashley B., Kulka, Isaac G.
• Granted Patent: US 9,854,328 B2
• Time in Patent Office: Days
• US Class Current: 345/502
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/43079 of additional data with content streams on multiple devices
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

Disclosed are methods and apparatus for augmenting a user'"'"'s multimedia consumption experience. The methods comprise whilst the user is consuming the multimedia presentation using a first device, that device provides (to one or more remote processors) information that may be used to identify a relevant location. The one or more processors use this information to identify the location and acquire a virtual environment. This virtual environment may be a virtual representation the location. The virtual environment is presented to the user on a second (companion) device. Using the second device, the user may explore the virtual environment and interact with virtual objects therein.

Claims

1. A method of augmenting a user'"'"'s multimedia consumption experience, the user being a user of a first user device and of a second user device, the user using the first device to consume a multimedia presentation, the method comprising:

providing, for use by one or more processors remote from the first device, information that may be used by the one or more processors to identify a location, the location being relevant to at least part of the multimedia presentation;

identifying, by the one or more processors, the location;

acquiring, by the one or more processors, a virtual environment, the virtual environment being representative of the identified location;

acquiring, by the one or more processors, one or more virtual objects, each virtual object relating to at least part of the multimedia presentation;

determining, by the one or more processors, for each virtual object, a position for that virtual object within the virtual environment;

for each virtual object, including that virtual object in the virtual environment at the position within the virtual environment determined for that virtual object; and

presenting to the user, using the second device, the virtual environment with the one or more virtual objects included therein;

wherein the virtual environment is to some extent explorable by the user using the second user device; and

wherein the virtual objects within the virtual environment are capable of being interacted with by the user.

2. A method according to claim 1:

wherein each virtual object corresponds to an object within the multimedia presentation;

the method further comprising changing one or more properties of a virtual object within the virtual environment in response to a change, in the multimedia presentation, to one or more properties of the object that corresponds to that virtual object.

3. A method according to claim 1:

wherein each virtual object corresponds to an object within the multimedia presentation;

the method further comprising, in response to the user interacting with a virtual object within the virtual environment, changing, in the multimedia presentation, one or more properties of the object that corresponds to that virtual object.

4. A method according to claim 1 wherein the location is a location at which events that occur in the multimedia presentation take place or that is referred to at some point in the multimedia presentation.

5. A method according to claim 4 wherein the location is a location at which events occurring in a portion of the multimedia presentation that is currently being consumed by the user take place or a location that is currently being referred to in a portion of the multimedia presentation that is currently being consumed by the user.

6. A method according to claim 1 wherein including comprises overlaying the one or more objects onto the virtual environment using an augmented-reality process.

7. A method according to claim 1 wherein the virtual environment is a three-dimensional virtual environment.

8. A method according to claim 1:

wherein the virtual environment is an environment in which the user may interact with a further user;

wherein the further user is a user of a further device; and

wherein the virtual environment is being presented or has been presented, using the further device, to the further user.

9. A method according to claim 1 wherein the information that may be used by the one or more processors to identify the location comprises first identification data, the first identification data identifying the multimedia presentation, and second identification data, the second identification data identifying a point or range in the multimedia presentation that is currently being consumed by the user.

10. A method according to claim 1 wherein a watermark that is present in a portion of the multimedia presentation comprises information that may be used by the one or more processors to identify the location.

11. A method according to claim 1 wherein acquiring the virtual environment comprises selecting one or more images of the identified location and forming the virtual environment using the selected images.

12. A method according to claim 1 wherein the virtual environment comprises a panoramic representation of the location.

13. A method according to claim 1 wherein the multimedia presentation is selected from a group of multimedia presentations consisting of:

a video presentation, a television program, and a movie.

14. A method according to claim 1 wherein the first device is the same as the second device.

15. A method according to claim 14:

wherein the second device may be moved between having a first orientation and a second orientation;

wherein when the second device is positioned in the first orientation, the second device displays the multimedia presentation; and

wherein when the second device is positioned in the second orientation, the second device displays the virtual environment.

16. A method of augmenting a user'"'"'s multimedia consumption experience, the user being a user of a first user device and of a second user device, the user using the first device to consume a multimedia presentation, the method comprising:

providing, for use by one or more processors remote from the first device, information that may be used by the one or more processors to identify a location, the location being relevant to at least part of the multimedia presentation;

identifying, by the one or more processors, the location;

acquiring, by the one or more processors, a virtual environment; and

presenting to the user, using the second device, the virtual environment;

wherein providing information that may be used by the one or more processors to identify a location comprises either;

the first device providing the information that may be used by the one or more processors to identify a location;

or

the one or more processors receiving the information that may be used by the one or more processors to identify a location from one or more servers, the one or more servers storing information that may be used to identify a location;

wherein the virtual environment is representative of the identified location; and

wherein the virtual environment is to some extent explorable by the user using the second user device.

17. A method according to claim 16 further comprising presenting to the user, using the second device, one or more virtual objects, the virtual objects being presented as objects within the virtual environment, the virtual objects being capable of being interacted with by the user.

18. A method according to claim 17:

wherein each virtual object corresponds to an object within the multimedia presentation;

the method further comprising;

changing one or more properties of a virtual object within the virtual environment in response to a change, in the multimedia presentation, to one or more properties of the object that corresponds to that virtual object;

or

in response to the user interacting with a virtual object within the virtual environment, changing, in the multimedia presentation, one or more properties of the object that corresponds to that virtual object.

19. Apparatus for augmenting a user'"'"'s multimedia consumption experience, the user being a user of a first user device and of a second user device, the user using the first device to consume a multimedia presentation, the apparatus comprising one or more processors arranged to:

use information to identify a location relevant to at least part of the multimedia presentation;

acquire a virtual environment, the virtual environment being representative of the identified location;

acquire one or more virtual objects, the virtual objects relating to at least part of the multimedia presentation;

determine, for each virtual object, a position for that virtual object within the virtual environment;

for each virtual object, include, in the virtual environment at the position within the virtual environment determined for that virtual object, that virtual object; and

present to the user, using the second device, the virtual environment with the one or more virtual objects included therein;

wherein the virtual environment is to some extent explorable by the user using the second user device; and

wherein the virtual objects within the virtual environment are capable of being interacted with by the user.

20. Apparatus for augmenting a user'"'"'s multimedia consumption experience, the user being a user of a first user device and of a second user device, the user using the first device to consume a multimedia presentation, the apparatus comprising one or more processors configured to:

receive information for identifying a location, the location being relevant to at least part of the multimedia presentation;

identify the location;

acquire a virtual environment; and

present to the user, using the second device, the virtual environment;

wherein the information that may be used by the one or more processors to identify a location is either;

received from the first device;

or

received from one or more servers, the one or more servers storing information that may be used to identify a location;

wherein the virtual environment is representative of the identified location; and

wherein the virtual environment is to some extent explorable by the user using the second user device.