Synchronization of User Interactive Events with On-Screen Events During Playback of Multimedia Stream

• US 20110081965A1
• Filed: 10/07/2009
• Published: 04/07/2011
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Klein, Craig A., Harp, Gregory O., Rushing, James D.
• Granted Patent: US 9,043,829 B2
• Time in Patent Office: Days
• US Class Current: 463/31
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/338 using television networks
• CPC Class Codes: A63F 13/88 Mini-games executed independently while main games are being loaded
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/6036 for offering a minigame in combination with a main game
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8173 End-user applications, e.g. Web browser, game
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

A method includes receiving, at a multimedia device, a data stream representing multimedia content, the data stream comprising first timing information for the multimedia content, and metadata representing interactive event information associated with the multimedia content, the metadata comprising second timing information for the interactive event information. During the playback of the multimedia content, a first user interactive event is identified that is associated with a first time point in the playback of the multimedia content based on the interactive event information, and the first user interactive event is performed concurrent with the first time point in the playback of the multimedia content.

Claims

1. A method comprising:

receiving, at a multimedia device, a data stream representing multimedia content, the data stream comprising first timing information for the multimedia content;

receiving, at the multimedia device, metadata representing interactive event information associated with the multimedia content, the metadata comprising second timing information for the interactive event information;

processing, at the multimedia device, the data stream to provide a playback of the multimedia content to a user; and

during the playback of the multimedia content;

identifying, at the multimedia device, a first user interactive event that is associated with a first time point in the playback of the multimedia content based on the interactive event information; and

performing, at the multimedia device, the first user interactive event concurrent with the first time point in the playback of the multimedia content.

2. The method of claim 1, wherein identifying the first user interactive event comprises:

determining a subset of the interactive event information that is associated with the first time point based on the second timing information in response to determining that a progress of the playback of the multimedia content is approaching the first time point based on the first timing information; and

identifying the first user interactive event from the subset of the interactive event information.

3. The method of claim 1, wherein the first timing information comprises a first set of time stamps interspersed within the data stream and the second timing information comprises a second set of time stamps interspersed within the metadata.

4. The method of claim 3, wherein the time stamps of the first set and the second set comprise time stamps synchronized to Coordinated Universal Time.

5. The method of claim 1, wherein receiving the metadata at the multimedia device comprises receiving the metadata at the multimedia device concurrent with receiving the data stream.

6. The method of claim 5, wherein the metadata is embedded in the data stream.

7. The method of claim 5, wherein receiving the metadata at the multimedia device comprises accessing the metadata from an external source via a network responsive to initiation of the playback of the multimedia content at the multimedia device.

8. The method of claim 1, wherein receiving the metadata at the multimedia device comprises accessing, at the multimedia device, the metadata from a portable storage medium.

9. The method of claim 1, further comprising:

providing, via the multimedia device, a video game application concurrent with the playback of the multimedia content, the video game application providing a video game experience emulating a situation presented in the playback of the multimedia content; and

wherein;

the interactive event information comprises game context information for use by the video game application, the game context information associated with multimedia content presented at the first time point; and

the first user interactive event comprises a gaming action of the video game application based on the game context information.

10. The method of claim 1, further comprising:

presenting, via the multimedia device, an on-screen display that overlies the playback of the multimedia content, the on-screen display presenting scoring information associated with the multimedia content; and

wherein;

the interactive event information comprises updated scoring information associated with the multimedia content presented at the first time point; and

the first user interactive event comprises updating the on-screen display to reflect the updated scoring information.

11. The method of claim 1, further comprising:

during the playback of the multimedia content;

identifying, at the multimedia device, a second user interactive event based on the interactive event information that is associated with a second time point in the playback of the multimedia content; and

performing, at the multimedia device, the second user interactive event concurrent with the second time point in the playback of the multimedia content.

12. A multimedia device comprising:

a first interface to receive a data stream representing multimedia content, the data stream comprising first timing information for the multimedia content;

a second interface to receive metadata representing interactive event information associated with the multimedia content, the metadata comprising second timing information for the interactive event information;

a display controller to process the data stream to provide a playback of the multimedia content to a user; and

a user interaction controller to identify, during the playback of the multimedia content, a first user interactive event that is associated with a first time point in the playback of the multimedia content based on the interactive event information, and the user interaction controller to control a performance of the first user interactive event concurrent with the first time point in the playback of the multimedia content.

13. The multimedia device of claim 12, wherein the user interaction controller identifies the first user interactive event by determining, based on the first timing information, that a progress of the playback of the multimedia content is approaching the first time point, determining a subset of the interactive event information that is associated with the first time point based on the second timing information, and identifying the first user interactive event from the subset of the interactive event information.

14. The multimedia device of claim 12, wherein the first timing information comprises a first set of time stamps interspersed within the data stream and the second timing information comprises a second set of time stamps interspersed within the metadata.

15. The multimedia device of claim 12, further comprising:

a portable storage interface to interface with a portable storage medium, the portable storage medium storing the metadata.

16. The multimedia device of claim 12, further comprising:

a storage device to store executable instructions for a video game application; and

a processor to execute the executable instructions to provide video game experience emulating a situation presented in the playback of the multimedia content concurrent with the playback of the multimedia content; and

wherein;

the interactive event information comprises game context information for use by the video game application, the game context information associated with the multimedia content presented at the first time point; and

the first user interactive event comprises a gaming action of the video game application based on the game context information.

17. The multimedia device of claim 12, further comprising:

a storage device to store executable instructions for an on-screen display application; and

a processor to execute the executable instructions to provide an on-screen display that overlies the playback of the multimedia content, the on-screen display presenting scoring information associated with the multimedia content; and

wherein;

the interactive event information comprises updated scoring information associated with the multimedia content presented at the first time point; and

the first user interactive event comprises updating the on-screen display to reflect the updated scoring information.

18. A video game console comprising:

a first interface to receive a data stream representing multimedia content, the data stream comprising first timing information for the multimedia content;

a second interface to receive metadata representing game context information associated with the multimedia content, the metadata comprising second timing information for the interactive event information;

a display controller to process the data stream to provide a playback of the multimedia content;

a storage component to store a software program comprising a set of executable instructions representative of a video game application; and

a processor to execute the software program to present a video game experience to a user concurrent with the playback of the multimedia content, the video game experience emulating a situation concurrently presented in the playback of the multimedia content;

whereby the set of executable instructions is configured to manipulate the processor to synchronize gaming actions of the video game experience to corresponding time points in the playback of the multimedia content based on the first timing information and the second timing information.

19. The video game console of claim 18, wherein:

the metadata is stored on a portable storage medium; and

the second interface comprises a portable storage interface to access the metadata from the portable storage medium.

20. A method comprising:

generating, at a multimedia distribution system, a data stream representing multimedia content, the data stream comprising first timing information for the multimedia content;

determining, at the multimedia distribution system, a set of time points in a playback of the multimedia content that correspond to identified on-screen events in the multimedia content;

determining, at the multimedia distribution system, interactive event information for each of the identified on-screen events;

generating, at the multimedia distribution system, metadata comprising the interactive event information for the identified events, the metadata further comprising second timing information associating a time point of an identified on-screen event with corresponding interactive event information;

providing the data stream to a multimedia device; and

providing the metadata to the multimedia device.

21. The method of claim 20, wherein the interactive event information comprises score update information for use by an on-screen display overlay at the multimedia device.

22. The method of claim 20, wherein the interactive event information comprises game context information for use by a video game application at the multimedia device.