Method and device for generating and detecting a fingerprint functioning as a trigger marker in a multimedia signal

• US 8,020,000 B2
• Filed: 07/05/2004
• Issued: 09/13/2011
• Est. Priority Date: 07/11/2003
• Status: Active Grant

• Current Assignee: Roku, Inc.
• Original Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Inventors: Oostveen, Job Cornelis, Ten Kate, Warner Rudolph Theophile, Roberts, David Keith, Denissen, Adrianus Johannes Maria
• Primary Examiner(s): Pich; Ponnoreay
• Application Number:
  US10/564,297
• Publication Number:
  US 20060190776A1
• Time in Patent Office: 2,626 Days
• Field of Search: None
• US Class Current: 713/176
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 21/23109 by placing content in organized collections, e.g. EPG data repository details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/433 Content storage operation, e.g. storage operation in response to a pause request, caching operations
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4586 Content update operation triggered locally, e.g. by comparing the version of software modules in a DVB carousel to the version stored locally deployment, distribution, installation, update of software G06F8/65; error detection or correction of the data by redundancy during software upgrading G06F11/1433; arrangements for updating broadcast information or broadcast-related information H04H60/25
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program

Abstract

This invention relates to a device and a method of relating one or more trigger actions with a multimedia signal and corresponding method and device for detecting one or more trigger actions in a multimedia signal. One fingerprint is generated on the basis of a segment of the multimedia signal at each trigger time point. The generated fingerprint(s) is stored in a database and communicated to playback devices. During playback fingerprints of the multimedia signal are generated and matched against fingerprints in the database. When a match is found, associated trigger actions is retrieved. The trigger time point is determined as a time point near or at a segment of the multimedia signal that caused a fingerprint to match. In this way, a simple and reliable way of detecting trigger markers for trigger actions is enabled without modifying the multimedia signal.

Claims

1. A method comprising:

providing a trigger time point (T_n;

T_n+1) of a plurality of trigger time points within a multimedia signal,

the trigger time point (T_n;

T_n+1) corresponding to a segment of a plurality of segments of the multimedia signal,

the trigger time point (T_n;

T_n+1) corresponding to at least one of a predetermined distance from the start of the segment or a predetermined distance from the end of the segment,

each segment of the plurality of segments being identified by one of the plurality of trigger time points and corresponding to specific content to be shown to a user;

providing a representation of an action that corresponds to the specific content to be shown in the segment of the multimedia signal,

the trigger time point (T_n;

T_n+1) indicating a time point within the multimedia signal at which the action is to be triggered during a playback of the multimedia signal;

deriving a fingerprint based on the segment of the multimedia signal; and

associating the derived fingerprint with the action.

2. The method according to claim 1, further comprising:

storing at least one of the derived fingerprint or the representation of the action in a first database.

3. The method according to claim 1, further comprising

transmitting at least one of the derived fingerprint or the representation of the action to a playback device.

4. The method according to claim 1, wherein

the multimedia signal includes at least one of an audio signal or a video signal.

5. The method according to claim 1, wherein

the action is selected from a group consisting of:

retrieving and displaying additional information on a display,

retrieving and playing additional information via a speaker,

playing another multimedia signal instead of the multimedia signal for a period of time,

interrupting the playback of the multimedia signal,

executing control commands, and

preparing a system for user inputs.

6. The method according to claim 2, further comprising

storing at least one of the derived fingerprint or the representation of the action in a second database;

and wherein

the derived fingerprint is at least one of an audio fingerprint or a video fingerprint.

7. A device comprising:

a fingerprint module implemented in hardware, the fingerprint module being configured to;

provide a trigger time point (T_n;

T_n+1) of a plurality of trigger time points within a multimedia signal,

the trigger time point (T_n;

T_n+1) corresponding to a segment of a plurality of segments of the multimedia signal,

the trigger time point (T_n;

T_n+1) corresponding to at least one of a predetermined distance from the start of the segment or a predetermined distance from the end of the segment,

each segment of the plurality of segments being identified by one of the plurality of trigger time points and corresponding to specific content to be shown to a user; and

provide a representation of an action that corresponds to the specific content to be shown in the segment of the multimedia signal,

the trigger time point (T_n;

T_n+1) indicating a time point within the multimedia signal at which the action is to be triggered during a playback of the multimedia signal; and

derive a fingerprint based on the segment of the multimedia signal; and

a database module configured to associate the derived fingerprint with the action.

8. The device according to claim 7, further comprising:

a first database to store at least one of the derived fingerprint or the representation of the action.

9. The device according to claim 7, further comprising:

a transmitter to transmit at least one of the derived fingerprint or the representation of the action to a playback device.

10. The device according to claim 7, wherein

the multimedia signal includes at least one of an audio signal or a video signal.

11. The device according to claim 7, wherein

the action is selected from a group consisting of:

retrieving and displaying additional information on a display,

retrieving and playing additional information via a speaker,

playing another multimedia signal instead of the multimedia signal for a period of time,

interrupting the playback of the multimedia signal,

executing control commands, and

preparing the device for user inputs.

12. The device according to claim 8, further comprising

a second database to store at least one of the derived fingerprint or the representation of the action;

and wherein

the derived fingerprint is at least one of an audio fingerprint or a video fingerprint.

13. A non-transitory computer readable storage medium comprising instructions that, when executed, cause one or more processors to execute operations comprising:

providing a trigger time point (T_n;

T_n+1) of a plurality of trigger time points within a multimedia signal,

the trigger time point (T_n;

T_n+1) corresponding to a segment of a plurality of segments of the multimedia signal,

the trigger time point (T_n;

T_n+1) corresponding to at least one of a a predetermined distance from the start of the segment or a predetermined distance from the end of the segment,

each segment of the plurality of segments being identified by one of the plurality of trigger time points and corresponding to specific content to be shown to a user;

providing a representation of an action that corresponds to the specific content to be shown by the segment of the multimedia signal,

the trigger time point (T_n;

T_n+1) indicating a time point within the multimedia signal at which the action is to be triggered during a playback of the multimedia signal;

deriving a fingerprint based on the segment of the multimedia signal; and

associating the derived fingerprint with the action.