Integration of multiple synchronization methodologies

• US 9,596,502 B1
• Filed: 12/21/2015
• Issued: 03/14/2017
• Est. Priority Date: 12/21/2015
• Status: Expired due to Fees

• Current Assignee: CustomPlay, LLC
• Original Assignee: CustomPlay, LLC
• Inventors: Blodnieks, Ivars, Abecassis, Max
• Primary Examiner(s): Corbo, Nicholas
• Application Number:
  US14/977,526
• Time in Patent Office: 449 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: H04H 20/18 Arrangements for synchronising broadcast or distribution via plural systems
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04N 21/232 Content retrieval operation locally within server, e.g. reading video streams from disk arrays storage management G06F3/0604; details of querying and searching of video data from a database G06F16/739
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4398 involving reformatting operations of audio signals details of audio signal transcoding G10L19/173
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

A method comprising the steps of: generating an acoustic fingerprint that is responsive to a playing of a movie; searching, responsive to the acoustic fingerprint, an acoustic database; generating a dialogue fingerprint that is responsive to the playing of the movie; searching, responsive to the dialogue fingerprint, a dialogue database; identifying, responsive to the searching of the acoustic database and responsive to the searching of the dialogue database, a location within the movie; and displaying, responsive to the identified location, an information item that is associated with the identified location; and wherein the generating the dialogue fingerprint comprises a conversion of a portion of dialogue to text; wherein the searching the acoustic database is initially limited to a portion of the acoustic database; wherein the identifying is further responsive to a confidence indicator; and wherein a subsequent searching is responsive to a synchronization suspend timer.

Claims

1. A method comprising the steps of:

generating, with a second screen device, an acoustic fingerprint that is responsive to a playing of a movie on a primary screen device;

searching, responsive to the acoustic fingerprint, an acoustic database;

generating, with the second screen device, a dialogue fingerprint that is responsive to the playing of the movie on the primary screen device;

searching, responsive to the dialogue fingerprint, a dialogue database;

identifying, responsive to the searching of the acoustic database and responsive to the searching of the dialogue database, a location within the movie;

displaying, on the second screen device and responsive to the identified location, an indication that information is available; and

displaying, on the second screen device and responsive to a request for information, an information item that is responsive to the displayed indication that information is available,

wherein the searching the acoustic database is initially limited to a portion of the acoustic database corresponding to a beginning portion of the movie, whereby a complete search of the acoustic database is ignored.

2. The method of claim 1 wherein the identifying, responsive to the searching of the acoustic database and responsive to the searching of the dialogue database, the location within the movie is further responsive to a confidence indicator associated with a search of the dialogue database.

3. The method of claim 1 wherein the identifying, responsive to the searching of the acoustic database and responsive to the searching of the dialogue database, the location within the movie is further responsive to a confidence indicator associated with a search of the acoustic database.

4. The method of claim 1 wherein the generating the dialogue fingerprint comprises a conversion of a portion of dialogue to text.

5. The method of claim 1 further comprising the step of downloading the acoustic database and the dialogue database from a remote information provider, and wherein the searching of the acoustic database, the searching of the dialogue database, and the identifying of the location within the movie are performed by the second screen device.

6. The method of claim 1 wherein subsequent to an initial identifying of a location within the movie, a subsequent searching of the acoustic database and of the dialogue database is responsive to a synchronization suspend timer.

7. The method of claim 1 wherein subsequent to an initial identifying of a location within the movie, and subsequent to a suspension of a searching of the acoustic database and of the dialogue database, a resumption of the searching of the acoustic database and of the dialogue database is limited to a portion of the databases that is responsive to a last identified location within the movie.

8. A method comprising the steps of:

generating an acoustic fingerprint that is responsive to a playing of a movie;

searching, responsive to the acoustic fingerprint, an acoustic database;

generating a dialogue fingerprint that is responsive to the playing of the movie;

searching, responsive to the dialogue fingerprint, a dialogue database;

identifying, responsive to the searching of the acoustic database and responsive to the searching of the dialogue database, a location within the movie; and

displaying, responsive to the identified location, an information item that is associated with the identified location,

wherein the searching the acoustic database is initially limited to a portion of the acoustic database corresponding to a beginning portion of the movie, whereby a complete search of the acoustic database is ignored.

9. The method of claim 8 wherein the identifying, responsive to the searching of the acoustic database and responsive to the searching of the dialogue database, the location within the movie is further responsive to a confidence indicator associated with a search of the acoustic database.

10. The method of claim 8 wherein the generating the dialogue fingerprint comprises a conversion of a portion of dialogue to text.

11. The method of claim 8 further comprising the step of downloading the acoustic database and the dialogue database from a remote information provider, and wherein the searching of the acoustic database, the searching of the dialogue database, and the identifying of the location within the movie are performed by a second screen device.

12. The method of claim 8 wherein subsequent to an initial identifying of a location within the movie, a subsequent searching of the acoustic database and of the dialogue database is responsive to a synchronization suspend timer.

13. The method of claim 8 wherein subsequent to an initial identifying of a location within the movie, and subsequent to a suspension of a searching of the acoustic database and of the dialogue database, a resumption of the searching of the acoustic database and of the dialogue database is limited to a portion of the databases that is responsive to a last identified location within the movie.

14. A second screen device comprising:

a communications interface capable of downloading, over a communications network from a remote information provider and to a memory of the second screen device, a plurality of information items each associated with a location within a movie, an acoustic database, and a dialogue database;

an integrated microphone capable of receiving audio information from a playing of the movie on a primary screen device; and

a processor executing an application capable of generating an acoustic fingerprint and a dialogue fingerprint that are responsive to the playing of the movie on the primary screen device;

capable of identifying, responsive to a searching of the acoustic database and responsive to a searching of the dialogue database, a location within the movie; and

capable of causing responsive to the identified location and on a display screen of the second screen device, a display of an information item,

wherein the searching the acoustic database is initially limited to a portion of the acoustic database corresponding to a beginning portion of the movie, whereby a complete search of the acoustic database is ignored.

15. The second screen device of claim 14 wherein the identifying, responsive to the searching of the acoustic database and responsive to the searching of the dialogue database, the location within the movie is further responsive to a confidence indicator associated with a search of the acoustic database.

16. The second screen device of claim 14 wherein subsequent to an initial identifying of a location within the movie, a subsequent searching of the acoustic database and of the dialogue database is responsive to a synchronization suspend timer.

17. The second screen device of claim 14 wherein subsequent to an initial identifying of a location within the movie, and subsequent to a suspension of a searching of the acoustic database and of the dialogue database, a resumption of the searching of the acoustic database and of the dialogue database is limited to a portion of the databases that is responsive to a last identified location within the movie.