Identifying media on a mobile device

• US 10,051,295 B2
• Filed: 03/10/2017
• Issued: 08/14/2018
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Li, Renxiang, Baum, Kevin L., Ishtiaq, Faisal, Needham, Michael L.
• Primary Examiner(s): Chae, Kyu
• Application Number:
  US15/456,179
• Publication Number:
  US 20170188061A1
• Time in Patent Office: 522 Days
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/6581 Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8352 involving content or source identification data, e.g. Unique Material Identifier [UMID]

Abstract

Disclosed is a method of associating, at a secondary device, secondary media content with primary media content being output at a primary device. The method includes receiving, at the secondary device, first information based upon the primary content being output at the primary device, wherein the first information includes at least one of an audio and a visual signal, determining at the secondary device second information corresponding to the first information, receiving at the secondary device one or more portions of secondary media content that have been made available by a third device, determining at the secondary device whether one or more of the portions of the secondary media content match one or more portions of the second information, and taking at least one further action upon determining that there is a match.

Claims

1. A method for associating secondary media content with primary media content, comprising:

determining, at a mobile device, a data signature associated with primary media content being output on a display device;

transmitting, at a first time point from the mobile device to a server, a notification to begin generating a plurality of media signatures included in one or more portions of secondary media content related to the primary media content;

transmitting, at a second time point from the mobile device to the server, a request for the one or more portions of secondary media content that includes the plurality of media signatures;

receiving, at the mobile device, the one or more portions of secondary media content that includes the plurality of media signatures in response to the request, wherein each of the plurality of media signatures spans a time between the first time point at which the notification to begin generating the plurality of media signatures was transmitted to the server and the second time point at which the request for the one or more portions of secondary media content was transmitted to the server;

determining, at the mobile device, whether at least one media signature included in the one or more portions of the secondary media content matches one or more portions of the data signature; and

in response to determining that the at least one media signature matches the one or more portions of the data signature, causing information associated with the primary media content to be presented based on the one or more portions of the secondary media content.

2. The method of claim 1, further comprising obtaining a data sample of the primary media content, wherein determining the data signature is in response to obtaining the data sample.

3. The method of claim 1, further comprising:

determining an identity of a program corresponding to the primary media content based on the at least one media signature that matches the one or more portions of the data signature;

transmitting the identity of the program to a second server; and

receiving second information associated with the primary media content from the second server.

4. The method of claim 1, wherein the size of each media signature in the plurality of media signatures is determined based on a bandwidth of a network connection to the server.

5. The method of claim 1, wherein a plurality of programs corresponding to each of the plurality of media signatures are selected based on a popularity metric.

6. The method of claim 1, wherein a plurality of programs corresponding to each of the plurality of media signatures are selected based on genres of each of the programs.

7. A system for associating secondary media content with primary media content, the system comprising:

a memory; and

 

a hardware processor of a mobile device, that, when executing computer executable instructions stored in the memory, is programmed to;

determine, at the mobile device, a data signature associated with primary media content being output on a display device;

transmit, at a first time point to a server, a notification to begin generating a plurality of media signatures included in one or more portions of secondary media content related to the primary media content;

transmit, at a second time point to the server, a request for the one or more portions of secondary media content that includes the plurality of media signatures;

receive the one or more portions of secondary media content that includes the plurality of media signatures in response to the request, wherein each of the plurality of media signatures spans a time between the first time point at which the notification to begin generating the plurality of media signatures was transmitted to the server and the second time point at which the request for the one or more portions of secondary media content was transmitted to the server;

determine whether at least one media signature included in the one or more portions of the secondary media content matches one or more portions of the data signature; and

in response to determining that the at least one media signature matches the one or more portions of the data signature, cause information associated with the primary media content to be presented based on the one or more portions of the secondary media content.

8. The system of claim 7, wherein the hardware processor is further programmed to obtain a data sample of the primary media content, wherein determining the data signature is in response to obtaining the data sample.

9. The system of claim 7, wherein the hardware processor is further programmed to:

determine an identity of a program corresponding to the primary media content based on the at least one media signature that matches the one or more portions of the data signature;

transmit the identity of the program to a second server; and

receive second information associated with the primary media content from the second server.

10. The system of claim 7, wherein the size of each media signature in the plurality of media signatures is determined based on a bandwidth of a network connection to the server.

11. The system of claim 7, wherein a plurality of programs corresponding to each of the plurality of media signatures are selected based on a popularity metric.

12. The system of claim 7, wherein a plurality of programs corresponding to each of the plurality of media signatures are selected based on genres of each of the programs.

13. A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for associating secondary media content with primary media content, the method comprising:

determining, at a mobile device, a data signature associated with primary media content being output on a display device;

transmitting, at a first time point from the mobile device to a server, a notification to begin generating a plurality of media signatures included in one or more portions of secondary media content related to the primary media content;

transmitting, at a second time point from the mobile device to the server, a request for the one or more portions of secondary media content that includes the plurality of media signatures;

receiving, at the mobile device, the one or more portions of secondary media content that includes the plurality of media signatures in response to the request, wherein each of the plurality of media signatures spans a time between the first time point at which the notification to begin generating the plurality of media signatures was transmitted to the server and the second time point at which the request for the one or more portions of secondary media content was transmitted to the server;

determining, at the mobile device, whether at least one media signature included in the one or more portions of the secondary media content matches one or more portions of the data signature; and

in response to determining that the at least one media signature matches the one or more portions of the data signature, causing information associated with the primary media content to be presented based on the one or more portions of the secondary media content.

14. The non-transitory computer-readable medium of claim 13, wherein the method further comprises obtaining a data sample of the primary media content, wherein determining the data signature is in response to obtaining the data sample.

15. The non-transitory computer-readable medium of claim 13, wherein the method further comprises:

determining an identity of a program corresponding to the primary media content based on the at least one media signature that matches the one or more portions of the data signature;

transmitting the identity of the program to a second server; and

receiving second information associated with the primary media content from the second server.

16. The non-transitory computer-readable medium of claim 13, wherein the size of each media signature in the plurality of media signatures is determined based on a bandwidth of a network connection to the server.

17. The non-transitory computer-readable medium of claim 13, wherein a plurality of programs corresponding to each of the plurality of media signatures are selected based on a popularity metric.

18. The non-transitory computer-readable medium of claim 13, wherein a plurality of programs corresponding to each of the plurality of media signatures are selected based on genres of each of the programs.