Identifying media on a mobile device

• US 9,628,829 B2
• Filed: 06/26/2012
• Issued: 04/18/2017
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Li, Renxiang, Baum, Kevin L., Ishtiaq, Faisal, Needham, Michael L.
• Primary Examiner(s): Chae, Kyu
• Application Number:
  US13/533,247
• Publication Number:
  US 20130347017A1
• Time in Patent Office: 1,757 Days
• Field of Search: 725 13, 725 19, 725 60, 709248
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/6581 Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8352 involving content or source identification data, e.g. Unique Material Identifier [UMID]

Abstract

Disclosed is a method of associating, at a secondary device, secondary media content with primary media content being output at a primary device. The method includes receiving, at the secondary device, first information based upon the primary content being output at the primary device, wherein the first information includes at least one of an audio and a visual signal, determining at the secondary device second information corresponding to the first information, receiving at the secondary device one or more portions of secondary media content that have been made available by a third device, determining at the secondary device whether one or more of the portions of the secondary media content match one or more portions of the second information, and taking at least one further action upon determining that there is a match.

Claims

1. A method of associating secondary media content with primary media content being output at a display device, the method comprising:

transmitting, at a first time point, from a mobile device to a server, a notification to begin generating a plurality of media signatures included in one or more portions of secondary media content, wherein the secondary media content is related to the primary media content;

receiving, at the mobile device at a second time point, first information based upon the primary media content being output at the display device by at least one of an audio input device and a visual input device of the mobile device, wherein the first information includes at least one of an audio and a visual signal;

determining, at the mobile device, a data signature based at least in part on the first information;

transmitting, at a third time point, a request to the server from the mobile device, for the one or more portions of secondary media content that includes the plurality of media signatures;

receiving, at the mobile device, the one or more portions of secondary media content that includes the plurality of media signatures from the server in response to the request, wherein each of the plurality of media signatures spans a time between the first time point and the third time point;

determining, at the mobile device, whether at least one media signature included in the one or more of the portions of the secondary media content matches one or more portions of the data signature; and

taking at least one further action in response to determining that at least one media signature included in the one or more portions of the secondary media content match the one or more portions of the data signature.

2. The method of claim 1 wherein the plurality of media signatures include at least one of audio-based and visual-based media signatures for one or more media programs.

3. The method of claim 1 further comprising:

upon determining that at least one media signature included in the one or more portions of the secondary media content match the one or more portions of the data signature, then additionally sending, for receipt by the server, at least one instruction configured to cause the server to refrain from sending additional portions of the secondary media content.

4. The method of claim 1 further comprising:

commencing an operation by which the mobile device is capable of receiving push communication signals transmitted by the server, wherein the one or more portions of the secondary media content are received via the push communication signals.

5. The method of claim 1 wherein receiving the first information includes obtaining, by use of the mobile device, a media sample of primary media content being output by the display device, wherein the taking of the at least one further action includes launching a service tied to an identified program directly, or indirectly, by at least one of forwarding program identification information and related information to an entity other than the mobile device.

6. The method of claim 1 wherein the one or more portions of the secondary media content include a signature chunk that includes the plurality media signatures.

7. The method of claim 1 wherein each media signature is attributable to at least one of a channel or program viewable during a media time window and associated with the primary media content.

8. The method of claim 7 wherein at least one media signature is grouped with one or more other media signatures derived from another point in time or another program to form one or more signature chunks.

9. The method of claim 8 wherein the one or more signature chunks include media signature information limited to a media time window.

10. The method of claim 8 wherein at least one of the media signatures chosen for partial inclusion in the one or more signature chunks is selected according to a prioritization scheme.

11. The method of claim 8 wherein at least one of the one or more signature chunks selected for transmission to the mobile device is selected according to a prioritization scheme.

12. The method of claim 11 wherein the mobile device is selected from the group consisting of:

a cell phone, a personal digital assistant, a mobile computer device, and a tablet computer device, and wherein the display device is a set-top box or a television set.

13. A mobile device comprising:

an audio input device and a visual input device of the mobile device configured to receive information based upon the primary media content being output by a display device distinct from the mobile device;

a memory device configured to store one or more portions of a data signature generated based upon the information, wherein the data signature was generated at a second time point;

a wireless transceiver configured to;

transmit to a server computer, at a first time point, a notification to begin generating a plurality of media signatures included in one or more portions of secondary media content, wherein the secondary media content is related to the primary media content;

transmit, at a third time point, a request to the server computer for the one or more portions of secondary media content that includes the plurality of media signatures;

receive, in response to the request, communication signals originating at the server computer, the communication signals including one or more portions of secondary media content that includes the plurality of media signatures, wherein each of the plurality of media signatures spans a time between the first time point and the third time point; and

a processing device coupled at least indirectly to each of the memory device and the wireless transceiver, the processing device configured to determine whether at least one media signature included in the one or more of the portions of the secondary media content matches one or more portions of the data signature;

wherein the mobile device is further configured to take at least one action in response to determining that at least one media signature included in the one or more portions of the secondary media content match the one or more portions of the data signature.

14. A method of associating secondary media content with primary media content being output at a display device, the method comprising:

obtaining, at at least one of a display device and a mobile device, a media sample related to the primary media content being viewed on the display device by at least one of an audio input device and a visual input device of the at least one of a display device and a mobile device;

transmitting at a first time point, to a first server, a notification to begin generating a plurality of media signatures included in one or more signature chunks derived from the primary media content;

generating one or more data signatures from the media sample at the at least one of the display device and the mobile device;

transmitting, at a second time point, a request to the first server for the one or more signature chunks that includes the plurality of media signatures;

determining, at the at least one of the display device and the mobile device, whether the one or more data signatures match one or more media signatures within the one or more signature chunks, wherein each of the media signatures spans a time between the first time point and the second time point, and wherein the at least one of the display device and the mobile device is further configured to take at least one action in response to determining that at least one media signature within the one or more signature chunks matches the one or more data signatures.

15. The method of claim 14 wherein the at least one action taken upon determining that there is a match includes communicating to at least one of the first server and a second server at least one of the identity of the primary media content and related information.

16. The method of claim 14 wherein the one or more signature chunks are selected based on a prioritization scheme obtained from at least one of the first server and the at least one of the display device and the mobile device.

17. The method of claim 14 wherein upon verification that a match has been obtained, the at least one of the display device and the mobile device sends a request to the first server to cease transmission of additional signature chunks.

18. The method of claim 14 wherein a plurality of media signatures are bundled to form the signature chunks, which are transmitted by the first server as either a discrete signature chunk, a limited number of signature chunks, or as a continuous stream of signature chunks.