Methods and systems for processing a sample of a media stream

• US 10,003,664 B2
• Filed: 02/08/2016
• Issued: 06/19/2018
• Est. Priority Date: 05/04/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Shazam Entertainment Limited (Apple Inc.)
• Inventors: Wang, Avery Li-Chun, Revill, Adrian Allan
• Primary Examiner(s): MAUNG, THOMAS H
• Application Number:
  US15/018,124
• Publication Number:
  US 20160156731A1
• Time in Patent Office: 862 Days
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04H 2201/90 characterised by the use of signatures
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739

Abstract

Methods and systems for processing a sample of a media stream are provided. In one example, a method includes receiving a sample of a media stream rendered by a media rendering source, and for matching media recordings from which the sample was obtained, the method also includes receiving a signature file including a temporally mapped collection of features describing content of the matching media recordings. Each received signature file may be stored in memory. In addition, a signature stream may be received that includes features extracted from the media stream, and a number of matching features between the signature stream of the media stream and the signature file for at least one matching media recording can be determined. In one example, when substantially matching features between the signature stream of the media stream and the signature file for a matching media recording stop occurring, a discontinuity can be identified.

Claims

1. A method comprising:

receiving from a server into a local cache on a computing device a signature file of fingerprints of media content and corresponding identifiers of the media content after the computing device sends one or more initial fingerprints determined from a media stream by the computing device, wherein the fingerprints in the signature file are for features anticipated to appear in the media stream at predetermined times; and

the computing device using the cached fingerprints of media content in the signature file to identify a media content item within the media stream based on one or more additional fingerprints determined from the media stream.

2. The method of claim 1, wherein the fingerprints of media content in the signature file include fingerprint sequences of the media content that follow the one or more initial fingerprints.

3. The method of claim 1, wherein the fingerprints of media content in the signature file include audio fingerprint sequences of potentially identified songs that share the one or more initial fingerprints.

4. The method of claim 1, further comprising:

the computing device using a microphone to record one or more samples of the media stream; and

extracting the one or more additional fingerprints from the one or more recorded samples.

5. The method of claim 1, further comprising using the fingerprints of media content in the signature file to identify another media content item within the media stream without intervention by a user.

6. The method of claim 1, further comprising:

when correspondences between the cached fingerprints of media content in the signature file and the one or more additional fingerprints determined from the media stream stop occurring at expected locations, determining a discontinuity of alignment; and

based on determining the discontinuity of alignment, performing a second content identification function of the media stream.

7. The method of claim 1, further comprising extracting the one or more initial fingerprints from the media stream by the computing device.

8. The method of claim 1, wherein the computing device includes a portable device operable according to executable instructions to continuously monitor the media stream and to identify media content items within the media stream.

9. The method of claim 1, wherein the fingerprints in the signature file correspond to candidate media content items matching the one or more initial fingerprints sent by the computing device, and the method further comprises:

the computing device resolving the candidate media content items to identify a match.

10. The method of claim 1, wherein the one or more fingerprints are sent by the computing device and correspond to a sample of the media stream, and the method further comprises:

the computing device determining a time offset corresponding to an offset of a sampling time of the sample within a reference media recording; and

the computing device determining an alignment of the cached fingerprints of media content in the signature file to the one or more additional fingerprints based on the time offset.

11. A method comprising:

receiving from a server into a local cache on a computing device a signature file of fingerprints of media content and corresponding identifiers of the media content, after the computing device sends one or more initial fingerprints determined from a media stream by the computing device, for a continuity tracking function and a content identification function performed by the computing device, wherein the fingerprints in the signature file are for features anticipated to appear in a media stream being monitored by the computing device;

the computing device performing the continuity tracking function using the cached fingerprints of media content in the signature file to track features matching between the media stream and the signature file; and

the computing device switching to the content identification function based on the continuity tracking function determining a discontinuity of features between the media stream and the signature file, in which a transition has occurred between a media content item identified through use of the cached fingerprints in the signature file and a new unidentified media content item of the media stream.

12. The method of claim 11, wherein switching comprises switching between the continuity tracking function, the content identification function, and a second content identification function in which the computing device relies upon a server to act upon fingerprints sent from the computing device which cannot be resolved by the computing device using the cached fingerprints of media content.

13. The method of claim 12, further comprising receiving into the local cache an updated signature file of fingerprints of media content and corresponding updated identifiers of the media content upon a successful identification of one or more media content items for the fingerprints which were not resolved using the cached fingerprints of media content.

14. The method of claim 11, wherein the continuity tracking function includes resolving a single media content item from among a plurality of candidates as matching to the media stream.

15. The method of claim 11, wherein switching between the continuity tracking function and the content identification function occurs without intervention by a user.

16. The method of claim 11, further comprising:

the computing device using a microphone to record one or more samples of the media stream; and

extracting the features of the media stream from the one or more recorded samples.

17. The method of claim 11, wherein the computing device is configured to use the fingerprints of media content in the signature file to identify a given media content item.

18. The method of claim 11, further comprising determining that the transition has occurred between the media content item identified through use of the cached fingerprints in the signature file and the new unidentified media content item based on determination of the discontinuity by the continuity tracking function.

19. The method of claim 18, further comprising identifying a plurality of candidates for the new unidentified media content item via a second content identification function in which the computing device relies upon a server to act upon fingerprints sent from the computing device which cannot be resolved by the computing device using the cached fingerprints of media content.

20. The method of claim 11, wherein the computing device includes a portable device operable according to executable instructions to continuously monitor the media stream and to identify media content items within the media stream.

21. A method executable by a computing device to continuously monitor a media stream and identify media content items within the media stream, the method comprising:

caching a signature file of predetermined fingerprints of media content and corresponding identifiers of the media content from a server after the computing device sends one or more initial fingerprints determined from the media stream by the computing device, wherein the predetermined fingerprints in the signature file are for features anticipated to appear in the media stream at predetermined times; and

the computing device using the cached fingerprints of media content in the signature file to identify a media content item within the media stream based on one or more additional fingerprints determining from the media stream.

22. The method of claim 21, further comprising extracting the one or more initial fingerprints from the media stream by the computing device.