Methods and systems for identifying content in a data stream

• US 9,256,673 B2
• Filed: 06/04/2012
• Issued: 02/09/2016
• Est. Priority Date: 06/10/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Shazam Entertainment Limited (Apple Inc.)
• Inventors: Wang, Avery Li-Chun
• Primary Examiner(s): SALAD, ABDULLAHI ELMI
• Application Number:
  US13/488,083
• Publication Number:
  US 20120317240A1
• Time in Patent Office: 1,345 Days
• Field of Search: 709218-219, 709/231
• US Class Current: 1/1
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID

Abstract

Methods and systems for identifying content in a data stream are provided. In one example, a client device receives a continuous data stream and substantially continuously performs a content identification of content in the data stream based on content patterns stored on the client device. The content patterns stored on the client device may include information associated with extracted features of a media file, or a temporally mapped collection of features describing a media file. The client device may determine whether the continuous data stream includes media content, and based on the determination, continuously perform the content identification of content in the data stream at the client device. The client device may query a server to determine an identity of content in the data stream based on receiving an instruction.

Claims

1. A method for identifying content in a data stream, comprising:

receiving at a client device one or more content patterns of content, wherein the one or more content patterns are selected from a group consisting of content patterns correlated to a user'"'"'s profile, content patterns associated with a location of the client device, content patterns related to content previously identified by the client device or a server, content patterns related to media content stored on the client device, and content patterns selected based on a statistical profile indicating a popularity of pieces of content;

receiving, at the client device, a continuous data stream collected from an ambient environment of the client device;

at the client device, substantially continuously performing a content identification of content in the data stream collected from the ambient environment of the client device based on the one or more content patterns received and stored on the client device, wherein the content patterns include information to identify pieces of content;

based on the content in the data stream collected from the ambient environment of the client device matching to any of the content patterns stored on the client device, the client device providing a notification indicating recognition of the content in the data stream collected from the ambient environment of the client device, wherein the recognition indicates an identity of the content; and

based on receiving an instruction, the client device querying the server to determine an identity of content in the data stream.

2. The method of claim 1, further comprising:

determining whether the continuous data stream collected from the ambient environment of the client device includes media content;

based on the determination that the continuous data stream collected from the ambient environment of the client device includes media content, continuously performing the content identification of content in the data stream at the client device; and

based on the determination that the continuous data stream collected from the ambient environment of the client device includes random ambient noise, the client device not continuously performing the content identification of content in the data stream at the client device.

3. The method of claim 2, further comprising:

if content in the data stream includes media content and is not identified at the client device, the client device providing a notification; and

receiving a user selection of the notification causing the client device to query the server to determine the identity of content in the data stream.

4. The method of claim 1, wherein continuously performing the content identification of content in the data stream comprises performing the content identification without user instruction.

5. The method of claim 1, further comprising:

the client device receiving from the server an instruction to continuously perform the content identification of content in the data stream at the client device.

6. The method of claim 1, wherein the one or more content patterns are selected by a third party.

7. The method of claim 1, further comprising:

the client device identifying content in the data stream; and

based on the identified content, the client device displaying advertisements related to the identified content.

8. The method of claim 1, further comprising:

the client device identifying content in the data stream; and

based on the identified content, the client device displaying synchronized media related to the identified content.

9. The method of claim 1, further comprising:

the client device identifying content in the data stream based on the one or more content patterns received and stored on the client device; and

the client device querying the server to verify the identification.

10. The method of claim 1, wherein the content patterns stored on the client device include information associated with extracted features of a media file.

11. The method of claim 1, wherein the content patterns stored on the client device include a temporally mapped collection of features describing a media file.

12. The method of claim 1, further comprising:

upon receiving, at the client device, the continuous data stream collected from the ambient environment of the client device, the client device initiating the continuous performance of the content identification of the data stream without user input.

13. The method of claim 1, wherein substantially continuously performing a content identification of content in the data stream collected from an ambient environment of the client device based on the one or more content patterns received and stored on the client device comprises:

determining fingerprints of the data stream;

comparing the fingerprints of the data stream to the content patterns received and stored on the client device; and

identifying a given content pattern received and stored on the client device that has a largest number of relative locations of fingerprints matching relative locations of fingerprints of the data stream.

14. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions of:

receiving at the computing device one or more content patterns of content, wherein the one or more content patterns are selected from a group consisting of content patterns correlated to a user'"'"'s profile, content patterns associated with a location of the computing device, content patterns related to content previously identified by the computing device or a server, content patterns related to media content stored on the computing device, and content patterns selected based on a statistical profile indicating a popularity of pieces of content;

receiving, at the computing device, a continuous data stream collected from an ambient environment of the computing device;

at the computing device, continuously performing a content identification of content in the data stream collected from the ambient environment of the computing device based on the one or more content patterns received and stored on the computing device, wherein the content patterns include information to identify pieces of content;

based on the content in the data stream collected from the ambient environment of the computing device matching to any of the content patterns stored on the computing device, the computing device providing a notification indicating recognition of the content in the data stream collected from the ambient environment of the computing device, wherein the recognition indicates an identity of the content; and

based on receiving an instruction, the computing device querying the server to determine an identity of content in the data stream.

15. The non-transitory computer readable medium of claim 14, wherein the instructions are executable to further perform functions of:

determining whether the continuous data stream collected from the ambient environment of the computing device includes media content;

based on the determination that the continuous data stream collected from the ambient environment of the computing device includes media content, continuously performing the content identification of content in the data stream at the computing device; and

based on the determination that the continuous data stream collected from the ambient environment of the computing device includes random ambient noise, the computing device not continuously performing the content identification of content in the data stream at the computing device.

16. The non-transitory computer readable medium of claim 14, wherein the instructions are executable to further perform functions of:

the computing device identifying content in the data stream; and

based on the identified content, the computing device displaying advertisements related to the identified content.

17. A system for identifying content in a data stream, comprising:

a processor; and

data storage storing instructions executable by the processor to perform functions of;

receiving at a computing device one or more content patterns of content, wherein the one or more content patterns are selected from a group consisting of content patterns correlated to a user'"'"'s profile, content patterns associated with a location of the computing device, content patterns related to content previously identified by the computing device or a server, content patterns related to media content stored on the computing device, and content patterns selected based on a statistical profile indicating a popularity of pieces of content;

receiving, at the computing device, a continuous data stream collected from an ambient environment of the computing device;

at the computing device, continuously performing a content identification of content in the data stream collected from the ambient environment of the computing device based on the one or more content patterns received and stored on the computing device, wherein the content patterns include information to identify pieces of content;

based on the content in the data stream collected from the ambient environment of the computing device matching to any of the content patterns stored on the computing device, the computing device providing a notification indicating recognition of the content in the data stream collected from the ambient environment of the computing device, wherein the recognition indicates an identity of the content; and

based on receiving an instruction, the computing device querying the server to determine an identity of content in the data stream.

18. The system of claim 17, wherein the instructions are executable to further perform functions of:

determining whether the continuous data stream collected from the ambient environment of the computing device includes media content;

based on the determination that the continuous data stream collected from the ambient environment of the computing device includes media content, continuously performing the content identification of content in the data stream at the computing device; and

based on the determination that the continuous data stream collected from the ambient environment of the computing device includes random ambient noise, the computing device not continuously performing the content identification of content in the data stream at the computing device.

19. The system of claim 17, wherein the instructions are executable to further perform functions of:

the computing device identifying content in the data stream; and

based on the identified content, the computing device displaying advertisements related to the identified content.