Efficient apparatus and method for audio signature generation using recognition history

• US 10,278,017 B2
• Filed: 06/30/2017
• Issued: 04/30/2019
• Est. Priority Date: 05/16/2014
• Status: Active Grant

• Current Assignee: Alphonso, Inc. (LG Corporation)
• Original Assignee: Alphonso, Inc. (LG Corporation)
• Inventors: Kalampoukas, Lampros, Chordia, Ashish, Kodige, Raghu Srinivas
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US15/639,007
• Publication Number:
  US 20180062778A1
• Time in Patent Office: 669 Days
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/27 characterised by the analysis technique
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04R 29/004 for microphones H04R29/007 takes precedence
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds

Abstract

Audio information is monitored by a user device that performs audio content recognition of any received audio content. The user device includes a scheduling logic unit, a probe, and an audio signature generator. The scheduling logic unit maintains a set of scheduling rules that define conditions that were present when previous audio content recognition of audio content received by the user device was successful. The scheduling logic unit receives currently present conditions of the user device, and compares the currently present conditions to the set of scheduling rules to determine if the currently present conditions match any scheduling rules. The user devices captures ambient audio content via the probe and generates audio signatures of the captured audio content using the audio signature generator if a match occurs, and inhibits capturing audio content by the user device and/or inhibits generating audio signatures if a match does not occur.

Claims

1. A method for monitoring audio information by a user device, the user device performing audio content recognition of any received audio content, the user device including (i) a scheduling logic unit, (ii) a probe that captures ambient audio content, and (iii) an audio signature generator that generates audio signatures for subsequent use in audio content recognition, the method comprising:

(a) maintaining in the scheduling logic unit a set of reusable scheduling rules that define one or more conditions that were present when previous audio content recognition of audio content received by the user device is determined as being successful, the determination of successful audio content recognition being when an audio signature of the audio content previously received by the user device matches a previously stored audio signature, wherein the previous audio content is audio content of media content that is broadcast to multiple users, and wherein the set of scheduling rules are not associated with scheduled broadcast times of any audio content, and wherein the one or more conditions in the scheduling rules are reusable over time without requiring any subsequent determination of a successful audio content recognition, and wherein the one or more conditions in the scheduling rules are unrelated to any time windows of a timestamp of audio signatures;

(b) receiving in the scheduling logic unit currently present conditions of the user device;

(c) comparing in the scheduling logic unit the currently present conditions of the user device to the set of scheduling rules to determine when the currently present conditions match any scheduling rules; and

(d) capturing ambient audio content by the user device via the probe and generating audio signatures of the captured audio content using the audio signature generator when a match occurs in step (c), and inhibiting capturing audio content by the user device and/or inhibiting generating audio signatures when a match does not occur in step (c),

wherein the inhibiting of capturing audio content and/or the inhibiting of generating audio signatures both inhibit performing audio content recognition because no audio signatures are available to perform the subsequent audio content recognition, and

wherein the captured ambient audio content is audio content of media content that is broadcast to multiple users.

2. The method of claim 1 wherein the user device further comprises (iv) a transmitter connected to the audio signature generator, the method further comprising:

(e) transmitting using the transmitter any generated audio signatures over a communication channel to a remote server for potential identification of the content represented by the audio signature.

3. The method of claim 1 wherein a time delay unit is associated with the scheduling logic unit, the method further comprising:

(f) delaying, by the time delay unit, receiving the currently present conditions of the user device by the scheduling logic unit when a match does not occur in step (c).

4. The method of claim 1 wherein the one or more conditions include environmental conditions.

5. The method of claim 1 wherein the one or more conditions include time of day.

6. The method of claim 1 wherein the one or more conditions include motion of the user device.

7. The method of claim 1 wherein the probe includes a microphone that converts captured audio content to an electrical signal for use by the audio signature generator.

8. The method of claim 1 wherein the user device is a smartphone or tablet.

9. The method of claim 1 wherein the inhibiting interrupts capturing audio content by the user device and/or interrupts generating audio signatures.

10. An apparatus for monitoring audio information and performing audio content recognition of any received audio content, the apparatus comprising:

(a) a probe that captures ambient audio content;

(b) an audio signature generator that generates audio signatures of the captured audio content for subsequent use in audio content recognition; and

(c) a scheduling logic unit configured to;

(i) maintain a set of reusable scheduling rules which define one or more conditions that were present when previous audio content recognition of audio content received by the apparatus is determined as being successful, the determination of successful audio content recognition being when an audio signature of the audio content previously received by the apparatus matches a previously stored audio signature, wherein the previous audio content is audio content of media content that is broadcast to multiple users, and wherein the set of scheduling rules are not associated with scheduled broadcast times of any audio content, and wherein the one or more conditions in the scheduling rules are reusable over time without requiring any subsequent determination of a successful audio content recognition, and wherein the one or more conditions in the scheduling rules are unrelated to any time windows of a timestamp of audio signatures,

(ii) receive currently present conditions of the apparatus,

(iii) compare in the scheduling logic unit the currently present conditions of the apparatus to the set of scheduling rules to determine when the currently present conditions match any scheduling rules,

(iv) output an operation control signal that allows the probe to capture audio content and allows the audio signature generator to generate audio signatures of any captured audio content when a match occurs, and

(v) inhibit the probe from capturing any audio content when a match does not occur and/or inhibit the audio signature generator from generating audio signatures when a match does not occur,

wherein the inhibiting of capturing audio content and/or the inhibiting of generating audio signatures both inhibit performing audio content recognition because no audio signatures are available to perform the subsequent audio content recognition, and

wherein the captured ambient audio content is audio content of media content that is broadcast to multiple users.

11. The apparatus of claim 10 further comprising:

(d) a transmitter connected to the audio signature generator that transmits using the transmitter any generated audio signatures over a communication channel to a remote server for potential identification of the content represented by the audio signature.

12. The apparatus of claim 10 further comprising:

(d) a time delay unit associated with the scheduling logic unit that delays receiving the currently present conditions of the apparatus by the scheduling logic unit when a match does not occur.

13. The apparatus of claim 10 wherein the one or more conditions include environmental conditions.

14. The apparatus of claim 10 wherein the one or more conditions include time of day.

15. The apparatus of claim 10 wherein the one or more conditions include motion of the apparatus.

16. The apparatus of claim 10 wherein the probe includes a microphone that converts captured audio content to an electrical signal for use by the audio signature generator.

17. The apparatus of claim 10 wherein the apparatus is a smartphone or tablet.