Accoustic context recognition using local binary pattern method and apparatus

• US 10,635,983 B2
• Filed: 04/29/2016
• Issued: 04/28/2020
• Est. Priority Date: 05/12/2015
• Status: Active Grant

• Current Assignee: Goodix Technology Co., Ltd. (Shenzhen Goodix Technology Co., Ltd.)
• Original Assignee: Goodix Technology Co., Ltd. (Shenzhen Goodix Technology Co., Ltd.)
• Inventors: Battaglino, Daniele, Lepauloux, Ludovick, Pilati, Laurent, Evans, Nicholas
• Primary Examiner(s): Louie, Jue
• Application Number:
  US15/141,942
• Publication Number:
  US 20160335553A1
• Time in Patent Office: 1,460 Days
• Field of Search: 704200-278, 706 12, 706 13
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: G10L 2019/0004 Design or structure of the codebook
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

Various exemplary aspects are directed to acoustic context recognition apparatuses and methods involving isolating and identifying context(s) of an acoustic environment. In one exemplary embodiment, source audio is converted into audio spectrograms, each spectrogram indicative of a period of time. The series of spectrograms are analyzed to identify audio patterns, over a period of time, which are indicative of an environmental context of the source audio. In many embodiments of the present disclosure, acoustic context recognition also includes comparing the identified audio patterns to known environmental contexts.

Claims

1. A method comprising:

receiving an audio signal at a microphone that is built in to an electronic device;

constructing an audio signal spectrogram, having adjacent pixels, indicative of the audio received at the microphone;

dividing the audio signal spectrogram into a plurality of blocks;

constructing a plurality of local binary patterns (LBP) based on a comparison of the adjacent pixels of the spectrogram;

creating, for each of the plurality of blocks of the spectrogram, a respective LBP histogram that is based on a number of times different LBPs occur in a corresponding block of the plurality of blocks;

identifying clusters of LBP histograms from the respective LBP histograms;

generating, for each of the clusters, a code word representing a corresponding cluster;

creating a codebook histogram based upon correspondence between the LBP histograms and the code words representing the clusters; and

classifying, using a machine learning model, the codebook histogram to identify environmental context that indicates a location of the electronic device at the time the audio signal was received at the microphone;

wherein the respective LBP histogram, for each of the plurality of blocks of the spectrogram, is indicative of an acoustic context of the audio input over a period of time;

wherein the audio signal spectrogram is a linear spectrum representation of the audio input over a frequency range and wherein each of the plurality of blocks for which an LBP histogram is created represents a particular sub-time over the period of time and a particular sub-frequency range that is within the frequency range of the audio signal.

2. The method of claim 1, wherein the plurality of LBPs are indicative of pixel values, threshold transitions, and respective pixel locations in each of the plurality of blocks of the spectrogram.

3. The method of claim 1, wherein creation of the codebook histogram further includes:

determining a cosine distance between the LBP histograms and the code words using the cosine distance between the LBP histograms and the code words, and

assigning the LBP histograms to code words in the codebook histogram based upon the cosine distance.

4. The method of claim 1, further comprising the step of interpolating, for each of the LBP histograms, by using pixel values outside a respective block for each of the LBP histograms.

5. The method of claim 1, wherein each respective LBP histogram is a histogram of a relationship between each of the plurality of blocks of the spectrogram as defined by a plurality of uniform configurations and a non-uniform configuration.

6. The method of claim 1, wherein the plurality of blocks of the spectrogram are defined by blocks within 0 Hz-900 Hz frequency-band, blocks within 900 Hz-2000 Hz frequency-band, and blocks within 2000 Hz-8000 Hz frequency-band.

7. The method of claim 1, wherein the LBP is defined by an algorithm:

8. The method of claim 1, wherein the electronic device is a cellphone.

9. The method of claim 1, wherein the electronic device is a mobile device.

10. A method comprising the steps of:

receiving an audio signal spectrogram indicative of an audio input;

dividing the audio signal spectrogram into a plurality of blocks;

constructing a plurality of local binary patterns (LBP) based on a comparison of adjacent pixels of the spectrogram;

creating, for each of the plurality of blocks of the spectrogram, a respective LBP histogram that is based on a number of times different LBPs occur in a corresponding block of the plurality of blocks;

identifying clusters of LBP histograms from the respective LBP histograms for each of the plurality of blocks of the spectrogram;

generating, based upon the clusters, a codebook containing a plurality of code words;

mapping the LBP histograms to the code words of the codebook to create a codebook histogram; and

training a machine learning algorithm using the LBP histograms as features therefore to identify environmental context that indicates a location of an electronic device at the time an audio signal is received at a microphone of the electronic device;

wherein the respective LBP histogram, for each of the plurality of blocks of the spectrogram, is indicative of an acoustic context of the audio input over a period of time;

wherein the audio signal spectrogram is a linear spectrum representation of the audio input over a frequency range and wherein each of the plurality of blocks for which an LBP histogram is created represents a particular sub-time over the period of time and a particular sub-frequency range that is within the frequency range of the audio signal.

11. The method of claim 10, further comprising interpolating, for each of the LBP histograms, using pixel values outside a respective block for each of the LBP histograms.

12. The method of claim 10, wherein each respective LBP histogram maps each of the plurality of blocks of the spectrogram into a plurality of uniform configurations and a non-uniform configuration.

13. The method of claim 10, wherein the plurality of blocks of the spectrogram are defined by blocks within 0 Hz-900 Hz frequency-band, blocks within 900 Hz-2000 Hz frequency-band, and blocks within 2000 Hz-8000 Hz frequency-band.

14. The method of claim 10, wherein the machine learning algorithm is within a support vector machine.

15. The method of claim 10, wherein the LBP is defined by an algorithm:

16. The method of claim 10, wherein the electronic device is a cellphone.

17. The method of claim 10, wherein the electronic device is a mobile device.