Formant based speech reconstruction from noisy signals

• US 9,015,044 B2
• Filed: 08/20/2012
• Issued: 04/21/2015
• Est. Priority Date: 03/05/2012
• Status: Active Grant

• Current Assignee: Malaspina Labs (Barbados), Inc.
• Original Assignee: Malaspina Labs (Barbados), Inc.
• Inventors: Zakarauskas, Pierre, Escott, Alexander, Stevenson, Shawn E., Chu, Clarence S. H.
• Primary Examiner(s): GUERRA-ERAZO, EDGAR X
• Application Number:
  US13/590,005
• Publication Number:
  US 20130231927A1
• Time in Patent Office: 974 Days
• Field of Search: 704/243, 704/244, 704/246, 704/256, 704/256.7, 704/235, 704/236, 704/240, 704/255, 704/277
• US Class Current: 704/244
• CPC Class Codes: G10L 19/0017 Lossless audio signal coding; Perfect reconstruction of coded audio signal by transmission of coding error G10L19/24 takes precedence
• CPC Class Codes: G10L 19/012 Comfort noise or silence coding
• CPC Class Codes: G10L 2019/0007 Codebook element generation
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information
• CPC Class Codes: G10L 25/75 for modelling vocal tract parameters
• CPC Class Codes: H04R 25/00 Deaf-aid sets , i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception electrical stimulation of auditory nerves to promote the auditory function A61N1/36038; optical stimulation of auditory nerves to promote the auditory function A61N5/0622

Abstract

Implementations of systems, method and devices described herein enable enhancing the intelligibility of a target voice signal included in a noisy audible signal received by a hearing aid device or the like. In particular, in some implementations, systems, methods and devices are operable to generate a machine readable formant based codebook. In some implementations, the method includes determining whether or not a candidate codebook tuple includes a sufficient amount of new information to warrant either adding the candidate codebook tuple to the codebook or using at least a portion of the candidate codebook tuple to update an existing codebook tuple. Additionally and/or alternatively, in some implementations systems, methods and devices are operable to reconstruct a target voice signal by detecting formants in an audible signal, using the detected formants to select codebook tuples, and using the formant information in the selected codebook tuples to reconstruct the target voice signal.

Claims

1. A method of generating a machine readable formant based codebook, the method comprising:

detecting one or more formants in a voice sample, wherein each formant is characterized by a respective spectral location and a respective amplitude value;

generating a candidate codebook tuple for the voice sample, wherein the candidate codebook tuple includes a formant spectrum value and one or more formant amplitude values, wherein the formant spectrum value is indicative of the spectral location of each of the one or more detected formants, and the one or more formant amplitude values are indicative of the corresponding amplitudes of the one or more detected formants; and

selectively adding at least a portion of the candidate codebook tuple to the codebook based at least on whether any portion of the candidate codebook tuple matches a corresponding portion of an existing codebook tuple.

2. The method of claim 1, further comprising accessing a storage medium including the plurality of voice samples to retrieve the voice sample, wherein the plurality of voice samples includes audible frequencies that are within the spectrum associated with human speech.

3. The method of claim 2, wherein a portion of the plurality of voice samples are each characterized an intelligibility value representative of intelligible speech.

4. The method of claim 3, wherein the respective intelligibility values each comprise a speech transmission index value greater than 0.45.

5. The method of claim 2, wherein a portion of the plurality of voice samples each have the same duration, wherein the duration comprises one or more time frames, and formants are detected on a per time frame basis.

6. The method of claim 2, wherein the plurality of voice samples comprises voice samples from a plurality of speakers.

7. The method of claim 1, wherein a respective spectral location of a formant is further characterized by at least one of a corresponding center frequency, a frequency offset and a bandwidth.

8. The method of claim 2, wherein the spectrum associated with human speech includes a plurality of sub-bands, and wherein the formant spectrum value indicates which of the plurality of sub-bands includes the one or more detected formants.

9. The method of claim 8, wherein the formant spectrum value comprises a binary pattern.

10. The method of claim 8, wherein the formant spectrum value comprises an encoded value.

11. The method of claim 8, wherein the plurality of sub-bands is contiguously distributed throughout the spectrum associated with human speech.

12. The method of claim 1, further comprising determining whether the candidate codebook tuple matches an existing codebook tuple by:

comparing the formant spectrum value of the candidate codebook tuple to a respective formant spectrum value of an existing codebook tuple to determine whether the formant spectrum value of the candidate codebook tuple includes a representation of the formants associated with the existing codebook tuple.

13. The method of claim 12, wherein the formant spectrum value of the candidate codebook tuple must at least contain a representation of all of the formants associated with the existing codebook tuple for the candidate codebook tuple to be considered a potential positive match.

14. The method of claim 12, wherein the comparison of the formant spectrum value of the candidate codebook tuple to the respective formant spectrum value of the existing codebook tuple is fault tolerant within a threshold.

15. The method of claim 12, wherein in response to determining that the formant spectrum value of the candidate codebook tuple includes a representation of the formants associated with the existing codebook tuple, the method further comprising:

comparing the one or more formant amplitude values of the candidate codebook tuple to the corresponding one or more formant amplitudes values of the existing codebook tuple to determine whether the candidate codebook tuple and the existing codebook tuple match.

16. The method of claim 15, wherein the candidate codebook tuple matches the existing codebook tuple when each of the one or more formant amplitude values of the candidate codebook tuple matches the corresponding one of the one or more formant amplitude values of the existing codebook tuple within a respective threshold.

17. The method of claim 16, wherein the respective threshold is 10 dB.

18. The method of claim 16, wherein in response to determining that the candidate codebook tuple matches the existing codebook tuple, the method further comprises:

adjusting the one or more formant amplitude values of the existing codebook tuple based at least on the one or more formant amplitude values of the candidate codebook tuple.

19. The method of claim 16, wherein in response to determining that the candidate codebook tuple matches the existing codebook tuple, the method further comprises:

adjusting a respective weight value associated with the existing codebook tuple based at least on the one or more formant amplitude values of the candidate codebook tuple.

20. The method of claim 1, further comprising scaling the respective one or more formant amplitude values for each of the codebook tuples based at least on one or more of the largest formant amplitude values in the codebook values.

21. A formant based codebook generation device comprising:

a formant detection module configured to detect one or more formants in a voice sample, wherein each formant is characterized by a respective spectral location and a respective amplitude value;

a tuple generation module configured to generate a candidate codebook tuple for the voice sample, wherein the candidate codebook tuple includes a formant spectrum value and one or more formant amplitude values, wherein the formant spectrum value is indicative of the spectral location of each of the one or more detected formants, and the one or more formant amplitude values are indicative of the corresponding amplitudes of the one or more detected formants; and

a tuple evaluation module configured to selective add at least a portion of the candidate codebook tuple to the codebook based at least on whether any portion of the candidate codebook tuple matches a corresponding portion of an existing codebook tuple.

22. A formant based codebook generation device comprising:

means for detecting one or more formants in a voice sample, wherein each formant is characterized by a respective spectral location and a respective amplitude value;

means for generating a candidate codebook tuple for the voice sample, wherein the candidate codebook tuple includes a formant spectrum value and one or more formant amplitude values, wherein the formant spectrum value is indicative of the spectral location of each of the one or more detected formants, and the one or more formant amplitude values are indicative of the corresponding amplitudes of the one or more detected formants; and

means for selectively adding at least a portion of the candidate codebook tuple to the codebook based at least on whether any portion of the candidate codebook tuple matches a corresponding portion of an existing codebook tuple.

23. A formant based codebook generation device comprising:

a processor; and

a memory including instructions, that when executed by the processor cause the device to;

detect one or more formants in a voice sample, wherein each formant is characterized by a respective spectral location and a respective amplitude value;

generate a candidate codebook tuple for the voice sample, wherein the candidate codebook tuple includes a formant spectrum value and one or more formant amplitude values, wherein the formant spectrum value is indicative of the spectral location of each of the one or more detected formants, and the one or more formant amplitude values are indicative of the corresponding amplitudes of the one or more detected formants; and

selectively add at least a portion of the candidate codebook tuple to the codebook based at least on whether any portion of the candidate codebook tuple matches a corresponding portion of an existing codebook tuple.