Method and device for spectral expansion for an audio signal

• US 10,043,535 B2
• Filed: 01/15/2014
• Issued: 08/07/2018
• Est. Priority Date: 01/15/2013
• Status: Active Grant

• Current Assignee: Staton Techiya, LLC
• Original Assignee: Staton Techiya, LLC
• Inventors: Usher, John, Ellis, Dan
• Primary Examiner(s): Abebe, Daniel
• Application Number:
  US14/155,724
• Publication Number:
  US 20140200883A1
• Time in Patent Office: 1,665 Days
• CPC Class Codes: G10L 21/0388 Details of processing therefor
• CPC Class Codes: H04R 1/028 associated with devices performing functions other than acoustics, e.g. electric candles
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 3/00 Circuits for transducers , loudspeakers or microphones
• CPC Class Codes: H04R 5/033 Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10

Abstract

A method and device for automatically increasing the spectral bandwidth of an audio signal including generating a “mapping” (or “prediction”) matrix based on the analysis of a reference wideband signal and a reference narrowband signal, the mapping matrix being a transformation matrix to predict high frequency energy from a low frequency energy envelope, generating an energy envelope analysis of an input narrowband audio signal, generating a resynthesized noise signal by processing a random noise signal with the mapping matrix and the envelope analysis, high-pass filtering the resynthesized noise signal, and summing the high-pass filtered resynthesized noise signal with the input narrowband audio signal. Other embodiments are disclosed.

Claims

1. A method for automatically expanding a spectral bandwidth of an audio signal comprising the steps of:

a first training step of generating a mapping matrix, by a digital signal processor, the mapping matrix being a transformation matrix to predict high frequency energy from a low frequency energy envelope, the first training step consisting of simultaneously recording a sentence by an ambient microphone and by an ear canal microphone located in an earphone, wherein the ambient microphone captures a reference wideband signal and the digital signal processor performs a frequency transform on the wideband signal, wherein the ear canal microphone captures a reference narrowband signal and the digital signal processor performs a frequency transform on the narrowband signal into a plurality of bands, wherein the mapping matrix is based on an analysis of the frequency transform on the wideband signal and the frequency transform on the narrowband signal into a plurality of bands;

generating, by the digital signal processor, an energy envelope analysis of an input narrowband audio signal;

generating, by the digital signal processor, a resynthesized noise signal by processing a random noise signal with the mapping matrix and the envelope analysis;

high-pass filtering, by the digital signal processor, the resynthesized noise signal; and

summing, by the digital signal processor, the high-pass filtered resynthesized noise signal with the input narrowband audio signal to automatically expand the spectral bandwidth of the input narrowband audio signal to produce a summed signal with spectral enhancement;

reproducing by a loudspeaker the summed signal with spectral enhancement.

2. The method of claim 1, wherein the reference wideband and narrowband signals consist of a simultaneous recording of a phonetically balanced sentence.

3. The method of claim 1, where the input narrowband audio signal is taken from the ear-canal microphone within the occluded ear canal and located in the earphone.

4. The method of claim 1, where the input narrowband audio signal is taken from a received speech audio signal in a speech telecommunications system.

5. The method of claim 1, where the summed signal is directed to a speech telecommunications system.

6. The method of claim 1, where the summed signal is directed to a voice controlled device.

7. The method of claim 1, where the mapping matrix is generated from a least squares fit analysis of the reference wideband and reference narrowband signals.

8. The method of claim 1, where the mapping matrix is generated by a linear regression model, where the input reference wideband signals and reference narrowband signals are first converted to a frequency domain representation using a Fast Fourier Transform, and secondly each frequency band envelope is converted to a decibel domain representation to perform a linear prediction.

9. A non-transitory computer readable medium containing instructions for spectral enhancement, the execution of the instructions by one or more processors of a computer system causing the one or more processors to perform operations comprising:

generating a mapping matrix, the mapping matrix being a transformation matrix to predict high frequency energy from a low frequency energy envelope, generating of the mapping matrix consisting of simultaneously recording of a sentence by an ambient microphone and by an ear canal microphone, wherein the ambient microphone captures a reference wideband signal and the digital signal processor performs a frequency transform on the wideband signal, wherein the ear canal microphone captures a reference narrowband signal and the digital signal processor performs a frequency transform on the narrowband signal into a plurality of bands, wherein the mapping matrix is based on an analysis of the frequency transform on the wideband signal and the frequency transform on the narrowband signal into a plurality of bands;

generating an energy envelope analysis of an input narrowband audio signal;

generating a resynthesized noise signal by processing a random noise signal with the mapping matrix and the envelope analysis;

high-pass filtering the resynthesized noise signal;

summing the high-pass filtered resynthesized noise signal with the input narrowband audio signal to produce a summed signal with spectral enhancement; and

reproducing by a loudspeaker the summed signal with spectral enhancement.

10. A system for automatically expanding the spectral bandwidth of an audio signal comprising:

one or more processors;

a memory having instructions and being operatively coupled to the one or more processors, the instructions when executed by the one or more processors performs the operations of;

generating a mapping matrix, consisting of simultaneously recording of a sentence by an ambient microphone and by an ear canal microphone, wherein the ambient microphone captures a reference wideband signal and the digital signal processor performs a frequency transform on the wideband signal, wherein the ear canal microphone captures a reference narrowband signal and the digital signal processor performs a frequency transform on the narrowband signal into a plurality of bands, wherein the mapping matrix is based on an analysis of the frequency transform on the wideband signal and the frequency transform on the narrowband signal into a plurality of bands;

generating an energy envelope analysis of an input narrowband audio signal;

generating a resynthesized noise signal by processing a random noise signal with the mapping matrix and the envelope analysis to provide a resynthesized noise signal;

high-pass filtering the resynthesized noise signal to provide a high-pass filtered resynthesized noise signal; and

summing the high-pass filtered resynthesized noise signal with the input narrowband audio signal to provide a wideband signal to produce a summed signal with spectral enhancement;

reproducing by a loudspeaker the summed signal with spectral enhancement.

11. The system of claim 10, wherein the mapping matrix is a transformation matrix predicting high frequency energy from a low frequency energy envelope.

12. The system of claim 10, wherein the summing provides the wideband signal as the output to the loudspeaker for reproducing the audio signal with an expanded spectral bandwidth.

13. The system of claim 12, where the mapping matrix is generated from a least squares fit analysis of the reference wideband signal and reference narrowband signal.

14. The system of claim 12, where the mapping matrix is generated by a linear regression model, where the input reference wideband signals and reference narrowband signals are first converted to a frequency domain representation providing frequency band envelopes, and secondly each frequency band envelope is converted to a decibel domain representation.

15. The system of claim 10, wherein the system further includes a smart watch having one or more microphones.

16. The system of claim 10, wherein the system further includes a set of smart glasses having one or more microphones.

17. The system of claim 10, wherein the system further includes a mobile phone having one or more microphones.

18. The system of claim 10, wherein the system further includes a hearing aid having one or more microphones.

19. The system of claim 10, wherein the system further includes steering wheel having one or more microphones.

20. The system of claim 10, wherein the system is formed in an earpiece device.