SYSTEM AND METHOD FOR TRACKING SOUND PITCH ACROSS AN AUDIO SIGNAL USING HARMONIC ENVELOPE

• US 20130041657A1
• Filed: 08/08/2011
• Published: 02/14/2013
• Est. Priority Date: 08/08/2011
• Status: Active Grant

• Current Assignee: Friday Harbor LLC
• Original Assignee: Intellisis Corporation
• Inventors: BRADLEY, David C., Gateau, Rodney, Goldin, Daniel S., Hilton, Robert N., Fisher, Nicholas K.
• Granted Patent: US 8,620,646 B2
• Time in Patent Office: Days
• US Class Current: 704/207
• CPC Class Codes: G10L 2025/906 Pitch tracking
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals
• CPC Class Codes: H04R 29/00 Monitoring arrangements; Testing arrangements for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308

Abstract

A system and method may be configured to analyze audio information derived from an audio signal. The system and method may track sound pitch across the audio signal. The tracking of pitch across the audio signal may take into account change in pitch by determining at individual time sample windows in the signal duration an estimated pitch and a representation of harmonic envelope at the estimated pitch. The estimated pitch and the representation of harmonic envelope may then be implemented to determine an estimated pitch for another time sample window in the signal duration with an enhanced accuracy and/or precision.

Claims

1. A system configured to analyze audio information, the system comprising:

one or more processors configured to execute computer program modules, the modules comprising;

an audio information module configured to obtain audio information derived from an audio signal representing one or more sounds, wherein the audio information includes audio information that corresponds to an audio signal during a first time sample window, and wherein such information includes transformed audio information that specifies a magnitude of an intensity coefficient related to an intensity of the audio signal as a function of frequency during the first time sample window;

an envelope vector module configured to determine, as a function of pitch in the first time sample window, an envelope vector having coordinates, wherein the envelope vector module is configured to determine the envelope vector for a given pitch in the first time sample window based on the values for the intensity coefficient at harmonic frequencies of the given pitch in the first time sample window;

an envelope correlation module configured to obtain an envelope vector for a sound represented by the audio signal during a second time sample window, and to determine, for the first time sample window, a correlation metric as a function of pitch, wherein the correlation metric for a given pitch in the first time sample window indicates a level of correlation between the envelope vector for the second time sample window and the envelope vector for the given pitch in the first time sample window; and

a pitch estimation module configured to determine an estimated pitch for the first time sample window based on the determination of the correlation metric for the first time sample window.

2. The system of claim 1, wherein the audio information corresponding to the first time sample window includes pitch likelihood information that specifies as a function of pitch, a pitch likelihood metric for the first time sample window, and wherein the computer program modules further comprise a weighting module configured to weight the pitch likelihood metric for the first time sample window such that relatively larger weights are applied to the pitch likelihood metric at pitches having correlation metric values in the first time sample window that indicate relatively high correlation with the envelope vector for the second time sample window, and relatively smaller weights are applied to the pitch likelihood metric at pitches having correlation metric values in the first time sample window that indicate relatively low correlation with the envelope vector for the second time sample window.

3. The system of claim 2, wherein the pitch estimation module is configured such that determining the estimated pitch for the first time sample window comprises identifying a pitch in the first time sample window for which the weighted pitch likelihood metric is a maximum.

4. The system of claim 1, wherein the second time sample window is adjacent to the first window of time before or after the first time sample window.

5. The system of claim 1, wherein the audio information corresponding to the first time sample window indicates chirp likelihood as a function of fractional chirp rate, wherein the chirp likelihood for a given fractional chirp rate indicates the likelihood of the sound having the estimated pitch also having the given fractional chirp rate during the first time sample window, and wherein the pitch estimation module is further configured to determine an estimated fractional chirp rate for the first time sample window based on the estimated pitch and the chirp likelihood for the first time sample window.

6. The system of claim 5, wherein the audio information specifies the pitch likelihood metric as a function of pitch and fractional chirp rate such that chirp likelihood is indicated in the audio information for the first time sample window by the specification of the pitch likelihood metric.

7. The system of claim 1, wherein the audio information further includes audio information that corresponds to the audio signal during a third time sample window, wherein the envelope vector module is configured to determine the envelope vector for the third time sample window as a function of pitch, wherein the envelope correlation module is further configured to determine the value of the correlation metric as a function of pitch in the time sample window to indicate correlation with the envelope vector for the estimated pitch of the first time sample window, wherein the weighting module is further configured to weight the pitch likelihood metric for the third time sample window based on the correlation metric, and wherein the pitch estimation module is further configured to determine an estimated pitch for the third time sample window based on the weighted pitch likelihood metric for the third time sample window.

8. A method analyzing audio information, the method comprising:

obtaining audio information derived from an audio signal representing one or more sounds, wherein the audio information includes audio information that corresponds to an audio signal during a first time sample window, and wherein such information includes transformed audio information that specifies a magnitude of an intensity coefficient related to an intensity of the audio signal as a function of frequency during the first time sample window;

determining, as a function of pitch in the first time sample window, an envelope vector having coordinates, wherein determination of the coordinates of the envelope vector for a given pitch in the first time sample window is based on the values for the intensity coefficient at harmonic frequencies of the given pitch in the first time sample window;

obtaining an envelope vector for a sound represented by the audio signal during a second time sample window;

determining, for the first time sample window, a correlation metric as a function of pitch, wherein the correlation metric for a given pitch in the first time sample window indicates a level of correlation between the envelope vector for the second time sample window and the envelope vector for the given pitch in the first time sample window; and

determining an estimated pitch for the first time sample window based on the determination of the correlation metric for the first time sample window.

9. The method of claim 8, wherein the audio information corresponding to the first time sample window includes pitch likelihood information that specifies as a function of pitch, a pitch likelihood metric for the first time sample window, and wherein the method further comprises weighting the pitch likelihood metric for the first time sample window such that relatively larger weights are applied to the pitch likelihood metric at pitches having correlation metric values in the first time sample window that indicate relatively high correlation with the envelope vector for the second time sample window, and relatively smaller weights are applied to the pitch likelihood metric at pitches having correlation metric values in the first time sample window that indicate relatively low correlation with the envelope vector for the second time sample window.

10. The method of claim 9, wherein determining the estimated pitch for the first time sample window comprises identifying a pitch in the first time sample window for which the weighted pitch likelihood metric is a maximum.

11. The method of claim 8, wherein the second time sample window is adjacent to the first window of time before or after the first time sample window.

12. The method of claim 8, wherein the audio information corresponding to the first time sample window indicates chirp likelihood as a function of fractional chirp rate, wherein the chirp likelihood for a given fractional chirp rate indicates the likelihood of the sound having the estimated pitch also having the given fractional chirp rate during the first time sample window, and wherein the method further comprises determining an estimated fractional chirp rate for the first time sample window based on the estimated pitch and the chirp likelihood for the first time sample window.

13. The method of claim 12, wherein the audio information specifies the pitch likelihood metric as a function of pitch and fractional chirp rate such that chirp likelihood is indicated in the audio information for the first time sample window by the specification of the pitch likelihood metric.

15. A system configured to analyze audio information, the system comprising:

one or more processors configured to execute computer program modules, the modules comprising;

an audio information module configured to obtain audio information derived from an audio signal representing one or more sounds, wherein the audio information includes audio information that corresponds to an audio signal during a set of discrete time sample windows, wherein such information includes for the individual time sample windows (i) transformed audio information that specifies a magnitude of an intensity coefficient related to an intensity of the audio signal as a function of frequency during an individual time sample window, and (ii) pitch likelihood information that specifies, as a function of pitch, a pitch likelihood metric for an individual time sample window;

a processing window module configured to define a processing time window within the signal duration, the processing time window including a plurality of time sample windows;

a pitch estimation module configured to determine, for the individual time sample windows in the processing time window, an estimated pitch;

an envelope vector module configured to determine, for the individual time sample windows, an envelope vector as a function of pitch such that the envelope vector for a given pitch in a given time sample window is determined based on the values for the intensity coefficient at harmonic frequencies of the given pitch in the given time sample window,

a primary window module configured to identify a primary time sample window within the processing time window, and to identify a primary pitch within the primary time sample window, wherein the primary pitch is considered to be the estimated pitch for the primary time sample window;

wherein the pitch estimation module is further configured to determine estimated pitch for the time sample windows in the processing time window other than the primary time sample window by iterating through the processing time window from the primary time sample window toward one or both of the boundaries of the processing time window and determining the estimated pitch for a given time sample window based on (i) the pitch likelihood metric specified by the transformed audio information for the given time sample window, and (ii) the envelope vector for an estimated pitch in a time sample window adjacent to the given time sample window.

14. The method of claim 18, wherein the audio information further includes audio information that corresponds to the audio signal during a third time sample window, wherein the method further comprises:

determining the value of the correlation metric as a function of pitch in the time sample window to indicate correlation with the envelope vector for the estimated pitch of the first time sample window;

weighting the pitch likelihood metric for the third time sample window based on the correlation metric; and

determining an estimated pitch for the third time sample window based on the weighted pitch likelihood metric for the third time sample window.

16. The system of claim 15, wherein the envelope vector module is configured such that the coordinates of an envelope vector of a given pitch in a given time sample window are values of the intensity coefficient at harmonic frequencies of the given pitch in the given time sample window.

17. The system of claim 15, wherein the processing window module is further configured to define a plurality of processing time windows within the signal duration, and wherein the pitch estimation module is further configured to determine estimated pitch for time sample windows within the different processing windows separately for the individual processing time windows.

18. The system of claim 17, wherein the processing window module is configured such that the plurality of processing time windows include overlapping processing time windows within at least a portion of the signal duration, thereby resulting in determination of a plurality of estimated pitches for a first time sample window that is within two or more of the overlapping processing time windows.

19. The system of claim 18, wherein the computer program modules further comprise an estimated pitch aggregation module configured to determine an aggregated estimated pitch for the first time sample window by aggregating the plurality of estimated pitches determined for the first time sample window.

20. The system of claim 19, wherein the estimated pitch aggregation module is configured such that determining the aggregated estimated pitch for the first time sample window includes one or more of determining a mean of the estimated pitches for the first time sample window, determining a mean of the estimated pitches for the first time sample window, selecting a most frequently determined estimated pitch for the first time sample window, or determining a weighted average of the estimated pitches for the first time sample window.

21. The system of claim 18, wherein the processing window module is configured to define the processing time windows by incrementing the boundaries of the processing time window over the span of the signal duration.