Harmonicity-dependent controlling of a harmonic filter tool

• US 10,679,638 B2
• Filed: 08/30/2018
• Issued: 06/09/2020
• Est. Priority Date: 07/28/2014
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Markovic, Goran, Helmrich, Christian, Ravelli, Emmanuel, Jander, Manuel, Doehla, Stefan
• Primary Examiner(s): Abebe, Daniel
• Application Number:
  US16/118,316
• Publication Number:
  US 20190057710A1
• Time in Patent Office: 649 Days
• Field of Search: 704205
• CPC Class Codes: G10L 19/025 Detection of transients or attacks for time/frequency resolution switching
• CPC Class Codes: G10L 19/028 Noise substitution, i.e. substituting non-tonal spectral components by noisy source comfort noise for discontinuous speech transmission G10L19/012
• CPC Class Codes: G10L 19/12 the excitation function being a code excitation, e.g. in code excited linear prediction [CELP] vocoders
• CPC Class Codes: G10L 19/22 Mode decision, i.e. based on audio signal content versus external parameters
• CPC Class Codes: G10L 19/26 Pre-filtering or post-filtering
• CPC Class Codes: G10L 19/265 Pre-filtering, e.g. high frequency emphasis prior to encoding
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

The coding efficiency of an audio codec using a controllable—switchable or even adjustable—harmonic filter tool is improved by performing the harmonicity-dependent controlling of this tool using a temporal structure measure in addition to a measure of harmonicity in order to control the harmonic filter tool. In particular, the temporal structure of the audio signal is evaluated in a manner which depends on the pitch. This enables to achieve a situation-adapted control of the harmonic filter tool so that in situations where a control made solely based on the measure of harmonicity would decide against or reduce the usage of this tool, although using the harmonic filter tool would, in that situation, increase the coding efficiency, the harmonic filter tool is applied, while in other situations where the harmonic filter tool may be inefficient or even destructive, the control reduces the appliance of the harmonic filter tool appropriately.

Claims

1. An apparatus for performing a harmonicity-dependent controlling of a harmonic filter tool of an audio codec, comprising

a harmonicity measurer configured to determine a measure of harmonicity of the audio signal,

a temporal structure analyzer configured to determine, depending on the pitch, at least one temporal structure measure measuring a characteristic of a temporal structure of the audio signal;

a controller configured to control the harmonic filter tool depending on the temporal structure measure and the measure of harmonicity.

2. The apparatus according to claim 1, wherein the harmonicity measurer is configured to determine the measure of harmonicity by computing a normalized correlation of the audio signal or a pre-modified version thereof at or around a pitch-lag of the audio signal.

3. The apparatus according to claim 1, further comprising a pitch estimator configured to determine a pitch of the audio signal.

4. The apparatus according to claim 3, wherein the pitch estimator is configured to, within a first stage, determine a preliminary estimation of the pitch at a down-sampled domain of a first sample rate and, within a second stage, refine the preliminary estimation of the pitch at a second sample rate, higher than the first sample rate.

5. The apparatus according to claim 3, wherein the pitch estimator is configured to determine the pitch using autocorrelation.

6. The apparatus according to claim 3, wherein the temporal structure analyzer is configured to determine the at least one temporal structure measure within a temporal region temporally placed depending on the pitch.

7. The apparatus according to claim 6, wherein the temporal structure analyzer is configured to position a temporally past-heading end of the temporal region, or of a region of higher influence onto the determination of the temporal structure measure, depending on the pitch.

8. The apparatus according to claim 3, wherein the temporal structure analyzer is configured to position the temporal past-heading end of the temporal region or, of the region of higher influence onto the determination of the temporal structure measure, such that the temporally past-heading end of the temporal region or, of the region of higher influence onto the determination of the temporal structure measure, is displaced into past direction by a temporal amount monotonically increasing with a decrease of the pitch.

9. The apparatus according to claim 7, wherein the temporal structure analyzer is configured to position a temporally future-heading end of the temporal region or, of the region of higher influence onto the determination of the temporal structure measure, depending on the temporal structure of the audio signal within a temporal candidate region extending from the temporally past-heading end of the temporal region, or of the region of higher influence onto the determination of the temporal structure measure, to a temporally future-heading end of a current frame.

10. The apparatus according to claim 9, wherein the temporal structure analyzer is configured to use an amplitude or ratio between maximum and minimum energy samples within the temporal candidate region in order to position the temporally future-heading end of the temporal region or, of the region of higher influence onto the determination of the temporal structure measure.

11. The apparatus according to claim 1, wherein the controller comprises

a logic configured to check whether a predetermined condition is met by the at least one temporal structure measure and the measure of harmonicity so as to achieve a check result;

and

a switch configured to switch between enabling and disabling the harmonic filter tool depending on the check result.

12. The apparatus according to claim 11, wherein the at least one temporal structure measure measures an average or maximum energy variation of the audio signal within the temporal region and the logic is configured such that the predetermined condition is met if

both the at least one temporal structure measure is smaller than a predetermined first threshold and the measure of harmonicity is, for a current frame and/or a previous frame, above a second threshold.

13. The apparatus according to claim 12, wherein the logic is configured such that the predetermined condition is also met if

the measure of harmonicity is, for a current frame, above a third threshold, and the measure of harmonicity is, for a current frame and/or a previous frame, above a fourth threshold which decreases with an increase of a pitch lag of the audio signal.

14. The apparatus according to claim 1, wherein the controller is configured to control the harmonic filter tool by

explicitly signaling a control signal via an audio codec'"'"'s data stream to a decoding side;

or

explicitly signaling a control signal via an audio codec'"'"'s data stream to a decoding side for controlling a post-filter at the decoding side and, in line with the control of the post-filter at the decoding side, controlling a pre-filter at an encoder side.

15. The apparatus according to claim 1, wherein the temporal structure analyzer is configured to determine the at least one temporal structure measure in a spectrally discriminating manner so as to acquire one value of the at least one temporal structure measure per spectral band of a plurality of spectral bands.

16. The apparatus according to claim 1, wherein the controller is configured to control the harmonic filter tool at units of frames, and the temporal structure analyzer is configured to sample an energy of the audio signal at a sample rate higher than a frame rate of the frames so as to acquire energy samples of the audio signal and to determine the at least one temporal structure measure on the basis of the energy samples.

17. The apparatus according to claim 16, wherein the temporal structure analyzer is configured to determine the at least one temporal structure measure within a temporal region temporally placed depending on a pitch of the audio signal and the temporal structure analyzer is configured to determine the at least one temporal structure measure on the basis of the energy samples by computing a set of energy change values measuring a change between pairs of immediately consecutive energy samples of the energy samples within the temporal region and subjecting the set of energy change values to a scalar function comprising a maximum operator or a sum over addends each of which depends on exactly one of the set of energy change values.

18. The apparatus according to claim 16, wherein the temporal spectrum analyzer is configured to perform the sampling of the energy of the audio signal within a high-pass filtered domain.

19. The apparatus according to claim 3, wherein the pitch estimator, the harmonicity measurer and the temporal structure analyzer perform its determination based on different versions of the audio signal comprising the original audio signal and some pre-modified version thereof.

20. The apparatus according to claim 1, wherein the controller is configured to, in controlling the harmonic filter tool, depending on the temporal structure measure and the measure of harmonicity

switch between enabling and disabling a pre-filter and/or a post-filter of the harmonic filter tool, or

gradually adapt a filter strength of the pre-filter and/or the post-filter of the harmonic filter tool,

wherein the harmonic filter tool is of a pre-filter plus post-filter approach and the pre-filter of the harmonic filter tool is configured to increase the quantization noise within a harmonic of a pitch of the audio signal and the post-filter of the harmonic filter tool is configured to reshape a transmitted spectrum accordingly, or the harmonic filter tool is of a post-filter only approach and the post-filter of the harmonic filter tool is configured to filter quantization noise occurring between the harmonics of the pitch of the audio signal.

21. An audio encoder or audio decoder, comprising a harmonic filter tool and the apparatus for performing a harmonicity-dependent controlling of the harmonic filter tool according to claim 1.

22. A system comprising

an apparatus for performing a harmonicity-dependent controlling of a harmonic filter tool according to claim 16, and

a transient detector configured to detect transients in an audio signal to be processed by the audio codec on the basis of the energy samples.

23. A transform-based encoder comprising the system of claim 22, configured to switch a transform block and/or overlap length depending on the detected transients.

24. An audio encoder comprising the system of claim 22, configured to support switching between a transform coded excitation mode and a code excited linear prediction mode depending on the detected transients.

25. The audio encoder according to claim 24, configured to switch a transform block and/or overlap length in the transform coded excitation mode depending on the detected transients.

26. A method for performing a harmonicity-dependent controlling of a harmonic filter tool of an audio codec, comprising

determining a measure of harmonicity of the audio signal;

determining, depending on the pitch, at least one temporal structure measure measuring a characteristic of a temporal structure of the audio signal;

controlling the harmonic filter tool depending on the temporal structure measure and the measure of harmonicity.

27. A non-transitory digital storage medium having a computer program stored thereon to perform a method for performing a harmonicity-dependent controlling of a harmonic filter tool of an audio codec, the method comprising:

determining a measure of harmonicity of the audio signal;

determining, depending on the pitch, at least one temporal structure measure measuring a characteristic of a temporal structure of the audio signal;

controlling the harmonic filter tool depending on the temporal structure measure and the measure of harmonicity;

when said computer program is run by a computer.