FREQUENCY BAND EXTENDING DEVICE AND METHOD, ENCODING DEVICE AND METHOD, DECODING DEVICE AND METHOD, AND PROGRAM

• US 20120243526A1
• Filed: 09/29/2010
• Published: 09/27/2012
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yamamoto, Yuki, Chinen, Toru, Honma, Hiroyuki, Mitsufuji, Yuhki
• Granted Patent: US 9,208,795 B2
• Time in Patent Office: Days
• US Class Current: 370/343
• CPC Class Codes: G10L 19/0208 Subband vocoders
• CPC Class Codes: G10L 21/038 using band spreading techniques
• CPC Class Codes: G10L 21/0388 Details of processing therefor

Abstract

The present invention relates to a frequency band extending device and method, an encoding device and method, a decoding device and method, and a program, whereby music signals can be played with higher sound quality due to the extension of frequency bands.

A bandpass filter 13 divides an input signal into multiple sub-band signals, a feature amount calculating circuit 14 calculates feature amount using at least one of the multiple divided sub-band signals and the input signal, a high frequency sub-band power estimating circuit 15 calculates an estimated value of a high frequency sub-band power based on the calculated feature amount, a high frequency signal generating circuit 16 generates a high frequency signal component based on the multiple sub-band signals divided by the bandpass filter 13, and the estimated value of the high frequency sub-band power calculated by the high frequency sub-band power estimating circuit 15. A frequency band extending device 10 extends the frequency band of the input signal using a high frequency signal component. The present invention may be applied to a frequency band extending device, for example.

Claims

1. A frequency band extending device comprising:

signal dividing means configured to divide an input signal into a plurality of sub-band signals;

feature amount calculating means configured to calculate feature amount which expresses a feature of said input signal using at least one of said plurality of sub-band signals divided by said signal dividing means, and said input signal;

high frequency sub-band power estimating means configured to calculate an estimated value of a high frequency sub-band power that is the power of a sub-band signal having a higher frequency band than said input signal based on said feature amount calculated by said feature amount calculating means; and

high frequency signal component generating means configured to generate a high frequency signal component based on said plurality of sub-band signals divided by said signal dividing means, and the estimated value of said high frequency sub-band power calculated by said high frequency sub-band power estimating means;

wherein the frequency band of said input signal is extended using said high frequency signal component generated by said high frequency signal component generating means.

2. The frequency band extending device according to claim 1, wherein said feature amount calculating means calculate a low frequency sub-band power that is a power of said plurality of sub-band signals as said feature amount.

3. The frequency band extending device according to claim 1, wherein said feature amount calculating means calculate a temporal variation of a low frequency sub-band power that is a power of said plurality of sub-band signals as said feature amount.

4. The frequency band extending device according to claim 1, wherein said feature amount calculating means calculate difference between the maximum and minimum powers in a predetermined frequency band, of said input signal, as said feature amount.

5. The frequency band extending device according to claim 1, wherein said feature amount calculating means calculate a temporal variation of difference between the maximum value and minimum value of power in a predetermined frequency band, of said input signal, as said feature amount.

6. The frequency band extending device according to claim 1, wherein said feature amount calculating means calculate the slope of a power in a predetermined frequency band, of said input signal, as said feature amount.

7. The frequency band extending device according to claim 1, wherein said feature amount calculating means calculate a temporal variation of the slope of a power in a predetermined frequency band, of said input signal, as said feature amount.

8. The frequency band extending device according to claim 1, wherein said high frequency sub-band power estimating means calculate of an estimated value of said high frequency sub-band power based on said feature amount, and a coefficient for each high frequency sub-band obtained beforehand by learning.

9. The frequency band extending device according to claim 8, wherein said coefficient for each high frequency sub-band is generated by performing clustering of the residual vector of said high frequency signal component calculated with the coefficient for each high frequency sub-band obtained by regression analysis with a plurality of teacher signals, and performing regression analysis, for each cluster obtained by said clustering, using said teacher signals belonging to said cluster.

10. The frequency band extending device according to claim 9, wherein said residual vector is normalized with the dispersion value of each component of a plurality of said residual vectors, and said vector after normalization is subjected to clustering.

11. The frequency band extending device according to claim 9, wherein said high frequency sub-band power estimating means calculate an estimated value of said high frequency sub-band power based on said feature amount, and the coefficient and constant for each of said high frequency sub-bands;

and wherein said constant is calculated from a center-of-gravity vector for said new clusters obtained by further calculating said residual vector using the coefficient for each high frequency sub-band obtained by regression analysis with said teacher signals belonging to said cluster, and performing clustering of said residual vector thereof to a plurality of new clusters.

12. The frequency band extending device according to claim 11, wherein said high frequency sub-band power estimating means record the coefficient for each of said high frequency sub-bands, and a pointer that determines the coefficient for said each high frequency sub-band, in a correlated manner, and also record a plurality of sets of said pointer and said constant, and some of a plurality of said sets include a pointer having the same value.

13. The frequency band extending device according to claim 1, wherein said high frequency signal generating means generate said high frequency signal component from a low frequency sub-band power that is a power of said plurality of sub-band signals, and an estimated value of said high frequency sub-band power.

14. A frequency band extending method comprising:

a signal dividing step arranged to divide an input signal into a plurality of sub-band signals;

a feature amount calculating step arranged to calculate feature amount which expresses a feature of said input signal using at least one of said plurality of sub-band signals divided by the processing in said signal dividing step, and said input signal;

a high frequency sub-band power estimating step arranged to calculate an estimated value of a high frequency sub-band power that is the power of a sub-band signal having a higher frequency band than said input signal based on said feature amount calculated by the processing in said feature amount calculating step; and

a high frequency signal component generating step arranged to generate a high frequency signal component based on said plurality of sub-band signals divided by the processing in said signal dividing step, and the estimated value of said high frequency sub-band power calculated by the processing in said high frequency sub-band power estimating step;

wherein the frequency band of said input signal is extended using said high frequency signal component generated by the processing in said high frequency signal component generating step.

15. A program comprising:

a signal dividing step arranged to divide an input signal into a plurality of sub-band signals;

a feature amount calculating step arranged to calculate feature amount which expresses a feature of said input signal using at least one of said plurality of sub-band signals divided by the processing in said signal dividing step, and said input signal;

a high frequency sub-band power estimating step arranged to calculate an estimated value of a high frequency sub-band power that is the power of a sub-band signal having a higher frequency band than said input signal based on said feature amount calculated by the processing in said feature amount calculating step; and

a high frequency signal component generating step arranged to generate a high frequency signal component based on said plurality of sub-band signals divided by the processing in said signal dividing step, and the estimated value of said high frequency sub-band power calculated by the processing in said high frequency sub-band power estimating step;

causing a computer to execute processing for extending the frequency band of said input signal using said high frequency signal component generated by the processing in said high frequency signal component generating step.

16. An encoding device comprising:

sub-band dividing means configured to divide an input signal into a plurality of sub-bands, and to generate a low frequency sub-band signal made up of a plurality of sub-bands at a low frequency side and a high frequency sub-band signal made up of a plurality of sub-bands at a high frequency side;

feature amount calculating means configured to calculate feature amount that expresses a feature of said input signal, using at least one of said low frequency sub-band signal generated by said sub-band dividing means, and said input signal;

pseudo high frequency sub-band power calculating means configured to calculate a pseudo high frequency sub-band power that is a pseudo power of said high frequency sub-band signal based on said feature amount calculated by said feature amount calculating means;

pseudo high frequency sub-band power difference calculating means configured to calculate a high frequency sub-band power that is the power of said high frequency sub-band signal from said high frequency sub-band signal generated by said sub-band dividing means, and to calculate pseudo high frequency sub-band power difference that is difference as to said pseudo high frequency sub-band power calculated by said pseudo high frequency sub-band power calculating means;

high frequency encoding means configured to encode said pseudo high frequency sub-band power difference calculated by said pseudo high frequency sub-band power difference calculating means to generate high frequency encoded data;

low frequency encoding means configured to encode a low frequency signal that is a low frequency signal of said input signal to generate low frequency encoded data; and

multiplexing means configured to multiplex said low frequency encoded data generated by said low frequency encoding means, and said high frequency encoded data generated by said high frequency encoding means to obtain an output code string.

17. The encoding device according to claim 16, further comprising:

low frequency decoding means configured to decode said low frequency encoded data generated by said low frequency encoding means to generate a low frequency signal;

wherein said sub-band dividing means generate said low frequency sub-band signal from said low frequency signal generated by said low frequency decoding means.

18. The encoding device according to claim 16, wherein said high frequency encoding means calculate similarity between said pseudo high frequency sub-band power difference, and a representative vector or representative value in predetermined plurality of pseudo high frequency sub-band power difference space to generate an index corresponding to a representative vector or representative value of which the similarity is the maximum, as said high frequency encoded data.

19. The encoding device according to claim 16, wherein said pseudo high frequency sub-band power difference calculating means calculate an evaluated value based on said pseudo high frequency sub-band power of each sub-band, and said high frequency sub-band power for every plurality of coefficients for calculating said pseudo high frequency sub-band power;

and wherein said high frequency encoding means generate an index indicating said coefficient of said evaluated value that is the highest evaluated value, as said high frequency encoded data.

20. The encoding device according to claim 19, wherein said pseudo high frequency sub-band power difference calculating means calculate said evaluated value based on at least any of sum of squares of said pseudo high frequency sub-band power difference of each sub-band, the maximum value of the absolute value of said pseudo high frequency sub-band power of said sub-band, or the mean value of said pseudo high frequency sub-band power difference of each sub-band.

21. The encoding device according to claim 20, wherein said pseudo high frequency sub-band power difference calculating means calculate said evaluated value based on said pseudo high frequency sub-band power difference of different frames.

22. The encoding device according to claim 20, wherein said pseudo high frequency sub-band power difference calculating means calculate said evaluated value using said pseudo high frequency sub-band power difference multiplied by weight that is weight for each sub-band such that the lower frequency side the sub-band is, the greater weight thereof is.

23. The encoding device according to claim 20, wherein said pseudo high frequency sub-band power difference calculating means calculate said evaluated value using said pseudo high frequency sub-band power difference multiplied by weight that is weight for each sub-band such that the greater said high frequency sub-band power of the sub-band is, the greater weight thereof is.

24. An encoding method comprising:

a sub-band dividing step arranged to divide an input signal into a plurality of sub-bands, and to generate a low frequency sub-band signal made up of a plurality of sub-bands at a low frequency side and a high frequency sub-band signal made up of a plurality of sub-bands at a high frequency side;

a feature amount calculating step arranged to calculate feature amount that expresses a feature of said input signal, using at least one of said low frequency sub-band signal generated by the processing in said sub-band dividing step, and said input signal;

a pseudo high frequency sub-band power calculating step arranged to calculate a pseudo high frequency sub-band power that is a pseudo power of said high frequency sub-band signal based on said feature amount calculated by the processing in said feature amount calculating step;

a pseudo high frequency sub-band power difference calculating step arranged to calculate a high frequency sub-band power that is the power of said high frequency sub-band signal from said high frequency sub-band signal generated by the processing in said sub-band dividing step, and to calculate pseudo high frequency sub-band power difference that is difference as to said pseudo high frequency sub-band power calculated by the processing in said pseudo high frequency sub-band power calculating step;

a high frequency encoding step arranged to encode said pseudo high frequency sub-band power difference calculated by the processing in said pseudo high frequency sub-band power difference calculating step to generate high frequency encoded data;

a low frequency encoding step arranged to encode a low frequency signal that is a low frequency signal of said input signal to generate low frequency encoded data; and

a multiplexing step arranged to multiplex said low frequency encoded data generated by the processing in said low frequency encoding step, and said high frequency encoded data generated by the processing in said high frequency encoding step to obtain an output code string.

25. A program causing a computer to execute processing comprising:

a sub-band dividing step arranged to divide an input signal into a plurality of sub-bands, and to generate a low frequency sub-band signal made up of a plurality of sub-bands at a low frequency side and a high frequency sub-band signal made up of a plurality of sub-bands at a high frequency side;

a feature amount calculating step arranged to calculate feature amount that expresses a feature of said input signal, using at least one of said low frequency sub-band signal generated by the processing in said sub-band dividing step, and said input signal;

a pseudo high frequency sub-band power calculating step arranged to calculate a pseudo high frequency sub-band power that is a pseudo power of said high frequency sub-band signal based on said feature amount calculated by the processing in said feature amount calculating step;

a pseudo high frequency sub-band power difference calculating step arranged to calculate a high frequency sub-band power that is the power of said high frequency sub-band signal from said high frequency sub-band signal generated by the processing in said sub-band dividing step, and to calculate pseudo high frequency sub-band power difference that is difference as to said pseudo high frequency sub-band power calculated by the processing in said pseudo high frequency sub-band power calculating step;

a high frequency encoding step arranged to encode said pseudo high frequency sub-band power difference calculated by the processing in said pseudo high frequency sub-band power difference calculating step to generate high frequency encoded data;

a low frequency encoding step arranged to encode a low frequency signal that is a low frequency signal of said input signal to generate low frequency encoded data; and

a multiplexing step arranged to multiplex said low frequency encoded data generated by the processing in said low frequency encoding step, and said high frequency encoded data generated by the processing in said high frequency encoding step to obtain an output code string.

26. A decoding device comprising:

demultiplexing means configured to demultiplex input encoded data into at least low frequency encoded data and an index;

low frequency decoding means configured to decode said low frequency encoded data to generate a low frequency signal;

sub-band dividing means configured to divide the band of said low frequency signal into a plurality of low frequency sub-bands to generate a low frequency sub-band signal for each of said low frequency sub-bands; and

generating means configured to generate said high frequency signal based on said index and said low frequency sub-band signal.

27. The decoding device according to claim 26, wherein said index is obtained, at a device which encodes an input signal and outputs said encoded data, based on said input signal before encoding, and said high frequency signal estimated from said input signal.

28. The decoding device according to claim 26, wherein said index has not been encoded.

29. The decoding device according to claim 26, wherein said index is information indicating an estimating coefficient used for generation of said high frequency signal.

30. The decoding device according to claim 29, wherein said generating means generate said high frequency signal based on, of a plurality of said estimating coefficients, said estimating coefficient indicated by said index.

31. The decoding device according to claim 29, said generating means comprising:

feature amount calculating means configured to calculate feature amount that expresses a feature of said encoded data using at least one of said low frequency sub-band signal and said low frequency signal;

high frequency sub-band power calculating means configured to calculate a high frequency sub-band power of a high frequency sub-band signal of said high frequency sub-band by calculation using said feature amount and said estimating coefficient regarding each of a plurality of high frequency sub-bands making up the band of said high frequency signal; and

high frequency signal generating means configured to generate said high frequency signal based on said high frequency sub-band power and said low frequency sub-band signal.

32. The decoding device according to claim 31, wherein said high frequency sub-band power calculating means calculate said high frequency sub-band power of said high frequency sub-band by linearly combining a plurality of said feature amount using said estimating coefficient prepared for each of said high frequency sub-bands.

33. The decoding device according to claim 32, wherein said feature amount calculating means calculate a low frequency sub-band power of said low frequency sub-and signal for each of said low frequency sub-bands as said feature amount.

34. The decoding device according to claim 31, wherein said index is information indicating said estimating coefficient whereby said high frequency sub-band power most approximate to said high frequency sub-band power obtained from said high frequency signal of said input signal before encoding is obtained as a result of comparison between said high frequency sub-band power obtained from said high frequency signal of said input signal before encoding and said high frequency sub-band power generated based on said estimating coefficient of a plurality of said estimating coefficients.

35. The decoding device according to claim 34, wherein said index is information indicating said estimating coefficient whereby the sum of squares of difference between said high frequency sub-band power obtained from said high frequency signal of said input signal before encoding, and said high frequency sub-band power generated based on said estimating coefficient obtained for each of said high frequency sub-bands, becomes the minimum.

36. The decoding device according to claim 34, wherein said encoded data further includes difference information indicating difference between said high frequency sub-band power obtained from said high frequency signal of said input signal before encoding, and said high frequency sub-band power generated based on said estimating coefficient.

37. The decoding device according to claim 36, wherein said difference information has been encoded.

38. The decoding device according to claim 36, wherein said high frequency sub-band power calculating means add said difference indicated with said difference information included in said encoded data to said high frequency sub-band power obtained by calculation using said feature amount and said estimating coefficient;

and wherein said high frequency signal generating means generate said high frequency signal based on said high frequency sub-band power to which said difference has been added, and said low frequency sub-band signal.

39. The decoding device according to claim 31, wherein said estimating coefficient is obtained by regression analysis using the least square method with said feature amount as an explanatory variable and said high frequency sub-band power as an explained variable.

40. The decoding device according to claim 31, further comprising, with said index being information indicating a difference vector made up of said difference for each of said high frequency sub-bands wherein difference between said high frequency sub-band power obtained from said high frequency signal of said input signal before encoding, and said high frequency sub-band power generated based on said estimating coefficient as an element:

coefficient output means configured to obtain distance between a representative vector or representative value in feature space of said difference with said difference of said high frequency sub-bands as an element, obtained beforehand for each of said estimating coefficients, and said difference vector indicated by said index, and to supply said estimating coefficient of said representative vector or said representative value whereby said distance is the shortest, of a plurality of said estimating coefficients, to said high frequency sub-band power calculating means.

41. The decoding device according to claim 29, wherein said index is information indicating said estimating coefficient of a plurality of said estimating coefficients whereby as a result of comparison between said high frequency signal of said input signal before encoding, and said high frequency signal generated based on said estimating coefficient, said high frequency signal most approximate to said high frequency signal of said input signal before encoding is obtained.

42. The decoding device according to claim 29, wherein said estimating coefficient is obtained by regression analysis.

43. The decoding device according to claim 26, wherein said generating means generate said high frequency signal based on information obtained by decoding said encoded index.

44. The decoding device according to claim 43, wherein said index has been subjected to entropy encoding.

45. A decoding method comprising:

a demultiplexing step arranged to demultiplex input encoded data into at least low frequency encoded data and an index;

a low frequency decoding step arranged to decode said low frequency encoded data to generate a low frequency signal;

a sub-band dividing step arranged to divide the band of said low frequency signal into a plurality of low frequency sub-bands to generate a low frequency sub-band signal for each of said low frequency sub-bands; and

a generating step arranged to generate said high frequency signal based on said index and said low frequency sub-band signal.

46. A program causing a computer to execute processing comprising:

a demultiplexing step arranged to demultiplex input encoded data into at least low frequency encoded data and an index;

a low frequency decoding step arranged to decode said low frequency encoded data to generate a low frequency signal;

a sub-band dividing step arranged to divide the band of said low frequency signal into a plurality of low frequency sub-bands to generate a low frequency sub-band signal for each of said low frequency sub-bands; and

a generating step arranged to generate said high frequency signal based on said index and said low frequency sub-band signal.

47. A decoding device comprising:

demultiplexing means configured to demultiplex input encoded data into low frequency encoded data and an index for obtaining an estimating coefficient used for generation of a high frequency signal;

low frequency decoding means configured to decode said low frequency encoded data to generate a low frequency signal;

sub-band dividing means configured to divide the band of said low frequency signal into a plurality of low frequency sub-bands to generate a low frequency sub-band signal for each of said low frequency sub-bands;

feature amount calculating means configured to calculate feature amount that expresses a feature of said encoded data using at least one of said low frequency sub-band signal and said low frequency signal;

high frequency sub-band power calculating means configured to calculate a high frequency sub-band power of the high frequency sub-band signal of said high frequency sub-band by multiplexing said feature amount by said estimating coefficient determined by said index of a plurality of said estimating coefficients prepared beforehand regarding each of a plurality of high frequency sub-bands making up the band of said high frequency signal, and obtaining the sum of said feature amount by which said estimating coefficient has been multiplied; and

high frequency signal generating means configured to generate said high frequency signal using said high frequency sub-band power and said low frequency sub-band signal.

48. The decoding device according to claim 47, wherein said feature amount calculating means calculate a low frequency sub-band power of said low frequency sub-band signal for each of said low frequency sub-bands as said feature amount.

49. The decoding device according to claim 48, wherein said index is information for obtaining said estimating coefficient of said plurality of said estimating coefficients whereby the sum of squares of difference obtained for each of said high frequency sub-bands, which is difference between said high frequency sub-band power obtained from the true value of said high frequency signal, and said high frequency sub-band power generated with said estimating coefficient, becomes the minimum.

50. The decoding device according to claim 49, wherein said index further include difference information indicating difference between said high frequency sub-band power obtained from said true value, and said high frequency sub-band power generated with said estimating coefficient;

and wherein said high frequency sub-band power calculating means further add said difference indicated by said difference information included in said index to said high frequency sub-band power obtained by obtaining the sum of said feature amount by which said estimating coefficient has been multiplied;

and wherein said high frequency signal generating means generate said high frequency signal using said high frequency sub-band power to which said difference has been added by said high frequency sub-band power calculating means, and said low frequency sub-band signal.

51. The decoding device according to claim 47, wherein said index is information indicating said estimating coefficient.

52. The decoding device according to claim 47, wherein said index is information obtained by information indicating said estimating coefficient being subjected to entropy encoding;

and wherein said high frequency sub-band power calculating means calculate said high frequency sub-band power using said estimating coefficient indicated by information obtained by decoding said index.

53. The decoding device according to claim 47, wherein said plurality of said estimating coefficients are obtained beforehand by regression analysis using the least square method with said feature amount as an explanatory variable and said high frequency sub-band power as an explained variable.

54. The decoding device according to claim 47, further comprising, with said index being information indicating a difference vector made up of said difference for each of said high frequency sub-bands wherein difference between said high frequency sub-band power obtained from the true value of said high frequency signal, and said high frequency sub-band power generated with said estimating coefficient as an element:

coefficient output means configured to obtain distance between a representative vector or representative value in feature space of said difference with said difference of said high frequency sub-bands as an element, obtained beforehand for each of said estimating coefficients, and said difference vector indicated by said index, and to supply said estimating coefficient of said representative vector or said representative value whereby said distance is the shortest, of a plurality of said estimating coefficients, to said high frequency sub-band power calculating means.

55. A decoding method comprising:

a demultiplexing step arranged to demultiplex input encoded data into low frequency encoded data and an index for obtaining an estimating coefficient used for generation of a high frequency signal;

a low frequency decoding step arranged to decode said low frequency encoded data to generate a low frequency signal;

a sub-band dividing step arranged to divide the band of said low frequency signal into a plurality of low frequency sub-bands to generate a low frequency sub-band signal for each of said low frequency sub-bands;

a feature amount calculating step arranged to calculate feature amount that expresses a feature of said encoded data using at least one of said low frequency sub-band signal and said low frequency signal;

a high frequency sub-band power calculating step arranged to calculate a high frequency sub-band power of the high frequency sub-band signal of said high frequency sub-band by multiplexing said feature amount by said estimating coefficient determined by said index of a plurality of said estimating coefficients prepared beforehand regarding each of a plurality of high frequency sub-bands making up the band of said high frequency signal, and obtaining the sum of said feature amount by which said estimating coefficient has been multiplied; and

a high frequency signal generating step arranged to generate said high frequency signal using said high frequency sub-band power and said low frequency sub-band signal.

56. A program causing a computer to execute processing comprising:

a demultiplexing step arranged to demultiplex input encoded data into low frequency encoded data and an index for obtaining an estimating coefficient used for generation of a high frequency signal;

a low frequency decoding step arranged to decode said low frequency encoded data to generate a low frequency signal;

a sub-band dividing step arranged to divide the band of said low frequency signal into a plurality of low frequency sub-bands to generate a low frequency sub-band signal for each of said low frequency sub-bands;

a feature amount calculating step arranged to calculate feature amount that expresses a feature of said encoded data using at least one of said low frequency sub-band signal and said low frequency signal;

a high frequency sub-band power calculating step arranged to calculate a high frequency sub-band power of the high frequency sub-band signal of said high frequency sub-band by multiplexing said feature amount by said estimating coefficient determined by said index of a plurality of said estimating coefficients prepared beforehand regarding each of a plurality of high frequency sub-bands making up the band of said high frequency signal, and obtaining the sum of said feature amount by which said estimating coefficient has been multiplied; and

a high frequency signal generating step arranged to generate said high frequency signal using said high frequency sub-band power and said low frequency sub-band signal.