Vector quantizer

• US 5,692,100 A
• Filed: 02/01/1995
• Issued: 11/25/1997
• Est. Priority Date: 02/02/1994
• Status: Expired due to Term

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Nakahashi, Junichi, Tsuboka, Eiichi
• Primary Examiner(s): Olms, Douglas W.
• Assistant Examiner(s): BLUM, RUSSELL
• Application Number:
  US08/382,753
• Time in Patent Office: 1,028 Days
• Field of Search: 395/2.1, 395/2.3, 395/2.31, 395/2.33, 395/2.4, 395/2.43, 395/2.45, 395/2.47, 395/2.48, 395/2.52, 395/2.53, 395/2.54, 395/2.55
• US Class Current: 704/222
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 2019/0002 Codebook adaptations

Abstract

A codebook for vector quantization is speaker-adapted or a speaker is normalized so that it complies with the codebook using a small number of samples for learning. A deviation vector is set for the centroid of each cluster or an input vector. The deviation vector is set so that a separately defined objective function is maximized if it is defined to be maximized, or minimized if it is defined to be minimized. The maximization or minimization is performed using samples for learning obtained from a speaker who uses the system when the centroid or input vector is moved by an amount corresponding to the deviation vector. By moving the centroid or input vector using the deviation vector, speaker adaptation is performed if the former is moved and speaker normalization is performed if the latter is moved.

Claims

1. A vector quantizer comprising:

a reference codebook for storing several representative vectors in a feature vector space so that they can be retrieved using labels corresponding thereto;

a learning vector storing means for storing several vectors for learning;

an objective function calculating means for calculating an objective function defined as a function of said representative vectors and said vectors for learning;

a deviation vector calculating means for calculating deviation vectors; and

an adaptation means for obtaining new representative vectors by adding said deviation vectors to said representative vectors, wherein;

input vectors are encoded by converting the input vectors into labels or membership vectors whose components are the membership values of said input vector for the labels using said new representative vectors; and

said deviation vector calculating means calculates so that said new representative vectors maximize said objective function relative to said vector for learning.

5. The vector quantizer according to claim 1 or claim 2, wherein said deviation vector calculating means obtains a deviation vector for each representative vector.

6. The vector quantizer according to claim 1 or claim 2, wherein said deviation vector calculating means obtains a common deviation vector for all the representative vectors.

7. The vector quantizer according to claim 1, comprising an adder for adding a deviation vector and each of the representative vectors in the reference codebook wherein vector quantization is performed based on the output of said adder.

8. The vector quantizer according to claim 1, comprising an adder for adding a deviation vector and each of the representative vectors in the reference codebook and an adaptation codebook for storing the output of said adder wherein vector quantization is performed based on the output of said adaptation codebook.

9. A signal transmitter comprising a label transmitting means for transmitting labels encoded by the vector quantizer according to claim 7 or claim 8 and a deviation vector transmitting means.

10. A signal transmitter comprising a label transmitting means for transmitting labels encoded by the vector quantizer according to claim 8 and an adaptation codebook transmitting means for transmitting an adaptation codebook.

11. A signal receiver comprising a deviation vector storing portion for storing deviation vectors transmitted by the signal transmitter according to claim 9, a reference codebook, an adder for adding a representative vector read out from said reference codebook associated with a label which has been received and a deviation vector read out from said deviation vector storing portion in accordance with said label, and a decoder for providing the output of said adder as a decoded vector for said label.

12. A signal receiver comprising a deviation vector storing portion for storing deviation vectors transmitted by the signal transmitter according to claim 9, a reference codebook, an adaptation codebook for storing the sums of representative vectors associate with labels in said reference codebook and said deviation vectors read out from said deviation vector storing portion in accordance with the labels, and a decoder for providing a code vector in said adaptation codebook associated with a label which has been received as a decoded vector for said label.

13. A signal receiver comprising an adaptation codebook storing portion for storing an adaptation codebook transmitted by the signal transmitter according to claim 10 and a decoder for providing a code vector in said adaptation codebook associated with a label which has been received as a decoded vector for said label.

14. A recognition device comprising the vector quantizer according to claim 1 for performing vector quantization on each of a series of feature vectors which has been input, for converting them into labels, and for consequently converting said series of feature vectors into a series of labels, an HMM storing means for storing a hidden Markov model for each unit for recognition wherein the probability of the occurrence of each label is defined for each state, and a likelihood calculating means for calculating the likelihood of each of said HMMs for said series of labels, the unit for recognition associated with the HMM that gives the maximum value of the likelihood being regarded as the result of recognition.

15. A recognition device comprising the vector quantizer according to claim 1 for performing vector quantization on each of a series of feature vectors which has been input, for converting them into membership vectors whose components are membership values of the vectors for individual labels, and for consequently converting said series of feature vectors into a series of said membership vectors, an HMM storing means for storing a hidden Markov model for each unit for recognition wherein the probability of the occurrence of each label is defined for each state, and a likelihood calculating means for calculating the likelihood of each of said HMMs for said series of membership vectors, the unit for recognition associated with the HMM that gives the maximum value of the likelihood being regarded as the result of recognition.

18. A recognition device comprising the vector quantizer according to claim 1 for performing vector quantization on each of a series of feature vectors which has been input, for converting them into labels, and for consequently converting said series of feature vectors into a series of labels, a recognition model storing means for storing a recognition model represented by a series of labels for each unit for recognition, and a distance calculating means for calculating the distance or similarity between said input series of labels and each of said recognition models, the unit for recognition associated with the recognition model that gives the minimum value of said distance or the maximum value of said similarity being regarded as the result of recognition.

19. A recognition device comprising the vector quantizer according to claim 1 for performing vector quantization on each of a series of feature vectors which has been input, for converting them into membership vectors whose components are membership values of the vectors for individual labels, and for consequently converting said series of feature vectors into a series of said membership vectors, a recognition model storing means for storing a recognition model represented by a series of labels for each unit for recognition, and a distance calculating means for calculating the distance or-similarity between said input series of membership values and each of said recognition models, the unit for recognition associated with the recognition model that gives the minimum value of said distance or the maximum value of said similarity being regarded as the result of recognition.

29. The vector quantizer according to claim 1, comprising a temporary storage means for serially storing predetermined constant signal segments of an input signal in real time wherein serial adaptation of a code book or a deviation vector is performed for each of said signal segments using the contents of said temporary storing means as vectors for learning.

31. The vector quantizer according to claim 1, comprising a multiplier and attenuation coefficient storing means for storing an attenuation coefficient which is set to attenuate an effect of a past input voice, wherein

the deviation vector is calculated so as to make an effect of the input voice near a present time large by multiplying the accumulated product of short time deviation vector and the sum of membership values and the accumulated sum of membership values vector with the attenuation coefficient, and thereby to adapt the code book or the deviation vector.

2. A vector quantizer comprising:

a reference codebook for storing several representative vectors in a feature vector space so that they can be retrieved using labels corresponding thereto;

a learning vector storing means for storing several vectors for learning;

an objective function calculating means for calculating an objective function defined as a function of said representative vectors and said vectors for learning;

a deviation vector calculating means for calculating a deviation vector; and

a normalization means for adding said deviation vector to input vectors, wherein;

the input vectors are encoded by adding the deviation vectors to the input vectors to obtain normalized input vectors and by converting them into labels or membership vectors whose components are the membership values of said input vectors for the labels; and

said deviation vector calculating means calculates so that said objective function is maximized when the sums of said vectors for learning and said deviation vector are placed in said reference codebook as new representative vectors.

20. The vector quantizer according to claim 2, comprising an adder for adding a deviation vector and input vectors wherein vector quantization is performed based on the output of said adder.

21. A signal transmitter comprising a label transmitting means for transmitting labels encoded by the vector quantizer according to claim 20 and a deviation vector transmitting means for transmitting a deviation vector.

22. A signal receiver comprising a deviation vector storing portion for storing deviation vectors transmitted by the signal transmitter according to claim 21, a reference codebook, an subtracter for subtracting a deviation vector read out from said deviation vector storing portion from a representative, vector read out from said reference codebook in accordance with a label which bas been received, and a decoder for providing the output of said subtracter as a decoded vector for said label.

23. A recognition device comprising the vector quantizer according to claim 2 for performing vector quantization on a normalized vector of each of a series of feature vectors which has been input, for converting them into labels, and for consequently converting said series of feature vectors into a series of labels, an HMM storing means for storing a hidden Markov model for each unit for recognition wherein the probability of the occurrence of each label is defined for each state, and a likelihood calculating means for calculating the likelihood of each of said HMMs for said series of membership vectors, the unit for recognition associated with the HMM that gives the maximum value of the likelihood being regarded as the result of recognition.

24. A recognition device comprising the vector quantizer according to claim 2 for performing vector quantization on the normalized vector of each of a series of feature vectors which has been input, for converting them into membership vectors whose components are membership values of the vectors for individual labels, and for consequently converting said series of feature vectors into a series of said membership vectors, an HMM storing means for storing a hidden Markov model for each unit for recognition wherein the probability of the occurrence of each label is defined for each state, and a likelihood calculating means for calculating the likelihood of each of said HMMs for said series of membership vectors, the unit for recognition associated with the HMM that gives the maximum value of the likelihood being regarded as the result of recognition.

27. A recognition device comprising the vector quantizer according to claim 2 for performing vector quantization on a normalized vector of each of a series of feature vectors which has been input, for converting them into labels, and for consequently converting said series of feature vectors into a series of labels, a recognition model storing means for storing a recognition model represented by a series of labels for each unit for recognition, and a distance calculating means for calculating the distance or similarity between said input series of labels and each of said recognition models, the unit for recognition associated with the recognition model that gives the minimum value of said distance or the maximum value of said similarity being regarded as the result of recognition.

28. A recognition device comprising the vector quantizer according to claim 2 for performing vector quantization on a normalized vector of each of a series of feature vectors which has been input, for converting them into membership vectors whose components are membership values of the vectors for individual labels, and for consequently converting said series of feature vectors into a series of said membership vectors, a recognition model storing means for storing a recognition model represented by a series of labels for each unit for recognition, a distance calculating means for calculating the distance or similarity between said input series of membership values and each of said recognition models, the unit for recognition associated with the recognition model that gives the minimum value of said distance or the maximum value of said similarity being regarded as the result of recognition.

30. The vector quantizer according to claim 2, comprising a temporary storage means for serially storing predetermined constant signal segments of an input signal in real time wherein deviation vectors for normalization of the input signal are serially calculated for each of said signal segments using the contents of said temporary storing means as vectors for learning.

32. The vector quantizer according to claim 2, comprising a multiplier and attenuation coefficient storing means for storing an attenuation coefficient which is set to attenuate an effect of a past input voice, wherein

the deviation vector is calculated so as to make an effect of the input voice near a present time large by multiplying the accumulated product of short time deviation vector and the sum of membership values and the accumulated sum of membership values vector with the attenuation coefficient, and thereby to calculate the deviation vector for normalization of the input signal in order.

3. A vector quantizer comprising:

a reference codebook for storing several representative vectors in a feature vector space so that each of them can be retrieved using labels associated therewith;

a speech input means for inputting speech;

a speech analysis means for performing feature extraction on said input speech to convert it into a series of feature vectors;

a vector quantizing means for converting said feature vectors into labels or a membership value vector whose components are the membership value of said feature vectors for the representative vectors;

an objective function calculating means for calculating an objective function defined as a function of said representative vectors and said series of feature vectors;

short time deviation vector calculating means for calculating a short time deviation vector;

a sum of membership values calculating means for calculating the sum of said membership values in the input speech segment used for the calculation of said short time deviation vector;

a product of short time deviation vector and the sum of membership values calculating means for calculating a vector which is the product of said short time deviation vector and said sum of membership values;

an accumulated sum of membership values storing means for accumulating and storing past sum of membership values;

an accumulated product of short time deviation vector and the sum of membership values storing means for accumulating and storing past product of said short time deviation vector and said sum of membership values;

a deviation vector calculating means for calculating past deviation vectors from the past accumulated product of short time deviation vector and said sum of membership values stored in said accumulated product of short time deviation vector and the sum of membership values storing means and the past accumulated sum of said membership values stored in said accumulated sum of membership values storing means; and

an adaptation means for adding said past deviation vector to said representative vectors to obtain new representative vectors, wherein said short time deviation vector calculating means calculates so that said new representative vectors cause said objective function to approach the extreme value relative to the feature vectors for the current input utterance and wherein said deviation vector calculating means calculates so that an overall objective function approaches the extreme value relative to feature vectors for past input utterances.

4. A vector quantizer comprising:

a reference codebook for storing several representative vectors in a feature vector space so that each of them can be retrieved using labels associated therewith;

a speech input means for inputting speech;

a speech analysis means for performing feature extraction on said input speech to convert it into a series of feature vectors;

a vector quantizing means for converting said feature vectors into labels or a membership value vector whose components are the membership value of said feature vectors for the representative vectors;

an objective function calculating means for calculating an objective function defined as a function of said representative vectors and said series of feature vectors;

short time deviation vector calculating means for calculating a short time deviation vector;

a sum of membership values calculating means for calculating the sum of said membership values in the input speech segment used for the calculation of said short time deviation vector;

a product of short time deviation vector and the sum of membership values calculating means for calculating a vector which is the product of said short time deviation vector and said sum of membership values;

an accumulated sum of membership values storing means for accumulating and storing past sum of membership values;

an accumulated product of short time deviation vector and the sum of membership values storing means for accumulating and storing past product of short time deviation vector and said sum of membership values;

a deviation vector calculating means for calculating past deviation vectors from the past accumulated product of short time deviation vector and said sum of membership values stored in said accumulated product of short time deviation vector and the sum membership values storing means and the past accumulated sum of said membership values stored in said accumulated sum of membership values storing means; and

a normalization means for adding said past deviation vector to a feature vector, wherein said short time deviation vector calculating means calculates so that said objective function approaches the extreme value relative to said reference codebook when the sum of the current feature vector and said deviation vector is replaced as a new vector for learning and wherein said deviation vector calculating means calculates so that an overall objective function approaches the extreme value relative to said reference codebook when the sums of past feature vectors and said deviation vector are replaced as new vectors for learning.

16. A vector quantizer comprising:

a reference codebook storing means for storing a finite number of representative vectors in a feature vector space so that they can be retrieved using labels assigned thereto;

a vector quantizing means for converting each of a series of feature vectors into a membership vector associated with a label using said reference codebook to convert said series of feature vectors into a series of membership vectors;

an HMM storing means for storing HMMs for which the label occurrence probability and the probability of a state transition are defined for each state thereof;

a feature vector series occurrence rate calculating means for calculating the rate of the occurrence of said series of feature vectors from said HMMs based on said label occurrence probability and said membership vectors;

a path probability calculating means for calculating the path based on said feature vector series occurrence rate and the probability of the transition of said HMM; and

a codebook correcting means for correcting said representative vectors, wherein said codebook correcting means comprises a correction vector calculating means for correcting said representative vectors to minimize the distortion of quantization error of said series of feature vectors from said reference codebook weighted by said path probability and is configured to correct said representative vectors.

17. A vector quantizer comprising:

a reference codebook storing means for storing a finite number of representative vectors in a feature vector space so that they can be retrieved using labels assigned thereto;

a vector quantizing means for converting each of a series of feature vectors into a membership vector associated with a label using said reference codebook to convert said series of feature vectors into a series of membership vectors;

an HMM storing means for storing HMMs for which the label occurrence probability and the probability of a state transition are defined for each state thereof;

a feature vector series occurrence rate calculating means for calculating the rate of the occurrence of said series of feature vectors from said HMMs based on said label occurrence probability and said membership vectors;

a path probability calculating means for calculating the path probability based on said feature vector series occurrence rate and the probability of the transition of said HMM;

a likelihood calculating means for calculating the likelihood of the HMM for each of said word relative to said series of feature vectors;

a comparison and determination means for determining the result of recognition;

a recognition candidate reliability calculating means for calculating the reliability of a candidate for recognition obtained by said comparison and determination means;

a code book correction execution determining means for instructing the execution of the correction of the reference codebook if the reliability of said candidate for recognition exceeds a predetermined threshold; and

a codebook correcting means for correcting each of said code vectors, wherein said reference codebook correcting means comprises a correction vector calculating means for correcting said representative vectors to minimize the distortion of quantization error of said series of feature vectors from said reference codebook weighted by said path probability and is configured to correct said reference codebook when the contents of speech is known by using the candidate for recognition as the contents of speech.

25. A vector quantizer comprising:

a reference codebook storing means for storing a finite number of representative vectors in a feature vector space so that they can be retrieved using labels assigned thereto;

a vector quantizing means for converting each of a series of feature vectors into a membership vector associated with a label using said reference codebook to convert said series of feature vectors into a series of membership vectors;

an HMM storing means for storing HMMs for which the label occurrence probability and the probability of a state transition are defined for each state thereof;

a feature vector series occurrence rate calculating means for calculating the rate of the occurrence of said series of feature vectors from said HMMs based on said label occurrence probability and said membership vectors;

a path probability calculating means for calculating the path probability based on said feature vector series occurrence rate and the probability of the transition of said HMM;

a feature vector correcting means for correcting said feature vectors; and

a normalization vector adjusting means for calculating a normalization vector for correcting said feature vectors, wherein said normalization vector adjusting means comprises a correction vector calculating means for correcting said representative vectors to minimize the distortion of quantization error of said series of feature vectors from said reference codebook weighted by said path probability and is configured to correct said representative vectors.

26. A vector quantizer comprising:

a reference codebook storing means for storing a finite number of representative vectors in a feature vector space so that they can be retrieved using labels assigned thereto;

a vector quantizing means for converting each of a series of feature vectors into a membership vector associated with a label using said reference codebook to convert said series of feature vectors into a series of membership vectors;

an HMM storing means for storing HMMs for which the label occurrence probability and the probability of a state transition are defined for each state thereof;

a feature vector series occurrence rate calculating means for calculating the rate of the occurrence of said series of feature vectors from said HMMs based on said label occurrence probability and said membership vectors;

a path probability calculating means for calculating the path probability based on said feature vector series occurrence rate and the probability of the transition of said HMM;

a likelihood calculating means for calculating the likelihood of the HMM for each of said word relative to said series of feature vectors;

a comparison and determination means for deterring the result of recognition;

a recognition candidate reliability calculating means for calculating the reliability of a candidate for recognition obtained by said comparison and determination means;

a code book correction execution determining means for instructing the execution of the correction of the reference codebook if the reliability of said candidate for recognition exceeds a predetermined threshold;

a feature vector correcting means for correcting said feature vectors; and

a normalization vector adjusting means for calculating a normalization vector for correcting said feature vectors, wherein said reference codebook correcting means comprises a correction vector calculating means for correcting said representative vectors to minimize the distortion of quantization error of said series of feature vectors from said reference codebook weighted by said path probability and is configured to correct said reference codebook when the contents of speech is unknown by using the candidate for recognition as the contents of speech.