Learning apparatus, learning method, recognition apparatus, recognition method, and recording medium

• US 20020184011A1
• Filed: 06/10/2002
• Published: 12/05/2002
• Est. Priority Date: 10/09/1998
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Tetsujiro, Yoshiwara, Norifumi
• Granted Patent: US 7,072,829 B2
• Time in Patent Office: Days
• US Class Current: 704/222
• CPC Class Codes: G06F 18/10 Pre-processing; Data cleansing
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 18/253 of extracted features
• CPC Class Codes: G06V 10/72 Data preparation, e.g. statistical preprocessing of image or video features
• CPC Class Codes: G06V 10/762 using clustering, e.g. of similar faces in social networks
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/806 of extracted features
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features

Abstract

With respect to each of codes corresponding to code vectors in a code book stored in a code book storage section 82, an expectation degree storage section 84 stores an expectation degree at which observation is expected when an integrated parameter with respect to a word as a recognition target is inputted. A vector quantization section 81 vector-quantizes the integrated parameter and outputs a series of codes of a code vector which has a shortest distance to the integrated parameter. Further, a chi-square test section 83 makes a chi-square test with use of the series of codes outputted from the vector quantization section 81 and an expectation degree of each code stored in the expectation degree storage section 84, thereby to obtain properness as to whether or not the integrated parameter corresponds to a recognition target. Further, recognition is performed, based on the chi-square test result. As a result of this, recognition can be performed without considering time components which a signal has.

Claims

1. A learning apparatus for carrying out learning an expectation degree at which a vector quantization result is observed and which is used for vector-quantizing an input series and recognizing whether or not the input series corresponds to a recognition target, based on the vector quantization result, comprising:

vector quantization means for vector-quantizing a time series of learning data pieces and for outputting a series of identifiers each indicating a code vector; and

calculator means for calculating an expectation degree of each of the identifiers, from the series of identifiers obtained from the time series of learning data pieces.

2. The apparatus according to claim 1, wherein with respect to each of the tine series of learning data pieces, the vector quantization means outputs an identifier of a code vector which has a shortest distance to the learning data piece.

3. The apparatus according to claim 1, wherein the calculator means obtains an average value of expectation degrees of an identifier with respect to each of a plurality of learning data pieces used for learning for recognition of one same recognition target, as a final expectation degree of the identifier.

4. A leaning method for carrying out learning an expectation degree at which a vector quantization result is observed and which is used for vector-quantizing an input series and recognizing whether or not the input series corresponds to a recognition target, based on the vector quantization result, comprising the steps of:

vector-quantizing a time series of learning data pieces and of outputting a series of identifiers each indicating a code vector; and

calculating an expectation degree of each of the identifiers, from the series of identifiers obtained from the tine series of learning data pieces.

5. A recording medium which records a program for making a computer execute learning an expectation degree at which a vector quantization result is observed and which is used for vector-quantizing an input series and recognizing whether or not the input series corresponds to a recognition target, based on the vector quantization result,

the program comprises:

a vector quantization step of vector-quantizing a time series of learning data pieces and of outputting a series of identifiers each indicating a code vector; and

a calculation step of calculating an expectation degree of each of the identifiers, from the series of identifiers obtained from the time series of learning data pieces.

6. A recognition apparatus for recognizing whether or not a time series of input data pieces corresponds to a recognition target, comprising:

storage means for storing an expectation degree at which observation is expected when the time series of input data pieces is inputted, with respect to each of identifiers corresponding to code vectors used for vector quantization;

vector quantization means for vector-quantizing the time series of input data pieces and for outputting a series of identifiers indicating code vectors;

properness detection means for obtaining properness as to whether or not the time series of input data pieces corresponds to the recognition target, with use of the series of identifiers obtained from the input data and the expectation degrees of the identifiers; and

recognition means for recognizing whether or not the time series of input data pieces corresponds to the recognition target, based on the properness.

7. The apparatus according to claim 6, the vector quantization means outputs an identifier of a code vector which has a shortest distance to the input data piece with respect to each of the time series of input data pieces.

8. The apparatus according to claim 6, wherein the properness detection means obtains the properness by making a chi-square test.

9. The apparatus according to claim 6, wherein the time series of input data pieces is an integrated parameter which integrates a characteristic parameter of a voice and a characteristic parameter of an image of lips when the voice is spoken.

10. A recognition method for recognizing whether or not a time series of input data pieces corresponds to a recognition target, comprising the steps of:

vector-quantizing the time series of input data pieces, thereby to output a series of identifiers indicating code vectors;

obtaining properness as to whether or not the time series of input data pieces corresponds to the recognition target, with use of the series of identifiers obtained from the input data pieces and expectation degrees of the identifiers at which the identifiers are expected to be observed; and

recognizing whether or not the time series of input data pieces corresponds to the recognition target, based on the properness.

11. A recording medium which records a program for making a computer execute recognition processing for recognizing whether or not a time series of input data pieces corresponds to a recognition target, wherein the program comprises:

a vector-quantization step of vector-quantizing the time series of input data pieces, thereby to output a series of identifiers indicating code vectors;

a properness detection step of obtaining properness as to whether or not the time series of input data pieces corresponds to the recognition target, with use of the series of identifiers obtained from the input data pieces and expectation degrees of the identifiers at which the identifiers are expected to be observed; and

a recognition step of recognizing whether or not the time series of input data pieces corresponds to the recognition target, based on the properness.

13. The apparatus according to claim 12, further comprising approximation means for approximating a polynomial to the transition of the distance.

14. The apparatus according to claim 13, wherein the approximation means determines a degree of the polynomial such that an accumulation value of a distance to one of code vectors in a code book that minimizes the distance to the standard series is minimized.

18. The apparatus according to claim 17, wherein the storage means stores, as the distance transition model, a polynomial which approximates transition of the distance between the standard series and the code vector.

19. The apparatus according to claim 17, further comprising normalization means for performing normalization of a time axis with respect to the time series of input data pieces, and wherein

the vector quantization means vector-quantizes the normalized input data pieces.

20. The apparatus according to claim 17, wherein the time series of input data pieces is an integrated parameter which integrates a characteristic parameter of a voice and a characteristic parameter of an image of lips when the voice is spoken.

21. The apparatus according to claim 17, wherein the recognition means accumulates the distance for each of the identifiers, based on the series of identifiers and the distance transition model, and recognizes whether or not the series of input data pieces corresponds to at least one recognition target, based on the accumulated distances.

22. The apparatus according to claim 17, wherein the vector quantization means outputs an identifier of the code vector which has a shortest distance to the input data pieces.

23. The apparatus according to claim 17, wherein the model storage means stores a plurality of distance transition models.

27. The apparatus according to claim 16, wherein the recognition means performs first matching processing based on the transition of the distance obtained from the vector based on the time series of integrated data pieces, and second matching processing based on a space distribution obtained from the vector based on the time series of integrated data pieces, and recognizes whether or not the time series of first or second input data pieces corresponds to at least one recognition target, based on results of the first and second matching processing.

28. A recognition apparatus according to claim 26, characterized in that from a code vector in a code book used for vector quantization and a distance transition model expressing transition of a distance to a standard series, the recognition means accumulates the distance when a vector quantization result obtained by vector-quantizing a vector based on the integrated data pieces, with use of the code book is observed, thereby to recognize whether or not the time series of the first or second input data pieces corresponds to at least one recognition target, based on an accumulation result.

12. A learning apparatus for obtaining a distance transition model expressing transition of a distance between a standard series and a code vector used for vector quantization, comprising:

normalization means for performing normalization of a time axis with respect to a time series of learning data pieces and for outputting the standard series; and

distance calculation means for calculating a distance between the standard series and the code vector and for outputting transition of the distance.

15. A learning method for obtaining a distance transition model expressing transition of a distance between a standard series and a code vector used for vector quantization, comprising the steps of:

performing normalization of a time axis with respect to a time series of learning data pieces and of outputting the standard series; and

calculating a distance between the standard series and the code vector and of outputting transition of the distance.

16. A recording medium which records a program for making a computer execute learning for obtaining a distance transition model expressing transition of a distance between a standard series and a code vector used for vector quantization, characterized by comprising:

a normalization step of performing normalization of a time axis with respect to a time series of learning data pieces and of outputting the standard series; and

a distance calculation step of calculating a distance between the standard series and the code vector and of outputting transition of the distance.

17. A recognition apparatus for recognizing whether or not a time series of input data pieces corresponds to at least one recognition target, comprising:

code book storage means which stores a plurality of code vectors and identifiers respectively indicating the code vectors;

model storage means which stores a distance transition model corresponding to at least one recognition target and expressing transition of a distance between the standard series and each code vector of the code book;

vector quantization means for vector-quantizing the time series of input data pieces, with use of the code book and for outputting a series of the identifiers; and

recognition means for recognizing whether or not the time series of input data pieces corresponds to at least one recognition target, based on the distance transition model and the series of identifiers with respect to the time series of input data pieces.

24. A recognition method for recognizing whether or not a time series of input data pieces corresponds to at least one recognition target, comprising the steps of:

vector-quantizing the time series of input data pieces, with use of a code book including a plurality of code vectors and identifiers respectively indicating the code vectors, and for outputting a series of the identifiers; and

recognizing whether or not the time series of input data pieces corresponds to at least one recognition target, based on a distance transition model expressing a distance between the standard series and the code vectors and the series of identifiers with respect to the time series of input data pieces.

25. A recording medium which records a program for making a computer execute processing for recognizing whether or not a time series of input data pieces corresponds to at least one recognition target, wherein the program comprises:

a vector-quantization step of vector-quantizing the time series of input data pieces, with use of a code book including a plurality of code vectors and identifiers respectively indicating the code vectors, and for outputting a series of the identifiers; and

a recognition step of recognizing whether or not the time series of input data pieces corresponds to at least one recognition target, based on a distance transition model expressing a distance between the standard series and the code vectors and the series of identifiers with respect to the time series of input data pieces.

26. A recognition apparatus for recognizing whether or not a time series of input data pieces corresponds to at least one recognition target, comprising:

integration means for integrating a time series of first input data pieces and a time series of second input data pieces, thereby to output a time series of integrated data pieces; and

recognition means for recognizing whether or not the time series of first or second input data pieces corresponds to at least one recognition target, based on transition of a distance obtained from a vector based on the time series of integrated data pieces.

29. A recognition method for recognizing whether or not a time series of input data pieces corresponds to at least one recognition target, comprising the steps of:

integrating a time series of first input data pieces and a time series of second input data pieces, thereby to output a time series of integrated data pieces; and

recognizing whether or not the time series of first or second input data pieces corresponds to at least one recognition target, based on transition of a distance obtained from a vector based on the time series of integrated data pieces.

30. A recording medium which records a program for making a computer execute recognition processing for recognizing whether or not a time series of input data pieces corresponds to at least one recognition target, wherein the program comprises:

an integration step of integrating a time series of first input data pieces and a time series of second input data pieces, thereby to output a time series of integrated data pieces; and

a recognition step of recognizing whether or not the time series of first or second input data pieces corresponds to at least one recognition target, based on transition of a distance obtained from a vector based on the time series of integrated data pieces.

31. A learning apparatus for performing learning for obtaining a normalization coefficient used for normalization of a characteristic parameter expressing each of a plurality of input data pieces, comprising:

characteristic parameter normalization means for normalizing each of a plurality of characteristic parameters, based on the normalization coefficient;

distance calculation means for calculating a distance to a standard parameter, with respect to each of the plurality of characteristic parameters normalized; and

change means for changing the normalization coefficient such that a distance with respect to an arbitrary one of the plurality of characteristic parameters and a distance with respect to another arbitrary one of the plurality of characteristic parameters are equal to each other.

32. The apparatus according to claim 31, further comprising:

code book generation data normalization means for normalizing code book generation data for generating a code book used for vector quantization, based on the normalization coefficient; and

code book generation means for generating the code book with use of the normalized code book generation data,

and wherein the distance calculation means calculates a distance to the characteristic parameter using one of code vectors in the code book that is closest to the characteristic parameter as the standard parameter.

33. The apparatus according to claim 31, wherein the plurality of input data pieces include at least data pieces of an image and a voice.

37. A recognition apparatus according to claim 36, wherein the normalization means normalizes the characteristic parameter, based on a normalization coefficient.

38. The apparatus according to claim 37, wherein the normalization coefficient is obtained by normalizing each of a plurality of characteristic parameters, based on the normalization coefficient, by calculating a distance to a standard parameter, with respect to each of the plurality of characteristic parameters normalized, and by changing the normalization coefficient such that a distance with respect to an arbitrary one of the plurality of characteristic parameters and a distance with respect to another arbitrary one of the plurality of characteristic parameters are equal to each other.

39. The apparatus according to claim 37, wherein the normalization coefficient is obtained by normalizing code book generation data for generating a code book used for vector quantization, based on the normalization coefficient, by generating the code book with use of the normalized code book generation data, by using one of code vectors in the code book that is closest to the characteristic parameter as the standard parameter, thereby to calculate a distance to the characteristic parameter, and by changing the normalization coefficient such that a distance with respect to an arbitrary one of the plurality of characteristic parameters and a distance with respect to another arbitrary one of the plurality of characteristic parameters are equal to each other.

40. The apparatus according to claim 36, wherein the plurality of input data pieces include at least data pieces of an image and a voice.

41. The apparatus according to claim 40, wherein the recognition means recognizes the voice.

42. The apparatus according to claim 36, further comprising time axis normalization means for normalizing the integrated parameter in a time axis direction.

34. A learning method for performing learning for obtaining a normalization coefficient used for normalization of a characteristic parameter expressing each of a plurality of input data pieces, comprising the steps of:

normalizing each of a plurality of characteristic parameters, based on the normalization coefficient;

calculating a distance to a standard parameter, with respect to each of the plurality of characteristic parameters normalized; and

changing the normalization coefficient such that a distance with respect to an arbitrary one of the plurality of characteristic parameters and a distance with respect to another arbitrary one of the plurality of characteristic parameters are equal to each other.

35. A recording medium which records a program for making a computer execute learning for obtaining a normalization coefficient used for normalization of a characteristic parameter expressing each of a plurality of input data pieces, comprising:

a characteristic parameter normalization step of normalizing each of a plurality of characteristic parameters, based on the normalization coefficient;

a distance calculation step of calculating a distance to a standard parameter, with respect to each of the plurality of characteristic parameters normalized; and

a change step of changing the normalization coefficient such that a distance with respect to an arbitrary one of the plurality of characteristic parameters and a distance with respect to another arbitrary one of the plurality of characteristic parameters are equal to each other.

36. A recognition apparatus comprising:

detection means for detecting a characteristic parameter with respect to each of a plurality of input data pieces;

normalization means for normalizing the characteristic parameter of each of the plurality of input data pieces;

integration means for integrating a plurality of normalized characteristic parameters into an integrated parameter; and

recognition means for recognizing whether or not one or more of the plurality of input data pieces correspond to a recognition target, based on the integrated parameter.

43. A recognition method comprising:

outputting a characteristic parameter with respect to each of a plurality of input data pieces;

normalizing the characteristic parameter of each of the plurality of input data pieces;

integrating a plurality of normalized characteristic parameters into an integrated parameter; and

recognizing whether or not one or more of the plurality of input data pieces correspond to a recognition target, based on the integrated parameter.

44. A recording medium which records a program to be executed by a computer, the program comprising:

a detection step of detecting a characteristic parameter with respect to each of a plurality of input data pieces;

a normalization step of normalizing the characteristic parameter of each of the plurality of input data pieces;

an integration step of integrating a plurality of normalized characteristic parameters into an integrated parameter; and

a recognition step of recognizing whether or not one or more of the plurality of input data pieces correspond to a recognition target, based on the integrated parameter.