Apparatus and method for recognition and apparatus and method for learning

• US 6,396,954 B1
• Filed: 08/24/1998
• Issued: 05/28/2002
• Est. Priority Date: 12/26/1996
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Tetsujiro
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Mariam, Daniel G.
• Application Number:
  US09/125,717
• Time in Patent Office: 1,373 Days
• Field of Search: 382/116,118,203,224,181,190,115,159,160,209,225,227,284 707/6,1,7,100,102
• US Class Current: 382/224
• CPC Class Codes: G10L 15/25 using position of the lips, movement of the lips or face analysis
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

Different types of data including voice data of a user, image data produced by picturing the mouth of the user, and ambient noise data are provided through an input unit 10. Those data are analyzed by preprocessors 20 to 23 respectively to determine characteristic parameters. In a classification data constructing unit 24, classification data is constructed from the characteristic parameters and transferred to a classification unit 25 for classification. Meanwhile, an integrated parameter constructing unit 26 constructs integrated parameters from the characteristic parameters provided by the preprocessors 20 to 23. An adaptivity determining unit 27 selects a table corresponding to the class determined by the classification unit 25. From the standard parameters saved in the table and the integrated parameter from the integrated parameter constructing unit 26, the voice emitted by a user is recognized. Accordingly, the accuracy of the voice recognition will be increased.

Claims

1. A recognition apparatus for recognizing a given subject from different types of input data, comprising:

a first classification data construction unit for constructing a first set of classification data from the different types of input data;

a first classification unit for classifying the different types of input data into a first class based on the first set of classification data from the first classification data construction unit;

preprocessing means for extracting a plurality of characteristic parameters from the different types of input data on the basis of said first class from said first classification unit;

a second classification data construction unit for constructing a second set of classification data based on the plurality of characteristic parameters from the preprocessing means;

a second classification unit for classifying the different types of input data into a second class based on the second set of classification data from the second classification data construction unit;

an integrated parameter construction unit for constructing an integrated parameter by integrating the plurality of characteristic parameters from the preprocessing means;

a plurality of standard parameter memories for storing a plurality of tables;

each standard parameter memory corresponding to a predetermined first class;

each table containing standard parameters corresponding to a predetermined second class; and

a matching unit for recognizing the given subject by matching the integrated parameter with the standard parameters from the table corresponding to the second class and located in the standard parameter memory corresponding to the first class.

2. The recognition apparatus according to claim 1, wherein the given subject is a speaker of interest and the different types of input data include at least the voice of the speaker and an image of the speaker'"'"'s mouth.

3. The recognition apparatus according to claim 1, wherein the different types of input data are collected by a corresponding plurality of different type sensors.

4. The recognition apparatus according to claim 1, wherein the preprocessing means comprises a first preprocessing means for extracting a first plurality of characteristic parameters for use by said second classification data construction unit, and a second preprocessing means for extracting a second plurality of characteristic parameters for use by said integrated parameter construction unit.

5. A method of recognizing a given subject from different types of input data, comprising the steps of:

constructing a first set of classification data from the different types of input data using a first classification data construction unit;

classifying the different types of input data into a first class based on the first set of classification data using a first classification unit;

extracting a plurality of characteristic parameters from the different types of input data on the basis of said first class from said first classification unit;

constructing a second set of classification data based on the plurality of characteristic parameters from the extracting step using a second classification data construction unit;

classifying the different types of input data into a second class based on the second set of classification data using a second classification unit;

constructing an integrated parameter by integrating the plurality of characteristic parameters from the extracting step; and

recognizing the given subject by matching the integrated parameter with standard parameters corresponding to the first class and the second class;

the standard parameters being stored in a plurality of standard parameter memories corresponding to predetermined first classes and containing a plurality of tables corresponding to predetermined second classes.

6. The method according to claim 5, wherein the given subject is a speaker of interest and the different types of input data include at least the voice of the speaker and an image of the speaker'"'"'s mouth.

7. The method according to claim 5, wherein the different types of input data are collected by a corresponding plurality of different type sensors.

8. The method according to claim 5, wherein the extracting step comprises a first preprocessing step of extracting a first plurality of characteristic parameters for use by said second classification data construction unit, and a second preprocessing step of extracting a second plurality of characteristic parameters for use in said integrated parameter constructing step.

9. A voice command recognition system for operating a voice-activated system, comprising:

an input for receiving different types of input data from a plurality of corresponding input sensors;

the different types of input data comprising voice audio data and at least one of ambient noise data and environmental conditions data;

a first classification data construction unit for constructing a first set of classification data from the different types of input data;

a first classification unit for classifying the different types of input data into a first class based on the first set of classification data from the first classification data construction unit;

an extractor for extracting a plurality of characteristic parameters from the different types of input data on the basis of said first class from said first classification unit;

a second classification data construction unit for constructing a second set of classification data based on the plurality of characteristic parameters from the extractor;

a second classification unit for classifying the different types of input data into a second class based on the second set of classification data from the second classification data construction unit;

an integrator for constructing an integrated parameter by integrating the plurality of characteristic parameters from the extractor;

a plurality of standard parameter memories for storing a plurality of tables;

each standard parameter memory corresponding to a predetermined first class;

each table containing standard parameters corresponding to a predetermined second class; and

a recognizer for recognizing a command from a given person by matching the integrated parameter with the standard parameters from the table corresponding to the second class and located in the standard parameter memory corresponding to the first class.

10. The voice command recognition system according to claim 9, wherein the extractor extracts a first plurality of characteristic parameters for use by said second classification data construction unit and extracts a second plurality of characteristic parameters for use by said integrated parameter construction unit.

11. The voice command recognition system according to claim 9, wherein the extractor comprises a plurality of preprocessors corresponding to each of the different types of input data and for extracting said plurality of characteristic parameters.

12. The voice command recognition system according to claim 9, wherein said recognizer recognizes said command on the basis of the smallest Euclidian distance calculated from each of the standard parameters in the standard parameter table corresponding to said second class and the integrated parameter.