Gesture recognition system

• US 6,804,396 B2
• Filed: 03/28/2001
• Issued: 10/12/2004
• Est. Priority Date: 03/28/2001
• Status: Expired due to Term

• Current Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Original Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Inventors: Fujimura, Kikuo, Yoshida, Yuichi, Higaki, Nobuo
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US09/820,130
• Publication Number:
  US 20020181773A1
• Time in Patent Office: 1,294 Days
• Field of Search: 382/165, 382/170, 382/181, 382/218, 704/231, 704/233, 704/235, 704/236, 704/246, 704/251, 725/19, 345/863, 345/728, 348/14.01, 348/14.1, 367/198, 340/5.21, 340/5.24
• US Class Current: 382/181
• CPC Class Codes: B25J 11/0005 Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
• CPC Class Codes: B25J 13/003 by means of an audio-responsive input audible safety signals B25J19/061
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

The present invention provides a system for recognizing gestures made by a moving subject. The system comprises a sound detector for detecting sound, one or more image sensors for capturing an image of the moving subject, a human recognizer for recognizing a human being from the image captured by said one or more image sensors, and a gesture recognizer, activated when human voice is identified by said sound detector, for recognizing a gesture of the human being.

In a preferred embodiment, the system includes a hand recognizer for recognizing a hand of the human being. The gesture recognizer recognizes a gesture of the human being based on movement of the hand identified by the hand recognizer. The system may further include a voice recognizer that recognizes human voice and determines words from human voice input to the sound detector. The gesture recognizer is activated when the voice recognizer recognizes one of a plurality of predetermined keywords such as “hello!”, “bye”, and “move”.

Claims

1. A computer system for recognizing gestures made by a moving subject, comprising:

a sound detector for detecting sound;

one or more image sensors for capturing an image of the moving subject;

a human recognizer for recognizing a human being from the image captured by said one or more image sensors; and

a gesture recognizer, activated when human voice is identified by said sound detector, for recognizing a gesture of the human being.

2. The system as recited in claim 1, further comprising:

a hand recognizer for recognizing a hand of the human being;

wherein said gesture recognizer recognizes a gesture of the human being based on movement of the hand identified by the hand recognizer.

3. The system as recited in claim 2, further including:

a head recognizer for recognizing the position of the head of the human being,

wherein the hand recognizer determines the position of the hand relative to the position of the head determined by the head recognizer.

4. The system as recited in claim 3, further comprising:

a storage for storing statistical features of one or more gestures relating to position of the hand relative to the position of the head;

an extractor for extracting features of the movement of the hand as recognized by said hand recognizer; and

a comparator for comparing the extracted features with the stored features to determine a gesture to which the extracted features belong.

5. The system as recited in claim 3, wherein the hand recognizer recognizes a hand by determining the portion that shows large difference of positions in a series of images captured by the image sensors.

6. The system as recited in claim 1, further comprising:

a voice recognizer recognizing human voice and determining words from the human voice input to said sound detector;

wherein said gesture recognizer is activated responsive to said voice recognizer recognizing one of predetermined keywords.

7. The system as recited in claim 6, wherein said sound detector includes at least two microphones placed at a predetermined distance for determining the direction of the human voice, and said human recognizer identifies a moving subject located in the direction of the human voice as a human being.

8. A walking robot incorporating a computer system for recognizing gestures made by a moving subject, said computer system comprising:

a sound detector for detecting sound;

one or more image sensors for capturing an image of the moving subject;

a human recognizer for recognizing a human being from the image captured by said one or more image sensors; and

a gesture recognizer, activated when human voice is identified by said sound detector, for recognizing a gesture of the human being.

9. A computer-implemented method for recognizing human gestures, the method comprising:

identifying a human body based on images captured by one or more image sensors;

recognizing a hand of the human body; and

recognizing a gesture of the hand based on movement of the hand, wherein the method is initiated when human voice is recognized.

10. The method according to claim 9, wherein the method is initiated when the human voice is recognized to include one of predetermined keywords.

11. The method according to claim 9, further including a step of determining the direction of the human voice, and a step of moving an image sensor to the direction of the human voice.