Image recognition apparatus and method

• US 20030048930A1
• Filed: 10/29/2002
• Published: 03/13/2003
• Est. Priority Date: 01/30/1998
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Mihara, Isao, Doi, Miwako
• Granted Patent: US 6,690,815 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An image recognition apparatus and method in which an image capture device captures a distance image stream of an object body. A mouth portion extraction mechanism extracts a mouth portion from the distance image stream that has been extracted by the image capture device. An image recognition mechanism recognizes at least one of a lip shape and a lip movement based on a mouth portion distance image stream that has been extracted by the mouth portion extraction mechanism. Instructions for causing a computer to perform the method may be stored in a computer readable memory.

Claims

1. An image recognition apparatus comprising:

an image capturing device which captures a distance image of an object;

a mouth portion extracting mechanism configured to extract a mouth portion from the distance image captured by the image capturing device; and

an image recognizing mechanism configured to recognize a lip shape based on a mouth portion distance image extracted by the mouth portion extracting mechanism.

2. The image recognition apparatus according to claim 1, further comprising:

a direction recognizing mechanism configured to recognize a direction of a face of a speaker based on the mouth shape obtained by the image recognition mechanism.

3. The image recognition apparatus according to claim 1, further comprising:

a face portion extracting mechanism configured to extract a face portion from the distance image captured by the image capturing mechanism; and

a direction recognizing mechanism configured to recognize a direction of a face of a speaker based on a face portion distance image extracted by the face portion extracting mechanism.

4. The image recognition apparatus according to claim 1, further comprising:

a speech recognizing mechanism configured to recognize an input voice; and

a control mechanism configured to perform at least one of a control of causing the speech recognizing mechanism to start speech recognition when a start of talking of a speaker has been detected based on a recognition result of the image recognizing mechanism and a control of causing the speech recognizing means to terminate speech recognition when an end of talking of a speaker has been detected based on a recognition result of the image recognizing mechanism.

5. The image recognition apparatus according to claim 2, further comprising:

a speech recognizing mechanism configured to recognize an input voice; and

control mechanism configured to perform at least one of a control of causing the speech recognizing means to start speech recognition when a recognition result of the direction recognizing mechanism is a square direction and a control of causing the speech recognizing mechanism to terminate speech recognition based on a recognition result of the direction recognizing mechanism.

6. The image recognition apparatus according to claim 1, further comprising:

an information mechanism configured to present prescribed information according to a prescribed output form; and

a control mechanism configured to detect at least one of a start and an end of a talk of a speaker based on a recognition result of the image recognizing means, and configured to perform, in accordance with a detection result, at least one of a control of causing the information presenting mechanism to start information presentation, a control of causing the information presenting mechanism to terminate information presentation, and a control of changing at least part of output forms in information presentation being performed by the information presenting mechanism.

7. The image recognition apparatus according to claim 2, further comprising:

an information presenting mechanism configured to present prescribed information according to a prescribed output form; and

a control means mechanism configured to perform, in accordance with a relationship between a square direction and a direction of a recognition result of the direction recognizing mechanism, a control of causing the information presenting mechanism, a control of causing the information presenting mechanism to start information presentation, a control of causing the information presenting mechanism to terminate information presentation, and a control of changing at least part of output forms used in information presentation being performed by the information presenting mechanism.

8. The image recognition apparatus according to claim 3, further comprising:

an information presenting mechanism configured to present prescribed information according to a prescribed output form; and

a control mechanism configured to perform, in accordance with a relationship between a square direction and a direction of a recognition result of the direction recognizing mechanism, a control of causing the information presenting mechanism configured to start information presentation, a control of causing the information presenting mechanism configured to terminate information presentation, and a control of changing at least part of output forms used in information presentation being performed by the information presenting mechanism.

9. The image recognition apparatus according to claim 1, further comprising:

a communicating mechanism configured to transmit information obtained by the image recognizing mechanism.

10. An image recognition apparatus comprising:

image capturing device which captures a distance image stream of an object;

a mouth portion extracting mechanism configured to extract a mouth portion from the distance image stream captured by the image capturing device; and

an image recognizing mechanism configured to recognize a lip movement based on a mouth portion distance image stream extracted by the mouth portion extracting mechanism.

11. The image recognition apparatus according to claim 10, further comprising:

a direction recognizing mechanism configured to recognize a direction of a face of a speaker based on the mouth movement obtained by the image recognition mechanism.

12. The image recognition apparatus according to claim 10, further comprising:

a face portion extracting mechanism configured to extract a face portion from the distance image stream captured by the image capturing mechanism; and

direction recognizing mechanism configured to recognize a direction of a face of a speaker based on a face portion distance image stream extracted by the face portion extracting mechanism.

13. The image recognition apparatus according to claim 10, further comprising:

a speech recognizing mechanism configured to recognizing an input voice; and

a control mechanism configured to perform at least one of a control of causing the speech recognizing mechanism to start speech recognition when a start of talking of a speaker has been detected based on a recognition result of the image recognizing mechanism and a control of causing the speech recognizing mechanism to terminate speech recognition when an end of taking of a speaker has been detected based on a recognition result of the image recognizing mechanism.

14. The image recognition apparatus according to claim 11, further comprising:

a speech recognizing mechanism configured to recognize an input voice; and

a control mechanism configured to perform at least one of a control of causing the speech recognizing mechanism to start speech recognition when a recognition result of the direction recognizing mechanism is a square direction and a control of causing the speech recognizing mechanism to terminate speech recognition when a recognition result of the direction recognizing mechanism is not a square direction.

15. The image recognition apparatus according to claim 10, further comprising:

an information presenting mechanism configured to present prescribed information according to a prescribed output form; and

a control mechanism configured to detect at least one of a start and an end of talking of a speaker based on a recognition result of the image recognizing mechanism, and for performing, in accordance with a detection result, a least one of a control of causing the information presenting mechanism to start information presentation, a control of causing the information presenting mechanism to terminate information presentation, and a control of changing at least part of output forms used in information presentation being performed by the information presenting mechanism.

17. The image recognition apparatus according to claim 12, further comprising:

an information presenting mechanism configured to present prescribed information according to a prescribed output form; and

a control mechanism configured to perform, in accordance with a relationship between a square direction and a direction of a recognition result of the direction recognizing mechanism, a control of causing the information presenting mechanism to start information presentation, a control of causing the information presenting mechanism to terminate information presentation, and a control of changing at least part of output forms used in information presentation being performed by the information presenting mechanism.

18. The image recognition apparatus according to claim 10, further comprising:

a communicating mechanism configured to transmit information obtained by the image recognizing mechanism.

16. The image recognition apparatus according to claim I 1, further comprising:

an information presenting mechanism configured to present prescribed information according to a prescribed output form; and

a control mechanism configured to perform, in accordance with a relationship between a square direction and a direction of a recognition result of the direction recognizing mechanism, a control of causing the information presenting mechanism to start information presentation, a control of causing the information presenting mechanism to terminate information presentation, and a control of changing at least part of output forms used in information presentation being performed by the information presenting mechanism.

19. An image recognition apparatus comprising:

an image capturing device which captures a distance image of an object;

face portion extracting means for extracting a face portion from the distance image captured by the image capturing device; and

an image recognizing mechanism configured to recognize a face shape based on a face portion distance image extracted by the face portion extracting mechanism.

20. The image recognition apparatus according to claim 19, further comprising:

a communicating mechanism configured to transmit information obtained by the image recognizing mechanism.

21. An image recognition apparatus comprising:

an image capturing device which captures a distance image stream of an object;

a face portion extracting mechanism configured to extract a face portion from the distance image stream captured by the image capturing device; and

an image recognizing mechanism configured to recognize at least one of a face shape and a face movement based on a face portion distance image stream extracted by the face portion extracting mechanism.

22. The image recognition apparatus according to claim 21, further comprising:

a communicating mechanism configured to transmit information obtained by the image recognizing mechanism.

23. An image recognition mechanism comprising the steps of:

capturing a distance image of an object;

extracting a mouth portion from the captured distance image; and

recognizing a lip shape based on an extracted mouth portion distance image.

24. An image recognition method comprising the steps of:

capturing a distance image stream of an object;

extracting a mouth portion from the captured distance image stream; and

recognizing a lip movement based on an extracted mouth portion distance image stream.

25. A computer-readable memory containing computer-readable instructions, comprising:

a first instruction configured to cause a computer to capture a distance image of an object;

a second instruction configured to cause the computer to extract a mouth portion from the captured distance image; and

a third instruction configured to cause the computer to recognize a lip shape based on an extracted mouth portion distance image.

26. A computer-readable memory containing computer-readable instructions, comprising:

a first instruction configured to cause a computer to capture a distance image stream of an object;

a second instruction configured to cause the computer to extract a mouth portion from the captured distance image stream; and

a third instruction configured to cause the computer to recognize a lip movement based on an extracted mouth portion distance image stream.