Image recognizing apparatus and method

• US 20030007682A1
• Filed: 04/26/2002
• Published: 01/09/2003
• Est. Priority Date: 05/02/2001
• Status: Active Grant

• Current Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Original Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Inventors: Tsujino, Hiroshi, Koshizen, Takamasa, Akatsuka, Koji
• Granted Patent: US 7,221,797 B2
• Time in Patent Office: Days
• US Class Current: 382/157
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An image recognizing apparatus and method is provided for recognizing behavior of a mobile unit accurately with an image of external environment acquired during the mobile unit is moving.

Behavior command output block 12 outputs behavior commands to cause the mobile unit 32 move. Local feature extraction block 16 extracts features of local areas of the image from the image of external environment acquired on the mobile unit 32 when the behavior command is output. Global feature extraction block 18 extracts feature of global area of the image using the features of local areas. Learning block 20 calculates probability models for recognizing behavior given to the mobile unit 32 based on the feature of global area of the image. After learning is finished, behavior of the mobile unit 32 may be recognized rapidly and accurately by applying the probability models to an image of external environment acquired in mobile unit 32 afresh.

Claims

1. An apparatus for recognizing images, comprising:

behavior command output means for outputting behavior commands to cause a mobile unit to move;

local feature extraction means for extracting features of local areas within an image of external environment acquired on said mobile unit when said behavior command is output;

global feature extraction means for extracting feature of global area of said image using said extracted features of said local areas; and

learning means for calculating probability models to recognize behavior of the mobile unit based on said extracted feature of said global area.

2. The apparatus according to claim 1, wherein said local feature extraction means extracts said features of the local areas within the image by utilizing image intensities obtained by applying both positive and negative component of Gabor filters on two images.

3. The apparatus according to claim 2, wherein each of said Gabor filters is applied on one of eight different directions respectively.

4. The apparatus according to claims 1 to 3, wherein said global feature extraction means combines said features of the local areas into said global feature with the use of Gaussian functions.

5. The apparatus according to claim 4, wherein said learning means calculates said probability models utilizing expectation maximization algorithm and supervised learning with the use of neural network.

6. The apparatus according to claim 5, further comprising behavioral recognition means for applying Bayes'"'"' rule with use of said probability model on an image acquired afresh, said behavioral recognition means calculating confidence for each of said behavior commands to recognize behavior of said mobile unit.

7. The apparatus according to claim 6, further comprising:

behavioral assessment means for comparing said confidence with a predetermined value to assess said recognized behavior;

attention generation means for generating attentional demanding which demands to cause said probability models to be updated based on the result of said assessment; and

attentional modulation means for changing specified parameter of said learning means in response to said attentional demanding;

wherein said learning means recalculates said probability model after said parameter has been changed.

8. The apparatus accoring to claim 7, wherein said behavioral recognition means selects one confidence having largest value among the plurality of confidences and supplies it to said behaviroral assessment means, said behavioral recognition means recognizing the behavior corresponding to the selected confidence as the behavior of said mobile unit if said confidence is resulted to be larger than the predetermined value by said compairing by said behaviral assessment means.

9. The apparatus according to claim 8, wherein said parameter is the number of Gaussian mixture in an equation of said learning means for calculating said probability models.

10. An apparatus for recognizing images, comprising:

behavior command output means for outputting behavior commands to cause a mobile unit to move;

local feature extraction means for extracting features of local areas within an image of external environment acquired on said mobile unit when said behavior command is output;

global feature extraction means for combining said extracted features of said local areas into global feature of global area of said image with the use of Gaussian functions;

learning means for calculating probability models by utilizing expectation maximization algorithm and supervised learning with the use of neural network based on said global feature;

behavioral recognition means for applying Bayes'"'"' rule with use of said probability model on an image acquired afresh and calculating confidence for each of said behavior commands to recognize behavior of said mobile unit;

wherein said behavioral recognition means selects one confidence having largest value among the plurality of confidence;

behavioral assessment means for comparing said selected confidence with a predetermined value to assess said recognized behavior;

wherein said behavioral recognition means recognizing the behavior corresponding to the selected confidence as the behavior of said mobile unit if said confidence is resulted to be larger than the predetermined value;

attention generation means for generating attentional demanding which demands to cause said probability model to be updated if said confidence is resulted to be equal or smaller than the predetermined value; and

attentional modulation means for changing the number of Gaussian mixture in an equation of said learning means for calculating said probability models in response to said attentional demanding;

wherein said learning means recalculates said probability model after said parameter has been changed.

11. A method for recognizing images in computing system, comprising:

outputting behavior commands to cause a mobile unit to move;

extracting features of local areas within an image of external environment acquired on said mobile unit when said behavior command is output;

extracting feature of global area of said image using said extracted features of said local areas; and

calculating probability models to recognize behavior of the mobile unit based on said extracted feature of said global area.

12. The method according to claim 11, wherein said step of extracting features of the local areas within the image includes utilizing image intensities obtained by applying both positive and negative component of Gabor filters on two images.

13. The method according to claim 12, wherein each of said Gabor filters is applied on one of eight different directions respectively.

14. The method according to claims 11 to 13, wherein said step of extracting feature of the global area includes combining said features of the local areas into said global feature with the use of Gaussian functions.

15. The method according to claims 14, wherein said step of calculating the probability models includes utilizing expectation maximization algorithm and supervised learning with the use of neural network.

16. The method according to claim 15, further comprising:

applying Bayes'"'"' rule with use of said probability models on an image acquired afresh; and

calculating confidence for each of said behavior commands to recognize behavior of said mobile unit.

17. The method according to claim 16, further comprising:

comparing said confidence with a predetermined value to assess said recognized behavior;

generating attentional demanding which demands to cause said probability models to be updated based on the result of said assessment; and

changing specified parameter of said learning step in response to said attentional demanding;

wherein said probability model is recalculated after said parameter has been changed.

18. The apparatus accoring to claim 17, further comprising:

subsequent to said step of calculating confidence, selecting one confidence having largest value among the plurality of confidences for said comparing step,

recognizing the behavior corresponding to the selected confidence as the behavior of said mobile unit if said confidence is resulted to be larger than the predetermined value in said compairing step.

19. The method according to claim 18, wherein said parameter is the number of Gaussian mixture in an equation of said calculating step for calculating said probability models.

20. A method for recognizing images in computing system, comprising:

outputting behavior commands to cause a mobile unit to move;

extracting features of local areas within an image of external environment acquired on said mobile unit when said behavior command is output;

combining said extracted features of said local areas into global feature of global area of said image with the use of Gaussian functions;

calculating probability models by utilizing expectation maximization algorithm and supervised learning with the use of neural network based on said global feature;

applying Bayes'"'"' rule with use of said probability model on an image acquired afresh and calculating confidence for each of said behavior commands to recognize behavior of said mobile unit;

selecting one confidence having largest value among the plurality of confidence for said comparing step;

comparing said selected confidence with a predetermined value to assess said recognized behavior;

recognizing the behavior corresponding to the selected confidence as the behavior of said mobile unit if said confidence is resulted to be larger than the predetermined value;

generating attentional demanding which demands to cause said probability model to be updated if said confidence is resulted to be equal or smaller than the predetermined value; and

changing the number of Gaussian mixture in an equation of said calculating step for calculating said probability models in response to said attentional demanding;

wherein said probability model is recalculated after said parameter has been changed.