Method and apparatus for object recognition

• US 20020001398A1
• Filed: 06/20/2001
• Published: 01/03/2002
• Est. Priority Date: 06/28/2000
• Status: Active Grant

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Naruoka, Tomonobu, Akimoto, Toshiaki, Nagao, Kenji, Shimano, Mihoko
• Granted Patent: US 7,031,496 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition

Abstract

Based on image data taken and acquired by any one of a plurality of cameras, a predetermined object included in the image is recognized by pattern matching. A plurality of databases associated with a plurality of cameras is provided and a database to be used is changed according to the camera to be used. The plurality of databases can also be changed according to conditions such as weather and environment.

Claims

1. An object recognition method that inputs images taken by a plurality of cameras to a common image processing apparatus, applies pattern matching processing to each of said images and acquires information on objects that exist in said images, comprising the steps of:

focusing the range of searching said objects from said images entered; and

switching between a plurality of databases provided in association with each of said plurality of cameras and performing said pattern matching processing.

2. The object recognition method according to claim 1, wherein each of said plurality of databases is created by taking pictures of one object in association with the position of each of said cameras while changing the distance to the object and registering image data about a model acquired based on the image data obtained by the picture taking in each of said plurality of databases in association with the distance to said object.

3. The object recognition method according to claim 1, wherein each of said plurality of databases is created in association with individual conditions to use the database as well as any one of said plurality of cameras.

4. An object recognition method that carries out object recognition processing on each of images taken by a plurality of cameras, comprising the steps of:

preparing a plurality of databases corresponding to said plurality of cameras, registering a plurality of feature vectors obtained by multiplying image data acquired by taking pictures of a plurality of objects using each of said plurality of cameras by a feature extraction matrix to extract features of a predetermined object;

focusing the range of searching an object in an image taken by one of said plurality of cameras;

determining a feature vector about the image taken by multiplying the image data within said search range by said feature extraction matrix; and

deciding the similarity by comparing said feature vector determined and each of said plurality of feature vectors registered in the database corresponding to the camera that has taken pictures of the image for which the range of said object search has been focused.

5. The object recognition method according to claim 4, wherein a plurality of typical feature vectors obtained by multiplying a typical value of image data for every classified object by said feature extraction matrix is registered in said database.

6. An object recognition apparatus that carries out object recognition processing on each of images taken by a plurality of cameras, comprising:

a plurality of databases associated with said plurality of cameras in which model data about object models is registered;

a search range focusing section that focuses the object search range in an image taken by any one of said plurality of cameras; and

an object recognition section that compares the image data within said search range and said model data registered in said database corresponding to the camera that has taken this image data and detects the object in the image by detecting the part showing the highest similarity with respect to said model data in said image data.

7. The object recognition apparatus according to claim 6, wherein each of said plurality of databases is created in association not only with any one of said plurality of cameras but also with individual conditions for using the database.

8. The object recognition apparatus according to claim 7, further comprising a database selection section that selects a database to be used from said plurality of databases.

9. An object recognition program that includes a program to make a computer operate as:

search range extracting means for focusing the range of searching an object in an image taken by any one of a plurality of cameras; and

object recognizing means for comparing the image data within said search range and model data about the object model registered in said database corresponding to the camera that has taken this image data and detects the object by detecting the part showing the highest similarity with respect to said model data in said image data.

10. A plurality of databases associated with a plurality of cameras or every picture taking environment that registers a plurality of feature vectors obtained by multiplying image data acquired by taking pictures of said plurality of objects or taking pictures of said plurality of objects for every picture taking environment by a feature extraction matrix to extract features of a predetermined object.

11. A car-mounted object recognition apparatus that carries out image processing including object recognition processing on each of images taken by a plurality of car-mounted cameras, comprising:

a plurality of databases associated with each of said plurality of cameras in which model data about object models is registered;

a search range focusing section that focuses the range of searching an object in an image taken by any one of said plurality of cameras; and

an object recognition section that compares the image data within said search range and said model data registered in said database corresponding to the camera that has taken the image data and detects the object in the image by detecting the part showing the highest similarity with respect to said model data in said image data.

12. The car-mounted object recognition apparatus according to claim 11, wherein each of said plurality of databases is created in association with individual conditions to use said database as well as any one of said plurality of cameras.

13. The car-mounted object recognition apparatus according to claim 12 that monitors the operation state of the vehicle on which said plurality of cameras is mounted, selects any one of said plurality of cameras so as to match the operation state of the vehicle and switches between said plurality of databases so as to correspond to the selected camera.

14. The car-mounted object recognition apparatus according to claim 11, wherein said object recognition apparatus further comprises an information supply section to supply the driver of the vehicle with information of the image taken by any one of said plurality of cameras or information on the object obtained by object recognition processing.

15. The car-mounted object recognition apparatus according to claim 14, wherein said information on the object is supplied in a mode appealing to visual perception, a mode appealing to the five senses other than visual perception or in combination thereof.

16. The car-mounted object recognition apparatus according to claim 11, wherein the data about object models to be registered in each of said plurality of databases is downloaded from outside the vehicle using radio communication.

17. The car-mounted object recognition apparatus according to claim 11, wherein said object recognition apparatus further comprises a distance measuring section that applies image processing to an image taken by any one of said plurality of cameras and measures the distance in the three-dimensional space to the object existing in the image.

18. A car-mounted image processing apparatus that carries out image processing including object recognition processing on each of images taken by a plurality of car-mounted cameras, comprising:

a plurality of databases in which model data about object models is registered;

a search range extraction section that focuses the range of searching an object in an image taken by any one of a plurality of cameras;

an object recognition section that compares the image data within said search range and said model data registered in any one of said plurality of databases and detects the position of the object in the image by detecting the part showing the highest similarity with respect to said model data in said image data; and

a distance detection section that detects the distance in the three-dimensional space from said camera that has taken pictures of said object to said object based on the information on the position of the object in the image detected by said object recognition section.

19. The car-mounted image processing apparatus according to claim 18, wherein at least one typical feature vector is registered in each of said plurality of databases.

20. The car-mounted image processing apparatus according to claim 18, wherein any one of said plurality of cameras is selected and used so as to match the operation state of said vehicle on which said plurality of cameras is mounted and switches between said plurality of databases so as to correspond to the selected camera.

21. The car-mounted image processing apparatus according to claim 18, wherein model data about one object obtained by taking pictures of said object by a camera while changing the distance to the object is registered in each of said plurality of databases in association with the distance from said camera to said object.

22. The car-mounted image processing apparatus according to claim 18, further comprising an information supply section to supply the driver of the vehicle with at least one of information on the image taken by any one of said plurality of cameras, information on the object detected by said object recognition section or distance information detected by said distance detection section.

23. The car-mounted image processing apparatus according to claim 18, wherein the model data to be registered in said database can be downloaded from outside the vehicle using radio communication.

24. An image processing apparatus that carries out image processing including object recognition on each of images taken by a plurality of car-mounted cameras, comprising:

a plurality of databases registering feature vectors obtained by multiplying image data for each of objects of different types acquired beforehand by taking pictures of said plurality of objects of different types by a feature extraction matrix to extract features of a predetermined object;

a search range extraction section that focuses the range of searching an object on an area in the image taken by any one of a plurality of cameras where the object is expected to be located;

a feature vector extraction section that obtains a feature vector about the image taken by multiplying image data within said search range by said feature extraction matrix;

an object recognition section that compares the feature vector obtained by said feature vector extraction section and each of said plurality of feature vectors registered in said database, decides the similarity and thereby detects that said predetermined object exists in the image taken by any one of said plurality of cameras and detects the position of the predetermined object in said image; and

a distance detection section that detects the distance in the three-dimensional space from said camera that has taken pictures of said object to said object based on the information of the position of the object in the image detected by said object recognition section.

25. The image processing apparatus according to claim 24, wherein said distance detection section can reconstruct the three-dimensional road structure, detect the position of said object in said reconstructed three-dimensional road structure based on the information on the position of the object in the image detected by said object recognition section and measure the distance from the camera to said object based on the information on the position of said object in the detected three-dimensional space;

and

each of said plurality of cameras can take pictures of different objects in different directions independently of one another.