Object recognition apparatus and method

• US 5,739,848 A
• Filed: 10/16/1996
• Issued: 04/14/1998
• Est. Priority Date: 09/08/1993
• Status: Expired due to Term

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Sumitomo Electric Industries Limited (Sumitomo Corporation)
• Inventors: Shimoura, Hiroshi, Tenmoku, Kenji
• Primary Examiner(s): Au, Amelia
• Application Number:
  US08/731,582
• Time in Patent Office: 545 Days
• Field of Search: 348/113, 348/116, 348/118, 348/119, 348/143, 348/148, 348/169, 382/103, 382/104, 382/106, 382/107, 382/190, 364/460, 364/424.01, 364/424.02, 180/169
• US Class Current: 348/119
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An object recognition apparatus that can recognize an object in an image acquired by an on-vehicle camera mounted on a vehicle with high efficiency and accuracy. A recognition processing is carried out in a cut off range, which is cut off from the image, and in which an object to be recognized is supposed to be present. The cut off range is obtained using support information supplied from a position detection device and/or an anti-lock brake device of the vehicle. Since the object in the image changes its position as the vehicle moves, the displacement must be estimated during the object recognition. The estimation requires camera attitude parameters of the on-vehicle camera with respect to the road, and the estimation is performed using support information supplied from the position detection device and/of the anti-lock brake device. This will improve the efficiency and accuracy of obtaining the camera attitude parameters.

Claims

1. A camera attitude parameter generating apparatus comprising:

an on-vehicle camera mounted on a vehicle to acquire an image ahead of or behind the vehicle;

means for detecting a pair of road parallel lines in the image acquired by said on-vehicle camera while the vehicle is traveling on a road, each of said road parallel lines being parallel with the longitudinal direction of the road;

means for obtaining the intersection of the pair of orad parallel lines, thereby generating the intersection as a road vanishing point;

parameter generating mean for generating at least one of camera attitude parameters on the basis of the pair of road parallel lines and the road vanishing point, said camera attitude parameters representing the attitude of said on-vehicle camera with respect to the road; and

a position detection device including sensors and road map data, said position detection device detecting first support information including a bearing, a position, and a mileage of the vehicle, wherein said parameter generating means uses the first support information in generating said camera attitude parameters, and

wherein said parameter generating means obtains the bearing of the vehicle from the position detection device, and inhibits the generation of the camera attitude parameters if the change in the bearing of the vehicle is greater than a predetermined value.

2. A camera attitude parameter generating apparatus comprising:

an on-vehicle camera mounted on a vehicle to acquire an image ahead of or behind the vehicle;

means for detecting a pair of road parallel lines in the image acquired by said on-vehicle camera while the vehicle is traveling on a road, each of said road parallel lines being parallel with the longitudinal direction of the road;

means for obtaining the intersection of the pair of orad parallel lines, thereby generating the intersection as a road vanishing point;

parameter generating mean for generating at least one of camera attitude parameters on the basis of the pair of road parallel lines and the road vanishing point, said camera attitude parameters representing the attitude of said on-vehicle camera with respect to the road; and

a position detection device including sensors and road map data, said position detection device detecting first support information including a bearing, a position, and a mileage of the vehicle, wherein said parameter generating means uses the first support information in generating said camera attitude parameters, and

wherein said position detection device provides said parameter generating means with information associated with the acceleration of the vehicle, and said parameter generating means inhibits the generation of the camera attitude parameters when the acceleration exceeds a predetermined value.

3. A camera attitude parameter generating apparatus comprising:

an on-vehicle camera mounted on a vehicle to acquire an image ahead of or behind the vehicle;

means for detecting a pair of road parallel lines in the image acquired by said on-vehicle camera while the vehicle is traveling on a road, each of said road parallel lines being parallel with the longitudinal direction of the road;

means for obtaining the intersection of the pair of orad parallel lines, thereby generating the intersection as a road vanishing point;

parameter generating mean for generating at least one of camera attitude parameters on the basis of the pair of road parallel lines and the road vanishing point, said camera attitude parameters representing the attitude of said on-vehicle camera with respect to the road; and

a position detection device including sensors and road map data, said position detection device detecting first support information including a bearing, a position, and a mileage of the vehicle, wherein said parameter generating means uses the first support information in generating said camera attitude parameters, and

wherein said parameter generating means generates the camera attitude parameters at a predetermined interval of mileage or time, and generates averages of the generated camera attitude parameters as parameters associated with the attitude of the on-vehicle camera with respect to the vehicle.

4. A camera attitude parameter generating apparatus comprising:

an on-vehicle camera mounted on a vehicle to acquire an image ahead of or behind the vehicle;

road parallel line detecting means for detecting a pair of road parallel lines in the image acquired by said on-vehicle camera while the vehicle is traveling on a road, each of said road parallel lines being parallel with the longitudinal direction of the road;

road vanishing point generating means for obtaining the intersection of the pair of road parallel lines, thereby generating the intersection as a road vanishing point;

yaw angle observation means for observing the yaw angle of the on-vehicle camera with respect to the road on the basis of a coordinate of the road vanishing point, thereby outputting an observed yaw angle;

a position detection device having sensors and road map data, said position detection device detecting first support information including a bearing, a position, a mileage, and changes in the bearing of the vehicle;

yaw angle estimation means for estimating the yaw angle of the on-vehicle camera with respect to the road by applying the previously obtained yaw angle with respect to the road and the changes in the direction of the vehicle to a recursion formula prepared in advance, thereby outputting an estimated yaw angle; and

yaw angle generating means for generating the yaw angle of the on-vehicle camera with respect to the road by using at least one of the observed yaw angle and the estimated yaw angle,

wherein said yaw angle generating means obtains the difference between the estimated yaw angle and the observed yaw angle, obtains a weighted average of the estimated yaw angle and the observed yaw angle, and outputs the weighted average as the yaw angle, if the difference is within a predetermined range.

5. A camera attitude parameter generating apparatus comprising:

an on-vehicle camera mounted on a vehicle to acquire an image ahead of or behind the vehicle;

road parallel line detecting means for detecting a pair of road parallel lines in the image acquired by said on-vehicle camera while the vehicle is traveling on a road, each of said road parallel lines being parallel with the longitudinal direction of the road;

road vanishing point generating means for obtaining the intersection of the pair of road parallel lines, thereby generating the intersection as a road vanishing point;

yaw angle observation means for observing the yaw angle of the on-vehicle camera with respect to the road on the basis of a coordinate of the road vanishing point, thereby outputting an observed yaw angle;

a position detection device having sensors and road map data, said position detection device detecting first support information including a bearing, a position, a mileage, and changes in the bearing one the vehicle;

yaw angle estimation means for estimating the yaw angle of the on-vehicle camera with respect to the road by applying the previously obtained yaw angle with respect to the road and the changes in the direction of the vehicle to a recursion formula prepared in advance, thereby outputting an estimated yaw angle; and

yaw angle generating means for generating the yaw angle of the on-vehicle camera with respect to the road by using at least one of the observed yaw angle and the estimated yaw angle,

wherein said yaw angle generating means obtains the difference between the estimated yaw angle and the observed yaw angle, and outputs the estimated yaw angle as the yaw angle of the on-vehicle camera with respect to the road if the difference is greater than a predetermined value.

6. The camera attitude parameter generating apparatus as claimed in claim 5, wherein said yaw angle generating means outputs the observed yaw angle as the yaw angle of the on-vehicle camera with respect to the road when the estimated yaw angle has been made the yaw angle of the on-vehicle camera for more than a predetermined interval.

7. A camera attitude parameter generating apparatus comprising:

an on-vehicle camera mounted on a vehicle to acquire an image ahead of or behind the vehicle;

road parallel line detecting means for detecting a pair of road parallel lines in the image acquired by said on-vehicle camera while the vehicle is traveling on a road, each of said road parallel lines being parallel with the longitudinal direction of the road;

road vanishing point generating means for obtaining the intersection of the pair of road parallel lines, thereby generating the intersection as a road vanishing point;

yaw angle observation means for observing the yaw angle of the on-vehicle camera with respect to the road on the basis of a coordinate of the road vanishing point, thereby outputting an observed yaw angle;

a position detection device having sensors and road map data, said position detection device detecting first support information including a bearing, a position, a mileage, and changes in the bearing of the vehicle;

yaw angle estimation means for estimating the yaw angle of the on-vehicle camera with respect to the road by applying the previously obtained yaw angle with respect to the road and the changes in the direction of the vehicle to a recursion formula prepared in advance, thereby outputting an estimated yaw angle;

yaw angle generating means for generating the yaw angle of the on-vehicle camera with respect to the road by using at least one of the observed yaw angle and the estimated yaw angle;

slope detecting means for detecting the slope of each road parallel line outputted from the road parallel line generating means;

traverse deviation distance observation means for observing a traverse deviation distance of said on-vehicle camera from a predetermined reference line parallel with the road by using the slope of the road parallel lines detected by said slope detecting means and the height of said on-vehicle camera from the surface of the road, thereby outputting an observed traverse deviation distance;

traverse deviation distance estimation means for estimating the traverse deviation distance of said on-vehicle camera with respect to the road by obtaining the mileage of the vehicle included in the first support information supplied from said position detection device, and by applying a previously obtained traverse deviation distance of the on-vehicle camera with respect to the road and the mileage of the vehicle to a recursion formula prepared in advance; and

traverse deviation distance generating means for generating the traverse deviation distance of the on-vehicle camera with respect to the road by using at least one of the observed traverse deviation distance and the estimated traverse deviation distance.

8. The camera attitude parameter generating apparatus as claimed in claim 7, wherein said traverse deviation distance generating means selects said estimated traverse deviation distance as the traverse deviation distance of the on-vehicle camera with respect to the road when the observed traverse deviation distance is not obtained by the traverse deviation distance observation means.

9. The camera attitude parameter generating apparatus as claimed in claim 7, wherein said traverse deviation distance generating means obtains the difference between the estimated traverse deviation distance and the observed traverse deviation distance, and outputs the estimated traverse deviation distance as the traverse deviation distance of the on-vehicle camera with respect to the road if the difference is greater than a predetermined value.

10. The camera attitude parameter generating apparatus as claimed in claim 7, wherein said traverse deviation distance generating means suppresses the output of the traverse deviation distance of the on-vehicle camera with respect to the road until a turn of the vehicle has been completed when the turn is detected on the basis of the first support information supplied from the position detection device.