Object recognition apparatus and method

• US 6,285,393 B1
• Filed: 12/22/1997
• Issued: 09/04/2001
• Est. Priority Date: 09/08/1993
• Status: Expired due to Term

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Sumitomo Electric Industries Limited (Sumitomo Corporation)
• Inventors: Shimoura, Hiroshi, Tenmoku, Kenji
• Primary Examiner(s): Britton, Howard
• Assistant Examiner(s): DIEP, NHON THANH
• Application Number:
  US08/996,090
• Time in Patent Office: 1,352 Days
• Field of Search: 348/113, 348/116, 348/118, 348/119, 348/143, 348/148, 348/169, 382/103, 382/104, 382/106, 382/107, 364/460, 364/424.01, 364/424.02, 180/169
• US Class Current: 348/119
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An object recognition apparatus that can recognize an object in an image acquired by an on-vehicle camera mounted on a vehicle with high efficiency and accuracy. A recognition processing is carried out in a cut off range, which is cut off from the image, and in which an object to be recognized is supposed to be present. The cut off range is obtained using support information supplied from a position detection device and/or an anti-lock brake device of the vehicle. Since the object in the image changes its position as the vehicle moves, the displacement must be estimated during the object recognition. The estimation requires camera attitude parameters of the on-vehicle camera with respect to the road, and the estimation is performed using support information supplied from the position detection device and/of the anti-lock brake device. This will improve the efficiency and accuracy of obtaining the camera attitude parameters.

Claims

1. A camera attitude parameter generating apparatus comprising:

an on-vehicle camera mounted on a vehicle to acquire an image ahead of or behind the vehicle;

detecting means for detecting a pair of road parallel lines in the image acquired by said on vehicle camera while the vehicle is traveling on a road, said pair of road parallel lines being parallel with each other in the direction that the vehicle is traveling;

intersection obtaining means for obtaining the intersection the pair of road parallel lines in the image, thereby generating the intersection as a road vanishing point; and

parameter generating means for generating at least one of camera attitude parameters on the basis of the pair of road parallel lines and the road vanishing point, said camera attitude parameters representing the attitude of said on-vehicle camera with respect to the road.

2. The camera attitude parameter generating apparatus as claimed in claim 1, wherein said camera attitude parameters include the yaw angle, roll angle, pitch angle of said on-vehicle camera with respect to a road, and a traverse deviation distance of the vehicle from a predetermined first reference line associated with the road.

3. The camera attitude parameter generating apparatus as claimed in claim 1, further comprising a position detection device including sensors and road map data, said position detection device detecting first support information including a bearing, a position, and a mileage of the vehicle, wherein said parameter generating means uses the first support information in generating said camera attitude parameters.

4. The camera attitude parameter generating apparatus as claimed in claim 3, wherein said position detection device provides said parameter generating means with the bearing of the vehicle and a bearing of the road from the position detection device, and said parameter generating means obtains an angle between the bearing of the vehicle and the road parallel lines on the basis of the difference between the bearing of the vehicle and the bearing of the road, and uses the angle for calculating said camera attitude parameters.

5. The camera attitude parameter generating apparatus as claimed in claim 4, wherein said parameter generating means considers the angle between the bearing of the vehicle and the road parallel lines to be zero if the difference between the change in the bearing of the vehicle and the change in the bearing of the road while the vehicle travels a predetermined distance is less than a predetermined value.

6. The camera attitude parameter generating apparatus as claimed in claim 4, wherein said parameter generating means considers the angle between the bearing of the vehicle and the road parallel lines to be zero if the change in the bearing of the vehicle continues to be less than a predetermined value for a predetermined distance.

7. The camera attitude parameter generating apparatus as claimed in claim 4, wherein said parameter generating means generates the attitude parameters only when the angle between the bearing of the vehicle and the road parallel lines is zero.

8. The camera attitude parameter generating apparatus as claimed in claim 3, wherein said position detection device provides said parameter generating means with information associated with the speed of the vehicle, and said parameter generating means generates the camera attitude parameters when the speed of the vehicle is within a predetermined range.

9. The camera attitude parameter generating apparatus as claimed in claim 3, further comprising a brake control device having second support information for controlling brakes of the vehicle, wherein said parameter generating means generates the camera attitude parameters using the second support information in addition to the first support information supplied from the position detection device.

10. The camera attitude parameter generating apparatus as claimed in claim 9, wherein said brake control device has information associated with the acceleration of the vehicle, and said parameter generating means inhibits the generation of the camera attitude parameters when the acceleration exceeds a predetermined value.

11. The camera attitude parameter generating apparatus as claimed in claim 9, wherein said brake control device has information associated with an operation of the brakes, and said parameter generating means inhibits the generation of the camera attitude parameters when at least one of the brakes is operated.

12. A camera attitude parameter generating apparatus comprising:

an on-vehicle camera mounted on a vehicle to acquire an image ahead of or behind the vehicle;

road parallel line detecting means for detecting a pair of road parallel lines in the image acquired by said on-vehicle camera while the vehicle is traveling on a road, said pair of road parallel lines being parallel with each other in the direction that the vehicle is traveling;

road vanishing point generating means for obtaining the intersection of the pair of road parallel lines in the image, thereby generating the intersection as a road vanishing point;

yaw angle observation means for observing the yaw angle of the on-vehicle camera with respect to the road on the basis of a coordinate of the road vanishing point, thereby outputting an observed yaw angle;

a position detection device having sensors and road map data, said position detection device detecting first support information including a bearing, a position, a mileage, and changes in the bearing of the vehicle;

yaw angle estimation means for estimating the yaw angle of the on-vehicle camera with respect to the road by applying the previously obtained yaw angle with respect to the road and the changes in the direction of the vehicle to a recursion formula prepared in advance, thereby outputting an estimated yaw angle; and

yaw angle generating means for generating the yaw angle of the on-vehicle camera with respect to the road by using at least one of the observed yaw angle and the estimated yaw angle.

13. The camera attitude parameter generating apparatus as claimed in claim 12, wherein said yaw angle generating means selects said estimated yaw angle as the yaw angle of the on-vehicle camera with respect to the road when the observed yaw angle is not obtained by the yaw angle observation means.

14. The camera attitude parameter generating apparatus as claimed in claim 12, wherein said yaw angle generating means suppresses the output of the yaw angle of the on-vehicle camera with respect to the road until a turn of the vehicle has been completed when the turn is detected on the basis of the first support information supplied from the position detection device.

15. The camera attitude parameter generating apparatus as claimed in claim 12, further comprising:

pitch angle observation means for observing a pitch angle of the on-vehicle camera with respect to the road on the basis of a coordinate of the road vanishing point outputted from the road vanishing point generating means; and

pitch angle generating means for estimating, if the road vanishing point is not obtained by said road vanishing point generating means and only a single road parallel line is obtained by said road parallel line detecting means, the coordinates of the road vanishing point on the basis of the single road parallel line and the yaw angle of the on-vehicle camera with respect to the road outputted from the yaw angle generating means, and outputting the pitch angle of the on-vehicle camera with respect to the road on the basis of the estimated coordinates of the road vanishing point.

16. The camera attitude parameter generating apparatus as claimed in claim 15, wherein said pitch angle generating means suppresses the output of the pitch angle of the on-vehicle camera with respect to the road until a turn of the vehicle has been completed when the turn is detected on the basis of the first support information supplied from the position detection device.

17. A camera attitude parameter generating method comprising:

detecting a pair of road parallel lines in an image acquired by an on-vehicle camera, which is mounted on a vehicle to acquire an image ahead of or behind the vehicle while the vehicle is traveling on a road, said pair of road parallel lines being parallel with each other in the direction that the vehicle is traveling;

obtaining the intersection of the pair of road parallel lines in the image, thereby generating the intersection as a road vanishing point; and

generating at least one of camera attitude parameters on the basis of the pair of road parallel lines and the road vanishing point, said camera attitude parameters representing the attitude of said on-vehicle camera with respect to the road.

18. A camera attitude parameter generating method comprising:

detecting a pair of road parallel lines in the image acquired by an on-vehicle camera, which is mounted on a vehicle to acquire an image ahead of or behind the vehicle while the vehicle is traveling on a road, said pair of road parallel lines being parallel with each other in the direction that the vehicle is traveling;

obtaining the intersection of the pair of road parallel lines in the image, thereby generating the intersection as a road vanishing point;

observing the yaw angle of the on-vehicle camera with respect to the road on the basis of a coordinate of the road vanishing point, thereby outputting an observed yaw angle;

detecting, by using sensors and road map data, first support information including a bearing, a position, a mileage, and changes in the bearing of the vehicle;

estimating the yaw angle of the on-vehicle camera with respect to the road by applying the previously obtained yaw angle with respect to the road and the changes in the direction of the vehicle to a recursion formula prepared in advance, thereby outputting an estimated yaw angle; and

generating the yaw angle of the on-vehicle camera with respect to the road by using at least one of the observed yaw angle and the estimated yaw angle.

19. A camera attitude parameter generating apparatus comprising:

an on-vehicle camera mounted on a vehicle to acquire an image ahead of or behind the vehicle;

means for detecting a pair of road parallel lines in the image acquired by said on-vehicle camera while the vehicle is traveling on a road, each of said pair of road parallel lines being parallel with the longitudinal direction of the road;

means for obtaining the intersection of the pair of road parallel lines, thereby generating the intersection as a road vanishing point; and

parameter generating means for generating at last one of camera attitude parameters on the basis of the pair of road parallel lines and the road vanishing point, said camera attitude parameters representing the attitude of said on-vehicle camera with respect to the road; and

recognizing means for recognizing another object excepting the road by using an image obtained by said on-vehicle camera.