Field recognition apparatus, method for field recognition and program for the same

• US 20070285217A1
• Filed: 04/11/2007
• Published: 12/13/2007
• Est. Priority Date: 04/27/2006
• Status: Active Grant

• Current Assignee: DENSO Corporation, Takanori Fukao
• Original Assignee: DENSO Corporation, Takanori Fukao
• Inventors: Ishikawa, Takahiro, Fukao, Takanori
• Granted Patent: US 8,036,424 B2
• Time in Patent Office: Days
• US Class Current: 340/435
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A field recognition apparatus includes a first camera for imaging a forward field and a second camera for imaging a road surface. The second camera has a light axis tilted toward a downward direction for accurately tracking a characteristic point in consecutively captured images to calculate an optical flow and for accurately recognizing road structure information. In this manner, three dimensional information of obstacles in a field is accurately processed and detected.

Claims

1. A field recognition apparatus for use in a movable body comprising:

a first imaging unit disposed on the movable body for imaging a field in a proximity of the movable body;

a second imaging unit disposed on the movable body for imaging a road surface that is traveled by the movable body, wherein the second imaging unit has a light axis pointing downward in comparison to the light axis of the first imaging unit;

a movement detection unit for detecting an optical flow of a characteristic point on the road surface based on an image of the road surface derived from the second imaging unit, wherein the optical flow of the characteristic point is used to determine a movement parameter that specifies the movement of the movable body; and

an obstacle detection unit for calculating three-dimensional data of a position and a shape of an obstacle based on the movement parameter derived from the movement detection unit and the image derived from the first imaging unit.

2. The field recognition apparatus as in claim 1,

wherein the obstacle detection unit includes:

a division unit for dividing the image derived from the first imaging unit into plural image pieces;

a divisional parameter calculation unit for calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection unit; and

a mapping unit for mapping each of the plural image pieces by using three-dimensional coordinates based on the divisional parameter derived from the divisional parameter calculation unit and the position of each of the plural image pieces in the image derived from the first imaging unit.

3. The field recognition apparatus as in claim 1,

a division unit for dividing an image derived from the first imaging unit into plural image pieces;

a movement correction unit for correcting the movement parameter derived from the movement detection unit based on extraction of road pieces from the plural image pieces as the road surface that includes the optical flow of the characteristic point;

a divisional parameter calculation unit for calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection unit; and

a mapping unit for mapping each of the plural image pieces by using three-dimensional coordinate based on the divisional parameter derived from the divisional parameter calculation unit and the position of each of the plural image pieces in the image derived from the first imaging unit.

4. The field recognition apparatus as in claim 1,

wherein the first imaging unit is disposed to have the light axis aligned in one of a horizontal direction and a direction lower than the horizontal direction toward a front field of the movable body.

5. The field recognition apparatus as in claim 4,

wherein the image derived from the first imaging unit has a landscape shape that has a greater length in a lateral direction.

6. The field recognition apparatus as in claim 1,

wherein the second imaging unit has the light axis aligned in a vertically downward direction.

7. The field recognition apparatus as in claim 1,

wherein the second imaging unit has the light axis tilted from a vertically downward direction toward a rear direction of the movable body.

8. The field recognition apparatus as in claim 1 further comprising:

a position control unit for controlling a tilt angle of the second imaging unit toward a reverse direction of movable body movement to have a greater value in an occasion that a speed of the movable body movement increases so that two consecutive images derived from the second imaging unit have an identical characteristic point included therein.

9. The field recognition apparatus as in claim 6,

wherein the image derived from the second imaging unit has a portrait shape that has a greater length in a longitudinal direction.

10. The field recognition apparatus as in claim 1,

wherein the second imaging unit is so disposed in terms of an angle and a position as to image at least a left marker on the road surface that shows a left edge of a lane, or one of a road side object that is disposed on a roadside of a left side traffic and a median at a center of a road in a right side traffic.

11. The field recognition apparatus as in claim 1,

wherein the second imaging unit is so disposed in terms of an angle and a position as to image at least a right marker on the road surface that shows a right edge of a lane, or one of a road side object that is disposed on a roadside of a right side traffic and a median at a center of a road in a left side traffic.

12. The field recognition apparatus as in claim 1,

wherein the second imaging unit is disposed at a rear side of the movable body, and

the second imaging unit additionally serves as a rear view camera for monitoring a dead angle of a driver

13. The field recognition apparatus as in claim 1,

wherein the second imaging unit is disposed on a bottom surface of the movable body.

14. The field recognition apparatus as in claim 13,

wherein the second imaging unit is disposed on a front portion of the movable body relative to a center of the movable body.

15. The field recognition apparatus as in claim 1 further comprising:

a position control unit for controlling an angle and a position of the second imaging unit so that the image derived from the second imaging unit is prevented from including a shadow of a subject movable body, a shadow of another movable body that travels along the subject movable body, or a shadow of a roadside object.

16. The field recognition apparatus as in claim 1 further comprising:

an illumination unit for illuminating an imaging area of the second imaging unit.

17. The field recognition apparatus as in claim 1,

wherein at least one of the first imaging unit and the second imaging unit generates the image having a luminous intensity value for respective pixels.

18. The field recognition apparatus as in claim 1,

wherein at least one of the first imaging unit and the second imaging unit generates the image having a chroma value for respective pixels.

19. The field recognition apparatus as in claim 1,

wherein at least one of the first imaging unit and the second imaging unit generates the image having an infrared ray intensity value for respective pixels.

20. The field recognition apparatus as in claim 1,

wherein at least one of the first imaging unit and the second imaging unit generates the image having an edge emphasis.

21. A field recognition method for recognizing an obstacle in a proximity of a movable body in terms of a position and a shape in a three-dimensional space based on images derived from a first imaging unit for imaging a field in the proximity of the movable body and a second imaging unit with a light axis aligned downward relative to the light axis of the first imaging unit for imaging a road surface that is traveled by the movable body, the method comprising:

detecting an optical flow of a characteristic point on the road surface based on the image of the road surface derived from the second imaging unit, wherein the optical flow of the characteristic point is used to determine a movement parameter that specifies the movement of the movable body; and

calculating three-dimensional data of the position and the shape of the obstacle based on the movement parameter derived from a preceding step and the image derived from the first imaging unit.

22. The field recognition method as in claim 21,

wherein calculation procedure of the three-dimensional data of the position and the shape of the obstacle further includes:

dividing the image derived from the first imaging unit into plural image pieces;

calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection in a preceding procedure; and

mapping each of the plural image pieces by using three-dimensional coordinates based on the divisional parameter derived from the divisional parameter calculated in a preceding procedure and the position of each of the plural image pieces in the image derived from the first imaging unit.

23. The method as in claim 21,

wherein calculation procedure of the three-dimensional data of the position and the shape of the obstacle further includes:

dividing the image derived from the first imaging unit into plural image pieces;

correcting the movement parameter derived from the movement detection in a preceding step based on extraction of road pieces from the plural image pieces as the road surface that includes the optical flow of the characteristic point;

calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection in a preceding procedure; and

mapping each of the plural image pieces by using three-dimensional coordinates based on the divisional parameter derived from the divisional parameter calculated in a preceding procedure and the position of each of the plural image pieces in the image derived from the first imaging unit.

24. A program recorded on a retrievable medium for a procedure of field recognition by a computer, the program for use in a movable body comprising:

step of providing a first imaging unit and a second imaging unit;

step of detecting an optical flow of a characteristic point on a road surface based on an image of the road surface derived from a second imaging unit, wherein the optical flow of the characteristic point is used to determine a movement parameter that specifies the movement of the movable body; and

step of calculating three-dimensional data of a position and a shape of the obstacle based on the movement parameter derived from the movement detection in a preceding step and an image derived from a first imaging unit,

wherein the first imaging unit and the second imaging unit are provided on the movable body,

the first imaging unit images a field in the proximity of the movable body, and

the second imaging unit with a light axis aligned downward relative to the light axis of the first imaging unit for imaging a road surface that is traveled by the movable body.

25. The program as in claim 24,

wherein step of calculating the three-dimensional data of the position and the shape of the obstacle further includes:

sub-step of dividing the image derived from the first imaging unit into plural image pieces;

sub-step of calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection in a preceding procedure; and

sub-step of mapping each of the plural image pieces by using three-dimensional coordinates based on the divisional parameter derived from the divisional parameter calculated in a preceding procedure and the position of each of the plural image pieces in the image derived from the first imaging unit.

26. The program as in claim 24,

wherein step of calculating the three-dimensional data of the position and the shape of the obstacle further includes:

sub-step of dividing the image derived from the first imaging unit into plural image pieces;

sub-step of correcting the movement parameter derived from the movement detection in a preceding step based on extraction of road pieces from the plural image pieces as the road surface that includes the optical flow of the characteristic point;

sub-step of calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection in a preceding procedure; and

sub-step of mapping each of the plural image pieces by using three-dimensional coordinates based on the divisional parameter derived from the divisional parameter calculated in a preceding procedure and the position of each of the plural image pieces in the image derived from the first imaging unit.