Field recognition apparatus, method for field recognition and program for the same

• US 8,036,424 B2
• Filed: 04/11/2007
• Issued: 10/11/2011
• Est. Priority Date: 04/27/2006
• Status: Active Grant

• Current Assignee: DENSO Corporation, Takanori Fukao
• Original Assignee: DENSO Corporation, Takanori Fukao
• Inventors: Fukao, Takanori, Ishikawa, Takahiro
• Primary Examiner(s): Le; Vu
• Assistant Examiner(s): LIEW, ALEX KOK SOON
• Application Number:
  US11/786,152
• Publication Number:
  US 20070285217A1
• Time in Patent Office: 1,644 Days
• Field of Search: 382/103, 382173-180
• US Class Current: 382/103
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A field recognition apparatus includes a first camera for imaging a forward field and a second camera for imaging a road surface. The second camera has a light axis tilted toward a downward direction for accurately tracking a characteristic point in consecutively captured images to calculate an optical flow and for accurately recognizing road structure information. In this manner, three dimensional information of obstacles in a field is accurately processed and detected.

Claims

1. A field recognition apparatus for use in a movable body comprising:

a first imaging unit disposed on the movable body imaging a field in a proximity of the movable body;

a second imaging unit disposed on the movable body imaging a road surface that is traveled by the movable body, wherein the second imaging unit has a light axis pointing downward in comparison to the light axis of the first imaging unit;

a movement detection unit detecting an optical flow of a characteristic point on the road surface based on an image of the road surface derived from the second imaging unit, wherein the optical flow of the characteristic point is used to determine a movement parameter that specifies the movement of the movable body; and

an obstacle detection unit calculating three-dimensional data of a position and a shape of an obstacle based on the movement parameter derived from the movement detection unit and the image derived from the first imaging unit;

wherein the obstacle detection unit includes;

a division unit dividing the image derived from the first imaging unit into plural image pieces;

a divisional parameter calculation unit calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection unit; and

a mapping unit mapping each of the plural image pieces by using three-dimensional coordinates based on the divisional parameter derived from the divisional parameter calculation unit and the position of each of the plural image pieces in the image derived from the first imaging unit.

3. The field recognition apparatus as in claim 1,

wherein the first imaging unit is disposed to have the light axis aligned in one of a horizontal direction and a direction lower than the horizontal direction toward a front field of the movable body.

4. The field recognition apparatus as in claim 3,

wherein the image derived from the first imaging unit has a landscape shape that has a greater length in a lateral direction.

5. The field recognition apparatus as in claim 1,

wherein the second imaging unit has the light axis aligned in a vertically downward direction.

6. The field recognition apparatus as in claim 1,

wherein the second imaging unit has the light axis tilted from a vertically downward direction toward a rear direction of the movable body.

7. The field recognition apparatus as in claim 1 further comprising:

a position control unit controlling a tilt angle of the second imaging unit toward a reverse direction of movable body movement to have a greater value in an occasion that a speed of the movable body movement increases so that two consecutive images derived from the second imaging unit have an identical characteristic point included therein.

8. The field recognition apparatus as in claim 5,

wherein the image derived from the second imaging unit has a portrait shape that has a greater length in a longitudinal direction.

9. The field recognition apparatus as in claim 1,

wherein the second imaging unit is so disposed in terms of an angle and a position as to image at least a left marker on the road surface that shows a left edge of a lane, or one of a road side object that is disposed on a roadside of a left side traffic and a median at a center of a road in a right side traffic.

10. The field recognition apparatus as in claim 1,

wherein the second imaging unit is so disposed in terms of an angle and a position as to image at least a right marker on the road surface that shows a right edge of a lane, or one of a road side object that is disposed on a roadside of a right side traffic and a median at a center of a road in a left side traffic.

11. The field recognition apparatus as in claim 1,

wherein the second imaging unit is disposed at a rear side of the movable body, and

the second imaging unit additionally serves as a rear view camera for monitoring a dead angle of a driver.

12. The field recognition apparatus as in claim 1,

wherein the second imaging unit is disposed on a bottom surface of the movable body.

13. The field recognition apparatus as in claim 12,

wherein the second imaging unit is disposed on a front portion of the movable body relative to a center of the movable body.

14. The field recognition apparatus as in claim 1 further comprising:

a position control unit controlling an angle and a position of the second imaging unit so that the image derived from the second imaging unit is prevented from including a shadow of a subject movable body, a shadow of another movable body that travels along the subject movable body, or a shadow of a roadside object.

15. The field recognition apparatus as in claim 1 further comprising:

an illumination unit illuminating an imaging area of the second imaging unit.

16. The field recognition apparatus as in claim 1,

wherein at least one of the first imaging unit and the second imaging unit generates the image having a luminous intensity value for respective pixels.

17. The field recognition apparatus as in claim 1,

wherein at least one of the first imaging unit and the second imaging unit generates the image having a chroma value for respective pixels.

18. The field recognition apparatus as in claim 1,

wherein at least one of the first imaging unit and the second imaging unit generates the image having an infrared ray intensity value for respective pixels.

19. The field recognition apparatus as in claim 1,

wherein at least one of the first imaging unit and the second imaging unit generates the image having an edge emphasis.

24. The field recognition apparatus as in claim 6,

wherein the image derived from the second imaging unit has a portrait shape that has a greater length in a longitudinal direction.

25. The field recognition apparatus as in claim 7,

wherein the image derived from the second imaging unit has a portrait shape that has a greater length in a longitudinal direction.

2. A field recognition apparatus for use in a movable body comprising:

a first imaging unit disposed on the movable body imaging a field in a proximity of the movable body;

a second imaging unit disposed on the movable body imaging a road surface that is traveled by the movable body, wherein the second imaging unit has a light axis pointing downward in comparison to the light axis of the first imaging unit;

a movement detection unit detecting an optical flow of a characteristic point on the road surface based on an image of the road surface derived from the second imaging unit, wherein the optical flow of the characteristic point is used to determine a movement parameter that specifies the movement of the movable body; and

an obstacle detection unit calculating three-dimensional data of a position and a shape of an obstacle based on the movement parameter derived from the movement detection unit and the image derived from the first imaging unit;

a division unit dividing an image derived from the first imaging unit into plural image pieces;

a movement correction unit correcting the movement parameter derived from the movement detection unit based on extraction of road pieces from the plural image pieces as the road surface that includes the optical flow of the characteristic point;

a divisional parameter calculation unit calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection unit; and

a mapping unit mapping each of the plural image pieces by using three-dimensional coordinate based on the divisional parameter derived from the divisional parameter calculation unit and the position of each of the plural image pieces in the image derived from the first imaging unit.

26. The field recognition apparatus as in claim 2,

wherein the first imaging unit is disposed to have the light axis aligned in one of a horizontal direction and a direction lower than the horizontal direction toward a front field of the movable body.

27. The field recognition apparatus as in claim 26,

wherein the image derived from the first imaging unit has a landscape shape that has a greater length in a lateral direction.

28. The field recognition apparatus as in claim 2,

wherein the second imaging unit has the light axis aligned in a vertically downward direction.

29. The field recognition apparatus as in claim 28,

wherein the image derived from the second imaging unit has a portrait shape that has a greater length in a longitudinal direction.

30. The field recognition apparatus as in claim 2,

wherein the second imaging unit has the light axis tilted from a vertically downward direction toward a rear direction of the movable body.

31. The field recognition apparatus as in claim 30,

wherein the image derived from the second imaging unit has a portrait shape that has a greater length in a longitudinal direction.

32. The field recognition apparatus as in claim 2 further comprising:

a position control unit controlling a tilt angle of the second imaging unit toward a reverse direction of movable body movement to have a greater value in an occasion that a speed of the movable body movement increases so that two consecutive images derived from the second imaging unit have an identical characteristic point included therein.

33. The field recognition apparatus as in claim 32,

wherein the image derived from the second imaging unit has a portrait shape that has a greater length in a longitudinal direction.

34. The field recognition apparatus as in claim 2,

wherein the second imaging unit is so disposed in terms of an angle and a position as to image at least a left marker on the road surface that shows a left edge of a lane, or one of a road side object that is disposed on a roadside of a left side traffic and a median at a center of a road in a right side traffic.

35. The field recognition apparatus as in claim 2,

wherein the second imaging unit is so disposed in terms of an angle and a position as to image at least a right marker on the road surface that shows a right edge of a lane, or one of a road side object that is disposed on a roadside of a right side traffic and a median at a center of a road in a left side traffic.

36. The field recognition apparatus as in claim 2,

wherein the second imaging unit is disposed at a rear side of the movable body, and

the second imaging unit additionally serves as a rear view camera for monitoring a dead angle of a driver.

37. The field recognition apparatus as in claim 2,

wherein the second imaging unit is disposed on a bottom surface of the movable body.

38. The field recognition apparatus as in claim 37,

wherein the second imaging unit is disposed on a front portion of the movable body relative to a center of the movable body.

39. The field recognition apparatus as in claim 2 further comprising:

a position control unit controlling an angle and a position of the second imaging unit so that the image derived from the second imaging unit is prevented from including a shadow of a subject movable body, a shadow of another movable body that travels along the subject movable body, or a shadow of a roadside object.

40. The field recognition apparatus as in claim 2 further comprising:

an illumination unit illuminating an imaging area of the second imaging unit.

41. The field recognition apparatus as in claim 2,

wherein at least one of the first imaging unit and the second imaging unit generates the image having a luminous intensity value for respective pixels.

42. The field recognition apparatus as in claim 2,

wherein at least one of the first imaging unit and the second imaging unit generates the image having a chroma value for respective pixels.

43. The field recognition apparatus as in claim 2,

wherein at least one of the first imaging unit and the second imaging unit generates the image having an infrared ray intensity value for respective pixels.

44. The field recognition apparatus as in claim 2,

wherein at least one of the first imaging unit and the second imaging unit generates the image having an edge emphasis.

20. A field recognition method used in a field recognition apparatus for recognizing an obstacle in a proximity of a movable body in terms of a position and a shape in a three-dimensional space based on images derived from a first imaging unit of the field recognition apparatus for imaging a field in the proximity of the .movable body and a second imaging unit of the field recognition apparatus with a light axis aligned downward relative to the light axis of the first imaging unit for imaging a road surface that is traveled by the movable body, the method comprising:

detecting an optical flow of a characteristic point on the road surface based on the image of the road surface derived from the second imaging unit, wherein the optical flow of the characteristic point is used to determine a movement parameter that specifies the movement of the movable body; and

calculating three-dimensional data of the position and the shape of the obstacle based on the movement parameter derived from a preceding step and the image derived from the first imaging unit;

wherein

calculating the three-dimensional data of the position and the shape of the obstacle further includes;

dividing the image derived from the first imaging unit into plural image pieces;

calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection in a preceding procedure; and

mapping each of the plural image pieces by using three-dimensional coordinates based on the divisional parameter derived from the divisional parameter calculated in a preceding procedure and the position of each of the plural image pieces in the image derived from the first imaging unit.

21. A field recognition method used in a field recognition apparatus for recognizing an obstacle in a proximity of a movable body in terms of a position and a shape in a three-dimensional space based on images derived from a first imaging unit of the field recognition apparatus for imaging a field in the proximity of the movable body and a second imaging unit of the field recognition apparatus with a light axis aligned downward relative to the light axis of the first imaging unit for imaging a road surface that is traveled by the movable body, the method comprising:

detecting an optical flow of a characteristic point on the road surface based on the image of the road surface derived from the second imaging unit, wherein the optical flow of the characteristic point is used to determine a movement parameter that specifies the movement of the movable body; and

calculating three-dimensional data of the position and the shape of the obstacle based on the movement parameter derived from a preceding step and the image derived from the first imaging unit;

wherein

calculating the three-dimensional data of the position and the shape of the obstacle further includes;

dividing the image derived from the first imaging unit into plural image pieces;

correcting the movement parameter derived from the movement detection in a preceding step based on extraction of road pieces from the plural image pieces as the road surface that includes the optical flow of the characteristic point;

calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection in a preceding procedure; and

mapping each of the plural image pieces by using three-dimensional coordinates based on the divisional parameter derived from the divisional parameter calculated in a preceding procedure and the position of each of the plural image pieces in the image derived from the first imaging unit.

22. A program recorded on a non-transitory computer-readable medium for a procedure of field recognition by a computer, the program for use in a movable body comprising:

providing a first imaging unit and a second imaging unit;

detecting an optical flow of a characteristic point on a road surface based on an image of the road surface derived from a second imaging unit, wherein the optical flow of the characteristic point is used to determine a movement parameter that specifies the movement of the movable body; and

calculating three-dimensional data of a position and a shape of the obstacle based on the movement parameter derived from the movement detection in a preceding step and an image derived from a first imaging unit,

wherein the first imaging unit and the second imaging unit are provided on the movable body,

the first imaging unit images a field in the proximity of the movable body, and

the second imaging unit with a light axis aligned downward relative to the light axis of the first imaging unit for imaging a road surface that is traveled by the movable body;

wherein the step of calculating the three-dimensional data of the position and the shape of the obstacle further includes;

dividing the image derived from the first imaging unit into plural image pieces;

calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection in a preceding procedure; and

mapping each of the plural image pieces by using three-dimensional coordinates based on the divisional parameter derived from the divisional parameter calculated in a preceding procedure and the position of each of the plural image pieces in the image derived from the first imaging unit.

23. A program recorded on a non-transitory computer-readable medium for a procedure of field recognition by a computer, the program for use in a movable body comprising:

providing a first imaging unit and a second imaging unit;

detecting an optical flow of a characteristic point on a road surface based on an image of the road surface derived from a second imaging unit, wherein the optical flow of the characteristic point is used to determine a movement parameter that specifies the movement of the movable body; and

calculating three-dimensional data of a position and a shape of the obstacle based on the movement parameter derived from the movement detection in a preceding step and an image derived from a first imaging unit,

wherein the first imaging unit and the second imaging unit are provided on the movable body,

the first imaging unit images a field in the proximity of the movable body, and

the second imaging unit with a light axis aligned downward relative to the light axis of the first imaging unit for imaging a road surface that is traveled by the movable body;

wherein the step of calculating the three-dimensional data of the position and the shape of the obstacle further includes;

dividing the image derived from the first imaging unit into plural image pieces;

correcting the movement parameter derived from the movement detection in a preceding step based on extraction of road pieces from the plural image pieces as the road surface that includes the optical flow of the characteristic point;

calculating a divisional parameter in each of the plural image pieces, wherein the divisional parameter specifies a position and an orientation of each of the plural image pieces in a three-dimensional field based on the optical flow of the characteristic point in each of the plural image pieces and the movement parameter derived from the movement detection in a preceding procedure; and

mapping each of the plural image pieces by using three-dimensional coordinates based on the divisional parameter derived from the divisional parameter calculated in a preceding procedure and the position of each of the plural image pieces in the image derived from the first imaging unit.