Method for processing stereo vision data using image density

• US 7,587,081 B2
• Filed: 01/30/2006
• Issued: 09/08/2009
• Est. Priority Date: 09/28/2005
• Status: Active Grant

• Current Assignee: Board of Trustees of The University of Illinois, Deere & Company
• Original Assignee: Deere & Company, University of Illinois
• Inventors: Zhang, Qin, Rovira-Mas, Francisco, Reid, John Franklin
• Primary Examiner(s): Bali; Vikkram
• Assistant Examiner(s): Zeilberger; Daniel
• Application Number:
  US11/343,142
• Publication Number:
  US 20070071311A1
• Time in Patent Office: 1,317 Days
• Field of Search: 382/103, 382/104, 382/154, 345/419, 701/301
• US Class Current: 382/154
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An image collection system collects stereo image data comprising left image data and right image data of a particular scene. An estimator estimates a reference three dimensional image density rating for corresponding reference volumetric cell within the particular scene at a reference range from an imaging unit. An image density compensator compensates for the change in image density rating between the reference volumetric cell and an evaluated volumetric cell by using a compensated three dimensional image density rating based on a range or relative displacement between the reference volumetric cell and the evaluated volumetric cell. An obstacle detector identifies a presence or absence of an obstacle in a particular compensated volumetric cell in the particular scene if the particular compensated volumetric cell meets or exceeds a minimum three dimensional image density rating

Claims

1. A method for processing stereo vision data using image density, the method comprising:

collecting, by an image collection system, image data comprising left image data and right image data of a particular scene;

estimating an evaluated three dimensional image density rating for one or more respective evaluated volumetric cells within the particular scene;

estimating a reference three dimensional image density rating for corresponding reference volumetric cell within the particular scene at a reference range from an imaging unit;

compensating, using an image processor, for the change in image density rating between the reference volumetric cell and each evaluated volumetric cell by using a compensated image density rating based on a range or relative displacement between the reference volumetric cell and the evaluated volumetric cell, and by establishing a compensation curve as a function of compensated three dimensional image density versus range, for volumetric cells between the reference volumetric cell and the evaluated volumetric cell; and

identifying a presence or absence of an obstacle in a particular compensated volumetric cell in the particular scene if the particular compensated volumetric cell meets or exceeds a minimum three dimensional image density rating.

2. The method according to claim 1 wherein the evaluated three dimensional image density rating comprises the number of stereo matched points or pixels within the first volumetric cell;

and the reference three dimensional image density rating comprises the number of stereo matched points or pixels within a second volumetric cell.

3. The method according to claim 1 wherein the volume of the volumetric cells remains constant prior to and after the compensation for the change in image density rating.

7. The method according to claim 1 wherein the compensating comprises establishing a compensation curve that is a function of baseline between a left imaging unit and a right imaging unit of the imaging unit, focal length of the imaging unit, and pixel size of the imaging unit.

8. The method according to claim 1 wherein the compensating conforms to the following equation:

$d<br><br><br><br>3<br><br><br><br>D_C<br><br>\left(Y\right)=d<br><br><br><br>3<br><br><br><br>D<br><br>\left(Y\right)·<br><br>\frac{d<br><br><br><br>3<br><br>D_{\mathrm{TH}}<br><br>\left(Y_R\right)}{d<br><br><br><br>3<br><br>D_{\mathrm{TH}}<br><br>\left(Y\right)},$

where the d3D_c(Y) is the compensated three dimensional image density as a function of the range Y, Y_R is the reference range, where d3D_TH=K_s·

10⁵·

Y<sup>−

2</sup>, and where

$K_s=\frac{b·<br><br>f}{w},$

where w is the pixel size, b is the baseline, f is the focal length of the lenses of the imaging unit.

4. The method according to clam 3 wherein the reference three dimensional image density comprises an image density associated with at least one of an average range, a mode range, or a mean range.

5. The method according to claim 4 wherein the compensating applies to volumetric cells having a further range from the imaging unit than the reference volumetric cell.

6. The method according to claim 5 wherein the compensating is not applied to volumetric cells having closer range from the imaging unit than the reference volumetric cell.

9. A system for processing stereo vision data using image density, the system comprising:

an image collection system for collecting image data comprising left image data and right image data of a particular scene;

an estimator for estimating an evaluated three dimensional image density rating for one or more respective evaluated volumetric cells within the particular scene and for estimating a reference three dimensional image density rating for corresponding reference volumetric cell within the particular scene at a reference range from an imaging unit;

an image density compensator for compensating for the change in image density rating between each reference volumetric cell and an evaluated volumetric cell by using a compensated image density rating based on a range or relative displacement between the reference volumetric cell and the evaluated volumetric cell, and wherein the image density compensator establishes a compensation curve as a function of compensated image density versus range for volumetric cells between the reference volumetric cell and the evaluated volumetric cell; and

an object detector for identifying a presence or absence of an obstacle in a particular compensated volumetric cell in the particular scene if the particular compensated volumetric cell meets or exceeds a minimum three dimensional image density rating.

10. The system according to claim 9 wherein the evaluated three dimensional image density rating comprises the number of stereo matched points or pixels within the first volumetric cell;

and the reference three dimensional image density rating comprises the number of stereo matched points or pixels within a second volumetric cell.

11. The system according to claim 9 wherein the volume of the volumetric cells remains constant prior to and after the compensation for the change in image density rating.

12. The system according to claim 9 wherein the reference three dimensional image density comprises an image density associated with at least one of an average range, a mode range, or a mean range.

13. The system according to claim 12 wherein the image density compensator applies compensation for intra-image density variation to volumetric cells having a further range from the imaging unit than the reference volumetric cell.

14. The system according to claim 13 wherein the image density compensator does not apply compensation for intra-image density variation for volumetric cells having closer range from the imaging unit than the reference volumetric cell.

15. The system according to claim 9 wherein the image density compensator comprises establishing a compensation curve that is a function of baseline between a left imaging unit and a right imaging unit of the imaging unit, focal length of the imaging unit, and pixel size of the image collection system.

16. The system according to claim 9 wherein the image density compensator conforms to the following equation:

$d<br><br><br><br>3<br><br><br><br>D_C<br><br>\left(Y\right)=d<br><br><br><br>3<br><br><br><br>D<br><br>\left(Y\right)·<br><br>\frac{d<br><br><br><br>3<br><br>D_{\mathrm{TH}}<br><br>\left(Y_R\right)}{d<br><br><br><br>3<br><br>D_{\mathrm{TH}}<br><br>\left(Y\right)},$

where the d3D_c(Y) is the compensated three dimensional image density as a function of the range Y, Y_R is the reference range, where d3D_TH=K_s·

10⁵·

Y<sup>−

2</sup>, and where

$K_s=\frac{b·<br><br>f}{w},$

where w is the pixel size, b is the baseline, f is the focal length of the lenses of the image collection system.