Image analysis device

• US 7,221,777 B2
• Filed: 06/27/2003
• Issued: 05/22/2007
• Est. Priority Date: 07/02/2002
• Status: Active Grant

• Current Assignee: Arriver Software AB (Qualcomm, Inc.)
• Original Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Inventors: Nagaoka, Nobuharu, Hattori, Hiroshi, Watanabe, Masahito, Takatsudo, Izumi, Tsuji, Takayui
• Primary Examiner(s): Wu; Jingge
• Assistant Examiner(s): Akhavannik; Hadi
• Application Number:
  US10/607,647
• Publication Number:
  US 20040032971A1
• Time in Patent Office: 1,425 Days
• Field of Search: 382/103
• US Class Current: 382/104
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2520/14 Yaw
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/182 Network patterns, e.g. roads or rivers
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 23/20 for generating image signals from infrared radiation only
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

An image analysis device installed on a vehicle for detecting objects that are likely to affect the vehicle. An image processing unit included in the image analysis device sets a seeking area in a left-hand image of the field, and calculates a moving increment for seeking “Step” using a variable “n”. A correlation calculation process is sequentially executed in the seeking area set in the left-hand image of the field while moving a reference image stepwise at the moving increment “Step” so as to obtain a correlation area that gives the highest degree of correlation, and then a new seeking area is set, which consists of the correlation area and horizontally arranged two areas that sandwich the correlation area, and the moving increment “Step” is decreased in order to reduce the amount of calculation.

Claims

1. An image analysis device comprising:

first and second cameras which together constitute a set of stereo cameras; and

a distance calculation section which is adapted to calculate distance to an object in such a manner that a first image of the object is extracted from an image of a field taken by the first camera while a second image of the object corresponding to the first image of the object is extracted from a seeking area being set, in another image of the field taken by the second camera, depending on the extracted first image using a correlation calculation process, and then a parallax between the first and second images is calculated, wherein

the distance calculation section is further adapted to set a moving increment based on the width of an object frame that is determined depending on the first image of the object, and to execute the correlation calculation process while moving the first image of the object stepwise at the moving increment in the seeking area so as to extract a new seeking area which consists of a correlation area that exhibits a high degree of correlation with the first image of the object and two areas that sandwich the correlation area.

2. An image analysis device according to claim 1, wherein the moving increment is set to be equal to the width of the object frame.

3. An image analysis device comprising:

first and second cameras which together constitute a set of stereo cameras; and

a distance calculation section which is adapted to calculate distance to an object in such a manner that a first image of the object is extracted from an image of a field taken by the first camera while a second image of the object corresponding to the first image of the object is extracted from a seeking area being set, in another image of the field taken by the second camera, depending on the extracted first image using a correlation calculation process, and then a parallax between the first and second images is calculated, wherein

the distance calculation section is further adapted to set a moving increment based on the width of an object frame that is determined depending on the first image of the object, and to execute the correlation calculation process while moving the first image of the object stepwise at the moving increment in the seeking area so as to extract a new seeking area which consists of a correlation area that exhibits a high degree of correlation with the first image of the object and two areas that sandwich the correlation area,

wherein the moving increment is set to be equal to the width of the object frame, and

wherein the distance calculation section is further adapted to reset the moving increment to a smaller moving increment based on the width of the object frame every time a new seeking area is extracted, and to execute the correlation calculation process in the new seeking area while moving the first image of the object stepwise at the smaller moving increment.

4. An image analysis device comprising:

first and second cameras which together constitute a set of stereo cameras; and

a distance calculation section which is adapted to calculate distance to an object in such a manner that

a first image of the object is extracted from a first field image of a field taken by the first camera

while a second image of the object corresponding to the first image of the object is extracted from a seeking area that is set, in a second field image of the field taken by the second camera, depending on the extracted first image using a correlation calculation process, and

then a parallax between the first and second images is calculated, wherein the distance calculation section is further adapted to

set a moving increment based on a width of an object frame that defines a periphery of the first image of the object, and

execute the correlation calculation process while moving the first image of the object within the seeking area stepwise a distance equal to the moving increment so as to extract a new seeking area which consists of an area that is the sum of

a correlation area that exhibits a high degree of correlation with the first image of the object, and

two areas that sandwich the correlation area.

5. The image analysis device of claim 4 wherein the moving increment is set to be equal to the width of the object frame.

6. The image analysis device of claim 4 wherein the moving increment is set to be equal to a fraction of the width of the object frame.

7. The image analysis device of claim 4 wherein the extraction of the second image of the object is achieved iteratively, the iteration step identified by the value n, where n is initially set to be zero, and

the moving increment is set according to the following equation:

moving increment=(width of the object frame)/2ⁿ.

8. A method of analyzing an image using an image processing device, the image processing device comprising:

first and second cameras which together constitute a set of stereo cameras; and

a distance calculation section,

wherein the distance calculation section is adapted to calculate distance to an object using the following method steps;

extracting a first object image of the object from a first field image of a field taken by the first camera, the first image comprising an object frame, the object frame comprising a frame width substantially corresponding to the width of the first object image,

setting an initial step width, the initial step width based on the frame width,

setting a seeking area in a second field image of the field taken by the second camera,

setting a search area within the seeking area, the search area comprising an area substantially corresponding to the area of the object frame, the search area set within the seeking area such that a left edge of the search area overlies a left edge of the seeking area, and executing a correlation calculation process within the search area,

moving the search area within the seeking area to the left within the seeking area a distance corresponding to the initial step width and executing a correlation calculation process within the search area,

repeating the step of moving the search area and executing a correlation calculation process across a width of the seeking area,

identifying the position of the search area having the highest degree of correlation with the first object image of the object,

extracting a second object image of the object, corresponding to the first object image of the object, from the seeking area,

calculating a parallax between the first and second object images.

9. The method of analyzing an image using an image processing device of claim 8, wherein the distance calculation section comprises the following additional method steps inserted following the step of identifying the position of the search area having the highest degree of correlation:

extracting a new seeking area in the second field image which consists of an area that is the sum of

the search area identified as having the highest degree of correlation with the first object image of the object, and

two areas that sandwich the correlation area, setting a new step width, the new step width comprising a fraction of the initial step width,

setting a new search area within the new seeking area, the new search area comprising an area corresponding to a fraction of the area of the object frame, the new search area set within the new seeking area such that a left edge of the new search area overlies a left edge of the new seeking area, and executing a correlation calculation process within the new search area,

moving the new search area within the new seeking area to the left within the new seeking area a distance corresponding to the new step width and executing a correlation calculation process within the new search area,

repeating the step of moving the new search area and executing a correlation calculation process until a right edge of the new search area lies outside the new seeking area,

identifying the position of the new search area having the highest degree of correlation with the first object image of the object.