Apparatus, program and method for detecting both stationary objects and moving objects in an image using optical flow

• US 7,062,071 B2
• Filed: 12/17/2002
• Issued: 06/13/2006
• Est. Priority Date: 12/28/2001
• Status: Active Grant

• Current Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Original Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Inventors: Kondo, Hiroshi, Miura, Atsushi, Tsujino, Hiroshi, Akatsuka, Koji, Nagai, Shinichi
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): KUHN, JORDAN M
• Application Number:
  US10/322,307
• Publication Number:
  US 20030152271A1
• Time in Patent Office: 1,274 Days
• Field of Search: 382/103, 382/104, 382/153
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

An object detection apparatus is provided for detecting both stationary objects and moving objects accurately from an image captured from a moving mobile unit.

The object detection apparatus of the present invention applies Gabor filter to two or more input images captured by an imaging device such as CCD camera mounted on a mobile unit, and calculates optical flow of local areas in the input images. Then the object detection apparatus closely removes optical flow produced by motion of the mobile unit by estimating optical flow produced from background of the input images. In other words, the object detection apparatus clarifies the area where object is not present (“ground”) in the input images. By removing such “ground” part, the area where objects seems to be present (“feature”) is extracted from the input images. Finally, the object detection apparatus determines whether objects are present or not using flow information of the extracted “feature” part.

Claims

1. An object detection apparatus for detecting objects in input images captured from a mobile unit, comprising:

a local flow processor for calculating flow information for each local area in said input images;

a global flow processor for calculating flow information for each global area in said input images using said flow information for each of a plurality of local areas;

a flow feature calculator for estimating motion of the mobile unit using said flow information for each of a plurality of global areas, calculating new flow information using the estimated motion, comparing said new flow information with said flow information for each local area to extract local areas having flow information not consistent with said estimated motion; and

an object presence/absence determiner for determining the presence or absence of objects in the extracted local areas.

2. The object detection apparatus according to claim 1, wherein said local flow processor applies Gabor filter to each local area to calculate the magnitude of optical flow in predetermined directions of the local area as said flow information of each local area.

3. The object detection apparatus according to claim 2, wherein said local flow processor set the greatest one as size in predetermined directions of the local area, sets the greatest one as the magnitude of flow for the local area and sets the direction of that flow as a flow direction of the local area.

4. The object detection apparatus according to claim 3, wherein said global flow processor obtains all flow information for local areas included in each global area, selects local areas having the flow direction predetermined for each global area based on the flow information for local areas and sets the average of the magnitude of flow for the selected local area as said flow information for the global area.

5. The object detection apparatus according to claim 1, wherein said global areas occupy different peripheral areas in said input images.

6. The object detection apparatus according to claim 1, wherein said flow feature calculator applies said flow information of each global area to a result of a first learning to estimate the motion of said mobile unit.

7. The object detection apparatus according to claim 6, wherein said first learning is performed by associating said flow information for each global area with the motion of the mobile unit using a neural network.

8. The object detection apparatus according to claim 7, wherein said neural network is Perceptron.

9. The object detection apparatus according to claim 1, wherein said extracted local areas are repeatedly processed in said global flow processor and said flow feature calculator.

10. The object detection apparatus according to claim 1, wherein said object presence/absence determiner determines the presence or absence of objects based on a second learning which associates said extracted local areas with the presence or absence of objects.

11. A computer program stored on a computer readable medium, executable on a computer for detecting objects in input images captured by an imaging device on a mobile unit, when executed, said computer program performing the steps of:

calculating flow information for each of a plurality of local areas in said input images;

calculating flow information for each of a plurality of global areas in said input images using said flow information for each local area;

estimating motion of the mobile unit using said flow information for each global area;

calculating new flow information using the estimated motion;

comparing said new flow information with said flow information for each local area to extract local areas having flow information not consistent with said estimated motion; and

determining the presence or absence of objects in the extracted local areas.

12. A computer program stored on a computer readable medium, executable on a computer for detecting objects in input images captured by an imaging device on a mobile unit, when executed, said computer program performing the steps of:

(a) calculating flow information for each of a plurality of local areas in said input images;

(b) calculating flow information for each of a plurality of global areas in said input images using said flow information for each local area;

(c) applying said flow information for global area to a result of a first learning to estimate the motion of said mobile unit;

(d) estimating motion of the mobile unit using said flow information for each global area;

(e) calculating new flow information using the estimated motion;

(f) comparing said new flow information with said flow information for each local area to extract local areas having flow information not consistent with said estimated motion;

(g) repeating the steps (a) to (f) on said extracted local areas to extract second local areas; and

(h) determining the presence or absence of objects in said second local areas.

13. An object detection method for detecting objects in input images captured from a mobile unit, comprising:

calculating flow information for each of a plurality of local areas in said input images;

calculating flow information for each of a plurality of global areas in said input images using said flow information for each local area;

estimating motion of the mobile unit using said flow information for each global area;

calculating new flow information using the estimated motion;

comparing said new flow information with said flow information for each local area to extract local areas having flow information not consistent with said estimated motion; and

determining the presence or absence of objects in the extracted local areas.

14. An object detection method for detecting objects in input images captured from a mobile unit, comprising:

(a) calculating flow information for each of a plurality of local areas in said input images;

(b) calculating flow information for each of a plurality of global areas in said input images using said flow information for each local area;

(c) applying said flow information for global area to a result of a first learning to estimate the motion of said mobile unit;

(d) estimating motion of the mobile unit using said flow information for each global area;

(e) calculating new flow information using the estimated motion;

(f) comparing said new flow information with said flow information for each local area to extract local areas having flow information not consistent with said estimated motion;

(g) repeating the steps (a) to (f) on said extracted local areas to extract second local areas; and

(h) determining the presence or absence of objects in said second local areas.