MOVING OBJECT DETECTION APPARATUS AND MOVING OBJECT DETECTION METHOD

• US 20110298988A1
• Filed: 04/26/2011
• Published: 12/08/2011
• Est. Priority Date: 06/04/2010
• Status: Abandoned Application

• Current Assignee: Toshiba Corporation
• Original Assignee: Toshiba Corporation
• Inventors: KAWAI, KIYOYUKI
• Application Number:
  US13/094,345
• Publication Number:
  US 20110298988A1
• Time in Patent Office: Days
• US Class Current: 348/699
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

An apparatus for detecting a moving object includes: an image input unit that inputs a camera image; a motion vector generation unit that generates motion vectors of a plurality of points P in the image; an estimation unit that estimates rotational components of vehicle movement parameters as being equal to the inclination from each point P to a vanishing point when the inclination of the motion vector of each point P is corrected by the rotational components; and a determination unit that corrects the inclination of the motion vector of a given point Q in the image, and detects the existence of a moving object that moves in a direction different from the vehicle movement direction when the coincidence degree between the inclinations is low, while detects the existence of an object that radially moves toward the vanishing point when the coincidence degree of the inclinations is high.

Claims

1. An apparatus for detecting a moving object, comprising:

an image input unit that inputs a camera image taken by an on-vehicle camera;

a motion vector generation unit that processes the image from the image input unit to generate motion vectors of a plurality of points P in the image;

an estimation unit that estimates rotational components (Rx, Ry, Rz) of vehicle movement parameters as being equal to the inclination from each point P to a vanishing point when the inclination of the motion vector of each point P is corrected by the rotational components (Rx, Ry, Rz) of the vehicle movement parameters; and

a determination unit that corrects the inclination of the motion vector of a given point Q in the image by using the rotational components (Rx, Ry, Rz) of the vehicle movement parameters, compares the corrected inclination of the motion vector and inclination of a straight line connecting the given point Q and vanishing point, and detects the existence of a moving object that moves in a direction different from the vehicle movement direction when the coincidence degree between the inclinations is low, while detects the existence of a stationary object or a moving object that radially moves toward the vanishing point when the coincidence degree of the inclinations is high.

2. The apparatus according to claim 1, wherein

when the rotational component=0, the vehicle movement parameter estimation unit updates the vanishing point with the intersection of the extended lines of the motion vectors of the plurality of points P.

3. The apparatus according to claim 1, comprising:

a stationary area candidate estimation unit that estimates a stationary object area based on the motion vectors of the plurality of points P in the image, wherein

the vehicle movement parameter estimation unit estimates the rotational components (Rx, Ry, Rz) of the vehicle movement parameters based on the motion vectors of the plurality of points P in the estimated stationary object area image and the vanishing point.

4. The apparatus according to claim 3, wherein

the stationary area candidate estimation unit calculates the rotational component Ry (or rotational components Ry and Rz) corresponding to the rudder angle of the vehicle based on the motion vectors of the plurality of points P in the estimated stationary object area image and the vanishing point and selects an area near the peak of a histogram concerning the rotational component Ry (or rotational components Ry and Rz) of each point P as a stationary object area candidate.

5. The apparatus according to claim 3, comprising a translational direction moving object detection unit that inputs thereto image information that has not been detected as a moving object in the moving object detection unit, calculates a translational component Tz/z of the vehicle movement parameter from the motion vector of a given point Q in the stationary object area candidate and rotational components (Rx, Ry, Rz), sets an estimated distance weighting coefficient z0 that associates the road surface with camera viewing angle, compares a translational component z0(Tz/z) obtained by multiplying the (Tz/z) by the coefficient z0 and an estimated vehicle translational component Tzm, and detects the existence of a moving object when the coincidence degree between the z0(Tz/z) and Tzm is low.

6. The apparatus according to claim 5, wherein

the translational direction moving object detection unit calculates the translational component Tz/z of the vehicle movement parameter from the motion vectors of the plurality of points P in the stationary object area candidate and rotational components (Rx, Ry, Rz) and sets the average of the translational component z0(Tz/z) obtained by multiplying the (Tz/z) by the coefficient z0 as the estimated vehicle translational component Tzm.

7. The apparatus according to claim 5 or claim 6, wherein

the translational direction moving object detection unit previously sets a correspondence between a pixel position of the camera image and a distance between the road surface and camera as the estimated distance weighting coefficient z0.

8. An apparatus for detecting a moving object, comprising:

an image input unit that inputs a camera image taken by an on-vehicle camera;

a motion vector generation unit that processes the image from the image input unit to generate a motion vector of the image;

an estimation unit that calculates motion vectors of a plurality of points P in the image and a vanishing point at which extended motion vectors obtained by extending the motion vectors converge, and estimates rotational components (Rx, Ry, Rz) and a translational component as vehicle movement parameters based on the motion vectors and the vanishing point; and

a determination unit that sets an estimated distance weighting coefficient z0 that associates the road surface with camera viewing angle, calculates an estimated vehicle translational component from the average of a translational component obtained by multiplying the estimated translational component by the coefficient z0, predicts a motion vector of a given point Q in the image from the rotational components (Rx, Ry, Rz), translational component, and coefficient z0, compares the predicted motion vector and an actual measurement motion vector, and determines the existence of a moving object when the coincidence degree between the predicted motion vector and actual measurement motion vector is low.

9. The apparatus according to claim 8, wherein

a correspondence between a pixel position of the camera image and a distance between the road surface and camera is previously set as the estimated distance weighting coefficient z0.

10. A method for detecting a moving object, comprising:

inputting a camera image taken by an on-vehicle camera;

processing the image to generate motion vectors of a plurality of points P in the image;

estimating rotational components (Rx, Ry, Rz) of vehicle movement parameters as being equal to the inclination from each point P to a vanishing point when the inclination of the motion vector of each point P is corrected by the rotational components (Rx, Ry, Rz) of the vehicle movement parameters; and

correcting the inclination of the motion vector of a given point Q in the image by using the rotational components (Rx, Ry, Rz) of the vehicle movement parameters, comparing the corrected inclination of the motion vector and inclination of a straight line connecting the given point Q and the vanishing point, and detecting the existence of a moving object that moves in a direction different from the vehicle movement direction when the coincidence degree between the inclinations is low, while detecting the existence of a stationary object or a moving object that radially moves toward the vanishing point when the coincidence degree of the inclinations is high.

11. The method according to claim 10, wherein

when the rotational component is 0, the vanishing point is updated with the intersection of the extended lines of the motion vectors of the plurality of points P.

12. The method according to claim 10, comprising:

estimating a stationary object area based on the motion vectors of the plurality of points P in the image, wherein

the rotational component Ry (or rotational components Ry and Rz) corresponding to the rudder angle of the vehicle is calculated based on the motion vectors of the plurality of points P in the estimated stationary object area image and vanishing point, and

an area near the peak of a histogram concerning the rotational component Ry (or rotational components Ry and Rz) of each point P is selected as a stationary object area candidate.

13. The method according to claim 12, comprising:

inputting thereto image information that has not been detected as a moving object;

calculating a translational component Tz/z of the vehicle movement parameter from the motion vector of a given point Q in the stationary object area candidate and rotational components (Rx, Ry, Rz);

setting an estimated distance weighting coefficient z0 that associates the road surface with camera viewing angle, comparing a translational component z0(Tz/z) obtained by multiplying the (Tz/z) by the coefficient z0 and an estimated vehicle translational component Tzm, and detecting the existence of a moving object when the coincidence degree between the z0(Tz/z) and Tzm is low.

14. The method according to claim 13, wherein

the translational component Tz/z of the vehicle movement parameter is calculated from the motion vectors of the plurality of points P in the stationary object area candidate and the rotational components (Rx, Ry, Rz), and the average of the translational component z0(Tz/z) obtained by multiplying the (Tz/z) by the coefficient z0 is set as the estimated vehicle translational component Tzm.

15. A method for detecting a moving object, comprising:

inputting a camera image taken by an on-vehicle camera;

processing the image to generate motion vectors of a plurality of points P in the image and calculating a vanishing point at which extended motion vectors obtained by extending the motion vectors converge, and estimating rotational components (Rx, Ry, Rz) and a translational component as vehicle movement parameters based on the motion vectors and the vanishing point; and

setting an estimated distance weighting coefficient z0 that associates the road surface with camera viewing angle, calculating an estimated vehicle translational component from the average of a translational component obtained by multiplying the estimated translational component by the coefficient z0, predicting a motion vector of a given point Q in the image from the rotational components (Rx, Ry, Rz), translational component, and coefficient z0, comparing the predicted motion vector and an actual measurement motion vector, and determining the existence of a moving object when the coincidence degree between the predicted motion vector and actual measurement motion vector is low.

16. An apparatus for detecting a moving object, comprising:

a division/conversion unit that divides a non-linear image taken by an on-vehicle camera having a wide imaging range into a plurality of images each having a preset viewing-angle range to generate a plurality of linear projection plane images having a common camera center;

a calculation unit that calculates motion vectors of the projection plane images;

a determination unit that analyzes the motion vectors of the plurality of projection plane images to determine the existence of a moving object; and

a display unit that displays the determination result of the moving object determination unit.

17. The apparatus according to claim 16, wherein

the division/conversion unit divides a wide-angle camera image having a viewing angle exceeding 180°

into a plurality of images each having a viewing angle not more than 180°

including an overlap portion to generate a plurality of projecting plane images.

18. The apparatus according to claim 16, wherein

the moving object determination unit determines presence/absence of a moving object approaching the vehicle based on the directions of the motion vectors calculated in the motion vector calculation unit.

19. The apparatus according to claim 16, comprising:

an original image conversion unit that superimposes the determination result of the moving object determination unit on a taken image before division; and

an approach direction determination unit that determines the approach direction of the moving object based on the image obtained in the original image conversion unit and displays image information representing the approach of the moving object on the display unit.

20. A method for detecting a moving object, comprising:

dividing a non-linear image taken by an on-vehicle camera having a wide imaging range into a plurality of images each having a preset viewing-angle range to generate a plurality of linear projection plane images having a common camera center;

calculating motion vectors of the projection plane images;

analyzing the motion vectors of the plurality of projection plane images to determine the existence of a moving object; and

displaying the determination result of the moving object existence on a display unit.

21. The method according to claim 20, wherein

presence/absence of a moving object approaching the vehicle is determined based on the directions of the calculated motion vectors.

22. The method according to claim 20, comprising:

superimposing the determination result into an original image before division;

determining the approach direction of the moving object based on the original image on which the determination result is superimposed; and

displaying image information representing the approach of the moving object on the display unit.