DETECTION DEVICE, METHOD AND PROGRAM THEREOF

• US 20080199050A1
• Filed: 02/12/2008
• Published: 08/21/2008
• Est. Priority Date: 02/16/2007
• Status: Abandoned Application

• Current Assignee: Omron Corporation
• Original Assignee: Omron Corporation
• Inventors: Koitabashi, Hiroyoshi
• Application Number:
  US12/029,992
• Publication Number:
  US 20080199050A1
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

A rotational movement component of a camera provided on a mobile object is to be detected in a precise and simple manner. Three motion vectors extracted from the forward images captured by a camera mounted on a vehicle are extracted on a random basis. A temporary rotation angle is calculated using the extracted motion vectors based on a relational expression that represents the relationship between a background vector and the rotation angle of the camera, the relational expression being a linear expression of the yaw angle, pitch angle, and roll angle of the rotational movement of the camera. An error when using the temporary rotation angle is calculated for other motion vectors, and the number of motion vectors for which the error is within a predetermined threshold value is counted. After repeating such a process for a predetermined number of times, the temporary rotation angle with the largest number of motion vectors for which the error is within the predetermined threshold value is selected as the rotation angle of the camera. The present invention can be applied to an in-vehicle obstacle detecting device.

Claims

1. A detection device that detects a rotational movement component of a camera mounted on a mobile object performing a translational movement in only one axis direction, the detection device comprising:

a detecting means for detecting the rotational movement component of the camera using a motion vector of a stationary object within an image captured by the camera and a relational expression that represents the relationship between the motion vector and the rotational movement component of the camera, based on a motion vector at feature points extracted within the image, the relational expression derived by expressing two-axis directional components among three-axis directional components of a translational movement of the camera using a remaining one-axis directional component.

2. The detection device according to claim 1, wherein the relational expression is expressed by a linear expression of a yaw angle, a pitch angle, and a roll angle of the rotational movement of the camera.

3. The detection device according to claim 1, wherein when the focal length of the camera is F, the x- and y-axis directional coordinates of the feature points are Xp and Yp, respectively, the x- and y-axis directional components of the motion vector at the feature points are v_x and v_y, respectively, the pitch angle, yaw angle, and roll angle of the rotational movement of the camera are θ

, ψ

, and φ

, respectively, the focal length of the camera is F, the translational movement component in the z-axis direction of the camera is t_z, and the translational movement components in the x- and y-axis direction of the camera are t_x=at_z (a;

constant) and t_y=bt_z (b;

constant), respectively, the detecting means detects the rotational movement component of the camera using the following relational expression.

$\left(\mathrm{Yp}-\mathrm{Fb}\right)<br><br>\left\{v_x-\left(-F<br><br><br><br>\mathrm{\psi <br><br>}+\mathrm{Yp}<br><br><br><br>\mathrm{\phi <br><br>}-\frac{{\mathrm{Xp}}^2}{F}<br><br>\mathrm{\psi <br><br>}+\frac{\mathrm{XpYp}}{F}<br><br>\mathrm{\theta <br><br>}\right)\right\}=\left(\mathrm{Xp}-\mathrm{Fa}\right)<br><br>\left\{v_y-\left(-\mathrm{Xp}<br><br><br><br>\mathrm{\phi <br><br>}+F<br><br><br><br>\mathrm{\theta <br><br>}-\frac{\mathrm{XpYp}}{F}<br><br>\mathrm{\psi <br><br>}+\frac{{\mathrm{Yp}}^2}{F}<br><br>\mathrm{\theta <br><br>}\right)\right\}$

4. The detection device according to claim 1, wherein when the direction of the mobile object performing the translational movement is substantially parallel or perpendicular to the optical axis of the camera, the detecting means detects the rotational movement component of the camera using a simplified expression of the relational expression by applying a model in which the direction of the translational movement of the camera is restricted to the direction of the mobile object performing the translational movement.

5. The detection device according to claim 4, wherein the mobile object is a vehicle,

the camera is mounted on the vehicle so that the optical axis of the camera is substantially parallel to the front-to-rear direction of the vehicle, and

the detecting means detects the rotational movement component of the camera using the simplified expression of the relational expression by applying the model in which the direction of the translational movement of the camera is restricted to the front-to-rear direction of the vehicle.

6. The detection device according to claim 1, wherein the detecting means detects the rotational movement component of the camera based on the motion vector at the feature point on the stationary object among the feature points.

7. The detection device according to claim 1, wherein the detecting means performs a robust estimation so as to suppress the effect on the detection results of the motion vector at the feature point on a moving object among the feature points.

8. A detection method of a detection device for detecting a rotational movement component of a camera mounted on a mobile object performing a translational movement in only one axis direction, the detection method comprising:

a detecting step of detecting the rotational movement component of the camera using a motion vector of a stationary object within an image captured by the camera and a relational expression that represents the relationship between the motion vector and the rotational movement component of the camera, based on a motion vector at feature points extracted within the image, the relational expression derived by expressing two-axis directional components among three-axis directional components of a translational movement of the camera using a remaining one-axis directional component.

9. A program for causing a computer to execute a detection process for detecting a rotational movement component of a camera mounted on a mobile object performing a translational movement in only one axis direction, the detection process comprising:

a detecting step of detecting the rotational movement component of the camera using a motion vector of a stationary object within an image captured by the camera and a relational expression that represents the relationship between the motion vector and the rotational movement component of the camera, based on a motion vector at feature points extracted within the image, the relational expression derived by expressing among three-axis directional movement of the camera using a component.