IMAGE PROCESSING METHOD FOR DETECTING OBJECTS USING RELATIVE MOTION

• US 20170124405A1
• Filed: 01/16/2017
• Published: 05/04/2017
• Est. Priority Date: 04/25/2011
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gupta, Nikhil
• Granted Patent: US 10,043,082 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An obstacle detection method includes acquiring a series of image frames captured by a vehicle camera whilst the vehicle is in motion. First and second feature points are selected from a first detected object and are spaced apart in a first image frame and tracked over at least a second image frame, and movement of those feature points is determined. Third and fourth feature points are selected from a second detected object and are spaced apart in a first image frame and tracked over at least a second image frame, and movement of the third and fourth feature points is determined. The movements of the first and second feature points are compared to the movements of the third and fourth feature points to distinguish the first detected object as a three dimensional object and the second detected object as a two dimensional object.

Claims

1. An obstacle detection method, comprising:

(a) disposing a camera at a vehicle, the camera having a field of view external of the vehicle;

(b) acquiring a series of image frames captured by the camera whilst the vehicle is in motion, the image frames defining an image plane having a vertical aspect and a horizontal aspect;

(c) detecting a first object present in the series of image frames by edge detection;

(d) detecting a second object present in the series of image frames by edge detection;

(e) for the first detected object,

(i) selecting first and second feature points from the first detected object that are spaced apart in a first image frame,

(ii) tracking the positions of the first and second feature points over at least a second image frame, and

(iii) determining the movement of the first and second feature points when projected to the image plane;

(f) for the second detected object,

(i) selecting third and fourth feature points from the second detected object that are spaced apart in a first image frame,

(ii) tracking the positions of the third and fourth feature points over at least a second image frame, and

(iii) determining the movement of the third and fourth feature points when projected to the image plane;

(g) comparing the movement of the first and second feature points when projected to the image plane to the movement of the third and fourth feature points when projected to the image plane; and

(h) distinguishing between the first detected object being a three dimensional object and the second detected object being a two dimensional object by determining that the movement of the first feature point when projected to the image plane is dissimilar to that of the second feature point and by determining that the movement of the third feature point when projected to the image plane is similar to that of the fourth feature point.

2. The method according to claim 1, wherein the second detected object comprises a shadow.

3. The method according to claim 1, wherein the second detected object comprises a pavement marking.

4. The method according to claim 1, wherein the second detected object comprises a ground marking.

5. The method according to claim 1, including unwarping a given pixel position to account for lens distortion prior to determining a physical location corresponding to the given pixel position.

6. The method according to claim 1, wherein the first and second feature points are portions of top and bottom edge contours of the first detected object.

7. The method according to claim 1, wherein the first and second feature points selected from the first detected object are spaced vertically apart in the first image frame.

8. The method according to claim 1, comprising:

estimating, from the first image frame, a physical distance D₁ of the first feature point O_1b to the camera in a selected physical horizontal plane;

estimating, from the first image frame, a physical location D_T of the second feature point O_1T assuming that the second feature point physically lies at a height T above the selected physical horizontal plane; and

estimating, from the second image frame, a physical distance D₂ of the first feature point O_2b to the camera in the selected physical horizontal plane, and determining the distance Δ

D between D₂ and D₁.

9. The method according to claim 8, wherein the selected physical horizontal plane is the ground plane.

10. The method according to claim 1, comprising providing extrinsic parameters of the camera, the extrinsic parameters comprising the height of the camera at the vehicle.

11. The method according to claim 10, comprising providing extrinsic parameters of the camera, the extrinsic parameters comprising angular orientation of the camera at the vehicle.

12. The method according to claim 10, comprising providing intrinsic parameters of the camera, the intrinsic parameters comprising focal length of a lens of the camera.

13. The method according to claim 10, comprising providing vehicle data of the vehicle the camera is disposed at, the vehicle data comprising speed and steering angle.

14. The method according to claim 13, wherein the vehicle data is provided via a controller area network.

15. The method according to claim 13, comprising making a trajectory estimation of the first detected object and, based on the trajectory estimation, determining whether the first detected object is in the path of movement of the vehicle.

16. The method according to claim 1, comprising operating in conjunction with a dynamic calibration system that is operable to dynamically ascertain the rotational angles of the camera so that any deviations over time from nominal angle are accounted for.

17. An obstacle detection method, comprising:

(a) disposing a camera at a vehicle, the camera having a field of view external of the vehicle;

(b) acquiring a series of image frames captured by the camera whilst the vehicle is in motion, the image frames defining an image plane having a vertical aspect and a horizontal aspect;

(c) detecting a first object present in the series of image frames by edge detection;

(d) detecting a second object present in the series of image frames by edge detection;

(e) for the first detected object,

(i) selecting first and second feature points from the first detected object that are spaced apart in a first image frame,

(ii) tracking the positions of the first and second feature points over at least a second image frame, and

(iii) determining the movement of the first and second feature points when projected to the image plane;

(f) for the second detected object,

(i) selecting third and fourth feature points from the second detected object that are spaced apart in a first image frame,

(ii) tracking the positions of the third and fourth feature points over at least a second image frame, and

(iii) determining the movement of the third and fourth feature points when projected to the image plane;

(g) comparing the movement of the first and second feature points when projected to the image plane to the movement of the third and fourth feature points when projected to the image plane;

(h) distinguishing between the first detected object being a three dimensional object and the second detected object being a two dimensional object by determining that the movement of the first feature point when projected to the image plane is dissimilar to that of the second feature point and by determining that the movement of the third feature point when projected to the image plane is similar to that of the fourth feature point;

(i) providing the height of the camera at the vehicle;

(j) providing angular orientation of the camera at the vehicle;

(k) providing the focal length of a lens of the camera;

(l) providing vehicle data of the vehicle the camera is disposed at, the vehicle data comprising speed and steering angle; and

(m) making a trajectory estimation of the first detected object and, based on the trajectory estimation, determining whether the first detected object is in the path of movement of the vehicle.

18. The method according to claim 17, wherein the second detected object comprises a shadow.

19. The method according to claim 17, wherein the second detected object comprises a ground marking.

20. The method according to claim 17, comprising operating in conjunction with a dynamic calibration system that is operable to dynamically ascertain the rotational angles of the camera so that any deviations over time from nominal angle are accounted for.

21. An obstacle detection method, comprising:

(a) disposing a camera at a vehicle, the camera having a field of view external of the vehicle;

(b) acquiring a series of image frames captured by the camera whilst the vehicle is in motion, the image frames defining an image plane having a vertical aspect and a horizontal aspect;

(c) detecting a first object present in the series of image frames by edge detection;

(d) detecting a second object present in the series of image frames by edge detection;

(e) for the first detected object,

(i) selecting first and second feature points from the first detected object that are spaced apart in a first image frame,

(ii) tracking the positions of the first and second feature points over at least a second image frame, and

(iii) determining the movement of the first and second feature points when projected to the image plane;

(f) for the second detected object,

(i) selecting third and fourth feature points from the second detected object that are spaced apart in a first image frame,

(ii) tracking the positions of the third and fourth feature points over at least a second image frame, and

(iii) determining the movement of the third and fourth feature points when projected to the image plane;

(g) comparing the movement of the first and second feature points when projected to the image plane to the movement of the third and fourth feature points when projected to the image plane;

(h) distinguishing between the first detected object being a three dimensional object and the second detected object being a two dimensional object by determining that the movement of the first feature point when projected to the image plane is dissimilar to that of the second feature point and by determining that the movement of the third feature point when projected to the image plane is similar to that of the fourth feature point; and

(i) operating in conjunction with a dynamic calibration system that is operable to dynamically ascertain the rotational angles of the camera so that any deviations over time from nominal angle are accounted for.

22. The method according to claim 21, comprising providing extrinsic parameters of the camera, the extrinsic parameters comprising the height and angular orientation of the camera at the vehicle, and comprising providing intrinsic parameters of the camera, the intrinsic parameters comprising focal length of a lens of the camera, and comprising providing vehicle data of the vehicle the camera is disposed at, the vehicle data comprising speed and steering angle.

23. The method according to claim 22, comprising making a trajectory estimation of the first detected object and, based on the trajectory estimation, determining whether the first detected object is in the path of movement of the vehicle.

24. The method according to claim 23, wherein the vehicle data is provided via a controller area network.

25. The method according to claim 24, wherein the second detected object comprises a shadow.