Processing method for distinguishing a three dimensional object from a two dimensional object using a vehicular system

• US 10,452,931 B2
• Filed: 08/06/2018
• Issued: 10/22/2019
• Est. Priority Date: 04/25/2011
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gupta, Nikhil
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Mangialaschi, Tracy
• Application Number:
  US16/055,262
• Publication Number:
  US 20180341823A1
• Time in Patent Office: 442 Days
• Field of Search: None
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A method of distinguishing a three dimensional object from a two dimensional object using a vehicular system includes acquiring image frames captured by a vehicle camera while the vehicle is in motion. First and second feature points are selected from a first detected object in a first captured image frame and tracked in at least a second captured image frame. Third and fourth feature points are selected from a second detected object in the first captured image frame and tracked over at least the second captured image frame. Movements of the first and second feature points over the multiple captured image frames are compared to movements of the third and fourth feature points the multiple captured image frames to distinguish the first object as a three dimensional object and the second object as a two dimensional object.

Claims

1. A method of distinguishing a three dimensional object from a two dimensional object using a vehicular vision system, said method comprising:

(a) disposing a camera at a vehicle, the camera having a field of view external of the vehicle;

(b) providing a control having a processor;

(c) providing to the control height of the camera at the vehicle and angular orientation of the camera at the vehicle;

(d) capturing image frames of image data via the camera while the vehicle is in motion, each of the captured image frames defining an image plane having a vertical aspect and a horizontal aspect;

(e) via processing of captured image frames by the processor, detecting by edge detection a first object present in multiple captured image frames;

(f) via processing of captured image frames by the processor, detecting by edge detection a second object present in the multiple captured image frames;

(g) for the first detected object,

(1) selecting, via the control, and responsive to processing of captured image frames by the processor, first and second feature points from the first detected object that are spaced apart in a first captured image frame of the multiple captured image frames,

(2) tracking, via the control, and responsive to processing of captured image frames by the processor, positions of the first and second feature points in at least a second captured image frame of the multiple captured image frames, and

(3) determining, via the control, and responsive to processing of captured image frames by the processor, movement of the first and second feature points over the multiple captured image frames;

(h) for the second detected object,

(1) selecting, via the control, and responsive to processing of captured image frames by the processor, third and fourth feature points from the second detected object that are spaced apart in the first captured image frame,

(2) tracking, via the control, and responsive to processing of captured image frames by the processor, positions of the third and fourth feature points in at least the second captured image frame, and

(3) determining, via the control, and responsive to processing of captured image frames by the processor, movement of the third and fourth feature points over the multiple captured image frames;

(i) comparing, via the control, movement of the first and second feature points over the multiple captured image frames to movement of the third and fourth feature points over the multiple captured image frames; and

(j) distinguishing, via the control, between the first detected object being a three dimensional object and the second detected object being a two dimensional object by determining, via the control, that movement of the first feature point over the multiple captured image frames is dissimilar to that of the second feature point and by determining, via the control, that movement of the third feature point over the multiple captured image frames is similar to that of the fourth feature point.

2. The method according to claim 1, comprising providing to the control a focal length of a lens of the camera.

3. The method according to claim 1, comprising providing to the control vehicle data of the vehicle the camera is disposed at, the vehicle data comprising speed and steering angle.

4. The method according to claim 3, wherein the vehicle data is provided to the control via a controller area network.

5. The method according to claim 1, wherein the second detected object comprises a shadow.

6. The method according to claim 1, wherein the second detected object comprises a pavement marking.

7. The method according to claim 1, wherein the second detected object comprises a ground marking.

8. The method according to claim 1, including unwarping a given pixel position to account for lens distortion prior to determining a physical location corresponding to the given pixel position.

9. The method according to claim 1, wherein the first and second feature points are portions of top and bottom edge contours of the first detected object.

10. The method according to claim 1, wherein the first and second feature points selected from the first detected object are spaced vertically apart in the first captured image frame.

11. The method according to claim 1, comprising:

estimating, via the control, from the first captured image frame, a physical distance D₁ of the first feature point O_1b to the camera in a selected physical horizontal plane;

estimating, via the control, from the first captured image frame, a physical location D_T of the second feature point O_1T assuming that the second feature point physically lies at a height T above the selected physical horizontal plane; and

estimating, via the control, from the second captured image frame, a physical distance D₂ of the first feature point O_2b to the camera in the selected physical horizontal plane, and determining the distance Δ

D between D₂ and D₁.

12. The method according to claim 11, wherein the selected physical horizontal plane is the ground plane.

13. The method according to claim 1, comprising operating in conjunction with a dynamic calibration system that is operable to dynamically ascertain rotational angles of the camera so that any deviations over time from nominal angle are accounted for.

14. A method of distinguishing a three dimensional object from a two dimensional object using a vehicular vision system, said method comprising:

(a) disposing a camera at a vehicle, the camera having a field of view external of the vehicle;

(b) providing a control having a processor;

(c) providing to the control height of the camera at the vehicle and angular orientation of the camera at the vehicle;

(d) providing to the control a focal length of a lens of the camera;

(e) providing to the control vehicle data of the vehicle the camera is disposed at, the vehicle data comprising speed and steering angle while the vehicle is in motion;

(f) capturing image frames of image data via the camera while the vehicle is in motion, each of the captured image frames defining an image plane having a vertical aspect and a horizontal aspect;

(g) via processing of captured image frames by the processor, detecting by edge detection a first object present in multiple captured image frames;

(h) via processing of captured image frames by the processor, detecting by edge detection a second object present in the multiple captured image frames;

(i) for the first detected object,

(1) selecting, via the control, and responsive to processing of captured image frames by the processor, first and second feature points from the first detected object that are spaced apart in a first captured image frame of the multiple captured image frames,

(2) tracking, via the control, and responsive to processing of captured image frames by the processor, positions of the first and second feature points in at least a second captured image frame of the multiple captured image frames, and

(3) determining, via the control, and responsive to processing of captured image frames by the processor, movement of the first and second feature points over the multiple captured image frames;

(j) for the second detected object,

(1) selecting, via the control, and responsive to processing of captured image frames by the processor, third and fourth feature points from the second detected object that are spaced apart in the first captured image frame,

(2) tracking, via the control, and responsive to processing of captured image frames by the processor, positions of the third and fourth feature points in at least the second captured image frame, and

(3) determining, via the control, and responsive to processing of captured image frames by the processor, movement of the third and fourth feature points over the multiple captured image frames;

(k) comparing, via the control, movement of the first and second feature points over the multiple captured image frames to movement of the third and fourth feature points over the multiple captured image frames; and

(l) distinguishing, via the control, between the first detected object being a three dimensional object and the second detected object being a two dimensional object by determining, via the control, that movement of the first feature point over the multiple captured image frames is dissimilar to that of the second feature point and by determining, via the control, that movement of the third feature point over the multiple captured image frames is similar to that of the fourth feature point.

15. The method according to claim 14, wherein the vehicle data is provided to the control via a controller area network.

16. The method according to claim 14, wherein the first and second feature points are portions of top and bottom edge contours of the first detected object.

17. A method of distinguishing a three dimensional object from a two dimensional object using a vehicular vision system, said method comprising:

(a) disposing a camera at a vehicle, the camera having a field of view external of the vehicle;

(b) providing a control having a processor;

(c) providing to the control height of the camera at the vehicle and angular orientation of the camera at the vehicle;

(d) providing to the control vehicle data of the vehicle the camera is disposed at, the vehicle data comprising speed and steering angle while the vehicle is in motion, wherein the vehicle data is provided to the control via a controller area network;

(e) capturing image frames of image data via the camera while the vehicle is in motion, each of the captured image frames defining an image plane having a vertical aspect and a horizontal aspect;

(f) via processing of captured image frames by the processor, detecting by edge detection a first object present in multiple captured image frames;

(g) via processing of captured image frames by the processor, detecting by edge detection a second object present in the multiple captured image frames;

(h) for the first detected object,

(1) selecting, via the control, and responsive to processing of captured image frames by the processor, first and second feature points from the first detected object that are spaced apart in a first captured image frame of the multiple captured image frames, wherein the first and second feature points selected from the first detected object are spaced vertically apart in the first captured image frame,

(2) tracking, via the control, and responsive to processing of captured image frames by the processor, positions of the first and second feature points in at least a second captured image frame of the multiple captured image frames, and

(3) determining, via the control, and responsive to processing of captured image frames by the processor, movement of the first and second feature points over the multiple captured image frames;

(i) for the second detected object,

(1) selecting, via the control, and responsive to processing of captured image frames by the processor, third and fourth feature points from the second detected object that are spaced apart in the first captured image frame,

(2) tracking, via the control, and responsive to processing of captured image frames by the processor, positions of the third and fourth feature points in at least the second captured image frame, and

(3) determining, via the control, and responsive to processing of captured image frames by the processor, movement of the third and fourth feature points over the multiple captured image frames;

(j) comparing, via the control, movement of the first and second feature points over the multiple captured image frames to movement of the third and fourth feature points over the multiple captured image frames; and

(k) distinguishing, via the control, between the first detected object being a three dimensional object and the second detected object being a two dimensional object by determining, via the control, that movement of the first feature point over the multiple captured image frames is dissimilar to that of the second feature point and by determining, via the control, that movement of the third feature point over the multiple captured image frames is similar to that of the fourth feature point.

18. The method according to claim 17, wherein the second detected object comprises an object selected from the group consisting of (i) a shadow, (ii) a pavement marking and (iii) a ground marking.

19. The method according to claim 17, including unwarping a given pixel position to account for lens distortion prior to determining a physical location corresponding to the given pixel position.

20. The method according to claim 17, wherein the first and second feature points are portions of top and bottom edge contours of the first detected object.