Vehicle vision system with vehicle detection

• US 10,210,404 B2
• Filed: 02/19/2018
• Issued: 02/19/2019
• Est. Priority Date: 05/11/2015
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gupta, Nikhil, Zhang, Liang
• Primary Examiner(s): Truong, Nguyen T
• Application Number:
  US15/899,116
• Publication Number:
  US 20180173972A1
• Time in Patent Office: 365 Days
• Field of Search: 348148
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A method for detecting a vehicle via a vehicular vision system includes equipping a vehicle with a camera and providing a control at the equipped vehicle. Frames of image data captured by the camera are processed, via an image processor of the control. Responsive at least in part to (i) vehicle motion information of the equipped vehicle and (ii) processing, via the image processor, of frames of image data captured by the camera, detecting a vehicle present in the field of view of the camera and determining motion of the detected vehicle relative to the moving equipped vehicle, wherein determining motion of the detected vehicle relative to the moving equipped vehicle includes (i) determining corresponding feature points of the detected vehicle in at least two frames of captured image data and (ii) estimating vehicle motion trajectory of the detected vehicle based on the determined corresponding feature points.

Claims

1. A method for detecting a vehicle via a vehicular vision system, said method comprising:

equipping a vehicle with a camera so that the camera has a field of view exterior of the equipped vehicle;

providing a control at the equipped vehicle, the control comprising an image processor;

capturing frames of image data with the camera;

processing, via the image processor, frames of image data captured by the camera;

while the equipped vehicle is moving, and responsive at least in part to (i) vehicle motion information of the equipped vehicle and (ii) processing, via the image processor, of frames of image data captured by the camera, detecting a vehicle present in the field of view of the camera and determining motion of the detected vehicle relative to the moving equipped vehicle; and

wherein determining motion of the detected vehicle relative to the moving equipped vehicle comprises (i) determining corresponding feature points of the detected vehicle in at least two frames of image data captured by the camera and (ii) estimating vehicle motion trajectory of the detected vehicle based on the determined corresponding feature points.

2. The method of claim 1, wherein vehicle motion information includes at least speed of the equipped vehicle.

3. The method of claim 2, wherein vehicle motion information includes trajectory of the equipped vehicle.

4. The method of claim 3, wherein determining motion of the detected vehicle relative to the moving equipped vehicle comprises determining structure of the detected vehicle based on the estimated vehicle motion trajectory of the detected vehicle.

5. The method of claim 1, comprising providing vehicle motion information to the control via a CAN bus of the equipped vehicle.

6. The method of claim 1, wherein determining motion of the detected vehicle relative to the moving equipped vehicle is responsive at least in part to (i) vehicle motion information and (ii) image processing, via the image processor, of at least four frames of image data captured by the camera.

7. The method of claim 1, wherein determining motion of the detected vehicle relative to the moving equipped vehicle is responsive at least in part to utilization of at least two constraints on trajectory.

8. The method of claim 7, wherein the at least two constraints comprise at least one of (i) object translation at a constant speed over a certain time period and (ii) object translation at an arbitrary speed over a certain time period.

9. The method of claim 1, wherein moving feature points are determined via the following equation:

x<sub>t−

1</sub>^TE{circumflex over (x)}_t=x<sub>t−

1</sub>^TEP_tΔ

X;

where x<sub>t−

1</sub> and x_t are projections at position X at time instants t−

1 and t, {circumflex over (x)}_t is projection at position X+Δ

X at time instant t, and P_t is a projection matrix of the camera.

10. The method of claim 9, comprising performing a feature correspondence analysis between two frames of captured image data and, given a number of correspondent feature pairs, solving the equation to provide an estimate of object motion Δ

X.

11. The method of claim 10, comprising calculating structure of an object point X by using image projection equations along with the estimated object motion Δ

X.

12. The method of claim 1, wherein the camera is part of a multi-camera surround view vision system of the equipped vehicle, and wherein said method comprises providing a display at the equipped vehicle for viewing by a driver of the equipped vehicle, and displaying, via the display, images derived from image data captured by the camera and at least some of the other cameras of the multi-camera surround view vision system to assist the driver in maneuvering the equipped vehicle.

13. A method for detecting a vehicle via a vehicular vision system, said method comprising:

equipping a vehicle with a camera so that the camera has a field of view exterior of the equipped vehicle;

providing a control at the equipped vehicle, the control comprising an image processor;

capturing frames of image data with the camera;

processing, via the image processor, frames of image data captured by the camera;

while the equipped vehicle is moving, and responsive at least in part to (i) vehicle motion information of the equipped vehicle and (ii) processing, via the image processor, of frames of image data captured by the camera, detecting a vehicle present in the field of view of the camera and determining motion of the detected vehicle relative to the moving equipped vehicle;

wherein vehicle motion information includes at least speed of the equipped vehicle and trajectory of the equipped vehicle;

wherein determining motion of the detected vehicle relative to the moving equipped vehicle comprises (i) determining corresponding feature points of the detected vehicle in at least two frames of image data captured by the camera and (ii) estimating vehicle motion trajectory of the detected vehicle based on the determined corresponding feature points; and

wherein the camera is part of a multi-camera surround view vision system of the equipped vehicle.

14. The method of claim 13, comprising providing vehicle motion information to the control via a communication bus of the equipped vehicle.

15. The method of claim 14, wherein the communication bus comprises a CAN bus of the equipped vehicle, and wherein said method comprises providing a display at the equipped vehicle for viewing by a driver of the equipped vehicle, and displaying, via the display, images derived from image data captured by the camera and at least some of the other cameras of the multi-camera surround view vision system to assist the driver in maneuvering the equipped vehicle.

16. The method of claim 14, wherein determining motion of the detected vehicle relative to the moving equipped vehicle comprises determining structure of the detected vehicle based on the estimated vehicle motion trajectory of the detected vehicle.

17. The method of claim 13, wherein determining motion of the detected vehicle relative to the moving equipped vehicle is responsive at least in part to utilization of at least two constraints on trajectory.

18. The method of claim 17, wherein the at least two constraints comprise (i) object translation at a constant speed over a certain time period and (ii) object translation at an arbitrary speed over a certain time period.

19. A method for detecting a vehicle via a vehicular vision system, said method comprising:

equipping a vehicle with a camera so that the camera has a field of view exterior of the equipped vehicle;

providing a control at the equipped vehicle, the control comprising an image processor;

capturing frames of image data with the camera;

processing, via the image processor, frames of image data captured by the camera;

while the equipped vehicle is moving, and responsive at least in part to (i) vehicle motion information of the equipped vehicle and (ii) processing, via the image processor, of at least four frames of image data captured by the camera, detecting a vehicle present in the field of view of the camera and determining motion of the detected vehicle relative to the moving equipped vehicle;

wherein vehicle motion information includes at least speed of the equipped vehicle; and

wherein determining motion of the detected vehicle relative to the moving equipped vehicle comprises (i) determining corresponding feature points of the detected vehicle in at least two frames of image data captured by the camera and (ii) estimating vehicle motion trajectory of the detected vehicle based on the determined corresponding feature points.

20. The method of claim 19, comprising providing vehicle motion information to the control via a CAN bus of the equipped vehicle.

21. The method of claim 20, wherein vehicle motion information includes trajectory of the equipped vehicle.

22. The method of claim 20, wherein determining motion of the detected vehicle relative to the moving equipped vehicle comprises determining structure of the detected vehicle based on the estimated vehicle motion trajectory of the detected vehicle.

23. The method of claim 19, wherein the camera is part of a multi-camera surround view vision system of the equipped vehicle, and wherein said method comprises providing a display at the equipped vehicle for viewing by a driver of the equipped vehicle, and displaying, via the display, images derived from image data captured by the camera and at least some of the other cameras of the multi-camera surround view vision system to assist the driver in maneuvering the equipped vehicle.

24. The method of claim 19, wherein determining motion of the detected vehicle relative to the moving equipped vehicle is responsive at least in part to utilization of at least two constraints on trajectory.

25. The method of claim 24, wherein the at least two constraints comprise (i) object translation at a constant speed over a certain time period and (ii) object translation at an arbitrary speed over a certain time period.