VEHICLE VISION SYSTEM USING KINEMATIC MODEL OF VEHICLE MOTION

• US 20140350834A1
• Filed: 05/20/2014
• Published: 11/27/2014
• Est. Priority Date: 05/21/2013
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Turk, James
• Granted Patent: US 9,205,776 B2
• Time in Patent Office: Days
• US Class Current: 701/300
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A vision system of a vehicle includes a camera disposed at the vehicle and having a field of view exterior of the vehicle. The camera is operable to capture frames of image data. Responsive to image processing by an image processor of captured image data, a control is operable to determine objects present in the field of view of the camera. Responsive to vehicle data, the control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle. The control determines movement of an object relative to the vehicle via image processing of at least two frames of captured image data during driving of the vehicle by the driver of the vehicle. The control compares the determined relative movement of the object to the determined vehicle motion vector, and responsive to the comparison, the control may determine a misalignment of the camera.

Claims

1. A vision system of a vehicle comprising:

a camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises a two dimensional array of a plurality of photosensing elements;

wherein said camera is operable to capture frames of image data;

a control, said control comprising an image processor operable to process image data captured by said camera;

wherein, responsive to image processing of captured image data, said control is operable to determine objects present in the field of view of said camera;

wherein, responsive to vehicle data, said control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle;

wherein the vehicle data includes at least two of (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle;

wherein said control determines movement of an object relative to the vehicle via image processing of at least two frames of captured image data during driving of the vehicle by the driver of the vehicle;

wherein said control compares the determined relative movement of the object to the determined vehicle motion vector; and

wherein, responsive to a difference between the determined relative movement of the object and the determined vehicle motion vector, said control determines a misalignment of said camera.

2. The vision system of claim 1, wherein the vehicle data is communicated to the control via a communication bus of the vehicle.

3. The vision system of claim 1, wherein a kinematic model comprising a bicycle model is used to represent vehicle motion.

4. The vision system of claim 3, wherein said kinematic model utilizes at least one of (i) a wheel steering angle, (ii) a wheel pulse, (iii) a heading angle and (iv) translational vehicle motion.

5. The vision system of claim 1, wherein the vehicle data includes (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle.

6. The vision system of claim 1, wherein, responsive to a determined misalignment of said camera, said vision system is operable to calibrate said camera.

7. The vision system of claim 1, wherein said control compares the determined vehicle motion vector to an object vector determined between a first position of the object captured in a first frame of image data and a second position of the object captured in a second frame of image data.

8. The vision system of claim 7, wherein said control determines a misalignment of said camera responsive to a difference between a direction or magnitude of the determined vehicle motion vector and a direction or magnitude of the determined object vector.

9. The vision system of claim 1, wherein said vision system comprises a plurality of cameras disposed at the vehicle, each having a respective field of view exterior of the vehicle.

10. The vision system of claim 9, wherein image data captured by at least some of said plurality of cameras is used for a surround view system of the vehicle.

11. The vision system of claim 10, wherein said plurality of cameras comprises (i) a rear camera disposed at a rear portion of the vehicle and having a rearward field of view, (ii) a driver side camera disposed at a driver side exterior rearview mirror assembly of the vehicle and at least having a generally sideward and rearward field of view and (iii) a passenger side camera disposed at a passenger side exterior rearview mirror assembly of the vehicle and at least having a generally sideward and rearward field of view.

12. The vision system of claim 1, wherein said camera comprises a CMOS camera.

13. A vision system of a vehicle comprising:

a plurality of cameras disposed at a vehicle, each having a respective field of view exterior of the vehicle;

wherein said plurality of cameras comprises (i) a rear camera disposed at a rear portion of the vehicle and having a rearward field of view, (ii) a driver side camera disposed at a driver side of the vehicle and at least having a generally sideward and rearward field of view and (iii) a passenger side camera disposed at a passenger side of the vehicle and at least having a generally sideward and rearward field of view;

wherein said cameras are operable to capture frames of image data;

a control, said control comprising an image processor operable to process image data captured by said cameras;

wherein, responsive to image processing of captured image data, said control is operable to determine objects present in the fields of view of said cameras;

wherein, responsive to vehicle data, said control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle;

wherein the vehicle data includes at least two of (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle;

wherein said control determines movement of an object relative to the vehicle via image processing of at least two frames of image data captured by at least one of said cameras during driving of the vehicle by the driver of the vehicle;

wherein said control compares the determined relative movement of the object to the determined vehicle motion vector; and

wherein, responsive to a difference between the determined relative movement of the object and the determined vehicle motion vector, said control determines a misalignment of said at least one of said cameras.

14. The vision system of claim 13, wherein each of said cameras comprises a two dimensional array of a plurality of photosensing elements.

15. The vision system of claim 13, wherein the vehicle data is communicated to the control via a communication bus of the vehicle.

16. The vision system of claim 13, wherein, responsive to a determined misalignment of said at least one of said cameras, said vision system is operable to calibrate said at least one of said cameras.

17. The vision system of claim 13, wherein said control compares the determined vehicle motion vector to an object vector determined between a first position of the object captured in a first frame of image data by said at least one of said cameras and a second position of the object captured in a second frame of image data by said at least one of said cameras.

18. The vision system of claim 17, wherein said control determines a misalignment of said at least one of said cameras responsive to a difference between a direction or magnitude of the determined vehicle motion vector and a direction or magnitude of the determined object vector.

19. A vision system of a vehicle comprising:

a plurality of cameras disposed at a vehicle, each having a respective field of view exterior of the vehicle;

wherein said plurality of cameras comprises (i) a rear camera disposed at a rear portion of the vehicle and having a rearward field of view, (ii) a driver side camera disposed at a driver side of the vehicle and at least having a generally sideward and rearward field of view and (iii) a passenger side camera disposed at a passenger side of the vehicle and at least having a generally sideward and rearward field of view;

wherein each of said cameras comprises a two dimensional array of a plurality of photosensing elements;

wherein said cameras are operable to capture frames of image data;

a control, said control comprising an image processor operable to process image data captured by said cameras;

wherein, responsive to image processing of captured image data, said control is operable to determine objects present in the fields of view of said cameras;

wherein, responsive to vehicle data, said control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle;

wherein the vehicle data includes at least two of (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle;

wherein the vehicle data is communicated to the control via a communication bus of the vehicle;

wherein said control determines movement of an object relative to the vehicle via image processing of at least two frames of image data captured by at least one of said cameras during driving of the vehicle by the driver of the vehicle;

wherein said control compares the determined relative movement of the object to the determined vehicle motion vector;

wherein, responsive to a difference between the determined relative movement of the object and the determined vehicle motion vector, said control determines a misalignment of said at least one of said cameras; and

wherein, responsive to a determined misalignment of said at least one of said cameras, said vision system is operable to calibrate said at least one of said cameras.

20. The vision system of claim 19, wherein said control compares the determined vehicle motion vector to an object vector determined between a first position of the object captured in a first frame of image data by said at least one of said cameras and a second position of the object captured in a second frame of image data by said at least one of said cameras, and wherein said control determines a misalignment of said at least one of said cameras responsive to a difference between a direction or magnitude of the determined vehicle motion vector and a direction or magnitude of the determined object vector.