VEHICULAR VISION SYSTEM

• US 20190054918A1
• Filed: 10/22/2018
• Published: 02/21/2019
• Est. Priority Date: 07/27/2009
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Lu, Yuesheng, Gibson, Joel S., Gebauer, Duane W., Shriner, Richard D., Miller, Patrick A.
• Granted Patent: US 10,875,526 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B62D 15/0295 by overlaying a vehicle path based on present steering angle over an image without processing that image
• CPC Class Codes: G08G 1/168 Driving aids for parking, e.g. acoustic or visual feedback on parking space
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2380/10 Automotive applications
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Abstract

A vehicular vision system includes a camera, a distance sensor and a controller having at least one processor, where image data captured by the camera and sensor data captured by the distance sensor are processed at the controller. The controller, responsive to processing of captured image data and of captured sensor data, detects an object. The controller determines the distance to the detected object based at least in part on difference between the positions of the detected object in captured image data and in captured sensor data. The controller, responsive to processing of captured image data and of captured sensor data, and responsive to the determined distance to the detected object, determines that the detected object represents a collision risk. The controller informs a driver of the vehicle of the collision risk and/or controls the vehicle to mitigate the collision risk.

Claims

1. A vehicular vision system, said vehicular vision system comprising:

a camera comprising a lens and an image sensor, wherein the camera is disposed at a vehicle equipped with said vehicular vision system so as to have a field of view exterior of the equipped vehicle;

a distance sensor disposed at the equipped vehicle so as to have a field of sensing exterior of the equipped vehicle;

wherein the distance sensor comprises a plurality of infrared light-emitting light sources, and wherein the distance sensor senses infrared light;

a controller comprising at least one processor, wherein image data captured by the camera and sensor data captured by the distance sensor are processed at the controller;

wherein the controller, responsive to processing at the controller of image data captured by the camera and of sensor data captured by the distance sensor, detects an object present in the field of view of the camera and in the field of sensing of the distance sensor;

wherein the controller determines the distance to the detected object based at least in part on difference between the position of the detected object in image data captured by the camera and the position of the detected object in sensor data captured by the distance sensor;

wherein the controller, responsive to processing at the controller of image data captured by the camera and of sensor data captured by the distance sensor, and responsive to the determined distance to the detected object, determines that the detected object represents a collision risk; and

wherein, responsive to determination that the detected object represents a collision risk, the controller carries out at least one action selected from the group consisting of (i) informing a driver of the equipped vehicle of the collision risk and (ii) controlling the equipped vehicle to mitigate the collision risk.

2. The vehicular vision system as claimed in claim 1, wherein the controller receives a depth map from the distance sensor and determines distance to the detected object based at least in part on the depth map.

3. The vehicular vision system as claimed in claim 1, wherein, responsive to determination that the detected object represents the collision risk, the controller alerts a driver of the equipped vehicle of the collision risk.

4. The vehicular vision system as claimed in claim 1, wherein, responsive to determination that the detected object represents the collision risk, the controller controls at least one vehicle component to mitigate collision by the equipped vehicle with the detected object.

5. The vehicular vision system as claimed in claim 4, wherein, responsive to determination that the detected object represents the collision risk, the controller controls braking of the equipped vehicle.

6. The vehicular vision system as claimed in claim 1, wherein image data captured by the camera has a first resolution and sensor data captured by the distance sensor has a second resolution that is lower than the first resolution.

7. The vehicular vision system as claimed in claim 1, wherein, prior to determining whether the detected object represents a collision risk, the controller determines whether the detected object falls within a category of objects that do not represent a collision risk based at least in part on processing of image data captured by the camera.

8. The vehicular vision system as claimed in claim 1, wherein, prior to determining whether the detected object represents a collision risk, the controller determines whether the detected object falls within a category of objects that represent objects of interest based at least in part on processing of image data captured by the camera.

9. The vehicular vision system as claimed in claim 8, wherein the category of objects that represent objects of interest include at least one selected from the group consisting of people, animals and other vehicles.

10. The vehicular vision system as claimed in claim 1, comprising a display disposed in an interior of the equipped vehicle and viewable by a driver of the equipped vehicle, and wherein the display displays video images derived from image data captured by the camera, and wherein the displayed video images include images of the detected object that represents the collision risk, and wherein the display displays an overlay that highlights the displayed detected object.

11. The vehicular vision system as claimed in claim 10, wherein the controller processes image data captured by the camera and processes sensor data captured by the sensor to determine a projected trajectory for the equipped vehicle, and wherein the controller applies a projected path overlay at the displayed video images, and wherein the projected path overlay comprises a representation of the projected trajectory.

12. The vehicular vision system as claimed in claim 10, wherein the controller modifies raw image data captured by the camera to produce a processed image, and wherein the controller determines the position of a selected feature in the field of view of the camera and applies an overlay on the selected feature in an initial raw image data to produce an initial processed image, and wherein the controller thereafter holds the overlay in a fixed position in subsequent processed images regardless of movement of the selected feature in subsequent raw image data.

13. The vehicular vision system as claimed in claim 12, wherein the selected feature is a feature on a trailer being towed by the equipped vehicle.

14. The vehicular vision system as claimed in claim 10, wherein the camera is positioned at an actual viewing angle, and wherein said vehicular vision system has a bird'"'"'s eye viewing mode in which the controller modifies image data captured by the camera so that the displayed video images appear to be at an apparent viewing angle that is more vertically oriented than the actual viewing angle of the camera.

15. The vehicular vision system as claimed in claim 14, wherein the controller compresses a lower portion of captured image data and stretches an upper portion of captured image data so that the apparent viewing angle is more vertically oriented than the actual viewing angle.

16. The vehicular vision system as claimed in claim 14, wherein the equipped vehicle has a towing hitch, and wherein the towing hitch is in the field of view of the camera, and wherein said vehicular vision system has a hitch viewing mode in which the towing hitch in the displayed video images is magnified to a magnification level that is greater than a magnification level provided in the bird'"'"'s eye viewing mode.

17. A vehicular vision system, said vehicular vision system comprising:

a camera comprising a lens and an image sensor, wherein the camera is disposed at a vehicle equipped with said vehicular vision system so as to have a field of view exterior of the equipped vehicle;

a distance sensor disposed at the equipped vehicle so as to have a field of sensing exterior of the equipped vehicle;

wherein the distance sensor comprises a plurality of infrared light-emitting light sources, and wherein the distance sensor senses infrared light;

a controller comprising at least one processor;

wherein image data captured by the camera and sensor data captured by the distance sensor are processed at the controller;

wherein the controller, responsive to processing at the controller of image data captured by the camera and of sensor data captured by the distance sensor, detects an object present in the field of view of the camera and in the field of sensing of the distance sensor;

wherein the controller receives a depth map from the distance sensor and determines distance to the detected object based at least in part on the depth map;

wherein the controller, responsive to processing at the controller of image data captured by the camera and of sensor data captured by the distance sensor, and responsive to the determined distance to the detected object, determines that the detected object represents a collision risk; and

wherein, responsive to determination that the detected object represents a collision risk, the controller controls the equipped vehicle to mitigate the collision risk.

18. The vehicular vision system as claimed in claim 17, comprising a display disposed in an interior of the equipped vehicle and viewable by a driver of the equipped vehicle, and wherein the display displays video images derived from image data captured by the camera, and wherein the displayed video images include images of the detected object.

19. The vehicular vision system as claimed in claim 18, wherein the display displays an overlay that highlights the displayed detected object.

20. The vehicular vision system as claimed in claim 18, wherein the camera is positioned at an actual viewing angle, and wherein said vehicular vision system has a bird'"'"'s eye viewing mode in which the displayed video images appear to be at an apparent viewing angle that is more vertically oriented than the actual viewing angle of the camera.

21. The vehicular vision system as claimed in claim 20, wherein the controller compresses a lower portion of captured image data and stretches an upper portion of captured image data so that the apparent viewing angle is more vertically oriented than the actual viewing angle.

22. The vehicular vision system as claimed in claim 20, wherein the equipped vehicle has a towing hitch, and wherein the towing hitch is in the field of view of the camera, and wherein said vehicular vision system has a hitch viewing mode in which the towing hitch in the displayed video images is magnified to a magnification level that is greater than a magnification level provided in the bird'"'"'s eye viewing mode.

23. A vehicular vision system, said vehicular vision system comprising:

a camera comprising a lens and an image sensor, wherein the camera is disposed at a vehicle equipped with said vehicular vision system so as to have a field of view exterior of the equipped vehicle;

a distance sensor disposed at the equipped vehicle so as to have a field of sensing exterior of the equipped vehicle;

wherein image data captured by the camera has a first resolution and sensor data captured by the distance sensor has a second resolution that is lower than the first resolution;

wherein the distance sensor comprises a plurality of infrared light-emitting light sources, and wherein the distance sensor senses infrared light;

a controller comprising at least one processor, wherein image data captured by the camera and sensor data captured by the distance sensor are processed at the controller;

a display disposed in an interior of the equipped vehicle and viewable by a driver of the equipped vehicle, and wherein the display displays video images derived from image data captured by the camera;

wherein the controller, responsive to processing at the controller of image data captured by the camera and of sensor data captured by the distance sensor, detects an object present in the field of view of the camera and in the field of sensing of the distance sensor;

wherein the controller determines the distance to the detected object based at least in part on difference between the position of the detected object in image data captured by the camera and the position of the detected object in sensor data captured by the distance sensor;

wherein the displayed video images include images of the detected object;

wherein the controller, responsive to processing at the controller of image data captured by the camera and of sensor data captured by the distance sensor, and responsive to the determined distance to the detected object, determines that the detected object represents a collision risk; and

wherein, responsive to determination that the detected object represents a collision risk, the controller carries out at least one action selected from the group consisting of (i) informing the driver of the equipped vehicle of the collision risk and (ii) controlling the equipped vehicle to mitigate the collision risk.

24. The vehicular vision system as claimed in claim 23, wherein the display displays an overlay that highlights the displayed detected object.

25. The vehicular vision system as claimed in claim 23, wherein the camera is positioned at an actual viewing angle, and wherein said vehicular vision system has a bird'"'"'s eye viewing mode in which the displayed video images appear to be at an apparent viewing angle that is more vertically oriented than the actual viewing angle of the camera.

26. The vehicular vision system as claimed in claim 25, wherein the equipped vehicle has a towing hitch, and wherein the towing hitch is in the field of view of the camera, and wherein said vehicular vision system has a hitch viewing mode in which the towing hitch in the displayed video images is magnified to a magnification level that is greater than a magnification level provided in the bird'"'"'s eye viewing mode.