Forward sensing system for vehicle

• US 10,670,713 B2
• Filed: 10/22/2018
• Issued: 06/02/2020
• Est. Priority Date: 01/25/2007
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Lynam, Niall R.
• Primary Examiner(s): Bythrow, Peter M
• Application Number:
  US16/166,333
• Publication Number:
  US 20190056493A1
• Time in Patent Office: 589 Days
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/93 for anti-collision purposes
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 2013/0245 Radar with phased array antenna
• CPC Class Codes: G01S 2013/9321 Velocity regulation, e.g. cruise control
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 2013/93276 in the windshield area
• CPC Class Codes: G01S 7/03 Details of HF subsystems specially adapted therefor, e.g. common to transmitter and receiver
• CPC Class Codes: G01S 7/032 Constructional details for solid-state radar subsystems
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A forward sensing system for a vehicle includes a radar sensor and an image sensor housed in a self-contained unit disposed behind and attached at the vehicle windshield. A control includes an image processor that has an image processing chip that processes image data captured by the image sensor to detect an object of interest present exterior of the vehicle. The control, responsive at least in part to processing of captured image data and to sensing by the radar sensor, determines that a potentially hazardous condition exists in a path of forward travel of the vehicle. The radar sensor and the image sensor collaborate in a way that enhances sensing capability of the sensing system. Responsive to determination that the object of interest is in the path of forward travel of the vehicle, the control may at least in part control a driver assistance system of the vehicle.

Claims

1. A forward sensing system for a vehicle, said forward sensing system comprising:

a radar sensor disposed within an interior cabin of a vehicle equipped with said forward sensing system;

wherein said radar sensor has a sensing direction forward of the equipped vehicle;

wherein said radar sensor transmits at a frequency of at least 60 GHz;

wherein said radar sensor utilizes digital beam forming;

an image sensor disposed within the interior cabin of the equipped vehicle behind a windshield of the equipped vehicle;

wherein said radar sensor and said image sensor are housed in a self-contained unit disposed behind and attached at the windshield of the equipped vehicle;

wherein said self-contained unit attaches at the windshield of the equipped vehicle and is removable therefrom as a unit;

wherein the windshield of the equipped vehicle comprises an opaque layer where said self-contained unit is disposed behind the windshield of the equipped vehicle;

wherein the opaque layer comprises a light-transmitting aperture, and wherein, with said self-contained unit attached at the windshield of the equipped vehicle, said image sensor views forward through the windshield of the equipped vehicle via the light-transmitting aperture;

wherein said image sensor comprises a pixelated imaging array sensor comprising a plurality of photo-sensing pixels;

a control comprising an image processor, wherein said image processor comprises an image processing chip that processes image data captured by said image sensor utilizing object detection software;

wherein said image processing chip processes image data captured by said image sensor in order to detect an object of interest present exterior of the equipped vehicle in a direction of forward travel of the equipped vehicle and being imaged at said pixelated imaging array sensor;

wherein said control, responsive at least in part to processing of captured image data by said image processing chip and to sensing by said radar sensor, determines that a potentially hazardous condition exists in a path of forward travel of the equipped vehicle;

wherein the determined potentially hazardous condition comprises the object of interest being in the path of forward travel of the equipped vehicle;

wherein said radar sensor and said image sensor collaborate in a way that enhances sensing capability of said forward sensing system for the object of interest in the path of forward travel of the equipped vehicle;

wherein said radar sensor and said image sensor share, at least in part, common circuitry;

wherein said image processing chip and said common circuitry are housed in said self-contained unit disposed behind and attached at the windshield of the equipped vehicle; and

wherein, responsive to determination that the object of interest is in the path of forward travel of the equipped vehicle, said control at least in part controls a driver assistance system of the equipped vehicle.

2. The forward sensing system of claim 1, wherein responsive to determination that the object of interest is in the path of forward travel of the equipped vehicle, a driver of the equipped vehicle is alerted to the determined potentially hazardous condition.

3. The forward sensing system of claim 2, wherein video images captured by said image sensor are displayed on a video display screen viewable to the driver of the equipped vehicle, and wherein, responsive to determination that the object of interest is in the path of forward travel of the equipped vehicle, a visual alert is displayed by said video display screen to alert the driver of the equipped vehicle to the determined potentially hazardous condition.

4. The forward sensing system of claim 3, wherein said image sensor is part of a night vision system of the equipped vehicle.

5. The forward sensing system of claim 2, wherein the driver of the equipped vehicle is audibly alerted to the determined potentially hazardous condition.

6. The forward sensing system of claim 1, wherein the driver assistance system of the equipped vehicle comprises a speed control system of the equipped vehicle.

7. The forward sensing system of claim 6, wherein the speed control system of the equipped vehicle comprises an adaptive cruise control system of the equipped vehicle.

8. The forward sensing system of claim 6, wherein the detected object of interest is a deer.

9. The forward sensing system of claim 1, wherein the detected object of interest is a pedestrian.

10. The forward sensing system of claim 1, wherein the detected object of interest is a vehicle.

11. The forward sensing system of claim 1, wherein said radar sensor transmits at a frequency of at least 77 GHz.

12. The forward sensing system of claim 11, wherein said radar sensor transmits at 79 GHz.

13. The forward sensing system of claim 1, wherein said image processing chip processes image data captured by said image sensor for an automatic headlamp control system of the equipped vehicle.

14. The forward sensing system of claim 13, wherein said image processing chip processes image data captured by said image sensor for a lane departure warning system of the equipped vehicle.

15. The forward sensing system of claim 1, wherein said common circuitry includes said image processing chip.

16. The forward sensing system of claim 15, wherein said radar sensor utilizes a phased array antenna.

17. The forward sensing system of claim 16, wherein the detected object of interest present exterior of the equipped vehicle in a direction of forward travel of the equipped vehicle is imaged at a sub-array of said pixelated imaging array sensor.

18. A forward sensing system for a vehicle, said forward sensing system comprising:

a radar sensor disposed within an interior cabin of a vehicle equipped with said forward sensing system;

wherein said radar sensor has a sensing direction forward of the equipped vehicle;

wherein said radar sensor transmits at a frequency of at least 60 GHz;

wherein said radar sensor utilizes digital beam forming;

an image sensor disposed within the interior cabin of the equipped vehicle behind a windshield of the equipped vehicle;

wherein said radar sensor and said image sensor are housed in a self-contained unit disposed behind and attached at the windshield of the equipped vehicle;

wherein said self-contained unit attaches at the windshield of the equipped vehicle and is removable therefrom as a unit;

wherein the windshield of the equipped vehicle comprises an opaque layer where said self-contained unit is disposed behind the windshield of the equipped vehicle;

wherein the opaque layer comprises a light-transmitting aperture, and wherein, with said self-contained unit attached at the windshield of the equipped vehicle, said image sensor views forward through the windshield of the equipped vehicle via the light-transmitting aperture;

wherein said image sensor comprises a pixelated imaging array sensor comprising a plurality of photo-sensing pixels;

a control comprising an image processor, wherein said image processor comprises an image processing chip that processes image data captured by said image sensor utilizing object detection software;

wherein said image processing chip processes image data captured by said image sensor in order to detect an pedestrian present exterior of the equipped vehicle in a direction of forward travel of the equipped vehicle and being imaged at said pixelated imaging array sensor;

wherein said control, responsive at least in part to processing of captured image data by said image processing chip and to sensing by said radar sensor, determines that a potentially hazardous condition exists in a path of forward travel of the equipped vehicle;

wherein the determined potentially hazardous condition comprises the pedestrian being in the path of forward travel of the equipped vehicle;

wherein said radar sensor and said image sensor collaborate in a way that enhances sensing capability of said forward sensing system for the pedestrian in the path of forward travel of the equipped vehicle;

wherein said radar sensor and said image sensor share, at least in part, common circuitry;

wherein said image processing chip and said common circuitry are housed in said self-contained unit disposed behind and attached at the windshield of the equipped vehicle; and

wherein, responsive to determination that the pedestrian is in the path of forward travel of the equipped vehicle, said control at least in part controls a speed control system of the equipped vehicle.

19. The forward sensing system of claim 18, wherein said image processing chip processes image data captured by said image sensor for an automatic headlamp control system of the equipped vehicle, and wherein said image processing chip processes image data captured by said image sensor for a lane departure warning system of the equipped vehicle.

20. The forward sensing system of claim 19, wherein the speed control system of the equipped vehicle comprises an adaptive cruise control system of the equipped vehicle.

21. The forward sensing system of claim 20, wherein said common circuitry includes said image processing chip.

22. The forward sensing system of claim 21, wherein said radar sensor utilizes a phased array antenna.

23. The forward sensing system of claim 22, wherein the detected pedestrian present exterior of the equipped vehicle in a direction of forward travel of the equipped vehicle is imaged at a sub-array of said pixelated imaging array sensor.

24. A forward sensing system for a vehicle, said forward sensing system comprising:

a radar sensor disposed within an interior cabin of a vehicle equipped with said forward sensing system;

wherein said radar sensor has a sensing direction forward of the equipped vehicle;

wherein said radar sensor transmits at a frequency of at least 60 GHz;

wherein said radar sensor utilizes digital beam forming;

an image sensor disposed within the interior cabin of the equipped vehicle behind a windshield of the equipped vehicle;

wherein said radar sensor and said image sensor are housed in a self-contained unit disposed behind and attached at the windshield of the equipped vehicle;

wherein said self-contained unit attaches at the windshield of the equipped vehicle and is removable therefrom as a unit;

wherein the windshield of the equipped vehicle comprises an opaque layer where said self-contained unit is disposed behind the windshield of the equipped vehicle;

wherein the opaque layer comprises a light-transmitting aperture, and wherein, with said self-contained unit attached at the windshield of the equipped vehicle, said image sensor views forward through the windshield of the equipped vehicle via the light-transmitting aperture;

wherein said image sensor comprises a pixelated imaging array sensor comprising a plurality of photo-sensing pixels;

a control comprising an image processor, wherein said image processor comprises an image processing chip that processes image data captured by said image sensor utilizing object detection software;

wherein said image processing chip processes image data captured by said image sensor in order to detect another vehicle present exterior of the equipped vehicle in a direction of forward travel of the equipped vehicle and being imaged at said pixelated imaging array sensor;

wherein said control, responsive at least in part to processing of captured image data by said image processing chip and to sensing by said radar sensor, determines that a potentially hazardous condition exists in a path of forward travel of the equipped vehicle;

wherein the determined potentially hazardous condition comprises the other vehicle being in the path of forward travel of the equipped vehicle;

wherein said radar sensor and said image sensor collaborate in a way that enhances sensing capability of said forward sensing system for the other vehicle in the path of forward travel of the equipped vehicle;

wherein said radar sensor and said image sensor share, at least in part, common circuitry;

wherein said image processing chip and said common circuitry are housed in said self-contained unit disposed behind and attached at the windshield of the equipped vehicle; and

wherein, responsive to determination that the other vehicle is in the path of forward travel of the equipped vehicle, said control at least in part controls a speed control system of the equipped vehicle.

25. The forward sensing system of claim 24, wherein said image processing chip processes image data captured by said image sensor for an automatic headlamp control system of the equipped vehicle, and wherein said image processing chip processes image data captured by said image sensor for a lane departure warning system of the equipped vehicle.

26. The forward sensing system of claim 25, wherein the speed control system of the equipped vehicle comprises an adaptive cruise control system of the equipped vehicle.

27. The forward sensing system of claim 26, wherein said common circuitry includes said image processing chip.

28. The forward sensing system of claim 25, wherein said radar sensor utilizes a phased array antenna.

29. The forward sensing system of claim 28, wherein the detected other vehicle present exterior of the equipped vehicle in a direction of forward travel of the equipped vehicle is imaged at a sub-array of said pixelated imaging array sensor.