Parking assist system with annotated map generation

• US 10,328,932 B2
• Filed: 06/01/2015
• Issued: 06/25/2019
• Est. Priority Date: 06/02/2014
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gieseke, Achim, Goseberg, Frank, Biemer, Michael, Boegel, Ruediger, Pflug, Goerg
• Primary Examiner(s): Mustafa, Imran K
• Application Number:
  US14/726,906
• Publication Number:
  US 20150344028A1
• Time in Patent Office: 1,485 Days
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2555/00 Input parameters relating to exterior conditions, not covered by groups B60W2552/00, B60W2554/00
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 30/06 Automatic manoeuvring for parking
• CPC Class Codes: B60W 30/095 Predicting travel path or likelihood of collision
• CPC Class Codes: B60W 50/0097 Predicting future conditions
• CPC Class Codes: B62D 15/028 Guided parking by providing commands to the driver, e.g. acoustically or optically

Abstract

A driver assistance system of a vehicle includes at least one vehicle environmental sensor having a field of sensing exterior of the vehicle. A sensor data processor is operable to process sensor data captured by the sensor. When the vehicle is operated, the sensor data processor receives captured sensor data that is representative of the vehicle surrounding scene. The sensor data processor fuses the sensor data with map data to generate an annotated master map of the scene captured by the sensor. A display device is operable to display information for viewing by a driver of the vehicle. The displayed information is derived at least in part from the annotated master map and is displayed for viewing by the driver of the vehicle who is executing a maneuver of the vehicle.

Claims

1. A driver assistance system of a vehicle, said driver assistance system comprising:

at least one sensor disposed at a vehicle equipped with said driver assistance system and having a field of sensing exterior of the equipped vehicle;

a sensor data processor operable to process sensor data captured by said at least one sensor;

map data derived at least in part from a navigational system of the equipped vehicle, wherein the map data comprises data that is at least one of (i) stored in memory of the equipped vehicle, (ii) downloaded to the equipped vehicle from a remote map data source and (iii) generated by a GPS system of the equipped vehicle;

a display device operable to display a master map derived from the map data for viewing by a driver of the equipped vehicle operating the equipped vehicle;

wherein, when the driver of the equipped vehicle is operating the equipped vehicle, said sensor data processor receives sensor data captured by said at least one sensor that is representative of a scene exterior the equipped vehicle;

wherein, responsive to processing of said sensor data by said sensor data processor, said system determines a parking space existing in the scene exterior the equipped vehicle and generates an indication label for the determined parking space existing in the scene exterior the equipped vehicle;

wherein, responsive to determination of the parking space existing in the scene exterior the equipped vehicle, said sensor data processor uses said sensor data and the map data to annotate the master map of the scene exterior the equipped vehicle being displayed by said display device and said system adds the generated indication label to the displayed master map to indicate the determined parking space existing in the scene exterior the equipped vehicle; and

wherein, while the driver is operating the equipped vehicle, said display device displays the annotated master map for viewing by the driver of the equipped vehicle who is operating the equipped vehicle.

2. The driver assistance system of claim 1, wherein said sensor data is selectively filtered by a filter before being used with the map data to annotate the master map.

3. The driver assistance system of claim 2, wherein said filter comprises a neural grid with neurons having filter parameters.

4. The driver assistance system of claim 2, wherein said filter comprises a classifier having filter parameters learned and optimized during a learning procedure.

5. The driver assistance system of claim 4, wherein said classifier comprises one of a Baysien type classifier and an Adaboost type classifier.

6. The driver assistance system of claim 2, wherein said filter is selectivity controlled by parameters derived from at least one of static sensor properties and dynamic sensor properties.

7. The driver assistance system of claim 6, wherein said filter is selectivity controlled by parameters derived from static sensor properties and wherein said static sensor properties comprises at least a lens map function.

8. The driver assistance system of claim 6, wherein said filter is selectivity controlled by parameters derived from static sensor properties and wherein said static sensor properties comprises at least an ultrasonic sensor sensitivity coil.

9. The driver assistance system of claim 6, wherein said filter is selectivity controlled by parameters derived from dynamic sensor properties and wherein said dynamic sensor properties comprises how steady said sensor'"'"'s map cell frame-wise classification is over time.

10. The driver assistance system of claim 6, wherein said filter is selectivity controlled by parameters derived from dynamic sensor properties and wherein said dynamic sensor properties comprises a time period which has lapsed upon the last sample time of said specific sensor.

11. The driver assistance system of claim 1, wherein said at least one sensor comprises at least one camera having an exterior field of view at the equipped vehicle and wherein said sensor data processor comprises an image data processor, and wherein operation of the equipped vehicle comprises a parking maneuver of the equipped vehicle.

12. The driver assistance system of claim 1, wherein said at least one sensor comprises a plurality of cameras having respective exterior fields of view at the equipped vehicle and wherein said sensor data processor comprises an image data processor operable to process image data captured by said cameras.

13. The driver assistance system of claim 1, wherein the annotated master map is input to a consecutive classification system trained for predicting probable parking spaces according to the annotated master map element'"'"'s scene arrangement.

14. The driver assistance system of claim 13, wherein said consecutive classification system comprises one of a Baysien type classifier and an Adaboost type classifier.

15. The driver assistance system of claim 1, wherein said sensor data processor is operable to process sensor data that were pre-processed by another processor.

20. The driver assistance system of claim 12, wherein, responsive to processing of said sensor data by said sensor data processor, said system determines objects in the field of view of at least one of said cameras and generates labels for the determined objects, and wherein said system adds the labels to the displayed master map to indicate objects determined, via processing of sensor data captured by said sensor, to be in the field of sensing of said sensor.

16. A driver assistance system of a vehicle, said driver assistance system comprising:

a plurality of cameras disposed at a vehicle equipped with said driver assistance system and having respective fields of view exterior of the equipped vehicle;

an image data processor operable to process image data captured by said cameras;

map data derived at least in part from a navigational system of the equipped vehicle, wherein the map data comprises data that is at least one of (i) stored in memory of the equipped vehicle, (ii) downloaded to the equipped vehicle from a remote map data source and (iii) generated by a GPS system of the equipped vehicle;

a display device operable to display a master map derived from the map data for viewing by a driver of the equipped vehicle executing a parking maneuver of the equipped vehicle;

wherein, when the driver of the equipped vehicle is executing the parking maneuver of the equipped vehicle, said image data processor receives image data captured by said cameras that is representative of a scene exterior the equipped vehicle;

wherein, responsive to processing of said image data by said image data processor, said system determines a parking space existing in the scene exterior the equipped vehicle and generates an indication label for the determined parking space existing in the scene exterior the equipped vehicle;

wherein, responsive to determination of the parking space existing in the scene exterior the equipped vehicle, said image data processor uses said image data and said map data to annotate the master map of the scene exterior the equipped vehicle being displayed by said display device and said system adds the generated indication label to the displayed master map to indicate the determined parking space existing in the scene exterior the equipped vehicle; and

wherein, while the driver is executing the parking maneuver of the equipped vehicle, said display device displays the annotated master map for viewing by the driver of the equipped vehicle who is executing the parking maneuver of the equipped vehicle.

17. The driver assistance system of claim 16, wherein the annotated master map is input to a consecutive classification system trained for predicting probable parking spaces according to the annotated master map element'"'"'s scene arrangement.

18. The driver assistance system of claim 16, wherein said image data is selectively filtered by a filter before being used with the map data to annotate the master map.

19. The driver assistance system of claim 16, wherein, responsive to processing of said image data by said image data processor, said system determines objects in the field of view of at least one of said cameras and generates labels for the determined objects, and wherein said system adds the labels to the displayed master map to indicate objects determined, via image processing of image data captured by said cameras, to be in the field of view of at least one of said cameras.