System and method for warning a driver of pedestrians and other obstacles when turning

• US 9,514,650 B2
• Filed: 02/05/2014
• Issued: 12/06/2016
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Ng-Thow-Hing, Victor, Tran, Cuong
• Primary Examiner(s): Harrison, Chante
• Application Number:
  US14/173,506
• Publication Number:
  US 20140266656A1
• Time in Patent Office: 1,035 Days
• Field of Search: 345/7, 345/633, 382/104, 382/107, 340/435, 340/436, 340/441, 340/461, 340/425.5, 715/708
• US Class Current: 1/1
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B62D 15/0295 by overlaying a vehicle path based on present steering angle over an image without processing that image
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A system for warning of potential hazards when a vehicle is turning has a sensor coupled to the vehicle and configured to capture data of objects located around the vehicle. A control unit is coupled to the sensor for processing the data captured by the sensor to generate graphical representations of objects captured by the sensor, graphical representations of projected paths of moving objects captured; and a graphical representation of a projected turning path of the vehicle.

Claims

1. A system for warning of potential hazards when a vehicle is turning comprising:

a sensor coupled to the vehicle and configured to capture data of objects located around the vehicle;

monitoring sensors coupled to the vehicle to measure a speed and a turning angle of the vehicle; and

a control unit coupled to the sensor for processing the data captured by the sensor to;

generate graphical representations of objects captured by the sensor;

generate graphical representations of projected paths of moving objects captured by the sensors based on speeds and travel directions of the moving objects;

calculate a projected turning path of the vehicle based on the speed and the turning angle of the vehicle; and

generate a graphical representation of the projected turning path, wherein the graphical representation indicating the projected turning path of the vehicle is divided into a plurality of sections, each section having a respective indicator, wherein each indicator illustrates a probability of impact with the moving object in the respective section of the projected turning path.

2. The system of claim 1, wherein each section is color coded to illustrate the probability of impact with the moving object in the respective section of the projected turning path.

3. The system of claim 1, further comprising monitoring sensors coupled to the vehicle and the control unit for monitoring at least one of the turning angle or the speed of the vehicle.

4. The system of claim 1, wherein the control unit generates the graphical representations of objects captured by the sensor, the graphical representations of projected paths of moving objects captured by the sensor, and the graphical representation of a projected turning path of the vehicle when the sensor indicates that the vehicle is turning.

5. The system of claim 1, further comprising a Global Positioning Satellite (GPS) system coupled to the vehicle and the control unit.

6. The system of claim 5, wherein the GPS system transfers imagery of a current location of the vehicle to the control unit, the control unit generating graphical representations of imagery transferred from the GPS of the current location.

7. The system of claim 1, further comprising a display coupled to the control unit to show the graphical representations of objects captured by the sensor, the graphical representations of projected paths of moving objects captured by the sensor, and the graphical representation indicating the projected turning path of the vehicle.

8. The system of claim 1, wherein the control unit comprises:

an image processing module coupled to the sensor to identify the objects detected by the sensors;

a path processing module to calculate the projected paths of moving objects detected by the sensor and the projected turning path of the vehicle; and

an Augmented Reality (AR) processing module to generate the graphical representations of the objects detected by the sensor, the graphical representations of the projected paths of the moving objects and the graphical representations indicating the projected turning path of the vehicle, wherein the projected turning path of the vehicle is dynamic and adjusts in size as the vehicle moves during the turn.

9. The system of claim 1, wherein the graphical representations of the objects detected by the sensor are two dimensional.

10. The system of claim 1, wherein the graphical representations of the objects detected by the sensor are three dimensional.

11. The system of claim 1, wherein the graphical representations of the objects include a first sign associated with at least one of the objects detected by the sensor which is in a Field of View (FOV) of a driver of the vehicle.

12. The system of claim 11, wherein the graphical representations of the objects include:

a second sign associated with at least one of the objects detected by the sensor which is outside of the FOV of the driver of the vehicle; and

a pointer sign associated with the second sign and positioned to indicate a location of the at least one of the objects detected by the sensor which is outside the FOV of the driver.

13. A system for warning of potential hazards when a vehicle is turning comprising:

image sensors coupled to the vehicle and configured to capture pedestrians located around the vehicle;

monitoring sensors coupled to the vehicle to measure speed and turning angle of the vehicle;

a control unit coupled to the image sensors and monitoring sensors for processing data from the image sensors and monitoring sensors and to;

generate graphical representations of pedestrians captured by the image sensors;

generate graphical representations of virtual crosswalks associated with pedestrians moving near the vehicle showing projected pathways of the pedestrians based on a speed and a travel direction of each of the pedestrians;

calculate a projected turning path of the vehicle based on a speed and turning angle of the vehicle; and

generate a graphical representation indicating the projected turning path of the vehicle, wherein the graphical representation indicating the projected turning path of the vehicle is divided into a plurality of sections, each section having a respective indicator, wherein each indicator illustrates a probability of impact with the moving object in the respective section of the projected turning path; and

a display to show the graphical representations of pedestrians captured by the sensor, the graphical representations of the virtual crosswalks associated with pedestrians moving near the vehicle; and

the graphical representation indicating the projected turning path of the vehicle.

14. The system of claim 13, further comprising a Global Positioning Satellite (GPS) system coupled to the vehicle and the control unit to transfer imagery of a current location of the vehicle to the control unit.

15. The system of claim 13, wherein the control unit comprises:

an image processing module coupled to the image sensors;

a path processing module to calculate the virtual crosswalks and the projected turning path of the vehicle; and

an Augmented Reality (AR) processing module to generate the graphical representations of the pedestrians detected by the sensor, the graphical representations of the virtual crosswalks and the graphical representations indicating the projected turning path of the vehicle, wherein the projected turning path of the vehicle is dynamic and adjusts in size as the vehicle moves during the turn.

16. The system of claim 13, wherein the control unit generates the graphical representations of pedestrians captured by the image sensors, the graphical representations of virtual crosswalks associated with pedestrians moving near the vehicle;

and the graphical representations indicating the projected turning path of the vehicle when the monitoring sensors indicate that the vehicle is turning.

17. A system for warning of potential hazards when a vehicle is turning comprising:

image sensors coupled to the vehicle and configured to capture pedestrians located around the vehicle;

monitoring sensors coupled to the vehicle to measure a speed and a turning angle of the vehicle;

a control unit coupled to the image sensors and monitoring sensors for processing data from the image sensors and monitoring sensors, wherein the control unit is configured to;

generate graphical representations of pedestrians captured by the sensor;

generate graphical representations of virtual crosswalks associated with pedestrians moving near the vehicle showing projected pathways of the pedestrians based on a speed and a travel direction of each of the pedestrians;

calculate a projected turning path of the vehicle based on a speed and turning angle of the vehicle; and

generate a graphical representation indicating the projected turning path of the vehicle when the monitoring sensors indicate that the vehicle is turning, wherein the graphical representation indicating the projected turning path of the vehicle is divided into a plurality of sections, each section having a respective indicator, wherein each indicator illustrates a probability of impact with the moving pedestrian in the respective section of the projected turning path; and

a Heads-Up Display HUD system coupled to the control unit to show the graphical representations of pedestrians captured by the sensor, the graphical representations of virtual crosswalks associated with pedestrians moving near the vehicle, and the graphical representations indicating the projected turning path of the vehicle.

18. The system of claim 17, wherein the control unit comprises:

an image processing module coupled to the image sensors;

a path processing module to calculate the virtual crosswalks and the projected turning path of the vehicle; and

an Augmented Reality (AR) processing module to generate the graphical representations of the pedestrians detected by the sensor, the graphical representations of the virtual crosswalks and the graphical representation indicating the projected turning path of the vehicle.