Sensor blind spot indication for vehicles

• US 10,328,949 B2
• Filed: 01/28/2016
• Issued: 06/25/2019
• Est. Priority Date: 01/28/2016
• Status: Expired due to Fees

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Kamata, Nobuhide
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US15/008,518
• Publication Number:
  US 20170221359A1
• Time in Patent Office: 1,244 Days
• Field of Search: 701 23- 28
• CPC Class Codes: B60K 2360/178 Warnings
• CPC Class Codes: B60K 2360/179 Distances to obstacles or vehicles
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 30/143 Speed control B60W30/16 takes precedence
• CPC Class Codes: B60W 50/085 Changing the parameters of the control units, e.g. changing limit values, working points by control input
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G07C 5/0825 using optical means
• CPC Class Codes: G07C 5/0833 using audio means
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

A vehicle can be configured to indicate sensor blind spots to a vehicle occupant (e.g., a driver). Using one or more sensors, the vehicle can acquire driving environment data of an external environment of the vehicle. It can be determined whether one or more portions of the acquired driving environment data is unreliable. Responsive to determining that one or more portions of the acquired driving environment data is unreliable, an alert can be caused to be presented within the vehicle. The alert can indicate one or more locations in the external environment in which the driving environment data that is determined to be unreliable. In some instances, the alert can be a visual alert and/or an audial alert.

Claims

1. A method of indicating sensor blind spots for an autonomous vehicle, the method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the autonomous vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including;

determining whether one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by at least one of the one or more sensors; and

responsive to determining that one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by the at least one of the one or more sensors, determining that the portion of the acquired driving environment data associated with the at least one of the one or more sensors that are occluded by one or more physical objects to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

2. The method of claim 1, wherein the alert is a visual alert, and wherein causing the visual alert to be presented within the autonomous vehicle includes causing the visual alert to be presented on a display within the autonomous vehicle.

3. The method of claim 2, wherein the visual alert includes a graphical representation of the autonomous vehicle and a graphical representation of the one or more portions of the external environment is not reliably detectable by the one or more sensors.

4. The method of claim 1, wherein the alert is a wherein the alert is an audial alert.

5. The method of claim 4, wherein causing the audial alert to be presented within the autonomous vehicle includes causing the audial alert to be presented within the autonomous vehicle in one or more locations that spatially correspond to the one or more locations in the external environment in which the acquired driving environment data that is determined to be unreliable.

7. The method of claim 1, further including:

responsive to determining that one or more portions of the acquired driving environment data is unreliable, determining a modification to a current driving maneuver of the autonomous vehicle; and

causing the determined modification to the current driving maneuver of the autonomous vehicle to be implemented.

8. The method of claim 7, wherein the determined modification to a current driving maneuver of the autonomous vehicle includes reducing a current speed of the autonomous vehicle.

9. The method of claim 1, wherein determining whether one or more portions of the acquired driving environment data is unreliable includes:

detecting whether any of the one or more sensors are inoperative or faulting; and

responsive to detecting that any of the one or more sensors are inoperative or faulting, determining that the portion of the acquired driving environment data associated with the inoperative or faulting sensors to be unreliable.

6. A method of indicating sensor blind spots for an autonomous vehicle, the method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the autonomous vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including determining whether sensor data is completely occluded in one or more portions of the external environment of the autonomous vehicle;

responsive to determining that sensor data is completely occluded in one or more portions of the external environment, causing the autonomous vehicle to transition to a manual operational mode or safety operational mode; and

causing an alert to be presented within the autonomous vehicle, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

10. A method of indicating sensor blind spots for a vehicle, the method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including;

comparing the acquired driving environment data to a map;

determining whether there is a difference between the acquired driving environment data and an expected feature included in the map; and

responsive to determining whether there is a difference between the acquired driving environment data and an expected feature included in the map, determining that that portion of the acquired driving environment data associated with the difference between the acquired driving environment data and an expected feature included in the map to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the vehicle, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

11. A sensor blind spot indication system for an autonomous vehicle, the system comprising:

one or more sensors configured to acquire driving environment data of at least a portion of an external environment of the autonomous vehicle; and

a processor operatively connected to the one or more sensors, the processor being programmed to initiate executable operations comprising;

determining whether one or more portions of acquired driving environment data is unreliable, the determining including;

determining whether one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by at least one of the one or more sensors; and

responsive to determining that one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by at least one of the one or more sensors, determining that the portion of the acquired driving environment data associated with the one or more sensors that are occluded by at least one of one or more physical objects to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

12. The system of claim 11, further including a display, wherein the display is operatively connected to the processor, wherein the alert is a visual alert, and wherein causing an alert to be presented within the autonomous vehicle includes causing the visual alert to be presented on the display.

13. The system of claim 11, wherein the alert is a visual alert, and further including:

a projector; and

a projection surface,

wherein causing an alert to be presented within the autonomous vehicle includes causing the projector to present the visual alert on the projection surface.

14. The system of claim 13, wherein the visual alert includes a graphical representation of the autonomous vehicle and a graphical representation of the one or more portions of the external environment is not reliably detectable by the one or more sensors.

15. The system of claim 11, further including a plurality of speakers, wherein the plurality of speakers are operatively connected to the processor, wherein the alert is a audial alert, and wherein causing an alert to be presented within the autonomous vehicle includes causing the audial alert to be presented by one or more of the plurality of speakers.

16. The system of claim 15, wherein causing the audial alert to be presented within the autonomous vehicle includes causing the audial alert to be presented within the autonomous vehicle by one or more of the plurality of speakers in one or more locations that spatially correspond to the one or more locations in the external environment in which the acquired driving environment data that is determined to be unreliable.

17. The system of claim 11, wherein the executable operations further include:

responsive to determining that one or more portions of the acquired driving environment data is unreliable, determining a modification to a current driving maneuver of the autonomous vehicle; and

causing the determined modification to the current driving maneuver of the autonomous vehicle to be implemented.

18. The system of claim 17, wherein the determined modification to a current driving maneuver of the autonomous vehicle includes reducing a current speed of the autonomous vehicle.

19. A computer program product for indicating sensor blind spots for an autonomous vehicle, the computer program product comprising a computer readable storage medium having program code embodied therein, the program code executable by a processor to perform a method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the autonomous vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including;

determining whether one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by at least one of the one or more sensors; and

responsive to determining that one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by at least one of the one or more sensors, determining that the portion of the acquired driving environment data associated with the at least one of the one or more sensors that are occluded by one or more physical objects to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

20. A sensor blind spot indication system for a vehicle, the system comprising:

one or more sensors configured to acquire driving environment data of at least a portion of an external environment of the vehicle; and

a processor operatively connected to the one or more sensors, the processor being programmed to initiate executable operations comprising;

determining whether one or more portions of acquired driving environment data is unreliable, the determining including;

comparing the acquired driving environment data to a map;

determining whether there is a difference between the acquired driving environment data and an expected feature included in the map; and

responsive to determining whether there is a difference between the acquired driving environment data and an expected feature included in the map, determining that that portion of the acquired driving environment data associated with the difference between the acquired driving environment data and an expected feature included in the map to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the vehicle, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

21. A computer program product for indicating sensor blind spots for a vehicle, the computer program product comprising a computer readable storage medium having program code embodied therein, the program code executable by a processor to perform a method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including;

comparing the acquired driving environment data to a map;

determining whether there is a difference between the acquired driving environment data and an expected feature included in the map; and

responsive to determining whether there is a difference between the acquired driving environment data and an expected feature included in the map, determining that that portion of the acquired driving environment data associated with the difference between the acquired driving environment data and an expected feature included in the map to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the vehicle, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

22. A method of indicating sensor blind spots for an autonomous vehicle, the method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the autonomous vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including;

determining whether map data is available for a current external environment of the autonomous vehicle; and

responsive to determining that map data is not available for at least a portion of the current external environment of the autonomous vehicle, determining that the acquired driving environment data associated with the at least a portion of the current external environment of the autonomous vehicle for which map data is not available to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

23. A sensor blind spot indication system for an autonomous vehicle, the system comprising:

one or more sensors configured to acquire driving environment data of at least a portion of an external environment of the autonomous vehicle; and

a processor operatively connected to the one or more sensors, the processor being programmed to initiate executable operations comprising;

determining whether one or more portions of acquired driving environment data is unreliable, the determining including;

determining whether map data is available for a current external environment of the autonomous vehicle; and

responsive to determining that map data is not available for at least a portion of the current external environment of the autonomous vehicle, determining that the acquired driving environment data associated with the at least a portion of the current external environment of the autonomous vehicle for which map data is not available to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

24. A method of indicating sensor blind spots for an autonomous vehicle, the method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the autonomous vehicle;

determining whether one or more portions of driving environment data is unreliable, the determining including;

determining whether one or more environment sensors is off or deactivated;

responsive to determining that one or more environment sensors is off or deactivated, determining that the portion of the driving environment data associated with the one or more environment sensors that are off or deactivated to be unreliable; and

responsive to determining that one or more portions of the driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the driving environment data that is determined to be unreliable.

25. A sensor blind spot indication system for an autonomous vehicle, the system comprising:

one or more sensors configured to acquire driving environment data of at least a portion of an external environment of the autonomous vehicle; and

a processor operatively connected to the one or more sensors, the processor being programmed to initiate executable operations comprising;

determining whether one or more portions of driving environment data is unreliable, the determining including;

determining whether one or more environment sensors is off or deactivated; and

responsive to determining that one or more environment sensors is off or deactivated, determining that the portion of the driving environment data associated with the one or more environment sensors that are off or deactivated to be unreliable; and

responsive to determining that one or more portions of the driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the driving environment data that is determined to be unreliable.

26. A sensor blind spot indication system for an autonomous vehicle, the system comprising:

one or more sensors configured to acquire driving environment data of at least a portion of an external environment of the autonomous vehicle; and

a processor operatively connected to the one or more sensors, the processor being programmed to initiate executable operations comprising;

determining whether one or more portions of acquired driving environment data is unreliable, the determining including determining whether sensor data is completely occluded in one or more portions of an external environment of the autonomous vehicle;

responsive to determining that sensor data is completely occluded in one or more portions of the external environment, causing the autonomous vehicle to transition to a manual operational mode or safety operational mode; and

causing an alert to be presented within the autonomous vehicle, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.