Autonomous vehicle operation in view-obstructed environments

• US 9,649,979 B2
• Filed: 01/29/2015
• Issued: 05/16/2017
• Est. Priority Date: 01/29/2015
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Prokhorov, Danil V., Okumura, Bunyo, Sakai, Katsuhiro, Harada, Masahiro, Kamata, Nobuhide, Nagasaka, Naoki
• Primary Examiner(s): Tarcza, Thomas
• Assistant Examiner(s): GOLDMAN, RICHARD A
• Application Number:
  US14/609,273
• Publication Number:
  US 20160221500A1
• Time in Patent Office: 838 Days
• Field of Search: 701 45, 701301, 701300, 701 49, 701 23, 701 28, 701 36, 701431, 701532, 701 96, 340435, 340436, 340903, 340902, 340904, 340438
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2510/101 Transmission neutral state
• CPC Class Codes: B60W 2510/18 Braking system
• CPC Class Codes: B60W 2510/20 Steering systems
• CPC Class Codes: B60W 2520/04 Vehicle stop
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/801 Lateral distance
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 2710/1005 Transmission ratio engaged
• CPC Class Codes: B60W 2710/18 Braking system
• CPC Class Codes: B60W 2710/20 Steering systems
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 40/00 : Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
• CPC Class Codes: B60W 50/0097 : Predicting future conditions
• CPC Class Codes: B60W 50/0098 : Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0018 : by employing degraded modes, e.g. reducing speed, in response to suboptimal conditions
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/59 : inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Arrangements related to operating an autonomous vehicle in view-obstructed environments are described. At least a portion of an external environment of the autonomous vehicle can be sensed to detect one or more objects located therein. An occupant viewable area of the external environment can be determined. It can be determined whether one or more of the detected one or more objects is located outside of the determined occupant viewable area. Responsive to determining that a detected object is located outside of the determined occupant viewable area, one or more actions can be taken. For instance, the action can include presenting an alert within the autonomous vehicle. Alternatively or in addition, the action can include causing a current driving action of the autonomous vehicle to be modified.

Claims

1. A method of operating an autonomous vehicle in a view-obstructed environment comprising:

acquiring, using one or more sensors, sensor data of at least a portion of an external environment of the autonomous vehicle;

detecting one or more objects located in the external environment based on the acquired sensor data;

determining, using one or more processors, an occupant viewable area of the external environment, the occupant viewable area being a portion of the external environment that is visible to a vehicle occupant, the occupant viewable area being determined based at least in part on a field of vision for the occupant and any obstructions located in the external environment;

determining whether one or more of the detected one or more objects is located outside of the determined occupant viewable area by comparing a location of the detected one or more objects to the determined occupant viewable area; and

responsive to determining that one or more of the detected one or more objects is located outside of the determined occupant viewable area, presenting an alert within the autonomous vehicle indicating that one or more of the detected one or more objects is determined to be located outside of the determined occupant viewable area, the alert being presented while the autonomous vehicle continues to operate in a highly automated mode,

wherein, in the highly automated mode, the autonomous vehicle is configured to navigate without receiving input from the vehicle occupant.

2. The method of claim 1, wherein the alert is an audible alert, and wherein presenting the alert within the autonomous vehicle includes emitting the audible alert within the autonomous vehicle.

3. The method of claim 2, wherein the audible alert is emitted in a location within the autonomous vehicle corresponding to the location of the detected one or more objects located outside of the determined occupant viewable area.

4. The method of claim 1, wherein the alert is a visual alert, and wherein presenting the alert within the autonomous vehicle includes displaying the visual alert within the autonomous vehicle.

5. The method of claim 4, wherein the visual alert includes a representation of the external environment, and wherein a representation of the one or more of the detected one or more objects located outside of the occupant viewable area is overlaid upon the representation of the external environment.

6. The method of claim 5, wherein a representation of at least one of a representation of the occupant viewable area and a representation of the sensed at least a portion of an external environment is overlaid upon the representation of the external environment.

7. The method of claim 1, wherein determining an occupant viewable area of the external environment based at least in part on any obstructions located in the external environment is only performed responsive to detecting the one or more objects located in the external environment based on the acquired sensor data.

8. A system for operating an autonomous vehicle in a view-obstructed environment, comprising:

a sensor system configured to acquire sensor data of at least a portion of an external environment of the autonomous vehicle;

a processor operatively connected to the sensor system, the processor being programmed to initiate executable operations comprising;

detecting one or more objects located in the external environment based on the acquired sensor data;

determining an occupant viewable area of the external environment, the occupant viewable area being a portion of the external environment that is visible to a vehicle occupant, the occupant viewable area being determined based at least in part on a field of vision for the vehicle occupant and any obstructions located in the external environment;

determining whether one or more of the detected one or more objects is located outside of the determined occupant viewable area by comparing a location of the detected one or more objects to the determined occupant viewable area; and

responsive to determining that one or more of the detected one or more objects is located outside of the determined occupant viewable area, presenting an alert within the autonomous vehicle indicating that one or more of the detected one or more objects is determined to be located outside of the determined occupant viewable area, the alert being presented while the autonomous vehicle continues to operate in a highly automated mode,

wherein, in the highly automated mode, the autonomous vehicle is configured to navigate without receiving input from the vehicle occupant.

9. The system of claim 8, further including:

a speaker operatively connected to the processor, wherein the alert is an audible alert, and wherein presenting the alert within the autonomous vehicle includes emitting the audible alert within the autonomous vehicle via the speaker.

10. The system of claim 9, wherein emitting the audible alert within the autonomous vehicle includes emitting the audible alert in a location within the autonomous vehicle corresponding to the location of the detected one or more objects located outside of the determined occupant viewable area.

11. The system of claim 8, further including:

a display operatively connected to the processor, wherein the alert is a visual alert, and wherein presenting the alert within the autonomous vehicle includes presenting the visual alert within the autonomous vehicle on the display.

12. The system of claim 11, wherein the visual alert includes a representation of the external environment, and wherein a representation of the one or more of the detected one or more objects located outside of the occupant viewable area is overlaid upon the representation of the external environment.

13. The system of claim 12, wherein at least one of a representation of the occupant viewable area and a representation of the sensed at least a portion of an external environment is overlaid upon the representation of the external environment.

14. A method of operating an autonomous vehicle in a view-obstructed environment comprising:

acquiring, using one or more sensors, sensor data of at least a portion of an external environment of the autonomous vehicle;

detecting one or more objects located in the external environment based on the acquired sensor data;

determining a human occupant physical measurement of at least a portion of a body of a current human occupant of the autonomous vehicle based on acquired data of the current human occupant of the autonomous vehicle;

determining, using one or more processors, an occupant viewable area of the external environment, the occupant viewable area being a portion of the external environment that is visible to the current human occupant of the autonomous vehicle, the occupant viewable area being determined based at least in part on a field of vision for the current human occupant of the autonomous vehicle and the determined human occupant physical measurement;

determining whether one or more of the detected one or more objects is located outside of the determined occupant viewable area by comparing a location of the detected one or more objects to the determined occupant viewable area; and

responsive to determining that one or more of the detected one or more objects is located outside of the determined occupant viewable area, presenting an alert within the autonomous vehicle indicating that one or more of the detected one or more objects is determined to be located outside of the determined occupant viewable area.

15. A system for operating an autonomous vehicle in a view-obstructed environment, comprising:

one or more sensors configured to acquire sensor data of at least a portion of an external environment of the autonomous vehicle;

a processor operatively connected to the one or more sensors and the one or more sensors, the processor being programmed to initiate executable operations comprising;

operating the autonomous vehicle in a highly automated mode, wherein, in the highly automated mode, the autonomous vehicle is configured to navigate without receiving input from a vehicle occupant;

detecting one or more objects located in the external environment based on sensor data acquired by the one or more sensors;

determining a human occupant physical measurement of a current human occupant of the autonomous vehicle;

determining an occupant viewable area of the external environment, the occupant viewable area being a portion of the external environment that is visible to the current human occupant of the autonomous vehicle, the occupant viewable area being determined based at least in part on a field of vision for the current human occupant of the autonomous vehicle and the determined human occupant physical measurement;

determining whether one or more of the detected one or more objects is located outside of the determined occupant viewable area by comparing a location of the detected one or more objects to the determined occupant viewable area; and

responsive to determining that one or more of the detected one or more objects is located outside of the determined occupant viewable area, presenting an alert within the autonomous vehicle indicating that one or more of the detected one or more objects is determined to be located outside of the determined occupant viewable area.