Systems and methods to obtain feedback in response to autonomous vehicle failure events

• US 10,559,140 B2
• Filed: 06/16/2017
• Issued: 02/11/2020
• Est. Priority Date: 06/16/2017
• Status: Active Grant

• Current Assignee: Aurora Operations Incorporated (Aurora Innovation, Inc.)
• Original Assignee: Uatc, LLC (Uber Technologies, Inc.)
• Inventors: Nix, Molly Castle, Chin, Sean, Zhao, Dennis
• Primary Examiner(s): Mancho, Ronnie M
• Application Number:
  US15/625,255
• Publication Number:
  US 20180365913A1
• Time in Patent Office: 970 Days
• Field of Search: 701 23
• CPC Class Codes: B60K 2360/115 Selection of menu items
• CPC Class Codes: B60K 2360/119 Icons
• CPC Class Codes: B60K 2360/1438 Touch screens
• CPC Class Codes: B60K 2360/175 Autonomous driving
• CPC Class Codes: B60K 2360/186 Displaying information according to relevancy
• CPC Class Codes: B60K 2360/566 Mobile devices displaying vehicle information
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/20 Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60K 35/80 Arrangements for controlling instruments
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2540/215 Selection or confirmation of options
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01C 21/3697 Output of additional, non-guidance related information, e.g. low fuel level G01C21/3679 takes precedence
• CPC Class Codes: G05D 1/021 specially adapted to land vehicles
• CPC Class Codes: G05D 1/69 Coordinated control of the position or course of two or more vehicles
• CPC Class Codes: G07C 5/0825 using optical means
• CPC Class Codes: G07C 5/0841 : Registering performance data recording measured values G01D; information storage G11B

Abstract

The present disclosure provides systems and methods to obtain feedback descriptive of autonomous vehicle failures. In particular, the systems and methods of the present disclosure can detect that a vehicle failure event occurred at an autonomous vehicle and, in response, provide an interactive user interface that enables a human located within the autonomous vehicle to enter feedback that describes the vehicle failure event. Thus, the systems and methods of the present disclosure can actively prompt and/or enable entry of feedback in response to a particular instance of a vehicle failure event, thereby enabling improved and streamlined collection of information about autonomous vehicle failures.

Claims

1. A computing system, comprising:

one or more processors; and

one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising;

determining that a vehicle failure event occurred at an autonomous vehicle;

in response to determining that the vehicle failure event occurred, providing an interactive user interface that enables a human located within the autonomous vehicle to enter feedback that describes the vehicle failure event, wherein providing the interactive user interface comprises providing the interactive user interface that provides a plurality of user-selectable icons that respectively correspond to a plurality of different types of vehicle failure events, each user-selectable icon selectable by the human to provide the feedback indicative of the corresponding type of vehicle failure event;

receiving the feedback entered by the human located within the autonomous vehicle via the interactive user interface; and

associating the feedback with one or more of;

a time at which the vehicle failure event occurred, a location at which the vehicle failure event occurred, and data collected by the autonomous vehicle contemporaneous to the vehicle failure event.

2. The computing system of claim 1, further comprising:

a front-seat display device that is accessible by both a passenger that is located in a front passenger seat of the autonomous vehicle and a driver that is located in a driver seat of the autonomous vehicle;

wherein providing the interactive user interface comprises displaying the interactive user interface on the front-seat display.

3. The computing system of claim 1, wherein providing the interactive user interface comprises wirelessly providing the interactive user interface to a handheld computing device associated with the human for display by the handheld computing device.

4. The computing system of claim 1, wherein the computing system comprises an autonomy system and a human machine interface device, wherein one or both of the autonomy system and the human machine interface device perform the operation of determining that the vehicle failure event occurred at the autonomous vehicle.

5. The computing system of claim 1, wherein the computing system comprises a central computing system that is configured to perform additional operations, the additional operations comprising:

aggregating multiple instances of triage data received via the interactive user interface relative to multiple detected vehicle failure events over time; and

determining an aggregate performance score for at least the autonomous vehicle relative to at least one physical location based at least in part on the aggregated multiple instances of triage data.

6. A computer-implemented method to obtain feedback descriptive of autonomous vehicles failures, the method comprising:

determining, by a computing system comprising one or more computing devices, that a vehicle failure event occurred at an autonomous vehicle;

in response to determining that the vehicle failure event occurred, providing, by the computing system, an interactive user interface that enables a human located within the autonomous vehicle to enter feedback that describes the vehicle failure event, wherein providing, by the computing system, the interactive user interface comprises providing, by the computing system, the interactive user interface that provides a plurality of user-selectable icons that respectively correspond to a plurality of different types of vehicle failure events, each user-selectable icon selectable by the human to provide the feedback indicative of the corresponding type of vehicle failure event;

receiving, by the computing system, the feedback entered by the human located within the autonomous vehicle via the interactive user interface; and

associating, by the computing system, the feedback with one or more of;

a time at which the vehicle failure event occurred, a location at which the vehicle failure event occurred, and data collected by the autonomous vehicle contemporaneous to the vehicle failure event.

7. The computer-implemented method of claim 6, wherein providing, by the computing system, the interactive user interface comprises providing, by the computing system, the interactive user interface that includes a plurality of nested hierarchical menus.

8. The computer-implemented method of claim 6, wherein:

determining, by the computing system, that the vehicle failure event occurred comprises detecting, by the computing system, that a human intervention event in which a human driver assumed control of the autonomous vehicle occurred at the autonomous vehicle; and

providing, by the computing system, the interactive user interface comprises automatically providing, by the computing system, the interactive user interface in response to detecting that the human intervention event occurred.

9. The computer-implemented method of claim 6, wherein:

determining, by the computing system, that the vehicle failure event occurred comprises receiving, by the computing system, a human input from the human requesting to enter the feedback; and

providing, by the computing system, the interactive user interface comprises transitioning, by the computing system, the interactive user interface to a feedback menu in response to receiving the human input requesting to enter the feedback.

10. The computer-implemented method of claim 9, wherein receiving, by the computing system, the human input from the human requesting to enter the feedback comprises receiving, by the computing system, a human selection of a feedback icon presented on the interactive user interface.

11. The computer-implemented method of claim 6, wherein providing, by the computing system, the interactive user interface comprises displaying the interactive user interface on a front-seat display device that is accessible by at least one of a front seat passenger located in a front passenger seat of the autonomous vehicle and a driver located in a driver seat of the autonomous vehicle.

12. The computer-implemented method of claim 6, wherein providing, by the computing system, the interactive user interface comprises:

providing for display, by the computing system, the interactive user interface that initially enables the human to indicate a criticality level of the vehicle failure event; and

receiving, by the computing system, a user input that indicates the criticality level of the vehicle failure event; and

in response to receiving the user input that indicates the criticality level of the vehicle failure event, providing for display, by the computing system, the interactive user interface that provides the plurality of user-selectable icons that respectively correspond to the plurality of different types of vehicle failure events.

13. The computer-implemented method of claim 12, wherein providing for display, by the computing system, the interactive user interface that provides the plurality of user-selectable icons in response to receiving the user input that indicates the criticality level of the vehicle failure event comprises providing for display, by the computing system, the interactive user interface that provides the plurality of user-selectable icons in response to receiving the user input that indicates that the criticality level of the vehicle failure event is greater than a threshold criticality level.

14. The computer-implemented method of claim 6,

further comprising:

in response to receiving the feedback that selects one of the plurality of user-selectable icons, causing, by the computing system, the user interface to provide a secondary menu that provides a second plurality of user-selectable icons that respectively correspond to different descriptions of the selected type of vehicle failure event.

15. The computer-implemented method of claim 6, wherein providing, by the computing system, the interactive user interface comprises providing for display, by the computing system, the interactive user interface that includes both a navigational portion and a feedback portion, wherein the navigational portion comprises at least one of navigational instructions and a map, and wherein the feedback portion enables the human located within the autonomous vehicle to enter the feedback that describes the vehicle failure event.

16. The computer-implemented method of claim 6, further comprising:

automatically dismissing, by the computing system, a feedback menu of the interactive user interface when the feedback is not entered by the user within a predetermined amount of time.

17. The computer-implemented method of claim 6, further comprising:

aggregating, by the computing system, multiple instances of triage data received via the interactive user interface relative to multiple detected vehicle failure events over time; and

determining, by the computing system, an aggregate performance score for at least the autonomous vehicle relative to at least one physical location based at least in part on the aggregated multiple instances of triage data.

18. The computer-implemented method of claim 6, wherein the one or more computing devices are located on-board the autonomous vehicle.

19. One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more processors, cause a computing system to perform operations, the operations comprising:

determining that a vehicle failure event occurred at an autonomous vehicle;

in response to determining that the vehicle failure event occurred, providing an interactive user interface that enables a human located within the autonomous vehicle to enter feedback that describes the vehicle failure event, wherein providing the interactive user interface comprises providing the interactive user interface that provides a plurality of user-selectable icons that respectively correspond to a plurality of different types of vehicle failure events, each user-selectable icon selectable by the human to provide the feedback indicative of the corresponding type of vehicle failure event;

receiving the feedback entered by the human located within the autonomous vehicle via the interactive user interface; and

associating the feedback with at least one of;

a time at which the vehicle failure event occurred, a location at which the vehicle failure event occurred, and data collected by the autonomous vehicle contemporaneous to the vehicle failure event.