User interface for displaying internal state of autonomous driving system

• US 10,082,789 B1
• Filed: 11/04/2016
• Issued: 09/25/2018
• Est. Priority Date: 04/28/2010
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Szybalski, Andrew Timothy, Prada Gomez, Luis Ricardo, Urmson, Christopher Paul, Thrun, Sebastian, Nemec, Philip
• Primary Examiner(s): Brown, Drew J
• Application Number:
  US15/344,007
• Time in Patent Office: 690 Days
• Field of Search: 701 11, 701 23, 701 24, 701 36, 701 41, 701 70
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 10/30 including control of auxiliary equipment, e.g. air-conditioning compressors or oil pumps
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B62D 1/286 Systems for interrupting non-mechanical steering due to driver intervention
• CPC Class Codes: B62D 15/025 Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation B62D1/28 takes precedence
• CPC Class Codes: G01S 19/10 providing dedicated supplementary positioning signals
• CPC Class Codes: G05D 1/0061 for transition from automatic pilot to manual pilot and vice versa
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials

Abstract

A passenger in an automated vehicle may relinquish control of the vehicle to a control computer when the control computer has determined that it may maneuver the vehicle safely to a destination. The passenger may relinquish or regain control of the vehicle by applying different degrees of pressure, for example, on a steering wheel of the vehicle. The control computer may convey status information to a passenger in a variety of ways including by illuminating elements of the vehicle. The color and location of the illumination may indicate the status of the control computer, for example, whether the control computer has been armed, is ready to take control of the vehicle, or is currently controlling the vehicle.

Claims

1. A method of controlling a vehicle, the method comprising:

determining, by one or more processors of a control computer, a status of the control computer, the control computer being configured to control the vehicle in an autonomous driving mode;

based on the status, selecting one of a plurality of predetermined visual status configurations, each of the plurality of predetermined visual status configurations includes at least one visual indication that indicates whether the control computer is ready to control the vehicle in the autonomous driving mode; and

displaying, by the one or more processors, the selected one to indicate the status,

wherein the visual indication of the selected one includes a color indication, such that displaying the selected one includes illuminating a portion of a display in a color of the color indication.

2. The method of claim 1, wherein determining a status of a control computer includes determining whether the control computer is able to control aspects of a vehicle in an autonomous driving mode.

3. The method of claim 2, further comprising receiving sensor information identifying a number of objects identified in the environment of the vehicle, and wherein the determining is based on a number of objects identified in the environment of the vehicle.

4. The method of claim 3, wherein the determining is based on a current geographic location of the vehicle.

5. The method of claim 4, wherein the determining is further based on map information for the current geographic location of the vehicle.

6. The method of claim 1, further comprising:

maneuvering, by the one or more processors, the control computer in the autonomous driving mode;

as the vehicle is maneuvered, re-evaluating the status of the control computer configured to control the vehicle in an autonomous driving mode; and

updating the displayed selected one based on the re-evaluated status.

7. The method of claim 1, further comprising, receiving, at a user input apparatus, input from an occupant of the vehicle, and wherein selecting one of the plurality of predetermined visual status configurations is further based on the input.

8. The method of claim 7, wherein the visual indication of another selected one indicates that there is no input at the user input apparatus.

9. The method of claim 1, wherein the visual indication of another selected one indicates that the control computer is armed and able to control the vehicle in the autonomous driving mode.

10. The method of claim 1, wherein the visual indication of another selected one indicates that the control computer is armed and the control computer is not able to control the vehicle in the autonomous driving mode.

11. The method of claim 10, wherein the visual indication of the another selected one includes a visual notification that an occupant of the vehicle needs to take control of a control aspect of the vehicle.

12. The method of claim 1, wherein the visual indication of another selected one indicates that the control computer is disarmed.

13. The method of claim 1, further comprising illuminating the display gradually in the color of the color indication.

14. A system for controlling a vehicle, the system comprising:

a control computer configured to control the vehicle in an autonomous driving mode, the control computer having one or more processors configured to;

determine a status of the control computer;

based on the status, select one of a plurality of predetermined visual status configurations, each of the plurality of predetermined visual status configurations includes at least one visual indication that indicates whether the control computer is ready to control the vehicle in the autonomous driving mode; and

display the selected one to indicate the status,

wherein the visual indication of the selected one includes a color indication, such that displaying the selected one includes illuminating a portion of a display in a color of the color indication.

15. The system of claim 14, further comprising the vehicle.

16. The system of claim 14, wherein the visual indication of another selected one indicates that the control computer is armed and able to control the vehicle in the autonomous driving mode.

17. The system of claim 14, wherein the visual indication of another selected one indicates that the control computer is armed and the control computer is not able to control the vehicle in the autonomous driving mode.

18. The system of claim 17, wherein the visual indication of the another selected one indicates that there is no input at at user input apparatus.

19. The system of claim 17, wherein the visual indication of the another selected one includes a visual notification that an occupant of the vehicle needs to take control of a control aspect of the vehicle.

20. The system of claim 14, wherein the visual indication of another selected one indicates that the control computer is disarmed.