User interface for displaying internal state of autonomous driving system

• US 9,134,729 B1
• Filed: 07/29/2014
• Issued: 09/15/2015
• Est. Priority Date: 04/28/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Szybalski, Andrew Timothy, Prada Gomez, Luis Ricardo, Urmson, Christopher Paul, Thrun, Sebastian, Nemec, Philip
• Primary Examiner(s): BROWN, DREW J
• Application Number:
  US14/445,493
• Time in Patent Office: 413 Days
• Field of Search: 701/1, 701 23- 25, 701/27, 701/28, 701/41, 701/44, 701408-410, 701/301, 701/302
• US Class Current: 1/1
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 10/30 including control of auxiliary equipment, e.g. air-conditioning compressors or oil pumps
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B62D 1/286 Systems for interrupting non-mechanical steering due to driver intervention
• CPC Class Codes: B62D 15/025 Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation B62D1/28 takes precedence
• CPC Class Codes: G01S 19/10 providing dedicated supplementary positioning signals
• CPC Class Codes: G05D 1/0061 for transition from automatic pilot to manual pilot and vice versa
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials

Abstract

A passenger in an automated vehicle may relinquish control of the vehicle to a control computer when the control computer has determined that it may maneuver the vehicle safely to a destination. The passenger may relinquish or regain control of the vehicle by applying different degrees of pressure, for example, on a steering wheel of the vehicle. The control computer may convey status information to a passenger in a variety of ways including by illuminating elements of the vehicle. The color and location of the illumination may indicate the status of the control computer, for example, whether the control computer has been armed, is ready to take control of the vehicle, or is currently controlling the vehicle.

Claims

1. A method comprising:

identifying, by one or more processors, that a user is ready to relinquish control of a vehicle to a control computer;

in response to the identifying, illuminating, by the one or more processors, a portion of the vehicle in a first color to indicate that the control computer is engaged and ready to control the vehicle in an autonomous driving mode; and

when the control computer begins to control the vehicle in the autonomous driving mode, illuminating, by the one or more processors, the portion of the vehicle in a second color, different from the first color, to indicate that the control computer is controlling the vehicle in the autonomous driving mode.

2. The method of claim 1, wherein identifying that the user is ready to relinquish control of a vehicle to the control computer includes:

receiving a first input at a first user input apparatus of the vehicle;

receiving a second input at a second user input apparatus of the vehicle, the second user input apparatus being different from the first user input apparatus, and

the portion of the vehicle is illuminated in the first color when the first input and the second input are received in conjunction with one another.

3. The method of claim 1, wherein identifying that the user is ready to relinquish control of a vehicle to the control computer includes receiving first input at a first user input apparatus of the vehicle, and the method further comprises:

receiving second input at a second user input apparatus of the vehicle prior to receiving the first input, the second user input apparatus being different from the first user input apparatus; and

in response to the second input, providing a notification requesting a user to complete the first input;

wherein the first input is received after the notification is provided.

4. The method of claim 3, wherein identifying that the user is ready to relinquish control of a vehicle to the control computer further includes receiving the second input.

5. The method of claim 3, further comprising, in response to the second input, illuminating, by the one or more processors, the portion of the vehicle in the first color.

6. The method of claim 1, wherein identifying that the user is ready to relinquish control of a vehicle to the control computer includes receiving an input at a user input apparatus of the vehicle.

7. The method of claim 1, further comprising:

in response to the identifying, determining whether the control computer is able to control the vehicle safely;

when the control computer is not able to control the vehicle safely, not illuminating the portion of the vehicle in the first color to indicate that the control computer is engaged and ready to control the vehicle in an autonomous driving mode.

8. The method of claim 1, further comprising:

determining whether the control computer is ready to control the vehicle in the autonomous driving mode,

wherein the portion of the vehicle is illuminated in the first color only when the control computer is ready to control the vehicle in an autonomous driving mode.

9. A system comprising one or more processors configured to:

identify that a user is ready to relinquish control of a vehicle to a control computer;

in response to the identifying, illuminate a portion of the vehicle in a first color to indicate that the control computer is engaged and ready to control the vehicle in an autonomous driving mode; and

when the control computer begins to control the vehicle in the autonomous driving mode, illuminate the portion of the vehicle in a second color, different from the first color, to indicate that the control computer is controlling the vehicle in the autonomous driving mode.

10. The system of claim 9, wherein the one or more processors are configured to:

identify that the user is ready to relinquish control of a vehicle to the control computer by;

receiving a first input at a first user input apparatus of the vehicle, and

receiving a second input at a second user input apparatus of the vehicle, the second user input apparatus being different from the first user input apparatus; and

illuminate the portion of the vehicle in the first color when the first input and the second input are received in conjunction with one another.

11. The system of claim 9, wherein identifying that the user is ready to relinquish control of a vehicle to the control computer includes receiving first input at a first user input apparatus of the vehicle, and the one or more processors are configured to:

receive second input at a second user input apparatus of the vehicle prior to receiving the first input, the second user input apparatus being different from the first user input apparatus; and

in response to the second input, provide a notification requesting a user to complete the first input such that the first input is received after the notification is provided.

12. The system of claim 11, wherein identifying that the user is ready to relinquish control of a vehicle to the control computer further includes receiving the second input.

13. The system of claim 11, the one or more processors are configured to, in response to the second input, illuminate, by the one or more processors, the portion of the vehicle in the first color.

14. The system of claim 9, wherein identifying that the user is ready to relinquish control of a vehicle to the control computer includes receiving an input at a user input apparatus of the vehicle.

15. The system of claim 9, wherein the one or more processors are further configured to:

in response to the input, determine whether the control computer is able to control the vehicle safely;

when the control computer is not able to control the vehicle safely, not illuminating the portion of the vehicle in the first color to indicate that the control computer is engaged and ready to control the vehicle in an autonomous driving mode.

16. The system of claim 9, wherein the one or more processors are further configured to:

determine whether the control computer is ready to control the vehicle in the autonomous driving mode,

wherein the portion of the vehicle is illuminated in the first color only when the control computer is ready to control the vehicle in an autonomous driving mode.

17. The system of claim 9, further comprising the vehicle.

18. A non-transitory computer-readable storage medium on which computer readable instructions of a program are stored, the instructions, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:

identifying that a user is ready to relinquish control of a vehicle to a control computer;

in response to the identifying, illuminating a portion of the vehicle in a first color to indicate that the control computer is engaged and ready to control the vehicle in an autonomous driving mode; and

when the control computer begins to control the vehicle in the autonomous driving mode, illuminating the portion of the vehicle in a second color, different from the first color, to indicate that the control computer is controlling the vehicle in the autonomous driving mode.

19. The medium of claim 18, wherein identifying that the user is ready to relinquish control of a vehicle to the control computer includes:

receiving a first input at a first user input apparatus of the vehicle;

receiving a second input at a second user input apparatus of the vehicle, the second user input apparatus being different from the first user input apparatus, and

the portion of the vehicle is illuminated in the first color when the first input and the second input are received in conjunction with one another.

20. The medium of claim 18, wherein identifying that the user is ready to relinquish control of a vehicle to the control computer includes receiving a first input at a first user input apparatus of the vehicle, and the method further comprises:

receiving a second input at a second user input apparatus of the vehicle prior to receiving the first input, the second user input apparatus being different from the first user input apparatus; and

in response to the second input, providing a notification requesting a user to complete the first input;

wherein the first input is received after the notification is provided.