User interface for displaying internal state of autonomous driving system

• US 8,352,110 B1
• Filed: 08/06/2012
• Issued: 01/08/2013
• Est. Priority Date: 04/28/2010
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Szybalski, Andrew, Prada Gomez, Luis Ricardo, Urmson, Christopher Paul, Thrun, Sebastian, Nemec, Philip
• Primary Examiner(s): BROWN, DREW J
• Application Number:
  US13/567,428
• Time in Patent Office: 155 Days
• Field of Search: 701/11, 701/23, 701/24, 701/36, 701/41, 701/70
• US Class Current: 701/23
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 10/30 including control of auxiliary equipment, e.g. air-conditioning compressors or oil pumps
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B62D 1/286 Systems for interrupting non-mechanical steering due to driver intervention
• CPC Class Codes: B62D 15/025 Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation B62D1/28 takes precedence
• CPC Class Codes: G01S 19/10 providing dedicated supplementary positioning signals
• CPC Class Codes: G05D 1/0061 for transition from automatic pilot to manual pilot and vice versa
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials

Abstract

A passenger in an automated vehicle may relinquish control of the vehicle to a control computer when the control computer has determined that it may maneuver the vehicle safely to a destination. The passenger may relinquish or regain control of the vehicle by applying different degrees of pressure, for example, on a steering wheel of the vehicle. The control computer may convey status information to a passenger in a variety of ways including by illuminating elements of the vehicle. The color and location of the illumination may indicate the status of the control computer, for example, whether the control computer has been armed, is ready to take control of the vehicle, or is currently controlling the vehicle.

Claims

1. A method comprising:

receiving input at a user input apparatus indicating that a user is ready to relinquish an aspect of control of a vehicle to a control computer, the aspect of control including one of control of braking, control of acceleration, and control of steering;

in response to the input, determining whether the control computer can control the aspect safely;

when the control computer can control the aspect safely, providing a notification to the user that the control computer is engaged and ready to control the aspect;

receiving a second input from a force input apparatus of the vehicle indicating that the user has released the force input apparatus of the vehicle; and

controlling, by the control computer, the aspect, in response to receiving the second input.

2. The method of claim 1, wherein controlling the aspect includes moving the vehicle to a new location, and the method further comprises:

determining that the control computer cannot control the aspect safely at the new location; and

after determining that the control computer cannot control the aspect safely at the new location, providing a notification to the user indicating a need to take control of the aspect.

3. The method of claim 2, wherein it is determined that the control vehicle cannot control the aspect safely when the vehicle is at a geographic location that the control computer cannot identify.

4. The method of claim 2, wherein:

the control computer uses map information to control the aspect; and

it is determined that the control computer cannot control the aspect safely when the vehicle is at a geographic location which is not sufficiently detailed in the map information.

5. The method of claim 2, wherein it is determined that the control vehicle cannot control the aspect safely when the control computer detects a plurality of obstacles proximate to the vehicle.

6. The method of claim 1, further comprising:

while the control computer is controlling the aspect, receiving, from the force input apparatus, third input; and

determining whether the user intended to take control of the aspect based on the third input.

7. The method of claim 6, wherein determining whether the user intended to take control of the aspect is based on a location of the third input relative to the force input apparatus.

8. The method of claim 6, wherein determining whether the user intended to take control of the aspect is based on a length of time associated the third input.

9. The method of claim 6, wherein determining whether the user intended to take control of the aspect is based on whether the third input meets a pressure threshold.

10. The method of claim 6, wherein determining whether the user intended to take control of the aspect is based on whether the third input indicates that two hands are touching the steering wheel.

11. The method of claim 1, further comprising:

receiving a third input from a second force input apparatus coupled to a foot pedal of the vehicle; and

determining whether the user intended to take control of the aspect based on the third input.

12. The method of claim 1, wherein the force input apparatus is associated with at least one of a brake pedal, acceleration pedal, or steering wheel.

13. A method comprising:

receiving input at a user input apparatus indicating that a user is ready to relinquish control of a vehicle to a control computer;

in response to receiving the input, determining whether the control computer can control the vehicle safely;

upon determining that the control computer can control the aspect safely, providing a notification to the user that the control computer is engaged and ready to control the aspect;

receiving a second input from a force input apparatus of the vehicle indicating that the user has released the force input apparatus of the vehicle; and

controlling the vehicle by the control computer in response to receiving the second input.

14. The method of claim 13, wherein the force input apparatus is associated with at least one of a brake pedal, acceleration pedal, or steering wheel.

15. The method of claim 13, wherein controlling the vehicle includes controlling steering, acceleration, and deceleration functions of the vehicle without input from the driver.

16. The method of claim 13, further comprising:

while the control computer is controlling the aspect, receiving, from the force input apparatus, third input; and

determining whether the user intended to take control of the vehicle based on the third input.

17. The method of claim 16, wherein determining whether the user intended to take control of the vehicle is based on a location of the third input relative to the force input apparatus.

18. The method of claim 16, wherein determining whether the user intended to take control of the aspect is based on a length of time associated the third input.

19. The method of claim 16, wherein determining whether the user intended to take control of the aspect is based on whether the third input meets a pressure threshold.

20. The method of claim 16, wherein determining whether the user intended to take control of the aspect is based on whether the third input indicates that two hands are touching the steering wheel.

21. The method of claim 16, further comprising:

receiving a third input, from a second force input apparatus coupled to a foot pedal of the vehicle, third input; and

determining whether the user intended to take control of the aspect based on the third input.

22. A system comprising:

a control computer for controlling a vehicle in an autonomous mode without continuous input from a driver, wherein the control computer is configured to;

receive input at a user input apparatus indicating that a user is ready to relinquish control of a vehicle to a control computer;

after receiving the input, determine whether the control computer can control the aspect safely;

upon determining that the control computer can control the aspect safely, providing a notification to the user that the control computer is engaged and ready to control the aspect;

receive a second input from a force input apparatus of the vehicle indicating that the user has released a steering wheel of the vehicle; and

when the control computer can control the vehicle safely and the second input is received, control the vehicle in the autonomous mode.