User interface for displaying internal state of autonomous driving system

• US 8,738,213 B1
• Filed: 02/28/2013
• Issued: 05/27/2014
• Est. Priority Date: 04/28/2010
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Szybalski, Andrew, Prada Gomez, Luis Ricardo, Nemec, Philip, Urmson, Christopher Paul, Thrun, Sebastian
• Primary Examiner(s): BROWN, DREW J
• Application Number:
  US13/781,173
• Time in Patent Office: 453 Days
• Field of Search: 701/1, 701 23- 25, 701/27, 701/28, 701/41, 701/44, 701408-410, 701/301, 701/302
• US Class Current: 701/28
• CPC Class Codes: B60K 2360/16 Type of output information
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2552/00 Input parameters relating to infrastructure
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2555/60 Traffic rules, e.g. speed limits or right of way
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 2720/106 Longitudinal acceleration
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G01C 21/3676 Overview of the route on the road map
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle

Abstract

Autonomous vehicles use various computing systems to transport passengers from one location to another. A control computer sends messages to the various systems of the vehicle in order to maneuver the vehicle safely to the destination. The control computer may display information on an electronic display in order to allow the passenger to understand what actions the vehicle may be taking in the immediate future. Various icons and images may be used to provide this information to the passenger.

Claims

1. A method comprising:

receiving location information of a vehicle from a geographic position component;

accessing stored map information including information indicating locations and orientations of a first lane and a second lane;

displaying an object indicating the location of the vehicle in the first lane based on the received location information and the stored map information;

determining, by a processor of the vehicle, that the vehicle will change from the first lane to the second lane;

prior to the vehicle physically changing from the first lane to the second lane, displaying on a display device of the vehicle a turning zone indicating a projected path of the vehicle; and

when the processor determines that the vehicle will change from the first lane to the second lane, displaying on the display device of the vehicle a changing lane icon indicating that the vehicle is going to change lanes.

2. The method of claim 1, wherein the changing lane icon indicates that the vehicle is going to change from the first lane to the second lane.

3. The method of claim 1, wherein the changing lane icon is a turn signal icon.

4. The method of claim 1, wherein the changing lane icon identifies an area of the second lane that the vehicle is going to move into.

5. The method of claim 1, wherein the turning zone is displayed in a first way to indicate that the vehicle is waiting until it is safe to move from first lane to the second lane.

6. The method of claim 5, further comprising:

changing the vehicle from the first lane to the second lane; and

after changing the vehicle from the first lane to the second lane, no longer displaying the turning zone on the display device.

7. The method of claim 1, further comprising:

receiving destination information from a user input device of the vehicle;

identifying a route to a destination based on the received location information and destination information;

before the processor determines that the vehicle will change from the first lane to the second lane, displaying a first location of the route in the first lane;

changing the vehicle from the first lane to the second lane; and

after changing the vehicle from the first lane to the second lane, adjusting the location of the route to the second lane.

8. The method of claim 1, further comprising continuing to display the changing lane icon until the vehicle has completely changed from the first lane to the second lane.

9. The method of claim 7, further comprising, after the vehicle has completely changed from the first lane to the second lane, fading the changing lane icon.

10. The device of claim 1, wherein the processor is further configured to continue to display the changing lane icon until the vehicle has completely changed from the first lane to the second lane.

11. The method of claim 10, wherein the processor is further configured to after the vehicle has completely changed from the first lane to the second lane, fade the changing lane icon.

12. A device comprising:

memory storing map information including information indicating locations and orientations of a first lane and a second lane;

a display device; and

a processor configured to;

access the memory;

receive location information of a vehicle from a geographic position component;

display on the display device an object indicating the location of a vehicle in the first lane based on the received location information and the stored map information;

determine that the vehicle will change from the first lane to the second lane;

prior to the vehicle physically changing from the first lane to the second lane, display on a display device of the vehicle a turning zone indicating a projected path of the vehicle; and

when the processor determines that the vehicle will change from the first lane to the second lane, display on the display device a changing lane icon indicating that the vehicle is going to change lanes.

13. The device of claim 12, wherein the changing lane icon indicates that the vehicle is going to change to the second lane.

14. The device of claim 12, wherein the changing lane icon is a turn signal icon.

15. The device of claim 12, wherein the changing lane icon identifies an area of the second lane that the vehicle is going move into.

16. The device of claim 12, wherein the turning zone is displayed in a first way to indicate that the vehicle is waiting until it is safe to move from the first lane to the second lane.

17. The device of claim 16, wherein the processor is further configured to:

change the vehicle from the first lane to the second lane; and

after changing the vehicle from the first lane to the second lane, no longer display the turning zone on the display device.

18. The device of claim 12, further comprising a user input device and wherein the processor is further configured to:

receive destination information from the user input device of the vehicle;

identify a route to the destination based on the received location information and destination information;

before the processor determines that the vehicle will change from the first lane to the second lane, display a first location of the route in the first lane;

change the vehicle from the first lane to the second lane; and

after changing the vehicle from the first lane to the second lane, adjust the location of the route to the second lane.

19. A non-transitory, tangible computer-readable storage medium on which computer readable instructions of a program are stored, the instructions, when executed by a processor, cause the processor to perform a method, the method comprising:

receiving location information of a vehicle from a geographic position component;

accessing stored map information including information indicating locations and orientations of a first lane and a second lane;

displaying an object indicating the location of the vehicle in the first lane based on the received location information and the stored map information;

determining that the vehicle will change from the first lane to the second lane;

prior to the vehicle physically changing from the first lane to the second lane, displaying on a display device of the vehicle a turning zone indicating a projected path of the vehicle; and

when the processor determines that the vehicle will change from the first lane to the second lane, displaying on the display device of the vehicle a changing lane icon indicating that the vehicle is going to change lanes.