User interface for displaying internal state of autonomous driving system

• US 8,818,610 B1
• Filed: 02/28/2013
• Issued: 08/26/2014
• Est. Priority Date: 04/28/2010
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Szybalski, Andrew, Prada Gomez, Luis Ricardo, Nemec, Philip, Urmson, Christopher Paul, Thrun, Sebastian
• Primary Examiner(s): BROWN, DREW J
• Application Number:
  US13/781,113
• Time in Patent Office: 544 Days
• Field of Search: 701/1, 701 23- 25, 701/27, 701/28, 701/41, 701/44, 701408-410, 701/301, 701/302
• US Class Current: 701/28
• CPC Class Codes: B60K 2360/16 Type of output information
• CPC Class Codes: B60K 2370/16 Type of information
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2552/00 Input parameters relating to infrastructure
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2555/60 Traffic rules, e.g. speed limits or right of way
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 2720/106 Longitudinal acceleration
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G01C 21/3676 Overview of the route on the road map
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle

Abstract

Autonomous vehicles use various computing systems to transport passengers from one location to another. A control computer sends messages to the various systems of the vehicle in order to maneuver the vehicle safely to the destination. The control computer may display information on an electronic display in order to allow the passenger to understand what actions the vehicle may be taking in the immediate future. Various icons and images may be used to provide this information to the passenger.

Claims

1. A method comprising:

receiving destination information from a user input device about a destination for a vehicle;

receiving location information of the vehicle from a geographic position component;

identifying a route to the destination based on the received location information;

accessing stored map information;

displaying on a display device of the vehicle the stored map information based on the received location information;

displaying along with the stored map information a path of the vehicle based on the route;

determining, by a processor of the vehicle, that in order to follow the route safely, the vehicle must wait before proceeding along the route; and

when the processor determines that the vehicle must wait, displaying a portion of the path in a first way to indicate to a user that the vehicle is waiting.

2. The method of claim 1, further comprising:

determining by the processor that the vehicle no longer needs to wait; and

when it is determined that the vehicle no longer needs to wait, displaying on the display device the portion of path in a second way, different from the first way, to indicate to the user that the vehicle is no longer waiting.

3. The method of claim 2, wherein the second way includes displaying the portion of the path as a solid line.

4. The method of claim 1, further comprising, when displaying the portion of the path in the first way, displaying on the display device text indicating that the vehicle is waiting.

5. The method of claim 1, wherein the first way includes displaying portion of the path as striped.

6. The method of claim 1, wherein determining by the processor that the vehicle must wait includes:

receiving data from an object detection apparatus indicating the object in a roadway; and

determining that in order to follow the route safely, the vehicle must wait for an action to be completed by the object.

7. The method of claim 6, wherein the action includes the object moving through an intersection.

8. The method of claim 6, wherein the action includes the object making a turn.

9. The method of claim 6, wherein the action includes the object passing the vehicle.

10. The method of claim 1, further comprising, when displaying the portion of the path in the first way, displaying on the display device a ticking clock indicating that the vehicle is waiting.

12. The device of claim 1, wherein the processor is further configured to:

determine by the processor that the vehicle no longer needs to wait; and

when it is determined that the vehicle no longer needs to wait, display the portion of path in a second way, different from the first way, to indicate to the user that the vehicle is no longer waiting.

13. The device of claim 12, wherein the second way includes displaying the portion of the path as a solid line.

14. The device of claim 1, wherein the processor is further configured to, when displaying the portion of the path in the first way, display on the display device text indicating that the vehicle is waiting.

16. The device of claim 1, wherein the processor is further configured to determine that the vehicle must wait by:

receiving data from an object detection apparatus indicating an object in a roadway; and

determining that in order to follow the route safely, the vehicle must wait for an action to be completed by the object.

17. The device of claim 16, wherein the action includes the object moving through an intersection.

18. The device of claim 16, wherein the action includes the object making a turn.

19. The device of claim 16, wherein the action includes the object passing the vehicle.

11. A device comprising:

memory storing map information;

a display device; and

a processor configured to;

access the memory;

receive destination information from a user input device about a destination for a vehicle;

receive location information of the vehicle from a geographic position component;

identify a route to the destination based on the received location information;

display on the display the stored map information based on the received location information;

display along with the stored map information a path of the vehicle based on the route;

determine that in order to follow the route safely, the vehicle must wait before proceeding along the route;

when the processor determines that the vehicle must wait, display on the display device a portion of the path in a first way order to indicate to a user that the vehicle is waiting.

15. The device of claim 11, wherein the first way includes displaying the portion of the path as striped.

20. The device of claim 11, wherein the processor is further configured to, when displaying the portion of the path in the first way, display on the display device a ticking clock indicating that the vehicle is waiting.

21. A non-transitory, tangible computer-readable storage medium on which computer readable instructions of a program are stored, the instructions, when executed by a processor, cause the processor to perform a method, the method comprising:

receiving destination information from a user input device about a destination for a vehicle;

receiving location information of the vehicle from a geographic position component;

identifying a route to the destination based on the received location information;

accessing stored map information;

displaying on a display device of the vehicle the stored map information based on the received location information;

displaying along with the stored map information a path of the vehicle based on the route;

determining that in order to follow the route safely, the vehicle must wait before proceeding along the route; and

when the processor determines that the vehicle must wait, displaying a portion of the path in a first way to indicate to a user that the vehicle is waiting.