User interface for displaying internal state of autonomous driving system

• US 9,132,840 B1
• Filed: 07/17/2014
• Issued: 09/15/2015
• Est. Priority Date: 04/28/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Szybalski, Andrew Timothy, Prada Gomez, Luis Ricardo, Nemec, Philip, Urmson, Christopher Paul, Thrun, Sebastian
• Primary Examiner(s): BROWN, DREW J
• Application Number:
  US14/333,804
• Time in Patent Office: 425 Days
• Field of Search: 701/1, 701 23- 25, 701/27, 701/28, 701/41, 701/44, 701408-410, 701/301, 701/302
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/16 Type of output information
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2552/00 Input parameters relating to infrastructure
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2555/60 Traffic rules, e.g. speed limits or right of way
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 2720/106 Longitudinal acceleration
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G01C 21/3676 Overview of the route on the road map
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle

Abstract

Autonomous vehicles use various computing systems to transport passengers from one location to another. A control computer sends messages to the various systems of the vehicle in order to maneuver the vehicle safely to the destination. The control computer may display information on an electronic display in order to allow the passenger to understand what actions the vehicle may be taking in the immediate future. Various icons and images may be used to provide this information to the passenger.

Claims

1. A method comprising:

determining, by one or more processors, a current location of a vehicle;

displaying, by the one or more processors, map information including a current lane of the vehicle based on the current location of a vehicle;

displaying on a display device of the vehicle, by the one or more processors, a path corresponding to a route that the vehicle is following;

determining, by the one or more processors, when the vehicle will turn in order to follow the route;

determining, by the one or more processors, when the vehicle must wait before making the turn;

based on the determination of when the vehicle will turn in order to follow the route and the determination of when the vehicle must wait before making the turn, displaying on the display device, by the one or more processors, a visual indicator identifying an area of the map information to indicate that the vehicle will be moving into the area from the current lane and also that the vehicle is waiting to make the turn into the area; and

maneuvering, by the one or more processors, the vehicle by turning the vehicle into a location corresponding to the area of the map information.

2. The method of claim 1, further comprising displaying on the display device a turn signal icon to indicate a direction in which the vehicle will be moving in order to maneuver into the area.

3. The method of claim 2, further comprising, when the vehicle has maneuvered into the area, removing the turn signal from the display device.

4. The method of claim 2, further comprising, when the vehicle has maneuvered into the area, removing the visual indicator from the display device.

5. The method of claim 1, further comprising, when the vehicle must turn, repositioning the path in the area of the map information on the display device.

6. The method of claim 1, wherein the turn includes changing lanes and the visual indicator indicates a second lane into which the vehicle will turn.

7. A system comprising one or more processors configured to:

determine a current location of a vehicle;

display map information including a current lane of the vehicle based on the current location of a vehicle;

display by the one or more processors, a path corresponding to a route that the vehicle is following;

determine when the vehicle will turn in order to follow the route;

determine when the vehicle must wait before making the turn;

based on the determination of when the vehicle will turn in order to follow the route and the determination of when the vehicle must wait before making the turn, display on the display device a visual indicator identifying an area of the map information to indicate that the vehicle will be moving into the area from the current lane and also that the vehicle is waiting to make the turn into the area; and

maneuver the vehicle by turning the vehicle into a location corresponding to the area of the map information.

8. The system of claim 7, wherein the one or more processors are further configured to display on the display device a turn signal icon to indicate a direction in which the vehicle will be moving in order to maneuver into the area.

9. The system of claim 8, wherein the one or more processors are further configured to, when the vehicle has maneuvered into the area, remove the turn signal from the display device.

10. The system of claim 8, wherein the one or more processors are further configured to, when the vehicle has maneuvered into the area, remove the visual indicator from the display device.

11. The system of claim 7, wherein the one or more processors are further configured to, when the vehicle must turn, reposition the path in the area of the map information on the display device.

12. The system of claim 7, wherein the turn includes changing lanes and the visual indicator indicates a second lane into which the vehicle will turn.

13. The system of claim 7, further comprising the vehicle.

14. The system of claim 7, further comprising the display device.

15. A non-transitory, tangible computer-readable storage medium on which computer readable instructions of a program are stored, the instructions, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:

determining a current location of a vehicle;

displaying map information including a current lane of the vehicle based on the current location of a vehicle;

displaying on a display device of the vehicle, by the one or more processors, a path corresponding to a route that the vehicle is following;

determining when the vehicle will turn in order to follow the route;

determining, by the one or more processors, when the vehicle must wait before making the turn;

based on the determination of when the vehicle will turn in order to follow the route and the determination of when the vehicle must wait before making the turn, displaying on the display device a visual indicator identifying an area of the map information to indicate that the vehicle will be moving into the area from the current lane and also that the vehicle is waiting to make the turn into the area; and

maneuvering the vehicle by turning the vehicle into a location corresponding to the area of the map information.

16. The medium of claim 15, wherein the method further comprises displaying on the display device a turn signal icon to indicate a direction in which the vehicle will be moving in order to maneuver into the area.

17. The medium of claim 16, wherein the method further comprises, when the vehicle has maneuvered into the area, removing the turn signal from the display device.

18. The medium of claim 16, wherein the method further comprises, when the vehicle has maneuvered into the area, removing the visual indicator from the display device.