User interface for displaying internal state of autonomous driving system

• US 8,346,426 B1
• Filed: 04/28/2010
• Issued: 01/01/2013
• Est. Priority Date: 04/28/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Szybalski, Andrew, Prada Gomez, Luis Ricardo, Nemec, Philip, Urmson, Christopher, Thrun, Sebastian
• Primary Examiner(s): BROWN, DREW J
• Application Number:
  US12/769,252
• Time in Patent Office: 979 Days
• Field of Search: 701/1, 701 23- 25, 701/27, 701/28, 701/41, 701/44, 701408-410, 701/301, 701/302
• US Class Current: 701/28
• CPC Class Codes: B60K 2360/16 Type of output information
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2552/00 Input parameters relating to infrastructure
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2555/60 Traffic rules, e.g. speed limits or right of way
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 2720/106 Longitudinal acceleration
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G01C 21/3676 Overview of the route on the road map
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle

Abstract

Autonomous vehicles use various computing systems to transport passengers from one location to another. A control computer sends messages to the various systems of the vehicle in order to maneuver the vehicle safely to the destination. The control computer may display information on an electronic display in order to allow the passenger to understand what actions the vehicle may be taking in the immediate future. Various icons and images may be used to provide this information to the passenger.

Claims

1. A vehicle comprising:

a plurality of control apparatuses including a braking apparatus, an acceleration apparatus, and a steering apparatus;

a user input device for inputting destination information;

a geographic position component for determining the current location of the vehicle;

an object detection apparatus for detecting and identifying a type of an object in or proximate to a roadway;

memory for storing a detailed roadway map including roadways, traffic signals, and intersections;

an electronic display for displaying information to a passenger; and

a processor programmed to;

receive the destination information;

identify a route to the destination;

determine, from location information received from the geographic position component and the stored map information, the current geographic location of the vehicle;

identify an object and object type based on object information received from the object detection apparatus;

determine at least one action to be taken including controlling at least one of the control apparatuses based on the identified object, the current geographic location of the vehicle, and the route, wherein the at least one action to be taken includes avoiding a headroom zone in front of the identified object;

select a headroom zone image to be displayed based on the at least one action to be taken and the identified object; and

display the selected headroom zone image on the electronic display.

2. The vehicle of claim 1, wherein the processor is further programmed to:

select a second image to be displayed, wherein the second image to be displayed is an icon representing a second vehicle; and

display the second selected image on the electronic display.

3. The vehicle of claim 2, wherein the icon representing the second vehicle is selected based on the type of vehicle.

4. The vehicle of claim 1, wherein the processor is further programmed to display, on the display, an image indicting a portion of the route to be traveled by the vehicle in the next few seconds.

5. The vehicle of claim 1, wherein the identified object is a third vehicle and the at least one action to be taken further includes maintaining a safe following distance behind the third vehicle.

6. The vehicle of claim 5, wherein the processor is further programmed to;

select a safe following distance image to be displayed; and

displaying the safe following distance image on the electronic display.

7. The vehicle of claim 1, wherein the at least one action to be taken further includes waiting and the vehicle is further programmed to display text indicating that the vehicle is waiting.

8. The vehicle of claim 1, wherein the processor is further programmed determine a geographic area to be displayed such that a larger geographic area is displayed where the vehicle is moving faster, and a smaller geographic area is displayed where the vehicle is moving slower.

9. The vehicle of claim 1, wherein the processor is further programmed to determine a geographic area to be displayed such that a larger geographic area is displayed where the roadway is associated with a relatively high speed limit, and a smaller geographic area is displayed where the roadway is associated with a relatively low speed limit.

10. The vehicle of claim 1, wherein the at least one action to be taken further includes stopping at an intersection and the processor is further programmed to select and display an image indicating where the vehicle will stop at the intersection.

11. The vehicle of claim 1, wherein the processor is further programmed to determine the geographic area to be displayed based on the action to be taken, where if the action to be taken is a turn, the geographic area includes a larger view in the direction opposite to the turn.

12. The vehicle of claim 1, wherein the identified object is a traffic signal and the processor is further programmed to:

select an image indicating a traffic signal, and

display the image indicating the traffic signal on the display proximate to the location of the traffic signal.

13. The vehicle of claim 12, wherein the processor is further programmed to:

determine the state of the traffic signal, wherein the image indicating the traffic signal is selected based on the state of the traffic signal; and

determine the at least one action to be taken based on the state of the traffic signal.

14. The vehicle of claim 1, wherein the at least one action to be taken further includes changing to a different lane, and the processor is further programmed to select and display an image indicating a turn signal.

15. A method for selecting images for display on an display apparatus of a vehicle, the method comprising:

receiving destination information from a user input device;

identifying a route to the destination;

receiving location information from a geographic position component;

accessing stored map information including roadways, traffic signals, and intersections;

determining, from the location information and the stored map information, the current geographic location of the vehicle;

identifying an object of a roadway and an object type based on object information received from an object detection apparatus;

determining at least one action to be taken including controlling at least one of a plurality of control apparatuses including a braking apparatus, an acceleration apparatus, and a steering apparatus, wherein the action to be taken is determined based on the identified object, the current geographic location of the vehicle, and the route, wherein the at least one action to be taken includes avoiding a headroom zone in front of the identified object; and

selecting a headroom zone image based on the at least one action to be taken and the identified object; and

displaying the selected headroom zone image on the electronic display.

16. The method of claim 15, further comprising determining a geographic area to be displayed such that a larger geographic area is displayed where the vehicle is moving faster, and a smaller geographic area is displayed where the vehicle is moving slower.

17. The method of claim 15, further comprising determining the geographic area to be displayed based on the action to be taken, where if the at least one action to be taken includes a turn, the geographic area includes a larger view in the direction opposite to the turn.