Occupant facing vehicle display

• US 9,950,619 B1
• Filed: 04/04/2016
• Issued: 04/24/2018
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Chan, Min Li, Nemec, Philip, Cullinane, Brian Douglas, Mercay, Julien Charles, Crandall, Peter
• Primary Examiner(s): Lee, Tyler J
• Application Number:
  US15/089,708
• Time in Patent Office: 750 Days
• Field of Search: 701 23, 701 36
• CPC Class Codes: B60K 2360/16 Type of output information
• CPC Class Codes: B60K 2360/166 Navigation
• CPC Class Codes: B60K 2360/175 Autonomous driving
• CPC Class Codes: B60K 2360/179 Distances to obstacles or vehicles
• CPC Class Codes: B60K 2360/48 Sensors
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G01C 21/3697 Output of additional, non-guidance related information, e.g. low fuel level G01C21/3679 takes precedence
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/015 with provision for distinguishing between two or more types of vehicles, e.g. between motor-cars and cycles
• CPC Class Codes: G08G 1/017 identifying vehicles G08G1/015, G08G1/054 take precedence
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Aspects of the present disclosure relate to a vehicle for maneuvering an occupant of the vehicle to a destination autonomously as well as providing information about the vehicle and the vehicle'"'"'s environment for display to the occupant.

Claims

1. A method comprising:

receiving, by one or more processors, sensor data from one or more sensor systems configured to detect objects in an external environment of a vehicle;

identifying, by the one or more processors, an object in the vehicle'"'"'s environment using the sensor data;

determining, by the one or more processors, that the vehicle should yield to the identified object in order to safely maneuver the vehicle to a destination;

generating, by the one or more processors, a three-dimensional image for a representation of the object for a scene display depicting the external environment of the vehicle, the representation of the object including a feature having a first color;

generating, by the one or more processors, a two-dimensional image for an intent indicator that indicates the determination that the vehicle should yield to the identified object for an intent display depicting the intent of the vehicle with respect to the external environment of the vehicle, the intent indicator including a feature having the first color; and

providing, by the one or more processors, the representation of the object and intent indicator for display to an occupant of the vehicle in order to indicate to the occupant a relationship between the representation of the object and the intent indicator through the first color.

2. The method of claim 1, wherein the representation of the object is provided for display in a first area of an electronic display corresponding to the scene display, and the intent indicator is provided for display in a second area of an electronic display distinct from the scene display.

3. The method of claim 1, wherein the representation of the object is provided for display in a first area of an electronic display corresponding to the scene display, the intent indicator is provided for display as an overlay on the first area of the electronic display over the scene display.

4. The method of claim 3, wherein an area within the intent indicator which overlays a portion the scene display includes a blur effect for the portion in order to distinguish the intent indicator from the scene display.

5. The method of claim 1, further comprising:

generating a three-dimensional background scene using detailed map information corresponding to the external environment of the vehicle, and

wherein the background scene is provided as a background for the three-dimensional image of the representation of the object and the two-dimensional image for the intent indicator is configured to be overlaid onto the three-dimensional background scene.

6. The method of claim 1, wherein the feature of the representation of the object is a first ring, and the feature of the intent indicator is a second ring.

7. The method of claim 1, wherein the feature of the representation of the object is a first ring drawn around a depiction of sensor data corresponding to the object and the feature of the intent indicator is a second ring drawn around a depiction of intent of one or more computing devices of the vehicle.

8. The method of claim 1, wherein the feature of the representation of the object is an abstract depiction of the object and the feature of the intent indicator is a ring drawn around a depiction of intent of one or more computing devices of the vehicle.

9. A system comprising one or more computing devices having one or more processors configured to:

receive sensor data from one or more sensor systems configured to detect objects in an external environment of a vehicle;

identify an object in the vehicle'"'"'s environment using the sensor data;

determine that the vehicle should yield to the identified object in order to safely maneuver the vehicle to a destination;

generate a three-dimensional image for a representation of the object for a scene display depicting the external environment of the vehicle, the representation of the object including a feature having a first color;

generate a two-dimensional image for an intent indicator that indicates the determination that the vehicle should yield to the identified object for an intent display depicting the intent of the vehicle with respect to the external environment of the vehicle, the intent indicator including a feature having the first color; and

provide the representation of the object and intent indicator for display to an occupant of the vehicle in order to indicate to the occupant a relationship between the representation of the object and the intent indicator through the first color.

10. The system of claim 9, wherein the one or more processors are further configured to:

provide the representation of the object for display in a first area of an electronic display corresponding to the scene display, and

provide intent indicator in a second area of an electronic display distinct from the scene display.

11. The system of claim 9, wherein the one or more processors are further configured to;

provide the representation of the object for display in a first area of an electronic display corresponding to the scene display, and

provide the intent indicator as an overlay on the first area of the electronic display over the scene display.

12. The system of claim 11, wherein the one or more processors are further configured to provide the intent indicator such that an area within the intent indicator which overlays a portion the scene display includes a blur effect for the portion in order to distinguish the intent indicator from the scene display.

13. The system of claim 9, wherein the one or more processors are further configured to:

generate a three-dimensional background scene using detailed map information corresponding to the external environment of the vehicle,

provide the background scene as a background for the three-dimensional image of the representation of the object, and

provide the two-dimensional image for the intent indicator configured to be overlaid onto the three-dimensional background scene.

14. The system of claim 9, wherein the feature of the representation of the object is a first ring, and the feature of the intent indicator is a second ring.

15. The system of claim 9, wherein the feature of the representation of the object is a first ring drawn around a depiction of sensor data corresponding to the object and the feature of the intent indicator is a second ring drawn around a depiction of intent of one or more computing devices of the vehicle.

16. The system of claim 9, wherein the feature of the representation of the object is an abstract depiction of the object and the feature of the intent indicator is a ring drawn around a depiction of intent of one or more computing devices of the vehicle.

17. The system of claim 9, further comprising the vehicle.

18. A non-transitory, tangible computer readable medium on which instructions are stored, the instructions when executed by one or more processors cause the one or more processors to perform a method, the method comprising:

receiving sensor data from one or more sensor systems configured to detect objects in an external environment of a vehicle;

identifying an object in the vehicle'"'"'s environment using the sensor data;

determining that the vehicle should yield to the identified object in order to safely maneuver the vehicle to a destination;

generating a three-dimensional image for a representation of the object for a scene display depicting the external environment of the vehicle, the representation of the object including a feature having a first color;

generating a two-dimensional image for an intent indicator that indicates the determination that the vehicle should yield to the identified object for an intent display depicting the intent of the vehicle with respect to the external environment of the vehicle, the intent indicator including a feature having the first color; and

providing the representation of the object and intent indicator for display to an occupant of the vehicle in order to indicate to the occupant a relationship between the representation of the object and the intent indicator through the first color.

19. The medium of claim 18, wherein the representation of the object is provided for display in a first area of an electronic display corresponding to the scene display, and the intent indicator is provided for display in a second area of an electronic display distinct from the scene display.

20. The medium of claim 18, wherein the representation of the object is provided for display in a first area of an electronic display corresponding to the scene display, the intent indicator is provided for display as an overlay on the first area of the electronic display over the scene display.