Occupant facing vehicle display

• US 10,140,870 B1
• Filed: 04/04/2016
• Issued: 11/27/2018
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Chan, Min Li, Nemec, Philip, Cullinane, Brian Douglas, Mercay, Julien Charles, Crandall, Peter
• Primary Examiner(s): Antonucci, Anne M
• Assistant Examiner(s): Stroud, James E
• Application Number:
  US15/089,736
• Time in Patent Office: 967 Days
• Field of Search: 701301
• CPC Class Codes: B60K 2360/16 Type of output information
• CPC Class Codes: B60K 2360/166 Navigation
• CPC Class Codes: B60K 2360/175 Autonomous driving
• CPC Class Codes: B60K 2360/179 Distances to obstacles or vehicles
• CPC Class Codes: B60K 2360/48 Sensors
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G01C 21/3697 Output of additional, non-guidance related information, e.g. low fuel level G01C21/3679 takes precedence
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/015 with provision for distinguishing between two or more types of vehicles, e.g. between motor-cars and cycles
• CPC Class Codes: G08G 1/017 identifying vehicles G08G1/015, G08G1/054 take precedence
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A vehicle for maneuvering an occupant of the vehicle to a destination autonomously as well as providing information about the vehicle and the vehicle'"'"'s environment for display to the occupant.

Claims

1. A method comprising:

receiving, by one or more processors, sensor data from a sensor configured to detect objects in an external environment of a vehicle;

identifying, by the one or more processors, a first object of a first type and a second object of a second type in the vehicle'"'"'s environment from the sensor data, the first type and the second type being different;

generating, by the one or more processors, a representation of the first object having a first color scheme for a scene display depicting the external environment of the vehicle having a second color scheme based on the first type, wherein the representation of the first object includes a display of at least a portion of the sensor data corresponding to the object;

generating, by the one or more processors, a representation of the second object having the second color scheme for the scene display based on the second type, wherein the representation of the second object is an abstract representation of an object;

generating, by the one or more processors, a representation of the vehicle and a path of the vehicle;

providing, by the one or more processors, the representation of the first object, the representation of the second object, and the representation of the vehicle and the path of the vehicle for display on an electronic display to an occupant of the vehicle in order to indicate to the occupant that the first object and the second objects are of different types and that the first object is more vulnerable to an impact with the vehicle than the second object

changing, by the one or more processors, a color scheme of the second object from the second color scheme to the first color scheme when the second object moves in a path of the vehicle; and

changing, by the one or more processors, a color scheme of the path of the vehicle from the first color scheme to the second color scheme when the vehicle stops or yields.

2. The method of claim 1, wherein the representation of the first object includes a laser point cloud that provides the occupant with an indication of a particular shape of the first object.

3. The method of claim 2, wherein the representation of the second object is a stylized cuboid that does not provide the occupant with an indication of the particular shape of the first object.

4. The method of claim 1, wherein the first object is a pedestrian and the second object is another vehicle.

5. The method of claim 1, wherein the first type is bicyclist and the second type is another vehicle.

6. The method of claim 1, further comprising:

generating a three-dimensional background scene using detailed map information corresponding to the external environment of the vehicle; and

providing the three-dimensional background scene for display with the representation of the first object and the representation of the second object.

7. The method of claim 1, further comprising:

identifying a third object of a third type in the vehicle'"'"'s environment from the sensor data;

generating a representation of the third object that is a same representation as the representation of the second object; and

providing the representation of the third object for display with the representation of the first object and the representation of the second object.

8. The method of claim 7, wherein the second object and the first object are each other vehicles.

9. The method of claim 7, wherein the third type is a different type from the second type.

10. The method of claim 7, wherein the second object is a car and the third object is a tractor trailer.

11. The method of claim 1, wherein the representation of the first object indicates an intent of one or more of the vehicle'"'"'s processors with regard to the first object, and the abstract representative of the second object indicates an intent of the one or more of the vehicle'"'"'s processors with regard to the second object.

12. A system comprising one or more computing devices having one or more processors configured to:

receive sensor data from a sensor configured to detect objects in an external environment of a vehicle;

identify a first object of a first type and a second object of a second type in the vehicle'"'"'s environment from the sensor data, the first type and the second type being different;

generate a representation of the first object having a first color scheme for a scene display depicting the external environment of the vehicle having a second color scheme based on the first type, wherein the representation of the first object includes a display of at least a portion of the sensor data corresponding to the object;

generate a representation of the second object having the second color scheme for the scene display based on the second type, wherein the representation of the second object is an abstract representation of an object;

generate a representation of the vehicle and a pathe of the vehicle;

provide the representation of the first object, the representation of the second object, and the representation of the vehicle and the path of the vehicle for display on an electronic display to an occupant of the vehicle in order to indicate to the occupant that the first object and the second objects are of different types and that the first object is more vulnerable to an impact with the vehicle than the second object;

changing, by the one or more processors, a color scheme of the second object from the second color scheme to the first color scheme when the second object moves in a path of the vehicle; and

change a color scheme of the path of the vehicle from the first color scheme to the second color scheme when the vehicle stops or yields.

13. The system of claim 12, wherein the representation of the first object includes a laser point cloud that provides the occupant with an indication of a particular shape of the first object.

14. The system of claim 13, wherein the representation of the second object is a stylized cuboid that does not provide the occupant with an indication of the particular shape of the first object.

15. The system of claim 12, wherein the first object is a pedestrian and the second object is another vehicle.

16. The system of claim 12, wherein the first type is bicyclist and the second type is another vehicle.

17. The system of claim 12, wherein the one or more processors are further configured to:

generate a three-dimensional background scene using detailed map information corresponding to the external environment of the vehicle; and

provide the three-dimensional background scene for display with the representation of the first object and the representation of the second object.

18. The system of claim 12, wherein the one or more processors are further configured to:

identify a third object of a third type in the vehicle'"'"'s environment from the sensor data;

generate a representation of the third object that is a same representation as the representation of the second object; and

provide the representation of the third object for display with the representation of the first object and the representation of the second object.

19. The system of claim 12, further comprising the vehicle.

20. A non-transitory, tangible computer readable medium on which instructions are stored, the instructions when executed by one or more processors cause the one or more processors to perform a method, the method comprising:

receiving sensor data from a sensor configured to detect objects in an external environment of a vehicle;

identifying a first object of a first type and a second object of a second type in the vehicle'"'"'s environment from the sensor data, the first type and the second type being different;

generating a representation of the first object having a first color scheme for a scene display depicting the external environment of the vehicle having a second color scheme based on the first type, wherein the representation of the first object includes a display of at least a portion of the sensor data corresponding to the object;

generating a representation of the second object having the second color scheme for the scene display based on the second type, wherein the representation of the second object is an abstract representation of an object;

generating a representation of the vehicle and a path of the vehicle;

providing the representation of the first object, the representation of the second object, and the representation of the vehicle and the path of the vehicle for display on an electronic display to an occupant of the vehicle in order to indicate to the occupant that the first object and the second objects are of different types and that the first object is more vulnerable to an impact with the vehicle than the second object;

changing a color scheme of the second object from the second color scheme to the first color scheme when the second object moves in a path of the vehicle; and

change a color scheme of the path of the vehicle from the first color scheme to the second color scheme when the vehicle stops or yields.