Occupant facing vehicle display

• US 9,849,784 B1
• Filed: 04/04/2016
• Issued: 12/26/2017
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Chan, Min Li, Nemec, Philip, Cullinane, Brian Douglas, Mercay, Julien Charles, Crandall, Peter
• Primary Examiner(s): Figueroa, Jaime
• Application Number:
  US15/089,716
• Time in Patent Office: 631 Days
• Field of Search: 701 1, 701 36
• CPC Class Codes: B60K 2360/16 Type of output information
• CPC Class Codes: B60K 2360/166 Navigation
• CPC Class Codes: B60K 2360/175 Autonomous driving
• CPC Class Codes: B60K 2360/179 Distances to obstacles or vehicles
• CPC Class Codes: B60K 2360/48 Sensors
• CPC Class Codes: B60K 2370/152 Displays
• CPC Class Codes: B60K 2370/16 Type of information
• CPC Class Codes: B60K 2370/166 Navigation
• CPC Class Codes: B60K 2370/175 Autonomous driving
• CPC Class Codes: B60K 2370/179 Distances to obstacles or vehicles
• CPC Class Codes: B60K 2370/193 for improving awareness
• CPC Class Codes: B60K 2370/48 Sensors
• CPC Class Codes: B60K 2370/52 Control of displays
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G01C 21/3697 Output of additional, non-guidance related information, e.g. low fuel level G01C21/3679 takes precedence
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/015 : with provision for distinguishing between two or more types of vehicles, e.g. between motor-cars and cycles
• CPC Class Codes: G08G 1/017 : identifying vehicles G08G1/015, G08G1/054 take precedence
• CPC Class Codes: G08G 1/04 : using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/0962 : having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Aspects of the present disclosure relate to a vehicle for maneuvering an occupant of the vehicle to a destination autonomously as well as providing information about the vehicle and the vehicle'"'"'s environment for display to the occupant.

Claims

1. A method comprising:

receiving, by one or more processors, sensor data from a sensor configured to detect objects in an external environment of a vehicle;

identifying, by the one or more processors, a first object and a second object in the vehicle'"'"'s environment from the sensor data;

determining, by the one or more processors, that the first object is relevant to safely maneuvering the vehicle to a destination;

determining, by the one or more processors, that the second object is not currently relevant to safely maneuvering the vehicle to the destination;

generating, by the one or more processors, a representation of the first object for a scene display depicting the external environment of the vehicle, the representation of the first object including a feature having a first color scheme;

generating, by the one or more processors, a representation of the second object for the scene display, the representation of the second object having a second color scheme, wherein the first color scheme is configured to stand out from the second color scheme;

generating, by the one or more processors, a three-dimensional background scene in the second color scheme using detailed map information corresponding to the external environment of the vehicle; and

providing, by the one or more processors, the representation of the first object, the representation of the second object, and the background scene for display to an occupant of the vehicle in order to indicate to the occupant that first object is relevant to safely maneuvering the vehicle to a destination and that the second object is not currently relevant to safely maneuvering the vehicle to the destination.

2. The method of claim 1, wherein the background scene includes lane lines in the second color scheme, and the representation of the first object and the representation of the second object both represent vehicles.

3. The method of claim 1, wherein the background scene includes static objects detected the vehicle'"'"'s external environment and included with the detailed map information.

4. The method of claim 1, wherein the sensor data indicates that both the first object and the second object are of a same type.

5. The method of claim 1, further comprising:

after providing the first representation for display, determining that the first object should no longer be shown as relevant to safely maneuvering the vehicle to the destination;

generating, based on the determination that the first object should no longer be shown as relevant, a new representation of the first object in the second color scheme; and

providing the new representation as a replacement for the representation of the first object in order to indicate to the occupant that the first object is no longer relevant to safely maneuvering the vehicle to the destination.

6. The method of claim 1, wherein the new representation is provided such that the representation of the first object will appear to fade into the background scene by being replaced with the new representation.

7. The method of claim 1, further comprising:

after providing the second representation for display, determining that the second object is relevant to safely maneuvering the vehicle to the destination;

generating a new representation of the second object in the second color scheme; and

providing the new representation as a replacement for the representation of the second object in order to indicate to the occupant that the first second is relevant to safely maneuvering the vehicle to the destination.

8. The method of claim 1, wherein the new representation is provided such that the new representation of the first object will appear to fade in from the background scene when replacing the representation of the first object.

9. The method of claim 1, wherein determining that the first object is relevant to safely maneuvering the vehicle to the destination is based on a determination that the vehicle must take a specific action in order to avoid the first object for a predetermined period of time into the future.

10. The method of claim 9, wherein determining that the second object is not currently relevant to safely maneuvering the vehicle to the destination is based on a determination that the vehicle need not take any specific action in order to avoid the second object for a predetermined period of time into the future.

11. The method of claim 1, further comprising:

determining a route to a destination;

generating a path for display in the scene display representing the route;

providing the route for display in a third color scheme different from the second color scheme in order to distinguish the path from the background scene; and

after providing the route for display, when the vehicle has stopped to avoid an object, providing the route for display in the second color scheme in order to indicate to the occupant that the vehicle is temporarily stopped but will shortly resume the route.

12. A system comprising one or more processors configured to:

receive sensor data from a sensor configured to detect objects in an external environment of a vehicle;

identify a first object and a second object in the vehicle'"'"'s environment from the sensor data;

determine that the first object is relevant to safely maneuvering the vehicle to a destination;

determine that the second object is not currently relevant to safely maneuvering the vehicle to the destination;

generate a representation of the first object for a scene display depicting the external environment of the vehicle, the representation of the first object including a feature having a first color scheme;

generate a representation of the second object for the scene display, the representation of the second object having a second color scheme, wherein the first color scheme is configured to stand out from the second color scheme;

generate a three-dimensional background scene in the second color scheme using detailed map information corresponding to the external environment of the vehicle; and

provide the representation of the first object, the representation of the second object, and the background scene for display to an occupant of the vehicle in order to indicate to the occupant that first object is relevant to safely maneuvering the vehicle to a destination and that the second object is not currently relevant to safely maneuvering the vehicle to the destination.

13. The system of claim 12, wherein the background scene includes lane lines in the second color scheme, and the representation of the first object and the representation of the second object both represent vehicles.

14. The system of claim 12, wherein the background scene includes static objects detected the vehicle'"'"'s external environment and included with the detailed map information.

15. The system of claim 12, wherein the sensor data indicates that both the first object and the second object are of a same type.

16. The system of claim 12, wherein the one or more processors are further configured to:

after providing the first representation for display, determine that the first object should no longer be shown as relevant to safely maneuvering the vehicle to the destination;

generate, based on the determination that the first object should no longer be shown as relevant, a new representation of the first object in the second color scheme; and

provide the new representation as a replacement for the representation of the first object in order to indicate to the occupant that the first object is no longer relevant to safely maneuvering the vehicle to the destination.

17. The system of claim 12, wherein the one or more processors are further configured to provide the new representation such that the representation of the first object will appear to fade into the background scene by being replaced with the new representation.

18. The system of claim 12, wherein the one or more computing devices are further configured to:

after providing the second representation for display, determine that the second object is relevant to safely maneuvering the vehicle to the destination;

generate a new representation of the second object in the second color scheme; and

provide the new representation as a replacement for the representation of the second object in order to indicate to the occupant that the first second is relevant to safely maneuvering the vehicle to the destination.

19. The system of claim 12, further comprising the vehicle.

20. A non-transitory, tangible computer readable medium on which instructions are stored, the instructions when executed by one or more processors cause the one or more processors to perform a method, the method comprising:

receive sensor data from a sensor configured to detect objects in an external environment of a vehicle;

identifying a first object and a second object in the vehicle'"'"'s environment from the sensor data;

determining that the first object is relevant to safely maneuvering the vehicle to a destination;

determining that the second object is not currently relevant to safely maneuvering the vehicle to the destination;

generating a representation of the first object for a scene display depicting the external environment of the vehicle, the representation of the first object including a feature having a first color scheme;

generating a representation of the second object for the scene display, the representation of the second object having a second color scheme, wherein the first color scheme is configured to stand out from the second color scheme;

generating a three-dimensional background scene in the second color scheme using detailed map information corresponding to the external environment of the vehicle; and

providing the representation of the first object, the representation of the second object, and the background scene for display to an occupant of the vehicle in order to indicate to the occupant that first object is relevant to safely maneuvering the vehicle to a destination and that the second object is not currently relevant to safely maneuvering the vehicle to the destination.