User interface for displaying object-based indications in an autonomous driving system

• US 10,168,710 B1
• Filed: 05/23/2017
• Issued: 01/01/2019
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Mariet, Robertus Christianus Elisabeth, Clement, Manuel Christian, Nemec, Philip, Cullinane, Brian Douglas
• Primary Examiner(s): Brown, Drew J
• Application Number:
  US15/602,423
• Time in Patent Office: 588 Days
• CPC Class Codes: B60T 2201/022 Collision avoidance systems
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2554/804 Relative longitudinal speed
• CPC Class Codes: B60W 2556/40 High definition maps
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/18159 Traversing an intersection
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G05D 1/0238 using obstacle or wall sensors G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/09626 where the origin of the information is within the own vehicle, e.g. a local storage device, digital map
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A vehicle has a plurality of control apparatuses, a user input, a geographic position component, an object detection apparatus, memory, and a display. A processor is also included and is programmed to receive the destination information, identify a route, and determine the current geographic location of the vehicle. The processor is also programmed to identify an object and object type based on object information received from the object detection apparatus and to determine at least one warning characteristic of the identified object based on at least one of: the object type, a detected proximity of the detected object to the vehicle, the location of the detected object relative to predetermined peripheral areas of the vehicle, the current geographic location of the vehicle, and the route. The processor is also configured to select and display on the display an object warning image based on the at least one warning characteristic.

Claims

1. A method comprising:

identifying, by one or more processors, a notification image including a representation of a vehicle and a plurality of segments representing physical areas in an environment around the vehicle;

detecting, by the one or more processors, an object at a particular location in the environment using a sensor system;

identifying, by the one or more processors, a segment of the plurality based on the particular location;

generating, by the one or more processors, a visual representation of the environment of the vehicle, the visual representation including a path of the vehicle and a representation of the vehicle; and

displaying, by the one or more processors, the visual representation, the representation of the vehicle, the path and the identified segment such that the representation of the vehicle appears to be overlaid on the path, and the identified segment is depicted as a highlighted area adjacent to the representation of the vehicle in order to identify a location of the object relative to the vehicle.

2. The method of claim 1, further comprising displaying all of the segments of the plurality with the visual representation, the representation of the vehicle, and the path.

3. The method of claim 2, wherein the identified segment is displayed in a different color from other displayed segments of the plurality.

4. The method of claim 1, wherein each of the segments of the plurality extend radially outward from the vehicle.

5. The method of claim 1, wherein the identified segment is displayed as an area that extends radially outward from the vehicle.

6. The method of claim 1, further comprising:

detecting a second object at a second particular location in the environment; and

identifying a second segment of the plurality based on the second particular location, and

displaying the second identified segment is displayed the visual representation, the representation of the vehicle, the path and the identified segment, the second identified segment being displayed as a highlighted area adjacent to the representation of the vehicle in order to identify a location of the second object relative to the representation of the vehicle.

7. The method of claim 1, further comprising:

detecting a second object, the second object corresponding to a second vehicle; and

displaying a representation of the second object in front of the visual representation relative to the path.

8. The method of claim 1, wherein an area of the identified segment overlaps with an area of another segment of the plurality of segments.

9. The method of claim 1, wherein displaying the identified segment includes displaying an image of an object representative of a type of the object.

10. A system comprising one or more processors configured to:

identify a notification image including a representation of a vehicle and a plurality of segments representing physical areas in an environment around the vehicle;

detect an object at a particular location in the environment using a sensor system;

identify a segment of the plurality based on the particular location;

generate a visual representation of the environment of the vehicle, the visual representation including a path of the vehicle and a representation of the vehicle; and

display the visual representation, the representation of the vehicle, the path and the identified segment such that the representation of the vehicle appears to be overlaid on the path, and the identified segment is depicted as a highlighted area adjacent to the representation of the vehicle in order to identify a location of the object relative to the vehicle.

11. The system of claim 10, further comprising the vehicle.

12. The system of claim 10, wherein the one or more processors are further configured to display all of the segments of the plurality with the visual representation, the representation of the vehicle, and the path.

13. The system of claim 12, wherein the one or more processors are further configured to display the identified segment in a different color from other displayed segments of the plurality.

14. The system of claim 10, wherein each of the segments of the plurality extend radially outward from the vehicle.

15. The system of claim 10, wherein the one or more processors are further configured to display the identified segment as an area that extends radially outward from the vehicle.

16. The system of claim 10, wherein the one or more processors are further configured to:

detect a second object at a second particular location in the environment; and

identify a second segment of the plurality based on the second particular location, and

displaying the second identified segment is displayed the visual representation, the representation of the vehicle, the path and the identified segment, the second identified segment being displayed as a highlighted area adjacent to the representation of the vehicle in order to identify a location of the second object relative to the representation of the vehicle.

17. The system of claim 10, wherein the one or more processors are further configured to:

detect a second object, the second object corresponding to a second vehicle; and

display a representation of the second object in front of the visual representation relative to the path.

18. The system of claim 10, wherein an area of the identified segment overlaps with an area of another segment of the plurality of segments.

19. The system of claim 10, wherein displaying the identified segment includes displaying an image of an object representative of a type of the object.

20. A non-transitory, machine readable medium on which instructions are stored, the instructions when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:

identifying a notification image including a representation of a vehicle and a plurality of segments representing physical areas in an environment around the vehicle;

detecting an object at a particular location in the environment using a sensor system;

identifying a segment of the plurality based on the particular location;

generating a visual representation of the environment of the vehicle, the visual representation including a path of the vehicle and a representation of the vehicle; and

displaying the visual representation, the representation of the vehicle, the path and the identified segment such that the representation of the vehicle appears to be overlaid on the path, and the identified segment is depicted as a highlighted area adjacent to the representation of the vehicle in order to identify a location of the object relative to the vehicle.