User interface for displaying object-based indications in an autonomous driving system

• US 10,852,742 B1
• Filed: 10/22/2018
• Issued: 12/01/2020
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Mariet, Robertus Christianus Elisabeth, Clement, Manuel Christian, Nemec, Philip, Cullinane, Brian Douglas
• Primary Examiner(s): Brown, Drew J
• Application Number:
  US16/166,876
• Time in Patent Office: 771 Days
• CPC Class Codes: B60T 2201/022 Collision avoidance systems
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2554/804 Relative longitudinal speed
• CPC Class Codes: B60W 2556/40 High definition maps
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/18159 Traversing an intersection
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G05D 1/0238 using obstacle or wall sensors G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/09626 where the origin of the information is within the own vehicle, e.g. a local storage device, digital map
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A vehicle has a plurality of control apparatuses, a user input, a geographic position component, an object detection apparatus, memory, and a display. A processor is also included and is programmed to receive the destination information, identify a route, and determine the current geographic location of the vehicle. The processor is also programmed to identify an object and object type based on object information received from the object detection apparatus and to determine at least one warning characteristic of the identified object based on at least one of: the object type, a detected proximity of the detected object to the vehicle, the location of the detected object relative to predetermined peripheral areas of the vehicle, the current geographic location of the vehicle, and the route. The processor is also configured to select and display on the display an object warning image based on the at least one warning characteristic.

Claims

1. A method comprising:

receiving, by one or more processors, sensor data generated by a sensor of a vehicle having an autonomous driving mode, the sensor data identifying at least one object in an environment of the vehicle;

determining, by the one or more processors, a potential problem with respect to the at least one object;

generating, by the one or more processors, visual alert information for the at least one object based on the potential problem; and

displaying, by the one or more processors, the visual alert information of a display of the vehicle with a portion of a route the vehicle is following in the autonomous driving mode

wherein the visual alert information includes an image surrounding a representation of the vehicle, and

wherein appearance of one or more portions of the image can be altered to indicate presence of a potentially problematic identified object.

2. The method of claim 1, wherein the determining the potential problem is based on a location of the at least one object relative to one or more predetermined peripheral areas of the autonomous vehicle.

3. The method of claim 1, wherein the determining the potential problem is based on a type of the at least one object.

4. The method of claim 1, wherein the determining the potential problem is based on a proximity of the at least one object to the vehicle.

5. The method of claim 1, wherein the determining the potential problem is based on a speed of the at least one object.

6. The method of claim 1, wherein the determining the potential problem is based on the route the vehicle is following in the autonomous driving mode.

7. The method of claim 1, wherein the determining the potential problem is based on a direction of the at least one object relative to the vehicle.

8. The method of claim 1, wherein the determining the potential problem is based on a braking behavior of the at least one object.

9. The method of claim 8, further comprising, controlling the vehicle in the autonomous driving mode in order to respond to the braking behavior of the at least one object.

10. The method of claim 9, wherein displaying the visual alert information comprises identifying a reason for the controlling of the vehicle.

11. The method of claim 1, wherein displaying the visual alert information includes overlaying the visual alert information on the portion.

12. The method of claim 1, wherein the visual alert information includes a warning image.

13. The method of claim 1, wherein the visual alert information includes changing a color of one or more of the one or more portions of the image to a first color.

14. The method of claim 13, wherein the first color is determined based on an urgency of the potential problem associated with the at least one object.

15. The method of claim 13, wherein the one or more portions of the image include a plurality of segments.

16. The method of claim 15, wherein each one of the plurality of segments corresponds to a predetermined peripheral area around the representation of the vehicle.

17. The method of claim 13, wherein the representation of the vehicle has a chevron shape.

18. The method of claim 13, wherein the representation of the vehicle has a triangular shape.

19. The method of claim 8, further comprising, controlling the vehicle in the autonomous driving mode in order to respond to the braking behavior of the at least one object.

20. The method of claim 1, further comprising, controlling the vehicle in the autonomous driving mode in order to respond to the potential problem.