User interface for displaying object-based indications in an autonomous driving system

• US 9,501,058 B1
• Filed: 11/17/2014
• Issued: 11/22/2016
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Mariet, Robertus Christianus Elisabeth, Clement, Manuel Christian, Nemec, Philip, Cullinane, Brian
• Primary Examiner(s): Brown, Drew
• Application Number:
  US14/542,799
• Time in Patent Office: 736 Days
• US Class Current: 1/1
• CPC Class Codes: B60T 2201/022 Collision avoidance systems
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2554/804 Relative longitudinal speed
• CPC Class Codes: B60W 2556/40 High definition maps
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/18159 Traversing an intersection
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G05D 1/0238 using obstacle or wall sensors G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/249 from positioning sensors located off-board the vehicle, e.g. from cameras
• CPC Class Codes: G05D 1/617 Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80
• CPC Class Codes: G05D 1/628 following the obstacle profile, e.g. a wall or undulated terrain
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/09626 where the origin of the information is within the own vehicle, e.g. a local storage device, digital map
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A vehicle has a plurality of control apparatuses, a user input, a geographic position component, an object detection apparatus, memory, and a display. A processor is also included and is programmed to receive the destination information, identify a route, and determine the current geographic location of the vehicle. The processor is also programmed to identify an object and object type based on object information received from the object detection apparatus and to determine at least one warning characteristic of the identified object based on at least one of: the object type, a detected proximity of the detected object to the vehicle, the location of the detected object relative to predetermined peripheral areas of the vehicle, the current geographic location of the vehicle, and the route. The processor is also configured to select and display on the display an object warning image based on the at least one warning characteristic.

Claims

1. A vehicle comprising:

a geographic position component configured to determine a current location of the vehicle;

an object detection apparatus configured to detect object information of one or more objects in proximity to the vehicle, the object detection apparatus including one or more selected from the group consisting of a laser, a camera, sonar or radar;

at least one display device in a passenger compartment of the vehicle; and

one or more processors in operative communication with the object detection apparatus, the at least one display device, and the geographic position component, the one or more processors being configured to;

receive the current location of the vehicle from the geographic position component;

identify a selected one of the one or more objects and an object type of the selected object from the detected object information;

determine a warning characteristic of the identified object; and

present an object warning indication corresponding to the warning characteristic visually on the at least one display device, wherein the visual presentation of the object warning indication comprises;

a vehicle icon representing the vehicle;

an object indicator image representing a current location of the identified object, the object indicator image being disposed remote from the vehicle icon;

a notification image having one or more segments surrounding the vehicle icon, the one or more segments being disposed between the vehicle icon and the object indicator image; and

a warning image representing a warning related to the identified object, the warning image being disposed adjacent to the object indicator image.

2. The vehicle of claim 1, wherein the one or more processors are configured to determine the warning characteristic of the identified object according to one or more of:

the object type, a detected proximity of the identified object to the vehicle, a location of the identified object relative to one or more predetermined peripheral areas of the vehicle, the current location of the vehicle, and a route of the vehicle.

3. The vehicle of claim 1, wherein the notification image having the one or more segments is presented as a multi-segmented shape surrounding the vehicle icon.

10. The vehicle of claim 1, wherein upon presentation of the warning image, a given one of the plurality of segments corresponding to a position of the identified object changes color.

11. The vehicle of claim 1, wherein the notification image includes a plurality of divided segments, and the divisions between the segments extend radially outward from a center of the vehicle icon, thereby dividing an area surrounding the vehicle icon into a set of quadrants.

12. The vehicle of claim 1, wherein the notification image includes a plurality of overlapping segments at least partly surrounding the vehicle icon.

13. The vehicle of claim 1, wherein the object warning indication is additionally presented audibly within the vehicle.

14. The vehicle of claim 1, wherein the object warning indication is presented as an escalated warning state.

15. The vehicle of claim 1, wherein, when operating the vehicle in an autonomous mode, the object warning indication presents a reason to a passenger why a corresponding action is being taken by the vehicle.

16. The vehicle of claim 1, further comprising memory configured to store display data including at least one of roadway and intersection information wherein the one or more processors are configured to present the object warning indication along with at least a portion of the roadway and intersection information to prospectively indicate a corresponding action to be taken by the vehicle.

17. The vehicle of claim 1, further comprising memory configured to store detailed roadway map information, wherein the one or more processors are configured to present the object warning indication along with the detailed roadway map information.

18. The vehicle of claim 1, wherein the notification image is presented as a single segment surrounding the vehicle icon.

19. The vehicle of claim 1, wherein the warning image is disposed between the object indicator image and the vehicle icon.

20. The vehicle of claim 1, wherein the warning image is configured to identify the object type of the identified object.

4. The vehicle of claim wherein the one or more segments of the notification image are a plurality of segments, and each segment corresponds to predetermined peripheral area adjacent an exterior portion of the vehicle.

5. The vehicle of claim 4, wherein the plurality of segments are arranged in multiple nested layers, each layer indicating a different level of proximity of objects to the vehicle.

6. The vehicle of claim 5, wherein upon presentation of the warning image, an area within a given one of the plurality of segments corresponding to a position of the identified object changes color.

7. The vehicle of claim 5, wherein each segment of the plurality of segments in a given layer is undivided and arranged in a concentric ring surrounding the vehicle icon.

8. The vehicle of claim 5, wherein the multiple nested layers are arranged as a grid surrounding the vehicle icon.

9. The vehicle of claim 8, wherein each segment of the plurality of segments is divided in multiple sub-segments along a portion of the grid.

21. A method for communicating warning information in a vehicle having an autonomous operating mode, the method comprising:

determining, by one or more processors using information from a geographic position component, a current location of the vehicle;

detecting object information of one or more objects in proximity to the vehicle using an object detection apparatus, the object detection apparatus including one or more selected from the group consisting of a laser, a camera, sonar or radar;

identifying, by the one or more processors, an object and an object type of the object from the detected object information;

determining, by the one or more processors using the current location of the vehicle, a warning characteristic of the identified object; and

presenting, by the one or more processors, an object warning indication corresponding to the warning characteristic visually on at least one display device;

wherein the one or more processors are configured to display on the at least one display device;

a vehicle icon representing the vehicle;

an object indicator image representing a current location of the identified object, the object indicator image being disposed remote from the vehicle icon;

a notification image having one or more segments surrounding the vehicle icon, the one or more segments being disposed between the vehicle icon and the object indicator image; and

a warning image representing a warning related to the identified object;

the warning image being disposed adjacent to the object indicator image.

22. The method of claim 21, wherein the warning characteristic of the identified object is determined according to one or more of:

the object type, a detected proximity of the identified object to the vehicle, a location of the identified object relative to one or more predetermined peripheral areas of the vehicle, the current location of the vehicle, and a route of the vehicle.

23. The method of claim 21, wherein the object warning indication is further presented audibly within the vehicle.

24. The method of claim 21, wherein the object warning indication is presented as an escalated warning state.

25. The method of claim 21, wherein, when operating the vehicle in autonomous mode, the object warning indication presents a reason to a passenger why a corresponding action is being taken by the vehicle.