Augmented reality heads up display (HUD) for yield to pedestrian safety cues

• US 9,064,420 B2
• Filed: 03/14/2013
• Issued: 06/23/2015
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Beckwith, Lee, Ng-Thow-Hing, Victor
• Primary Examiner(s): WANG, JIN CHENG
• Application Number:
  US13/803,288
• Publication Number:
  US 20140267398A1
• Time in Patent Office: 831 Days
• Field of Search: 345633-634, 348/148
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/1438 Touch screens
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/308 by overlaying the real scene, e.g. through a head-up display on the windscreen
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

An augmented reality driver system, device, and method safely guide a vehicle driver to yield to pedestrians. A vehicle navigator determines a turn lane based upon proximity to a vehicle. A target sensor detects a pedestrian entering the turn lane and to determine a crosswalk path across the turn lane. An augmented reality controller three dimensionally maps a forward view including the pedestrian, and spatially overlays an augmented reality display on the volumetric heads up display for a driver of the vehicle by projecting a yielding indication adjacent to the crosswalk path.

Claims

1. A method for safely guiding a vehicle driver to yield to a pedestrian, comprising:

determining a turn lane based upon a proximity of a vehicle to the turn lane prior to the vehicle entering the turn lane;

detecting a pedestrian entering the turn lane;

three dimensionally mapping a forward view including the pedestrian and the turn lane; and

spatially overlaying an augmented reality display on a volumetric heads up display for the vehicle driver comprising projecting a yielding indication adjacent to a determined crosswalk path across the turn lane.

2. The method of claim 1, comprising projecting the yielding indication aligned as a barrier to the pedestrian adjacent to the crosswalk path.

3. The method of claim 1, comprising:

detecting a building structure adjacent to the pedestrian entering the turn lane;

three dimensionally mapping the building structure in the forward view; and

overlaying the augmented reality display with the yielding indication that is skinned to the building structure.

4. The method of claim 1, wherein determining the turn lane comprises:

accessing a digital map, a location of the vehicle, and a current trajectory of the vehicle; and

receiving a user input indicative of a planned turn onto the turn lane.

5. The method of claim 4, further comprising receiving the user input indicative of the planned turn by receiving a route map based upon the digital map.

6. The method of claim 4, further comprising receiving the user input indicative of the planned turn by receiving a turn signal.

7. The method of claim 1, further comprising:

detecting an absence of pedestrians entering or in the crosswalk path subsequent to detecting the vehicle stopped in response to the yielding indication; and

replacing the yielding indication with a success indication.

8. An augmented reality driver system for safely guiding a vehicle driver to yield to pedestrians, comprising:

a vehicle navigator to determine a turn lane based upon a proximity to of a vehicle to the turn lane;

a target sensor to detect a pedestrian entering the turn lane and to determine a crosswalk path across the turn lane;

a volumetric heads up display; and

an augmented reality controller to three dimensionally map a forward view including the pedestrian and the turn lane, and to spatially overlay an augmented reality display on the volumetric heads up display for the vehicle driver by projecting a yielding indication adjacent to the crosswalk path.

9. The augmented reality driver system of claim 8, wherein the augmented reality controller projects the yielding indication aligned as a barrier to the pedestrian adjacent to the crosswalk path.

10. The augmented reality driver system of claim 8, wherein the target sensor detects a building structure adjacent to the pedestrian entering the turn lane;

and

wherein the augmented reality controller three dimensionally maps the building structure in the forward view, and to overlay the augmented reality display with the yielding indication that is skinned to the building structure.

11. The augmented reality driver system of claim 8, wherein the vehicle navigator determines the turn lane by:

accessing a digital map, a location of the vehicle, and a current trajectory of the vehicle; and

receiving a user input indicative of a planned turn onto the turn lane.

12. The augmented reality driver system of claim 11, wherein the vehicle navigator receives the user input indicative of the planned turn by receiving a route map based upon the digital map.

13. The augmented reality driver system of claim 11, wherein the vehicle navigator receives the user input indicative of the planned turn by receiving a turn signal.

14. The augmented reality driver system of claim 8, wherein the target sensor detects an absence of pedestrians entering or in the crosswalk path;

and

wherein the augmented reality controller replaces the yielding indication with a success indication subsequent to detecting the vehicle stopped in response to the yielding indication.

15. A device for safely guiding a vehicle driver to yield to pedestrians, comprising:

a volumetric heads up display; and

a controller in communication with the volumetric heads up display, wherein the controller comprises at least one processor that executes software instructions to perform operations comprising;

determining a turn lane based upon proximity of a vehicle to the turn lane before the vehicle enters the turn lane,

detecting a pedestrian entering the turn lane and determining a crosswalk path across the turn lane,

three dimensionally mapping a forward view including the pedestrian and the turn lane, and

spatially overlaying an augmented reality display on a volumetric heads up display for the vehicle driver by projecting a yielding indication adjacent to the crosswalk path.

16. The device of claim 15, wherein the controller performs operations that comprise projecting the yielding indication aligned as a barrier to the pedestrian adjacent to the crosswalk path.

17. The device of claim 15, wherein the controller performs operations that comprise:

detecting a building structure adjacent to the pedestrian entering the turn lane;

three dimensionally mapping the building structure in the forward view; and

overlaying the augmented reality display with the yielding indication that is skinned to the building structure.

18. The device of claim 15, wherein the controller performs operations that comprise:

accessing a digital map, a location of the vehicle, and a current trajectory of the vehicle; and

receiving a user input indicative of a planned turn onto the turn lane.

19. The device of claim 18, wherein the controller receives the user input indicative of the planned turn by receiving a route map based upon the digital map.

20. The device of claim 18, wherein the controller performs operations that comprise receiving the user input indicative of the planned turn by receiving a turn signal.