Reducing Driver Distraction Using a Heads-Up Display

• US 20120224060A1
• Filed: 02/10/2012
• Published: 09/06/2012
• Est. Priority Date: 02/10/2011
• Status: Abandoned Application

• Current Assignee: Integrated Night Vision Systems Incorporated
• Original Assignee: Integrated Night Vision Systems Incorporated
• Inventors: Gurevich, Mikhail, Carrasco, Luis, Griswold, Wesley
• Application Number:
  US13/371,382
• Publication Number:
  US 20120224060A1
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60K 2360/1438 Touch screens
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/785 on or in relation to the windshield or windows
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/308 by overlaying the real scene, e.g. through a head-up display on the windscreen
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

Driver distraction is reduced by providing information only when necessary to assist the driver, and in a visually pleasing manner. Obstacles such as other vehicles, pedestrians, and road defects are detected based on analysis of image data from a forward-facing camera system. An internal camera images the driver to determine a line of sight. Navigational information, such as a line with an arrow, is displayed on a windshield so that it appears to overlay and follow the road along the line of sight. Brightness of the information may be adjusted to correct for lighting conditions, so that the overlay will appear brighter during daylight hours and dimmer during the night. A full augmented reality is modeled and navigational hints are provided accordingly, so that the navigational information indicates how to avoid obstacles by directing the driver around them. Obstacles also may be visually highlighted.

Claims

1. A method of reducing the distraction of a driver of a motor vehicle, the motor vehicle having a windshield in front of the driver, the method comprising:

receiving an image from a generally front facing camera system mounted on the motor vehicle, the image including data regarding a portion of a road surface generally in front of the motor vehicle and an ambient brightness;

receiving data pertaining to the position and orientation of the motor vehicle from at least one location sensing device;

computing a desired route between the position of the motor vehicle and a destination; and

displaying, on the windshield, a navigational image that is computed as a function of the desired route, the position and orientation of the motor vehicle, a curvature of the portion of the road surface, and a line of sight of the driver, the navigational image appearing, to the driver, to be superimposed on the road surface in front of the motor vehicle.

2. A method according to claim 1, wherein the navigational image has a brightness and a transparency that are calculated as a function of the ambient brightness.

3. A method according to claim 1, wherein receiving an image includes receiving an active infrared image or receiving a visible light spectrum image.

4. A method according to claim 1, wherein the line of sight of the driver is determined by analyzing an image of the driver'"'"'s face.

5. A method according to claim 1, wherein the motor vehicle is positioned on a road having an intersection, and the navigational image indicates that the driver should turn the motor vehicle at the intersection.

6. A method according to claim 1, further comprising displaying on the windshield a shape that appears, to the driver, to surround an object outside the motor vehicle, the object being one of:

a point of interest, a road defect, an elevated highway sign, a roadside traffic sign, a pedestrian, animal, or other road debris.

7. A method according to claim 6, wherein the displayed shape further comprises an iconic label that identifies the object.

8. A method according to claim 6, further comprising displaying, in a fixed position on the windshield, a textual image that conveys information relating to the highlighted object.

9. A method according to claim 6, wherein when the object is a road defect, the shape includes a column of light that appears to the driver to rise vertically from the road defect.

10. A method according to claim 6, wherein when the object is a pedestrian, animal, or road debris, the shape includes a shaded box that surrounds the detected object.

11. A method according to claim 6, wherein when the object is an elevated highway sign or a roadside traffic sign, the shape includes a shaded box that surrounds the sign.

12. A method according to claim 11, further comprising displaying the text of the sign in a fixed position on the windshield.

13. A method according to claim 1, further comprising:

projecting a light on the road surface in front of the motor vehicle, the light having a transmission pattern;

imaging a reflection from the road of the shined light, the reflection having a reflection pattern;

in a computing processor, determining a difference between the transmission pattern and the reflection pattern, the difference being indicative of a defect in the road surface; and

displaying, on the windshield, an image representing the defect, the displayed image being based on a line of sight of the driver so that the image appears, to the driver, to be superimposed on the road surface in front of the motor vehicle.

14. A method according to claim 13, wherein shining the light comprises shining light having infrared frequencies.

15. A method according to claim 1, further comprising:

using a histogram of orientated gradients to identify, in the received image, an object having a bodily symmetry of a life form; and

displaying, on the windshield, an image representative of the identified life form.

16. A method according to claim 1, further comprising:

determining that the received image includes a depiction of a road sign;

analyzing the image to determine a shape of the road sign;

if a meaning of the road sign cannot be determined from its detected shape, analyzing the image to determine any text present on a face of the road sign; and

displaying, on the windshield, an image relating to the road sign based on the line of sight of the driver.

17. A method according to claim 16, further comprising displaying, on a fixed position of the windshield, an image comprising the text of the sign.

18. A system for reducing the distraction of a driver of a motor vehicle, the motor vehicle having a windshield in front of the driver, the windshield having a given three-dimensional shape, the system comprising:

an imaging system configured to produce images on the windshield;

a first camera for imaging the interior of the motor vehicle, the first camera being oriented to capture images of the driver;

a second camera for imaging a road in front of the motor vehicle;

a touch screen for configuring the system;

a location sensing device for obtaining data that indicate the current position and orientation of the motor vehicle; and

a computing processor coupled to the imaging system, first camera, second camera, touch screen, and location sensing device, the computing processor being configured to;

(i) determine a line of sight of the driver based on images received from the first camera;

(ii) create navigational images based on data received from the second camera, the location sensing device, data received from the touch screen, and the line of sight;

(iii) transform the navigational images according to the given three-dimensional shape of the windshield, and

(iv) cause the imaging system to display the transformed images on the windshield so that the images appear, to the driver, to be superimposed on the road surface in front of the motor vehicle.

19. A system according to claim 18, wherein the second camera is configured to detect an ambient brightness, and the navigational image has a brightness and a transparency that are calculated as a function of the ambient brightness.

20. A system according to claim 18, wherein the at least one location sensing device is one of:

a global positioning system receiver, an inertial gyroscope, an accelerometer, or a camera.

21. A system according to claim 18, wherein the processor determines the line of sight by analyzing an image of the driver'"'"'s face.

22. A system according to claim 18, wherein the imaging system is further configured to display a shape that appears, to the driver, to surround an object outside the motor vehicle, the object being one of:

a point of interest, a road defect, an elevated highway sign, a roadside traffic sign, a pedestrian, animal, or other road debris.

23. A system according to claim 22, wherein the displayed shape further comprises a textual label or an iconic label that identifies the object.

24. A system according to claim 22, wherein the imaging system is further configured to display, in a fixed position on the windshield, a textual image that conveys information relating to the highlighted object.

25. A system according to claim 22, wherein when the object is a road defect, the shape includes a column of light that appears to the driver to rise vertically from the road defect.

26. A system according to claim 22, wherein when the object is a pedestrian, animal, or road debris, the shape includes a shaded box that surrounds the detected object.

27. A system according to claim 22, wherein when the object is an elevated highway sign or a roadside traffic sign, the shape includes a shaded box that surrounds the sign.

28. A system according to claim 27, wherein the imaging system is further configured to display the text of the sign in a fixed position on the windshield.

29. A system according to claim 18, further comprising:

a light having a transmission pattern aimed at the road surface in front of the motor vehicle;

wherein the second camera is configured to image a reflection from the road of the light, the reflection having a reflection pattern; and

the computer processor is further configured to

determine a difference between the transmission pattern and the reflection pattern, the difference being indicative of a defect in the road surface, and

cause the imaging system to display, on the windshield, an image representing the defect, the displayed image being based on a line of sight of the driver so that the image appears, to the driver, to be superimposed on the road surface in front of the motor vehicle.

30. A system according to claim 29, wherein the light is an infrared light.

31. A system according to claim 18, wherein the computer processor is further configured to use a histogram of orientated gradients to identify, in the received image, an object having a bodily symmetry of a life form, and to cause the imaging system to display, on the windshield, an image representative of the identified life form.

32. A system according to claim 18, wherein the computer processor is further configured to:

determine that the received image includes a depiction of a road sign;

analyze the image to determine a shape of the road sign;

if a meaning of the road sign cannot be determined from its detected shape, analyze the image to determine any text present on a face of the road sign; and

cause the imaging system to display, on the windshield, an image relating to the road sign based on the line of sight of the driver.

33. A system according to claim 32, wherein the imaging system displays, on a fixed position of the windshield, an image comprising the text of the sign.

34. A system according to claim 18, wherein the first camera is configured to capture video of one of the driver'"'"'s hands, the video comprising a succession of images, each image consisting of a plurality of pixels, and the computer processor is further configured to detect the motion of the one of the driver'"'"'s hands by calculating a motion gradient based on differences between the pixels of successive images of the video, and to issue commands to configure the system based on the direction of the detected motion gradient of the one of the driver'"'"'s hands relative to a coordinate system.

35. A method according to claim 34, wherein the system includes a menu function, a zoom function, and a rotate function, and wherein the direction of the detected motion gradient and a current state of the system together indicate whether to issue, to the system, a selection command, a menu navigation command, a zoom command, or a rotate command.