Enhanced heads-up display system

• US 10,067,341 B1
• Filed: 02/04/2015
• Issued: 09/04/2018
• Est. Priority Date: 02/04/2014
• Status: Active Grant

• Current Assignee: Intelligent Technologies International, Inc.
• Original Assignee: Intelligent Technologies International, Inc.
• Inventors: Breed, David S, Shostak, Oleksandr T
• Primary Examiner(s): Kumar, Srilakshmi K
• Assistant Examiner(s): Wills-Burns, Chineyere
• Application Number:
  US14/614,105
• Time in Patent Office: 1,308 Days
• Field of Search: 345 8, 345173
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness

Abstract

Arrangement for a vehicle includes a heads-up display (HUD) system that displays content to an occupant of a vehicle, a processor that directs the heads-up display system to display content related to a lane being traveled by the vehicle, a scene data generating system that obtains information about an object in an environment around the vehicle, and an occupant monitoring system that monitors the occupant and detects movement of at least one part of the occupant. The processor is coupled to the scene data generating system and the occupant monitoring system and causes enhanced display of content by the heads-up display system additional to the content related to the lane being traveled by the vehicle, based on the information obtained by the scene data generating system and movement of the part of the occupant as detected by the occupant monitoring system.

Claims

1. An arrangement for a vehicle, comprising:

a heads-up display (HUD) system that displays content to an occupant of a vehicle in a position in front of the occupant when the occupant is present in the vehicle;

a scene data generating system that obtains information about an environment around the vehicle;

an occupant monitoring system that monitors the occupant when present in the vehicle and detects movement of at least one part of the occupant; and

a processor coupled to said HUD system, to said scene data generating system and to said occupant monitoring system,

said processor directing said HUD system to display first content related to a road or lane being traveled by the vehicle and to selectively initiate display of additional, second content that is different than and additional to the display of the first content related to the road or lane being traveled by the vehicle, while the first content is being displayed, whereby prior to initiation of the display of the additional, second content, the additional, second content is not displayed,

said processor directing said HUD system to display the additional, second content in one of a plurality of different locations relative to a location in which the first content is being displayed, the location in which the additional, second content is displayed being in a direction of movement of the at least one part of the occupant from an initial state, as detected by said occupant monitoring system, such that movement of the at least one part of the occupant in different directions from the initial state causes the additional, second content to be displayed by said HUD system in different locations,

said processor determining the additional, second content to display based on the information obtained by said scene data generating system and based on the same direction of movement of the at least one part of the occupant from the initial state, as detected by said occupant monitoring system, that determines in which one of the plurality of different locations the additional, second content is displayed by said HUD system such that movement of the at least one part of the occupant in different directions from the initial state causes different additional, second content to be displayed by said HUD system, and

whereby the additional, second content includes at least one of an object exterior of the vehicle, an edge of the road or lane on which the vehicle is traveling, an indication relating to traffic management structure for the road or lane on which the vehicle is traveling and a road sign for the road or lane on which the vehicle is traveling.

2. The arrangement of claim 1, wherein said scene data generating system comprises at least one imaging device, said processor including a pattern recognition algorithm that processes images obtained by said at least one imaging device into an identity of any objects in the images, the additional, second content being displayed based on the identity of any objects in the images.

3. The arrangement of claim 1, further comprising:

a position determining system that determines position of the vehicle; and

a map database containing traffic management signs having known positions,

said HUD system receiving data from said map database about position of traffic management signs and being configured to cause the display of one or more traffic management signs by said HUD system as the additional, second content based on position of the vehicle determined by said position determining system whereby different positions of the vehicle cause different additional, second content to be displayed by said HUD system.

4. The arrangement of claim 1, wherein said HUD system is configured to display the additional, second content on a discrete part of a windshield of the vehicle while the first content related to the road or lane being traveled by the vehicle is displayed on a different part of the windshield such that there is no overlap between the additional, second content and the first content related to the road or lane being traveled by the vehicle.

5. The arrangement of claim 4, wherein said scene data generating system is configured to be located relative to the vehicle in a position in which said scene data gathering system operatively obtains information about an object alongside and behind the vehicle and in the environment around the vehicle, whereby the additional, second content corresponds to a view by the occupant through a side-mounted rearview mirror of the vehicle.

6. The arrangement of claim 4, wherein said scene data generating system is configured to be located relative to the vehicle in a position in which said scene data generating system operatively obtains information about any objects behind the vehicle and in the environment around the vehicle, whereby the additional, second content corresponds to a view by the occupant through a compartment-mounted rearview mirror of the vehicle.

7. The arrangement of claim 1, wherein said scene data generating system is configured to obtain data about the presence of any objects exterior of the vehicle over time and in the environment around the vehicle and determine speed or velocity of any present objects from the obtained data, said HUD system being configured to display the determined speed or velocity in association with any present objects as the additional, second content.

8. The arrangement of claim 1, wherein said scene data generating system is configured to determine an identity of any objects in the environment around the vehicle, said HUD system being configured to display an icon representative of the determined identity of any of the objects as the additional, second content.

9. The arrangement of claim 1, wherein said processor is further configured to control said HUD system to display status of a variable traffic management structure for the road or lane being traveled by the vehicle and that have a plurality of different conditions as the additional, second content, the status of the variable traffic management structure including current condition of the variable traffic management structure and time remaining in the current condition of the variable traffic management structure.

10. The arrangement of claim 9, wherein at least one of the plurality of conditions of the variable traffic management structure requires compliance by the vehicle and said processor is configured to determine whether the vehicle can comply with the at least one of the plurality of conditions of the variable traffic management structure by analyzing the current condition of the variable traffic management structure, position and speed of the vehicle relative to the current condition and if said processor determines that the vehicle cannot comply with the current condition of the variable traffic management structure based on such analysis, said processor being configured to direct command signals to said HUD system in order to control said HUD system to provide a warning of potential non-compliance with the current condition.

11. The arrangement of claim 9, wherein the variable traffic management structure is controllable to change its condition, further comprising a communications system coupled to said processor and configured to transmit a communication to cause a change in operation of the variable traffic management structure, said processor being configured to control the variable traffic management structure based on the position and speed of the vehicle by causing transmission of communications to effect control of the variable traffic management structure using said communications system.

12. The arrangement of claim 1, wherein said occupant monitoring system is configured to monitor the head of the occupant and detect turning of the head of the occupant in a predetermined direction based on such monitoring and initiate display of the additional, second content based on detection of the turning of the head of the occupant in the predetermined direction, the head constituting the at least one part of the occupant and turning of the head constituting movement of the head from the initial state, whereby turning of the head of the occupant in different predetermined directions causes display of the different additional, second content at one of the plurality of different locations which is directly in front of the occupant after having turned their head in the predetermined direction.

13. The arrangement of claim 12, wherein the turning of the head of the occupant in the predetermined direction is turning of the occupant'"'"'s head to a side from a forward-facing orientation which is the initial state, said processor being configured to respond to the detection of the turning of the head of the occupant in the predetermined direction by said occupant monitoring system and initiate display of objects by said HUD system to one of the plurality of different locations which is located on the side of the vehicle to which the occupant turned their head.

14. The arrangement of claim 1, wherein said occupant monitoring system is configured to monitor a hand of the occupant and detect movement of the hand of the occupant in a predetermined direction based on such monitoring and initiate display of the additional, second content based on detection of the movement of the hand of the occupant in the predetermined direction, the hand constituting the at least one part of the occupant, whereby movements of the hand of the occupant in different predetermined directions cause display of the different additional, second content at one of the plurality of different locations.

15. The arrangement of claim 1, wherein said occupant monitoring system is further configured to monitor sound emanating from the occupant and detect a predetermined oral command by the occupant based on such monitoring and initiate display of the additional, second content based on detection of the predetermined oral command by the occupant, whereby different predetermined oral commands by the occupant cause display of the different additional, second content.

16. The arrangement of claim 1, wherein said processor directs said heads-up display to remove the additional, second content, when displayed, after detecting movement of the at least one part of the occupant in a direction opposite to the direction of movement of the at least one part of the occupant that caused the display of the additional, second content.

17. An arrangement for a vehicle, comprising:

a heads-up display (HUD) system that displays content to an occupant of a vehicle, said HUD system being configured to separately display content on a first and a second different, discrete portions of a windshield of the vehicle such that there is no overlap between the content on the first portion and the content on the second portion;

a scene data generating system that obtains information about an environment around the vehicle, said scene data generating system comprising at least one imaging device that images the environment around the vehicle or a map database containing data relating to roadway structures on a road or lane on which the vehicle is travelling and near the vehicle;

an occupant monitoring system that monitors the occupant and detects movement of at least one part of the occupant;

a processor coupled to said HUD system, to said scene data generating system, and to said occupant monitoring system,

said processor directing said HUD system to display first content related to the road or lane being traveled by the vehicle in the first portion of the windshield and to selectively cause display of additional, second content in the second portion of the windshield that is different than and additional to the display of the first content related to the road or lane being traveled by the vehicle, while the first content is being displayed whereby prior to initiation of the display of the additional, second content, the additional, second content is not displayed,

said processor directing said HUD system to display the additional, second content in one of a plurality of different locations relative to a location in which the first content is being displayed, the location in which the additional, second content is displayed being in a direction of movement of the at least one part of the occupant from an initial state, as detected by said occupant monitoring system, such that movement of the at least one part of the occupant in different directions from the initial state causes the additional, second content to be displayed by said HUD system in different locations,

said processor determining the additional, second content to display based on the information obtained by said scene data generating system and based on the same direction of movement of the at least one part of the occupant from the initial state as detected by said occupant monitoring system that determines in which one of the plurality of different locations the additional, second content is displayed by said HUD system such that movement of the at least one part of the occupant in different directions from the initial state causes different additional, second content to be displayed by said HUD system,

whereby the additional, second content includes at least one of an object exterior of the vehicle, an edge of the road or lane on which the vehicle is traveling, an indication relating to traffic management structure for the road or lane on which the vehicle is traveling and a road sign for the road or lane on which the vehicle is traveling.

18. The arrangement of claim 17, wherein said processor directs said heads-up display to remove the additional, second content, when displayed, after detecting movement of the at least one part of the occupant in a direction opposite to the direction of movement of the at least one part of the occupant that caused the display of the additional, second content.

19. A method for controlling a heads-up display in a vehicle, comprising:

displaying first content on a first portion of a windshield of the vehicle, the first content being related to a road or lane being traveled by the vehicle;

obtaining information about an environment around the vehicle using a scene data generating system; and

selectively displaying, using the heads-up display, additional, second content on a second portion of the windshield discrete from the first portion that is different than and additional to the display of the first content related to the road or lane being traveled by the vehicle, while the first content is being displayed and such that there is no overlap between the content being displayed in the first portion and the content being displayed in the second portion,

the step of selectively displaying the additional, second content on the second portion of the windshield comprising;

detecting movement of at least one part of the occupant using an occupant monitoring system;

directing the HUD system to display the additional, second content in one of a plurality of different locations relative to a location in which the first content is being displayed, the location in which the additional, second content is displayed being in a direction of movement of the at least one part of the occupant from an initial state, as detected by the occupant monitoring system, such that movement of the at least one part of the occupant in different directions from the initial state causes the additional, second content to be displayed by the HUD system in different locations,

determining, using a processor, the additional, second content to display based on the obtained information and based on the same direction of the detected movement of the at least one part of the occupant from the initial state that determines in which one of the plurality of different locations the additional, second content is displayed by the HUD system such that detected movement of the at least one part of the occupant in different directions from the initial state causes display of different additional, second content on the second portion of the windshield; and

initiating display of the additional, second content based on the direction of movement of the at least one part of the occupant from the initial state whereby prior to initiation of the display of the additional, second content based on the direction of movement of the at least one part of the occupant from the initial state, the additional, second content is not displayed,

whereby the additional, second content includes at least one of an object exterior of the vehicle, an edge of the road or lane on which the vehicle is traveling, an indication relating to traffic management structure for the road or lane on which the vehicle is traveling and a road sign for the road or lane on which the vehicle is traveling.

20. The method of claim 19, wherein the step of detecting movement of at least one part of the occupant comprises detecting turning of the head of the occupant in a predetermined direction to a side of the vehicle such that the head constitutes the at least one part of the occupant and turning of the head constituting movement of the head from the initial state whereby turning of the head of the occupant in different predetermined directions cause display of the different additional, second content, the step of displaying additional, second content on the second portion of the windshield comprises displaying on the second portion, a view by the occupant rearward and to the side of the vehicle to which the occupant turned their head, the second portion being one of the plurality of different locations which is directly in front of the occupant after having turned their head in the predetermined direction to a side of the vehicle.

21. The method of claim 19, wherein the step of selectively displaying additional, second content on the second portion of the windshield comprises only temporarily displaying the additional, second content on the second portion of the windshield.

22. The method of claim 19, further comprising rendering the second portion of the windshield temporarily opaque while the additional, second content is being displayed on the second portion of the windshield.

23. The method of claim 19, further comprising, when the additional, second content is displayed based on movement of the at least one part of the occupant in a predetermined direction from the initial state, directing the heads-up display to remove the displayed additional, second content when movement of the at least one part of the occupant in a direction opposite to the predetermined direction is detected by the occupant monitoring system.