UNCONSTRAINED SPATIALLY ALIGNED HEAD-UP DISPLAY

• US 20110052009A1
• Filed: 08/25/2010
• Published: 03/03/2011
• Est. Priority Date: 08/27/2009
• Status: Active Grant

• Current Assignee: Rafael Advanced Defense Systems Limited (Government of Israel)
• Original Assignee: Rafael Advanced Defense Systems Limited (Government of Israel)
• Inventors: BERKOVICH, Erez, LIFSCHUETZ, Gad
• Granted Patent: US 8,467,598 B2
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: G02B 2027/0132 comprising binocular systems
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays

Abstract

A system for providing a spatially aligned head-up display to a user viewing a scene through a display the system includes: an image sensing system including at least one image sensor deployed to sample images of the user'"'"'s face, the image sensing system generating data derived at least in part from the images; a display for displaying visible indications to the user superimposed on the scene; and a control system associated with the image sensing system and the display, the control system being configured to: process data from the image sensing system to determine a position and attitude of the user'"'"'s face, determine a viewing direction from at least one eye of the user to a point of interest within the scene, and actuate the display to display a visible indication aligned with the viewing direction that provides a spatially aligned head-up display to the user.

Claims

1. A system for providing a spatially aligned head-up display to a user viewing a scene through a display the system comprising:

(a) an image sensing system including at least one image sensor deployed to sample images of the user'"'"'s face, said image sensing system generating data derived at least in part from said images;

(b) a display for displaying visible indications to the user superimposed on the scene; and

(c) a control system associated with said image sensing system and said display, said control system including at least one processor, said control system being configured to;

(i) process data from said image sensing system to determine a position and attitude of the user'"'"'s face,

(ii) determine a viewing direction from at least one eye of the user to a point of interest within the scene, and

(iii) actuate said display to display a visible indication aligned with said viewing direction that provides a spatially aligned head-up display to the user.

2. The system of claim 1, wherein said control system is configured to process said data to derive a three-dimensional model of at least a part of the face of the user, said three-dimensional model being used in determining the position and attitude of the user'"'"'s face.

3. The system of claim 2, wherein said image sensing system includes at least two of said image sensors deployed to sample images of the user'"'"'s face from differing viewpoints to generate stereo image pairs, and wherein said three-dimensional model of at least part of the face of the user is derived at least in part from said stereo image pairs.

4. The system of claim 1, wherein said display is a non-collimated display associated with a surface of a window.

5. The system of claim 4, wherein said control system is configured to process said data to determine a position and attitude of a plurality of faces, and to actuate said display to display visible indications in a corresponding plurality of positions relative to the window, each of said visible indications being aligned with said viewing direction for a corresponding one of said faces.

6. The system of claim 1, wherein said control system is additionally configured to vary dynamically a visible indication on a single non-collimated display, said display providing a spatially aligned head-up display, such that a user looking binocularly through said display at an object of interest perceives a visible indicator that indicates the object of interest, the method comprising the steps of:

(a) calculating the user'"'"'s apparent binocular displacement on said display relative to the object of interest; and

(b) varying dynamically, based on said apparent binocular displacement, the width of said visible indication.

7. A method for providing a spatially aligned head-up display to a user viewing a scene through a display, the method comprising the steps of:

(a) sampling images of the user'"'"'s face;

(b) processing said images to determine a position and attitude of the user'"'"'s face;

(c) determining a viewing direction from at least one eye of the user to a point of interest within the scene; and

(d) displaying on a display a visible indication aligned with said viewing direction that provides a spatially aligned head-up display to the user.

8. The method of claim 7, wherein said processing derives a three-dimensional model of at least a part of the face of the user, said three-dimensional model being used in determining the position and attitude of the user'"'"'s face.

9. The method of claim 7, wherein said display is a non-collimated display associated with a surface of a window.

10. The method of claim 9, wherein said processing determines a position and attitude of a plurality of faces, and wherein said display displays visible indications in a corresponding plurality of positions relative to the window, each of said visible indications being aligned with said viewing direction for a corresponding one of said faces.

11. The method of claim 7, further comprising varying dynamically a visible indication on a single non-collimated display, said display providing a spatially aligned head-up display, such that a user looking binocularly through said display at an object of interest perceives a visible indicator that indicates the object of interest, the method comprising the steps of:

(a) calculating the user'"'"'s apparent binocular displacement on said display relative to the object of interest; and

(b) varying dynamically, based on said apparent binocular displacement, the width of said visible indication.

12. A system for providing a spatially aligned head-up display, the system comprising:

(a) a detection system for determining a position and attitude of each face of a plurality of users;

(b) a common display for displaying visible indications to the plurality of users viewing scenes through said common display; and

(c) a control system associated with said detection system and said common display, said control system including at least one processor, said control system being configured to;

(i) process data from said detection system to determine a position and attitude of each of the user'"'"'s faces,

(ii) determine a viewing direction from at least one eye of each of the users to an associated point of interest within a corresponding scene, and

(iii) actuate said common display to display a visible indication aligned with each of said viewing directions that provides a spatially aligned head-up display to the users.

13. The system of claim 12, wherein said common display is a non-collimated display associated with a surface of a window.

14. The system of claim 12, wherein said control system is configured to actuate a plurality of displays, wherein each of said displays provides a spatially aligned head-up display to one or more users.

15. A method for providing a spatially aligned head-up display, the method comprising the steps of:

(a) determining a position and attitude of each face of a plurality of users;

(b) determining a viewing direction from at least one eye of each of the users to an associated point of interest within a corresponding scene; and

(c) displaying on a common display a visible indication aligned with each of said viewing directions, which provides a spatially aligned head-up display to the users.

16. The method of claim 15, wherein said common display is a non-collimated display associated with a surface of a window.

17. The method of claim 15, wherein said control system is configured to actuate a plurality of displays, wherein each of said displays provides a spatially aligned head-up display to one or more users.

18. A method for varying dynamically a visible indication on a single non-collimated display, said display providing a spatially aligned head-up display, such that a user looking binocularly through said display at an object of interest perceives a visible indicator that indicates the object of interest, the method comprising the steps of:

(a) calculating the user'"'"'s apparent binocular displacement on said display relative to the object of interest; and

(b) varying dynamically, based on said apparent binocular displacement, the width of said visible indication.

19. The method of claim 18, wherein the height of said visible indication is varied dynamically based at least on a distance from the user to the object of interest such that the perceived height of said visible indicator relative to the object of interest is substantially maintained.

20. The method of claim 18, wherein the height of said visible indication is varied dynamically based on at least a distance from the user to said display such that the perceived height of said visible indicator relative to the user is substantially maintained.

21. The method of claim 18, wherein said visible indication is displayed such that, in the user'"'"'s perceived binocular view, the object of interest is framed by the visible indicator.

22. The method of claim 18, wherein said visible indication is displayed such that in the user'"'"'s perceived view a proportion of horizontal overlap of said visible indication with itself due to the apparent binocular displacement is maintained substantially constant.

23. The method of claim 18, wherein said visible indication includes a first visible indication portion displayed in a first position aligned with said viewing direction from a first eye of the user to the point of interest in the scene and a second visible indication portion displayed in a second position aligned with said viewing direction from a second eye of the user to the point of interest in the scene such that said first visible indication portion and said second visible indication portion combine in the perceived view of the user to indicate the point of interest.