HEAD-MOUNTED DISPLAY EYEBALL TRACKER INTEGRATED SYSTEM
1. A see-through optical system for vehicle safety, comprising:
- a transparent substrate having at least two major surfaces and edges;
an image projector configured to project an image made up of light waves of at least a second optical spectrum, said image projector being optically coupled to the substrate so as to inject the image into the substrate so that the image propagates within the substrate by internal reflection at the two major surfaces;
an array of partially-reflective surfaces carried by said substrate and obliquely angled relative to said major surfaces, said array of partially-reflective surfaces coupling out the image from the substrate so as to be viewable by the eye of a viewer;
an illumination source generating illumination light waves within a first optical spectrum distinct from said second optical spectrum;
a chromatically-selective reflector carried by the substrate and configured for coupling-in light waves generated by said illumination source reflected from the eye of the viewer;
a detector arrangement deployed to derive an image of at least part of the eye of the viewer from said light waves reflected from the eye of the viewer; and
a processing system associated with said detector arrangement and configured to process said image of at least part of the eye of the viewer to detect an open/closed state of the eye and to derive an indication of driver drowsiness.
Head-mounted display with an eye-tracking system and including a light-transmitting substrate (20) having two major surfaces and edges, optical means for coupling light into said substrate (20) by total internal reflection, partially-reflecting surfaces (22a-22c) carried by the substrate (20) that are not parallel with the major surfaces of the substrate (20), a near-infrared light source (78) and a display source (92) projecting within the photopic spectrum, wherein light from the light source (78) and light from the display source (92) are coupled into the substrate (20) by total internal reflection.
- 1. A see-through optical system for vehicle safety, comprising:
a transparent substrate having at least two major surfaces and edges; an image projector configured to project an image made up of light waves of at least a second optical spectrum, said image projector being optically coupled to the substrate so as to inject the image into the substrate so that the image propagates within the substrate by internal reflection at the two major surfaces; an array of partially-reflective surfaces carried by said substrate and obliquely angled relative to said major surfaces, said array of partially-reflective surfaces coupling out the image from the substrate so as to be viewable by the eye of a viewer; an illumination source generating illumination light waves within a first optical spectrum distinct from said second optical spectrum; a chromatically-selective reflector carried by the substrate and configured for coupling-in light waves generated by said illumination source reflected from the eye of the viewer; a detector arrangement deployed to derive an image of at least part of the eye of the viewer from said light waves reflected from the eye of the viewer; and a processing system associated with said detector arrangement and configured to process said image of at least part of the eye of the viewer to detect an open/closed state of the eye and to derive an indication of driver drowsiness.
The present invention relates to integrated head-mounted display (HMD) systems, and in particular, to systems that include two combined units: a head-mounted unit and an eyeball tracking unit.
The invention can be implemented to advantage in a large number of imaging applications, such as portable DVDs, cellular phones, mobile TV receivers, video games, portable media players or other mobile display devices.
One important application for compact optical elements, is in HMDs wherein an optical module serves both as an imaging lens and a combiner, in which a two-dimensional image source is imaged to infinity and reflected into an eye of an observer. The display source can be obtained directly from, e.g., a spatial light modulator (SLM) such as a cathode ray tube (CRT), a liquid crystal display (LCD), an organic light emitting diode array (OLED), a scanning source, or indirectly, by means of a relay lens, or an optical fiber bundle. The display source comprises an array of elements (pixels) imaged to infinity by a collimating lens and transmitted into the eye of a viewer by means of a reflecting or partially reflecting surface acting as a combiner for non-see-through and see-through applications, respectively. Typically, a conventional, free-space optical module is used for these purposes. As the desired field-of-view (FOV) of a HMD system increases, however, such a conventional optical module becomes larger, heavier and bulkier, and therefore, even for a moderate-performance device, is impractical. This is a major drawback for all kinds of displays and especially in head-mounted applications, wherein the system should necessarily be as light and compact as possible.
The strive for compactness has led to several different complex optical solutions, all of which, on the one hand, are still not sufficiently compact for most practical applications and, on the other hand, suffer major drawbacks in terms of manufacturability. Furthermore, the eye-motion-box (EMB) of the optical viewing angles resulting from these designs is usually very small, typically less than 8 mm. Hence, the performance of the optical system is very sensitive, even for small movements of the optical system relative to the eye of a viewer, and does not allow sufficient pupil motion for comfortable reading of text from such displays.
The teachings included in Publication Nos. WO01/95027, WO03/081320, WO2005/024485, WO2005/024491, WO2005/024969, WO2005/124427, WO2006/013565, WO2006/085309, WO2006/085310, WO2006/087709, WO2007/054928, WO2007/093983, WO2008/023367, WO2008/129539 and WO2008/149339, all in the name of Applicant, are herein incorporated by references.
The present invention facilitates the exploitation of very compact light-guide optical elements (LOEs) for, amongst other applications, HMDs. The invention allows for relatively wide FOVs together with relatively large EMB values. The resulting optical system offers a large, high-quality image, which also accommodates large movements of the eye. The optical system offered by the present invention is particularly advantageous because it is substantially more compact than the state-of-the-art implementations and yet, it can be readily incorporated even into optical systems having specialized configurations.
Another optical function which could prove to be useful for HMD designs is eyeball tracking, or sensing the direction the eyeball is looking at, relative to the direction of the head. A typical eye tracker will combine a miniature CCD camera and an infrared LED to illuminate the pupil. By measuring the changes in shape and position of the pupil, it is possible to perceive the direction in which the viewer'"'"'s eye is looking, with very reasonable accuracy once calibrated. Combining measurements of head position and eye position would solve the problems inherent in existing HMD technology, since the projected symbols and boresight could be slaved to the direction in which the viewer is looking, thus retaining existing human tracking behavior. It will be useful to combine the HMD and the eyeball tracker in the same optical module.
A broad object of the present invention is therefore to alleviate the drawbacks of prior art compact optical display devices and to provide other optical components and systems having improved performance, according to specific requirements.
In accordance with the invention there is therefore provided an optical system, comprising a light-transmitting substrate having at least two major surfaces and edges, at least one optical means for coupling light waves into the substrate by total internal reflection, at least two partially reflecting surfaces carried by the substrate wherein the partially reflecting surfaces are not parallel to the main surfaces of the substrate, at least one light source projecting light waves located within a first optical spectrum, and at least one display source projecting light waves located within a second optical spectrum, characterized in that the light waves from the light source and light waves from the display source are coupled into the substrate by total internal reflection.
The invention is described in connection with certain preferred embodiments, with reference to the following illustrative figures so that it may be more fully understood.
With specific reference to the figures in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention only, and are presented in the cause of providing what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention. The description taken with the drawings are to serve as direction to those skilled in the art as to how the several forms of the invention may be embodied in practice.
In the drawings:
As seen in
The trapped rays arrive at the reflecting surface from the second direction 30 after an odd number of reflections from the substrate surfaces 26 and 27, where the off-axis angle is a α′in=180°−αin and the incident angle between the trapped ray and the normal to the reflecting surface is as indicated in
As further illustrated in
A solution for this requirement that exploits the angular sensitivity of thin film coatings was previously proposed in the Publications referred-to above. The desired discrimination between the two incident directions can be achieved if one angle is significantly smaller than the other one. It is possible to provide a coating with very low reflectance at high incident angles, and a high reflectance for low incident angles. This property can be exploited to prevent undesired reflections and ghost images by eliminating the reflectance in one of the two directions. For example choosing βref˜25°, then it can be calculated that:
If a reflecting surface is now determined for which β′ref is not reflected but βref is, then the desired condition is achieved.
Referring now specifically to
In general, all the potential configurations of the LOEs considered in the Publications referred-to above, offer several important advantages over alternative compact optics for display applications, which include that:
1) the input display source can be located very close to the substrate, so that the overall optical system is compact and lightweight, offering an unparalleled form-factor;
2) in contrast to other compact display configurations, the LOE technology offers flexibility as to location of the input display source relative to the eyepiece. This flexibility, combined with the ability to locate the display source close to the expanding substrate, alleviates the need to use an off-axis optical configuration that is common to other display systems. In addition, since the input aperture of the LOE is much smaller than the active area of the output aperture, the numerical aperture of the collimating lens is much smaller than required for a comparable conventional imaging system. Consequently, a significantly more convenient optical system can be implemented and the many difficulties associated with off-axis optics and high numerical-aperture lenses, such as field or chromatic aberrations, can be compensated-for relatively easily and efficiently;
3) the reflectance coefficients of the selectively reflecting surfaces in the present invention, are essentially identical over the entire relevant spectrum. Hence, both monochromatic and polychromatic light sources may be used as display sources. The LOE has a negligible wavelength-dependence, ensuring high-quality color images with high resolutions;
4) since each point from the input image is transformed into a plane light wave that is reflected into the eye of a viewer from a large part of the reflecting array, the tolerances on the exact location of the eye can be significantly relaxed. As such, the viewer can see the entire FOV, and the EMB can be significantly larger than in other compact display configurations, and
5) since a large part of the intensity from the display source is coupled into the substrate, and since a large portion of this coupled energy is “recycled” and coupled out into an eye of a viewer, a display of comparatively high brightness can be achieved even with display sources having low-power consumption.
Some of the current HMD technology uses head position measurements to approximate line-of-sight, which may cause significant disparity between what a viewer is intended to look at, and what the viewer is actually looking at, as a result of at least ±20° eye movement. Therefore, it is necessary to integrate eyeball tracking capability into HMDs in some applications. Eyeball tracking is the process of measuring either the point of gaze or the motion of an eye relative to the head. An eyeball tracker is a device for measuring eye positions and eye movement. The most popular method for operating this device is by utilizing an optical method for measuring eye motion. Light from an external source, typically infrared, is reflected from the eye and sensed by a video camera, or some other specially designed optical sensors. The information is then analyzed to extract eye rotation from changes in reflections. Video-based eye trackers typically use corneal reflection and the center of the pupil as features to track over time. As a result, an HMD-eyeball tracker integrated system would be able to display stereoscopic virtual images as would a classical HMD, and also be able to track the ‘direction of gaze’ of a viewer.
In accordance with the present invention, it would be advantageous to physically combine the two optical units, the HMD and the eyeball tracker. Moreover, it would be beneficial to utilize the same LOE for projecting the light from the display source into a viewer'"'"'s eye, as described above, as well as for illuminating the eye with light from the eye tracker source, and to collect light which reflects from the eye into the detector. These two optical units should work properly without interfering with each other. To achieve this goal, two main characteristics of the combined optical system are exploited in the present invention: a separate partially reflecting surface or facet, dedicated for transferring light from a light source to the inspected eye and backwards, and a light having a wavelength substantially different from the photopic region utilized for the eye tracking.
In order to avoid ghost images, it is important that only one of the facets of the surfaces of the LOE (partially reflecting surface 22a in the shown Figure) will reflect light waves in the range of about λir. Otherwise, light waves from other surfaces will also be reflected from the eye and cause a noise on the detector 66, thus severely degrading the quality of the imaged eyeball. In addition, reflecting surface 22a should be transparent to the photopic range in the relevant angular spectra of the LOE, in the lower region, as well as the upper one.
Another problem that should be addressed is the possibility that a ghost image might also for a single surface. As illustrated in
The embodiments described above with regard to the reflecting surface 16 are examples of a method for coupling the input waves into the substrate. Input waves could, however, also be coupled into the substrate by other optical means, including, but not limited to, folding prisms, fiber optic bundles, diffraction gratings, and other solutions. In some of these methods, which were described in the Publications referred to above, the input surface of the LOE is not a reflecting surface but rather a transparent aperture. In these cases, it is required that the filter will be reflective to light waves having a wavelength of λtr, while still transparent to the photopic range.
The combination of a display source with a light source for illuminating an eye tracker utilizing light waves having a wavelength of λtr, is illustrated in
The two spectrally separated s-polarized input light waves 80 and 106 are now coupled through the reflecting surface 16 of the LOE, by total internal reflection. The light waves 80 are utilized for forming a virtual image projected by partially reflecting surfaces 22a-22e into a viewer'"'"'s eye 24, while the light waves 106 are utilized to illuminate the eye 24 for eye-tracking. The light waves 106 having the wavelength of λtr, are reflected from the eye 24 coupled again into the LOE by the partially reflecting surface 22a, coupled out from the LOE through reflecting surface 16, and as seen, pass again through the polarizing beam splitter 88 and through the dichroic surface 82, and coupled into the eyeball tracker 108, where they are focused onto the detector 110.
In all the configurations described so far, the optical reflecting 16 is utilized to couple light waves from the display source having wavelengths in the photopic range, as well as light waves from the eyeball tracker 108 having wavelength of λtr, into the LOE, by total internal reflection. There are, however, configurations wherein different coupling elements are utilized to couple separately the light waves from the display source and the light waves from the eyeball tracker. These configuration include, but are not limited to, two different elements wherein the first one is substantially transparent for the photopic range, while it is reflective for the spectral range wavelength of Air, and the second element is substantially transparent for the spectral range wavelength of λtr, while it is reflective for photopic range.
In all the configurations described so far, the two partially reflecting surfaces, 22a and 22b, are laterally separated. However, there are configurations wherein, for the sake of compactness or for enlarging the EMB of the optical system, it is required that the two surfaces will be adjacent to each other.
In all the hereinbefore described embodiments, the light waves from the eyeball tracker, as well as from the display source, are coupled into the substrate by the same coupling-in element. However, there are embodiments wherein, for the sake of simplicity or because of geometrical constraints, it is required that the eyeball tracker and the display source will be separated, and hence, the two different light waves will impinge on the substrate at two different locations.
In some of methods described in the prior art Publications referred to above, the input surface of the LOE is not a reflecting surface, but rather a transparent aperture. In these cases, it is required that the second aperture will be reflective to light waves having a wavelength of λtr while still being transparent to the photopic range.
So far, it was assumed that the main purpose of the eyeball tracker is to measure eye positions and eye movements. When, however, an eyelid of a viewer'"'"'s eye is closed, the pattern of the optical waves which are reflected from the eye, is significantly changed. The eyeball tracker can easily detect if a viewer'"'"'s eyelid is open or closed. Since the LOE-based eyeglasses illustrated in