Head mounted display with full field of view and high resolution

• US 6,529,331 B2
• Filed: 04/20/2001
• Issued: 03/04/2003
• Est. Priority Date: 04/20/2001
• Status: Active Grant

• Current Assignee: Razer (Asia-Pacific) Pte. Ltd. (Razer Incorporated)
• Original Assignee: Johns Hopkins University
• Inventors: Brown, Lawrence G., Shapiro, Marc D., Massof, Robert W.
• Primary Examiner(s): Epps, Georgia
• Assistant Examiner(s): Spector, David N.
• Application Number:
  US09/839,670
• Publication Number:
  US 20020181115A1
• Time in Patent Office: 683 Days
• Field of Search: 345/9, 359/630, 349/13
• US Class Current: 359/630
• CPC Class Codes: G02B 2027/0132 comprising binocular systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

A head mounted virtual environment display with high visual resolution and a full field of view is disclosed. The display uses an optical system in which the video displays and corresponding lenses are positioned tangent to hemispheres with centers located at the centers of rotation of a user'"'"'s eyes. Centering the optical system on the center of rotation of the eye allows both a full field of view and high fidelity visual resolution, without compromising visual resolution. A multi-panel video wall design for the video displays allows each lens facet to image a miniature flat panel display at optical infinity. The flat panel displays are centered on the optical axes of the lens facets. A parallel graphics architecture uses data from high resolution and accuracy head trackers and built-in eye trackers to generate high detail 3D models at high frame rates with minimal perceptible lag. Resolution for central vision is optimized with a roaming high level of detail window, and slip artifacts associated with rapid head movements are eliminated using freeze-frame.

Claims

1. A head mounted display comprising for each eye of a user of the display:

a plurality of lenses positioned relative to one another as though each of the lenses is tangent to a surface of a first sphere having a center that is located substantially at a center of rotation of an eye of a user of the head mounted display; and

a plurality of displays positioned relative to one another as though each of the displays is tangent to a surface of a second sphere having a radius larger than the first sphere'"'"'s radius and having a center that is located at the center of rotation of the eye of the user;

wherein each of the displays corresponds to at least one of the lenses, and is imaged by the corresponding lens.

2. A head mounted display as recited in claim 1, wherein the plurality of lenses are joined together to form a wide field array of lens facets.

3. A head mounted display as recited in claim 2, wherein the wide field array of lens facets forms a montage of display images on the eye'"'"'s retina.

4. A head mounted display as recited in claim 3, wherein the montage of display images formed on the retina is substantially seamless.

5. A head mounted display as recited in claim 3, wherein the montage of display images formed on the retina extends to substantially 85°

temporal field and substantially 50°

nasal field.

6. A head mounted display as recited in claim 1, wherein each of the lenses is a Fresnel lens.

7. A head mounted display as recited in claim 1, wherein the plurality of displays are paneled together to form a video wall providing a wide field of view.

8. A head mounted display as recited in claim 1, wherein each of the displays is a miniature flat panel video display.

9. A head mounted display as recited in claim 1, wherein each of the displays is a high resolution video display.

10. A head mounted display as recited in claim 1, wherein each of the displays is an emissive display.

11. A head mounted display as recited in claim 1, wherein each of the displays is a reflective display.

12. A head mounted display as recited in claim 11 further comprising:

a light source for illuminating the displays; and

a beam splitter positioned between the displays and the lenses.

13. A head mounted display as recited in claim 12, wherein the lenses are arranged in two rows, the displays are arranged in two rows, and the light source is positioned outside a field of view through the lenses.

14. A head mounted display as recited in claim 1 further comprising a glass plate mounted on each display for magnifying a display image.

15. A head mounted display as recited in claim 1 further comprising for each eye an eye tracker for measuring eye position.

16. A head mounted display as recited in claim 15 wherein each eyetracker includes

a beam splitter positioned between a central display of said plurality of displays and a corresponding central lens of said plurality of lenses;

and

a video camera mounted above the beam splitter.

17. A head mounted display as recited in claim 15 wherein the video camera is a charge coupled device video camera.

18. A head mounted display comprising:

a first array of lenses positioned tangential to a surface of a first imaginary sphere having a center that is located substantially at a center of rotation of a first eye of a user of the head mounted display;

a first wall of video displays positioned tangential to a surface of a second sphere having a radius larger than the first sphere'"'"'s radius and having a center that is located at the center of rotation of the first eye;

a second array of lenses positioned tangential to a surface of a third imaginary sphere having a center that is located substantially at a center of rotation of a second eye of the user of the display;

a second wall of video displays positioned tangential to a surface of a fourth sphere having a radius larger than the third sphere'"'"'s radius and having a center that is located at the center of rotation of the second eye;

each display included in the first wall of video displays having a corresponding lens in the first array of lenses, and being imaged by the corresponding lens; and

each display included in the second wall of video displays having a corresponding lens in the second array of lenses, and being imaged by the corresponding lens.

19. A head mounted display as recited in claim 18, wherein each of the first and second arrays of lens forms a montage of display images on the retinas of the first and second eyes, respectively.

20. A head mounted display as recited in claim 19, wherein the montages of display images formed on the retinas are substantially seamless.

21. A head mounted display as recited in claim 19, wherein each of the montages of display images formed on the retinas extends to substantially 85°

temporal field and substantially 50°

nasal field.

22. A head mounted display as recited in claim 18, wherein each of the lenses forming the first and second arrays is a Fresnel lens.

23. A head mounted display as recited in claim 18, wherein the first and second arrays of lenses are each a wide field array.

24. A head mounted display as recited in claim 18, wherein each of the first and second video walls of displays provides a wide field of view.

25. A head mounted display as recited in claim 18, wherein each of the displays forming the first and second video walls is a miniature flat panel video display.

26. A head mounted display as recited in claim 18, wherein each of the displays forming the first and second video walls is a high resolution video display.

27. A head mounted display as recited in claim 18, wherein each of the displays forming the first and second video walls is an emissive display.

28. A head mounted display as recited in claim 18, wherein each of the displays forming the first and second video walls is a reflective display.

29. A head mounted display as recited in claim 18 further comprising a glass plate mounted on each display for magnifying a display image.

30. A head mounted display as recited in claim 18 further comprising

a first eye tracker including

a first beam splitter positioned between a first central display of said first wall of video displays and a corresponding first central lens of said first array of lenses;

a first video camera mounted above the beam splitter; and

a second eye tracker including;

a second beam splitter positioned between a second central display of said second wall of video displays and a corresponding central lens of said second array of lenses; and

a second video camera mounted above the beam splitter.

31. A head mounted display as recited in claim 30 wherein the first and second video cameras are each charge coupled device video cameras.

32. A system for displaying a virtual environment comprising:

a head mounted display including;

a first array of lenses positioned tangential to a surface of a first imaginary sphere having a center that is located substantially at a center of rotation of a first eye of a user of the head mounted display;

a first wall of video displays for displaying the virtual environment, said first wall of video displays being positioned tangential to a surface of a second sphere having a radius larger than the first sphere'"'"'s radius and having a center that is located at the center of rotation of the first eye;

a second array of lenses positioned tangential to a surface of a third imaginary sphere having a center that is located substantially at a center of rotation of a second eye of the user of the head mounted display;

a second wall of video displays for displaying the virtual environment, said second wall of video displays being positioned tangential to a surface of a fourth sphere having a radius larger than the third sphere'"'"'s radius and having a center that is located at the center of rotation of the second eye, each display included in the first wall of video displays having a corresponding lens in the first array of lenses, and being imaged by the corresponding lens; and

each display included in the second wall of video displays having a corresponding lens in the second array of lenses, and being imaged by the corresponding lens;

a first tracking system for tracking positions of said first and second eyes;

a second tracking system for sensing and tracking position, orientation and movement of said user'"'"'s head relative to a predetermined coordinate system; and

a distributed computer system for driving said first and second walls of video displays to display the virtual environment.

33. A system for displaying a virtual environment as recited in claim 32 wherein the first tracking system includes:

a first beam splitter positioned between a first central display of said first wall of video displays and a corresponding first central lens of said first array of lenses;

a first video camera mounted above the beam splitter;

a second beam splitter positioned between a second central display of said second wall of video displays and a corresponding second central lens of said second array of lenses;

a second video camera mounted above the beam splitter; and

a computer for tracking eye position data received from said first and second video cameras.

34. A system for displaying a virtual environment as recited in claim 32 wherein the second tracking system includes:

a measuring unit to sense position, orientation and movement of a user'"'"'s head relative to a predetermined coordinate system; and

a computer for tracking head position, orientation and movement data received from said measuring unit.

35. A system for displaying a virtual environment as recited in claim 32 wherein the distributed computer system includes:

a first plurality of slave computers and drivers for driving said first wall of video displays to display the virtual environment;

a second plurality of slave computers and drivers for driving said second wall of video displays to display the virtual environment; and

a master computer for monitoring and controlling the operation of said first and second slave computers and drivers.

36. A system for displaying a virtual environment as recited in claim 35 wherein the distributed computer system implements a “

freeze frame”

feature when a user'"'"'s head is moving rapidly to avoid image slip.

37. A system for displaying a virtual environment as recited in claim 32 wherein the distributed computer system further includes first and second separate slaves computers for generating first and second high level of detail roaming windows inside respective central displays of said first and seconds wall of video displays for said first and second eyes.

38. A system for displaying a virtual environment as recited in claim 32 wherein each of the first and second arrays of lens forms a montage of display images on the retinas of the first and second eyes, respectively.

39. A system for displaying a virtual environment as recited in claim 32 wherein each of the lenses forming the first and second arrays is a Fresnel lens.

40. A system for displaying a virtual environment as recited in claim 32 wherein the first and second arrays of lenses are each a wide field array.

41. A system for displaying a virtual environment as recited in claim 32 wherein each of the first and second video walls of displays provides a wide field of view.

42. A system for displaying a virtual environment as recited in claim 32 wherein each of the displays forming the first and second video walls is a miniature flat panel video display.

43. A system for displaying a virtual environment as recited in claim 32 wherein each of the displays forming the first and second video walls is a high resolution video display.

44. A system for displaying a virtual environment as recited in claim 32 wherein each of the displays forming the first and second video walls is an emissive display.

45. A system for displaying a virtual environment as recited in claim 32 wherein each of the displays forming the first and second video walls is a reflective display.

46. A system for displaying a virtual environment as recited in claim 32 wherein the head mounted display further comprises a helmet for supporting and a cover for enclosing the first and second arrays of lenses and the first and second walls of video displays.

47. A system for displaying a virtual environment comprising:

a head mounted display including;

a first array of lenses positioned tangential to a surface of a first imaginary sphere having a center that is located substantially at a center of rotation of a first eye of a user of the head mounted display;

a first wall of video displays for displaying the virtual environment, said first wall of video displays being positioned tangential to a surface of a second sphere having a radius larger than the first sphere'"'"'s radius and having a center that is located at the center of rotation of the first eye;

a second array of lenses positioned tangential to a surface of a third imaginary sphere having a center that is located substantially at a center of rotation of a second eye of the user of the head mounted display;

a second wall of video displays for displaying the virtual environment, said second wall of video displays being positioned tangential to a surface of a fourth sphere having a radius larger than the third sphere'"'"'s radius and having a center that is located at the center of rotation of the second eye;

each display included in the first wall of video displays having a corresponding lens in the first array of lenses, and being imaged by the corresponding lens; and

each display included in the second wall of video displays having a corresponding lens in the second array of lenses, and being imaged by the corresponding lens;

an eye tracking system including;

a first beam splitter positioned between a first central display of said first wall of video displays and a corresponding first central lens of said first array of lenses, a first video camera mounted above the beam splitter;

a second beam splitter positioned between a second central display of said second wall of video displays and a corresponding second central lens of said second array of lenses;

a second video camera mounted above the beam splitter; and

a first computer for tracking eye position data received from said first and second video cameras;

a head tracking system including;

a measuring unit to sense position, orientation and movement of a user'"'"'s head relative to a predetermined coordinate system; and

a second computer for tracking head position, orientation and movement data received from said measuring unit; and

a control system including;

a plurality of third computers for driving said first wall of video displays to display the virtual environment;

a plurality of fourth computers for driving said second wall of video displays to display the virtual environment; and

a fifth computer for monitoring and controlling the operation of said first, second third and fourth computers.

48. A head mounted display comprising for each eye of a user of the display:

a plurality of lenses positioned relative to one another as though each of the lenses is tangent to a surface of a first sphere having a center that is located substantially at a center of rotation of an eye of a user of the head mounted display; and

a plurality of displays positioned relative to one another as though each of the displays is tangent to a surface of a second sphere having a radius larger than the first sphere'"'"'s radius and having a center that is located at the center of rotation of the eye of the user;

each of the displays corresponding to one of the lenses, whereby the displays are imaged with their corresponding lenses to form overlapping adjacent display images.

49. The head mounted display as recited in claim 48, wherein there is an overlap of at least 5 degrees between the adjacent display images.

50. The head mounted display as recited in claim 49, wherein the overlap is in a range of approximately 10 to 15 degrees.

51. The head mounted display as recited in claim 49, wherein the overlapping display images form a montage of display images on the eye'"'"'s retina.

52. A head mounted display as recited in claim 51, wherein the montage of display images formed on the retina is substantially seamless.

53. A head mounted display as recited in claim 51, wherein the montage of display images formed on the retina extends to substantially 85°

temporal field and substantially 50°

nasal field.