Virtual Monitor Display Technique for Augmented Reality Environments

• US 20160320625A1
• Filed: 04/21/2016
• Published: 11/03/2016
• Est. Priority Date: 04/21/2016
• Status: Active Grant

• Current Assignee: Von und ZU Liechtenstein Maximilian Ralph Peter
• Original Assignee: Von und ZU Liechtenstein Maximilian Ralph Peter
• Inventors: von und zu Liechtenstein, Maximilian Ralph Peter
• Granted Patent: US 9,726,896 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/011 comprising device for correcting geometrical aberrations, distortion
• CPC Class Codes: G02B 2027/0132 comprising binocular systems
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G02B 3/14 of variable focal length
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04N 13/00 Stereoscopic video systems; Multi-view video systems; Details thereof

Abstract

Methods, apparatus, and computer-readable media are described herein related to a virtual monitor display technique for augmented reality environments targeted at allowing a user of a display-enabled computing device to substitute their conventional hardware-realized display screen with a virtualized display screen of equal or better usability characteristics than the hardware-realized device. A virtual screen is rendered via a micro display in an augmented reality environment to a human user wearing a see through head mountable device. The system architecture makes use of liquid lens technology in order to adjust the relative position of the display as well as the focal distance by optical means only thereby ensuring that the virtual screen is rendered at the maximum resolution of the micro display at all times. The system architecture also comprises an occlusion matrix thereby ensuring that the virtual screen is free of ghosting. The system is configured in such a way that a display auto-hide function is triggered whenever certain threshold parameters are exceeded. The virtual monitor display technique described herein has been designed with the aim of reducing the effects simulator sickness during prolonged use of the virtual display.

Claims

1. Display-enabled computing system comprising

a see through head mountable device, wherein the head mountable device is adapted for displaying augmented reality content to a wearer of the said head mountable device, and

a plurality of at least one micro display, wherein the micro display is configured to display a screen image relating to a visual output of the said display-enabled computer system, and

a plurality of at least one liquid lens, wherein the liquid lens is placed in an optical path of the said screen image, and

a binocular occlusion mask, and

virtualizing means for virtualizing a hardware-realized screen device.

2. The display-enabled computing system of claim 1 wherein the virtualizing means is adapted to control the optical properties of the said liquid lens in such a way that an augmented reality image of the at least one micro display can change relative position.

3. The display-enabled computing system of claim 1 wherein the virtualizing means is adapted to control the optical properties of the said liquid lens in such a way that an augmented reality image of the at least one micro display can change relative position in response to head or body movements made by the said wearer of the head mountable device.

4. The display-enabled computing system of claim 1 wherein the virtualizing means is adapted to control the optical properties of the said liquid lens in such a way that an augmented reality image of the at least one micro display can change in terms of focal distance.

5. The display-enabled computing system of claim 1 wherein the virtualizing means is adapted to control the optical properties of the said liquid lens in such a way that an augmented reality image of the at least one micro display can change in terms of focal distance in response to head or body movements by the said wearer of the head mountable device.

6. The display-enabled computing system of claim 1 wherein the plurality of at least one micro display comprises a single micro display and wherein the plurality of the at least one liquid lens comprises a single liquid lens.

7. The display-enabled computing system of claim 1 wherein the binocular occlusion mask comprises a spatial light modulator.

8. The display-enabled computing system of claim 1 wherein the binocular occlusion mask comprises a multi-cell liquid crystal shutter.

9. A method for enabling a user of a see through head mountable device to be presented with a virtualized display screen, the method comprising:

displaying a mostly opaque virtualized display screen to a user of a display-enabled device by means of retinal projection; and

in response to an action by the said user, adjusting a plurality of at least one property of the virtualized display screen in such a way that the said user visually experiences the said virtualized display screen in a similar way to a stationary, non-virtualized display screen.

10. The method of claim 9, wherein the display-enabled device is a personal computer.

11. The method of claim 9, wherein the display-enabled device is a television set.

12. The method of claim 9, wherein the display-enabled device is a touch screen operated mobile computing device.

13. The method of claim 9, wherein the adjusting step comprises changing at least one optical property of a liquid lens.

14. The method of claim 13, wherein the at least one optical property is focus.

15. The method of claim 13, wherein the at least one optical property is tilt.

16. The method of claim 9 comprising a step of hiding the said virtualized display screen in response to at least one movement by the said user.

17. The method of claim 16, wherein the at least one movement comprises a head movement.

18. The method of claim 16 wherein the at least one movement comprises a change of the said user'"'"'s location.

19. The method of claim 16 comprising the step of quantifying the at least one movement by the said user and making the execution of the hiding step dependent on the result of the quantifying step exceeding a threshold.

20. The method of claim 9, wherein the adjusting step comprises rendering a binocular occlusion mask in such a way that the said virtualized display screen appears opaque to the said user.

21. The method of claim 9 comprising the step of enabling the said user to set at least one parameter relating to an initial offset of the said virtualized display screen.

22. The method of claim 9 comprising the step of enabling the said user to set at least one parameter relating to an initial focal distance of the said virtualized display screen.

23. The method of claim 9 comprising the step of enabling the said user to set at least one parameter relating to a threshold distance in such a way that the said virtualized display is hidden once the said threshold distance is exceeded.

24. A nontransitory computer-readable medium on which are stored program instructions that, when executed by a processor, cause the processor to perform the operations of

presenting a virtualized display screen to a wearer of a see through head mountable device, and

controlling a liquid lens in such a way that an image of the said virtual display screen is altered in such a way that the said wearer of a see through head mountable device can experience the subjective impression of the said virtual screen being a stationary real-world object.

25. The nontransitory computer-readable medium of claim 24 comprising an operation of controlling an occlusion matrix in such a way that an image of the said virtual display screen is occluded.