Virtual monitor display technique for augmented reality environments

• US 9,726,896 B2
• Filed: 04/21/2016
• Issued: 08/08/2017
• Est. Priority Date: 04/21/2016
• Status: Active Grant

• Current Assignee: Von und ZU Liechtenstein Maximilian Ralph Peter
• Original Assignee: Von und ZU Liechtenstein Maximilian Ralph Peter
• Inventors: von und zu Liechtenstein, Maximilian Ralph Peter
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US15/134,398
• Publication Number:
  US 20160320625A1
• Time in Patent Office: 474 Days
• Field of Search: 345633, 345634, 351359
• CPC Class Codes: G02B 2027/011 comprising device for correcting geometrical aberrations, distortion
• CPC Class Codes: G02B 2027/0132 comprising binocular systems
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G02B 3/14 of variable focal length
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04N 13/00 Stereoscopic video systems; Multi-view video systems; Details thereof

Abstract

Methods, apparatus, and computer-readable media are described herein related to a virtual monitor display technique for augmented reality environments targeted at allowing a user of a display-enabled computing device to substitute their conventional hardware-realized display screen with a virtualized display screen of equal or better usability characteristics than the hardware-realized device. A virtual screen is rendered via a micro display in an augmented reality environment to a human user wearing a see through head mountable device. The system architecture makes use of liquid lens technology in order to adjust the relative position of the display as well as the focal distance by optical means only thereby ensuring that the virtual screen is rendered at the maximum resolution of the micro display at all times. The system architecture also comprises an occlusion matrix thereby ensuring that the virtual screen is free of ghosting. The system is configured in such a way that a display auto-hide function is triggered whenever certain threshold parameters are exceeded. The virtual monitor display technique described herein has been designed with the aim of reducing the effects simulator sickness during prolonged use of the virtual display.

Claims

1. An augmented reality type head mountable device comprising

a graphics system interface programmed to accept a video input of at least VGA standard, and

at least one micro display, wherein the micro display has a resolution at least equal to that of the said video input, and

at least one liquid lens, wherein the liquid lens is placed in an optical path of an image of the said micro display, and

a binocular occlusion matrix, and

a liquid lens controller, wherein the liquid lens controller is programmed to adjust tilt and focus settings on the at least one liquid lens in such a way as to shift a position of a projected image of the said plurality of the at least one micro display in any direction and wherein the liquid lens controller is programmed to interface with an occlusion matrix control circuit in such a way that said image of said micro display is rendered opaque.

2. The augmented reality type head mountable device of claim 1 wherein the liquid lens controller is programmed to adjust the optical properties of said at least one liquid lens in such a way that, in response to voluntary head or body movements made by a wearer of the head mountable device, said position of said projected image is adjusted in such a way as to anchor said projected image to a fixed position in the real world and wherein the voluntary nature of said movements is characterized either by an angular movement exceeding an angle of 15 degrees or a lateral movement exceeding a distance of 10 inches.

3. The augmented reality type head mountable device of claim 1 wherein the at least one micro display comprises a single monocular micro display.

4. The augmented-reality type head mountable device of claim 3 wherein the binocular occlusion matrix comprises at least two multi-cell liquid crystal shutters and wherein the said liquid lens controller is programmed to adjust the multi-cell liquid crystal shutters in such a way as to render opaque any cells of said shutters which are situated on an optical path behind the said projected image.

5. A method for enabling a wearer of a head mountable device to be presented with a virtualized computer monitor, the method comprising:

displaying a virtualized computer monitor wherein the virtualized computer monitor is interfaced with a video output source and wherein said video output source is not an integral part of the said head mountable device; and

in response to a movement of the head or of the body of said wearer;

using a liquid lens controller to shift a retinally projected image of the virtualized computer monitor in a direction selected from the group consisting of;

back, forth, left, right, up and down; and

interfacing said liquid lens controller with an occlusion matrix control circuit in such a way that said retinally projected image is rendered opaque.

6. The method of claim 5, wherein the video output source is a personal computer.

7. The method of claim 5, wherein the video output source is a television set.

8. The method of claim 5, wherein the video output source is a touch screen operated mobile computing device.

9. The method of claim 5, wherein in response to a head or body movement by the said wearer, adjusting at least one property of said liquid lens in such a way that the virtualized computer monitor appears as anchored to a real-world location all the time.

10. The method of claim 9, wherein the at least one optical property is focal length.

11. The method of claim 9, wherein the at least one optical property is tilt.

12. The method of claim 5 comprising a step of hiding said virtualized computer monitor in response to at least one movement by the said wearer.

13. The method of claim 12, wherein the at least one movement comprises a head movement in excess of a pre-set angle from a looking straight ahead attitude.

14. The method of claim 12 wherein the at least one movement comprises a change of the said wearer'"'"'s location away from a plurality of at least one pre-set locations in excess of a pre-set threshold distance.

15. The method of claim 5, wherein said movement is a voluntary movement and wherein the voluntary nature of said movement is characterized either by an angular movement exceeding an angle of 15 degrees or a lateral movement exceeding a distance of 10 inches.

16. The method of claim 5 comprising the step of enabling the said wearer to set at least one parameter relating to default offset in terms of X,Y or Z coordinates from a real-world anchor point of the said virtualized computer monitor.

17. The method of claim 5 comprising the step of enabling the wearer to set a default distance of said virtualized computer monitor in relation to the said wearer.

18. A nontransitory computer-readable medium on which are stored program instructions that, when executed by a processor, cause the processor to perform the operations of

controlling at least one liquid lens in such a way that, in response to a head or a body movement of a user of an augmented reality type head mountable device, a retinally projected image of a micro display is repositioned on an axis selected from the group consisting of:

X, Y and Z and;

interfacing a controller for said liquid lens with an occlusion matrix control circuit in such a way that said retinally projected image of said micro display is rendered opaque.

19. The nontransitory computer-readable medium of claim 18 wherein said head or body movement is a voluntary movement and wherein the voluntary nature of said movement is characterized either by an angular movement exceeding an angle of 15degrees or a lateral movement exceeding a distance of 10 inches.