Method and apparatus for securely displaying private information using an augmented reality headset

• US 10,504,289 B2
• Filed: 05/01/2018
• Issued: 12/10/2019
• Est. Priority Date: 05/01/2018
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Yildiz, Yagiz C., Files, Jace W., Sanaullah, Abu S.
• Primary Examiner(s): McDowell, Jr., Maurice L.
• Application Number:
  US15/968,518
• Publication Number:
  US 20190340815A1
• Time in Patent Office: 588 Days
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06T 1/0021 Image watermarking
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G09G 2300/023 Display panel composed of stacked panels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2356/00 Detection of the display position w.r.t. other display screens
• CPC Class Codes: G09G 2358/00 Arrangements for display data security
• CPC Class Codes: G09G 5/12 Synchronisation between the display unit and other units, e.g. other display units, video-disc players
• CPC Class Codes: H04N 1/32144 embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
• CPC Class Codes: H04N 13/204 : using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals

Abstract

An information handling system operating an augmented reality assisted display security system may comprise a head mounted display device, a three-dimensional camera co-located with the head mounted display device capturing a three-dimensional image of a host display device displaying digital content, and a processor executing code instructions to determine a position and orientation of the head mounted display device with respect to a placeholder displayed by the host display device based on the three-dimensional image, generate a three-dimensional primitive of the head mounted display device and the placeholder based on the determined position and orientation, receive secure content, and render a three-dimensional image of the secure content within the placeholder primitive. The head mounted display device may display the three-dimensional image of secure content according to the three-dimensional primitive such that the rendered three-dimensional image of the secure content appears within the placeholder displayed on the host display device.

Claims

1. An information handling system operating an augmented reality assisted display security system comprising:

a head mounted display within a head mounted display device;

a three-dimensional camera co-located with the head mounted display capturing a three-dimensional image of a host display device displaying a placeholder for the display of a secure content and other digital content;

a processor executing code instructions of the augmented reality assisted display security system to;

determine a position and orientation of the head mounted display with respect to the placeholder of the host display device based on the three-dimensional image captured by the three-dimensional camera;

generate a three-dimensional primitive of the head mounted display and the placeholder of the host display device based on the determined position and orientation;

receive the secure content;

render a three-dimensional image of the secure content within the three-dimensional primitive at the location within the three-dimensional primitive of the placeholder; and

the head mounted display displaying the three-dimensional image of the secure content according to the three-dimensional primitive such that the rendered three-dimensional image of the secure content appears to be within the placeholder displayed on the host display device and appears to be coplanar with the host display device.

2. The information handling system of claim 1 further comprising:

the processor executing code instructions to;

identify a plurality of physical boundaries of the host video display within the three-dimensional image captured by the three-dimensional camera; and

determine the position and orientation of the head mounted display with respect to the placeholder further based on the identification of the plurality of physical boundaries and an indication received from the host display device of a location of the placeholder with respect to the plurality of physical boundaries.

3. The information handling system of claim 1 further comprising:

the three-dimensional image including a watermark displayed on the host display device;

the processor executing code instructions to;

identify the watermark within the three-dimensional image captured by the three-dimensional camera; and

determine the position and orientation of the head mounted display with respect to the placeholder further based on the identification of the watermark and an indication received from the host display device of a location of the placeholder with respect to the watermark.

4. The information handling system of claim 3, wherein the location of the watermark displayed on the host display device matches the location of the placeholder displayed on the host display device.

5. The information handling system of claim 1, wherein the secure content includes text information.

6. The information handling system of claim 1, wherein the secure content includes a digital image.

7. The information handling system of claim 1, wherein the placeholder for the display of the secure content and the digital content are displayed within the same software application of the host display device.

8. A method of routing the display of secure content to a head mounted display device via a host information handling system comprising:

capturing a three-dimensional image of a head-mounted display via a host three-dimensional camera co-located with a host video display;

receiving an identification of a secure content;

receiving an instruction to display digital content including the secure content via the host video display;

disallowing the display of the secure content via the host video display;

displaying via the host video display a placeholder for the display of the secure content and other digital content;

determining, via the processor, a position and orientation of the head mounted display device with respect to the host video display based on the three-dimensional image;

generating a primitive of the placeholder of the host display device from the perspective of the head mounted display based on the determined position and orientation of the head mounted display with respect to the host video display and a position of the placeholder with respect to the host video display; and

rendering a three-dimensional image of the secure content such that the three-dimensional image of the secure content appears to be within the primitive of the placeholder.

9. The method of claim 8 comprising:

transmitting the rendered three-dimensional image of the secure content to a head mounted display device for display via the head mounted display.

10. The method of claim 8 comprising:

displaying a watermark via the display device; and

transmitting to the head mounted display device an indication of a location of the placeholder with respect to the watermark, for placement of the primitive of the placeholder within a three-dimensional primitive at the head mounted display device and display of the rendered three-dimensional image of the secure content.

11. The method of claim 10, wherein the location of the watermark displayed on the host display device matches the location of the placeholder displayed on the host display device.

12. The method of claim 8, wherein the secure content includes text information.

13. The method of claim 8, wherein the secure content includes a digital image.

14. The method of claim 8, wherein the placeholder for the display of the secure content and the digital content are not displayed within the same software application of the host display device.

15. An information handling system operating an augmented reality assisted display security system comprising:

a head mounted display;

a three-dimensional camera co-located with the head mounted display capturing a three-dimensional image of a host display device displaying a placeholder for the display of a secure digital content item and other digital content;

a processor executing code instructions of the augmented reality assisted display security system to;

receive a rendered three-dimensional image of a secure content item;

determine a position and orientation of the head mounted display with respect to the placeholder based on an identification of the plurality of physical boundaries of the host display within the three-dimensional image captured by the three-dimensional camera, and an identification of the position and orientation of the placeholder with respect to the plurality of physical boundaries of the host display received from the host display;

generate a three-dimensional primitive of the head mounted display and the placeholder of the host display device based on the determined position and orientation;

place the received rendered three-dimensional image of the secure content item within the three-dimensional primitive of the placeholder; and

the head mounted display displaying the three-dimensional image of the secure content according to the three-dimensional primitive such that the rendered three-dimensional image of the secure content appears to be within the placeholder.

16. The information handling system of claim 15, wherein the placeholder indicates a location within the host video display where the secure content item has been removed from display.

17. The information handling system of claim 16, wherein the placeholder indicates a location within the host video display where the secure content item has been covered such that the secure content item is not viewable via the host video display.

18. The information handling system of claim 15, wherein the secure content includes text information.

19. The information handling system of claim 15, wherein the secure digital content includes a digital image.

20. The information handling system of claim 15, wherein the placeholder for the display of the secure content and the digital content are displayed within the same software application of the host display device.