CONFIDENTIAL INFORMATION OCCLUSION USING AUGMENTED REALITY

• US 20170337352A1
• Filed: 05/23/2016
• Published: 11/23/2017
• Est. Priority Date: 05/23/2016
• Status: Abandoned Application

• Current Assignee: RPX Corporation
• Original Assignee: DAQRI, LLC
• Inventors: Williams, Nicole
• Application Number:
  US15/162,081
• Publication Number:
  US 20170337352A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06F 21/84 output devices, e.g. displays or monitors
• CPC Class Codes: G06F 2221/2111 Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06F 2221/2149 Restricted operating environment
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A device comprises a camera, a display, and a processor. The camera captures an image of a physical object. The device identifies the physical object based on the image of the physical object and determines that the physical object includes confidential content. The device displays an AR content in the display in response to determining that the physical object includes the confidential content. The AR content occludes only the confidential content.

Claims

1. A device comprising:

a camera configured to capture an image of a physical object;

a display;

a processor comprising an augmented reality (AR) occlusion application configured to;

identify the physical object based on the image of the physical object;

determine that the physical object includes confidential content based on an identification of the physical object; and

display an AR content in the display in response to determining that the physical object includes the confidential content, the AR content configured to occlude only the confidential content.

2. The device of claim 1, wherein the AR occlusion application is further configured to:

determine a confidentiality level of the confidential content;

generate an occlusion configuration based on the confidentiality level of the confidential content; and

generate the AR content based on the occlusion configuration.

3. The device of claim 1, wherein the AR occlusion application is further configured to:

identify a user of the device;

determine a permission level associated with an identification of the user;

determine a confidentiality level of the confidential content;

generate an occlusion configuration based on the confidentiality level of the confidential content exceeding the permission level of the user; and

generate the AR content based on the occlusion configuration, the AR content configured to occlude the physical object to the user.

4. The device of claim 1, wherein the AR occlusion application is further configured to:

identify a user of the device;

determine a permission level associated with an identification of the user;

determine a confidentiality level of the confidential content;

generate an occlusion configuration based on the confidentiality level and the permission level, the occlusion configuration identifying a portion of the physical object to be occluded to the user;

generate the AR content based on the occlusion configuration, the AR content configured to occlude only the confidential content to the user.

5. The device of claim 1, wherein the AR occlusion application is further configured to:

identify a user of the device;

determine a permission level associated with an identification of the user;

determine that the confidential content includes a document;

determine a confidentiality level associated with the document;

generate an occlusion configuration based on the confidentiality level and the permission level, the occlusion configuration identifying a portion of the document to be occluded to the user; and

generate the AR content based on the occlusion configuration, the AR content occluding only the portion of the document to the user.

6. The device of claim 1, wherein the AR occlusion application is further configured to:

identify a user of the device;

determine a permission level associated with an identification of the user; and

generate a replacement content consistent with the permission level of the user, the AR content including the replacement content, the replacement content configured to occlude and replace the confidential content.

7. The device of claim 1, wherein the AR content includes a notification of the confidential content to a user of the device.

8. The device of claim 1, wherein the AR content includes a portion of an image of a background of the physical object.

9. The device of claim 1, wherein the AR occlusion application is further configured to:

detect a removal of the display from a line of sight between an eye of the user and the confidential content; and

generate a notification message to a server in response to detecting the removal, the notification message identifying the user and the confidential content.

10. The device of claim 1, further comprising:

a first set of sensors configured to generate user-based sensor data related to a user of the device; and

a second set of sensors configured to generate ambient-based sensor data related to the device,

wherein the AR occlusion application is configured to;

determine a user-based context based on the user-based sensor data, and an ambient-based context based on the ambient-based sensor data;

generate an occlusion configuration based on the user-based context and the ambient-based context; and

generate the AR content based on the occlusion configuration.

11. A method comprising:

capturing an image of a physical object with a camera of a device;

identifying, using a hardware processor of the device, the physical object based on the image of the physical object;

determining, using the hardware processor, that the physical object includes confidential content based on an identification of the physical object; and

displaying an AR content in a display of the device in response to determining that the physical object includes the confidential content, the AR content configured to occlude only the confidential content.

12. The method of claim 11, further comprising:

determining a confidentiality level of the confidential content;

generating an occlusion configuration based on the confidentiality level of the confidential content; and

generating the AR content based on the occlusion configuration.

13. The method of claim 11, further comprising:

identifying a user of the device;

determining a permission level associated with an identification of the user;

determining a confidentiality level of the confidential content;

generating an occlusion configuration based on the confidentiality level of the confidential content exceeding the permission level of the user; and

generating the AR content based on the occlusion configuration, the AR content configured to occlude the physical object to the user.

14. The method of claim 11, further comprising:

identifying a user of the device;

determining a permission level associated with an identification of the user;

determining a confidentiality level of the confidential content;

generating an occlusion configuration based on the confidentiality level and the permission level, the occlusion configuration identifying a portion of the physical object to be occluded to the user; and

generating the AR content based on the occlusion configuration, the AR content configured to occlude only the confidential content to the user.

15. The method of claim 11, further comprising:

identifying a user of the device;

determining a permission level associated with an identification of the user;

determining that the confidential content includes a document;

determining a confidentiality level associated with the document;

generating an occlusion configuration based on the confidentiality level and the permission level, the occlusion configuration identifying a portion of the document to be occluded to the user; and

generating the AR content based on the occlusion configuration, the AR content configured to occlude only the portion of the document to the user.

16. The method of claim 11, further comprising:

identifying a user of the device;

determining a permission level associated with an identification of the user; and

generating a replacement content consistent with the permission level of the user, the AR content including the replacement content, the replacement content configured to occlude and replace the confidential content.

17. The method of claim 11, wherein the AR content includes a notification of the confidential content to a user of the device or a portion of an image of a background of the physical object.

18. The method of claim 11, further comprising:

detecting a removal of the display from a line of sight between an eye of the user and the confidential content; and

generating a notification message to a server in response to detecting the removal, the notification message identifying the user and the confidential content.

19. The method of claim 11, further comprising:

generating user-based sensor data related to a user of the device; and

generating ambient-based sensor data related to the device,

determining a user-based context based on the user-based sensor data, and an ambient-based context based on the ambient-based sensor data;

generating an occlusion configuration based on the user-based context and the ambient-based context; and

generating the AR content based on the occlusion configuration.

20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

capturing an image of a physical object with a camera of a device;

identifying the physical object based on the image of the physical object;

determining that the physical object includes confidential content based on an identification of the physical object; and

displaying an AR content in a display of the device in response to determining that the physical object includes the confidential content, the AR content configured to occlude only the confidential content.