Augmented reality systems and methods utilizing reflections

• US 10,430,985 B2
• Filed: 01/18/2017
• Issued: 10/01/2019
• Est. Priority Date: 03/14/2014
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Harrises, Christopher M., Samec, Nicole Elizabeth, Robaina, Nastasja U., Baerenrodt, Mark, Wright, Adam Carl, Kaehler, Adrian
• Primary Examiner(s): Mazumder, Saptarshi
• Application Number:
  US15/409,430
• Publication Number:
  US 20170206691A1
• Time in Patent Office: 986 Days
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A display system comprises a wearable display device for displaying augmented reality content. The display device comprises a display area comprising light redirecting features that are configured to direct light to a user. The display area is at least partially transparent and is configured to provide a view of an ambient environment through the display area. The display device is configured to determine that a reflection of the user is within the user'"'"'s field of view through the display area. After making this determination, augmented reality content is displayed in the display area with the augmented reality content augmenting the user'"'"'s view of the reflection. In some embodiments, the augmented reality content may overlie on the user'"'"'s view of the reflection, thereby allowing all or portions of the reflection to appear to be modified to provide a realistic view of the user with various modifications made to their appearance.

Claims

1. A display system, comprising:

a wearable display device comprising;

a display area comprising light redirecting features configured to direct light to a user, wherein the display area is at least partially transparent and is configured to provide a view of an ambient environment through the display area;

one or more hardware processors; and

a non-transitory computer-readable storage medium including computer-executable instructions that, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations comprising;

determining that a specular reflection is within the user'"'"'s field of view through the display area, wherein determining that the specular reflection is within the user'"'"'s field of view comprises;

receiving an indication of diffusion of detected reflected electromagnetic radiation or acoustical waves; and

determining that a specular reflection is present if the indicated diffusion is below a threshold.

2. The display system of claim 1, wherein the display area comprises a plurality of sets of light redirecting features configured to output a plurality of images on a plurality of depth planes, each set of light directing features configured to output light to form an image on one of the plurality of depth planes.

3. The display system of claim 2, wherein the display device comprises a stack of waveguides, at least some of the waveguides comprising the light redirecting features, wherein the waveguides are configured to in-couple light from a light source and to outcouple light to output the light to form the plurality of images, wherein each waveguide is configured to output images for a single corresponding depth plane.

4. The display system of claim 1, further comprising subsequently displaying augmented reality content in the display area, the augmented reality content augmenting the specular reflection.

5. The display system of claim 4, wherein the display device comprises a camera configured to detect one or both of user stare duration or pupil area, wherein subsequently displaying augmented reality content comprises modifying one or both of a type and timing of the augmented reality content depending on one or both of user stare duration or pupil area.

6. The display system of claim 4, wherein the operations further comprise determining an identity of the user, wherein subsequently displaying augmented reality content comprises displaying information unique to the user.

7. The display system of claim 4, wherein the operations further comprise determining a location of the user before subsequently displaying augmented reality content, wherein subsequently displaying augmented reality content comprises:

displaying augmented reality content specific to the location of the user.

8. The display system of claim 7, wherein displaying augmented reality content comprises displaying sales offers from a store or a portion of a store in which the user is situated.

9. The display system of claim 4, wherein the user is in the specular reflection, and wherein the augmented reality content comprises images of clothing overlaying the user'"'"'s reflection.

10. The display system of claim 4, wherein the augmented reality content comprises an alert to replenish recurring-use products.

11. The display system of claim 4, wherein the operations further comprise:

capturing one or more images of the user across an extended time interval;

storing the images or data derived from the images; and

performing a comparison between a current image with the stored images or data, wherein subsequently displaying augmented reality content comprises displaying results of the comparison.

12. The display system of claim 11, wherein displaying results of the comparison comprises displaying one or more earlier images of the user.

13. The display system of claim 11, wherein performing a comparison comprises performing a health analysis.

14. The display system of claim 13, wherein performing a health analysis comprises detecting changes in body shape, skin pallor, skin features.

15. The display system of claim 4, wherein the operations further comprise accessing data from one or more of a scale, a thermometer, a sphygmomanometer, and a heart rate monitor.

16. The display system of claim 15, wherein subsequently displaying augmented reality content comprises displaying one or both of the accessed data or information derived from the accessed data.

17. The display system of claim 1, wherein the user is in the specular reflection, and wherein the operations further comprise:

collecting information regarding the user'"'"'s activities; and

transmitting information regarding the user'"'"'s activities to a receiving station associated with a reflective surface providing the user'"'"'s reflection.

18. The display system of claim 1, wherein the operations further comprise:

determining an identity of the user;

accessing a purchase or browsing history of the user; and

generating sales offers based upon the purchase or browsing history.

19. The display system of claim 1, wherein determining that the specular reflection is within the user'"'"'s field of view further comprises analyzing an image of the specular reflection and determining whether directional features in the specular reflection are reversed.

20. The display system of claim 1, wherein the user is in the specular reflection and wherein the operations further comprise:

capturing an image of a reflection of the user; and

performing a health analysis based upon the captured image of the user.

21. The display system of claim 20, wherein the health analysis comprises a neurological test.

22. The display system of claim 20, wherein the health analysis comprises one or more of:

a cranial nerve test, a motor examination, an examination of observable body abnormalities, an inspection of muscle tone and bulk, functional testing of muscles, a test of a strength of individual muscle groups, a reflex test, a coordination test, and a gait test.

23. A method for displaying images, the method comprising:

providing a wearable display device comprising;

a display area comprising light redirecting features configured to direct light to a user, wherein the display area is at least partially transparent and is configured to provide a view of an ambient environment through the display area; and

determining that a specular reflection is within the user'"'"'s field of view through the display area, wherein determining that the specular reflection is within the user'"'"'s field of view comprises;

receiving an indication of diffusion of detected reflected electromagnetic radiation or acoustical waves; and

determining that a specular reflection is present if the diffusion is below a threshold.

24. The method of claim 23, wherein the user is in the specular reflection and further comprising:

capturing an image of a reflection of the user; and

performing a health analysis of the user based upon the captured image of the user.

25. The method of claim 23, further comprising attracting the user to a reflective surface by providing one or both of optical or auditory signals to the user.

26. The method of claim 23, further comprising subsequently displaying augmented reality content in the display area, the augmented reality content augmenting the specular reflection.

27. The method of claim 26, further comprising determining a location of the user before subsequently displaying augmented reality content, wherein subsequently displaying augmented reality content comprises:

displaying augmented reality content specific to the location of the user.

28. The method of claim 27, wherein displaying augmented reality content comprises displaying sales offers from a store or a portion of a store in which the user is situated.

29. The method of claim 27, wherein the user is in the specular reflection, and wherein the augmented reality content comprises images of clothing overlaying the user'"'"'s reflection.