Augmented reality display system for evaluation and modification of neurological conditions, including visual processing and perception conditions

• US 10,332,315 B2
• Filed: 06/19/2017
• Issued: 06/25/2019
• Est. Priority Date: 06/20/2016
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Samec, Nicole Elizabeth, Harrises, Christopher M., Baerenrodt, Mark, Mangiat, Stephen Vincent, Robaina, Nastasja U., Wright, Adam Carl
• Primary Examiner(s): Buttram, Todd
• Application Number:
  US15/627,208
• Publication Number:
  US 20170365101A1
• Time in Patent Office: 736 Days
• CPC Class Codes: A61B 3/09 for testing accommodation
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/398 Electrooculography [EOG], e.g. detecting nystagmus; Electroretinography [ERG]
• CPC Class Codes: A61B 5/4082 Diagnosing or monitoring movement diseases, e.g. Parkinson, Huntington or Tourette
• CPC Class Codes: A61B 5/4088 Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
• CPC Class Codes: A61B 5/4094 Diagnosing or monitoring seizure diseases, e.g. epilepsy
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: G02B 2027/0105 Holograms with particular structures
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G16H 20/70 relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

In some embodiments, a display system comprising a head-mountable, augmented reality display is configured to perform a neurological analysis and to provide a perception aid based on an environmental trigger associated with the neurological condition. Performing the neurological analysis may include determining a reaction to a stimulus by receiving data from the one or more inwardly-directed sensors; and identifying a neurological condition associated with the reaction. In some embodiments, the perception aid may include a reminder, an alert, or virtual content that changes a property, e.g. a color, of a real object. The augmented reality display may be configured to display virtual content by outputting light with variable wavefront divergence, and to provide an accommodation-vergence mismatch of less than 0.5 diopters, including less than 0.25 diopters.

Claims

1. A display system comprising:

a head-mountable, augmented reality display configured to output light with variable wavefront divergence to display virtual content;

one or more inwardly-directed sensors;

one or more outwardly-directed sensors;

one or more processors; and

one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

performing a neurological analysis by;

determining a reaction to a stimulus by receiving data from the one or more inwardly-directed sensors; and

identifying a neurological condition associated with the reaction;

determining environmental triggers associated with the neurological condition;

monitoring an ambient environment with the one or more outwardly-directed sensors;

detecting a presence of an environmental trigger in the ambient environment; and

providing a perception aid based on the detected presence of the environmental trigger.

2. The display system of claim 1, wherein providing the perception aid comprises displaying virtual content.

3. The display system of claim 2, wherein the neurological condition comprises memory loss, wherein the perception aid comprises one or more of a reminder and an alert.

4. The display system of claim 2, wherein providing the perception aid comprises altering a perceived color of a real object.

5. The display system of claim 1, wherein the perception aids comprise sounds associated with the environmental trigger.

6. The display system of claim 1, wherein performing the neurological analysis is conducted automatically a plurality of times over a plurality of months, further comprising updating a user neurological profile based on performing the neurological analysis.

7. The display system of claim 1, wherein the display comprises a waveguide comprising diffractive optical elements configured to output the light by extracting the light out of the waveguide, wherein the waveguide is one of a stack of waveguides, wherein each of the stack waveguides is configured to output light with different wavefront divergence.

8. A display system comprising:

a head-mountable, augmented reality display configured to output light with variable wavefront divergence to display virtual content;

one or more inwardly-directed sensors;

one or more processors; and

one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

performing a neurological analysis by;

determining a reaction to a stimulus by receiving data from the one or more inwardly-directed sensors; and

identifying a neurological abnormality or deficiency associated with the reaction.

9. The display system of claim 8, wherein the operations further comprise causing the display system to display a perception aid.

10. The display system of claim 9, wherein the perception aid is selected based on one or more of the identified neurological abnormality or deficiency and the reaction.

11. The display system of claim 9, wherein the perception aid is selected based on a user profile.

12. The display system of claim 8, wherein performing the neurological analysis comprises providing the stimulus, wherein the stimulus comprises virtual content outputted by the display.

13. The display system of claim 8, wherein the stimulus comprises a virtual object moved from a distant depth plane to a near depth plane.

14. The display system of claim 8, wherein the neurological abnormality or deficiency is at least one of:

a visual processing deficiency and a memory deficiency.

15. The display system of claim 8, wherein the stimulus is a stimulus present in an ambient environment.

16. The display system of claim 8, wherein identifying the neurological abnormality or deficiency comprises generating a list of potential neurological conditions.

17. The display system of claim 8, wherein the operations further comprise:

automatically repeating the neurological analysis a plurality of times over a plurality of months,

wherein repeating the neurological analysis comprises updating the identified neurological abnormality or deficiency.

18. The display system of claim 8, wherein the operations further comprise transmitting the identified neurological abnormality or deficiency to a plurality of other display systems.

19. The display system of claim 18, wherein the operations comprise identifying the neurological abnormality or deficiency based upon a norm determined from a population of users of the other display systems.

20. The display system of claim 8, wherein the display is configured to output virtual content with an accommodation-vergence mismatch of less than 0.25 diopters.

21. The display system of claim 8, wherein the one or more inwardly-directed sensors comprises an electrode configured to measure electrical potentials.

22. A method performed by a display system comprising one or more processors, one or more inwardly-directed sensors, and a head-mounted display, the method comprising:

performing a neurological analysis by;

determining a user reaction to a stimulus by collecting data from the one or more inwardly-directed sensors; and

identifying a neurological abnormality or deficiency associated with the reaction.

23. The method of claim 22, further comprising displaying a perception aid.

24. The method of claim 22, wherein the perception aid is based on the identified neurological abnormality or deficiency, the reaction, or a user profile.

25. The method of claim 22, further comprising automatically repeating the neurological analysis a plurality of times over a plurality of months and updating the identified neurological abnormality or deficiency.