Augmented reality spectroscopy

• US 10,558,047 B2
• Filed: 09/22/2017
• Issued: 02/11/2020
• Est. Priority Date: 09/22/2016
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Samec, Nicole Elizabeth, Robaina, Nastasja U., Kaehler, Adrian, Baerenrodt, Mark, Baerenrodt, Eric, Harrises, Christopher M., Powers, Tammy Sherri
• Primary Examiner(s): McLoone, Peter D
• Application Number:
  US15/713,420
• Publication Number:
  US 20180081179A1
• Time in Patent Office: 872 Days
• CPC Class Codes: A61B 2017/00216 with eye tracking or head position tracking control
• CPC Class Codes: A61B 2090/309 using white LEDs
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/1121 Determining geometric values, e.g. centre of rotation or angular range of movement
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/14555 specially adapted for the eye fundus
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/743 Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
• CPC Class Codes: A61B 5/744 Displaying an avatar, e.g. an animated cartoon character
• CPC Class Codes: G01J 2003/106 the two sources being alternating or selectable, e.g. in two ranges or line:continuum
• CPC Class Codes: G01J 2003/2826 Multispectral imaging, e.g. filter imaging
• CPC Class Codes: G01J 2003/425 Reflectance
• CPC Class Codes: G01J 3/0248 using a sighting port, e.g. camera or human eye
• CPC Class Codes: G01J 3/0256 Compact construction
• CPC Class Codes: G01J 3/0264 Electrical interface; User interface
• CPC Class Codes: G01J 3/10 Arrangements of light sources specially adapted for spectrometry or colorimetry
• CPC Class Codes: G01J 3/108 for measurement in the infrared range
• CPC Class Codes: G01J 3/2823 : Imaging spectrometer
• CPC Class Codes: G01J 3/42 : Absorption spectrometry; Double beam spectrometry; Flicker spectrometry; Reflection spectrometry beam switching arrangements G01J3/08
• CPC Class Codes: G01J 3/427 : Dual wavelengths spectrometry
• CPC Class Codes: G01N 21/359 : using near infrared light
• CPC Class Codes: G01N 33/483 : Physical analysis of biological material
• CPC Class Codes: G02B 2027/0125 : Field-of-view increase by wavefront division
• CPC Class Codes: G02B 2027/0138 : comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 : comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 : slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 : with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 : Head mounted
• CPC Class Codes: G02B 27/0172 : characterised by optical features
• CPC Class Codes: G02B 6/0076 : Stacked arrangements of multiple light guides of the same or different cross-sectional area
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 : Head tracking input arrangements
• CPC Class Codes: G06F 3/013 : Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 : Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

In some embodiments, a system comprises a head-mounted frame removably coupleable to the user'"'"'s head; one or more light sources coupled to the head-mounted frame and configured to emit light with at least two different wavelengths toward a target object in an irradiation field of view of the light sources; one or more electromagnetic radiation detectors coupled to the head-mounted member and configured to receive light reflected after encountering the target object; and a controller operatively coupled to the one or more light sources and detectors and configured to determine and display an output indicating the identity or property of the target object as determined by the light properties measured by the detectors in relation to the light properties emitted by the light sources.

Claims

1. A wearable spectroscopy system comprising:

a head-mounted display system removably coupleable to a user'"'"'s head;

at least one eye tracking camera configured to detect a gaze of the user;

one or more light sources coupled to the head-mounted display system and configured to emit light with at least two different wavelengths in an irradiated field of view, wherein the spectroscopy system is configured to determine a gaze direction of the user and to direct light emission along substantially the same direction as the determined gaze direction;

one or more electromagnetic radiation detectors coupled to the head-mounted member and configured to receive reflected light from a target object within the irradiated field of view;

a controller operatively coupled to the one or more light sources and the one or more electromagnetic radiation detectors, the controller configured to cause the one or more light sources to emit pulses of light while also causing the one or more electromagnetic radiation detectors to detect levels of light absorption related to the emitted pulses of light and reflected light from the target object;

an absorption database of light absorption properties of at least one material; and

a graphics processor unit to display an output to the user

wherein the head-mounted display system comprises a waveguide stack configured to output light with selectively variable levels of wavefront divergence.

2. The system of claim 1, wherein the waveguide stack comprises waveguides having optical power.

3. A wearable spectroscopy system comprising:

a head-mounted display system removably coupleable to a user'"'"'s head;

at least one eye tracking camera configured to detect a gaze of the user;

one or more light sources coupled to the head-mounted display system and configured to emit light with at least two different wavelengths in an irradiated field of view substantially in the same direction as the detected gaze;

one or more electromagnetic radiation detectors coupled to the head-mounted member and configured to receive reflected light from a target object within the irradiated field of view;

a controller operatively coupled to the one or more light sources and the one or more electromagnetic radiation detectors, the controller configured to cause the one or more light sources to emit pulses of light while also causing the one or more electromagnetic radiation detectors to detect levels of light absorption related to the emitted pulses of light and reflected light from the target object;

an absorption database of light absorption properties of at least one material;

a graphics processor unit to display an output to the user; and

an inertial measurement unit positional system, wherein the inertial measurement systems determines a pose orientation of the user'"'"'s head, and wherein the irradiated field of view is at least as wide as the pose orientation.

4. The system of claim 3, wherein the head-mounted display system comprises a waveguide stack configured to output light with selectively variable levels of wavefront divergence.

5. A wearable spectroscopy system comprising:

a head-mounted display system removably coupleable to a user'"'"'s head;

at least one eye tracking camera configured to detect a gaze of the user;

one or more light sources coupled to the head-mounted display system and configured to emit light with at least two different wavelengths in an irradiated field of view substantially in the same direction as the detected gaze;

one or more electromagnetic radiation detectors coupled to the head-mounted member and configured to receive reflected light from a target object within the irradiated field of view;

a controller operatively coupled to the one or more light sources and the one or more electromagnetic radiation detectors, the controller configured to cause the one or more light sources to emit pulses of light while also causing the one or more electromagnetic radiation detectors to detect levels of light absorption related to the emitted pulses of light and reflected light from the target object;

an absorption database of light absorption properties of at least one material; and

a graphics processor unit to display an output to the user,

wherein the head-mounted display system comprises a waveguide stack configured to output light with selectively variable levels of wavefront divergence.

6. The system of claim 5, wherein the one or more light sources comprises a plurality of light emitting diodes.

7. The system of claim 5, wherein the one or more light sources are configured to emit electromagnetic radiation at two or more predetermined wavelengths.

8. The system of claim 7, wherein the one or more light sources are configured to emit electromagnetic radiation at a first wavelength of about 660 nanometers, and a second wavelength of about 940 nanometers.

9. The system of claim 7, wherein the one or more light sources are configured to emit electromagnetic radiation at the two predetermined wavelengths sequentially.

10. The system of claim 7, wherein the one or more light sources are configured to emit electromagnetic radiation at the two predetermined wavelengths simultaneously.

11. The system of claim 5, wherein the controller is further configured to cause the one or more light sources to emit a cyclic pattern of a first wavelength on, then a second wavelength on, then both first and second wavelengths off, such that the one or more electromagnetic radiation detectors detect the first and second wavelengths separately.

12. The system of claim 5, wherein the controller is configured to calculate a ratio of first wavelength light measurement to second wavelength light measurement, and wherein the system is configured to convert the ratio to a tissue property based on the absorption database.

13. The system of claim 12, wherein the controller is operatively coupled to an optical element coupled to the head-mounted member and viewable by the user, wherein the system is configured to provide an output based on the tissue property, wherein the output is viewable by the user through the optical element.

14. The system of claim 5, wherein the one or more electromagnetic radiation detectors comprises a device selected from the group consisting of:

a photodiode, a photodetector.

15. The system of claim 5, wherein the one or more electromagnetic radiation detectors comprises a digital image sensor.

16. The system of claim 15, wherein the digital image sensor comprises a plurality of pixels, and wherein the controller is configured to automatically detect a subset of pixels which are receiving the light reflected after encountering a predetermined tissue property and to produce an output that displays the location of the subset of pixels indicating the predetermined tissue property.

17. The system of claim 5, wherein the head-mounted member further comprises an inertial measurement unit positional system.

18. The system of claim 17, wherein the inertial measurement systems determines a pose orientation of the user'"'"'s head.

19. The system of claim 18, wherein the irradiated field of view is at least as wide as the pose orientation.

20. The system of claim 5, wherein the waveguide stack comprises waveguides having optical power.

21. A wearable spectroscopy system comprising:

a head-mounted display system removably coupleable to a user'"'"'s head;

at least one eye tracking camera configured to detect a gaze of the user;

one or more light sources coupled to the head-mounted display system and configured to emit light with at least two different wavelengths in an irradiated field of view, wherein the spectroscopy system is configured to determine a gaze direction of the user and to direct light emission along substantially the same direction as the determined gaze direction;

one or more electromagnetic radiation detectors coupled to the head-mounted member and configured to receive reflected light from a target object within the irradiated field of view;

a controller operatively coupled to the one or more light sources and the one or more electromagnetic radiation detectors, the controller configured to cause the one or more light sources to emit pulses of light while also causing the one or more electromagnetic radiation detectors to detect levels of light absorption related to the emitted pulses of light and reflected light from the target object;

an absorption database of light absorption properties of at least one material;

a graphics processor unit to display an output to the user; and

an inertial measurement unit positional system, wherein the inertial measurement systems determines a pose orientation of the user'"'"'s head, and wherein the irradiated field of view is at least as wide as the pose orientation.