WEARABLE IMAGE MANIPULATION AND CONTROL SYSTEM WITH CORRECTION FOR VISION DEFECTS AND AUGMENTATION OF VISION AND SENSING

• US 20180249151A1
• Filed: 04/25/2018
• Published: 08/30/2018
• Est. Priority Date: 03/17/2015
• Status: Active Grant

• Current Assignee: Raytrx, LLC
• Original Assignee: Raytrx, LLC
• Inventors: Freeman, Michael Hayes, Freeman, Richard C., Freeman, Mitchael C., Boss, Chad, Boss, Jordan
• Granted Patent: US 11,956,414 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/502 Headgear, e.g. helmet, spectacles
• CPC Class Codes: A61B 2560/0242 adapted to measure environmental factors, e.g. temperature, pollution
• CPC Class Codes: A61B 2562/0204 Acoustic sensors
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 2562/0223 Magnetic field sensors
• CPC Class Codes: A61B 3/0025 characterised by electronic signal processing, e.g. eye models
• CPC Class Codes: A61B 3/005 Constructional features of the display
• CPC Class Codes: A61B 3/0058 for multiple images
• CPC Class Codes: A61B 3/024 for determining the visual field, e.g. perimeter types
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 3/145 by video means
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: A61B 90/39 Markers, e.g. radio-opaque or breast lesions markers
• CPC Class Codes: A61F 9/00 Methods or devices for treatment of the eyes; Devices for putting-in contact lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand caps with means for protecting the eyes A42B1/018; visors for helmets A42B3/22; retractors A61B17/02; manipulators specially adapted for use in surgery A61B34/70; appliances to aid invalids to move about A61H3/00; exercisers for the eyes A61H5/00; eye baths A61H35/02; sunglasses or goggles having the same features as spectacles G02C
• CPC Class Codes: A61F 9/08 : Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
• CPC Class Codes: G02B 2027/0138 : comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 : comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 : Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 : Head mounted
• CPC Class Codes: G02B 27/0172 : characterised by optical features
• CPC Class Codes: G02C 11/10 : Electronic devices other than hearing aids
• CPC Class Codes: G02C 5/14 : Side-members
• CPC Class Codes: G06F 1/163 : Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 : Head tracking input arrangements
• CPC Class Codes: G06F 3/013 : Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 : Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 40/193 : Preprocessing; Feature extraction
• CPC Class Codes: G16H 20/30 : relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: H04N 13/189 : Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/332 : Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 13/366 : using viewer tracking

Abstract

A wearable image manipulation system comprising a camera input system, an image projection system, where the image projection system is capable of being worn by a user, and a processor in communication with the camera input system and the image projection system such that the processor is capable of receiving an image from the camera input system, modifying the image to produce a modified image, and displaying the modified image on the image projection system. The camera input system may comprise a contact lens with a camera mounted thereon. Additionally or alternately, the system may be capable of tracking a user'"'"'s eye movement to accurately capture where the user is looking with the camera input system.

Claims

1. A wearable image manipulation system comprising:

a camera input system;

an image projection system, where the image projection system is capable of being worn by a user; and

a processor in communication with the camera input system and the image projection system such that the processor is capable of receiving an image from the camera input system, modifying the image to produce a modified image, and displaying the modified image on the image projection system.

2. The system of claim 1 where the camera input system comprises a contact lens with a camera mounted thereon or at least partially embedded therein.

3. The system of claim 1 further comprising an eye tracking camera, where the eye tracking camera is capable of tracking movement of a user'"'"'s eye and the system is capable of adjusting the camera input system such that the image received by the processor from the camera input system is based on the user'"'"'s eye movement.

4. The system of claim 3 where the eye tracking camera is positioned such that it is pointed toward an eye of the user during use.

5. The system of claim 3 further comprising software utilized by the processor to adjust the displayed modified image based on eye tracking data from the eye tracking camera.

6. The system of claim 3 where modifying the image comprises correcting for epipolar geometry and line of sight.

7. The system of claim 1 further comprising headgear, where the headgear houses the image projection system.

8. The system of claim 7 where the headgear houses one or more of the following sensors:

motion sensors, degrees of freedom sensors, gesture recognition sensors, fiducial marker sensors, accelerometer sensors, infrared sensors, motion sensors, alert sensors, gyroscope technology and related sensors, positional tracking sensors, sound sensors, optical sensors, magnetic sensors, acoustic sensors, and/or inertial sensors.

9. The system of claim 7 where the headgear houses one or more of the following subsystems:

head and eye tracking;

hand and arm tracking;

body tracking; and

/or environment mapping interfaces.

10. The system of claim 7 where the headgear houses the processor.

11. The system of claim 7 where the headgear houses the camera input system.

12. The system of claim 7 where the headgear is capable of being worn on the head of the user and positioned over the user'"'"'s eyes and nose like glasses.

13. The system of claim 7 where the headgear is a helmet and the display is a face shield.

14. The system of claim 1 where the camera input system comprises a video camera, a still camera, a night-vision camera, an infrared camera, a thermal imaging camera, a 3D camera, or any combination thereof.

15. The system of claim 1 where the system is capable of recording images from the camera input system.

16. The system of claim 1 where the image projection system is capable of displaying at least part of the image and at least part of the modified image simultaneously.

17. The system of claim 1 where modifying the image comprises correcting for eye defects.