CONTEXTUAL IMAGE STABILIZATION

• US 20140303687A1
• Filed: 11/05/2012
• Published: 10/09/2014
• Est. Priority Date: 11/04/2011
• Status: Active Grant

• Current Assignee: Massachusetts Eye And Ear Infirmary (Foundation of The Massachusetts Eye and EAR Infirmary Incorporated), US Department of Veterans Affairs (Government of the United States of America)
• Original Assignee: Massachusetts Eye And Ear Infirmary (Foundation of The Massachusetts Eye and EAR Infirmary Incorporated), US Department of Veterans Affairs (Government of the United States of America)
• Inventors: Wall, Conrad III, Rizzo, Joseph F. III, Xu, Minnan
• Granted Patent: US 9,389,431 B2
• Time in Patent Office: Days
• US Class Current: 607/54
• CPC Class Codes: A61F 9/08 Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
• CPC Class Codes: A61H 2201/0184 by raising an alarm
• CPC Class Codes: A61H 2201/165 Wearable interfaces
• CPC Class Codes: A61H 2201/5007 computer controlled
• CPC Class Codes: A61H 2201/501 connected to external computer devices or networks
• CPC Class Codes: A61H 2201/503 Inertia activation, i.e. activated by movement
• CPC Class Codes: A61H 2201/5048 Audio interfaces, e.g. voice or music controlled
• CPC Class Codes: A61H 2201/5069 Angle sensors
• CPC Class Codes: A61H 2201/5079 Velocity sensors
• CPC Class Codes: A61H 2201/5084 Acceleration sensors
• CPC Class Codes: A61H 2201/5092 Optical sensor
• CPC Class Codes: A61H 2201/5097 wireless
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: A61N 1/36046 of the eye
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G02C 5/001 specially adapted for particular purposes, not otherwise provided for or not fully classifiable according to technical characteristics, e.g. therapeutic glasses
• CPC Class Codes: G03B 2205/0007 Movement of one or more optical elements for control of motion blur imaging systems in general using optical elements for stabilisation of the position of the image G02B27/64
• CPC Class Codes: G03B 2217/005 Blur detection
• CPC Class Codes: G16H 20/70 relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: H04N 23/6812 : based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6815 : by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/683 : performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/687 : by shifting the lens or sensor position

Abstract

Image stabilization systems and methods include a detector configured to detect images, an actuator coupled to the detector, a sensor coupled to the detector and configured to detect motion of the detector, and an electronic processor in communication with the sensor and the actuator, where the electronic processor is configured to, for example: (a) receive information about motion of the detector from the sensor; (b) determine components of the motion of the detector, and associate a class with each of the determined components; (c) identify components to be compensated from among the determined components based on the associated classes; and (d) generate a control signal that causes the actuator to adjust a position of at least a portion of the detector to compensate for the identified components.

Claims

1. An image stabilization system, comprising:

a detector configured to detect images;

an actuator coupled to the detector;

a sensor coupled to the detector and configured to detect motion of the detector; and

an electronic processor in communication with the sensor and the actuator, wherein the electronic processor is configured to;

(a) receive information about motion of the detector from the sensor;

(b) determine components of the motion of the detector, and associate a class with each of the determined components;

(c) identify components to be compensated from among the determined components based on the associated classes; and

(d) generate a control signal that causes the actuator to adjust a position of at least a portion of the detector to compensate for the identified components.

2. The system of claim 1, wherein the sensor comprises at least one of an accelerometer and a gyroscope.

3. The system of any one of claims 1 and 2, wherein the detector comprises a camera.

4. The system of any one of claims 1 to 3, wherein the actuator comprises at least one of a mechanical actuator and a piezoelectric actuator.

5. The system of any one of claims 1 to 4, wherein the motion of the detector comprises at least one of motion along a linear direction and angular motion about an axis.

6. The system of any one of claims 1 to 5, wherein the actuator is configured to adjust the position by at least one of translating and rotating the at least a portion of the detector.

7. The system of any one of claims 1 to 6, further comprising a support structure to which the detector, actuator, sensor, and electronic processor are attached.

8. The system of claim 7, wherein the support structure comprises eyeglass frames.

9. The system of any one of claims 1 to 8, further comprising a receiver in communication with the detector and configured to:

(a) receive information from the detector, wherein the information is derived from one or more images detected by the detector; and

(b) transmit a representation of the received information to a human.

10. The system of claim 9, wherein the receiver comprises a visual implant positioned in an eye of the human.

11. The system of any one of claims 1 to 10, further comprising at least one additional sensor coupled to the detector, wherein each sensor is configured to detect linear motion along any of three orthogonal axes or angular motion about any one of the three orthogonal axes, and wherein each sensor detects a different motion of the detector.

12. The system of any one of claims 1 to 11, wherein the system is worn by a human, and wherein one of the associated classes comprises involuntary motion of the detector by the human.

13. The system of claim 12, wherein another one of the classes comprises voluntary motion of the detector by the human.

14. A method for image stabilization, the method comprising:

obtaining image information using a detector;

detecting motion of the detector while the image information is obtained;

determining components of the motion of the detector and associating a class with each of the determined components;

identifying components to be compensated from among the determined components based on the associated classes; and

adjusting a position of at least a portion of the detector to compensate for the identified components while the image information is obtained.

15. The method of claim 14, wherein obtaining image information comprises detecting one or more images.

16. The method of any one of claims 14 and 15, wherein detecting motion of the detector comprises at least one of detecting a linear displacement of the detector along a direction and detecting an angular displacement of the detector about an axis.

17. The method of claim 16, wherein detecting motion of the detector comprises at least one of detecting linear displacements of the detector along at least two orthogonal coordinate directions, and detecting angular displacements of the detector about at least two orthogonal axes.

18. The method of any one of claims 14 to 17, wherein one of the classes comprises involuntary motion of the detector by a wearer of the detector.

19. The method of any one of claims 14 to 18, wherein one of the classes comprises voluntary motion of the detector by the wearer.

20. The method of any one of claims 14 to 19, wherein adjusting the position of at least a portion of the detector to compensate for the identified components comprises directing an actuator coupled to the detector to:

(a) linearly displace the detector along a direction opposite to a linear displacement corresponding to at least one of the identified components;

or

(b) angularly displace the detector about an axis in a direction opposite to an angular displacement about the same axis corresponding to at least one of the identified components;

or

(c) both (a) and (b).

21. The method of any one of claims 14 to 20, wherein determining components of the motion of the detector comprises detecting a magnitude of a displacement of the detector relative to a reference position, and identifying components of the motion based upon the magnitude of the displacement.

22. The method of claim 21, further comprising associating a class with at least some of the determined components based upon the magnitude of the displacement.

23. The method of any one of claims 14 to 22, wherein determining components of the motion of the detector comprises detecting a magnitude of a displacement of the detector relative to a reference position, determining one or more frequencies associated with the displacement of the detector, and identifying components of the motion based upon the one or more frequencies.

24. The method of claim 23, further comprising associating a class with at least some of the determined components based upon the determined frequencies.

25. The method of any one of claims 14 to 24, further comprising transmitting the image information from the detector to a receiver.

26. The method of claim 25, wherein the detector is worn by a human, and wherein the receiver is a visual implant positioned in an eye of the human.

27. The method of claim 25, further comprising determining a velocity of the detector, and halting transmission of the image information to the receiver when a magnitude of the velocity exceeds a threshold value.

28. A method for transmitting image information, the method comprising:

detecting motion of a detector while the detector measures image information;

decomposing the detected motion into a plurality of components to identify a portion of the motion to be compensated based on at least one of a magnitude, a frequency, and a velocity of the motion; and

while the detector measures image information;

(a) moving the detector to offset the identified portion of the motion; and

(b) transmitting information derived from the image information to a receiver.

29. A method for image correction, the method comprising:

obtaining an image using a detector;

detecting motion of the detector while the image is obtained;

transmitting information about the motion of the detector to an electronic processor; and

using the electronic processor to correct the image based on the information about the motion of the detector.

30. The method of claim 29, wherein correcting the image comprises reducing artifacts in the image that arise from the motion of the detector.

31. The method of any one of claims 29 and 30, further comprising:

determining components of the motion of the detector and associating a class with each of the determined components;

identifying components to be compensated from among the determined components based on the associated classes; and

correcting the image by compensating for the effects of the identified components in the image.