Apparatus and method for enhancing human visual performance in a head worn video system

• US 9,516,283 B2
• Filed: 06/13/2013
• Issued: 12/06/2016
• Est. Priority Date: 06/13/2012
• Status: Active Grant

• Current Assignee: eSight Corporation
• Original Assignee: eSight Corporation
• Inventors: Hilkes, Robert, Jones, Frank, Rankin, Kevin
• Primary Examiner(s): Bailey, Frederick
• Assistant Examiner(s): Nawaz, Talha M
• Application Number:
  US13/916,806
• Publication Number:
  US 20130335543A1
• Time in Patent Office: 1,272 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61F 9/08 Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
• CPC Class Codes: A61H 2201/1604 Head
• CPC Class Codes: A61H 2201/165 Wearable interfaces
• CPC Class Codes: A61H 2201/5007 computer controlled
• CPC Class Codes: A61H 2201/501 connected to external computer devices or networks
• CPC Class Codes: A61H 2201/5092 Optical sensor
• CPC Class Codes: A61H 2201/5097 wireless
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G02B 2027/011 comprising device for correcting geometrical aberrations, distortion
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 3/12 Fluid-filled or evacuated lenses
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 13/332 Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control

Abstract

Visual impairment, or vision impairment, refers to the vision loss of an individual to such a degree as to require additional support for one or more aspects of their life. Such a significant limitation of visual capability may result from disease, trauma, congenital, and/or degenerative conditions that cannot be corrected by conventional means, such as refractive correction, such as eyeglasses or contact lenses, medication, or surgery. According to embodiments of the invention a method of augmenting a user'"'"'s sight is provided comprising obtaining an image of a scene using a camera carried by the individual, transmitting the obtained image to a processor, selecting an algorithm of a plurality of spectral, spatial, and temporal image modification algorithms to be applied to the image by the processor, modifying the image using the algorithm substantially in real time, and displaying the modified image on a display device worn by the individual.

Claims

1. A method comprising:

processing image data relating to an image with a microprocessor to separate the image data into at least two image data files of a plurality of image data files, each image data file relating to a predetermined portion of the image;

processing the image data within each image data file with the microprocessor to generate processed image data;

inserting image modification data into a header of each image data file, the image modification data relating to an image processing effect that will be automatically applied by a display to that portion of the image data relating to the image represented by that image data file when displaying that predetermined portion of the image to a predetermined user, wherein

the inserted image modification data comprises data relating to at least one of a location of retinal damage of an eye of the predetermined user, a vision loss relating to degradation in a visual field of view of the predetermined user, and a vision issue external to a physical eyeball of the predetermined user.

2. The method according to claim 1, wherein

each predetermined portion of the image is determined in dependence upon at least a characteristic of a display upon which the image data will be displayed to the predetermined user.

3. The method according to claim 1, wherein

processing the image data to generate processed image data comprises processing the image data in dependence upon a characteristic of a region of a display associated with the predetermined portion of the image to which the image data file relates upon which the processed image data will be displayed to the predetermined user.

4. The method according to claim 1, wherein

adding image modification data relating to an image processing effect comprises adding data relating to at least one of a spectral shift and a spatial shift to be applied to the image file data by the display.

5. The method according to claim 1, wherein

the predetermined portions of the image are either:

distinguishable in that the regions of the display on which they are displayed have different physical characteristics;

or

differentiated by different effective pixel characteristics which are applied to a display comprising an array of uniform display pixels.

6. The method according to claim 1, wherein

each predetermined portion of the image is determined in dependence upon at least a characteristic of an image sensor which captures the image data to be presented to the predetermined user.

7. The method according to claim 1, wherein

each predetermined portion of the image is determined in dependence upon at least a characteristic of vision of the predetermined user to whom the image data will be displayed.

8. The method according to claim 1, wherein

processing the image data to generate processed image data comprises processing the image data in dependence upon a characteristic of vision of the predetermined user relating to that portion of the user'"'"'s vision to which the image data file relates.

9. The method according to claim 1, wherein

adding image modification data relating to an image processing effect comprises adding data relating to an image processing algorithm to be applied to the image file data by the display.

10. The method according to claim 1, wherein

adding image modification data comprises adding a predetermined portion of a patient calibration file relating to visual perception of the predetermined user.

16. The non-transitory computer readable medium storing computer readable instructions according to claim 1, wherein the electronic device is selected from the group comprising a camera associated with the user, a digital set-top box, a portable electronic device, a television, a computer display, a multimedia player, a gaming console, a personal video recorder, and a digital set-top box.

11. A non-transitory computer readable medium storing computer readable instructions for execution by a microprocessor, the instructions when executed relating to a process within an electronic device comprising the microprocessor, the process comprising:

processing image data relating to an image with a microprocessor to separate the image data into at least two image data files of a plurality of image data files, each image data file relating to a predetermined portion of the image;

processing the image data within each image data file with the microprocessor to generate processed image data;

inserting image modification data into a header of each image data file, the image modification data relating to an image processing effect that will be automatically applied by a display to that portion of the image data relating to the image represented by that image data file when displaying that predetermined portion of the image to a predetermined user, wherein

the inserted image modification data comprises data relating to at least one of a location of retinal damage of an eye of the predetermined user, a vision loss relating to degradation in a visual field of view of the predetermined user, and a vision issue external to a physical eyeball of the predetermined user.

12. A non-transitory computer readable medium storing computer readable instructions according to claim 11, wherein

each predetermined portion of the image is determined in dependence upon at least one of:

a characteristic of a display upon which the image data will be displayed to the predetermined user;

a characteristic of the image sensor which captures the image data to be presented to the predetermined user; and

a characteristic of vision of the predetermined user to whom the image data will be displayed.

13. A non-transitory computer readable medium storing computer readable instructions according to claim 11, wherein,

processing the image data to generate processed image data comprising processing the image data in dependence upon at least one of:

a characteristic of a region of a display associated with the image data file upon which the processed image data will be displayed to the predetermined user; and

a characteristic of vision of a user relating to that portion of the predetermined user'"'"'s vision to which the image data file relates.

14. A non-transitory computer readable medium storing computer readable instructions according to claim 11, wherein;

adding image modification data relating to an image processing effect comprises adding data relating to at least one of a spectral shift and a spatial shift to be applied to the image file data by the display.

15. A non-transitory computer readable medium storing computer readable instructions according to claim 11, wherein,

the predetermined portions of the image are either:

distinguishable in that the regions of the display on which they are displayed have different physical characteristics, or

the predetermined portions of the image are differentiated by different effective pixel characteristics which are applied to a display comprising an array of uniform display pixels.

17. A non-transitory computer readable medium storing computer readable instructions according to claim 11, wherein

the display is head-mounted display.

18. The non-transitory computer readable medium storing computer readable instructions according to claim 11, wherein the electronic device is selected from the group comprising a camera associated with the user, a digital set-top box, a portable electronic device, a television, a computer display, a multimedia player, a gaming console, a personal video recorder, and a digital set-top box.

19. The method according to claim 11, wherein

adding image modification data comprises adding a predetermined portion of a patient calibration file relating to visual perception of the predetermined user.