Method and apparatus to produce re-focusable vision with detecting re-focusing event from human eye

• US 9,699,433 B2
• Filed: 01/24/2014
• Issued: 07/04/2017
• Est. Priority Date: 01/24/2013
• Status: Active Grant

• Current Assignee: Yuchen Zhou
• Original Assignee: Yuchen Zhou
• Inventors: Zhou, Yuchen
• Primary Examiner(s): Villecco, John
• Application Number:
  US14/162,758
• Publication Number:
  US 20150215601A1
• Time in Patent Office: 1,257 Days
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 26/0825 the reflecting element being a flexible sheet or membrane, e.g. for varying the focus flexible mirrors for cosmetic use A45D42/24
• CPC Class Codes: G02B 26/0833 the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD G02B26/0825 takes precedence; micromechanical devices in general B81B
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/322 using varifocal lenses or mirrors
• CPC Class Codes: H04N 13/332 : Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 13/341 : using temporal multiplexing
• CPC Class Codes: H04N 13/344 : with head-mounted left-right displays
• CPC Class Codes: H04N 13/383 : for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

The current invention relates to the method to achieve re-focusable vision, including re-focusable stereo vision, with detecting the re-focusing event from a human eye. The method comprises utilizing optical and electrical sensing apparatus to detect the physiological change of viewer'"'"'s eye without viewer'"'"'s active participation or physical action, and retrieving the intended focus depth information of the viewer from such physiological information to update the visual impression perceived by the viewer that matches the intended focus depth, to achieve a re-focusable vision. The sensing apparatus includes both “glass” type and “contact-lens” type of see-through substrates that contain optical and electrical components that are necessary for obtaining the physiological information of viewer'"'"'s eye and controlling the visual impression that the viewer perceives.

Claims

1. A method to produce re-focusable vision for a viewer by detecting a desired focus depth of an eye of said viewer comprising the steps of:

providing an image display, an image generator, a vision re-focus sensor comprising at least one optical emitter and at least one optical detector, and a processor;

displaying a first image to said viewer by said image display;

a scanning probing light being emitted towards said eye by said at least one optical emitter with varying incident angles;

said probing light being reflected by cornea of said eye into at least one reflection light that is received by said at least one optical detector;

said at least one optical detector detecting eye information including shape change of said cornea with intensity and angle of said reflection light;

said eye information being utilized by said processor to calculate a desired focus depth of said eye;

said desired focus depth being used to produce one second image by said image generator;

and said image display displaying said second image to said viewer.

2. The method according to claim 1, wherein said image display is any of:

a screen of an electronic device, a screen for image projection, a projector to directly project image into human eye, a head-mount image viewing device.

3. The method according to claim 1, wherein said image display is a MEMS-based mirror array having at least one input light source for projecting image directly onto said viewer'"'"'s eye retina.

4. The method according to claim 1, wherein said image generator is any of:

a camera, a camcorder, a computerized image generator.

5. The method according to claim 1, wherein said image generator is a stereo vision image generation device.

6. The method according to claim 1, wherein said vision re-focus sensor is a see-through substrate comprises a first electronics that controls said at least one optical emitter and at least one optical detector.

7. The method according to claim 6, wherein said see-through substrate is in direct contact with said eye of said viewer and substantially covering the pupil of said eye.

8. The method according to claim 6, wherein said see-through substrate is embedded in a supporting frame disposed close to said eye of said viewer.

9. The method according to claim 1, wherein said probing light includes one of:

a directional beam, a scanning beam, a pulsed light beam.

10. The method according to claim 1, wherein said at least one optical detector detects property of said at least one reflection light including:

the dispersion of said at least one reflection light, duration of said at least one reflection light pulse, delay between said at least one reflection light pulses, said at least one reflection light direction.

11. The method according to claim 10, wherein said property is used in calculating said desired focus depth-by said processor.

12. The method according to claim 1, wherein an optical imaging system with a variable effective optical focus existing between said image display and said viewer'"'"'s eye, whereas said effective optical focus of said optical imaging system is adjusted according to said desired focus depth;

and

wherein said optical imaging system includes one of;

a mechanically positioned optical lens;

an optical component composed of electrically-controlled refractive index material; and

an micro-electro-mechanical-system (MEMS) actuated lens array.

13. The method according to claim 1, wherein said processor is a computing device embedded in a supporting frame disposed close to said eye of said viewer.

14. A method to realize human-machine interaction by detecting a desired focus depth of an eye of a viewer comprising the steps of:

providing an image display, an image capture device, a vision sensor comprising at least one optical emitter and at least one optical detector, a processor, and a communication means;

capturing a first image of the viewing space of said viewer with said image capture device;

a scanning probing light being emitted towards said eye by said at least one optical emitter with varying incident angles;

said probing light being reflected by cornea of said eye into at least one reflection light that is received by said at least one optical detector;

said at least one optical detector detecting eye information including shape change of said cornea and pupil position of said eye with intensity and angle of said reflection light;

said eye information being utilized by said processor to calculate a desired focus point of said eye in said viewing space and identifying at least one first object in said viewing space at said focus point from said first image;

displaying at least one imaginary object by said image display to said viewer exhibiting at least one property of said first object;

said viewer generating a command and said signal processor interpreting said command into a first information;

said first information being sent through said communication means;

and said first object performing at least one action after receiving said first information and following said command.

15. The method according to claim 14, wherein said image capture device includes one of:

a camera, a camcorder.

16. The method according to claim 14, wherein said communication means includes a wireless data network.

17. The method according to claim 14, wherein said command includes one of:

(i) eye gestures including one of;

time of stare by said viewer on said imaginary object;

movement of said eye of said viewer;

change of eye lips open width of said viewer; and

shift of said focus point in said viewing space;

(ii) body gestures including finger pointing; and

(iii) vocal commands.