Smart contact lens with eye driven control system and method

• US 10,353,463 B2
• Filed: 02/26/2017
• Issued: 07/16/2019
• Est. Priority Date: 03/16/2016
• Status: Active Grant

• Current Assignee: Raayonnova LLC
• Original Assignee: Raayonnova LLC
• Inventors: Shtukater, Aleksandr
• Primary Examiner(s): Villecco, John
• Application Number:
  US15/442,674
• Publication Number:
  US 20170270636A1
• Time in Patent Office: 870 Days
• CPC Class Codes: G02B 2027/0123 comprising devices increasing the field of view
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G02C 7/04 Contact lenses for the eyes disinfection or sterilisation of contact lenses A61L12/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 3/20 Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/57 : Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/60 : Control of cameras or camera modules
• CPC Class Codes: H04N 23/62 : Control of parameters via user interfaces
• CPC Class Codes: H04N 23/675 : comprising setting of focusing regions

Abstract

A smart contact lens system is provided. The contact lens system comprises a contact lens substrate, an embedded display, and an onboard, embedded or off board, remote processor. The embedded display may form a part of the contact lens substrate. The embedded display is configured for outputting variety of UI components to the user. The processor is configured to shift at least a part of an image to a central position of the embedded display to bring the part of the image into focus on the embedded display and select UI component and optionally trigger it; thereby generating appropriate system event. Hence the entire system operation may be controlled with the eye(s).

Claims

1. A smart contact lens system (SCL) comprising:

a contact lens substrate movable in synchronism with an eye of a user;

an embedded display integrated into the said contact lens substrate;

a processor module configured to select a user interface (UI) component of one or more UI components on a user interface on the said embedded display and optionally trigger the selected UI component, wherein the user interface (UI) component selected is displayed in focus, at the time of selection, on the said embedded display.

2. The smart contact lens system of claim 1, further comprising:

an eye position or eye orientation tracking module.

3. The smart contact lens system of claim 1, further comprising:

an eye focus tracking module.

4. The smart contact lens system of claim 1, further comprising:

an external environment tracking module, comprising at least one of an image sensor or depth sensor or range sensor or infrared sensor, configured to track an environment around a user of the smart contact lens system or to track gestures and hand commands of the user of the smart contact lens system.

5. The smart contact lens system of claim 4, further comprising:

a the processor module configured to track changes in focus of the eye and identify a UI component of the one or more UI components being looked at and select a UI component of the one or more UI components being looked at.

6. The smart contact lens system of claim 4,

wherein the processor module tracks body gestures to either shift an image including the UI components on the embedded display to position part of the image including the UI components of interest at the center of the embedded display, in front of the retina of the eye or to select a UI component of the one or more UI components.

7. The smart contact lens system of claim 1, further comprising:

the processor module configured to determine a UI component of the one or more UI components to be selected or UI component of the one or more UI components to be selected and triggered.

8. The smart contact lens system of claim 1, further comprising:

the processor module configured to;

a. determine an anchor point;

b. track focus changes or eye orientation vector changes;

c. shift an image including the UI components on the embedded display to position part of the image including the UI components of interest at the center of the embedded display and in front of the retina of the eye;

d. determine that a criteria for UI component selection has been met and select a UI component of the one or more UI components, by generating a selection event.

9. The smart contact lens system of claim 1, further comprising:

an integrated, into the contact lens substrate, image capture device;

the embedded display configured to display the one or more UI components connoting start or stop or pause of image capturing or audio recording commands;

the processor module, configured to start image capturing on selection of a UI component of the one or more UI components;

the processor module configured to stop the image capturing on selection of a UI component of the one or more UI components, and

the processor module configured to pause image capturing on selection of a UI component of the one or more UI components.

10. The smart contact lens system of claim 1 further comprising a focus tracking module which tracks ciliary muscle of the eye and correlating a measured actual focus level of the eye against expected focus level for current depth of the environment.

11. The smart contact lens system of claim 1 further comprising a focus tracking module which identifies and tracks the shape of a crystalline lens of the eye.

13. A method of controlling the SCL of claim 1, with an eye, comprising the steps of:

depicting the one or more user interface (UI) components on the embedded display of said contact lens substrate; and

selecting or triggering a UI component of the one or more UI components.

14. The method of claim 13, further comprising:

a step of tracking the eye orientation;

computing an image shift factor;

shifting an image including the UI components on the said embedded display using the image shift factor.

15. The method of claim 13, further comprising:

a step of tracking eye focus to determine transition between a frame of reference or to determine an anchor point and a shift factor;

computing the shift factor by measuring the extent of change in the eye gaze'"'"'s direction.

16. The method of claim 13, further comprising:

a step of tracking an environment of a user to determine the environment around the user.

17. The method of claim 16, further comprising:

wherein the step of tracking environment of the user includes identifying the user'"'"'s gestures and the user'"'"'s hand commands.

18. The method of claim 13, further comprising:

a step of controlling an embedded image capture device, by selecting the a UI component of the one or more UI components on the embedded display.

19. The method of claim 18 where the one or more UI components consist of control components selected from a group consisting of start recording, end recording and pause control components.

20. The method of claim 13 where a focus change is detected by tracking ciliary muscle of the eye and correlating the a measured actual focus level of the eye against expected focus level for current depth of the environment.

21. The method of claim 13 where a focus change is detected by identifying or tracking a shape of a crystalline lens of the eye.

12. A smart contact lens system comprising:

a contact lens substrate movable in synchronism with an eye of a user;

an embedded display integrated into the said contact lens substrate; and

a processor module configured to shift an image including UI components on the embedded display to position part of the image including the UI components of interest to the center of the embedded display and select a UI component of one or more UI components on the UI on the embedded display and optionally trigger the selected UI component, wherein the selected UI component is displayed in focus, at the time of selection, on the embedded display.