SMART CONTACT LENS SYSTEM WITH COGNITIVE ANALYSIS AND AID

• US 20190332168A1
• Filed: 04/25/2018
• Published: 10/31/2019
• Est. Priority Date: 04/25/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Weldemariam, Komminist, Kwatra, Shikhar, Pandit, Sushain, Diriye, Abdigani, Kinai, Andrew
• Granted Patent: US 10,963,045 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 3/112 for measuring diameter of pupils
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/145 Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissuemeasuring of blood pressure or blood flow A61B5/02; non-radiation detecting or locating of foreign bodies in blood A61B5/06
• CPC Class Codes: A61B 5/14532 for measuring glucose, e.g. by tissue impedance measurement
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/6821 Eye
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G02C 7/04 Contact lenses for the eyes disinfection or sterilisation of contact lenses A61L12/00
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

A smart lens system may include a hardware processor coupled to a smart contact lens, which may include a microprocessor, an image capturing sensor, and a wireless communication interface. The smart lens system may determine reading factors associated with a user with respect to the user reading content item, based on data related to the user'"'"'s physiological movements captured by the smart contact lens. The reading factors may include a user'"'"'s cognitive state determined at least from analyzing data representing the user'"'"'s physiological movements. Responsive to determining that the user'"'"'s cognitive state is above a threshold value, the smart lens system may generate an assistive action associated with the content item and transmit a signal to at least one assistive device to perform the assistive action. The assistive device performs the assistive action responsive to receiving the signal.

Claims

1. A method executed by at least one hardware processor communicating with a smart contact lens comprising at least a microprocessor, an image capturing lens, and a wireless communication interface, the method comprising:

receiving from the smart contact lens, data captured by the smart contact lens, the data associated with a user wearing the smart lens reading content item from a data source, the data comprising at least data representing eye movements of the user while reading the content item;

based on the data, determining reading factors associated with the user with respect to the content item, the reading factors comprising at least a user'"'"'s cognitive state determined at least from analyzing the data representing eye movements;

responsive to determining that the user'"'"'s cognitive state is above a threshold value, generating an assistive action associated with the content item;

transmitting a signal to at least on assistive device to perform the assistive action, wherein the assistive device performs the assistive action responsive to receiving the signal.

2. The method of claim 1, wherein the data further comprises images of the user'"'"'s hand movement, facial expression, head movement while reading the content item, glucose level, and pupil dilation.

3. The method of claim 1, wherein the reading factors further comprises user context, user profile and characteristics associated with the content item.

4. The method of claim 1, wherein the assistive action comprises at least one of highlighting a portion of the content item on a display of the data source from which the user is reading the content item, summarizing the content item, triggering a sound-controlled intelligent personal assistant service.

5. The method of claim 1, wherein the reading factors further comprises user context, wherein the user context comprises at least one of a degree of engagement between the user and the content item, an engagement pattern between the user and the content item, a degree of the user'"'"'s confusion state with respect to the reading of the content item, and the user'"'"'s reading environment context.

6. The method of claim 1, further comprising determining an area in the content item where the user has left reading to generate the assistive action.

7. The method of claim 1, further comprising storing the data for temporary duration for a predetermined time window.

8. A smart lens system comprising:

at least one hardware processor communicatively coupled via a communication network to a smart contact lens comprising at least a microprocessor, an image capturing sensor, and a wireless communication interface, the at least one hardware processor operable to at least;

receive from the smart contact lens data captured by the smart contact lens, the data associated with a user wearing the smart lens reading content item from a data source, the data comprising at least data representing eye movements of the user while reading the content item;

based on the data, determine reading factors associated with the user with respect to the content item, the reading factors comprising at least a user'"'"'s cognitive state determined at least from analyzing the data representing eye movements;

responsive to determining that the user'"'"'s cognitive state is above a threshold value, generate an assistive action associated with the content item; and

transmit a signal to at least one assistive device to perform the assistive action, wherein the assistive device performs the assistive action responsive to receiving the signal.

9. The system of claim 8, wherein the data further comprises images of the user'"'"'s hand movement, facial expression, head movement, glucose level, and pupil dilation while viewing the content item.

10. The system of claim 8, wherein the reading factors further comprises user context, user profile and characteristics associated with the content item.

11. The system of claim 8, wherein the assistive action comprises at least one of highlighting a portion of the content item on a display of the data source from which the user is reading the content item, summarizing the content item, triggering a sound-controlled intelligent personal assistant service.

12. The system of claim 8, wherein the reading factors further comprises user context, wherein the user context comprises at least one of a degree of engagement between the user and the content item, an engagement pattern between the user and the content item, a degree of the user'"'"'s confusion state with respect to the reading of the content item, and the user'"'"'s reading environment context.

13. The system of claim 8, further comprising determining an area in the content item where the user has left reading to generate the assistive action.

14. The system of claim 8, further comprising storing the data for temporary duration for a predetermined time window.

15. The system of claim 8, wherein the assistive device comprises at least one of an intelligent personal assistant device, an electronic book, a laptop, an audio device, a smartphone, and a device running an instant messaging app.

16. A computer readable storage medium storing a program of instructions executable by a machine to perform a method of executed by at least one hardware processor communicating with a smart contact lens comprising at least a microprocessor, an image capturing sensor, and a wireless communication interface, the method comprising:

receiving from the smart contact lens data captured by the smart contact lens, the data associated with a user wearing the smart lens reading content item from a data source, the data comprising at least data representing eye movements of the user while reading the content item;

based on the data, determining reading factors associated with the user with respect to the content item, the reading factors comprising at least a user'"'"'s cognitive state determined at least from analyzing the data representing eye movements;

responsive to determining that the user'"'"'s cognitive state is above a threshold value, generating an assistive action associated with the content item;

transmitting a signal to at least on assistive device to perform the assistive action, wherein the assistive device performs the assistive action responsive to receiving the signal.

17. The computer readable storage medium of claim 16, wherein the data further comprises images of the user'"'"'s hand movement, facial expression, head movement while reading the content item, glucose level, and pupil dilation.

18. The computer readable storage medium of claim 16, wherein the reading factors further comprises user context, user profile and content item characteristics associated with the content item.

19. The computer readable storage medium of claim 16, wherein the assistive action comprises at least one of highlighting a portion of the content item on a display of the data source from which the user is reading the content item, summarizing the content item, triggering a sound-controlled intelligent personal assistant service.

20. The computer readable storage medium of claim 16, wherein the reading factors further comprises user context, wherein the user context comprises at least one of a degree of engagement between the user and the content item, an engagement pattern between the user and the content item, a degree of the user'"'"'s confusion state with respect to the reading of the content item, and the user'"'"'s reading environment context.