Communication and control system and method

• US 10,747,315 B2
• Filed: 07/03/2019
• Issued: 08/18/2020
• Est. Priority Date: 10/30/2013
• Status: Active Grant

• Current Assignee: Technology Against ALS
• Original Assignee: Technology Against ALS
• Inventors: Tungare, Sandeep, Gupte, Milind, Patil, Amitabh, Reddy, Naveen, Haberland, Bert
• Primary Examiner(s): Soto Lopez, Jose R
• Application Number:
  US16/502,112
• Publication Number:
  US 20190324535A1
• Time in Patent Office: 412 Days
• CPC Class Codes: A61F 4/00 Methods or devices enabling patients or disabled persons to operate an apparatus or a device not forming part of the body operating or control means for prostheses A61F2/48, A61F2/68
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B

Abstract

An input system includes at least one sensor and a controller communicatively coupled to the at least one sensor. The at least one sensor is configured to identify a position of an eye. The controller is configured to receive a first signal indicative of a first position of an eye from the at least one sensor and map the eye position to a user'"'"'s selection.

Claims

1. A system, comprising:

a support structure supporting a plurality of visible indicia;

a feedback mechanism, the feedback mechanism configured to provide at least one of visual feedback and audio feedback;

a sensor supported by the support structure, the sensor configured to identify a position of a pupil of an eye of a user; and

a controller communicatively coupled to the sensor and to the feedback mechanism, the controller configured to;

receive a first signal from the sensor, the first signal indicative of a gaze direction of the pupil,

identify, based on the first signal, a gazed upon visible indicia of the plurality of visible indicia, the gazed upon visible indicia being the visible indicia that most closely corresponds to the gaze direction,

map the gazed upon visible indicia to a virtual reference frame, the virtual reference frame comprising a plurality of members of a set,

identify a selected member from the plurality of members of the set,

cause the feedback mechanism to notify the user that the selected member has been identified,

update the virtual reference frame if the selected member is not one of a group of predetermined final selections, and

transmit a second signal to at least one of the feedback mechanism and a first device to initiate an action based on the selected member if the selected member is one of the group of predetermined final selections.

2. The system of claim 1, wherein the notification that the selected member has been identified includes at least one of changing a brightness of at least one of the plurality of visible indicia and blinking at least one of the plurality of visible indicia.

3. The system of claim 2, wherein changing the brightness of at least one of the plurality of visible indicia includes gradually increasing the brightness of the at least one of the plurality of visible indicia.

4. The system of claim 1, wherein the support structure includes a frame which supports an infrared light source for illuminating the eye.

5. The system of claim 1, wherein the feedback mechanism includes at least one of a speaker, an earpiece, a mounted heads-up display, and a visual display.

6. The system of claim 1, wherein the support structure includes glasses for being worn by the user.

7. The system of claim 6, wherein the support structure further includes an arm coupled to the glasses and extending in front of the glasses, wherein the sensor includes a camera, and wherein the arm supports the camera such that, when the glasses are worn by the user, a lens of the camera is facing toward at least one of the user'"'"'s eyes.

8. The system of claim 7, wherein the support structure further includes a housing coupled to the glasses, wherein the controller is disposed in the housing, and wherein the arm is coupled to and extends from the housing.

9. The system of claim 1, wherein the feedback mechanism includes a visual display, and wherein the visual display is configured to display a representation of the virtual reference frame.

10. The system of claim 1, wherein the group of final selections includes at least one of a letter, a number, a word, a phrase, and a sentence.

11. The system of claim 1, wherein the controller is configured to transmit the second signal to the first device if the selected member is one of the group of final selections, and wherein the first device is selected from the group consisting of a television, a medical support device, and a light.

12. The system of claim 1, wherein the first device is a speaker, and wherein the controller transmits the selected member to the speaker if the selected member is one of the group of final selections such that the speaker broadcasts the selected member.

13. A method, comprising:

receiving, by a processor, a first signal from a sensor, the first signal indicative of a gaze direction of a pupil of an eye of a user gazing at at least one of a plurality of visible indicia;

identify, based on the first signal, a gazed upon visible indicia, the gazed upon visible indicia being one of the plurality of visible indicia;

mapping, by the processor, the gazed upon visible indicia to a virtual reference frame, the virtual reference frame comprising a plurality of members of a set;

identifying a selected member from the plurality of members of the set;

causing a feedback mechanism to provide visual or audio feedback that the selected member has been identified;

updating the virtual reference frame if the selected member is not one of a group of predetermined final selections; and

transmitting a second signal to the feedback mechanism or to a first device to initiate an action based on the selected member if the selected member is one of the group of predetermined final selections.

14. The method of claim 13, further comprising adjusting a brightness of at least one of the plurality of visible indicia in response to identifying the selected member.

15. The method of claim 14, wherein adjusting the brightness of at least one of the plurality of visible indicia includes gradually increasing the brightness.

16. The method of claim 13, further comprising causing at least one of the plurality of visible indicia to blink in response to identifying the selected member.

17. The method of claim 13, wherein the second signal is transmitted to the first device, the first device includes a controllable device, and the method further comprises causing the controllable device to perform a function in response to receiving the second signal.

18. A system, comprising:

a support structure comprising glasses and an arm coupled to the glasses and extending in front of the glasses;

a plurality of visible indicia supported by the support structure;

a feedback mechanism, the feedback mechanism configured to provide at least one of visual feedback and audio feedback;

a sensor configured to identify a position of a pupil of an eye, the sensor including a camera, wherein the camera is supported by the arm of the support structure such that, when the glasses are worn by a user, a lens of the camera is facing toward at least one of the user'"'"'s eyes; and

a controller communicatively coupled to the camera and to the feedback mechanism, the controller configured to;

receive a first signal from the camera, the first signal indicative of a gaze direction of the pupil,

identify, based on the first signal, a gazed upon visible indicia, the gazed upon visible indicia being one of the plurality of visible indicia that most closely corresponds to the gaze direction,

map the gazed upon visible indicia to a virtual reference frame, the virtual reference frame comprising a plurality of members of a set,

identify a selection of one of the plurality of members of the set, and

transmit a second signal to the feedback mechanism or to a first device to initiate an action based on the selection.

19. The system of claim 18, wherein the plurality of visible indicia are affixed to a lens of the glasses.

20. The system of claim 18, wherein the support structure further includes a housing coupled to the glasses, wherein the controller is disposed in the housing, and wherein the arm is coupled to and extends from the housing.