Methods and systems for augmentative and alternative communication

• US 9,595,171 B2
• Filed: 01/12/2016
• Issued: 03/14/2017
• Est. Priority Date: 07/12/2013
• Status: Active Grant

• Current Assignee: University of Iowa Research Foundation
• Original Assignee: University of Iowa Research Foundation
• Inventors: Hurtig, Richard, Berkowitz, Benjamin, Martinson, Blake Earl, Zhu, Zihan, Bird, Richard Rives
• Primary Examiner(s): HOLLOWAY III, EDWIN C
• Application Number:
  US14/993,889
• Publication Number:
  US 20160125705A1
• Time in Patent Office: 427 Days
• Field of Search: 340/4.11
• US Class Current: 1/1
• CPC Class Codes: A61F 4/00 Methods or devices enabling patients or disabled persons to operate an apparatus or a device not forming part of the body operating or control means for prostheses A61F2/48, A61F2/68
• CPC Class Codes: G06F 2203/038 Indexing scheme relating to G06F3/038
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08B 5/222 Personal calling arrangements or devices, i.e. paging systems selective calling systems and call receivers H04W84/00, H04W68/00
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: H03K 17/962 Capacitive touch switches
• CPC Class Codes: H03K 2217/96003 using acoustic waves, e.g. ultrasound

Abstract

Methods and systems for augmentative and alternative communication are disclosed. An example method can comprise receiving a candidate input, classifying the candidate input as an intentional input, and generating a signal in response to the intentional input.

Claims

1. A method for communication, comprising:

detecting, by a computing device, a connection with a sensor configured to detect only a single gesture, wherein the computing device is configured to connect with a plurality of sensors of different types;

determining, by the computing device, a type of the sensor;

determining, by the computing device, a filter based on the type of the sensor, wherein the filter is generated based on analysis of signals associated with the type of sensor and configured the computing device to classify gestures as intentional or unintentional based on a first timing threshold and a frequency characteristic;

receiving, by the computing device from the sensor, a signal representing a plurality of candidate inputs detected by the sensor;

classifying, by the computing device, the plurality of candidate inputs as one or more intentional inputs based on the first timing threshold and the frequency characteristic;

determining, by the computing device, timing information indicative of a time difference between receiving a first input signal of the one or more intentional inputs and receiving a second input signal of the one or more intentional inputs;

determining, by the computing device, that the one or more intentional inputs are relevant to a single command based on the timing information;

counting, by the computing device, a number of the one or more intentional inputs relevant to the single command;

selecting, by the computing device, a target device from a plurality of devices, based on the number of the one or more intentional inputs relevant to the single command, wherein each of the plurality of devices is associated with a respective number and a number of the target device matches the number of the one or more intentional outputs relevant to the single command; and

providing a signal to the selected target device in response to the intentional input.

2. The method of claim 1, wherein the sensor comprises a touch sensor, a pressure sensor, a mechanical sensor, an optical reflectance sensor, an infrared reflectance sensor, a light sensor, a proximity detector, a capacitance sensor, an audio sensor, or a camera.

3. The method of claim 1, wherein providing the signal to the selected target device in response to the intentional input comprises activating one or more of a nurse call switch, a computer, a television, a fan, a light, a telephone, and a medical device.

4. The method of claim 1, wherein providing the signal to the selected target device in response to the intentional input comprises controlling one or more controlled devices.

5. The method of claim 1, wherein the type of the sensor is determined based on a voltage level of a signal received from the sensor.

6. The method of claim 1, wherein the type of the sensor is determined based on a code received from the sensor.

7. The method of claim 1, further comprising determining an electrical resistance value of the connection with the sensor, wherein the type of the sensor is determined based on the electrical resistance value.

8. The method of claim 1, further comprising:

receiving a second plurality of candidate inputs from the sensor;

classifying the second plurality of candidate inputs as one or more second intentional inputs; and

determining a command for controlling the target device based on the one or more second intentional inputs, wherein providing the signal to the selected target device comprises providing a signal indicative of the command to the selected target device in response to the intentional input.

9. The method of claim 8, wherein the command comprises a command for an application on the target device.

10. The method of claim 8, wherein the command is determined based on a number of the one or more second intentional inputs received during a time window.

11. The method of claim 8, wherein the command is determined based on which target device is selected from among the plurality of devices.

12. The method of claim 1, wherein the sensor comprises a single switch configured to provide signals based on a single gesture from a user.

13. A system, comprising:

a memory comprising computer-executable instructions; and

a processor functionally coupled to the memory and configured, by the computer-executable instructions, to perform at least the following actions,

detecting a connection with a sensor configured to detect only a single gesture, wherein the processor is configured to connect with a plurality of sensors of different types,

determining a type of the sensor,

determining a filter based on the type of the sensor, wherein the filter is generated based on analysis of signals associated with the type of sensor and configures the processor to classify gestures as intentional or unintentional based on a first timing threshold and a frequency characteristic,

receiving, from the sensor, a signal representing a plurality of candidate inputs detected by the sensor,

classifying the plurality of candidate inputs as one or more intentional inputs based on the first timing threshold and the frequency characteristic,

determining timing information indicative of a time difference between receiving a first input signal of the one or more intentional inputs and receiving a second input signal of the one or more intentional inputs,

determining that the one or more intentional inputs are relevant to a single command based on the timing information,

counting a number of the one or more intentional inputs relevant to the single command,

selecting a target device from a plurality of devices based on the number of the one or more intentional inputs relevant to the single command, wherein each of the plurality of devices is associated with a respective number and a number of the target device matches the number of the one or more intentional outputs relevant to the single command, and

providing a signal to the selected target device in response to the intentional input.

14. The system of claim 13, wherein the sensor comprises a touch sensor, a pressure sensor, a mechanical sensor, an optical reflectance sensor, an infrared reflectance sensor, a light sensor, a proximity detector, a capacitance sensor, an audio sensor, or a camera.

15. The system of claim 13, wherein providing the signal to the selected target device in response to the intentional input comprises activating one or more of a nurse call switch, a computer, a television, a fan, a light, a telephone, and a medical device.

16. The system of claim 13, wherein providing the signal to the selected target device in response to the intentional input comprises controlling one or more controlled devices.

17. The system of claim 13, wherein the type of the sensor is determined based on a voltage level of a signal received from the sensor.

18. The system of claim 13, wherein the type of the sensor is determined based on a code received from the sensor.

19. The system of claim 13, wherein the processor is further configured for determining an electrical resistance value of the connection with the sensor, wherein the type of the sensor is determined based on the electrical resistance value.

20. The system of claim 13, wherein the processor is further configured for:

receiving a second plurality of candidate inputs from the sensor;

classifying the second plurality of candidate inputs as one or more second intentional inputs; and

determining a command for controlling the target device based on the one or more second intentional inputs, wherein providing the signal to the selected target device comprises providing a signal indicative of the command to the selected target device in response to the intentional input.

21. The system of claim 20, wherein the command comprises a command for an application on the target device.

22. The system of claim 20, wherein the command is determined based on a number of the one or more second intentional inputs received during a time window.

23. The system of claim 20, wherein the command is determined based on which target device is selected from among the plurality of devices.

24. The system of claim 13, wherein the sensor comprises a single switch configured to provide signals based on a single gesture from a user.

25. A non-transitory computer-readable medium comprising computer-executable instructions that when executed by a processor perform the method comprising:

detecting, by a computing device, a connection with a sensor configured to detect only a single gesture, wherein the computing device is configured to connect with a plurality of sensors of different types;

determining, by the computing device, a type of the sensor;

determining, by the computing device, a filter based on the type of the sensor, wherein the filter is generated based on analysis of signals associated with the type of sensor and configures the computing device to classify gestures as intentional or unintentional based on a first timing threshold and a frequency characteristic;

receiving, by he computing device from the sensor, a signal representing a plurality of candidate inputs detected by the sensor;

classifying, by the computing device, the plurality of candidate inputs as one or more intentional inputs based on the first timing threshold and the frequency characteristic;

determining, by the computing device, timing information indicative of a time difference between receiving a first input signal of the one or more intentional inputs and receiving a second input signal of the one or more intentional inputs;

determining, by the computing device, that the one or more intentional inputs are relevant to a single command based on the timing information;

counting, by the computing device, a number of the one or more intentional inputs relevant to the single command;

selecting, by the computing device, a target device from a plurality of devices, based on the number of the one or more intentional inputs relevant to the single command, wherein each of the plurality of devices is associated with a respective number and a number of the target device matches the number of the one or more intentional outputs relevant to the single command; and

providing a signal to the selected target device in response to the intentional input.