FACILITATING AWARENESS AND CONVERSATION THROUGHPUT IN AN AUGMENTATIVE AND ALTERNATIVE COMMUNICATION SYSTEM

• US 20170103679A1
• Filed: 10/09/2015
• Published: 04/13/2017
• Est. Priority Date: 10/09/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Campbell, Jon, Paradiso, Ann, Beavers, Jay, Shah, Mira E., Morris, Meredith, Fiannaca, Alexander, Kulkarni, Harish
• Granted Patent: US 10,262,555 B2
• Time in Patent Office: Days
• CPC Class Codes: A61F 4/00 Methods or devices enabling patients or disabled persons to operate an apparatus or a device not forming part of the body operating or control means for prostheses A61F2/48, A61F2/68
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Speech generating devices, communication systems, and methods for communicating using the devices and systems are disclosed herein. In certain examples, a communication system is configured to receive a generated communication, establish a connection between a speech generating device and a computing device subsequent to receipt of the generated communication, and transmit the generated communication to the computing device. In other examples, a computing device is configured to establish a connection with a speech generating device, and receive a transmission generated by the speech generating device following the connection, the transmission including previously generated communications or real-time communication segments or proxies. In other examples, a speech generating device is configured to establish a connection with one or more computing devices, receive one or more suggestions from at least one computing device during generation of the communication, and display a suggestion on the display device as a shortcut input key.

Claims

1. A communication system comprising:

a server configured to wirelessly communicate with a speech generating device and a computing device over a communication network, wherein the server is configured to;

receive a generated communication from the speech generating device;

establish a connection between the speech generating device and the computing device subsequent to receipt of the generated communication; and

transmit the generated communication to the computing device.

2. The communication system of claim 1, wherein the speech generating device comprises a display device and an input device configured to generate the communication to be displayed on the display device.

3. The communication system of claim 1, wherein the generated communication is selected from the group consisting of:

a communication preference, a pre-composed block of text, multimedia, an activity log, and a combination thereof.

4. The communication system of claim 3, wherein the communication preference is a request:

(1) to ask only yes or no questions, (2) to view or not view over a shoulder of a user of the speech generating device as the user is composing a message, (3) to not finish a user'"'"'s thought as the user is composing the message, or (4) to assist the user in completing or correcting the message using a computing device connected to the speech generating device.

5. The communication system of claim 3, wherein the multimedia comprises an image or video captured by the speech generating device.

6. The communication system of claim 3, wherein the activity log comprises an internet browsing history, a television or movie viewing history, a book reading or listening history, a music listening history, an article reading history, or an application usage history of a user of the speech generating device for a defined period of time.

7. The communication system of claim 1, wherein the server is configured to transmit the communication to the computing device based on whether the computing device is categorized in a specific conversation group.

8. A computing device comprising:

at least one processor; and

at least one memory coupled to the at least one processor, the at least one memory including computer program code for one or more programs;

the at least one memory and the computer program code configured to, with the at least one processor, cause the computing device to;

establish a connection with a speech generating device via a communication network; and

receive a transmission generated by the speech generating device, wherein the transmission comprises at least one of;

(1) a previously generated communication, (2) a segment of a communication being generated in real-time, (3) a social cue proxy of the real-time communication, or (4) an emotional proxy of the previously generated communication or the real-time communication.

9. The computing device of claim 8, wherein the computing device is a personal computer, tablet computer, laptop computer, mobile phone, or wearable electronic device.

10. The computing device of claim 8, wherein the previously generated communication is a communication preference, a pre-composed block of text, multimedia, an activity log, or a combination thereof.

11. The computing device of claim 8, wherein the segment of the real-time communication is a predefined segment length, the predefined segment length being a character, word, phrase, sentence, or block of text, and

wherein the predefined segment length is based on a relationship between a user of the speech generating device and a conversation partner of the computing device.

12. The computing device of claim 11, wherein the at least one memory and the computer program code are further configured to cause the computing device to:

assist the user of the speech generating device in generation of the real-time communication through a submission of (1) one or more corrections to words, phrases or sentences in the received transmission, and/or (2) one or more suggestions of a next word, phrase, or sentence.

13. The computing device of claim 8, wherein the social cue proxy is (1) an indication that a user of the speech generating device is typing or composing a message, (2) an indication that the user is speaking or about to speak, (3) an indication that the user is requesting to speak, (4) a request for assistance, (5) an identification of an emergency, (6) an indication that the user is calibrating an input device of the speech generating device, (7) an indication that the speech generating device is idle, or (8) an indication that the user is sleeping.

14. The computing device of claim 8, wherein the emotional proxy or the social cue proxy is displayed as a color or an arrangement of lights on a display screen of the computing device, the color or the arrangement of lights configured to identify an emotion or social cue.

15. A speech generating device comprising:

a display device;

an input device configured to generate a communication to be displayed on the display device; and

at least one processor coupled to the display device and input device,

wherein the speech generating device, with the at least one processor, is configured to;

establish a connection with one or more computing devices via a communication network,

receive one or more suggestions from at least one computing device during generation of the communication; and

display at least one suggestion on the display device as a shortcut input key.

16. The speech generating device of claim 15, wherein the speech generating device, with the at least one processor, is further configured to:

define a relationship between a user of the speech generating device and a conversation partner of the computing device; and

determine which of the one or more suggestions to display on the display screen based on the defined relationship.

17. The speech generating device of claim 15, wherein the speech generating device, with the at least one processor, is further configured to:

define a relationship between a user of the speech generating device and a conversation partner of the computing device;

identify a geographic location of the speech generating device; and

determine which of the one or more suggestions to display on the display screen based on the defined relationship and the geographic location.

18. The speech generating device of claim 15, wherein the speech generating device, with the at least one processor, is further configured to:

provide text predictions on the display screen as additional shortcut input keys, wherein the at least one suggestion is identified on the display screen in a different font, color, or shortcut input key size as the additional shortcut input keys.

19. The speech generating device of claim 18, wherein the at least one suggestion is identified in the different font, color, or shortcut input key size based on a relationship between a user of the speech generating device and a conversation partner of the computing device.

20. The speech generating device of claim 15, wherein the speech generating device, with the at least one processor, is further configured to:

provide text predictions on the display screen as additional shortcut input keys,

wherein the at least one suggestion is identified on the display screen in a same font, color, and shortcut input key size as the additional shortcut input keys.