Facilitating awareness and conversation throughput in an augmentative and alternative communication system

• US 10,262,555 B2
• Filed: 10/09/2015
• Issued: 04/16/2019
• Est. Priority Date: 10/09/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Campbell, Jon, Paradiso, Ann, Beavers, Jay, Shah, Mira E., Morris, Meredith, Fiannaca, Alexander, Kulkarni, Harish
• Primary Examiner(s): Azad, Abul K
• Application Number:
  US14/880,061
• Publication Number:
  US 20170103679A1
• Time in Patent Office: 1,285 Days
• Field of Search: None
• CPC Class Codes: A61F 4/00 Methods or devices enabling patients or disabled persons to operate an apparatus or a device not forming part of the body operating or control means for prostheses A61F2/48, A61F2/68
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Speech generating devices, communication systems, and methods for communicating using the devices and systems are disclosed herein. In certain examples, a communication system is configured to receive a generated communication, establish a connection between a speech generating device and a computing device subsequent to receipt of the generated communication, and transmit the generated communication to the computing device. In other examples, a computing device is configured to establish a connection with a speech generating device, and receive a transmission generated by the speech generating device following the connection, the transmission including previously generated communications or real-time communication segments or proxies. In other examples, a speech generating device is configured to establish a connection with one or more computing devices, receive one or more suggestions from at least one computing device during generation of the communication, and display a suggestion on the display device as a shortcut input key.

Claims

1. A communication system for relaying asynchronous communications between a user of a speech generating device and a conversation partner of a computing device during a synchronous conversation, the communication system comprising:

a server configured to wirelessly communicate with the speech generating device and the computing device over a communication network, wherein the server is configured to;

receive a generated communication for asynchronous communication composed by the user of the speech generating device prior to establishing a connection between the speech generating device and the computing device;

establish the connection between the speech generating device and the computing device subsequent to receipt of the generated communication;

facilitate an exchange of one or more messages between the speech generating device of the user and the computing device of the conversation partner as part of the synchronous conversation;

transmit the generated communication to the computing device, the generated communication including content that is shared with the conversation partner to fill a conversational gap occurring from a low throughput of communication as part of the synchronous conversation by the user;

receive, via the speech generating device, additional speech constructed by the user for the synchronous conversation during presentation of the generated communication to the conversation partner at the computing device; and

transmit the additional speech to the computing device for presentation to the conversation partner.

2. The communication system of claim 1, wherein the speech generating device comprises a display device and an input device configured to generate the communication to be displayed on the display device.

3. The communication system of claim 1, wherein the generated communication is selected from the group consisting of:

a communication preference, a pre-composed block of text, multimedia, an activity log, and a combination thereof.

4. The communication system of claim 3, wherein the multimedia comprises an image or video captured by the speech generating device.

5. The communication system of claim 3, wherein the activity log comprises an internet browsing history, a television or movie viewing history, a book reading or listening history, a music listening history, an article reading history, or an application usage history of the user of the speech generating device for a defined period of time.

6. The communication system of claim 1, wherein the generated communication is a communication preference requesting the conversation partner of the computing device to ask only yes or no questions to the user of the speech generating device.

7. The communication system of claim 1, wherein the server is configured to transmit the communication to the computing device based on whether the computing device is categorized in a specific conversation group.

8. The communication system of claim 1, wherein the generated communication is a communication preference requesting the conversation partner of the computing device to view or not view over a shoulder of the user of the speech generating device as the user of the speech generating device is composing one of the one or more messages.

9. The communication system of claim 1, wherein the generated communication is a communication preference requesting the conversation partner of the computing device to not finish a thought of the user of the speech generating device as the user of the speech generating device is composing one of the one or more messages.

10. The communication system of claim 1, wherein the generated communication is a communication preference requesting the conversation partner of the computing device to assist the user of the speech generating device in completing or correcting one of the one or more messages using the computing device connected to the speech generating device.

11. A computing device comprising:

a display screen;

at least one processor; and

at least one memory coupled to the at least one processor, the at least one memory including computer program code for one or more programs;

the at least one memory and the computer program code configured to, with the at least one processor, cause the computing device to;

establish a connection with a speech generating device via a communication network for a synchronous conversation between a user of the speech generating device and a conversation partner of the computing device;

receive a transmission generated by the speech generating device for asynchronous communication composed by the user of the speech generating device prior to establishing the connection between the speech generating device and the computing device;

facilitate an exchange of one or more messages between the speech generating device of the user and the computing device of the conversation partner as part of the synchronous conversation;

display the transmission on the display screen, the transmission including content that is shared with the conversation partner to fill a conversational gap occurring from a low throughput of communication as part of the synchronous conversation by the user;

receive, via the speech generating device, additional speech constructed by the user for the synchronous conversation during display of the transmission to the conversation partner at the computing device; and

display the additional speech to the conversation partner,

wherein the transmission comprises a previously generated communication composed by the user of the speech generating device.

12. The computing device of claim 11, wherein the computing device is a personal computer, tablet computer, laptop computer, mobile phone, or wearable electronic device.

13. The computing device of claim 11, wherein the previously generated communication is a communication preference defining how to interact with the user of the speech generating device.

14. The computing device of claim 11, wherein the at least one memory and the computer program code are further configured to cause the computing device to:

assist the user of the speech generating device in generation of a real-time communication through a submission of (1) one or more corrections to words, phrases or sentences in the received transmission, and/or (2) one or more suggestions of a next word, phrase, or sentence.

15. The computing device of claim 11, wherein the previously generated communication is a pre-composed block of text.

16. The computing device of claim 11, wherein the previously generated communication is a multimedia.

17. The computing device of claim 11, wherein the previously generated communication is an activity log of the user of the speech generating device.