Proxies for speech generating devices

• US 9,679,497 B2
• Filed: 10/09/2015
• Issued: 06/13/2017
• Est. Priority Date: 10/09/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Campbell, Jon, Paradiso, Ann, Beavers, Jay, Shah, Mira E., Morris, Meredith, Fiannaca, Alexander, Kulkarni, Harish
• Primary Examiner(s): GUERRA-ERAZO, EDGAR X
• Application Number:
  US14/880,092
• Publication Number:
  US 20170103680A1
• Time in Patent Office: 613 Days
• Field of Search: 704235, 704246, 704270, 7042701, 704275, 704277
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation
• CPC Class Codes: H04M 1/72475 specially adapted for disabled users

Abstract

Speech generating devices, communication systems, and methods for communicating using the devices and systems are disclosed herein. In certain examples, the speech generating device includes: a first display device positioned in a first direction, the first display device configured to face a user of the speech generating device; an input device configured to generate a communication to be displayed on the first display device; and a second display device positioned in a second direction different from the first direction, wherein the second display device is configured to display a social cue proxy or an emotional proxy of the generated communication.

Claims

1. A speech generating device comprising:

a first display device positioned in a first direction, the first display device configured to face a user of the speech generating device;

an input device configured to generate a message to be displayed on the first display device;

at least one processor;

at least one memory coupled to the at least one processor, the at least one memory including computer program code for one or more programs, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the speech generating device to;

analyze the generated message, and

identify an emotion or a social cue from the generated message; and

a second display device positioned in a second direction different from the first direction, wherein the second display device is configured to display a social cue proxy or an emotional proxy of the generated message representing the identified emotion or social cue, respectively.

2. The speech generating device of claim 1, wherein the input device is an eye gaze tracking device.

3. The speech generating device of claim 1, wherein the emotional proxy or the social cue proxy is displayed as a color or an arrangement of lights on a display screen of the second display device, the color or the arrangement of lights configured to represent the identified emotion or social cue.

4. The speech generating device of claim 1, wherein the second display device comprises an array of lighting elements.

5. The speech generating device of claim 1, wherein the second display device comprises an electronic-paper display, a liquid crystal display, or a light emitting diode display.

6. The speech generating device of claim 1, wherein the second display device comprises a display screen configured to display communication instructions for a conversation partner of the user of the speech generating device.

7. The speech generating device of claim 1,

wherein the emotion is identified from the generated message, and

wherein the identified emotion is displayed on the second display device as the emotional proxy.

8. The speech generating device of claim 1,

wherein the social cue is identified from the generated message, and

wherein the identified social cue is displayed on the second display device as the social cue proxy.

9. The speech generating device of claim 8, wherein the social cue proxy is (1) an indication that the user is typing or composing the message, (2) an indication that the user is speaking or about to speak, (3) an indication that the user is requesting to speak, (4) a request for assistance, or (5) an identification of an emergency.

10. The speech generating device of claim 1, wherein the first display device and the second display device are positioned within a single housing, and wherein the first direction is at a 180°

angle from the second direction.

11. A communication device comprising:

a speech generating device comprising;

a first display device configured to face a user of the speech generating device;

an input device configured to generate a message to be displayed on the first display device;

at least one processor; and

at least one memory coupled to the at least one processor, the at least one memory including computer program code for one or more programs;

the at least one memory and the computer program code configured to, with the at least one processor, cause the speech generating device to;

analyze the generated message, and

identify an emotion or a social cue from the generated message; and

a computing device separate from and wirelessly in communication with the speech generating device, the computing device comprising a second display device configured to display the social cue proxy or the emotional proxy of the generated message representing the identified emotion or social cue, respectively.

12. The communication device of claim 11, wherein the computing device is a personal computer, tablet computer, laptop computer, mobile phone, or wearable electronic device.

13. The communication device of claim 11, wherein the emotional proxy or the social cue proxy is displayed as a color or an arrangement of lights on a display screen of the second display device of the computing device, the color or the arrangement of lights configured to represent the identified emotion or social cue.

14. The communication device of claim 11,

wherein the emotion is identified from the generated message, and

wherein the identified emotion is displayed on the second display device as the emotional proxy.

15. The communication device of claim 11,

wherein the social cue is identified from the generated message, and

wherein the identified social cue is displayed on the second display device as the social cue proxy.

16. A method of communicating comprising:

generating a message using an input device of a speech generating device;

displaying the generated message on a first display device of the speech generating device;

analyzing the generated message using at least one processor and at least one memory including computer program code for one or more programs of the speech generating device;

identifying an emotion or a social cue in the analyzing; and

displaying a social cue proxy or an emotional proxy of the generated message on a second display device representing the identified emotion or social cue, respectively.

17. The method of claim 16, wherein the emotion is identified in the analyzing, and wherein the identified emotion is displayed on the second display device as the emotional proxy.

18. The method of claim 16, wherein the social cue is identified in the analyzing, and wherein the identified social cue is displayed on the second display device as the social cue proxy.

19. The method of claim 16, wherein the speech generating device comprises the second display device.

20. The method of claim 16, wherein the generated message comprises the emotion or the social cue input by the input device.