Voice control for asynchronous notifications

• US 8,959,023 B2
• Filed: 05/13/2013
• Issued: 02/17/2015
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Bringert, Bjorn Erik, Pietryka, Pawel, Hodgson, Peter John, Burke, Dave, Penha, Henrique, Tickner, Simon, Cohen, Richard Zarek, Zanolin, Luca, LeBeau, Michael J.
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US13/892,485
• Publication Number:
  US 20130253928A1
• Time in Patent Office: 645 Days
• Field of Search: 704/257, 704/270, 704/275
• US Class Current: 704/275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04L 51/224 providing notification on incoming messages, e.g. pushed notifications of received messages
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00

Abstract

A computing device may receive an incoming communication and, in response, generate a notification that indicates that the incoming communication can be accessed using a particular application on the communication device. The computing device may further provide an audio signal indicative of the notification and automatically activate a listening mode. The computing device may receive a voice input during the listening mode, and an input text may be obtained based on speech recognition performed upon the voice input. A command may be detected in the input text. In response to the command, the computing device may generate an output text that is based on at least the notification and provide a voice output that is generated from the output text via speech synthesis. The voice output identifies at least the particular application.

Claims

1. A method comprising:

providing, by a computing device, an audio signal indicative of at least one notification in a plurality of notifications and automatically activating a listening mode on the computing device, wherein the plurality of notifications relate to a plurality of incoming communications and a plurality of different applications on the computing device, such that each notification in the plurality of notifications indicates that a respective incoming communication in the plurality of incoming communications can be accessed using a respective application in the plurality of different applications;

receiving a voice input during the listening mode;

obtaining an input text based on speech recognition performed upon the voice input;

detecting a command in the input text;

generating an output text in response to detecting the command in the input text, wherein the output text is based on the plurality of notifications; and

providing, by the computing device, a voice output that identifies each of the different applications prior to receiving a selection of an application in the plurality of different applications.

2. The method of claim 1, further comprising:

after providing the voice output, receiving an additional voice input;

obtaining an additional input text based on speech recognition performed upon the additional voice input;

determining that the additional input text identifies a particular application in the plurality of different applications; and

in response to determining that the additional input text identifies the particular application, invoking the particular application.

3. The method of claim 2, wherein invoking the particular application comprises invoking the particular application in a voice mode.

4. The method of claim 1, wherein the plurality of different applications includes at least one of an email application, a voicemail application, a short message service (SMS) application, a social network messaging application, a multi-media messaging service (MMS) application, or an instant messaging application.

5. The method of claim 1, further comprising:

receiving, by the computing device, the plurality of incoming communications; and

generating, by the computing device, the plurality of notifications such that each notification in the plurality of notifications is generated in response to receiving a respective incoming communication in the plurality of incoming communications.

6. The method of claim 5, wherein the plurality of notifications includes at least one newly-generated notification and one or more previously-generated notifications, and wherein the audio signal is indicative of the at least one newly-generated notification.

7. The method of claim 1, further comprising:

displaying an indication of a total number of notifications in the plurality of notifications.

8. The method of claim 1, further comprising:

displaying identifications of each of the different applications and indications of how many notifications in the plurality of notifications relate to each of the different applications.

9. A computer-readable medium having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations comprising:

providing an audio signal indicative of at least one notification in a plurality of notifications and automatically activating a listening mode on the computing device, wherein the plurality of notifications relate to a plurality of incoming communications and a plurality of different applications on the computing device, such that each notification in the plurality of notifications indicates that a respective incoming communication in the plurality of incoming communications can be accessed using a respective application in the plurality of different applications;

receiving a voice input during the listening mode;

obtaining an input text based on speech recognition performed upon the voice input;

detecting a command in the input text;

generating an output text in response to detecting the command in the input text, wherein the output text is based on the plurality of notifications; and

providing a voice output that identifies each of the different applications prior to receiving a selection of an application in the plurality of different applications.

10. The computer-readable medium of claim 9, wherein the operations further comprise:

after providing the voice output, receiving an additional voice input;

obtaining an additional input text based on speech recognition performed upon the additional voice input;

determining that the additional input text identifies a particular application in the plurality of different applications; and

in response to determining that the additional input text identifies the particular application, invoking the particular application.

11. The computer-readable medium of claim 10, wherein invoking the particular application comprises invoking the particular application in a voice mode.

12. The computer-readable medium of claim 9, wherein the plurality of different applications includes at least one of an email application, a voicemail application, a short message service (SMS) application, a social network messaging application, a multi-media messaging service (MMS) application, or an instant messaging application.

13. The computer-readable medium of claim 9, wherein the operations further comprise:

displaying an indication of a total number of notifications in the plurality of notifications.

14. The computer-readable medium of claim 9, wherein the operations further comprise:

displaying identifications of each of the different applications and indications of how many notifications in the plurality of notifications relate to each of the different applications.

15. A computing device, comprising:

an audio input unit;

an audio output unit;

a display;

at least one processor; and

a memory, wherein the memory stores instructions that upon execution by the at least one processor cause the computing device to perform operations comprising;

providing, via the audio output unit, an audio signal indicative of at least one notification in a plurality of notifications and automatically activating a listening mode on the computing device, wherein the plurality of notifications relate to a plurality of incoming communications and a plurality of different applications on the computing device, such that each notification in the plurality of notifications indicates that a respective incoming communication in the plurality of incoming communications can be accessed using a respective application in the plurality of different applications;

receiving, via the audio input unit, a voice input during the listening mode;

obtaining an input text based on speech recognition performed upon the voice input;

detecting a command in the input text;

generating an output text in response to detecting the command in the input text, wherein the output text is based on the plurality of notifications;

providing, via the audio output unit, a voice output that identifies each of the different applications prior to receiving a selection of an application in the plurality of different applications.

16. The computing device of claim 15, wherein the operations further comprise:

after providing the voice output, receiving an additional voice input via the audio input unit;

obtaining an additional input text based on speech recognition performed upon the additional voice input;

determining that the additional input text identifies a particular application in the plurality of different applications; and

in response to determining that the additional input text identifies the particular application, invoking the particular application.

17. The computing device of claim 16, wherein invoking the particular application comprises invoking the particular application in a voice mode.

18. The computing device of claim 15, wherein the plurality of different applications includes at least one of an email application, a voicemail application, a short message service (SMS) application, a social network messaging application, a multi-media messaging service (MMS) application, or an instant messaging application.

19. The computing device of claim 15, wherein the operations further comprise:

displaying, on the display, an indication of a total number of notifications in the plurality of notifications.

20. The computing device of claim 15, wherein the operations further comprise:

displaying, on the display, identifications of each of the different applications and indications of how many notifications in the plurality of notifications relate to each of the different applications.