Systems and methods for continual speech recognition and detection in mobile computing devices

• US 8,452,597 B2
• Filed: 09/15/2012
• Issued: 05/28/2013
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Bringert, Bjorn Erik, Pietryka, Pawel, Hodgson, Peter John, Tickner, Simon, Penha, Henrique, Cohen, Richard Zarek, Zanolin, Luca, Burke, Dave
• Primary Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US13/621,068
• Publication Number:
  US 20130085755A1
• Time in Patent Office: 255 Days
• Field of Search: 704231-257, 704270-276, 379/455
• US Class Current: 704/251
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 21/10 Transforming into visible information

Abstract

The present application describes systems, articles of manufacture, and methods for continuous speech recognition for mobile computing devices. One embodiment includes determining whether a mobile computing device is receiving operating power from an external power source or a battery power source, and activating a trigger word detection subroutine in response to determining that the mobile computing device is receiving power from the external power source. In some embodiments, the trigger word detection subroutine operates continually while the mobile computing device is receiving power from the external power source. The trigger word detection subroutine includes determining whether a plurality of spoken words received via a microphone includes one or more trigger words, and in response to determining that the plurality of spoken words includes at least one trigger word, launching an application corresponding to the at least one trigger word included in the plurality of spoken words.

Claims

1. A method comprising:

determining whether a mobile computing device is receiving operating power from an external power source, wherein the mobile computing device has a trigger word detection subroutine that is activatable by a user input and automatically in response to determining that the mobile computing device is receiving external power; and

in response to determining that the mobile computing device is receiving operating power from the external power source, activating a trigger word detection subroutine, wherein the trigger word detection subroutine includes;

receiving spoken input via a microphone of the mobile computing device,

based on speech recognition performed on the spoken input, obtaining text,

determining whether the text includes one or more trigger words associated with a voice command prompt application, and

in response to determining that the text includes the one or more trigger words associated with the voice command prompt application, launching the voice command prompt application, wherein the voice command prompt application is configured to receive via the microphone additional spoken input that causes the mobile computing device to launch one or more other applications, and wherein launching the voice command prompt application comprises displaying a voice command prompt on the mobile computing device.

2. The method of claim 1, further comprising:

executing the trigger word detection routine continually until it is deactivated.

3. The method of claim 1, further comprising:

deactivating the trigger word detection subroutine in response to determining that the mobile computing device is receiving operating power from one or more batteries.

4. The method of claim 1, further comprising:

receiving the user input; and

activating the trigger word detection subroutine in response to receiving the user input regardless of whether the mobile computing device is receiving operating power from the external power source.

5. The method of claim 1, wherein the one or more other applications include at least one of a communications application, a multimedia application, or an application for configuring settings of the mobile computing device.

6. The method of claim 1, wherein the external power source is one of a charger associated with the mobile computing device and a docking station for the mobile computing device.

7. The method of claim 1, wherein one or more of the determining step, the activating step, and the trigger word detection subroutine are components of a mobile computing device operating system.

8. The method of claim 1, further comprising:

after launching the voice command prompt application, receiving a spoken command string via the microphone, wherein the spoken command string includes an action and an object of the action; and

in response to receiving the spoken command string, launching on the mobile computing device an application corresponding to at least the action and executing on the mobile computing device the application based on at least the object of the action.

9. The method of claim 8, wherein the spoken command string further includes one or more parameters associated with the action, and wherein launching on the mobile computing device an application corresponding to at least the action comprises launching on the mobile computing device an application corresponding to the action and the one or more parameters associated with the action.

10. The method of claim 8, wherein the spoken command string further includes one or more parameters associated with the object of the action, and wherein executing on the mobile computing device the application based on at least the object of the action comprises executing on the mobile computing device the application based on the object of the action and the one or more parameters associated with the object of the action.

11. A mobile computing device comprising:

a sound detector;

a display;

a power selector configured to selectively power the mobile computing device from one of either an external power source or one or more batteries; and

one or more processors configured to continually execute a trigger word detection subroutine when the power selector is powering the mobile computing device from the external power source and halt execution of the trigger word detection subroutine when the power selector is powering the mobile computing device from the one or more batteries, wherein the trigger word detection subroutine comprises determining whether spoken input received via the sound detector includes one or more trigger words associated with a voice command prompt application, and in response to determining that the spoken input includes the one or more trigger words associated with the voice command prompt application, launching the voice command prompt application, wherein the voice command prompt application is configured to receive via the sound detector additional spoken input that causes the mobile computing device to launch one or more other applications, and wherein launching the voice command prompt application comprises displaying a voice command prompt on the display.

12. The mobile computing device of claim 11, further comprising:

a user interface configured to receive an input from a user to activate the trigger word detection subroutine when the power selector is powering the mobile computing device from the one or more batteries.

13. The mobile computing device of claim 11, wherein the one or more other applications include at least one of a communications application, a multimedia application, or an application for configuring settings of the mobile computing device.

14. The mobile computing device of claim 11, wherein the one or more processors are further configured to, in response to the additional spoken input including an action and an object of the action, launch an application corresponding to at least the action and execute the application based on at least the object of the action.

15. The mobile computing device of claim 14, wherein the additional spoken input further includes one or more parameters associated with the action, and wherein launching an application corresponding to at least the action comprises launching an application corresponding to the action and the one or more parameters associated with the action.

16. The mobile computing device of claim 14, wherein the additional spoken input further includes one or more parameters associated with the object of the action, and wherein executing the application based on at least the object of the action comprises executing on the mobile computing device the application based on the object of the action and the one or more parameters associated with the object of the action.

17. A non-transitory computer readable medium having stored therein executable instructions that, when executed by a computing device having at least one processor, causes the computing device to perform functions comprising:

determining whether a mobile computing device is receiving operating power from either an external power source or an internal battery source; and

activating a trigger word detection subroutine in response to determining that the mobile computing device is receiving power from the external power source, wherein the trigger word detection subroutine comprises continually determining whether spoken input received via a microphone includes one or more trigger words associated with a voice command prompt application and, in response to determining that the spoken input includes the one or more trigger words associated with the voice command prompt application, launching the voice command prompt application, wherein the voice command prompt application is configured to receive via the microphone additional spoken input that causes the mobile computing device to launch one or more other applications, and wherein launching the voice command prompt application comprises displaying a voice command prompt on the mobile computing device.

18. The non-transitory computer readable medium of claim 17, wherein the functions further comprise:

deactivating the trigger word detection subroutine in response to determining that the mobile computing device is receiving operating power from the internal battery source.

19. The non-transitory computer readable medium of claim 17, wherein the functions further comprise:

receiving a user input; and

in response to receiving the user input, activating the trigger word detection subroutine regardless of whether the mobile computing device is receiving operating power from the external power source or the internal battery source.

20. The non-transitory computer readable medium of claim 17, wherein the functions further comprise, in response to the additional spoken input including an action and an object of the action, launching an application corresponding to at least the action and executing the application based on at least the object of the action.