Automatically Monitoring for Voice Input Based on Context

• US 20130095805A1
• Filed: 12/06/2012
• Published: 04/18/2013
• Est. Priority Date: 08/06/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Michael J. Lebeau
• Inventors: LeBeau, Michael J., Jitkoff, John Nicholas, Burke, Dave
• Granted Patent: US 8,918,121 B2
• Time in Patent Office: Days
• US Class Current: 455/414.1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: H04M 1/04 Supports for telephone transmitters or receivers
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 1/72409 by interfacing with external accessories hands-free H04M1/60
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/046 involving vehicles, e.g. floating traffic data [FTD] or vehicle traffic prediction

Abstract

In one implementation, a computer-implemented method includes detecting a current context associated with a mobile computing device and determining, based on the current context, whether to switch the mobile computing device from a current mode of operation to a second mode of operation during which the mobile computing device monitors ambient sounds for voice input that indicates a request to perform an operation. The method can further include, in response to determining whether to switch to the second mode of operation, activating one or more microphones and a speech analysis subsystem associated with the mobile computing device so that the mobile computing device receives a stream of audio data. The method can also include providing output on the mobile computing device that is responsive to voice input that is detected in the stream of audio data and that indicates a request to perform an operation.

Claims

1. A computer-implemented method comprising:

detecting a context associated with a mobile computing device, the context indicating that the mobile computing device is traveling in a vehicle based at least in part on a calculated speed of the mobile computing device;

determining, based on the context, to switch the mobile computing device from a first mode of operation, during which the mobile computing device does not monitor ambient sounds for voice input that indicates a request to perform an operation, to a second mode of operation during which the mobile computing device monitors ambient sounds for voice input that indicates a request to perform an operation;

in response to determining to switch to the second mode of operation, activating one or more microphones and a speech analysis system associated with the mobile computing device; and

providing output using the mobile computing device, the output being responsive to voice input that is detected in a stream of audio data and that indicates a request to perform an operation.

2. The computer-implemented method of claim 1, wherein the context further indicates a direction in which the mobile computing device is traveling.

3. The computer-implemented method of claim 1, further comprising:

receiving a stream of audio data via the one or more microphones;

monitoring the stream of audio data for voice input using the speech analysis system;

upon detecting voice input from the audio stream, determining whether the voice input indicates a request for the mobile computing device to perform an operation; and

based on the determination of whether the voice input indicates a request to perform an operation, causing the requested operation to be performed.

4. The computer-implemented method of claim 3, wherein the request to perform an operation comprises a search request;

and

wherein causing the requested operation to be performed comprises causing the search request to be performed.

5. The computer-implemented method of claim 3, wherein causing the requested operation to be performed comprises:

providing first information regarding the requested operation over a network to a remote computer system that is configured to perform the requested operation; and

receiving second information over the network from the remote computer system that indicates output generated from performance of the requested operation, wherein at least a portion of the second information is provided as the output on the mobile computing device.

6. The computer-implemented method of claim 3, further comprising:

identifying whether any of a predetermined group of keywords is present in the voice input; and

wherein the determination of whether the voice input indicates a request to perform an operation is based on the determination of whether any of the keywords are present in the voice input.

7. The computer-implemented method of claim 3, further comprising:

determining whether the voice input is a command or a question directed at the mobile computing device based on, at least, syntactical analysis of the voice input;

wherein the determination of whether the voice input indicates a request to perform an operation is based on, at least, the determination of whether the voice input is a command or a question directed at the mobile computing device.

8. The computer-implemented method of claim 3, further comprising:

identifying changes in a structure associated with the voice input; and

determining whether the voice input is directed at the mobile computing device based on the identified changes;

wherein the determination of whether the voice input indicates a request to perform an operation is based on, at least, the determination of whether the voice input is directed at the mobile computing device.

9. The computer-implemented method of claim 1, further comprising:

determining, based on the context, whether a user associated with the mobile computing device has at least a threshold likelihood of providing voice input to the mobile computing device;

wherein determining whether to switch to the second mode of operation is additionally based on, at least, the determination of whether the user has at least the threshold likelihood of providing voice input.

10. The computer-implemented method of claim 1, further comprising:

determining, based on the context, whether monitoring audio data for voice input will have at least a threshold level of convenience for a user associated with the mobile computing device and for the mobile computing device;

wherein determining whether to switch to the second mode of operation is based on, at least, the determination of whether the user has at least the threshold likelihood of providing voice input.

11. The computer-implemented method of claim 10, wherein the context indicates at least the threshold level of convenience for the user when manual operation of the mobile computing device is not readily available.

12. The computer-implemented method of claim 1, wherein detecting the context, determining to switch to the second mode of operation, and activating the microphones and the speech analysis system is performed without direction from a user.

13. A system for automatically monitoring for voice input, the system comprising:

a mobile computing device;

one or more microphones that receive ambient audio signals and provide electronic audio data to the mobile computing device;

a context determination unit that detects a context associated with the mobile computing device, the context indicating that the mobile computing device is traveling in a vehicle based at least in part on a calculated speed of the mobile computing device;

a mode selection unit that determines, based on the context determined by the context determination unit, to switch the mobile computing device from a first mode of operation, during which the mobile computing device does not monitor ambient sounds for voice input that indicates a request to perform an operation, to a second mode of operation during which the mobile computing device monitors ambient sounds for voice input that indicates a request to perform an operation;

an input system that activates the one or more microphones and a speech analysis system associated with the mobile computing device in response to determining to switch to the second mode of operation; and

an output that provides output via the mobile computing device, the output being responsive to voice input that is detected in the stream of audio data and that indicates a request to perform an operation.

14. The system of claim 13, wherein the context further indicates a direction in which the device is traveling.

15. The system of claim 13 wherein the one or more microphones receive a stream of audio data, the speech analysis system monitors the stream of audio data for voice input, and upon detecting voice input from the audio stream, determines whether the voice input indicates a request for the mobile computing device to perform an operation;

and

wherein based on the determination of whether the voice input indicates a request to perform an operation, the output system causes the requested operation to be performed.

16. The system of claim 13 wherein the speech analysis system identifies whether any of a predetermined group of keywords is present in the voice input, and determines whether the voice input indicates a request to perform an operation based on the determination of whether any of the keywords are present in the voice input.

17. The system of claim 13 wherein the speech analysis system determines whether the voice input is a command or a question directed at the mobile computing device based on, at least, syntactical analysis of the voice input, and determines whether the voice input indicates a request to perform an operation based on, at least, the determination of whether the voice input is a command or a question directed at the mobile computing device.

18. The system of claim 13 wherein the speech analysis system identifying changes in a structure associated with the voice input, and determines whether the voice input is directed at the mobile computing device based on the identified changes;

wherein the determination of whether the voice input indicates a request to perform an operation is based on, at least, the determination of whether the voice input is directed at the mobile computing device.

19. The system of claim 13 wherein the input system is a subsystem of the mobile computing device, the speech analysis system is a subsystem of the mobile computing device, and the output system is a subsystem of the mobile computing device.

20. The system of claim 13, wherein the context indicates whether the device is accelerating.