Method, apparatus, and system for automatically monitoring for voice input based on context

• US 8,918,121 B2
• Filed: 12/06/2012
• Issued: 12/23/2014
• Est. Priority Date: 08/06/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: LeBeau, Michael J., Jitkoff, John Nicholas, Burke, Dave
• Primary Examiner(s): Sarwar, Babar
• Application Number:
  US13/706,696
• Publication Number:
  US 20130095805A1
• Time in Patent Office: 747 Days
• Field of Search: 4554041-4042, 455/414.1, 455/418, 455/420, 455/550.1, 704/235, 704/233, 704/275, 704/E15.044, 704/E15.039
• US Class Current: 455/456.3
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: H04M 1/04 Supports for telephone transmitters or receivers
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 1/72409 by interfacing with external accessories hands-free H04M1/60
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/046 involving vehicles, e.g. floating traffic data [FTD] or vehicle traffic prediction

Abstract

In one implementation, a computer-implemented method includes detecting a current context associated with a mobile computing device and determining, based on the current context, whether to switch the mobile computing device from a current mode of operation to a second mode of operation during which the mobile computing device monitors ambient sounds for voice input that indicates a request to perform an operation. The method can further include, in response to determining whether to switch to the second mode of operation, activating one or more microphones and a speech analysis subsystem associated with the mobile computing device so that the mobile computing device receives a stream of audio data. The method can also include providing output on the mobile computing device that is responsive to voice input that is detected in the stream of audio data and that indicates a request to perform an operation.

Claims

1. A method comprising:

detecting a context associated with a mobile computing device;

determining, based at least on the context associated with the mobile computing device, to automatically switch from a first mode of operation in which ambient sounds are not monitored to determine whether the ambient sounds include one or more trigger terms that correspond to a request from a user for the mobile computing device to perform an operation, to a second mode of operation in which the ambient sounds are monitored to determine whether the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform an operation;

in response to determining to automatically switch to the second mode of operation, activating one or more microphones to receive the ambient sounds, and activating a speech analysis system for determining whether the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform the operation;

receiving data indicating that the speech analysis system has determined that the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform the operation; and

based at least on receiving the data indicating that the speech analysis system has determined that the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform the operation, providing an output that is responsive to the request.

2. The method of claim 1 including using the mobile computing device in conjunction with a separate computer system to monitor for voice input indicating the request from the user for the mobile computing device to perform an operation.

3. The automated method of claim 1 including processing the request from the user for the mobile computing device to perform operation by the mobile computing device interacting with an information server system.

4. The method of claim 1 including providing the output on an output device separate from the mobile computing device.

5. The method of claim 1 including providing the output on a display.

6. The method of claim 1 wherein the context associated with the mobile computing device indicates a physical location at which the mobile computing device is located.

7. The method of claim 1 wherein the context associated with the mobile computing device indicates that the mobile computing device is located in a vehicle.

8. The method of claim 1 including detecting that the mobile computing device is traveling in the vehicle based at least in part on a calculated speed of the mobile computing device.

9. The method of claim 1 further including:

automatically determining to switch from the second mode of operation to the first mode of operation; and

in response to automatically determining to switch to the first mode of operation, deactivating the one or more microphones and deactivating the speech analysis system.

10. The method of claim 1 including, after activating the speech analysis system, using the speech analysis system to analyze and determine whether any voice input is present in the audio data received by the one or microphones.

11. The method of claim 1 including displaying a message indicating that audio signal monitoring currently is taking place.

12. A system comprising:

one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising;

detecting a context associated with a mobile computing device;

determining, based at least on the context associated with the mobile computing device, to automatically switch from a first mode of operation in which ambient sounds are not monitored to determine whether the ambient sounds include one or more trigger terms that correspond to a request from a user for the mobile computing device to perform an operation, to a second mode of operation in which the ambient sounds are monitored to determine whether the ambient sounds include the one or more trigger terms that correspond to the request from a user for the mobile computing device to perform an operation;

in response to determining to automatically switch to the second mode of operation, activating one or more microphones to receive the ambient sounds, and activating a speech analysis system for determining whether the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform the operation;

receiving data indicating that the speech analysis system has determined that the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform the operation; and

based at least on receiving the data indicating that the speech analysis system has determined that the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform the operation, providing an output that is responsive to the request.

13. The system of claim 12, the operations including using the mobile computing device in conjunction with a separate computer system to monitor for voice input indicating the request from the user for the mobile computing device to perform an operation.

14. The system of claim 12, the operations including processing the request from the user for the mobile computing device to perform an operation by the mobile computing device interacting with an information server system.

15. The system of claim 12, the operations including providing the output on an output device separate from the mobile computing device.

16. The system of claim 12, the operations including providing the output on a display.

17. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

detecting a context associated with a mobile computing device;

determining, based at least on the context associated with the mobile computing device, to automatically switch from a first mode of operation in which ambient sounds are not monitored to determine whether the ambient sounds include one or more trigger terms that correspond to a request from a user for the mobile computing device to perform an operation, to a second mode of operation in which the ambient sounds are monitored to determine whether the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform an operation;

in response to determining to automatically switch to the second mode of operation, activating one or more microphones to receive the ambient sounds, and activating a speech analysis system for determining whether the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform the operation;

receiving data indicating that the speech analysis system has determined that the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform the operation; and

based at least on receiving the data indicating that the speech analysis system has determined that the ambient sounds include the one or more trigger terms that correspond to the request from the user for the mobile computing device to perform the operation, providing an output that is responsive to the request.

18. The system of claim 17, the operations including using the mobile computing device in conjunction with a separate computer system to monitor for voice input indicating the request from the user for the mobile computing device to perform an operation.

19. The system of claim 17, the operations including processing the request from the user for the mobile computing device to perform an operation by the mobile computing device interacting with an information server system.

20. The system of claim 17, the operations including providing the output on an output device separate from the mobile computing device.