Method, apparatus, and system for automatically monitoring for voice input based on context

• US 9,251,793 B2
• Filed: 07/07/2015
• Issued: 02/02/2016
• Est. Priority Date: 08/06/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: LeBeau, Michael J., Jitkoff, John Nicholas, Burke, Dave
• Primary Examiner(s): Sarwar, Babar
• Application Number:
  US14/793,253
• Publication Number:
  US 20150310867A1
• Time in Patent Office: 210 Days
• Field of Search: 455/436, 455/411, 455418-4221, 455/456.1, 4554041-4042, 455/414.1, 455/420, 455/550.1, 370/328, 370/338
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: H04M 1/04 Supports for telephone transmitters or receivers
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 1/72409 by interfacing with external accessories hands-free H04M1/60
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/046 involving vehicles, e.g. floating traffic data [FTD] or vehicle traffic prediction

Abstract

In one implementation, a computer-implemented method includes detecting a current context associated with a mobile computing device and determining, based on the current context, whether to switch the mobile computing device from a current mode of operation to a second mode of operation during which the mobile computing device monitors ambient sounds for voice input that indicates a request to perform an operation. The method can further include, in response to determining whether to switch to the second mode of operation, activating one or more microphones and a speech analysis subsystem associated with the mobile computing device so that the mobile computing device receives a stream of audio data. The method can also include providing output on the mobile computing device that is responsive to voice input that is detected in the stream of audio data and that indicates a request to perform an operation.

Claims

1. A computer-implemented method comprising:

determining a context associated with a mobile computing device;

classifying the context as a pre-defined context that indicates a likelihood that ambient sounds will include one or more trigger terms that correspond to a request from a user for the computing device to perform an operation satisfies a likelihood threshold;

in response to classifying the context as a pre-defined context that indicates the likelihood that ambient sounds will include one or more trigger terms that correspond to a request from a user for the computing device to perform an operation satisfies the likelihood threshold, switching the computing device to a mode of operation in which the computing device monitors the ambient sounds for one or more trigger terms that correspond to a request from the user for the computing device to perform an operation;

receiving data indicating that the ambient sounds include one or more trigger terms that correspond to a request from the user for the mobile computing device to perform the operation; and

in response to receiving data indicating that the ambient sounds include one or more trigger terms that correspond to a request from the user for the mobile computing device to perform the operation, providing an output that is responsive to the request.

2. The method of claim 1, comprising:

receiving input that defines the pre-defined context from the user.

3. The method of claim 1, comprising:

determining that the user provided a request for the computing device to perform an operation in a particular context; and

in response to determining that the user provided the request for the computing device to perform the operation in the particular context, defining the particular context as the pre-defined context.

4. The method of claim 1, wherein switching the computing device to a mode of operation in which the computing device monitors the ambient sounds for one or more trigger terms that correspond to a request from the user for the computing device to perform an operation comprises:

switching the computing device from a prior mode of operation in which the computing device does not monitor the ambient sounds for one or more trigger terms that correspond to a request from the user for the computing device to perform an operation.

5. The method of claim 4, comprising:

detecting the context associated with the mobile computing device has changed to another context; and

in response to detecting the context associated with the mobile computing device has changed to the other context, switching the computing device back to the prior mode of operation in which the computing device does not monitor the ambient sounds for one or more trigger terms that correspond to a request from the user for the computing device to perform an operation.

6. The method of claim 1, comprising:

using the mobile computing device in conjunction with a separate computer system to monitor for voice input indicating the request from the user for the mobile computing device to perform an operation.

7. The method of claim 1, comprising:

processing the request from the user for the mobile computing device to perform an operation by the mobile computing device interacting with an information server system.

8. The method of claim 1, comprising:

providing the output on an output device separate from the mobile computing device.

9. The method of claim 1, comprising:

providing the output on a display.

10. The method of claim 1, wherein the context associated with the mobile computing device indicates a physical location at which the mobile computing device is located.

11. The method of claim 1, wherein the context associated with the mobile computing device indicates that the mobile computing device is located in a vehicle.

12. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

determining a context associated with a mobile computing device;

classifying the context as a pre-defined context that indicates a likelihood that ambient sounds will include one or more trigger terms that correspond to a request from a user for the computing device to perform an operation satisfies a likelihood threshold;

in response to classifying the context as a pre-defined context that indicates the likelihood that ambient sounds will include one or more trigger terms that correspond to a request from a user for the computing device to perform an operation satisfies the likelihood threshold, switching the computing device to a mode of operation in which the computing device monitors the ambient sounds for one or more trigger terms that correspond to a request from the user for the computing device to perform an operation;

receiving data indicating that the ambient sounds include one or more trigger terms that correspond to a request from the user for the mobile computing device to perform the operation; and

in response to receiving data indicating that the ambient sounds include one or more trigger terms that correspond to a request from the user for the mobile computing device to perform the operation, providing an output that is responsive to the request.

13. The system of claim 12, the instructions comprising:

receiving input that defines the pre-defined context from the user.

14. The system of claim 12, the instructions comprising:

determining that the user provided a request for the computing device to perform an operation in a particular context; and

in response to determining that the user provided the request for the computing device to perform the operation in the particular context, defining the particular context as the pre-defined context.

15. The system of claim 12, wherein switching the computing device to a mode of operation in which the computing device monitors the ambient sounds for one or more trigger terms that correspond to a request from the user for the computing device to perform an operation comprises:

switching the computing device from a prior mode of operation in which the computing device does not monitor the ambient sounds for one or more trigger terms that correspond to a request from the user for the computing device to perform an operation.

16. The system of claim 15, the instructions comprising:

detecting the context associated with the mobile computing device has changed to another context; and

in response to detecting the context associated with the mobile computing device has changed to the other context, switching the computing device back to the prior mode of operation in which the computing device does not monitor the ambient sounds for one or more trigger terms that correspond to a request from the user for the computing device to perform an operation.

17. The system of claim 12, the instructions comprising:

using the mobile computing device in conjunction with a separate computer system to monitor for voice input indicating the request from the user for the mobile computing device to perform an operation.

18. The system of claim 12, the instructions comprising:

processing the request from the user for the mobile computing device to perform an operation by the mobile computing device interacting with an information server system.

19. The system of claim 12, the instructions comprising:

providing the output on an output device separate from the mobile computing device.

20. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such

execution, cause the one or more computers to perform operations comprising:

determining a context associated with a mobile computing device;

classifying the context as a pre-defined context that indicates a likelihood that ambient sounds will include one or more trigger terms that correspond to a request from a user for the computing device to perform an operation satisfies a likelihood threshold;

in response to classifying the context as a pre-defined context that indicates the likelihood that ambient sounds will include one or more trigger terms that correspond to a request from a user for the computing device to perform an operation satisfies the likelihood threshold, switching the computing device to a mode of operation in which the computing device monitors the ambient sounds for one or more trigger terms that correspond to a request from the user for the computing device to perform an operation;

receiving data indicating that the ambient sounds include one or more trigger terms that correspond to a request from the user for the mobile computing device to perform the operation; and

in response to receiving data indicating that the ambient sounds include one or more trigger terms that correspond to a request from the user for the mobile computing device to perform the operation, providing an output that is responsive to the request.