AUGMENTATION OF KEY PHRASE USER RECOGNITION

• US 20160125879A1
• Filed: 08/14/2015
• Published: 05/05/2016
• Est. Priority Date: 11/03/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lovitt, Andrew William
• Granted Patent: US 10,262,655 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/10 Multimodal systems, i.e. based on the integration of multiple recognition engines or fusion of expert systems
• CPC Class Codes: G10L 17/24 the user being prompted to utter a password or a predefined phrase
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

Examples for augmenting user recognition via speech are provided. One example method comprises, on a computing device, monitoring a use environment via one or more sensors including an acoustic sensor, detecting utterance of a key phrase via data from the acoustic sensor, and based upon the selected data from the acoustic sensor and also on other environmental sensor data collected at different times than the selected data from the acoustic sensor, determining a probability that the key phrase was spoken by an identified user. The method further includes, if the probability meets or exceeds a threshold probability, then performing an action on the computing device.

Claims

1. On a computing device, a method comprising:

monitoring a use environment via one or more sensors including an acoustic sensor;

detecting utterance of a key phrase via selected data from the acoustic sensor;

based upon the selected data from the acoustic sensor and also on other environmental sensor data collected at different times than the selected data from the acoustic sensor, determining a probability that the key phrase was spoken by an identified user; and

if the probability meets or exceeds a threshold probability, then performing an action on the computing device.

2. The method of claim 1, wherein the other environmental sensor data comprises acoustic sensor data.

3. The method of claim 1, wherein the other environmental sensor data comprises image data.

4. The method of claim 3, further comprising identifying one or more persons in the use environment based on the image data, and wherein determining the probability comprises determining the probability based at least in part upon a determined identity of the one or more persons in the use environment.

5. The method of claim 1, wherein the other environmental sensor data comprises location data.

6. The method of claim 5, wherein the location data comprises proximity data from a proximity sensor.

7. The method of claim 5, wherein the location data comprises calendar information for the identified user.

8. The method of claim 1, further comprising detecting a user behavioral pattern, and wherein determining the probability comprises determining the probability based at least in part upon the user behavioral pattern.

9. The method of claim 8, wherein the user behavioral pattern comprises information regarding how often the identified user speaks.

10. A computing system, comprising:

one or more sensors including at least an acoustic sensor;

a logic machine; and

a storage machine holding instructions executable by the logic machine to

monitor a use environment via the one or more sensors including the acoustic sensor;

detect utterance of a key phrase via selected data from the acoustic sensor;

based upon the selected data from the acoustic sensor and also on other environmental sensor data collected at different times than the selected data from the acoustic sensor, determine a probability that the key phrase was spoken by an identified user; and

if the probability meets or exceeds a threshold probability, then perform an action on the computing system.

11. The computing system of claim 10, wherein the other environmental sensor data comprises image data, and wherein the instructions are further executable to identify one or more persons in the use environment based on the image data, and to determine the probability based at least in part upon a determined identity of the one or more persons in the use environment.

12. The computing system of claim 10, wherein the other environmental sensor data comprises location data, the location data comprising one or more of proximity data from a proximity sensor and calendar information for the identified user.

13. The computing system of claim 12, wherein the instructions are further executable to determine whether the identified user is scheduled to be in the use environment during a time that the utterance of key phrase was detected based on the calendar information, and if the identified user is scheduled to be in the use environment, increase the probability that the key phrase was spoken by the identified user.

14. The computing system of claim 10, wherein the instructions are further executable to detect a user behavioral pattern based upon prior user behaviors detected via environmental sensing, the user behavioral pattern including information regarding how frequently the identified user speaks, and to determine the probability based on the average frequency the identified user speaks.

15. The computing system of claim 10, wherein the other environmental sensor data collected at different times than the selected data from the acoustic sensor comprises additional acoustic data collected before and/or after the utterance of the key phrase.

16. The computing system of claim 15, wherein to determine the probability that the key phrase was spoken by the identified user, the instructions are further executable to analyze the additional acoustic data to determine if the identified user was also speaking before or after the key phrase was uttered, and

increase the probability that the key phrase was spoken by the identified user if the identified user was also speaking before or after the key phrase was uttered.

17. The computing system of claim 16, wherein the instructions are further executable to decrease the probability that the key phrase was spoken by the identified user if the analysis indicates the identified user was not speaking before or after the utterance of the key phrase.

18. A computing system, comprising:

one or more sensors including an acoustic sensor;

a processor configured to execute computer-readable instructions; and

a storage device comprising instructions stored thereon that are executable by the processor to

monitor a use environment via the one or more sensors including the acoustic sensor;

detect utterance of a key phrase followed by a command via selected data from the acoustic sensor, the key phrase configured to identify a user, the command configured to invoke a computing system action related to the user;

if an identity of a user who uttered the key phrase is determined based on the selected data from the acoustic sensor, then perform the action on the computing system for the identified user;

if an identity of a user who uttered the key phrase is not determined based on the selected data from the acoustic sensor, then determine a probability that the key phrase was uttered by one or more possible users based on other environmental sensor data collected at different times than the selected data from the acoustic sensor; and

if the probability meets or exceeds a threshold probability for a selected user of the one or more other possible users, then perform the action on the computing system for the selected user.

19. The computing system of claim 18, wherein the instructions are further executable to detect a user behavioral pattern based upon prior user behaviors detected via environmental sensing, and wherein determining the probability comprises determining the probability based at least in part upon the user behavioral pattern.

20. The computing system of claim 19, wherein the instructions executable to detect the user behavioral pattern comprise instructions executable to detect information regarding how often the identified user speaks.