Speech recognition

• US 10,140,988 B2
• Filed: 08/05/2016
• Issued: 11/27/2018
• Est. Priority Date: 03/01/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Froelich, Raymond J.
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US15/229,978
• Publication Number:
  US 20170256261A1
• Time in Patent Office: 844 Days
• Field of Search: 704251, 704275, 704236, 704231, 704235, 704238, 704243, 704244, 704249, 704254, 704257, 704270, 7042701, 704 9, 715751
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/222 Barge in, i.e. overridable guidance for interrupting prompts
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

A computer system comprises an input configured to receive voice input from a user, the voice input having speech intervals separated by non-speech intervals; an ASR system configured to identify individual words in the voice input during speech intervals of the voice input, and store the identified words in memory; a speech overload detection module configured to detect at a time during a speech interval of the voice input a speech overload condition; and a notification module configured to output to the user, in response to said to detection, a notification of the speech overload condition.

Claims

1. A computer system comprising:

an input configured to receive voice input from a user, the voice input having speech intervals separated by non-speech intervals;

a system configured to identify individual words in the voice input during the speech intervals of the voice input, and store the identified individual words in memory;

a speech overload detection module configured to detect a speech overload condition at a time during a speech interval of the voice input, the speech overload condition being based on one or more of a rate at which words are identified during the speech interval, or a state of an artificial intelligence (AI) tree being driven by the voice input; and

a notification module configured to output, in response to the speech overload detection module detecting the speech overload condition, a notification of the speech overload condition.

2. A computer system according to claim 1, wherein the speech overload condition is detected based on a number of words that the system has identified so far in the speech interval.

3. A computer system according to claim 1, wherein the notification module is configured to output the notification during the speech interval whilst the user is speaking.

4. A computer system according to claim 3, wherein the notification comprises an audible notification.

5. A computer system according to claim 1, comprising a video generation module, wherein the notification comprises a visual notification by the video generation module.

6. A computer system according to claim 5, wherein the video generation module is configured to generate and output a moving image of an avatar, wherein the visual notification is a visual action performed by the avatar.

7. A computer system according to claim 1, comprising a speech detection module configured to detect the speech and non-speech intervals.

8. A computer system according to claim 7, wherein the speech detection module is configured to monitor the identified individual words in the memory as it is updated by the system.

9. A computer system according to claim 1, wherein each word of the identified individual words is stored in the memory as a string of one or more characters.

10. A computer system according to claim 1, further comprising:

a response module configured to generate and output a response based on the identified individual words in the speech interval.

11. A computer system according to claim 10, wherein the response module is configured to output the response during a next non-speech interval if no speech overload condition is detected in the speech interval.

12. A computer-implemented method comprising:

receiving at a system voice input from a user, the voice input having speech intervals separated by non-speech intervals;

identifying, by the system, individual words in the voice input during speech intervals of the voice input, and storing the identified individual words in memory;

detecting a speech overload condition at a time during a speech interval of the voice input, the speech overload condition being based on one or more of a rate at which words are identified during the speech interval, or a state of an artificial intelligence (AI) tree being driven by the voice input; and

outputting, in response to detecting the speech overload condition, a notification of the speech overload condition.

13. A method according to claim 12, wherein the voice input is received from a user device via a communication network, wherein the outputting step is performed by a software agent transmitting the notification to the user device via the network so as to cause the user device to output the notification to the user.

14. A method according to claim 13, wherein the voice input is received and the notification is outputted in real-time, thereby effecting a real-time communication event between the user device and the software agent via the network.

15. A method according to claim 12, the method being implemented in a communication system, wherein the communication system comprises a user account database storing, for each of a plurality of users of the communication system, a user identifier that uniquely identifies each user within the communication system;

wherein a user identifier of a software agent is also stored in the user) account database so that the software agents appears as another user of the communication system.

16. A method according to claim 12, further comprising:

monitoring the identified individual words in the memory as the memory is updated by the system, wherein the speech overload condition is detected based on the monitoring of the identified individual words.

17. A computer-readable memory storage device) storing a software agent, the software agent being configured when executed to perform operations comprising:

detecting at a time during a speech interval of voice input a speech overload condition the speech overload condition being based on one or more of a rate at which words are identified during the speech interval, or a state of an artificial intelligence (AI) tree being driven by the voice input; and

outputting, in response to detecting the speech overload condition, a notification of the speech overload condition.

18. A computer-readable memory storage device according to claim 17, wherein the notification is output by a user device.

19. A computer-readable memory storage device according to claim 17, wherein said outputting comprises outputting the notification during the speech interval.

20. A computer-readable memory storage device according to claim 17, wherein the state of the AI tree comprises an indication that an end of the AI tree has been reached.