Speech Recognition

• US 20170256259A1
• Filed: 08/05/2016
• Published: 09/07/2017
• Est. Priority Date: 03/01/2016
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Froelich, Raymond J.
• Application Number:
  US15/229,916
• Publication Number:
  US 20170256259A1
• Time in Patent Office: Days
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/222 Barge in, i.e. overridable guidance for interrupting prompts
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

A computer system comprises an input configured to receive voice input from a user, the voice input having speech intervals separated by non-speech intervals; an ASR system configured to identify individual words in the voice input during speech intervals thereof, and store the identified words in memory; a response generation module configured to generate based on the words stored in the memory an audio response for outputting to the user; and a response delivery module configured to begin outputting the audio response to the user during a non-speech interval of the voice input, wherein the outputting of the audio response is terminated before it has completed in response to a subsequent speech interval of the voice input commencing whilst the audio response is still being outputted.

Claims

1. A computer system comprising:

an input configured to receive voice input from a user, the voice input having speech intervals separated by non-speech intervals;

an ASR system configured to identify individual words in the voice input during speech intervals thereof, and store the identified words in memory;

a response generation module configured to generate based on the words stored in the memory an audio response for outputting to the user; and

a response delivery module configured to begin outputting the audio response to the user during a non-speech interval of the voice input, wherein the outputting of the audio response is terminated before it has completed in response to a subsequent speech interval of the voice input commencing whilst the audio response is still being outputted.

2. A computer system according to claim 1, comprising a speech detection module configured to configured monitor the set of identified words in the memory as it is updated by the ASR system, and detect the commencement of the subsequent speech interval based on the monitoring.

3. A computer system according to claim 2, wherein the commencement is detected by detecting that a newly-identified word has been added to the set by the ASR system.

4. A computer system according to claim 2, wherein the speech recognition module is configured to detect the non-speech interval based on the monitoring.

5. A computer system according to claim 4, further comprising a language model, wherein detecting the non-speech interval comprises detecting, by the speech detection module, when the set of identified words forms a grammatically complete sentence according to the language model.

6. A computer system according to claim 4, wherein detecting the non-speech interval comprises detecting, by the speech detection module, that no new words have been identified by the ASR system for a pre-determined duration, wherein the interval of time commences with the expiry of the pre-determined duration.

7. A computer system according to claim 1, wherein the response is an audio response for playing out to the user in audible form.

8. A computer system according to claim 1, comprising a video generation module configured to, in response to the response module determining that the ASR system has not identified any more words in the interval of time, output to the user a visual indication that the outputting of the response is about to begin.

9. A computer system according to claim 8, wherein the video generation module is configured to generate and output to the user a moving image of an avatar, wherein the visual indication is a visual action performed by the avatar.

10. A computer system according to claim 1, wherein each word of the set is stored in the memory as a string of one or more characters.

11. A computer system according to claim 1, further comprising:

a lookup module configured to receive at least one word from the set in the memory at a first time whilst updates to the set by the ASR system are still ongoing, and perform a lookup to pre-retrieve information associated with the at least one word;

wherein the response generation module is configured to access the set in the memory at a later time, the set having been updated by the ASR system at least once between the first time and the later time, the response being generated by the response generation module based on the set as accessed at the later time, wherein the response conveys the information pre-retrieved by the lookup module.

12. A computer system according to claim 1, further comprising:

a speech overload detection module configured to detect at a time during a speech interval of the voice input a speech overload condition; and

a notification module configured to output to the user, in response to said to detection, a notification of the speech overload condition, wherein the overload condition is detected based on;

a number of words that the ASR system has identified so far in that speech interval, and/or

a rate at which words are being identified by the ASR system in that speech interval, and/or

a state of an AI tree being driven by the voice input.

13. A computer-implemented method of effecting communication between a user and an artificial intelligence software agent executed on a computer, the method comprising:

receiving at an ASR system voice input from a user, the voice input having speech intervals separated by non-speech intervals;

identifying, by the ASR system, individual words in the voice input during speech intervals thereof, and storing the identified words in memory; and

generating, by the software agent, based on the words stored in the memory an audio response for outputting to the user;

wherein the software agent begins outputting the audio response to the user during a non-speech interval of the voice input, wherein the outputting of the audio response is terminated before it has completed in response to a subsequent speech interval of the voice input commencing whilst the audio response is still being outputted.

14. A method according to claim 13, wherein the voice input is received from a user device via a communication network, wherein the outputting step is performed by the software agent transmitting the response to the user device via the network so as to cause the user device to output the response to the user.

15. A method according to claim 14, wherein the voice input is received from and the response outputted to the user in real-time, thereby effecting a real-time communication event between the user and the software agent via the network.

16. A method according to claim 13, the method being implemented in a communication system, wherein the communication system comprises a user account database storing, for each of a plurality of users of the communication system, a user identifier that uniquely identifies that user within the communication system;

wherein a user identifier of the software agent is also stored in the user account database so that the software agents appears as another user of the communication system.

17. A method according to claim 13, further comprising:

monitoring the set of identified words in the memory as it is updated by the ASR system, wherein the speech and non-speech intervals are detected based on the monitoring of the set of identified words.

18. A method according to claim 13, wherein the response is an audio response for playing out to the user in audible form.

19. A computer program product comprising an artificial intelligence software agent stored on a computer readable storage medium, the software agent for communicating with a user based on the output of an ASR system, the ASR system configured to identify individual words in a voice input during speech intervals thereof, and store the identified words in memory, the software agent configured when executed to:

generate based on the words stored in the memory an audio response for outputting to the user; and

begin outputting the audio response to the user during a non-speech interval of the voice input, wherein the outputting of the audio response is terminated before it has completed in response to a subsequent speech interval of the voice input commencing whilst the audio response is still being outputted.

20. A computer program product according to claim 19, wherein the response is outputted to the user by transmitting it to a user device available to the user via a network so as to cause the user device to output the response to the user.