Real-time interactive voice recognition and response over the internet

• US 9,947,321 B1
• Filed: 02/21/2017
• Issued: 04/17/2018
• Est. Priority Date: 10/04/1999
• Status: Expired due to Fees

• Current Assignee: Pearson Education Incorporated (Pearson plc)
• Original Assignee: Pearson Education Incorporated (Pearson plc)
• Inventors: Jochumson, Christopher S.
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US15/438,096
• Time in Patent Office: 420 Days
• Field of Search: 704235, 704236, 704270, 7042701, 704275, 434185
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 19/0018 Speech coding using phonetic or linguistical decoding of the source; Reconstruction using text-to-speech synthesis

Abstract

Methods and systems for handling speech recognition processing in effectively real-time, via the Internet, in order that users do not experience noticeable delays from the start until they receive responsive feedback. A user uses a client to access the Internet and a server supporting speech recognition processing. The user inputs speech to the client, which transmits the user speech to the server in approximate real-time. The server evaluates the user speech, and provides responsive feedback to the client, again, in approximate real-time, with minimum latency delays. The client upon receiving responsive feedback from the server, displays, or otherwise provides, the feedback to the user.

Claims

1. A method comprising:

receiving, by a server from a client device, a data transmission including one or more packets encoding voice data, a first value, and a question count identifier;

processing, by the server, the voice data to generate a response to the voice data, wherein the server processes the voice data for a period of time at least partially determined by the first value; and

transmitting, by the server in response to the data transmission, the response to the voice data to the client device, wherein the response is determined by;

identifying a portion of an audio file using the question count identifier, and

transmitting the portion of the audio file to the client device.

2. The method of claim 1, wherein the first value includes an accuracy value and the server is configured to process the voice data for an amount of processor time determined by the accuracy value.

3. The method of claim 1, wherein the first value includes a threshold value and the server is configured to process the voice data by applying a threshold level of recognition to a spoken word in the voice data, wherein the threshold level of recognition is determined by the threshold value.

4. The method of claim 1, wherein the response includes text data and the client device is configured to process the text data into audio data.

5. The method of claim 1, wherein transmitting the response to the voice data to the client device over the Internet includes causing a display of visual information on a screen of the client device.

6. The method of claim 1, wherein transmitting the response to the voice data to the client device over the Internet includes causing an output of audio information on an audio output device of the client device.

7. The method of claim 1, wherein the response is communicated from server to a client over the Internet, and causes a speech output on an audio output device of the client device.

8. The method of claim 1, wherein transmitting the response to the voice data to the client device over the Internet includes causing an output of audio and visual information by the client device.

9. A method, comprising:

transmitting, by a client device to a server, a data transmission including one or more packets encoding voice data and a first value;

receiving, by the client device from the server, a response to the voice data, wherein the voice data has been processed by the server for a period of time at least partially determined by the first value;

transmitting, to the server and after receiving the response to the voice data, a request for a correct speech file; and

receiving, from the server, an audio file that is the correct speech file.

10. The method of claim 9, wherein the first value includes an accuracy value and the server is configured to process the voice data for an amount of processor time determined by the accuracy value.

11. The method of claim 9, wherein the first value includes a threshold value and the server is configured to process the voice data by applying a threshold level of recognition to a spoken word in the voice data, wherein the threshold level of recognition is determined by the threshold value.

12. The method of claim 9, wherein the response includes text data and including processing the text data into audio data.

13. The method of claim 9, including:

processing the response to the voice data into visual information; and

displaying the visual information on a screen of the client device.

14. The method of claim 9, including:

processing the response to the voice data into audio information; and

outputting the audio information on an audio output device of the client device.

15. The method of claim 9, including:

processing the response to the voice data into speech output; and

outputting the speech output on an audio output device of the client device.

16. A system, comprising:

a server including a processor configured to;

receive a data transmission from a client device, the data transmission including one or more packets encoding voice data, a first value, and a question count identifier;

process the voice data to generate a response to the voice data, wherein the voice data is processed for a period of time at least partially determined by the first value; and

transmit, in response to the data transmission, the response to the voice data to the client device, wherein the response is determined by;

identifying a portion of an audio file using the question count identifier, and

transmitting the portion of the audio file to the client device.

17. The system of claim 16, wherein the first value includes an accuracy value and the processor is configured to process the voice data for an amount of processor time determined by the accuracy value.

18. The system of claim 16, wherein the first value includes a threshold value and the processor is configured to process the voice data by applying a threshold level of recognition to a spoken word in the voice data, wherein the threshold level of recognition is determined by the threshold value.

19. The system of claim 16, wherein the processor is configured to:

execute one or more speech processing threads; and

input the voice data into the one or more speech processing threads for processing.

20. The system of claim 16, including a storage device configured to store a plurality of response data files and wherein processing the voice data to generate a response to the voice data includes identifying a first response data file in the plurality of response data files using the voice data and the response to the voice data includes at least a portion of the first response data file.