Client-server speech recognition with processing level based on value received from client

• US 9,111,541 B1
• Filed: 02/28/2012
• Issued: 08/18/2015
• Est. Priority Date: 10/04/1999
• Status: Expired due to Term

• Current Assignee: Pearson Education Incorporated (Pearson plc)
• Original Assignee: GlobalEnglish Corporation (Pearson plc)
• Inventors: Jochumson, Christopher S.
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US13/407,611
• Time in Patent Office: 1,267 Days
• Field of Search: 704/255, 704/270, 704/270.1, 704/275
• US Class Current: 1/1
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 19/0018 Speech coding using phonetic or linguistical decoding of the source; Reconstruction using text-to-speech synthesis

Abstract

Methods and systems for handling speech recognition processing in effectively real-time, via the Internet, in order that users do not experience noticeable delays from the start until they receive responsive feedback. A user uses a client to access the Internet and a server supporting speech recognition processing. The user inputs speech to the client, which transmits the user speech to the server in approximate real-time. The server evaluates the user speech using processing level based on the value received from each client.

Claims

1. A method comprising:

at a first client comprising a capability to receive audio speech from a user, after receiving audio speech, encoding the received audio speech;

from the first client, after encoding the received audio speech, transmitting packets of encoded audio speech along with a first value over the Internet;

at a second client comprising a capability to receive audio speech from a user, after receiving audio speech, encoding the received audio speech;

from the second client, after encoding the received audio speech, transmitting packets of encoded audio speech along with a second value over the Internet;

at a server, receiving packets of encoded audio speech from two or more clients; and

at the server, servicing the encoded audio speech in an amount of processing time based on the first value received from the first client and the second value received from the second client,

wherein the server comprises the capability to transmit responses to the first and second clients, and the response is a result of the server'"'"'s servicing of the encoded audio speech based on the first and second values received from the clients.

2. The method of claim 1 wherein a client of the two or more clients comprises the capability to receive the response from the server.

3. The method of claim 1 wherein a client further comprises an audio output device, and the capability to receive the packets of text format, convert the packets of text format to audio data and play the audio data to a user.

4. The method of claim 1 wherein the response is communicated from server to a client over the Internet, and causes displaying of visual information on a screen of the client.

5. The method of claim 1 wherein the response is communicated from server to a client over the Internet, and causes output of audio information on an audio output device of the client.

6. The method of claim 1 wherein the response is communicated from server to a client over the Internet, and causes a speech output on an audio output device of the client.

7. The method of claim 1 wherein the response is communicated from server to a client over the Internet, and causes output of audio and visual information at the client.

8. The method of claim 1 wherein the server comprises two or more stored files, and the server selects a file to transmit to a client of the two or more clients as a result of the server'"'"'s servicing of the encoded audio speech received from the client.

9. A method of on-line speech recognition comprising:

at a first client, after receiving audio speech from a user, encoding the audio speech before all of the audio speech is received;

at the first client comprising the capability to receive audio speech from a user in an uncompressed audio format, storing the encoded speech in one or more buffers;

at the first client, packaging the encoded audio speech into one or more packets to be transmitted over the Internet before all of the audio speech is received;

from the first client, transmitting a packet of encoded audio speech along with a first value over the Internet before all of the audio speech is received;

at a second client comprising the capability to receive audio speech from a user in an uncompressed audio format, after receiving audio speech from a user, encoding the audio speech before all of the audio speech is received;

at the second client, storing the encoded speech in one or more buffers;

at the second client, packaging the encoded audio speech into one or more packets to be transmitted over the Internet before all of the audio speech is received;

from the second client, transmitting a packet of encoded audio speech along with a second value over the Internet before all of the audio speech is received;

providing a server, the server comprising the capability to receive packets of encoded audio speech from two or more clients;

at the server, decoding each of the packets of audio speech and storing the resultant speech into one or more buffers for the respective client; and

at the server, evaluating the resultant speech received from each of the two or more clients,

wherein the server comprises the capability to transmit a response to the first and second clients, and the response is a result of the server'"'"'s evaluating of the resultant speech in an amount of processing time based on the first and second values received from the clients.

10. The method of claim 9 wherein a client of the two or more clients further comprises the capability to receive the response from the server.

11. The method of claim 9 wherein a client further comprises an audio output device, and the capability to receive the packets of text format, convert the packets of text format to audio data and play the audio data to a user.

12. The method of claim 9 wherein the response is communicated from server to a client over the Internet, and causes displaying of visual information on a screen of the client.

13. The method of claim 9 wherein the response is communicated from server to a client over the Internet, and causes output of audio information on an audio output device of the client.

14. The method of claim 9 wherein the response is communicated from server to a client over the Internet, and causes a speech output on an audio output device of the client.

15. The method of claim 9 wherein the response is communicated from server to a client over the Internet, and causes output of audio and visual information at the client.

16. The method of claim 9 wherein the server comprises two or more stored files, and the server selects a file to transmit to a client of the two or more clients as a result of the server'"'"'s evaluation of the resultant speech received from the client.

17. The method of claim 9 wherein the server comprises the capability to partition a stored text format file into two or more packets for the transmission over the Internet, and to transmit each packet over the Internet to a client.

18. The method of claim 9 wherein the server, based on its evaluation of the resultant speech from a first client of the two or more two clients, generates a textual response and sends the textual response to the first client over the Internet.

19. A computer program product stored in a non-transitory storage medium providing program instructions comprising:

first executable code executing at two or more clients, the first executable code enabling each client the capability to receive audio speech from a user, store the audio speech in one or more buffers in an uncompressed audio format, each buffer comprising a portion of the received audio speech, encode the stored audio speech in the one or more buffers, package the encoded audio speech into one or more packets to be transmitted over the Internet, and transmit a packet of encoded audio speech along with a respective value for each client over the Internet; and

second executable code executing at a server, the executable code enabling the server the capability to receive packets of encoded audio speech from the two or more clients, decode each of the packets of audio speech and store the resultant speech into one or more buffers for the respective client, and evaluate the resultant speech received from each of the two or more clients in an amount of processing time based on the respective value received from a respective client, and the server has the capability to transmit a response to a client based on the evaluated speech speech.

20. The computer program product of claim 19 wherein the response is communicated from server to a client over the Internet, and causes a speech output on an audio output device of the client.