Timing of speech recognition over lossy transmission systems

• US 7,496,503 B1
• Filed: 12/18/2006
• Issued: 02/24/2009
• Est. Priority Date: 06/30/1998
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Marcus, Stephen Michael, Rahim, Mazin G., Sharp, Robert Douglas, Seshadri, Nambirajan, Cox, Richard Vandervoort
• Primary Examiner(s): Vo, Huyen X.
• Application Number:
  US11/611,983
• Time in Patent Office: 799 Days
• Field of Search: 704/231, 704/233, 704/235, 704/240, 704/270, 704/270.1, 704/201, 704/244, 704/243, 704/245, 704/250
• US Class Current: 704/201
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence

Abstract

Recognizing a stream of speech received as speech vectors over a lossy communications link includes constructing for a speech recognizer a series of speech vectors from packets received over a lossy packetized transmission link, wherein some of the packets associated with each speech vector are lost or corrupted during transmission. Each constructed speech vector is multi-dimensional and includes associated features. After waiting for a predetermined time, speech vectors are generated and potentially corrupted features within the speech vector are indicated to the speech recognizer when present. Speech recognition is attempted at the speech recognizer on the speech vectors when corrupted features are present. This recognition may be based only on certain or valid features within each speech vector. Retransmission of a missing or corrupted packet is requested when corrupted values are indicated by the indicating step and when the attempted recognition step fails.

Claims

1. A method of recognizing speech, the method comprising:

receiving packets associated with input speech transmitted over a link at a buffering and decoding unit;

after waiting a predetermined time, generating from the received packets a speech vector and data identifying corrupt or likely to be corrupt features in the speech vector;

transmitting the speech vector and data to a speech recognition unit;

comparing the speech vector to stored recognition models based on non-corrupt features in the speech vector; and

either recognizing the speech input or requesting a retransmission of at least one packet based on the comparison.

2. The method of claim 1, further comprising:

determining whether features within the speech vector are missing or corrupt; and

if so, then requesting retransmission of packets associated with the missing or corrupt features; and

if not, then requesting either that a speaker of the input speech make another utterance or to negotiate a higher bandwidth over the link.

3. The method of claim 2, wherein negotiation of higher bandwidth comprises at least one of an alteration of compression techniques, more robust error correction, use of partially redundant parallel data streams, or the use of principle component analysis.

4. The method of claim 2, wherein the negotiation is transparent to the speaker.

5. The method of claim 1, wherein the method further comprises:

based on the comparison step, determining a plurality of probabilities associated with a likelihood that the speech vector is associated with each stored recognition model.

6. The method of claim 1, wherein the predetermined time is based on at least one of:

assumptions about the link, present network conditions and/or a tradeoff between recognizing speech in real time and receiving error free data.

7. A system for recognizing speech, the system comprising:

a module configured to receive packets associated with input speech transmitted over a link at a buffering and decoding unit;

a module configured to, after waiting a predetermined time, generate from the received packets a speech vector and data identifying corrupt or likely to be corrupt features in the speech vector;

a module configured to transmit the speech vector and data to a speech recognition unit;

a module configured to compare the speech vector to stored recognition models based on non-corrupt features in the speech vector; and

a module configured to either recognize the speech input or request a retransmission of at least one packet based on the comparison.

8. The system of claim 7, further comprising:

a module configured to determine whether features within the speech vector are missing or corrupt; and

if so, then request retransmission of packets associated with the missing or corrupt features; and

if not, then request either that a speaker of the input speech make another utterance or a negotiate a higher bandwidth over the link.

9. The system of claim 8, wherein negotiation of higher bandwidth comprises at least one of an alteration of compression techniques, more robust error correction, use of partially redundant parallel data streams, or the use of principle component analysis.

10. The system of claim 8, wherein the negotiation is transparent to the speaker.

11. The system of claim 7, further comprising:

a module configured to, based on the comparison, determine a plurality of probabilities associated with a likelihood that the speech vector is associated with each stored recognition model.

12. The system of claim 7, wherein the predetermined time is based on at least one of:

assumptions about the link, present network conditions and/or a tradeoff between recognizing speech in real time and receiving error free data.

13. A tangible computer readable medium storing a computer program having instructions for controlling a computing device to recognize speech, the instructions comprising:

receiving packets associated with input speech transmitted over a link at a buffering and decoding unit;

after waiting a predetermined time, generating from the received packets a speech vector and data identifying corrupt or likely to be corrupt features in the speech vector;

transmitting the speech vector and data to a speech recognition unit;

comparing the speech vector to stored recognition models based on non-corrupt features in the speech vector; and

either recognizing the speech input or requesting a retransmission of at least one packet based on the comparison.

14. The computer readable medium of claim 13, further comprising:

determining whether features within the speech vector are missing or corrupt; and

if so, then requesting retransmission of packets associated with the missing or corrupt features; and

if not, then requesting either that a speaker of the input speech make another utterance or a negotiate a higher bandwidth over the link.

15. The computer readable medium of claim 14, wherein negotiation of higher bandwidth comprises at least one of an alteration of compression techniques, more robust error correction, use of partially redundant parallel data streams, or the use of principle component analysis.

16. The computer readable medium of claim 14, wherein the negotiation is transparent to the speaker.

17. The computer readable medium of claim 13, wherein the instructions further comprises:

based on the comparison, determining a plurality of probabilities associated with a likelihood that the speech vector is associated with each stored recognition model.

18. The computer readable medium of claim 13, wherein the predetermined time is based on at least one of:

assumptions about the link, present network conditions and/or a tradeoff between recognizing speech in real time and receiving error free data.