VOIP barge-in support for half-duplex DSR client on a full-duplex network

• US 7,848,314 B2
• Filed: 05/10/2006
• Issued: 12/07/2010
• Est. Priority Date: 05/10/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Li, Yan, Cross, Charles W. Jr., McCobb, Gerald M.
• Primary Examiner(s): Harold; Jefferey F
• Assistant Examiner(s): Kassim; Khaled
• Application Number:
  US11/382,575
• Publication Number:
  US 20070274296A1
• Time in Patent Office: 1,672 Days
• Field of Search: 370/276, 370/278, 370/294, 370/412, 370/352, 370/356
• US Class Current: 370/352
• CPC Class Codes: G10L 15/222 Barge in, i.e. overridable guidance for interrupting prompts
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 2207/40 terminals with audio html browser
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

Providing VOIP barge-in support for a half-duplex DSR client on a full-duplex network by buffering, in a half-duplex DSR client, input audio from the full-duplex network; playing, through the half-duplex DSR client, the buffered input audio; pausing, during voice activity on the half-duplex DSR client, the playing of the buffered input audio; sending, during voice activity on the half-duplex DSR client, speech for recognition through the full-duplex network to a voice server; receiving in the half-duplex DSR client through the full-duplex network from the voice server notification of speech recognition, the notification bearing a time stamp; and, responsive to receiving the notification, resuming the playing of the buffered input audio, including playing only buffered VOIP audio data bearing time stamps later than the time stamp of the recognition notification.

Claims

1. A method of providing voice over the Internet Protocol (‘

VOIP’

) barge-in support for a half-duplex distributed speech recognition (‘

DSR’

) client on a full-duplex network, the method comprising;

buffering, in the half-duplex DSR client, input audio from the full-duplex network, the input audio comprising VOIP audio data for output through the half-duplex DSR client, the buffered input audio bearing time stamps;

playing, through the half-duplex DSR client, the buffered input audio;

pausing, during voice activity on the half-duplex DSR client, the playing of the buffered input audio;

sending, during voice activity on the half-duplex DSR client, speech for recognition through the full-duplex network to a voice server;

receiving in the half-duplex DSR client through the full-duplex network from the voice server notification of speech recognition, the notification bearing a time stamp; and

responsive to receiving the notification, resuming the playing of the buffered input audio, including playing only buffered input audio bearing time stamps later than the time stamp of the recognition notification.

2. The method of claim 1 wherein buffering input audio further comprises:

receiving by the half-duplex DSR client audio data from an input audio stream;

storing the audio data in an input memory buffer; and

inserting time stamps in the buffered input audio, the time stamps indicating when the input audio was received by the half-duplex DSR client.

3. The method of claim 1 further comprising inserting by the voice server time stamps in the VOIP audio data, the time stamps indicating when the audio was transmitted from the voice serve to the half duplex DSR client.

4. The method of claim 1 wherein:

buffering input audio further comprises receiving, by a receive thread, VOIP audio data from an input audio stream in accordance with a first VOIP data communications protocol; and

receiving notification of speech recognition further comprises receiving the notification through the first VOIP data communications protocol.

5. The method of claim 1 wherein:

buffering input audio further comprises receiving, by a receive thread, VOIP audio data from an input audio stream in accordance with a first VOIP data communications protocol; and

receiving notification of speech recognition further comprises receiving the notification through a second VOIP data communications protocol.

6. The method of claim 1 further comprising affixing, by the voice server to the notification of speech recognition, the time stamp borne by the notification, the time stamp representing the time when the notification is sent from the voice server to the half-duplex DSR client.

7. Apparatus for providing voice over the Internet Protocol (‘

VOIP’

) barge-in support for a half-duplex distributed speech recognition ('"'"'DSR'"'"') client on a full-duplex network, the apparatus comprising;

a computer processor; and

a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, perform;

buffering, in the half-duplex DSR client, input audio from the full-duplex network, the input audio comprising VOIP audio data for output through the half-duplex DSR client, the buffered input audio bearing time stamps;

playing, through the half-duplex DSR client, the buffered input audio;

pausing, during voice activity on the half-duplex DSR client, the playing of the buffered input audio;

sending, during voice activity on the half-duplex DSR client, speech for recognition through the full-duplex network to a voice server;

receiving in the half-duplex DSR client through the full-duplex network from the voice server notification of speech recognition, the notification bearing a time stamp; and

responsive to receiving the notification, resuming the playing of the buffered input audio, including playing only buffered input audio bearing time stamps later than the time stamp of the recognition notification.

8. The apparatus of claim 7 wherein buffering input audio further comprises:

receiving by the half-duplex DSR client audio data from an input audio stream;

storing the audio data in an input memory buffer; and

inserting time stamps in the buffered input audio, the time stamps indicating when the input audio was received by the half-duplex DSR client.

9. The apparatus of claim 7 further comprising computer program instructions that, when executed, perform inserting, by the voice server time stamps in the VOIP audio data, the time stamps indicating when the audio was transmitted from the voice serve to the half duplex DSR client.

10. The apparatus of claim 7 wherein:

buffering input audio further comprises receiving, by a receive thread, VoIP audio data from an input audio stream in accordance with a first VOIP data communications protocol; and

receiving notification of speech recognition further comprises receiving the notification through the first VOIP data communications protocol.

11. The apparatus of claim 7 wherein:

buffering input audio further comprises receiving, by a receive thread, VOIP audio data from an input audio stream in accordance with a first VOIP data communications protocol; and

receiving notification of speech recognition further comprises receiving the notification through a second VOIP data communications protocol.

12. The apparatus of claim 7 further comprising computer program instructions that, when executed, perform affixing, by the voice server to the notification of speech recognition, the time stamp borne by the notification, the time stamp representing the time when the notification is sent from the voice server to the half-duplex DSR client.

13. A computer program product comprising at least one tangible computer-readable storage medium having instructions encoded thereon which, when executed, perform a method of providing voice over the Internet Protocol (‘

VOIP’

) barge-in support for a half-duplex distributed speech recognition (‘

DSR’

) client on a full-duplex network, the method comprising;

buffering, in the half-duplex DSR client, input audio from the full-duplex network, the input audio comprising VOIP audio data for output through the half-duplex DSR client, the buffered input audio bearing time stamps;

playing, through the half-duplex DSR client, the buffered input audio;

pausing, during voice activity on the half-duplex DSR client, the playing of the buffered input audio;

sending, during voice activity on the half-duplex DSR client, speech for recognition through the full-duplex network to a voice server;

receiving in the half-duplex DSR client through the full-duplex network from the voice server notification of speech recognition, the notification bearing a time stamp; and

responsive to receiving the notification, resuming the playing of the buffered input audio, including playing only buffered input audio bearing time stamps later than the time stamp of the recognition notification.

14. The computer program product of claim 13 wherein buffering input audio further comprises:

receiving by the half-duplex DSR client audio data from an input audio stream;

storing the audio data in an input memory buffer; and

inserting time stamps in the buffered input audio, the time stamps indicating when the input audio was received by the half-duplex DSR client.

15. The computer program product of claim 13, wherein the method further comprises inserting, by the voice server, time stamps in the VOIP audio data, the time stamps indicating when the audio was transmitted from the voice serve to the half duplex DSR client.

16. The computer program product of claim 13 wherein:

buffering input audio further comprises receiving, by a receive thread, VOIP audio data from an input audio stream in accordance with a first VOIP data communications protocol; and

receiving notification of speech recognition further comprises receiving the notification through the first VOIP data communications protocol.

17. The computer program product of claim 13 wherein:

buffering input audio further comprises receiving, by a receive thread, VOIP audio data from an input audio stream in accordance with a first VOIP data communications protocol; and

receiving notification of speech recognition further comprises receiving the notification through a second VOIP data communications protocol.

18. The computer program product of claim 13, wherein the method further comprises affixing, by the voice server to the notification of speech recognition, the time stamp borne by the notification, the time stamp representing the time when the notification is sent from the voice server to the half-duplex DSR client.