Method and apparatus for streaming data

• US 8,626,939 B2
• Filed: 04/10/2006
• Issued: 01/07/2014
• Est. Priority Date: 04/30/2005
• Status: Active Grant

• Current Assignee: ServiceNow Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Jackson, Callum Peter, Smith, Nanami
• Primary Examiner(s): Najjar, Saleh
• Assistant Examiner(s): Shepperd, Eric W
• Application Number:
  US11/400,952
• Publication Number:
  US 20060248214A1
• Time in Patent Office: 2,829 Days
• Field of Search: 709231-235, 709201-202, 709/204, 709/217, 709/219, 709/226, 709/229, 379 8813- 8814, 379/83.15, 379/83.24, 379/100.13, 379/908, 704/1, 704/258, 704/262, 704270-278
• US Class Current: 709/231
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/612 for unicast

Abstract

Method and apparatus for negotiation of streaming data, suitable for application in an environment wherein a server generates and serves large, continuous amounts of speech data to a client in real time. A method of controlling the play out of a generated data stream from a data stream player includes estimating a time for generating the data stream; estimating a time for playing the data stream; generating a data stream using a data stream generation resource, for output by the data stream player; and alerting the stream player if the remaining generation time is less than the play time. The server may calculate what it can do, given knowledge of the network, and send a message to guarantee a level of service. A “can start play” message may be sent to client when the client can reliably start playing the signal.

Claims

1. A method of controlling play out of a generated audio data stream from a client data stream player, the method comprising:

at a server, receiving a request, from a client data stream player, to generate an audio stream by converting textual data received from the client data stream player;

at the server, receiving, from a client, textual data to be generated by the server to an audio stream;

at the server, estimating a generation time for generating the audio stream;

at the server, estimating a time to play the generated audio stream at a data stream player;

at the server, generating the audio stream using data received from the client and a data stream generation resource, the audio stream for output by the client data stream player;

buffering at the client, the audio stream generated at the server,

waiting at the client for an alert from the server indicating that a remaining generation time is not more than a remaining play time;

sending an alert from the server to the client if remaining generation time for generating the audio stream using the data stream generation resource is not more than remaining play time; and

maintaining a rate of generation at an initial rate or faster after the alert has been sent,

wherein generating the audio stream has a priority and the priority is raised in order to maintain the rate of generation of the audio stream after the alert has been sent.

2. The method of claim 1, wherein the remaining generation time is obtained from the data stream generation resource during the generation of the audio stream.

3. The method of claim 1, wherein the alert is sent from the server to the client and wherein remaining generation time is calculated using size of text for conversion from the client and workload of a server text-to-speech engine.

4. The method of claim 1, wherein the audio stream is speech only and the generation resource is a server text-to-speech engine.

5. The method of claim 1 wherein estimating the generation time includes considering the size of the textual data received from the client and considering the current workload of the data stream generation resource and determining if a critical buffer point is zero or less, the critical buffer point being determined by subtracting play time of voice data at a client media stream player from text-to-speech generation time by the data stream generation resource.

6. A system for controlling play out of a generated media data stream from a client data stream player, said system comprising:

a processor configured to estimate a generation time for generating a media data stream,

the processor executing instructions that cause the processor to calculate a critical buffer point for a buffer of a client data stream player, the critical buffer point representing a text-to-speech generation time minus a playtime of the speech at a client data stream player,

the processor also executing instructions that cause the processor to estimate a play time for the media data stream at a client data stream player;

means for generating the media data stream using a data stream generation resource, for output by the client data stream player,

wherein the means for generating the media data stream is configured to convert text data received from the client to generate a media data stream consisting of audible speech,

wherein the means for generating is further configured to keep a controller updated with text-to-speech generation time throughout a transmission of a text-to-speech data stream;

means for sending an alert to the client data stream player if remaining generation time for generating the media data stream using the data stream generation resource is not more than the play time by a client data stream player; and

means for maintaining the rate of generation of the media data stream at an initial rate or faster after an alert has been sent,

wherein the means for maintaining the rate of generation of the media data stream generation is configured to set a priority for generating the media data stream and to further raise the priority for generating the media data stream after the alert has been sent.

7. The system of claim 6, wherein remaining generation time is determined by the data stream generation resource during the generation of the media data stream and wherein remaining generation time is obtained each time from the data stream generation resource each time generation time is compared to play time by a client data stream player.

8. The system of claim 6, wherein the means for sending an alert is in a server.

9. The system of claim 6, wherein the means for generating the media data stream is configured to convert text to speech.

10. The system of claim 6 wherein the means for generating the media data stream is further configured to consider the size of the textual data received from a client and consider the means for generating.

11. A computer program product for controlling the play out of a generated media data stream from a data stream player, said computer program product comprising a non-transitory computer readable medium having computer readable program code tangibly embedded therein, the computer readable program code comprising:

computer readable program code configured to repeatedly estimate a remaining generation time for generating an audio voice stream from textual data received from a client;

computer readable program code configured to repeatedly estimate a play time at a client for playing a stream consisting of audio voice converted from the textual data received from the client;

computer readable program code configured to generate an audio voice stream using textual data received from a client and a data stream generation resource at a server, for output by a data stream player at the client;

computer readable program code configured to send an alert to the data stream player at the client if the remaining generation time is not more than the estimated play time at the client data stream player; and

computer readable program code configured to maintain the rate of generation of the audio voice stream at an initial rate or faster after the alert has been sent,

wherein the audio voice stream generation has a priority and the priority is raised to maintain the rate of generation of the audio voice stream after the alert has been sent.

12. The computer program product of claim 11, wherein the remaining generation time of the audio voice stream is obtained from the data stream generation resource during the generation of the audio voice stream.

13. The computer program product of claim 11, wherein the data stream generation resource is a text-to-speech engine.

14. The computer program product as in claim 11 wherein the computer readable program code is further configured to consider the size of the textual data received from the client and consider the current workload of the data stream generation resource.

15. A system for controlling play out of a generated media data stream from a client data stream player, said system comprising:

a processor and memory, the memory storing instructions for execution on the processor, the instructions, which when executed by the processor, cause the processor to;

estimate a generation time for generating a media data stream;

estimate a play time at a client data stream player for a generated media data stream;

calculate a critical buffer point for a buffer of a client data stream player, the critical buffer point representing a text-to-speech generation time minus a playtime at the client data stream player;

generate the media data stream by converting text data received from a client to a media data stream consisting of audible speech;

send an alert to the client data stream player if remaining generation time for generating the media data stream is not more than the play time at the client data stream player;

maintain the rate of generation of the media data stream at an initial rate or faster after an alert has been sent; and

set a priority for generating the media data stream and later raise the priority for generating the media data stream after the alert has been sent.

16. The system of claim 15, wherein remaining generation time is determined during the generation of the media data stream and wherein remaining generation time is obtained each time generation time is compared to play time by a client data stream player.

17. The system of claim 15, wherein the instructions, which when executed by the processor, further cause the processor to:

consider the size of textual data received from the client before sending the alert and consider the current workload of a data stream generation resource before sending the alert.

18. The system of claim 15, wherein the instructions, when executed by the processor, further cause the processor to:

obtain remaining generation time of an audio voice stream from a data stream generation resource in communication with the processor during the generation of the audio voice stream.