Digital audio and/or video streaming system

• US 20040170159A1
• Filed: 02/28/2003
• Published: 09/02/2004
• Est. Priority Date: 02/28/2003
• Status: Abandoned Application

• Current Assignee: On-Demand Technologies Incorporated
• Original Assignee: On-Demand Technologies Incorporated
• Inventors: Yerkes, Arthur, Kim, Myong Gi
• Application Number:
  US10/376,866
• Publication Number:
  US 20040170159A1
• Time in Patent Office: Days
• US Class Current: 370/352
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/70 Media network packetisation

Abstract

A digital multimedia streaming system has an encoder having an input port that receives input digital multimedia (video and audio) signals and an output port that outputs encoded digital multimedia signals. The encoded digital multimedia signals are encoded from the input digital multimedia signals. The system also includes a player having an input port that receives the encoded digital multimedia signals and an output port that outputs an output digital multimedia signal. The output digital multimedia signals are decoded from the encoded digital multimedia signals. Latency between the input digital multimedia signals and the output digital multimedia signals are less than one second. The system also has a server having at least one input port, which receives the encoded digital multimedia signals from the encoder, operatively connected to the output port of the encoder, and at least one output port that outputs the encoded digital multimedia signals. A method for multimedia streaming is also disclosed.

Claims

1. A digital streaming system, comprising:

an encoder having an input port that receives at least one of input digital video signals and input digital audio signals and an output port that outputs an encoded digital multimedia signal, the encoded digital multimedia signal being encoded from the at least one of input digital video signals and input digital audio signals; and

a player having an input port that receives the encoded digital signal and an output port that outputs at least one of output digital video signals and output digital audio signals, the output digital video and audio signals being decoded from the encoded digital signal, a latency between the at least one of input digital video signals and input digital audio signals and at least one of output digital video signals and output digital audio signals being less than one second.

3. The system according to claim 1, wherein the system further comprises a server having at least one input port that receives the encoded digital video and audio signals from the encoder and at least one output port that outputs the encoded digital video and audio signals to the player.

4. The system according to claim 1, wherein the encoded digital video and audio signals form media data, wherein the digital audio streaming system distributes real-time media data to computers via the internet, and wherein the latency is a delay of about 500 ms.

5. The system according to claim 1, wherein the encoded digital multimedia signals form media data, wherein the system effects bi-directional exchange of real-time media data between computers via the internet, and wherein the latency is a delay of about 500 ms.

6. The system according to claim 1, wherein recording of the multimedia presentation, with all timing data intact, in a manner that all aural, visual, textual, and other media data can be replayed accurately in a well synchronized manner.

7. The system according to claim 1, including seeking a particular time in the presentation allowing a synchronous reproduction of the recorded media.

8. The system according to claim 1, wherein the system further comprises a storage medium for storing at least one of encoded digital multimedia signals and output digital video and audio signals, whereby the stored media signals are supplied by the player after a length of time after the receiving of the encoded digital multimedia signals.

39. The system according to claim 1, wherein the encoder has an mpeg-4 encoding module, and wherein the player has an mpeg-4 decoding module.

2. A digital multimedia streaming system, comprising:

an encoder having an input port that receives input digital video and audio signals and an output port that outputs an encoded digital multimedia signal, the encoded digital multimedia signal being encoded from the input digital video and audio signals; and

a player having an input port that receives the encoded digital multimedia signals and an output port that outputs output digital video and audio signals, the output digital video and audio signals being decoded from the encoded digital multimedia signal, a latency between the input digital video and audio signals and the output digital video and audio signals being less than one second.

9. A method for digital multimedia video and audio streaming including video and audio, for use with an encoder, a server, and a player, comprising the steps of:

in an encoder;

receiving video frames using a DirectX layer, via a video capture card, and simultaneously receiving audio signals in PCM samples via an audio input;

for each encoder, converting the sampled audio and video signals into data objects respectively, along with the capture characteristics consisting of at least one of sample rate, bits and channels for audio and x, y and color space for video;

encoding the converted data into encoded data, each encoder producing a view of the sample compatible with its input by converting and re-sampling the input data;

partitioning the encoded data into smaller data packets;

creating and attaching time-stamps to respective data packets, and, as a function of a transport mode, creating at least one of unicast RTP/UDP or TCP packets or multicast packets for transmission;

in a server;

determining, as a function of a network configuration of a network node on which the player is running, determine the type of network transport (RTP/UDP or TCP/Tunneled HTTP) and routing method (multicast or unicast) for the player;

sending the data packets to all players that are connected thereto;

in a player;

placing each received packet in a sorted queue, a packet with one of an earliest time-stamp or a smallest sequence number being a first data packet in the queue;

selecting the first packet out of the queue, copying the first packet to a synch buffer, and processing the first packet as follows;

if a sleep time is less than 10 ms, processing the sample immediately;

if the sleep time is greater than 50 ms, processing the sample after a 50 ms wait;

if the sleep time is between 10 ms and 50 ms, sleeping for a predetermined number of milliseconds and then processing the sample;

decoding each received frame, adding via a ring buffer a small audio lead time, and keeping one video frame in a buffer for a repaint;

clearing, in response to a new audio frame, the ring buffer when the ring buffer is full, a new video frame replacing a previous video frame;

feeding decoded frames to DirectX to be played back;

updating the video frames, and playing back the audio stream; and

sending an outgoing IRC message, when there is an IRC message to be sent, to an IRC server, and, when there are incoming IRC messages, displaying the IRC messages.

10. A digital audio streaming system, comprising:

an encoder having an input port that receives an input digital audio signal and an output port that outputs an encoded digital audio signal, the encoded digital audio signal being encoded from the input digital signal; and

a player having an input port that receives the encoded digital audio signal and an output port that outputs an output digital audio signal, the output digital signal being decoded from the encoded digital audio signal, a latency between the input digital audio signal and the output digital audio signal being less than one second.

11. The system according to claim 10, wherein the system further comprises a server having at least one input port that receives the encoded digital audio signal from the encoder and at least one output port that outputs the encoded digital audio signal to the player.

12. The system according to claim 10, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system distributes real-time audio data to computers via the internet, and wherein the latency is a delay of about 500 ms.

13. The system according to claim 10, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system effects bi-directional exchange of real-time audio data between computers via the internet, and wherein the latency is a delay of about 500 ms.

14. The system according to claim 10, wherein the system further comprises a conversion module for capturing and encoding continuous real-time audio from a telephone system and forming the input digital signal therefrom.

15. The system according to claim 14, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system effects bi-directional exchange of real-time audio data between computers via the internet, and wherein the latency is a delay of about 500 ms.

16. The system according to claim 10, wherein the system further comprises a storage medium for storing encoded digital audio signals, whereby the stored audio signals are supplied to the player after a length of time after receipt of the encoded digital audio signals.

17. A digital audio streaming system, comprising:

an encoder having an input port that receives an input digital audio signal and an output port that outputs an encoded digital audio signal, the encoded digital audio signal being encoded from the input digital signal; and

a player having an input port that receives the encoded digital audio signal and an output port that outputs an output digital audio signal, the output digital signal being decoded from the encoded digital audio signal.

18. The system according to claim 17, wherein the system further comprises a server having at least one input port that receives the encoded digital audio signal from the encoder and at least one output port that outputs the encoded digital audio signal to the player.

19. The system according to claim 17, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system distributes real-time audio data to computers via the internet, and wherein the latency is a delay of about 500 ms.

20. The system according to claim 17, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system effects bi-directional exchange of real-time audio data between computers via the internet, and wherein the latency is a delay of about 500 ms.

21. The system according to claim 17, wherein the system further comprises a conversion module for capturing and encoding continuous real-time audio from a telephone system and forming the input digital signal therefrom.

22. The system according to claim 21, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system effects bi-directional exchange of real-time audio data between computers via the internet, and wherein the latency is a delay of about 500 ms.

23. The system according to claim 17, wherein the system further comprises a storage medium for storing encoded digital audio signals, whereby the stored audio signals are supplied by the player after a length of time after receipt of the encoded digital audio signals.

24. A digital audio streaming system, comprising:

an encoder having an input port that receives an input digital audio signal and an output port that outputs an encoded digital audio signal, the encoded digital audio signal being encoded from the input digital signal;

a server having at least one input port, which receives the encoded digital audio signal, operatively connected to the output port of the encoder from the encoder and at least one output port that outputs the encoded digital audio signal; and

a player having an input port, which receives the encoded digital audio signal, operatively connected to the output port of the server, and an output port that outputs an output digital audio signal, the output digital signal being decoded from the encoded digital audio signal, the latency between the input digital audio signal and the output digital audio signal being less than one second.

25. The system according to claim 24, wherein the system further comprises a server having at least one input port that receives the encoded digital audio signal from the encoder and at least one output port that outputs the encoded digital audio signal to the player.

26. The system according to claim 24, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system distributes real-time audio data to computers via the internet, and wherein the latency is a delay of about 500 ms.

27. The system according to claim 24, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system effects bi-directional exchange of real-time audio data between computers via the internet, and wherein the latency is a delay of about 500 ms.

28. The system according to claim 24, wherein the system further comprises a conversion module for capturing and encoding continuous real-time audio from a telephone system and forming the input digital signal therefrom.

29. The system according to claim 28, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system effects bi-directional exchange of real-time audio data between computers via the internet, and wherein the latency is a delay of about 500 ms.

30. The system according to claim 28, wherein the system further comprises a storage medium for storing at least one of encoded digital audio signals an output digital audio signals, whereby the stored audio signals are supplied by the player after a length of time after the receiving of the encoded digital audio signals.

31. A digital audio streaming system, comprising:

an encoder having an input port that receives an input digital audio signal and an output port that outputs an encoded digital audio signal, the encoded digital audio signal being encoded from the input digital signal, the encoder having a first latency;

a server having at least one input port, which receives the encoded digital audio signal, operatively connected to the output port of the encoder from the encoder and at least one output port that outputs the encoded digital audio signal, the server having a second latency;

at least one player having an input port, which receives the encoded digital audio signal, operatively connected to the output port of the server, and an output port that outputs an output digital audio signal, the output digital signal being decoded from the encoded digital audio signal, the player having a third latency; and

a system latency between a sum of the first, second, and third latencies, the system latency being less than one second.

32. The system according to claim 31, wherein the system further comprises a server having at least one input port that receives the encoded digital audio signal from the encoder and at least one output port that outputs the encoded digital audio signal to the player.

33. The system according to claim 31, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system distributes real-time audio data to computers via the internet, and wherein the latency is a delay of about 500 ms.

34. The system according to claim 31, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system effects bi-directional exchange of real-time audio data between computers via the internet, and wherein the latency is a delay of about 500 ms.

35. The system according to claim 31, wherein the system further comprises a conversion module for capturing and encoding continuous real-time audio from a telephone system and forming the input digital signal therefrom.

36. The system according to claim 35, wherein the encoded digital audio signals form audio data, wherein the digital audio streaming system effects bi-directional exchange of real-time audio data between computers via the internet, and wherein the latency is a delay of about 500 ms.

37. The system according to claim 35, wherein the system further comprises a storage medium for storing encoded digital audio signals, whereby the stored audio signals are supplied by the player after a length of time after receipt of the encoded digital audio signals.

38. A digital audio streaming system, comprising:

an encoder having an input port that receives an input digital audio signal and an output port that outputs an encoded digital audio signal, the encoded digital audio signal being encoded from the input digital signal, the output port being operatively connected to the internet and the encoded digital signal being output to the internet in packets;

a server having at least one input port, which receives the packets from the encoder, operatively connected to the output port of the encoder via the internet, and at least one output port that outputs the encoded digital audio signal;

at least one player having an input port, which receives the encoded digital audio signal from the server, operatively connected to the output port of the server, and an output port that outputs an output digital audio signal, the output digital signal being decoded from the encoded digital audio signal; and

a system latency between the input digital audio signal received by the encoder and the output digital audio signal output by the player being less than one second.

40. A method for digital audio streaming, comprising the steps of:

in an encoder;

waiting for the phone to ring;

picking up, when a call is made, via a modem program of the encoder, the phone;

recording 8 kHz PCM samples from speech input generated from the modem to produce audio signals;

dividing the audio signals into 20 ms long frames;

using a GSM codec to compress the 20 ms long frame into a data packet representing particular excitation sequence and amplitude by using short-term and long-term predictors; and

time-stamping the packet with a current time;

in a server;

depending on the network configuration of the network node the player resides in, determining the type of network transport (RTP/UDP or TCP/Tunneled HTTP) and routing method (multicast or unicast) for the player; and

sending the data packets to all the players that are connected to the server;

in a player;

placing each received packet in a sorted queue, a packet with one of an earliest time-stamp or a smallest sequence number being a first data packet in the queue;

selecting the first packet out of the queue, coping the first packet to a synch buffer, and processing the first packet as follows;

if a sleep time is less than 10 ms, processing the sample immediately;

if the sleep time is greater than 50 ms, processing the sample after a 50 ms wait;

if the sleep time is between 10 ms and 50 ms, sleeping for a predetermined number of milliseconds and then processing the sample;

decoding each received frame, a ring buffer adding a small audio lead-time;

clearing, in response to a new audio frame, the ring buffer when the ring buffer is full;

feeding excitation signals in the frames through short-term and long-term synthesis filters to reconstruct the audio streams; and

feeding the decoded audio streams to DirectX to be played back through a sound card.