Accelerated media coding for robust low-delay video streaming over time-varying and bandwidth limited channels

• US 7,652,994 B2
• Filed: 03/31/2006
• Issued: 01/26/2010
• Est. Priority Date: 03/31/2006
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: van Beek, Petrus J. L.
• Primary Examiner(s): Ngo; Ricky
• Assistant Examiner(s): SINKANTARAKORN, PAWARIS
• Application Number:
  US11/394,573
• Publication Number:
  US 20070236599A1
• Time in Patent Office: 1,397 Days
• Field of Search: 370229-234, 370465-468
• US Class Current: 370/232
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/44227 Monitoring of local network, e.g. connection or bandwidth variations; Detecting new devices in the local network configuring of peripheral devices in general G06F9/4411; monitoring connectivity in data switched networks H04L43/0811
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 7/163 by receiver means only

Abstract

A system includes effective transmission of compressed audio/video stream as a sequence of audio/video (AV) data units. The system includes a transcoder that transcodes the data at different data unit rates. The system also includes a transmitter that transmits the data units at different average data unit rates. A controller controls the bit rate at which the transcoder transcodes the data units in conjunction with the data unit rate per second at which the transmitter transmits the said transcoded data stream.

Claims

1. A system for transmitting a data stream including at least one of audio and video over a transmission channel to a receiver at a bit rate capable of being varied, said data stream arranged in frames, said system comprising:

(a) a transcoder that receives said data stream and transcodes said data stream at a first average number of frames per second for a first period of time and at a second average number of frames per second for a second period of time, where said first period of time is immediately prior to said second period of time, where said first average number of frames per second is greater than said second average number of frames per second, where said first average number of frames per second is greater than a playback rate of frames per second at said receiver and said second average number of frames per second is no greater than a playback rate of frames per second at said receiver, and where the transition between said first period of time and second period of time is based on a measure of the fullness of a buffer at said receiver;

(b) a transmitter that transmits the transcoded data stream at an average bit rate for a third period of time that is different than an average bit rate over a fourth period of time, where said first period of time is overlapping in time with said third period of time, where said second period of time is overlapping in time with said fourth period of time;

(c) a controller that optimizes the bit rate at which said transcoder transcodes said data stream based on an estimate of transmission characteristics of said transmission channel in conjunction with the number of frames per second at which said transmitter transmits the said transcoded data stream.

2. The system of claim 1 wherein said data stream is stored on a storage medium prior to being transcoded.

3. The system of claim 2 wherein said first number of controller controls said bit rate and said frame rate based on a constraint on the end-to-end delay between a sender and a receiver.

4. The system of claim 1 wherein said data stream includes video.

5. The system of claim 1 wherein said data stream includes audio.

6. The system of claim 5 wherein said data stream includes video.

7. The system of claim 1 wherein said transcoding includes modifying the spatial and temporal resolution of the data stream.

8. The system of claim 1 wherein said transcoding includes modifying the compression format.

9. The system of claim 1 further comprising a transmission monitor that estimates at least one of the bandwidth of a channel that said transcoded data stream is transmitted through and the status of the system.

10. The system of claim 9 wherein said transmission monitor estimates a fullness of a transmission buffer of said system.

11. The system of claim 9 wherein said transmission monitor estimates a fullness of a receiving buffer of said system.

12. The system of claim 9 wherein said transmission monitor estimates the bandwidth of said channel.

13. The system of claim 9 wherein said transcoding is based upon said estimation.

14. The system of claim 9 wherein said transmitting is based upon said estimation.

15. The system of claim 1 wherein a bit rate of said transmitted data units is greater during said first period than said second period.

16. The system of claim 1 wherein a bit rate of said transmitted frames is greater during said second period than said first period.

17. The system of claim 1 further comprising a transmission monitor estimating the available bandwidth of a channel that said transcoded data stream is transmitted through and said transcoding said data stream in such a manner so that the resulting data rate of said transmitted data stream is less than said available bandwidth.

18. The system of claim 1 wherein said first time period has a predetermined time duration.

19. The system of claim 1 wherein said transcoding is based upon information provided by a decoder.

20. The system of claim 1 wherein said transcoding results in a different bit rate than provided by said transmitter of said transcoded data stream.

21. The system of claim 1 wherein said bit rate of said frames for at least one of said first and second time period is based upon the type of video frames in said data stream.