Method and device for transmitting variable rate video data

• US 8,812,724 B2
• Filed: 06/18/2009
• Issued: 08/19/2014
• Est. Priority Date: 06/19/2008
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Nassor, Eric, Maze, Frederic
• Primary Examiner(s): Etienne, Ario
• Assistant Examiner(s): RUBIN, BLAKE J
• Application Number:
  US12/487,348
• Publication Number:
  US 20090319682A1
• Time in Patent Office: 1,888 Days
• Field of Search: 709/235
• US Class Current: 709/235
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/234309 by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo conversion of standards in analog television systems H04N7/01
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/238 : Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams hybrid fiber coaxial [HFC] networks for downstream channel allocation for video distribution H04L12/2801; flow control in packet networks H04L47/10; real-time communication protocols in data switching networks H04L65/00; scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2402 : Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/6437 : Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 21/658 : Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6583 : Acknowledgement

Abstract

A video consisting of data organized in the form of a plurality of images is transmitted in a communication network. The method comprises a step of coding images with motion compensation, which consists in compressing the images of the video and in creating dependencies between compressed images, a step of scheduling the transmission of packets representing the compressed images, which consists in sending the compressed images over the network in a selected order, and a step of controlling the rate of the video. At least one of reconsidering the selected order of sending already compressed but not yet transmitted images and deleting at least one compressed image is performed at the time of coding a new image. Furthermore, the dependencies between the new image to be compressed and the compressed images are selected by taking into account the reconsidered sending order at the time of coding the new image.

Claims

1. A method for transmitting a video stream including a plurality of compressed images in a communication network, said method comprising:

a step of coding a plurality of original images with motion estimation and motion compensation, including compressing the plurality of original images of the video and creating dependencies between compressed images of the plurality of compressed images, said step of coding a plurality of original images being performed based on a current rate of the video;

a step of storing packets representing the compressed images into a buffer;

a step of scheduling a transmission of packets representing the compressed images of the video stream, including ordering the compressed images together according to a selected order for their transmission over the communication network;

a step of controlling the rate of the video;

a step of modifying a state of the compressed images in the buffer based on a new rate of the video;

at least one step of reconsidering the selected order of transmission already compressed but not yet transmitted images in the buffer and deleting at least one compressed image in the buffer based on said step of modifying and at a time of coding a new image that is not in the buffer;

a step of coding the new image based on the new rate of the video; and

wherein dependencies between the new image to be compressed and the compressed images are selected by taking into account the selected order reconsidered at the time of coding the new image.

2. The method according to claim 1, wherein dependencies between images and the selected order for transmission of sending the compressed images are selected as a function of an available bandwidth of the communication network, and/or a number of packets waiting to be sent and content of the video.

3. The method according to claim 2, wherein the method further comprises a step of evaluating said available bandwidth via a mechanism for controlling congestion on the communication network.

4. The method according to claim 3, wherein the mechanism for controlling the congestion is of a TFRC (“

TCP Friendly Rate Control”

) or AIMD (“

Additive Increase/Multiple Decrease”

) type.

5. The method according to claim 2, wherein a selection of the dependencies between the new image and the compressed images includes defining a coding mode for the new image with or without dependency.

6. The method according to claim 5, wherein, when a coding mode with dependency is defined, said selection of the dependencies between images further includes defining at least one possible reference image.

7. The method according to any one of claims 1 and 2-6, wherein deleting at least one compressed image in the buffer that has not been transmitted is performed after a reordering of the compressed images.

8. A method according to claim 1, wherein a selection of the order of transmission of the packets includes deciding to delay or bring forward a sending of at least one compressed image.

9. The method according to claim 1, wherein the packets representing each of the compressed images are assigned a certain priority and a certain expiry time, after which sending of the packets representing the compressed images becomes useless, and the packets are sent in a decreasing priority order and in an increasing expiry time order.

10. The method according to claim 1, wherein the video is coded in H.264 format.

22. A non-transitory computer-readable recording medium configured to store information readable by a computer or a microprocessor holding instructions for a computer program for carrying out the method according to any one of claims 1 and 2-6.

23. A non-transitory computer-readable recording medium storing a computer program product configured to be loaded onto a programmable apparatus, that when executed causes a computer to perform sequences of instructions for carrying out the method according to any one of claims 1 and 2-6.

11. A server for transmitting a video stream including a plurality of compressed images in a communication network, said server comprising:

a processor coupled to a memory configured to perform;

coding a plurality of original images with motion estimation and motion compensation, which includes compressing the plurality of original images of the video and creating dependencies between compressed images of the plurality of compressed images, said coding step being performed based on a current rate of the video;

storing packets representing the compressed images into a buffer;

scheduling a transmission of packets representing the compressed images of the video stream, which includes ordering the compressed images together according to a selected order for their transmission over the communication network;

controlling the rate of the video;

modifying a state of the compressed images in the buffer based on a new rate of the video;

at least one of reconsidering the selected order of transmission sending already compressed but not yet transmitted images in the buffer and deleting at least one compressed image in the buffer based on said modifying and at a time of coding a new image that is not in the buffer;

coding the new image based on the new rate of the video; and

wherein dependencies between the new image to be compressed and the compressed images are selected by taking into account the selected order reconsidered at the time of coding the new image.

12. The server according to claim 11, wherein the processor is further configured to select the dependencies between images and the selected order of transmission of sending the compressed images as a function of an available bandwidth of the communication network, and/or a number of packets waiting to be sent and content of the video.

13. The server according to claim 12, wherein the processor is further configured to evaluate said available bandwidth via a mechanism for controlling congestion on the communication network.

14. The server according to claim 13, wherein the 1 mechanism for controlling the congestion is of a TFRC (“

TCP Friendly Rate Control”

) or AIMD (“

Additive Increase/Multiple Decrease”

) type.

15. The server according to claim 11, wherein the processor is further configured to define a coding mode for the new image with or without dependency.

16. The server according to claim 15, wherein defining a coding mode with dependency further includes defining at least one possible reference image.

17. The server according to any one of claims 11 and 12-16, wherein the processor is further configured to decide to delete at least one compressed image in the buffer that has not been transmitted after a reordering of the compressed images.

18. The server according to claim 11, wherein the processor is further configured to decide to delay or bring forward a transmission of at least one compressed image.

19. The server according to 11, wherein the processor is further configured to assign to packets representing each of the compressed images are assigned a certain priority and a certain expiry time, after which the transmission of the packets representing the compressed images becomes useless, and the packets are adapted to be sent in a decreasing priority order and in an increasing expiry time order.

20. The server according to claim 11, wherein the video is coded in H.264 format.

21. A telecommunications system comprising a plurality of terminal devices connected via a telecommunications network, wherein the telecommunications system comprises at least one terminal device equipped with a server according to any one of claims 11 and 12-16.