Method of coding, decoding, coder and decoder

• US 8,755,436 B2
• Filed: 04/23/2009
• Issued: 06/17/2014
• Est. Priority Date: 04/25/2008
• Status: Active Grant

• Current Assignee: InterDigital Madison Patent Holdings (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Francois, Edouard, Le Meur, Olivier, Vieron, Jerome
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): EDWARDS, TYLER B
• Application Number:
  US12/736,613
• Publication Number:
  US 20110170599A1
• Time in Patent Office: 1,881 Days
• Field of Search: None
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A method for coding a picture portion of a picure of a video sequence is disclosed, at least one picture of the video sequence being a key picture. The method comprises the following steps:

• • a) calculate a saliency map of the key picture,
  • b) estimate for the picture portion to be coded, at least one motion vector pointing towards a portion of the key picture,
  • c) motion compensate at least one portion of the saliency map using an estimated motion vector to obtain, for said picture portion, at least one portion of the predicted saliency map, and
  • d) code the picture portion according to the saliency level of the portion of the predicted saliency map.

The invention also relates to a method for decoding a binary stream, a coder able to code pictures of a video sequence and a decoder of a binary stream.

Claims

1. A method for coding pictures of a video sequence, the video sequence comprises at least one key picture and at least one intermediary picture, the method comprising the following steps:

a) calculating a saliency map of at least one of the key picture(s);

b) estimating for a picture portion of an intermediary picture to be coded, at least one motion vector pointing towards a portion of the key picture(s);

c) motion compensating at least one portion of the saliency map using one or each estimated motion vector to obtain, for said picture portion, at least one portion of a predicted saliency map, and

d) coding the picture portion according to a saliency level of each portion(s) of the predicted saliency map,

wherein step d) comprises the following steps;

determining the saliency level of each portion(s) of the predicted saliency map;

comparing a predefined saliency level and the saliency level of the portion of the predicted saliency map, and

coding the picture portion using a first coding method when the saliency level of the portion is greater than the predefined saliency level and using a second coding method when the saliency level of the portion is lower than the predefined saliency level, said first coding method being a coding method with motion compensation and residue coding and said second method being a coding method with a more precise motion compensation than the motion compensation of the first coding method and without residue coding.

2. A method for coding according to claim 1, in which the first coding method is defined by H.264 standard, the second coding method being a method based on the H.264 standard in which an amplitude of components of each motion vector is greater than 1/4 pixel.

3. A method for coding according to claim 1, in which the second coding method comprises coding and transmission of motion information defined by affine parameters.

4. A method for coding according to claim 1, in which the picture portion of the intermediary picture of the video sequence is coded according to a compression method that is scalable by layers, the key picture(s) being destined to be coded in a base layer, the picture portion being coded in a high layer.

5. A method for coding according to claim 1, that comprises the following steps:

estimating a motion vector from a portion of a second intermediary picture and pointing towards at least one part of a portion of a first intermediary picture,

counting a number of pixels of the part of the portion of a first intermediary picture,

when the number of pixels counted is greater than a predefined number,

coding the portion of the second intermediary picture using the portion of the first intermediary picture and by the method selected to code the portion of the first intermediary picture.

6. A method for coding according to claim 1, in which during a bi-directional coding of an intermediary picture from a portion of a first key picture and a portion of a second key picture, the saliency level compared to the predefined saliency level being a function of the saliency level of a first portion of the predicted saliency map from a first motion vector, and the saliency level of a second portion of the predicted saliency map from a second motion vector;

said intermediary picture portion being coded from a mean of predictions of the portion of the first key picture and the portion of the second key picture, said mean being weighted by said saliency levels of the first and the second portions of the predicted saliency map.

7. A method for coding according to claim 1, in which step

a) comprises a coding step of the key picture(s), said step generating reconstructed key picture(s), the saliency map being calculated from the reconstructed key picture(s).

8. A method for decoding a binary stream comprising compressed data representative of motion vectors, of intermediary pictures and of key picture(s) of a video sequence with a view to reconstructing a picture portion of an intermediary picture to be reconstructed, the method comprising the following steps:

a) reconstructing the key picture(s) from the compressed data,

b) calculating a saliency map of at least one of the reconstructed key picture(s),

c) decoding at least one motion vector for the picture portion to be reconstructed pointing towards a portion of the key picture(s),

d) motion compensating at least one portion of the saliency map using the decoded motion vector to obtain for said picture portion to be reconstructed, a portion of a predicted saliency map, and

e) reconstructing the picture portion to be reconstructed according to a saliency level of the predicted saliency map,

wherein step e) comprises the following steps;

determining the saliency level of each portion(s) of the predicted saliency map,

comparing a predefined saliency level and the saliency level of the portion of the predicted saliency map and

reconstructing the picture portion to be reconstructed using a first decoding method when the saliency level of the portion is greater than the predefined saliency level and using a second decoding method when the saliency level of the portion is lower than the predefined saliency level, said first decoding method being a decoding method with motion compensation and residue decoding and said second method being a decoding method with a more precise motion compensation than the motion compensation of the first decoding method and without residue decoding.

9. A method for decoding according to claim 8, in which the picture portion of the intermediary picture of the video sequence is decoded according to a compression method that is scalable by layers, the reconstruction of compressed data of a base layer generating the key picture(s), the reconstruction of compressed data of a high layer generating the picture portion to be reconstructed.

10. A method for decoding according to claim 8, that comprises the following steps:

decoding at least one motion vector going from a portion of a second intermediary picture and pointing towards at least one part of a portion of a first decoded intermediary picture;

the method also comprising the following steps;

counting the number of pixels of the part of the portion of the first decoded intermediary picture,

when the number of pixels counted is greater than a predefined number, reconstructing the portion of the second intermediary picture using the portion of the first intermediary picture and by the method selected to reconstruct the portion of the first intermediary picture.

11. A method for decoding according to claim 8, in which during a reconstruction of an intermediary picture portion coded according to a bi-directional prediction from a portion of the first key picture and a portion of the second key picture, the saliency level compared with the predefined saliency level is a function of the saliency level of a first portion of the saliency map predicted from a first motion vector, and the saliency level of a second portion of the saliency map predicted from a second motion vector, said intermediary picture portion being reconstructed from a mean of predictions of the portion of the first key picture and the portion of the second key picture, said mean being weighted by said saliency levels of the first and second portions of the predicted saliency map.

12. A coder able to code pictures of a video sequence, the video sequence comprising at least one key picture and at least one intermediary picture, the coder comprising:

a calculation unit of a saliency map of at least one the key picture(s);

an estimation unit able to estimate for a picture portion of an intermediary picture to be coded at least one motion vector pointing towards a portion of the key picture(s);

a compensation unit able to motion compensate the portion(s) of the saliency map using the estimated motion vector(s) to obtain for said picture portion, at least one portion of a predicted saliency map, and

a coding unit able to code the picture portion according to a saliency level of the portion(s) of the predicted saliency map, said coding unit comprising ;

a unit able to determine the saliency level of each portion(s) of the predicted saliency map;

a unit able to compare a predefined saliency level and the saliency level of the portion of the predicted saliency map;

a unit able to code the picture portion using a first coding method when the saliency level of the portion is greater than the predefined saliency level and using a second coding method when the saliency level of the portion is lower than the predefined saliency level, said first coding method being a coding method with motion compensation and residue coding and said second method being a coding method with a more precise motion compensation than the motion compensation of the first coding method and without residue coding.

13. A decoder of a binary stream comprising the compressed data representative of motion vectors, of intermediary pictures and of key picture(s) of a video sequence with a view to reconstruct a picture portion of an intermediary picture to be reconstructed, the decoder comprising:

a reconstruction unit of the key picture(s) from the compressed data,

a calculation unit of a saliency map of at least one of the reconstructed key picture(s),

a decoding unit able to decode at least one motion vector for said picture portion to be reconstructed pointing towards a portion of the key picture(s),

a motion compensation unit able to motion compensate at least one portion of the saliency map using a decoded motion vector to obtain, for said picture portion to be reconstructed, a portion of the predicted saliency map, and

a reconstruction unit able to reconstruct the picture portion to be reconstructed according to the saliency level of the portion of the predicted saliency map, said reconstruction unit comprising;

a unit able to determine the saliency level of each portion(s) of the predicted saliency map,

a unit able to compare a predefined saliency level and the saliency level of the portion of the predicted saliency map and

a unit able to reconstruct the picture portion to be reconstructed using a first decoding method when the saliency level of the portion is greater than the predefined saliency level and using a second decoding method when the saliency level of the portion is lower than the predefined saliency level, said first decoding method being a decoding method with motion compensation and residue decoding and said second method being a decoding method with a more precise motion compensation than the motion compensation of the first decoding method and without residue decoding.