Video coder/decoder

• US 6,560,284 B1
• Filed: 06/13/2001
• Issued: 05/06/2003
• Est. Priority Date: 09/12/1997
• Status: Expired due to Term

• Current Assignee: VID SCALE Inc. (InterDigital, Inc.)
• Original Assignee: Netergy Microelectronics, Inc. (8X8, Inc.)
• Inventors: Girod, Bernd, Wiegand, Thomas
• Primary Examiner(s): BRITTON, HOWARD W
• Application Number:
  US09/880,582
• Time in Patent Office: 692 Days
• Field of Search: H04/N.737, H04/N.750, 375/240.16, 375/240.12, 382/236
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/58 Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one H04N19/23 takes precedence
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding

Abstract

Coding and decoding of video images includes the use of multiple reference pictures and determining motion vector and time delay information for the video images. Video images are quantized and transmitted with the motion vector and time delay information from a first station to a second station. A new video image is predicted for display at the second station as function of the motion parameter codes and multiple reference pictures. The invention may be used in connection with a video communication and/or video conferencing system as a long-term memory predictor to improve image-coding efficiencies and motion compensation. The long-term memory prediction extends the spatial displacement utilized in block-based hybrid video coding by a variable time delay. The arrangement and method employ a proposed bit allocation scheme to provide increased coding efficiencies of the video coder. Applications include video reproduction of repetitive sequences, such as when the video camera is shaking or an image portion is repetitively moving between previously-displayed positions.

Claims

1. For use in a system that is adapted for coding and decoding video images for transmission from a first station to a second station, a method of processing the video images, the method comprising:

in at least one of the first and second stations, providing respective sets of multiple reference pictures to permit use of up to N frames for prediction, where N is a positive integer;

at the first station,

encoding the video images,

determining motion parameters, including a frame selection parameter and a spatial displacement parameter, for the video images as a function of at least one of;

a distortion measure and a data transmission rate, and

providing the encoded video images and the motion parameters for processing at the second station; and

wherein a new video image for display at the second station can be predicted as function of the motion parameters and a group of the N frames.

2. The method of claim 1, wherein determining the motion parameters includes determining a motion vector and time delay information.

3. The method of claim 1, wherein determining the motion parameters includes determining the spatial displacement parameters as polynomial motion coefficients.

4. The method of claim 1, wherein the step of determining includes executing a minimization of a Lagrangian cost function.

5. The method of claim 1, wherein the motion parameters are determined as a function of both a distortion measure and a data transmission rate.

6. The method of claim 1, wherein the step of determining the motion parameters is a computed as a function of a minimization of a Lagrangian cost function based on both a distortion measure and a data transmission rate.

11. The arrangement of claim 1, wherein the motion parameters are determined as a function of a distortion measure and not a data transmission rate.

12. The arrangement of claim 1, wherein the motion parameters are determined as a function of a data transmission rate and not a distortion measure.

7. For use in a system that is adapted for coding and decoding video images for transmission from a first station to a second station, an arrangement for processing the video images, comprising:

memories for storing respective sets of multiple reference pictures at the first and second stations to permit use of up to frames N for prediction, where N is a positive integer;

an encoder responsive to video images provided at the first station and arranged to generate encoded video images;

means for determining motion parameters, including a frame selection parameter and a spatial displacement parameter, for the video images as a function of at least one of;

a distortion measure and a data transmission rate; and

means, responsive to the encoded video images and the motion parameters, for predicting a new video image for display at the second station as function of the motion parameters and a group of the N frames.

8. The arrangement of claim 7, wherein said motion parameters are determined as a function of a distortion measure and a data transmission rate.

9. The arrangement of claim 7, wherein the motion parameters are determined as a function of a distortion measure and not a data transmission rate.

10. The arrangement of claim 7, wherein the motion parameters are determined as a function of a data transmission rate and not a distortion measure.

13. For use in a system that is adapted for coding and decoding video images for transmission from a first station to a second station, a method of processing the video images, the method comprising:

in at least one of the first and second stations, providing respective sets of multiple reference pictures to permit use of up to N frames for prediction, where N is a positive integer;

at the second station

predicting a new video image for display as function of motion parameters and a group of the N frames;

wherein the motion parameters and the group of the N frames are processed at the first station, by

encoding the video images,

determining motion parameters, including a frame selection parameter and a spatial displacement parameter, for the video images as a function of at least one of;

a distortion measure and a data transmission rate.

14. For use in a system that is adapted for coding and decoding video images for transmission from a first station to a second station, an arrangement for processing the video images, the arrangement comprising:

in at least one of the first and second stations, means for providing respective sets of multiple reference pictures to permit use of up to N frames for prediction, where N is a positive integer;

at the second station

means for predicting a new video image for display as function of motion parameters and a group of the N frames;

wherein the motion parameters and the group of the N frames are processed at the first station, by

encoding the video images,

determining motion parameters, including a frame selection parameter and a spatial displacement parameter, for the video images as a function of at least one of;

a distortion measure and a data transmission rate.

15. The arrangement of claim 14, wherein the second station further includes means for receiving and storing a series of images provided by and forwarded from the first station.

16. The arrangement of claim 15, wherein the series of images provided by and forwarded from'"'"' the first station include one of:

images captured via a video-conference session, and rich-media images.

17. For use in a system that is adapted for coding and decoding video images for transmission from a first station to a second station, an arrangement for processing the video images, the arrangement comprising:

in at least one of the first and second stations, means for providing respective sets of multiple reference pictures to permit use of up to N frames for prediction, where N is a positive integer;

at the first station,

means for encoding the video images,

means for determining motion parameters, including a frame selection parameter and a spatial displacement parameter, for the video images as a function of at least one of;

a distortion measure and a data transmission rate, and

means for providing the encoded video images and the motion parameters for processing at the second station; and

wherein a new video image for display at the second station can be predicted as function of the motion parameters and a group of the N frames.

18. The arrangement of claim 17, wherein the first station further includes means for storing images corresponding to the encoded video images, and means for retrieving the stored images for providing the encoded video images for processing at the second station.

19. The arrangement of claim 18, wherein the encoded video images include one of:

images captured via a video-conference session, and rich-media images.