METHOD AND APPARATUS FOR ENCODING AND DECODING VIDEO IMAGES

• US 20080123747A1
• Filed: 11/28/2007
• Published: 05/29/2008
• Est. Priority Date: 11/28/2006
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: AHN, Tae-gyoung, LEE, Jae-hun, YOO, Ki-won
• Application Number:
  US11/946,168
• Publication Number:
  US 20080123747A1
• Time in Patent Office: Days
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/543 using regions

Abstract

A method of and apparatus for encoding and/or decoding an image are provided, in which motion estimation and compensation are performed by using a reference background image and a reference object image extracted from an input image frame. In the method and apparatus, a background image of each of input image frames is used, and a background area which any one image frame lacks is updated by using a background image of another image frame. In this way, a reference background image is generated and stored, and an object image in each image frame is stored as a reference object image. Then, by using the stored reference background images and reference object images, motion estimation and compensation are performed, thereby increasing encoding efficiency.

Claims

1. A method of encoding an image comprising:

dividing a current image frame into a background area and an object area;

comparing the current image frame with a previous image frame, thereby determining whether a background area or an object area has changed between the current image frame and the previous image frame;

according to the result of the determination, updating a reference background image which is generated from backgrounds of previous image frames, by using the background area of the current image frame, and storing the updated reference background image; and

performing motion estimation of an image frame subsequent to the current image frame by using the updated reference background image.

2. The method of claim 1, wherein the dividing of the current image frame into the background area and the object area comprises:

performing block based motion estimation of the current image frame, and generating motion vectors;

calculating the magnitudes of each of the motion vectors of the current image frame; and

for a block in which the magnitude of the motion vector of the block is equal to or greater than a predetermined threshold, determining that the block is included in an object area, and for a block in which the magnitude of the motion vector of the block is less than the predetermined threshold, determining that the block is included in a background area.

3. The method of claim 1, wherein the dividing of the current image frame into the background area and the object area comprises:

performing block based motion estimation of the current image frame, and generating motion vectors;

calculating a global motion vector representing all motion vectors of blocks in the current image frame; and

among the blocks in the current image frame, determining blocks for which the difference between the global motion vector and the motion vector of the block is equal to or greater than a predetermined threshold, as an object area, and determining a block for which the difference between the global motion vector and the motion vector of the block is less than the predetermined threshold, as a background area.

4. The method of claim 1, wherein the updating of the reference background image comprises:

calculating clearness of an area which is determined to be a background area in both the current image frame and the previous image frame;

selecting a background area having higher clearness from the background areas of the current image frame and the previous image frame; and

replacing an area of the reference background image, corresponding to the selected background area, with the selected background area, thereby updating the reference background image.

5. The method of claim 4, wherein the selecting of background area having higher clearness comprises:

transforming the current image frame and the previous image frame into frequency domain; and

comparing the high frequency components of the current image frame and the previous image frame, and selecting a background area having higher high frequency components of between the current image frame and the previous image frame, as the background area having higher clearness.

6. The method of claim 1, wherein in the updating of the reference background image, an area of the reference background image corresponding to a background area of the current image frame, which is determined to be an object area in the previous image frame and a background area in the current image frame, is replaced with the background area of the current image frame, thereby updating the reference background image.

7. The method of claim 1, further comprising storing an object area of the current image frame, which is determined to be a background area in the previous image frame and an object area in the current image frame, as reference object image.

8. An apparatus for encoding an image comprising:

an area separation unit which divides a current image frame into a background area and an object area;

a determination unit which compares the current image frame with a previous image frame, thereby determining whether a background area or an object area changes between the current image frame and the previous image frame;

an update unit which updates a reference background image which is generated from backgrounds of previous image frames, by using the background area of the current image frame, according to the result of the determination, and storing the updated reference background image; and

a motion estimation unit which performs motion estimation of an image frame subsequent to the current image frame, by using the updated reference background image.

9. The apparatus of claim 8, wherein the area separation unit comprises:

a motion vector magnitude calculation unit which calculates the magnitude of a motion vector of each of blocks of the current image frame, by using motion vectors generated by performing block based motion estimation of the current image frame; and

an area determination unit which determines for a block in which the magnitude of a motion vector of the block is equal to or greater than a predetermined threshold, that the block is included in an object area, and which determines, for a block in which the magnitude of a motion vector of the block is less than the predetermined threshold, that the block is included in a background area.

10. The apparatus of claim 8, wherein the area separation unit comprises:

a global motion vector calculation unit which calculates a global motion vector representing motion vectors of all of blocks of the current image frame; and

an area determination unit which determines, among the blocks of the current image frame, a block for which the difference between the global motion vector and a motion vector of the block is equal to or greater than a predetermined threshold, as an object area, and which determines a block for which the difference between the global motion vector and a motion vector of the block is less than the predetermined threshold, as a background area.

11. The apparatus of claim 8, wherein the update unit comprises:

an update area selection unit which calculates clearness of an area which is determined to be a background area in both the current image frame and the previous image frame, and which selects a background area having higher clearness from background areas of the current image frame and the previous image frame; and

an updating unit which replaces an area in the reference background image, corresponding to the selected background area, with the selected background area, thereby updating the reference background image.

12. The apparatus of claim 11, wherein the update area selection unit transforms the current image frame and the previous image frame into frequency domain, compares the high frequency components of the current image frame and the previous image frame, and selects a background area having higher high frequency components of between the current image frame and the previous image frame, as the background area having higher clearness.

13. The apparatus of claim 8, wherein the update unit replaces an area of the reference background image corresponding to a background area of the current image frame, which is determined to be an object area in the previous image frame and a background area in the current image frame, with the background area of the current image frame, thereby updating the reference background image.

14. The apparatus of claim 8, wherein the storage unit stores an object area of the current image frame, which is determined to be a background area in the previous image frame and an object area in the current image frame, as a reference object image.

15. A method of decoding an image comprising:

separating a background area and an object area from each of at least two image frames, including a first image frame and a second image frame, restored before a current image frame is decoded;

comparing the restored first image frame and second image frame, and determining whether or not a background area or an object area changes between the first image frame and the second image frame;

according to the result of the determination, updating a reference background image which is generated from background areas of previously restored image frames, by using the backgrounds of the first image frame and the second image frame, and storing the updated reference background image; and

by using the updated reference background image, performing motion compensation of the current image frame.

16. The method of claim 15, wherein the separating of the background area and object area from each of the first image frame and the second image frame comprises:

by using motion vectors of blocks of the first image frame and the second image frame included in an input bitstream, determining the magnitude of the motion vector of each of the blocks included in the first image frame and the second image frame; and

for a block in which the magnitude of a motion vector of the block is equal to or greater than a predetermined threshold, determining that the block is included in an object area, and for a block in which the magnitude of a motion vector of the block is less than the predetermined threshold, determining that the block is included in a background area.

17. The method of claim 15, wherein the separating of the background area and object area from each of the first image frame and the second image frame comprises:

by using motion vectors of the first image frame and the second image frame included in the input bitstream, calculating a first global motion vector representing the motion vectors of the blocks in the first image frame, and a second global motion vector representing the motion vectors of the blocks in the second image frame; and

for each of the blocks of the first image frame and the second image frame, comparing the motion vector of the block with a corresponding one of the first global motion vector and the second global motion vector, and determining a block in which the difference between the global motion vector and the motion vector of the block is equal to or greater than a predetermined threshold, as an object area, and determining a block in which the difference between the global motion vector and the motion vector of the block is less than the predetermined threshold, as a background area.

18. The method of claim 15, wherein updating of the reference background image comprises:

calculating clearness of an area which is determined to be a background area both in the first image frame and the second image frame;

selecting a background area having higher clearness from the background areas of the first image frame and the second image frame; and

replacing an area in the reference background image corresponding to the selected background area, with the selected background area, thereby updating the reference background image.

19. The method of claim 18, wherein the selecting of the background area having higher clearness comprises:

transforming the first image frame and the second image frame into frequency domain; and

comparing high frequency components of the first image frame and the second image frame, and selecting a background area having higher high frequency components of between the first image frame and the second image frame, as the background area having higher clearness.

20. The method of claim 15, wherein in the updating of the reference background image, an area of the reference background image corresponding to a background area of the second image frame, which is determined to be an object area in the first image frame and a background area in the second image frame, is replaced by the background area of the second image frame, thereby updating the reference background image.

21. The method of claim 15, further comprising storing an object area of the second image frame, which is determined to be a background area in the first image frame and an object area in the second image frame, as a reference object image.

22. An apparatus for decoding an image comprising:

an area separation unit which separates a background area and an object area from each of at least two image frames, including a first image frame and a second image frame, restored before a current image frame is decoded;

a determination unit which compares the restored first image frame and second image frame, and which determines whether or not a background area or an object area changes between the first image frame and the second image frame;

an update unit which updates a reference background image which is generated from background areas of previously restored image frames, by using backgrounds of the first image frame and the second image frame, according to the result of the determination;

a storage unit which stores the updated reference background image; and

a motion compensation unit which performs motion compensation of the current image frame, by using the updated reference background image.

23. The apparatus of claim 22, wherein the area separation unit comprises:

a motion vector magnitude calculation unit which calculates the magnitude of a motion vector of each of the blocks included in the first image frame and the second image frame, by using motion vectors of each of the blocks of the first image frame and the second image frame included in an input bitstream; and

an area determination unit, which determines, for a block in which the magnitude of the motion vector of the block is equal to or greater than a predetermined threshold, that the block is included in an object area, and which determines, for a block in which the magnitude of the motion vector of the block is less than the predetermined threshold, that the block is included in a background area.

24. The apparatus of claim 22, wherein the area separation unit comprises:

a global motion vector calculation unit which calculates a first global motion vector representing motion vectors of the blocks in the first image frame, and a second global motion vector representing the motion vectors of the blocks in the second image frame, by using the motion vectors of each of the blocks of the first image frame and the second image frame included in the input bitstream; and

an area determination unit which compares, for each of the blocks of the first image frame and the second image frame, the motion vector of the block with a corresponding one of the first global motion vector and the second global motion vector, and which determines a block in which the difference between the global motion vector and the motion vector of the block is equal to or greater than a predetermined threshold, as an object area, and determining a block in which the difference between the global motion vector and the motion vector of the block is less than the predetermined threshold, as a background area.

25. The apparatus of claim 22, wherein the update unit comprises:

an update area selection unit which calculates clearness of an area which is determined to be a background area both in the first image frame and the second image frame, and which selects a background area having higher clearness from the background areas of the first image frame and the second image frame; and

an updating unit which replaces an area in the reference background image corresponding to the selected background area, with the selected background area, thereby updating the reference background image.