Method for improving accuracy of block based motion compensation

• US 6,810,081 B2
• Filed: 12/15/2000
• Issued: 10/26/2004
• Est. Priority Date: 12/15/2000
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Dantwala, Nehal R.
• Primary Examiner(s): Vo, Tung T.
• Application Number:
  US09/738,644
• Publication Number:
  US 20020075959A1
• Time in Patent Office: 1,411 Days
• Field of Search: H04/B.166, H04/N.544, 375/240.01, 375/240.9, 375/240.12, 375/240.14, 375/240.16, 375/240.17, 348/448, 348/452, 348/699, 348/700, 382/236, 382/238, 382/240
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/112 according to a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence

Abstract

For use in a video image upconversion unit of the type that uses motion compensation to generate an interpolated field using motion vectors, an improved method of motion compensation is disclosed. The method performs motion compensation on a pixel and determines whether a motion vector assigned to the pixel is correct or incorrect. If the motion vector is incorrect, the method sets the value of the pixel to a previously recorded pixel value. The previously recorded pixel value is obtained by calculating for the pixel the difference between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field. The difference is then compared with a threshold value. If the difference is less than the threshold value, then the value of the pixel is set equal to the average of the value of the corresponding motion compensated pixel from the previous frame and the value of the corresponding motion compensated pixel from the next field.

Claims

1. For use in a video image upconversion unit of the type that uses motion compensation to generate an interpolated field using motion vectors, a method of motion compensation comprising the steps of:

calculating for a pixel within an array of n pixels by m pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of said pixel within said array of n pixels by m pixels equal to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field if the value of said difference is less than said threshold value; and

recording said value for said pixel within said array of n pixels by m pixels.

2. The method as claimed in claim 1 further comprising the steps of:

calculating for each of a plurality of pixels within an array of n pixels by m pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of each of said plurality of pixels within said array of n pixels by m pixels equal to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field if the value of said difference is less than said threshold value; and

recording said values for said plurality of pixels within said array of n pixels by m pixels.

3. The method as claimed in claim 2 wherein said threshold value is not greater than ten.

4. The method as claimed in claim 2 further comprising the steps of:

calculating for a pixel within a block of pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of said pixel within said block of pixels to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field, if the value of said difference is less than said threshold value; and

setting the value of said pixel within said block of pixels equal to a value that was previously recorded for said pixel when said pixel was evaluated as a pixel element within said array of n pixels by m pixels, if the value of said difference is not less than said threshold value, and if a recorded value for said pixel was previously recorded.

5. The method as claimed in claim 4 further comprising the steps of:

calculating for each of a plurality of pixels within said block of pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of each of said plurality of pixels within said block of pixels to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field, if the value of said difference is less than said threshold value; and

setting the value of each of said plurality of pixels within said block of pixels equal to a value that was previously recorded for each of said plurality of pixels when each of said plurality of pixels was evaluated as a pixel element within said array of n pixels by m pixels, if the value of said difference is not less than said threshold value, and if a recorded value for said pixel was previously recorded.

6. The method as claimed in claim 5 wherein said threshold value is not greater than ten.

7. For use in a video image upconversion unit of the type that uses motion compensation to generate an interpolated field using motion vectors, a method of motion compensation comprising the steps of:

performing motion compensation on a pixel;

determining whether a motion vector assigned to said pixel is incorrect;

obtaining a previously recorded pixel value for said pixel when it is determined that said motion vector assigned to said pixel is incorrect, said previously recorded pixel value being determined by a method comprising the steps of;

calculating for said pixel when said pixel is a pixel element within an array of n pixels by m pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field,

comparing said difference with a threshold value, and

setting the value of said pixel within said array of n pixels by m pixels equal to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field if the value of said difference is less than said threshold value; and

setting the value of said pixel equal to said previously recorded pixel value for said pixel.

8. The method as claimed in claim 7 wherein said threshold value is not greater than ten.

9. The method as claimed in claim 7 wherein said step of determining whether a motion vector assigned to said pixel is incorrect comprises the steps of:

obtaining a value of a corresponding motion compensated pixel from a previous frame;

obtaining a value of a corresponding motion compensated pixel from a next field;

calculating the difference in value between said value of said corresponding motion compensated pixel from said previous frame and said value of said corresponding motion compensated pixel from said next field;

comparing said difference with a threshold value;

determining that said motion vector assigned to said pixel is incorrect if said difference is greater than said threshold value.

10. The method as claimed in claim 9 wherein said threshold value is not greater than ten.

11. For use in a video image upconversion unit of the type that uses motion compensation to generate an interpolated field using motion vectors, a method of motion compensation comprising the steps of:

performing motion compensation on a first pixel in a plurality of pixels within an array of n pixels by m pixels;

determining pixel values for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel, said pixel values being determined by a method comprising the steps of;

calculating for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field,

comparing said difference with a threshold value, and

setting the value of each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel equal to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field if the value of said difference is less than said threshold value; and

recording said pixel values for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel.

12. The method as claimed in claim 11 Wherein said threshold value is not greater than ten.

13. The method as claimed in claim 11 further comprising the steps of:

performing motion compensation on each of a plurality of pixels in a block of pixels; and

determining whether a motion vector assigned to a pixel of said plurality of pixels in said block of pixels is incorrect.

14. The method as claimed in claim 13 further comprising the steps of:

obtaining a previously recorded pixel value for said pixel when it is determined that said motion vector assigned to said pixel is incorrect; and

setting the value of said pixel equal to said previously recorded pixel value for said pixel, where said previously recorded pixel value is one of said pixel values recorded for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel.

15. The method as claimed in claim 13 wherein said motion vector assigned to a pixel of said plurality of pixels in said block of pixels is one of a plurality of motion vectors assigned to pixels in said block of pixels.

16. The method as claimed in claim 13 wherein said array of n pixels by m pixels is repeatedly located in different positions with respect to said plurality of pixels in said block of pixels to obtain pixel values for each of said plurality of pixels in said block of pixels.

17. The method as claimed in claim 13 wherein the step of determining whether a motion vector assigned to a pixel of said plurality of pixels in said block of pixels is incorrect comprises the steps of:

obtaining a value of a corresponding motion compensated pixel from a previous frame;

obtaining a value of a corresponding motion compensated pixel from a next field;

calculating the difference in value between said value of said corresponding motion compensated pixel from said previous frame and said value of said corresponding motion compensated pixel from said next field;

comparing said difference with a threshold value;

determining that said motion vector assigned to said pixel of said plurality of pixels in said block of pixels is incorrect if said difference is greater than said threshold value.

18. A high definition progressive to interlace converter comprising a high definition video image upconversion unit that uses motion compensation to generate an interpolated field using motion vectors, wherein said high definition video image upconversion unit is capable of:

calculating for a pixel within an array of n pixels by m pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of said pixel within said array of n pixels by m pixels equal to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field if the value of said difference is less than said threshold value; and

recording said value for said pixel within said array of n pixels by m pixels.

19. The high definition progressive to interlace converter as claimed in claim 18 wherein said high definition video image upconversion unit is capable of:

calculating for each of a plurality of pixels within an array of n pixels by m pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of each of said plurality of pixels within said array of n pixels by m pixels equal to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field if the value of said difference is less than said threshold value; and

recording said values for said plurality of pixels within said array of n pixels by m pixels.

20. The high definition progressive to interlace converter as claimed in claim 19 wherein said high definition video image upconversion unit is capable of:

calculating for a pixel within a block of pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of said pixel within said block of pixels to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field, if the value of said difference is less than said threshold value; and

setting the value of said pixel within said block of pixels equal to a value that was previously recorded for said pixel when said pixel was evaluated as a pixel element within said array of n pixels by m pixels, if the value of said difference is not less than said threshold value, and if a recorded value for said pixel was previously recorded.

21. The high definition progressive to interlace converter as claimed in claim 20 wherein said high definition video image upconversion unit is capable of:

calculating for each of a plurality of pixels within said block of pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of each of said plurality of pixels within said block of pixels to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field, if the value of said difference is less than said threshold value; and

setting the value of each of said plurality of pixels within said block of pixels equal to a value that was previously recorded for each of said plurality of pixels when each of said plurality of pixels was evaluated as a pixel element within said array of n pixels by m pixels, if the value of said difference is not less than said threshold value, and if a recorded value for said pixel was previously recorded.

22. The high definition progressive to interlace converter as claimed in claim 21 wherein said threshold value is not greater than ten.

23. The high definition progressive to interlace converter as claimed in claim 18 wherein said high definition video image upconversion unit is capable of:

performing motion compensation on a first pixel in a plurality of pixels within an array of n pixels by m pixels;

determining pixel values for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel;

recording said pixel values for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel;

performing motion compensation of each of a plurality of pixels in a block of pixels;

determining whether a motion vector assigned to a pixel of said plurality of pixels in said block of pixels in incorrect;

obtaining a previously recorded pixel value for said pixel when it is determined that said motion vector assigned to said pixel is incorrect; and

setting the value of said pixel equal to said previously recorded pixel value for said pixel, where said previously recorded pixel value is one of said pixel values recorded for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel.

24. A high definition receiver comprising a high definition progressive to interlace converter comprising a high definition video image upconversion unit that uses motion compensation to generate an interpolated field using motion vectors, wherein said high definition video image upconversion unit is capable of:

calculating for a pixel within an array of n pixels by m pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of said pixel within said array of n pixels by m pixels equal to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field if the value of said difference is less than said threshold value; and

recording said value for said pixel within said array of n pixels by m pixels.

25. The high definition receiver as claimed in claim 24 wherein said high definition video image upconversion unit is capable of:

calculating for each of a plurality of pixels within an array of n pixels by m pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of each of said plurality of pixels within said array of n pixels by m pixels equal to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field if the value of said difference is less than said threshold value; and

recording said values for said plurality of pixels within said array of n pixels by m pixels.

26. The high definition receiver as claimed in claim 25 wherein said high definition video image upconversion unit is capable of:

calculating for a pixel within a block of pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of said pixel within said block of pixels to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field, if the value of said difference is less than said threshold value; and

setting the value of said pixel within said block of pixels equal to a value that was previously recorded for said pixel when said pixel was evaluated as a pixel element within said array of n pixels by m pixels, if the value of said difference is not less than said threshold value, and if a recorded value for said pixel was previously recorded.

27. The high definition receiver as claimed in claim 26 wherein said high definition video image upconversion unit is capable of:

calculating for each of a plurality of pixels within said block of pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of each of said plurality of pixels within said block of pixels to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field, if the value of said difference is less than said threshold value; and

setting the value of each of said plurality of pixels within said block of pixels equal to a value that was previously recorded for each of said plurality of pixels when each of said plurality of pixels was evaluated as a pixel element within said array of n pixels by m pixels, if the value of said difference is not less than said threshold value, and if a recorded value for said pixel was previously recorded.

28. The high definition receiver as claimed in claim 27 wherein said threshold value is not greater than ten.

29. The high definition progressive to interlace converter as claimed in claim 24 wherein said high definition video image upconversion unit is capable of:

performing motion compensation on a first pixel in a plurality of pixels within an array of n pixels by m pixels;

determining pixel values for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel;

recording said pixel values for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel;

performing motion compensation of each of a plurality of pixels in a block of pixels;

determining whether a motion vector assigned to a pixel of said plurality of pixels in said block of pixels in incorrect;

obtaining a previously recorded pixel value for said pixel when it is determined that said motion vector assigned to said pixel is incorrect; and

setting the value of said pixel equal to said previously recorded pixel value for said pixel, where said previously recorded pixel value is one of said pixel values recorded for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel.

30. A signal comprising:

an output signal of a high definition progressive to interlace converter comprising a high definition video image upconversion unit that uses motion compensation to generate an interpolated field using motion vectors, wherein said high definition video image upconversion unit generates said output signal by;

calculating for a pixel within an array of n pixels by m pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of said pixel within said array of n pixels by m pixels equal to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field if the value of said difference is less than said threshold value; and

recording said value for said pixel within said array of n pixels by m pixels.

31. The signal as claimed in claim 30 wherein said high definition video image upconversion unit generates said output signal by:

calculating for each of a plurality of pixels within an array of n pixels by m pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of each of said plurality of pixels within said array of n pixels by m pixels equal to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field if the value of said difference is less than said threshold value; and

recording said values for said plurality of pixels within said array of n pixels by m pixels.

32. The signal as claimed in claim 31 wherein said high definition video image upconversion unit generates said output signal by:

calculating for a pixel within a block of pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of said pixel within said block of pixels to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field, if the value of said difference is less than said threshold value; and

setting the value of said pixel within said block of pixels equal to a value that was previously recorded for said pixel when said pixel was evaluated as a pixel element within said array of n pixels by m pixels, if the value of said difference is not less than said threshold value, and if a recorded value for said pixel was previously recorded.

33. The signal as claimed in claim 32 wherein said high definition video image upconversion unit generates said output signal by:

calculating for each of a plurality of pixels within said block of pixels the difference in value between the value of a corresponding motion compensated pixel from a previous frame and the value of a corresponding motion compensated pixel from a next field;

comparing said difference with a threshold value;

setting the value of each of said plurality of pixels within said block of pixels to the average of the value of said corresponding motion compensated pixel from said previous frame and the value of said corresponding motion compensated pixel of said next field, if the value of said difference is less than said threshold value; and

setting the value of each of said plurality of pixels within said block of pixels equal to a value that was previously recorded for each of said plurality of pixels when each of said plurality of pixels was evaluated as a pixel element within said array of n pixels by m pixels, if the value of said difference is not less than said threshold value, and if a recorded value for said pixel was previously recorded.

34. The signal as claimed in claim 33 wherein said threshold value is not greater than ten.

35. The signal as claimed in claim 30 wherein said high definition video image upconversion unit generates said output signal by:

performing motion compensation on a first pixel in a plurality of pixels within an array of n pixels by m pixels;

determining pixel values for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel;

recording said pixel values for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel;

performing motion compensation of each of a plurality of pixels in a block of pixels;

determining whether a motion vector assigned to a pixel of said plurality of pixels in said block of pixels in incorrect;

obtaining a previously recorded pixel value for said pixel when it is determined that said motion vector assigned to said pixel is incorrect; and

setting the value of said pixel equal to said previously recorded pixel value for said pixel, where said previously recorded pixel value is one of said pixel values recorded for each of said plurality of pixels within said array of n pixels by m pixels other than said first pixel.