Occlusion adaptive motion compensated interpolator

• US 8,576,341 B2
• Filed: 03/01/2010
• Issued: 11/05/2013
• Est. Priority Date: 03/01/2010
• Status: Active Grant

• Current Assignee: STMicroelectronics Incorporated (STMicroelectronics NV)
• Original Assignee: STMicroelectronics Incorporated (STMicroelectronics NV)
• Inventors: Petrides, Gordon
• Primary Examiner(s): Harold, Jefferey
• Assistant Examiner(s): HAIEM, SEAN N
• Application Number:
  US12/715,233
• Publication Number:
  US 20110211128A1
• Time in Patent Office: 1,345 Days
• Field of Search: None
• US Class Current: 348/616
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/543 using regions
• CPC Class Codes: H04N 19/553 Motion estimation dealing with occlusions
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

In one embodiment of the present invention, motion compensated interpolation is performed by locating full frame conceal and reveal areas, determining intermediate frame occlusion areas of an interpolated frame of the displayable output by locating intermediate frame conceal areas based on projected locations of pixels within the full frame conceal areas using forward motion vectors and information about a time slot for the interpolated frame, and by locating intermediate frame reveal areas based on projected locations of pixels within the full frame reveal areas using backward motion vectors and information about the time slot for the interpolated frame; for any pixels in the interpolated frame to which there is neither a forward vector nor a backward vector projecting: including the pixel in an intermediate frame conceal area if it is not located within the full frame reveal area; including the pixel in an intermediate frame reveal area if it is not located within the full frame conceal area; and using the intermediate frame conceal and reveal areas to interpolate values for pixels within an area of the interpolated frame.

Claims

1. A method for performing motion compensated interpolation using a previous frame and a current frame of a displayable output, the method comprising:

determining if motion of an object within the displayable output with respect to motion of a background within the displayable output is such that a particular area of an interpolated frame is a reveal region, a conceal region, or neither;

when the area is neither a reveal region nor a conceal region, using both forward vectors applied to the current frame and backward vectors applied to the previous frame to calculate values for pixels within the area;

when the area is a reveal region, using forward vectors applied to the current frame and to a frame subsequent to the current frame to calculate values for pixels within the area, without using backward vectors; and

when the area is a conceal region, using backward vectors applied to the previous frame and to a frame prior to the previous frame to calculate values for pixels within the area, without using forward vectors.

2. The method of claim 1, wherein when the area is neither a reveal region nor a conceal region, a value for a pixel within the area is calculated by taking the median of a pixel value generated using a corresponding forward vector applied to the current frame, a pixel value generated using a corresponding backward vector applied to the previous frame, and a temporal average pixel.

3. The method of claim 2, wherein the area is either a reveal region or a conceal region, a temporal average pixel is not used to determine pixel values for pixels within the area.

4. The method of claim 1, wherein the area is determined to be a reveal region when a pixel in the area in the current frame does not have a match in the previous frame.

5. The method of claim 1, wherein the area is determined to be a conceal region when a pixel in the area in the previous frame does not have a match in the current frame.

6. The method of claim 1, further comprising weighting the forward and backward vectors based upon their respective temporal distances from the interpolated frame.

7. An occlusion adaptive motion compensated temporal interpolator comprising:

a picture correlation vector confidence block configured to determine if motion of an object within a displayable output with respect to motion of a background within the displayable output is such that a particular area of an interpolated frame is a reveal region, a conceal region, or neither; and

a selector coupled to the picture correlation vector confidence block and configured to use forward vectors applied to a current frame and to a frame subsequent to the current frame to select a motion compensated pixel from the current frame as a value for a pixel in the interpolated frame if the area is a reveal region and to use backward vectors applied to a previous frame and to a frame prior to the previous frame to select a motion compensated pixel from the previous frame if the area is a conceal region.

8. The occlusion adaptive motion compensated temporal interpolator of claim 7, further comprising an assignment status histogram filter coupled between the picture correlation vector confidence block and the selector.

9. The occlusion adaptive motion compensated temporal interpolator of claim 7, wherein the selector is connected to a first motion compensated pixel interpolator and a second motion compensated pixel interpolator, wherein the first motion compensated pixel interpolator is configured to motion compensate a pixel from the current frame by using a forward vector, and wherein the second motion compensated pixel interpolator is configured to motion compensate a pixel from the previous frame by using a backward vector.

10. The occlusion adaptive motion compensated temporal interpolator of claim 7, wherein the selector is further configured to select a median of the motion compensated pixel from the current frame, a motion compensated pixel from the previous frame, and a temporal average pixel if the area is neither a reveal region nor a conceal region.

11. The occlusion adaptive motion compensated interpolator of claim 10, wherein the selector is coupled to a temporal average.

12. A display device comprising:

a digital display; and

a display controller comprising;

a picture correlation vector confidence block configured to determine if motion of an object within a displayable output with respect to motion of a background within the displayable output is such that a particular area of an interpolated frame is a reveal region, a conceal region, or neither; and

a selector coupled to the picture correlation vector confidence block and configured to use forward vectors applied to a current frame and to a frame subsequent to the current frame to select a motion compensated pixel from the current frame as a value for a pixel in the interpolated frame if the area is a reveal region and to use backward vectors applied to a previous frame and to a frame prior to the previous frame to select a motion compensated pixel from the previous frame if the area is a conceal region.

13. The display device of claim 12, wherein the display device is a television set.

14. The display device of claim 13, wherein the display device is a Liquid Crystal Display (LCD) device.

15. The display device of claim 12, wherein the selector is connected to a three tap median filter.

16. An occlusion adaptive motion compensated temporal interpolator comprising:

means for determining if motion of an object within the displayable output with respect to motion of a background within the displayable output is such that a particular area of an interpolated frame is a reveal region, a conceal region, or neither;

means for, when the area is neither a reveal region nor a conceal region, using both forward vectors applied to the current frame and backward vectors applied to the previous frame to calculate values for pixels within the area;

means for, when the area is a reveal region, using forward vectors applied to the current frame and to a frame subsequent to the current frame to calculate values for pixels within the area, without using backward vectors; and

means for, when the area is a conceal region, using backward vectors applied to the previous frame and to a frame prior to the previous frame to calculate values for pixels within the area, without using forward vectors.

17. The occlusion adaptive motion compensated temporal interpolator of claim 16, wherein the means for determining are located in a display controller.

18. The occlusion adaptive motion compensated temporal interpolator of claim 17, wherein the display controller is located in a televisions set.

19. A computer chip configured to:

determine if motion of an object within the displayable output with respect to motion of a background within the displayable output is such that a particular area of an interpolated frame is a reveal region, a conceal region, or neither;

when the area is neither a reveal region nor a conceal region, use both forward vectors applied to the current frame and backward vectors applied to the previous frame to calculate values for pixels within the area;

when the area is a reveal region, use forward vectors applied to the current frame and to a frame subsequent to the current frame to calculate values for pixels within the area, without using backward vectors; and

when the area is a conceal region, use backward vectors applied to the previous frame and to a frame prior to the previous frame to calculate values for pixels within the area, without using forward vectors.