Motion compensated video signal processing by interpolation of correlation surfaces and apparatus for doing the same

• US 5,485,224 A
• Filed: 02/04/1994
• Issued: 01/16/1996
• Est. Priority Date: 04/08/1993
• Status: Expired due to Term

• Current Assignee: Sony United Kingdom Limited (Sony Group Corp.)
• Original Assignee: Sony United Kingdom Limited (Sony Group Corp.)
• Inventors: Burns, James E., Dorricott, Martin R., David, Morgan W. A.
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US08/191,924
• Time in Patent Office: 711 Days
• Field of Search: 348/699, 348/409, 348/413, 348/419, 348/414, 348/700, 348/412, 348/405, 348/416
• US Class Current: 348/699
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: G06T 7/231 using full search
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

Motion compensated video signal processing apparatus, in which motion vectors are generated to represent image motion between a pair of input images of an input video signal, comprises means for comparing search blocks within one of the pair of input images with respective search areas, comprising a plurality of blocks, in the other of the pair of input images, to generate a first plurality of original correlation surfaces, each comprising an array of correlation values representing correlation between the respective search block and search area; means for generating a second plurality of interpolated correlation surfaces by interpolation from the original correlation surfaces; and means for generating a respective motion vector from each interpolated correlation surface, in dependence on a point of maximum correlation in that interpolated correlation surface.

Claims

1. Motion compensated video signal processing apparatus in which motion vectors are generated to represent image motion between a pair of input images of an input video signal, said apparatus comprising:

means for comparing search blocks within one of said pair of input images with respective search areas, comprising a plurality of blocks, in the other of said pair of input images, to generate a first plurality of original correlation surfaces, each comprising an array of correlation values representing correlation between said respective search block and search area;

means for generating a second plurality of interpolated correlation surfaces by interpolation from said plurality of original correlation surfaces; and

means for generating a respective motion vector from each interpolated correlation surface, in dependence on a point of maximum correlation in said interpolated correlation surface.

2. Apparatus according to claim 1, in which each original correlation surface comprises an array of correlation values representing a difference between a content of said respective search block and said respective search area.

3. Apparatus according to claim 2, in which said correlation values represent a difference between a luminance content of said search block and said search area.

4. Apparatus according to claim 2, in which said means for generating a motion vector comprises means for detecting a correlation value indicative of minimum difference between said search block and said search area.

5. Apparatus according to claim 1, in which said second plurality is greater than said first plurality.

6. Apparatus according to claim 1, in which:

said first plurality of original correlation surfaces comprises a rectangular array of original correlation surfaces, a position of each original correlation surface within said rectangular array depending on a position of a corresponding search block within said respective input image; and

said second plurality of interpolated correlation surfaces comprises a rectangular array of interpolated correlation surfaces.

7. Apparatus according to claim 1, in which each interpolated correlation surface comprises an array of correlation values, said apparatus comprising:

means for multiplying said array of correlation values in each interpolated correlation surface by a weighing value dependent on a position of said correlation value within said array, thereby generating a weighted interpolated correlation surface comprising an array of weighted correlation values; and

means for generating a motion vector from said weighted interpolated correlation surface, in dependence on a weighted correlation value indicative of a point of maximum correlation in said weighted correlation surface.

12. Apparatus according to claim 1, comprising a motion compensated interpolator for interpolating an output image of an output video signal according to said motion vectors generated from said pair of input images.

14. A motion compensated television standards converter comprising apparatus according to claim 1.

18. Apparatus according to claim 1, comprising means for performing a predetermined confidence test on each motion vector to determine whether said detected point of maximum correlation in said interpolated correlation surface represents a significant maximum in said interpolated correlation surface.

8. Motion compensated video signal processing apparatus in which motion vectors are generated to represent image motion between a pair of input images of an input video signal, said apparatus comprising:

means for comparing a search block within one of said pair of input images with a respective search area, comprising a plurality of blocks, in the other of said pair of input images, to generate a correlation surface comprising an array of correlation values representing correlation between said search block and said search area;

means for multiplying each correlation value by a weighting value dependent on a position of said correlation value within said array, thereby generating a weighted correlation surface comprising an array of weighted correlation values wherein said weighting values increase monotonically with increasing distance from a predetermined point within said array of correlation values and said weighting value at said predetermined point is a nonzero value; and

means for generating a motion vector from said weighted correlation surface, in dependence on a weighted correlation value indicative of a point of maximum correlation in said weighted correlation surface.

9. Apparatus according to claim 8, in which said predetermined point is a point indicative of zero image motion.

10. Apparatus according to claim 8, in which said weighting values increase in proportion to a square root of distance from said predetermined point.

11. Apparatus according to claim 8, comprising means for performing a predetermined confidence test on each motion vector to determine whether said detected point of maximum correlation in said weighted correlation surface represents a significant correlation maximum in said weighted correlation surface.

13. Apparatus according to claim 8, comprising a motion compensated interpolator for interpolating an output image of an output video signal according to said motion vectors generated from said pair of input images.

15. A motion compensated television standards converter comprising apparatus according to claim 8.

16. A method of motion compensated video signal processing in which motion vectors are generated to represent image motion between a pair of input images of an input video signal, said method comprising the steps of:

comparing search blocks within one of said pair of input images with respective search areas, comprising a plurality of blocks, in the other of said pair of input images, to generate a first plurality of original correlation surfaces, each comprising an array of correlation values representing correlation between said respective search block and said respective search area;

generating a second plurality of interpolated correlation surfaces by interpolation from said plurality of original correlation surfaces; and

generating a respective motion vector from each interpolated correlation surface, in dependence on a point of maximum correlation in said interpolated correlation surface.

17. A method of motion compensated video signal processing in which motion vectors are generated to represent image motion between a pair of input images of input video signals, said method comprising the steps of:

comparing a search block within one of said pair of input images with a respective search area, comprising a plurality of blocks, in the other of said pair of input images, to generate a correlation surface comprising an array of correlation values representing correlation between said search block and said search area;

multiplying each correlation value by a weighting value dependent on a position of said correlation value within said array, thereby generating a weighted correlation surface comprising an array of weighted correlation values, wherein said weighted values increase monotonically with increasing distance from a predetermined point within said array of correlation values and said weighting value at said predetermined point is a nonzero value; and

generating a motion vector from said weighted correlation surface, in dependence on a weighted correlation value indicative of a point of maximum correlation in said weighted correlation surface.