Motion vector detecting apparatus, motion vector detecting method, and program

• US 8,401,318 B2
• Filed: 07/23/2009
• Issued: 03/19/2013
• Est. Priority Date: 07/30/2008
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tetsukawa, Hiroki, Kondo, Tetsujiro, Takahashi, Kenji, Yoshikawa, Kazushi
• Primary Examiner(s): Newman, Michael A
• Application Number:
  US12/508,136
• Publication Number:
  US 20100027900A1
• Time in Patent Office: 1,335 Days
• Field of Search: 382/236, 382/103, 382/107, 382/197, 375/240.15, 375/240.16, 348/401.1, 348/402.1
• US Class Current: 382/236
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 19/57 Motion estimation characterised by a search window with variable size or shape

Abstract

Generation of evaluation value information about a motion vector, extraction of candidates for the motion vector on the basis of the evaluation value information, and determination of the motion vector to be allocated from the candidates for the motion vector are performed. The possibility that a reference pixel a in a second frame is a candidate for a motion from a target point in a first frame is evaluated on the basis of pixel-value correlation information to generate an evaluation value table. A constant area is adaptively set around each of the target pixel and the reference pixel corresponding to each extracted candidate for the motion vector in accordance with the state of the continuity of a constant pixel value. The motion vector from the first frame to the second frame is allocated on the basis of the result of comparison between values calculated for the constant areas.

Claims

1. A motion vector detecting apparatus comprising:

an evaluation-value information generating unit that generates evaluation value information indicating the evaluation value of a motion vector on the basis of pixel-value correlation information indicating the correlation between a target pixel in a first frame among multiple frames for moving image data and a reference pixel in a search area in a second frame, the evaluation value information indicating the result of evaluation of the possibility that the reference pixel is a candidate for a motion from the target pixel;

a motion-vector extracting unit that extracts a candidate for the motion vector corresponding to each pixel in the frames of the moving image data on the basis of the evaluation value information generated by the evaluation-value information generating unit; and

a motion-vector allocating unit that adaptively sets a constant area around each of the target pixel and the reference pixel corresponding to the candidate for the motion vector extracted by the motion-vector extracting unit in accordance with a state indicating the continuity of a constant pixel value and allocates the motion vector from the first frame to the second frame for each pixel on the basis of the result of comparison of the correlation between the constant areas, the motion-vector allocating unit setting the constant area to be an area including the target pixel and at least one other pixel, an absolute value of a difference between a luminance of the target pixel and a luminance of each of the at least one other pixel is less than a threshold, and each of the at least one other pixel is spatially contiguous to the target pixel.

2. The motion vector detecting apparatus according to claim 1, wherein the motion-vector allocating unit calculates a difference between the pixel value of each pixel in the constant area around the target pixel in the first frame and the pixel value of the corresponding pixel in the constant area around the reference pixel in the second frame, adds up the absolute values of the calculated differences in the constant area in the second frame to calculate the sum of the absolute differences, and allocates the motion vector corresponding to the area having the smallest sum of the absolute differences.

3. The motion vector detecting apparatus according to claim 2, wherein the motion-vector allocating unit sets a second reference pixel corresponding to the reference pixel in the first frame in a third frame that is opposite to the second frame in which the reference pixel exists with respect to the first frame in which the target pixel exists on a time axis, calculates a difference between the pixel value of each pixel in the constant area around the second reference pixel and the pixel value of the corresponding pixel in the constant area around the target pixel in the first frame, adds up the absolute values of the calculated differences in the constant area in the third frame to calculate the sum of the absolute differences, and allocates the motion vector from the first frame to the second frame on the basis of the result of the comparison for the constant area around the second reference pixel.

4. The motion vector detecting apparatus according to claim 3, wherein the sum of the absolute differences calculated for the constant area around the reference pixel in the second frame is compared with the sum of the absolute differences calculated for the constant area around the second reference pixel in the third frame, the motion vector connecting the reference pixel in the second frame to the target pixel is allocated if the comparison with the constant area around the target pixel in the first frame shows that the sum of the absolute differences calculated for the constant area around the reference pixel in the second frame is smaller than the sum of the absolute differences calculated for the constant area around the second reference pixel in the third frame, and the motion vector that results from extension of the motion vector between the second reference pixel and the target pixel and that extends from the first frame to the second frame is allocated if the comparison with the constant area around the target pixel in the first frame shows that the sum of the absolute differences calculated for the constant area around the second reference pixel in the third frame is smaller than the sum of the absolute differences calculated for the constant area around the reference pixel in the second frame.

5. The motion vector detecting apparatus according to claim 2, wherein information about a frequency of a candidate vector in an evaluation value table is acquired and the candidate vector having the highest frequency is allocated as the motion vector if multiple candidate vectors have the smallest sum of the absolute differences.

6. The motion vector detecting apparatus according to claim 2, wherein an average of the absolute differences resulting from normalization of the sum of the absolute differences calculated for each constant area by using a size of the constant area or the number of pixels in the constant area is used in the comparison.

7. The motion vector detecting apparatus according to claim 1, wherein the constant area having a constant pixel value is set around each of the target pixel and the reference pixel, the size of an area where the constant area around the reference pixel overlaps with the constant area around the target pixel is determined, and the motion vector corresponding to the reference pixel in the largest constant area overlapping with the constant area around the target pixel is allocated.

8. The motion vector detecting apparatus according to any of claims 1 to 7, wherein the motion-vector extracting unit sorts the candidates for the motion vector in frequency order on the basis of the evaluation value information generated by the evaluation-value information generating unit and extracts the candidates for the motion vector up to a predetermined order from the top.

9. A motion vector detecting method comprising:

generating evaluation value information indicating the evaluation value of the motion vector on the basis of pixel-value correlation information indicating the correlation between a target pixel in a first frame among multiple frames for moving image data and a reference pixel in a search area in a second frame, the evaluation value information indicating the result of evaluation of the possibility that the reference pixel is a candidate for a motion from the target pixel;

extracting a candidate for the motion vector corresponding to each pixel in the frames of the moving image data on the basis of the generated evaluation value information; and

setting a constant area having a constant pixel value around each of the target pixel and the reference pixel corresponding to the extracted candidate for the motion vector and allocating the motion vector from the first frame to the second frame for each pixel on the basis of the result of comparison between values calculated for the constant areas, the setting including setting the constant area to be an area including the target pixel and at least one other pixel, an absolute value of a difference between a luminance of the target pixel and a luminance of each of the at least one other pixel is less than a threshold, and each of the at least one other pixel is spatially contiguous to the target pixel.

10. A non-transitory computer readable medium encoded with a program which, when installed in an information processing apparatus, causes the apparatus to execute a method comprising:

generating evaluation value information indicating the evaluation value of a motion vector on the basis of pixel-value correlation information indicating the correlation between a target pixel in a first frame among multiple frames for moving image data and a reference pixel in a search area in a second frame, the evaluation value information indicating the result of evaluation of the possibility that the reference pixel is a candidate for a motion from the target pixel;

extracting a candidate for the motion vector corresponding to each pixel in the frames of the moving image data on the basis of the generated evaluation value information; and

setting a constant area having a constant pixel value around each of the target pixel and the reference pixel corresponding to the extracted candidate for the motion vector and allocating the motion vector from the first frame to the second frame for each pixel on the basis of the result of comparison between values calculated for the constant areas, the setting including setting the constant area to be an area including the target pixel and at least one other pixel, an absolute value of a difference between a luminance of the target pixel and a luminance of each of the at least one other pixel is less than a threshold, and each of the at least one other pixel is spatially contiguous to the target pixel.