MOTION VECTOR DETECTING APPARATUS, MOTION VECTOR DETECTING METHOD, AND PROGRAM

• US 20100027900A1
• Filed: 07/23/2009
• Published: 02/04/2010
• Est. Priority Date: 07/30/2008
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tetsukawa, Hiroki, Takahashi, Kenji, Kondo, Tetsujiro, Yoshikawa, Kazushi
• Granted Patent: US 8,401,318 B2
• Time in Patent Office: Days
• US Class Current: 382/236
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 19/57 Motion estimation characterised by a search window with variable size or shape

Abstract

Generation of evaluation value information about a motion vector, extraction of candidates for the motion vector on the basis of the evaluation value information, and determination of the motion vector to be allocated from the candidates for the motion vector are performed. The possibility that a reference pixel a in a second frame is a candidate for a motion from a target point in a first frame is evaluated on the basis of pixel-value correlation information to generate an evaluation value table. A constant area is adaptively set around each of the target pixel and the reference pixel corresponding to each extracted candidate for the motion vector in accordance with the state of the continuity of a constant pixel value. The motion vector from the first frame to the second frame is allocated on the basis of the result of comparison between values calculated for the constant areas.

Claims

1. A motion vector detecting apparatus comprising:

an evaluation-value information generating unit that generates evaluation value information indicating the evaluation value of a motion vector on the basis of pixel-value correlation information indicating the correlation between a target pixel in a first frame among multiple frames for moving image data and a reference pixel in a search area in a second frame, the evaluation value information indicating the result of evaluation of the possibility that the reference pixel is a candidate for a motion from the target point;

a motion-vector extracting unit that extract a candidate for the motion vector corresponding to each pixel in the frames of the moving image data on the basis of the evaluation value information generated by the evaluation-value information generating unit; and

a motion-vector allocating unit that adaptively sets a constant area around each of the target pixel and the reference pixel corresponding to the candidate for the motion vector extracted by the motion-vector extracting unit in accordance with a state indicating the continuity of a constant pixel value and allocates the motion vector from the first frame to the second frame for each pixel on the basis of the result of comparison of the correlation between the constant areas.

2. The motion vector detecting apparatus according to claim 1,

wherein the motion-vector allocating unit calculates a difference between the pixel value of each pixel in the constant area around the target point in the first frame and the pixel value of the corresponding pixel in the constant area around the reference pixel in the second frame, adds up the absolute values of the calculated differences in the constant area in the second frame to calculate the sum of the absolute differences, and allocates the motion vector corresponding to the area having the smallest sum of the absolute differences.

3. The motion vector detecting apparatus according to claim 2,

wherein the motion-vector allocating unit sets a second reference pixel corresponding to the reference point in the first frame in a third frame that is opposite to the second frame in which the reference pixel exists with respect to the first frame in which the target pixel exists on a time axis, calculates a difference between the pixel value of each pixel in the constant area around the second reference pixel and the pixel value of the corresponding pixel in the constant area around the target pixel in the first frame, adds up the absolute values of the calculated differences in the constant area in the third frame to calculate the sum of the absolute differences, and allocates the motion vector from the first frame to the second frame on the basis of the result of the comparison for the constant area around the second reference pixel.

4. The motion vector detecting apparatus according to claim 3,

wherein the sum of the absolute differences calculated for the constant area around the reference pixel in the second frame is compared with the sum of the absolute differences calculated for the constant area around the second reference pixel in the third frame, the motion vector connecting the reference pixel in the second frame to the target pixel is allocated if the comparison with the constant area around the target pixel in the first frame shows that the sum of the absolute differences calculated for the constant area around the reference pixel in the second frame is smaller than the sum of the absolute differences calculated for the constant area around the second reference pixel in the third frame, and the motion vector that results from extension of the motion vector between the second reference pixel and the target point and that extends from the first frame to the second frame is allocated if the comparison with the constant area around the target pixel in the first frame shows that the sum of the absolute differences calculated for the constant area around the second reference pixel in the third frame is smaller than the sum of the absolute differences calculated for the constant area around the reference pixel in the second frame.

5. The motion vector detecting apparatus according to claim 2,

wherein information about a frequency of a candidate vector in an evaluation value table is acquired and the candidate vector having the highest frequency is allocated as the motion vector if multiple candidate vectors have the smallest sum of the absolute differences.

6. The motion vector detecting apparatus according to claim 2,

wherein an average of the absolute differences resulting from normalization of the sum of the absolute differences calculated for each constant area by using a size of the constant area or the number of pixels in the constant area is used in the comparison.

7. The motion vector detecting apparatus according to claim 1,

wherein the constant area having a constant pixel value is set around each of the target pixel and the reference pixel, the size of an area where the constant area around the reference pixel overlaps with the constant area around the target pixel is determined, and the motion vector corresponding to the reference point in the largest constant area overlapping with the constant area around the target pixel is allocated.

8. The motion vector detecting apparatus according to any of claims 1 to 7,

wherein the motion-vector extracting unit sorts the candidates for the motion vector in frequency order on the basis of the evaluation value information generated by the evaluation-value information generating unit and extracts the candidates for the motion vector up to a predetermined order from the top.

9. A motion vector detecting method comprising the steps of:

generating evaluation value information indicating the evaluation value of the motion vector on the basis of pixel-value correlation information indicating the correlation between a target pixel in a first frame among multiple frames for moving image data and a reference pixel in a search area in a second frame, the evaluation value information indicating the result of evaluation of the possibility that the reference pixel is a candidate for a motion from the target point;

extracting a candidate for the motion vector corresponding to each pixel in the frames of the moving image data on the basis of the generated evaluation value information; and

setting a constant area having a constant pixel value around each of the target pixel and the reference pixel corresponding to the extracted candidate for the motion vector and allocating the motion vector from the first frame to the second frame for each pixel on the basis of the result of comparison between values calculated for the constant areas.

10. A program installed in an information processing apparatus, the program comprising the steps of:

generating evaluation value information indicating the evaluation value of a motion vector on the basis of pixel-value correlation information indicating the correlation between a target pixel in a first frame among multiple frames for moving image data and a reference pixel in a search area in a second frame, the evaluation value information indicating the result of evaluation of the possibility that the reference pixel is a candidate for a motion from the target point;

extracting a candidate for the motion vector corresponding to each pixel in the frames of the moving image data on the basis of the generated evaluation value information; and

setting a constant area having a constant pixel value around each of the target pixel and the reference pixel corresponding to the extracted candidate for the motion vector and allocating the motion vector from the first frame to the second frame for each pixel on the basis of the result of comparison between values calculated for the constant areas.