Image processing apparatus and image processing method

• US 8,199,805 B2
• Filed: 10/01/2008
• Issued: 06/12/2012
• Est. Priority Date: 10/22/2007
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Numata, Satoshi, Kurata, Tohru, Wada, Kimitaka
• Primary Examiner(s): Orgad, Edan
• Assistant Examiner(s): Baum, Ronald
• Application Number:
  US12/243,567
• Publication Number:
  US 20090103621A1
• Time in Patent Office: 1,350 Days
• Field of Search: 375/240
• US Class Current: 375/240
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 19/102 characterised by the element, parameter or selection affected or controlled by the adaptive coding
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/521 for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

Described herein is an image processing apparatus for calculating a motion vector between two screen images including a target screen image and a reference screen image, including: a base face motion vector calculation section; a high-accuracy base face motion vector calculation section; a reduction face motion vector calculation section; a high-accuracy reduction face motion vector calculation section; first and second base face search range determination sections; and a selection section configured to select a calculation section to be used from among the base face motion vector calculation section, high-accuracy base face motion vector calculation section, reduction face motion vector calculation section and high-accuracy reduction face motion vector calculation section and select whether the first or second base face search range determination section should be used and then select, where use of any of the determination sections is determined, a determination section to be used from between the determination sections.

Claims

1. An image processing apparatus for calculating a motion vector between two screen images including a target screen image and a reference screen image, comprising:

base face motion vector calculation means for detecting a maximum correlation base face reference block having a high correlation with a target block of a predetermined size which is set at a predetermined position on the target screen image from among a plurality of reference blocks which individually have a size same as that of the target block and which are set in a first search range set on the reference screen image and calculating a base face motion vector based on a position displacement amount of the detected maximum correlation base face reference block on the reference screen image with respect to the target block;

high-accuracy base face motion vector calculation means for carrying out an interpolation process using a correlation value of the maximum correlation base face reference block and correlation values of the plural reference blocks positioned at neighboring positions with the maximum correlation base face reference block to detect a position of a high-accuracy maximum correlation base face reference block and calculating a high-accuracy base face motion vector based on a position displacement amount of the high-accuracy maximum correlation base face reference block on the reference screen image with respect to the target block;

reduction face motion vector calculation means for reducing the target screen image and the reference screen image by a predetermined reduction ratio to produce a reduction target screen image and a reduction reference screen image, detecting a maximum correlation reduction face reference block having a high correlation with a reduction face target block which has a predetermined size and is set at a predetermined position on the reduction target screen image from among a plurality of reduction face reference blocks which individually have a size same as that of the reduction face target block and which are set in a second search range set on the reduction reference screen image and calculating a reduction face motion vector based on a position displacement amount of the detected maximum correlation reduction face reference block on the reduction reference screen image with respect to the reduction face target block;

high-accuracy reduction face motion vector calculation means for carrying out an interpolation process using the correlation value of the maximum correlation reduction face reference block and the correlation values of the plural reduction face reference blocks positioned at the neighboring positions with the maximum correlation reduction face reference block to detect a position of a high-accuracy maximum correlation reduction face reference block and calculating a high-accuracy reduction face motion vector based on a position displacement amount of the high-accuracy maximum correlation reduction face reference block on the reduction reference screen image with respect to the reduction face target block;

first base face search range determination means for determining the first search range based on the reduction face motion vector calculated by said reduction face motion vector calculation means;

second base face search range determination means for determining the second search range based on the high-accuracy reduction face motion vector calculated by said high-accuracy reduction face motion vector calculation means; and

selection means for selecting a calculation means to be used from among said base face motion vector calculation means, high-accuracy base face motion vector calculation means, reduction face motion vector calculation means and high-accuracy reduction face motion vector calculation means and selecting whether or not said first or second base face search range determination means should be used and then selecting, where use of any of the determination means is determined, a determination means to be used from between said first and second base face search range determination means.

2. The image processing apparatus according to claim 1, wherein said selection means carries out the means after a selection operation input from the user is received.

3. The image processing apparatus according to claim 1, wherein said selection means automatically carries out the selection at least based on a processing speed and a detection accuracy of the motion vector.

4. The image processing apparatus according to claim 1, wherein said selection means automatically carries out the selection based on whether an object image whose motion vector is to be detected is a static picture image or a dynamic picture image.

5. An image processing method for calculating a motion vector between two screen images including a target screen image and a reference screen image, the image processing method being executed by an image processing apparatus including a base face motion vector calculation means, a high-accuracy base face motion vector calculation means, a reduction face motion vector calculation means, a high-accuracy reduction face motion vector calculation means, a first base face search range determination means and a second base face search range determination means, the image processing method comprising:

a base face motion vector calculation step, carried out by the base face motion vector calculation means, of setting a plurality of reference blocks which individually have a size same as that of a target block which has a predetermined size and is set at a predetermined position on the target screen image in a first search range set on the reference screen image to detect a maximum correlation base face reference block having a high correlation with the target block from among the plural reference blocks and calculating a base face motion vector based on a position displacement amount of the detected maximum correlation base face reference block on the reference screen image with respect to the target block;

a high-accuracy base face motion vector calculation step, carried out by the high-accuracy base face motion vector calculation means, of carrying out an interpolation process using a correlation value of the maximum correlation base face reference block and correlation values of the plural reference blocks positioned at neighboring positions with the maximum correlation base face reference block to detect a position of a high-accuracy maximum correlation base face reference block and calculating a high-accuracy base face motion vector based on a position displacement amount of the high-accuracy maximum correlation base face reference block on the reference screen image with respect to the target block;

a reduction face motion vector calculation step, carried out by the reduction face motion vector calculation means, of reducing the target screen image and the reference screen image by a predetermined reduction ratio to produce a reduction target screen image and a reduction reference screen image, setting a plurality of reduction face reference blocks which individually have a size same as that of a reduction face target block which has a predetermined size and is set at a predetermined position on the reduction target screen image in a second search range set on the reduction reference screen image, detecting a maximum correlation reduction face reference block having a high correlation with the reduction face target block from among the plural reduction face reference blocks and calculating a reduction face motion vector based on a position displacement amount of the detected maximum correlation reduction face reference block on the reduction reference screen image with respect to the reduction face target block;

a high-accuracy reduction face motion vector calculation step, carried out by the high-accuracy reduction face motion vector calculation means, of carrying out an interpolation process using a correlation value of the maximum correlation reduction face reference block and correlation values of the plural reduction face reference blocks positioned at the neighboring positions with the maximum correlation reduction face reference block to detect a position of a high-accuracy maximum correlation reduction face reference block to calculate a high-accuracy reduction face motion vector based on a position displacement amount of the high-accuracy maximum correlation reduction face reference block on the reduction reference screen image with respect to the reduction face target block;

a first base face search range determination step, carried out by the first base face search range determination means, of determining the first search range based on the reduction face motion vector calculated by the reduction face motion vector calculation means;

a second base face search range determination step, carried out by the second base face search range determination means, of determining the second search range based on the high-accuracy reduction face motion vector calculated by the high-accuracy reduction face motion vector calculation means; and

a selection step of selecting, when the motion vector between the two screen images is to be calculated, a calculation step to be used from among the reduction face motion vector calculation step, high-accuracy reduction face motion vector calculation step, base face motion vector calculation step and high-accuracy base face motion vector calculation step, selecting whether or not the first or second base face search range determination step should be used and then selecting, where use of any of the determination steps is determined, a determination means to be used from between the first and second base face search range determination steps.

6. The image processing method according to claim 5, wherein, at the selection step, the selection is carried out in response to a selection operation input received from the user.

7. The image processing method according to claim 5, wherein, at the selection step, the selection is automatically carried out at least based on a processing speed and a detection accuracy of the motion vector.

8. The image processing method according to claim 5, wherein, at the selection step, the selection is automatically carried out based on whether a target image whose motion vector is to be detected is a still picture image or a dynamic picture image.

9. An image processing apparatus for calculating a motion vector between two screen images including a target screen image and a reference screen image, comprising:

a base face motion vector calculation section configured to detect a maximum correlation base face reference block having a high correlation with a target block of a predetermined size which is set at a predetermined position on the target screen image from among a plurality of reference blocks which individually have a size same as that of the target block and which are set in a first search range set on the reference screen image and calculate a base face motion vector based on a position displacement amount of the detected maximum correlation base face reference block on the reference screen image with respect to the target block;

a high-accuracy base face motion vector calculation section configured to carry out an interpolation process using a correlation value of the maximum correlation base face reference block and correlation values of the plural reference blocks positioned at neighboring positions with the maximum correlation base face reference block to detect a position of a high-accuracy maximum correlation base face reference block and calculate a high-accuracy base face motion vector based on a position displacement amount of the high-accuracy maximum correlation base face reference block on the reference screen image with respect to the target block;

a reduction face motion vector calculation section configured to reduce the target screen image and the reference screen image by a predetermined reduction ratio to produce a reduction target screen image and a reduction reference screen image, detect a maximum correlation reduction face reference block having a high correlation with a reduction face target block which has a predetermined size and is set at a predetermined position on the reduction target screen image from among a plurality of reduction face reference blocks which individually have a size same as that of the reduction face target block and which are set in a second search range set on the reduction reference screen image and calculate a reduction face motion vector based on a position displacement amount of the detected maximum correlation reduction face reference block on the reduction reference screen image with respect to the reduction face target block;

a high-accuracy reduction face motion vector calculation section configured to carry out an interpolation process using the correlation value of the maximum correlation reduction face reference block and the correlation values of the plural reduction face reference blocks positioned at the neighboring positions with the maximum correlation reduction face reference block to detect a position of a high-accuracy maximum correlation reduction face reference block and calculate a high-accuracy reduction face motion vector based on a position displacement amount of the high-accuracy maximum correlation reduction face reference block on the reduction reference screen image with respect to the reduction face target block;

a first base face search range determination section configured to determine the first search range based on the reduction face motion vector calculated by said reduction face motion vector calculation section;

a second base face search range determination section configured to determine the second search range based on the high-accuracy reduction face motion vector calculated by said high-accuracy reduction face motion vector calculation section; and

a selection section configured to select a calculation section to be used from among said base face motion vector calculation section, high-accuracy base face motion vector calculation section, reduction face motion vector calculation section and high-accuracy reduction face motion vector calculation section and select whether or not said first or second base face search range determination section should be used and then select, where use of any of the determination sections is determined, a determination section to be used from between said first and second base face search range determination sections.