IMAGE PROCESSING APPARATUS, METHOD, AND PROGRAM

• US 20090079876A1
• Filed: 09/18/2008
• Published: 03/26/2009
• Est. Priority Date: 09/26/2007
• Status: Abandoned Application

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Ida, Takashi, Kaneko, Toshimitsu, TAKESHIMA, Hidenori
• Application Number:
  US12/233,030
• Publication Number:
  US 20090079876A1
• Time in Patent Office: Days
• US Class Current: 348/699
• CPC Class Codes: G06T 3/4069 by subpixel displacements
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/205 for correcting amplitude versus frequency characteristic
• CPC Class Codes: H04N 7/0125 one of the standards being a high definition standard
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

An image processing apparatus includes a first receiving unit configured to receive a plurality of frames of an image containing pixel values, a first setting unit configured to set one of the frames as a reference frame, a second setting unit configured to set, of the frames received by the first receiving unit, one frame other than the reference frame as an other frame, a first storage unit configured to store at least one subpixel shift value that is a preset fractional value, and an estimation unit configured to estimate a fractional part of a position of a corresponding point corresponding to a pixel of the other frame to readily select a value closer to the subpixel shift value.

Claims

1. An image processing apparatus comprising:

a first receiving unit configured to receive a plurality of frames of an image containing pixel values;

a first setting unit configured to set one of the frames as a reference frame;

a second setting unit configured to set, of the frames received by the first receiving unit, one frame other than the reference frame as an other frame;

a first storage unit configured to store at least one subpixel shift value that is a preset real value; and

an estimation unit configured to estimate a fractional part of a position of a corresponding point corresponding to a pixel of the other frame to readily select a value closer to the subpixel shift value.

4. The apparatus according to claim 1, wherein the estimation unit comprises:

a generation unit configured to generate a temporary reference frame by shifting the reference frame by the subpixel shift value using pixel value interpolation;

a third setting unit configured to set a position of interest in the other frame;

a fourth setting unit configured to set a block of interest based on the position of interest;

a first calculation unit configured to calculate a first position in the temporary reference frame, the first position having a pixel value pattern similar to a pixel value pattern in the block of interest; and

an acquisition unit configured to acquire, as the corresponding point, a second position in the reference frame corresponding to the position of interest by correcting the first position by the subpixel shift value.

6. The apparatus according to claim 1, further comprising:

a second receiving unit configured to receive an output resolution;

a second storage unit configured to store a point spread function which calculates a first pixel value of a resolution of the reference frame from a second pixel value corresponding to the output resolution; and

a second calculation unit configured to calculate a third pixel value of the reference frame at the output resolution using a fourth pixel value at the corresponding point as a corresponding pixel value of the other frame.

8. The apparatus according to claim 6, wherein the estimation unit comprises:

a generation unit configured to generate a temporary reference frame by shifting the reference frame by the subpixel shift value using pixel value interpolation;

a third setting unit configured to set a position of interest in the other frame;

a fourth setting unit configured to set a block of interest based on the position of interest;

a first calculation unit configured to calculate a first position in the temporary reference frame, the first position having a pixel value pattern similar to a pixel value pattern in the block of interest; and

an acquisition unit configured to acquire, as the corresponding point, a second position in the reference frame corresponding to the position of interest by correcting the first position by the subpixel shift value.

9. The apparatus according to claim 8, wherein the first calculation unit comprises:

a fifth setting unit configured to sequentially set, in the reference frame, a plurality of candidate blocks each of which is a block having the same size as the block of interest;

a third calculation unit configured to calculate, as a sum of error values, a first block error by obtaining, for each pixel in the block of interest, an error value which is a value obtained by inputting a difference between a pixel value in the block of interest and a pixel value in the candidate block corresponding to the pixel value into an error function for obtaining an error between pixel values;

a selection unit configured to select, from the candidate blocks, a candidate block having a minimum block error as an optimum candidate block;

a fourth calculation unit configured to calculate a second block error for at least one neighboring candidate block set near the optimum candidate block;

a fifth calculation unit configured to calculate a coefficient of the error function based on the first block error and the second block error; and

a sixth calculation unit configured to calculate, as the position in the reference frame, a position where the error function including the calculated coefficient is minimum.

10. The apparatus according to claim 1, wherein the estimation unit comprises:

a sixth setting unit configured to set an energy function including a term which increases a first energy value as a magnitude of a difference between a pixel value of a pixel of interest in the other frame and a pixel value at a position to which a pixel of interest in the reference frame is shifted by a motion vector becomes large, and a term which increases a second energy value as a magnitude of a difference between a fractional part of the motion vector and the subpixel shift value becomes large; and

a seventh calculation unit configured to calculate, as the corresponding point, a motion vector which minimizes the energy function.

2. An image processing apparatus comprising:

a first receiving unit configured to receive a plurality of frames of an image containing pixel values;

a first setting unit configured to set one of the frames as a reference frame;

a second setting unit configured to set the reference frame as an other frame;

a first storage unit configured to store at least one subpixel shift value that is a preset real value; and

an estimation unit configured to estimate a fractional part of a position of a corresponding point corresponding to a pixel of the other frame to readily select a value closer to the subpixel shift value.

3. The apparatus according to claim 2, wherein the second setting unit is configured to set a plurality of frame included in the reference frame as other frames.

5. The apparatus according to claim 2, wherein the estimation unit comprises:

a generation unit configured to generate a temporary reference frame by shifting the reference frame by the subpixel shift value using pixel value interpolation;

a third setting unit configured to set a position of interest in the other frame;

a fourth setting unit configured to set a block of interest based on the position of interest;

a first calculation unit configured to calculate a first position in the temporary reference frame, the first position having a pixel value pattern similar to a pixel value pattern in the block of interest; and

an acquisition unit configured to acquire, as the corresponding point, a second position in the reference frame corresponding to the position of interest by correcting the first position by the subpixel shift value.

7. The apparatus according to claim 2, further comprising:

a second receiving unit configured to receive an output resolution;

a second storage unit configured to store a point spread function which calculates a first pixel value of a resolution of the reference frame from a second pixel value corresponding to the output resolution; and

a second calculation unit configured to calculate a third pixel value of the reference frame at the output resolution using a fourth pixel value at the corresponding point as a corresponding pixel value of the other frame.

11. The apparatus according to claim 2, wherein the estimation unit comprises:

a sixth setting unit configured to set an energy function including a term which increases a first energy value as a magnitude of a difference between a pixel value of a pixel of interest in the other frame and a pixel value at a position to which a pixel of interest in the reference frame is shifted by a motion vector becomes large, and a term which increases a second energy value as a magnitude of a difference between a fractional part of the motion vector and the subpixel shift value becomes large; and

a seventh calculation unit configured to calculate, as the corresponding point, a motion vector which minimizes the energy function.

12. An image processing method comprising:

receiving a plurality of frames of an image containing pixel values;

setting one of the frames as a reference frame;

setting, of the received frames, one frame other than the reference frame as an other frame.

storing, in a first storage unit, at least one subpixel shift value that is a preset fractional value; and

estimating a fractional part of a position of a corresponding point corresponding to a pixel of the other frame to readily select a value closer to the subpixel shift value.