Image Processing Apparatus, Image Processing Method, and Program

• US 20100183072A1
• Filed: 06/10/2008
• Published: 07/22/2010
• Est. Priority Date: 06/18/2007
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tanaka, Junichi, Yagasaki, Yoichi, Nakagami, Ohji
• Granted Patent: US 8,885,716 B2
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/11 among a plurality of spatial predictive coding modes
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/91 Television signal processing therefor
• CPC Class Codes: H04N 7/0132 the field or frame frequency of the incoming video signal being multiplied by a positive integer, e.g. for flicker reduction
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51
• CPC Class Codes: H04N 9/8045 using predictive coding

Abstract

There is provided an image processing apparatus including a an acquiring unit for acquiring moving image data containing a plurality of successive frames, and one or a plurality of image data corresponding to the frames and having a spatial resolution higher than the frames; a motion prediction unit for detecting a motion vector between the frames using the moving image data; a difference amount calculation unit for calculating a difference amount between a predetermined frame and the frame corresponding to the image data; and an image generation unit capable of generating motion compensated image data corresponding to the predetermined frame based on the frame corresponding to the image data and the motion vector.

Claims

1. An image processing apparatus, comprising:

a receiving unit for receiving first video data containing a plurality of frames and second video data having a resolution higher than the frames;

a motion prediction unit for detecting a motion vector between the frames contained in the first video data received by the receiving unit;

a difference amount calculation unit for calculating a difference amount between a frame of the first video data and a frame of the second video data; and

an image generation unit for, based on the frame of the second video data and the motion vector detected by the motion prediction unit, generating motion compensated image data having a resolution same as the second video data if the difference amount calculated by the difference amount calculation unit is smaller than a predetermined value, and generates a frame of video data in which the frame of the first video data is magnified to a resolution same as the frame of the second video data image data if the difference amount calculated by the difference amount calculation unit is greater than the predetermined value.

2. The image processing apparatus according to claim 1, further comprising:

a storage unit for storing the first video data and the second video data,

wherein

the receiving unit receives the first video data and the second video data from the storage unit.

3. The image processing apparatus according to claim 1, wherein

the first video data has lower spatial resolution than the second video data, and

the second video data has higher spatial resolution than the first video data.

5. The image processing apparatus according to claim 2, wherein the storage unit stores image data obtained by down sampling an imaged image signal as the first video data.

9. The image processing apparatus according to claim 1, wherein the image generation unit changes a ratio of combining the frame of the first video data, which is a reference source of the motion vector, and the frame of the second video data, which is a reference destination of the motion vector, according to the difference amount calculated by the difference amount calculation unit when performing the motion compensation if the difference amount calculated by the difference amount calculation unit is greater than the predetermined value.

10. The image processing apparatus according to claim 1, wherein

the first video data has higher temporal resolution than the second video data, and

the second video data has lower temporal resolution than the first video data.

11. The image processing apparatus according to claim 2, wherein the storage unit includes,

a first storage unit for storing the first video data, and

a second storage unit for storing the second video data.

12. The image processing apparatus according to claim 2, further comprising a recording unit for recording the first video data and the second data in the storage unit.

13. The image processing apparatus according to claim 12, further comprising an imaging unit for imaging an image of a subject and generating an image signal.

4. (canceled)

6. (canceled)

7. An image processing method, comprising the steps of:

receiving first video data containing a plurality of frames and second video data having a resolution higher than the frames;

detecting a motion vector between the frames contained in the first video data received in the receiving step;

calculating a difference amount between a frame of the first video data and a frame of the second video data; and

generating, based on the frame of the second video data and the motion vector detected in the motion prediction step, motion compensated image data having a resolution same as the second video data if the difference amount calculated in the difference amount calculation step is smaller than a predetermined value, and a frame of video data in which the frame of the first video data is magnified to a resolution same as the frame of the second video data image data if the difference amount calculated in the difference amount calculation step is greater than the predetermined value.

8. A program for causing a computer to execute the functions of:

receiving first video data containing a plurality of frames and second video data having a resolution higher than the frames;

detecting a motion vector between the frames contained in the first video data received by the receiving function;

calculating a difference amount between a frame of the first video data and a frame of the second video data; and

generating, based on the frame of the second video data and the motion vector detected in the motion prediction function, motion compensated image data having a resolution same as the second video data if the difference amount calculated by the difference amount calculation function is smaller than a predetermined value, and a frame of video data in which the frame of the first video data is magnified to a resolution same as the frame of the second video data image data if the difference amount calculated by the difference amount calculation function is greater than the predetermined value.