IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND PROGRAM

• US 20100118963A1
• Filed: 05/21/2008
• Published: 05/13/2010
• Est. Priority Date: 06/18/2007
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tanaka, Junichi, Yagasaki, Yoichi, Nakagami, Ohji
• Granted Patent: US 8,804,832 B2
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: H04N 1/217 Interfaces allowing access to a single user
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/3247 Data linking a set of images to one another, e.g. sequence, burst or continuous capture mode
• CPC Class Codes: H04N 2201/3267 of motion picture signals, e.g. video clip
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 25/42 by switching between different modes of operation using different resolutions or aspect ratios, e.g. switching between interlaced and non-interlaced mode
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 7/0125 one of the standards being a high definition standard
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

There is provided an image processing apparatus including an acquiring unit for acquiring moving image data containing a plurality of successive frames, and one or a plurality of image data corresponding to the frames and having a spatial resolution higher than the frames; a motion prediction unit for detecting a motion vector between the frames using the moving image data; and an image generation unit for generating motion compensated image data corresponding to a predetermined frame based on the image data and the motion vector. The image generation unit generates the motion compensated image data being positioned between the predetermined frame and a frame corresponding to the image data and corresponding to the frame, and generates motion compensated image data corresponding to the predetermined frame based on the motion compensated image data and the motion vector.

Claims

1. An image processing apparatus, comprising:

a receiving unit for receiving first video data containing a plurality of frames and second video data having a resolution higher than the frames contained in the first video data;

a motion prediction unit for detecting a motion vector between the frames contained in the first video data received by the receiving unit; and

an image generation unit for, based on the second video data received by the receiving unit and the motion vector detected by the motion prediction unit, performing motion compensation on a frame positioned between a predetermined frame contained in the first video data received by the receiving unit and a frame contained in the second video data, and generating motion compensated image data corresponding to the predetermined frame based on the motion compensated frame and the motion vector detected by the motion prediction unit.

2. The image processing apparatus according to claim 1, wherein the image generation unit executes motion compensation in order from a frame corresponding to a time close to the second video data, and sequentially generates motion compensated image data corresponding to the predetermined frame based on the motion compensated frame and the motion vector.

3. The image processing apparatus according to claim 1, further comprising:

a storage unit for storing the first video data and the second video data,

wherein

the receiving unit receives the first video data and the second video data from the storage unit.

4. The image processing apparatus according to claim 1, wherein

the first video data has lower spatial resolution than the second video data, and

the second video data has higher spatial resolution than the first video data.

6. The image processing apparatus according to claim 3, wherein the storage unit stores image data obtained by down sampling an imaged image signal as the first video data.

8. The image processing apparatus according to claim 1, wherein if a difference amount between a predetermined frame contained in the first video data and a frame contained in the second video data is smaller than a predetermined value, the image generation unit outputs the image data corresponding to the predetermined frame without performing motion compensation on the second video data.

11. The image processing apparatus according to claim 1, wherein

the first video data has higher temporal resolution than the second video data, and

the second video data has lower temporal resolution than the first video data.

12. The image processing apparatus according to claim 3, wherein the storage unit includes,

a first storage unit for storing the first video data, and

a second storage unit for storing the second video data.

13. The image processing apparatus according to claim 3, further comprising a recording unit for recording the first video data and the second data in the storage unit.

14. The image processing apparatus according to claim 6, further comprising an imaging unit for imaging an image of a subject and generating an image signal.

5. (canceled)

7. (canceled)

9. An image processing method, comprising the steps of:

receiving first video data containing a plurality of frames and second video data having a resolution higher than the frames contained in the first video data;

detecting a motion vector between the frames contained in the first video data received in the receiving step; and

performing, based on the second video data received in the receiving step and the motion vector detected in the motion prediction step, motion compensation on a frame positioned between a predetermined frame contained in the first video data received in the receiving step and a frame contained in the second video data, and generating motion compensated image data corresponding to the predetermined frame based on the motion compensated frame and the motion vector detected in the motion prediction step.

10. A program for causing a computer to execute the functions of each unit of an image processing apparatus comprising:

a receiving unit for receiving first video data containing a plurality of frames and second video data having a resolution higher than the frames contained in the first video data;

a motion prediction unit for detecting a motion vector between the frames contained in the first video data received by the receiving unit; and

an image generation unit for, based on the second video data received by the receiving unit and the motion vector detected by the motion prediction unit, performing motion compensation on a frame positioned between a predetermined frame contained in the first video data received by the receiving unit and a frame contained in the second video data, and generating motion compensated image data corresponding to the predetermined frame based on the motion compensated frame and the motion vector detected by the motion prediction unit.