IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM

• US 20120189167A1
• Filed: 01/12/2012
• Published: 07/26/2012
• Est. Priority Date: 01/21/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kurata, Tohru, Kimura, Atsushi, Hayashi, Naoki, Iwase, Akira
• Granted Patent: US 8,818,046 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/238 using non-full search, e.g. three-step search
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

An image processing device, includes: a first motion vector calculation unit calculating a motion vector of a target block regarding the target block made with a plurality of pixels set up in a target image by detecting a block position maximally correlated with the target block in a search range set up in a reference image; a second motion vector calculation unit calculating a motion vector between the target image and the reference image from the motion vector for each of a plurality of target blocks set up in the target image; and a predictive motion vector setup unit setting up a predictive motion vector using a motion vector calculated between images in further past than the target image. The first motion vector calculation unit sets up the search range on the reference image based on a position of the target block and the predictive motion vector.

Claims

1. An image processing device comprising:

a first motion vector calculation unit configured to calculate a motion vector of a target block regarding the target block made with a plurality of pixels set up in a target image by detecting a block position maximally correlated with the target block in a search range set up in a reference image;

a second motion vector calculation unit configured to calculate a motion vector between the target image and the reference image from the motion vector for each of a plurality of target blocks set up in the target image; and

a predictive motion vector setup unit configured to set up a predictive motion vector using a motion vector calculated between images in further past than the target image

wherein the first motion vector calculation unit configured to set up the search range on the reference image based at least in part on a position of the target block and the predictive motion vector.

2. The image processing device according to claim 1, wherein the first motion vector calculation unit configured to set up the search range, by moving the block position on the reference image equivalent to the position of the target block on the target image based at least in part on the predictive motion vector, on the basis of a block position after movement.

3. The image processing device according to claim 1, further comprising:

a function value setup unit configured to setup of a function value;

wherein

the predictive motion vector setup unit configured to set up the predictive motion vector by adding the function value as an offset component to an inter-image motion vector.

4. The image processing device according to claim 3, wherein the image is an image generated by moving an image pickup direction, and the function value is a function value of a function approximating a motion to move the image pickup direction.

5. The image processing device according to claim 4, wherein the function value is a constant value.

6. The image processing device according to claim 4, wherein the function value is a function value of a function linearly changing relative to time.

7. The image processing device according to claim 4, wherein the function value is a function value of a function parabolically changing relative to time.

8. The image processing device according to claim 3, further comprising:

an input unit configured to generate an operational signal in accordance with a user operation;

wherein

the predictive motion vector setup unit is configured to set up a sign of an initial value of the function value based at least in part on the operational signal.

9. The image processing device according to claim 1, wherein the predictive motion vector setup unit is configured to set up the predictive motion vector using at least one inter-image motion vectors calculated in the past.

10. The image processing device according to claim 9, wherein the predictive motion vector setup unit is configured to carry out linear prediction using the at least one inter-image motion vectors calculated in the past to set up the predictive motion vector using a result of the linear prediction.

11. The image processing device according to claim 1, further comprising:

a motion detection sensor configured to detect a motion of an image pickup device generating the target image and the reference image;

wherein the predictive motion vector setup unit is configured to carry out a setup of the predictive motion vector by using an output of the motion detection sensor.

12. The image processing device according to claim 11, wherein the motion detection sensor is a sensor detecting at least one of an angular velocity, acceleration, and/or magnetism.

13. The image processing device according to claim 1, further comprising:

an image synthesis processing unit configured to generate a wide angle image, by setting up a region of a partial image to be a part of the wide angle image based at least in part on the motion vector calculated in the second motion vector calculation unit, by linking the partial image.

14. An image processing method comprising:

calculating a motion vector of a target block regarding the target block made with a plurality of pixels set up in a target image by detecting a block position maximally correlated with the target block in a search range set up in a reference image;

calculating a motion vector between the target image and the reference image from the motion vector for each of a plurality of target blocks set up in the target image; and

carrying out a setup of a predictive motion vector using a motion vector calculated between images in further past than the target image

wherein, the search range is set up on the reference image based at least in part on a position of the target block and the predictive motion vector.

15. A removable recording medium storing a program that, when executed by a processor, performs a method comprising:

calculating a motion vector of a target block regarding the target block made with a plurality of pixels set up in a target image by detecting a block position maximally correlated with the target block in a search range set up in a reference image;

calculating a motion vector between the target image and the reference image from the motion vector for each of a plurality of target blocks set up in the target image;

carrying out a setup of a predictive motion vector using a motion vector calculated between images in further past than the target image; and

setting up the search range on the reference image based at least in part on a position of the target block and the predictive motion vector.