Image processing device, image processing method, and program

• US 8,818,046 B2
• Filed: 01/12/2012
• Issued: 08/26/2014
• Est. Priority Date: 01/21/2011
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kurata, Tohru, Hayashi, Naoki, Iwase, Akira, Kimura, Atsushi
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): Azima, Shaghayegh
• Application Number:
  US13/348,749
• Publication Number:
  US 20120189167A1
• Time in Patent Office: 957 Days
• Field of Search: 382/107, 375/240.16, 375/240
• US Class Current: 382/107
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/238 using non-full search, e.g. three-step search
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

An image processing device, includes: a first motion vector calculation unit calculating a motion vector of a target block regarding the target block made with a plurality of pixels set up in a target image by detecting a block position maximally correlated with the target block in a search range set up in a reference image; a second motion vector calculation unit calculating a motion vector between the target image and the reference image from the motion vector for each of a plurality of target blocks set up in the target image; and a predictive motion vector setup unit setting up a predictive motion vector using a motion vector calculated between images in further past than the target image. The first motion vector calculation unit sets up the search range on the reference image based on a position of the target block and the predictive motion vector.

Claims

1. An image processing device comprising:

a microcomputer operable to;

calculate a plurality of motion vectors for each of a plurality of target blocks set up in a current target image, wherein the plurality of motion vectors are calculated by detecting a plurality of block positions corresponding to each of the plurality of target blocks in a search range set up in a current reference image, wherein the plurality of block positions are maximally correlated with each of the plurality of target blocks;

calculate an inter-image motion vector between the current target image and the current reference image based on the plurality of motion vectors for each of the plurality of target blocks set up in the current target image; and

set up a predictive motion vector for the current target image based on a plurality of inter-image motion vectors calculated between a plurality of target images and a plurality of reference images in further past than the current target image,

wherein the search range is set up on the current reference image based on a position of a target block from the plurality of target blocks and the predictive motion vector.

2. The image processing device according to claim 1, wherein the microcomputer is operable to:

move a first block position from the plurality of block positions on the current reference image based on the predictive motion vector, wherein the first block position is equivalent to the position of the target block on the current target image; and

set up the search range based on a second block position after movement.

3. The image processing device according to claim 1, wherein the microcomputer is further operable to:

setup a function value, wherein

the predictive motion vector is set up by adding the function value as an offset component to the inter-image motion vector.

4. The image processing device according to claim 3, wherein a picked up image is an image generated by moving the image processing device in an image pickup direction, and the function value corresponds to a function approximating a motion to move in the image pickup direction.

5. The image processing device according to claim 3, wherein the function value is a constant value.

6. The image processing device according to claim 3, wherein the function value corresponds to a function linearly changing relative to time.

7. The image processing device according to claim 3, wherein the function value corresponds to a function parabolically changing relative to time.

8. The image processing device according to claim 3, wherein the microcomputer is further operable to:

generate an operational signal in accordance with a user operation, wherein

a sign of an initial value of the function value is set based on the operational signal.

9. The image processing device according to claim 1, wherein the microcomputer is operable to set up the predictive motion vector using the inter-image motion vector calculated in the past.

10. The image processing device according to claim 9, wherein the microcomputer is operable to carry out linear prediction using the inter-image motion vector calculated in the past to set up the predictive motion vector using a result of the linear prediction.

11. The image processing device according to claim 1, further comprising:

a motion detection sensor configured to detect a motion of the image processing device generating the current target image and the current reference image,

wherein the microcomputer is configured to set up the predictive motion vector based on the detected motion of the image processing device.

12. The image processing device according to claim 11, wherein the motion detection sensor is configured to detect one or more of:

an angular velocity, acceleration, and magnetism.

13. The image processing device according to claim 1, wherein the microcomputer is further operable to:

set up partial images based on the calculated inter-image motion vector; and

generate a wide angle image by linking the partial images.

14. An image processing method comprising:

calculating a plurality of motion vectors for each of a plurality target blocks set up in a current target image, wherein the plurality of motion vectors are calculated by detecting a plurality of block positions corresponding to each of the plurality of target blocks in a search range set up in a current reference image, wherein the plurality of block positions are maximally correlated with each of the plurality of target blocks;

calculating an inter-image motion vector between the current target image and the current reference image based on the plurality of motion vectors for each of the plurality of target blocks set up in the current target image; and

setting up a predictive motion vector for the current target image based on a plurality of inter-image motion vectors calculated between a plurality of target images and a plurality of reference images in further past than the current target image,

wherein the search range is set up on the current reference image based on a position of a target block from the plurality of target blocks and the predictive motion vector.

15. A non-transitory removable recording medium storing a program that, when executed by a processor, performs the steps comprising:

calculating a plurality of motion vectors for each of a plurality target blocks set up in a current target image, wherein the plurality of motion vectors are calculated by detecting a plurality of block positions corresponding to each of the plurality of target blocks in a search range set up in a current reference image, wherein the plurality of block positions are maximally correlated with each of the plurality of target blocks;

calculating an inter-image motion vector between the current target image and the current reference image based on the plurality of motion vectors for each of the plurality of target blocks set up in the current target image; and

setting up a predictive motion vector for the current target image based on a plurality of inter-image motion vectors calculated between a plurality of target images and a plurality of reference images in further past than the current target image; and

setting up the search range on the current reference image based on a position of a target block from the plurality of target blocks and the predictive motion vector.