Image processing method and image processing apparatus

• US 20080240588A1
• Filed: 03/29/2007
• Published: 10/02/2008
• Est. Priority Date: 03/29/2007
• Status: Abandoned Application

• Current Assignee: Monolith Company Limited
• Original Assignee: Monolith Company Limited
• Inventors: Borovikov, Igor, Tsoupko-Sitnikov, Mikhail
• Application Number:
  US11/730,018
• Publication Number:
  US 20080240588A1
• Time in Patent Office: Days
• US Class Current: 382/236
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches
• CPC Class Codes: H04N 19/521 for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

An image processing technology is provided that lends itself to improving precision of image matching. Keyframe to keyframe matching point information is generated by combining image frame to image frame corresponding point information obtained by computing matching in a group of image frames which includes a first keyframe and a second keyframe as a source and a destination, respectively. Image matching between the first and second keyframes is directly computed by using, of the entire keyframe to keyframe corresponding point information, the corresponding point information evaluated to be highly reliable as a constraint condition.

Claims

1. An image processing method comprising:

concatenating, whereby keyframe to keyframe corresponding point information is generated by combining corresponding point information indicating correspondence between image frames obtained by subjecting an image frame group which includes a first keyframe and a second keyframe as a source and a destination, respectively, to a matching process; and

refining, whereby direct image matching is computed between the first keyframe and the second keyframe by using, of the entire keyframe to keyframe corresponding point information, the corresponding point information evaluated to be highly reliable as a constraint condition.

2. The image processing method according to claim 1, wherein

the refining comprises:

characteristic point detecting, whereby a characteristic point in an image of the first keyframe is detected;

constraint condition defining, whereby a point in the second keyframe corresponding to the characteristic point in the first keyframe thus detected is obtained from the keyframe to keyframe corresponding point information, and a pair comprising the characteristic point and the corresponding point is defined as a constraint condition; and

keyframe to keyframe matching, whereby image matching is directly computed between the first keyframe and the second keyframe under the constraint condition.

3. The image processing method according to claim 2, wherein

in the characteristic point detecting, a point included in an object determined to move between the first and second keyframes by referring to the keyframe to keyframe corresponding point information is detected as a characteristic point in the first keyframe.

4. The image processing method according to claim 2, wherein

in the characteristic point detecting, a characteristic point is detected in an area other than the periphery of the image of the first keyframe.

5. The image processing method according to claim 2, wherein

in the keyframe to keyframe matching, corresponding point information indicating correspondence between the first and second keyframes is obtained by applying a multiresolutional critical point filter to the first and second keyframes under the constraint condition.

6. The image processing method according to claim 1, further comprising:

inspecting, whereby, when it is determined, with reference to a preset standard for inspection, that the result of computation in the refining approximates the keyframe to keyframe corresponding point information generated in the concatenating, the corresponding point information indicating correspondence between the first and second keyframes is updated according to the result of computation.

7. An image processing method comprising:

preparatory matching, whereby corresponding point information indicating correspondence between a first keyframe and a second keyframe is preparatorily generated by computing image matching between the first and second keyframes; and

primary matching, whereby, subsequently the corresponding point information indicating correspondence between the keyframes is updated by re-computing image matching between the first and second keyframes under a constraint condition defined based on the corresponding point information indicating correspondence between the first and second keyframes.

8. The image processing method according to claim 7, wherein

an algorithm for performing preparatory matching and an algorithm for performing primary matching includes a common image matching algorithm.

9. An image processing method comprising:

preparatory matching, whereby corresponding point information indicating correspondence between a first keyframe and a second keyframe is preparatorily generated by computing image matching between the first and second keyframes; and

primary matching, whereby image matching between the first and second keyframes is computed with the same level of resolution as used in preparatory matching, under a constraint defined based on corresponding point information indicating correspondence between the first and second keyframes.

10. The image processing method according to claim 9, wherein

an algorithm for performing preparatory matching and an algorithm for performing primary matching includes a common image matching algorithm.

11. An image processing apparatus comprising:

a concatenation processor operative to generate keyframe to keyframe corresponding point information by combining corresponding point information obtained by computing matching between adjacent two image frames in a group of image frames which includes a first keyframe and a second keyframe as a source and a destination, respectively; and

a refinement processor operative to directly compute image matching between the first and second keyframes by using, of the entire keyframe to keyframe corresponding point information, the corresponding point information evaluated to be highly reliable as a constraint condition.

12. The image processing apparatus according to claim 11, wherein the refinement processor comprises:

a characteristic point detecting unit operative to detect a characteristic point in an image of the first keyframe;

a constraint condition defining unit operative to obtain a point in the second keyframe corresponding to the characteristic point in the first keyframe thus detected from the keyframe to keyframe corresponding point information, and define a pair comprising the characteristic point and the corresponding point as a constraint condition; and

a keyframe to keyframe matching processor operative to directly compute matching between the first keyframe and the second keyframe matching under the constraint condition.

13. The image processing apparatus according to claim 12, wherein

the characteristic point detecting unit detects a point included in an object determined to move between the first and second keyframes by referring to the keyframe to keyframe corresponding point information as a characteristic point in the first keyframe.

14. The image processing apparatus according to claim 12, wherein

the characteristic point detecting unit detects a characteristic point in an area other than the periphery of the image of the first keyframe.

15. The image processing apparatus according to claim 11, further comprising:

an inspection processor operative to use, when it is determined, with reference to a preset standard for inspection, that the result of computation in the refining approximates the keyframe to keyframe corresponding point information generated in the concatenating, the result of computation as the corresponding point information indicating correspondence between the first and second keyframes.

16. An image processing apparatus comprising:

a preparatory matching processor operative to preparatorily generate corresponding point information indicating correspondence between a first keyframe and a second keyframe by computing image matching between the first and second keyframes;

a primary matching processor operative to subsequently update the corresponding point information indicating correspondence between the keyframes by re-computing image matching between the first and second keyframes under a constraint condition defined based on the corresponding point information indicating correspondence between the first and second keyframes.

17. An image processing apparatus comprising:

a preparatory matching processor operative to preparatorily generate corresponding point information indicating correspondence between a first keyframe and a second keyframe by computing image matching between the first and second keyframes;

a primary matching processor operative to compute image matching between the first and second keyframes with the same level of resolution as used in preparatory matching, under a constraint defined based on corresponding point information indicating correspondence between the first and second keyframes.

18. An image processing method which processes n image frames including a first through nth image frames, the method comprising:

identifying correspondence between the first image frame and the nth image frame by tracking a fast moving object from the first image frame to the nth image frame; and

directly identifying correspondence between the first image frame and the nth image frame with respect to a slowly moving object.