Image processing apparatus and method, program, and recording medium

• US 7,409,075 B2
• Filed: 01/29/2004
• Issued: 08/05/2008
• Est. Priority Date: 02/14/2003
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Watanabe, Yoshinori
• Primary Examiner(s): Johns; Andrew W.
• Assistant Examiner(s): Bitar; Nancy
• Application Number:
  US10/765,859
• Publication Number:
  US 20040252861A1
• Time in Patent Office: 1,650 Days
• Field of Search: 382/103, 382/181, 382/199, 382/154, 382/107, 707 6- 10, 375/240.16, 356/27, 734/88, 348/416, 345/154
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99939 Privileged access

Abstract

In each of frames of specified image data, a covered background area or an uncovered background area is detected. In an image in the first frame, a user inputs the contour of an object in the covered background area. In an image in the last frame, the user inputs the contour of the object in the uncovered background area. On the basis of the contour of the object, which is input in the image in the first frame, the frames are played in the forward direction to extract the object. At the same time, on the basis of the contour of the object, which is input in the image in the last frame, the frames are played in the backward direction to extract the object.

Claims

1. An image processing apparatus for extracting an object in an image, comprising;

image obtaining means for obtaining image data of a specified image;

motion analyzing means for analyzing motion of the object included in the image based on the obtained image data, including frame detecting means for detecting a first frame containing a first frame covered background area that corresponds to a background area that is gradually covered by the object due to frame-to-frame apparent movement of the object and for detecting a last frame containing a last frame uncovered background area that corresponds to a background area that gradually changes from being covered by the object to being uncovered by the object due to the apparent frame-to-frame movement of the object;

means for prompting a user to input a first frame contour of a first subset of the contour of the object to be extracted within the first frame covered background area and to input a last frame contour of a second subset of the contour of the object to be extracted within the last frame uncovered background area, respectively, wherein the first subset differs from the second subset and neither the first subset nor the second subset includes the entire contour of the object; and

object extracting means for extracting the object from a plurality of frames from the first frame to the last frame based on the respective first and last frame contours.

2. An image processing apparatus according to claim 1, wherein the motion analyzing means includes:

motion computing means for computing the motion of the object in the image relative to a background; and

area determining means for determining the first frame covered background area and the last frame uncovered background area based on the motion computed by the motion computing means.

3. An image processing apparatus according to claim 2 further comprising image presenting means for presenting an image in a frame in which the object to be extracted from the image is specified, wherein the image presenting means displays the first frame covered background area and the last frame uncovered background area.

4. An image processing apparatus according to claim 2, wherein the motion computing means includes distance computing means for setting a plurality of feature points in the image and computing distances between adjacent feature points.

5. An image processing apparatus according to claim 4, wherein the area determining means includes:

comparison means for comparing a distance between adjacent feature points in a temporally prior frame with a distance between adjacent feature points in a temporally subsequent frame, the distances being computed by the motion computing means; and

setting means for setting, based on a comparison result obtained by the comparison means, in the background of the image, the first frame covered background area and the last frame uncovered background area.

6. An image processing apparatus according to claim 1, further comprising object displaying means for displaying the object extracted by the object extracting means.

7. An image processing method for extracting an object in an image, comprising steps of;

obtaining image data of a specified image;

analyzing motion of an object included in the image based on the obtained image data, including detecting a first frame containing a first frame covered background area that corresponds to a background area that is gradually covered by the object due to frame-to-frame apparent movement of the object and detecting a last frame containing a last frame uncovered background area that corresponds to a background area that gradually changes from being covered by the object to being uncovered by the object due to the apparent frame-to-frame movement of the object;

prompting a user to input a first frame contour of a first subset of the contour of the object to be extracted within the first frame covered background area and to input a last frame contour of a second subset of the contour of the object to be extracted within the last frame uncovered background area, respectively, wherein the first subset differs from the second subset and neither the first subset nor the second subset includes the entire contour of the object; and

extracting the object from a plurality of frames from the first frame to the last frame based on the respective first and last frame contours.

9. The method according to claim 7, wherein the analyzing step includes:

computing the motion of the object in the image relative to a background; and

determining the first frame covered background area and the last frame uncovered background area based on the computed motion.

10. The method according to claim 9 further comprising presenting an image in a frame in which the object to be extracted from the image is specified, wherein the image presenting step includes displaying the first frame covered background area and the last frame uncovered background area.

11. The method according to claim 9, wherein the motion computing step includes setting a plurality of feature points in the image and computing distances between adjacent feature points.

12. The method according to claim 11, wherein the area determining step includes:

comparing a distance between adjacent feature points in a temporally prior frame with a distance between adjacent feature points in a temporally subsequent frame; and

setting in the background of the image the first frame covered background area as-an and the last frame uncovered background area.

8. A computer readable storage medium having a computer-executable program recorded thereon configured to execute a method comprising:

obtaining image data of a specified image;

analyzing motion of an object included in the image based on the obtained image data, including detecting a first frame containing a first frame covered background area that corresponds to a background area that is gradually covered by the object due to frame-to-frame apparent movement of the object and detecting a last frame containing a last frame uncovered background area that corresponds to a background area that gradually changes from being covered by the object to being uncovered by the object due to the apparent frame-to-frame movement of the object;

prompting a user to input a first frame contour of a first subset of the contour of the object to be extracted within the first frame covered background area and to input a last frame contour of a second subset of the contour of the object to be extracted within the last frame uncovered background area, respectively, wherein the first subset differs from the second subset and neither the first subset nor the second subset includes the entire contour of the object; and

extracting the object from a plurality of frames from the first frame to the last frame based on the respective first and last frame contours.