Color image processing method and apparatus for same, which automatically detects a contour of an object in an image

• US 5,475,507 A
• Filed: 10/12/1993
• Issued: 12/12/1995
• Est. Priority Date: 10/14/1992
• Status: Expired due to Term

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Suzuki, Shoji, Tokuyo, Masanaga
• Primary Examiner(s): BRINICH, STEPHEN M
• Application Number:
  US08/134,826
• Time in Patent Office: 791 Days
• Field of Search: 358/500, 358/517, 358/532, 382/22, 382/10, 382/9, 382/17, 382/48, 382/54, 364/448, 364/521, 395/128, 395/131
• US Class Current: 358/500
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation

Abstract

A point at a position near the contour of a desired object in an image being processed is designated, and a boundary point on the contour of the object is detected on the basis of the designation. From the detected boundary point, other points on the object-contour are subsequently detected to detect the object-contour. When the object-contour has been detected, either the inside or the outside of the object-contour is uniformly filled with prescribed image data, so as to extract the object from the image. Extracted image data resulting from the extraction is output. The extracted image data may include mask data indicating the color mixing ratio at a boundary portion between the object and the background thereof so that color adjustment or image composition can be performed using the mask data.

Claims

1. A color image processing method for detecting a contour of an object in an image, comprising the steps of:

a contour position designation step wherein a position in a color image near the contour of a desired object contained in said image is designated by a user;

a boundary point detection step wherein a boundary point on said contour of said object is automatically detected on the basis of designation in said contour position designation step;

an object-contour tracking step wherein said contour of said object is detected by subsequently automatically detecting other points on said contour from said boundary point which has been automatically detected in said boundary point detection step; and

a termination step wherein contour detection in said object-contour tracking step is checked, and said contour detection is terminated when said contour detection satisfies a prescribed condition.

2. A color image processing method according to claim 1, wherein said contour position designation step comprises a first sub-step wherein said color image is displayed on a display device, and a second sub-step wherein at least one position in said displayed image is designated.

3. A color image processing method according to claim 1, wherein said contour position designation step comprises a first sub-step wherein said color image is displayed on a display device, and a second sub-step wherein a boundary region in said displayed image is designated.

4. A color image processing method according to claim 1, wherein said boundary point detection step comprises:

a first sub-step wherein the coordinates of the position designated in said contour position designation step are detected;

a second sub-step wherein image data of a plurality of pixels near said designated position with said detected coordinates are detected;

a third sub-step wherein differences of said image data from image data at said designated position are calculated for a plurality of directions from said designated position;

a fourth sub-step wherein the maximum of said differences calculated for said plurality of directions is stored;

a fifth sub-step wherein a determination is made as to whether or not such difference calculations have been completed at all the pixels within a previously set detection range including said designated position;

a sixth sub-step wherein, when said difference calculations have not been completed at all said pixels, the coordinates of another pixel within said detection range are detected for a further difference calculation at said another pixel; and

a seventh sub-step wherein, when said difference calculations have been completed at all said pixels, the coordinates of a particular one of said pixels within said detection range that has the maximum difference are detected as boundary coordinates.

5. A color image processing method according to claim 1, wherein said boundary point detection step comprises:

a first sub-step wherein the coordinates of the position designated in said contour position designation step are detected;

a second sub-step wherein image data of a plurality of pixels near said designated position with said detected coordinates are detected;

a third sub-step wherein differences of said image data from image data at said designated position are calculated for a plurality of directions from said designated position;

a fourth sub-step wherein a particular one of said directions having the maximum of said calculated differences is detected;

a fifth sub-step wherein said image data of said plurality of pixels near said designated position are expressed by an additive color mixing process, and the color mixing ratios of image data of certain plurality of pixels lying in said particular direction having the maximum difference are calculated and stored;

a sixth sub-step wherein a determination is made as to whether or not such difference calculations have been completed at all the pixels within a previously set detection range including said designated position;

a seventh sub-step wherein, when said difference calculations have not been completed at all said pixels, the coordinates of another pixel within said detection range are detected for a further difference calculation at said another pixel; and

an eighth sub-step wherein, when said difference calculations have been completed at all said pixels, the coordinates of a particular one of said pixels within said detection range that has a color mixing ratio closest to a previously set value are detected as boundary coordinates.

6. A color image processing method according to claim 4, wherein said image data used in said difference calculations comprise means each obtained as a mean of image data of a plurality of pixels.

7. A color image processing method according to claim 5, wherein said image data used in said color mixing ratio calculations comprise means each obtained as a mean of image data of a plurality of pixels.

8. A color image processing method according to claim 1, wherein said object-contour tracking step comprises:

a first sub-step wherein the boundary point which has been detected in said boundary point detection step is used as a starting point, and wherein a plurality of peripheral pixels within a first detection range are set around said starting point;

a second sub-step wherein said set peripheral pixels are tracked to detect a subsequent boundary point, and such tracking of peripheral pixels and such detection of a subsequent boundary point are repeatedly effected to track contour portions;

a third sub-step wherein the results of tracking are checked to determine whether or not said starting point has been again reached, and said tracking is terminated when said starting point has been again reached; and

a fourth sub-step wherein a determination is made as to whether or not said tracking has been effected either a set number of times or for a set period of time, and said tracking is forcibly terminated even when said starting point has not been again reached by tracking effected either said set number of times or for said set period of time.

9. A color image processing method according to claim 1, wherein said object-contour tracking step comprises:

a first sub-step wherein one of a plurality of boundary points detected in accordance with designation of a plurality of points is used as a starting point to detect coordinates between said starting point and another one of said boundary points;

a second sub-step wherein boundary points are subsequently detected on the basis of such detected coordinates to track contour portions; and

a third sub-step wherein the results of tracking are checked to determine whether or not said starting point has been again reached, and said tracking is terminated when said starting point has been again reached.

10. A color image processing method according to claim 1, wherein said object-contour tracking step is adapted to detect boundary points by tracking a designated boundary region.

11. A color image processing method according to claim 1, wherein said object-contour tracking step comprises:

a first sub-step wherein a first point in said object in said image and a second point in a background in said image are designated;

a second sub-step wherein image data on an object color and image data on a background color are detected on the basis of designation of said first and second points, respectively;

a third sub-step wherein a mean of said image data on said object color and said image data on said background color are calculated, said mean comprising image data concerning a plurality of color components;

a fourth sub-step wherein said mean is used as a threshold for dividing the entire image into two regions; and

a fifth step wherein a boundary region between said two regions is detected.

12. A color image processing method according to claim 1, wherein said object-contour tracking step comprises:

a first sub-step wherein a first point in said object in said image and a second point on a background in said image are designated;

a second sub-step wherein image data on an object color and image data on a background color are detected on the basis of designation of said first and second points, respectively;

a third sub-step wherein color mixing ratios are calculated by expressing the entire image data by an additive process as color mixture of said object color and said background color;

a fourth sub-step wherein a threshold comprising a certain color mixing ratio is used to divide the entire image into two regions; and

a fifth step wherein a boundary region between said two regions is detected.

13. A color image processing method according to claim 1, further comprising a smoothing step wherein the contour of said object which has been detected in said object-contour tracking step is smoothed.

14. A color image processing method according to claim 1, wherein said object-contour tracking step further comprises a sub-step of deleting at least a portion of the detected contour of said object, and again detecting the deleted portion of the contour.

15. A color image processing method according to claim 1, further comprising:

an extraction step wherein said object in said image is extracted from said image on the basis of the results of detection of the contour of said object; and

an outputting step wherein extracted image data obtained in said extraction step is output,

said method being adapted to extract said object from said image by effecting the above steps.

16. A color image processing method according to claim 15, wherein said extraction step comprises:

a first sub-step wherein a position inside said contour of said object is designated;

a second sub-step wherein the coordinates of points on said contour are detected on the basis of designation inside said contour; and

a third sub-step wherein the coordinates of points inside said contour are detected on the basis of detection of said coordinates of said points on said contour.

17. A color image processing method according to claim 15, wherein said extraction step comprises:

a first sub-step wherein coordinates outside said contour of said object are detected by detecting said contour from end portions of said image; and

a second sub-step wherein the coordinates of points inside said contour are detected on the basis of detection of said coordinates outside said contour.

18. A color image processing method according to claim 15, wherein said extraction step further comprises:

a fourth sub-step wherein the contour of an inner object inside said contour of said object is detected;

a fifth sub-step wherein a position inside said detected contour of said inner object is designated; and

a sixth sub-step wherein the inside of said contour of said inner object is deleted on the basis of designation of said position inside the contour of said inner object.

19. A color image processing method according to claim 15, wherein said outputting step comprises:

a first sub-step wherein the color mixing ratio at which an object color and a background color are mixed with each other at an object-contour portion of said object extracted in said extraction step is calculated; and

a second sub-step wherein mask data indicating the results of calculation of the color mixing ratio, object image data indicating an object color and a boundary between said object and the background thereof, and background image data indicating said object color and said boundary, are output.

20. A color image processing method according to claim 1 and 15, wherein:

said boundary point detection step comprises a candidate detection sub-step wherein at least one candidate for a point on said contour of said object is detected at each of a plurality of positions on the periphery of said object; and

said object-contour tracking step comprises;

a contour portions detection sub-step wherein contour portions constituting at least one continuous line are detected on the basis of the at least one candidate detected in said candidate detection sub-step; and

a contour area detection sub-step wherein a contour area having a width varying between various positions of said contour area is detected on the basis of said at least one line detected in said contour portions detection sub-step.

21. A color image processing method according to claim 20, wherein, said candidate detection sub-step is adapted such that a pixel which is at a position on the periphery of said object and which has a greater degree of change in color than a threshold set for a prescribed range, is detected as a candidate.

22. A color image processing method according to claim 20, wherein said contour area detection sub-step is adapted such that pixels which are within a detected region and which are not included in the detected candidates are set as contour pixels.

23. A color image processing method according to claim 1, further comprising:

a color adjustment step wherein the color of either said object or the background thereof which has been extracted from said image in the extraction step is changed, and wherein a color to be applied to a boundary portion between said object and said background after the color-changing is calculated on the basis of the color mixing ratio between the object color and the background color which has been possessed by said image before said color-changing.

24. A color image processing method according to claim 1, further comprising:

an image composition step wherein either said object or the background thereof which has been extracted from said image in the extraction step is composed with either a background or an object extracted from another image, and a color to be applied to a boundary portion between the composed object and the composed background after the image composition is calculated on the basis of the color mixing ratio between an object color and a background color which has been possessed by said image before said image composition.

25. A color image processing method according to claim 23, wherein each of said color adjustment step and said image composition step is adapted to effect color adjustment and image composition, respectively, on the basis of mask data indicating the color mixing ratio at a boundary between a region having said object color and another region having said background color, object image data indicating said object color and said boundary, and background image data indicating said background color and said boundary, all of which data have been output in said outputting step.

26. A color image processing apparatus comprising:

storage means for storing input color image data;

display means for displaying said input color image data;

contour position designation means for receiving input data from a user designating a position near the contour of a desired object in said image;

boundary point detection means for automatically detecting a boundary point on said contour of said object on the basis of designation through said contour position designation means;

object-contour tracking means for subsequently automatically detecting other points on said contour from said boundary point which has been automatically detected by said boundary point detection means so as to detect said contour of said object; and

termination means for checking contour detection by said object-contour tracking means, and for terminating said contour detection when said contour detection satisfies a prescribed condition.

27. A color image processing apparatus according to claim 26, further comprising:

extraction means for extracting said object in said image on the basis of object-contour detection by said object-contour tracking means; and

outputting means for outputting extracted image data obtained by said extraction means.

28. A color image processing apparatus according to claim 27, wherein said outputting means is adapted to output at least one of object extraction data including an object-contour, background extraction data including said object-contour, and color mixing ratio data indicating the color mixing ratio at a boundary region between said object and the background thereof.

29. A color image processing apparatus according to claim 26, further comprising:

color adjustment means for changing the color of either said object or the background thereof which has been extracted from said image by said extraction means, and for calculating a color to be applied after the color-changing to a boundary portion between said object and the background thereof on the basis of the color mixing ratio between the object color and the background color which has been possessed by said image before the color-changing.

30. A color image processing apparatus according to claim 26, further comprising:

image composition means for composing either said object or the background thereof which has been extracted from said image by said extraction means with either a background or an object extracted from another image, and for calculating a color to be applied after the image composition to a boundary between the composed object and the composed background on the basis of the color mixing ratio between an object color and a background color which has been possessed by said image before the image composition.