Image processing method, image processing apparatus, and storage medium

• US 6,674,905 B1
• Filed: 01/21/2000
• Issued: 01/06/2004
• Est. Priority Date: 01/22/1999
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Matsugu, Masakazu, Takahashi, Fumiaki, Kondo, Toshiaki
• Primary Examiner(s): Werner, Brian
• Assistant Examiner(s): Lu, Tom Y.
• Application Number:
  US09/488,792
• Time in Patent Office: 1,446 Days
• Field of Search: 382/181, 382/199, 382/162
• US Class Current: 382/199
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/20096 Interactive definition of curve of interest
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling

Abstract

When a dispersion value of a shading value is equal to or more than a threshold value in the neighborhood of a standard point, it is determined that an image pattern has a texture dominant characteristic. When the value is less than the threshold value, a change of the shading value is determined to be moderate (S11). When the dispersion value is equal to or more than the threshold value (S11), a texture characteristic amount is loaded (S12), a characteristic amount suitable for area growth (e.g., texture energy) is calculated (S13), and a distance D_T concerning the texture characteristic amount is obtained (S14). When the dispersion value is less than the threshold value (S11), a color component characteristic amount is loaded (S16), a characteristic amount suitable for the area growth (e.g., a characteristic vector obtained from a local histogram concerning a color component or a list of representative color components) is extracted (S17), and a distance D_C concerning the color component is obtained (S18). The area is grown based on the corresponding characteristic amount (S15, S19), and a renewal position is determined based on a new borderline of a core area obtained by unifying area growth results from standard points (S20).

Claims

1. An image extracting method comprising the steps of:

setting a core by designating at least one of a predetermined point, line segment and area in an image;

calculating an evaluation function including a texture characteristic amount in a local area, the local area including at least one standard point corresponding to the core; and

determining a borderline of a predetermined area in the image by renewing a shape of the core based on a value of the evaluation function,

wherein said step of calculating an evaluation function comprises

determining an attention point using a predetermined method;

determining a first distance between the attention point and the at least one standard point in a characteristic space based on a color component characteristic amount;

determining a second distance between the attention point and the at least one standard point in a characteristic space based on a texture characteristic amount; and

weighting and adding the first distance and the second distance, wherein weighting factors are determined based on a dispersion value concerning a shading value in a neighborhood of the at least one standard point, and

wherein said step of determining a borderline comprises

obtaining a similarity between a neighborhood area of the at least one standard point and a neighborhood area of the attention point, based on a function value obtained by calculating the evaluation function;

renewing the shape of the core by combining (i) the attention point or a neighborhood area of the attention point and (ii) a predetermined extraction area that includes the at least one standard point, when the similarity is larger than a predetermined threshold value; and

renewing the shape of the core by removing the attention point and the neighborhood area from the extraction area when the similarity is equal to or less than the threshold value and the attention point is included in the extraction area.

2. The image extracting method according to claim 1, wherein the evaluation function is represented as a function of a predetermined occurrence probability value concerning the texture characteristic amount in the vicinity of the standard point, and the core shape is renewed by moving each standard point by a predetermined amount in a direction in which the function value of the evaluation function is maximized.

3. The image extracting method according to claim 1 wherein when a value of an edge ratio of the point on the borderline of the core reaches a predetermined standard value or more, a renewal processing of the extraction area is automatically stopped.

4. The image extracting method according to claim 1, wherein when an absolute value of a change rate of an edge ratio of the point on the borderline of the core reaches a predetermined standard value or less, a renewal processing of the extraction area is automatically stopped.

5. The image extracting method according to claim 1, wherein the threshold value is renewed based on a signal from predetermined instruction selection means.

6. The image extracting method according to claim 1, wherein the evaluation function is a predetermined energy function including a term for giving a minimum value in a border of the texture characteristic amount, and the core shape is renewed by moving a position of each standard point in a direction in which the function value of the evaluation function is minimized.

7. The image extracting method according to claim 1, wherein the first distance is represented by a weighted sum of differences between, for each color component, a value of the at least one standard point and a value of the attention point, or the absolute values of the differences, and the second distance is represented by the weighted sum of differences between, for each texture characteristic component, a value of the at least one standard point and a value of the attention point or the absolute values of the differences.

8. The image extracting method according to claim 1, wherein the first distance is represented by a weighted sum of differences between, for each color component, a value of the at least one standard point and a value of the attention point or the absolute value of the differences, and the second distance is represented by a predetermined function value of the differences, for an energy value represented by a weighted sum of texture characteristic components, between a value of the at least one standard point and a value of the attention point or the absolute value of the differences.

9. An image extracting method comprising:

a core setting step of setting a core by designating at least one of a predetermined point, line segment and area in an image;

a standard point setting step of setting a representative point on the core as a standard point;

a similarity calculating step of obtaining a similarity of a neighborhood area of the standard point and a neighborhood area of an attention point, said similarity calculating step comprising the steps of

determining an attention point using a predetermined method;

determining a first distance between the attention point and the at least one standard point in a characteristic space based on a color component characteristic amount;

determining a second distance between the attention point and the at least one standard point in a characteristic space based on a texture characteristic amount; and

weighting and adding the first distance and the second distance, wherein weighting factors are determined based on a dispersion value concerning a shading value in the neighborhood of the standard point;

an extraction area renewing step of combining the attention point or a neighborhood area with a predetermined extraction area including the standard point, when the similarity is larger than a predetermined threshold value, or removing the attention point or the neighborhood area from the extraction area when the similarity is equal to or less than the threshold value and when the attention point is included in the extraction area, to renew a borderline of the extraction area; and

a borderline display step of displaying the borderline of the extraction area after renewal.

10. The image extracting method according to claim 9, wherein the extraction area renewing step comprises a threshold value renewing step of increasing or decreasing the threshold value for each renewal of the borderline in a predetermined method.

11. The image extracting method according to claim 10, wherein the threshold value renewing step comprises an edge ratio calculating step of calculating a ratio of a point on the predetermined borderline on the extraction area being an edge point, and a renewing step of renewing the threshold value based on one of a value of the edge ratio and a change rate.

12. The image extracting method according to claim 9, wherein the threshold value is renewed based on a signal from predetermined instruction selection means.

13. The image extracting method according to claim 9, wherein the first distance is represented by a weighted sum of differences between, for each color component, a value of the at least one standard point and a value of the attention point, or the absolute values of the differences, and the second distance is represented by the weighted sum of differences between, for each texture characteristic component, a value of the at least one standard point and a value of the attention point or the absolute values of the differences.

14. The image extracting method according to claim 9, wherein the first distance is represented by a weighted sum of differences between, for each color component, a value of the at least one standard point and a value of the attention point or the absolute value of the differences, and the second distance is represented by a predetermined function value of the differences, for an energy value represented by a weighted sum of texture characteristic components, between a value of the at least one standard point and a value of the attention point or the absolute value of the differences.

15. An image extracting method comprising:

an initial outline setting step of setting an initial outline present inside or outside an object to be extracted;

a standard point setting step of setting a representative point of a plurality of predetermined representative points on the initial outline as a standard point;

a similarity calculating step of obtaining a similarity of a neighborhood area of the standard point and a neighborhood area of an attention point,

wherein said similarity calculating step comprises the steps of

determining an attention point using a predetermined method;

determining a first distance between the attention point and the standard point based in a characteristic space based on a color component characteristic amount;

determining a second distance between the attention point and the standard point in a characteristic space based on a texture characteristic amount; and

weighting and adding the first distance and the second distance, wherein weighting factors are determined based on a dispersion value concerning a shading value in a neighborhood of the standard point;

an extraction area renewing step of combining the attention point or a neighborhood area with a predetermined extraction area including the standard point, when the similarity is larger than a predetermined threshold value, or removing the attention point or the neighborhood area from the extraction area when the similarity is equal to or less than the threshold value and when the attention point is included in the extraction area, to renew a borderline of the extraction area a predetermined number of times; and

a threshold value setting step of determining the threshold value based on an edge ratio on an inner borderline of the renewed extraction area when the entire initial outline is present outside the object, or determining the threshold value based on the edge ratio on an outer borderline of the renewed extraction area when the entire initial outline is present inside the object.

16. The image extracting method according to claim 15, wherein the first distance is represented by a weighted sum of differences between, for each color component, a value of the at least one standard point and a value of the attention point, or the absolute values of the differences, and the second distance is represented by the weighted sum of differences between, for each texture characteristic component, a value of the at least one standard point and a value of the attention point or the absolute values of the differences.

17. The image extracting method according to claim 15, wherein the first distance is represented by a weighted sum of differences between, for each color component, a value of the at least one standard point and a value of the attention point or the absolute value of the differences, and the second distance is represented by a predetermined function value of the differences, for an energy value represented by a weighted sum of texture characteristic components, between a value of the at least one standard point and a value of the attention point or the absolute value of the differences.

18. An image extraction apparatus comprising:

core setting means for setting a core by designating at least one of a predetermined point, line segment and area in an image;

calculating means for calculating an evaluation function including a texture characteristic amount in a local area, the local area including at least one standard point corresponding to the core; and

determining means for determining a borderline of a predetermined area in the image by renewing a shape of the core based on a value of the evaluation function,

wherein said calculating means comprises

means for determining an attention point using a predetermined method;

means for determining a first distance between the attention point and the at least one standard point in a characteristic space based on a color component characteristic amount;

means for determining a second distance between the attention point and the at least one standard point in a characteristic space based on a texture characteristic amount; and

means for weighting and adding the first distance and the second distance, wherein weighting factors are determined based on a dispersion value concerning a shading value in a neighborhood of the at least one standard point, and

wherein said determining means for determining a borderline comprises

means for obtaining a similarity between a neighborhood area of the at least one standard point and a neighborhood area of the attention point, based on a function value obtained by calculating the evaluation function; and

means for (a) renewing the shape of the core by combining (i) the attention point or a neighborhood area of the attention point and (ii) a predetermined extraction area that includes the at least one standard point, when the similarity is larger than a predetermined threshold value, and for (b) renewing the shape of the core by removing the attention point and the neighborhood area from the extraction area when the similarity is equal to or less than the threshold value and the attention point is included in the extraction area.

19. An image extraction apparatus comprising:

core setting means for setting a core by designating at least one of a predetermined point, line segment and area in an image;

standard point setting means for setting a representative point on the core as a standard point;

similarity calculating means for obtaining a similarity of a neighborhood area of the standard point and a neighborhood area of an attention point, said similarity calculating means comprising

means for determining an attention point using a predetermined method;

means for determining a first distance between the attention point and the at least one standard point in a characteristic space based on a color component characteristic amount;

means for determining a second distance between the attention point and the at least one standard point in a characteristic space based on a texture characteristic amount; and

means for weighting and adding the first distance and the second distance, wherein weighting factors are determined based on a dispersion value concerning a shading value in the neighborhood of the standard point;

extraction area renewing means for combining the attention point or a neighborhood area with a predetermined extraction area including the standard point, when the similarity is larger than a predetermined threshold value, or removing the attention point or the neighborhood area from the extraction area when the similarity is equal to or less than the threshold value and when the attention point is included in the extraction area, to renew a borderline of the extraction area; and

borderline display means or displaying the borderline of the extraction area after renewal.

20. An image extraction apparatus comprising;

initial outline setting means for setting an initial outline present inside or outside an object to be extracted;

standard point setting means for setting a representative point of a plurality of predetermined representative points on the initial outline as a standard point;

similarity calculating means for obtaining a similarity of a neighborhood area of the standard point and a neighborhood area of an attention point,

wherein said similarity calculating means comprises

means for determining an attention point using a predetermined method;

means for determining a first distance between the attention point and the standard point based in a characteristic space based on a color component characteristic amount;

means for determining a second distance between the attention point and the standard point in a characteristic space based on a texture characteristic amount; and

means for weighting and adding the first distance and the second distance, wherein weighting factors are determined based on a dispersion value concerning a shading value in a neighborhood of the standard point;

extraction area renewing means for combining the attention point or a neighborhood area with a predetermined extraction area including the standard point, when the similarity is larger than a predetermined threshold value, or removing the attention point or the neighborhood area from the extraction area when the similarity is equal to or less than the threshold value and when the attention point is included in the extraction area, to renew a borderline of the extraction area a predetermined number of times; and

threshold value setting means for determining the threshold value based on an edge ratio on an inner borderline of the renewed extraction area when the entire initial outline is present outside the object, or determining the threshold value based on the edge ratio on an outer borderline of the renewed extraction area when the entire initial outline is present inside the object.

21. A storage medium for storing a program, said program comprising the steps of:

setting a core by designating at least one of a predetermined point, line segment and area in an image;

calculating an evaluation function including a texture characteristic amount in a local area, the local area including at least one standard point corresponding to the core, and

renewing a shape of the core based on a value of the evaluation function to determine a borderline of a predetermined area in the image,

wherein said step of calculating an evaluation function comprises

determining an attention point using a predetermined method;

determining a first distance between the attention point and the at least one standard point in a characteristic space based on a color component characteristic amount;

determining a second distance between the attention point and the at least one standard point in a characteristic space based on a texture characteristic amount; and

weighting and adding the first distance and the second distance, wherein weighting factors are determined based on a dispersion value concerning a shading value in a neighborhood of the at least one standard point, and

wherein said step of renewing a shape of the core comprises

obtaining a similarity between a neighborhood area of the at least one standard point and a neighborhood area of the attention point, based on a function value obtained by calculating the evaluation function,

renewing the shape of the core by combining (i) the attention point or a neighborhood area of the attention point and (ii) a predetermined extraction area that includes the at least one standard point, when the similarity is larger than a predetermined threshold value; and

renewing the shape of the core by removing the attention point and the neighborhood area from the extraction area when the similarity is equal to or less than the threshold value and the attention point is included in the extraction area.

22. A storage medium for storing a program, said program comprising the steps of:

setting a core by designating at least one of a predetermined point, line segment and area in an image;

setting a representative point on the core as a standard point;

obtaining a similarity of a neighborhood area of the standard point and a neighborhood area of an attention point, said similarity calculating step comprising the steps of

determining an attention point using a predetermined method;

determining a first distance between the attention point and the at least one standard point in a characteristic space based on a color component characteristic amount;

determining a second distance between the attention point and the at least one standard point in a characteristic space based on a texture characteristic amount; and

weighting and adding the first distance and the second distance, wherein weighting factors are determined based on a dispersion value concerning a shading value in the neighborhood of the standard point;

combining the attention point or a neighborhood area with a predetermined extraction area including the standard point, when the similarity is larger than a predetermined threshold value, or removing the attention point or the neighborhood area from the extraction area when the similarity is equal to or less than the threshold value and when the attention point is included in the extraction area, to renew a borderline of the extraction area; and

displaying the borderline of the extraction area after renewal.

23. A storage medium for storing a program, said program comprising the steps of:

setting an initial outline present inside or outside an object to be extracted;

setting a representative point of a plurality of predetermined representative points on the initial outline as a standard point;

obtaining a similarity of a neighborhood area of the standard point and a neighborhood area of an attention point,

wherein said step of obtaining a similarity comprises the steps of

determining an attention point using a predetermined method;

determining a first distance between the attention point and the standard point based in a characteristic space based on a color component characteristic amount;

determining a second distance between the attention point and the standard point in a characteristic space based on a texture characteristic amount; and

weighting and adding the first distance and the second distance, wherein weighting factors are determined based on a dispersion value concerning a shading value in a neighborhood of the standard point;

combining the attention point or a neighborhood area with a predetermined extraction area including the standard point, when the similarity is larger than a predetermined threshold value, or removing the attention point or the neighborhood area from the extraction area when the similarity is equal to or less than the threshold value and when the attention point is included in the extraction area, to renew a borderline of the extraction area a predetermined number of times; and

determining the threshold value based on an edge ratio on an inner borderline of the renewed extraction area when the entire initial outline is present outside the object, or determining the threshold value based on the edge ratio on an outer borderline of the renewed extraction area when the entire initial outline is present inside the object.