Image texture retrieving method and apparatus thereof

• US 6,624,821 B1
• Filed: 02/04/2000
• Issued: 09/23/2003
• Est. Priority Date: 02/05/1999
• Status: Expired due to Term

• Current Assignee: Samsung Electronics Co. Ltd., Regents of the University of California (University of California)
• Original Assignee: Samsung Electronics Co. Ltd., Regents of the University of California (University of California)
• Inventors: Wu, Peng, Manjunath, Bangalore, Choi, Yang-lim, Shin, Hyun-doo
• Primary Examiner(s): Bella, Matthew C.
• Assistant Examiner(s): CUNNINGHAM, GREGORY F
• Application Number:
  US09/497,504
• Time in Patent Office: 1,327 Days
• Field of Search: 345/582, 345/585, 345/586, 345/588, 345/211.FOR, 382/108, 382/260-265, 382/190
• US Class Current: 345/582
• CPC Class Codes: G06F 16/5862 using texture
• CPC Class Codes: G06F 18/23213 with fixed number of clusters, e.g. K-means clustering
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 2207/20068 Projection on vertical or horizontal image axis
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 7/42 using transform domain methods
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 30/1988 Graph matching graphical image representation G06V30/18181

Abstract

A method for retrieving an image texture descriptor for describing texture features of an image, including the steps of (a) filtering input images using predetermined filters having different orientation coefficients, (b) projecting the filtered images onto axes of each predetermined direction to obtain data groups consisting of averages of each directional pixel values, (c) selecting candidate data groups among the data groups by a predetermined classification method, (d) determining a plurality of indicators based on orientation coefficients of the filters used in filtering the candidate data groups, and (e) determining the plurality of indicators as the texture descriptor of the image. The texture descriptors which allow kinds of texture structure present in an image to be perceptually captured can be retrieved.

Claims

1. A method for describing texture features of an image, comprising the steps of:

(a) filtering input images using predetermined filters having different orientation coefficients;

(b) projecting the filtered images onto axes of each predetermined direction to obtain data groups consisting of averages of directional pixel values;

(c) selecting candidate data groups among the data groups by a predetermined classification method;

(d) determining a plurality of indicators based on orientation coefficients of the filters used in filtering the candidate data groups;

(e) determining the plurality of indicators as the texture descriptor of the image; and

(f) determining another indicator based on a presence of graphs filtered by filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected candidate data groups.

2. The describing method according to claim 1, further comprising the step of calculating a mean and a variance of pixels with respect to each of the filtered images, and obtaining a predetermined vector using the calculated mean and variance.

3. A method for describing texture features of an image, comprising the steps of:

(a) filtering input images using predetermined filters having different orientation coefficients and different scale coefficients;

(b) projecting the filtered images onto axes of each predetermined direction to obtain graphs consisting of averages of directional pixel values;

(c) selecting candidate graphs among the graphs obtained in the step (b) by a predetermined classification method;

(d) determining another indicator based on the presence of graphs filtered by filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected candidate graphs;

(e) determining a plurality of indicators based on scale coefficients or orientation coefficients of the filters used in filtering the determined candidate graphs; and

(f) determining the indicator determined in the step (d) and the plurality of indicators determined in the step (e) as the texture descriptor of the image.

4. The describing method according to claim 3, further comprising the step of calculating a mean and a variance of pixels with respect to the filtered images, and obtaining a predetermined vector using a the calculated mean and variance.

5. A method for describing texture features of an image, comprising the steps of:

(a) filtering input images using predetermined filters having different orientation coefficients and different scale coefficients;

(b) projecting the filtered images onto horizontal and vertical axes to obtain horizontal-axis projection graphs and vertical-axis projection graphs;

(c) calculating normalized auto-correlation values for each graph;

(d) obtaining a local maximum and a local minimum for each normalized auto-correlation value, at which the calculated normalized auto-correlation values forms a local peak and a local valley at a predetermined section;

(e) defining an average of the local maximums and an average of the local minimums as contrast;

(f) selecting graphs in which the ratio of a standard deviation to the average of the local maximums is less than or equal to a predetermined threshold as first candidate graphs;

(g) determining a type of second candidate graphs according to a number of graphs filtered by the filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected second candidate graphs;

(h) counting numbers of graphs belonging to respective types of second candidate graphs and determining weights for each type of second candidate graphs;

(i) calculating a sum of products of the counted numbers of graphs and the determined weights to determine a calculation result value as a first indicator constituting a texture descriptor;

(j) determining the orientation coefficients and scale coefficients of the second candidate graphs having a largest contrast as second through fifth indicators; and

(k) setting indicators, including the first indicator and the second through fifth indicators, as the texture descriptors of the corresponding image.

6. The describing method according to claim 5, further comprising the step of calculating a mean and a variance of pixels with respect to the filtered images, and obtaining a predetermined vector using the calculated mean and variance, wherein the step (k) includes the step of determining indicators including the first indicator, the second through fifth indicators and the predetermined vector as the texture descriptors of the corresponding image.

7. The describing method according to claim 5, wherein the normalized auto-correlation, denoted by NAC(k), is calculated by the following formula:

$\mathrm{NAC}<br><br>\left(k\right)=\frac{\underset{m=k}{\overset{N-1}{\sum <br><br>}}<br><br>P<br><br>\left(m-k\right)<br><br>P<br><br>\left(m\right)}{\sqrt{\underset{m=k}{\overset{N-1}{\sum <br><br>}}<br><br>P^2<br><br>\left(m-k\right)<br><br>\underset{m=k}{\overset{N-1}{\sum <br><br>}}<br><br>P^2<br><br>\left(m\right)}}$

wherein N is a predetermined positive integer, an input image consists of N×

N pixels, a pixel position is represented by i, where i is a number from 1 to N, the projection graphs, expressed by pixels of the pixel position i are represented by P(i) and k is a number from 1 to N.

8. The describing method according to claim 5, wherein the contrast is determined as:

$\mathrm{contrast}=\frac{1}{M}<br><br>\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>\mathrm{P\_magn}<br><br>\left(i\right)-\frac{1}{L}<br><br>\underset{i=1}{\overset{L}{\sum <br><br>}}<br><br>\mathrm{V\_magn}<br><br>\left(i\right)$

wherein P_magn (i) and V_magn (i) are the local maximums and local minimums determined in the step (d).

9. The describing method according to claim 5, wherein in the step (f), the graphs satisfying the following formula are selected as first candidate graphs:

$\frac{S}{d}\le <br><br>\mathrm{\alpha <br><br>}$

wherein d and S are the average and standard deviation of the local maximums and a is a predetermined threshold.

10. The describing method according to claim 5, wherein the step (g) comprises the sub-steps of:

(g-1) when there are one or more graphs having scale or orientation coefficients identical with those of a pertinent candidate graph and one or more graphs having scale or orientation coefficients close to those of the pertinent candidate graph, classifying the pertinent candidate graph as a first type graph;

(g-2) when there are one or more graphs having scale or orientation coefficients identical with those of a pertinent candidate graph but there is no graph having scale or orientation coefficients close to those of the pertinent candidate graph, classifying the pertinent candidate graph as a second type graph; and

(g-3) when there is no graph having scale or orientation coefficients identical with or close to those of a pertinent candidate graph, classifying the pertinent candidate graph as a third type graph.

11. The describing method according to claim 5, wherein the step (h) includes the step of counting a number of graphs belonging to each of first through third types of graphs and determining predetermined weights for each of the types of graphs.

12. The describing method according to claim 5, after the step of (f), further comprising the step of applying a predetermined clustering algorithm to the first candidate graphs to select second candidate graphs.

13. The describing method according to claim 12, wherein the predetermined clustering algorithm is a modified agglomerative clustering.

14. The describing method according to claim 5, wherein in the step (j), the orientation coefficient of a graph having the largest contrast, among the horizontal-axis projection graphs, is determined as a second indicator;

the orientation coefficient of a graph having the largest contrast, among the vertical-axis projection graphs, is determined as a second indicator;

the scale coefficient of a graph having the largest contrast, among the horizontal-axis projection graphs, is determined as a fourth indicator; and

the scale coefficient of a graph having the largest contrast, among the vertical-axis projection graphs, is determined as a fifth indicator.

15. The describing method according to claim 5, wherein the step (j) includes the step of determining indicators including the first indicator, the second through fifth indicators and a predetermined vector as the texture descriptors of the corresponding image.

16. The describing method according to claim 5, wherein the predetermined filters include Gabor filters.

17. The describing method according to claim 6, wherein the predetermined filters include Gabor filters.

18. The describing method according to claim 7, wherein the predetermined filters include Gabor filters.

19. The describing method according to claim 8, wherein the predetermined filters include Gabor filters.

20. The describing method according to claim 9, wherein the predetermined filters include Gabor filters.

21. The describing method according to claim 10, wherein the predetermined filters include Gabor filters.

22. The describing method according to claim 11, wherein the predetermined filters include Gabor filters.

23. A computer readable medium having program code executable by a computer to perform a method for describing texture features of an image, the method comprising the steps of:

(a) filtering input images using predetermined filters having different orientation coefficients and different scale coefficients;

(b) projecting the filtered images onto horizontal and vertical axes to obtain horizontal-axis projection graphs and vertical-axis projection graphs;

(c) calculating normalized auto-correlation values for each graph;

(d) obtaining a local maximum and a local minimum for each of the normalized auto-correlation values, at which the calculated normalized auto-correlation value forms a local peak and a local valley at a predetermined section;

(e) defining an average of the local maximums and an average the local minimums as contrast;

(f) selecting graphs in which the ratio of a standard deviation to the average of the local maximums is less than or equal to a predetermined threshold as first candidate graphs;

(g) determining a type of second candidate graphs according to a number of graphs filtered by the filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected second candidate graphs;

(h) counting numbers of graphs belonging to respective types of second candidate graphs and determining weights for each type for second candidate graph;

(i) calculating a sum of products of the counted numbers of graphs and the determined weights to determine a calculation result value as a first indicator constituting a texture descriptor;

(j) determining the orientation coefficients and scale coefficients of the second candidate graphs having a largest contrast as second through fifth indicators; and

(k) setting indicators including the first indicator and the second through fifth indicators as the texture descriptors of the corresponding image.

24. The computer readable medium according to claim 23, wherein the image texture descriptor retrieving method further comprises the step of calculating a mean and a variance of pixels with respect to the filtered images, and obtaining a predetermined vector using the calculated mean and variance, and wherein the step (k) includes the step of determining indicators including the first indicator, the second through fifth indicators and the predetermined vector as the texture descriptors of the corresponding image.

25. An apparatus for describing texture features of an image, comprising:

filtering means for filtering input images using predetermined filters having different orientation coefficients and different scale coefficients;

projecting means for projecting the filtered images onto axes of each predetermined direction to obtain data groups consisting of averages of each directional pixel values;

classifying means for selecting candidate data groups among the data groups by a predetermined classification method;

first indicator determining means for determining an indicator based on a number of graphs filtered by filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected candidate graph; and

second indicator determining means for determining a plurality of indicators based on scale coefficients and orientation coefficients of the filters used in filtering the determined candidate graphs.

26. The describing method according to claim 25, further comprising mean/variance calculating means for calculating a mean and a variance of pixels with respect to each of the filtered images, and obtaining a predetermined vector using the calculated mean and variance.

27. An apparatus for describing texture features of an image, comprising:

a filtering unit for filtering input images using predetermined filters having different orientation coefficients and different scale coefficients;

an image mean/variance calculating unit for calculating a mean and a variance of pixels with respect to each of the filtered images, and obtaining a predetermined vector using the calculated mean and variance;

a projecting unit for projecting the filtered images onto horizontal and vertical axes to obtain horizontal-axis projection graphs and vertical-axis projection graphs;

a calculating unit for calculating a normalized auto-correlation value for each graph;

a peak detecting/analyzing unit for detecting a local maximum and a local minimum for each auto-correlation value, at which the calculated normalized auto-correlation values forms a local peak and a local valley at a predetermined section;

a mean/variance calculating unit for calculating an average of the local maximums and an average of the local minimums;

a first candidate graph selecting/storing unit for selecting the graphs satisfying the requirement that the ratio of a standard deviation to the average of the local maximums be less than or equal to a predetermined threshold, as first candidate graphs;

a second candidate graph selecting/storing unit for applying a predetermined clustering algorithm to the first candidate graphs to select the same as second candidate graphs;

a classifying unit for counting a number of graphs belonging to each of the respective types of the second candidate graphs, outputting data signals indicative of the number of graphs of each type, determining weights for the graphs belonging to the respective types and outputting data signals indicative of weights to be applied to each type;

a first indicator determining unit for calculating a sum of a products of the data representing the number of graphs belonging to each type, and the data representing the weights to be applied to each type, determining and outputting a calculation result as a first indicator constituting a texture descriptor;

a contrast calculating unit for calculating the contrast using the averages output from the mean/variance calculating unit and outputting a signal indicating for which the calculated contrast is largest;

a second candidate graph selecting/storing unit for outputting the candidate graphs having the largest contrast among the second candidate graphs stored therein in response to the signal indicating the candidate graphs for which the calculated contrast is largest;

a second-to-fifth indicator determining unit for determining the orientation coefficient of a graph having the largest contrast, among the horizontal-axis projection graphs;

the orientation coefficient of a graph having the largest contrast, among the vertical-axis projection graphs, as a second indicator;

the scale coefficient of a graph having the largest contrast, among the horizontal-axis projection graphs, as a fourth indicators and the scale coefficient of a graph having the largest contrast, among the vertical-axis projection graphs, as a fifth indicator; and

a texture descriptor output unit for combining the first indicator, the second through fifth indicators and the predetermined vector and outputting a combination result as the texture descriptors of the corresponding image.