System and method for color characterization using fuzzy pixel classification with application in color matching and color match location

• US 7,046,842 B2
• Filed: 12/13/2000
• Issued: 05/16/2006
• Est. Priority Date: 08/17/1999
• Status: Expired due to Term

• Current Assignee: National Instruments Corporation (Emerson Electric Company)
• Original Assignee: National Instruments Corporation (Emerson Electric Company)
• Inventors: Nair, Dinesh, Lin, Siming, Schmidt, Darren
• Primary Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US09/737,531
• Publication Number:
  US 20020102018A1
• Time in Patent Office: 1,980 Days
• Field of Search: 382162-167, 382/305, 382/170, 707/104.1
• US Class Current: 382/165
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30141 Printed circuit board [PCB]
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/66 Trinkets, e.g. shirt buttons or jewellery items recognising microscopic objects G06V20/69

Abstract

A system and method for measuring the similarity of multiple-color images and for locating regions of a target image having color information that matches, at least to a degree, the color information of a template image. A color characterization method operates to characterize the colors of an image and to measure the similarity between multiple-color images. For each image pixel, the method determines a color category or bin for the respective pixel based on HSI values of the respective pixel, wherein the color category is one of a plurality of possible color categories in HSI color space. In various embodiments, the weight of the pixel may be fractionally distributed across a plurality of color categories, e.g., as determined by applying fuzzy pixel classification with a fuzzy membership function. The percentage of pixels assigned to each category is then determined. The percentage of pixels in each color category is then used as a color feature vector to represent the color information of the color image. A quantitative measurement of the color similarity between color images is then computed based on the distance between their color feature vectors. Once the color information of a template image has been characterized, a target image may be searched in order to locate regions within the target image having matching color information. In one embodiment, a coarse-to-fine heuristic may be utilized, in which multiple search stages of decreasing granularity are performed. A first-stage search may operate to identify a list of candidate match regions based on the city-block distance of the color feature vector computed using a sub-sampling scheme. These candidate match regions may then be examined in further detail in order to determine final matches.

Claims

1. A computer-implemented method for characterizing colors of an image, wherein the image comprises a plurality of pixels, the method comprising:

for each respective pixel of at least a subset of pixels of the image, assigning values to one or more color categories based on color information of the pixel;

wherein, for each of one or more first pixels, said assigning comprises assigning values to a plurality of the color categories based on color information of the pixel, wherein, for each of the one or more first pixels, said assigning comprises assigning a percentage of the pixel to each of the plurality of color categories; and

determining information regarding the total values of pixels assigned to each of the color categories, wherein said information characterizes colors of the image.

2. The method of claim 1,

wherein, for each of the one or more first pixels, the sum of the percentages assigned to each of the plurality of color categories is 100 percent.

3. The method of claim 1,

wherein each of the one or more color categories corresponds to a portion of a color space.

4. The method of claim 3,

wherein, for each respective pixel of the at least a subset of pixels, said assigning values to the one or more color categories based on color information of the pixel comprises:

determining a location of the pixel within the color space;

applying a function based on the location of the pixel within the color space to determine the values assigned to the one or more color categories.

5. The method of claim 4,

wherein said determining the location of the pixel within the color space comprises examining color information of the pixel.

6. The method of claim 5,

wherein the color space is the Hue, Saturation, Intensity (HSI) color space;

wherein said examining color information of the pixel comprises examining HSI information of the pixel.

7. The method of claim 4,

wherein the function is a fuzzy membership function.

8. The method of claim 7,

wherein the fuzzy membership function is one of:

a triangle fuzzy membership function;

a trapezoidal fuzzy membership function; and

a step fuzzy membership function.

9. The method of claim 1,

wherein said assigning values to one or more color categories based on color information of each pixel comprises creating a data structure having values representing the total values of pixels assigned to each of the color categories;

wherein said determining information regarding the total contributions of pixels to each of the color categories comprises determining the values of the data structure.

10. A computer-implemented method for characterizing colors of an image, wherein the image comprises a plurality of pixels, the method composing:

for each respective pixel of at least a subset of pixels of the image, determining contributions of the pixel to one or more color categories;

wherein, for each of one or more first pixels, said determining comprises determining contributions of the pixel to a plurality of the color categories, wherein, for each of the one or more first pixels, said determining comprises determining a percentage of the pixel that is contributed to each of the plurality of color categories; and

determining information regarding the total contributions of pixels to each of the color categories, wherein said information characterizes colors of the image.

11. The method of claim 10,

wherein, for each of the one or more first pixels, said determining comprises determining a percentage of the pixel that is contributed to each of the plurality of color categories.

12. The method of claim 11,

wherein, for each of the one or more first pixels, the sum of the percentages contributed to each of the plurality of color categories is 100 percent.

13. The method of claim 10,

wherein each of the one or more color categories corresponds to a portion of a color space.

14. The method of claim 13,

wherein, for each respective pixel of the at least a subset of pixels, said determining the contributions of the pixel to the one or more color categories comprises:

determining a location of the pixel within the color space;

applying a function based on the location of the pixel within the color space to determine the contributions of the pixel to the one or more color categories.

15. The method of claim 14,

wherein said determining the location of the pixel within the color space comprises examining color information of the pixel.

16. The method of claim 15,

wherein the color space is the Hue, Saturation, Intensity (HSI) color space;

wherein said examining color information of the pixel comprises examining HSI information of the pixel.

17. The method of claim 14,

wherein the function is a fuzzy membership function.

18. The method of claim 17,

wherein the fuzzy membership function is one of:

a triangle fuzzy membership function;

a trapezoidal fuzzy membership function; and

a step fuzzy membership function.

19. The method of claim 10,

wherein said determining contributions of each pixel to one or more color categories comprises creating a data structure having values representing the total contributions of pixels to each of the color categories;

wherein said determining information regarding the total contributions of pixels to each of the color categories comprises determining the values of the data structure.

20. A computer-implemented method for characterizing the color information of an image, wherein the image comprises a plurality of pixels, wherein a color space representing possible colors of the pixels is divided into a plurality of bins, the method comprising:

for each of at least a subset of pixels of the image;

examining color information of the pixel to determine a bin corresponding to the color information of the pixel;

applying a function based on a location of the pixel within the bin to determine a contribution of the pixel to one or more neighboring bins, wherein the contribution comprises a respective percentage of the pixel for each of the one or more neighboring bins;

assigning values to the bin and the one or more neighboring bins based on the determined contributions of the pixel to the one or more neighboring bins;

wherein the total assigned values across the bins of the color space characterize the color information of the image.

21. The method of claim 20,

wherein the color space is the Hue, Saturation, Intensity (HSI) color space;

wherein said examining color information of the pixel comprises examining HSI color information of the pixel.

22. The method of claim 20,

wherein the function is a fuzzy membership function.

23. The method of claim 20,

wherein a first pixel falls within a first bin;

wherein the location of the first pixel within the first bin is a first distance away from a second bin, with respect to color space difference, wherein the second bin neighbors the first bin, with respect to the portion of color space to which the first and second bins correspond;

wherein a second pixel falls within the first bin;

wherein the location of the second pixel within the first bin is a second distance away from the second bin, with respect to color space difference;

wherein the second distance is less than the first distance;

wherein applying the function based on the location of the first pixel within the first bin results in determining a contribution of the first pixel to the second bin;

wherein applying the function based on the location of the second pixel within the first bin results in determining a contribution of the second pixel to the second bin;

wherein the contribution of the second pixel to the second bin is greater than the contribution of the first pixel to the second bin.

24. A system for characterizing colors of an image, wherein the image comprises a plurality of pixels, the system comprising:

a processor;

a memory medium coupled to the processor, wherein the memory medium stores color characterization software;

wherein the processor is operable to execute the color characterization software to;

for each respective pixel of at least a subset of pixels of the image, assign values to one or more color categories based on color information of the pixel;

wherein, for each of one or more first pixels, said assigning comprises assigning values to a plurality of the color categories based on color information of the pixel, wherein, for each of the one or more first pixels, said assigning comprises assigning a percentage of the pixel to each of the plurality of color categories;

wherein the processor is operable to determine information regarding the total values of pixels assigned to each of the color categories, wherein said information characterizes colors of the image.

25. The system of claim 24,

wherein each of the one or more color categories corresponds to a portion of a color space.

26. The system of claim 25,

wherein, for each respective pixel of the at least a subset of pixels, in assigning values to the one or more color categories based on color information of the pixel, the processor is operable to:

determine a location of the, pixel within the color space;

apply a function based on the location of the pixel within the color space to determine the values assigned to the one or more color categories.

27. The system of claim 26,

wherein the function is a fuzzy membership function.

28. The system of claim 24,

wherein, in performing said assigning values to one or more color categories based on color information of each pixel, the processor is operable to create a data structure having values representing the total values of pixels assigned to each of the color categories;

wherein said determining information regarding the total values of pixels assigned to each of the color categories comprises determining the values of the data structure.

29. A computer readable memory medium comprising program instructions operable to:

for each respective pixel of at least a subset of pixels of an image, assign values to one or more color categories based on color information of the pixel;

wherein, for each of one or more first pixels, said assigning comprises assigning values to a plurality of the color categories based on color information of the pixel, wherein, for each of the one or more first pixels, said assigning comprises assigning a percentage of the pixel to each of the plurality of color categories; and

determine information regarding the total values of pixels assigned to each of the color categories, wherein said information characterizes colors of the image.

30. The computer readable memory medium of claim 29,

wherein each of the one or more color categories corresponds to a portion of a color space.

31. The computer readable memory medium of claim 30,

wherein, for each respective pixel of the at least a subset of pixels, said assigning values to the one or more color categories based on color information of the pixel comprises:

determining a location of the pixel within the color space;

applying a function based on the location of the pixel within the color space to determine the values assigned to the one or more color categories.

32. The computer readable memory medium of claim 31,

wherein the function is a fuzzy membership function.

33. The computer readable memory medium of claim 29,

wherein said assigning values to one or more color categories based on color information of each pixel comprises creating a data structure having values representing the total values of pixels assigned to each of the color categories;

wherein said determining information regarding the total values of pixels assigned to each of the color categories comprises determining the values of the data structure.