Color naming, color categorization and describing color composition of images

• US 20060087517A1
• Filed: 09/20/2002
• Published: 04/27/2006
• Est. Priority Date: 09/20/2002
• Status: Active Grant

• Current Assignee: Aleksandra Mojsilovic
• Original Assignee: Aleksandra Mojsilovic
• Inventors: Mojsilovic, Aleksandra
• Granted Patent: US 7,415,153 B2
• Time in Patent Office: Days
• US Class Current: 345/593
• CPC Class Codes: G01J 3/462 Computing operations in or between colour spaces; Colour management systems
• CPC Class Codes: G01J 3/463 Colour matching
• CPC Class Codes: G01J 3/465 taking into account the colour perception of the eye; using tristimulus detection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 1/644 using a reduced set of representative colours, e.g. each representing a particular range in a colour space

Abstract

The invention provides methods, systems and apparatus for assigning color names to individual image pixels, and generating verbal description of color composition in images, suitable for a wide variety of information processing applications. For an isolated image pixel (individual color sample or color value) an illustrative embodiment of this invention uses a predetermined vocabulary of color names, and then generates a distance measure describing the best color match and corresponding color name for the given pixel. For the input image, an illustrative embodiment of this invention computes the simplified representation of the scene, consistent with human perception, and uses the predetermined vocabulary, syntax rules and metric to assign color names to all perceptually significant objects and regions and obtain the verbal description of the overall color composition. The invention is also applicable to other types of information signals, such as sequences of video frames, web pages, etc.

Claims

1. A method comprising employing a vocabulary of color names in assigning a color name to a pixel in a digital representation, including the steps of:

providing said vocabulary of color names with a plurality of color prototypes, each said color prototype having a prototype color name and a corresponding prototype color value;

comparing a pixel color value of said pixel to said prototype color value of each color prototype in the vocabulary, and obtaining a color match value for said each color prototype in the vocabulary;

determining a closest color match value resulting from said step of comparing; and

assigning to said pixel a particular prototype color name corresponding to said closest match value.

2. A method as recited in claim 1, further comprising forming said vocabulary of color names, includes the steps of:

obtaining at least one general set of essential color names, each essential color name having a corresponding color value;

whereby each color name includes a hue descriptor, brightness modifier, and saturation modifier; and

selecting a subset of said at least one general set in meeting an application desire for said vocabulary of color names.

3. A method as recited in claim 2, wherein the step of obtaining includes starting with a large number of arbitrary color names having corresponding color values and modify said color names through subjective experiments to obtain a general set that models human behavior in color naming.

4. A method as recited in claim 1, wherein the step of comparing includes the steps of:

computing a color distance in a selected color space between the color value of said pixel and said prototype color value;

finding the closest perceptual match to the prototype color value for said color distance;

computing an estimate value of perceptual dissimilarity between said closest perceptual match and color value of said pixel; and

using said estimate value to modify said color distance, thereby obtaining said color match value.

5. A method as recited in claim 1, wherein the step of determining includes finding the prototype color value having the minimum color naming distance to the color value of said pixel.

6. A method as recited in claim 1, further comprising:

acquiring a digital representation of a scene;

employing said vocabulary in providing a description of a color composition in said scene.

7. An article of manufacture comprising a computer usable medium having computer readable program code means embodied therein for causing assignment of a color name, the computer readable program code means in said article of manufacture comprising computer readable program code means for causing a computer to effect the steps of claim 1.

8. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for assigning a color name, said method steps comprising the steps of claim 1.

9. An apparatus comprising means for employing a vocabulary of color names in assigning a color name to a pixel in a digital representation, including:

means for providing said vocabulary of color names with a plurality of color prototypes, each said color prototype having a prototype color name and a corresponding prototype color value;

means for comparing a pixel color value of said pixel to said prototype color value of each color prototype in the vocabulary, and obtaining a color match value for said each color prototype in the vocabulary;

means for determining a closest color match value resulting from said step of comparing; and

means for assigning to said pixel a particular prototype color name corresponding to said closest match value.

10. An apparatus as recited in claim 9, further comprising

means for forming said vocabulary of color names, including:

means for obtaining at least one general set of essential color names, each essential color name having a corresponding color value;

means for whereby each color name includes a hue descriptor, brightness modifier, and saturation modifier; and

means for selecting a subset of said at least one general set in meeting an application desire for said vocabulary of color names.

13. A computer program product comprising a computer usable medium having computer readable program code means embodied therein for causing assignment of a color name to a pixel in a digital representation, the computer readable program code means in said computer program product comprising computer readable program code means for causing a computer to effect the functions of claim 9.

11. An apparatus comprising:

means for obtaining a vocabulary of color names;

means for acquiring a digital representation of a scene; and

means for employing said vocabulary in providing a description of a color composition in said scene.

12. Apparatus as recited in claim 11,

wherein the means for providing includes means for employing color names in describing the scene and in describing objects in said scene.

14. A computer program product comprising a computer usable medium having computer readable program code means embodied therein for causing provision of a description of a color composition in a scene, the computer readable program code means in said computer program product comprising computer readable program code means for causing a computer to effect the functions of claim 11.

15. A method comprising:

obtaining a vocabulary of color names;

acquiring a digital representation of a scene; and

employing said vocabulary in providing a description of a color composition in said scene.

16. A method as recited in claim 15, further comprising at least one limitation taken from a group of limitations consisting of:

wherein the step of providing includes employing color names in describing the scene and in describing objects in said scene;

wherein the step of employing includes;

computing a simplified representation of said scene to model a human perception of said scene;

segmenting a digital image representing said scene into a set of meaningful regions;

computing a perceived color value for each said meaningful region;

assigning a region color name from said vocabulary to each said meaningful region,

using said region color names to describe the color composition of said scene at different accuracy levels;

further comprising computing a simplified representation of said scene corresponding to a human observation of said scene;

wherein the step of computing a simplified representation of said scene includes;

applying a chromatic transformation to accommodate for a particular illumination condition of said scene;

labeling said each pixel in said image with a label of a descriptive type to correspond with the descriptive type of an element of said scene to which said each pixel belongs;

performing adaptive smoothing of said image with an amount of smoothing around said each pixel in correspondence with the descriptive type of said each pixel;

for said each pixel, determining whether said each pixel is a perceptually significant pixel or a perceptually insignificant pixels; and

reapplying chromatic transformation to preserve a color appearance as perceived in said human observation;

wherein the step of labeling includes;

determining if said each pixel in said image represents an edge;

estimating distribution of edge density in a neighborhood of said each pixel in said image; and

using said distribution to determine a descriptive type of said each pixel and labeling said each pixel accordingly;

wherein the descriptive type includes a type taken from a group of types including;

uniform, noise, contour, region boundary texture boundary transition region, coarse texture, and fine texture;

wherein said step of determining includes deeming said each pixel to be perceptually significant if said each pixel is at least one of;

uniform, in a dominant object, in a dominant background, coarse texture and fine texture;

wherein said step of determining includes deeming said each pixel to be perceptually insignificant if said each pixel is at least one of;

noise, contour, region boundary texture boundary, and transition region;

wherein the step of labeling produces a label for said each pixel, and the step of smoothing includes;

obtaining the label for said each pixel in said image; and

using said label for said each pixel in determining a smoothing filter to be used for said each pixel in said image;

wherein the step of smoothing is performed only on perceptually significant pixels;

further comprising changing said smoothing filter depending on a particular descriptive type of a particular pixel in a center of the smoothing filter;

wherein said support of a smoothing filter is largest for pixels labeled uniform and smallest for pixels labeled coarse texture;

wherein the step of segmentation includes at least one of the following;

color segmentation, texture segmentation, region growing, region merging, region splitting, multiresolution segmentation; and

wherein the step of computing a perceived color value for a meaningful color region includes;

changing a pixel value of each perceptually insignificant pixel in said region into a new value equal to the pixel value of a closest perceptually significant pixel from said region; and

averaging color values of all pixels in said region.

29. A method as recited in claim 16, wherein the step of assigning includes:

providing said vocabulary of color names with a plurality of color prototypes, each said color prototype having a prototype color name and a corresponding prototype color value;

comparing said perceived color value of each said meaningful region to said prototype color value of each color prototype in the vocabulary, and obtaining a color match value for said each color prototype in the vocabulary;

determining a closest color match value resulting from said step of comparing; and

assigning to said meaningful region a particular prototype color name corresponding to said closest match value.

30. A method as recited in claim 29, further comprising forming said vocabulary of color names, including the steps of:

obtaining at least one general set of essential color names, each essential color name having a corresponding color value;

wherein each color name includes a hue descriptor, brightness modifier, and saturation modifier; and

selecting a subset of said at least one general set in meeting an application desire for said vocabulary of color names.

31. A method as recited in claim 16, having at least one limitation taken from a group of limitations consisting of:

wherein the step of using includes;

providing a set of rules defining accuracy levels in color naming,

selecting an accuracy level meeting an application desire, and

computing a histogram of color names for said scene, for said selected accuracy level;

further comprising forming said rules, includes the steps of;

obtaining a set of accuracy levels describing human behavior in color naming, and

deriving rules whereby color names and each accuracy level include at least one of;

a hue descriptor, a combination of brightness modifiers, and a saturation modifier;

wherein the set of accuracy levels is determined employing subjective experiments modeling human behavior in color naming;

wherein the step of computing a histogram of color names for said scene includes;

determining color names present in said scene.

computing a total area for each said color name,

discarding color names not meeting an area criterion, and

combining color names according to said rule for said selected accuracy level; and

further comprising;

providing color names for objects in said scene; and

pointing out examples of named color in said scene.

37. An article of manufacture comprising a computer usable medium having computer readable program code means embodied therein for causing provision of a description of a color composition in a scene, the computer readable program code means in said article of manufacture comprising computer readable program code means for causing a computer to effect the steps of claim 15.

38. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for providing a description of a color composition in a scene, said method steps comprising the steps of claim 15.

17-28. -28. (canceled)

32-35. -35. (canceled)

36. A method comprising computing a simplified representation of a scene consistent with human observation of said scene, including the steps of:

applying chromatic transformation to accommodate for different illumination conditions;

labeling pixels in said scene into different pixel types;

performing adaptive smoothing of said image where the amount of smoothing around each pixel depends on the type of said pixel;

determining perceptually significant pixels and perceptually insignificant pixels; and

reapplying chromatic transformation to preserve the color appearance to the human observer;

39. An article of manufacture comprising a computer usable medium having computer readable program code means embodied therein for causing provision of a description of a color composition in a scene, the computer readable program code means in said article of manufacture comprising computer readable program code means for causing a computer to effect the steps of claim 36.

40. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for providing a description of a color composition in a scene, said method steps comprising the steps of claim 36. (original)