System and method for automatically identifying and matching a color of a structure's external surface

• US 9,384,542 B1
• Filed: 09/30/2015
• Issued: 07/05/2016
• Est. Priority Date: 02/10/2014
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Maurer, Jonathan D., Christopulos, Nicholas U., Drinan, Lee C.
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Rosario, Dennis
• Application Number:
  US14/870,209
• Time in Patent Office: 279 Days
• US Class Current: 1/1
• CPC Class Codes: G01J 2003/466 Coded colour; Recognition of predetermined colour; Determining proximity to predetermined colour
• CPC Class Codes: G01J 3/0264 Electrical interface; User interface
• CPC Class Codes: G01J 3/463 Colour matching
• CPC Class Codes: G01J 3/52 using colour charts
• CPC Class Codes: G01J 3/524 Calibration of colorimeters
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

The method and system may be used to provide an indication of a color value for a particular siding sample and to color match a specific siding product to the color value of the siding sample. The system receives a digital image of a siding sample and a desired color value to be matched. A color query module plots this desired color value as a desired color point in a multidimensional color space together with a plurality of color reference points. Each color reference point represents the color value of an existing siding product. The system determines a “distance” between the desired color point and each plotted color reference point within the color space and identifies the siding product associated with the color reference point that is located the shortest distance to the desired color point within the color space.

Claims

1. A computer-implemented method for matching the color of siding, the method comprising:

displaying, via one or more processors, a positional overlay framing a digital image of a siding sample to be taken with an image capture device, the positional overlay including a color card reference area and a region of interest indicating a portion of the siding sample that has a first color value,

wherein the positional overlay indicates instructions for (i) a distance to be maintained from the siding sample with respect to the image capture device, (ii) an orientation of the siding sample with respect to the image capture device before the digital image is captured, and (iii) a determination of a correct physical placing of a color reference card having a reference color value on the siding sample;

capturing, via the image capture device, the digital image at the distance, orientation, and correct physical placing of the color reference card according to the instructions;

color normalizing, by one or more processors, the digital image based upon the reference color value of the color reference card positioned within the color card reference area when the digital image is captured, the color normalizing resulting in the first color value within the region of interest being adjusted to a second color value;

plotting, via one or more processors, the second color value as a plotted color point in a color space;

determining, via one or more processors, a plurality of spatial distances between the plotted color point and a plurality of reference color points within the color space, respectively,

wherein each of the plurality of reference color points represents a quantified color of a respective replacement siding product from among a plurality of replacement siding products, and

identifying, via one or more processors, which of the plurality of replacement siding products has a quantified color point associated with the shortest spatial distance to the plotted color point within the color space as the replacement siding product having the closest matching color to the siding sample.

2. The method of claim 1, further comprising:

calculating, via one or more processors, a confidence level as a numeric score indicative of how similar the quantified color of the replacement siding product having the closest matching color to the siding sample is to the plotted color point within the color space based upon the shortest spatial distance.

3. The method of claim 2, wherein the act of calculating the confidence level further comprises:

calculating the confidence level based upon whether the length of the shortest spatial distance is less than one or more spatial length thresholds, and

wherein a shorter spatial length threshold is indicative of a greater similarity between the quantified color of the replacement siding product having the closest matching color to the siding sample and the plotted color point within the color space.

4. The method of claim 1, wherein the color space is a three-dimensional (3D) color space and includes at least one of a red-green-blue (RGB) color space, a cyan-magenta-yellow-black (CMYK) color space, a hue-saturation-value (HSV) color space, a hue-saturation-light (HSL) color space, or a Lab color space (CIELab).

5. The method of claim 1, further comprising:

identifying, via one or more processors, at least one of a product name, a product number, or a manufacturer associated with the replacement siding product having the closest matching color to the siding sample.

6. The method of claim 1, further comprising:

receiving, from the digital image of the siding sample, the region of interest as an area within the positional overlay that is selected by a user.

7. The method of claim 1, wherein the act of displaying the positional overlay comprises:

displaying the positional overlay such that, when the digital image of the siding sample is captured, flat portions of the siding are parallel to a lens of the image capture device that captures the digital image to reduce glare.

8. A non-transitory computer-readable medium having stored thereon a set of instructions, that when executed by a processor, cause the processor to:

display a positional overlay framing a digital image of a siding sample to be taken with an image capture device, the positional overlay including a color card reference area and a region of interest indicating a portion of the siding sample that has a first color value,

wherein the positional overlay indicates instructions for (i) a distance to be maintained from the siding sample with respect to the image capture device, (ii) an orientation of the siding sample with respect to the image capture device before the digital image is captured, and (iii) a determination of a correct physical placing of a color reference card having a reference color value on the siding sample;

capture the digital image with the image capture device at the distance, orientation, and correct physical placing of the color reference card according to the instructions;

color normalize the digital image based upon the reference color value of the color reference card positioned within the color card reference area when the digital image is captured, the color normalizing resulting in the first color value within the region of interest being adjusted to a second color value;

plot the second color value as a plotted color point in a color space;

determine a plurality of spatial distances between the plotted color point and a plurality of reference color points within the color space, respectively,

wherein each of the plurality of reference color points represents a quantified color of a respective replacement siding product from among a plurality of replacement siding products, and

identify which of the plurality of replacement siding products has a quantified color point associated with the shortest spatial distance to the plotted color point within the color space as the replacement siding product having the closest matching color to the siding sample.

9. The non-transitory computer-readable medium of claim 8, further including instructions that, when executed by the processor, cause the processor to:

calculate a confidence level as a numeric score indicative of how similar the quantified color of the replacement siding product having the closest matching color to the siding sample is to the plotted color point within the color space based upon the shortest spatial distance.

10. The non-transitory computer-readable medium of claim 9,

wherein the instructions to calculate the confidence level further include instructions that, when executed by the processor, cause the processor to:

calculate the confidence level based upon whether the length of the shortest spatial distance is less than one or more spatial length thresholds, and

wherein a shorter spatial length threshold is indicative of a greater similarity between the quantified color of the replacement siding product having the closest matching color to the siding sample and the plotted color point within the color space.

11. The non-transitory computer-readable medium of claim 8, wherein the color space is a three-dimensional (3D) color space and includes at least one of a red-green-blue (RGB) color space, a cyan-magenta-yellow-black (CMYK) color space, a hue-saturation-value (HSV) color space, a hue-saturation-light (HSL) color space, or a Lab color space (CIELab).

12. The non-transitory computer-readable medium of claim 8, further including instructions that, when executed by the processor, cause the processor to:

identify at least one of a product name, a product number, or a manufacturer associated with the replacement siding product having the closest matching color to the siding sample.

13. The non-transitory computer-readable medium of claim 8, further including instructions that, when executed by the processor, cause the processor to:

receive, from the digital image of the siding sample, the region of interest as an area within the positional overlay that is selected by a user.

14. The non-transitory computer-readable medium of claim 8, wherein the instructions to display the positional overlay further include instructions that, when executed by the processor, cause the processor to:

display the positional overlay such that, when the digital image of the siding sample is captured, flat portions of the siding are parallel to a lens of the image capture device that captures the digital image to reduce glare.

15. A system, comprising:

a mobile computing device configured to;

display a positional overlay framing a digital image of a siding sample to be taken with an image capture device, the positional overlay including a color card reference area and a region of interest indicating a portion of the siding sample that has a first color value,

wherein the positional overlay indicates instructions for (i) a distance to be maintained from the siding sample with respect to the image capture device, (ii) an orientation of the siding sample with respect to the image capture device before the digital image is captured, and (iii) a determination of a correct physical placing of a color reference card having a reference color value on the siding sample;

capture the digital image with the image capture device at the distance, orientation, and correct physical placing of the color reference card according to the instructions; and

a color matching engine configured to;

receive the captured digital image from the mobile computing device as a color matching request;

color normalize the digital image based upon the reference color value of the color reference card positioned within the color card reference area when the digital image is captured, the color normalizing resulting in the first color value within the region of interest being adjusted to a second color value;

plot the second color value as a plotted color point in a color space;

determine a plurality of spatial distances between the plotted color point and a plurality of reference color points within the color space, each of the plurality of reference color points representing a quantified color of a respective replacement siding product from among a plurality of replacement siding products, respectively,

identify which of the plurality of replacement siding products has a quantified color point associated with the shortest spatial distance to the plotted color point within the color space as the replacement siding product having the closest matching color to the siding sample, and

transmit an indication of the replacement siding product to the mobile computing device.

16. The system of claim 15, wherein the color matching engine is further configured to calculate a confidence level as a numeric score indicative of how similar the quantified color of the replacement siding product having the closest matching color to the siding sample is to the plotted color point within the color space based upon the shortest spatial distance.

17. The system of claim 16, wherein the color matching engine is further configured to calculate the confidence level based upon whether the length of the shortest spatial distance is less than one or more spatial length thresholds, and

wherein a shorter spatial length threshold is indicative of a greater similarity between the quantified color of the replacement siding product having the closest matching color to the siding sample and the plotted color point within the color space.

18. The system of claim 15, wherein the color space is a three-dimensional (3D) color space and includes at least one of a red-green-blue (RGB) color space, a cyan-magenta-yellow-black (CMYK) color space, a hue-saturation-value (HSV) color space, a hue-saturation-light (HSL) color space, or a Lab color space (CIELab).

19. The system of claim 15, wherein the color matching engine is further configured to identify at least one of a product name, a product number, or a manufacturer associated with the replacement siding product having the closest matching color to the siding sample.

20. The system of claim 15, wherein the mobile computing device is further configured to receive, from the digital image of the siding sample, the region of interest as an area within the positional overlay that is selected by a user.

21. The system of claim 15, wherein the mobile computing device is further configured to display the positional overlay such that, when the digital image of the siding sample is captured, flat portions of the siding are parallel to a lens of the image capture device that captures the digital image to reduce glare.