System and method for automatically identifying and matching a color of a structure's external surface

• US 9,183,641 B2
• Filed: 02/10/2014
• Issued: 11/10/2015
• Est. Priority Date: 02/10/2014
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Maurer, Jonathan D., Christopulos, Nicholas U., Drinan, Lee C.
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Rosario, Dennis
• Application Number:
  US14/176,734
• Publication Number:
  US 20150228086A1
• Time in Patent Office: 638 Days
• Field of Search: 382/162, 382/167
• US Class Current: 1/1
• CPC Class Codes: G01J 2003/466 Coded colour; Recognition of predetermined colour; Determining proximity to predetermined colour
• CPC Class Codes: G01J 3/0264 Electrical interface; User interface
• CPC Class Codes: G01J 3/463 Colour matching
• CPC Class Codes: G01J 3/52 using colour charts
• CPC Class Codes: G01J 3/524 Calibration of colorimeters
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

The method and system may be used to provide an indication of a color value for a particular siding sample and to color match a specific siding product to the color value of the siding sample. The system receives a digital image of a siding sample and a desired color value to be matched. A color query module plots this desired color value as a desired color point in a multidimensional color space together with a plurality of color reference points. Each color reference point represents the color value of an existing siding product. The system determines a “distance” between the desired color point and each plotted color reference point within the color space and identifies the siding product associated with the color reference point that is located the shortest distance to the desired color point within the color space.

Claims

1. A computer-implemented method for matching the color of siding, the method comprising:

displaying, via one or more processors, a positional overlay framing a digital image of a siding sample, the positional overlay including a color card reference area and indicating instructions for (i) a distance to be maintained from the siding sample with respect to an image capture device, (ii) an orientation of the siding sample with respect to the image capture device, before the digital image is captured, and (iii) determining the correct physical placing of a color reference card on the siding sample;

capturing, via one or more processors, the digital image according to the instructions such that the image capture device is at the distance from the siding sample as specified by the instructions, the siding sample is at the orientation specified by the instructions, and the reference card is placed as specified by the instructions;

receiving, via one or more processors from the captured digital image of a the siding sample, (i) a region of interest, and (ii) a region of reference color corresponding to the color reference card positioned within the color card reference area, the color reference card having a predetermined color value, and the region of interest indicating a portion of the siding sample having a desired color value to be matched;

color normalizing, by one or more processors, the digital image based upon the predetermined color value to provide an adjusted desired color value to be matched within the region of interest;

plotting, via one or more processors, the adjusted desired color value as a desired color point in a color space;

determining, via one or more processors, a plurality of spatial distances between the desired color point and a plurality of reference color points within the color space, each of the plurality of reference color points representing a quantified color estimate of a respective replacement siding product from among a plurality of replacement siding products;

identifying, via one or more processors, one of the plurality of replacement siding products having a respective desired color point associated with the shortest spatial distance from among the plurality of spatial distances as the replacement siding product having the closest matching color to the siding sample; and

calculating, via one or more processors, a confidence level indicative of how similar the quantified color estimate for the replacement siding product having the closest matching color to the siding sample is to the adjusted desired color value based upon the shortest spatial distance.

2. The method of claim 1, wherein receiving the region of reference color includes automatically determining the region of reference color based on at least a shape and/or a size of the region of reference color.

3. The method of claim 1, wherein the color space is a three-dimensional (3D) color space.

4. The method of claim 3, wherein the 3D color space includes at least one of a red-green-blue (RGB) color space, a cyan-magenta-yellow-black (CMYK) color space, a hue-saturation-value (HSV) color space, a hue-saturation-light (HSL) color space, or a Lab color space (CIELab).

5. The method of claim 1, further comprising:

identifying, via one or more processors, at least one of a product name, a product number, or a manufacturer associated with the replacement siding product having the closest matching color to the siding sample.

6. The method of claim 1, wherein calculating the confidence level further comprises:

calculating the confidence level based upon whether the length of the shortest spatial distance is less than one or more spatial length thresholds, and

wherein a shorter spatial length threshold is indicative of a greater similarity between a quantified color estimate for the replacement siding product having the closest matching color to the siding sample and the adjusted desired color value.

7. The method of claim 1, wherein the confidence level is a numeric score.

8. The method of claim 1, wherein the act of displaying the positional overlay comprises:

displaying the positional overlay such that, when the digital image of the siding sample is captured, flat portions of the siding are parallel to a lens of the image capture device that captures the digital image to reduce glare.

9. The method of claim 1, wherein the act of receiving from the digital image of the siding sample further comprises:

receiving, from the digital image of the siding sample, the region of interest as an area within the positional overlay that is selected by a user.

10. A non-transitory computer-readable medium having stored thereon a set of instructions, executable by a processor, for matching the color of siding, the instructions comprising:

instructions for displaying a positional overlay framing a digital image of a siding sample, the positional overlay including a color card reference area and indicating instructions for (i) a distance to be maintained from the siding sample with respect to an image capture device, (ii) an orientation of the siding sample with respect to the image capture device, before the digital image is captured, and (iii) determining the correct physical placing of a color reference card on the siding sample;

capturing the digital image according to the instructions such that the image capture device is at the distance from the siding sample as specified by the instructions, the siding sample is at the orientation specified by the instructions, and the reference card is placed as specified by the instructions;

instructions for receiving, from the digital image of the siding sample, (i) a region of interest, and (ii) a region of reference color corresponding to a color reference card positioned within the color card reference area, the color reference card having a predetermined color value, and the region of interest indicating a portion of the siding sample having a desired color value to be matched;

color normalizing the digital image based upon (i) the predetermined color value, and (ii) lighting conditions present when the image was captured, to provide an adjusted desired color value to be matched within the region of interest;

instructions for causing one or more processors to plot the desired color point in a color space;

instructions for causing one or more processors to determine a plurality of spatial distances between the desired color point and a plurality of reference color points within the color space, each of the plurality of reference color points representing a quantified color estimate of a respective replacement siding product from among a plurality of replacement siding products;

instructions for causing one or more processors to determine one of the plurality of replacement siding products having a respective desired color point associated with the shortest spatial distance from among the plurality of spatial distances as the replacement siding product having the closest matching color to the siding sample; and

instructions for causing one or more processors to calculate a confidence level indicative of how similar the quantified color estimate for the replacement siding product having the closest matching color to the siding sample is to the adjusted desired color value based upon the shortest spatial distance.

11. The non-transitory computer-readable medium of claim 10, wherein the instructions for receiving the region of reference color include instructions for automatically determining the region of reference color based on at least a shape and/or a size of the region of reference color.

12. The non-transitory computer-readable medium of claim 10, wherein the color space is a three-dimensional (3D) color space.

13. The non-transitory computer-readable medium of claim 12, wherein the 3D color space includes at least one of a red-green-blue (RGB) color space, a cyan-magenta-yellow-black (CMYK) color space, a hue-saturation-value (HSV) color space, a hue-saturation-light (HSL) color space, or a Lab color space (CIELab).

14. The non-transitory computer-readable medium of claim 10, further comprising:

instructions for causing one or more processors to identify at least one of a product name, a product number, or a manufacturer associated with the replacement siding product having the closest matching color to the siding sample.

15. The non-transitory computer-readable medium of claim 10, wherein the instruction to calculate the confidence level further include instructions comprising:

instructions for calculating the confidence level based upon whether the length of the shortest spatial distance is less than one or more spatial length thresholds, and

wherein a shorter spatial length threshold is indicative of a greater similarity between quantified color estimate for the replacement siding product having the closest matching color to the siding sample and the adjusted desired color value.

16. The non-transitory computer-readable medium of claim 10, wherein the confidence level is a numeric score.

17. The non-transitory computer-readable medium of claim 10, the instructions to display the positional overlay further comprising:

instructions to display the positional overlay such that, when the digital image of the siding sample is captured, flat portions of the siding are parallel to a lens of the image capture device that captures the digital image to reduce glare.

18. The non-transitory computer-readable medium of claim 10, wherein the instructions for receiving, from the digital image of the siding sample, the region of interest further comprises:

receiving, from the digital image of the siding sample, the region of interest as an area within the positional overlay that is selected by a user.

19. A computer-implemented method for matching the color of siding, the method comprising:

receiving, via one or more processors, a location of a desired color point in a color space, the desired color point corresponding to a desired color value of an identified region of interest within a digital image of a siding sample to be matched, the desired color value being the result of an a color normalization of the digital image and being based upon a predetermined color value associated with a region of reference color, the predetermined color value corresponding to a color reference card positioned within a color card reference area within the digital image of the siding sample;

determining, via one or more processors, a plurality of spatial distances between the received desired color point and a plurality of reference color points within the color space, each of the plurality of reference color points representing a quantified color estimate of a respective replacement siding product from among a plurality of replacement siding products;

identifying, via one or more processors, one of the plurality of replacement siding products having a respective desired color point associated with the shortest spatial distance from among the plurality of spatial distances as the replacement siding product having the closest matching color to the siding sample; and

calculating, via one or more processors, a confidence level indicative of how similar the quantified color estimate for the replacement siding product having the closest matching color to the siding sample is to the adjusted desired color value based upon the shortest spatial distance, and

wherein the digital image of the siding sample is captured using a positional overlay framing the digital image of the siding sample, the positional overlay including the color card reference area and indicating instructions for (i) a distance to be maintained from the siding sample with respect to an image capture device, (ii) an orientation of the siding sample with respect to the image capture device, before the digital image is captured, and (iii) determining the correct physical placing of the color reference card on the siding sample;

capturing the digital image according to the instructions such that the image capture device is at the distance from the siding sample as specified by the instructions, the siding sample is at the orientation specified by the instructions, and the the reference card is placed as specified by the instructions.

20. The method of claim 19, wherein the positional overlay is displayed such that, when the digital image of the siding sample is captured, flat portions of the siding are parallel to a lens of the image capture device that captures the digital image to reduce glare.

21. The method of claim 19, further comprising:

receiving, via one or more processors, the identified region of interest as an area within the positional overlay that is selected by a user.