Multi-agent training of a color identification neural network

• US 10,346,710 B2
• Filed: 09/29/2016
• Issued: 07/09/2019
• Est. Priority Date: 09/29/2016
• Status: Active Grant

• Current Assignee: Datacolor, Inc.
• Original Assignee: Datacolor, Inc.
• Inventors: Xu, Zhiling
• Primary Examiner(s): Strege, John B
• Application Number:
  US15/280,506
• Publication Number:
  US 20180089532A1
• Time in Patent Office: 1,013 Days
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks

Abstract

Various embodiments of the systems and methods described herein are directed towards training an artificial neural network to identify color values of a sample by providing image data obtained through multiple image capture devices under a plurality of lighting conditions. The present invention also includes using a pre-trained neural network to identify the color values of a sample having an unknown color value by capturing an image of an unknown color sample and known color reference samples under any illumination or hardware configuration.

Claims

1. A system for training a neural network to output estimated CIE color values of a sample having unknown color values, the system comprising

at least one image capture device, where the at least one image capture device is configured to capture an image of at least one color sample and a color reference chart comprised of known color reference elements, the color reference chart including a plurality of reference color element groups, wherein each of the plurality of reference color element groups includes a matrix of color reference elements, each color sample having known CIE color values, and wherein the image capture device is configured to capture the image of the at least one color sample and color reference chart under at least one of a plurality of known illuminants, each image capture device configured to transmit the captured image;

a processor having memory configured by program code executed thereby to;

receive the transmitted images;

obtain the known CIE color values corresponding to the color sample;

extract from the transmitted images the RGB color values of the pixels corresponding to area of the image depicting the sample and the reference color elements and

train the neural network using the plurality of images and corresponding known CIE color values, where training of the neural network includes;

for each image, apply the extracted RGB color values for the sample and the reference color elements as input nodes in an input layer of an artificial neural network and assign the known CIE color values for the sample color as the output nodes of the artificial neural network.

2. The neural network training system of claim 1, wherein the color reference groups are arranged to have fourfold rotational symmetry about the sample.

3. The neural network training system of claim 1, wherein the image capture device is configured to transmit the known color value of the sample along with the captured image.

4. The neural network training system of claim 1, wherein the processor is configured by code executing thereby to access the pre-determined color value from one or more storage devices accessible to the processor and store, for each image, the RGB color values of the pixels corresponding to area of the image depicting the sample and the reference color elements.

5. The neural network training system of claim 1, wherein the processor is further configured to:

access from a storage location accessible by the processor, a collection of image data objects, where each image data object includes the RGB color values of the pixels corresponding to area of the image depicting the sample, the reference color elements of a subset of the plurality of extracted RGB values for a given image, and the known CIE color values for the sample;

evaluate the RGB color values as input node values of the neural network;

receive the output of the neural network as CIE color values for color sample;

compare the output CIE color values to the known CIE color values stored in the image data object; and

provide an alert to a user of the neural network where the known CIE values and the output CIE values are not substantially similar.

6. The neural network training system of claim 1, wherein each image device is configured to transmit a hardware profile to the processor along with the captured image.

7. The neural network training system of claim 6, wherein the training of the artificial neural network further includes:

storing the captured image data and associated hardware profile in one or more databases;

filtering the entries of the database for images captured with a particular hardware profile to obtain a hardware specific image dataset;

applying the hardware specific image dataset as input node values of the artificial neural network.

8. The neural network training system of claim 6, wherein the hardware profile of at least one image capture device differs from the hardware profile of at least one other image capture device.

9. The neural network training system of claim 1, wherein at least one image of the sample is captured under each of the plurality of illuminants.

10. The neural network training system of claim 1, wherein each image capture device is remote to the processor and configured to transmit the captured image via a wireless communication channel.

11. A system for training a neural network to output estimated CIE color values of a sample having unknown color values, the system comprising:

at least one image capture device, where each image capture device is configured to capture an image of at least one color sample and a color reference chart comprised of known color reference elements, the color reference chart including one or more reference color elements, the at least one color sample having a known CIE color value, and wherein the image capture device is configured to capture the image of the at least one color sample and color reference chart under at least one of a plurality of known illuminants, each image capture device configured to transmit the captured image;

a processor having memory configured by program code executed thereby to;

receive the transmitted images;

obtain the known CIE color values corresponding to the color sample;

extract from the transmitted images the RGB color values of the pixels corresponding to area of the image depicting the sample and the reference color elements and

train the neural network using the plurality of images and corresponding known CIE color values, where training of the neural network includes;

access from a storage location accessible by the processor, a collection of image data objects, where each image data object includes the RGB color values of the pixels corresponding to area of the image depicting the sample, the reference color elements of a subset of the plurality of extracted RGB values for a given image, and the known CIE color values for the sample;

evaluate the RGB color values as input node values of the neural network;

receive the output of the neural network as CIE color values for color sample;

compare the output CIE color values to the known CIE color values stored in the image data object; and

provide an alert to a user of the neural network where the known CIE values and the output CIE values are not substantially similar.

12. A system for training a neural network to output estimated CIE color values of a sample having unknown color values, the system comprising

at least one image capture device, where each image capture device is configured to capture an image of at least one color sample and a color reference chart comprised of known color reference elements, the color reference chart including one or more reference color elements, the at least one color sample having a known CIE color value, and wherein the image capture device is configured to capture the image of the at least one color sample and color reference chart under at least one of a plurality of known illuminants, each image capture device configured to transmit the captured image and a hardware profile of the image capture device used to capture the image;

a processor having memory configured by program code executed thereby to;

receive the transmitted images and hardware profile;

obtain the known CIE color values corresponding to the color sample;

extract from the transmitted images the RGB color values of the pixels corresponding to area of the image depicting the sample and the reference color elements and

train the neural network using the plurality of images and corresponding known CIE color values, where training of the neural network includes;

storing the captured image data and associated hardware profile in one or more databases;

filtering the entries of the database for images captured with a particular hardware profile to obtain a hardware specific image dataset;

applying the hardware specific image dataset as input node values of the artificial neural network.