Color-based post-processing of images

• US 9,805,289 B2
• Filed: 12/18/2015
• Issued: 10/31/2017
• Est. Priority Date: 12/18/2015
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Narasimha, Ramya, Moraleda, Jorge, Berkner, Kathrin, Bhat, Srikrishna, Pavani, Sri Kaushik, Garg, Anshul
• Primary Examiner(s): Liu, Li
• Application Number:
  US14/975,639
• Publication Number:
  US 20170177978A1
• Time in Patent Office: 683 Days
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: H04N 9/79 Processing of colour television signals in connection with recording
• CPC Class Codes: H04N 9/7973 by dividing the luminance or colour component signal samples or frequency bands among a plurality of recording channels H04N9/804, H04N9/825 take precedence

Abstract

The disclosure includes a system and method for distinguishing objects of similar appearance that vary in color. An image recognition application receives a query image of a plurality of objects, determines a candidate list of indexed images that match an object in the query image based on luminance features, crops an image of the object from the query image to generate a cropped image, computes color features for the object, ranks the candidate list of indexed images based on the color features, and provides the candidate list of indexed images to a user.

Claims

1. A method comprising:

receiving, by one or more processors, a query image of a plurality of objects;

determining, by the one or more processors, a candidate list of indexed images that match an object in the query image based on luminance features;

cropping, by the one or more processors, an image of the object from the query image to generate a cropped image;

computing, by the one or more processors, a color transfer function using sample colors collected from the cropped image and an indexed image of the candidate list;

ranking, by the one or more processors, the candidate list of indexed images based on one or more distance measures computed using the color transfer function; and

providing, by the one or more processors, the ranked candidate list of indexed images to a user.

2. The method of claim 1, further comprising:

determining, by the one or more processors, a region of interest for the object in the query image based on the luminance features; and

determining, by the one or more processors, a region of interest for the indexed image based on the luminance features.

3. The method of claim 2, wherein computing the color transfer function further comprises collecting the sample colors from the region of interest determined for the cropped image and the region of interest determined for the indexed image.

4. The method of claim 3, wherein computing the color transfer function further comprises:

applying, by the one or more processors, the color transfer function to the cropped image to generate a transformed cropped image;

applying, by the one or more processors, the color transfer function to the indexed image to generate a transformed indexed image;

computing, by the one or more processors, a first distance measure between the cropped image and the transformed indexed image; and

computing, by the one or more processors, a second distance measure between the indexed image and the transformed cropped image.

5. The method of claim 4, wherein ranking the candidate list of indexed images is based on the first distance measure and the second distance measure.

6. The method of claim 4, wherein the first distance and the second distance are an earth mover'"'"'s distance.

7. The method of claim 1, further comprising filtering the candidate list of indexed images based on the ranking.

8. A system comprising;

one or more processors; and

a memory, the memory storing instructions, which when executed cause the one or more processor to;

receive a query image of a plurality of objects;

determine a candidate list of indexed images that match an object in the query image based on luminance features;

crop an image of the object from the query image to generate a cropped image;

compute a color transfer function using sample colors collected from the cropped image and an indexed image of the candidate list;

rank the candidate list of indexed images based on one or more distance measures computed using the color transfer function; and

provide the ranked candidate list of indexed images to a user.

9. The system of claim 8, wherein the instructions further cause the one or more processors to:

determine a region of interest for the object in the query image based on the luminance features; and

determine a region of interest for the indexed image based on the luminance features.

10. The system of claim 9, wherein to compute the color transfer function, the instructions cause the one or more processors to collect the sample colors from the region of interest determined for the cropped image and the region of interest determined for the indexed image.

11. The system of claim 10, wherein to compute the color transfer function, the instructions cause the one or more processors to:

apply the color transfer function to the cropped image to generate a transformed cropped image;

apply the color transfer function to the indexed image to generate a transformed indexed image;

compute a first distance measure between the cropped image and the transformed indexed image; and

compute a second distance measure between the indexed image and the transformed cropped image.

12. The system of claim 11, wherein the instructions cause the one or more processors to rank the candidate list of indexed images based on the first distance measure and the second distance measure.

13. The system of claim 11, wherein the first distance measure and the second distance measure are a Pearson correlation coefficient.

14. The system of claim 8, wherein the instructions further cause the one or more processors to filter the candidate list of indexed images based on the ranking.

15. A computer program product comprising a non-transitory computer readable medium storing a computer readable program, wherein the computer readable program when executed causes a computer to perform operations comprising:

receiving a query image of a plurality of objects;

determining a candidate list of indexed images that match an object in the query image based on luminance features;

cropping an image of the object from the query image to generate a cropped image;

computing a color transfer function using sample colors collected from the cropped image and an indexed image of the candidate list;

ranking the candidate list of indexed images based on one or more distance measures computed using the color transfer function; and

providing the ranked candidate list of indexed images to a user.

16. The computer program product of claim 15, wherein the computer readable program further causes the computer to:

determine a region of interest for the object in the query image based on the luminance features; and

determine a region of interest for the indexed image based on the luminance features.

17. The computer program product of claim 16, wherein computing the color transfer function further comprises collecting the sample colors from the region of interest determined for the cropped image and the region of interest determined for the indexed image.

18. The computer program product of claim 17, wherein computing the color transfer function further comprises:

applying the color transfer function to the cropped image to generate a transformed cropped image;

applying the color transfer function to the indexed image to generate a transformed indexed image;

computing a first distance measure between the cropped image and the transformed indexed image; and

computing a second distance measure between the indexed image and the transformed cropped image.

19. The computer program product of claim 18, wherein ranking the candidate list of indexed images is based on the first distance measure and the second distance measure.

20. The computer program product of claim 15, wherein the computer readable program further causes the computer to filter the candidate list of indexed images based on the ranking.