Generating and utilizing normalized scores for classifying digital objects

• US 9,846,822 B2
• Filed: 12/31/2015
• Issued: 12/19/2017
• Est. Priority Date: 12/31/2015
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Kriegman, David J., Belhumeur, Peter N., Berg, Thomas, Welinder, Nils Peter
• Primary Examiner(s): WOLDEMARIAM, AKLILU K
• Application Number:
  US14/986,585
• Publication Number:
  US 20170193337A1
• Time in Patent Office: 719 Days
• Field of Search: 382118, 382124, 382160, 382255, 382103, 382190, 382197, 382100
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data

Abstract

The present disclosure is directed toward systems and methods that enable more accurate digital object classification. In particular, disclosed systems and methods address inaccuracies in digital object classification introduced by variations in classification scores. Specifically, in one or more embodiments, disclosed systems and methods generate probability functions utilizing digital test objects and transform classifications scores into normalized classification scores utilizing probability functions. Disclosed systems and methods utilize normalized classification scores to more accurately classify and identify digital objects in a variety of applications.

Claims

1. A method comprising:

identifying a set of one or more digital training images tagged with information identifying a known object, the known object portrayed in each image in the set of one or more tagged digital training images;

generating, by at least one processor and utilizing the information identifying the known object, a classification score with regard to an unknown object portrayed in a probe digital image, the classification score indicating a likelihood that the unknown object portrayed in the probe digital image corresponds to the known object portrayed in the set of one or more tagged digital training images;

transforming the classification score into a normalized classification score based on the number of tagged digital training images in the set of one or more tagged digital training images; and

determining whether the unknown object portrayed in the probe digital image corresponds to the known object portrayed in the set of one or more tagged digital training images by comparing the normalized classification score with a second normalized classification score corresponding to a second known object portrayed in a second set of one or more tagged digital training images and generated based on the number of tagged digital training images in the second set of one or more tagged digital training images.

2. The method of claim 1, wherein the normalized classification score comprises a probability that the known object portrayed in the set of one or more digital training images is the unknown object portrayed in the probe digital image.

3. The method of claim 1, wherein transforming the classification score into a normalized classification score based on the number of tagged digital training images further comprises:

calculating a probability that, assuming the unknown object portrayed in the probe digital image corresponds to the known object portrayed in the set of one or more tagged digital training images, the generated classification score would result based on the number of tagged digital training images.

4. The method of claim 1, wherein transforming the classification score into a normalized classification score based on the number of tagged digital training images further comprises:

generating a probability function from a repository of digital test images, the probability function reflecting the probability of generating classification scores given the number of tagged digital training images; and

transforming the classification score into the normalized classification score based on the probability function.

5. The method of claim 4, further comprising:

identifying the second set of one or more tagged digital training images, wherein the second set of one or more tagged digital training images is tagged with information identifying the second known object, the second known object portrayed in each image in the second set of one or more tagged digital training images;

generating, by the at least one processor, a second classification score with regard to the unknown object portrayed in the probe digital image, the second classification score indicating a likelihood that the unknown object portrayed in the probe digital image corresponds to the second known object portrayed in the second set of one or more tagged digital training images;

transforming the second classification score into the second normalized classification score based on the number of tagged digital training images in the second set of one or more tagged digital training images; and

based on the normalized classification score and the second normalized classification score, determining whether the unknown object portrayed in the probe digital image corresponds to the known object portrayed in the set of one or more tagged digital training images.

6. The method of claim 5, wherein:

transforming the classification score based on the number of tagged digital training images in the set of one or more tagged digital training images comprises utilizing a first probability function specific to the number of tagged digital training images in the set of one or more tagged digital training images to transform the classification score into the normalized classification score; and

transforming the second classification score based on the number of tagged digital training images in the second set of one or more tagged digital training images comprises utilizing a second probability function specific to the number of tagged digital training images in the second set of one or more tagged digital training images to transform the second classification score into the second normalized classification score.

7. The method of claim 5, wherein

transforming the classification score into the normalized classification score based on the number of tagged digital training images in the set of one or more tagged digital training images further comprises:

generating an imposter probability function, the imposter probability function reflecting the probability of returning classification scores based on the number of tagged digital training images in the set of one or more tagged digital training images assuming that the unknown object portrayed in the probe digital image does not correspond to the known object portrayed in the set of one or more tagged digital training images; and

transforming the second classification score into the second normalized classification score based on the number of tagged digital training images in the second set of one or more tagged digital training images further comprises;

generating a second imposter probability function, the second imposter probability function reflecting the probability of returning classification scores based on the number of tagged digital training images in the second set of one or more tagged digital training images assuming that the unknown object portrayed in the probe digital image does not correspond to the known object portrayed in the second set of one or more tagged digital training images.

8. The method of claim 7,

wherein transforming the classification score into a normalized classification score further comprises:

calculating a probability of generating the classification score based on the number of tagged digital training images in the set of one or more tagged digital training images utilizing the probability function; and

calculating a probability of generating the classification score based on the number of tagged digital training images in the set of one or more tagged digital training images utilizing the imposter probability function; and

wherein transforming the second classification score into a second normalized classification score further comprises;

calculating a probability of generating the second classification score based on the number of tagged digital training images in the second set of one or more tagged digital training images utilizing the second probability function; and

calculating a probability of generating the second classification score based on the number of tagged digital training images in the second set of one or more tagged digital training images utilizing the second imposter probability function.

9. The method of claim 4, wherein generating the probability function from the repository of digital test images comprises:

training a plurality of test training models from the repository of digital test images, each test training model being trained with a number of digital test images corresponding to the number of tagged digital training images;

generating a plurality of test classification scores utilizing the plurality of test training models;

generating a histogram of the plurality of test classification scores; and

generating the probability function based on the generated histogram.

10. A system comprising:

at least one processor; and

at least one non-transitory computer readable storage medium storing instructions that, when executed by the at least one processor, cause the system to;

identify a set of one or more digital training images tagged with information identifying a known object, the known object portrayed in each image in the set of one or more tagged digital training images;

generate, utilizing the information identifying the known object, a classification score with regard to an unknown object portrayed in a probe digital image, the classification score indicating a likelihood that the unknown object portrayed in the probe digital image corresponds to the known object portrayed in the set of one or more tagged digital training images;

transform the classification score into a normalized classification score based on the number of tagged digital training images in the set of one or more tagged digital training images; and

determine whether the unknown object portrayed in the probe digital image corresponds to the known object portrayed in the set of one or more tagged digital training images by comparing the normalized classification score with a second normalized classification score corresponding to a second known object portrayed in a second set of one or more tagged digital training images and generated based on the number of tagged digital training images in the second set of one or more tagged digital training images.

11. The system of claim 10, wherein transforming the classification score into a normalized classification score based on the number of tagged digital training images further comprises:

calculating a probability that, assuming the unknown object portrayed in the probe digital image corresponds to the known object portrayed in the set of one or more tagged digital training images, the generated classification score would result based on the number of tagged digital training images.

12. The system of claim 10, wherein transforming the classification score into a normalized classification score based on the number of tagged digital training images further comprises:

generating a probability function from a repository of digital test images, the probability function reflecting the probability of generating classification scores given the number of tagged digital training images; and

transforming the classification score into the normalized classification score based on the probability function.

13. The system of claim 10, further comprising instructions that, when executed by the at least one processor, cause the system to:

identify the second set of one or more tagged digital training objects, wherein the second set of one or more tagged digital training images is tagged with information identifying the second known object, the second known object portrayed in each image in the second set of one or more tagged digital training images;

generate a second classification score with regard to the unknown object portrayed in the probe digital image, the second classification score indicating a likelihood that the unknown object portrayed in the probe digital image corresponds to the second known object portrayed in the second set of one or more tagged digital training images;

transform the second classification score into the second normalized classification score based on the number of tagged digital training images in the second set of one or more tagged digital training images; and

based on the normalized classification score and the second normalized classification score, determine whether the unknown object portrayed in the probe digital image corresponds to the known object portrayed in the set of one or more tagged digital training images.

14. The system of claim 13, wherein:

transforming the classification score based on the number of tagged digital training images in the set of one or more tagged digital training images comprises utilizing a first probability function specific to the number of tagged digital training images in the set of one or more tagged digital training images to transform the classification score into the normalized classification score; and

transforming the second classification score based on the number of tagged digital training images in the second set of one or more tagged digital training images comprises utilizing a second probability function specific to the number of tagged digital training images in the second set of one or more tagged digital training images to transform the second classification score into the second normalized classification score.

15. The system of claim 10, wherein transforming the classification score into the normalized classification score based on the number of tagged digital training images in the set of one or more tagged digital training images further comprises:

generating an imposter probability function, the imposter probability function reflecting the probability of returning classification scores based on the number of tagged digital training images in the set of one or more tagged digital training images assuming that the unknown object portrayed in the probe digital image does not correspond to the known object portrayed in the set of one or more tagged digital training images.

16. A method comprising:

identifying a set of one or more digital training objects tagged with information identifying a known classification, the known classification corresponding to each of the digital training objects in the set of one or more tagged digital training objects;

generating, by at least one processor and utilizing the information identifying the known classification, a classification score with regard to an unknown digital object, the classification score indicating a likelihood that the unknown digital object corresponds to the known classification;

transforming the classification score into a normalized classification score based on the number of tagged digital training objects; and

determining whether the unknown digital object corresponds to the known classification by comparing the normalized classification score with a second normalized classification score corresponding to a second known classification of a second set of one or more digital training objects and generated based on the number of tagged digital training objects in the second set of one or more tagged digital training objects.

17. The method of claim 16, wherein transforming the classification score into a normalized classification score based on the number of tagged digital objects further comprises:

calculating a probability that, assuming the unknown digital object corresponds to the known classification, the generated classification score would result based on the number of tagged digital training objects.

18. The method of claim 16, wherein transforming the classification score into a normalized classification score based on the number of tagged digital training objects further comprises:

generating a probability function from a repository of digital test objects, the probability function reflecting the probability of generating classification scores given the number of tagged digital training objects; and

transforming the classification score into the normalized classification score based on the probability function.

19. The method of claim 16, further comprising:

identifying the second set of one or more digital training objects, wherein the second set of one or more digital training objects is tagged with information identifying the second known classification, the second known classification corresponding to each of the digital training objects in the second set of one or more digital objects;

generating, by the at least one processor, a second classification score with regard to the unknown digital object;

transforming the second classification score into the second normalized classification score based on the number of tagged digital training objects in the second set of one or more tagged digital training objects; and

based on the normalized classification score and the second normalized classification score, determining whether the unknown digital object corresponds to the known classification.

20. The method of claim 19, wherein:

transforming the classification score based on the number of tagged digital training objects in the set of one or more tagged digital training objects comprises utilizing a first probability function specific to the number of tagged digital training objects in the set of one or more tagged digital training objects to transform the classification score into the normalized classification score; and

transforming the second classification score based on the number of tagged digital training objects in the second set of one or more tagged digital training objects comprises utilizing a second probability function specific to the number of tagged digital training objects in the second set of one or more tagged digital training objects to transform the second classification score into the second normalized classification score.