Methods and apparatus for object recognition using textons

• US 7,680,330 B2
• Filed: 11/03/2004
• Issued: 03/16/2010
• Est. Priority Date: 11/14/2003
• Status: Expired due to Fees

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Leung, Thomas King-Hong
• Primary Examiner(s): Strege; John B
• Application Number:
  US10/979,175
• Publication Number:
  US 20050147302A1
• Time in Patent Office: 1,959 Days
• Field of Search: 382/190, 382/118, 382/181, 382/225
• US Class Current: 382/181
• CPC Class Codes: G06V 10/424 Syntactic representation, e.g. by using alphabets or grammars
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations

Abstract

Methods and an apparatuses for automatically recognizing and/or verifying objects in a digital image are presented. In one example, a method automatically recognizes objects in digital image data by detecting an object of interest in input digital image data, obtaining a normalized object of interest, assigning texton representations of the normalized object of interest to produce a first a texton array, and determining a similarity between the texton representations and previously determined texton representations of at least one other object. In another example, an apparatus for automatically recognizing objects in digital image data is presented, which includes an image processing control-operably coupled to memory and functional processing units for controlling recognition processing, where the functional processing units further include an object detection unit, a normalizing unit, a texton generation unit, and a similarity unit.

Claims

1. A method for automatically recognizing objects in digital image data, comprising:

detecting an object of interest in input digital image data;

obtaining a normalized object of interest;

assigning texton representations of the normalized object of interest to produce a first a texton array;

determining a similarity between the texton representations and previously determined texton representations of at least one other object;

detecting and normalizing a plurality of reference objects from a plurality of images;

obtaining a first plurality of filter responses based upon pixels forming the plurality of normalized reference objects;

forming a first plurality of vectors based upon the first plurality of filter responses; and

clustering the first plurality of vectors into a plurality of groups, wherein each group is assigned a texton label.

2. The method according to claim 1, further comprising:

obtaining a second plurality of filter responses based upon pixels forming the normalized object of interest;

forming a second plurality of vectors based upon the second plurality of filter responses; and

creating a first texton label array by assigning the texton labels to each of the pixels forming the normalized object of interest.

3. The method according to claim 2, wherein the first and second plurality of filter responses are obtained using horizontal and vertical derivatives of a plurality of circular symmetric Gaussian filters, further wherein the plurality of circular symmetric Gaussian filters comprise a plurality of scales.

4. The method according to claim 3, wherein the circular symmetric Gaussian filters are separable.

5. The method according to claim 3, wherein the each of the first and second plurality of vectors has a length based upon the number of filters used to obtain the first and second filter responses.

6. The method according to claim 1, wherein the clustering further comprises a K-means clustering algorithm.

7. The method according to claim 1, further comprising:

forming a plurality of texton arrays from images belonging to a training set taken from the plurality of normalized reference objects;

selecting a pair of texton arrays from the plurality of texton arrays;

determining, for each element in the pair of texton arrays, whether a texton label pair are in a same class;

incrementing an element by one in a first conditional probability table based upon the determining finding the texton label pair are in the same class;

incrementing an element by one in a second conditional probability table based upon the determining finding the texton label pair are in a different class;

repeating the selecting, the determining, and each incrementing until all of the texton array pairs have been processed; and

normalizing the first and second conditional probability tables.

8. The method according to claim 7, wherein the training set is based upon in-class image variations.

9. The method according to claim 8, wherein the training set is based upon images of the same reference object under different imaging conditions.

10. The method according to claim 7, wherein the first and second conditional probability tables are computed based upon the assumption that texton labels are independent of location in a reference object from said plurality of reference objects.

11. The method according to claim 7, wherein the similarity measure is based upon a likelihood ratio of the first and second probabilities.

12. The method according to claim 7, further comprising:

assigning texton labels to a probe image to produce a second texton array;

determining first probabilities, for each location in the first and second texton arrays, that texton values of the normalized object of interest and the probe image are the same using the first conditional probability table;

determining probability of total similarity by multiplying all of the first probabilities;

determining second probabilities, for each location in the first and second texton array, that texton values of the normalized object of interest and the probe image are different using the second conditional probability table;

determining probability of total dissimilarity by multiplying all of the second probabilities;

computing a likelihood ratio using the probability of total similarity and probability of total dissimilarity; and

determining a similarity between the normalized object of interest and the probe image using the likelihood ratio.

13. The method according to claim 1, further comprising:

assigning texton labels to the plurality of normalized reference objects to create a plurality of texton arrays;

transforming the plurality of texton arrays into a vector; and

computing a linear discriminant analysis using the vector to form basis vectors.

14. The method according to claim 13, wherein the linear discriminant analysis is a Fisher linear discriminant analysis and the basis vectors are Fisher Textons.

15. The method according to claim 14, further comprising:

generating a first vector using the first texton array;

determining first projections by projecting the first vector onto the Fisher Textons;

assigning texton labels to a probe image to produce a second texton array;

generating a second vector using the second texton array;

determining second projections by projecting the second vector onto the Fisher Textons; and

determining a similarity between the normalized object of interest and the probe image using the first and second projections.

16. The method according to claim 1, wherein the object of interest is a face and the method further comprises performing face recognition.

17. The method according to claim 1, wherein the object of interest is a face and the method further comprises performing face verification.

18. The method of claim 1, wherein the object of interest is a face and the texton representations encode facial features.

19. The method of claim 7, further comprising:

generating a training set to distinguish facial features which discriminate within a class.

20. The method according to claim 7, further comprising:

generating a training set to distinguish facial features which discriminate across at least one different class.

41. The method according to claim 1, wherein said assigning step assigns texton representations of the normalized object of interest, using textons from a predetermined set of textons representing local surface characteristics for pixels of said normalized object of interest, to produce said first texton array.

42. The method according to claim 1, wherein said assigning step assigns texton representations of the normalized object of interest to produce said first texton array using labels for individual pixels of said normalized object of interest, the labels being selected from a set of texton labels which has a predetermined size.

43. The method according to claim 1, wherein said assigning step assigns texton representations of the normalized object of interest to produce said first texton array in which values corresponding to individual pixels of said normalized object of interest depend upon individual textons from a texton vocabulary and wherein said textons describe local surface characteristics of said pixels.

21. An apparatus for automatically recognizing objects in digital image data, comprising:

an image processing control operably coupled to memory and functional processing units for controlling recognition processing, wherein the functional processing units comprise;

an object detection unit for detecting an object of interest in input digital image data;

a normalizing unit for obtaining a normalized object of interest;

a texton generation unit for assigning texton representations of the normalized object of interest to produce a first texton array; and

a similarity unit for determining a similarity between the texton representations and previously determined texton representations of at least one other object,

wherein the detecting unit and normalizing unit detects and normalizes, respectively, a plurality of reference objects from a plurality of images, and further wherein the texton generation unit

obtains a first plurality of filter responses based upon pixels forming the plurality of normalized reference objects,

forms a first plurality of vectors based upon the first plurality of filter responses, and

clusters the first plurality of vectors into a plurality of groups, wherein each group is assigned a texton label.

22. The apparatus according to claim 21, wherein the texton generation unit

obtains a second plurality of filter responses based upon pixels forming the normalized object of interest,

forms a second plurality of vectors based upon the second plurality of filter responses, and

creates a first texton label array by assigning the texton labels to each of the pixels forming the normalized object of interest.

23. The apparatus according to claim 22, wherein the first and second plurality of filter responses are obtained using horizontal and vertical derivatives of a plurality of circular symmetric Gaussian filters, further wherein the plurality of Gaussian filters comprise a plurality of scales.

24. The apparatus according to claim 23, wherein the Gaussian filters are separable.

25. The apparatus according to claim 23, wherein the each of the first and second plurality of vectors has a length based upon the number of filters used to obtain the first and second filter responses.

26. The apparatus according to claim 21, wherein the clustering further comprises a K-means clustering algorithm.

27. The apparatus according to claim 21, wherein the texton generation unit forms a plurality of texton arrays from images belonging to a training set taken from the plurality of normalized reference objects, and further wherein the similarity determination unit

selects a pair of texton arrays from the plurality of texton arrays,

determines, for each element in the pair of texton arrays, whether the texton label pair are in a same class,

increments an element by one in a first conditional probability table based upon the determining finding the texton label pair are in the same class,

increments an element by one in a second conditional probability table based upon the determining finding the texton label pair are in a different class,

repeats the selecting, the determining, and each incrementing until all of the texton array pairs have been processed, and

normalizes the first and second conditional probability tables.

28. The apparatus according to claim 27, wherein the training set is based upon in-class image vanations.

29. The apparatus according to claim 28, wherein the training set is based upon images of the same object under different imaging conditions.

30. The apparatus according to claim 27, wherein the first and second conditional probability tables are computed based upon the assumption that texton labels are independent of location in the object of interest.

31. The apparatus according to claim 27, wherein the similarity measure is based upon a likelihood ratio of the first and second probabilities.

33. The apparatus according to claim 21, wherein

the texton generation unit assigns texton labels to the plurality of normalized reference objects to create a plurality of texton arrays, and further wherein

the similarity determination unit transforms the plurality of texton arrays into a vector, and

computes a linear discriminant analysis using the vector to from basis vectors.

34. The apparatus according to claim 33, wherein the linear discriminant analysis is a Fisher linear discriminant analysis and the basis vectors are Fisher Textons.

35. The apparatus according to claim 34, wherein the similarity determination unit

generates a first vector using the first texton array,

determines first projections by projecting the first vector onto the Fisher Textons,

assigns texton labels to a probe image to produce a second texton array,

generates a second vector using the second texton array,

determines second projections by projecting the second vector onto the Fisher Textons, and

determines a similarity between the normalized object of interest and the probe image using the first and second projections.

36. The apparatus according to claim 21, wherein the object of interest is a face and the similarity determination unit performs face recognition.

37. The apparatus according to claim 21, wherein the object of interest is a face and the similarity determination unit performs face verification.

38. The apparatus according to claim 21, wherein the object of interest is a face and the texton representations encode facial features.

39. The apparatus of claim 27, wherein the similarity determination unit generates a training set to distinguish facial features which discriminate within a class.

40. The apparatus according to claim 27, wherein the similarity determination unit

generates a training set to distinguish facial features which discriminate across at least one different class.

44. The apparatus according to claim 21, wherein said texton generation unit assigns texton representations of the normalized object of interest, using textons from a predetermined set of textons representing local surface characteristics for pixels of said normalized object of interest, to produce said first texton array.

45. The apparatus according to claim 21, wherein said texton generation unit assigns texton representations of the normalized object of interest to produce said first texton array using labels for individual pixels of said normalized object of interest, the labels being selected from a set of texton labels which has a predetermined size.

46. The apparatus according to claim 21, wherein said texton generation unit assigns texton representations of the normalized object of interest to produce said first texton array in which values corresponding to individual pixels of said normalized object of interest depend upon individual textons from a texton vocabulary and wherein said textons describe local surface characteristics of said pixels.

32. An apparatus for automatically recognizing objects in digital image data, comprising:

an image processing control operably coupled to memory and functional processing units for controlling recognition processing, wherein the functional processing units comprise;

an object detection unit for detecting an object of interest in input digital image data;

a normalizing unit for obtaining a normalized object of interest;

a texton generation unit for assigning texton representations of the normalized object of interest to produce a first texton array; and

a similarity unit for determining a similarity between the texton representations and previously determined texton representations of at least one other object,

wherein the similarity determination unit

assigns texton labels to a probe image to produce a second texton array,

determines first probabilities, for each location in the first and second texton arrays, that texton values of the normalized object of interest and the probe image are the same using the first conditional probability table,

determines a probability of total similarity by multiplying all of the first probabilities;

determines second probabilities, for each location in the first and second texton array, that texton values of the normalized object of interest and the probe image are different using the second conditional probability table,

determines a probability of total dissimilarity by multiplying all of the second probabilities,

computes a likelihood ratio using the probability of total similarity and probability of total dissimilarity, and

determines a similarity between the normalized object of interest and the model image using the likelihood ratio.