Methods and systems for occlusion tolerant face recognition

• US 8,452,107 B2
• Filed: 10/02/2009
• Issued: 05/28/2013
• Est. Priority Date: 10/02/2009
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Zheng, Thomas, Liu, Yinyin, Bender, Paul E.
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Dunphy, David F
• Application Number:
  US12/573,051
• Publication Number:
  US 20110081053A1
• Time in Patent Office: 1,334 Days
• Field of Search: None
• US Class Current: 382/197
• CPC Class Codes: G06F 18/2135 based on approximation criteria, e.g. principal component analysis
• CPC Class Codes: G06F 18/2136 based on sparsity criteria, e.g. with an overcomplete basis
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Certain embodiments of the present disclosure relate to a method for face recognition that is occlusion tolerant and scale/shift invariant based on a combination of hierarchical maximization and adaptive representation technique.

Claims

1. A method for signal processing, comprising:

filtering a signal related to a subject of a plurality of subjects;

adaptively generating a signal subspace according to the subject;

finding a sparse solution associated with the filtered signal by using the signal subspace;

performing identification of the subject based on the sparse solution and the signal subspace, if sparsity of the sparse solution is greater than a defined threshold value;

updating statistics of bases of the filtered signal in a set of bases, if identity of the subject is confirmed; and

adaptively inserting one or more new bases into the set of bases until the identity of the subject is confirmed, if the sparsity of the sparse solution is greater than the defined threshold value.

2. The method of claim 1, further comprising:

finding another sparse solution associated with another filtered signal, if the sparsity of the sparse solution is greater than the defined threshold value and if identity of the subject is not confirmed.

3. The method of claim 1, further comprising:

adaptively inserting one or more new bases into the set of bases and finding another sparse solution associated with the filtered signal, if the sparsity of the sparse solution is not greater than the defined threshold value.

4. The method of claim 3, wherein adaptively inserting the one or more new bases comprises:

performing spatial sampling of a training signal associated with the subject according to an adaptive probability density function (PDF) to generate one or more patches of the training signal;

tagging the generated patches;

expanding each of the tagged patches; and

inserting each of the tagged patches into the set, if the tagged patch is not linearly dependent with patches from the set and if a number of the patches in the set is smaller than a defined maximum number of patches.

5. The method of claim 4, further comprising:

finding, in a portion of the set with linearly dependent bases according to their cumulative scores, a least effective patch for the subject identification, if the tagged patch is linearly dependent with the patches in the set; and

replacing, in the set, the found patch with the tagged patch.

6. The method of claim 4, further comprising:

removing, from the set, one or more least effective patches for the subject identification according to their cumulative scores, wherein less effective patch for the subject identification is associated with a smaller cumulative score; and

inserting the tagged patch into the set, if the tagged patch is not linearly dependent with the patches in the set and if the number of patches in the set is not smaller than the defined maximum number of patches.

7. The method of claim 1, wherein updating statistics of the bases in the set comprises:

updating a cumulative score for each basis in the set based on the found sparse solution, wherein less effective basis for the subject identification is associated with a smaller cumulative score;

ranking the bases in the set by their cumulative scores, wherein a higher ranked basis in the set is more effective for the subject identification;

updating another cumulative score for each basis in the set based on a location of the basis in the signal subspace; and

adapting a probability density function (PDF) for spatial sampling of the filtered signal based on the updated other cumulative scores associated with all basis in the set.

8. The method of claim 1, wherein performing identification of the subject comprises:

reconstructing identity signals based on the sparse solution and the signal subspace;

finding, based on the reconstructed identity signals, a minimum reconstruction error by using bases for the plurality of subjects; and

obtaining the identity of the subject of the plurality of subjects according to the found minimum reconstruction error.

9. The method of claim 1, wherein finding the sparse solution associated with the filtered signal comprises:

performing reconstruction of an image related to the filtered signal by using the signal subspace; and

obtaining the sparse solution by using the reconstructed image according to the l₁-norm minimization scheme.

10. The method of claim 9, wherein adaptively generating the signal subspace associated with the filtered signal comprises:

removing all empty rows from the signal subspace;

deleting each row from the signal subspace, if the row is under-represented in the signal subspace; and

preserving the row and removing all empty columns from the signal subspace, if the row is not under-represented in the signal subspace.

11. An apparatus for signal processing, comprising:

an interface for generating a signal related to a subject of a plurality of subjects; and

a recognition engine configured to filter the signal, adaptively generate a signal subspace according to the subject, find a sparse solution associated with the filtered signal by using the signal subspace, perform identification of the subject based on the sparse solution and the signal subspace if sparsity of the sparse solution is greater than a defined threshold value, update statistics of bases of the filtered signal in the set of bases, if identity of the subject is confirmed, and adaptively insert one or more new bases into the set of bases until the identity of the subject is confirmed, if the sparsity of the sparse solution is greater than the defined threshold value.

12. The apparatus of claim 11, wherein the signal corresponds to an image of a face of a person.

13. The apparatus of claim 11, wherein the recognition engine is further configured to:

find another sparse solution associated with another filtered signal, if the sparsity of the sparse solution is greater than the defined, threshold value and if identity of the subject is not confirmed.

14. The apparatus of claim 11, wherein the recognition engine is further configured to:

adaptively insert one or more new bases into the set of bases and find another sparse solution associated with the filtered signal, if the sparsity of the sparse solution is not greater than the defined threshold value.

15. The apparatus of claim 14, wherein the recognition engine is further configured to:

perform spatial sampling, of a training signal associated with the subject according to an adaptive probability density function (PDF) to generate one or more patches of the training signal;

tag the generated patches;

expand each of the tagged patches; and

insert each of the tagged patches into the set, if the tagged patch is not linearly dependent with patches from the set and if a number of the patches in the set is smaller than a defined maximum number of patches.

16. The apparatus of claim 15, wherein the recognition engine is further configured to:

find, in a portion of the set with linearly dependent bases according to their cumulative scores, a least effective patch for the subject identification, if the tagged patch is linearly dependent with the patches in the set; and

replace, in the set, the found patch with the tagged patch.

17. The apparatus of claim 15, wherein the recognition engine is further configured to:

remove, from the set, one or more least effective patches for the subject identification according to their cumulative scores, wherein less effective patch for the subject identification is associated with a smaller cumulative score; and

insert the tagged patch into the set, if the tagged patch is not linearly dependent with the patches in the set and if the number of patches in the set is not smaller than the defined maximum number of patches.

18. The apparatus of claim 11, wherein the recognition engine is configured to update statistics of the bases in the set by:

updating a cumulative score for each basis in the set based on the found sparse solution, wherein less effective basis for the subject identification is associated with a smaller cumulative score;

ranking the bases in the set by their cumulative scores, wherein a higher ranked basis in the set is more effective for the subject identification;

updating another cumulative score for each basis in the set based on a location of the basis in the signal subspace; and

adapting a probability density function (PDF) for spatial sampling of the filtered, signal based on the updated other cumulative scores associated with all basis in the set.

19. The apparatus of claim 11, wherein the recognition engine is configured to perform identification of the subject by:

reconstructing identity signals based on the sparse solution and the signal subspace;

finding, based on the reconstructed identity signals, a minimum reconstruction error by using bases for the plurality of subjects; and

obtaining the identity of the subject of the plurality of subjects according to the found minimum reconstruction error.

20. The apparatus of claim 11, wherein the recognition engine is further configured to find the sparse solution associated with the filtered signal by:

performing reconstruction of an image related to the filtered signal by using the signal subspace; and

obtaining the sparse solution by using the reconstructed image according to the l₁-norm minimization scheme.

21. An apparatus for signal processing, comprising:

means for filtering a signal related to a subject of a plurality of subjects;

means for adaptively generating a signal subspace according to the subject;

means for finding a sparse solution associated with the filtered signal by using the signal subspace;

means for performing identification of the subject based on the sparse solution and the signal subspace, if sparsity of the sparse solution is greater than a defined threshold value;

means for updating statistics of bases of the filtered signal in a set of bases, if identity of the subject is confirmed; and

means for adaptively inserting one or more new bases into the set of bases until the identity of the subject is confirmed, if the sparsity of the sparse solution is greater than the defined threshold value.

22. The apparatus of claim 21, further comprising:

means for finding another sparse solution associated with another filtered signal, if the sparsity of the sparse solution is greater than the defined threshold value and if identity of the subject is not confirmed.

23. The apparatus of claim 21, further comprising:

means for adaptively inserting one or more new bases into the set of bases and means for finding another sparse solution associated with the filtered signal, if the sparsity of the sparse solution is not greater than the defined threshold value.

24. The apparatus of claim 23, wherein the means for adaptively inserting the one or more new bases comprises:

means for performing spatial sampling of a training signal associated with the subject according to an adaptive probability density function (PDF) to generate one or more patches of the training signal;

means for tagging the generated patches;

means for expanding each of the tagged patches; and

means for inserting each of the tagged patches into the set, if the tagged patch is not linearly dependent with patches from the set and if a number of the patches in the set is smaller than a defined maximum number of patches.

25. The apparatus of claim 24, further comprising:

means for finding, in a portion of the set with linearly dependent bases according to their cumulative scores, a least effective patch for the subject identification, if the tagged patch is linearly dependent with the patches in the set; and

means for replacing, in the set, the found patch with the tagged patch.

26. The apparatus of claim 24, further comprising:

means for removing, from the set, one or more least effective patches for the subject identification according to their cumulative scores, wherein less effective patch for the subject identification is associated with a smaller cumulative score; and

means for inserting the tagged patch into the set, if the tagged patch is not linearly dependent with the patches in the set and if the number of patches in the set is not smaller than the defined maximum number of patches.

27. The apparatus of claim 21, wherein the means for updating statistics of the bases in the set comprises:

means for updating a cumulative score for each basis in the set based on the found sparse solution, wherein less effective basis for the subject identification is associated with a smaller cumulative score;

means for ranking the bases in the set by their cumulative scores, wherein a higher ranked basis in the set is more effective for the subject identification;

means for updating another cumulative score for each basis in the set based on a location of the basis in the signal subspace; and

means for adapting a probability density function (PDF) for spatial sampling of the filtered signal based on the updated other cumulative scores associated with all basis in the set.

28. The apparatus of claim 21, wherein the means for performing identification of the subject comprises;

means for reconstructing identity signals based on the sparse solution and the signal subspace;

means for finding, based on the reconstructed identity signals, a minimum reconstruction error by using bases for the plurality of subjects; and

means for obtaining the identity of the subject of the plurality of subjects according to the found minimum reconstruction error.

29. The apparatus of claim 21, wherein the means for finding the sparse solution associated with the filtered signal comprises:

means for performing reconstruction of an image related to the filtered signal by using the signal subspace; and

means for obtaining the sparse solution by using the reconstructed image according to the l₁-norm minimization scheme.

30. The apparatus of claim 29, wherein the means for adaptively generating the signal subspace associated with the filtered signal comprises:

means for removing all empty rows from the signal subspace;

means for deleting each row from the signal subspace, if the row is under-represented in the signal subspace; and

means for preserving the row and removing all empty columns from the signal subspace, it the row is not under-represented in the signal subspace.

31. A computer-program product for signal processing, comprising a non transitory computer-readable medium having instructions stored thereon, the instructions being executable by one or more processors and the instructions comprising:

instructions for filtering a signal related to a subject of a plurality of subjects;

instructions for adaptively generating a signal subspace according to the subject;

instructions for finding a sparse solution associated with the filtered signal by using the signal subspace;

instructions for performing identification of the subject based on the sparse solution and the signal subspace, if sparsity of the sparse solution is greater than a defined threshold value;

instructions for updating statistics of bases of the filtered signal in a set of bases, if identity of the subject is confirmed; and

instructions for adaptively inserting one or more new bases into the set of bases until the identity of the is confirmed, if the sparsity of the sparse solution is greater than the defined threshold value.

32. The computer-program product of claim 31, wherein the instructions further comprise:

instructions for finding another sparse solution associated with another filtered signal, if the sparsity of the sparse solution is greater than the defined threshold value and if identity of the subject is not confirmed.

33. The computer-program product of claim 31, wherein the instructions further comprise:

instructions for adaptively inserting one or more new bases into the set of bases and instructions for finding another sparse solution associated with the filtered signal, if the sparsity of the sparse solution is not greater than the defined threshold value.

34. The computer-program product of claim 33, wherein the instructions for adaptively inserting the one or more new bases comprise:

instructions for performing spatial sampling of a training signal associated with the subject according to an adaptive probability density function (PDF) to generate one or more patches of the training signal;

instructions for tagging the generated patches;

instructions for expanding each of the tagged patches; and

instructions for inserting each of the tagged patches into the set, if the tagged patch is not linearly dependent with patches from the set and if a number of the patches in the set is smaller than a defined maximum number of patches.

35. The computer-program product of claim 34, wherein the instructions further comprise:

instructions for finding, in a portion of the set with linearly dependent bases according to their cumulative scores, a least effective patch for the subject identification, if the tagged patch is linearly dependent with the patches in the set; and

instructions for replacing, in the set, the found patch with the tagged patch.

36. The computer-program product of claim 34, wherein the instructions further comprise:

instructions for removing, from the set, one or more least effective patches for the subject identification according to their cumulative scores, wherein less effective patch for the subject identification is associated with a smaller cumulative score; and

instructions for inserting the tagged patch into the set, if the tagged patch is not linearly dependent with the patches in the set and if the number of patches in the set is not smaller than the defined maximum number of patches.

37. The computer-program product of claim 31, wherein the instructions for updating statistics of the bases in the set comprise:

instructions for updating a cumulative score for each basis in the set based on the found sparse solution, wherein less effective basis for the subject identification is associated with a smaller cumulative score;

instructions for ranking the bases in the set by their cumulative scores, wherein a higher ranked basis in the set is more effective for the subject identification;

instructions for updating another cumulative score for each basis in the set based on a location of the basis in the signal subspace; and

instructions for adapting a probability density function (PDF) for spatial sampling of the filtered signal based on the updated other cumulative scores associated with all basis in the set.

38. The computer-program product of claim 31, wherein the instructions for performing identification of the subject comprise:

instructions for reconstructing identity signals based on the sparse solution and the signal subspace;

instructions for finding, based on the reconstructed identity signals, a minimum reconstruction error by using bases for the plurality of subjects; and

instructions for obtaining the identity of the subject of the plurality of subjects according to the found minimum reconstruction error.

39. The computer-program product of claim 31, wherein the instructions for finding the sparse solution associated with the filtered signal comprise:

instructions for performing reconstruction of an image related to the filtered signal by using the signal subspace; and

instructions for obtaining the sparse solution by using the reconstructed image according to the l₁-norm minimization scheme.

40. The computer-program product of claim 39, wherein the instructions for adaptively generating the signal subspace associated with the filtered signal comprise:

instructions for removing all empty rows from the signal subspace;

instructions for deleting each row from the signal subspace, if the row is under-represented in the signal subspace; and

instructions for preserving the row and removing all empty columns from the signal subspace, if the row is not under-represented in the signal subspace.