Techniques for enabling or establishing the use of face recognition algorithms

• US 8,571,272 B2
• Filed: 03/12/2007
• Issued: 10/29/2013
• Est. Priority Date: 03/12/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gokturk, Salih Burak, Anguelov, Dragomir, Torresani, Lorenzo, Vanhoucke, Vincent, Shah, Munjal, Vu, Diem, Lee, Kuang-Chih
• Primary Examiner(s): Newman, Michael A
• Application Number:
  US11/685,106
• Publication Number:
  US 20070258645A1
• Time in Patent Office: 2,423 Days
• Field of Search: 382/118, 382/115, 382/154, 382/285, 382/300
• US Class Current: 382/118
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Embodiments described herein facilitate or enhance the implementation of image recognition processes which can perform recognition on images to identify objects and/or faces by class or by people.

Claims

1. A non-transitory computer readable medium that stores instructions, the stored instructions including instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

identifying, from first image data, an object;

determining whether one or more regions of the object are occluded;

determining, based on the one or more occluded regions, whether the object is associated with a likelihood of recognition error that exceeds a threshold;

in response to determining that the object is associated with the likelihood of recognition error that exceeds the threshold;

synthesizing second image data for at least some of the one or more occluded regions of the object, wherein synthesizing the second image data includes using pixel values associated with the first image data to generate pixel values for at least some of the second image data corresponding to the one or more occluded regions; and

performing recognition of the object using the synthesized second image data.

2. The computer readable medium of claim 1, wherein the object is a face of a person.

3. The computer readable medium of claim 1, wherein synthesizing the second image data includes using a three-dimensional data model of the object.

4. The computer readable medium of claim 3, wherein synthesizing the second image data includes augmenting the three-dimensional data model of the object with attributes of the first image data.

5. The computer readable medium of claim 4, wherein augmenting the three-dimensional data model includes posing the three-dimensional data model to correspond to a view of the object as provided by the first image data.

6. The computer readable medium of claim 1, wherein synthesizing the second image data includes computing an intensity value of one or more discrete locations of the second image data from an intensity value of one or more corresponding discrete locations associated with the first image data.

7. The computer readable medium of claim 1, wherein the synthesized second image data is presented in a frontal view.

8. The computer readable medium of claim 1, wherein the synthesized second image data is presented in a user specified position.

24. The computer readable medium of claim 1, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations that include:

in response to the determination that the object is associated with the likelihood of recognition error that does not exceed the threshold;

performing recognition of the object using the first image data having the one or more occluded regions.

26. The computer readable medium of claim 1, wherein the instructions for synthesizing the second image data include instructions for using the first image data to generate at least some of the intensity values for the second image data based on one or more assumptions about object symmetry.

27. The computer readable medium of claim 1, wherein the one or more occluded regions include one or more features of the object that are occluded, and wherein the instructions for synthesizing the second image data include instructions for generating the pixel values for the second image data using pixel values associated with at least a portion of the first image data that represents one or more features of the object that are at least similar in appearance to the one or more features that are occluded.

9. A system comprising:

one or more processors; and

a memory coupled to the one or more processors, wherein the memory stores instructions that, when executed by the one or more processors, cause the one or more processors to;

identify, from first image data, an object;

determine whether one or more regions of the object are occluded;

determine, based on the one or more occluded regions, whether the object is associated with a likelihood of recognition error that exceeds a threshold;

in response to determining that the object is associated with the likelihood of recognition error that exceeds the threshold;

use the first image data to synthesize second image data for at least some of the one or more occluded regions of the object, wherein pixel values associated with the first image data are used to generate pixel values for at least some of the second image data; and

perform recognition of the object using the synthesized second image data.

10. The system of claim 9, wherein the memory stores instructions to cause the one or more processors to synthesize the second image data using a three-dimensional data model of the object.

11. The system of claim 10, wherein the memory stores instructions to cause the one or more processors to augment the three-dimensional data model of the object with attributes of the first image data.

12. The system of claim 11, wherein the memory stores instructions to cause the one or more processors to pose the three-dimensional data model that corresponds to a view of the object as provided by the first image data.

13. The system of claim 9 wherein the memory stores instructions to cause the one or more processors to use the first image data to synthesize second image data by computing an intensity value of one or more discrete locations of the second image data from an intensity value of one or more corresponding discrete locations associated with the first image data.

14. The system of claim 9, wherein the synthesized second image data is presented in a frontal view.

15. The system of claim 9, wherein the synthesized second image data is presented in a user specified position.

25. The system of claim 9, wherein the one or more processors are configured to:

in response to the determination that the object is associated with the likelihood of recognition error that does not exceed the threshold, perform recognition of the object using the first image data having the one or more occluded regions.

28. The system of claim 9, wherein the one or more processors use the first image data to generate at least some of the pixel values for the second image data based on one or more assumptions about object symmetry.

29. The system of claim 9, wherein the one or more occluded regions include one or more occluded features of the object, and wherein the one or more processors synthesize the second image data by generating the pixel values for the second image data using pixel values associated with at least a portion of the first image data that represents one or more features of the object that are at least similar in appearance to the one or more features that are occluded.

16. A computer-implemented method comprising:

identifying, from first image data, an object;

making a determination as to whether the object includes one or more occluded regions;

making a determination, based on the one or more occluded regions, as to whether the object is associated with a likelihood of recognition error that exceeds a threshold;

in response to the determination that the object is associated with the likelihood of recognition error that exceeds the threshold;

synthesizing second image data for at least some of the one or more occluded regions of the object, wherein synthesizing the second image data includes using pixel values associated with the first image data to generate pixel values for at least some of the second image data corresponding to the one or more occluded regions; and

performing recognition of the object using the synthesized second image data.

17. The method of claim 16, wherein the object is a face of a person.

18. The method of claim 16, wherein synthesizing the second image data includes using a three-dimensional data model of the object.

19. The method of claim 18, wherein synthesizing the second image data includes augmenting the three-dimensional data model of the object with attributes of the first image data.

20. The method of claim 19, wherein augmenting the three-dimensional data model includes posing the three-dimensional data model to correspond to a view of the object provided by the first image data.

21. The method of claim 20, wherein augmenting the three-dimensional data model includes computing an intensity value of one or more discrete locations of the second image data from an intensity value of one or more corresponding discrete locations associated with the first image data.

22. The method of claim 16, wherein the synthesized second image data is presented in a frontal view.

23. The method of claim 16, wherein the synthesized second image data is presented.

30. The method of claim 16, wherein synthesizing the second image data comprises using the first image data to generate at least some of the pixel values for the second image data based on one or more assumptions about object symmetry.

31. The method of claim 16, wherein the one or more occluded regions include one or more occluded features of the object, and wherein synthesizing the second image data comprises generating the pixel values for the second image data using pixel values associated with at least a portion of the first image data that represents one or more features of the object that are at least similar in appearance to the one or more features that are occluded.