DETECTING ORIENTATION OF DIGITAL IMAGES USING FACE DETECTION INFORMATION

• US 20100165150A1
• Filed: 12/02/2009
• Published: 07/01/2010
• Est. Priority Date: 06/26/2003
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Blonk, Leo, Gângea, Mihnea, Vertran, Constantin, Steinberg, Eran, Prilutsky, Yury, Bigioi, Petronel, Corcoran, Peter
• Granted Patent: US 8,265,399 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A method of automatically establishing the correct orientation of an image using facial information. This method is based on the exploitation of the inherent property of image recognition algorithms in general and face detection in particular, where the recognition is based on criteria that is highly orientation sensitive. By applying a detection algorithm to images in various orientations, or alternatively by rotating the classifiers, and comparing the number of successful faces that are detected in each orientation, one may conclude as to the most likely correct orientation. Such method can be implemented as an automated method or a semi automatic method to guide users in viewing, capturing or printing of images.

Claims

1. A digital image acquisition device, comprising a lens and image sensor for acquiring a digital image, a processor, and memory having code embedded therein for programming the processor to perform a method of determining a state of a facial feature, wherein the method comprises:

acquiring a digital image;

detecting a face region within the acquired digital image;

determining an orientation of the face region;

extracting and rotating a crop of the face region, thereby providing a normalized face crop;

applying at least one set of classifiers to said normalized face crop; and

based on results of said applying at least one set of classifiers to said normalized face crop, determining a state of at least one facial feature within said normalized face crop.

2. The device of claim 1, wherein the method further comprises calculating an integral image representation of said normalized face crop.

3. The device of claim 2, wherein the applying at least one set of classifiers to said normalized face region is performed with regard to the integral image representation.

4. The device of claim 1, wherein the normalized face crop comprises a portrait or upright face image, or both.

5. The device of claim 1, wherein the method further comprises resizing said face crop to a standardized size.

6. The device of claim 5, wherein the resizing precedes said rotating.

7. The device of claim 5, wherein the rotating precedes said resizing.

8. The device of claim 1, wherein the at least one set of classifiers comprises one or more binary classifiers.

9. The device of claim 1, wherein the at least one set of classifiers comprises a cascade of classifiers.

10. The device of claim 1, wherein the at least one set of classifiers comprises one or more orientation-sensitive classifiers.

11. The device of claim 1, wherein the applying at least one set of classifiers comprises adjusting one or more of the classifiers according to the resizing.

12. The device of claim 1, wherein the applying at least one set of classifiers comprises applying said set of classifiers at more than one scale.

13. The device of claim 1, wherein the method further comprises generating in-camera, capturing or otherwise obtaining in-camera a collection of low resolution images including said faces, wherein said determining said orientation is based on analyzing said low resolution images.

14. The device of claim 1, wherein the method further comprises determining a further action based on said state of said facial feature within said normalized face crop determined from said applying said at least one set of classifiers to said normalized face crop.

15. A method of determining a state of a facial feature, comprising:

using a processor programmed by embedded digital code within a processor-readable medium;

acquiring a digital image using an image sensor;

detecting a face region within the acquired digital image;

determining an orientation of the face region;

extracting and rotating a crop of the face region, thereby providing a normalized face crop;

applying at least one set of classifiers to said normalized face crop; and

based on results of said applying at least one set of classifiers to said normalized face crop, determining a state of at least one facial feature within said normalized face crop.

16. The method of claim 15, further comprising calculating an integral image representation of said normalized face crop.

17. The method of claim 16, wherein the applying at least one set of classifiers to said normalized face region is performed with regard to the integral image representation.

18. The method of claim 15, wherein the normalized face crop comprises a portrait or upright face image, or both.

19. The method of claim 15, further comprising resizing said face crop to a standardized size.

20. The method of claim 19, wherein the resizing precedes said rotating.

21. The method of claim 19, wherein the rotating precedes said resizing.

22. The method of claim 15, wherein the at least one set of classifiers comprises one or more binary classifiers.

23. The method of claim 15, wherein the at least one set of classifiers comprises a cascade of classifiers.

24. The method of claim 15, wherein the at least one set of classifiers comprises one or more orientation-sensitive classifiers.

25. The method of claim 15, wherein the applying at least one set of classifiers comprises adjusting one or more of the classifiers according to the resizing.

26. The method of claim 15, wherein the applying at least one set of classifiers comprises applying said set of classifiers at more than one scale.

27. The method of claim 15, further comprising generating in-camera, capturing or otherwise obtaining in-camera a collection of low resolution images including said faces, wherein said determining said orientation is based on analyzing said low resolution images.

28. The method of claim 15, further comprising determining a further action based on said state of said normalized face crop determined from said applying at least one set of classifiers to said normalized face crop.

29. One or more processor-readable media having code embedded therein for programming the processor to perform a method of determining a state of a facial feature, wherein the method comprises:

using a processor programmed by the embedded digital code within a processor-readable medium;

acquiring a digital image using an image sensor;

detecting a face region within an acquired digital image;

determining an orientation of the face region;

extracting and rotating a crop of the face region, thereby providing a normalized face crop;

applying at least one set of classifiers to said normalized face crop; and

based on results of said applying at least one set of classifiers to said normalized face crop, determining a state of at least one facial feature within said normalized face crop.

30. The one or more processor-readable media of claim 29, wherein the method further comprises calculating an integral image representation of said normalized face crop.

31. The one or more processor-readable media of claim 30, wherein the applying at least one set of classifiers to said normalized face region is performed with regard to the integral image representation.

32. The one or more processor-readable media of claim 29, wherein the normalized face crop comprises a portrait or upright face image, or both.

33. The one or more processor-readable media of claim 29, wherein the method further comprises resizing said face crop to a standardized size.

34. The one or more processor-readable media of claim 33, wherein the resizing precedes said rotating.

35. The one or more processor-readable media of claim 33, wherein the rotating precedes said resizing.

36. The one or more processor-readable media of claim 29, wherein the at least one set of classifiers comprises one or more binary classifiers.

37. The one or more processor-readable media of claim 29, wherein the at least one set of classifiers comprises a cascade of classifiers.

38. The one or more processor-readable media of claim 29, wherein the at least one set of classifiers comprises one or more orientation-sensitive classifiers.

39. The one or more processor-readable media of claim 29, wherein the applying at least one set of classifiers comprises adjusting one or more of the classifiers according to the resizing.

40. The one or more processor-readable media of claim 29, wherein the applying at least one set of classifiers comprises applying said set of classifiers at more than one scale.

41. The one or more processor-readable media of claim 29, wherein the method further comprises generating in-camera, capturing or otherwise obtaining in-camera a collection of low resolution images including said faces, wherein said determining said orientation is based on analyzing said low resolution images.

42. The one or more processor-readable media of claim 29, wherein the method further comprises determining a further action based on said state of said normalized face crop determined from said applying at least one set of classifiers to said normalized face crop.