Detecting orientation of digital images using face detection information

• US 8,265,399 B2
• Filed: 12/02/2009
• Issued: 09/11/2012
• Est. Priority Date: 06/26/2003
• Status: Expired due to Term

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Corcoran, Peter, Bigioi, Petronel, Blonk, Leo, Gângea, Mihnea, Vertran, Constantin
• Primary Examiner(s): Moe, Aung S
• Assistant Examiner(s): Chen, Chia-Wei A
• Application Number:
  US12/629,450
• Publication Number:
  US 20100165150A1
• Time in Patent Office: 1,014 Days
• Field of Search: 382/165, 382115-118, 382181-231
• US Class Current: 382/216
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A method of automatically establishing the correct orientation of an image using facial information. This method is based on the exploitation of the inherent property of image recognition algorithms in general and face detection in particular, where the recognition is based on criteria that is highly orientation sensitive. By applying a detection algorithm to images in various orientations, or alternatively by rotating the classifiers, and comparing the number of successful faces that are detected in each orientation, one may conclude as to the most likely correct orientation. Such method can be implemented as an automated method or a semi automatic method to guide users in viewing, capturing or printing of images.

Claims

1. A digital image acquisition device, comprising a lens and image sensor for acquiring a digital image, a processor, and memory having code embedded therein for programming the processor to perform a method of determining an orientation state of a digital image, wherein the method comprises:

acquiring a digital image;

detecting one or more face regions within the acquired digital image, including contemporaneously determining an orientation state of the digital image, including;

applying a combined face classifier set, that includes one or more rotation invariant classifiers and three or more sets of orientation-specific classifiers, to said digital image, wherein each of the one or more rotation invariant classifiers is common to individual versions of each of the orientation-specific classifiers and included only once in the combined face classifier, and wherein each set of the three or more sets of orientation-specific classifiers is rotated relative to the other sets of orientation-specific classifiers; and

based on results of said applying said combined face classifier set to the digital image, determining an orientation state of the digital image as an orientation among three or more orientations corresponding respectively to the three or more orientation-specific classifier sets with a highest probability of detection of one or more face regions.

2. The device of claim 1, wherein the method further comprises calculating an integral image representation of said digital image.

3. The device of claim 2, wherein the applying said combined face classifier set to said digital image is performed with regard to the integral image representation.

4. The device of claim 1, wherein the digital image comprises a portrait or upright face image, or both.

5. The device of claim 1, wherein the method further comprises cropping and resizing said one or more face regions to a standardized size.

6. The device of claim 5, wherein the cropping and resizing precedes application of certain classifiers of the combined face classifier set.

7. The device of claim 5, wherein the applying said combined face classifier set to said digital image precedes said resizing.

8. The device of claim 1, wherein the combined face classifier set comprises one or more binary classifiers.

9. The device of claim 1, wherein the combined face classifier set comprises a cascade of classifiers.

10. The device of claim 1, wherein the combined face classifier set comprises more than three orientation-sensitive classifiers.

11. The device of claim 1, wherein the applying said combined face classifier set further comprises resizing one or more face regions to a standardized size, and adjusting one or more classifiers of the combined face classifier set according to the resizing.

12. The device of claim 1, wherein the applying said combined face classifier set comprises applying said combined face classifier set at more than one scale.

13. The device of claim 1, wherein the method further comprises generating in-camera, capturing or otherwise obtaining in-camera a collection of low resolution images including said one or more face regions, wherein said determining said orientation state is based on analyzing said low resolution images.

14. The device of claim 1, wherein the method further comprises determining a further action based on said orientation state.

15. A method of determining an orientation state of a digital image, comprising:

using a processor programmed by embedded digital code within a processor-readable medium;

acquiring a digital image using an image sensor;

detecting one or more face regions within the acquired digital image, including contemporaneously determining an orientation state of the digital image, including;

applying a combined face classifier set, that includes one or more rotation invariant classifiers and three or more sets of orientation-specific classifiers, to said digital image, wherein each of the one or more rotation invariant classifiers is common to individual versions of each of the orientation-specific classifiers and included only once in the combined face classifier, and wherein each set of the three or more sets of orientation-specific classifiers is rotated relative to the other sets of orientation-specific classifiers; and

based on results of said applying said combined face classifier set to the digital image, determining an orientation state of the digital image as an orientation among three or more orientations corresponding respectively to the three or more orientation-specific classifier sets with a highest probability of detection of one or more face regions.

16. The method of claim 15, further comprising calculating an integral image representation of said digital image.

17. The method of claim 16, wherein the applying said combined face classifier set to said digital image is performed with regard to the integral image representation.

18. The method of claim 15, wherein the digital image comprises a portrait or upright face image, or both.

19. The method of claim 15, further comprising cropping and resizing said one or more face regions to a standardized size.

20. The method of claim 19, wherein the cropping and resizing precedes application of certain classifiers of the combined face classifier set.

21. The method of claim 19, wherein the applying said combined face classifier set to said digital image precedes said resizing.

22. The method of claim 15, wherein the combined face classifier set comprises one or more binary classifiers.

23. The method of claim 15, wherein the combined face classifier set comprises a cascade of classifiers.

24. The method of claim 15, wherein the combined face classifier set comprises more than three orientation-sensitive classifiers.

25. The method of claim 15, wherein the applying said combined face classifier set further comprises resizing one or more face regions to a standardized size, and adjusting one or more classifiers of the combined face classifier set according to the resizing.

26. The method of claim 15, wherein the applying said combined face classifier set comprises applying said combined face classifier set at more than one scale.

27. The method of claim 15, further comprising generating in-camera, capturing or otherwise obtaining in-camera a collection of low resolution images including said one or more face regions, wherein said determining said orientation state is based on analyzing said low resolution images.

28. The method of claim 15, further comprising determining a further action based on said orientation state of said digital image.

29. One or more non-transitory processor-readable media having code embedded therein for programming the processor to perform a method of determining an orientation state of an acquired digital image, wherein the method comprises:

detecting one or more face regions within an acquired digital image, including contemporaneously determining an orientation state of the digital image, including;

applying a combined face classifier set, that includes one or more rotation invariant classifiers and three or more sets of orientation-specific classifiers, to said digital image, wherein each of the one or more rotation invariant classifiers is common to individual versions of each of the orientation-specific classifiers and included only once in the combined face classifier, and wherein each set of the three or more sets of orientation-specific classifiers is rotated relative to the other sets of orientation-specific classifiers; and

based on results of said applying said combined face classifier set to the digital image, determining an orientation state of the digital image as an orientation among three or more orientations corresponding respectively to the three or more orientation-specific classifier sets with a highest probability of detection of one or more face regions.

30. The one or more processor-readable media of claim 29, wherein the method further comprises calculating an integral image representation of said digital image.

31. The one or more processor-readable media of claim 30, wherein the applying said combined face classifier set to said digital image is performed with regard to the integral image representation.

32. The one or more processor-readable media of claim 29, wherein the digital image comprises a portrait or upright face image, or both.

33. The one or more processor-readable media of claim 29, wherein the method further comprises cropping and resizing said one or more face regions to a standardized size.

34. The one or more processor-readable media of claim 33, wherein the cropping and resizing precedes application of certain classifiers of the combined face classifier set.

35. The one or more processor-readable media of claim 33, wherein the applying said combined face classifier set to said digital image precedes said resizing.

36. The one or more processor-readable media of claim 29, wherein the combined face classifier set comprises one or more binary classifiers.

37. The one or more processor-readable media of claim 29, wherein the combined face classifier set comprises a cascade of classifiers.

38. The one or more processor-readable media of claim 29, wherein the combined face classifier set comprises more than three orientation-sensitive classifiers.

39. The one or more processor-readable media of claim 29, wherein the applying said combined face classifier set further comprises resizing one or more face regions to a standardized size, and adjusting one or more classifiers of the combined face classifier set according to the resizing.

40. The one or more processor-readable media of claim 29, wherein the applying said combined face classifier set comprises applying said combined face classifier set at more than one scale.

41. The one or more processor-readable media of claim 29, wherein the method further comprises generating in-camera, capturing or otherwise obtaining in-camera a collection of low resolution images including said one or more face regions, wherein said determining said orientation state is based on analyzing said low resolution images.

42. The one or more processor-readable media of claim 29, wherein the method further comprises determining a further action based on said orientation state.