Detecting orientation of digital images using face detection information

• US 20060204110A1
• Filed: 12/27/2004
• Published: 09/14/2006
• Est. Priority Date: 06/26/2003
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Gangea, Mihnea, Bigioi, Petronel, Corcoran, Peter, Blonk, Leo, Vertran, Constantin
• Granted Patent: US 7,565,030 B2
• Time in Patent Office: Days
• US Class Current: 382/224
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A method of automatically establishing the correct orientation of an image using facial information. This method is based on the exploitation of the inherent property of image recognition algorithms in general and face detection in particular, where the recognition is based on criteria that is highly orientation sensitive. By applying a detection algorithm to images in various orientations, or alternatively by rotating the classifiers, and comparing the number of successful faces that are detected in each orientation, one may conclude as to the most likely correct orientation. Such method can be implemented as an automated method or a semi automatic method to guide users in viewing, capturing or printing of images.

Claims

1. A method of detecting an orientation of a digital image using statistical classifier techniques comprising:

(a) applying a set of classifiers to a digital image in a first orientation and determining a first level of match between said digital image at said first orientation and said classifiers;

(b) rotating said digital image to a second orientation, applying the classifiers to said rotated digital image at said second orientation, and determining a second level of match between said rotated digital image at said second orientation and said classifiers;

(c) comparing said first and second levels of match between said classifiers and said digital image and between said classifiers and said rotated digital image, respectively; and

(d) determining which of the first orientation and the second orientations has a greater probability of being a correct orientation based on which of the first and second levels of match, respectively, comprises a higher level of match.

2. The method of claim 1, further comprising:

(e) rotating said digital image to a third orientation, applying the classifiers to said rotated digital image at said third orientation, and determining a third level of match between said rotated digital image at said third orientation and said classifiers;

(f) comparing said third level of match with said first level of match or said second level of match, or both; and

(g) determining which of two or more of the first orientation, the second orientations and the third orientation has a greater probability of being a correct orientation based on which of the two or more the first, second and third levels of match, respectively, comprises a higher level of match.

3. The method of claim 2, wherein the rotating to the second orientation and the rotating to the third orientation comprise rotations in opposite directions.

4. The method of claim 2, wherein the second orientation and the third orientation comprise orientations of the digital image that are relatively rotated from the first orientation, each by an acute or obtuse amount, in opposite directions.

5. The method of claim 2, wherein said classifiers comprise face detection classifiers.

6. The method of claim 5, wherein said classifiers comprise elliptical classifiers.

7. The method of claim 6, wherein said elliptical classifiers are oriented at known orientations.

8. The method of claim 5, wherein said classifiers correspond to regions of a detected face.

9. The method of claim 8, wherein said regions include an eye, two eyes, a nose, a mouth, or an entire face, or combinations thereof.

10. The method of claim 2, wherein said classifiers comprise color based classifiers.

11. The method of claim 2, wherein said classifiers comprise image classifiers for scene analysis.

12. The method of claim 11, wherein said classifiers based on scene analysis comprise perception-based classifiers.

13. The method of claim 2, wherein said classifiers comprise face detection classifiers, color classifiers, semantic-based classifiers, scene analysis classifiers, or combinations thereof.

14. The method of claim 2, further comprising preparing said digital image prior to applying said classifiers to said digital image and determining said level of match between said digital image and said classifiers.

15. The method of claim 14, wherein said preparing said digital image comprises subsampling, color conversion, edge enhancement, blurring, sharpening, tone reproduction correction, exposure correction, gray scale transformation, region segmentation, cropping, or combinations thereof.

16. The method of claim 14, wherein said preparing said digital image includes subsampling.

17. The method of claim 14, wherein said preparing said digital image includes image quality correcting.

18. The method of claim 2, further comprising:

(h) rotating said digital image to a fourth orientation, applying the classifiers to said rotated digital image at said fourth orientation, and determining a fourth level of match between said rotated digital image at said fourth orientation and said classifiers;

(f) comparing said fourth level of match with said first level of match, said second level of match, or said third level of match, or combinations thereof; and

(g) determining which of two or more of the first orientation, the second orientation, the third orientation, and the fourth orientation has a greater probability of being a correct orientation based on which of the two or more of the first, second, third, and fourth levels of match, respectively, comprises a higher level of match.

19. The method of claim 18, wherein said second and third orientations comprise 90°

opposite rotations of said digital image from said first orientation, and said fourth rotations comprises a 180°

rotation of said digital image from said first orientation.

20. The method of claim 2, wherein said second and third orientations comprise 90°

opposite rotations of said digital image from said first orientation.

21. A method of detecting an orientation of a digital image using statistical classifier techniques comprising:

(a) applying a set of classifiers to a digital image in a first orientation and determining a first level of match between said digital image at said first orientation and said classifiers;

(b) rotating said set of classifiers a first predetermined amount, applying the classifiers rotated said first amount to said digital image at said first orientation, and determining a second level of match between said digital image at said first orientation and said classifiers rotated said first amount;

(c) comparing said first and second levels of match between said classifiers and said digital image and between said rotated classifiers and said digital image, respectively; and

(d) determining which of the first and second levels of match, respectively, comprises a higher level of match in order to determine whether said first orientation is a correct orientation of said digital image.

22. The method of claim 21, further comprising:

(e) rotating said set of classifiers a second predetermined amount, applying the classifiers rotated said second amount to said digital image at said first orientation, and determining a third level of match between said digital image at said first orientation and said classifiers rotated said second amount;

(f) comparing said third level of match with said first level of match or said second level of match, or both; and

(g) determining which of two or more of the first orientation, the second orientations and the third orientation has a greater probability of being a correct orientation based on which of the two or more of the first, second and third levels of match, respectively, comprises a higher level of match.

23. The method of claim 22, wherein the rotating by the first and second amounts comprise rotations in opposite directions.

24. The method of claim 22, wherein the first and second amounts comprise acute or obtuse amounts equal in magnitude and opposite in direction.

25. The method of claim 22, wherein said classifiers comprise face detection classifiers.

26. The method of claim 25, wherein said classifiers comprise elliptical classifiers.

27. The method of claim 26, wherein said elliptical classifiers are initially oriented at known orientations and, when rotated by said first and second amounts, are rotated to different known orientations.

28. The method of claim 25, wherein said classifiers correspond to regions of a detected face.

29. The method of claim 28, wherein said regions include an eye, two eyes, a nose, a mouth, or an entire face, or combinations thereof.

30. The method of claim 22, wherein said classifiers comprise color based classifiers.

31. The method of claim 22, wherein said classifiers comprise image classifiers for scene analysis.

32. The method of claim 31, wherein said classifiers based on scene analysis comprise perception-based classifiers.

33. The method of claim 22, wherein said classifiers comprise face detection classifiers, color classifiers, semantic-based classifiers, scene analysis classifiers, or combinations thereof.

34. The method of claim 22, further comprising preparing said digital image prior to applying said classifiers to said digital image and determining said level of match between said digital image and said classifiers.

35. The method of claim 34, wherein said preparing said digital image comprises subsampling, color conversion, edge enhancement, blurring, sharpening, tone reproduction correction, exposure correction, gray scale transformation, region segmentation, cropping, or combinations thereof.

36. The method of claim 34, wherein said preparing said digital image includes subsampling.

37. The method of claim 34, wherein said preparing said digital image includes image quality correcting.

38. The method of claim 22, further comprising:

(h) rotating said set of classifiers a third predetermined amount, applying the classifiers rotated by said third amount to said digital image at said first orientation, and determining a fourth level of match between said digital image at said first orientation and said classifiers rotated by said third amount;

(f) comparing said fourth level of match with two or more of said first level of match, said second level of match, and said third level of match; and

(g) determining which of the two or more of the unrotated classifiers, and those rotated by the first amount, the second amount, and the third amount has a greater probability of matching the first orientation of the digital image based on which of the two or more of the first, second, third, and fourth levels of match, respectively, comprises a higher level of match.

39. The method of claim 38, wherein said first and second amounts comprise 90°

opposite rotations of said set of classifiers from an initial orientation, and said third amount comprises a 180°

rotation of said set of classifiers.

40. The method of claim 22, wherein said first and second amounts comprise 90°

opposite rotations of said set of classifiers from an initial orientation.

41. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of detecting an orientation of a digital image using statistical classifier techniques, the method comprising:

(a) applying a set of classifiers to a digital image in a first orientation and determining a first level of match between said digital image at said first orientation and said classifiers;

(b) rotating said digital image to a second orientation, applying the classifiers to said rotated digital image at said second orientation, and determining a second level of match between said rotated digital image at said second orientation and said classifiers;

(c) comparing said first and second levels of match between said classifiers and said digital image and between said classifiers and said rotation digital image, respectively; and

(d) determining which of the first orientation and the second orientations has a greater probability of being a correct orientation based on which of the first and second levels of match, respectively, comprises a higher level of match.

42. The one or more storage devices of claim 41, the method further comprising:

(e) rotating said digital image to a third orientation, applying the classifiers to said rotated digital image at said third orientation, and determining a third level of match between said rotated digital image at said third orientation and said classifiers;

(f) comparing said third level of match with said first level of match or said second level of match, or both; and

(g) determining which of two or more of the first orientation, the second orientations and the third orientation has a greater probability of being a correct orientation based on which of the two or more the first, second and third levels of match, respectively, comprises a higher level of match.

43. The one or more storage devices of claim 42, wherein the rotating to the second orientation and the rotating to the third orientation comprise rotations in opposite directions.

44. The one or more storage devices of claim 42, wherein the second orientation and the third orientation comprise orientations of the digital image that are relatively rotated from the first orientation, each by an acute or obtuse amount, in opposite directions.

45. The one or more storage devices of claim 42, wherein said classifiers comprise face detection classifiers.

46. The one or more storage devices of claim 45, wherein said classifiers comprise elliptical classifiers.

47. The one or more storage devices of claim 46, wherein said elliptical classifiers are oriented at known orientations.

48. The one or more storage devices of claim 45, wherein said classifiers correspond to regions of a detected face.

49. The one or more storage devices of claim 48, wherein said regions include an eye, two eyes, a nose, a mouth, or an entire face, or combinations thereof.

50. The one or more storage devices of claim 42, wherein said classifiers comprise color based classifiers.

51. The one or more storage devices of claim 42, wherein said classifiers comprise image classifiers for scene analysis.

52. The one or more storage devices of claim 51, wherein said classifiers based on scene analysis comprise perception-based classifiers.

53. The one or more storage devices of claim 42, wherein said classifiers comprise face detection classifiers, color classifiers, semantic-based classifiers, scene analysis classifiers, or combinations thereof.

54. The one or more storage devices of claim 42, the method further comprising preparing said digital image prior to applying said classifiers to said digital image and determining said level of match between said digital image and said classifiers.

55. The one or more storage devices of claim 54, wherein said preparing said digital image comprises subsampling, color conversion, edge enhancement, blurring, sharpening, tone reproduction correction, exposure correction, gray scale transformation, region segmentation, cropping, or combinations thereof.

56. The one or more storage devices of claim 54, wherein said preparing said digital image includes subsampling.

57. The one or more storage devices of claim 54, wherein said preparing said digital image includes image quality correcting.

58. The one or more storage devices of claim 42, the method further comprising:

(h) rotating said digital image to a fourth orientation, applying the classifiers to said rotated digital image at said fourth orientation, and determining a fourth level of match between said rotated digital image at said fourth orientation and said classifiers;

(f) comparing said fourth level of match with said first level of match, said second level of match, or said third level of match, or combinations thereof; and

(g) determining which of two or more of the first orientation, the second orientation, the third orientation, and the fourth orientation has a greater probability of being a correct orientation based on which of the two or more of the first, second, third, and fourth levels of match, respectively, comprises a higher level of match.

59. The one or more storage devices of claim 58, wherein said second and third orientations comprise 90°

opposite rotations of said digital image from said first orientation, and said fourth rotations comprises a 180°

rotation of said digital image from said first orientation.

60. The one or more storage devices of claim 42, wherein said second and third orientations comprise 90°

opposite rotations of said digital image from said first orientation.

61. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of detecting an orientation of a digital image using statistical classifier techniques, the method comprising:

(a) applying a set of classifiers to a digital image in a first orientation and determining a first level of match between said digital image at said first orientation and said classifiers;

(b) rotating said set of classifiers a first predetermined amount, applying the classifiers rotated said first amount to said digital image at said first orientation, and determining a second level of match between said digital image at said first orientation and said classifiers rotated said first amount;

(c) comparing said first and second levels of match between said classifiers and said digital image and between said rotated classifiers and said digital image, respectively; and

(d) determining which of the first and second levels of match, respectively, comprises a higher level of match in order to determine whether said first orientation is a correct orientation of said digital image.

62. The one or more storage devices of claim 61, the method further comprising:

(e) rotating said set of classifiers a second predetermined amount, applying the classifiers rotated said second amount to said digital image at said first orientation, and determining a third level of match between said digital image at said first orientation and said classifiers rotated said second amount;

(f) comparing said third level of match with said first level of match or said second level of match, or both; and

(g) determining which of two or more of the first orientation, the second orientations and the third orientation has a greater probability of being a correct orientation based on which of the two or more of the first, second and third levels of match, respectively, comprises a higher level of match.

63. The one or more storage devices of claim 62, wherein the rotating by the first and second amounts comprise rotations in opposite directions.

64. The one or more storage devices of claim 62, wherein the first and second amounts comprise acute or obtuse amounts equal in magnitude and opposite in direction.

65. The one or more storage devices of claim 62, wherein said classifiers comprise face detection classifiers.

66. The one or more storage devices of claim 65, wherein said classifiers comprise elliptical classifiers.

67. The one or more storage devices of claim 66, wherein said elliptical classifiers are initially oriented at known orientations and, when rotated by said first and second amounts, are rotated to different known orientations.

68. The one or more storage devices of claim 65, wherein said classifiers correspond to regions of a detected face.

69. The one or more storage devices of claim 68, wherein said regions include an eye, two eyes, a nose, a mouth, or an entire face, or combinations thereof.

70. The one or more storage devices of claim 62, wherein said classifiers comprise color based classifiers.

71. The one or more storage devices of claim 62, wherein said classifiers comprise image classifiers for scene analysis.

72. The one or more storage devices of claim 71, wherein said classifiers based on scene analysis comprise perception-based classifiers.

73. The one or more storage devices of claim 62, wherein said classifiers comprise face detection classifiers, color classifiers, semantic-based classifiers, scene analysis classifiers, or combinations thereof.

74. The one or more storage devices of claim 62, the method further comprising preparing said digital image prior to applying said classifiers to said digital image and determining said level of match between said digital image and said classifiers.

75. The one or more storage devices of claim 74, wherein said preparing said digital image comprises subsampling, color conversion, edge enhancement, blurring, sharpening, tone reproduction correction, exposure correction, gray scale transformation, region segmentation, cropping, or combinations thereof.

76. The one or more storage devices of claim 74, wherein said preparing said digital image includes subsampling.

77. The one or more storage devices of claim 74, wherein said preparing said digital image includes image quality correcting.

78. The one or more storage devices of claim 62, the method further comprising:

(h) rotating said set of classifiers a third predetermined amount, applying the classifiers rotated by said third amount to said digital image at said first orientation, and determining a fourth level of match between said digital image at said first orientation and said classifiers rotated by said third amount;

(f) comparing said fourth level of match with two or more of said first level of match, said second level of match, and said third level of match; and

(g) determining which of the two or more of the unrotated classifiers, and those rotated by the first amount, the second amount, and the third amount has a greater probability of matching the first orientation of the digital image based on which of the two or more of the first, second, third, and fourth levels of match, respectively, comprises a higher level of match.

79. The one or more storage devices of claim 78, wherein said first and second amounts comprise 90°

opposite rotations of said set of classifiers from an initial orientation, and said third amount comprises a 180°

rotation of said set of classifiers.

80. The one or more storage devices of claim 62, wherein said first and second amounts comprise 90°

opposite rotations of said set of classifiers from an initial orientation.