Method of improving orientation and color balance of digital images using face detection information

• US 7,912,245 B2
• Filed: 06/20/2007
• Issued: 03/22/2011
• Est. Priority Date: 06/26/2003
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Bigioi, Petronel, Corcoran, Peter
• Primary Examiner(s): Seth; Manav
• Application Number:
  US11/765,899
• Publication Number:
  US 20080013799A1
• Time in Patent Office: 1,371 Days
• Field of Search: 382/103, 382/115, 382/118, 382/276, 382286-298, 348/61, 348/77, 348/78
• US Class Current: 382/103
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 9/68 for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

A method of generating one or more new spatial and chromatic variation digital images uses an original digitally-acquired image which including a face or portions of a face. A group of pixels that correspond to a face within the original digitally-acquired image is identified. A portion of the original image is selected to include the group of pixels. Values of pixels of one or more new images based on the selected portion are automatically generated, or an option to generate them is provided, in a manner which always includes the face within the one or more new images. Such method may be implemented to automatically establish the correct orientation and color balance of an image. Such method can be implemented as an automated method or a semi automatic method to guide users in viewing, capturing or printing of images.

Claims

1. A method of digital image processing using face detection to achieve desired relative positions, orientations and pose angles of two or more detected faces, comprising:

identifying two or more groups of pixels that correspond to two or more faces within a digital image;

identifying one or more sub-groups of pixels that correspond to one or more facial features of each of the two or more faces;

determining initial values of relative positions, orientations and pose angles of pixels of the two or more groups of pixels based on values of said one or more sub-groups of pixels of said each of the two or more faces, including calculating spatial orientations of axes of ellipses fit to said groups;

determining initial relative positions, orientations and pose angles of the two or more faces within the digital image based on the initial values; and

determining adjusted values of pixels within the digital image for adjusting the relative positions, orientations or pose angles, or combinations thereof, respectively to the desired relative positions, orientations or pose angles.

2. The method of claim 1, further comprising determining which of the sub-group of pixels belong to which said group of face pixels.

3. The method of claim 1, said determining initial values of relative positions or orientations, or both, of pixels being calculated based on the spatial orientation of said one or more sub-groups that correspond to one or more facial features.

4. The method of claim 3, the spatial orientation of said one or more sub-groups that correspond to one or more facial features being calculated based on an axis of an ellipse fit to said sub-group.

5. The method of claim 1, said adjusted values of pixels within the digital image being rounded to a closest multiple of 90 degrees.

6. The method of claim 1, further comprising adjusting the initial values to adjusted values for re-orienting the image to an adjusted orientation.

7. The method of claim 1, the one or more facial features including an eye.

8. The method of claim 1, the one or more facial features including two eyes.

9. The method of claim 1, the one or more facial features including two eyes and a mouth.

10. The method of claim 1, the one or more facial features including an eye, a mouth, lips, hairline, ears, nostrils, nose bridge, eyebrows, chin, neck or a nose, or combinations thereof.

11. The method of claim 10, the one or more facial features including an eye, a mouth, hairline, two eyes, a nose, an ear, two ears, a neck, shoulders, one or more other facial features, one or more personal features other than a facial feature, a feature associated with a person other than a personal feature, another feature that is indicative of orientation, or combinations thereof.

12. The method of claim 11, the feature that is indicative of orientation including an article of clothing, furniture, transportation, outdoor environment, horizon, tree, water, indoor environment, doorway, hallway, ceiling, floor, wall, or combinations thereof.

13. The method of claim 1, the one or more facial features including at least two features, the initial values including positions of the at least two features, the method further comprising determining relative positions of the at least two features based on the initial values, and the initial orientation being determined based on the relative positions of the at least two features.

14. The method of claim 1, further comprising automatically adjusting the initial orientation to an adjusted orientation.

15. The method of claim 1, further comprising automatically providing an option for adjusting the initial orientation to a suggested orientation.

16. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of digital image processing using face detection to achieve a desired relative positions, orientations and pose angles of two or more detected faces, comprising:

identifying two or more groups of pixels that correspond to two or more faces within a digital image;

identifying one or more sub-groups of pixels that correspond to one or more facial features of each of the two or more faces;

determining initial values of relative positions, orientations and pose angles of pixels of the two or more groups of pixels based on values of said one or more sub-groups of pixels of said each of the two or more faces, including calculating spatial orientations of axes of ellipses fit to said groups;

determining initial relative positions, orientations and pose angles of the two or more faces within the digital image based on the initial values; and

determining adjusted values of pixels within the digital image for adjusting the relative positions, orientations or pose angles, or combinations thereof, respectively to the desired relative positions, orientations or pose angles.

17. The one or more storage devices of claim 16, the method further comprising determining which of the sub-group of pixels belong to which said group of face pixels.

18. The one or more storage devices of claim 16, said determining initial values of relative positions or orientations, or both, of pixels being calculated based on the spatial orientation of said one or more sub-groups that correspond to one or more facial features.

19. The one or more storage devices of claim 18, the spatial orientation of said one or more sub-groups that correspond to one or more facial features being calculated based on an axis of an ellipse fit to said sub-group.

20. The one or more storage devices of claim 19, said adjusted values of pixels within the digital image being rounded to a closest multiple of 90 degrees.

21. The one or more storage devices of claim 18, the method further comprising adjusting the initial values to adjusted values for re-orienting the image to an adjusted orientation.

22. The one or more storage devices of claim 18, the one or more facial features including an eye.

23. The one or more storage devices of claim 18, the one or more facial features including two eyes.

24. The one or more storage devices of claim 18, the one or more facial features including two eyes and a mouth.

25. The one or more storage devices of claim 18, the one or more facial features including an eye, a mouth, hairline, ears, neck, chin, nostrils, nose bridge, eyebrows or a nose, or combinations thereof.

26. The one or more storage devices of claim 18, the one or more facial features including an eye, a mouth, hairline, two eyes, a nose, an ear, two ears, a neck, shoulders, one or more other facial features, one or more personal features other than a facial feature, a feature associated with a person other than a personal feature, another feature that is indicative of orientation, or combinations thereof.

27. The one or more storage devices of claim 26, the feature that is indicative of orientation including an article of clothing, furniture, transportation, outdoor environment, horizon, tree, water, indoor environment, doorway, hallway, ceiling, floor, wall, or combinations thereof.

28. The one or more storage devices of claim 16, the one or more facial features including at least two features, the initial values including positions of the at least two features, the method further comprising determining relative positions of the at least two features based on the initial values, and the initial orientation being determined based on the relative positions of the at least two features.

29. The one or more storage devices of claim 16, the method further comprising automatically adjusting the initial orientation to an adjusted orientation.

30. The one or more storage devices of claim 16, the method further comprising automatically providing an option for adjusting the initial orientation to a suggested orientation.

31. The one or more storage devices of claim 16, wherein said processor is located within a digital printing device.

32. The one or more storage devices of claim 16, wherein said processor is located within a digital image acquisition device.

33. The one or more storage devices of claim 16, wherein said processor is located within a digital display device.

34. The one or more storage devices of claim 16, further comprising the step of displaying multiple images at once after said adjusting of orientation.