Method of Improving Orientation and Color Balance of Digital Images Using Face Detection Information

• US 20080013799A1
• Filed: 06/20/2007
• Published: 01/17/2008
• Est. Priority Date: 06/26/2003
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Bigioi, Petronel, Corcoran, Peter
• Granted Patent: US 7,912,245 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 9/68 for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

A method of generating one or more new spatial and chromatic variation digital images uses an original digitally-acquired image which including a face or portions of a face. A group of pixels that correspond to a face within the original digitally-acquired image is identified. A portion of the original image is selected to include the group of pixels. Values of pixels of one or more new images based on the selected portion are automatically generated, or an option to generate them is provided, in a manner which always includes the face within the one or more new images. Such method may be implemented to automatically establish the correct orientation and color balance of an image. Such method can be implemented as an automated method or a semi automatic method to guide users in viewing, capturing or printing of images.

Claims

1. A method of digital image processing using face detection to achieve a desired orientation, comprising:

(a) identifying one or more groups of pixels that correspond to a face within a digital image;

(b) identifying one or more sub-groups of pixels that correspond to one or more facial features of the face;

(c) determining initial values of one or more parameters of pixels of the one or more sub-groups of pixels;

(d) determining an initial orientation of the face within the digital image based on the initial values; and

(e) determining adjusted values of pixels within the digital image for adjusting the orientation to the desired orientation.

2. The method of claim 1, further comprising determining which of the sub-group of pixels belong to which said group of face pixels.

3. The method of claim 1, said determining initial values of one or more parameters of pixels being calculated based on the spatial orientation of said one or more sub-groups that correspond to one or more facial features.

4. The method of claim 3, the spatial orientation of said one or more sub-groups that correspond to one or more facial features being calculated based on an axis of an ellipse fit to said sub-group.

5. The method of claim 1, said adjusted values of pixels within the digital image being rounded to a closest multiple of 90 degrees.

6. The method of claim 1, further comprising adjusting the initial values to adjusted values for re-orienting the image to an adjusted orientation.

7. The method of claim 1, the one or more facial features including an eye.

8. The method of claim 1, the one or more facial features including two eyes.

9. The method of claim 1, the one or more facial features including two eyes and a mouth.

10. The method of claim 1, the one or more facial features including an eye, a mouth, lips, hairline, ears, nostrils, nose bridge, eyebrows, chin, neck or a nose, or combinations thereof.

11. The method of claim 10, the one or more facial features including an eye, a mouth, hairline, two eyes, a nose, an ear, two ears, a neck, shoulders, one or more other facial features, one or more personal features other than a facial feature, a feature associated with a person other than a personal feature, another feature that is indicative of orientation, or combinations thereof.

12. The method of claim 11, the feature that is indicative of orientation including an article of clothing, furniture, transportation, outdoor environment, horizon, tree, water, indoor environment, doorway, hallway, ceiling, floor, wall, or combinations thereof.

13. The method of claim 1, the one or more facial features including at least two features, the initial values including positions of the at least two features, the method further comprising determining relative positions of the at least two features based on the initial values, and the initial orientation being determined based on the relative positions of the at least two features.

14. The method of claim 1, the one or more facial features including at least three features, the initial values including positions of the at least three features, the method further comprising determining a shape based on the initial values, and the initial orientation being determined based on one or more properties of the shape.

15. The method of claim 14, the shape comprising a polygon having points corresponding to the at least three features as vertices.

16. The method of claim 15, the shape comprising a triangle.

17. The method of claim 1, further comprising automatically adjusting the initial orientation to an adjusted orientation.

18. The method of claim 1, further comprising automatically providing an option for adjusting the initial orientation to a suggested orientation.

19. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of digital image processing using face detection to achieve a desired orientation, comprising:

(a) identifying one or more groups of pixels that correspond to a face within a digital image;

(b) identifying one or more sub-groups of pixels that correspond to one or more facial features of the face;

(c) determining initial values of one or more parameters of pixels of the one or more sub-groups of pixels;

(d) determining an initial orientation of the face within the digital image based on the initial values; and

(e) determining adjusted values of pixels within the digital image for adjusting the orientation to the desired orientation.

20. The one or more storage devices of claim 19, the method further comprising determining which of the sub-group of pixels belong to which said group of face pixels.

21. The one or more storage devices of claim 19, said determining initial values of one or more parameters of pixels being calculated based on the spatial orientation of said one or more sub-groups that correspond to one or more facial features.

22. The one or more storage devices of claim 21, the spatial orientation of said one or more sub-groups that correspond to one or more facial features being calculated based on an axis of an ellipse fit to said sub-group.

23. The one or more storage devices of claim 22, said adjusted values of pixels within the digital image being rounded to a closest multiple of 90 degrees.

24. The one or more storage devices of claim 21, the method further comprising adjusting the initial values to adjusted values for re-orienting the image to an adjusted orientation.

25. The one or more storage devices of claim 21, the one or more facial features including an eye.

26. The one or more storage devices of claim 21, the one or more facial features including two eyes.

27. The one or more storage devices of claim 21, the one or more facial features including two eyes and a mouth.

28. The one or more storage devices of claim 21, the one or more facial features including an eye, a mouth, hairline, ears, neck, chin, nostrils, nose bridge, eyebrows or a nose, or combinations thereof.

29. The one or more storage devices of claim 21, the one or more facial features including an eye, a mouth, hairline, two eyes, a nose, an ear, two ears, a neck, shoulders, one or more other facial features, one or more personal features other than a facial feature, a feature associated with a person other than a personal feature, another feature that is indicative of orientation, or combinations thereof.

30. The one or more storage devices of claim 29, the feature that is indicative of orientation including an article of clothing, furniture, transportation, outdoor environment, horizon, tree, water, indoor environment, doorway, hallway, ceiling, floor, wall, or combinations thereof.

31. The one or more storage devices of claim 19, the one or more facial features including at least two features, the initial values including positions of the at least two features, the method further comprising determining relative positions of the at least two features based on the initial values, and the initial orientation being determined based on the relative positions of the at least two features.

32. The one or more storage devices of claim 19, the one or more facial features including at least three features, the initial values including positions of the at least three features, the method further comprising determining a shape based on the initial values, and the initial orientation being determined based on one or more properties of the shape.

33. The one or more storage devices of claim 32, the shape comprising a polygon having points corresponding to the at least three features as vertices.

34. The one or more storage devices of claim 33, the shape comprising a triangle.

35. The one or more storage devices of claim 19, the method further comprising automatically adjusting the initial orientation to an adjusted orientation.

36. The one or more storage devices of claim 19, the method further comprising automatically providing an option for adjusting the initial orientation to a suggested orientation.

37. The one or more storage devices of claim 19, wherein said processor is located within a digital printing device.

38. The one or more storage devices of claim 19, wherein said processor is located within a digital image acquisition device.

39. The one or more storage devices of claim 19, wherein said processor is located within a digital display device.

40. The one or more storage devices of claim 19, further comprising the step of displaying multiple images at once after said adjusting of orientation.