Method of improving orientation and color balance of digital images using face detection information

• US 7,440,593 B1
• Filed: 06/26/2003
• Issued: 10/21/2008
• Est. Priority Date: 06/26/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Bigioi, Petronel, Corcoran, Peter
• Primary Examiner(s): Seth; Manav
• Application Number:
  US10/608,772
• Time in Patent Office: 1,944 Days
• Field of Search: 382/103, 382115-118, 348/61, 348 77- 78, 348/143, 348/152, 348/155, 348/158, 348/207.99
• US Class Current: 382/118
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 9/68 for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

A method of generating one or more new spatial and chromatic variation digital images uses an original digitally-acquired image which including a face or portions of a face. A group of pixels that correspond to a face within the original digitally-acquired image is identified. A portion of the original image is selected to include the group of pixels. Values of pixels of one or more new images based on the selected portion are automatically generated, or an option to generate them is provided, in a manner which always includes the face within the one or more new images. Such method may be implemented to automatically establish the correct orientation and color balance of an image. Such method can be implemented as an automated method or a semi automatic method to guide users in viewing, capturing or printing of images.

Claims

1. A method of digital image processing using face detection for achieving a desired spatial parameter, comprising:

(a) identifying a group of pixels that correspond to a face within a main digital image;

(b) generating in-camera, capturing or otherwise obtaining in-camera a collection of one or more low resolution images including said face;

(c) tracking said face within said collection of one or more low resolution images;

(d) identifying one or more sub-groups of pixels that correspond to one or more facial features of the face, said identifying of said group or sub-groups of pixels, or both, being based on the tracking of said face within said collection of one or more low resolution images;

(e) determining initial values of one or more parameters of pixels of the one or more sub-groups of pixels;

(f) determining an initial spatial parameter of the face within the main digital image based on the initial values;

(g) determining adjusted values of pixels within the digital image for adjusting the main digital image based on a comparison of the initial and desired spatial parameters;

(h) generating an adjusted version of the digital image including said adjusted values of said pixels; and

(i) storing, displacing, transmitting, transferring, printing, uploading or downloading the adjusted version of the digital image, or a further processed version, or combinations thereof.

2. The method of claim 1, the initial spatial parameter comprising orientation.

3. The method of claim 1, further comprising automatically adjusting the values of the pixels within the digital image to adjust the initial spatial parameter approximately to the desired spatial parameter.

4. The method of claim 1, further comprising automatically providing an option for adjusting the values of the pixels within the digital image to adjust the initial spatial parameter to the desired spatial parameter.

9. The method of claim 2, wherein the determining initial values of one or more parameters of pixels comprises calculating based on said orientation of said one or more sub-groups that correspond to one or more facial features.

10. The method of claim 9, wherein said calculating based on said orientation of said one or more sub-groups that correspond to one or more facial features comprises calculating based on an axis of an ellipse fit to said sub-group.

11. The method of claim 2, wherein said adjusted values of pixels within the digital image are rounded to a closest multiple of 90 degrees.

12. The method of claim 2, further comprising adjusting the initial values to adjusted values for re-orienting the image to an adjusted orientation.

13. The method of claim 2, wherein the one or more facial features include at least two features, and the initial values include positions of the at least two features, and the method further comprises determining relative positions of the at least two features based on the initial values, and wherein determining the initial orientation based on the relative positions of the at least two features.

14. The method of claim 2, further comprising automatically adjusting initial orientation to an adjusted orientation.

15. The method of claim 2, further comprising automatically providing an option for adjusting initial orientation to a suggested orientation.

16. The method of claim 1, wherein the tracking comprises tracking said face within said main digital image and said collection of one or more low resolution images.

17. The method of claim 16, wherein said identifying of said group or sub-groups of pixels, or both, is based on said tracking of said face within said main digital image and said collection of one or more low resolution images.

5. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of digital image processing using face detection for achieving a desired spatial parameter, comprising:

(a) identifying a group of pixels that correspond to a face within a main digital image;

(b) generating in-camera, capturing or otherwise obtaining in-camera a collection of one or more low resolution images including said face;

(c) tracking said face within said collection of one or more low resolution images;

(d) identifying one or more sub-groups of pixels that correspond to one or more facial features of the face, said identifying of said group or sub-groups of pixels, or both, being based on the tracking of said face within said collection of one or more low resolution images;

(e) determining initial values of one or more parameters of pixels of the one or more sub-groups of pixels;

(f) determining an initial spatial parameter of the face within the main digital image based on the initial values;

(g) determining adjusted values of pixels within the digital image for adjusting the main digital image based on a comparison of the initial and desired spatial parameters;

(h) generating an adjusted version of the digital image including said adjusted values of said pixels; and

(i) storing, displaying, transmitting, transferring, printing, uploading or downloading the adjusted version of the digital image or a further processed version or combinations thereof.

6. The one or more storage devices of claim 5, the initial spatial parameter comprising orientation.

7. The one or more storage devices of claim 5, the method further comprising automatically adjusting the values of the pixels within the digital image to adjust the initial spatial parameter approximately to the desired spatial parameter.

8. The one or more storage devices of claim 5, the method further comprising automatically providing an option for adjusting the values of the pixels within the digital image to adjust the initial spatial parameter to the desired spatial parameter.

18. The one or more storage devices of claim 6, wherein the determining initial values of one or more parameters of pixels comprises calculating based on said orientation of said one or more sub-groups that correspond to one or more facial features.

19. The one or more storage devices of claim 18, wherein said calculating based on said orientation of said one or more sub-groups that correspond to one or more facial features comprises calculating based on an axis of an ellipse fit to said sub-group.

20. The one or more storage devices of claim 6, wherein said adjusted values of pixels within the digital image are rounded to a closest multiple of 90 degrees.

21. The one or more storage devices of claim 6, wherein the method further comprises adjusting the initial values to adjusted values for re-orienting the image to an adjusted orientation.

22. The one or more storage devices of claim 6, wherein the one or more facial features include at least two features, and the initial values include positions of the at least two features, and wherein the method further comprises determining relative positions of the at least two features based on the initial values, and wherein determining the initial orientation based on the relative positions of the at least two features.

23. The one or more storage devices of claim 6, wherein the method further comprises automatically adjusting initial orientation to an adjusted orientation.

24. The one or more storage devices of claim 6, wherein the method further comprises automatically providing an option for adjusting initial orientation to a suggested orientation.

25. The method of claim 5, wherein the tracking comprises tracking said face within said main digital image and said collection of one or more low resolution images.

26. The method of claim 25, wherein said identifying of said group or sub-groups of pixels, or both, is based on said tracking of said face within said main digital image and said collection of one or more low resolution images.