Method of improving orientation and color balance of digital images using face detection information

• US 7,986,811 B2
• Filed: 12/14/2009
• Issued: 07/26/2011
• Est. Priority Date: 06/26/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Bigioi, Petronel, Corcoran, Peter
• Primary Examiner(s): Seth; Manav
• Application Number:
  US12/637,664
• Publication Number:
  US 20100220899A1
• Time in Patent Office: 589 Days
• Field of Search: 382/103, 382115-118, 382/274, 348/61, 348 77- 78, 348/143, 348/152, 348/155, 348/207.99
• US Class Current: 382/103
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 9/68 for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

A method of generating one or more new spatial and chromatic variation digital images uses an original digitally-acquired image which including a face or portions of a face. A group of pixels that correspond to a face within the original digitally-acquired image is identified. A portion of the original image is selected to include the group of pixels. Values of pixels of one or more new images based on the selected portion are automatically generated, or an option to generate them is provided, in a manner which always includes the face within the one or more new images. Such method may be implemented to automatically establish the correct orientation and color balance of an image. Such method can be implemented as an automated method or a semi automatic method to guide users in viewing, capturing or printing of images.

Claims

1. A method of digital image processing using face detection for achieving desired chromatic and luminance parameters for a face, comprising:

using a digital image acquisition device or external image processing device, or a combination thereof, that includes a processor that is programmed to perform the method, wherein the method comprises;

identifying a group of pixels that correspond to a face within a digital image;

generating in-camera, capturing or otherwise obtaining in-camera a collection of reference images including said face;

tracking said face within said collection of reference images;

identifying one or more sub-groups of pixels that correspond to one or more facial features of the face including at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof;

determining initial chrominance values of one or more chromatic parameters of pixels of the one or more sub-groups of pixels including said at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof;

determining initial luminance values of one or more luminance parameters of said pixels of the one or more sub-groups of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof;

determining at least one initial chromatic parameter based on the initial chrominance values;

determining at least one initial luminance parameter based on the initial luminance values;

determining adjusted values of the one or more chromatic parameters of the pixels of the one or more sub-groups of pixels, including said at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof, based on a comparison of the initial chromatic parameter with the desired chromatic parameter; and

determining adjusted values of the one or more luminance parameters of the pixels of the one or more sub-groups of pixels, including said at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof, based on a comparison of the initial luminance parameter with the desired luminance parameter.

2. The method of claim 1, the one or more chromatic parameters including color.

3. The method of claim 1, the one or more chromatic parameters including tone.

4. The method of claim 1, further comprising adjusting values of the pixels from the initial luminance and chromatic values to the adjusted luminance and chromatic values.

5. The method of claim 1, further comprising automatically adjusting values of the pixels from the initial luminance and chromatic values to the adjusted luminance and chromatic values.

6. The method of claim 1, further comprising automatically providing an option for adjusting values of the pixels from the initial luminance and chromatic values to the adjusted luminance and chromatic values.

13. The method of claim 1, the one or more luminance parameters including exposure.

14. The method of claim 1, the one or more luminance parameters including sharpness.

15. The method of claim 1, the one or more luminance parameters including contrast.

16. The method of claim 4, wherein the adjusting values from initial to adjusted luminance values comprises applying fill-flash.

17. The method of claim 5, wherein the adjusting values from initial to adjusted luminance values comprises applying fill-flash.

18. The method of claim 6, wherein the adjusting values from initial to adjusted luminance values comprises applying fill-flash.

7. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of digital image processing using face detection for achieving desired chromatic and luminance parameters for a face, the method comprising:

identifying a group of pixels that correspond to a face within a digital image;

generating in-camera, capturing or otherwise obtaining in-camera a collection of reference images including said face;

tracking said face within said collection of reference images;

identifying one or more sub-groups of pixels that correspond to one or more facial features of the face including at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof;

determining initial chrominance values of one or more chromatic parameters of pixels of the one or more sub-groups of pixels including said at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof;

determining initial luminance values of one or more luminance parameters of said pixels of the one or more sub-groups of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof;

determining at least one initial chromatic parameter based on the initial chrominance values;

determining at least one initial luminance parameter based on the initial luminance values;

determining adjusted values of the one or more chromatic parameters of the pixels of the one or more sub-groups of pixels, including said at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof, based on a comparison of the initial chromatic parameter with the desired chromatic parameter; and

determining adjusted values of the one or more luminance parameters of the pixels of the one or more sub-groups of pixels, including said at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof, based on a comparison of the initial luminance parameter with the desired luminance parameter.

8. The one or more storage devices of claim 7, the one or more chromatic parameters including color.

9. The one or more storage devices of claim 7, the one or more chromatic parameters including tone.

10. The one or more storage devices of claim 7, the method further comprising adjusting values of the pixels from the initial luminance and chromatic values to the adjusted luminance and chromatic values.

11. The one or more storage devices of claim 7, the method further comprising automatically adjusting values of the pixels from the initial luminance and chromatic values to the adjusted luminance and chromatic values.

12. The one or more storage devices of claim 7, the method further comprising automatically providing an option for adjusting values of the pixels from the initial luminance and chrominance values to the adjusted luminance and chrominance values.

19. The one or more storage devices of claim 7, the one or more luminance parameters including exposure.

20. The one or more storage devices of claim 7, the one or more luminance parameters including sharpness.

21. The one or more storage devices of claim 7, the one or more luminance parameters including contrast.

22. The one or more storage devices of claim 10, wherein the adjusting values from initial to adjusted luminance values comprises applying fill-flash.

23. The one or more storage devices of claim 11, wherein the adjusting values from initial to adjusted luminance values comprises applying fill-flash.

24. The one or more storage devices of claim 12, wherein the adjusting values from initial to adjusted luminance values comprises applying fill-flash.

25. A digital image acquisition device, comprise a lens, an image sensor, a processor, and a processor readable storage device having processor readable code embodied thereon for programming the processor to perform a method of digital image processing using face detection for achieving desired luminance and chromatic parameters for a face, the method comprising:

identifying a group of pixels that correspond to a face within a digital image;

generating in-camera, capturing or otherwise obtaining in-camera a collection of reference images including said face;

tracking said face within said collection of reference images;

identifying one or more sub-groups of pixels that correspond to one or more facial features of the face including at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof;

determining initial chrominance values of one or more chromatic parameters of pixels of the one or more sub-groups of pixels including said at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof;

determining initial luminance values of one or more luminance parameters of said pixels of the one or more sub-groups of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof;

determining at least one initial chromatic parameter based on the initial chrominance values;

determining at least one initial luminance parameter based on the initial luminance values;

determining adjusted values of the one or more chromatic parameters of the pixels of the one or more sub-groups of pixels, including said at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof, based on a comparison of the initial chromatic parameter with the desired chromatic parameter; and

determining adjusted values of the one or more luminance parameters of the pixels of the one or more sub-groups of pixels, including said at least one sub-group of pixels that substantially comprise one or two eyes, a mouth, a chin or a nose, or combinations thereof, based on a comparison of the initial luminance parameter with the desired luminance parameter.

26. The device of claim 25, the one or more chromatic parameters including color.

27. The device of claim 25, the one or more chromatic parameters including tone.

28. The device of claim 25, the method further comprising adjusting values of the pixels from the initial luminance and chromatic values to the adjusted luminance and chromatic values.

29. The device of claim 25, the method further comprising automatically adjusting values of the pixels from the initial luminance and chromatic values to the adjusted luminance and chromatic values.

30. The device of claim 25, the method further comprising automatically providing an option for adjusting values of the pixels from the initial luminance and chromatic values to the adjusted luminance and chromatic values.

31. The device of claim 25, the one or more luminance parameters including exposure.

32. The device of claim 25, the one or more luminance parameters including sharpness.

33. The device of claim 25, the one or more luminance parameters including contrast.

34. The device of claim 28, wherein the adjusting values from initial to adjusted luminance values comprises applying fill-flash.

35. The device of claim 29, wherein the adjusting values from initial to adjusted luminance values comprises applying fill-flash.

36. The device of claim 30, wherein the adjusting values from initial to adjusted luminance values comprises applying fill-flash.