Eye Beautification

• US 20110002506A1
• Filed: 06/30/2010
• Published: 01/06/2011
• Est. Priority Date: 07/30/2008
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Florea, Corneliu, Ciuc, Mihai, Capata, Adrian
• Granted Patent: US 8,520,089 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/181 involving edge growing; involving edge linking
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

Sub-regions within a face image are identified to be enhanced by applying a localized smoothing kernel to luminance data corresponding to the sub-regions of the face image. An enhanced face image is generated including an enhanced version of the face that includes certain original pixels in combination with pixels corresponding to the one or more enhanced sub-regions of the face.

Claims

1. A method of enhancing an appearance of a face within a digital image, comprising using a processor in:

acquiring a digital image of a scene including a face, including capturing the image using a lens and an image sensor of a processing device that includes the processor, or receiving said image following capture by another device that includes a lens and an image sensor, or a combination thereof;

detecting the face within the digital image;

segmenting skin tone portions of the face from face features including one or two eyes;

within at least one of the eyes, identifying a border between an iris and a sclera of the eye using luminance information;

enhancing pixels of the iris at an iris side of the border;

displaying, transmitting, communicating or digitally storing or otherwise outputting the enhanced image or a further processed version, or combinations thereof.

2. The method of claim 1, wherein the enhancing of the iris comprises linear contrast stretching.

3. The method of claim 1, wherein the enhancing of the iris comprises spatial blurring.

4. The method of claim 1, wherein the method further comprises applying one or more localized color smoothing kernels to the iris, such that the iris of the enhanced image comprises pixels modified from original pixels of the face at least by localized color smoothing.

5. The method of claim 1, wherein the method further comprises applying noise reduction or enhancement, or both, to the iris, such that the iris of the enhanced image comprises pixels modified from original pixels of the face at least by localized noise reduction or enhancement, or both.

6. The method of claim 1, wherein the method further comprises finding a border point as a maximum gradient point along a horizontal line through an upright eye, and growing the border including adding immediate neighboring points characterized by largest horizontal intensity gradients.

7. The method of claim 1, wherein the method further comprises identifying one or more further segments of a periphery of the sclera.

8. The method of claim 7, wherein the method further comprises enhancing pixels of the sclera within the periphery.

9. The method of claim 7, wherein the identifying of one or more further segments of a periphery of the sclera comprises determining points along multiple horizontal lines, wherein the points are located each at a next local intensity minimum that follows a local intensity maximum.

10. The method of claim 1, wherein the method further comprises enhancing pixels of the sclera at a sclera side of the border.

11. The method of claim 1, wherein the identifying said border between said iris and said sclera of the eye comprises not using color information.

12. A method of enhancing an appearance of a face within a digital image, comprising using a processor in:

acquiring a digital image of a scene including a face, including capturing the image using a lens and an image sensor of a processing device that includes the processor, or receiving said image following capture by another device that includes a lens and an image sensor, or a combination thereof;

detecting the face within the digital image;

segmenting skin tone portions of the face from face features including one or two eyes;

within at least one of the eyes, identifying a border between an iris and a sclera of the eye using luminance information;

enhancing pixels of the sclera at a sclera side of the border;

displaying, transmitting, communicating or digitally storing or otherwise outputting the enhanced image or a further processed version, or combinations thereof.

13. The method of claim 12, wherein the method further comprises applying one or more localized color smoothing kernels to the sclera, such that the sclera of the enhanced image comprises pixels modified from original pixels of the face at least by localized color smoothing.

14. The method of claim 12, wherein the method further comprises applying noise reduction or enhancement, or both, to the sclera, such that the sclera of the enhanced image comprises pixels modified from original pixels of the face at least by localized noise reduction or enhancement, or both.

15. The method of claim 12, wherein the method further comprises identifying one or more further segments of a periphery of the sclera.

16. The method of claim 15, wherein the enhancing pixels of the sclera comprises utilizing information regarding said periphery.

17. The method of claim 15, wherein the identifying of one or more further segments of a periphery of the sclera comprises determining points along multiple horizontal lines, wherein the points are located each at a next local intensity minimum that follows a local intensity maximum.

18. The method of claim 12, wherein the identifying said border between said iris and said sclera of the eye comprises not using color information.

19. A digital image acquisition device, comprising a lens, an image sensor and a processor, and a processor-readable memory having embodied therein processor-readable code for programming the processor to perform a method of enhancing an appearance of a face within a digital image, wherein the method comprises:

acquiring a digital image of a scene including a face, including capturing the image using the lens and the image sensor, or receiving said image following capture by another device with its own lens and image sensor, or a combination thereof;

detecting the face within the digital image;

segmenting skin tone portions of the face from face features including one or two eyes;

within at least one of the eyes, identifying a border between an iris and a sclera of the eye using luminance information;

enhancing pixels of the iris at an iris side of the border;

displaying, transmitting, communicating or digitally storing or otherwise outputting the enhanced image or a further processed version, or combinations thereof.

20. The device of claim 19, wherein the enhancing of the iris comprises linear contrast stretching.

21. The device of claim 19, wherein the enhancing of the iris comprises spatial blurring.

22. The device of claim 19, wherein the method further comprises applying one or more localized color smoothing kernels to the iris, such that the iris of the enhanced image comprises pixels modified from original pixels of the face at least by localized color smoothing.

23. The device of claim 19, wherein the method further comprises applying noise reduction or enhancement, or both, to the iris, such that the iris of the enhanced image comprises pixels modified from original pixels of the face at least by localized noise reduction or enhancement, or both.

24. The device of claim 19, wherein the method further comprises finding a border point as a maximum gradient point along a horizontal line through an upright eye, and growing the border including adding immediate neighboring points characterized by largest horizontal intensity gradients.

25. The device of claim 19, wherein the method further comprises identifying one or more further segments of a periphery of the sclera.

26. The device of claim 25, wherein the method further comprises enhancing pixels of the sclera within the periphery.

27. The device of claim 25, wherein the identifying of one or more further segments of a periphery of the sclera comprises determining points along multiple horizontal lines, wherein the points are located each at a next local intensity minimum that follows a local intensity maximum.

28. The device of claim 19, wherein the method further comprises enhancing pixels of the sclera at a sclera side of the border.

29. The device of claim 19, wherein the identifying said border between said iris and said sclera of the eye comprises not using color information.

30. A digital image acquisition device, comprising a lens, an image sensor and a processor, and a processor-readable memory having embodied therein processor-readable code for programming the processor to perform a method of enhancing an appearance of a face within a digital image, wherein the method comprises:

acquiring a digital image of a scene including a face, including capturing the image using the lens and the image sensor, or receiving said image following capture by another device, or a combination thereof;

detecting the face within the digital image;

segmenting skin tone portions of the face from face features including one or two eyes;

within at least one of the eyes, identifying a border between an iris and a sclera of the eye using luminance information;

enhancing pixels of the sclera at a sclera side of the border;

displaying, transmitting, communicating or digitally storing or otherwise outputting the enhanced image or a further processed version, or combinations thereof.

31. The device of claim 30, wherein the method further comprises applying one or more localized color smoothing kernels to the sclera, such that the sclera of the enhanced image comprises pixels modified from original pixels of the face at least by localized color smoothing.

32. The device of claim 30, wherein the method further comprises applying noise reduction or enhancement, or both, to the sclera, such that the sclera of the enhanced image comprises pixels modified from original pixels of the face at least by localized noise reduction or enhancement, or both.

33. The device of claim 30, wherein the method further comprises identifying one or more further segments of a periphery of the sclera.

34. The device of claim 33, wherein the enhancing pixels of the sclera comprises utilizing information regarding said periphery.

35. The device of claim 33, wherein the identifying of one or more further segments of a periphery of the sclera comprises determining points along multiple horizontal lines, wherein the points are located each at a next local intensity minimum that follows a local intensity maximum.

36. The device of claim 30, wherein the identifying said border between said iris and said sclera of the eye comprises not using color information.

37. One or more computer-readable storage media having embodied therein code for programming one or more processors to perform a method of enhancing an appearance of a face within a digital image, wherein the method comprises:

acquiring a digital image of a scene including a face, including capturing the image using a lens and an image sensor of a processing device that includes the processor, or receiving said image following capture by another device that includes a lens and an image sensor, or a combination thereof;

detecting the face within the digital image;

segmenting skin tone portions of the face from face features including one or two eyes;

within at least one of the eyes, identifying a border between an iris and a sclera of the eye using luminance information;

enhancing pixels of the iris at an iris side of the border;

displaying, transmitting, communicating or digitally storing or otherwise outputting the enhanced image or a further processed version, or combinations thereof.

38. The one or more computer-readable storage media of claim 37, wherein the enhancing of the iris comprises linear contrast stretching.

39. The one or more computer-readable storage media of claim 37, wherein the enhancing of the iris comprises spatial blurring.

40. The one or more computer-readable storage media of claim 37, wherein the method further comprises applying one or more localized color smoothing kernels to the iris, such that the iris of the enhanced image comprises pixels modified from original pixels of the face at least by localized color smoothing.

41. The one or more computer-readable storage media of claim 37, wherein the method further comprises applying noise reduction or enhancement, or both, to the iris, such that the iris of the enhanced image comprises pixels modified from original pixels of the face at least by localized noise reduction or enhancement, or both.

42. The one or more computer-readable storage media of claim 37, wherein the method further comprises finding a border point as a maximum gradient point along a horizontal line through an upright eye, and growing the border including adding immediate neighboring points characterized by largest horizontal intensity gradients.

43. The one or more computer-readable storage media of claim 37, wherein the method further comprises identifying one or more further segments of a periphery of the sclera.

44. The one or more computer-readable storage media of claim 43, wherein the method further comprises enhancing pixels of the sclera within the periphery.

45. The one or more computer-readable storage media of claim 43, wherein the identifying of one or more further segments of a periphery of the sclera comprises determining points along multiple horizontal lines, wherein the points are located each at a next local intensity minimum that follows a local intensity maximum.

46. The one or more computer-readable storage media of claim 37, wherein the method further comprises enhancing pixels of the sclera at a sclera side of the border.

47. The one or more computer-readable storage media of claim 37, wherein the identifying said border between said iris and said sclera of the eye comprises not using color information.

48. One or more computer-readable storage media having embodied therein code for programming one or more processors to perform a method of enhancing an appearance of a face within a digital image, wherein the method comprises:

acquiring a digital image of a scene including a face, including capturing the image using a lens and an image sensor of a processing device that includes the processor, or receiving said image following capture by another device that includes a lens and an image sensor, or a combination thereof;

detecting the face within the digital image;

segmenting skin tone portions of the face from face features including one or two eyes;

within at least one of the eyes, identifying a border between an iris and a sclera of the eye using luminance information;

enhancing pixels of the sclera at a sclera side of the border;

displaying, transmitting, communicating or digitally storing or otherwise outputting the enhanced image or a further processed version, or combinations thereof.

49. The one or more computer-readable storage media of claim 48, wherein the method further comprises applying one or more localized color smoothing kernels to the sclera, such that the sclera of the enhanced image comprises pixels modified from original pixels of the face at least by localized color smoothing.

50. The one or more computer-readable storage media of claim 48, wherein the method further comprises applying noise reduction or enhancement, or both, to the sclera, such that the sclera of the enhanced image comprises pixels modified from original pixels of the face at least by localized noise reduction or enhancement, or both.

51. The one or more computer-readable storage media of claim 48, wherein the method further comprises identifying one or more further segments of a periphery of the sclera.

52. The one or more computer-readable storage media of claim 51, wherein the enhancing pixels of the sclera comprises utilizing information regarding said periphery.

53. The one or more computer-readable storage media of claim 51, wherein the identifying of one or more further segments of a periphery of the sclera comprises determining points along multiple horizontal lines, wherein the points are located each at a next local intensity minimum that follows a local intensity maximum.

54. The one or more computer-readable storage media of claim 48, wherein the identifying said border between said iris and said sclera of the eye comprises not using color information.