Image modification based on red-eye filter analysis

• US 7,352,394 B1
• Filed: 02/04/2004
• Issued: 04/01/2008
• Est. Priority Date: 10/09/1997
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Drimbarean, Alexandru, Pososin, Alexei, Steinberg, Eran, Prilutsky, Yury, Capata, Adrian, Nanu, Florin, Bigioi, Petronel, Corcoran, Peter, DeLuca, Michael J.
• Primary Examiner(s): Vu; Ngoc-Yen
• Assistant Examiner(s): QUIETT, CARRAMAH J
• Application Number:
  US10/772,767
• Time in Patent Office: 1,518 Days
• Field of Search: 348/222.1, 348/241, 348/246, 348/247, 348/370, 348/371, 348/576, 382/275, 382/167, 382/117
• US Class Current: 348/241
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A digital camera has an integral flash and stores and displays a digital image. Under certain conditions, a flash photograph taken with the camera may result in a red-eye phenomenon due to a reflection within an eye of a subject of the photograph. A digital apparatus has a red-eye filter which analyzes the stored image for the red-eye phenomenon and modifies the stored image to eliminate the red-eye phenomenon by changing the red area to black. The modification of the image is enabled when a photograph is taken under conditions indicative of the red-eye phenomenon. The modification is subject to anti-falsing analysis which further examines the area around the red-eye area for indicia of the eye of the subject. The detection and correction can be optimized for performance and quality by operating on subsample versions of the image when appropriate.

Claims

1. A portable digital camera having no photographic film, comprising:

(a) an integral flash for providing illumination during image acquisition;

(b) a digital image capturing apparatus for acquiring an image; and

(c) a red-eye filter for modifying an area within the image indicative of a red-eye phenomenon based on an analysis of a subsample representation of selected regions of the image.

2. The camera of claim 1, wherein the analysis is performed at least in part for determining said area.

3. The camera of claim 1, wherein the analysis is performed at least in part for determining said modifying.

4. The camera of claim 1, wherein said selected regions of said acquired image comprise the entire image.

5. The camera of claim 1, wherein said selected regions of said acquired image comprise multi resolution encoding of said image.

6. The camera of claim 1, wherein at least one region of the entire image is not included among said selected regions of said image.

7. The camera of claim 1, wherein said analysis is performed in part on a full resolution image and in part on a subsample resolution of said image.

8. The camera of claim 1, further comprising a module for changing the degree of said subsampling.

9. The camera of claim 8, wherein said changing the degree of said subsampling is determined empirically.

10. The camera of claim 8, wherein said changing the degree of said subsampling is determined based on a size of said image.

11. The camera of claim 8, wherein said changing the degree of said subsampling is determined based on a size of selected regions of the image.

12. The camera of claim 8, wherein said changing the degree of said subsampling is determined based on data obtained from the camera relating to the settings of the camera at the time of image capture.

13. The camera of claim 12, wherein the data obtained from the camera includes an aperture setting or focus of the camera, or both.

14. The camera of claim 12, wherein the data obtained from the camera includes the distance of the subject from the camera.

15. The camera of claim 8, wherein said changing the degree of said subsampling is determined based image metadata information.

16. The camera of claim 8, wherein said modifying the area is performed including the full resolution of said image.

17. The camera of claim 8, wherein said red-eye filter comprises of a plurality of sub filters.

18. The camera of claim 17, wherein said subsampling for said sub filters operating on selected regions of said image is determined by one or more of the image size, suspected as red eye region size, filter computation complexity, empirical success rate of said sub filter, empirical false detection rate of said sub filter, falsing probability of said sub filter, relations between said suspected regions as red eye, results of previous analysis of other said sub filters.

19. The camera of claim 1, further comprising memory for saving said image after applying said filter for modifying pixels as a modified image.

20. The camera of claim 1, further comprising memory for saving said subsample representation of said image.

21. The camera of claim 1, wherein said subsample representation of selected regions of said image is determined in hardware.

22. The camera of claim 1, wherein said analysis is performed in part on the full resolution image and in part on a subsample resolution of said image.

23. The camera of claim 1, further comprising means for changing the degree of said subsampling.

24. The camera of claim 23, wherein said changing the degree of said subsampling is determined empirically.

25. The camera of claim 23, wherein said changing the degree of said subsampling is determined based on a size of said image.

26. The camera of claim 23, wherein said changing the degree of said subsampling is determined based on a region size.

27. The camera of claim 23, wherein said changing the degree of said subsampling is determined based on a complexity of calculation for said filter.

28. The camera of claim 1, wherein said subsample representation is determined using spline interpolation.

29. The camera of claim 1, wherein said subsample representation is determined using bi-cubic interpolation.

30. The camera of claim 1, wherein said modifying the area is performed on the full resolution of said image.

31. The camera of claim 1, wherein said red-eye filter comprises a plurality of sub-filters.

32. The camera according to claim 31, wherein said subsampling for said sub-filters operating on selected regions of said image is determined by one or more of the image size, a suspected red eye region size, filter computation complexity, empirical success rate of said sub-filter, empirical false detection rate of said sub-filter, falsing probability of said sub-filter, relations between said suspected red eye regions, or results of previous analysis of one or more other sub-filters.

33. The camera of claim 1, further comprising:

(d) a pixel locator for locating pixels having a color indicative of the red-eye phenomenon;

(e) a shape analyzer for determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

(f) a pixel modifier for modifying the color of the pixels within the grouping.

34. The camera of claim 33, further comprising a falsing analyzer for further processing the image in a vicinity of the grouping for details indicative of an eye, and for enabling the pixel modifier in response thereto.

35. The camera of claim 33, further comprising an exposure analyzer for determining if the image was acquired in a condition indicative of the red-eye phenomenon.

36. A portable digital camera having no photographic film, comprising:

(a) an integral flash for providing illumination during image recording;

(b) a digital image capturing apparatus for recording an image; and

(c) a red-eye filter for modifying an area within the image indicative of a red-eye phenomenon based on an analysis of a subsample representation of selected regions of the image.

37. The camera of claim 36, wherein the analysis is performed at least in part for determining said area.

38. The camera of claim 36, wherein the analysis is performed at least in part for determining said modifying.

39. The camera of claim 36, wherein said selected regions of said recorded image comprise the entire image.

40. The camera of claim 36, wherein said selected regions of said recorded image comprise multi resolution encoding of said image.

41. The camera of claim 36, wherein at least one region of the entire image is not included among said selected regions of said image.

42. The camera of claim 36, wherein said analysis is performed in part on a full resolution image and in part on a subsample resolution of said image.

43. The camera of claim 36, further comprising a module for changing the degree of said subsampling.

44. The camera of claim 43, wherein said changing the degree of said subsampling is determined empirically.

45. The camera of claim 43, wherein said changing the degree of said subsampling is determined based on a size of said image.

46. The camera of claim 43, wherein said changing the degree of said subsampling is determined based on a size of selected regions of the image.

47. The camera of claim 43, wherein said changing the degree of said subsampling is determined based on data obtained from the camera relating to the settings of the camera at the time of image capture.

48. The camera of claim 47, wherein the data obtained from the camera includes an aperture setting or focus of the camera, or both.

49. The camera of claim 47, wherein the data obtained from the camera includes the distance of the subject from the camera.

50. The camera of claim 43, wherein said changing the degree of said subsampling is determined based image metadata information.

51. The camera of claim 43, wherein said modifying the area is performed including the full resolution of said image.

52. The camera of claim 43, wherein said red-eye filter comprises of a plurality of sub filters.

53. The camera of claim 52, wherein said subsampling for said sub filters operating on selected regions of said image is determined by one or more of the image size, suspected as red eye region size, filter computation complexity, empirical success rate of said sub filter, empirical false detection rate of said sub filter, falsing probability of said sub filter, relations between said suspected regions as red eye, results of previous analysis of other said sub filters.

54. The camera of claim 36, further comprising memory for saving said digitized image after applying said filter for modifying pixels as a modified image.

55. The camera of claim 36, further comprising memory for saving said subsample representation of said image.

56. The camera of claim 36, wherein said subsample representation of selected regions of said image is determined in hardware.

57. The camera of claim 36, wherein said analysis is performed in part on the full resolution image and in part on a subsample resolution of said image.

58. The camera of claim 36, further comprising means for changing the degree of said subsampling.

59. The camera of claim 58, wherein said changing the degree of said subsampling is determined empirically.

60. The camera of claim 58, wherein said changing the degree of said subsampling is determined based on a size of said image.

61. The camera of claim 58, wherein said changing the degree of said subsampling is determined based on a region size.

62. The camera of claim 58, wherein said changing the degree of said subsampling is determined based on a complexity of calculation for said filter.

63. The camera of claim 36, wherein said subsample representation is determined using spline interpolation.

64. The camera of claim 36, wherein said subsample representation is determined using bi-cubic interpolation.

65. The camera of claim 36, wherein said modifying the area is performed on the full resolution of said image.

66. The camera of claim 36, wherein said red-eye filter comprises a plurality of sub-filters.

67. The camera of claim 66, wherein said subsampling for said sub-filters operating on selected regions of said image is determined by one or more of the image size, a suspected red eye region size, filter computation complexity, empirical success rate of said sub-filter, empirical false detection rate of said sub-filter, falsing probability of said sub-filter, relations between said suspected red eye regions, or results of previous analysis of one or more other sub-filters.

68. The camera of claim 36, further comprising:

(d) a pixel locator for locating pixels having a color indicative of the red-eye phenomenon;

(e) a shape analyzer for determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

(f) a pixel modifier for modifying the color of the pixels within the grouping.

69. The camera of claim 68, further comprising a falsing analyzer for further processing the image in a vicinity of the grouping for details indicative of an eye, and for enabling the pixel modifier in response thereto.

70. The camera of claim 68, further comprising an exposure analyzer for determining if the image was recorded in a condition indicative of the red-eye phenomenon.