Optimized performance and performance for red-eye filter method and apparatus

• US 20050140801A1
• Filed: 02/04/2004
• Published: 06/30/2005
• Est. Priority Date: 08/05/2003
• Status: Abandoned Application

• Current Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Capata, Adrian, Nanu, Florin, Drimbarean, Alexandru, Bigioi, Petronel, Corcoran, Peter, Pososin, Alexei
• Application Number:
  US10/773,092
• Publication Number:
  US 20050140801A1
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G03B 15/05 Combinations of cameras with electronic flash apparatus; Electronic flash units
• CPC Class Codes: G03B 2215/05 Combinations of cameras with electronic flash units
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A digital camera has an integral flash and stores and displays a digital image. Under certain conditions, a flash photograph taken with the camera may result in a red-eye phenomenon due to a reflection within an eye of a subject of the photograph. A digital apparatus has a red-eye filter which analyzes the stored image for the red-eye phenomenon and modifies the stored image to eliminate the red-eye phenomenon by changing the red area to black. The modification of the image is enabled when a photograph is taken under conditions indicative of the red-eye phenomenon. The modification is subject to anti-falsing analysis which further examines the area around the red-eye area for indicia of the eye of the subject. The detection and correction can be optimized for performance and quality by operating on subsample versions of the image when appropriate.

Claims

1. A digital apparatus comprising a red-eye filter for modifying an area within a digitized image indicative of a red-eye phenomenon based on an analysis of a subsample representation of selected regions of said digitized image.

2. The apparatus of claim 1, wherein the analysis is performed at least in part for determining said area.

3. The apparatus of claim 1, wherein the analysis is performed at least in part for determining said modifying.

4. The apparatus of claim 1, wherein said selected regions of said digitized image comprise the entire image.

5. The apparatus of claim 1, wherein said selected regions of said digitized image comprise multi resolution encoding of said image.

6. The apparatus of claim 1, wherein at least one region of the entire image is not included among said selected regions of said image.

7. The apparatus of claim 1, wherein said analysis is performed in part on a full resolution image and in part on a subsample resolution of said digital image.

8. The apparatus of claim 1, further comprising a module for changing the degree of said subsampling.

9. The apparatus of claim 8, wherein said changing the degree of said subsampling is determined empirically.

10. The apparatus of claim 8, wherein said changing the degree of said subsampling is determined based on a size of said image.

11. The apparatus of claim 8, wherein said changing the degree of said subsampling is determined based on a size of selected regions of the image.

12. The apparatus of claim 8, wherein said changing the degree of said subsampling is determined based on data obtained from the camera relating to the settings of the camera at the time of image capture.

13. The apparatus of claim 12, wherein the data obtained from the camera includes an aperture setting or focus of the camera, or both.

14. The apparatus of claim 12, wherein the data obtained from the camera includes the distance of the subject from the camera.

15. The apparatus of claim 8, wherein said changing the degree of said subsampling is determined based digitized image metadata information.

16. The apparatus of claim 8, wherein said modifying the area is performed including the full resolution of said digital image.

17. The apparatus of claim 8, wherein said red-eye filter comprises of a plurality of sub filters.

18. The apparatus of claim 17, wherein said subsampling for said sub filters operating on selected regions of said image is determined by one or more of the image size, suspected as red eye region size, filter computation complexity, empirical success rate of said sub filter, empirical false detection rate of said sub filter, falsing probability of said sub filter, relations between said suspected regions as red eye, results of previous analysis of other said sub filters.

19. The apparatus of claim 1, further comprising memory for saving said digitized image after applying said filter for modifying pixels as a modified image.

20. The apparatus of claim 1, further comprising memory for saving said subsample representation of said image.

21. The apparatus of claim 1, wherein said subsample representation of selected regions of said image is determined in hardware.

22. The apparatus of claim 1, wherein said analysis is performed in part on the full resolution image and in part on a subsample resolution of said image.

23. The apparatus of claim 1, further comprising means for changing the degree of said subsampling.

24. The apparatus of claim 23, wherein said changing the degree of said subsampling is determined empirically.

25. The apparatus of claim 23, wherein said changing the degree of said subsampling is determined based on a size of said image.

26. The apparatus of claim 23, wherein said changing the degree of said subsampling is determined based on a region size.

27. The apparatus of claim 23, wherein said changing the degree of said subsampling is determined based on a complexity of calculation for said filter.

28. The apparatus of claim 1, wherein said subsample representation is determined using spline interpolation.

29. The apparatus of claim 1, wherein said subsample representation is determined using bi-cubic interpolation.

30. The apparatus of claim 1, wherein said modifying the area is performed on the full resolution of said image.

31. The apparatus of claim 1, wherein said red-eye filter comprises a plurality of sub-filters.

32. The apparatus according to claim 31, wherein said subsampling for said sub-filters operating on selected regions of said image is determined by one or more of the image size, a suspected red eye region size, filter computation complexity, empirical success rate of said sub-filter, empirical false detection rate of said sub-filter, falsing probability of said sub-filter, relations between said suspected red eye regions, or results of previous analysis of one or more other sub-filters.

33. A digital apparatus, comprising:

(a) an image store for holding;

(i) a temporary copy of an unprocessed image known as a pre-capture image;

(ii) a permanent copy of a digitally processed, captured image, and

(iii) a subsample representation of selected regions of the pre-capture image; and

(b) a red-eye filter for modifying an area within said at least one of the images indicative of a red-eye phenomenon based on an analysis of the subsample representation.

34. The apparatus of claim 33, wherein said at least one of the images comprises the digitally processed, captured image.

35. The apparatus of claim 34, wherein said subsample representation of selected regions of said image is determined in hardware.

36. The apparatus of claim 34, wherein said analysis is performed in part on the full resolution image and in part on a subsample resolution of said image.

37. The apparatus of claim 34, further comprising a module for changing the degree of said subsampling.

38. The apparatus of claim 37, wherein said changing the degree of said subsampling is determined empirically.

39. The apparatus of claim 37, wherein said changing the degree of said subsampling is determined based on a size of said image.

40. The apparatus of claim 37, wherein said changing the degree of said subsampling is determined based on a region size.

41. The apparatus of claim 37, wherein said changing the degree of said subsampling is determined based on a complexity of calculation for said red eye filter.

42. The apparatus of claim 37, wherein said subsample representation is determined using a spline interpolation.

43. The apparatus of claim 37, wherein said subsample representation is determined using bi-cubic interpolation.

44. The apparatus of claim 37, wherein said changing the degree of said subsampling is determined based on data obtained from the camera relating to the settings of the camera at the time of image acquisition.

45. The apparatus of claim 44, wherein the data obtained from the camera includes an aperture setting or focus of the camera, or both.

46. The apparatus of claim 44, wherein the data obtained from the camera includes the distance of the subject from the camera.

47. The apparatus of claim 37, wherein said changing the degree of said subsampling is determined based on data obtained from the camera relating to image processing analysis of said precapture images.

48. The apparatus of claim 47, wherein said image processing analysis is based on histogram data obtained from said pre-capture image.

49. The apparatus of claim 47, wherein said image processing analysis is based on color correlogram data obtained from said pre-capture image.

50. The apparatus of claim 47, wherein said image processing analysis is based on global luminance or white balance image data, or both, obtained from said pre-capture image.

51. The apparatus of claim 47, wherein said image processing analysis is based on face detection analysis of said pre-capture image.

52. The apparatus of claim 47, wherein said image processing analysis is based on determining pixel regions with a color characteristic indicative of redeye.

53. The apparatus of claim 47, wherein said image processing analysis is performed in hardware.

54. The apparatus of claim 37, wherein said changing the degree of said subsampling is determined based on image metadata information.

55. The apparatus of claim 34, wherein said modifying the area is performed including the full resolution of said image.

56. The apparatus of claim 34, wherein said red-eye filter comprises a plurality of sub filters.

57. A method of filtering a red eye phenomenon from a digitized image comprising a multiplicity of pixels indicative of color, the method comprising determining whether one or more regions within a subsample representation of said digitized image are suspected as including red eye artifact.

58. The method of claim,57, further comprising varying a degree of the subsample representation for each region of said one or more regions based on said image.

59. The method of claim 57, further comprising generating the subsample representation based on said image.

60. The method of claim 57, further comprising generating the subsample presentation utilizing a hardware-implemented subsampling engine.

61. The method of claim 57, further comprising testing one or more regions within said subsample representation determined as including red eye artifact for determining any false redeye groupings.

62. The method of claim 57, further comprising

(c) associating said one or more regions within said subsample presentation of said image with one or more corresponding regions within said image;

and

(d) modifying said one or more corresponding regions within said image.

63. The method of claim 57, wherein the determining comprises analyzing meta-data information including image acquisition device-specific information.

64. The method of claim 57, further comprising analyzing the subsample representation of selected regions of said digitized image, and modifying an area determined to include red eye artifact.

65. The method of claim 64, wherein the analysis is performed at least in part for determining said area.

66. The method of claim 64, wherein the analysis is performed at least in part for determining said modifying.

67. The method of claim 64, wherein said selected regions of said digitized image comprise the entire image.

68. The method of claim 64, wherein said selected regions of said digitized image comprise multi resolution encoding of said image.

69. The method of claim 64, wherein at least one region of the entire image is not included among said selected regions of said image.

70. The method of claim 64, wherein said analysizing is performed in part on a full resolution image and in part on a subsample resolution of said image.

71. The method of claim 64, further comprising changing the degree of said subsampling.

72. The method of claim 71, wherein said changing the degree of said subsampling is determined empirically.

73. The method of claim 71, wherein said changing the degree of said subsampling is determined based on a size of said image.

74. The method of claim 71, wherein said changing the degree of said subsampling is determined based on a size of selected regions.

75. The method of claim 64, further comprising saving said digitized image after applying said filter for modifying pixels as a modified image.

76. The method of claim 64, further comprising saving said subsample representation of said image.

77. The method of claim 64, further comprising determining said subsample representation of said image in hardware.

78. The method of claim 64, further comprising determining said subsample representation using spline interpolation.

79. The method of claim 64, further comprising determining said subsample representation using bi-cubic interpolation.

80. The method of claim 64, wherein said modifying of the area is performed including the full resolution of said image.

81. The method of claim 57, further comprising determining said subsample representation utilizing a plurality of sub-filters.

82. The method of claim 81, wherein said subsampling for said sub-filters operating on selected regions of said image is determined by one or more of the image size, a suspected red eye region size, filter computation complexity, empirical success rate of said sub-filter, empirical false detection rate of said sub-filter, falsing probability of said sub-filter, relations between said suspected red eye regions, or results of previous analysis of one or more other sub-filters.