Digital image processing using face detection information

• US 20060204055A1
• Filed: 06/26/2003
• Published: 09/14/2006
• Est. Priority Date: 06/26/2003
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Bigioi, Petronel, Corcoran, Peter
• Granted Patent: US 7,574,016 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A method of processing a digital image using face detection within the image achieves one or more desired image processing parameters. A group of pixels is identified that correspond to an image of a face within the digital image. Default values are determined of one or more parameters of at least some portion of the digital image. Values are adjusted of the one or more parameters within the digitally-detected image based upon an analysis of the digital image including the image of the face and the default values.

Claims

1. A method of processing a digital image using face detection within said image to achieve one or more desired image processing parameters, comprising the steps of:

(a) identifying a group of pixels that correspond to an image of a face within the digital image;

(b) determining default values of one or more parameters of at least some portion of said digital image; and

(c) adjusting values of the one or more parameters within the digitally-detected image based upon an analysis of said digital image including said image of said face and said default values.

2. The method of claim 1, the digital image comprising a digitally-acquired image.

3. The method of claim 1, further comprising a decision for processing said digital image based on said face detection, the decision being performed manually.

4. The method of claim 1, further comprising a decision for adjusting values of the one or more parameters, the decision being automated.

5. The method of claim 1, further comprising a decision for adjusting values of the one or more parameters, the decision being performed manually.

6. The method of claim 1, the one or more parameters including orientation, color, tone, size, luminance, relative exposure, relative spatial location, tone reproduction, sharpness or focus or combinations thereof.

7. The method of claim 6, the one or more parameters comprise of a mask that defines one or more regions where the one or more parameters are valid.

8. The method of claim 7, the mask further comprising a continuous presentation of varying strength within different sub-regions of said one or more regions.

9. The method of claim 7, said one or more parameters comprising identical parameters that differ in value based on said mask.

10. The method of claim 6, at least two parameters being concatenated into a single parameter.

11. The method of claim 6, further comprising a selection of one or more parameters, the selection being manually performed.

12. The method of claim 6, further comprising transforming said digital image based on said values of said one or more parameters.

13. The method of claim 6, further comprising creating an operation list for said digital image based on said values of said one or more parameters.

14. The method of claim 13, wherein said operation list is embedded within said digital image.

15. The method of claim 13, wherein said operation list is external to said digital image.

16. A method of processing a digital image using face detection as recited in claim 6, said adjusting the values of said orientation comprising determining a rotation value of the digital image.

17. A method of processing a digital image using face detection as recited in claim 6, said adjusting the values of said color comprising manipulating a color balance of the digital image.

18. A method of processing a digital image using face detection as recited in claim 6, said adjusting the values of said tone comprising manipulating a tonal balance of said digital image.

19. A method of processing a digital image using face detection as recited in claim 6, said adjusting the values of said size comprising adjusting a size of said image of said face relative to at least one other region of said digital image.

20. A method of processing a digital image using face detection as recited in claim 6, said adjusting the values of said luminance comprising manipulating a luminance balance of said digital image.

21. A method of processing a digital image using face detection as recited in claim 6, said adjusting the values of said relative exposure comprising digitally simulating a fill flash.

22. A method including digitally simulating a fill flash as recited in claim 21, said values being manually adjustable by a user to create one or more different degrees of said simulated fill flash.

23. A method including digitally simulating a fill flash as recited in claim 21, said analysis of said image of a face comprising a comparison of an overall exposure to an exposure around said identified face.

24. A method including digitally simulating a fill flash as recited in claim 23, said exposure being calculated based on a histogram.

25. A method of processing a digital image using face detection as recited in claim 21, said digitally simulating a fill flash further comprising optionally adjusting said tone reproduction.

26. A method of processing a digital image using face detection as recited in claim 25, said digitally simulating a fill flash further comprising optionally locally adjusting said sharpness.

27. A method including digitally simulating a fill flash as recited in claim 26, said digitally simulating a fill flash being performed by operating on one or more objects estimated to be closer to the camera.

28. A method including digitally simulating a fill flash as recited in claim 27, said objects determined to be closer to the camera comprising one or more identified faces.

29. A method including digitally simulating a fill flash as recited in claim 26, said digitally simulating a fill flash being performed on one or more objects estimated to be of higher importance in said digital image.

30. A method including digitally simulating a fill flash as recited in claim 29, whereas said objects estimated to be of higher importance in said digital image comprise of said identified faces.

31. A method of processing a digital image using face detection as recited in claim 6, said adjusting the values of said relative spatial location comprising adjusting a spatial location of said image of said face relative to at least one other region of said digital image.

32. A method of processing a digital image using face detection as recited in claim 6, said adjusting the values of said tone reproduction comprising manipulating a tonal reproduction balance within said digital image.

33. A method of processing a digital image using face detection as recited in claim 6, said adjusting the values of said sharpness comprising manipulating a sharpness balance within said digital image.

34. A method of processing a digital image using face detection as recited in claim 6, said adjusting the values of said focus comprising adjusting values of focus for enhancing a focus of said image of said face within said digital image.

35. The method of claim 1, the method being performed within a digital acquisition device.

36. The method of claim 1, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually removing a false indication of another face within the image.

37. The method of claim 2, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually adding an indication of at least one other face within the image.

38. The method of claim 1, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually verifying correct detection of at least one face within the image.

39. The method of claim 1, the digitally-detected image comprising a digitally-captured image.

40. The method of claim 39, further comprising the step of automatically providing a fill flash.

41. The method of claim 39, further comprising the step of automatically providing an option for providing a suggested fill-flash.

42. A method of digital image processing using face detection to achieve a desired image parameter, comprising the steps of:

(a) identifying a group of pixels that correspond to an image of a face within a digitally-detected image;

(b) determining initial values of one or more parameters of at least some of the pixels; and

(c) determining an initial parameter of the digitally-detected image based on the initial values; and

(d) automatically adjusting values of the one or more parameters of pixels within the digitally-detected image based upon comparison of the initial parameter with the desired parameter.

43. The method of claim 42, the digitally-detected image comprising a digitally-acquired image and the initial parameter of the digitally-detected image comprising an initial parameter of the face image.

44. The method of claim 43, the one or more parameters including orientation, color, tone, size, luminance, relative exposure, relative spatial location, tone reproduction, sharpness, or focus, or combinations thereof.

45. The method of claim 43, the method being performed within a digital camera.

46. The method of claim 43, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually removing a false indication of another face within the image.

47. The method of claim 43, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually adding an indication of another face within the image.

48. The method of claim 42, the digitally-detected image comprising a digitally-captured image and the initial parameter of the digitally-detected image comprising an initial parameter of the face image.

49. The method of claim 48, the one or more parameters including orientation, color, tone, size, luminance, relative exposure, relative spatial location, tone reproduction, sharpness, or focus, or combinations thereof.

50. The method of claim 48, the method being performed within a digital camera.

51. The method of claim 48, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually removing a false indication of another face within the image.

52. The method of claim 48, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually adding an indication of another face within the image.

53. A method of digital image processing using face detection for achieving a desired image parameter, comprising the steps of:

(a) identifying a group of pixels that correspond to a face within a digitally-detected image;

(b) determining initial values of one or more parameters of pixels of the group of pixels;

(c) determining an initial parameter of the digitally-detected image based on the initial values; and

(d) automatically providing an option for adjusting values of the one or more parameters of pixels within the digitally-detected image based upon comparison of the initial parameter with the desired parameter.

54. The method of claim 53, the digitally detected image comprising a digitally-acquired image and the initial parameter of the digitally-detected image comprising an initial parameter of the face image.

55. The method of claim 54, the one or more parameters including at least one parameter selected from the group consisting of orientation, color, tone, size, luminance, and focus.

56. The method of claim 54, the method being performed within a digital camera.

57. The method of claim 54, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually removing a false indication of another face within the image.

58. The method of claim 54, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually adding an indication of another face within the image.

59. The method of claim 53, the digitally-detected image comprising a digitally-captured image and the initial parameter of the digitally-detected image comprising an initial parameter of the face image.

60. The method of claim 59, the one or more parameters including orientation, color, tone, size, luminance, relative exposure, relative spatial location, tone reproduction, sharpness, or focus, or combinations thereof.

61. The method of claim 59, the method being performed within a digital camera.

62. The method of claim 59, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually removing a false indication of another face within the image.

63. The method of claim 59, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually adding an indication of another face within the image.

64. Within a digital camera, a method of digital image processing using face detection for achieving a desired image parameter, comprising the steps of:

(a) identifying a group of pixels that correspond to a face within a digital image;

(b) determining first initial values of a parameter of pixels of the group of pixels;

(c) determining second initial values of a parameter of pixels other than pixels of the group of pixels;

(d) comparing the first and second initial values; and

(e) determining adjusted values of the parameter based on the comparing of the first and second initial values and on a comparison of the parameter corresponding to at least one of the first and second initial values and the desired image parameter.

65. The method of claim 64, the parameter comprising luminance, the method further comprising the step of automatically generating the adjusted digital image using the adjusted values.

66. The method of claim 64, the parameter comprising luminance, the method further comprising automatically providing an option to generate the adjusted digital image using the adjusted values.

67. The method of claim 64, the parameter comprising luminance, the adjusted values of the luminance being provided by a fill flash.

68. The method of claim 64, the parameter comprising luminance, the adjusted values of the luminance being provided by a digitally-simulated fill flash.

69. Within a digital camera, method of digital image processing using face detection to achieve a desired luminance contrast, comprising the steps of:

(a) identifying a group of pixels that correspond to a face within a digital image;

(b) determining first initial values of luminance of pixels of the group of pixels;

(c) determining second initial values of luminance of pixels other than pixels of the group of pixels;

(d) comparing the first and second initial values to determine an initial luminance contrast; and

(e) determining properties of a fill flash for providing adjusted values of luminance for at least some of the pixels of the digital image based on a comparison of the initial luminance contrast and the desired luminance contrast.

70. The method of claim 69, the digitally-detected image comprising a digitally-captured image.

71. The method of claim 70, further comprising the step of automatically providing the determined fill flash.

72. The method of claim 70, further comprising the step of automatically providing an option for providing a suggested fill-flash.

73. The method of claim 70, the parameter comprising luminance, the method further comprising the step of automatically generating the adjusted digital image using the adjusted values.

74. The method of claim 70, the parameter comprising luminance, the method further comprising automatically providing an option to generate the adjusted digital image using the adjusted values.

75. The method of claim 70, the parameter comprising luminance, the adjusted values of the luminance being provided by a fill flash.

76. The method of claim 70, the parameter comprising luminance, the adjusted values of the luminance being provided by a digitally-simulated fill flash.

77. The method of claim 69, the digitally-detected image comprising a digitally-acquired image.

78. The method of claim 77, further comprising the step of automatically providing a determined fill flash.

79. The method of claim 77, further comprising the step of automatically providing an option for providing a suggested fill-flash.

80. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of processing a digital image using face detection in said image to achieve a desired image processing parameter, the method comprising:

(a) identifying a group of pixels that correspond to an image of a face within the digital image;

(b) determining default values of one or more parameters of at least some portion of said digital image; and

(c) adjusting values of the one or more parameters within the digitally-detected image based upon an analysis of said digital image including said image of said face and said default values.

81. The one or more storage devices of claim 80, the digital image comprising a digitally-acquired image.

82. The one or more storage devices of claim 80, the method further comprising manually deciding processing of said digital image based on said face detection.

83. The one or more storage devices of claim 80, the method further comprising an automated decision for adjusting values of the one or more parameters.

84. The one or more storage devices of claim 80, the method further comprising manually deciding adjusting values of the one or more parameters.

85. The one or more storage devices of claim 80, the one or more parameters including orientation, color, tone, size, luminance, relative exposure, relative spatial location, tone reproduction, sharpness or focus or combinations thereof.

86. The one or more storage devices of claim 85, the one or more parameters comprising of a mask that defines regions where the parameters are valid.

87. The one or more storage devices of claim 86, where the mask further comprises a continuous presentation of varying strength within different sub-regions of said region.

88. The one or more storage devices of claim 86, said one or more parameters comprising same parameters differing in value based on said mask.

89. The one or more storage devices of claim 85, at least two parameters being concatenated into a single parameter.

90. The one or more storage devices of claim 85, the method further comprising manually selecting one or more parameters.

91. The one or more storage devices of claim 85, the method further comprising transforming said digital image based on said values of said one or more parameters.

92. The one or more storage devices of claim 85, the method further comprising creating an operation list for said digital image based on said values of said one or more parameters.

93. The one or more storage devices of claim 92, wherein said operation list is embedded in said digital image.

94. The one or more storage devices of claim 92, wherein said operation list is external to said digital image.

95. The one or more storage devices of claim 85, said adjusting the values of said orientation comprising determining a rotation value of the digital image.

96. The one or more storage devices of claim 85, said adjusting the values of said color comprising manipulating a color balance of the digital image.

97. The one or more storage devices of claim 85, said adjusting the values of said tone comprising manipulating a tonal balance of said digital image.

98. The one or more storage devices of claim 85, said adjusting the values of said size comprising adjusting a size of said image of said face relative to at least one other region of said digital image.

99. The one or more storage devices of claim 85, said adjusting the values of said luminance comprising manipulating a luminance balance of said digital image.

100. The one or more storage devices of claim 85, said adjusting the values of said relative exposure comprising digitally simulating a fill flash.

101. The one or more storage devices of claim 100, said values being manually adjustable by a user to create one or more selected degrees of said simulated fill flash.

102. The one or more storage devices of claim 100, said analysis of said image of a face comprising comparing an overall exposure to an exposure around said identified face.

103. The one or more storage devices of claim 102, said exposure being calculated based on a histogram.

104. The one or more storage devices of claim 100, said digitally simulating a fill flash further comprising optionally adjusting said tone reproduction.

105. The one or more storage devices of claim 104, said digitally simulating a fill flash further comprising optionally locally adjusting said sharpness.

106. The one or more storage devices of claim 105, said digitally simulating a fill flash being performed by operating on objects estimated to be closer to the camera.

107. The one or more storage devices of claim 106, said objects determined to be closer to the camera comprising one or more identified faces.

108. The one or more storage devices of claim 105, said digitally simulating a fill flash being performed on objects estimated to be of higher importance in said digital image.

109. The one or more storage devices of claim 108, said objects estimated to be of higher importance in said digital image comprising one or more identified faces.

110. The one or more storage devices of claim 85, said adjusting the values of said relative spatial location comprising adjusting a spatial location of said image of said face relative to at least one other region of said digital image.

111. The one or more storage devices of claim 85, whereas said adjusting the values of said tone reproduction comprises manipulating a tonal reproduction balance within said digital image.

112. The one or more storage devices of claim 85, said adjusting the values of said sharpness comprising manipulating a sharpness balance within said digital image.

113. The one or more storage devices of claim 85, said adjusting the values of said focus comprising adjusting values of focus for enhancing a focus of said image of said face within said digital image.

114. The one or more storage devices of claim 80, the method being performed within a digital acquisition device.

115. The one or more storage devices of claim 80, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually removing a false indication of another face within the image.

116. The one or more storage devices of claim 80, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually adding an indication of at least one other face within the image.

117. The one or more storage devices of claim 80, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually verifying correct detection of at least one face within the image.

118. The one or more storage devices of claim 80, the digitally-detected image comprising a digitally-captured image.

119. The one or more storage devices of claim 118, further comprising the step of automatically providing a fill flash.

120. The one or more storage devices of claim 118, further comprising the step of automatically providing an option for providing a suggested fill-flash.

121. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of digital image processing using face detection to achieve a desired image parameter, comprising the steps of:

(a) identifying a group of pixels that correspond to an image of a face within a digitally-detected image;

(b) determining initial values of one or more parameters of at least some of the pixels; and

(c) determining an initial parameter of the digitally-detected image based on the initial values; and

(d) automatically adjusting values of the one or more parameters of pixels within the digitally-detected image based upon comparison of the initial parameter with the desired parameter.

122. The one or more storage devices of claim 121, the digitally-detected image comprising a digitally-acquired image and the initial parameter of the digitally-detected image comprising an initial parameter of the face image.

123. The one or more storage devices of claim 122, the one or more parameters including orientation, color, tone, size, luminance, relative exposure, relative spatial location, tone reproduction, sharpness, or focus, or combinations thereof.

124. The one or more storage devices of claim 122, the method being performed within a digital camera.

125. The one or more storage devices of claim 122, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually removing a false indication of another face within the image.

126. The one or more storage devices of claim 122, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually adding an indication of another face within the image.

127. The one or more storage devices of claim 121, the digitally-detected image comprising a digitally-captured image and the initial parameter of the digitally-detected image comprising an initial parameter of the face image.

128. The one or more storage devices of claim 127, the one or more parameters including orientation, color, tone, size, luminance, relative exposure, relative spatial location, tone reproduction, sharpness, or focus, or combinations thereof.

129. The one or more storage devices of claim 127, the method being performed within a digital camera.

130. The one or more storage devices of claim 127, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually removing a false indication of another face within the image.

131. The one or more storage devices of claim 127, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually adding an indication of another face within the image.

132. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of digital image processing using face detection to achieve a desired image parameter, comprising the steps of:

(a) identifying a group of pixels that correspond to a face within a digitally-detected image;

(b) determining initial values of one or more parameters of pixels of the group of pixels;

(c) determining an initial parameter of the digitally-detected image based on the initial values; and

(d) automatically providing an option for adjusting values of the one or more parameters of pixels within the digitally-detected image based upon comparison of the initial parameter with the desired parameter.

133. The one or more storage devices of claim 132, the digitally detected image comprising a digitally-acquired image and the initial parameter of the digitally-detected image comprising an initial parameter of the face image.

134. The one or more storage devices of claim 133, the one or more parameters including at least one parameter selected from the group consisting of orientation, color, tone, size, luminance, relative exposure, relative spatial location, tone reproduction, sharpness, or focus, or combinations thereof.

135. The one or more storage devices of claim 133, the method being performed within a digital camera.

136. The one or more storage devices of claim 133, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually removing a false indication of another face within the image.

137. The one or more storage devices of claim 133, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually adding an indication of another face within the image.

138. The one or more storage devices of claim 132, the digitally-detected image comprising a digitally-captured image and the initial parameter of the digitally-detected image comprising an initial parameter of the face image.

139. The one or more storage devices of claim 138, the one or more parameters including orientation, color, tone, size, luminance, relative exposure, relative spatial location, tone reproduction, sharpness, or focus, or combinations thereof.

140. The one or more storage devices of claim 138, the method being performed within a digital camera.

141. The one or more storage devices of claim 138, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually removing a false indication of another face within the image.

142. The one or more storage devices of claim 138, the face pixels identifying step being automatically performed by an image processing apparatus, the method further comprising manually adding an indication of another face within the image.

143. Within a digital camera, one or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of digital image processing using face detection for achieving a desired image parameter, the method comprising:

(a) identifying a group of pixels that correspond to a face within a digital image;

(b) determining first initial values of a parameter of pixels of the group of pixels;

(c) determining second initial values of a parameter of pixels other than pixels of the group of pixels;

(d) comparing the first and second initial values; and

(e) determining adjusted values of the parameter based on the comparing of the first and second initial values and on a comparison of the parameter corresponding to at least one of the first and second initial values and the desired image parameter.

144. The one or more storage devices of claim 143, the parameter comprising luminance, the method further comprising the step of automatically generating the adjusted digital image using the adjusted values.

145. The one or more storage devices of claim 143, the parameter comprising luminance, the method further comprising automatically providing an option to generate the adjusted digital image using the adjusted values.

146. The one or more storage devices of claim 143, the parameter comprising luminance, the adjusted values of the luminance being provided by a fill flash.

147. The one or more storage devices of claim 143, the parameter comprising luminance, the adjusted values of the luminance being provided by a digitally-simulated fill flash.

148. Within a digital camera, one or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of digital image processing using face detection for achieving a desired image parameter, the method comprising:

(a) identifying a group of pixels that correspond to a face within a digital image;

(b) determining first initial values of luminance of pixels of the group of pixels;

(c) determining second initial values of luminance of pixels other than pixels of the group of pixels;

(d) comparing the first and second initial values to determine an initial luminance contrast; and

(e) determining properties of a fill flash for providing adjusted values of luminance for at least some of the pixels of the digital image based on a comparison of the initial luminance contrast and the desired luminance contrast.

149. The one or more storage devices of claim 148, the digitally-detected image comprising a digitally-captured image.

150. The one or more storage devices of claim 149, the method further comprising automatically providing the determined fill flash.

151. The one or more storage devices of claim 149, the method further comprising automatically providing an option for providing a suggested fill-flash.

152. The one or more storage devices of claim 149, the parameter comprising luminance, the method further comprising the step of automatically generating the adjusted digital image using the adjusted values.

153. The one or more storage devices of claim 149, the parameter comprising luminance, the method further comprising automatically providing an option to generate the adjusted digital image using the adjusted values.

154. The one or more storage devices of claim 149, the parameter comprising luminance, the adjusted values of the luminance being provided by a fill flash.

155. The one or more storage devices of claim 149, the parameter comprising luminance, the adjusted values of the luminance being provided by a digitally-simulated fill flash.

156. The one or more storage devices of claim 148, the digitally-detected image comprising a digitally-acquired image.

157. The one or more storage devices of claim 156, further comprising the step of automatically providing a determined fill flash.

158. The one or more storage devices of claim 156, further comprising the step of automatically providing an option for providing a suggested fill-flash.