Systems and methods of detecting and correcting redeye in an image suitable for embedded applications

• US 20050047656A1
• Filed: 08/29/2003
• Published: 03/03/2005
• Est. Priority Date: 08/29/2003
• Status: Active Grant

• Current Assignee: Daniel Tretter, Huitao Luo, Jonathan Yen, Niaz R. Pavel
• Original Assignee: Daniel Tretter, Huitao Luo, Jonathan Yen, Niaz R. Pavel
• Inventors: Tretter, Daniel, Yen, Jonathan, Luo, Huitao, Pavel, Niaz R.
• Granted Patent: US 7,454,040 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

Systems and methods of detecting and correcting redeye in an image are described. In one aspect, the input image is sub-sampled to generate a thumbnail image, redeye pixel areas are detected in the thumbnail image. In another aspect, an input image having lines of pixels with original color values is processed. One or more redeye pixel areas corresponding to respective areas in the input image are detected. Each pixel in the input image corresponding to the detected redeye pixel areas is classified as a redeye pixel or a non-redeye pixel on a line-by-line basis without reference to pixels in adjacent lines. The original color values of pixels in the input image classified as redeye pixels are corrected.

Claims

1. A method of processing an input image, comprising:

sub-sampling the input image to generate a thumbnail image; and

detecting redeye pixel areas in the thumbnail image.

2. The method of claim 1, wherein detecting redeye pixel areas comprises computing measures of pixel redness in the thumbnail image, and identifying a preliminary set of candidate redeye pixel areas based on the computed pixel redness measures.

3. The method of claim 2, wherein pixel redness measures are computed based on a ratio of a measure of a red component of pixel energy to a measure of total pixel energy.

4. The method of claim 2, wherein identifying the preliminary set of candidate redeye pixel areas comprises applying a two-dimensional redness filter to the computed pixel redness measures, wherein the redness filter is operable to compute a redness score based on a central kernel pixel area and a pixel area surrounding the kernel pixel area.

5. The method of claim 4, further comprising applying a prescribed threshold to the computed redness scores to identify candidate redeye pixels.

6. The method of claim 5, wherein detecting redeye pixel areas further comprises segmenting redeye pixels by scanning a redness map of the redness measures in stripes of one or more pixel lines and tracking objects containing candidate redeye pixels connected across stripes.

7. The method of claim 4, wherein detecting redeye pixel areas further comprises filtering from the preliminary set each candidate redeye pixel area having a computed redness contrast relative to at least one respective neighboring pixel area less than a prescribed redness contrast threshold.

8. The method of claim 7, wherein each candidate redeye pixel area having a computed redness contrast relative to each of a set of corresponding surrounding pixel areas less than the prescribed redness contrast threshold is filtered from the preliminary set.

9. The method of claim 2, wherein identifying the preliminary set of candidate redeye pixel areas comprises enlarging a given candidate redeye pixel area having a dimension below a threshold size to generate an enlarged pixel area.

10. The method of claim 9, wherein identifying the preliminary set of candidate redeye pixel areas comprises comparing the enlarged pixel area to multiple pixel areas surrounding the enlarged pixel area, and selecting a pixel area to replace the given candidate redeye pixel area from among the enlarged pixel area and the surrounding pixel areas based on measures of redness computed for each of the enlarged pixel area and the surrounding pixel areas.

11. The method of claim 2, wherein detecting redeye pixel areas further comprises filtering from the preliminary set each candidate redeye pixel area located in an area of the digital image having a computed grayscale contrast relative to at least one respective neighboring pixel area less than a prescribed grayscale contrast threshold.

12. The method of claim 11, further comprising:

computing measures of pixel grayscale in the digital image;

computing, for a given candidate redeye pixel area, a candidate iris area centered at the given candidate redeye pixel area and having a size maximizing grayscale contrast between the candidate iris area and areas surrounding the candidate iris area;

computing a measure of grayscale contrast between the candidate iris area and at least a portion of the areas surrounding the candidate iris area;

and applying a threshold to the computed grayscale contrast measure to filter candidate redeye pixel areas from the preliminary set.

13. The method of claim 2, further comprising:

identifying a pixel boundary of a pixel region surrounding a given candidate redeye pixel area;

classifying pixels within the pixel boundary as red pixels and non-red pixels by applying a threshold to the computed pixel redness measures; and

filtering the given candidate redeye pixel area from the preliminary set when a set of contiguous red pixels extends from the given candidate redeye pixel area to the pixel boundary.

14. The method of claim 13, further comprising identifying the set of contiguous pixels by scanning a redness map of the redness measures in stripes of one or more pixel lines and tracking objects containing red pixels connected across stripes.

15. The method of claim 2, further comprising filtering candidate redeye pixel areas from the preliminary set based on proportions of detected skin tone pixels in regions respectively surrounding the candidate redeye pixels areas.

16. The method of claim 2, further comprising pairing candidate redeye pixel areas in the preliminary set, and filtering unpaired candidate redeye pixels areas from the preliminary set.

17. The method of claim 16, wherein pairing candidate redeye pixel areas comprises comparing a candidate texture pattern computed for a candidate pair of candidate redeye pixel areas in the preliminary set with a reference texture pattern.

18. The method of claim 17, wherein comparing the candidate texture pattern with the reference texture pattern comprises generating a feature vector representative of the candidate texture pattern and comparing the generated feature vector with a statistical model of the reference texture pattern.

19. The method of claim 18, wherein generating the feature vector representative of the candidate texture pattern comprises mapping a candidate redeye pair region encompassing the candidate redeye pair to a standardized candidate redeye pair template.

20. The method of claim 19, wherein mapping the candidate redeye pair region comprises cropping a pixel region from a grayscale map of the thumbnail image, rotating the cropped pixel region, and scaling the rotated pixel region.

21. The method of claim 20, wherein mapping the candidate redeye pair region comprises normalizing and equalizing the scaled pixel region.

22. The method of claim 19, wherein generating the feature vector representative of the candidate texture pattern comprises converting the mapped candidate redeye pair region to the feature vector.

23. The method of claim 1, further comprising detecting redeye pixel areas in the input image, and generating a set of detected redeye pixel areas by merging redeye pixel areas detected in the input image with redeye pixel areas detected in the thumbnail image.

24. The method of claim 1, further comprising correcting redeye in the input image based on redeye pixel areas detected in the thumbnail image.

25. The method of claim 24, wherein correcting redeye comprises mapping the detected redeye pixel areas to the input image.

26. The method of claim 25, wherein correcting redeye comprises enlarging redeye pixel areas mapped to the input image.

27. The method of claim 26, wherein the mapped redeye pixel areas are enlarged by amounts decreasing inversely with respect to original sizes of the mapped redeye pixel areas.

28. The method of claim 26, further comprising cropping corners from each of the enlarged redeye pixel areas.

29. The method of claim 25, further comprising classifying pixels as redeye pixels for correction before mapping detected redeye pixel areas to the input image.

30. The method of claim 24, wherein correcting redeye comprises identifying discrete redeye pixel areas separated from eyelid regions.

31. The method of claim 30, wherein identifying discrete redeye pixel areas comprises comparing at least one redeye pixel area size dimension to a threshold.

32. The method of claim 31, wherein a discrete redeye pixel area is identified based at least in part on a prescribed fraction of a respective grayscale iris area centered at a corresponding pixel area and having a size maximizing grayscale contrast between the grayscale iris area and areas surrounding the grayscale iris area.

33. The method of claim 30, wherein correcting redeye comprises classifying pixels in each non-discrete redeye pixel area based on skin tone coloration.

34. The method of claim 24, wherein correcting redeye comprises classifying pixels in each redeye pixel area based on a redness threshold.

35. The method of claim 24, wherein pixels are classified on a pixel-by-pixel basis.

36. The method of claim 24, wherein each pixel is classified with reference to an adjacent, previously-classified pixel.

37. The method of claim 24, wherein correcting redeye comprises classifying pixels between concentric inner and outer bounding regions based on a grayscale threshold.

38. The method of claim 37, further comprising correcting original color values of pixels in a redeye pixel correction region encompassing pixels classified as redeye pixels.

39. The method of claim 38, wherein original color values of pixels in the redeye pixel correction region are corrected by desaturating original color values.

40. The method of claim 39, wherein original color values are desaturated by respective amounts varying with pixel location in the final pixel mask.

41. The method of claim 39, wherein original color values of pixels in the redeye pixel correction region are corrected by darkening the original color values.

42. The method of claim 38, further comprising correcting original color values of pixels in a smoothing region surrounding the redeye pixel correction region.

43. The method of claim 42, wherein original color values of pixels in the smoothing region are corrected by an amount decreasing with distance from the given redeye pixel correction region.

44. The method of claim 43, wherein original color values of pixels in the redeye pixel correction region are corrected without reference to position within the redeye pixel correction region.

45. The method of claim 37, further comprising computing a size of the inner bounding region between a given redeye pixel area size and a corresponding grayscale iris area size, and computing a size of the outer bounding region larger than the computed size of the inner bounding region by a predetermined relative amount.

46. The method of claim 24, wherein original color values of pixels are corrected based on integer arithmetic computations.

47. The method of claim 1, further comprising correcting redeye in the thumbnail image based on redeye pixel areas detected in the thumbnail image.

48. The method of claim 47, further comprising displaying the thumbnail image with corrected redeye, and correcting redeye in the input image based on redeye pixel areas detected in the thumbnail image in respond to a user command.

49. A method of processing an input image having lines of pixels with original color values, comprising:

detecting one or more redeye pixel areas corresponding to respective areas in the input image;

classifying each pixel in the input image corresponding to the detected redeye pixel areas as a redeye pixel or a non-redeye pixel on a line-by-line basis without reference to pixels in adjacent lines; and

correcting the original color values of pixels in the input image classified as redeye pixels.

50. The method of claim 49, wherein a pixel in a given line is classified with reference to an adjacent, previously-classified pixel in the given line.

51. The method of claim 49, wherein correcting redeye comprises identifying discrete redeye pixel areas separated from eyelid regions.

52. The method of claim 51, wherein identifying discrete redeye pixel areas comprises comparing at least one redeye pixel area size dimension to a threshold.

53. The method of claim 52, wherein a discrete redeye pixel area is identified based at least in part on a prescribed fraction of a respective grayscale iris area centered at a corresponding pixel area and having a size maximizing grayscale contrast between the grayscale iris area and areas surrounding the grayscale iris area.

54. The method of claim 51, wherein correcting redeye comprises classifying pixels in each non-discrete redeye pixel area based on skin tone coloration.

55. The method of claim 49, wherein correcting redeye comprises classifying pixels in each redeye pixel area based on a redness threshold.

56. The method of claim 49, further comprising correcting original color values of pixels classified as redeye pixels by desaturating original color values.

57. The method of claim 49, further comprising correcting original color values of pixels classified as redeye pixels by darkening the original color values.

58. A system for processing an input image, comprising a redeye detection module operable to:

sub-sample the input image to generate a thumbnail image; and

detect redeye pixel areas in the thumbnail image.

59. The system of claim 58, wherein the redeye detection module computes measures of pixel redness in the thumbnail image and identifies a preliminary set of candidate redeye pixel areas based on the computed pixel redness measures.

60. The system of claim 59, wherein the redeye detection module applies a two-dimensional redness filter to the computed pixel redness measures, wherein the redness filter is operable to compute a redness score based on a central kernel pixel area and a pixel area surrounding the kernel pixel area.

61. The system of claim 59, wherein the redeye detection module enlarges a given candidate redeye pixel area having a dimension below a threshold size to generate an enlarged pixel area.

62. The system of claim 59, wherein the redeye detection module filters from the preliminary set each candidate redeye pixel area located in an area of the digital image having a computed grayscale contrast relative to at least one respective neighboring pixel area less than a prescribed grayscale contrast threshold.

63. The system of claim 59, wherein the redeye detection module is further operable to:

identify a pixel boundary of a pixel region surrounding a given candidate redeye pixel area;

classify pixels within the pixel boundary as red pixels and non-red pixels by applying a threshold to the computed pixel redness measures; and

filter the given candidate redeye pixel area from the preliminary set when a set of contiguous red pixels extends from the given candidate redeye pixel area to the pixel boundary.

64. The system of claim 59, wherein the redeye detection module filters candidate redeye pixel areas from the preliminary set based on proportions of detected skin tone pixels in regions respectively surrounding the candidate redeye pixels areas.

65. The system of claim 59, wherein the redeye detection module pairs candidate redeye pixel areas in the preliminary set and filters unpaired candidate redeye pixels areas from the preliminary set.

66. The system of claim 58, wherein the redeye detection module detects redeye pixel areas in the input image and generates a set of detected redeye pixel areas by merging redeye pixel areas detected in the input image with redeye pixel areas detected in the thumbnail image.

67. The system of claim 58, further comprising a redeye correction module operable to correct redeye in the input image based on redeye pixel areas detected in the thumbnail image.

68. A system for processing an input image having lines of pixels with original color values, comprising:

a redeye detection module operable to detect one or more redeye pixel areas corresponding to respective areas in the input image; and

a redeye correction module operable to classify each pixel in the input image corresponding to the detected redeye pixel areas as a redeye pixel or a non-redeye pixel on a line-by-line basis without reference to pixels in adjacent lines, and to correct the original color values of pixels in the input image classified as redeye pixels.