Method and apparatus for red-eye detection in an acquired digital image based on image quality pre and post filtering

• US 7,436,998 B2
• Filed: 05/06/2005
• Issued: 10/14/2008
• Est. Priority Date: 10/28/2004
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Nanu, Florin, Bigioi, Petronel, Corcoran, Peter
• Primary Examiner(s): Do; Anh Hong
• Application Number:
  US11/123,971
• Publication Number:
  US 20060093213A1
• Time in Patent Office: 1,257 Days
• Field of Search: 382/167, 382/118, 382/165, 382/275, 382/115, 382/117, 382/299, 382/103, 382/291, 353/28, 715/718, 463/41, 348/E5.139, 348/E9.027
• US Class Current: 382/167
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

A method for red-eye detection in an acquired digital image comprises acquiring a first image and analyzing the first acquired image to provide characteristics indicative of image quality. The process then determines if one or more corrective processes can be beneficially applied to the first acquired image according to the characteristics. Any such corrective processes are applied and red-eye defects are detected in an established order established according to determination and analysis of characteristics of the corrective process and/or resulting image qualities. Defect correction can comprise applying one or more image correction operations, or a chain of two or more red-eye filters to the first acquired image. In this case, prior to the detecting step, it is determined if the red-eye filter or red eye filter chain should be applied before or after image corrective processes and whether the red eye filter can be adapted in accordance with the characteristics. The red-eye filter and the order in which it is executed in relation to the various image corrective operations may be adapted accordingly.

Claims

1. A method for red-eye detection and image correction in an acquired digital image comprising:

a) acquiring a first image;

b) analyzing the first acquired image to provide one or more characteristics indicative of image quality;

c) determining if one or more corrective processes can be beneficially applied to said first acquired image according to said characteristics;

d) establishing an order for applying one or more of such corrective processes and detecting red-eye defects, and

e) applying said one or more of such corrective processes to said first acquired image and detecting red-eye defects according to said established order.

2. A method according to claim 1 wherein said one or more corrective processes are applied before said detecting red-eye when said one or more characteristics of image quality are determined to be outside acceptable tolerances and it is determined that said one or more corrective processes can be reliably applied.

3. A method according to claim 1, wherein said detecting red-eye precedes applying said one or more corrective processes when said one or more characteristics are determined to be within acceptable tolerances.

4. A method according to claim 3, wherein said red-eye detecting is applied to an uncorrected main image.

5. A method according to claim 1, wherein said detecting red-eye comprises applying a chain of one or more red-eye filters to said first acquired image;

and further comprising, prior to said detecting;

f) determining if said red-eye filter chain can be adapted in accordance with said one or more of characteristics; and

g) adapting said red-eye filter chain accordingly.

6. A method according to claim 5, wherein said one or more red eye filters comprise a chain of two or more red eye filters.

7. A method according to claim 6, wherein said adapting comprises providing an altered set of parameters for one or more filters of said filter chain.

8. A method according to claim 6 wherein said adapting comprises re-ordering a sequence in which said filters are applied to said first acquired image.

9. A method according to claim 6, wherein more than one of said filters are applied simultaneously.

10. A method according to claim 6, wherein said adapting comprises determining which filters are to be applied in said chain.

11. A method according to claim 5, wherein said red-eye filter chain comprises:

(i) a pixel locator and segmentor;

(ii) a shape analyzer; and

(iii) a falsing analyzer.

12. A method according to claim 11, wherein said pixel locator and segmentor includes a pixel transformer.

13. A method according to claim 1, wherein said first acquired image is a sub-sampled copy of an original acquired image.

14. A method according to claim 1, wherein said analyzing comprises determining one or more of:

a degree of blur;

a degree of dust contamination;

color balance;

white balance;

a gamma correction which might be applied;

texture characteristics;

noise characteristics; and

characteristics of regions;

in the first acquired image.

15. A method according to claim 1 wherein said corrective processes comprise one or more of:

(i) contrast normalization and image sharpening;

(ii) image color adjustment and tone scaling;

(iii) exposure adjustment and digital fill flash;

(iv) brightness adjustment with color space matching;

(v) image auto-gamma determination with image enhancement;

(vi) image enhancement; and

(vii) face based image enhancement.

16. A method as claimed in claim 1, further comprising, responsive to determining that one or more corrective processes can be applied, and further to analyzing the one or more characteristics indicative of image quality said establishing the order of said image correction and said red-eye detection is based on a degree of corrective processing that is determined for correcting said image.

17. A method as claimed in claim 16, wherein when said degree of corrective processing that is determined for correcting said image is below a threshold amount, said order is established such that first said red-eye is applied followed by image correction.

18. A method as claimed in claim 16, wherein when said degree of corrective processing that is determined for correcting said image is above a threshold amount, said order is established such that first said one or more corrective processes are applied followed by detecting said red-eye.

19. A method as claimed in claim 1, further comprising determining that corrective processes cannot be reliably applied without user input, and thereby interacting with a user to determine the corrective processes to be performed.

20. A method according to claim 1, wherein the applying of a corrective process includes performing a color space transformation.

21. A method as claimed in claim 1, wherein one or more of (a) to (e) are performed in a digital camera.

22. A method as claimed in claim 1, further comprising determining a sequence for performing more than one corrective process that are determined can be beneficially applied to said first acquired image according to said characteristics.

23. A method as claimed in claim 1, wherein said one or more characteristics comprise a plurality of at least two characteristics.

24. A method as claimed in claim 1, wherein said establishing an order comprises running image correction and red-eye detection processes in different orders, corn paring image qualities, and selecting an order corresponding to higher image quality.

25. A method as claimed in claim 1, wherein said red-eye detection comprises detection of red-eye defects and correction thereof, and wherein said establishing an order comprises determining to apply a corrective process before or after red-eye defect detection and before or after red-eye defect correction.

26. A method as claimed in claim 1, wherein multiple orders are established for applying corrective processes relative to applying red-eye detection.

27. A method as claimed in claim 1, wherein an order is established for applying multiple corrective processes relative to applying red-eye detection.

28. A method as claimed in claim 27, wherein said order is selected based on a statistical analysis of multiple possible orders.

29. A method as claimed in claim 1, wherein said establishing an order comprises determining and comparing qualities of resulting images when image correction and red-eye detection processes are applied in different orders.

30. A method as claimed in claim 1, wherein said establishing an order is based at least in part on an analysis of characteristics of the corrective processes to be applied.

31. A method as claimed in claim 1, wherein said order is selected based on a statistical analysis of multiple possible orders.

32. A computer-readable storage medium containing a set of instructions for being executed on a digital image processing device and thereby programming the device to perform a method for red-eye detection and image correction in an acquired digital image, wherein the method comprises:

a) acquiring a first image;

b) analyzing the first acquired image to provide one or more characteristics indicative of image quality;

c) determining if one or more corrective processes can be beneficially applied to said first acquired image according to said characteristics;

d) establishing an order for applying one or more of such corrective processes and detecting red-eye defects, and

e) applying said one or more of such corrective processes to said first acquired image and detecting red-eye defects according to said established order.

33. The computer-readable storage medium of claim 32, wherein said one or more corrective processes are applied before said detecting red-eye when said one or more characteristics of image quality are determined to be outside acceptable tolerances and it is determined that said one or more corrective processes can be reliably applied.

34. The computer-readable storage medium according to claim 32, wherein said detecting red-eye precedes applying said one or more corrective processes when said one or more characteristics are determined to be within acceptable tolerances.

35. The computer-readable storage medium according to claim 34, wherein said red-eye detecting is applied to an uncorrected main image.

36. The computer-readable storage medium according to claim 32, wherein said detecting red-eye comprises applying a chain of one or more red-eye filters to said first acquired image;

and further comprising, prior to said detecting;

f) determining if said red-eye filter chain can be adapted in accordance with said one or more of characteristics; and

g) adapting said red-eye filter chain accordingly.

37. The computer-readable storage medium according to claim 36, wherein said one or more red eye filters comprise a chain of two or more red eye filters.

38. The computer-readable storage medium according to claim 37, wherein said adapting comprises providing an altered set of parameters for one or more filters of said filter chain.

39. The computer-readable storage medium according to claim 37 wherein said adapting comprises re-ordering a sequence in which said filters are applied to said first acquired image.

40. The computer-readable storage medium according to claim 37, wherein more than one of said filters are applied simultaneously.

41. The computer-readable storage medium according to claim 37, wherein said adapting comprises determining which filters are to be applied in said chain.

42. The computer-readable storage medium according to claim 36, wherein said red-eye filter chain comprises:

(i) a pixel locator and segmentor;

(ii) a shape analyzer; and

(iii) a falsing analyzer.

43. The computer-readable storage medium according to claim 42, wherein said pixel locator and segmentor includes a pixel transformer.

44. The computer-readable storage medium according to claim 32, wherein said first acquired image is a sub-sampled copy of an original acquired image.

45. The computer-readable storage medium according to claim 32, wherein said analyzing comprises determining one or more of:

a degree of blur;

a degree of dust contamination;

color balance;

white balance;

a gamma correction which might be applied;

texture characteristics;

noise characteristics; and

characteristics of regions;

in the first acquired image.

46. The computer-readable storage medium according to claim 32 wherein said corrective processes comprise one or more of:

(i) contrast normalization and image sharpening;

(ii) image color adjustment and tone scaling;

(iii) exposure adjustment and digital fill flash;

(iv) brightness adjustment with color space matching;

(v) image auto-gamma determination with image enhancement;

(vi) image enhancement; and

(vii) face based image enhancement.

47. The computer-readable storage medium as claimed in claim 32, further comprising, responsive to determining that one or more corrective processes can be applied, and further to analyzing the one or more characteristics indicative of image quality said establishing the order of said image correction and said red-eye detection is based on a degree of corrective processing that is determined for correcting said image.

48. The computer-readable storage medium as claimed in claim 47, wherein when said degree of corrective processing that is determined for correcting said image is below a threshold amount, said order is established such that first said red-eye is applied followed by image correction.

49. The computer-readable storage medium as claimed in claim 47, wherein when said degree of corrective processing that is determined for correcting said image is above a threshold amount, said order is established such that first said one or more corrective processes are applied followed by detecting said red-eye.

50. The computer-readable storage medium as claimed in claim 32, further comprising determining that corrective processes cannot be reliably applied without user input, and thereby interacting with a user to determine the corrective processes to be performed.

51. The computer-readable storage medium according to claim 32, wherein the applying of a corrective process includes performing a color space transformation.

52. The computer-readable storage medium as claimed in claim 32, wherein one or more of (a) to (e) are performed in a digital camera.

53. The computer-readable storage medium as claimed in claim 32, further comprising determining a sequence for performing more than one corrective process that are determined can be beneficially applied to said first acquired image according to said characteristics.

54. The computer-readable storage medium as claimed in claim 32, wherein said one or more characteristics comprise a plurality of at least two characteristics.

55. The computer-readable storage medium as claimed in claim 32, wherein said establishing an order comprises running image correction and red-eye detection processes in different orders, corn paring image qualities, and selecting an order corresponding to higher image quality.

56. The computer-readable storage medium as claimed in claim 32, wherein said red-eye detection comprises detection of red-eye defects and correction thereof, and wherein said establishing an order comprises determining to apply a corrective process before or after red-eye defect detection and before or after red-eye defect correction.

57. The computer-readable storage medium as claimed in claim 32, wherein multiple orders are established for applying corrective processes relative to applying red-eye detection.

58. The computer-readable storage medium as claimed in claim 32, wherein an order is established for applying multiple corrective processes relative to applying red-eye detection.

59. The computer-readable storage medium as claimed in claim 58, wherein said order is selected based on a statistical analysis of multiple possible orders.

60. The computer-readable storage medium as claimed in claim 32, wherein said establishing an order comprises determining and comparing qualities of resulting images when image correction and red-eye detection processes are applied in different orders.

61. The computer-readable storage medium as claimed in claim 32, wherein said establishing an order is based at least in part on an analysis of characteristics of the corrective processes to be applied.

62. The computer-readable storage medium as claimed in claim 32, wherein said order is selected based on a statistical analysis of multiple possible orders.

63. A digital image processing device, comprising electronics for acquiring digital images, a processor, and a processor-readable medium having digital code embedded therein for programming the processor to perform a method for red-eye detection and image correction in an acquired digital image, wherein the method comprises:

a) acquiring a first image;

b) analyzing the first acquired image to provide one or more characteristics indicative of image quality;

c) determining if one or more corrective processes can be beneficially applied to said first acquired image according to said characteristics;

d) establishing an order for applying one or more of such corrective processes and detecting red-eye defects, and

e) applying said one or more of such corrective processes to said first acquired image and detecting red-eye defects according to said established order.

64. A digital image processing device as claimed in claim 63 comprising a digital printer.

65. A digital image processing device as claimed in claim 63 comprising a digital docking station.

66. A digital image processing device as claimed in claim 63, comprising a digital camera.