Method and Apparatus for Selective Disqualification of Digital Images

• US 20070201724A1
• Filed: 07/26/2006
• Published: 08/30/2007
• Est. Priority Date: 02/24/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Bigioi, Petronel, Corcoran, Peter
• Granted Patent: US 7,792,335 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 10/98 Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure

Abstract

An unsatisfactory scene is disqualified as an image acquisition control for a camera. An image is acquired. One or more eye regions are determined. The eye regions are analyzed to determine whether they are blinking, and if so, then the scene is disqualified as a candidate for a processed, permanent image while the eye is completing the blinking.

Claims

1. A method of disqualifying an acquired image as unsatisfactory for permanent storage based on facial expression, comprising:

(a) acquiring a present image of a scene including a face region;

(b) identifying one or more groups of pixels corresponding to one or more face portion regions within the face region;

(c) determining whether one or more expressions of said face portions are unsatisfactory; and

if so, then

(d) rejecting the present image for storage.

2. The method of claim 1, further comprising automatically initiating capturing a consequent image of the same scene.

3. The method of claim 2, wherein the timing of said capturing a consequent image corresponds to an estimated time for said expression to change.

4. The method of claim 1, further comprising providing a notification that said present image is unsatisfactory.

5. The method of claim 4, wherein said notification is visual, audible or a combination thereof.

6. The method of claim 1, further comprising:

(i) acquiring one or more preview images; and

(ii) determining presence and location of one or more eyes in said one or more preview images; and

(iii) wherein the identifying of eye region pixels in the present image is based at least in part on said determining of said presence and location of eyes in said preview images.

7. The method of claim 1, wherein the identifying comprises identifying one or more groups of pixels corresponding to an eye region within the face region;

and the determining comprises determining whether the eye region is in a blinking process; and

if so, then the rejecting comprises disqualifying the scene as a candidate for a processed, permanent image while the eye is completing the blinking.

8. The method of claim 7, wherein the present image comprises a preview image, and the disqualifying comprises delaying full resolution capture of an image of the scene.

9. The method of claim 8, wherein the delaying comprises predicting when the blinking will be completed and ending the disqualifying at approximately the predicted blink completion time.

10. The method of claim 7, wherein the determining whether the eye is blinking comprises determining a degree to which the eye is open or shut.

11. The method of claim 7, wherein the present image comprises a full resolution capture image and the disqualifying comprises foregoing further processing of the present image.

12. The method of claim 7, wherein the present image comprises a full resolution capture image, and the method further comprises assembling a combination image including pixels from the present image and open-eye pixels corresponding to the eye that is blinking from a different image.

13. The method of claim 7, wherein the identifying of the eye region comprises identifying the face region, and analyzing the face region to determine the eye region therein.

14. The method of claim 7, further comprising capturing a new image due to the disqualifying to replace the present image.

15. The method of claim 7, further comprising providing a warning signal regarding the blinking.

16. The method of claim 7, further comprising identifying a second eye region of a second face within the scene;

determining whether the second eye region is in a blinking process; and

if so, then disqualifying the scene as a candidate for a processed, permanent image while the second eye is completing the blinking.

17. The method of claim 7, further comprising:

(i) acquiring one or more preview images;

(ii) determining presence and location of one or more eyes in said one or more preview images; and

(iii) wherein the identifying of eye region pixels in the present image is based at least in part on said determining of said presence and location of eyes in said preview images.

18. A method of preventing the occurrence of eyes that are in the process of blinking in an acquired digital image comprising:

(i) acquiring one or more preview images of a scene;

(ii) determining presence and amount of faces in said scene; and

(iii) determining a number of multiple images to acquire based on said amount of faces determined in the scene; and

(iv) wherein the number is determined based on a probability that one or more of the multiple images will not have one or more blinking eyes present.

19. A method for automatically disqualifying an unsatisfactory scene as an image acquisition control of a camera, comprising:

(a) acquiring multiple preview images;

(b) extracting information from said multiple preview images;

(c) analyzing one or more changes in the scene between individual images of said multiple temporary images;

(d) based on the analyzing, determining whether one or more unsatisfactory features exist within the scene;

(e) disqualifying the scene as a candidate for a processed, permanent image while said one or more unsatisfactory features continue to exist.

20. The method of claim 19, wherein the analyzing comprises identifying one or more groups of pixels that correspond to a facial feature having an unsatisfactory configuration.

21. The method of claim 20, wherein the one or more groups of pixels comprises an eye group, and the unsatisfactory configuration comprises a blinking configuration.

22. The method of claim 19, further comprising determining a disqualifying interval during which no processed, permanent image is to be acquired.

23. A method for automatically disqualifying an unsatisfactory scene as an image acquisition control of a camera, comprising:

(a) acquiring a preview image;

(b) extracting information from said preview image;

(c) determining whether one or more key features exist or should exist within the scene;

(d) analyzing one or more regions of pixels within the preview image corresponding to the one or more key features within the scene;

(e) based on the analyzing of the one or more regions of pixels within the preview image, determining whether one or more unsatisfactory expressions, shadowing, or occluding, or combinations thereof, of the one or more key features exist within the scene;

(f) disqualifying the scene as a candidate for a processed, permanent image while a number of one or more unsatisfactory expressions, shadowing, occluding or combinations thereof, of said one or more key features continue to exist.

24. The method of claim 23, wherein the analyzing comprises identifying one or more groups of pixels that correspond to a facial feature having an unsatisfactory configuration.

25. The method of claim 24, wherein the one or more groups of pixels comprises an eye group, and the unsatisfactory configuration comprises a blinking configuration.

26. The method of claim 24, further comprising determining a disqualifying interval during which no processed, permanent image is to be acquired.

27. An image acquisition apparatus including an imaging optical system and digital image detector and storage medium, and having stored therein program instructions for programming one or more processors to perform a method of disqualifying an acquired image as unsatisfactory for permanent storage based on facial expression, the method comprising:

(a) acquiring a present image of a scene including a face region;

(b) identifying one or more groups of pixels corresponding to one or more face portion regions within the face region;

(c) determining whether one or more expressions of said face portions are unsatisfactory; and

if so, then

(d) rejecting the present image for storage.

28. The apparatus of claim 27, wherein the method further comprises automatically initiating capturing a consequent image of the same scene.

29. The apparatus of claim 28, wherein a timing of said capturing a consequent image corresponds to an estimated time for said expression to change.

30. The apparatus of claim 27, wherein the method further comprises providing a notification that said present image is unsatisfactory.

31. The apparatus of claim 30, wherein said notification is visual, audible or a combination thereof.

32. The apparatus of claim 27, wherein the method further comprises:

(i) acquiring one or more preview images;

(ii) determining presence and location of one or more eyes in said one or more preview images; and

(iii) wherein the identifying of eye region pixels in the present image is based at least in part on said determining of said presence and location of eyes in said preview images.

33. The apparatus of claim 27, wherein the identifying comprises identifying one or more groups of pixels corresponding to an eye region within the face region;

and the determining comprises determining whether the eye region is in a blinking process; and

if so, then the rejecting comprises disqualifying the scene as a candidate for a processed, permanent image while the eye is completing the blinking.

34. The apparatus of claim 33, wherein the present image comprises a preview image, and the disqualifying comprises delaying full resolution capture of an image of the scene.

35. An image acquisition apparatus including an imaging optical system and digital image detector and storage medium, and having stored therein program instructions for programming one or more processors to perform a method for automatically disqualifying an unsatisfactory scene as an image acquisition control of a camera, the method comprising:

(a) acquiring a preview image;

(b) extracting information from said preview image;

(c) determining whether one or more key features exist or should exist within the scene;

(d) analyzing one or more regions of pixels within the preview image corresponding to the one or more key features within the scene;

(e) based on the analyzing of the one or more regions of pixels within the preview image, determining whether one or more unsatisfactory configurations, shadowing, occluding, or combinations thereof, of the one or more key features exist within the scene;

(f) disqualifying the scene as a candidate for a processed, permanent image while a number of one or more unsatisfactory configurations, shadowing, occluding or combinations thereof, of said one or more key features continue to exist.

36. The apparatus of claim 35, wherein the analyzing comprises identifying one or more groups of pixels that correspond to a facial feature having an unsatisfactory configuration.

37. The apparatus of claim 36, wherein the one or more groups of pixels comprises an eye group, and the unsatisfactory configuration comprises a blinking configuration.

38. The apparatus of claim 36, further comprising determining a disqualifying interval during which no processed, permanent image is to be acquired.

39. The apparatus of claim 35, wherein the apparatus comprises a digital camera;

a mobile phone equipped with a image capture module, a handheld computer, or a digital video camera, or a combination thereof.

40. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of disqualifying an acquired image as unsatisfactory for permanent storage based on facial expression, the method comprising:

(a) acquiring a present image of a scene including a face region;

(b) identifying one or more groups of pixels corresponding to one or more face portion regions within the face region;

(c) determining whether one or more expressions of said face portions are unsatisfactory; and

if so, then

(d) rejecting the present image for storage.

43. The one or more storage devices as recited in claim 40, the method further comprising providing a notification that said present image is unsatisfactory

45. The one or more storage devices of claim 40, the method further comprising:

(i) acquiring one or more preview images;

(ii) determining presence and location of one or more eyes in said one or more preview images; and

(iii) wherein the identifying of eye region pixels in the present image is based at least in part on said determining of said presence and location of eyes in said preview images.

46. The one or more storage devices of claim 40, wherein the identifying comprises identifying one or more groups of pixels corresponding to an eye region within the face region;

and the determining comprises determining whether the eye region is in a blinking process; and

if so, then the rejecting comprises disqualifying the scene as a candidate for a processed, permanent image while the eye is completing the blinking.

47. The one or more storage devices of claim 46, wherein the present image comprises a preview image, and the disqualifying comprises delaying full resolution capture of an image of the scene.

48. The one or more storage devices of claim 47, wherein the delaying comprises predicting when the blinking will be completed and ending the disqualifying at approximately the predicted blink completion time.

49. The one or more storage devices of claim 48, wherein the predicting comprising determining a point of a complete blinking process the scene is at, and calculating a remainder time for completion of the blinking.

50. The one or more storage devices of claim 49, wherein the calculating comprises multiplying a fraction of the complete blinking process remaining times a predetermined complete blinking process duration.

51. The one or more storage devices of claim 50, wherein the predetermined complete blinking process duration is programmed based on an average blinking process duration.

52. The one or more storage devices of claim 50, wherein the predetermined complete blinking process duration is determined based on estimating a time from a beginning of the blinking to the present and in view of the fraction representing the point of the complete blinking process the scene is at.

53. The one or more storage devices of claim 52, wherein the estimating is based on analyzing a temporal capture parameter of one or more previous preview images relative to that of the present preview image.

54. The one or more storage devices of claim 50, further comprising determining said fraction including determining whether the eye that is blinking is opening or closing in the present preview image, and a degree to which the eye is open or shut.

55. The one or more storage devices of claim 47, wherein the delaying comprises ending the disqualifying after a predetermined wait time.

56. The one or more storage devices of claim 46, wherein the determining whether the eye is blinking comprises determining a degree to which the eye is open or shut.

57. The one or more storage devices of claim 56, wherein the determining the degree to which the eye is open or shut comprises relatively analyzing the present preview image and one or more other preview images relatively acquired within less than a duration of a complete blinking process.

58. The one or more storage devices of claim 56, wherein the determining whether the eye is blinking further comprises determining a degree of blurriness of one or both eye lids.

59. The one or more storage devices of claim 56, the method further comprising determining a portion of a pupil, an iris, one or both eye lids or an eye white that is/are showing, or combinations thereof.

60. The one or more storage devices of claim 59, the method further comprising performing a color analysis of the eye and differentiating pixels corresponding to an eye lid tone from pixels corresponding to an iris tone or pupil tone or eye white tone, or combinations thereof.

61. The one or more storage devices of claim 59, the method further comprising performing a shape analysis of the eye d differentiating pixels corresponding to an eye lid shape from pixels corresponding to an iris shape or pupil shape or eye white shape, or combinations thereof.

62. The one or more storage devices of claim 46, wherein the present image comprises a full resolution capture image and the disqualifying comprises foregoing further processing of the present image.

63. The one or more storage devices of claim 62, wherein the determining whether the eye is blinking comprises determining a degree to which the eye is open or shut.

64. The one or more storage devices of claim 63, wherein the determining the degree to which the eye is open or shut comprises relatively analyzing the present preview image and one or more other preview images relatively acquired within less than a duration of a complete blinking process.

65. The one or more storage devices of claim 63, wherein the determining whether the eye is blinking further comprises determining a degree of blurriness of one or both eye lids.

66. The one or more storage devices of claim 63, the method further comprising determining a portion of a pupil, an iris, one or both eye lids or an eye white that is/are showing, or combinations thereof.

67. The one or more storage devices of claim 66, the method further comprising performing a color analysis of the eye and differentiating pixels corresponding to an eye lid tone from pixels corresponding to an iris tone or pupil tone or eye white tone, or combinations thereof.

68. The one or more storage devices of claim 66, the method further comprising performing a shape analysis of the eye and differentiating pixels corresponding to an eye lid shape from pixels corresponding to an iris shape or pupil shape or eye white shape, or combinations thereof.

69. The one or more storage devices of claim 46, wherein the present image comprises a full resolution capture image, and the method further comprises assembling a combination image including pixels from the present image and open-eye pixels corresponding to the eye that is blinking from a different image.

70. The one or more storage devices of claim 69, wherein the different image comprises a preview image or a post-view image.

71. The one or more storage devices of claim 70, wherein the different image comprises a lower resolution than the present image, and the assembling comprises upsampling the different image or downsampling the present image, or a combination thereof.

72. The one or more storage devices of claim 71, the method further comprising aligning the present image and the different image, including matching an open-eye pixel region to a blinking eye region in the present image.

73. The one or more storage devices of claim 69, the method further comprising aligning the present image and the different image, including matching an open-eye pixel region to a blinking eye region in the present image.

74. The one or more storage devices of claim 46, wherein the identifying of the eye region comprises identifying the face region, and analyzing the face region to determine the eye region therein.

75. The one or more storage devices of claim 46, the method further comprising capturing a new image due to the disqualifying to replace the present image.

76. The one or more storage devices of claim 46, the method further comprising providing a warning signal regarding the blinking.

77. The one or more storage devices of claim 46, the method further comprising identifying a second eye region of a second face within the scene;

determining whether the second eye region is in a blinking process; and

if so, then disqualifying the scene as a candidate for a processed, permanent image while the second eye is completing the blinking.

78. The one or more storage devices of claim 77, the method further comprising delaying capturing a full resolution image until the eye regions of each face region within the scene comprises no blinking eye.

79. The one or more storage devices of claim 46, the method further comprising:

(i) acquiring one or more preview images;

(ii) determining presence and location of one or more eyes in said one or more preview images; and

(iii) wherein the identifying of eye region pixels in the present image is based at least in part on said determining of said presence and location of eyes in said preview images.

41. The one or more storage devices as recited in 40, the method further comprising automatically initiating capturing a consequent image of the same scene.

42. The one or more storage devices as recited in claim 41, wherein the timing of said capturing a consequent image corresponds to an estimated time for said expression to change.

44. The one or more storage devices of 43, wherein said notification is visual, audible or a combination thereof.

80. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method for automatically disqualifying an unsatisfactory scene as an image acquisition control of a camera, the method comprising:

(a) acquiring a preview image;

(b) extracting information from said preview image;

(c) determining whether one or more key features exist or should exist within the scene;

(d) analyzing one or more regions of pixels within the preview image corresponding to the one or more key features within the scene;

(e) based on the analyzing of the one or more regions of pixels within the preview image, determining whether one or more unsatisfactory configurations, shadowing, occluding, or combinations thereof, of the one or more key features exist within the scene;

(f) disqualifying the scene as a candidate for a processed, permanent image while a number of one or more unsatisfactory configurations, shadowing, occluding or combinations thereof, of said one or more key features continue to exist.

81. The one or more storage devices of claim 80, wherein the analyzing comprises identifying one or more groups of pixels that correspond to a facial feature having an unsatisfactory configuration.

82. The one or more storage devices of claim 81, wherein the one or more groups of pixels comprises an eye group, and the unsatisfactory configuration comprises a blinking configuration.

83. The one or more storage devices of claim 81, the method further comprising determining a disqualifying interval during which no processed, permanent image is to be acquired.