Method and apparatus for selective rejection of digital images

• US 7,551,754 B2
• Filed: 07/26/2006
• Issued: 06/23/2009
• Est. Priority Date: 02/24/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Bigioi, Petronel, Corcoran, Peter
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US11/460,227
• Publication Number:
  US 20070201726A1
• Time in Patent Office: 1,063 Days
• Field of Search: 382/100, 382/112, 382/181, 382/190, 382/192, 382/194, 382/195, 382/254, 382/255, 382/274, 382/275, 358/3.24, 358/3.26, 358/537, 250/559, 250/571, 348/135, 707/100, 707/101, 707/102
• US Class Current: 382/112
• CPC Class Codes: G06V 10/98 Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

An unsatisfactory scene is disqualified as an image acquisition control for a camera. An image is acquired. One or more regions of facial or other key features are determined in the image. These regions are analyzed to determine whether they are unsatisfactorily blocked or shadowed, and if so, then the scene is disqualified as a candidate for a processed, permanent image while the feature continues to be blocked or shadowed.

Claims

1. A method of disqualifying a stored image as unsatisfactory for further image processing, comprising:

(a) acquiring and storing a present image of a scene including a key feature region;

(b) extracting information from the stored image during post-acquisition digital image processing;

(c) identifying one or more groups of pixels corresponding to the key feature region;

(d) determining whether one or more details regarding said key feature region are unsatisfactory; and

if so, then

(e) rejecting the present image for further post-acquisition digital image processing;

(f) wherein the identifying comprises identifying one or more groups of pixels corresponding to the key feature; and

the determining comprises determining whether the key feature is significantly blocked, occluded or shadowed; and

if so, then the rejecting comprises disqualifying the image as a candidate for further processing, as well as any other temporally proximately captured images that are similarly blocked, occluded or shadowed,

(g) wherein the present image comprises a full resolution capture image, and the method further comprises assembling a combination image including pixels from the present image and non-blocked, non-occluded or non-shadowed pixels corresponding to the key feature from a different image; and

wherein a processor is used to perform the method.

2. The method of claim 1, further comprising:

(f) acquiring one or more preview images; and

(g) determining presence and location of one or more faces in said one or more preview images; and

(h) wherein the identifying of the one or more groups of pixels in the present image is based at least in part on said determining of said presence and location of faces in said preview images.

3. The method of claim 1, wherein the determining whether the key feature is blocked or shadowed comprises determining a degree to which the key feature is blocked or shadowed.

4. The method of claim 1, wherein the present image comprises a full resolution capture image and the key feature comprises a face region or portion of a face region.

5. The method of claim 1, wherein the identifying of the key feature region comprises identifying a face region, and analyzing the face region to determine an eye or mouth region therein.

6. The method of claim 1, further comprising processing a different image due to the disqualifying to replace the present image.

7. A method of disqualifying an acquired image as unsatisfactory for permanent storage based on a blocked or occluded feature, comprising:

(a) acquiring a present image of a scene including a key feature;

(b) identifying one or more groups of pixels corresponding to the key feature;

(c) determining whether the key feature is shadowed or occluded or both in the present image; and

if so, then

(d) rejecting the present image as a candidate for post-acquisition digital image processing, and automatically acquiring a new image to replace the present image after delaying for a period of time,

(e) wherein the delaying of said acquiring the new image corresponds to an estimated time for said shadowing or occluding of said key feature to change; and

wherein a processor is used to perform the method.

8. The method of claim 7, further comprising providing a notification that said present image is unsatisfactory.

9. The method of claim 8, wherein said notification is visual, audible or a combination thereof.

10. The method of claim 7, further comprising:

(i) acquiring one or more preview images; and

(ii) determining presence and location of shadows or occlusions or both in said one or more preview images; and

(iii) wherein the determining in said present image is based at least in part on said determining in said preview images.

11. The method of claim 7, wherein the identifying comprises identifying one or more groups of pixels corresponding to a face region;

and the determining further comprises determining whether the face region has an unsatisfactory configuration; and

if so, then the rejecting comprises disqualifying the scene as a candidate for a processed, permanent image while the face region has the unsatisfactory configuration.

12. The method of claim 11, wherein the unsatisfactory configuration comprises blinking or frowning or both.

13. The method of claim 7, wherein the determining whether the key feature is shadowed or occluded comprises determining a degree to which the key feature is shadowed or occluded.

17. An image acquisition apparatus including an imaging optical system and digital image detector and storage medium, and having stored therein program instructions for programming one or more processors to perform a method of disqualifying an acquired image as recited at any of claims 2, 3-6, 7-13 and 15-16.

14. A method of disqualifying an acquired image as unsatisfactory for permanent storage based on a blocked or occluded feature, comprising:

(a) acquiring a present image of a scene including a key feature;

(b) identifying one or more groups of pixels corresponding to the key feature;

(c) determining whether the key feature is shadowed or occluded or both in the present image; and

if so, then

(d) rejecting the present image as a candidate for post-acquisition digital image processing, and automatically acquiring a new image to replace the present image after delaying for a period of time,

(e) wherein the delaying comprises predicting when the occluding or shadowing will stop and ending the delaying at approximately a predicted occluding or shadowing completion time; and

wherein a processor is used to perform the method.

15. A method for automatically disqualifying an unsatisfactory scene as an image acquisition control of a camera, comprising:

(a) acquiring multiple preview images;

(b) extracting information from said multiple preview images;

(c) analyzing one or more changes in the scene between individual images of said multiple temporary images;

(d) based on the analyzing, determining whether one or more key features within the scene is being shadowed or occluded; and

(e) disqualifying the scene as a candidate for a processed, permanent image while said shadowing or occluding continue to exist,

(f) wherein the analyzing comprises estimating a period of time that the shadowing or occluding will continue to exist based on detected motion of a shadow or occluding object; and

wherein a processor is used to perform the method.

16. The method of claim 15, fun her comprising determining a disqualifying interval during which the shadowing or occluding continues and no processed, permanent image is to be acquired, and ending the disqualifying at the end of the interval.

18. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of disqualifying a stored image as unsatisfactory for further image processing, the method comprising:

(a) acquiring and storing a present image of a scene including a key feature region;

(b) extracting information from the stored image during post-acquisition digital image processing;

(c) identifying one or more groups of pixels corresponding to the key feature region;

(d) determining whether one or more details regarding said key feature region are Unsatisfactory; and

if so, then

(e) rejecting the present image for further post-acquisition digital image processing,

(f) wherein the identifying comprises identifying one or more groups of pixels corresponding to the key feature; and

the determining comprises determining whether the key feature is significantly blocked, occluded or shadowed; and

if so, then the rejecting comprises disqualifying the image as a candidate for further processing, as well as any other temporally proximately captured images that are similarly blocked, occluded or shadowed,

(g) wherein the present image comprises a full resolution capture image, and the method further comprises assembling a combination image including pixels from the present image and non-blocked, non-occluded or non-shadowed pixels corresponding to the key feature from a different image.

19. The one or more storage devices of claim 18, the method further comprising:

(f) acquiring one or more preview images; and

(g) determining presence and location of one or more faces in said one or more preview images; and

(h) wherein the identifying of the one or more groups of pixels in the present image is based at least in part on said determining of said presence and location of faces in said preview images.

20. The one or more storage devices of claim 18, wherein the determining whether the key feature is blocked or shadowed comprises determining a degree to which the key feature is blocked or shadowed.

21. The one or more storage devices of claim 18, wherein the present image comprises a full resolution capture image and the key feature comprises a face region or portion of a face region.

22. The one or more storage devices of claim 18, wherein the identifying of the key feature region comprises identifying a face region, and analyzing the face region to determine an eye or mouth region therein.

23. The one or more storage devices of claim 18, the method further comprising processing a different image due to the disqualifying to replace the present image.

24. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of disqualifying an acquired image as unsatisfactory for permanent storage based on a blocked or occluded feature, the method comprising:

(a) acquiring a present image of a scene including a key feature;

(b) identifying one or more groups of pixels corresponding to the key feature;

(c) determining whether the key feature is shadowed or occluded or both in the present image; and

if so, then

(d) rejecting the present image as a candidate for post-acquisition digital image processing, and automatically acquiring a new image to replace the present image after delaying for a period of time,

(e) wherein the delaying of said acquiring the new image corresponds to an estimated time for said shadowing or occluding of said key feature to substantially end.

25. The one or more storage devices of claim 24, the method further comprising providing a notification that said present image is unsatisfactory.

26. The one or more storage devices of claim 25, wherein said notification is visual, audible or a combination thereof.

27. The one or more storage devices of claim 24, the method further comprising:

(i) acquiring one or more preview images; and

(ii) determining presence and location of shadows or occlusions or both in said one or more preview images; and

(iii) wherein the determining in said present image is based at least in part on said determining in said preview images.

28. The one or more storage devices of claim 24, wherein the identifying further comprises identifying one or more groups of pixels corresponding to a face region;

and the determining further comprises determining whether the face region has an unsatisfactory configuration; and

if so, then the rejecting comprises disqualifying the scene as a candidate for a processed, permanent image while the face region has the unsatisfactory configuration.

29. The one or more storage devices of claim 28, wherein the unsatisfactory configuration comprises blinking or frowning or both.

30. The one or more storage devices of claim 24, wherein the determining whether the key feature is shadowed or occluded comprises determining a degree to which the key feature is shadowed or occluded.

31. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of disqualifying an acquired image as unsatisfactory for permanent storage based on a blocked or occluded feature, the method comprising:

(a) acquiring a present image of a scene including a key feature;

(b) identifying one or more groups of pixels corresponding to the key feature;

(c) determining whether the key feature is shadowed or occluded or both in the present image; and

if so, then

(d) rejecting the present image as a candidate for post-acquisition digital image processing, and automatically acquiring a new image to replace the present image after delaying for a period of time,

(e) wherein the delaying comprises predicting when the occluding or shadowing will stop and ending the delaying at approximately a predicted occluding or shadowing stop time.

32. One or more processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of automatically disqualifying an unsatisfactory scene as an image acquisition control of a camera, comprising:

(a) acquiring multiple preview images;

(b) extracting information from said multiple preview images;

(c) analyzing one or more changes in the scene between individual images of said multiple temporary images;

(d) based on the analyzing, determining whether one or more key features within the scene is being shadowed or occluded; and

(e) disqualifying the scene as a candidate for a processed, permanent image while said shadowing or occluding continue to exist, (f) wherein the analyzing comprises estimating a period of time that the shadowing or occluding will continue to exist based on detected motion of a shadow or occluding object.

33. The one or more storage devices of claim 32, the method further comprising determining a disqualifying interval during which the shadowing or occluding continues and no processed, permanent image is to be acquired, and ending the disqualifying at the end of the interval.