Face detecting camera and method

• US 6,940,545 B1
• Filed: 02/28/2000
• Issued: 09/06/2005
• Est. Priority Date: 02/28/2000
• Status: Expired due to Term

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Ray, Lawrence A., Nicponski, Henry
• Primary Examiner(s): Ho, Tuan
• Assistant Examiner(s): Ye, Lin
• Application Number:
  US09/514,436
• Time in Patent Office: 2,017 Days
• Field of Search: 348/207.99, 348/169, 348/77, 348/268, 348/208.13, 348/208.14, 382/118, 382/116, 382/282, 382/289, 382/237, 382/170
• US Class Current: 348/222.1
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure

Abstract

A method for determining the presence of a face from image data includes a face detection algorithm having two separate algorithmic steps: a first step of prescreening image data with a first component of the algorithm to find one or more face candidate regions of the image based on a comparison between facial shape models and facial probabilities assigned to image pixels within the region; and a second step of operating on the face candidate regions with a second component of the algorithm using a pattern matching technique to examine each face candidate region of the image and thereby confirm a facial presence in the region, whereby the combination of these components provides higher performance in terms of detection levels than either component individually. In a camera implementation, a digital camera includes an algorithm memory for storing an algorithm comprised of the aforementioned first and second components and an electronic processing section for processing the image data together with the algorithm for determining the presence of one or more faces in the scene. Facial data indicating the presence of faces may be used to control, e.g., exposure parameters of the capture of an image, or to produce processed image data that relates, e.g., color balance, to the presence of faces in the image, or the facial data may be stored together with the image data on a storage medium.

Claims

1. A digital camera for capturing an image of a scene, said digital camera comprising:

a capture section for capturing an image and producing image data;

an electronic processing section for processing the image data to determine the presence of one or more faces in the image;

face data means associated with the processing section for generating face data corresponding to at least one attribute of at least one of the faces in the image;

a storage medium for storing the image data; and

recording means associated with the processing section for recording the face data with the image data on the storage medium;

wherein the electronic processing section utilizes a face detection algorithm having a first component and a second component;

the first component comprising a pre-screening pattern recognizer that searches for image windows likely to contain faces, the first component having a first rate of false positives and determining a plurality of face candidates;

the second component comprising a posterior probability function classifier, the second component having a second rate of false positives lower than the first rate of false positives, and processing the plurality of face candidates to determine the presence of the one or more faces in the image.

2. The digital camera as claimed in claim 1 wherein the face data corresponds to at least one of the location, orientation, scale and pose of at least one of the faces in the image.

3. The digital camera as claimed in claim 1 wherein the electronic processing section further provides an indication that one or more faces have been detected.

4. The digital camera as claimed in claim 3 further comprising a framing device for framing the image, and wherein the electronic processing section provides an indication in the framing device identifying the one or more faces that have been detected.

5. The digital camera as claimed in claim 4 wherein the framing device is either an optical viewfinder that images the scene or an electronic display device that reproduces the image data.

6. The digital camera as claimed in claim 1 wherein the recording means records the captured image data in the storage medium in digital folders dedicated to images with a particular number of faces in the scenes.

7. The digital camera as claimed in claim 1 wherein the electronic processing section further includes a face recognition algorithm and a data base of known faces for generating facial identities, and wherein the recording means labels one or more images in the storage medium with the facial identities of known faces.

8. The digital camera as claimed in claim 1 wherein the capture section further includes an exposure control section responsive to the presence of one or more faces for optimally exposing the image for at least one of the faces in the scene.

9. The digital camera as claimed in claim 8 wherein the exposure control section optimally exposes the image for either the preponderance of faces in the scene or the largest face in the scene.

10. The digital camera as claimed in claim 8 wherein the capture section further includes a flash unit, and wherein the electronic processing section controls activation of the flash unit in order to optimize exposure for at least one of the faces in the scene.

14. The digital camera as claimed in claim 1 wherein the posterior probability function classifier comprises a maximum a posteriori classifier.

11. A digital camera for capturing an image of a scene, said digital camera comprising:

a capture section for capturing an image and producing image data;

an electronic processing section for processing the image data to determine the presence of one or more faces in the scene;

face data means associated with the processing section for generating face data corresponding to at least one attribute of at least one of the faces in the image;

a composition algorithm associated with the processing section for processing the face data and generating composition suggestions for a user of the digital camera in response to the processed face data; and

a display device for displaying the composition suggestions to the user.

12. A digital camera as claimed in claim 11 wherein the composition suggestions include at least one of (a) an indication that a main subject is too small in the image, (b) an indication that following the law of thirds will lead to a more pleasing composition, (c) an indication that one or faces have been cut off in the image, and (c) an indication that a horizontal alignment of subjects should be avoided in the image.

13. A digital camera for capturing an image of a scene, said digital camera comprising:

a capture section for capturing an image and producing image data;

an electronic processing section for processing the image data to determine the presence of one or more faces in the scene and generating face data therefrom;

an orientation algorithm associated with the processing section for generating orientation data indicating orientation of the image based on the orientation of at least one of the faces in the image;

a storage medium for storing the image data; and

recording means associated with the processing section for recording the orientation data with the image data on the storage medium.

15. A digital camera for capturing an image of a scene and producing image data, said digital camera comprising:

an algorithm memory storing an algorithm for determining the presence of one or more faces in the image, said algorithm comprised of a first component and a second component; and

an electronic processing section for processing the image data together with the algorithm for determining the presence of one or more faces in the image;

the first component of the algorithm comprising a pre-screening pattern recognizer that searches for image windows likely to contain faces, the first component having a first rate of false positives and determining a plurality of face candidates;

the second component of the algorithm comprising a posterior probability function classifier, the second component having a second rate of false positives lower than the first rate of false positives, and processing the plurality of face candidates to determine the presence of the one or more faces in the image.

16. A digital camera for capturing an image of a scene and producing image data, said digital camera comprising:

an algorithm memory storing an algorithm for determining the presence of one or more faces in the image, said algorithm comprised of a first component and a second component;

an electronic processing section for processing the image data together with the algorithm for determining the presence of one or more faces in the image, said processing section generating processed image data and face data corresponding to at least one of the location, orientation, scale and pose of at least one of the faces in the image;

a storage medium; and

an interface for transferring the processed image data and the face data to the storage medium whereby the face data is stored with the processed image data on the storage medium;

the first component of the algorithm comprising a pre-screening pattern recognizer that searches for image windows likely to contain faces, the first component having a first rate of false positives and determining a plurality of face candidates;

the second component of the algorithm comprising a posterior probability function classifier, the second component having a second rate of false positives lower than the first rate of false positives, and processing the plurality of face candidates to determine the presence of the one or more faces in the image.

17. A digital camera for capturing an image of a scene, said digital camera comprising:

an image capture section for capturing an image with an image sensor and producing image data;

an algorithm memory storing an algorithm for determining the presence of one or more faces in the image, said algorithm comprised of a first component and a second component; and

an electronic processing section for processing the image data together with the algorithm for determining the presence of one or more faces in the image and controlling the capture of images by the capture section as a function of the presence of faces in the image;

the first component of the algorithm comprising a pre-screening pattern recognizer that searches for image windows likely to contain faces, the first component having a first rate of false positives and determining a plurality of face candidates;

the second component of the algorithm comprising a posterior probability function classifier, the second component having a second rate of false positives lower than the first rate of false positives, and processing the plurality of face candidates to determine the presence of the one or more faces in the image.

18. The camera as claimed in claim 17 wherein the electronic processing section controls the exposure of the image in relation to the presence of faces in the image.

19. The camera as claimed in claim 17 wherein the electronic processing section controls the focus of the image in relation to the presence of faces in the image.

20. The camera as claimed in claim 17 wherein the electronic processing section controls the illumination of the image in relation to the presence of faces in the image.

21. A digital camera for capturing an image of a scene and producing image data, said digital camera comprising:

an algorithm memory storing an algorithm for determining the presence of one or more faces in the image, said algorithm comprised of a first component and a second component; and

an electronic processing section for processing the image data together with the algorithm for determining the presence of one or more faces in the image in order to produce processed image data that relates to the presence of faces in the image;

the first component of the algorithm determining a plurality of face candidates utilizing a pattern matching technique that identifies image windows likely to contain faces based on color and shape information;

the second component of the algorithm processing the plurality of face candidates using a maximum a posteriori classifier to determine the presence of the one or more faces in the image.

22. The camera as claimed in claim 21 wherein the electronic processing section is responsive to the location of faces in the image in order to produce processed image data that is color balanced for the faces in the image.

23. The camera as claimed in claim 21 wherein the electronic processing section is responsive to the location of faces in the image in order to produce processed image data that is corrected for red eye in the faces in the image.

24. The camera as claimed in claim 21 further including a display device and wherein the electronic processing section is responsive to the location of faces in the image in order to produce processed data that is applied to the display device to identify the location of faces in image data displayed on the display device.

25. The camera as claimed in claim 21 wherein the electronic processing section is responsive to the location of faces in the image in order to produce processed data that provides composition aids to a photographer using the camera.

26. A digital camera for capturing an image of a scene and producing image data, said digital camera comprising:

an algorithm memory storing a face detection algorithm for determining the presence of one or more faces in the image data and a composition algorithm for suggesting composition adjustments based on certain predetermined composition principles; and

an electronic processing section for processing the image data together with the algorithms for determining the presence of one or more faces in the scene and their relation to certain predetermined composition principles, said processing section generating face data corresponding to at least one of the location, orientation, scale and pose of at least one of the faces in the image and composition suggestions corresponding to deviation of the face data from the predetermined composition principles.

27. The camera as claimed in claim 26 wherein the predetermined composition principle is location of the faces in relation to a rule of thirds.

28. The camera as claimed in claim 26 wherein the predetermined composition principle is the size of the faces in relation to the overall image.

29. The camera as claimed in claim 26 wherein the predetermined composition principle is location of the faces so as to prevent occlusion by edges of the frame.

30. The camera as claimed in claim 26 wherein the predetermined composition principle is variety in location of the faces with respect to a common line.

36. The camera as claimed in claim 28 wherein the composition suggestions include moving the camera closer to the main subject.

31. A hybrid camera for capturing an image of a scene on both an electronic medium and a film medium having a magnetic layer, said hybrid camera comprising:

an image capture section for capturing an image with an image sensor and producing image data;

means for capturing the image on the film medium;

an electronic processing section for processing the image data to determine the presence of one or more faces in the scene;

face data means associated with the electronic processing section for generating face data corresponding to at least one of the location, scale and pose of at least one of the faces in the image;

means for writing the face data on the magnetic layer of the film medium; and

orientation data means associated with the electronic processing section for generating orientation data indicating orientation of the image based on the orientation of at least one of the faces in the image;

wherein recording means associated with the processing section records the orientation data on the magnetic layer of the film medium.

32. The hybrid camera as claimed in claim 31 further comprising:

a storage medium for storing the image data; and

recording means associated with the processing section for recording the face data with the image data on the storage medium.

33. The hybrid camera as claimed in claim 31 wherein the electronic processing section provides an indication that one or more faces have been detected.

34. The hybrid camera as claimed in claim 33 further comprising a framing device for framing the image, and wherein the electronic processing section provides an indication in the framing device identifying the one or more faces that have been detected.

35. The hybrid camera as claimed in claim 31 wherein the electronic processing section further includes a face recognition algorithm and a data base of known faces for generating facial identities, and wherein the recording means records the facial identities of known faces on the magnetic layer of the film medium.

37. The hybrid camera as claimed in claim 31 further comprising an exposure control section responsive to the presence of one or more faces for optimally exposing the film medium for at least one of the faces in the scene.

38. The hybrid camera as claimed in claim 37 further comprising a flash unit, and wherein the exposure control section controls activation of the flash unit in order to optimize exposure for at least one of the faces in the scene.