Method of determining an exposure amount using optical recognition of facial features

• US 5,629,752 A
• Filed: 10/27/1995
• Issued: 05/13/1997
• Est. Priority Date: 10/28/1994
• Status: Expired due to Fees

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Kinjo, Naoto
• Primary Examiner(s): Grimley, Arthur T.
• Assistant Examiner(s): KERNER, HERBERT
• Application Number:
  US08/549,181
• Time in Patent Office: 564 Days
• Field of Search: 355/18, 355/32, 355/35, 355/38, 355/67, 355/68, 355/69, 355/70, 355/71, 355/77, 355/40, 382/118, 382/274, 382/174, 382/162, 382/190, 382/254
• US Class Current: 355/35
• CPC Class Codes: G03B 27/80 in dependence upon automatic analysis of the original G03B27/73 takes precedence
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

In a method of determining an exposure amount, image data is fetched, and an image is divided into a plurality of regions by binarization or another technique, a shape pattern representing a contour of the head of a human figure is detected, and a face candidate region is set in accordance with the detected pattern. Next, a shape pattern representing a contour of the face of the human figure, a shape pattern representing an internal structure of the face of the human figure, and a shape pattern representing a contour of the body of the human figure are detected respectively. The consistency of a face candidate region is determined consecutively by the detected patterns, and a region having the highest probability of being a region corresponding to the face of the human figure is determined. The above processing is carried out a plurality of times while a threshold for binarization and parameters for controlling the relative fineness of the division of the image are varied. The data on the finally determined facial region is outputted as a result of determination.

Claims

1. A method of determining an exposure amount, comprising the steps of:

searching a first shape pattern which is present in an original image and is peculiar to a first portion of a human figure;

setting a region estimated to correspond to a face of the human figure, in accordance with a size and a direction of the searched first shape pattern and a positional relationship between the face of the human figure and the first portion;

determining a region having a high probability of being a region corresponding to the face of the human figure in the original image, by searching whether or not a second shape pattern peculiar to at least one second portion different from the first portion of the human figure is present in a range of search set in accordance with the set size and direction of the first shape pattern in the original image and a positional relationship between the face of the human figure and the second portion; and

determining an exposure amount for printing the original image onto a printing material on the basis of at least one of a color and a density of the determined region.

2. A method of determining an exposure amount according to claim 1, wherein the first shape pattern and the second shape pattern include at least one of a shape pattern representing a contour of the head of the human figure, a shape pattern representing a contour of the face of the human figure, a shape pattern representing an internal structure of the face of the human figure, and a shape pattern representing a contour of the body of the human figure.

3. A method of determining an exposure amount according to claim 1, wherein the search of the first shape pattern and the second shape pattern is effected by dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar, and by using contours of the plurality of regions.

4. A method of determining an exposure amount according to claim 1, wherein the search of the first shape pattern and the second shape pattern is effected by detecting an edge which is present in the original image, by determining a center line of the detected edge, and by using the determined center line.

5. A method of determining an exposure amount according to claim 1, wherein, prior to the search of the first shape pattern, a region having a low probability of corresponding to the human figure in the original image is excluded from a subject of search of the first shape pattern.

6. A method of determining an exposure amount according to claim 2, wherein the search of the shape pattern representing the contour of the head of the human figure as the first shape pattern or the second shape pattern is effected by:

dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar;

detecting at least one concave or convex portion on the basis of contours of the plurality of regions;

determining whether or not the at least one concave or convex portion detected corresponds to the contour of the head of the human figure, on the basis of characteristic quantities of the at least one concave or convex portion.

7. A method of determining an exposure amount according to claim 2, wherein the search of the shape pattern representing the contour of the head of the human figure as the first shape pattern or the second shape pattern is effected by:

detecting an edge which is present in the original image;

determining a center line of the detected edge;

detecting at least one concave or convex portion on the basis of the determined center line; and

determining whether or not the at least one concave or convex portion detected corresponds to the contour of the head of the human figure, on the basis of characteristic quantities of the at least one concave or convex portion.

8. A method of determining an exposure amount according to claim 2, wherein the search of the shape pattern representing the contour of the face of the human figure as the first shape pattern or the second shape pattern is effected by:

dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar;

detecting at least two concave and/or convex portions on the basis of contours of the plurality of regions; and

determining whether or not the at least two concave and/or convex portions detected correspond to the contour of the face of the human figure according to a degree of axial symmetry, on the basis of relationships of characteristic quantities of the at least two concave and/or convex portions.

9. A method of determining an exposure amount according to claim 2, wherein the search of the shape pattern representing the contour of the face of the human figure as the first shape pattern or the second shape pattern is effected by:

detecting an edge which is present in the original image;

determining a center line of the detected edge;

detecting at least two concave and/or convex portions on the basis of the determined center line; and

determining whether or not the at least two concave and/or convex portions detected corresponds to the contour of the face of the human figure according to a degree of axial symmetry, on the basis of characteristic quantities of the at least two concave and/or convex portions.

10. A method of determining an exposure amount according to claim 2, wherein the search of a shape pattern representing a pair of eyes among shape patterns representing the contour of the internal structure of the human figure as the first shape pattern or the second shape pattern is effected by:

dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar;

extracting a plurality of elliptical regions from among regions divided by contours of the plurality of regions;

detecting a pair of elliptical regions whose similarity is greater than or equal to a predetermined value, on the basis of matching processing using characteristic quantities of the detected plurality of elliptical regions; and

determining whether or not the pair of elliptical regions corresponds to the pair of eyes of the face of the human figure, according to the degree of axial symmetry of the detected pair of elliptical regions.

11. A method of determining an exposure amount according to claim 2, wherein the search of a shape pattern representing a pair of eyes among shape patterns representing the contour of the internal structure of the human figure as the first shape pattern or the second shape pattern is effected by:

detecting an edge which is present in the original image;

determining a center line of the detected edge;

extracting a plurality of elliptical regions from among regions divided by the determined center line;

detecting a pair of elliptical regions whose similarity is greater than or equal to a predetermined value, on the basis of matching processing using characteristic quantities of the detected plurality of elliptical regions; and

determining whether or not the pair of elliptical regions corresponds to the pair of eyes of the face of the human figure, according to the degree of axial symmetry of the detected pair of elliptical regions.

12. A method of determining an exposure amount according to claim 2, wherein the search of the shape pattern representing the contour of the body of the human figure as the first shape pattern or the second shape pattern is effected by:

dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar;

detecting at least two concave and/or convex portions on the basis of contours of the plurality of regions;

detecting a plurality of sets of concave and/or convex portions corresponding to a contour continuing from the neck to each shoulder of the human figure, and a contour from each shoulder to an arm portion or to a lower portion of the body, on the basis of relationships of characteristic quantities of the at least two concave and/or convex portions; and

determining whether the at least two concave and/or convex portions detected corresponds to the contour of the body of the human figure according to a degree of axial symmetry of the sets.

13. A method of determining an exposure amount according to claim 2, wherein the search of the shape pattern representing the contour of the body of the human figure as the first shape pattern or the second shape pattern is effected by:

detecting an edge which is present in the original image;

determining a center line of the detected edge;

detecting at least two concave and/or convex portions on the basis of the determined center line;

detecting a plurality of sets of concave and/or convex portions corresponding to a contour continuing from the neck to each shoulder of the human figure, and a contour from each shoulder to an arm portion or to a lower portion of the body, on the basis of relationships of characteristic quantities of the at least two concave and/or convex portions; and

determining whether the at least two concave and/or convex portions detected corresponds to the contour of the body of the human figure according to a degree of axial symmetry of the sets.

14. A method of determining an exposure amount according to claim 5, wherein, among regions obtained by dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar, the following regions are excluded as regions having a low probability of corresponding to the human figure in the original image:

a region in which the ratio of a linear portion occupied in the contour of the region is greater than or equal to a predetermined value;

a region whose axial symmetry is less than or equal to a predetermined value;

a region in which the number of concave and/or convex portions is greater than or equal to a predetermined value;

a region whose ratio of contact with an outer edge of the image is greater than or equal to a predetermined value;

a region whose contrast in the density within the region is less than or equal to a predetermined value; and

a region whose density changes with a predetermined pattern or whose density shows a repetition of a predetermined pattern of change.

15. A method of determining an exposure amount according to claim 5, wherein, among regions obtained by detecting an edge which is present in the original image, by determining a center line of the detected edge, and by being divided by the determined center line of the edge, the following regions are excluded as regions having a low probability of corresponding to the human figure in the original image:

a region in which the ratio of a linear portion occupied in the contour of the region is greater than or equal to a predetermined value;

a region whose axial symmetry is less than or equal to a predetermined value;

a region in which the number of concave and/or convex portions is greater than or equal to a predetermined value;

a region whose ratio of contact with an outer edge of the image is greater than or equal to a predetermined value;

a region whose contrast in the density within the region is less than or equal to a predetermined value; and

a region whose density changes with a predetermined pattern or whose density shows a repetition of a predetermined pattern of change.

16. A method of determining an exposure amount, comprising the steps of:

searching a plurality of kinds of shape patterns which are present in an original image and are peculiar to particular portions of a human figure;

setting a plurality of regions estimated to correspond to a face of the human figure, in accordance with a size and a direction of each of the searched shape patterns and a positional relationship between the face of the human figure and each of the particular portions;

weighting the set regions by using as subjects of search the plurality of shape patterns peculiar to the particular portions of the human figure;

determining a region having a highest probability of being a region corresponding to the face of the human figure in the original image, on the basis of ranges of the plurality of regions set as regions to be estimated to correspond to the face of the human figure and weighting factors imparted to the respective regions; and

determining an exposure amount for printing the original image onto a printing material on the basis of at least one of a color and a density of the determined region.

17. A method of determining an exposure amount according to claim 16, wherein the shape patterns peculiar to the particular portions of the human figure include a shape pattern representing a contour of the head of the human figure, a shape pattern representing a contour of the face of the human figure, a shape pattern representing an internal structure of the face of the human figure, and a shape pattern representing a contour of the body of the human figure.

18. A method of determining an exposure amount according to claim 16, wherein the search of each of the shape patterns is effected by dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar, and by using contours of the plurality of regions.

19. A method of determining an exposure amount according to claim 16, wherein the search of each of the shape patterns is effected by detecting an edge which is present in the original image, by determining a center line of the detected edge, and by using the determined center line.

20. A method of determining an exposure amount according to claim 16, wherein a weighting factor with a positive sign is imparted to the set region estimated to correspond to the face of the human figure, a region having a low probability of corresponding to the face of the human figure in the original image is determined, and a weighting factor with a negative sign is imparted to the determined region.

21. A method of determining an exposure amount according to claim 17, wherein the search of the shape pattern representing the contour of the head of the human figure as the shape pattern is effected by:

dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar;

detecting at least one concave or convex portion on the basis of contours of the plurality of regions;

determining whether or not the at least one concave or convex portion detected corresponds to the contour of the head of the human figure, on the basis of characteristic quantities of the at least one concave or convex portion.

22. A method of determining an exposure amount according to claim 17, wherein the search of the shape pattern representing the contour of the head of the human figure is effected by:

detecting an edge which is present in the original image;

determining a center line of the detected edge;

detecting at least one concave or convex portion on the basis of the determined center line; and

determining whether or not the at least one concave or convex portion detected corresponds to the contour of the head of the human figure, on the basis of characteristic quantities of the at least one concave or convex portion.

23. A method of determining an exposure amount according to claim 17, wherein the search of the shape pattern representing the contour of the face of the human figure is effected by:

dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar;

detecting at least two concave and/or convex portions on the basis of contours of the plurality of regions; and

determining whether or not the at least two concave and/or convex portions detected correspond to the contour of the face of the human figure according to a degree of axial symmetry, on the basis of relationships of characteristic quantities of the at least two concave and/or convex portions.

24. A method of determining an exposure amount according to claim 17, wherein the search of the shape pattern representing the contour of the face of the human figure is effected by:

detecting an edge which is present in the original image;

determining a center line of the detected edge;

detecting at least two concave and/or convex portions on the basis of the determined center line; and

determining whether or not the at least two concave and/or convex portions detected corresponds to the contour of the face of the human figure according to a degree of axial symmetry, on the basis of characteristic quantities of the at least two concave and/or convex portions.

25. A method of determining an exposure amount according to claim 17, wherein the search of a shape pattern representing a pair of eyes among shape patterns representing the contour of the internal structure of the human figure is effected by:

dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar;

extracting a plurality of elliptical regions from among regions divided by contours of the plurality of regions;

detecting a pair of elliptical regions whose similarity is greater than or equal to a predetermined value, on the basis of matching processing using characteristic quantities of the detected plurality of elliptical regions; and

determining whether or not the pair of elliptical regions corresponds to the pair of eyes of the face of the human figure, according to the degree of axial symmetry of the detected pair of elliptical regions.

26. A method of determining an exposure amount according to claim 17, wherein the search of a shape pattern representing a pair of eyes among shape patterns representing the contour of the internal structure of the human figure is effected by:

detecting an edge which is present in the original image;

determining a center line of the detected edge;

extracting a plurality of elliptical regions from among regions divided by the determined center line;

detecting a pair of elliptical regions whose similarity is greater than or equal to a predetermined value, on the basis of matching processing using characteristic quantities of the detected plurality of elliptical regions; and

determining whether or not the pair of elliptical regions corresponds to the pair of eyes of the face of the human figure, according to the degree of axial symmetry of the detected pair of elliptical regions.

27. A method of determining an exposure amount according to claim 17, wherein the search of the shape pattern representing the contour of the body of the human figure is effected by:

dividing the original image into a plurality of regions each consisting of a plurality of pixels whose densities or colors are identical or similar;

detecting at least two concave and/or convex portions on the basis of contours of the plurality of regions;

detecting a plurality of sets of concave and/or convex portions corresponding to a contour continuing from the neck to each shoulder of the human figure, and a contour from each shoulder to an arm portion or to a lower portion of the body, on the basis of relationships of characteristic quantities of the at least two concave and/or convex portions; and

determining whether the at least two concave and/or convex portions detected corresponds to the contour of the body of the human figure according to a degree of axial symmetry of the sets.

28. A method of determining an exposure amount according to claim 17, wherein the search of the shape pattern representing the contour of the body of the human figure is effected by:

detecting an edge which is present in the original image;

determining a center line of the detected edge;

detecting at least two concave and/or convex portions on the basis of the determined center line;

detecting a plurality of sets of concave and/or convex portions corresponding to a contour continuing from the neck to each shoulder of the human figure, and a contour from each shoulder to an arm portion or to a lower portion of the body, on the basis of relationships of characteristic quantities of the at least two concave and/or convex portions; and

determining whether the at least two concave and/or convex portions detected corresponds to the contour of the body of the human figure according to a degree of axial symmetry of the sets.

29. A method of extracting a figure, comprising the steps of:

setting in advance a plurality of partial shape patterns on the basis of a plurality of characteristic partial shapes constituting a figure subject to extraction;

effecting for each of the plurality of partial shape patterns the searching of the partial shape pattern which is present in an image and the setting of a candidate region in which a particular portion or the whole of the figure subject to extraction is estimated to be present in the image, in accordance with a size and a direction of the searched partial shape pattern and a position of the partial shape represented by the partial shape pattern in the figure subject to extraction;

determining for each of the set candidate regions the consistency as a region in which the particular portion or the whole of the figure subject to extraction is present, on the basis of a relationship with each of the detected partial shape patterns;

determining a candidate region having a high probability of the particular portion or the whole of the figure subject to extraction being present, on the basis of the consistency determined for each of the candidate regions; and

extracting the candidate region determined to have the high probability of the particular portion or the whole of the figure subject to extraction being present.

30. A method of extracting a figure according to claim 29, wherein states of the figure subject to extraction in the image are classified in advance into a plurality of states;

a plurality of kinds of parameters are set in advance for each of the plurality of partial shape patterns in correspondence with respective ones of the plurality of classified states as parameters defining search conditions at the time of searching the partial shape patterns;

by using each of the plurality of kinds of parameters corresponding to the plurality of states, processing is effected for performing the search of a partial shape pattern and the setting of a candidate region for each of the plurality of partial shape patterns, for determining consistency for each of the set candidate regions, and for determining and extracting a candidate region having a high probability of the particular portion or the whole of the figure subject to extraction being present;

or

if it is determined that there is no candidate region having the high probability of the particular portion or the whole of the figure subject to extraction being present as a result of effecting the processing by using a parameter corresponding to a predetermined one of the plurality of states, the processing is repeated by using another one of the parameters corresponding to a state different from the predetermined state, and if is determined that there is a candidate region having the high probability of the particular portion or the whole of the figure subject to extraction being present, the processing ends.

31. A method of extracting a figure according to claim 30, wherein the states of the figure subject to extraction in the image include at least one of the size of the figure subject to extraction in the image, the direction of the figure subject to extraction in the image, and the direction of an object represented by the figure subject to extraction.