Apparatus, method, and program for discriminating subjects

• US 20050100195A1
• Filed: 09/08/2004
• Published: 05/12/2005
• Est. Priority Date: 09/09/2003
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Li, Yuanzhong
• Granted Patent: US 7,920,725 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

A characteristic amount calculating means calculates first characteristic amounts, which do not require normalization, and normalized second characteristic amounts. A first discriminating portion discriminates whether a candidate for a face is included in the target image, by referring to first reference data with the first characteristic amounts, calculated from the target image. The first reference data is obtained by learning the first characteristic amounts of a plurality of images, which are known either to be of faces or to not be of faces. In the case that the candidate is included, a second discriminating portion discriminates whether the candidate is a face, by referring to second reference data, obtained by learning the second characteristic amounts of a plurality of images, which known either to be of faces or to not to be of faces.

Claims

1. A subject discriminating apparatus comprising:

an image input means for receiving input of a target image, which is a target of discrimination;

a first characteristic amount calculating means for calculating first characteristic amounts, which do not require normalization and are employed to discriminate a predetermined subject, from the target image;

a first discriminating means for discriminating whether a candidate of the predetermined subject is included in the target image, by referring to first reference data, in which the first characteristic amounts and discrimination conditions corresponding to the first characteristic amounts are defined in advance, with the first characteristic amounts calculated from the target image;

a second characteristic amount calculating means for calculating second characteristic amounts, which are normalized and employed to discriminate the predetermined subject, from the candidate of the predetermined subject in the case that the first discriminating means judges that the candidate is included in the target image; and

a second discriminating means for discriminating whether the candidate is the predetermined subject, by referring to second reference data, in which the second characteristic amounts and discrimination conditions corresponding to the second characteristic amounts are defined in advance, with the normalized second characteristic amounts.

2. A subject discriminating apparatus as defined in claim 1, wherein:

the first discriminating means obtains the first reference data in advance, by learning the first characteristic amounts within a plurality of sample images, which are known to include the predetermined subject, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique.

3. A subject discriminating apparatus as defined in claim 2, wherein:

in the case that the predetermined subject is a face, the first discriminating means obtains the first reference data, by learning the first characteristic amounts within a first region that includes a left eye and left cheek, and within a second region that includes a right eye and right cheek, of the sample images which are known to include the predetermined subject, and by learning the first characteristic amounts within regions corresponding to the first and second regions, of the sample images which are known to not include the predetermined subject; and

the first characteristic amount calculating means calculates the first characteristic amounts of regions that correspond to the first and second regions within the target image.

4. A subject discriminating apparatus as defined in claim 3, wherein:

the first discriminating means obtains the first reference data, by learning the first characteristic amounts within a third region that includes both eyes, of the sample images which are known to include the predetermined subject, and by learning the first characteristic amounts within regions corresponding to the third region, of the sample images which are known to not include the predetermined subject; and

the first characteristic amount calculating means calculates the first characteristic amounts of regions that correspond to the first, second, and third regions within the target image.

5. A subject discriminating apparatus as defined in claim 1, wherein:

the second discriminating means obtains the second reference data in advance, by learning the second characteristic amounts within a plurality of sample images, which are known to include the predetermined subject, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique.

6. A subject discriminating apparatus as defined in claim 5, wherein:

in the case that the predetermined subject is a face, the second discriminating means obtains the second reference data, by learning the second characteristic amounts within a first region that includes a left eye and left cheek, and within a second region that includes a right eye and right cheek, of the sample images which are known to include the predetermined subject, and by learning the second characteristic amounts within regions corresponding to the first and second regions, of the sample images which are known to not include the predetermined subject; and

the second characteristic amount calculating means calculates the second characteristic amounts of regions that correspond to the first and second regions within the target image.

7. A subject discriminating apparatus as defined in claim 6, wherein:

the second discriminating means obtains the second reference data, by learning the second characteristic amounts within a third region that includes both eyes, of the sample images which are known to include the predetermined subject, and by learning the first characteristic amounts within regions corresponding to the third region, of the sample images which are known to not include the predetermined subject; and

the second characteristic amount calculating means calculates the second characteristic amounts of regions that correspond to the first, second, and third regions within the target image.

8. A subject discriminating apparatus as defined in claim 1, wherein:

the first characteristic amounts are the directions of gradient vectors or color data, of each pixel of an image.

9. A subject discriminating apparatus as defined in claim 1, wherein:

the second characteristic amounts are the directions and magnitudes of gradient vectors of each pixel of an image.

10. A subject discriminating apparatus as defined in claim 1, further comprising:

a control means, for judging whether the discrimination result by the first discriminating means satisfies predetermined requirements, and in the case that the judgment is affirmative, controlling the first characteristic amount calculating means, the first discriminating means, the second characteristic amount calculating means, and the second discriminating means so that the candidate discriminated by the first discriminating means is judged to be the predetermined subject.

11. A subject discriminating apparatus as defined in claim 1, further comprising:

at least one other discriminating means for discriminating whether the predetermined subject, judged to be included in the target image by the second discriminating means, is truly the predetermined subject, based on other characteristic amounts, which are calculated from the candidate.

12. A subject discriminating apparatus as defined in claim 1, further comprising:

an extracting means for extracting the predetermined subject from the target image.

13. A subject discriminating apparatus as defined in claim 1, further comprising:

an output means for attaching data that represents the position of the predetermined subject within the target image, and outputting the data with the target image.

14. A photography apparatus equipped with the subject discriminating apparatus defined in claim 1.

15. A subject discriminating method comprising the steps of:

receiving input of a target image, which is a target of discrimination;

calculating first characteristic amounts, which do not require normalization and are employed to discriminate a predetermined subject, from the target image;

discriminating whether a candidate of the predetermined subject is included in the target image, by referring to first reference data, in which the first characteristic amounts and discrimination conditions corresponding to the first characteristic amounts are defined in advance, with the first characteristic amounts calculated from the target image;

calculating second characteristic amounts, which are normalized and employed to discriminate the predetermined subject, from the candidate of the predetermined subject in the case that the first discriminating means judges that the candidate is included in the target image; and

discriminating whether the candidate is the predetermined subject, by referring to second reference data, in which the second characteristic amounts and discrimination conditions corresponding to the second characteristic amounts are defined in advance, with the normalized second characteristic amounts.

16. A program that causes a computer to execute a subject discriminating method, comprising the procedures of:

receiving input of a target image, which is a target of discrimination;

calculating first characteristic amounts, which do not require normalization and are employed to discriminate a predetermined subject, from the target image;

discriminating whether a candidate of the predetermined subject is included in the target image, by referring to first reference data, in which the first characteristic amounts and discrimination conditions corresponding to the first characteristic amounts are defined in advance, with the first characteristic amounts calculated from the target image;

calculating second characteristic amounts, which are normalized and employed to discriminate the predetermined subject, from the candidate of the predetermined subject in the case that the first discriminating means judges that the candidate is included in the target image; and

discriminating whether the candidate is the predetermined subject, by referring to second reference data, in which the second characteristic amounts and discrimination conditions corresponding to the second characteristic amounts are defined in advance, with the normalized second characteristic amounts.

37. A computer readable medium having the program defined in claim 16 recorded therein.

17. A subject discriminating apparatus comprising:

an image input means for receiving a target image, which is a target of discrimination;

a characteristic amount calculating means for calculating at least one characteristic amount, which is employed in discriminating a predetermined subject that includes at least one structural component, from the target image;

a first discriminating means for discriminating whether the predetermined subject is included in the target image, by referring to first reference data in which the at least one characteristic amount and discrimination conditions corresponding to the at least one characteristic amount are defined in advance, with the at least one characteristic amount calculated from the target image while the target image is deformed in a stepwise manner within a variance corresponding to a first predetermined allowable range, wherein the first reference data is obtained by learning the at least one characteristic amount within a plurality of sample images, in which the positions and/or the positional relationships of the at least one structural component are normalized within the first predetermined allowable range, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique; and

a second discriminating means for discriminating the positions of the at least one structural component included in the target image, by referring to second reference data in which the at least one characteristic amount and discrimination conditions corresponding to the at least one characteristic amount are defined in advance, with the at least one characteristic amount calculated from the target image while the target image is deformed in a stepwise manner within a variance corresponding to a second predetermined allowable range which is smaller than the first allowable range in the case that the first discriminating means judges that the predetermined subject is included in the target image, wherein the second reference data is obtained by learning the at least one characteristic amount within a plurality of sample images, in which the positions and/or the positional relationships of the at least one structural component are normalized within the second predetermined allowable range, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique.

18. A subject discriminating apparatus as defined in claim 17, wherein:

in the case that the predetermined subject is a face, the first discriminating means obtains the first reference data, by learning the at least one characteristic amount within a first region that includes a left eye and left cheek, and within a second region that includes a right eye and right cheek, of the sample images which are known to include the predetermined subject, and by learning the at least one characteristic amount within regions corresponding to the first and second regions, of the sample images which are known to not include the predetermined subject; and

the first characteristic amount calculating means calculates the first characteristic amounts of regions that correspond to the first and second regions within the target image.

19. A subject discriminating apparatus as defined in claim 18, wherein:

the first discriminating means obtains the first reference data, by learning the at least one characteristic amount within a third region that includes both eyes, of the sample images which are known to include the predetermined subject, and by learning the at least one characteristic amount within regions corresponding to the third region, of the sample images which are known to not include the predetermined subject; and

the first characteristic amount calculating means calculates the at least one characteristic amount of regions that correspond to the first, second, and third regions within the target image.

20. A subject discriminating apparatus as defined in claim 17, wherein:

in the case that the predetermined subject is a face, the second discriminating means obtains the second reference data, by learning the at least one characteristic amount within a first region that includes a left eye and left cheek, and within a second region that includes a right eye and right cheek, of the sample images which are known to include the predetermined subject, and by learning the at least one characteristic amount within regions corresponding to the first and second regions, of the sample images which are known to not include the predetermined subject; and

the second characteristic amount calculating means calculates the at least one characteristic amount of regions that correspond to the first and second regions within the target image.

21. A subject discriminating apparatus as defined in claim 20, wherein:

the second discriminating means obtains the second reference data, by learning the at least one characteristic amounts within a third region that includes both eyes, of the sample images which are known to include the predetermined subject, and by learning the at least one characteristic amounts within regions corresponding to the third region, of the sample images which are known to not include the predetermined subject; and

the second characteristic amount calculating means calculates the at least one characteristic amount of regions that correspond to the first, second, and third regions within the target image.

22. A subject discriminating apparatus as defined in claim 17, wherein:

the at least one characteristic amount is at least one of the direction of a gradient vector, the magnitude of the gradient vector, and color data, of each pixel of the target image.

23. A subject discriminating apparatus as defined in claim 17, further comprising:

an extracting means for extracting the predetermined subject from the target image, employing the positions of the discriminated structural components as a reference.

24. A subject discriminating apparatus as defined in claim 17, further comprising:

an output means for attaching data that represents the positions of the structural components within the target image, and outputting the data with the target image.

25. A photography apparatus equipped with the subject discriminating apparatus as defined in claim 17.

26. A subject discriminating method, comprising the steps of:

receiving a target image, which is a target of discrimination;

calculating at least one characteristic amount, which is employed in discriminating a predetermined subject that includes at least one structural component, from the target image;

discriminating whether the predetermined subject is included in the target image, by referring to first reference data in which the at least one characteristic amount and discrimination conditions corresponding to the at least one characteristic amount are defined in advance, with the at least one characteristic amount calculated from the target image while the target image is deformed in a stepwise manner within a variance corresponding to a first predetermined allowable range, wherein the first reference data is obtained by learning the at least one characteristic amount within a plurality of sample images, in which the positions and/or the positional relationships of the at least one structural component are normalized within the first predetermined allowable range, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique; and

discriminating the positions of the at least one structural component included in the target image, by referring to second reference data in which the at least one characteristic amount and discrimination conditions corresponding to the at least one characteristic amount are defined in advance, with the at least one characteristic amount calculated from the target image while the target image is deformed in a stepwise manner within a variance corresponding to a second predetermined allowable range which is smaller than the first allowable range in the case that the first discriminating means judges that the predetermined subject is included in the target image, wherein the second reference data is obtained by learning the at least one characteristic amount within a plurality of sample images, in which the positions and/or the positional relationships of the at least one structural component are normalized within the second predetermined allowable range, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique.

27. A program that causes a computer to execute a subject discriminating method, comprising the procedures of:

receiving a target image, which is a target of discrimination;

calculating at least one characteristic amount, which is employed in discriminating a predetermined subject that includes at least one structural component, from the target image;

discriminating whether the predetermined subject is included in the target image, by referring to first reference data in which the at least one characteristic amount and discrimination conditions corresponding to the at least one characteristic amount are defined in advance, with the at least one characteristic amount calculated from the target image while the target image is deformed in a stepwise manner within a variance corresponding to a first predetermined allowable range, wherein the first reference data is obtained by learning the at least one characteristic amount within a plurality of sample images, in which the positions and/or the positional relationships of the at least one structural component are normalized within the first predetermined allowable range, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique; and

discriminating the positions of the at least one structural component included in the target image, by referring to second reference data in which the at least one characteristic amount and discrimination conditions corresponding to the at least one characteristic amount are defined in advance, with the at least one characteristic amount calculated from the target image while the target image is deformed in a stepwise manner within a variance corresponding to a second predetermined allowable range which is smaller than the first allowable range in the case that the first discriminating means judges that the predetermined subject is included in the target image, wherein the second reference data is obtained by learning the at least one characteristic amount within a plurality of sample images, in which the positions and/or the positional relationships of the at least one structural component are normalized within the second predetermined allowable range, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique.

38. A computer readable medium having the program defined in claim 27 recorded therein.

28. A subject discriminating apparatus comprising:

an image input means for receiving input of a target image, which is a target of discrimination;

a characteristic amount calculating means for calculating at least one characteristic amount, which is employed in discriminating a predetermined subject that includes at least one structural component, from the target image;

a discriminating means for discriminating whether the predetermined subject is included in the target image, by referring to reference data in which the at least one characteristic amount and discrimination conditions corresponding to the at least one characteristic amount are defined in advance, with the at least one characteristic amount calculated from the target image, while the target image is deformed in a stepwise manner within a variance corresponding to a first predetermined allowable range, wherein the reference data is obtained by learning the at least one characteristic amount within a plurality of sample images, in which the positions and/or the positional relationships of the at least one structural component are normalized within the first predetermined allowable range, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique, and for discriminating the positions of the at least one structural component included in the predetermined subject in the case that the predetermined subject is included in the target image.

29. A subject discriminating apparatus as defined in claim 28, wherein:

in the case that the predetermined subject is a face and the structural components are eyes, the discriminating means obtains the reference data by learning the at least one characteristic amount within a first region that includes a left eye and left cheek, and within a second region that includes a right eye and right cheek, of the sample images which are known to include the predetermined subject, and by learning the at least one characteristic amount within regions corresponding to the first and second regions, of the sample images which are known to not include the predetermined subject; and

the characteristic amount calculating means calculates the at least one characteristic amount of regions that correspond to the first and second regions within the target image.

30. A subject discriminating apparatus as defined in claim 29, wherein:

the discriminating means obtains the reference data, by learning the at least one characteristic amount within a third region that includes both eyes, of the sample images which are known to include the predetermined subject, and by learning the at least one characteristic amount within regions corresponding to the third region, of the sample images which are known to not include the predetermined subject; and

the characteristic amount calculating means calculates the at least one characteristic amount of regions that correspond to the first, second, and third regions within the target image.

31. A subject discriminating apparatus as defined in claim 28, wherein:

the at least one characteristic amount is at least one of the direction of a gradient vector, the magnitude of the gradient vector, and color data, of each pixel of the target image.

32. A subject discriminating apparatus as defined in claim 28, further comprising:

an extracting means for extracting the predetermined subject from the target image, employing the positions of the discriminated structural components as a reference.

33. A subject discriminating apparatus as defined in claim 28, further comprising:

an output means for attaching data that represents the positions of the structural components within the target image, and outputting the data with the target image.

34. A photography apparatus equipped with the subject discriminating apparatus as defined in claim 28.

35. A subject discriminating method, comprising the steps of:

receiving input of a target image, which is a target of discrimination;

calculating at least one characteristic amount, which is employed in discriminating a predetermined subject that includes at least one structural component, from the target image;

discriminating whether the predetermined subject is included in the target image, by referring to reference data in which the at least one characteristic amount and discrimination conditions corresponding to the at least one characteristic amount are defined in advance, with the at least one characteristic amount calculated from the target image, while the target image is deformed in a stepwise manner within a variance corresponding to a first predetermined allowable range, wherein the reference data is obtained by learning the at least one characteristic amount within a plurality of sample images, in which the positions and/or the positional relationships of the at least one structural component are normalized within the first predetermined allowable range, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique; and

discriminating the positions of the at least one structural component included in the predetermined subject, in the case that the predetermined subject is included in the target image.

36. A program that causes a computer to execute a subject discriminating method, comprising the procedures of:

receiving input of a target image, which is a target of discrimination;

calculating at least one characteristic amount, which is employed in discriminating a predetermined subject that includes at least one structural component, from the target image;

discriminating whether the predetermined subject is included in the target image, by referring to reference data in which the at least one characteristic amount and discrimination conditions corresponding to the at least one characteristic amount are defined in advance, with the at least one characteristic amount calculated from the target image, while the target image is deformed in a stepwise manner within a variance corresponding to a first predetermined allowable range, wherein the reference data is obtained by learning the at least one characteristic amount within a plurality of sample images, in which the positions and/or the positional relationships of the at least one structural component are normalized within the first predetermined allowable range, and a plurality of sample image groups, which are known to not include the predetermined subject, with a machine learning technique; and

discriminating the positions of the at least one structural component included in the predetermined subject, in the case that the predetermined subject is included in the target image.

39. A computer readable medium having the program defined in claim 36 recorded therein.