Method and device for generating a person's portrait, method and device for communications, and computer product

• US 20020070945A1
• Filed: 05/02/2001
• Published: 06/13/2002
• Est. Priority Date: 12/08/2000
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Renesas Technology Corporation (Renesas Electronics Corporation)
• Inventors: Kage, Hiroshi
• Granted Patent: US 6,885,761 B2
• Time in Patent Office: Days
• US Class Current: 345/581
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 2007/145 Handheld terminals

Abstract

The person'"'"'s portrait generation device comprises an image input section which picks up a two-dimensional image containing a person'"'"'s face by using an image sensor. A head area extracting section extracts the head of the person from the image. A feature detection section detects position of characteristic feature(s) of the face of that person. A face outline determining section determines a border between a face outline and a background. An image processing section generates a person'"'"'s portrait in which the characteristic feature(s) of the face is emphasized.

Claims

1. A person'"'"'s portrait generation device comprising:

an image input section which picks up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting section which extracts a head area from a differential image of a plurality of the two-dimensional images picked up by the image input section;

a feature detection section which detects position of characteristic features of the face within the extracted head area;

a face outline determining section which determines a border between a face outline and a background within the head area;

an image processing section which generates a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extracting section, the feature detection section and the face outline determining section.

2. The person'"'"'s portrait generation device according to claim 1,

wherein in the head area extracting section, the outline of the head area is obtained by combining a right-side profile and a lift-side profile of the differential image.

3. The person'"'"'s portrait generation device according to claim 2,

wherein the outline of the head area is obtained by eliminating noise components of the right-side profile and left-side profile through filtering.

4. The person'"'"'s portrait generation device according to claim 2,

wherein the head area extracting section provides the head area obtained by a head rectangular shape in which a left side, a right side and an upper side are determined by the outline of the head area and a lower side is determined by a shape ratio constant of a general person'"'"'s head.

5. A person'"'"'s portrait generation device comprising:

an image input section which picks up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting section which extracts a head area from the image picked up by the image input section;

a feature detection section which detects position of characteristic features of the face by dividing the head area thus extracted into face part areas for respective face parts and finding projection data of binarized images for the respective face parts areas;

a face outline determining section which determines a border between a face outline and a background within the head area; and

an image processing section which generates a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extracting section, the feature detection section and the face outline determining section.

6. The person'"'"'s portrait generation device according to claim 5,

wherein the face parts area is determined by relative positional information of the face parts with respect to the head area preliminarily found.

7. The person'"'"'s portrait generation device according to claim 5,

wherein the position of characteristic features of the face is detected based on a position of a center-of-gravity of the projection data.

8. The person'"'"'s portrait generation device according to claim 5,

wherein the position of characteristic features of the face is detected by based on a position of a maximum value of the projection data.

9. A person'"'"'s portrait generation device comprising:

an image input section which picks up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting section which extracts a head area from the image picked up by the image input section;

a feature detection section which detects position of characteristic features within the extracted head area;

a face outline determining section which determines a border between a face outline and a background within the head area by specifying a skin color area of the two-dimensional image;

an image processing section which generates a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extracting section, the feature detection section and the face outline determining section.

10. The person'"'"'s portrait generation device according to claim 9,

wherein the face outline determining section determines the outline of the jaw.

11. The person'"'"'s portrait generation device according to claim 9,

wherein in the face outline determining section, the skin color area is determined by converting an inputted RGB value to an HSV value.

12. The person'"'"'s portrait generation device according to claim 9,

wherein, with respect to an average color in the vicinity of the characteristic features of the face, areas having similar colors are used as face area candidates.

14. The person'"'"'s portrait generation device according to claim 13,

wherein the image processing section expresses a person'"'"'s emotions by changing the shape of a partial image at each of an eye portion, a noise portion and a mouth portion that are the characteristic features of the face.

15. The person'"'"'s portrait generation device according to claim 13,

wherein the image processing section replaces partial images at an eye portion, a noise portion and a mouth portion that are characteristic features of the face.

17. The communication terminal according to claim 16,

wherein in the head area extracting section, the outline of the head area is obtained by combining a right-side profile and a lift-side profile of the differential image.

18. The communication terminal according to claim 17,

wherein the outline of the head area is obtained by eliminating noise components of the right-side profile and left-side profile through filtering.

19. The communication terminal according to claim 17,

wherein the head area extracting section provides the head area obtained by a head rectangular shape in which a left side, a right side and an upper side are determined by the outline of the head area and a lower side is determined by a shape ratio constant of a general person'"'"'s head.

13. A person'"'"'s portrait generation device comprising:

an image input section which picks up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting section which extracts a head area from the image picked up by the image input section;

a feature detection section which detects position of characteristic features of the face for each of the face parts within the head area that has been extracted;

a face outline determining section which determines a border between a face outline and a background within the head area; and

an image processing section which generates a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized by changing the two-dimensional image with respect to each of the face parts.

16. A communication terminal comprising:

an image input section which picks up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting section which extracts a head area from a differential image of a plurality of the two-dimensional images picked up by the image input section;

a feature detection section which detects position of characteristic features within the extracted head area;

a face outline determining section which determines a border between a face outline and a background within the head area; and

an image processing section which generates a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extracting section, the feature detection section and the face outline determining section,

wherein the resulting person'"'"'s portrait is transmitted and received through a communication unit.

20. A communication terminal comprising:

an image input section which picks up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting section which extracts a head area from the image picked up by the image input section;

a feature detection section which detects position of characteristic features of the face by dividing the head area thus extracted into face part areas for respective face parts and finding projection data of binarized images for the respective face parts areas;

a face outline determining section which determines a border between a face outline and a background within the head area; and

an image processing section which generates a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extracting section, the feature detection section and the face outline determining section,

where in the resulting person'"'"'s portrait is transmitted and received through a communication unit.

21. The communication terminal according to claim 20, wherein the face parts area is determined by relative positional information of the face parts with respect to the head area preliminarily found.

22. The communication terminal according to claim 20, wherein the position of characteristic features of the face is detected based on a position of a center-of-gravity of the projection data.

23. The communication terminal according to claim 20, wherein the position of characteristic features of the face is detected by based on a position of a maximum value of the projection data.

25. The communication terminal according to claim 24,

wherein the face outline determining section determines the outline of the jaw.

26. The communication terminal according to claim 24,

wherein in the face outline determining section, the skin color area is determined by converting an inputted RGB value to an HSV value.

27. The communication terminal according to claim 24,

wherein determining the skin color area in the face outline determining section, with respect to an average color in the vicinity of the characteristic features of the face, areas having similar colors are used as face area candidates.

29. The communication terminal according to claim 28,

wherein the image processing section expresses a person'"'"'s emotions by changing the shape of a partial image at each of an eye portion, a noise portion and a mouth portion that are the characteristic features of the face.

30. The communication terminal according to claim 28,

wherein the image processing section replaces partial images at an eye portion, a noise portion and a mouth portion that are characteristic features of the face.

24. A communication terminal comprising:

an image input section which picks up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting section which extracts a head area from the image picked up by the image input section;

a feature detection section which detects position of characteristic features within the extracted head area;

a face outline determining section which determines a border between a face outline and a background within the head area by specifying a skin color area of the two-dimensional image; and

an image processing section which generates a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extracting section, the feature detection section and the face outline determining section,

wherein the resulting person'"'"'s portrait is transmitted and received through a communication unit.

28. A communication terminal comprising:

an image input section which picks up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting section which extracts a head area from the image picked up by the image input section;

a feature detection section which detects position of characteristic features of the face for each of the face parts within the head area that has been extracted;

a face outline determining section which determines a border between a face outline and a background within the head area; and

an image processing section which generates a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized by changing the two-dimensional image with respect to each of the face parts,

wherein the resulting person'"'"'s portrait is transmitted and received through a communication unit.

31. A person'"'"'s portrait generation method comprising:

an image input step of picking up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting step of extracting a head area from a differential image of a plurality of the two-dimensional images picked up in the image input step;

a feature detection step of detecting position of characteristic features within the extracted head area;

a face outline determining step of determining a border between a face outline and a background within the head area; and

an image processing step of creating a person'"'"'s portrait in which the characteristics in the person'"'"'s face are emphasized based upon the two-dimensional image by using output data obtained in the head area extracting step, the feature detection step and the face outline determining step.

32. The person'"'"'s portrait generation method according to claim 31,

wherein in the head area extracting step, the outline of the head are a is obtained by combining a right-side profile and a lift-side profile of the differential image.

33. The person'"'"'s portrait generation method according to claim 32,

wherein the outline of the head area is obtained by eliminating noise components of the right-side profile and left-side profile through a filtering step.

34. The person'"'"'s portrait generation method according to claim 32,

wherein the head area extracting step provides the head area that is obtained by a head rectangular shape in which a left side, a right side and an upper side are determined by the outline of the head area and a lower side is determined by a shape ratio constant of a general person'"'"'s head.

36. The person'"'"'s portrait generation method according to claim 35,

wherein the face parts area is determined by relative positional information of the face parts with respect to the head area preliminarily found.

37. The person'"'"'s portrait generation method according to claim 35,

wherein the position of characteristic features of the face is detected based on a position of a center-of-gravity of the projection data.

38. The person'"'"'s portrait generation method according to claim 35,

wherein the position of characteristic features of the face is detected by based on a position of a maximum value of the projection data.

40. The person'"'"'s portrait generation method according to claim 39,

wherein in the face outline determining step, the outline of the jaw is determined.

41. The person'"'"'s portrait generation method according to claim 39,

wherein in the face outline determining step, the skin color area is determined by converting an inputted RGB value to an HSV value.

42. The person'"'"'s portrait generation method according to claim 39,

wherein the determining method in the face outline determining step, with respect to an average color in the vicinity of the characteristic features of the face, areas having similar colors are used as face area candidates.

35. A person'"'"'s portrait generation method comprising:

an image input step of picking up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting step of extracting a head area from the image picked up in the image input step;

a feature detection step of detecting position of characteristic features of the face by dividing the head area thus extracted into face part areas for respective face parts and finding projection data of binarized images for the respective face parts areas;

a face outline determining step of determining a border between a face outline and a background within the head area; and

an image processing step of creating a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extraction step, the feature detection step and the face outline determining step.

39. A person'"'"'s portrait generation method comprising:

an image input step of picking up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting step of extracting a head area from the image picked up in the image input step;

a feature detection step of detecting position of characteristic features within the extracted head area;

a face outline determining step of determining a border between a face outline and a background within the head area by specifying a skin color area of the two-dimensional image; and

an image processing step of creating a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extraction step, the feature detection step and the face outline determining step.

43. A person'"'"'s portrait generation method comprising:

an image input step of picking up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting step of extracting a head area from the image picked up in the image input step;

a feature detection step of detecting position of characteristic features of the face for each of the face parts within the head area that has been extracted;

a face outline determining step of determining a border between a face outline and a background within the head area; and

an image processing step of creating a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized by changing the two-dimensional image with respect to each of the face parts.

44. The person'"'"'s portrait generation method according to claim 43,

wherein the image processing step expresses a person'"'"'s emotions by changing the shape of a partial image at each of an eye portion, a noise portion and a mouth portion that are the characteristic features of the face.

45. The person'"'"'s portrait generation method according to claim 43,

wherein the image processing step replaces partial images at an eye portion, a noise portion and a mouth portion that are characteristic features of the face.

46. A communication method using a communication terminal that comprises:

an image input step of picking up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting step of extracting a head area from a differential image of a plurality of the two-dimensional image picked up in the image input step;

a feature detection step of detecting position of characteristic features within the extracted head area;

a face outline determining step of determining a border between a face outline and a background within the head area by specifying a skin color area of the two-dimensional image;

an image processing step of creating a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extraction step, the feature detection step and the face outline determining step; and

a step in which, when a communication is received, the person'"'"'s portrait thus created is displayed so as to inform the user of the receipt of the communication.

47. A communication method using a communication terminal that comprises:

an image input step of picking up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting step of extracting a head area from the image picked up in the image input step;

a feature detection step of detecting position of characteristic features of the face by dividing the head area thus extracted into face part areas for respective face parts and finding projection data of binarized images for the respective face parts areas;

a face outline determining step of determining a border between a face outline and a background within the head area;

an image processing step of creating a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extraction step, the feature detection step and the face outline determining step; and

a step in which, when a communication is received, the person'"'"'s portrait thus created is displayed so as to inform the user of the receipt of the communication.

48. A communication method using a communication terminal that comprises:

an image input step of picking up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting step of extracting a head area from the image picked up in the image input step;

a feature detection step of detecting position of characteristic features within the extracted head area;

a face outline determining step of determining a border between a face outline and a background within the head area by specifying a skin color area of the two-dimensional image;

an image processing step of creating a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized based upon the two-dimensional image by using data acquired in the head area extraction step, the feature detection step and the face outline determining step; and

a step in which, when a communication is received, the person'"'"'s portrait thus created is displayed so as to inform the user of the receipt of the communication.

49. A communication method using a communication terminal that comprises:

an image input step of picking up a two-dimensional image containing a person'"'"'s face by using an image sensor;

a head area extracting step of extracting a head area from the image picked up in the image input step;

a feature detection step of detecting position of characteristic features of the face for each of the face parts within the head area that has been extracted;

a face outline determining step of determining a border between a face outline and a background within the head area;

an image processing step of creating a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized by changing the two-dimensional image with respect to each of the face parts; and

a step in which, when a communication is received, the person'"'"'s portrait thus created is displayed so as to inform the user of the receipt of the communication.

50. A recording medium in which is recorded a person'"'"'s portrait generation program which is executed by a computer, and comprises:

a head area extracting program for extracting a head area of a person from a differential image of a plurality of two-dimensional images picked up by an image input section;

a feature detection program for detecting positions of characteristic features of the person'"'"'s portrait by dividing the head area into face part areas for respective face parts, finding projection data of binarized images for the respective face parts areas and detecting the position of each of the face parts based upon a center-of-gravity position of the projection data;

a face outline determining program for determining a border between a face outline and a background within the head area by specifying a skin color area of the two-dimensional image; and

an image processing program for creating a person'"'"'s portrait in which the characteristics in the person'"'"'s face is emphasized by changing the two-dimensional image with respect to each of the face parts.