Image processing apparatus

• US 6,141,431 A
• Filed: 02/01/1996
• Issued: 10/31/2000
• Est. Priority Date: 02/02/1995
• Status: Expired due to Term

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Munetsugu, Toshihiko, Kado, Yoshiyasu, Maehara, Fumio, Nakagawa, Masamichi
• Primary Examiner(s): Chang, Jon
• Assistant Examiner(s): Patel, Jayanti K.
• Application Number:
  US08/595,224
• Time in Patent Office: 1,734 Days
• Field of Search: 382/100, 382/162, 382/266, 382/282, 382/284, 382/294, 382/115, 382/117, 382/118, 382/283, 348/77, 348/584, 348/586, 348/590, 348/597, 348/78, 358/453, 358/538, 358/452, 395/135, 434/94, 434/371
• US Class Current: 382/100
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 2005/2726 for simulating a person's appearance, e.g. hair style, glasses, clothes

Abstract

An image processing apparatus has a hair region selecting part 11 for selecting hair and shade of the hair in a front face image 13, as a removal region 15, on the basis of each pixel value of the front face image 13 which is a front image of a face, and a face surface model 14 showing structural information relative to a surface of the face in the front face image 13, and a removal region changing part 12 for changing each pixel value in whole or part of the removal region 15, on the basis of the structural information in the face surface model 14, and at least one pixel value in the front image.

Claims

1. An image processing apparatus comprising

a region selecting part for automatically selecting a region of hair or hair and hair shadow in a front face image of a person, on the basis of each pixel value in whole or part of the front image, and a surface model showing structural information including positions of at least eyes, mouth and eyebrows, relative to a surface of a face of the person in the front face image, and

a region changing part for automatically changing each pixel value in whole or part of the selected region, on the basis of (a) color information of at least one pixel in the front face image, the at least one pixel being obtained by using said surface model and (b) said surface model,

wherein

the front image is a front face image of the face of the person,

the surface model shows structural information relative to a surface of the face of the person in the front face image including positions of the eyes, nose, mouth and eyebrows in the front face image,

the region selecting part selects a hair region of the person in the front face image in a removal region, and

the region changing part changes each pixel value in whole or part of the removal region, on the basis of the structural information in the face surface model including the positions of the eyes, nose, mouth and eyebrows in the front face image, and color information obtained from the at least one pixel value in the front face image.

2. An image processing apparatus of claim 1, wherein the hair has a shade and the hair region selecting part selects the hair of the person as the removal region using the shade of the hair.

3. An image processing apparatus of claim 2, wherein the hair region includes pixel values and the hair region selecting part further changes the pixel values in the hair region to other values.

4. An image processing apparatus of claim 2, wherein the removal region includes an area having first pixel values and a remainder area having second pixel values, and when the first pixel values in the area of the removal region are changed, the removal region changing part further changes the second pixel values in the remainder area to other values than the pixel values in the front face image corresponding to inside of head contour in the face surface model.

5. An image processing apparatus of claim 2, wherein the structural information includes the head contour and eye position and eyebrow position the hair region selecting part selects the hair in response to the head contour and the eye position and the eyebrow position.

6. An image processing apparatus of claim 2, wherein the front face image has pixels having a lightness value and the hair region selecting part selects the hair and the shade of the hair in the removal region in response to the lightness value of each pixel in the front face image.

7. An image processing apparatus of claim 2, wherein the front face image includes pixels having lightness values and saturation values and the hair region selecting part selects the hair in response to the lightness values and the saturation values of each of the pixels.

8. An image processing apparatus of claim 2, further comprising a fluctuation generating part for adding specific fluctuation component to each pixel value in the removal region, on the basis of color information of at least one pixel in the front face image,

wherein at least one pixel in the front face image is not included in the removal region.

9. An image processing apparatus of claim 2, further comprising a face surface model generating part for generating the face surface model, on the basis of the front face image.

10. An image processing apparatus of claim 1, wherein the hair region includes pixel values and the hair region selecting part further changes the pixel values in the hair region to other values.

11. An image processing apparatus of claim 1, wherein the removal region includes an area having first pixel values and a remainder area having second pixel values, when the first pixel values in the area of the removal region are changed, the removal region changing part further changes the second pixel values in the remainder area to other values than the pixel values in the front face image corresponding to inside of head contour in the face surface model.

12. An image processing apparatus comprising:

an image adjusting part

on the basis of a surface model showing structural information, including positions of at least eyes and nose, relative to a surface of a face of a person in a front image of the person, and a surface model showing structural information including positions of at least eyes and nose, relative to a surface of a face of another person in a reference image which is a front image of the other person, for applying a specific conversion to said reference image and further to attribute information of each pixel of said reference image, so that a hair or hair and hair shadow of said other person may be suited to the front image of said person, and

a synthesis processing part for assigning the hair or hair and hair shadow of said other person in the reference image in which the specific conversion is applied, to said front image, according to the attribute information in which the specific conversion is applied wherein said attribute information of each pixel of said reference image, has at least two kinds of attributes with regard to the hair or hair and hair shadow of the other person.

13. An image processing apparatus of claim 12, wherein

the front image of the person is a front face image which is a front image of a face of the person,

the surface model is a face surface model showing structural information relative to a surface of the face of the person,

the reference image is a head image which is a front image of a face of the other person having hair,

the information model is a face information model showing structural information relative to a surface of the face of the other person in the head image,

the specific portion is head portion comprising the hair, or the hair and shade of the hair,

the attribute information is hair information held by corresponding the attribute of each pixel in the head image to the pixel,

the image adjusting part is a head image adjusting part for applying the specific conversion to the hair information and the head image, and

the synthesis processing part is a head synthesis processing part for assigning the front face image for the head portion in the head image in which the specific conversion is applied, according to the hair information in which the specific conversion is applied.

14. An image processing apparatus of claim 13, wherein the head image adjusting part applies the specific conversion to the head image and the hair information, by utilizing

information of head contour and whole or part of eyes, nose, mouth and eyebrows in the face surface model, and

information of head contour and whole or part of eyes, nose, mouth and eyebrows in the face information model.

15. An image processing apparatus of claim 13, wherein the head synthesis processing part comprises:

a generating part for generating pixel values of the head portion to be assigned to the front face image, according to the attribute of each pixel of the hair information, by using pixel value of the head image corresponding to the pixel of the attribute, or using the pixel value of the head image and pixel value of the front face image, corresponding to the pixel of the attribute, and

a mapping processing part for assigning the front face image for each pixel value of the head portion generated by the generating part.

16. An image processing apparatus of claim 15, further comprising:

a synthesis image correcting part for correcting the front face image in which the head portion is assigned by the mapping processing part.

17. An image processing apparatus of claim 15, wherein the generating part comprises at least one of:

a hue synthesis part for generating, according to the attribute, a hue by using the pixel value of the head image corresponding to the pixel of the attribute, or using the pixel value of the head image and the pixel value of the front face image, corresponding to the pixel of the attribute,

a lightness synthesis part for generating, according to the attribute, a lightness by using the pixel value of the head image corresponding to the pixel of the attribute, or using the pixel value of the head image and the pixel value of the front face image, corresponding to the pixel of the attribute, and

a saturation synthesis part for generating, according to the attribute, a saturation by using the pixel value of the head image corresponding to the pixel of the attribute, or using the pixel value of the head image and the pixel value of the front face image, corresponding to the pixel of the attribute.

18. An image processing apparatus of claim 13, further comprising a face surface model generating part for generating the face surface model on the basis of the front face image.

19. An image processing apparatus comprising:

a region selecting part for automatically selecting a region of hair or hair and hair shadow in a front face image of a person, on the basis of each pixel value in whole or part of the front image, and a surface model showing structural information including positions of at least eyes, mouth and eyebrows, relative to a surface of a face of the person in the front face image, and

a region changing part for automatically changing each pixel value in whole or part of the selected region, on the basis of (a) color information of at least one pixel in the front face image, the at least one pixel being obtained by using said surface model and (b) said surface model,

an image adjusting part, on the basis of said surface model of said person, and a surface model showing structural information including positions of at least eyes and nose, relative to a surface of a face of another person in a reference image which is a front image of the other person, for applying a specific conversion to said reference image and further to attribute information of each pixel of said reference image, so that a hair or hair and hair shadow of said other person may be suited to the front image of said person, and

a synthesis processing part for assigning the hair or hair and hair shadow of said other person in the reference image to which the specific conversion is applied, to said changed region of said selected region, according to the attribute information to which the specific conversion is applied.