IMAGE PROCESSING APPARATUS AND METHOD OF GENERATING FACE IMAGE

• US 20160267339A1
• Filed: 03/03/2016
• Published: 09/15/2016
• Est. Priority Date: 03/11/2015
• Status: Active Application

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Nakano, Shunsuke, Sato, Hiroshi, Kaneda, Yuji, Nomoto, Atsuo
• Granted Patent: US 10,558,851 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2210/44 Morphing
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

An image processing apparatus stores, as instance images, a plurality of face images obtained by photographing respective faces of a plurality of persons on a plurality of photographing conditions, while associating the persons with the photographing conditions for each part; obtains the photographing condition of the input face image; approximates the respective parts of the input face images, by a combination of the instance images of the plurality of persons stored in association with the parts and the obtained photographing conditions; decides, for each part of the input face image, a combination corresponding to the combination in the approximation from the instance images of the plurality of persons stored in association with the part and a predetermined photographing condition; and generates an image obtained by photographing the input face image on the predetermined photographing condition, by synthesizing the obtained combination on the whole input face image.

Claims

1. An image processing apparatus comprising:

a storing unit configured to store, as instance images, a plurality of face images obtained by photographing respective faces of a plurality of persons on a plurality of photographing conditions, while associating the persons with the photographing conditions for each part;

an obtaining unit configured to obtain the photographing condition of the input face image;

an approximating unit configured to approximate respective parts of the input face images, by a combination of the instance images of the plurality of persons stored in the storing unit in association with the parts and the photographing conditions obtained by the obtaining unit;

a deciding unit configured to decide, for each part of the input face image, a combination corresponding to the combination by the approximating unit from the instance images of the plurality of persons stored in the storing unit in association with the part and a predetermined photographing condition; and

a generating unit configured to generate an image obtained by photographing the input face image on the predetermined photographing condition, by synthesizing combinations obtained by the deciding unit on the whole of the input face image.

2. The image processing apparatus according to claim 1, wherein the combination includes a weighted linear combination.

3. The image processing apparatus according to claim 2, wherein the approximating unit calculates the weight of the linear combination by representing the input face image and the instance image with vectors having pixel values arranged.

4. The image processing apparatus according to claim 1, wherein the approximating unit obtains the combination while giving priority to the instance image similar to the input face image.

5. The image processing apparatus according to claim 1, wherein the obtaining unit obtains the photographing condition by detecting it from the input face image.

6. The image processing apparatus according to claim 1, wherein

the deciding unit decides the combination for each of the plurality of photographing conditions, and

the generating unit generates the image obtained by photographing the input face image on each of the plurality of photographing conditions.

7. The image processing apparatus according to claim 6, further comprising:

a registering unit configured to, for each of the plurality of input face images, associate the face image generated by the generating unit for each of the plurality of photographing conditions with the person and register the associated face image as a registration image;

a calculating unit configured to calculate similarity between a newly input face image and each of the plurality of registration images; and

a discriminating unit configured to discriminate the newly input face image based on the similarity calculated for each of the plurality of registration images.

8. The image processing apparatus according to claim 1, further comprising:

a registering unit configured to, for each of the plurality of input face images, associate the face image generated by the generating unit with the person and register the associated face image as a registration image;

a calculating unit configured to calculate similarity between the face image generated from a newly input face image by the generating unit and each of the plurality of registration images; and

a discriminating unit configured to discriminate the newly input face image based on the similarity calculated for each of the plurality of registration images.

9. The image processing apparatus according to claim 1, wherein the photographing condition includes a direction of the face image.

10. The image processing apparatus according to claim 1, wherein the photographing condition includes an illumination condition.

11. The image processing apparatus according to claim 1, wherein the photographing condition includes resolution.

12. The image processing apparatus according to claim 1, wherein the photographing condition includes a photographing time.

13. The image processing apparatus according to claim 1, wherein the photographing condition includes an expression of the face.

14. An image processing method comprising:

storing, as instance images in a storing unit, a plurality of face images obtained by photographing respective faces of a plurality of persons on a plurality of photographing conditions, while associating the persons with the photographing conditions for each part;

obtaining the photographing condition of the input face image;

approximating the respective parts of the input face images, by a combination of the instance images of the plurality of persons stored in the storing unit in association with the parts and the obtained photographing conditions;

deciding, for each part of the input face image, a combination corresponding to the combination used in the approximation from the instance images of the plurality of persons stored in the storing unit in association with the part and a predetermined photographing condition; and

generating an image obtained by photographing the input face image on the predetermined photographing condition, by synthesizing the decided combination on the whole of the input face image.

15. A non-transitory computer-readable storage medium storing a program which causes a computer to execute steps of:

storing, as instance images in a storing unit, a plurality of face images obtained by photographing respective faces of a plurality of persons on a plurality of photographing conditions, while associating the persons with the photographing conditions for each part;

obtaining the photographing condition of the input face image;

approximating the respective parts of the input face images, by a combination of the instance images of the plurality of persons stored in the storing unit in association with the parts and the obtained photographing conditions;

deciding, for each part of the input face image, a combination corresponding to the combination used in the approximation from the instance images of the plurality of persons stored in the storing unit in association with the part and a predetermined photographing condition; and

generating an image obtained by photographing the input face image on the predetermined photographing condition, by synthesizing the decided combination on the whole of the input face image.