Method And System For Example-Based Face Hallucination

• US 20110305404A1
• Filed: 08/18/2010
• Published: 12/15/2011
• Est. Priority Date: 06/14/2010
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Lin, Chia-Wen, Hsu, Chih-Chung
• Granted Patent: US 8,488,913 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole

Abstract

A method for example-based face hallucination uses manifold learning to project a plurality of training images in a training database and an input low resolution (LR) face image into a same manifold domain, then iteratively refines the reconstruction basis by selecting a training set having k projected training images which best match the parts of the projected LR face image, where k≦N and N is the number of projected training images. Through the best-match training set, a set of prototype faces are learned, and the set of prototype faces are used as the reconstruction basis to reconstruct a high resolution face image for the input LR face image.

Claims

1. A method for example-based face hallucination on a face hallucination system, comprising:

preparing a training database with a plurality of training images and inputting a low resolution (LR) face image to be hallucinated;

projecting said plurality of training images of said training database and said LR face image onto a same manifold by using manifold learning, where y_L representing projected said LR face image and y_train representing projected said training images;

selecting a training set best matching y_L from N projected training images y_train, where N is not greater than number of said plurality of training images;

learning a set of basis images by using basis decomposition on said training set and y_L, said set of basis images including a high and low resolution prototype faces in said training set and a low resolution prototype face of y_L; and

reconstructing a high resolution face image of said LR face image by using said basis images.

2. The method as claimed in claim 1, wherein said training set further includes k face images selected from said N projected training images y_train being most similar to projected said LR face image y_L, k≦

N.

3. The method as claimed in claim 1, wherein said plurality of training images and said LR face image use a dimensionality reduction algorithm for projection onto said same manifold domain.

4. The method as claimed in claim 1, wherein difference between said high resolution prototype face set of said training set and said y_L prototype face matches a threshold requirement.

5. The method as claimed in claim 1, wherein said basis decomposition scheme uses principal component analysis (PCA) as a basis decomposition function to extract said set of basis images.

6. The method as claimed in claim 1, wherein said selecting said training set best matching y_L, from N projected training images y_train, and said learning said set of basis images by using basis decomposition on said training set and y_L, further include

selecting k face images from N projected training images y_train being most matching to projected said LR face image y_L, k≦

N;

through said k faces y_train being most matching to projected LR face y_L, extracting an LR prototype face and a set of high resolution prototype faces by using basis decomposition; and

when difference between said LR prototype face and said high resolution prototype face set matching a threshold requirement, using said LR prototype face and said high resolution prototype face set as basis images;

otherwise, incrementing k and repeating above two steps until obtaining said set of basis images.

7. The method as claimed in claim 6, said method uses a cost function to determine k value, and said k value determined by said cost function minimizes the difference between said LR face image and a linear combination of a prototype face set of said training set.

8. A system for example-based face hallucination, comprising:

a training database, for collecting and storing a plurality of training images;

a projection module, for receiving said plurality of training images, using a manifold learning method to obtain a projection matrix, and projecting said plurality of training images and an input low resolution (LR) face image onto said manifold domain to obtain N projected training images y_train and a projected said LR face image y_L;

a matching module, for selecting a training set of k best matching to y_L from said N projected training images y_train, k≦

N;

a basis decomposition module, for using basis decomposition on said training set and y_L, to learn an LR prototype face and a set of high resolution prototype faces of said training set, and difference between said LR prototype face and said set of high resolution prototype faces matching a threshold requirement; and

a face hallucination module, for reconstruct a high resolution face image for said LR face image by using said LR prototype face and said set of high resolution prototype faces as a set of basis image.

9. The system as claimed in claim 8, wherein when the difference between said LR prototype face and said high resolution prototype face set does not match a threshold requirement, said basis decomposition module increments k and said matching module selects another training set according to k value from N projected training images y_train, said basis decomposition module extracts another set of high resolution prototype faces until the difference between said LR prototype face and said high resolution prototype face set matches said threshold requirement.

10. The system as claimed in claim 8, said face hallucination system is executed on a computer system, said computer system at least comprises:

a memory device, for realizing said training database; and

a processor, further including said projection module, said matching module, said basis decomposition module, and said face hallucination module, and for receiving said input LR face and executing functions of said projection module, said matching module, said basis decomposition module, and said face hallucination module, and by combining a plurality of parameters to generate said high resolution face image for said LR face image.

11. The system as claimed in claim 8, wherein said k is automatically determined.

12. The system as claimed in claim 8, wherein projected high resolution and low resolution images of said plurality of training images onto same manifold domain have similar distribution.

13. The system as claimed in claim 8, wherein said face hallucination module combines a plurality of appropriate parameters and said set of basis images to obtain said high resolution face image.