APPARATUS AND METHOD FOR RECONSTRUCTING THREE-DIMENSIONAL FACE AVATAR THROUGH STEREO VISION AND FACE DETECTION

• US 20110148868A1
• Filed: 12/20/2010
• Published: 06/23/2011
• Est. Priority Date: 12/21/2009
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: HWANG, Dae Hwan, CHO, Jae Il, CHANG, Ji Ho, LIM, Eul Gyoon
• Application Number:
  US12/973,326
• Publication Number:
  US 20110148868A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/20028 Bilateral filtering
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

Provided are an apparatus and method for reconstructing 3D face avatar. The apparatus includes a face detection unit, a stereo matching unit, a bilateral filter, and a texture mapping unit. The face detection unit receives a left image and right image of a user, and detects a face image of the user from the left and right images using a face detection algorithm. The stereo matching unit receives the left and right images of the user, and creates a depth map image from the left and right images through a stereo matching operation which uses disparity between the left and right images. The bilateral filter abstracts the detected face image through a bilateral filtering operation. The texture mapping unit texture-maps the abstracted face image on the created depth map image to reconstruct a 3D avatar.

Claims

1. An apparatus for reconstructing Three-Dimensional (3D) face avatar, comprising:

a face detection unit receiving a left image and right image of a user, and detecting a face image of the user from the left and right images using a face detection algorithm;

a stereo matching unit receiving the left and right images of the user, and creating a depth map image from the left and right images through a stereo matching operation which uses disparity between the left and right images;

a bilateral filter abstracting the detected face image through a bilateral filtering operation; and

a texture mapping unit texture-mapping the abstracted face image on the created depth map image to reconstruct a 3D avatar.

2. The apparatus of claim 1, further comprising:

an image obtainment unit comprising left and right cameras which are arranged in parallel, obtaining the left image of the user through the left camera, obtaining the right image of the user through the right camera, and transferring the obtained left and right images to the face detection unit and the stereo matching unit.

3. The apparatus of claim 2, wherein the left and right cameras are Complementary Metal-Oxide-Semiconductor (CMOS) cameras which are mounted on mobile equipment.

4. The apparatus of claim 1, further comprising:

a position reallocation unit extracting a feature point of the abstracted face image and a feature point of the depth map image, and reallocating the extracted feature points to be 1;

1 matched.

5. The apparatus of claim 4, further comprising:

a depth map refinement unit receiving the depth map image from the stereo matching unit, and removing a noise and error of the depth map image to provide the depth map image to the position reallocation unit.

6. The apparatus of claim 1, wherein the bilateral filter determines an abstraction level according to a parameter which is inputted by the user, and controls abstraction of the detected face image according to the determined abstraction level.

7. The apparatus of claim 1, wherein the face detection unit, the stereo matching unit, the bilateral filter and the texture mapping unit are comprised in one module.

8. The apparatus of claim 7, wherein the one module is mounted on mobile equipment.

9. A method for reconstructing Three-Dimensional (3D) face avatar, the method comprising:

detecting a face image of a user from the left and right images of the user by using a face detection algorithm;

receiving the left and right images of the user, and creating a depth map image from the left and right images through a stereo matching operation which uses disparity between the left and right images;

abstracting the detected face image through a bilateral filtering operation; and

texture-mapping the abstracted face image on the created depth map image to reconstruct a 3D avatar.

10. The method of claim 9, further comprising:

obtaining the left and right images of the user through a left camera and a right camera which are arranged in parallel, respectively.

11. The method of claim 9, further comprising:

extracting a feature point of the abstracted face image and a feature point of the depth map image, and reallocating the extracted feature points to be 1;

1 matched.

12. The method of claim 9, wherein the abstracting of the detected face image comprises:

determining an abstraction level according to a parameter which is inputted by the user; and

controlling abstraction of the detected face image according to the determined abstraction level.