SYSTEMS AND METHODS FOR PRODUCING A THREE-DIMENSIONAL FACE MODEL

• US 20150009207A1
• Filed: 07/08/2013
• Published: 01/08/2015
• Est. Priority Date: 07/08/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Guo, Feng, Bi, Ning, Qi, Yingyong
• Granted Patent: US 9,842,423 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

A method for three-dimensional face generation is described. An inverse depth map is calculated based on a depth map and an inverted first matrix. The inverted first matrix is generated from two images in which pixels are aligned vertically and differ horizontally. The inverse depth map is normalized to correct for distortions in the depth map caused by image rectification. A three-dimensional face model is generated based on the inverse depth map and one of the two images.

Claims

1. A method for three-dimensional face generation, comprising:

calculating an inverse depth map based on a depth map and an inverted first matrix, wherein the inverted first matrix is generated from two images in which pixels are aligned vertically and differ horizontally;

normalizing the inverse depth map to correct for distortions in the depth map caused by image rectification; and

generating a three-dimensional face model based on the inverse depth map and one of the two images.

2. The method of claim 1, further comprising:

obtaining a first matrix that corresponds to a first image of the two images;

obtaining a second matrix that corresponds to a second image of the two images;

applying the first matrix to the first image to obtain a first rectified image; and

applying the second matrix to the second image to obtain a second rectified image.

3. The method of claim 2, wherein the first matrix is a homographic matrix corresponding to the first image, and wherein the second matrix is a homographic matrix corresponding to the second image.

4. The method of claim 2, wherein calculating the inverse depth map comprises:

inverting the first matrix to obtain the inverted first matrix; and

applying the inverted first matrix to the depth map.

5. The method of claim 2, further comprising determining the depth map from the first rectified image and the second rectified image.

6. The method of claim 2, further comprising determining a fundamental matrix from the two images.

7. The method of claim 6, wherein obtaining the first matrix and the second matrix comprises decomposing the fundamental matrix.

8. The method of claim 1, wherein the two images are different perspectives relative to a human face.

9. The method of claim 1, wherein normalizing the inverse depth map comprises normalizing a subset of pixels in the inverse depth map.

10. The method of claim 1, wherein normalizing the inverse depth map comprises one of rotating, tilting, and scaling the inverse depth map.

11. The method of claim 10, wherein tilting the inverse depth map comprises:

determining depth derivatives for each column of pixels in a subset selection of the inverse depth map;

determining, for the subset selection, a mean value of all the depth derivatives;

comparing the mean value to a threshold value; and

tilting the inverse depth map based on the comparison.

12. The method of claim 11, wherein the subset selection comprises a rectangular area.

13. The method of claim 11, wherein the subset selection comprises a first area and a second area, and wherein comparing comprises comparing a difference between the mean value of the first area and the mean value of the second area to a threshold value.

14. The method of claim 10, wherein rotating the inverse depth map comprises:

detecting symmetrical features in the inverse depth map, wherein the symmetrical features comprise a left feature and a right feature; and

adjusting the inverse depth map proportionally to make the left feature and the right feature horizontally level.

15. The method of claim 10, wherein scaling the inverse depth map comprises:

detecting symmetrical features in the inverse depth map, wherein the symmetrical features comprise a left feature and a right feature; and

adjusting the inverse depth map to make the left feature and the right feature proportionally even and at the same depth level.

16. An apparatus for three-dimensional face generation, comprising:

a processor;

memory in electronic communication with the processor;

instructions stored in the memory, the instructions being executable to;

calculate an inverse depth map based on a depth map and an inverted first matrix, wherein the inverted first matrix is generated from two images in which pixels are aligned vertically and differ horizontally;

normalize the inverse depth map to correct for distortions in the depth map caused by image rectification; and

generate a three-dimensional face model based on the inverse depth map and one of the two images.

17. The apparatus of claim 16, further comprising instructions executable to:

obtain a first matrix that corresponds to a first image of the two images;

obtain a second matrix that corresponds to a second image of the two images;

apply the first matrix to the first image to obtain a first rectified image; and

apply the second matrix to the second image to obtain a second rectified image.

18. The apparatus of claim 17, wherein the first matrix is a homographic matrix corresponding to the first image, and wherein the second matrix is a homographic matrix corresponding to the second image.

19. The apparatus of claim 17, wherein the instructions being executable to calculate the inverse depth map comprise instructions being executable to:

invert the first matrix to obtain the inverted first matrix; and

apply the inverted first matrix to the depth map.

20. The apparatus of claim 17, further comprising instructions executable to determine the depth map from the first rectified image and the second rectified image.

21. The apparatus of claim 17, further comprising instructions executable to determine a fundamental matrix from the two images.

22. The apparatus of claim 21, wherein the instructions being executable to obtain the first matrix and the second matrix comprise instructions being executable to decompose the fundamental matrix.

23. The apparatus of claim 16, wherein the two images are different perspectives relative to a human face.

24. The apparatus of claim 16, wherein the instructions being executable to normalize the inverse depth map comprise instructions being executable to normalize a subset of pixels in the inverse depth map.

25. The apparatus of claim 16, wherein the instructions being executable to normalize the inverse depth map comprise one of instructions being executable to rotate, instructions being executable to tilt, and instructions being executable to scale the inverse depth map.

26. The apparatus of claim 25, wherein the instructions being executable to tilt the inverse depth map comprise instructions being executable to:

determine depth derivatives for each column of pixels in a subset selection of the inverse depth map;

determine, for the subset selection, a mean value of all the depth derivatives;

compare the mean value to a threshold value; and

tilt the inverse depth map based on the comparison.

27. The apparatus of claim 26, wherein the subset selection comprises a rectangular area.

28. The apparatus of claim 26, wherein the subset selection comprises a first area and a second area, and wherein the instructions being executable to compare comprise instructions being executable to compare a difference between the mean value of the first area and the mean value of the second area to a threshold value.

29. The apparatus of claim 25, wherein the instructions being executable to rotate the inverse depth map comprise instructions being executable to:

detect symmetrical features in the inverse depth map, wherein the symmetrical features comprise a left feature and a right feature; and

adjust the inverse depth map proportionally to make the left feature and the right feature horizontally level.

30. The apparatus of claim 25, wherein the instructions being executable to scale the inverse depth map comprise instructions being executable to:

detect symmetrical features in the inverse depth map, wherein the symmetrical features comprise a left feature and a right feature; and

adjust the inverse depth map to make the left feature and the right feature proportionally even and at the same depth level.

31. An apparatus for three-dimensional face generation, comprising:

means for calculating an inverse depth map based on a depth map and an inverted first matrix, wherein the inverted first matrix is generated from two images in which pixels are aligned vertically and differ horizontally;

means for normalizing the inverse depth map to correct for distortions in the depth map caused by image rectification; and

means for generating a three-dimensional face model based on the inverse depth map and one of the two images.

32. The apparatus of claim 31, wherein the means for normalizing the inverse depth map comprise means for normalizing a subset of pixels in the inverse depth map.

33. The apparatus of claim 31, wherein the means for normalizing the inverse depth map comprise one of means for rotating, tilting, and scaling the inverse depth map.

34. The apparatus of claim 33, wherein the means for tilting the inverse depth map comprise:

means for determining depth derivatives for each column of pixels in a subset selection of the inverse depth map;

means for determining, for the subset selection, a mean value of all the depth derivatives;

means for comparing the mean value to a threshold value; and

means for tilting the inverse depth map based on the comparison.

35. The apparatus of claim 34, wherein the subset selection comprises a first area and a second area, and wherein the means for comparing comprise means for comparing a difference between the mean value of the first area and the mean value of the second area to a threshold value.

36. The apparatus of claim 33, wherein the means for rotating the inverse depth map comprises:

means for detecting symmetrical features in the inverse depth map, wherein the symmetrical features comprise a left feature and a right feature; and

means for adjusting the inverse depth map proportionally to make the left feature and the right feature horizontally level.

37. A computer-program product for three-dimensional face generation, comprising a non-transitory computer-readable medium having instructions thereon, the instructions comprising:

code for causing an electronic device to calculate an inverse depth map based on a depth map and an inverted first matrix, wherein the inverted first matrix is generated from two images in which pixels are aligned vertically and differ horizontally;

code for causing the electronic device to normalize the inverse depth map to correct for distortions in the depth map caused by image rectification; and

code for causing the electronic device to generate a three-dimensional face model based on the inverse depth map and one of the two images.

38. The computer-program product of claim 37, wherein the code for causing the electronic device to normalize the inverse depth map comprises code for causing the electronic device to normalize a subset of pixels in the inverse depth map.

39. The computer-program product of claim 37, wherein the code for causing the electronic device to normalize the inverse depth map comprises one of code for causing the electronic device to rotate, tilt, and scale the inverse depth map.

40. The computer-program product of claim 39, wherein the code for causing the electronic device to tilt the inverse depth map comprises:

code for causing the electronic device to determine depth derivatives for each column of pixels in a subset selection of the inverse depth map;

code for causing the electronic device to determine, for the subset selection, a mean value of all the depth derivatives;

code for causing the electronic device to compare the mean value to a threshold value; and

code for causing the electronic device to tilt the inverse depth map based on the comparison.

41. The computer-program product of claim 40, wherein the subset selection comprises a first area and a second area, and wherein comparing comprises comparing a difference between the mean value of the first area and the mean value of the second area to a threshold value.

42. The computer-program product of claim 39, wherein the code for causing the electronic device to rotate the inverse depth map comprises:

code for causing the electronic device to detect symmetrical features in the inverse depth map, wherein the symmetrical features comprise a left feature and a right feature; and

code for causing the electronic device to adjust the inverse depth map proportionally to make the left feature and the right feature horizontally level.