METHODS OF GENERATING PERSONALIZED 3D HEAD MODELS OR 3D BODY MODELS

• US 20190035149A1
• Filed: 08/15/2016
• Published: 01/31/2019
• Est. Priority Date: 08/14/2015
• Status: Active Grant

• Current Assignee: Metail Limited
• Original Assignee: Metail Limited
• Inventors: CHEN, Yu, SHIN, Dongjoe, TOWNSEND, Joe, DOWNING, Jim, ROBERTSON, Duncan, ADEYOOLA, Tom
• Granted Patent: US 10,796,480 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G06T 2219/2021 Shape modification
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

There is provided a method of generating an image file of a personalized 3D head model of a user, the method comprising the steps of: (i) acquiring at least one 2D image of the user'"'"'s face; (ii) performing automated face 2D landmark recognition based on the at least one 2D image of the user'"'"'s face; (iii) providing a 3D face geometry reconstruction using a shape prior; (iv) providing texture map generation and interpolation with respect to the 3D face geometry reconstruction to generate a personalized 3D head model of the user, and (v) generating an image file of the personalized 3D head model of the user. A related system and computer program product are also provided.

Claims

1. A method of generating an image file of a personalized 3D head model of a user, the method comprising the steps of:

(i) acquiring at least one 2D image of the user'"'"'s face;

(ii) performing automated face 2D landmark recognition based on the at least one 2D image of the user'"'"'s face;

(iii) providing a 3D face geometry reconstruction using a shape prior;

(iv) providing texture map generation and interpolation with respect to the 3D face geometry reconstruction to generate a personalized 3D head model of the user, and

(v) generating an image file of the personalized 3D head model of the user.

2. The method of claim 1, wherein the at least one 2D image of the user'"'"'s face is acquired via a network communication.

3. The method of claim 2, wherein the at least one 2D image of the user'"'"'s face is acquired via the network communication, from a smartphone including a camera.

4. The method of claim 1, wherein the at least one 2D image of the user'"'"'s face is a front image of the user'"'"'s face.

5. The method of claim 1, wherein the at least one 2D image of the user'"'"'s face is a smartphone camera image of the user'"'"'s face.

6. The method of claim 1, wherein the automated face 2D landmark recognition includes using a 2D face landmark detector.

18. The method of claim 1, in which the image file is a 3D image file.

19. The method of claim 1, in which the image file is a 2D image file.

20. The method of claim 1, in which the image file is an animation file.

21. The method of claim 1, in which the image file is a personalised sticker set.

44. The method of claim 1, in which head size is estimated from body shape parameters.

45. The method of claim 1, in which an automatic image analysis is performed to help users quickly acquire input data of good quality so that they have a better chance of creating a photo-realistic personalised avatar.

46. The of claim 45, in which prior to starting the video or image capture, the user is presented with a live view of the camera feed, and a feedback mechanism analyses the live view and, if necessary, provides the user with recommendations on how to improve the conditions in order to achieve a high quality end result.

7-17. -17. (canceled)

22-43. -43. (canceled)

47. A system configured to perform a method of generating an image file of a personalized 3D head model of a user, the method comprising the steps of:

(i) acquiring at least one 2D image of the user'"'"'s face

(ii) performing automated face 2D landmark recognition based on the at least one 2D image of the user'"'"'s face;

(iii) providing a 3D face geometry reconstruction using a shape prior

(iv) providing texture map generation and interpolation with respect to the 3D face geometry reconstruction to generate a personalized 3D head model of the user, and

(v) generating an image file of the personalized 3D head model of the user.

48. A computer program product executable on a processor to generate an image file of a personalized 3D head model of a user, the computer program product executable on the processor to:

(i) receive at least one 2D image of the user'"'"'s face;

(ii) perform an automated face 2D landmark recognition based on the at least one 2D image of the user'"'"'s face;

(iii) provide a 3D face geometry reconstruction using a shape prior;

(iv) provide texture map generation and interpolation with respect to the 3D face geometry reconstruction to generate a personalized 3D head model of the user, and

(v) generate an image file of the personalized 3D head model of the user.

49. The computer program product of claim 48, executable on the processor to perform a method of generating an image file of a personalized 3D head model of a user, the method comprising the steps of:

(i) acquiring at least one 2D image of the user'"'"'s face;

(ii) performing automated face 2D landmark recognition based on the at least one 2D image of the user'"'"'s face;

(iii) providing a 3D face geometry reconstruction using a shape prior;

(iv) providing texture map generation and interpolation with respect to the 3D face geometry reconstruction to generate a personalized 3D head model of the user, and

(v) generating an image file of the personalized 3D head model of the user.

50-83. -83. (canceled)