METHOD AND A SYSTEM FOR GENERATING A REALISTIC 3D RECONSTRUCTION MODEL FOR AN OBJECT OR BEING

• US 20150178988A1
• Filed: 05/13/2013
• Published: 06/25/2015
• Est. Priority Date: 05/22/2012
• Status: Abandoned Application

• Current Assignee: Telefonica SA
• Original Assignee: Telefonica SA
• Inventors: Divorra Escoda, Oscar, Montserrat Mora, Tomas, Quelen, Juien, Ferran Bernstrom, Christian, Pages Scasso, Rafael, Berjon Diez, Daniel, Arnaldo Duart, Sergio, Moran Burgos, Francisco
• Application Number:
  US14/402,999
• Publication Number:
  US 20150178988A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation

Abstract

A method for generating a realistic 3D reconstruction model for an object or being, comprising:

• • a) capturing a sequence of images of an object or being from a plurality of surrounding cameras;
  • b) generating a mesh of said an object or being from said sequence of images captured;
  • c) creating a texture atlas using the information obtained from said sequence of images captured of said object or being;
  • d) deforming said generated mesh according to higher accuracy meshes of critical areas;and
  • e) rigging said mesh using an articulated skeleton model and assigning bone weights to a plurality of vertices of said skeleton model; the method comprises generating said 3D reconstruction model as an articulation model further using semantic information enabling animation in a fully automatic framework.

The system is arranged to implement the method of the invention.

Claims

1. A method for generating a realistic 3D reconstruction model for an object or being, comprising:

a) capturing a sequence of images of an object or being from a plurality of surrounding cameras;

b) generating a mesh of said an object or being from said sequence of images captured;

c) creating a texture atlas using the information obtained from said sequence of images captured of said object or being;

d) deforming said generated mesh according to higher accuracy meshes of critical areas;

and

e) rigging said mesh using an articulated skeleton model and assigning bone weights to a plurality of vertices of said skeleton model;

wherein the method is characterized in that it further comprises generating said 3D reconstruction model as an articulation model further using semantic information enabling animation in a fully automatic framework.

2. The method of claim 1, wherein said step a) is performed in a capture room.

3. The method of claim 2, comprising regulating said plurality of surrounding cameras under controlled local structured light conditions.

4. The method of claim 3, wherein said plurality of surrounding cameras are synchronized.

5. The method of claim 1, wherein said model is a closed and manifold mesh generated by means of at least one of a:

Shape from Silhouette techniques, Shape from Structured light techniques, Shape from Shading, Shape from Motion or any total or partial combination thereof.

6. The method of claim 5, comprising obtaining a three-dimensional scalar field, represented in voxels, of a volume of said mesh or model.

7. The method of claim 6, further comprising applying multi-view stereo methods for representing critical areas with higher detail of objects or beings.

8. The method of claim 7, comprising using a local high density mesh for representing said critical areas.

9. The method of claim 8, wherein said local high density mesh is based on shape from structured light techniques.

10. The method of claim 8, wherein said local high density mesh uses an algorithm based on depth maps.

11. The method of claim 8, further comprising using said local high density mesh with a synthetic 3D model.

12. The method of claim 8, comprising merging said generating mesh of said step b) and said local high density mesh by means of obtaining a quality improved global mesh.

13. The method of claim 1, wherein said information used for creating said texture atlas in said step c) is directly employed to texture said mesh.

14. The method of claim 13, further comprising a pre-processing step for said sequence of images captured.

15. The method of claim 14, wherein said information used for creating said texture atlas comprises information concerning the colour of said sequence of images.

16. The method of claim 1, wherein said assigned bone weights to said plurality of vertices of said skeleton model in said step e) further comprises using a combination of a Laplace diffusion equation and a Flexible Skinning weight method.

17. The method of claim 1, comprising concatenating said rigged and weighted mesh and said texturized mesh by means of being used in a compatible CAD application.

18. The method of claim 1, wherein said object or being reconstruction model comprises a human model.

20. The system of claim 19, wherein said system implementing a method according to any of the previous claims.

19. A system for generating a realistic 3D reconstructionmodel for an object or being, comprising:

a capture room equipped with a plurality of cameras surrounding an object or being to be scanned; and

a plurality of capture servers for storing images of said object or being from said plurality of cameras,

characterized in that said images of said object or being are used for fully automatically generate said 3D reconstruction model as an articulation model.

21. The system of claim 19, wherein said capture room is equipped with a plurality of pheripheral cameras arrange to reconstruct a Visual Hull (VH) of said object or being scanned.

22. The system of claim 19, wherein said capture room is equipped with a plurality of local cameras arranged to obtain a high quality local mesh for critical areas of said object or being scanned.

23. The system of claim 22, comprising a structured light pattern arranged to project on said object or being to be scanned.

24. The system of claim 19, wherein said object or being is a human or living being.