Method of reconstruction of tridimensional scenes and corresponding reconstruction device and decoding system

• US 20020110273A1
• Filed: 11/20/2001
• Published: 08/15/2002
• Est. Priority Date: 07/29/1997
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Inventors: Dufour, Cecile
• Granted Patent: US 6,661,914 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 17/005 Tree description, e.g. octree, quadtree
• CPC Class Codes: G06T 7/55 from multiple images

Abstract

The invention relates to a new method of reconstruction of tridimensional scenes. While conventional methods are often limited to the 3D reconstruction of the bounding volume of the concerned objects, the proposed method of recovery of a 3D geometric model from 2D views taken by one single camera, giving an information even about the parts which are hidden in each view, is implemented according to a first depth labeling step, implemented in a sub-system (11) and including initialization and projection sub-steps followed by a refinement process, and to a second reconstruction step, implemented in a sub-system (12). By means of a close cooperation of the 3D depth maps thus obtained for two views of a scene, a 3D model is identified and extracted.

Claims

1. A method of reconstruction of a tridimensional scene from a bidimensional video sequence corresponding to N successive images of a real scene, comprising in series, for each image, segmented into triangular regions, of the sequence:

(A) a first depth labeling step, in which, each view being considered as the projection of a continuous 3D sheet, a multi-view matching is performed independently on each view in order to get a disparity map corresponding to the depth map of said 3D sheet;

(B) a second 3D model extraction step, in which an octree subdivision of the 3D space is performed and the voxels lying in the intersection of all 3D depth sheets are kept.

2. A method according to claim 1, wherein said depth labeling step comprises in series an initialisation sub-step, provided for defining during a first iteration a preliminary 3D depth sheet for the concerned image, and a refinement sub-step, provided for defining, for each vertex of each region, an error vector corresponding for each sampled depth to the summation of correlated costs between each of the (N−

1) pairs of views on a window specifically defined for said vertex and storing the index that provides the minimum correlation cost, an additional operation being intended to replace after the first iteration the initialisation sub-step by a projection sub-step provided first for adjusting the position and field of view of the image acquisition device according to its parameters and the vertex map near to the image plane, and then for listing for each vertex the voxels that intersect the line passing through the vertex and the optical center of said acquisition device, in the viewing direction, and selecting the nearest voxel to the image plane.

3. A method according to claim 2, wherein said 3D model extraction step comprises in series a resolution definition sub-step, provided for defining the resolution of the voxel grid, and a voxel selection sub-step, provided for keeping for each view the voxels lying inside the non-empty spaces provided by each depth map and then only keeping voxels lying at the intersection of all non-empty spaces.

4. A device for reconstructing a tridimensional scene from a bidimensional video sequence corresponding to N successive images of a real scene, characterized in that:

(I) each of the N images of the sequence is segmented into triangular regions

(II) said device comprises, for processing each image of said sequence;

(A) a depth labeling sub-system, comprising itself in series;

(1) an initialisation device, provided for defining during a first iteration an error vector corresponding for a set of sampled depths to the summation of correlation costs between each of the (N−

1) pairs of views and the index providing the minimum correlation cost, the depth value of each vertex of the regions being computed by interpolation between the depths obtained for the neighboring regions;

(2) a refinement device, provided for defining similarly for each vertex an error vector on a previously delimited window and, correspondingly, the index providing the minimum correlation cost;

(B) a reconstruction sub-system provided for selecting the resolution of the voxel grid and keeping, for each view, the voxels lying inside the non-empty spaces provided by each depth map and, finally, only the voxels lying at the intersection of all non-empty spaces;

(III) said depth labeling sub-system also comprises a projection device intended to replace during the following iterations the initialisation device and provided for adjusting the position and field of view of the image acquisition device, and the vertex map very near to the image plane, and, for each vertex, listing the voxels that intersect the line passing through the vertex and the optical center of said acquisition device in the viewing direction and selecting the nearest voxel to the image plane.

5. A video decoding system comprising a reconstruction device according to claim 4.