MODEL AND METHOD FOR PRODUCING 3D PHOTOREALISTIC MODELS

• US 20140055570A1
• Filed: 03/19/2013
• Published: 02/27/2014
• Est. Priority Date: 03/19/2012
• Status: Active Grant

• Current Assignee: FittingBox S.A.
• Original Assignee: FittingBox S.A.
• Inventors: Dehais, Christophe, Le Gallou, Sylvain, Mammou, Khaled, Choukroun, Ariel
• Granted Patent: US 9,357,204 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence

Abstract

Disclosed is model and a method for producing dynamic photorealistic 3D models that is adaptive for visualization and real time fitting. The invention aims at a method for digitizing a three-dimensional object possibly including transparent or reflective parts from two-dimensional views of this object, the method including: A step for movable fastening of the object to a support, pivoting about a vertical axis, placed opposite at least two vertically spaced image shooting devices, A step for acquisition of object images by the image shooting devices during a rotation of the object about the vertical axis, A colorimetric correction step for the acquired images.

Claims

1. Method for digitizing a three-dimensional object, said object possibly including transparent or reflecting parts, from two-dimensional images of this object, characterized in that it comprises the following steps:

Movable fastening of the object to a support pivoting about a vertical axis placed opposite at least two vertically spaced image shooting devices;

Acquisition of views of the object by the image shooting devices during a rotation of the object about the vertical axis;

Colorimetric correction step for the acquired images.

2. Method for digitizing according to claim 1, characterized in that the colorimetric correction step comprises:

Obtaining a set of couples real color/recorded color by the image shooting devices,

Establishing a parametric model for the transfer function real color/recorded color using a generic model of non rigid transformation of color space,

the generic model advantageously taking the form of a thin-plate spline.

3. Method for digitizing according to claim 1, characterized in that it also comprises a geometric calibration step (103) consisting in shooting images of a target with a geometry known by the image shooting devices and the calculation of intrinsic and extrinsic parameters of the cameras of the acquisition machine.

4. Method for digitizing according to claim 3, from-motion technique and uses a calibration target consisting of colored metallic spheres with a known diameter and maintained at positions adapted to limit the occultation of spheres with each other and to uniformly distribute their positions in the field of view of the cameras,

whereby each camera acquires a sequences of images of the target at different orientations obtained by placing the target on a turntable about the vertical axis,

whereby the method comprises a step for locating the position of projection of the centers of each colored sphere in a set of images of the calibration target,

the calculation based on these positions of the internal parameters and of the positions and orientations of each image shooting device.

5. Method for digitizing according to claim 4, characterized in that the calculation step determines:

3D rigid transformations enabling to express a point in the benchmark of the object in a point in the benchmarks of each camera,

whereby parameters of the projection operation enabling to calculate 2D coordinates in the image produced by each camera of a point expressed in the benchmark thereof,

a scaling factor enabling to express the 3D points of the object in a metric benchmark.

6. Method for digitizing according to claim 3, characterized in that the geometric calibration step (103) comprises three main steps:

Retrieval of the projections of the 3D centers of the spheres;

this step comprises a series of processing operations of the input images leading to the determination of the center of the (assumed) circular imprints of the projections of the spheres;

Metric reconstruction of the scene;

this step enables an initial reconstruction of the scene (3D points) and of the cameras (internal and external parameters). It requires to determine the elliptical paths of each center of a sphere;

Adjustment of bundles;

this step enables to refine the parameters found in the preceding step, both internal and external parameters of the cameras and of the positions of the 3D centers of the spheres.

7. Method for digitizing according to claim 3, characterized in that the geometric calibration step (103) comprises:

A preliminary step consisting in determining the distribution of colors in each sphere in an adapted color space, step in which, considering an image comprising the set of spheres and a selections of areas completely covered by the image of each sphere, the characteristics of this distribution of color are determined;

A step of retrieval of the spheres during which the following steps are repeated for each image and for each sphere;

i—

Classification of the pixels in two classes sphere/background according to the probability of being in the image of the considered sphere,

ii—

Retrieval of the related components in the mask of the sphere class,

iii—

Rejection of the unsatisfactory related components according to their geometric characteristic;

elongation value, area,

iv—

On each component that has not been rejected in step iii, retrieval of the contour points in the original image,

v—

Robust and accurate matching of a circle on the contour points,

vi—

Retrieval of the center of the circle,

A step of determining the presence or absence of the considered sphere in the image

and, in case the sphere is actually visible in its entirety in the image, obtaining the center of its projection.

8. Method for constructing virtual images of a three-dimensional object from two-dimensional images of this object taken from different angles in at least two directions, said method comprising a digitizing method according to claim 1, characterized in that it also comprises the following phases:

1. Clipping (104, 105) of at least one part of the acquired 2D images in order to calculate a set of 2D silhouettes of the glasses 500 to be reconstructed;

2. Calculation, on the base of the obtained 2D silhouettes, of a volumetric representation of the object 500;

3. Calculation with a surface extraction algorithm of a surface representation of the object 500;

4. Smoothing and simplification of 3D surface mesh of the object 500.

9. Method for constructing virtual images according to claim 8, characterized in that it also comprises:

A smoothing step adapted to manufactured goods having constant curvatures per piece;

A step of dynamic representation of the appearance of an object;

A step of photorealistic rendering of an object simulating appearance variation related to the angle from which the object is viewed.

10. Method for constructing images according to claim 8, characterized in that the clipping uses a filling of the unwanted holes in the 3D volume by exploiting the reprojection of the 3D volume that has been previously clipped in the image to be clipped.

11. Method for constructing images of a pair of glasses comprising a method according to claim 8, characterized in that it comprises

A positioning aid step for the pairs of glasses in the field of view.

12. Method for constructing images of a pair of glasses comprising a method according to claim 11, characterized that it comprises:

A step of segmentation of the digital model of glasses in 2 bows and the rest of the frame with definition of the pivot points,

13. Method for constructing images of a pair of glasses comprising a method according to claim 11, characterized that it comprises:

A symmetrization step for the models of pairs of glasses

14. Method for constructing images of a pair of glasses comprising a method according to claim 11, characterized that it comprises:

A step enabling the semi-automatic reconstruction of a glasses lens using the profile of a three-dimensional path that is itself determined semi-automatically.

15. Method for constructing images of a pair of glasses comprising a method according to claim 11, characterized that it comprises:

A step for generation of glasses lenses by parametric approximation that guarantees a constant curvature.

16. Method for constructing images of a pair of glasses comprising a method according to claim 11, characterized that it comprises:

A step for computer-aided production of a model of a pair of glasses nearest to an existing model.

17. Method for digitizing according to claim 2, characterized in that it also comprises a geometric calibration step (103) consisting in shooting images of a target with a geometry known by the image shooting devices and the calculation of intrinsic and extrinsic parameters of the cameras of the acquisition machine.

18. Method for digitizing according to claim 4, characterized in that the geometric calibration step (103) comprises three main steps:

Retrieval of the projections of the 3D centers of the spheres;

this step comprises a series of processing operations of the input images leading to the determination of the center of the (assumed) circular imprints of the projections of the spheres;

Metric reconstruction of the scene;

this step enables an initial reconstruction of the scene (3D points) and of the cameras (internal and external parameters). It requires to determine the elliptical paths of each center of a sphere;

Adjustment of bundles;

this step enables to refine the parameters found in the preceding step, both internal and external parameters of the cameras and of the positions of the 3D centers of the spheres.

19. Method for digitizing according to claim 5, characterized in that the geometric calibration step (103) comprises three main steps:

Retrieval of the projections of the 3D centers of the spheres;

this step comprises a series of processing operations of the input images leading to the determination of the center of the (assumed) circular imprints of the projections of the spheres;

Metric reconstruction of the scene;

this step enables an initial reconstruction of the scene (3D points) and of the cameras (internal and external parameters). It requires to determine the elliptical paths of each center of a sphere;

Adjustment of bundles;

this step enables to refine the parameters found in the preceding step, both internal and external parameters of the cameras and of the positions of the 3D centers of the spheres.

20. Method for digitizing according to claim 4, characterized in that the geometric calibration step (103) comprises:

A preliminary step consisting in determining the distribution of colors in each sphere in an adapted color space, step in which, considering an image comprising the set of spheres and a selections of areas completely covered by the image of each sphere, the characteristics of this distribution of color are determined;

A step of retrieval of the spheres during which the following steps are repeated for each image and for each sphere;

i—

Classification of the pixels in two classes sphere/background according to the probability of being in the image of the considered sphere,

ii—

Retrieval of the related components in the mask of the sphere class,

iii—

Rejection of the unsatisfactory related components according to their geometric characteristic;

elongation value, area,

iv—

On each component that has not been rejected in step iii, retrieval of the contour points in the original image,

v—

Robust and accurate matching of a circle on the contour points,

vi—

Retrieval of the center of the circle,

A step of determining the presence or absence of the considered sphere in the image

and, in case the sphere is actually visible in its entirety in the image, obtaining the center of its projection.