Combined stereovision, color 3D digitizing and motion capture system

• US 6,664,531 B2
• Filed: 10/22/2002
• Issued: 12/16/2003
• Est. Priority Date: 04/25/2000
• Status: Active Grant

• Current Assignee: Creaform, Inc. (Ametek Incorporated)
• Original Assignee: InSpeck, Inc. (Ametek Incorporated)
• Inventors: Beauchamp, Dominique, Bourassa, Yvan, Breton, Martin, Gärtner, Hansjörg, Song, Li, Rouleau, Patryck, Lemelin, Guylain
• Primary Examiner(s): Le, Que T.
• Application Number:
  US10/277,424
• Publication Number:
  US 20030042401A1
• Time in Patent Office: 420 Days
• Field of Search: 250/208.1, 250/559.22, 250/559.4, 250/559.46, 250/216, 250/461.2, 348/208.12, 358/462, 382/131, 382/133, 382/154, 382/203, 382/204, 128/664, 128/665, 128/673
• US Class Current: 250/208.1
• CPC Class Codes: G01B 11/245 using a plurality of fixed, simultaneously operating transducers G01B11/2408 - G01B11/2425, G01B11/255 take precedence
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo

Abstract

A digitizer combining functions of stereovision, color 3D digitizing and motion capture of a target object. The digitizer has a base supporting two cameras and two projection arrangements with projectors, one of which is provided with a grating element for projection of an encoded pattern on a surface of the target object and used for an active 3D range sensing, the other being used for an acquisition of texture information of the target object. The cameras and the light projectors are angled with respect to each other and arranged so that they have optical axes converging through a single point. A computer operates the projectors and processes the video signals generated by the cameras according to the selected function.

Claims

1. A digitizer combining functions of stereovision, color 3D digitizing and motion capture of a target object, comprising:

a first camera;

a second camera;

a first projection arrangement having a first light projector providing lighting for an active 3D range sensing for each of the cameras, and a grating element in front of the first light projector for projection of an encoded pattern on a surface of the target object;

a second projection arrangement having a second light projector providing lighting for an acquisition of texture information of the target object;

a base onto which the first and second cameras and the first and second projection arrangements are mounted in fixed relative positions with respect to one another, the cameras having optical axes converging through a single point, one of the light projectors having an optical axis intersecting with the optical axes of the cameras at the single point; and

a communication port connected to the cameras and the light projectors, for reception of control signals setting operation of the cameras and the light projectors and transmission of video signals from the cameras.

2. The digitizer according to claim 1, wherein the first and second projection arrangements are aligned in angled directions with respect to each other so that projected light fields cover each other at a predetermined measurement distance from the projection arrangements, and the first and second cameras are aligned in angled directions with respect to each other so that the cameras have fields of view having significant overlapping portions over a depth of measurement around the measurement distance.

3. The digitizer according to claim 2, wherein:

the light projectors are white light projectors;

the cameras are color cameras; and

the encoded pattern comprises a repetition of a sequence of color lines, one of which being used as an identifiable reference having an absolute 3D position determinable in images captured by the cameras with the lighting provided by the first projection arrangement using experimentally predefined functions.

4. The digitizer according to claim 3, wherein the sequence of color lines comprises white, black, yellow, black, green, black, cyan and black, the black line closest to an optical axis of the first projection arrangement being replaced by a magenta reference line.

5. The digitizer according to claim 1, wherein the first projection arrangement has a projection lens optically coupled with the first light projector and having an optical axis substantially perpendicular to a surface of the encoded pattern.

6. The digitizer according to claim 2, wherein the other one of the first and second light projectors has an optical axis intersecting with the optical axes of the cameras at the single point, the projection arrangements and the cameras being positioned symmetrically with each other in a cross-like arrangement.

7. The digitizer according to claim 6, wherein the first projection arrangement has a projection lens optically coupled with the first light projector, the optical axis of the first camera having an angle with a projection axis of the projection lens, the angle being adjusted so that the overlapping portions equal or exceed 80% of the field of view of each camera.

8. The digitizer according to claim 1, wherein the cameras turn into action and the communication port transfers stereoscopic images captured by the cameras at video rate in response to control data in the control signals associated with the function of stereovision.

9. The digitizer according to claim 1, wherein the first light projector turns on and the communication port transfers images captured by the cameras in response to control data in the control signals associated with the function of color 3D digitizing.

10. The digitizer according to claim 1, wherein the communication port transfers images captured by the cameras in response to control data in the control signals associated with the function of motion capture.

11. The digitizer according to claim 1, further comprising lamp driver boards connected to and controlling respective ones of the light projectors, and a main control board coupled between the communication port and the lamp driver boards.

12. The digitizer according to claim 1, further comprising a casing enclosing the cameras and the projection arrangements, the base being an integral portion of the casing.

13. A digitizing system comprising:

a digitizer combining functions of stereovision, color 3D digitizing and motion capture of a target object, comprising;

a first camera;

a second camera;

a first projection arrangement having a first light projector providing lighting for an active 3D range sensing for each of the cameras, and a grating element in front of the first light projector for projection of an encoded pattern on a surface of the target object;

a second projection arrangement having a second light projector providing lighting for an acquisition of texture information of the target object;

a base onto which the first and second cameras and the first and second projection arrangements are mounted in fixed relative positions with respect to one another, the cameras having optical axes converging through a single point, one of the light projectors having an optical axis intersecting with the optical axes of the cameras at the single point; and

a communication port connected to the cameras and the light projectors, for reception of control signals setting operation of the cameras and the light projectors and transmission of video signals from the cameras; and

a computer having a port connectable with the communication port of the digitizer, functions controlling operation of the digitizer by generating the control signals for the stereovision, color 3D digitizing and motion capture, and functions for a processing of the video signals received through the port and generation of digitized data as a result of the processing.

14. The digitizing system according to claim 13, wherein the functions for the color 3D digitizing comprise a modelling process causing the computer to successively:

acquire basic images in the video signals from the cameras without illumination of the object from the projectors;

acquire structured images in the video signals from the cameras with illumination of the object from the first projector;

acquire texture images in the video signals from the cameras with illumination of the object from the second projector;

identify elements of the encoded pattern in the structured images;

determine a position of the elements to produce a set of measured points;

determine 3D coordinates of the measured points using calibration information stored in the computer; and

determine coordinates corresponding to each measured point in the texture images to produce a digitized image of the object, the digitized data comprising the digitized image of the object.

15. The digitizing system according to claim 14, wherein:

the elements are identified by detecting projected fringes in the structured images, the projected fringes forming the elements of the encoded pattern identified in the structured images, the projected fringes being segmented;

the position of the elements is determined by determining a relative position of the fringes in lines of the structured images by finding a maximum intensity or calculating a center of gravity of an intensity distribution of the fringes, the position of the fringes forming the measured points; and

the 3D coordinates are determined by locating a reference fringe in the structured images based on a distinctive one of features thereof, numbering the fringes according to the feature thereof beginning from the reference fringe to edges of the structured images, converting the relative position of the fringes into real world coordinates in real world units with respect to the reference fringe, and converting the position in real world coordinates into the 3D coordinates in geometric units using a calibration table created in a separate calibration operation for each one of the cameras, the calibration tables forming the calibration information.

16. The digitizing system according to claim 15, wherein:

the distinctive one of the features is a color, the fringes being numbered according to the color thereof;

the converting uses a function;

17. The digitizing system according to claim 16, wherein the computer further determines for each point of the point cloud a ratio of each red, green and blue color channels to a maximum of the channels, and applies an inverse of the ratios to corresponding points in the structured image.

18. The digitizing system according to claim 16, wherein the modelling process is performed separately on the images from the first and the second cameras respectively to produce two independent 3D models which are later merged together.

19. The digitizing system according to claim 13, further comprising a rotational table controlled by the computer for rotation of one of the digitizer and the object for creation of a textured 3D model of the object, the digitized data comprising the textured 3D model of the object.

20. The digitizing system according to claim 13, wherein the functions for the stereovision cause the computer to calculate a stereo pair of the images for display with a 3D impression of a scene, the digitized data comprising the stereo pair of the images.

21. The digitizing system according to claim 13, wherein the functions for the motion capture cause the computer to:

acquire in parallel sequences of images in the video signals from the first and second cameras;

detect control points in a first image of each sequence;

track the control points in the sequences of images;

determine disparities between the control points in the images from the first camera and the images from the second camera;

determine 3D positions of the control points in corresponding ones of the images from the first and second cameras taken at a same time by using the disparities and calibration information comprising relative position and angular alignment of the cameras; and

generate trajectories of the control points as sequences of the 3D positions of the control points respectively, the digitized data comprising the 3D positions and the trajectories of the control points.

22. The digitizing system according to claim 21, wherein the control points comprise specific markers positioned on the object.

23. The digitizing system according to claim 21, wherein the control points comprise points in the images of the object with high derivatives in intensity.

24. The digitizing system according to claim 13, wherein the computer further has compressing and decompressing functions applicable on the digitized data.

25. The digitizing system according to claim 13, wherein the computer further has functions for transmitting the digitized data through a communication port of the computer.

26. A method for digitizing a target object, comprising steps of:

capturing basic images of the object with first and second cameras without additional illumination of the object, the cameras having optical axes converging through a single point and being aligned in angled directions with respect to each other so that the cameras have fields of view having significant overlapping portions over a depth of measurement including the single point;

illuminating the object with light in which an encoded pattern is projected;

capturing structured images with the cameras;

illuminating the object with light deprived of a pattern;

capturing texture images with the cameras;

identifying elements of the encoded pattern in the structured images;

determining a position of the elements to produce a set of measured points;

determining 3D coordinates of the measured points using calibration information in respect with position and alignment of the cameras;

determining coordinates corresponding to each measured point in the texture images to produce a digitized image of the object.

27. The method according to claim 26, wherein:

the elements are identified by detecting projected fringes in the structured images, the projected fringes forming the elements of the encoded pattern identified in the structured images, the projected fringes being segmented;

the position of the elements is determined by determining a relative position of the fringes in lines of the structured images by finding a maximum intensity or calculating a center of gravity of an intensity distribution of the fringes, the position of the fringes forming the measured points; and

the 3D coordinates are determined by locating a reference fringe in the structured images based on a distinctive one of features thereof, numbering the fringes according to the feature thereof beginning from the reference fringe to edges of the structured images, converting the relative position of the fringes into real world coordinates in real world units with respect to the reference fringe, and converting the position in real world coordinates into the 3D coordinates in geometric units using a calibration table created in a separate calibration operation for each one of the cameras, the calibration tables forming the calibration information.

28. The method according to claim 27, wherein:

the distinctive one of the features is a color, the fringes being numbered according to the color thereof;

the converting uses a function;

29. The method according to claim 28, further comprising steps of:

for each point of the point cloud, determining a ratio of each red, green and blue color channels to a maximum of the channels; and

applying an inverse of the ratios to corresponding points in the structured image.

30. A method for tracking motion of a target object, comprising steps of:

capturing in parallel sequences of images of the target object with first and second cameras having optical axes converging through a single point and being aligned in angled directions with respect to each other so that the cameras have fields of view having significant overlapping portions over a depth of measurement including the single point;

detecting control points in a first image of each sequence;

tracking the control points in the sequences of images;

determining disparities between the control points in the images from the first camera and the images from the second camera;

determining 3D positions of the control points in corresponding ones of the images taken at a same time by the first and second cameras by using the disparities and calibration information comprising relative position and angular alignment of the cameras; and

generating trajectories of the control points as sequences of the 3D positions of the control points respectively.

31. The method according to claim 30, wherein the control points comprise specific markers positioned on the object.

32. The method according to claim 31, wherein the control points comprise points in the images of the object with high derivatives in intensity.