Generating 3D models by combining models from a video-based technique and data from a structured light technique

• US 6,529,627 B1
• Filed: 11/10/2000
• Issued: 03/04/2003
• Est. Priority Date: 06/24/1999
• Status: Expired due to Fees

• Current Assignee: Geometrix Inc. (Alive Tech Inc.)
• Original Assignee: Geometrix Inc. (Alive Tech Inc.)
• Inventors: Zwern, Arthur, Fejes, Sandor, Callari, Francesco
• Primary Examiner(s): Patel, Jayanti K.
• Assistant Examiner(s): Bayat, Ali
• Application Number:
  US09/710,438
• Time in Patent Office: 844 Days
• Field of Search: 382/154, 382/276, 382/285, 382/291, 382/293, 345/419, 345/426, 345/427, 345/653, 345/661, 345/664, 345/619, 345/676, 345/679, 356/3.1, 356/3.01
• US Class Current: 382/154
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings

Abstract

A 3D modeling system employing a commercially available structured light source is disclosed. The light source projecting a light stripe is swung manually across an object or a particular area of interest on the surface of the object. Under the illumination of the swinging light stripe, the object is imaged to produce a sequence of images. For each of the images, the position and orientation of the structured light source are independently and dynamically derived. From each of the images, a set of curvilinear points from the (deformed) projected line falling on the object or the area of interest are calculated in conjunction with a calibration model of the imager. The curvilinear points from all the images are merged to provide accurate and complete 3D data about the object or the area of interest.

Claims

1. A method for generating a 3D model of an object, the method comprising:

generating an initial 3D model of the object from a sequence of first images from an imager based on a calibration model of the imager, the object placed in reference to a platform, wherein the calibration model determines relationships between the imager and the platform;

obtaining, respectively, 3D data of certain areas of the object from a sequence of second images taken by the imager when a structured light beam is swung across respectively the certain areas of the object; and

deriving the 3D model by integrating the 3D data into the initial 3D model.

2. The method of claim 1, wherein each of the certain areas of the object has its own geometrical shape which, without being illuminated by the structured light beam, is unobservable or otherwise difficult to be imaged by the imager.

3. The method of claim 1 further comprising obtaining the calibration model of the imager before the first images are taken.

4. The method of claim 3, wherein the generating of the initial 3D model comprises:

receiving the sequence of first images from the imager, each of the first images representing a snapshot of the object from an angle; and

deriving the initial 3D model from the sequence of first images based on the calibration model.

5. The method of claim 4, wherein the initial 3D model includes inaccuracy for at least one of the certain areas of the object.

6. The method of claim 5, wherein the deriving of the 3D model comprises:

applying the 3D data to the at least one of the certain areas of the object so that the inaccuracy can be corrected.

7. The method of claim 4, wherein the obtaining of the 3D data comprises:

detecting, from the sequence of second images, respective deformations of projection of the structured light beam falling on each of the certain areas of the objects;

deriving 3D contours from the respective deformations; and

merging the 3D contours to generate the 3D data.

8. The method of claim 7, wherein the platform includes a pair of a first panel and a second panel, the first panel and the second panel are configured to a known angle and behind the object.

9. The method of claim 8, wherein the laser stripe is projected on the first panel and the second panel simultaneously at any given time.

10. The method of claim 7, wherein the platform includes a pair of a first panel and a second panel, the first panel and the second panel are so configured that each of the first images shows that the object is surrounded by the first panel and the second panel.

11. The method of claim 10, wherein the laser stripe is projected on the first panel and the second panel simultaneously at any given time.

12. The method of claim 1, wherein the structured light beam is a laser stripe projected from a laser projector.

13. The method of claim 12, wherein the laser projector is not positioned a priori in reference to the platform.

14. The method of claim 13, wherein the laser projector is a handheld type and manually operated.

15. The method of claim 1, wherein the initial 3D model includes inaccuracy for at least one of the certain areas of the object.

16. The method of claim 15, wherein the deriving of the 3D model comprises applying the 3D data to the at least one of the certain areas of the object so that the inaccuracy can be corrected.

17. A method for generating a 3D model of an object, the method comprising:

providing a modeling system including an imager and a platform, the imager and the platform maintaining relationships expressed In a calibration model of the imager;

taking a sequence of first images of the object by the imager to generate an initial 3D model of the object, wherein the object is placed in reference to the platform; and

taking a sequence of second images of the object by the imager while using a structured light beam to illuminate respectively certain areas of the object to obtain 3D data of the certain areas of the object, wherein the 3D model is derived by integrating the 3D data into the initial 3D model.

18. The method of claim 17, wherein each of the first images represents a snapshot of the object from an angle.

19. The method of claim 18, wherein the platform includes a turntable and the object is placed upon the turntable;

and wherein, when the turntable rotates, the imager takes the snapshot successively to produce the sequence of the images.

20. The method of claim 17, wherein each of the certain areas of the object, without being illuminated by the structured light beam, is unobservable or otherwise difficult to be imaged by the imager.

21. The method of claim 20, wherein the taking a sequence of second images of the object comprises:

swinging the structured light beam across respectively the certain areas of the object; and

imaging the object when each of the certain areas of the object is being illuminated by the structured light beam.

22. The method of claim 21, wherein the structured light beam is a laser strip projected from a laser projector.

23. The method of claim 22, wherein the laser projector is not positioned a priori in reference to the platform.

24. The method of claim 22, wherein the laser projector is a handheld type and manually operated.

25. The method of claim 21, wherein the platform includes a pair of a first panel and a second panel, the first panel and the second panel are configured to a known angel and behind the object.

26. The method of claim 25, wherein the swinging of the structured light beam is to project the structured light beam on the first panel and the second panel simultaneously at any given time.

27. The method of claim 21, wherein the platform includes a pair of a first panel and a second panel, the first panel and the second panel are so configured that each of the first images shows that the object is surrounded by the first panel and the second panel.

28. The method of claim 21, wherein the swinging of the structured light beam is to project the structured light beam on the first panel and the second panel simultaneously at any given time.

29. The method of claim 17, wherein the structured light mean is manually swung to illuminate across each of the certain areas of the object.

30. A machine-readable media for generating a 3D model of an object, the machine-readable media comprising:

program code for generating an initial 3D model of the object from a sequence of first images from an imager based on a calibration model of the imager, the object placed in reference to a platform, wherein the calibration model determines relationships between the imager and the platform;

program code for obtaining, respectively, 3D data of certain areas of the object from a sequence of second images taken by the imager when a structured light beam is swung across respectively the certain areas of the object; and

program code for deriving the 3D model by integrating the 3D data into the initial 3D model.

31. The machine-readable media of claim 30, wherein each of the certain areas of the object, without being illuminated by the structured light beam, is unobservable or otherwise difficult to be imaged by the imager.

32. The machine-readable media of claim 30 further comprising program code for obtaining the calibration model of the imager before the first images are taken.

33. The machine-readable media of claim 32, wherein the program code for generating the initial 3D model comprises:

program code for receiving the sequence of first images from the imager, each of the first images representing a snapshot of the object from an angle; and

program code for deriving the initial 3D model from the sequence of first images based on the calibration model.

34. The machine-readable media of claim 33, wherein the initial 3D model includes inaccuracy for at least one of the certain areas of the object.

35. The machine-readable media of claim 34, wherein the program code for deriving the 3D model comprises:

program code for applying the 3D data to the at least one of the certain areas of the object so that the inaccuracy can be corrected.

36. The machine-readable media of claim 35, wherein the program code for obtaining the 3D data comprises:

program code for detecting, from the sequence of second images, respective deformations of the structured light beam falling on each of the certain areas of the objects;

program code for deriving respective contours from the respective deformations; and

program code for merging the respective contours to generate the 3D data.

37. The machine-readable media of claim 36, wherein the platform includes a pair of a first panel and a second panel, the first panel and the second panel are configured to a known angel and behind the object.

38. The machine-readable media of claim 37, wherein the laser stripe is projected on the first panel and the second panel simultaneously at any given time.

39. The machine-readable media of claim 36, wherein the platform includes a pair of a first panel and a second panel, the first panel and the second panel are so configured that each of the first images shows that the object is surrounded by the first panel and the second panel.

40. The machine-readable media of claim 39, wherein the laser strip is projected on the first panel and the second panel simultaneously at any given time.

41. The machine-readable media of claim 30, wherein the structured light beam is a laser strip projected from a laser projector.

42. The machine-readable media of claim 41, wherein the laser projector is not positioned in reference to the platform.

43. The machine-readable media of claim 42, wherein the laser projector is a handheld type and manually operated.