System and method for three dimensional modeling

• US 20050128196A1
• Filed: 10/08/2004
• Published: 06/16/2005
• Est. Priority Date: 10/08/2003
• Status: Active Grant

• Current Assignee: Purdue Research Foundation (Purdue University)
• Original Assignee: Purdue Research Foundation (Purdue University)
• Inventors: Sacks, Elisha, Popescu, Voicu S.
• Granted Patent: US 7,747,067 B2
• Time in Patent Office: Days
• US Class Current: 345/420
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01S 17/48 Active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images

Abstract

An apparatus and method for creating a 3D model of a scene. A plurality of light beams are projected onto the scene and their imaged projections are analyzed to capture surface information of the scene at a plurality of dots where the light beams intersect the scene. Surface information is used to register consecutive video frames and to construct a 3D model of the scene.

Claims

1. A system for creating three-dimensional models of scenes, the system including:

a video camera configured to capture scene information and to transfer scene information to a computer;

a structured light frame coupled to the video camera, the structured light frame including a light projector configured to project a plurality of light beams onto the surface of the scene such that the projections of the light beams are visible in the scene;

a computer configured to receive scene information; and

software configured to process the scene information to create a three-dimensional model of the scene.

2. The system of claim 1, where the video camera is a digital camera.

3. The system of claim 1, where the video camera is an analog camera.

4. The system of claim 1, where the light projector includes at least eight light sources.

5. The system of claim 1, where the structured light projector includes at least sixteen light sources.

6. The system according to claim 1, where the plurality of light beams are projected by a plurality of light sources.

7. The system according to claim 6, where each of the light beams is generated by a separate light source.

8. The system according to claim 1, where the light projector comprises a light source and a diffraction grating configured to project a pluarality of light beams.

9. The system according to claim 8, where the diffraction grating projects a matrix pattern of light beams.

10. The system of claim 1, where the video camera is capable of continuous bi-directional electronic communication with the computer.

11. The system of claim 10, where the continuous bi-directional electronic communication is accomplished by use of a cable.

12. The system of claim 11, where the cable is an optical cable.

13. The system of claim 10, where the continuous bi-directional electronic communication is accomplished with wireless communication.

14. The system of claim 1, where scene information is transferred to the computer using recordable media.

15. The system of claim 1, where the computer is a wearable computer.

16. The system of claim 1, where the computer is integral to the video camera.

17. The system of claim 1, where the computer display is integral to the video camera.

18. A method of creating a three-dimensional model of a scene including the steps of:

projecting structured light beams onto the surface of a scene such that the projections of the structured light beams create visible dots in the scene;

capturing scene image information with a video camera;

transferring scene information from the video camera to a computer; and

processing scene information in the computer to create a three-dimensional model of the scene.

19. The method of claim 18, where the transferring step is continuous.

20. The method of claim 18, where the transferring step is interactive.

21. The method of claim 18, where the transferring step is discrete.

22. The method of claim 18, where the processing step includes the steps of:

finding dots in video frames,

calculating scene depth information at the dot locations;

obtaining scene color information at the dot locations;

registering pairs of continuous frames using scene depth and color information;

creating model depth images; and

creating a model of the scene by merging the model depth images.

23. A method of creating a three-dimensional model of a scene in real-time including the steps of:

projecting structured light beams onto the surface of a scene such that the projections of the structured light beams create visible dots in the scene;

capturing scene image information with a video camera;

transferring video frames from the video camera to a computer in real-time;

finding dots in the video frames;

calculating scene depth information for each frame at the dot locations;

obtaining scene color information for each frame at the dot locations;

registering pairs of continuous frames using scene depth and color information;

creating model depth images; and

creating an evolving model of the scene by merging the model depth images.

24. The method of claim 23 further including the steps of:

displaying an evolving model of the scene;

providing the user with modeling feedback; and

providing the user means for improving the model.

25. The method of claim 24 where the modeling feedback includes oversampling and undersampling signals.

26. The method of claim 24 where the modeling feedback includes loss of registration signals.

27. The method of claim 24 where the means for improving the model include instructions for repositioning the camera when registration has been lost such that the user may re-acquire registration.

28. The method of claim 24 where the means for improving the model include instructions for repositioning the camera when the camera is oversampling or undersampling.

29. The method of claim 24 where the means for improving the model include prompting and acceptance of input via an input device to establish desirable computing parameters.

30. A method of creating a depth enhanced panorama of a scene including the steps of:

projecting structured light beams onto the surface of a scene such that the projections of the structured light beams create visible dots in the scene;

capturing scene image information with a video camera by moving the camera about its center of projection without translating the camera;

transferring video frames from the video camera to a computer in real-time;

finding dots in the video frames;

calculating scene depth information for each frame at the dot locations;

obtaining scene color information for each frame at the dot locations;

registering pairs of continuous frames using scene color information; and

creating depth enhanced panoramas of the scene.

31. The method of claim 30 further including the steps of:

displaying an evolving depth enhanced panorama of the scene;

providing the user with modeling feedback; and

providing the user means for improving the model.

32. The method of claim 30 where the modeling feedback includes oversampling and undersampling signals.

33. The method of claim 30 where the modeling feedback includes loss of registration signals.

34. The method of claim 30 where the means for improving the model include instructions for repositioning the camera when registration has been lost such that the user may re-acquire registration.

35. The method of claim 30 where the means for improving the model include instructions for repositioning the camera when the camera is oversampling or undersampling.

36. The method of claim 30 where the means for improving the model include prompting and acceptance of input via an input device to establish desirable computing parameters.