Systems and methods for 2D image and spatial data capture for 3D stereo imaging

• US 20110222757A1
• Filed: 03/07/2011
• Published: 09/15/2011
• Est. Priority Date: 03/10/2010
• Status: Abandoned Application

• Current Assignee: Shapequest, Inc.
• Original Assignee: GBO 3D Technology Pte., Ltd.
• Inventors: Yeatman, Hoyt H. JR., Robertson, Gary
• Application Number:
  US12/932,789
• Publication Number:
  US 20110222757A1
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/00 Stereoscopic video systems; Multi-view video systems; Details thereof
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 13/327 Calibration thereof

Abstract

Systems and methods for 2D image and spatial data capture for 3D stereo imaging are disclosed. The system utilizes a cinematography camera and at least one reference or “witness” camera spaced apart from the cinematography camera at a distance much greater that the interocular separation to capture 2D images over an overlapping volume associated with a scene having one or more objects. The captured image date is post-processed to create a depth map, and a point cloud is created form the depth map. The robustness of the depth map and the point cloud allows for dual virtual cameras to be placed substantially arbitrarily in the resulting virtual 3D space, which greatly simplifies the addition of computer-generated graphics, animation and other special effects in cinemagraphic post-processing.

Claims

1. A method of converting two-dimensional (2D) images of a scene having therein at least one object to one or more three-dimensional (3D) images of the scene, comprising:

simultaneously capturing at least first and second 2D images of the scene from corresponding at least first and second cameras having respective camera positions and orientations measured relative to a reference coordinate system;

forming a disparity map from the at least first and second 2D images, wherein the disparity map has a gray scale that corresponds to distance information of the at least one object relative to the reference coordinate system; and

forming from the disparity map a 3D point cloud P(x,y,z) representative of the at least one object, wherein the point cloud is configured to support first and second virtual cameras to create a stereo camera pair arrangeable in substantially arbitrary virtual locations.

2. The method of claim 1, further comprising forming the disparity map using photogrammetric triangulation operation on the at least first and second 2D images.

3. The method of claim 1, further comprising:

defining the first and second virtual cameras for the 3D point cloud to define a stereo camera pair.

4. The method of claim 1, further comprising:

defining one or more regions of interest in one of the at least first and second 2D images;

forming the disparity map for the one or more regions of interest;

transforming the one or more regions of interest to conform to the other of the least first and second 2D images; and

forming the point cloud to include the one or more regions of interest.

5. The method of claim 1, further comprising:

calibrating the at least first and second cameras for one or more camera parameters.

6. The method of claim 5, further comprising:

performing a registration operation for the at least first and second cameras relative to a reference coordinate system.

7. The method of claim 3, further comprising:

combining the point cloud with at least one visual effect.

8. The method of claim 7, further comprising forming the at least one visual effect as:

a) one or more computer-generated objects;

c) one or more animated objects; and

b) one or more computer-generated environments.

9. The method of claim 7, further comprising:

forming a 3D stereo movie from the combined point cloud and the at least-one visual effect.

10. The method of claim 7, wherein the 3D stereo movie includes a plurality of 3D stereo frames, and further comprising providing different positions for the first and second virtual cameras for at least two of the frames.

11. The method of claim 7, further comprising defining for the first and second virtual camera an interocular distance and convergence values for the scene, thereby creating a true left and a true right eye for the stereo pair for theatrical viewing of the 3D movie.

12. The method of claim 1, further comprising:

supporting the at least first and second cameras on a camera support device.

13. The method of claim 1, further comprising:

collecting an amount of image data in respective removable memories respectively operably arranged in the at least first and second cameras; and

downloading the image data from the removable memories to a memory module.

14. The method of claim 1, further comprising employing three cameras, and using one of the three cameras as a cinemagraphic camera and the other two cameras for capturing spatial date of the scene.

15. A method of forming a distance representation of a scene from two-dimensional (2D) images of the scene, comprising:

simultaneously capturing at least first and second 2D images of the scene from corresponding at least first and second cameras having respective camera positions and orientations measured relative to a reference coordinate system;

defining one or more regions of interest in the at least first and second 2D images;

associating differences between pixels in the at least first and second cameras with distances from a reference point; and

assigning different gray-scale intensities to different ones of the distances.

16. The method of claim 15, further comprising transforming the at least one or more regions of interest from the at least first and second 2D images so that like regions of interest in the different images match despite being taken with different cameras at different orientations.

17. The method of claim 16, further comprising:

forming from the disparity map a 3D point cloud P (x,y,z) representing spatial positions of points in three-space that make up the 3D scene.

18. The method of claim 17, further comprising defining a stereo pair of first and second virtual cameras.

19. The method of claim 18, further comprising forming a 3D stereo movie from the 3D point cloud and the stereo pair of first and second virtual cameras.

20. The method of claim 15, further comprising forming from the disparity map a polygonal mesh PM(x,y,z;

RGB) of spatial positions and corresponding RGB color.

21. The method of claim 20, further comprising:

mapping 2D photographed images on to the polygonal mesh to form a textured 3D geometry;

defining first and second virtual cameras for the polygonal mesh; and

re-imaging the textured 3D geometry with the first and second virtual cameras to form a 3D movie, including setting an interocular distance and convergence values for the scene, thereby creating a true left and a true right eye stereo pair for theatrical viewing of the 3D movie.