Methods, systems, and computer-readable storage media for creating three-dimensional (3D) images of a scene

• US 8,508,580 B2
• Filed: 07/23/2010
• Issued: 08/13/2013
• Est. Priority Date: 07/31/2009
• Status: Active Grant

• Current Assignee: 3DMedia Corporation
• Original Assignee: 3DMedia Corporation
• Inventors: McNamer, Michael, Robers, Marshall, Markas, Tassos, Hurst, Jason Paul
• Primary Examiner(s): Rao, Andy
• Application Number:
  US12/842,084
• Publication Number:
  US 20110025825A1
• Time in Patent Office: 1,117 Days
• Field of Search: 348 42- 60, 348/E13.009, 348/E13.074, 382/154, 345/419
• US Class Current: 348/43
• CPC Class Codes: G03B 35/06 with axial movement of lens or gate between exposures
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

Methods, systems, and computer program products for generating three-dimensional images of a scene are disclosed herein. According to one aspect, a method includes receiving a plurality of images of a scene. The method also includes determining attributes of the plurality of images. Further, the method includes determining, based on the attributes, a pair of images from among the plurality of images for use in creating a three-dimensional image. A user can, by use of the subject matter disclosed herein, use an image capture device for capturing a plurality of different images of the same scene and for converting the images into a three-dimensional, or stereoscopic, image of the scene. The subject matter disclosed herein includes a three-dimensional conversion process. The conversion process can include identification of suitable pairs of images, registration, rectification, color correction, transformation, depth adjustment, and motion detection and removal.

Claims

1. A method for creating a three-dimensional image of a scene, the method comprising:

using at least one processor for;

receiving a plurality of captured images;

determining matching image features among the plurality of captured images;

determining a transformation to align at least one of the matched image features;

applying the transformation to a first captured image among the plurality of captured images such that the first captured image is aligned to a second captured image among the plurality of captured images to create a plurality of transformed images;

defining a transformed image pair using one of transformed and captured images;

calculating parallax values of the matched image features of the transformed image pair;

determining whether the parallax values do not meet a predetermined criteria;

in response to determining that the parallax values do not meet the predetermined criteria, altering a stereoscopic characteristic of the transformed image pair to create an adjusted image pair; and

defining a three-dimensional image based on the adjusted image pair.

2. The method of claim 1, wherein the plurality of captured images are captured from the same image capture device.

3. The method of claim 1, wherein the plurality of captured images are a first set of captured images, and

wherein the method further comprises:

comparing attributes of the first set of captured images;

determining a second set of captured images having attributes within a predetermined threshold; and

selecting images from the second set of captured images for creating a three-dimensional image.

4. The method of claim 3, wherein comparing attributes of the first set of captured images comprises comparing color attributes.

5. The method of claim 3, wherein comparing attributes of the first set of captured images comprises comparing extracted features of the images.

6. The method of claim 3, wherein comparing attributes of the first set of captured images comprises identifying matching edges of the images, and

wherein determining a second set of captured images comprises determining the second set of images having a number of matched edges that meets a predetermined threshold.

7. The method of claim 3, further comprising:

applying a Hough transform to identify lines within the images; and

determining perspective view changes between the images based on the identified lines.

8. The method of claim 1, further comprising determining whether images among the plurality of captured images are suitable for creating a stereoscopic pair by examining one of:

large occluded areas, large non-overlapping regions, and large vertical parallax.

9. The method of claim 1, further comprising:

comparing color attributes of the captured images;

determining pixels in the captured images having color attributes that are above a predetermined color threshold; and

modifying a plurality of pixels having color that is above a predetermined threshold to match the color of the corresponding pixels in another image.

10. The method of claim 1, further comprising changing a position of a screen plane of a transformed pair of images.

11. The method of claim 1, further comprising changing parallax of an object within at least one of the captured and transformed images.

12. The method of claim 1, further comprising enhancing a depth of a transformed pair of images.

13. The method of claim 1, further comprising automatically determining a left view image and a right view of an image pair for creating a stereoscopic pair.

14. The method of claim 1, wherein determining matching image features comprises:

performing horizontal and vertical edge detection;

filtering for strong edges of a minimum length; and

identifying crossing points of the strong edges.

15. The method of claim 1, further comprising utilizing motion estimation techniques to identify matching features between captured images.

16. The method of claim 1, further comprising utilizing epipoles along with an interest point set to generate a pair of rectifying homographies.

17. The method of claim 1, further comprising evaluating whether focal axes are parallel and proceeding with image registration if the angle of the axes is below a predetermined threshold.

18. The method of claim 1, further comprising:

determining a parallax disparity between a left view image and a right view image among the captured images;

determining whether the parallax disparity meets a predetermined criteria; and

in response to determining that the parallax disparity does not meet the predetermined criteria, adjusting an attribute of at least one pixel in one of a left view image and a right view image such that the parallax disparity meets predetermined criteria.

19. The method of claim 1, further comprising cropping one of the left view image and the right view of the transformed image pair.

20. The method of claim 1, further comprising:

determining a parallax disparity between at least a portion of an object in a left view image and a right view image among one of the captured and transformed images;

determining whether the parallax disparity is greater than a predetermined threshold level due to an object moving in the scene; and

in response to determining that the parallax disparity is greater than the predetermined threshold level, adjusting at least one of images in the pair.

21. The method of claim 20, further comprising creating a new image by performing one of:

moving one object to another position of the same image; and

removing one object from an image; and

adding an object from one image to the other image.

22. The method of claim 20, further comprising capturing multiple images of the same scene to remove objects from a scene.

23. The method of claim 20, further comprising capturing multiple images of the same scene by moving the camera laterally to remove objects from a scene.

24. The method of claim 20, further comprising:

identifying at least of one moving object within a stereoscopic pair;

identifying the same object in a second set of captured images;

copying the identified object to one of left or right images; and

adjusting disparities.

25. The method of claim 20, further comprising identifying the most suitable image from the set of captured pictures from which to extract the moving object.

26. The method of claim 20, further comprising identifying an object that exists in a first set of images but not on a second set of images and remove from the second set of images.

27. The method of claim 20, further comprising interpolating pixels on areas from where an object has been removed utilizing pixels from the other captured images.

28. The method of claim 1, further comprising calculating and saving disparity information of a plurality of matched image features.