System and method for real time 2D to 3D conversion of a video in a digital camera
First Claim
1. A processor based method, comprising:
- employing an image sensing device to receive a first set of two dimensional (2D) images;
employing one or more processors to perform actions, including;
generating a depth map for a scene, the scene comprising a plurality of pixels, each of the pixels corresponding to one of a plurality of depths in the depth map;
identifying a plurality of principal depths for the scene using the generated depth map, each of the principal depths having a number of corresponding pixels that is greater than a number of pixels corresponding to any non-principal depth of the depth map;
identifying a plurality of focus positions, the focus positions respectively corresponding to the principal depths;
capturing, using the image sensing device, a 2D image at each of the focus positions to form a second set of 2D images;
determining a depth correspondence between the second set of 2D images of the scene and the principal depths by associating each coordinate in the second set of 2D images with a principal depth that is closest to an actual depth of the coordinate; and
for each of a plurality of regions of interest in the scene, generating a three dimensional (3D) view including a right-eye image and a left-eye image by;
i) selecting a corresponding principal depth for a respective region of interest, and ii) performing a translation and rotation transformation mapping for each pixel from a 2D image captured at the focus position for the principal depth for the respective region of interest.
3 Assignments
0 Petitions
Accused Products
Abstract
Embodiments are directed towards enabling digital cameras to create a 3D view, which can be re-rendered onto any object within a scene, so that it is both in focus and a center of perspective, based on capturing a single set of multiple 2D images of the scene. From capturing a single set of 2D images for a scene, a depth map of the scene may be generated, and used to calculate principal depths, which are then used to capture an image focused at each of the principal depths. A correspondence between a 2D image of the scene and the principal depths are determined that corresponds to a specific principal depth. For different coordinates of the 2D image, different 3D views of the scene are created that are each focused at a principal dept that corresponds to the given coordinate.
-
Citations
21 Claims
-
1. A processor based method, comprising:
-
employing an image sensing device to receive a first set of two dimensional (2D) images; employing one or more processors to perform actions, including; generating a depth map for a scene, the scene comprising a plurality of pixels, each of the pixels corresponding to one of a plurality of depths in the depth map; identifying a plurality of principal depths for the scene using the generated depth map, each of the principal depths having a number of corresponding pixels that is greater than a number of pixels corresponding to any non-principal depth of the depth map; identifying a plurality of focus positions, the focus positions respectively corresponding to the principal depths; capturing, using the image sensing device, a 2D image at each of the focus positions to form a second set of 2D images; determining a depth correspondence between the second set of 2D images of the scene and the principal depths by associating each coordinate in the second set of 2D images with a principal depth that is closest to an actual depth of the coordinate; and for each of a plurality of regions of interest in the scene, generating a three dimensional (3D) view including a right-eye image and a left-eye image by;
i) selecting a corresponding principal depth for a respective region of interest, and ii) performing a translation and rotation transformation mapping for each pixel from a 2D image captured at the focus position for the principal depth for the respective region of interest. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. An image system, comprising:
-
an image sensing device configured to receive a first set of two dimensional (2D) images; and one or more circuits having a plurality of components thereon and configured to perform a plurality of actions, including; generating a depth map for a scene, the scene comprising a plurality of pixels, each of the pixels corresponding to one of a plurality of depths in the depth map; identifying a plurality of principal depths for the scene using the generated depth map, each of the principal depths having a number of corresponding pixels that is greater than a number of pixels corresponding to any non-principal depth of the depth map; identifying a plurality of focus positions, the focus positions respectively corresponding to the principal depths; capturing, using the image sensing device, a 2D image at each of the focus positions to form a second set of 2D images; determining a depth correspondence between the second set of 2D images of the scene and the principal depths by associating each coordinate in the second set of 2D images with a principal depth that is closest to an actual depth of the coordinate; and for each of a plurality of regions of interest in the scene, generating a three dimensional (3D) view by;
i) selecting a corresponding principal depth for a respective region of interest, and ii) performing a translation and rotation transformation mapping for each pixel from a 2D image captured at the focus position for the principal depth for the respective region of interest. - View Dependent Claims (10, 11, 12, 13, 14, 15)
-
-
16. A storage device having stored thereon a plurality of computer-executable instructions that when executed by a digital camera, perform a plurality of actions, comprising:
-
generating a depth map for a scene, the scene comprising a plurality of pixels, each of the pixels corresponding to one of a plurality of depths in the depth map; identifying a plurality of principal depths for the scene using the generated depth map, each of the principal depths having a number of corresponding pixels that is greater than a number of pixels corresponding to any non-principal depth of the depth map; identifying a plurality of focus positions, the focus positions respectively corresponding to the principal depths; capturing, using an image sensing device of the digital camera, a 2D image at each of the focus positions; determining a depth correspondence between the 2D images of the scene and the principal depths by associating each coordinate in the 2D images with a principal depth that is closest to an actual depth of the coordinate; and for each of a plurality of regions of interest in the scene, generating a three dimensional (3D) view by;
i) selecting a corresponding principal depth for a respective region of interest, and ii) performing a translation and rotation transformation mapping for each pixel from a 2D image captured at the focus position for the principal depth for the respective region of interest. - View Dependent Claims (17, 18, 19, 20, 21)
-
Specification