Methods and Apparatus for Coherent Manipulation and Stylization of Stereoscopic Images

• US 20130136337A1
• Filed: 11/13/2012
• Published: 05/30/2013
• Est. Priority Date: 11/30/2011
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Asente, Paul, Northam, Lesley Ann
• Granted Patent: US 9,105,117 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2024 Style variation

Abstract

Methods and apparatus for coherent manipulation and stylization of stereoscopic images. A stereo image manipulation method may use the disparity map for a stereo image pair to divide the left and right images into a set of slices, each of which is the portion of the images that correspond to a certain, small depth range. The method may merge the left and right slices for a depth into a single image. The method may then apply a stylization technique to each slice. The method may then extract the left and right portions of each stylized slice, and stack them together to create a coherent stylized stereo image. As an alternative to first extracting slices from a merged image and then applying a stylization technique to the slices, the method may first apply the stylization technique to the merged image and then extract slices from the stylized merged image.

Claims

1. A method, comprising:

performing, by one or more computing devices;

obtaining a stereo image pair comprising a left image and a right image;

obtaining a disparity map for the stereo image pair;

for each of a plurality of depth ranges indicated by the disparity map, generating a merged image that includes image data from the left image and image data from the right image at the respective depth range; and

generating a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images.

2. The method as recited in claim 1, further comprising applying a stylization technique to at least one of the one or more merged images prior to said generating a new stereo image pair.

3. The method as recited in claim 2, wherein said applying a stylization technique to at least one of the one or more merged images comprises, for each of the at least one of the one or more merged images:

extracting a slice image from the respective merged image at a particular depth as indicated by the disparity map; and

applying the stylization technique to the slice image to generate a stylized slice image.

4. The method as recited in claim 2, wherein said applying a stylization technique to at least one of the one or more merged images comprises, for each of the at least one of the one or more merged images:

applying the stylization technique to the respective merged image to generate a stylized merged image; and

extracting a stylized slice image from the stylized merged image at a particular depth.

5. The method as recited in claim 2, further comprising applying a different stylization technique to at least one other of the one or more merged images prior to said generating a new stereo image pair.

6. The method as recited in claim 1, wherein said generating a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images comprises:

extracting a slice image from each of the one or more merged images at a particular depth as indicated by the disparity map;

extracting a leftmost portion from each of the one or more slice images and stacking the one or more leftmost portions to generate the new left image; and

extracting a rightmost portion from each of the one or more slice images and stacking the one or more rightmost portions to generate the new right image.

7. The method as recited in claim 1, further comprising:

extracting a slice image from each of the one or more merged images at a particular depth as indicated by the disparity map;

extracting a portion from each of the one or more slice images and stacking the one or more portions to generate an intermediate image between the left image and the right image.

8. The method as recited in claim 1, wherein the disparity map indicates a disparity for each pixel in the stereo image pair, wherein each disparity indicates an offset from a corresponding pixel in the left image or the right image, and wherein each disparity corresponds to a particular depth in the stereo image pair.

9. A system, comprising:

one or more processors; and

a memory comprising program instructions, wherein the program instructions are executable by the at least one of the one or more processors to;

obtain a stereo image pair comprising a left image and a right image;

obtain a disparity map for the stereo image pair;

for each of a plurality of depth ranges indicated by the disparity map, generate a merged image that includes image data from the left image and image data from the right image at the respective depth range; and

generate a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images.

10. The system as recited in claim 9, wherein the program instructions are further executable by the at least one of the one or more processors to apply a stylization technique to at least one of the one or more merged images prior to said generating a new stereo image pair.

11. The system as recited in claim 10, wherein, to apply a stylization technique to at least one of the one or more merged images, the program instructions are executable by the at least one of the one or more processors to, for each of the at least one of the one or more merged images:

extract a slice image from the respective merged image at a particular depth as indicated by the disparity map; and

apply the stylization technique to the slice image to generate a stylized slice image.

12. The system as recited in claim 10, wherein, to apply a stylization technique to at least one of the one or more merged images, the program instructions are executable by the at least one of the one or more processors to, for each of the at least one of the one or more merged images:

apply the stylization technique to the respective merged image to generate a stylized merged image; and

extract a stylized slice image from the stylized merged image at a particular depth.

13. The system as recited in claim 10, wherein the program instructions are executable by the at least one of the one or more processors to apply a different stylization technique to at least one other of the one or more merged images prior to said generating a new stereo image pair.

14. The system as recited in claim 9, wherein, to generate a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images, the program instructions are executable by the at least one of the one or more processors to:

extract a slice image from each of the one or more merged images at a particular depth as indicated by the disparity map;

extract a leftmost portion from each of the one or more slice images and stack the one or more leftmost portions to generate the new left image; and

extract a rightmost portion from each of the one or more slice images and stack the one or more rightmost portions to generate the new right image.

15. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions are computer-executable to implement:

obtaining a stereo image pair comprising a left image and a right image;

obtaining a disparity map for the stereo image pair;

for each of a plurality of depth ranges indicated by the disparity map, generating a merged image that includes image data from the left image and image data from the right image at the respective depth range; and

generating a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images.

16. The non-transitory computer-readable storage medium as recited in claim 15, wherein the program instructions are further computer-executable to implement applying a stylization technique to at least one of the one or more merged images prior to said generating a new stereo image pair.

17. The non-transitory computer-readable storage medium as recited in claim 16, wherein, in said applying a stylization technique to at least one of the one or more merged images, the program instructions are computer-executable to implement, for each of the at least one of the one or more merged images:

extracting a slice image from the respective merged image at a particular depth as indicated by the disparity map; and

applying the stylization technique to the slice image to generate a stylized slice image.

18. The non-transitory computer-readable storage medium as recited in claim 16, wherein, in said applying a stylization technique to at least one of the one or more merged images, the program instructions are computer-executable to implement, for each of the at least one of the one or more merged images:

applying the stylization technique to the respective merged image to generate a stylized merged image; and

extracting a stylized slice image from the stylized merged image at a particular depth.

19. The non-transitory computer-readable storage medium as recited in claim 16, wherein the program instructions are further computer-executable to implement applying a different stylization technique to at least one other of the one or more merged images prior to said generating a new stereo image pair.

20. The non-transitory computer-readable storage medium as recited in claim 15, wherein, in said generating a new stereo image pair comprising a new left image and a new right image from one or more of the plurality of merged images, the program instructions are computer-executable to implement:

extracting a slice image from each of the one or more merged images at a particular depth as indicated by the disparity map;

extracting a leftmost portion from each of the one or more slice images and stacking the one or more leftmost portions to generate the new left image; and

extracting a rightmost portion from each of the one or more slice images and stacking the one or more rightmost portions to generate the new right image.