Methods and Apparatus for Rendering Output Images with Simulated Artistic Effects from Focused Plenoptic Camera Data

• US 20130128069A1
• Filed: 11/30/2010
• Published: 05/23/2013
• Est. Priority Date: 08/27/2010
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Georgiev, Todor G., Chunev, Georgi N.
• Granted Patent: US 8,665,341 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

Methods, apparatus, and computer-readable storage media for simulating artistic effects in images rendered from plenoptic data. An impressionistic-style artistic effect may be generated in output images of a rendering process by an “impressionist” 4D filter applied to the microimages in a flat captured with focused plenoptic camera technology. Individual pixels are randomly selected from blocks of pixels in the microimages, and only the randomly selected pixels are used to render an output image. The randomly selected pixels are rendered to generate the artistic effect, such as an “impressionistic” effect, in the output image. A rendering technique is applied that samples pixel values from microimages using a thin sampling kernel, for example a thin Gaussian kernel, so that pixel values are sampled only from one or a few of the microimages.

Claims

1. A method, comprising:

obtaining an input image comprising a plurality of separate microimages of an image of a scene, each of the microimages having a plurality of blocks of pixels;

for each of the plurality of blocks of pixels, applying a random selection technique to randomly select one pixel from the block; and

applying a rendering technique that uses the randomly selected pixels to render an output image of the scene from the plurality of separate microimages.

2. The method as recited in claim 1, wherein said applying a rendering technique that uses the randomly selected pixels to render an output image of the scene from the plurality of separate microimages comprises, for each point on an image plane of the output image:

determining a line of projection through the microimages in optical phase space according to a current point on the image plane and an angle of projection determined from a current depth of focus;

for each microimage intersected by the line of projection, applying a sampling kernel centered at an intersection point of the line of projection with the microimage to sample values from pixels of the microimage covered by the sampling kernel, wherein only the randomly selected pixels in the microimage are sampled by the sampling kernel; and

generating a value for a pixel at the current point in the output image from the values sampled from one or more of the randomly selected pixels by the sampling kernel.

3. The method as recited in claim 2, wherein, in said generating a value for a pixel at the current point in the output image from the values sampled from one or more of the randomly selected pixels by the sampling kernel, the values of the one or more randomly selected pixels sampled by the sampling kernel are weighted according to distance of the pixels from the intersection point on the respective microimage.

4. The method as recited in claim 2, wherein the sampling kernel is a thin Gaussian kernel.

5. The method as recited in claim 2, wherein, for a given point on the image plane of the output image, pixel values are not sampled from at least one of the microimages intersected by the respective line of projection when the sampling kernel does not cover at least one of the randomly selected pixels in said at least one of the microimages.

6. The method as recited in claim 2, wherein, for a given point on the image plane of the output image, the sampling kernel samples randomly selected pixels from at most two of the microimages intersected by the respective line of projection.

7. The method as recited in claim 1, wherein the blocks are m×

n blocks, where m and n are integers each greater than or equal to 2.

8. The method as recited in claim 7, where m and n are each equal to 3.

9. A system, comprising

at least one processor;

and

a memory comprising program instructions, wherein the program instructions are executable by the at least one processor to;

obtain an input image comprising a plurality of separate microimages of an image of a scene;

subdivide each of the plurality of separate microimages into a plurality of blocks of pixels;

for each of the plurality of blocks of pixels in each microimage, apply a random selection technique to randomly select one pixel in the block; and

apply a rendering technique that uses the randomly selected pixels to render an output image of the scene from the plurality of separate microimages.

10. The system as recited in claim 9, wherein, to apply a rendering technique that uses the randomly selected pixels to render an output image of the scene from the plurality of separate microimages includes to, for each point on an image plane of the output image:

determine a line of projection through the microimages in optical phase space according to a current point on the image plane and an angle of projection determined from a current depth of focus;

for each microimage intersected by the line of projection, apply a sampling kernel centered at an intersection point of the line of projection with the microimage to sample values from pixels of the microimage covered by the sampling kernel, wherein only the randomly selected pixels in the microimage are sampled by the sampling kernel; and

generate a value for a pixel at the current point in the output image from the values sampled from one or more of the randomly selected pixels by the sampling kernel.

11. The system as recited in claim 10, wherein, to generate a value for a pixel at the current point in the output image from the values sampled from one or more of the randomly selected pixels by the sampling kernel includes to weight the values of the one or more randomly selected pixels sampled by the sampling kernel according to distance of the pixels from the intersection point on the respective microimage.

12. The system as recited in claim 10, wherein the sampling kernel is a thin Gaussian kernel.

13. The system as recited in claim 10, wherein, for a given point on the image plane of the output image, pixel values are not sampled from at least one of the microimages intersected by the respective line of projection when the sampling kernel does not cover at least one of the randomly selected pixels in said at least one of the microimages.

14. The system as recited in claim 10, wherein, for a given point on the image plane of the output image, the sampling kernel samples randomly selected pixels from at most two of the microimages intersected by the respective line of projection.

15. A tangible computer-readable storage medium comprising stored program instructions, wherein the stored program instructions are computer-executable to implement a method comprising:

obtaining an input image comprising a plurality of separate microimages of an image of a scene, each of the microimages having a plurality of blocks of pixels;

for each of the plurality of blocks of pixels, applying a random selection technique to randomly select one pixel from the block; and

applying a rendering technique that uses the randomly selected pixels to render an output image of the scene from the plurality of separate microimages.

16. The tangible computer-readable storage medium as recited in claim 15, wherein, said applying a rendering technique that uses the randomly selected pixels to render an output image of the scene from the plurality of separate microimages includes, for each point on an image plane of the output image:

determining a line of projection through the microimages in optical phase space according to a current point on the image plane and an angle of projection determined from a current depth of focus;

for each microimage intersected by the line of projection, applying a sampling kernel centered at an intersection point of the line of projection with the microimage to sample values from pixels of the microimage covered by the sampling kernel, wherein only the randomly selected pixels in the microimage are sampled by the sampling kernel; and

generating a value for a pixel at the current point in the output image from the values sampled from one or more of the randomly selected pixels by the sampling kernel.

17. The tangible computer-readable storage medium as recited in claim 16, wherein, said generating a value for a pixel at the current point in the output image from the values sampled from one or more of the randomly selected pixels by the sampling kernel includes weighting the values of the one or more randomly selected pixels sampled by the sampling kernel according to distance of the pixels from the intersection point on the respective microimage.

18. The tangible computer-readable storage medium as recited in claim 16, wherein the sampling kernel is a thin Gaussian kernel.

19. The tangible computer-readable storage medium as recited in claim 16, wherein pixel values are not sampled from at least one of the microimages intersected by the respective line of projection when the sampling kernel does not cover at least one of the randomly selected pixels in said at least one of the microimages.

20. The tangible computer-readable storage medium as recited in claim 16, wherein, for a given point on the image plane of the output image, the sampling kernel samples randomly selected pixels from at most two of the microimages intersected by the respective line of projection.