IMAGE PROCESSING DEVICE FOR GENERATING RECONSTRUCTION IMAGE, IMAGE GENERATING METHOD, AND STORAGE MEDIUM

• US 20130162861A1
• Filed: 12/20/2012
• Published: 06/27/2013
• Est. Priority Date: 12/27/2011
• Status: Active Grant

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: YAMAMOTO, Ryohei
• Granted Patent: US 8,941,750 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/30 Image reproducers optical systems for producing stereoscopic or other three-dimensional effects G02B30/00

Abstract

A sub-image extractor extracts a target sub-image from a light field image. A partial area definer defines a predetermined area in the target sub-image as a partial area. A pixel extractor extracts pixels from the partial area, the number of pixels meeting correspondence areas of a generation image. The pixel arranger arranges the extracted pixels to the correspondence areas of the generation image in an arrangement according to the optical path of the optical system which photographs the light field image. Pixels are extracted for all sub-images in the light field image, and are arranged to the generation image to generate a reconstruction image.

Claims

1. An image processing device comprising:

an image obtainer that obtains a multi-view image in which sub-images from viewpoints are aligned;

a clipper that clips partial images from the sub-images; and

a generator that generates a reconstruction image by arranging the partial images based on an alignment position of the sub-images corresponding to the partial images.

2. The image processing device according to claim 1, wherein

the generator inverts the partial images in point-symmetric manner respectively, and arranges the inverted partial images.

3. The image processing device according to claim 1, further comprising:

a distance setter that sets a re-focal length of the reconstruction image;

an interval determiner that determines an interval, at which the partial images are arranged on the reconstruction image, according to the re-focal length; and

a first size determiner that determines a clipping size of the partial images so that the partial images are overlapped on the reconstruction image.

4. The image processing device according to claim 3, further comprising:

a blurring degree determiner that determines a blurring degree,

wherein the first size determiner determines a larger clipping size as the blurring degree is larger.

5. The image processing device according to claim 1, further comprising:

a second size determiner that determines a clipping size of the partial images; and

a position determiner that determines clipping positions of the partial images,

wherein

the generator combines partial images corresponding to the clipping positions based on the clipping size and the alignment position of the sub-images, to generate the reconstruction image.

6. The image processing device according to claim 5, further comprising:

a distance setter that sets a re-focal length of the reconstruction image,

wherein the second size determiner determining the clipping size according to the re-focal length.

7. The image processing device according to claim 6, wherein

the second size determiner determines a smaller clipping size as the re-focal length is larger.

8. The image processing device according to claim 5, wherein

the position determiner determines the clipping positions according to a shape of diaphragm by which the multi-view image is photographed.

9. The image processing device according to claim 5, wherein

the generator includes:

an intermediate image generator that generates an intermediate image each time the position determiner determines the clipping positions, by arranging the partial images on the intermediate image based on the alignment position of the sub-images and the clipping positions currently determined by the position determiner; and

a combiner that combines each of the generated intermediate images to generate the reconstruction image,

wherein the image processing device further comprises a display that displays the reconstruction image in which the intermediate images are combined each time the position determiner determines the clipping positions.

10. The image processing device according to claim 1, further comprising:

an imager that is provided with an optical system which includes a main lens, imaging element, and a micro lens array in which respective micro lenses cover pixels of the imaging element,

wherein the image obtainer obtains the image imaged by the imager.

11. The image processing device according to claim 1, further comprising:

an image displacement obtainer that obtains image displacement degrees for the sub-images respectively, wherein each of the image displacement degrees indicating a displacement between a position of a predetermined part in the sub-image and a position of a part correspond to a photographic object captured in the predetermined part in another sub-image; and

a third size determiner that determines a larger clipping size for the partial images with a larger image displacement degree.

12. The image processing device according to claim 11, wherein

the image displacement obtainer calculates an image displacement degree for a sub-image selected as a calculation object from the sub-images, and estimates the image displacement degree for a sub-image other than the calculation object based on the calculated image displacement degree.

13. The image processing device according to claim 12, wherein

the image obtainer sequentially obtains multi-view images,

the image displacement obtainer selects the sub-images for the calculation object in cyclic manner, for each of the multi-view images sequentially obtained, to obtain the image displacement degrees for the sub-images, and

the generator sequentially generates the reconstruction image from the sequentially obtained multi-view images using the sequentially obtained image displacement degrees.

14. The image processing device according to claim 1, further comprising:

a fourth size determiner that determines a clipping size of the partial images to fit arrangement areas which is obtained by dividing the reconstruction image according to the sub-images.

15. The image processing device according to claim 1, further comprising:

a fifth size determiner that determines a clipping size of the partial images,

wherein the generator (a) divides the reconstruction image into arrangement areas corresponding to the sub-images, and (b) reduces or expands the partial images to fit a size of corresponding arrangement area, and (c) arranges the partial images.

16. An image processing device comprising:

an image obtainer that obtains a multi-view image in which sub-images from viewpoints are aligned;

a first selector that selects a first image from the sub-images;

a pixel selector that selects a first pixel included in the first image;

a second selector that selects a second image from the sub-images;

a searcher that searches a second pixel corresponding to the first pixel among pixels included in the second image;

a pixel processor that processes the first pixel and the second pixel based on a position relationship between the first pixel and the second pixel; and

a reconstructor that reconstructs an image from the sub-images processed by the pixel processor.

17. The image processing device according to claim 16, wherein

the reconstructor includes

a clipper that clips partial images from the sub-images;

and

a generator that arranges the partial images based on an alignment position of the sub-images corresponding to the partial images to generate the reconstruction image.

18. An image generating method comprising steps of:

obtaining a multi-view image in which sub-images from viewpoints are aligned;

clipping partial images from the sub-images; and

arranging the partial images based on an alignment position of the sub-images corresponding to the partial images to generate the reconstruction image.

19. A non-transitory computer-readable storage medium storing a computer-executable program, the program causing a computer to execute:

obtaining a multi-view image in which sub-images from viewpoints are aligned;

clipping partial images from the sub-images; and

arranging the partial images based on an alignment position of the sub-images corresponding to the partial images to generate the reconstruction image.