IMAGING DEVICE, IMAGE DISPLAY METHOD, AND STORAGE MEDIUM FOR DISPLAYING RECONSTRUCTION IMAGE

• US 20130162779A1
• Filed: 12/21/2012
• Published: 06/27/2013
• Est. Priority Date: 12/27/2011
• Status: Active Grant

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: Yamamoto, Ryohei
• Granted Patent: US 9,749,601 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/30 Image reproducers optical systems for producing stereoscopic or other three-dimensional effects G02B30/00

Abstract

A sub-image extractor extracts a target sub-image from a light field image. A partial area definer defines a predetermined area in the target sub-image as a partial area. A pixel extractor extracts pixels from the partial area, the number of pixels meeting correspondence areas of a generation image. The pixel arranger arranges the extracted pixels to the correspondence areas of the generation image in an arrangement according to the optical path of the optical system which photographs the light field image. Pixels are extracted for all sub-images in the light field image, and are arranged to the generation image to generate a reconstruction image.

Claims

1. An imaging device comprising:

an imager that images a multi-view image in which sub-images from viewpoints are aligned;

a display that displays images;

a first generator that generates a first reconstruction image to be displayed prior to photographing by a first image generation process from the multi-view image;

a second generator that generates a second reconstruction image for display after photographing by a second image generation process from the multi-view image, wherein the second image generation process uses more calculations than the first image generation process to improve an image quality; and

a controller that controls the display to display the first reconstruction image prior to photographing, and to display the second reconstruction image after photographing.

2. The imaging device according to claim 1, further comprising:

a clipper that clips partial images from a predetermined range of the sub-images; and

an arranger that arranges the partial images in accordance with an alignment sequence of the sub-images corresponding to the partial images, wherein,

the first image generation process is a process which generates the first reconstruction image using the clipper and the arranger,

the second image generation process is a process which generates the second reconstruction image using the clipper and the arranger, and

the second image generation process is a process that includes an additional process for improving the image quality to the first image generation process, or includes an alternative process using more calculations.

3. The imaging device according to claim 2, further comprising:

an image displacement obtainer that obtains image displacement degrees for the sub-images respectively, wherein each of the image displacement degrees indicating a displacement between a position of a predetermined part in the sub-image and a position of a part correspond to a photographic object captured in the predetermined part in another sub-image.

4. The imaging device according to claim 3, wherein

the image displacement obtainer calculates an image displacement degree for a sub-image selected as a calculation object from the sub-images, and estimates the image displacement degree for a sub-image other than the calculation object based on the calculated image displacement degree.

5. The imaging device according to claim 4, wherein

the imager sequentially images multi-view images,

the image displacement obtainer selects the sub-images for the calculation object in cyclic manner, for each of the multi-view images sequentially imaged, to obtain the image displacement degrees for the sub-images, and

the first generator sequentially generates first reconstruction images from the multi-view images sequentially imaged by the imager, using the image displacement degrees obtained sequentially by the image displacement obtainer.

6. The imaging device according to claim 3, further comprising:

a size determiner that determines a clipping size of the partial images,

wherein,

in the first image generation process, the size determiner determines a same clipping size for the sub-images,

in the second image generation process,

the second generator causes the image displacement obtainer to obtain the image displacement degrees, and

the size determiner determines a larger clipping size for the sub-image with a larger image displacement degree.

7. The imaging device according to claim 3, further comprising

a blurring determiner that determines a blurring degree of pixels of the sub-images.

8. The imaging device according to claim 7, wherein

the blurring determiner obtains pixel displacement degrees for each of the pixels of the sub-images, wherein each of the pixel displacement degrees indicating a displacement between a position of the pixel in the sub-image and a position of photographic object corresponding to the pixel in another sub-image, and determines the blurring degree for each of the pixels based on the pixel displacement degrees of the pixel.

9. The imaging device according to claim 7, further comprising:

a size determiner that determines a clipping size of the partial images,

wherein, in the first image generation process, the size determiner determines a same clipping size for the sub-images,

in the second image generation process,

the second generator causes the blurring determiner to determine the blurring degree, performs a blurring process based on the blurring degree, and causes the image displacement obtainer to obtain the image displacement degrees, and

the size determiner determines a larger clipping size for the sub-image with a larger image displacement degree.

10. The imaging device according to claim 3, further comprising:

a size determiner that determines a clipping size of the partial images,

wherein, in the first image generation process,

the first generator causes the image displacement obtainer to obtain the image displacement degrees for the predetermined part, and

the size determiner determines a larger clipping size for the sub-image with a larger image displacement degree,

in the second image generation process,

the second generator causes the image displacement obtainer to obtain the image displacement degree for a part larger than the predetermined part of the first image generation process, and

the size determiner determines a larger clipping size for the sub-image with a larger image displacement degree.

11. The imaging device according to claim 7, further comprising:

a size determiner that determines a clipping size of the partial images,

wherein, in the first image generation process,

the first generator causes the image displacement obtainer to obtain the image displacement degrees, and

the size determiner determines a larger clipping size for the sub-image with a larger image displacement degree,

wherein, in the second image generation process,

the second generator causes the blurring determiner to determines the blurring degree, performs a blurring process based on the blurring degree, and causes the image displacement obtainer to obtain the image displacement degrees, and

the size determiner determines a larger clipping size for the sub-image with a larger image displacement degree.

12. The imaging device according to claim 1, further comprising:

a third generator that generates a third reconstruction image by a third image generation process, and

a memory that stores the third reconstruction image,

wherein the third image generation process uses more calculations than the second image generation process to improve an image quality.

13. An image display method comprising:

imaging a multi-view image in which sub-images from viewpoints are aligned;

generating a first reconstruction image to be displayed prior to photographing by a first image generation process from the multi-view image;

generating a second reconstruction image for display after photographing by a second image generation process from the multi-view image, wherein the second image generation process uses more calculations than the first image generation process to improve an image quality; and

displaying the first reconstruction image on a display device prior to photographing, and displaying the second reconstruction image on the display device after photographing.

14. A non-transitory computer readable storage medium storing a computer executable program, the program causing a computer to execute:

acquiring a multi-view image in which sub-images from viewpoints are aligned from imaging device;

generating a first reconstruction image to be displayed prior to photographing by a first image generation process from the multi-view image;

generating a second reconstruction image for display after photographing by a second image generation process from the multi-view image, wherein the second image generation process uses more calculations than the first image generation process to improve an image quality; and

displaying the first reconstruction image on a display device prior to photographing, and displaying the second reconstruction image on the display device after photographing.