Image processing device, image processing method, and program processing image which is developed as a panorama

• US 10,559,068 B2
• Filed: 03/14/2018
• Issued: 02/11/2020
• Est. Priority Date: 09/29/2015
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Kishine, Yasunobu
• Primary Examiner(s): Elahi, Shan E
• Application Number:
  US15/920,755
• Publication Number:
  US 20180204311A1
• Time in Patent Office: 699 Days
• Field of Search: 348 36
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 3/08 Projecting images onto non-planar surfaces, e.g. geodetic screens
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 25/615 involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]

Abstract

The present invention provides an image processing device, an image processing method, and a program capable of suppressing the calculation load of processing of enhancing mutually different resolutions for each of a sagittal direction and a tangential direction of a lens. In the aspect of the present invention, the image processing device 35 includes: an image acquisition section 40 that acquires a first image which is acquired from an imaging element by imaging a subject image using an optical system; a second image generation section 41 that generates a second image which is developed as a panorama by performing polar coordinate transformation on the first image acquired by the image acquisition section; and a resolution enhancement processing section 42 that performs resolution enhancement processing on the second image asymmetrically in a horizontal direction and a vertical direction of the second image.

Claims

1. An image processing device comprising:

an image acquisition section that acquires a first image which is acquired from an imaging element by imaging a subject image using an optical system;

a second image generation section that generates a second image which is developed as a panorama by performing polar coordinate transformation on the first image acquired by the image acquisition section, wherein the sagittal direction and the tangential direction of the first image correspond to the horizontal direction and the vertical direction of the second image such that the tangential direction and the sagittal direction of the first image and the vertical direction and the horizontal direction of the second image are aligned; and

a resolution enhancement processing section that performs resolution enhancement processing on the second image asymmetrically in the horizontal direction and the vertical direction of the second image,

wherein the resolution enhancement processing section performs the resolution enhancement processing on only a partial area of the second image in the vertical direction.

4. The image processing device according to claim 1, wherein the resolution enhancement processing, which is performed by the resolution enhancement processing section, is recovery processing using a recovery filter corresponding to the second image on the basis of a point spread function of the optical system.

5. The image processing device according to claim 1, wherein the resolution enhancement processing, which is performed by the resolution enhancement processing section, is outline enhancement processing using an outline enhancement filter.

6. The image processing device according to claim 1, wherein a lens composing the optical system is a fish-eye lens.

7. The image processing device according to claim 6, further comprising

a distortion correction section that performs distortion correction on the second image,

wherein the resolution enhancement processing section performs resolution enhancement processing on the second image, which is subjected to the distortion correction, asymmetrically in the horizontal direction and the vertical direction of the second image.

8. The image processing device according to claim 1, further comprising a third image generation section that performs reverse panorama development, which is reverse processing of the panorama development performed by the second image generation section, on the second image so as to generate a third image.

2. An image processing device comprising:

an image acquisition section that acquires a first image which is acquired from an imaging element by imaging a subject image using an optical system;

a second image generation section that generates a second image which is developed as a panorama by performing polar coordinate transformation on the first image acquired by the image acquisition section, wherein the sagittal direction and the tangential direction of the first image correspond to the horizontal direction and the vertical direction of the second image such that the tangential direction and the sagittal direction of the first image and the vertical direction and the horizontal direction of the second image are aligned; and

a resolution enhancement processing section that performs resolution enhancement processing on the second image asymmetrically in the horizontal direction and the vertical direction of the second image,

wherein the resolution enhancement processing section changes a level of enhancement of the resolution enhancement processing in accordance with a position of the second image in the vertical direction.

3. The image processing device according to claim 2, wherein the resolution enhancement processing section reduces the level of enhancement of the resolution enhancement processing in accordance with the position of the second image in the vertical direction.

9. An image processing method comprising:

an image acquisition step of acquiring a first image which is acquired from an imaging element by imaging a subject image using an optical system;

a second image generation step of generating a second image which is developed as a panorama by performing polar coordinate transformation on the first image acquired by the image acquisition step, wherein the sagittal direction and the tangential direction of the first image correspond to the horizontal direction and the vertical direction of the second image such that the tangential direction and the sagittal direction of the first image and the vertical direction and the horizontal direction of the second image are aligned; and

a resolution enhancement processing step of performing resolution enhancement processing on the second image asymmetrically in the horizontal direction and the vertical direction of the second image,

wherein the resolution enhancement processing step performs the resolution enhancement processing on only a partial area of the second image in the vertical direction.

10. A non-transitory computer-readable tangible medium recording a program causing a computer to execute the image processing method according to claim 9.

11. An image processing method comprising:

an image acquisition step of acquiring a first image which is acquired from an imaging element by imaging a subject image using an optical system;

a second image generation step of generating a second image which is developed as a panorama by performing polar coordinate transformation on the first image acquired by the image acquisition step, wherein the sagittal direction and the tangential direction of the first image correspond to the horizontal direction and the vertical direction of the second image such that the tangential direction and the sagittal direction of the first image and the vertical direction and the horizontal direction of the second image are aligned; and

a resolution enhancement processing step of performing resolution enhancement processing on the second image asymmetrically in the horizontal direction and the vertical direction of the second image,

wherein the resolution enhancement processing step changes a level of enhancement of the resolution enhancement processing in accordance with a position of the second image in the vertical direction.

12. A non-transitory computer-readable tangible medium recording a program causing a computer to execute the image processing method according to claim 11.