IMAGE PROCESSING APPARATUS AND IMAGE PROCESSING METHOD

• US 20190312996A1
• Filed: 05/09/2017
• Published: 10/10/2019
• Est. Priority Date: 07/13/2016
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: ASAI, HARUKA, MATSUI, AKIRA, UNE, HIDEHO
• Granted Patent: US 10,750,084 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/663 for controlling interchangeable camera parts based on electronic image sensor signals
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 25/611 Correction of chromatic aberration

Abstract

[Object] To estimate lens information with no change in photographing conditions. [Solution] An image processing apparatus according to the present disclosure includes: a positional displacement estimation unit configured to estimate positional displacement between a plurality of input images obtained by photographing an identical photographic subject; an alignment processing unit configured to align positions of the plurality of input images on the basis of the estimated positional displacement; and a lens information estimation unit configured to estimate information of a lens used to photograph the input image depending on a model indicating a deterioration component of the lens on the basis of the aligned input images.

Claims

1. An image processing apparatus comprising:

a positional displacement estimation unit configured to estimate positional displacement between a plurality of input images obtained by photographing an identical photographic subject;

an alignment processing unit configured to align positions of the plurality of input images on a basis of the estimated positional displacement; and

a lens information estimation unit configured to estimate lens information of a lens used to photograph the input image depending on a deterioration model indicating a deterioration component of the lens on a basis of the aligned input images.

2. The image processing apparatus according to claim 1,

wherein the alignment processing unit includes

a first alignment unit configured to align positions of original image components of the plurality of input images on the basis of the positional displacement, and

a second alignment unit configured to align positions of the deterioration models corresponding to the plurality of input images on the basis of the positional displacement.

3. The image processing apparatus according to claim 1,

wherein the lens information estimation unit estimates the lens information by comparing a deterioration degree of the input image and the deterioration model, the deterioration degree being obtained by canceling an original image component from the plurality of input images.

4. The image processing apparatus according to claim 3,

wherein the lens information estimation unit estimates the lens information in such a way to minimize a difference between the deterioration degree of the input image and the deterioration model.

5. The image processing apparatus according to claim 1,

wherein the deterioration model is a model prescribing deterioration of the image depending on the lens information and a position in the image.

6. The image processing apparatus according to claim 1,

wherein the deterioration component is a characteristic of luminance shading, color shading, or aberration.

7. The image processing apparatus according to claim 1,

wherein the lens information is an aperture value, a focal length or a focus distance, or an exit pupil diameter or an exit pupil distance.

8. The image processing apparatus according to claim 1, comprising:

a correction processing unit configured to correct a photographed image on a basis of the lens information and the deterioration model.

9. The image processing apparatus according to claim 1,

wherein the alignment unit aligns the positions of the input images using block matching.

10. The image processing apparatus according to claim 1, comprising:

a luminance level adjustment unit configured to adjust luminance of the plurality of aligned input images.

11. The image processing apparatus according to claim 1, comprising:

a reduction unit configured to reduce the plurality of input images,

wherein the positional displacement estimation unit estimates the positional displacement on a basis of the plurality of reduced input images.

12. The image processing apparatus according to claim 1,

wherein the lens information estimation unit

estimates, with respect to the three or more input images, the lens information for the input images of a plurality of aligned combinations, and

estimates lens information that minimizes a difference from the deterioration model among pieces of the lens information estimated for the plurality of combinations.

13. An image processing method comprising:

estimating positional displacement between a plurality of input images obtained by photographing an identical photographic subject;

aligning positions of the plurality of input images on a basis of the estimated positional displacement; and

estimating lens information of a lens used to photograph the input image depending on a deterioration model indicating a deterioration component of the lens on a basis of the aligned input images.