Image sharpness classification system

• US 9,251,439 B2
• Filed: 08/18/2011
• Issued: 02/02/2016
• Est. Priority Date: 08/18/2011
• Status: Active Grant

• Current Assignee: Nikon Corporation
• Original Assignee: Nikon Corporation
• Inventors: Hong, Li
• Primary Examiner(s): Nazrul, Shahbaz
• Application Number:
  US14/118,966
• Publication Number:
  US 20140078320A1
• Time in Patent Office: 1,629 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 10/431 Frequency domain transformation; Autocorrelation
• CPC Class Codes: G06V 10/52 Scale-space analysis, e.g. wavelet analysis multi-scale boundary representations G06V10/42
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: H04N 23/617 Upgrading or updating of programs or applications for camera control
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation

Abstract

A method for predicting whether a test image (318) is sharp or blurred includes the steps of: training a sharpness classifier (316) to discriminate between sharp and blurred images, the sharpness classifier (316) being trained based on a set of training sharpness features (314) computed from a plurality of training images (306), the set of training sharpness features (314) for each training image (306) being computed by (i) resizing each training image (306) by a first resizing factor; (ii) identifying texture regions (408, 410) in the resized training image; and (iii) computing the set of sharpness features in the training image (412) from the identified texture regions; and applying the trained sharpness classifier (316) to the test image (318) to determine if the test image (318) is sharp or blurred based on a set of test sharpness features (322) computed from the test image (318), the set of test sharpness features (322) for each test image (318) being computed by (i) resizing the test image (318) by a second resizing factor that is different than the first resizing factor; (ii) identifying texture regions (408, 410) in the resized test image; and (iii) computing the set of sharpness features in the test image (412) from the identified texture regions.

Claims

1. A method for determining if a test image is either sharp or blurred, the method comprising the steps of:

training a sharpness classifier to discriminate between sharp and blurred images, the sharpness classifier being trained based on a set of training sharpness features computed from a plurality of training images, the set of training sharpness features for each training image being computed by (i) resizing each training image by a first resizing factor;

(ii) identifying texture regions in the resized training image; and

(iii) computing the set of sharpness features in the training image from the identified texture regions; and

applying the trained sharpness classifier to the test image to determine if the test image is sharp or blurred based on a set of test sharpness features computed from the test image, the set of test sharpness features for each test image being computed by (i) resizing the test image by a second resizing factor that is different than the first resizing factor;

(ii) identifying texture regions in the resized test image; and

(iii) computing the set of sharpness features in the test image from the identified texture regions.

2. The method of claim 1 wherein the first resizing factor is smaller than the second resizing factor.

3. The method of claim 1 wherein the step of training includes the steps of generating a high pass training image from the resized training image, and identifying texture regions in the high pass training image by constraining texture regions in the high pass training image to be pixels with values in the high pass training image that exceed a preset texture threshold and constraining texture regions in the high pass training image to be pixels with corresponding gradient values exceed a preset gradient threshold;

and wherein the step of applying includes the steps of generating a high pass test image from the resized test image, and identifying texture regions in the high pass test image by constraining texture regions in the high pass test image to be pixels with values in the high pass test image that exceed a preset texture threshold and constraining texture regions in the high pass test image to be pixels with corresponding gradient values exceed a preset gradient threshold.

4. The method of claim 3 wherein the step of training includes the steps of generating a band pass training image from the resized training image, and identifying texture regions in the band pass training image by constraining texture regions in the band pass training image to be pixels with values in the band pass training image that exceed a preset texture threshold and constraining texture regions in the band pass training image to be pixels with corresponding gradient values exceed a preset gradient threshold;

and wherein the step of applying includes the steps of generating a band pass test image from the resized test image, and identifying texture regions in the band pass test image by constraining texture regions in the band pass test image to be pixels with values in the band pass test image that exceed a preset texture threshold and constraining texture regions in the band pass test image to be pixels with corresponding gradient values exceed a preset gradient threshold.

5. The method of claim 4 further comprising the step of determining if there is motion blur in the test image, and wherein if it is determined that there is motion blur in the test image, the step of generating the high pass test image and the step of generating the band pass test image includes the use of one or more oriented filters or if it is determined that there is not motion blur in the test image, the step of generating the high pass test image and the step of generating the band pass test image includes the use of one or more omni-directional filters.

6. The method of claim 4 wherein the step of computing the set of sharpness features in the training image and/or computing the set of sharpness features in the test image includes computing one or more of the following sharpness features:

(i) a measure of the texture percentage in the band pass image;

(ii) a measure of the mean energy in the textured regions of the high pass image;

(iii) a measure of the mean energy in the textured regions of the band pass image;

(iv) a measure of the ratio of mean energy difference between textures in the band pass image to the mean energy of textures in the high pass image; and

(v) a measure of the mean ratio of per texture pixel energy in the band pass image to the combined energy of the band pass image and the high pass image.

7. The method of claim 1 further comprising the step of (i) calculating a hand-holdable factor for the test image;

(ii) comparing the hand-holdable factor to a hand-holdable factor threshold; and

(iii) if the hand-holdable factor exceeds the hand-holdable factor threshold, then testing the test image for motion blur.

8. The method of claim 1 further comprising the steps of:

identifying metadata associated with the test image; and

using the metadata associated with the test image to aid in determining if the test image is sharp or blurred, wherein the metadata includes one or more of the following;

ISO value associated with the test image;

an exposure value associated with the test image;

magnification factor associated with the test image; and

a depth of field factor associated with the test image.

9. The method of claim 8 further comprising the steps of categorizing the test image as a macro type image or a non-macro type image based on the magnification factor;

and resizing the test image by a different resizing factor depending on whether the test image is categorized as a macro type image or a non-macro type image.

10. The method of claim 8 further comprising the steps of categorizing the test image as a macro-level one type image, a macro-level two type image or a non-macro type image based on the magnification factor;

and resizing the test image by a different resizing factor depending on whether the test image is categorized as a macro-level one type image, a macro-level two type image or a non-macro type image.

11. The method of claim 8 further comprising the steps of:

identifying a test image with high noise level based on the ISO value;

determining if the exposure value of the test image is low;

applying strong noise reduction to pixels in the texture regions of the test image if the test image has high noise level and the exposure value is low; and

applying relaxed noise reduction to pixels in the texture regions of the test image if the test image has high noise level and the exposure value is not low.

12. The method of claim 8 further comprising the steps of:

determining if the exposure value of a test image is not low;

evaluating the test image to determine if the test image contains weak texture regions; and

excluding pixels in the weak texture regions from the step of computing the set of sharpness features in the test image from the identified texture regions if the test image has the exposure value that is not low.

13. The method of claim 8 further comprising the steps of:

identifying a high contrast edge in the test image;

approximating the gradient direction of the high contrast edge;

determining if the high contrast edge is blurry; and

excluding pixels in the texture regions of the test image from the step of computing the set of sharpness features in the test image from the identified texture regions if they define blurry high contrast edges.

14. The method of claim 8 wherein the step of identifying texture regions in the resized test image includes the steps of constraining texture regions in the resized test image to be pixels with values in the resized test image that exceed a preset texture threshold and constraining texture regions in the resized test image to be pixels with corresponding gradient values that exceed a preset gradient threshold;

and further comprising the steps of;

determining if the resized test image includes rich texture;

categorizing the resized test image as a macro type image or a non-macro type image based on the magnification factor; and

increasing one or more of the preset texture threshold and the preset gradient threshold for identifying texture regions in the resized test image if the test image includes rich texture and is a macro-type image.

15. A camera including a capturing system for capturing a test image, and a control system that utilizes the method of claim 1 to estimate if the test image is either sharp or blurred.