Image sharpness classification system

• US 8,805,112 B2
• Filed: 06/24/2010
• Issued: 08/12/2014
• Est. Priority Date: 05/06/2010
• Status: Active Grant

• Current Assignee: Nikon Corporation
• Original Assignee: Nikon Corporation
• Inventors: Hong, Li
• Primary Examiner(s): Cunningham, Gregory F
• Application Number:
  US13/640,458
• Publication Number:
  US 20130033608A1
• Time in Patent Office: 1,510 Days
• Field of Search: 348/252, 348/606, 348/625, 382/159, 382/255, 382/263, 382/264
• US Class Current: 382/260
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics

Abstract

A method for predicting whether a test image (318) is sharp or blurred includes the steps of: providing a sharpness classifier (316) that is trained to discriminate between sharp and blurred images; computing a set of sharpness features (322) for the test image (318) by (i) generating a high pass image (404) from the test image (318), (ii) generating a band pass image (406) from the test image (318), (iii) identifying textured regions (408) in the high pass image, (iv) identifying texture regions (410) in the band pass image, and (v) evaluating the identified textured regions in the high pass image and the band pass image to compute the set of test sharpness features (412); and evaluating the sharpness features using the sharpness classifier (324) to estimate if the test image (318) is sharp or blurry (20).

Claims

1. A method for determining if a test image is either sharp or blurred, the method comprising the steps of;

training a sharpness classifier to discriminate between sharp and blurred images, the sharpness classifier is trained based on a set of training sharpness features computed from a plurality of training images; and

applying the trained classifier to the test image to determine if the test image is sharp or blurred based on a set of test sharpness features computed from the test image;

wherein, the set of training sharpness features for each training image and the set of test sharpness features for each test image are computed by (i) preprocessing the respective image;

(ii) generating a high pass image from the preprocessed image;

(iii) generating a band pass image from the preprocessed image;

(iv) identifying textured regions in the high pass image;

(v) identifying texture regions in the band pass image; and

(vi) computing the set of sharpness features from the identified textured regions.

2. The method of claim 1, wherein the step of preprocessing the respective image includes the steps of:

(i) border-cropping;

(ii) down sampling; and

(iii) converting the image into a grey scale image.

3. The method of claim 1, wherein the step of generating the high pass image includes computing the absolute difference between the preprocessed image and a low pass filtered version of the preprocessed image.

4. The method of claim 1, wherein the step of generating the band pass image includes the step of computing the absolute difference between two levels of low pass filtering of the preprocessed image.

5. The method of claim 1, wherein the steps of generating a high pass image and generating a band pass image include utilizing an omni-directional type filter.

6. The method of claim 1, wherein the step of computing the set of sharpness features includes computing one or more of the following sharpness features:

(i) a measure of the texture percentage in the band pass image;

(ii) a measure of the mean energy in the textured regions of the high pass image;

(iii) a measure of the mean energy in the textured regions of the band pass image;

(iv) a measure of the ratio of mean energy difference between textures in the band pass image to the mean energy of textures in the high pass image; and

(v) a measure of the mean ratio of per texture pixel energy in the band pass image to the combined energy of the band pass image and the high pass image.

7. The method of claim 1 wherein the step of generating the high pass image and the step of generating the band pass image includes the use of an oriented filter.

8. The method of claim 7, wherein the oriented band pass image is generated by computing the absolute difference between two levels of oriented-type filtering of the grey scale image.

9. The method of claim 1, wherein the step of identifying the textured regions in the high pass image includes the steps of:

constraining texture regions in the high pass image to be pixels with values in the high pass image that exceed a preset texture threshold; and

constraining texture regions in the high pass image to be pixels with corresponding gradient values exceed a preset gradient threshold.

10. The method of claim 1 wherein the step of identifying the textured regions in the band pass image includes the steps of:

constraining texture regions in the oriented band pass image to be pixels with values in the band pass image that exceed a preset texture threshold; and

constraining texture regions in the band pass image to be pixels with corresponding gradient values exceed a preset gradient threshold.

11. The method of claim 1, further comprising the step of determining if there is motion blur in the test image, and wherein if it is determined that there is motion blur in the test image, the step of generating the high pass image and the step of generating the band pass image for the test image including the use of an oriented filter or if it is determined that there is not motion blur in the test image, the step of generating the high pass image and the step of generating the band pass image for the test image including the use of an omni-directional filter.

12. The method of claim 1, further comprising the step of determining if there is motion blur in the test image, (i) wherein if it is determined that there is not motion blur, the step of computing the sets of test sharpness features includes computing one or more of the following test sharpness features:

(a) a measure of the texture percentage in the band pass image;

(b) a measure of the mean energy in the textured regions of the high pass image;

(c) a measure of the mean energy in the textured regions of the band pass image;

(d) a measure of the ratio of mean energy difference between textures in the band pass image to the mean energy of textures in the high pass image; and

(e) a measure of the mean ratio of per texture pixel energy in the band pass image to the combined energy of the band pass image and the high pass image, and (ii) wherein if it is determined that there is motion blur in the test image, the step of computing the sets of test sharpness features further includes computing one or more of the following test sharpness features;

(a) a measure of the texture percentage from an oriented band pass image;

(b) a measure of the mean energy in the textured regions in an oriented high pass image;

(c) a measure of the mean energy in the textured regions of the oriented band pass image;

(d) a measure of the ratio of mean energy difference between textures in the oriented band pass and the oriented band pass image to the mean energy of textures in the oriented high pass image; and

(e) a measure of the mean ratio of per texture pixel energy in the oriented band pass image to the combined energy of the oriented band pass image and the oriented high pass image.

13. The method of claim 1, further comprising the steps of:

identifying metadata associated with the test image; and

using the metadata associated with the test image to aid in determining if the test image is sharp or blurred.

14. The method of claim 13, wherein the metadata includes one or more of the following:

ISO noise level associated with the test image;

an exposure value associated with the test image;

magnification factor associated with the test image; and

a depth of field factor associated with the test image.

15. The method of claim 13, further comprising the steps of:

identifying a test image with high noise level based on an ISO value; and

reducing noisy pixels in texture regions in the high pass image.

16. The method of claim 13, further comprising the steps of:

identifying a test image with a very low exposure value; and

removing pixels associated with highlights in the texture regions in the high pass image and the band pass image.

17. The method of claim 13, further comprising the steps of:

identifying a test image with a high magnification factor; and

slightly relaxing sharpness standard for the test image.

18. The method of claim 13, further comprising the steps of:

identifying a test image with very shallow depth of field, and removing weak texture structures in the texture regions of the high pass image.

19. A method for estimating if a test image is either sharp or blurred, the method comprising the steps of:

providing a sharpness classifier that is trained to discriminate between sharp and blurred images;

computing a set of test sharpness features for the test image using one or more of the following steps;

(i) generating a test high pass image from the test image;

(ii) generating a test band pass image from the test image;

(iii) identifying test textured regions in the test high pass image;

(iv) identifying test textured regions in the test band pass image;

(v) evaluating the identified test textured regions in the test high pass image; and

(vi) evaluating the identified test textured regions in the test band pass image; and

evaluating the test sharpness features using the sharpness classifier to estimate if the test image is sharp or blurry.

20. The method of claim 19 wherein the step of providing a sharpness classifier includes the step of training the sharpness classifier with a set of training sharpness features for each of a plurality of training images;

and wherein the step of training the sharpness classifier includes the step of computing the set of training sharpness features for each training image by (i) generating a training high pass image from the training image;

(ii) generating a training band pass image from the training image;

(iii) identifying training textured regions in the training high pass image and the training band pass image; and

(iv) evaluating the identified training textured regions in the training high pass image and the training band pass image.

21. The method of claim 19, wherein the steps of generating a high pass image and generating a band pass image include utilizing an omni-directional type filter.

22. The method of claim 19, wherein the step of computing the set of test sharpness features includes computing one or more of the following sharpness features:

(i) a measure of the texture percentage in the test band pass image;

(ii) a measure of the mean energy in the textured regions of the test high pass image;

(iii) a measure of the mean energy in the textured regions of the test band pass image;

(iv) a measure of the ratio of mean energy difference between textures in the test band pass image to the mean energy of textures in the test high pass image; and

(v) a measure of the mean ratio of per texture pixel energy in the test band pass image to the combined energy of the test band pass image and the high pass image.

23. The method of claim 19 wherein the step of generating the test high pass image and the step of generating the test band pass image includes the use of an oriented filter.

24. The method of claim 19, further comprising the steps of:

identifying a test image with high noise level based on an ISO value; and

reducing noisy pixels in texture regions in the test high pass image.

25. The method of claim 19, further comprising the steps of:

identifying a test image with a very low exposure value; and

removing pixels associated with highlights in the texture regions in the test high pass image and the test band pass image.

26. The method of claim 19, further comprising the steps of:

identifying a test image with a high magnification factor; and

slightly relaxing sharpness standard for the test image.

27. The method of claim 19, further comprising the steps of:

identifying a test image with very shallow depth of field, and removing weak texture structures in the texture regions of the test high pass image.

28. The method of claim 19 further comprising the steps of evaluating the test image to determine if it has an extremely low texture percentage, and classifying the test image with the extremely low texture percentage as being blurred.

29. The method of claim 19 further comprising the step of determining a mean energy in the textured regions of the test high pass image, and classifying the test image as sharp if this mean energy is greater than a predetermined level.

30. The method of claim 19 further comprising the step of determining a ratio of mean energy difference between textures in the test band pass image to the mean energy of textures in the test high pass image, and classifying the test image as sharp if the ratio is greater than a predetermined level.

31. A camera including a capturing system for capturing a test image, and a control system that utilizes the method of claim 19 to estimate if the test image is either sharp or blurred.