Method for detecting and selecting good quality image frames from video

• US 20050286802A1
• Filed: 06/20/2005
• Published: 12/29/2005
• Est. Priority Date: 06/22/2004
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Ilbery, Peter William Mitchell, Politis, George, Clark, James David
• Granted Patent: US 7,916,173 B2
• Time in Patent Office: Days
• US Class Current: 382/286
• CPC Class Codes: H04N 17/00 Diagnosis, testing or measuring for television systems or their details
• CPC Class Codes: H04N 17/06 for recorders
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

A method of determining a quality value for an image frame is disclosed. The method comprises dividing (in a step 202) the frame into a plurality of tiles (906) and determining attributes (in a step 206) of each said tile based upon pixel values of the tile, and pixel values of a corresponding tile of a preceding frame. The method then establishes (in steps 210, 212) the quality value of the frame by testing the tile attributes of the frame against pre-determined criteria. The method then defines (in a step 220) the quality value of the frame depending upon results of the testing.

Claims

1. A method of determining a quality value of a frame, the method comprising the steps of:

dividing the frame into a plurality of tiles;

determining attributes of each said tile based upon (i) pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and

establishing the quality value of the frame by;

testing the tile attributes of the frame against pre-determined criteria; and

defining the quality value of the frame depending upon results of said testing.

2. A method according to claim 1, wherein if the tiles of the frame do not meet the criteria of the testing step, the defining step defines a default quality value for the frame.

3. A method according to claim 1, wherein a said pixel value comprises the luminance value of the pixel which is represented in one of YUV and YCbCr video colour space.

4. A method according to claim 1, wherein:

the frame is one of a sequence of video frames;

the method is applied to each frame in the sequence; and

the best frame in the sequence is identified as that frame having the best quality value.

5. A method according to claim 3, wherein:

the attributes comprise a motion definition state and a motion magnitude, these being determined by;

classifying the tile as having a motion defined state if one of (i) an intensity range of pixels in the tile or in the corresponding tile of the preceding frame exceeds a pre-determined luminance threshold; and

(ii) a difference between mid-range pixel values for the tile and the corresponding tile of the preceding frame exceeds a pre-determined mid-range difference threshold; and

determining, if the tile has a motion defined state, said motion magnitude dependent upon a difference in pixel value distributions between a pixel value histogram of the tile and a pixel value histogram of the corresponding tile in the previous frame; and

wherein;

the attributes further comprise a focus definition state and a focus magnitude, these being determined by;

classifying the tile as having a focus undefined state if (i) the intensity range of pixels in the tile is too small, or there is too much motion in the tile or (ii) the number of defined difference quotients in the tile is less than a threshold; and

determining the focus magnitude depending upon the largest difference quotient values of pixel luminance of the tile.

6. A method according to claim 5, wherein the criteria are:

(a) that the number of tiles in the frame having a motion defined state exceed a pre-defined motion defined threshold;

(b) that the ratio of (i) tiles in the frame having a motion defined state whose motion magnitude exceeds a pre-defined motion threshold, to (ii) the total number of tiles in the frame having a motion defined state, exceeds a pre-defined motion defined threshold;

(c) that the number of tiles in the frame having a focus defined state exceeds a pre-defined focus defined threshold; and

(d) that the average of the largest focus magnitude values of tiles in the frame having a focus defined state exceeds a pre-defined focus defined threshold.

7. A method according to claim 6, wherein if the tiles of the frame meet all the criteria of the testing step, the quality value depends upon the ratio and the average determined in the steps (b) and (d) of claim 6.

8. A method according to claim 1, wherein:

the quality value is determined based upon a sub-region of the frame; and

the dividing step divides the sub-region into the plurality of tiles.

9. A method according to claim 8, wherein the defining step is dependent upon a weight establishing the importance of the sub-region in the frame, and a weight establishing the size of the sub-region relative to the frame.

10. A method according to claim 6, wherein the criterion (b) is dependent upon the spatial location of tiles in the frame wherein:

the criterion (b) is that the ratio of (i) tiles in a pre-defined region of the frame having a motion defined state whose motion magnitude exceeds a pre-defined motion threshold, to (ii) the total number of tiles in the frame having a motion defined state, exceeds a pre-defined motion defined threshold.

11. A method according to claim 6, wherein the criterion (d) is dependent upon the spatial location of tiles in the frame wherein:

the criterion (d) is that the average of the largest focus magnitude values of tiles in the frame having a focus defined state exceeds a pre-defined focus defined threshold, wherein said focus magnitude values are weighted according to the relative distance of respective tiles from the centre of the frame.

12. A method according to claim 7, wherein the quality value further depends upon a spatial distribution of the tiles having one of a motion defined state and a focus defined state.

13. An apparatus for determining a quality value of a frame, the apparatus comprising:

means for dividing the frame into a plurality of tiles;

means for determining attributes of each said tile based upon (i) pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and

means for establishing the quality value of the frame comprising;

means for testing the tile attributes of the frame against pre-determined criteria; and

means for defining the quality value of the frame depending upon results of said testing.

14. A computer program product having a computer readable medium having a computer program recorded therein for directing a processor to execute a method of determining a quality value of a frame, said computer program product comprising:

code for dividing the frame into a plurality of tiles;

code for determining attributes of each said tile based upon (i) pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and

code for establishing the quality value of the frame comprising;

code for testing the tile attributes of the frame against pre-determined criteria; and

code for defining the quality value of the frame depending upon results of said testing.

15. A computer program for directing a processor to execute a method of determining a quality value of a frame, said computer program product comprising:

code for dividing the frame into a plurality of tiles;

code for determining attributes of each said tile based upon (i) pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and

code for establishing the quality value of the frame comprising;

code for testing the tile attributes of the frame against pre-determined criteria; and

code for defining the quality value of the frame depending upon results of said testing.

16. A video frame selected from a sequence of video frames dependent upon a quality value of the frame, the frame being selected by a method of determining a quality value of a video frame, said method being applied to each frame in the sequence, said method comprising, in regard to a particular frame in the sequence, the steps of:

dividing the frame into a plurality of tiles;

determining attributes of each said tile based upon (i) pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and

establishing the quality value of the frame by;

testing the tile attributes of the frame against pre-determined criteria; and

defining the quality value of the frame depending upon results of said testing.

35. A method according to claim 16 wherein said selecting at least one image is determined by ranking quality metric values and selecting the highest value images as the best image.

17. A method of estimating quality of an image in a video, the video comprising a plurality of images, each image comprising a plurality of pixels, said method comprising the steps of:

arranging, for a current image of said video, pixels of the current image into a plurality of tiles, each said tile containing a plurality of pixels;

determining the number of the tiles of the current image having a defined characteristic; and

estimating quality of the current image based on the number of said tiles having the characteristic as a proportion of the number of tiles in the current image.

19. A method according to claim 17 or claim 18 wherein said characteristic is dependent on luminance.

20. A method according to claim 17 or claim 18 wherein said tiles of said current image comprise a predetermined number of pixels N_p.

21. A method according to claim 17 or claim 18 wherein said characteristic is one of a motion characteristic, a focus characteristic, and an exposure characteristic.

22. A method according to claim 21 wherein said motion characteristic classifies a tile as either motion-defined or motion-undefined.

23. A method according to claim 21 wherein said motion characteristic establishes a motion magnitude in said tile if said tile is determined to be motion-defined.

24. A method according to claim 21 wherein said focus characteristic classifies a tile as either focus-defined or focus-undefined.

25. A method according to claim 21 wherein said focus characteristic establishes a focus magnitude in said tile if said tile is determined to be focus-defined.

26. A method according to claim 22 wherein determining said motion characteristics of a tile comprises the steps of:

obtaining, for each colour channel of tile pixels, a respective range of sample values, and;

setting a motion characteristic of said tile of said current image to indicate that the tile is motion-undefined if;

said colour channel range of said tile of said current image does not exceed a colour channel range threshold value, and;

said colour channel range of a corresponding tile of a previous image does not exceed said colour channel range threshold value, and;

the absolute difference between a colour channel mid-range value of said tile of said current image and a corresponding colour channel mid-range value of said corresponding tile of said previous image does not exceed a colour channel mid-range difference threshold value;

otherwise

setting said motion characteristic of said tile of said current image indicate that said tile is motion-defined.

27. A method according to claim 25 wherein determination of the focus characteristic of a tile comprises the steps of:

generating, for each sample of said colour channel, a difference quotient for said sample value in the horizontal and vertical directions; and

defining said focus magnitude of said tile to be equal to the maximum absolute value of said difference quotient values of said tile if said tile of said current image is focus-defined.

28. A method according to claims 17 wherein determining said proportion comprises the steps of:

determining the respective numbers of tiles in the image having respective good, bad and unknown classifications;

determining an image metric value for said image; and

classifying the frame as being one of acceptable and unacceptable dependent upon (i) the respective numbers of tiles in the image having good, bad and unknown classification and (ii) the image metric value.

29. A method according to claim 28 wherein said image metric value is an image motion metric value defined as the number of motion-defined tiles with a motion magnitude greater than a motion magnitude threshold value, normalised against the total number of motion-defined tiles.

30. A method according to claim 28 wherein said image metric value is an image focus metric value defined as the average of the focus magnitudes of focus-defined tiles with a focus magnitude greater than a focus magnitude threshold value.

31. A method according to claim 28 wherein said image metric value is an image exposure metric value dependent upon the entropy of the luminance values of the frame.

32. A method according to claims 17 wherein said quality of the current image is defined as the weighted contribution from motion, focus and exposure characteristic values.

33. A method according to claim 32 wherein said quality of current image is assigned a normalised value of 0.0 for the worst quality image, 1.0 for the best quality image, and corresponding intermediate values for images of intermediate quality.

34. A method according to claim 33 wherein said quality of a current image is classified as acceptable if any of the following criteria are satisfied:

the proportion of known tiles exceeds a first threshold value;

the proportion of good tiles exceeds a second threshold value; and

the exposure metric exceeds a third threshold value.

18. A method of selecting an image from a video comprising a plurality of images, each image comprising a plurality of pixels, and said method comprising the steps of:

(a) performing, for each image of said plurality of images, the steps of;

arranging pixels of the image into a plurality of tiles, each said tile containing a plurality of pixels;

determining the number of the tiles of the image having a defined characteristic; and

estimating quality of the image based on the number of said tiles having the characteristic as a proportion of the number of tiles in the image; and

(b) selecting an image from said video in accordance with the estimated quality.