Extracting image frames suitable for printing and visual presentation from the compressed image data

• US 6,298,145 B1
• Filed: 01/19/1999
• Issued: 10/02/2001
• Est. Priority Date: 01/19/1999
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Company (HP Inc.)
• Inventors: Zhang, Hong Jiang, Marichal, Xavier
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): Tabatabai, A
• Application Number:
  US09/233,500
• Time in Patent Office: 987 Days
• Field of Search: 382/103, 382/166, 345/441, 354/430, 396/55
• US Class Current: 382/103
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

An image processing system for automatically extracting image frames suitable for printing and/or visual presentation from a compressed image data is described. The image processing system includes a face detector that detects if an image frame contains at least a face. The image processing system also includes a blur detector that determines the blur indicator value of the image frame directly using the information contained in the compressed image data if the image frame is determined to contain a face. The blur detector indicates that the image frame is suitable for printing and/or visual presentation if the blur indicator value of the image frame is less than a predetermined threshold. The image processing system may also include a motion analyzer that determines if the image frame is a super-resolution image frame suitable for printing and/or visual presentation if the image frame does not contain any face. The image processing system may also include a face tracker that detects if the image frame contains a non-frontal face.

Claims

1. An image processing system for automatically extracting image frames suitable for printing or visual presentation from a compressed image data, comprising:

a face detector that detects if an image frame contains at least a face;

a blur detector coupled to the face detector to determine the blur indicator value of the image frame, if the image frame is determined to contain a face, directly using the information contained in the compressed image data, wherein the blur detector indicates that the image frame is suitable for printing and/or visual presentation if the blur indicator value is less than a predetermined threshold, wherein the blur detector further comprises

an extractor that extracts DCT (Discrete Cosine Transform) coefficients directly from the compressed image frame;

a blur calculation module that computes the blur indicator value by examining the occurrence histogram of non-zero DCT coefficients of the image frame, wherein the blur indicator value is normalized by the size of the image such that the blur indicator value is independent of the content and size of the image frame.

2. The image processing system of claim 1, further comprising a face tracker coupled to the face detector to track the location of a detected face through subsequently skipped image frames by the face detector in order to guide the face detector to detect the face in a next image frame next to the skipped image frames in the face detector.

3. The image processing system of claim 2, wherein the face tracker also tracks the appearance or disappearance of a face using forward or backward tracking.

4. The image processing system of claim 2, wherein the compressed image data is compressed in accordance with MPEG standard and the face detector detects if an I image frame contains a face and skips all image frames between any two I frames.

5. The image processing system of claim 1, further comprising a motion analyzer coupled to the face detector to detect if the image frame contains motion when the face detector determines that the image frame does not contain any face, wherein the image frame is selected as suitable candidate frame for printing or visual presentation if the motion analyzer determines that the image frame does not contain motion.

6. The image processing system of claim 5, wherein the motion analyzer further comprises

(I) a motion vector selector that selects macro-blocks from the image frame that contain reliable motion vectors;

(II) a motion estimation module coupled to the motion vector selector to use the motion vectors of the selected macro-blocks to determine motion of the image frame with respect to related image frames.

7. The image processing system of claim 6, wherein the motion vector selector selects the macro-blocks from the image frame that contain reliable motion vectors by determining if a small number of DCT coefficients of the macro-block are non-zero DCT coefficients.

8. The image processing system of claim 5, further comprising a blur detector coupled to the motion analyzer to determine the blur indicator value of the image frame if the image frame is determined not to contain motion.

9. The image processing system of claim 1, further comprising a shot boundary detector and key frame extractor coupled to the face detector to provide shot boundary and key frame information to the face detector when the face detector detects face image frames from the compressed image data.

10. An image processing system for automatically extracting image frames suitable for printing or visual presentation from a compressed image data, comprising:

a blur detector that determines the blur indicator value of the image frame if the image frame is determined not to contain motion, wherein the blur detector indicates that the image frame is suitable for printing and/or visual presentation if the blur indicator value is less than a predetermined threshold, wherein the blur detector further comprises

an extractor that extracts DCT (Discrete Cosine Transform) coefficients directly from the compressed image frame;

a blur calculation module that computes the blur indicator value by examining the occurrence histogram of non-zero DCT coefficients of the image frame, wherein the blur indicator value is normalized by the size of the image such that the blur indicator value is independent of the content and size of the image frame.

11. The image processing system of claim 10, wherein the compressed image data is compressed in accordance with MPEG or JPEG standard.

12. The image processing system of claim 10, wherein the blur calculation module computes the blur indicator value by

calculating the occurrence histogram of non-zero DCT coefficients of the image frame;

normalizing the occurrence histogram of non-zero DCT coefficients of the image frame by dividing (1) the histogram with the number of DCT blocks of the image frame, and (2) the number of non-zero occurrences of a non-DC DCT coefficient with the number of non-zero occurrences of the DC DCT coefficient;

calculating the blur indicator value by summing the value of all cells of the occurrence histogram of non-zero DCT coefficients weighted by a predetermined DCT coefficient weighting table.