Method for semantic scene classification using camera metadata and content-based cues

• US 7,555,165 B2
• Filed: 11/13/2003
• Issued: 06/30/2009
• Est. Priority Date: 11/13/2003
• Status: Expired due to Fees

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Luo, Jiebo, Boutell, Matthew R.
• Primary Examiner(s): Wu; Jingge
• Assistant Examiner(s): Krasnic; Bernard
• Application Number:
  US10/712,657
• Publication Number:
  US 20050105776A1
• Time in Patent Office: 2,056 Days
• Field of Search: 382/224, 382/228, 382/165
• US Class Current: 382/224
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

A method for scene classification of a digital image includes extracting pre-determined camera metadata tags from the digital image. The method also includes obtaining estimates of image class based on the extracted metadata tags. In addition, the method includes obtaining estimates of image class based on image and producing a final estimate of image class based on a combination of metadata-based estimates and image content-based estimates.

Claims

1. A method for scene classification of a digital image comprising the steps of:

(a) extracting one or more pre-determined camera metadata tags from the digital image;

(b) generating an estimate of image class of the digital image based on (1) the extracted camera metadata tags and not (2) image content features using a first data processing path, thereby providing a metadata-based estimate based only on the extracted camera metadata tags or generating a metadata null estimate;

(c) generating, separately from the metadata-based estimate, another estimate of image class of the digital image based on (1) image content features and not (2) the extracted camera metadata tags using a second data processing path separate from the first data processing path, thereby providing an image content-based estimate based only the image content features or generating a content-based null estimate; and

(d) producing a final integrated estimate of image class of the digital image using a Bayesian network based on a combination of

     1) the metadata-based estimate and the image content-based estimate,

     2) the metadata-based estimate and the image-based null estimate, or

     3) the image content-based estimate and the metadata null estimate;

wherein steps (b), (c) and (d) are each implemented using a computing device.

2. The method as claimed in claim 1 wherein the metadata extracted in step (a) includes one or more of exposure time, aperture, shutter speed, brightness value, subject distance and flash fired.

3. The method as claimed in claim 1 wherein the image content features in step (c) include one or more of color, texture and semantic features.

4. The method as claimed in claim 1, further comprising the step of applying a customized image enhancement procedure to the digital image in response to the final estimate of image class of the digital image.

5. The method as claimed in claim 4, wherein the customized image enhancement procedure is color balancing and the customized image enhancement procedure includes retaining or boosting brilliant colors in images classified as sunset scenes and removing warm-colored cast from indoor images classified as tungsten-illuminated scenes.

6. A computer-readable medium storing a computer program for causing a computer to implement a method for scene classification of a digital image comprising the steps of:

(a) extracting one or more pre-determined camera metadata tags from the digital image;

(b) generating an estimate of image class of the digital image based on (1) the extracted camera metadata tags and not (2) image content features using a first data processing path, thereby providing a metadata-based estimate based only on the extracted camera metadata tags or generating a metadata null estimate;

(c) generating, separateIy from the metadata-based estimate, another estimate of image class of the digital image based on (1) image content features and not (2) the extracted camera metadata tags using a second data processing path separate from the first data processing path, thereby providing an image content-based estimate based only the image content features or generating a content-based null estimate; and

(d) producing a final integrated estimate of image class of the digital image using a Bayesian network based on a combination of

     1) the metadata-based estimate and the image content-based estimate,

     2) the metadata-based estimate and the image-based null estimate, or

     3) the image content-based estimate and the metadata null estimate.