Classifying image regions based on picture location

• US 7,860,320 B2
• Filed: 06/26/2006
• Issued: 12/28/2010
• Est. Priority Date: 06/26/2006
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Luo, Jiebo, Gallagher, Andrew C.
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): ANSARI, TAHMINA N
• Application Number:
  US11/426,468
• Publication Number:
  US 20070297683A1
• Time in Patent Office: 1,646 Days
• Field of Search: 382/224, 382/109, 382/180, 382/173, 382/190, 354/106, 354/76, 701/207, 358/109, 358/107
• US Class Current: 382/227
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A method of classifying regions in a digital image or video captured by an image capture device includes providing a geographic location determining device associated with the image capture device; using the location determining device to measure the image capture location at substantially the time that the digital image or video was captured; and classifying regions in the captured digital image or video into one or more classes based on the image-capture location.

Claims

1. A method of classifying regions of image pixels in a digital image or video captured by an image capture device comprising:

a) providing a geographic location determining device associated with the image capture device that provides an image capture GPS location associated with an image;

b) using the location determining device to provide the image capture GPS location at substantially the time that the digital image or video was captured; and

c) using a data processor for classifying regions of image pixels in the captured digital image or video into one or more material classes based on a spatial context model that indicates the likelihood that specific material classes occur or co-occur in images or videos captured at the particular image capture GPS location.

2. The method of claim 1, wherein the one or more material classes include sky, grass, water, clouds, pavement, snow, or walls.

3. The method of claim 1 wherein step (c) further includes:

i) obtaining homogenous regions of image pixels from the captured digital image or a frame of the video;

ii) extracting features from the homogeneous regions of image pixels; and

iii) classifying the region of image pixels into one or more material classes, with associated class probabilities, based on the image capture GPS location and the extracted features.

4. The method of claim 3, wherein the extracted features include color or texture features associated with the region of image pixels.

5. The method of claim 1, wherein step (c) further includes using the image-capture time to classify regions of image pixels of the digital image.

6. The method of claim 3, wherein step (ii) further includes using the image-capture time to classify regions of image pixels of the digital image.

7. The method of claim 1, wherein step (c) further includes using the camera-facing direction to classify regions of image pixels of the digital image.

8. The method of claim 3, wherein step (ii) further includes using the camera-facing direction to classify regions of image pixels of the digital image.

9. The method of claim 3 wherein step (iii) includes associating prior probabilities of material classes according to geographic location and using such prior probabilities to determine the class of a region of image pixels.

10. The method of claim 3 wherein step (iii) includes associating conditional probabilities between material classes according to geographic location and using such conditional probabilities to determine the respective material class of each region of image pixels.

11. The method of claim 10 wherein the conditional probabilities between material classes are specified as pair-wise probabilities according to one or more of spatial relationships including above, far above, below, far below, beside, enclosed, or enclosing.

12. The method of claim 3 wherein step

(iii) further includes

(i) producing initial material class probabilities based only on the extracted features;

(ii) using a graphical model to modify the initial material class probabilities to produce posterior material class probabilities in response to the prior probabilities and the conditional probabilities between material classes according to geographic location.

13. The method of claim 3 further includes labeling a region of image pixels as the material class with highest posterior material class probability.

14. A method of enhancing regions of image pixels in a digital image or a video frame captured by an image capture device comprising:

a) providing a geographic location determining device associated with the image capture device that provides an image capture GPS location associated with an image;

b) using the geographic location determining device to provide the image capture GPS location at substantially the time that the digital image or video was captured;

c) using a data processor for classifying regions of image pixels in the captured digital image or video into one or more classes based on a spatial context model that indicates the likelihood that specific classes occur or co-occur in images or videos captured at the particular image capture GPS location; and

d) applying image enhancement to classified regions of image pixels depending on the class of each region.

15. A method of searching in a database a digital image or a video frame captured by an image capture device comprising:

a) providing a geographic location determining device associated with the image capture device that provides an image capture GPS location associated with an image;

b) using the geographic location determining device to provide the image capture GPS location at substantially the time that the digital image or video was captured;

c) using a data processor for classifying regions of image pixels in the captured digital image or video into one or more classes based on a spatial context model that indicates the likelihood that specific classes occur or co-occur in images or videos captured at the particular image capture GPS location, storing the classified regions of image pixels in a database; and

d) searching in the database for images containing regions of image pixels of one or more pre-determined classes.

16. A method of classifying regions of image pixels in a digital image or video captured by an image capture device comprising:

a) providing a geographic location determining device associated with the image capture device that provides an image capture GPS location associated with an image;

b) using the location determining device to provide the image capture GPS location at substantially the time that the digital image or video was captured;

c) providing a time determining device associated with the image capture device;

d) using the time determining device to measure the image capture time at substantially the time that the digital image or video was captured; and

e) using a data processor for classifying regions of image pixels in the captured digital image or video into one or more classes based on a spatial context model that indicates the likelihood that specific classes occur or co-occur in images or videos captured at the particular image capture GPS location and the image capture time.