Event clustering of images using foreground/background segmentation

• US 20020168108A1
• Filed: 03/28/2001
• Published: 11/14/2002
• Est. Priority Date: 03/28/2001
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Loui, Alexander C., Sun, Zhaohui, Jeanson, Matthieu
• Granted Patent: US 6,915,011 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

An event clustering method uses foreground and background segmentation for clustering images from a group into similar events. Initially, each image is divided into a plurality of blocks, thereby providing block-based images. Utilizing a block-by-block comparison, each block-based image is segmented into a plurality of regions comprising at least a foreground and a background. One or more luminosity, color, position or size features are extracted from the regions and the extracted features are utilized to estimate and compare the similarity of the regions comprising the foreground and background in successive images in the group. Then, a measure of the total similarity between successive images is computed, thereby providing image distance between successive images, and event clusters are delimited from the image distances.

Claims

1. An event clustering method using foreground and background segmentation for clustering images from a group into similar events, said method including the steps of:

(a) segmenting each image into a plurality of regions comprising at least a foreground and a background;

(b) extracting one or more features from the regions comprising the foreground and background, said features including at least one of luminosity, color, position and size of the regions;

(c) utilizing the features to compute the similarity of the regions comprising the foreground and background of successive images in the group;

(d) computing a measure of the total similarity between successive images, thereby providing a measure of image distance between successive images; and

(e) delimiting event clusters from the image distances, whereby the event clusters include groups of images pertaining to the same events.

2. The method as claimed in claim 1 wherein the step (c) utilizes the features to generate a distance measure that indicates the similarity or dissimilarity between the regions.

3. The method as claimed in claim 1 wherein if a predetermined number of regions formed in step (a) are each less than a predetermined size, then a fixed region is generated for the foreground.

4. The method as claimed in claim 1 wherein the group of images are arranged in a chronological order and step (c) further utilizes the features to estimate and compare the similarity of regions comprising foreground and background in every other image in the group and step (d) computes a measure of the total similarity between every other image, thereby providing image distance between successive images and every other image.

5. The method as claimed in claim 1 wherein the group of images are arranged in a chronological order and step (c) further utilizes the features to estimate and compare the similarity of regions comprising foreground and background in every other two images in the group and step (d) computes a measure of the total similarity between every other two images, thereby providing image distance between successive images and every other two images.

6. A computer storage medium having instructions stored therein for causing a computer for perform the method of claim 1.

7. An event clustering method using foreground and background segmentation for clustering images from a group into similar events, said method including the steps of:

(a) dividing each image into a plurality of blocks, thereby providing block-based images;

(b) utilizing a block-by-block comparison to segment each block- based image into a plurality of regions comprising at least a foreground and a background;

(c) extracting one or more features from the regions comprising the foreground and background, said features including at least one of luminosity, color, position and size of the regions;

(d) utilizing the features to compute the similarity of the regions comprising the foreground and background of successive images in the group, thereby leading to a measure of image distance between successive images; and

(e) delimiting event clusters from the image distances, whereby the event clusters include groups of images pertaining to the same events.

8. The method as claimed in claim 7 wherein the block-by-block comparison in step (b) comprises extracting one or more of said features from the blocks, utilizing the features to compute the similarity of each block with respect to its neighboring blocks, forming regions from similar blocks and merging similar regions into a background and a foreground.

9. A computer storage medium having instructions stored therein for causing a computer for perform the method of claim 7.

10. The method as claimed in claim 7 wherein if a predetermined number of regions formed in step (b) are each less than a predetermined size, then a fixed regions is generated for the foreground.

11. An event clustering method using foreground and background segmentation for clustering images from a group into similar events, said method including the steps of:

(a) dividing each image into a plurality of blocks, thereby providing block-based images;

(b) utilizing a block-by-block comparison to segment each block-based image into a plurality of regions, wherein a first combination of regions comprises a foreground and a second combination of regions comprises a background;

(c) extracting one or more features from the regions comprising the foreground and background, said features including at least one of luminosity, color, position and size of the regions;

(d) utilizing the features to compute the similarity between each region of the combination comprising the foreground of one image in the group and each region comprising the foreground of another image in the group, and further computing the similarity between each region of the combination comprising the background of said one image in the group and each region comprising the background of said another image in the group;

(e) computing a mean value measure of the total similarity between successive images based on the similarity of all regions included in the combinations comprising the foreground and background, thereby providing a measure of image distance between said images; and

(f) delimiting event clusters from the image distances, whereby the event clusters include groups of images pertaining to the same events.

12. The method as claimed in claim 11 wherein the computation of the similarity between each region in step (d) includes a component to account for the relative sizes of the regions

13. A computer storage medium having instructions stored therein for causing a computer for perform the method of claim 11.

15. The method as claimed in claim 14 wherein said low-level features include at least one of luminosity, color, position and size of the regions

18. The system as claimed in claim 17 wherein the group of images are arranged in a chronological order and said third module further utilizes the features to estimate and compare the similarity of regions comprising foreground and background in every other image in the group and computes a measure of the total similarity between every other image, thereby providing image distance between successive images and every other image.

19. The system as claimed in claim 17 wherein the group of images are arranged in a chronological order and the third module further utilizes the features to estimate and compare the similarity of regions comprising foreground and background in every other two images in the group and computes a measure of the total similarity between every other two images, thereby providing image distance between successive images and every other two images.

14. A method for clustering a sequence of images into events based on similarities between the images, said method comprising the steps of:

(a) segmenting each image into regions, including combinations of one or more regions comprising a foreground and a background;

(b) extracting low-level features from the regions;

(c) utilizing the low-level features to compare the regions comprising the foreground and background of successive images, said comparison generating an image similarity measure for the regions comprising the foreground and background of the successive images;

(d) combining the image similarity measures for the regions comprising the foreground and background of the successive images to obtain a global similarity measure; and

(e) delimiting event clusters by using the global similarity measure.

16. A method for segmenting an image into a foreground and a background comprising the steps of:

(a) dividing each image into a plurality of blocks;

(b) extracting one or more features from the blocks, said features including at least one of luminosity, color, position and size of the regions;

(c) utilizing the features to generate a similarity measure between each block and one or more of its neighboring blocks;

(d) identifying boundary separations between groups of blocks having the least similarity;

(e) connecting the boundary separations to form regions; and

(f) merging similar regions to form a foreground and a background.

17. A system using foreground and background segmentation for clustering images from a group into similar events, said system comprising:

(a) a first module for dividing each image into a plurality of blocks, thereby providing block-based images, said first module then utilizing a block-by-block comparison to segment each block-based image into a plurality of regions comprising at least a foreground and a background;

(b) a second module for extracting one or more features from the regions comprising the foreground and background, said features including at least one of luminosity, color, position and size of the regions;

(c) a third module for utilizing the features to compute the similarity of the regions comprising the foreground and background of successive images in the group, whereby said similarity includes a component to account for the relative sizes of the regions, said third module computing a mean value measure of the total similarity between successive images, thereby providing a measure of image distance between successive images; and

(d) a fourth module for delimiting event clusters from the image distances, whereby the event clusters include groups of images pertaining to the same events.