Method and Apparatus for Determining Motion in Images

• US 20080037869A1
• Filed: 07/13/2006
• Published: 02/14/2008
• Est. Priority Date: 07/13/2006
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Zhou, Hui
• Granted Patent: US 7,783,118 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20152 Watershed segmentation
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A method of determining motion in a set of input images comprises registering the images in the set to identify stationary background content therein and generating difference images based on the registered images. The input images are segmented into regions and local motion within each of the regions is estimated. A determination as to whether the regions include moving content or static content is determined based on the difference images and the estimated local motion within the regions.

Claims

1. A method of determining motion in a set of input images, said method comprising:

registering the images in said set to identify stationary background content therein;

generating difference images based on the registered images;

segmenting the input images into regions;

estimating local motion within each of said regions; and

determining whether the regions include moving content or static content based on the difference images and the local estimated motion within said regions.

2. The method of claim 1 wherein during said registering, one of said input images is identified as a reference image, the remaining input images in said set being registered thereto.

3. The method of claim 2 wherein said difference image generating comprises determining the difference between each remaining image in said set and said reference image to yield absolute intensity difference images.

4. The method of claim 1 further comprising binarizing the difference images.

5. The method of claim 4 further comprising noise filtering the binarized difference images.

6. The method of claim 5 wherein during said registering, one of said input images is identified as a reference image, the remaining input images in said set being registered thereto.

7. The method of claim 6 wherein said difference image generating comprises determining the difference between each remaining image in said set and said reference image to yield absolute intensity difference images.

8. The method of claim 7 wherein said noise filtering is performed by morphing said binarized images.

9. The method of claim 1 wherein said segmenting comprises applying the Watershed algorithm to said input images and compensating for over-segmentation of said input images.

10. The method of claim 9 wherein said compensating comprises merging each region of each input image that is below a threshold size with a boundary region.

11. The method of claim 10 wherein for each region, said boundary region is selected from one of a region greater than said threshold size and the largest boundary region smaller than said threshold size.

12. The method of claim 11 further comprising binarizing the difference images.

13. The method of claim 1 wherein said local motion estimating comprises generating a motion vector for each region in each of said images with the exception of the last image in said set.

14. The method of claim 13 wherein said motion vector generating for each region comprises:

for each pixel in the region, calculating the absolute difference between the pixel and a corresponding pixel in the subsequent input image in said set;

summing the calculated absolute differences;

shifting the position of the region according to a search pattern and for each position in the search pattern reperforming the calculating and summing; and

selecting the motion vector associated with the position of the region that yields the minimum absolute difference sum.

15. The method of claim 13 wherein said region content determining for each region comprises:

determining whether the motion vector for the region signifies movement above a threshold and if so labeling the region as representing moving content;

if not, determining whether the percentage of pixels in the region have a pixel value above a threshold in the corresponding difference image and if so labeling the region as representing moving content; and

if not labeling the region as representing static content.

16. The method of claim 15 further comprising refining the region labeling based on classification of corresponding regions in other input images and classification of neighbouring regions.

17. The method of claim 16 wherein said refining is based on temporal and spatial continuity of pixels in said regions.

18. The method of claim 13 wherein for said last input image in the set, the local motion estimating comprises projecting motion shifted pixels from regions of the second last input image onto the last input image and wherein the region content determining comprises for each corresponding region in the last input image determining whether a threshold number of projected motion shifted pixels fall within the region.

19. A method of determining motion in a set of input images, said method comprising:

designating one of the images in said set as a reference image;

creating a set of dominant motion compensated images from said set of images based on differences between the remaining images in said set and said reference image;

creating a set of intensity difference images from said dominant motion compensated images based on differences between the dominant motion compensated images and said reference image;

segmenting each of said input images into regions;

generating motion vectors for said regions based upon aggregate motion of pixels in each region;

classifying each region in each of the images in said set with the exception of the last image as including moving or static content on the basis of at least one of the motion vectors and the corresponding intensity difference images; and

using backward tracking to classify the regions in said last image.

20. An apparatus for determining motion in a set of input images, said apparatus comprising:

an image alignor registering the images in said set to identify stationary background content therein;

a comparator generating difference images based on the registered images;

a segmentor segmenting the input images into regions; and

an estimator estimating local motion within each of said regions and determining whether the regions include moving content or static content based on the difference images and the local estimated motion within said regions.