Automatic object extraction

• US 20030081836A1
• Filed: 10/31/2001
• Published: 05/01/2003
• Est. Priority Date: 10/31/2001
• Status: Active Grant

• Current Assignee: F Poszat Hu LLC (Intellectual Ventures LLC)
• Original Assignee: Infowrap, Inc.
• Inventors: Averbuch, Amir, Miller, Ofer
• Granted Patent: US 7,085,401 B2
• Time in Patent Office: Days
• US Class Current: 382/199
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns

Abstract

A method for automatic, stable and robust object extraction of moving objects in color video frames, achieved without any prior knowledge of the video content. For high rate video, the method includes providing at least a first and a second high frame rate video frames, performing a reciprocal illumination correction of the first and second video frames to yield respective first and second smoothed frames, performing a change detection operation between the first and second smoothed frames to obtain a difference image, and performing a local adaptive thresholding operation on the difference image to generate a binary image containing extracted objects, the local thresholding operation using a weight test to determine a boundary of each of the extracted objects. For an extracted object with a fragmented boundary, the method further comprises re-unifying the boundary. For low rate video, additional steps include: an edge correction applied on the first image to yield a first edge-corrected image, a global thresholding applied to the first edge-corrected image to yield a first binary edge image, and an ANDing operation on the difference image and the first binary edge image to generate a second binary image which is fed to the local adaptive thresholding operation.

Claims

1. A method for automatic extraction of objects in video sequences, each object represented by an array of pixels, the method comprising:

a. providing at least a first and a second high frame rate video frames;

b. performing a reciprocal illumination correction of said first and second video frames to yield respective first and second smoothed frames;

c. performing a change detection operation between said first and second smoothed frames to obtain a difference image; and

d. performing a local adaptive thresholding operation on said difference image to generate a binary image containing extracted objects, said local thresholding operation using a weight test to determine a boundary of each of said extracted objects,

whereby the extraction of said extracted objects by the method is automatic, stable and robust, and whereby no advance knowledge of the video sequences is needed to achieve the extraction.

2. The method of claim 1, wherein said first and second video frames are colored video frames, and wherein said step of providing at least a first and a second video frames is preceded by a logical test for said high frame rate, said logical test checking whether said high rate is at least fifteen frames per second.

3. The method of claim 1, wherein said step of reciprocal illumination correction includes the application of a low pass filter on each of said first and second video frames.

4. The method of claim 3, wherein said low pass filter is selected from the group consisting of bilateral, Gaussian, non-decimated scaling wavelet and spline filters.

5. The method of claim 1, wherein said step of reciprocal illumination correction further includes normalization of pixel values of said first and second smoothed frames.

6. The method of claim 5, wherein said normalization further includes a pixel-by-pixel division of said first and second smoothed frames by respectively said first and second video frames.

7. The method of claim 1, wherein said step of performing a change detection operation includes performing a statistical model-based change detection.

8. The method of claim 1, wherein said step of performing a local adaptive thresholding operation further includes the substep of using the pixels as seeds for a search operation resulting in the determination of said boundary.

9. The method of claim 1, wherein said boundary is fragmented, said method further comprising re-unifying said fragmented boundary.

10. The method of claim 9, wherein said step of performing a local adaptive thresholding further includes performing a region-growing operation.

11. The method of claim 1, wherein said weight test use includes checking for a monotonic change in said weight, said change showing a monotonic increase for pixels inside said boundary, and a monotonic decrease leading to a plateau in said weight for pixels outside said boundary.

12. The method of claim 11, wherein said step of re-unifying is carried out using a contour extraction algorithm.

23. The method of claim 1, wherein said weight test use includes checking for a monotonic change in said weight, said change showing a monotonic increase for pixels inside said boundary, and a monotonic decrease leading to a plateau in said weight for pixels outside said boundary.

24. The method of claim 23, wherein said step of re-unifying is carried out using a contour extraction algorithm.

13. A method for automatic extraction of objects in video sequences, each object represented by an array of pixels, the method comprising:

a. providing at least a first and a second low frame rate video frames;

b. performing a reciprocal illumination correction of said first and second video frames to yield respective first and second smoothed frames;

c. performing an edge detection operation on said first video frame to produce a first edge image;

d. performing a change detection operation between said first and second smoothed frames to obtain a difference image;

e. performing a global thresholding operation on said first edge image to produce a first binary edge image;

f. ANDing said difference image and said first binary edge image to produce a first edge-corrected difference image; and

g. performing a local adaptive thresholding operation on said first edge-corrected difference image to generate a second binary image containing extracted objects, said local thresholding operation using a weight test to determine a boundary of each of said extracted objects,

whereby the extraction of said extracted objects by the method is automatic, stable and robust, and whereby no advance knowledge of the video sequences is needed to achieve the extraction.

14. The method of claim 13, wherein said first and second video frames are colored video frames, and wherein said step of providing at least a first and a second low frame rate video frames is preceded by a logical test for said low frame rate, said logical test checking whether said low rate is less than fifteen frames per second.

15. The method of claim 13, wherein said step of reciprocal illumination correction includes the application of a low pass filter on each of said first and second video frames.

16. The method of claim 15, wherein said low pass filter is selected from the group consisting of bilateral, Gaussian, non-decimated scaling wavelet and spline filters.

17. The method of claim 13, wherein said step of reciprocal illumination correction further includes normalization of pixel values of said first and second smoothed frames.

18. The method of claim 17, wherein said normalization includes a pixel-by-pixel division of said first and second smoothed frames by respectively said first and second video frames.

19. The method of claim 13, wherein said step of performing a change detection operation includes performing a statistical model-based change detection.

20. The method of claim 13, wherein said step of performing a global thresholding operation on said first edge image includes:

iteratively;

i. choosing an initial threshold based on a first average of gray level values of a group of pixels in the image;

ii. computing a new threshold based on a second average of all gray level values that are below said initial threshold, and assigning said new threshold to said initial threshold to get a modified threshold;

iii. computing a third average of all gray level values that are above said intial threshold, and averaging said third average with said modified threshold to get a modified new threshold;

iv. comparing said modified new threshold with said modified threshold; and

v. assigning said modified new threshold to said modified threshold if said modified new and modified thresholds are not equal, and continuing a looping procedure until said modified new threshold and said modified threshold are equal.

21. The method of claim 13, wherein said boundary is fragmented, said method further comprising re-unifying said fragmented boundary.

22. The method of claim 21, wherein said step of performing a local adaptive thresholding further includes performing a region-growing operation.

25. A method for automatic extraction of objects in color video sequences, each object represented by an array of pixels, the method comprising:

a. providing at least a first and a second high frame rate video frames;

b. obtaining a first edge image of said first video frame;

c. obtaining a difference image based on first and second illumination corrected, smoothed frames derived from said first and second video frames respectively;

d. performing a global thresholding operation on said first edge image to produce a first binary edge image;

e. ANDing said difference image and said first binary edge image to produce a first edge-corrected difference image; and

f. based on said first edge-corrected difference image, generating a second binary image containing extracted objects,

whereby the extraction of said extracted objects by the method is automatic, stable and robust, and whereby no advance knowledge of the video sequences is needed to achieve the extraction.