Moving object detection detection within a video stream using object texture

• US 8,559,670 B2
• Filed: 02/04/2010
• Issued: 10/15/2013
• Est. Priority Date: 11/19/2009
• Status: Expired due to Fees

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Wang, Jian-Cheng, Lien, Cheng-Chang, Huang, Ya-Lin, Jiang, Yue-Min
• Primary Examiner(s): Wang, Claire X
• Application Number:
  US12/700,706
• Publication Number:
  US 20110116682A1
• Time in Patent Office: 1,349 Days
• Field of Search: 382/103, 382/104, 382/170, 382/173, 382/228
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20032 Median filtering
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/44 using image operators, e.g. filters, edge density metrics or local histograms
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

An object detection method and an object detection system, suitable for detecting moving object information of a video stream having a plurality of images, are provided. The method performs a moving object foreground detection on each of the images, so as to obtain a first foreground detection image comprising a plurality of moving objects. The method also performs a texture object foreground detection on each of the images, so as to obtain a second foreground detection image comprising a plurality of texture objects. The moving objects in the first foreground detection image and the texture objects in the second foreground detection image are selected and filtered, and then the remaining moving objects or texture objects after the filtering are output as real moving object information.

Claims

1. An object detection method, suitable for detecting moving objection information of a video stream comprising a plurality of images, the method comprising:

performing a moving object foreground detection on each of the images to obtain a first foreground detection image comprising a plurality of moving objects;

performing a texture object foreground detection on each of the images to obtain a second foreground detection image comprising a plurality of texture objects; and

filtering the moving objects in the first foreground detection image and the texture objects in the second foreground image, and outputting the remaining moving objects or texture objects after the filtering as the moving object information,

wherein the step of performing the texture object foreground detection on each of the images to obtain the second foreground detection image comprising the plurality of texture objects comprises;

for a plurality pixels in the consecutive images calculating a local binary pattern for each pixel in each image and using the local binary pattern as texture information of the pixel;

comparing the texture information of each of the pixels in adjacent images to determine whether the pixel is a texture pixel; and

labeling the texture pixels which are connected as the texture object and gathering the texture objects to form the second foreground detection image.

2. The object detection method according to claim 1, wherein the step of performing the moving object foreground detection on each of the images to obtain the first foreground detection image comprising the plurality of moving objects comprises:

establishing a background probability model by using a plurality of consecutive images in the video stream; and

performing the moving object foreground detection on other images by using the background probability model to obtain the first foreground detection image.

3. The object detection method as according to claim 2, wherein the step of establishing the background probability model by using the plurality of consecutive images in the video stream comprises:

for a plurality of pixels in the consecutive images calculating a mean and a variation of pixel values of each pixel in the images; and

establishing the background probability model by using the mean and the variation of each of the pixels, and using the background probability model to perform the moving object foreground detection on other images.

4. The object detection method according to claim 2, wherein the step of performing the moving object foreground detection on other images by using the background probability model to obtain the first foreground detection image comprises:

placing a plurality of color characteristics of each of the pixels in the background probability model for comparison to obtain a plurality of comparison results;

voting for the comparison results to determine whether the pixel is a moving pixel; and

labeling the moving pixels which are connected as the moving object and gathering the moving objects to form the first foreground detection image.

5. The object detection method according to claim 4, wherein the step of labeling the moving pixels which are connected as the moving object further comprising:

calculating a number of the connected moving pixels and comparing the number with a threshold;

labeling the connected moving pixels as the moving object when the calculated number of the connected moving pixels is larger than or equal to the threshold; and

not labeling the connected moving pixels as the moving object when the calculated number of the connected moving pixels is less than the threshold.

6. The object detection method according to claim 1, wherein the step of calculating the local binary pattern for each pixel in each image comprises:

calculating differences between a plurality of neighboring pixels around the pixel and the pixel itself; and

classifying the differences into two binary values by using a dichotomy to form the local binary pattern of the pixel.

7. The object detection method according to claim 1, wherein the step of comparing the texture information of each of the pixels in adjacent images to determine whether the pixel is the texture pixel comprises:

calculating a number of the pixels having different binary values in the local binary patterns of neighboring images and comparing the number with a threshold; and

determining that the pixel is the texture pixel when the number of pixels is larger than the threshold.

8. The object detection method according to claim 1, wherein the step of labeling the texture pixels which are connected as the texture object further comprising:

calculating a number of the connected texture pixels and comparing the number with a threshold;

labeling the connected texture pixels as the texture object when the calculated number of the connected texture pixels is larger than or equal to the threshold; and

not labeling the moving pixels as the texture object when the calculated number of the connected texture pixels is less than the threshold.

9. The object detection method according to claim 1, wherein the step of filtering the moving objects in the first foreground detection image and the texture objects in the second foreground image, and outputting the remaining moving objects or texture objects after the filtering as the moving object information comprises:

filtering out the moving objects in the first foreground detection image having positions overlapped with the texture objects in the second foreground image, and outputting the remaining moving objects after the filtering as the moving object information.

10. The object detection method according to claim 9, wherein the step of filtering out the moving objects in the first foreground detection image having positions overlapped with the texture objects in the second foreground image comprises:

verifying an area covered by the moving objects and an area covered by the texture objects, so as to remove a portion of the area covered by the moving objects that is overlapped with the texture objects; and

filtering the moving objects according to the verified area covered by the moving objects.

11. The object detection method according to claim 9, wherein the step of filtering the moving objects according to the verified area covered by the moving objects comprises:

calculating a reliability indicating whether each of the moving objects after the verification is existed or not; and

filtering the moving objects according to the calculated reliabilities.

12. The object detection method according to claim 11, wherein the reliability used for determining whether the moving object is existed or not comprises a number of the moving pixels in the moving object.

13. The object detection method according to claim 12, wherein the step of filtering the moving objects according to the calculated reliabilities comprises:

comparing each of the calculated reliabilities with a reliability threshold;

regarding the moving object as existed and reserving the information of the moving object when the corresponding reliability is larger than or equal to the reliability threshold; and

regarding the moving object as not existed and removing the information of the moving object when the corresponding reliability is less than the reliability threshold.

14. The object detection method according to claim 1, wherein the step of filtering the moving objects in the first foreground detection image and the texture objects in the second foreground image, and outputting the remaining moving objects or texture objects after the filtering as the moving object information comprises:

filtering out the texture objects in the second foreground detection image having positions overlapped with the moving objects in the first foreground image, and outputting the remaining texture objects after the filtering as the moving object information.

15. The object detection method according to claim 1, wherein before the step of performing the moving object foreground detection and the texture object foreground detection on each of the images, the method further comprises:

reducing a resolution of each of the images and performing the moving object foreground detection and the texture object foreground detection on each of the images having a reduced resolution.

16. A non-transitory computer-readable medium encoded with a computer program comprising:

a moving object detection module, performing a moving object foreground detection on each of a plurality of images in a video stream captured by an image capturing device to obtain a first foreground detection image comprising a plurality of moving objects;

a texture object detection module, performing a texture object foreground detection on each of the images to obtain a second foreground detection image comprising a plurality of texture objects; and

an object filtering module, filtering the moving objects in the first foreground detection image and the texture objects in the second foreground image, and outputting the remaining moving objects or texture objects after the filtering as the moving object information,

wherein the texture object detection module comprises;

a dynamic texture model establishing unit, a plurality of pixels in the consecutive images calculating a local binary pattern for each pixel in each image and using the local binary pattern as texture information of the pixel;

a texture comparing unit, comparing the texture information of each of the pixels in adjacent images to determine whether the pixel is a texture pixel; and

a connected component labeling unit, labeling the texture pixels which are connected as the texture object and gathering the texture objects to form the second foreground detection image.

17. The non-transitory computer-readable medium according to claim 16, wherein the moving object detection module comprises:

a background probability model establishing unit, a plurality of pixels in the consecutive images calculating a mean and a variation of the pixel values of each pixel in the images; and

a characteristic comparing unit, placing a plurality of color characteristics of each of the pixels in the background probability model for comparison to obtain a plurality of comparison results;

a voting unit, voting for the comparison results obtained by the characteristic comparing unit to determine whether the pixel is a moving pixel; and

a connected component labeling unit, labeling the connected moving pixels as the moving object and gathering the moving objects to form the first foreground detection image.

18. The non-transitory computer-readable medium according to claim 16, wherein the object filtering module comprises filtering out the moving objects in the first foreground detection image having positions overlapped with the texture objects in the second foreground image, and outputting the remaining moving objects after the filtering as the moving object information.

19. The non-transitory computer-readable medium according to claim 16, wherein the object filtering module comprises:

a verification unit, verifying an area covered by the moving objects and an area covered by the texture objects, so as to remove a portion of the area covered by each of the moving objects that is overlapped with the texture objects; and

a filtering unit, filtering the moving objects according to the area covered by the moving objects verified by the verification unit.

20. The non-transitory computer-readable medium according to claim 19, wherein the filtering unit comprises calculating a reliability indicating whether each of the moving objects after the verification is existed or not, and filtering the moving objects according to the reliabilities.

21. The non-transitory computer-readable medium according to claim 20, wherein the reliability used for determining whether the moving object is existed or not comprises a number of the moving pixels in the moving object.

22. The non-transitory computer-readable medium according to claim 16, wherein the object filtering module comprises filtering out the texture objects in the second foreground detection image having positions overlapped with the moving objects in the first foreground image, and outputting the remaining texture objects after the filtering as the moving object information.

23. The non-transitory computer-readable medium according to claim 16, wherein the image capturing device further comprises reducing a resolution of each of the images in the captured video stream and providing the images having a reduced resolution for the processing device to perform the moving object foreground detection and the texture object foreground detection.