IMAGE SURVEILLANCE SYSTEM AND METHOD OF DETECTING WHETHER OBJECT IS LEFT BEHIND OR TAKEN AWAY

• US 20120183177A1
• Filed: 01/17/2012
• Published: 07/19/2012
• Est. Priority Date: 01/17/2011
• Status: Active Grant

• Current Assignee: Hanwha Vision Company Limited (Hanwha Corporation), Postech Academy-Industry Foundation
• Original Assignee: Samsung Techwin Co., Ltd. (Hanwha Corporation), Postech Academy-Industry Foundation
• Inventors: Ku, Ja-yeong, Lim, Jeong-eun, Kim, Ji-man, Kim, Dai-jin
• Granted Patent: US 8,712,099 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion

Abstract

An image surveillance system and a method of detecting whether an object is left behind or taken away are provided. The image surveillance system includes: a foreground detecting unit which detects a foreground region based on a pixel information difference between a background image and a current input image; a still region detecting unit which detects a candidate still region by clustering foreground pixels of the foreground region, and determines whether the candidate still region is a falsely detected still region or a true still region; and an object detecting unit which determines whether an object is left behind or taken away, based on edge information about the true still region.

Claims

1. An image surveillance system comprising:

a foreground detecting unit which detects a foreground region based on a difference between pixel information about a background image and pixel information about a current input image;

a still region detecting unit which detects a candidate still region by clustering foreground pixels of the foreground region, and determines whether the candidate still region is a falsely detected still region or a true still region; and

an object detecting unit which determines whether an object is left behind or taken away, based on edge information about the true still region.

2. The image surveillance system of claim 1, wherein the foreground detecting unit comprises:

a background generating unit which obtains and stores pixel values of respective pixels in a predetermined number of image frames, generates the background image by using a pixel value which is obtained most frequently with respect to each of the pixels from among the stored pixel values;

a foreground region detecting unit which determines, as one of the foreground pixels, a pixel of which a pixel value difference between the background image and the current input image is equal to or greater than a first threshold value; and

a shadow removing unit which removes a shadow pixel from the foreground pixels, based on hue and brightness information about the foreground pixels.

3. The image surveillance system of claim 1, wherein the still region detecting unit comprises:

a candidate still region detecting unit which sets, as one of still pixels, a pixel which is detected as the foreground pixel for a predetermined period of time, and clusters the still pixels to detect the candidate still region; and

a false-detection determining unit which determines whether the candidate still region satisfies at least one of a plurality of conditions for determining whether the candidate still region is falsely detected,

wherein the plurality of conditions use at least one of an area of another foreground region with respect to an area of the candidate still region, variance of the area of the foreground region in the predetermined period of time, variance in a coordinate of the foreground region, a sustained time period for which the candidate still region is detected as the candidate still region, the size of the candidate still region, a motion of the candidate still region, darkness change of the candidate still region, and a relationship between an average pixel value of the candidate still region and an average pixel value of a background region of the background image corresponding to the candidate still region, and

wherein the still region detecting unit determines the candidate still region as the true still region if the candidate still region satisfies the at least one of the plurality of conditions.

4. The image surveillance system of claim 3, wherein the information about the candidate still region comprises coordinates of top left and bottom right pixels of the candidate still region, the size of the candidate still region, the average pixel value of the candidate still region, the average pixel value of the background region corresponding to the candidate still region, and the sustained time period for which the candidate still region is detected as the candidate still region, and

wherein the information about the candidate still region of the current input image is updated based on a information about a candidate still region of a previous input image.

5. The image surveillance system of claim 3, wherein the determining unit comprises:

a boundary region extracting unit which performs an exclusive logic sum (XOR) calculation on a dilation region generated by dilating the true still region and a reversed erosion region generated by eroding and reversing the true still region to extract a boundary region of the true still region in the current input image;

an edge extracting unit which extracts edge information about the boundary region of the true still region in the current input image and edge information about a boundary region of the background image in a previous input image; and

a determining unit which determines whether the object is left behind or taken away based on a change between an edge intensity of the true still region in the current input image and an edge intensity of the background image in the previous input image.

6. The image surveillance system of claim 3, wherein the determining unit determines that the object is left behind if the edge intensity of the true still region in the current image increases compared to the edge intensity of the background image in a previous input image, and

wherein the determining unit determines that the object is taken away if the edge intensity of the still region in the current input image decreases compared to the edge intensity of the background image in the previous input image.

7. The image surveillance system of claim 1, wherein the still region detecting unit comprises:

a candidate still region detecting unit which sets, as one of still pixels, a pixel which is detected for a predetermined period of time as the foreground pixel, and clusters the still pixels to detect the candidate still region; and

a false-detection determining unit which determines whether the candidate still region satisfies a plurality of conditions for determining whether the candidate still region is falsely detected,

wherein the plurality of conditions use an area of another foreground region with respect to an area of the candidate still region, variance of the area of the foreground region in the predetermined period of time, variance in a coordinate of the foreground region, a sustained time period for which the candidate still region is detected as the candidate still region, the size of the candidate still region, a motion of the candidate still region, darkness change of the candidate still region, and a relationship between an average pixel value of the candidate still region and an average pixel value of a background region of the background image corresponding to the candidate still region, respectively, and

wherein the still region detecting unit determines the candidate still region as the true still region if the candidate still region satisfies all of the plurality of conditions.

8. The image surveillance system of claim 7, wherein, if a difference in the average pixel values between the candidate still region and the background region is equal to or greater than a second threshold value, the candidate still region is determined as the true still region.

9. The image surveillance system of claim 1, wherein the object detecting unit comprises:

a boundary region extracting unit which performs an exclusive logic sum (XOR) calculation on a dilation region generated by dilating the true still region and a reversed erosion region generated by eroding and reversing the true still region to extract a boundary region of the true still region in the current input image;

an edge extracting unit which extracts edge information about the boundary region of the true still region in the current input image and edge information about a boundary region of the background image in a previous input image; and

a determining unit which detects whether the object is left behind or taken away based on a change between an edge intensity of the true still region in the current input image and an edge intensity of the background image in the previous input image.

10. The image surveillance system of claim 1, wherein the object detecting unit determines that the object is left behind if an edge intensity of the true still region in the current input image increases compared to an edge intensity of the background image in a previous input image, and

wherein the determining unit determines that the object is taken away if the edge intensity of the true still region in the current input image decreases compared to the edge intensity of the background image in the previous input image.

11. An image surveillance system comprising:

a foreground detecting unit which detects a plurality of foreground regions based on difference between pixel information about a background image and pixel information about a current input image;

a still region detecting unit which detects a plurality of candidate still regions from among the plurality of foreground regions, and selects a true still region by removing a falsely detected still region, from among the plurality of candidate still regions; and

an object detecting unit which determines whether an object corresponding to the true still region is an object which is left behind or taken away, based on edge information about the true still region.

12. The image surveillance system of claim 11, wherein the still region detecting unit comprises:

a candidate still region detecting unit which sets, as one of still pixels, a pixel which is detected for a predetermined period of time as a foreground pixel of the foreground regions, and clusters the still pixels to detect the plurality of candidate still regions; and

a false-detection determining unit which determines whether the plurality of candidate still regions are falsely detected, based on at least one of a plurality of conditions which uses an area of each of the foreground regions with respect to an area of each of the candidate still regions, variance of each of the area of the foreground regions in the predetermined period of time, variance in a coordinate of each of the foreground regions, a sustained time period for which each of the candidate still regions is detected as the candidate still region, the size of each of the candidate still regions, a motion of each of the candidate still regions, darkness change of each of the candidate still regions, and a relationship between an average pixel value of each of the candidate still regions and an average pixel value of a background region of the background image corresponding to each of the candidate still regions, respectively, and

wherein the still region detecting unit determines at least one of the candidate still regions as the true still region if the at least one candidate still region satisfies the at least one of the plurality of conditions.

13. The image surveillance system of claim 11, wherein the object detecting unit comprises:

a boundary region extracting unit which performs an exclusive logic sum (XOR) calculation on a dilation region generated by dilating the true still region and a reversed erosion region generated by eroding and reversing the true still region to extract a boundary region of the true still region in the current input image;

an edge extracting unit which extracts edge information about the boundary region of the true still region in the current input image and edge information about a boundary region of the background image in a previous input image; and

a determining unit which detects whether the object corresponding to the true still region is left behind or taken away based on a change between an edge intensity of the true still region in the current input image and an edge intensity of the background image in the previous input image.

14. A method of detecting whether an object is left behind or taken away using an image surveillance system, the method comprising:

detecting a foreground region based on difference between pixel information about a background image and pixel information about a current input image;

detecting a candidate still region by clustering foreground pixels of the foreground region, and determining whether the candidate still region is a falsely detected still region or a true still region; and

determining whether an object is left behind or taken away, based on edge information about the true still region.

15. The method of claim 14, wherein the detecting the foreground region comprises:

obtaining and storing pixel values for respective pixels in a predetermined number of image frames, generating the background image by using a pixel value which is obtained most frequently with respect to each of the pixels from among the stored pixel values;

determining, as one of the foreground pixels, a pixel of which a pixel value difference between the background image and the current input image is equal to or greater than a first threshold value; and

removing a shadow pixel from the foreground pixels, based on hue and brightness information about the foreground pixels.

16. The method of claim 14, wherein the determining whether the object is left behind or taken away comprises:

setting, as one of still pixels, a pixel which is detected as the foreground pixel for a predetermined period of time;

clustering the still pixels to detect the candidate still region; and

determining whether the candidate still region satisfies at least one of a plurality of conditions for determining whether the candidate still region is falsely detected,

wherein the plurality of conditions use at least one of an area of the foreground region with respect to an area of the candidate still region, variance of the area of the foreground region in the predetermined period of time, variance in a coordinate of the foreground region, a sustained time period for which the candidate still region is detected as the candidate still region, the size of the candidate still region, a motion of the candidate still region, darkness change of the candidate still region, and a relationship between an average pixel value of the candidate still region and an average pixel value of a background region of the background image corresponding to the candidate still region, and

wherein the candidate still region is determined as the true still region if the candidate still region satisfies the at least one of the plurality of conditions.

17. The method of claim 16, wherein the information about the candidate still region comprises coordinates of top left and bottom right pixels of the candidate still region, the size of the candidate still region, the average pixel value of the candidate still region, the average pixel value of the background region corresponding to the candidate still region, and the sustained time period for which the candidate still region is detected as the candidate still region, and

wherein the information about the candidate still region of the current input image is updated based on a information about a candidate still region of a previous input image.

18. The method of claim 16,

wherein, if a difference in the average pixel values between the candidate still region and the background region is equal to or greater than a second threshold value, the candidate still region is determined as the true still region.

19. The method of claim 14, wherein the determining whether the object is left behind or taken away comprises:

performing an exclusive logic sum (XOR) calculation on a dilation region generated by dilating the true still region and a reversed erosion region generated by eroding and reversing the true still region to extract a boundary region of the true still region;

extracting edge information about the boundary region of the true still region in the current image and edge information about a boundary region of the background image in a previous input image; and

detecting whether the object is left behind or taken away based on a change between an edge intensity of the true still region in the current input image and an edge intensity of the background image in the previous input image.

20. The method of claim 14, wherein the determining whether the object is left behind or taken away comprises:

determining that the object is left behind if an edge intensity of the true still region in the current input image increases compared to an edge intensity of the background image in a previous input image; and

determining that the object is taken away if the edge intensity of the true still region in the current input image decreases compared to the edge intensity of the background image in the previous input image.