Method and apparatus to distinguish deposit and removal in surveillance videio

• US 20020141637A1
• Filed: 03/28/2001
• Published: 10/03/2002
• Est. Priority Date: 03/28/2001
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Philips Electronics North America Corporation (Koninklijke Philips N.V.)
• Inventors: Lyons, Damian M., Brodsky, Tomas, Trajkovic, Miroslav, Cohen, Robert A., Cohen-Solal, Eric
• Granted Patent: US 6,731,805 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A system and method that detects and identifies an event from a sequence of images received. The system comprises a control unit that receives and processes the sequence of images from the camera. The control unit identifies at least one stationary image prior to the event and one stationary image subsequent to the event. The control unit compares one or more representations corresponding to one or more edges in the first stationary image with one or more representations corresponding to one or more edges in the second stationary image to determine whether an object has been deposited or removed during the event.

Claims

1. A system that detects and identifies an event from a sequence of images received, the system comprising a control unit that receives and processes the sequence of images from the camera, the control unit identifying at least one stationary image prior to the event and one stationary image subsequent to the event, the control unit comparing one or more representations corresponding to one or more edges in the first stationary image with one or more representations corresponding to one or more edges in the second stationary image to determine whether an object has been deposited or removed during the event.

2. The system as in claim 1, wherein the control unit creates an outline of the first stationary image that comprises the edges in the first stationary image.

3. The system as in claim 2, wherein the control unit identifies and creates an electronic representation of edges in the second stationary image and compares the electronic representations of edges of the second stationary image and the outline of the first stationary image.

4. The system as in claim 3, wherein the control unit determines that a removal of an object has occurred when there are one or more portions of the outline of the first stationary image that have no comparable electronic representation of edges in the second stationary image.

5. The system as in claim 3, wherein the control unit determines a net length of one or more curves comprising the outline of the first stationary image and determines a net length of the one or more edges of the second stationary image and compares the net length of the one or more curves of the first stationary image with the net length of the one or more edges of the second stationary image.

6. The system as in claim 5, wherein the control unit determines that a removal of an object has occurred when the net length of the one or more curves of the first stationary image has no comparable net length of the one or more edges in the second stationary image.

7. The system as in claim 3, wherein the control unit creates one or more templates from the one or more edges of the second stationary image and compares the templates with one or more areas comprised of one or more curves comprising the outline of the first stationary image.

8. The system as in claim 7, wherein the control unit determines that a removal of an object has occurred when an area enveloped by the outline of the first stationary image is not covered by a template created from the second stationary image.

9. A system that detects and identifies an event from a sequence of images received, the system comprising a control unit that receives and processes the sequence of images from the camera, the control unit identifying at least one stationary image selected from the group of stationary images prior and subsequent to the event, the control unit analyzing an identified boundary location in the at least one stationary image to determine whether an object has been deposited or removed during the event.

10. The system as in claim 9, wherein the control unit integrates the image gradient around the identified boundary location in a subsequent stationary image and compares the integral with a threshold value to determine whether the object has been deposited or removed during the event.

11. The system as in claim 10, wherein the control unit determines that a removal event has occurred when the integral of the image gradient is less than the threshold value.

12. The system as in claim 9, wherein the control unit integrates the image gradient around the identified boundary location in a prior stationary image and compares the integral with a threshold value to determine whether the object has been deposited or removed during the event.

13. The system as in claim 12, wherein the control unit determines that a removal event has occurred when the integral of the image gradient is greater than the threshold value.

14. The system as in claim 9, wherein the control unit integrates the image gradient around the identified boundary location in each of a prior stationary image and a subsequent stationary image and compares the integrals of the prior stationary image and the subsequent stationary image to determine whether the object has been deposited or removed during the event.

15. The system as in claim 14, wherein the control unit determines that a removal event has occurred when the integral of the image gradient of the prior stationary image is greater than the image gradient of the subsequent stationary image.

16. The system as in claim 9, wherein the control unit compares the color of at least one pixel on the interior of the boundary location with the color of at least one pixel on the exterior of the boundary location to determine whether the object has been deposited or removed during the event.

17. The system as in claim 16, wherein the image is a stationary image subsequent to the event and the control unit determines that there is a removal event if the color of the at least one pixel on the interior of the boundary location differs from the color of the at least one pixel on the exterior of the boundary location by less than a threshold value.

18. The system as in claim 16, wherein the image is a stationary image prior to the event and the control unit determines that there is a removal event if the color of the at least one pixel on the interior of the boundary location differs from the color of the at least one pixel on the exterior of the boundary location by more than a threshold value.

19. The system as in claim 9, wherein the control unit compares a color histogram for a number of pixels on the interior of the boundary location with a color histogram for a number of pixels on the exterior of the boundary location to determine whether the object has been deposited or removed during the event.

20. The system as in claim 19, wherein the image is a stationary image subsequent to the event and the control unit determines that there is a removal event if the color distribution of the histogram for the pixels on the interior of the boundary location corresponds to the color distribution of the histograms for the pixels on the exterior of the boundary location by greater than a threshold value.

21. The system as in claim 19, wherein the image is a stationary image prior to the event and the control unit determines that there is a removal event if the color distribution of the histogram for the pixels on the interior of the boundary location corresponds to the color distribution of the histograms for the pixels on the exterior of the boundary location by less than a threshold value.

22. A method for detecting and identifying an event from a sequence of received images, the method comprising the steps of:

a) identifying at least one stationary image prior to the event and one stationary image subsequent to the event,

b) comparing one or more representations corresponding to one or more edges in the first stationary image with one or more representations corresponding to one or more edges in the second stationary image; and

c) determining whether an object has been deposited or removed during the event based on the comparison of step b).

23. The method of claim 22, including the step of generating an outline of the first stationary image for the one or more representations corresponding to one or more edges in the first stationary image.

24. The method of claim 23, including the step of determining a net length of one or more curves of the one or more representations corresponding to one or more edges in the first stationary image and the step of determining a net length of the one or more edges of the second stationary image for use as the one or more representations corresponding to one or more edges in the second stationary image.

25. The method of claim 23, including the step of creating one or more templates from the one or more edges of the second stationary image for use as the one or more representations corresponding to one or more edges in the second stationary image

27. The method of claim 26, wherein the step of analyzing an identified boundary location in the at least one stationary image comprises integrating the image gradient around the identified boundary location in an image.

28. The method of claim 27, wherein the step of determining whether an object has been deposited or removed during the event comprises comparing the integral of the image gradient with a threshold value.

29. The method of claim 26, wherein the step of identifying comprises identifying one stationary image prior to the event and one stationary image subsequent to the event, the step of analyzing comprises integrating the image gradient around the identified boundary location in the stationary image prior to the event and integrating the image gradient around the identified boundary location in the stationary image subsequent to the event, and the step of determining includes comparing the integral of the image gradient for the stationary image prior to the event with the integral of the image gradient for the stationary image subsequent to the event.

30. The method of claim 26, wherein the step of analyzing an identified boundary location in the at least one stationary image comprises determining a color of at least one pixel on the interior of the boundary location of an image with the color of at least one pixel on the exterior of the boundary of the image.

31. The method of claim 30, wherein the step of determining whether an object has been deposited or removed during the event comprises comparing the color of the at least one pixel on the interior of the boundary location with the color of the at least one pixel on the exterior of the boundary.

32. The method of claim 26, wherein the step of analyzing an identified boundary location in the at least one stationary image comprises creation of a color histogram for a number of pixels on the interior of the boundary location of an image and a color histogram for a number of pixels on the exterior of the boundary location of the image.

33. The method of claim 32, wherein the step of determining whether an object has been deposited or removed during the event comprises comparing the color histogram for the interior of the boundary location with the color histogram for the exterior of the boundary location.

26. A method for detecting and identifying an event from a sequence of received images, the method comprising the steps of:

a) receiving and processing the sequence of images from the camera;

b) identifying at least one stationary image selected from the group of stationary images prior and subsequent to the event;

c) analyzing an identified boundary location in the at least one stationary image; and

d) determining whether an object has been deposited or removed during the event.