Master-slave automated video-based surveillance system

• US 20050134685A1
• Filed: 12/22/2003
• Published: 06/23/2005
• Est. Priority Date: 12/22/2003
• Status: Abandoned Application

• Current Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Original Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Inventors: Haering, Niels, Zhang, Zhong, Venetianer, Peter L., Lipton, Alan J., Yin, Weihong, Egnal, Geoffrey, Chosak, Andrew
• Application Number:
  US10/740,511
• Publication Number:
  US 20050134685A1
• Time in Patent Office: Days
• US Class Current: 348/157
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A video surveillance system comprises a first sensing unit; a second sensing unit; and a communication medium connecting the first sensing unit and the second sensing unit. The first sensing unit provides information about a position of a target to the second sensing unit via the communication medium, and the second sensing unit uses the position information to locate the target.

Claims

1. A video surveillance system comprising:

a first sensing unit;

a second sensing unit; and

a communication medium connecting the first sensing unit and the second sensing unit;

wherein the first sensing unit provides information about a position of a target to the second sensing unit via the communication medium, the second sensing unit using the position information to locate the target.

2. The video surveillance system of claim 1, further comprising:

a third sensing unit, wherein the third sensing unit provides further position information to the second sensing unit.

3. The video surveillance system of claim 2, further comprising:

a fourth sensing unit, wherein the fourth sensing unit receives and utilizes position information received from the third sensing unit to locate a target.

4. The video surveillance system according to claim 2, wherein the second sensing unit employs a conflict resolution algorithm to determine whether to utilize position information from the first sensing unit or from the third sensing unit.

5. The video surveillance system of claim 1, wherein the second sensing unit provides position information to the first sensing unit via the communication medium, the first sensing unit using the position information to locate the target.

6. The video surveillance system of claim 1, wherein the first sensing unit comprises:

a sensing device;

a vision module to process output of the sensing device;

an inference module to process output of the vision module; and

a response module to perform one or more actions based on the output of the inference module.

7. The video surveillance system of claim 6, wherein the sensing device comprises at least one of a camera, an infra-red sensor, and a thermal sensor.

8. The video surveillance system of claim 6, wherein the vision module detects at least one of blobs and targets.

9. The video surveillance system of claim 6, wherein the vision module comprises:

a change detection module to separate background pixels from foreground pixels;

a blobizer to receive the foreground pixels from the change detection module and to determine coherent blobs;

a target tracker to process the coherent blobs, determine when they are targets, and to obtain position information for each target;

a classifier to determine a target type for each target; and

a primitive generation module to generate summary statistics to be sent to the inference module.

10. The video surveillance system of claim 6, wherein the inference module determines when at least one specified condition has been either met or violated.

11. The video surveillance system of claim 6, wherein the response module is adapted to perform at least one of the following:

sending an e-mail alert;

sounding an audio alarm;

providing a visual alarm;

transmitting a message to a personal digital assistant; and

providing position information to another sensing unit.

12. The video surveillance system of claim 1, wherein the second sensing unit comprises:

a sensing device;

a receiver to receive position information from another sensing unit;

a PTZ controller module to filter and translate the position information received by the receiver into PTZ angles and velocities; and

a PTZ unit physically coupled to the sensing device; and

a response unit to transmit commands to the PTZ unit based on output from the PTZ controller module.

13. The video surveillance system of claim 12, wherein the second sensing unit further comprises:

a vision module to actively track a target based on at least one of position information received by the receiver and information received from the sensing device,

wherein the vision module provides position information derived from its input to the PTZ controller module.

14. A video-based security system, comprising the video surveillance system according to claim 1.

15. A video-based system for monitoring a scientific experiment, comprising the video surveillance system according to claim 1.

16. A video-based system for monitoring a sporting event, comprising the video surveillance system according to claim 1.

17. A video-based marketing information system, comprising the video surveillance system according to claim 1.

56. The apparatus of claim 1, wherein said first sensing unit comprises application specific hardware to emulate a computer and/or software, wherein said hardware is adapted to perform said video surveillance.

57. The apparatus of claim 1, wherein said second sensing unit comprises application specific hardware to emulate a computer and/or software, wherein said hardware is adapted to perform said video surveillance.

18. A method of operating a video surveillance system, the video surveillance system including at least two sensing units, the method comprising the steps of:

using a first sensing unit to detect the presence of a target;

sending position information about the target from the first sensing unit to at least one second sensing unit; and

training the at least one second sensing unit on the target, based on the position information, to obtain a higher resolution image of the target than one obtained by the first sensing unit.

19. The method of claim 18, wherein the step of using a first sensing unit comprises the steps of:

obtaining image information;

processing the image information with a vision module to detect and locate at least one object; and

determining if at least one predetermined condition has been violated by at least one object.

20. The method of claim 19, wherein the step of processing the image information comprises the step of:

geo-locating the at least one object in 3D space.

21. The method of claim 19, wherein the step of processing the image information comprises the steps of:

classifying pixels in the image information as background pixels or foreground pixels; and

using the foreground pixels to determine at least one blob.

22. The method of claim 21, further comprising the step of tracking at least one possible target based on the at least one blob.

23. The method of claim 22, wherein the step of tracking comprises the steps of:

determining when at least one blob merges or splits into one or more possible targets; and

filtering and predicting location of at least one of the possible targets.

24. The method of claim 23, wherein the step of tracking further comprises the step of:

calculating a 3D position of at least one of the possible targets.

25. The method of claim 22, further comprising the step of classifying at least one possible target.

26. The method of claim 25, further comprising the step of providing summary statistics to aid in the step of determining if at least one predetermined condition has been violated by at least one object.

27. The method of claim 18, wherein the step of training the at least one second sensing unit on the target comprises the steps of:

converting the position information received from the first sensing unit into pan-tilt-zoom (PTZ) information; and

converting the PTZ information into control commands to train a sensing device of the at least one second sensing unit on the target.

28. The method of claim 18, wherein the step of training the at least one second sensing unit on the target comprises the steps of:

obtaining second image information using a sensing device of the at least one second sensing unit;

tracking the target using the second image information and the position information received from the first sensing unit;

generating pan-tilt-zoom (PTZ) information based on the results of the tracking step; and

converting the PTZ information into control commands to train the sensing device of the at least one second sensing unit on the target.

29. The method of claim 18, further comprising the steps of:

determining a best shot of the target; and

directing the at least one second sensing unit to obtain the best shot.

30. The method of claim 29, wherein the step of determining a best shot is performed by the first sensing unit.

31. The method of claim 29, wherein the step of determining a best shot is performed by the at least one second sensing unit.

32. The method of claim 29, further comprising the steps of:

zooming in on the target the at least one second sensing unit; and

zooming the at least one second sensing unit back out.

33. The method of claim 18, further comprising the steps of:

feeding back positioning information from the at least one second sensing unit to the first sensing unit; and

utilizing, by the first sensing unit, the fed back positioning information to obtain improved geo-location.

34. The method of claim 18, further comprising the steps of:

tracking the target using the at least one second sensing unit;

if the at least one second sensing unit is unable to track the target, transmitting information from the at least one second sensing unit to the first sensing unit to cause the first sensing unit to track the target.

35. The method of claim 34, further including the steps of:

using the information received from the at least one second sensing unit to obtain pan-tilt-zoom (PTZ) information; and

converting the PTZ information into control commands to train a sensing device of the first sensing unit on the target.

36. A computer-readable medium containing software implementing the method of claim 18.

37. A video surveillance system, comprising:

at least two sensing units;

a computer system; and

the computer-readable medium of claim 36.

38. A video-based security system, comprising the video surveillance system according to claim 36.

39. A video-based system for monitoring a scientific experiment, comprising the video surveillance system according to claim 37.

40. A video-based system for monitoring a sporting event, comprising the video surveillance system according to claim 37.

41. A video-based marketing information system, comprising the video surveillance system according to claim 37.

42. A method of implementing a video-based security system, comprising the method according to claim 18.

43. A method of monitoring a scientific experiment, comprising the method according to claim 18.

44. The method according to claim 43, further comprising:

detecting at least one predetermined behavior of a subject of the experiment.

45. A method of monitoring a sporting event, comprising the method according to claim 18.

46. The method of claim 45, further comprising:

detecting at least one predetermined behavior of a participant in the sporting event.

47. A method of obtaining marketing information, comprising the method according to claim 18.

48. The method of claim 47, further comprising:

monitoring at least one behavior of at least one subject.

49. The method of claim 48, wherein said monitoring comprises:

detecting interest in a given product.

50. The method of claim 49, wherein said detecting interest comprises:

detecting when a customer reaches for the given product.

51. The method of claim 18, further comprising the steps of:

using at least one additional first sensing unit to detect one or more targets;

sending position information about the one or more targets to at least one second sensing unit; and

utilizing a conflict resolution algorithm to determine on which target to train at least one second sensing unit.

52. A method of operating a guidable sensing unit in a video surveillance system,

receiving position information about at least one target from at least two sensing units;

employing a conflict resolution algorithm to select the sensing unit whose position information will be used; and

using the position information to train the guidable sensing unit on a target corresponding to the selected sensing unit.

53. The method according to claim 52, wherein employing a conflict resolution algorithm comprises:

selecting the sensing unit whose position information is received first by the guidable sensing unit.

54. The method according to claim 52, wherein employing a conflict resolution algorithm comprises:

allocating a predetermined period of time during which each sensing unit is selected.

55. The method according to claim 52, wherein employing a conflict resolution algorithm comprises:

selecting a sensing unit having a highest priority.