Object tracking and alerts

• US 20070182818A1
• Filed: 05/30/2006
• Published: 08/09/2007
• Est. Priority Date: 09/02/2005
• Status: Active Grant

• Current Assignee: Johnson Controls Tyco IP Holdings LLP (Johnson Controls International plc)
• Original Assignee: Sensormatic Electronics LLC (Johnson Controls International plc)
• Inventors: Buehler, Christopher
• Granted Patent: US 9,036,028 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07C 9/28 the pass enabling tracking or indicating presence
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 13/1966 Wireless systems, other than telephone systems, used to communicate with a camera
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: G08B 13/19697 Arrangements wherein non-video detectors generate an alarm themselves
• CPC Class Codes: G08B 13/248 EAS system combined with another detection technology, e.g. dual EAS and video or other presence detection system
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An integrated surveillance system combining video surveillance and data from other sensor-based security networks is used to identify activities that may require attention.

Claims

1. A system for generating alerts based on events in an environment, the system comprising:

a plurality of cameras for producing video signals representing at least a portion of a environment;

a video-processing module for analyzing the video signals to identify objects within the environment;

a data-input module for receiving non-video data from at least one sensor-based monitoring system, the data representing one or more events relating to at least one of the identified objects; and

an alert module for generating alerts based on at least one of the events.

2. The system of claim 1 wherein the at least one sensor-based monitoring system is a point-of-sale system.

3. The system of claim 1 wherein the at least one sensor-based monitoring system is a electronic article surveillance system.

4. The system of claim 1 further comprising a data storage module for storing rules relating to at least one activity within the environment on which the alerts are based.

5. The system of claim 4 wherein the at least one activity comprises one or more of the identified objects, the events in the environment, the absence of identified objects, and the absence of events.

6. The system of claim 1 further comprising a communications module for transmitting the generated alerts to a remote device.

7. The system of claim 6 wherein the remote device is a hand-held wireless receiver.

8. The system of claim 6 further including a geographic information service module for rendering visual representations of the environment.

9. The system of claim 8 wherein the visual representations of the environment are transmitted to a remote device.

10. The system of claim 9 further comprising annotating the visual representations of the environment.

11. The system of claim 10 wherein the annotations comprise one or more of text, color, a directional indication, and a velocity indication.

12. A method of monitoring activity within an environment, the method comprising:

receiving a plurality of image signals representing at least a portion of the environment;

identifying an object within the image signals;

receiving non-video data representing an event occurring in the environment; and

generating an alert based on the identified object and the event.

13. The method of claim 12 wherein the image signals are received from a plurality of video cameras.

14. The method of claim 12 wherein the objects are people.

15. The method of claim 12 wherein the non-video data is received from a point-of-sale system.

16. The method of claim 12 wherein the non-video data is received from an electronic article surveillance system.

17. The method of claim 12 wherein the alerts are generated based on an identified object violating a rule relating to activity within the environment.

18. The method of claim 12 wherein the alerts are generated based on an event violating a rule relating to activity within the environment.

19. The method of claim 12 further comprising generating a graphical representation of the environment.

20. The method of claim 19 further comprising annotating the graphical representation of the environment with one or more of geographic information system data, color, text, directional indicators and velocity indicators.

21. The method of claim 20 further comprising transmitting the annotated graphical representation of the environment to a remote device.

22. The method of claim 21 wherein the transmission to remote devices comprises wireless transmission.

23. A surveillance system comprising:

a video surveillance subsystem comprising plurality of cameras each having a field of view, each field of view comprising a plurality of camera sub-regions;

a radio-frequency identification subsystem comprising a plurality of stations capable of detecting radio-frequency identification tags within an operational radius; and

an association module in communication with the video surveillance subsystem and the radio-frequency identification subsystem for associating one or more of the camera sub-regions with one or more of the operational radii of the radio-frequency stations.

24. The system of claim 23 wherein the associations are based on locational correspondences between the camera sub-regions and the operational radii.

25. The system of claim 24 where the associations between the camera sub-regions and the operational radii of the stations are inferred from data captured by the video surveillance subsystem.

26. The system of claim 25 wherein the data captured by the video surveillance subsystem comprises visual representations of objects moving among the plurality of cameras.

27. The system of claim 25 wherein the data captured by the video surveillance subsystem comprises visual representations of objects moving among the plurality of sub-regions of the cameras.

28. The system of claim 24 wherein the associations between the camera sub-regions and the operational radii of the stations are inferred from data captured by the radio-frequency identification subsystem.

29. The system of claim 28 wherein the data captured by the radio-frequency identification subsystem comprises spatial information associated with objects moving among the operational radii of the stations.

30. The system of claim 23 wherein the associations between the camera sub-regions and the operational radii of the stations are inferred from data captured by the video surveillance subsystem and the radio-frequency identification subsystem.

31. The system of claim 23 further comprising a data storage module for storing information associated with objects being tracked by the surveillance system.

32. The system of claim 31 wherein the information includes one or more rules relating to the objects.

33. The system of claim 31 further comprising a tracking module for chronologically representing, based on data received from the video surveillance subsystem and the radio-frequency identification subsystem, physical locations of the objects tracked by the surveillance system.

34. The system of claim 33 wherein the video surveillance subsystem and the radio-frequency identification subsystem operate within a region, the region comprising at least one ineffective area outside the fields of view of the cameras and the operational radii of the stations.

35. The system of claim 34 wherein the tracking module infers the location of one or more of the objects being tracked by the surveillance system while one or more objects pass through one or more ineffective areas.

36. The system of claim 34 wherein the tracking module infers a next camera location for an object being tracked by the surveillance system, while the object passes through an ineffective area, based on a previously known location of the object.

37. The system of claim 36 wherein the previously known location is based on data received from one or more of the radio-frequency identification subsystem and the video surveillance subsystem.

38. The system of claim 33 further comprising an alarm module, responsive to the data storage module and the tracking module, for comparing one or more of the rules and the chronological representations of the physical locations of one or more of the objects being tracked by the surveillance system.

39. The system of claim 38 wherein the alarm module creates one or more alerts upon determining one or more of the rules are violated by one or more of the chronological representations.

40. The system of claim 39 further comprising a transmission module for transmitting the alerts to a wireless communication device.

41. The system of claim 39 wherein the cameras are responsive to the alert modules such that an alert further instructs one or more cameras to focus on an object of interest.

42. The system of claim 39 wherein the cameras are responsive to the alert modules such that an alert further instructs one or more cameras to focus on an area of interest.

43. The system of claim 39 wherein the cameras are responsive to the alert modules such that an alert further instructs one or more cameras to focus on a person of interest.

44. The system of claim 23 wherein the video surveillance subsystem and the radio-frequency identification subsystem are calibrated independently of each other.

45. The system of claim 23 wherein the association of the camera sub-regions with one or more of operational radii of the radio-frequency stations is a many-to-many association.

46. A method of monitoring activity within an environment, the method comprising:

receiving a plurality of image signals generated by a plurality of cameras each having a field of view comprising a plurality of sub-regions;

receiving data generated by a plurality of radio-frequency stations, the data being based on interaction with a plurality of radio-frequency tags associated with objects within the environment;

associating one or more of the camera sub-regions with one or more of operational radii of the stations; and

tracking, independent of calibration, a plurality of objects within the environment as the objects move between fields of view of the cameras and the operational radii of the stations.

47. The method of claim 46 wherein associations between the camera sub-regions and the operational radii of the stations are inferred from data captured by the plurality of video signals generated by a plurality of cameras.

48. The method of claim 47 wherein the plurality of video signals generated by a plurality of cameras comprises visual representations of objects moving among the plurality of cameras.

49. The method of claim 47 wherein the plurality of video signals generated by a plurality of cameras comprises visual representations of objects moving among the plurality of sub-regions of the cameras.

50. The method of claim 46 where the associations between the camera sub-regions and the operational radii of the stations are inferred from the data generated by a plurality of radio-frequency stations.

51. The method of claim 50 wherein the data generated by a plurality of radio-frequency stations comprises spatial information associated with objects moving among the operational radii of the radio-frequency stations.

52. The method of claim 46 where the associations between the camera sub-regions and the operational radii of the stations are inferred from the plurality of video signals generated by the plurality of cameras and the data generated by the plurality of radio-frequency stations.

53. The method of claim 46 further comprising chronologically representing, based on the plurality of video signals generated by the plurality of cameras and the data generated by the plurality of radio-frequency stations, the physical locations of the plurality of objects within the environment.

54. The method of claim 46 further comprising inferring the location of one or more of the objects within the environment while the objects pass through one or more ineffective areas, the ineffective areas being outside the fields of view of the cameras and the operational radii of the stations.

55. The method of claim 46 further comprising inferring a next camera location for an object within the environment, while the object passes through an ineffective area, based on a previously known location of the object.

56. The method of claim 55 wherein the previously known location is based on the plurality of video signals generated by the plurality of cameras and the data generated by the plurality of radio-frequency stations.

57. The method of claim 53 further comprising applying one or more rules to one or more of the chronological representations of the physical location of the plurality of objects within the environment.

58. The method of claim 55 further comprising creating one or more alerts upon determining one or more of the rules are violated by one or more of the chronological representations.

59. The method of claim 58 further comprising transmitting the alerts to a wireless communication device.

60. The method of claim 58 wherein the alerts further instruct one or more cameras to focus on an object of interest within the environment.

61. The method of claim 58 wherein the alerts further instruct one or more cameras to focus on an area of interest within the environment.

62. The method of claim 58 wherein the alerts further instruct one or more cameras to focus on a person of interest within the environment.

63. An article of manufacture having computer-readable program portions embodied thereon for monitoring activity within an environment, the article comprising computer-readable instructions for:

receiving a plurality of image signals representing at least a portion of the environment;

identifying an object within the image signals;

receiving non-video data representing an event occurring in the environment; and

generating an alert based on the identified object and the event.