Systems and methods for business process monitoring

• US 8,354,926 B2
• Filed: 03/04/2012
• Issued: 01/15/2013
• Est. Priority Date: 10/04/2007
• Status: Active Grant

• Current Assignee: SecureNet Solutions Group, LLC
• Original Assignee: KD Secure LLC
• Inventors: Donovan, John J, Hussain, Daniar
• Primary Examiner(s): La, Anh V
• Application Number:
  US13/411,602
• Publication Number:
  US 20120226526A1
• Time in Patent Office: 317 Days
• Field of Search: 340/506, 340/531, 340/539.25, 340/517, 340/521, 340/532, 340/541, 340/3.1, 340/519, 340/540, 340/545.2, 340/933, 340/934, 340/937, 340/942, 340/286.07, 340/573.4, 348/14.01, 348/143, 348/148, 348/149, 382/103, 382/104, 702/159, 701/119, 705/7.36
• US Class Current: 340/540
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19693 using multiple video sources viewed on a single or compound screen
• CPC Class Codes: G08B 13/19697 Arrangements wherein non-video detectors generate an alarm themselves
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 41/0654 using network fault recovery ring fault isolation or reconfiguration in loop networks without recovery actions by a network management system H04L12/437
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

The present invention is a method to ensure business processes are followed, comprising the steps of capturing sensory data from one or more sensors; storing the sensory data from the one or more sensors in a data storage device; processing the sensory data from the sensors to detect primitive events in the sensory data using a computer processor; correlating two or more primitive events, weighted by attribute data representing information about the sensors used to sense the primitive events, to determine one or more correlated events using the computer processor; and performing one or more actions to ensure business processes are followed based on the correlation performed in the correlating step.

Claims

1. A business process monitoring system to ensure business processes are followed, comprising:

one or more sensors for capturing sensory data;

a data storage device for storing the sensory data from the one or more sensors;

one or more processors, operatively coupled to the one or more sensors; and

one or more memories, operatively coupled to the one or more processors, the one or more memories comprising program code which when executed causes the one or more processors to;

capture sensory data from the one or more sensors;

process the sensory data from the one or more sensors to detect primitive events in the sensory data;

correlate two or more primitive events, weighted by attribute data representing information about the sensors used to sense the primitive events, to determine one or more correlated events; and

perform one or more actions to ensure business processes are followed based on the correlation performed in the correlating step.

2. The system of claim 1, wherein the one or more sensors are selected from the group consisting of surveillance camera, surveillance microphone, temperature probe, pressure probe, speedometer, chlorine meter, radon meter, dust particle meter, CO₂ meter, water meter, electrical meter, blood pressure meter, and heart rate meter.

3. The system of claim 1, further comprising program code to:

detect compound events composed of two or more primitive events.

4. The system of claim 3, further comprising program code to:

time correlate the primitive events and the compound events across time;

space correlate the primitive events and the compound events across space; and

evaluate one or more rules based on the correlation performed in the time correlating step and the space correlating step.

5. The system of claim 1, further comprising program code to:

generate one or more new rules based on the primitive events correlated in the correlating step and the actions performed in the action step.

6. The system of claim 1, further comprising program code to:

receive tip data from one or more external sources;

determine attribute data for the tip data, the attribute data representing a reliability of the source of the tip data; and

generate tip events based on the tip data and the attribute data.

7. The system of claim 1, further comprising program code to:

manage storage and cascade of the sensory data in a hierarchy of data storage devices based on the attribute data corresponding to a source of the sensory data.

8. The system of claim 7,

wherein the hierarchy of data storage devices includes at least a first-tier device and a second-tier device, the first-tier device having a higher data access performance and a lower storage capacity than the second-tier device.

9. The system of claim 8, further comprising program code to:

determine storage locations for segments of sensory data based on a set of rules based on an importance of the sensory data.

10. The system of claim 1, further comprising program code to:

monitor network status of the one or more sensors; and

generate network events reflective of the network status of the one or more sensors.

11. The system of claim 1, further comprising program code to:

retrieve information about a vehicle detected in the sensory data based on the detected vehicle'"'"'s license plate; and

generate vehicle events based on the information retrieved about the vehicle.

12. The system of claim 1, wherein the attribute data comprises a quality of sensory data produced by the sensors.

13. The system of claim 1, wherein the attribute data comprises an age of the sensors used to capture the sensory data.

14. The system of claim 1, wherein the attribute data comprises a time since the sensors were last maintained.

15. The system of claim 1, wherein the attribute data comprises an integrity of the sensors used to capture the sensory data.

16. The system of claim 1, wherein the attribute data comprises a reliability of the sensors used to capture the sensory data.

17. The system of claim 1, further comprising program code, which when executed causes the processors to:

normalize the primitive events detected in the sensory data into a standardized format before correlating two or more normalized primitive events.

18. The system of claim 1, wherein the primitive events include video events from video analytics devices, audio events from audio sensory devices, tip events from third-party tips, network events from a network management subsystem, and vehicle events from a vehicle information subsystem.

19. The system of claim 1, further comprising program code, which when executed causes the processors to:

filter out primitive events based on a set of privacy rules, wherein the set of privacy rules are defined by a system administrator and are designed to protect a privacy of individuals where said system is being used.

20. The system of claim 1, further comprising program code, which when executed causes the processors to:

correlate events across time for historical event correlations between events detected now and events that occurred historically.

21. The system of claim 1, further comprising program code, which when executed causes the processors to:

correlate events across space for events occurring substantially simultaneously across multiple sensors located in different locations across space.

22. The system of claim 9, wherein the importance of the sensory data is based on an attribute selected from the group consisting of a resolution of the sensor used to capture the sensory data, an age of the sensor used to capture the sensory data, a time since last maintenance of the sensor used to capture the sensory data, a location of the sensor used to capture the sensory data, and a reliability of the sensor used to capture the sensory data.

23. The system of claim 1, further comprising program code, which when executed causes the processors to:

receive multimedia tips from one or more external sources, wherein the multimedia tips contain multimedia information of an event of interest and associated tip data.

24. The system of claim 1, further comprising program code, which when executed causes the processors to:

assign the attribute data based on past evidence of usefulness of sensory data from a given sensor.

25. The system of claim 1, further comprising program code, which when executed causes the processors to:

assign the attribute data based on susceptibility of the sensor to noise and interference.

26. The system of claim 1, wherein the primitive events include an event if no person is in a designated area when a person is required.

27. The system of claim 1, wherein the primitive events include an event from a personnel system indicating an experience level of a person.

28. The system of claim 1, further comprising program code, which when executed causes the processors to:

perform one or more actions based on an overall, system-wide correlation level exceeding a threshold.

29. The system of claim 1, wherein the one or more actions include an action to reboot a sensor upon detection of a failure of said sensor.

30. The system of claim 3, wherein the compound events include an event if more people entered a designated area than left the designated area.

31. The system of claim 3, wherein the compound events include an event if a person spends too short a time in a designated area.

32. The system of claim 3, wherein the compound events include an event if a person spends too long a time in a designated area.

60. The method of claim 25, wherein the compound events include an event if more people entered a designated area than left the designated area.

61. The method of claim 25, wherein the compound events include an event if a person spends too short a time in a designated area.

62. The method of claim 25, wherein the compound events include an event if a person spends too long a time in a designated area.

33. A business process monitoring system to ensure business processes are followed, comprising:

one or more sensors for capturing sensory data;

a data storage device for storing the sensory data from the one or more sensors;

one or more data processing units, adapted to process the sensory data from one or more of the sensors and to detect primitive events in the sensory data;

a correlation engine for correlating two or more primitive events from the data processing units, weighted by attribute data representing information about the sensors used to sense the primitive events, to determine one or more correlated events; and

an action engine for performing one or more actions to ensure business processes are followed based on the correlation performed by the correlation engine.

34. A method to ensure business processes are followed, comprising the steps of:

capturing sensory data from one or more sensors;

storing the sensory data from the one or more sensors in a data storage device;

processing the sensory data from the sensors to detect primitive events in the sensory data using a computer processor;

correlating two or more primitive events, weighted by attribute data representing information about the sensors used to sense the primitive events, to determine one or more correlated events using the computer processor; and

performing one or more actions to ensure business processes are followed based on the correlation performed in the correlating step.

35. The method of claim 34, further comprising:

detecting compound events composed of two or more primitive events.

36. The method of claim 35, further comprising:

time correlating the primitive events and the compound events across time;

space correlating the primitive events and the compound events across space; and

evaluating one or more rules based on the correlation performed in the time correlating step and the space correlating step.

37. The method of claim 34, further comprising:

receiving tip data from one or more external sources;

determining attribute data for the tip data, the attribute data representing a reliability of the source of the tip data; and

generating tip events based on the tip data and the attribute data.

38. The method of claim 34, further comprising:

managing storage and cascade of the sensory data in a hierarchy of data storage devices based on the attribute data corresponding to a source of the sensory data.

39. The method of claim 38, further comprising:

determining storage locations for segments of sensory data based on a set of rules based on an importance of the sensory data.

40. The method of claim 34, further comprising:

monitoring network status of the one or more sensors; and

generating network events reflective of the network status of the one or more sensors.

41. The method of claim 34, further comprising:

retrieving information about a vehicle detected in the sensory data based on the detected vehicle'"'"'s license plate; and

generating vehicle events based on the information retrieved about the vehicle.

42. The method of claim 34, wherein the attribute data comprises a quality of sensory data produced by the sensors.

43. The method of claim 34, wherein the attribute data comprises an age of the sensors used to capture the sensory data.

44. The method of claim 34, wherein the attribute data comprises a time since the sensors were last maintained.

45. The method of claim 34, wherein the attribute data comprises an integrity of the sensors used to capture the sensory data.

46. The method of claim 34, wherein the attribute data comprises a reliability of the sensors used to capture the sensory data.

47. The method of claim 34, further comprising:

normalizing the primitive events detected in the sensory data into a standardized format before correlating two or more normalized primitive events.

48. The method of claim 34, wherein the primitive events include video events from video analytics devices, audio events from audio sensory devices, tip events from third-party tips, network events from a network management subsystem, and vehicle events from a vehicle information subsystem.

49. The method of claim 34, further comprising:

filtering out primitive events based on a set of privacy rules, wherein the set of privacy rules are defined by a system administrator and are designed to protect a privacy of individuals where said method is being utilized.

50. The method of claim 34, further comprising:

correlating events across time for historical event correlations between events detected now and events that occurred historically.

51. The method of claim 34, further comprising:

correlating events across space for events occurring substantially simultaneously across multiple sensors located in different locations across space.

52. The method of claim 39, wherein the importance of the sensory data is based on an attribute selected from the group consisting of a resolution of the sensor used to capture the sensory data, an age of the sensor used to capture the sensory data, a time since last maintenance of the sensor used to capture the sensory data, a location of the sensor used to capture the sensory data, and a reliability of the sensor used to capture the sensory data.

53. The method of claim 34, further comprising:

receiving multimedia tips from one or more external sources, wherein the multimedia tips contain multimedia information of an event of interest and associated tip data.

54. The method of claim 34, further comprising:

assigning the attribute data based on past evidence of usefulness of sensory data from a given sensor.

55. The method of claim 34, further comprising:

assigning the attribute data based on susceptibility of the sensors to noise and interference.

56. The method of claim 34, wherein the primitive events include an event if no person is in a designated area when a person is required.

57. The method of claim 34, wherein the primitive events include an event from a personnel system indicating an experience level of a person.

58. The method of claim 34, further comprising:

performing one or more actions based on an overall, system-wide correlation level exceeding a threshold.

59. The method of claim 34, wherein the one or more actions include an action to reboot a sensor upon detection of a failure of said sensor.