Methods for defining, detecting, analyzing, indexing and retrieving events using video image processing

• US 7,801,328 B2
• Filed: 03/31/2005
• Issued: 09/21/2010
• Est. Priority Date: 03/31/2005
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Bedros, Saad J., Curtner, Keith L., Au, KwongWing
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): Thomas; Mia M
• Application Number:
  US10/907,413
• Publication Number:
  US 20060227997A1
• Time in Patent Office: 2,000 Days
• Field of Search: 382/103, 382/106, 382/107, 382/224, 382/305, 382/181, 382/199, 345/156, 348/143, 348/169, 725/61, 725/93, 725/105, 725/116
• US Class Current: 382/103
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: G08B 13/19669 Event triggers storage or change of storage policy
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: G08B 25/14 Central alarm receiver or annunciator arrangements

Abstract

Methods and systems for defining, detecting, analyzing, indexing, and retrieving events using video image processing techniques are described herein. An illustrative method of detecting and analyzing events occurring within a region of interest of a video monitoring system may include the steps of extracting information regarding one or more objects located within a region of interest, analyzing the extracted information to detect the occurrence of one or more events within the region of interest, matching at least one of the detected events with a corresponding user-specified event, and performing an action response upon the detection of such an event. In certain embodiments, the detected events may be defined as a set of semantic parameters relating to the object'"'"'s state and/or physical features, which can be compared against a corresponding set of semantic parameters programmed within a video image-processing appliance manager.

Claims

1. A computer assisted method of detecting and analyzing events occurring within a region of interest of a video monitoring system, the method comprising the steps of:

providing a video monitoring system including a user interface having a display, the display displaying event condition details in a first portion of the display and action response details in a second portion of the display;

defining one or more user-specified events using the event condition details in the first portion of the user interface of the video monitoring system;

the video monitoring system storing the one or more user-specified events;

the video monitoring system extracting information regarding one or more objects located within the region of interest;

the video monitoring system analyzing the extracted information to detect an occurrence of one or more of the one or more user-specified events within the region of interest;

defining one or more action responses using the second portion of the user interface of the video monitoring system; and

the video monitoring system performing the one or more action responses upon the detection of the occurrence of one or more of the one or more user-specified events.

2. The method of claim 1, wherein the video monitoring system comprises a video image-processing appliance manager in communication with one or more appliance modules.

3. The method of claim 2, wherein said step of extracting information regarding one or more objects located within a region of interest is accomplished using at least one of said one or more appliance modules.

4. The method of claim 2, wherein said step of extracting information regarding one or more objects located within a region of interest is accomplished using the appliance manager.

5. The method of claim 2, wherein said analyzing and performing steps are accomplished using the appliance manager.

6. The method of claim 2, wherein said step of analyzing the extracted information to detect the occurrence of the one or more user-specified events occurring within the region of interest is accomplished using an event detection module.

7. The method of claim 2, wherein said one or more appliance modules includes a video detection module, a video motion tracking module, an object classification module, and an event detection module.

8. The method of claim 2, wherein said step of performing the one or more action responses upon the detection of the one or more user-specified events includes the step of performing an action request using an action dispatcher module.

9. The method of claim 2, wherein the user interface includes a graphical user interface in communication with the appliance manager.

10. The method of claim 9, wherein said step of defining one or more user-specified events is accomplished using the graphical user interface.

11. The method of claim 1, wherein said step of analyzing the extracted information to detect the occurrence of the one or more user-specified events occurring within the region of interest includes the step of defining a set of semantic parameters describing the object'"'"'s state and/or physical features relative to the region of interest.

12. The method of claim 11, wherein said step of analyzing the extracted information to detect the occurrence of the one or more user-specified events within the region of interest includes the step of comparing said set of object semantic parameters with a set of user-specified semantic parameters.

13. The method of claim 12, further comprising the video monitoring system indexing each detected event using said set of object semantic parameters.

14. The method of claim 12, wherein said step of performing an action response upon the detection of the one or more user-specified events includes the step of recording a video clip containing the event and storing the video clip within a database.

15. The method of claim 14, further comprising the step of subsequently retrieving the video clip containing the one or more user-specified events using said set of object semantic parameters.

16. The method of claim 1, wherein said step of performing an action response upon the detection of the one or more user-specified events includes the step of sending a video signal to a terminal station.

17. A computer assisted method of detecting and analyzing events occurring within a region of interest of a video monitoring system, the method comprising the steps of:

providing a modular video monitoring system comprising a video image-processing appliance manager in communication with one or more plug-in appliance modules configured to perform user-defined imaging processing functions;

initiating an algorithm or routine within the appliance manager and extracting information regarding one or more objects located within a region of interest;

the video monitoring system analyzing the extracted information to detect the occurrence of one or more user-specified events within the region of interest; and

the video monitoring system performing an action response upon the detection of one or more of the user-specified events, wherein the action response includes issuing an alarm and/or initiating a recording of a video clip containing the one or more user-specified events.

18. The method of claim 17, wherein said step of extracting information regarding one or more objects located within a region of interest is accomplished using at least one of said one or more appliance modules.

19. The method of claim 17, wherein said step of extracting information regarding one or more objects located within a region of interest is accomplished using the appliance manager.

20. The method of claim 17, wherein said analyzing and performing steps are accomplished using the appliance manager.

21. The method of claim 17, wherein said step of analyzing the extracted information to detect the occurrence of the one or more user-specified events occurring within the region of interest is accomplished using an event detection module.

22. The method of claim 17, wherein said one or more appliance modules includes a video detection module, a video motion tracking module, an object classification module, and an event detection module.

23. The method of claim 17, wherein said step of performing an action response upon the detection of one or more of the user-specified events includes the step of performing an action request using an action dispatcher module.

24. The method of claim 17, wherein the video monitoring system comprises a graphical user interface in communication with the appliance manager.

25. The method of claim 24, further comprising the video monitoring system defining the one or more user-specified events using the graphical user interface.

26. The method of claim 17, wherein said step of analyzing the extracted information to detect the occurrence of the one or more user-specified events occurring within the region of interest includes the step of defining a set of semantic parameters describing the object'"'"'s state and/or physical features relative to the region of interest.

27. The method of claim 26, wherein said step of analyzing the extracted information to detect the occurrence of one or more user-specified events within the region of interest includes the step of comparing said set of object semantic parameters with a set of user-specified semantic parameters.

28. The method of claim 27, further comprising the video monitoring system indexing each detected event using said set of object semantic parameters.

29. The method of claim 27, wherein said recording of a video clip containing the one or more user-specified events includes storing the video clip within a database.

30. The method of claim 29, further comprising the video monitoring system subsequently retrieving the video clip containing the one or more user-specified events using said set of object semantic parameters.

31. The method of claim 17, wherein said step of performing an action response upon the detection of one or more of the user user-specified events includes the step of sending a video signal to a terminal station.

32. A computer assisted method of detecting and analyzing events occurring within a region of interest of a video monitoring system, the method comprising the steps of:

the video monitoring system prompting a user to define one or more user-specified event for detection in a region of interest using a user interface of the video monitoring system, wherein for each user-specified event defined by the user, the user can select two or more semantic parameters using the user interface of the video monitoring system;

the video monitoring system storing a set of user-specified semantic parameters corresponding to each user-specified event;

the video monitoring system extracting information regarding one or more objects located within a region of interest;

the video monitoring system analyzing the extracted information to define a set of semantic parameters describing the object'"'"'s state and/or physical features relative to the region of interest;

the video monitoring system comparing said set of object semantic parameters with the set of user-specified semantic parameters corresponding to each user-specified event; and

the video monitoring system detecting the user-specified event when at least one of the object semantic parameters matches a corresponding user-specified semantic parameter.

33. A computer assisted method of detecting and analyzing events occurring within a region of interest of a video monitoring system, the method comprising the steps of:

providing a modular video monitoring system comprising a video image-processing appliance manager in communication with one or more plug-in appliance modules and a graphical user interface;

the video monitoring system prompting a user to define one or more user-specified event for detection in the region of interest using the graphical user interface, wherein for each user-specified event defined by the user, the user can select two or more semantic parameters using the graphical user interface;

the video monitoring system monitoring the region of interest to detect the one or more user-specified event in real or near real-time; and

the video monitoring system performing an action response upon the detection of the one or more user-specified event.

34. A method of extracting object information using video image processing, the method comprising the steps of:

providing a video monitoring system that provides at least one video frame containing an object of interest;

defining at least one region of interest within the video frame;

providing a boundary box about the object of interest;

determining the location of the boundary box relative to a boundary area of the region of interest;

assigning a set of coordinates to the boundary box and the boundary area of the region of interest;

providing the video monitoring system one or more user-definable semantic parameters relating to the object of interest; and

analyzing the object of interest with the video monitoring system to detect a user-definable event when the object of interest has the one or more user-definable semantic parameters.

35. The method of claim 34, wherein said step of determining the location of the boundary box relative to a boundary area of the region of interest includes the step of:

comparing the corresponding coordinate values of the boundary box relative to the coordinate values of the boundary area.

36. The method of claim 34, wherein said boundary box is a minimum boundary rectangle.

37. A method of extracting object information using video image processing, the method comprising the steps of:

providing a video monitoring system including at least one video frame containing an object of interest;

defining at least one region of interest within the video frame;

providing a minimum boundary rectangle about the object of interest;

assigning a set of coordinates to the minimum boundary rectangle and a boundary area of the region of interest;

comparing the corresponding coordinate values of the minimum boundary rectangle relative to the coordinate values of the boundary area and determining the location of the minimum boundary rectangle relative to the boundary area;

defining one or more user-definable semantic parameters relating to the object of interest; and

analyzing the object of interest with the video monitoring system to detect a user-definable event when the object of interest has the one or more user-definable semantic parameters.

38. A video monitoring system, comprising:

a video image-processing appliance manager in communication with one or more appliance modules, said appliance manager configured to run an algorithm or routine for extracting information regarding one or more objects located within a region of interest and defining a set of semantic parameters according to one or more user-specified semantic parameters, the set of semantic parameters describing the status and/or physical state of each object relative to the region of interest; and

a graphical user interface including a first portion for configuring the one or more user-specified semantic parameters within the appliance manager and/or an event detection module and the graphical user interface includes a second portion for selecting an action response upon the detection of an event by the video monitoring system.

39. The video monitoring system of claim 38, wherein said graphical user interface includes a means for selecting a region of interest for the detection of an event.

40. The video monitoring system of claim 38, wherein said graphical user interface includes a means for selecting one or more event conditions for detection within the region of interest.

41. The video monitoring system of claim 38, wherein said means for selecting an action response upon the detection of an event by the video monitoring system includes a means for setting an alarm response.

42. The video monitoring system of claim 38, wherein said means for selecting an action response upon the detection of an event by the video monitoring system includes a means for recording a video clip.

43. The video monitoring system of claim 38, wherein said means for selecting an action response upon the detection of an event by the video monitoring system includes a means for sending video to a terminal station.