Video analysis, archiving and alerting methods and apparatus for a distributed, modular and extensible video surveillance system

• US 20090322881A1
• Filed: 08/11/2009
• Published: 12/31/2009
• Est. Priority Date: 10/12/2004
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hampapur, Arun, Lu, Max, Shu, Chiao-Fe
• Application Number:
  US12/462,926
• Publication Number:
  US 20090322881A1
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: G08B 13/19693 using multiple video sources viewed on a single or compound screen
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23109 by placing content in organized collections, e.g. EPG data repository details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Systems and methods are disclosed for analyzing, managing and cataloguing video tracks generated by a video surveillance system. Systems and methods operating in accordance with the present invention receive video tracks from a video surveillance system; analyze them for particular categories of, for example, objects or events, assign unique identifying tags to the video tracks; and save them in highly-cross-referenced databases so that video tracks containing, for example, particular objects or events can be easily searched for and recalled. In addition, the present invention provides browsing facilities for viewing events that have just elapsed, events that are occurring in near real-time.

Claims

1-97. -97. (canceled)

98. A signal-bearing medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus of a computer system to perform operations comprising management of video surveillance activities using an electronic video surveillance system, the operations comprising:

applying a video analysis application tool to video gathered by a video surveillance camera in real time, wherein the video analysis application tool comprises at least one user-specified video analysis criterion, and where application of the video analysis application tool to the video generates a data stream related to the at least one user-specified video analysis criterion;

transmitting the data stream to an electronic database and application system;

storing information comprising the data stream in the electronic database and application system;

selecting at least a portion of the information comprising the data stream for forwarding to a video surveillance analyst performing video surveillance activities, wherein the selection is made based on at least one user-specified information forwarding criterion; and

forwarding the selected information to the video surveillance analyst for display on an interface device of the electronic video surveillance system.

99. The signal-bearing medium of claim 98 wherein forwarding the selected information occurs in real time.

100. The signal-bearing medium of claim 98 where the user-specified information forwarding criterion comprises an alarm event related to the at least one user-specified video analysis criterion, wherein the alarm event is intended to alert the video surveillance analyst.

101. The signal-bearing medium of claim 100 where the user-specified video analysis criterion comprises vehicle speed and where the user-specified information forwarding criterion comprises an alarm triggered when a speed limit is exceeded by a vehicle observed by the video surveillance camera.

102. The signal-bearing medium of claim 100 where the user-specified video analysis criterion comprises loitering activity and where the user-specified information forwarding criterion comprises an alarm triggered by the occurrence of loitering activity in a field of view observed by the video surveillance camera.

103. The signal-bearing medium of claim 100 where the user-specified video analysis criterion comprises a vehicular accident condition and where the user-specified information forwarding criterion comprises an alarm triggered upon occurrence of a vehicular accident occurring in a field of view monitored by the video surveillance camera.

104. The signal-bearing medium of claim 100 where the user-specified video analysis criterion comprises identification of a specific human being and where the user-specified forwarding criterion comprises an alarm triggered when the specific human being is observed in a field of view monitored by the video surveillance camera.

105. The signal-bearing medium of claim 100 where the user-specified video analysis criterion comprises the detection of a specific activity and where the user-specified forwarding criterion comprises an alarm triggered when the specific activity occurs in a field of view monitored by the video surveillance camera.

106. The signal-bearing medium of claim 98 where the operations further comprise:

assigning unique identifiers to segments of video gathered by the video surveillance camera; and

transmitting the segments to the electronic database and application system for storage.

107. The signal-bearing medium of claim 106 where the data stream comprises at least pointers to video segments satisfying the at least one user-specified video analysis criterion, wherein the pointers comprise video tracks.

108. The signal-bearing medium of claim 107 where the operations further comprise:

forwarding video corresponding to video segments identified by the video track to video surveillance analysts for viewing on a display device of the video surveillance system, wherein the forwarding occurs in dependence on at least one user-specified video forwarding criterion.

109. The signal-bearing medium of claim 108 where forwarding the video corresponding to video segments identified by video track occurs in real time.

110. The signal-bearing medium of claim 108 where the user-specified video forwarding criterion comprises an alarm event related to the at least one user-specified video analysis criterion, wherein the alarm event is intended to alert the video surveillance analyst.

111. The signal-bearing medium of claim 108 where the operations further comprise:

generating at least one keyframe each for at least a portion of the video segments identified by video tracks;

associating each keyframe with its respective video segment through a unique identifier; and

saving the keyframe to the electronic database and application system.

112. The signal-bearing medium of claim 111 where the operations further comprise:

selecting at least one keyframe for forwarding to a video surveillance analyst performing video surveillance activities, wherein the selection is made based on at least one user-specified keyframe forwarding criterion; and

forwarding the selected keyframe to the video surveillance analyst for display on a display device of the electronic video surveillance system.

113. The signal-bearing medium of claim 112 where forwarding the at least one keyframe occurs in real time.

114. The signal-bearing medium of claim 112 where the user-specified keyframe forwarding criterion comprises an alarm event related to the at least one user-specified video analysis criterion, wherein the alarm event is intended to alert the video surveillance analyst.

115. The signal-bearing medium of claim 112 where the operations further comprise:

receiving a request for the video segment corresponding to the keyframe from the video surveillance analyst;

retrieving the video segment corresponding to the keyframe from the electronic database and application system using the track; and

transmitting the video segment to a display device of the video surveillance system for viewing by the video surveillance analyst.

116. The signal-bearing medium of claim 111 where the operations further comprise:

selecting a plurality of keyframes for forwarding to a display device of the video surveillance system for viewing by a video surveillance analyst;

forwarding the plurality of keyframes to the display device; and

simultaneously displaying the plurality of keyframes on the display device.

117. The signal-bearing medium of claim 116 where the operations further comprise:

receiving a request for the video segment corresponding to at least one of the keyframes from the video surveillance analyst;

retrieving the video segment corresponding to the at least one keyframe from the electronic database and application system using the track; and

transmitting the video segment for display on the display device of the video surveillance system for viewing by the video surveillance analyst.

118. The signal-bearing medium of claim 98 where the operations further comprise:

receiving a query for information stored in the electronic database and application system;

retrieving information responsive to the query from the electronic database and application system; and

forwarding the information responsive to the query to a display device of the video surveillance system for viewing by a video surveillance analyst.

119. The signal-bearing medium of claim 107 where the operations further comprise:

receiving a query for information stored in the electronic database and application system, wherein the query comprises at least a request for video satisfying a user-specified selection criterion;

retrieving information responsive to the query from the electronic database and application system, wherein the information comprises video track;

retrieving the video segments corresponding to the video track from the electronic database and application system; and

transmitting the video segments to a display device of the video surveillance system for viewing by a video surveillance analyst.

120. The signal-bearing medium of claim 111 where the operations further comprise:

receiving a query for information stored in the electronic database and application system, wherein the query comprises at least a request for keyframes satisfying a user-specified selection criterion;

retrieving information responsive to the query from the electronic database and application system, wherein the information comprises keyframe identification data;

retrieving the keyframes corresponding to the keyframe identification data from the electronic database and application system; and

transmitting the keyframes to a display device of the video surveillance system for viewing by a video surveillance analyst.

121. The signal-bearing medium of claim 119 wherein the operations further comprise:

receiving a request for video segments corresponding to the keyframes from the video surveillance analyst;

retrieving the video segments corresponding to the keyframes from the electronic database and application system using video tracks; and

transmitting the video segments for display on the display device of the video surveillance system for viewing by the video surveillance analyst.