Storage of video analysis data for real-time alerting and forensic analysis

• US 20070058842A1
• Filed: 09/12/2006
• Published: 03/15/2007
• Est. Priority Date: 09/12/2005
• Status: Abandoned Application

• Current Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Original Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Inventors: Vallone, Robert, Pitts, Colvin, Frederick, Timothy, Haupt, Gordon, Fleischer, Stephen, Faulhaber, Thomas, Marinelli, Marcus
• Application Number:
  US11/520,532
• Publication Number:
  US 20070058842A1
• Time in Patent Office: Days
• US Class Current: 382/115
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19676 Temporary storage, e.g. cyclic memory, buffer storage on pre-alarm
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/45455 applied to a region of the image
• CPC Class Codes: H04N 21/45457 applied to a time segment
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739

Abstract

A method and apparatus for storing video data is provided. Video data, that comprises a series of frames, is received. Information about changes that are detected in the series of frames is generated. The information is aggregated to generate a plurality of video data change records. Each video data change record corresponds to a plurality of frames and includes change information that indicates changes that were detected relative to the corresponding plurality of frames. Events of interest that satisfy specified search criteria are searched for by comparing the specified search criteria against change information in one or more of the plurality of video data change records.

Claims

1. A machine-implemented method, comprising:

generating information about changes that are detected in video data that includes a series of frames;

aggregating the information to generate a plurality of video data change records, wherein each video data change record corresponds to a plurality of frames and includes change information that indicates changes that were detected in video data relative to the corresponding plurality of frames; and

storing said aggregated video data change records for subsequent forensic analysis.

2. The method of claim 1, wherein:

the plurality of video data change records includes a particular video data change record that corresponds to an event detected within the video data; and

the change information contained in the particular video data change record indicates changes that occurred in the frames that correspond to said event.

3. The method of claim 1, wherein:

the plurality of video data change records includes a set of video data change records that each corresponds to a specified time interval; and

the change information contained in each video data change record of the set of video data change records indicates changes that occurred in the frames that correspond to the corresponding specified time interval.

4. The method of claim 1, wherein the change information contained in each video data change record of the plurality of video data change records includes change type information that indicates at least one of:

(a) a change in pixel values that occurred during the plurality of frames that correspond to said each video data change record, and

(b) a detection of motion of one or more objects in the plurality of frames that correspond to said each video data change record.

5. The method of claim 4, wherein:

the change type information of each video data change record indicates the detection of motion of one or more objects captured in the plurality of frames that correspond to said each video data change record; and

said each video data change record further indicates at least one of a direction and a speed of said motion.

6. The method of claim 1, wherein:

said plurality of video data change records includes a plurality of region change records; and

each region change record of the plurality of region change records corresponds to a separate region of a frame and contains region change information about the changes that were detected in the separate region relative to the plurality of frames that correspond to the particular video data change record.

7. The method of claim 6, wherein:

the region change information of said each region change record is filtered based on other region change information of other region change records of the plurality of region change records; and

the other region change records correspond to other regions, of the frame, that are adjacent to the region that corresponds to said each region change record.

8. The method of claim 1, wherein the change information of a particular video data change record of the plurality of video data change records is filtered, based on the plurality of frames that correspond to said particular video data change record.

9. The method of claim 1, further comprising searching for events of interest that satisfy specified search criteria by comparing the specified search criteria against change information in one or more of said plurality of video data change records.

10. The method of claim 6, wherein:

the method further comprises searching for events of interest that satisfy specified search criteria by comparing the specified search criteria against change information in one or more of said plurality of video data change records;

said specified search criteria indicates a first region of a set of frames in which the events of interest must have occurred;

searching for the events of interest that satisfy the specified search criteria includes;

identifying a first set of video data change records that correspond to an area of the view that includes said first region and is larger than said first region; and

based on the first set of video data change records, identifying a second set of video data change records that correspond to said first region, wherein the change information of each video data change record in the second set satisfies the specified search criteria.

11. The method of claim 1, wherein:

the method further comprises searching for events of interest that satisfy specified search criteria by comparing the specified search criteria against change information in one or more of said plurality of video data change records;

said specified search criteria specifies time at which the events of interest must have occurred;

searching for the events of interest that satisfy the specified search criteria includes;

identifying a first set of video data change records that correspond to a first time interval that includes said specified time; and

based on said first set of video data change records, identifying a second set of video data change records that aggregate change information at a finer level of granularity than the video data change records in said first set of video data change records;

wherein the change information of each video data change record in the second set includes said specified time and satisfies the specified search criteria.

12. The method of claim 1, wherein at least one video data change record of said plurality of video data change records corresponds to a time interval that was established, at least in part, by an event that is not reflected in said video data.

13. The method of claim 9, further comprising:

receiving second specified search criteria;

detecting an event in said video data as the video data is being received;

generating aggregated change information associated with said event;

before a video data change record is generated for said event, comparing said second specified search criteria against said aggregated change information; and

if said aggregated change information satisfies said second specified search criteria, immediately generating an alert that indicates that said second specified search criteria has been satisfied.

17. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

18. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

19. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

20. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

21. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

22. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

23. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

24. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

25. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

26. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.

27. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 11.

28. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 12.

29. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 13.

14. A machine-implemented method, comprising:

generating information about changes that are detected in video data that includes a series of frames; and

aggregating the information to generate a first plurality of video data change records, wherein each video data change record in the first plurality of change records corresponds to a plurality of frames and includes change information, aggregated at a relatively fine level of granularity, that indicates changes that were detected relative to the corresponding plurality of frames; and

aggregating the information to generate a second plurality of video data change records, wherein each video data change record in the second plurality of change records corresponds to a plurality of frames and includes change information, aggregated at a relatively coarse level of granularity, that indicates changes that were detected relative to the corresponding plurality of frames.

15. The method of claim 14 wherein:

each of the first plurality of video data change records corresponds to regions of a view; and

each of the second plurality of video data change records corresponds to the entire view.

16. The method of claim 14 wherein:

each of the first plurality of video data change records corresponds to time intervals of a first duration; and

each of the second plurality of video data change records corresponds to time intervals of a second duration, wherein the second duration is longer than said first duration.

30. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 14.

31. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 15.

32. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 16.