Specifying search criteria for searching video data

• US 20070061696A1
• Filed: 09/12/2006
• Published: 03/15/2007
• Est. Priority Date: 09/12/2005
• Status: Active Grant

• Current Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Original Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Inventors: Vallone, Robert, Hale, Shannon, Pitts, Colvin, Haupt, Gordon, Fleischer, Stephen, Wells, Michael
• Granted Patent: US 8,553,084 B2
• Time in Patent Office: Days
• US Class Current: 715/255
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19676 Temporary storage, e.g. cyclic memory, buffer storage on pre-alarm
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/45455 applied to a region of the image
• CPC Class Codes: H04N 21/45457 applied to a time segment
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739

Abstract

A method and apparatus is described for specifying regions of interest within a two-dimensional view of visual information that comprises a series of frames. Visual changes that occur in the view are stored. A user enters search criteria that specify at least one first region of interest within the view and a visual change. A visual change may include a change in pixel values or a detection of motion of one or more objects within the view. The first search criteria are compared against the stored visual changes to identify a sequence of frames in which the specified visual change occurred within the first region of interest. The search criteria may specify multiple regions of interest, each with one or more types of visual changes. If a motion is specified, then a direction, speed, and behavior of a moving object may also be specified.

Claims

1. A machine-implemented method, comprising:

storing change information about visual changes that occur in a two-dimensional view of visual information;

receiving, from a user, first search criteria that specifies;

a first region of interest within the view, and

a visual change; and

comparing the first search criteria against said change information to identify a sequence of frames, within the visual information, in which said visual change occurred within said first region of interest.

2. The method of claim 1, wherein the user specifies the first search criteria using a graphical interface.

3. The method of claim 1, wherein said first search criteria further specifies that the visual change must occur (1) within any portion of the first region of interest, or (2) within the entire portion of the first region of interest.

4. The method of claim 1, wherein said visual change indicates at least one of (a) a change in pixel values that occurred during said sequence of frames, and (b) a detection of motion in said sequence of frames.

5. The method of claim 4, wherein said visual change indicates:

the detection of motion in said sequence of frames, and

a direction of said motion.

6. The method of claim 5, wherein said visual change further indicates that the direction must occur (1) within any portion of the first region of interest, or (2) within the entire portion of the first region of interest.

7. The method of claim 4, wherein said visual change indicates:

the detection of motion in said sequence of frames; and

a speed of said motion.

8. The method of claim 7, wherein said visual change further indicates that the speed must occur (1) within any portion of the first region of interest, or (2) within the entire portion of the first region of interest.

9. The method of claim 4, wherein said visual change indicates:

the detection of motion in said sequence of frames, and

a shape of said motion.

10. The method of claim 4, wherein said visual change indicates:

the detection of motion in said sequence of frames, and

one or more characteristics of said motion.

11. The method of claim 1, wherein:

said visual information comprises a plurality of frames;

comparing the first search criteria against said change information to identify the sequence of frames is performed on change data corresponding to frames that are appended to the plurality of frames after the first search criteria is received.

12. The method of claim 1, wherein:

said visual information comprises a plurality of frames;

comparing the first search criteria against said change information to identify the sequence of frames is performed on change data corresponding to frames that were added to the plurality of frames before the first search criteria is received.

13. The method of claim 1, wherein:

said first search criteria further specifies a second region of interest within the view;

said second region of interest occupies a portion of the view that is not occupied by said first area of interest;

comparing the first search criteria includes comparing the first search criteria against said change information to identify a sequence of frames, within the visual information, in which said visual change occurred within at least one of said first region of interest and said second region of interest.

14. The method of claim 13, where said second region of interest occupies a portion of the view that is occupied by said first area of interest.

15. The method of claim 13, wherein said first search criteria further specifies that the visual change must occur (1) within any portion of the first region of interest or any portion of the second region of interest, (2) within any portion of the first region of interest and any portion of the second region of interest, (3) within the entire portion of the first region of interest or the entire portion of the second region of interest, or (4) within the entire portion of the first region of interest and the entire portion of the second region of interest.

16. The method of claim 15, wherein:

the visual change indicates a detection of motion of one or more objects in said sequence of frames; and

said first search criteria further specifies a direction of said motion.

17. The method of claim 15, wherein:

the visual change indicates a detection of motion of one or more objects in said sequence of frames; and

said first search criteria further specifies a speed of said motion.

18. The method of claim 1, wherein:

comparing said first search criteria includes;

identifying a first set of video data change records based on a second region within the view, wherein;

a video data change record corresponds to a plurality of frames of the visual information and includes change data that indicates visual changes that were detected relative to the corresponding plurality of frames, and

the second region includes said first region of interest and is larger than said first region of interest; and

based on the first set of video data change records and said first region, identifying a second set of video data change records, wherein the change data of each video data change record in the second set satisfies the first search criteria.

19. The method of claim 1, wherein:

comparing said first search criteria includes;

identifying a first set of video data change records based on a first time interval that includes said at least single point in time, wherein a video data change record;

corresponds to a plurality of frames of the visual information, and

includes change data that indicates visual changes that were detected relative to the corresponding plurality of frames; and

based on said first set of video data change records, identifying a second set of video data change records, wherein the change information of each video data change record in the second set satisfies the first search criteria.

20. The method of claim 1, wherein:

the first search criteria further specifies at least one time instance;

at least one frame in said sequence of frames occurred during said at least one time instance.

21. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

22. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

23. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

24. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

25. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

26. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

27. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

28. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

29. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

30. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.

31. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 11.

32. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 12.

33. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 13.

34. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 14.

35. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 15.

36. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 16.

37. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 17.

38. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 18.

39. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 19.

40. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 20.