Specifying search criteria for searching video data

• US 9,224,047 B2
• Filed: 09/24/2013
• Issued: 12/29/2015
• Est. Priority Date: 09/12/2005
• Status: Active Grant

• Current Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Original Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Inventors: Vallone, Robert P., Wells, Michael E., Hale, Shannon P., Fleischer, Stephen D., Pitts, Colvin H., Haupt, Gordon T.
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US14/035,098
• Publication Number:
  US 20140022387A1
• Time in Patent Office: 826 Days
• Field of Search: 348/143
• US Class Current: 1/1
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19676 Temporary storage, e.g. cyclic memory, buffer storage on pre-alarm
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/45455 applied to a region of the image
• CPC Class Codes: H04N 21/45457 applied to a time segment
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739

Abstract

A method and apparatus is described for specifying regions of interest within a two-dimensional view of visual information that comprises a series of frames. Visual changes that occur in the view are stored. A user enters search criteria that specify at least one first region of interest within the view and a visual change. A visual change may include a change in pixel values or a detection of motion of one or more objects within the view. The first search criteria are compared against the stored visual changes to identify a sequence of frames in which the specified visual change occurred within the first region of interest. The search criteria may specify multiple regions of interest, each with one or more types of visual changes. If a motion is specified, then a direction, speed, and behavior of a moving object may also be specified.

Claims

1. A machine-implemented method, comprising:

causing a graphical user interface to be displayed that allows a user to specify search criteria;

wherein causing the graphical user interface to be displayed includes;

causing to be displayed a plurality of selectable behavior option controls, wherein each selectable behavior option control of the plurality of selectable behavior option controls corresponds to a pre-defined behavior;

causing to be displayed, concurrently with the plurality of selectable behavior option controls, a plurality of selectable object shape controls, wherein each selectable object shape control of the plurality of selectable object shape controls corresponds to an object shape;

receiving, from the user through the graphical user interface, a selection of a particular selectable behavior option control of the plurality of selectable behavior-option controls;

wherein the particular selectable behavior option control corresponds to a particular pre-defined behavior;

receiving, from the user through the graphical user interface, a selection of a particular selectable object shape control of the plurality of selectable object shape controls;

wherein the particular selectable object shape control corresponds to a particular object shape;

determining whether any object having the particular object shape exhibited the particular pre-defined behavior within video data;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein the pre-defined behaviors corresponding to the plurality of selectable behavior option controls include one or more of loitering, running, snooping, or swerving.

3. The method of claim 1, further comprising:

in response to receiving the selection of the particular selectable object shape control and based on the particular object shape, identifying a plurality of pre-defined behaviors that are associated with the particular object shape;

wherein the graphical user interface displays the plurality of selectable behavior option controls in response to identifying the plurality of pre-defined behaviors.

4. The method of claim 3, wherein object shapes corresponding to the plurality of selectable object shape controls includes one or more of the following types:

a person, an automobile, or suitcase.

5. The method of claim 4, wherein the particular object shape is a person and the pre-defined behaviors corresponding to the plurality of selectable behavior option controls includes running, snooping, swerving, or loitering.

7. The method of claim 1, wherein the graphical user interface presents second controls for specifying one or more characteristics that are to be associated with a behavior option of the plurality of selectable behavior option controls.

8. The method of claim 1, wherein the particular pre-defined behavior is associated with a particular speed or a particular amount of time.

6. A method comprising:

causing a graphical user interface to be displayed that allows a user to select from among a plurality of selectable behavior option controls, wherein each selectable behavior option control of the plurality of selectable behavior option controls corresponds to a pre-defined behavior;

receiving, through the graphical user interface, first user input that selects a particular selectable behavior option control of the plurality of selectable behavior option controls, wherein the particular selectable behavior option control corresponds to a particular pre-defined behavior;

prior to or subsequent to receiving the first user input, receiving second user input that defines a region of interest in a camera view;

determining whether any object exhibited the particular pre-defined behavior within video data that corresponds to the region of interest.

9. One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause:

causing a graphical user interface to be displayed that allows a user to specify search criteria;

wherein causing the graphical user interface to be displayed includes;

causing to be displayed a plurality of selectable behavior option controls, wherein each selectable behavior option control of the plurality of selectable behavior option controls corresponds to a pre-defined behavior;

causing to be displayed a plurality of selectable object shape controls, wherein each selectable object shape control of the plurality of selectable object shape controls corresponds to an object shape;

receiving, from the user through the graphical user interface a selection of a particular selectable behavior option control of the plurality of selectable behavior option controls;

wherein the particular selectable behavior option control corresponds to a particular pre-defined behavior;

receiving, from the user through the graphical user interface, a selection of a particular selectable object shape control of the plurality of selectable object shape controls;

wherein the particular selectable object shape control corresponds to a particular object shape;

determining whether any object having the particular object shape exhibited the particular pre-defined behavior within video data.

10. The one or more non-transitory computer-readable media of claim 9, wherein the pre-defined behaviors corresponding to the plurality of selectable behavior option controls include one or more of loitering, running, snooping, or swerving.

11. The one or more non-transitory computer-readable media of claim 9, wherein the instructions, when executed by the one or more processors, further cause:

in response to receiving the selection of the particular selectable object shape control and based on the particular object shape, identifying a plurality of pre-defined behaviors that are associated with the particular object shape;

wherein the graphical user interface displays the plurality of selectable behavior option controls in response to identifying the plurality of pre-defined behaviors.

12. The one or more non-transitory computer-readable media of claim 11, wherein object shapes corresponding to the plurality of selectable object shape controls includes one or more of the following types:

a person, an automobile, or suitcase.

13. The one or more non-transitory computer-readable media of claim 12, wherein the particular object shape is a person and the pre-defined behaviors corresponding to the plurality of selectable behavior option controls includes running, snooping, swerving, or loitering.

15. The one or more non-transitory computer-readable media of claim 9, wherein the graphical user interface presents second controls for specifying one or more characteristics that are to be associated with a behavior option of the plurality of selectable behavior option controls.

16. The one or more non-transitory computer-readable media of claim 9, wherein the particular pre-defined behavior is associated with a particular speed or a particular amount of time.

14. One or more non-transitory computer-readable media storing instructions which, when executed by one or more processor, cause:

causing a graphical user interface to be displayed that allows a user to select from among a plurality of selectable behavior option controls, wherein each selectable behavior option control of the plurality of selectable behavior option controls corresponds to a pre-defined behavior;

receiving, through the graphical user interface, first user input that selects a particular selectable behavior option control of the plurality of selectable behavior option controls, wherein the particular selectable behavior option control corresponds to a particular pre-defined behavior;

prior to or subsequent to receiving the first user input, receiving second user input that defines a region of interest in a camera view;

determining whether any object exhibited the particular pre-defined behavior within video data that corresponds to the region of interest.