Anomalous stationary object detection and reporting

• US 9,723,271 B2
• Filed: 06/28/2013
• Issued: 08/01/2017
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Omni AI, Inc.
• Inventors: Xu, Gang, Cobb, Wesley Kenneth
• Primary Examiner(s): Atala, Jamie
• Assistant Examiner(s): NGUYEN, KATHLEEN V
• Application Number:
  US13/930,190
• Publication Number:
  US 20140002647A1
• Time in Patent Office: 1,495 Days
• Field of Search: 348143
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

Techniques are disclosed for analyzing a scene depicted in an input stream of video frames captured by a video camera. The techniques include receiving data for an object within the scene and determining whether the object has remained substantially stationary within the scene for at least a threshold period. If the object is determined to have remained stationary for at least the threshold period, a rareness score is calculated for the object to indicate a likelihood of the object being stationary to an observed degree at an observed location. The rareness score may use a learning model to take into account previous stationary and/or non-stationary behavior of objects within the scene. In general, the learning model may be updated based on observed stationary and/or non-stationary behaviors of the objects. If the rareness score meets reporting conditions, the stationary object event may be reported.

Claims

1. A computer-implemented method for analyzing a scene depicted in an input stream of video frames captured by a video camera, the method comprising:

receiving, from the video camera, the input stream of video frames;

identifying an object depicted in the input stream of video frames received from the video camera;

determining, via one or more processors, whether the object has remained at the observed location in the scene for at least a threshold period;

determining whether the object has remained substantially stationary during the threshold period;

upon determining the object has remained substantially stationary and remained within the scene for at least the threshold period, determining a rareness score for the object, wherein the rareness score indicates a likelihood of the object being substantially stationary at the observed location, and wherein the rareness score is determined based at least on automatically learned stationary and non-stationary count maps indicating observed frequencies of stationary and non-stationary objects in regions of the scene, respectively, and wherein the rareness score has form

2. The method of claim 1, wherein determining whether the object has remained substantially stationary includes determining whether the bounding box of the object in each of a given number of the input stream of video frames overlap for at least the threshold period.

3. The method of claim 1, further comprising:

storing a list of substantially stationary objects no longer visible in the scene;

upon determining the object appears from behind an other object, comparing the object to the substantially stationary objects in the list; and

upon determining the object matches one of the substantially stationary objects in the stored list, adding time the object is substantially stationary before leaving the scene, time between the object leaving the scene and appearing from behind the other object, and time during which the object is substantially stationary after appearing from behind the other object, to obtain the time the object has remained at the observed location in the scene.

4. The method of claim 1, further comprising:

storing a list of objects previously associated with rareness scores that exceeded the threshold value; and

if the object is determined to have remained stationary and remained within the scene for at least the threshold period, comparing pixel locations of the object with pixel locations of objects in the stored list,

wherein the alert is generated or the object is reported upon determining pixel locations of the object do not match the pixel locations of any of the objects in the stored list.

5. The method of claim 1, further comprising, updating one or both of the stationary and non-stationary count maps based on the observed stationary and/or non-stationary behavior of the object.

6. The method of claim 5, wherein the updating includes updating pixel count value(s) in area(s) covered by the object in the stationary count map or the non-stationary count map if the object remained stationary or non-stationary, respectively.

7. A non-transitory computer-readable storage medium storing instructions, which when executed by a computer system, perform operations for analyzing a scene depicted in an input stream of video frames captured by a video camera, the operations comprising:

receiving, from the video camera, the input stream of video frames;

identifying an object depicted in the input stream of video frames received from the video camera;

determining, via one or more processors, whether the object has remained at the observed location in the scene for at least a threshold period;

determining whether the object has remained substantially stationary during the threshold period;

upon determining the object has remained substantially stationary and remained within the scene for at least the threshold period, determining a rareness score for the object, wherein the rareness score indicates a likelihood of the object being substantially stationary at the observed location, wherein the rareness score is determined based at least on automatically learned stationary and non-stationary count maps indicating observed frequencies of stationary and non-stationary objects in regions of the scene, respectively, and wherein the rareness score has form

8. The computer-readable storage medium of claim 7, wherein determining whether the object has remained substantially stationary includes determining whether the bounding box of the object in each of a given number of the input stream of video frames overlap for at least the threshold period.

9. The computer-readable storage medium of claim 7, the operations further comprising:

storing a list of substantially stationary objects no longer visible in the scene;

upon determining the object appears from behind an other object, comparing the object to the substantially stationary objects in the list; and

upon determining the object matches one of the substantially stationary objects in the stored list, adding time the object is substantially stationary before leaving the scene, time between the object leaving the scene and appearing from behind the other object, and time during which the object is substantially stationary after appearing from behind the other object, to obtain the time the object has remained at the observed location in the scene.

10. The computer-readable storage medium of claim 7, the operations further comprising:

storing a list of objects previously associated with rareness scores that exceeded the threshold value; and

if the object is determined to have remained stationary and remained within the scene for at least the threshold period, comparing pixel locations of the object with pixel locations of objects in the stored list,

wherein the alert is generated or the object is reported upon determining pixel locations of the object do not match the pixel locations of any of the objects in the stored list.

11. The computer-readable storage medium of claim 7, the operations further comprising, updating one or both of the stationary and non-stationary count maps based on the observed stationary and/or non-stationary behavior of the object.

12. The computer-readable storage medium of claim 11, wherein the updating includes updating pixel count value(s) in area(s) covered by the object in the stationary count map or the non-stationary count map if the object remained stationary or non-stationary, respectively.

13. A system, comprising:

a video camera;

a processor; and

a memory, wherein the memory includes an application program configured to perform operations for analyzing a scene depicted in an input stream of video frames captured by the video camera, the operations comprising;

receiving, from the video camera, the input stream of video frames,

identifying an object depicted in the input stream of video frames received from the video camera,

determining, via the application execution the processor, whether the object has remained at the observed location in the scene for at least a threshold period,

determining whether the object has remained substantially stationary during the threshold period,

upon determining the object has remained substantially stationary and remained within the scene for at least the threshold period, determining a rareness score for the object, wherein the rareness score indicates a likelihood of the object being substantially stationary at the observed location, wherein the rareness score is determined based at least on automatically learned stationary and non-stationary count maps indicating observed frequencies of stationary and non-stationary objects in regions of the scene, respectively, and wherein the rareness score has form

14. The system of claim 13, wherein determining whether the object has remained substantially stationary includes determining whether the bounding box of the object in each of a given number of the input stream of video frames overlap for at least the threshold period.

15. The system of claim 13, the operations further comprising:

storing a list of substantially stationary objects no longer visible in the scene;

upon determining the object appears from behind an other object, comparing the object to the substantially stationary objects in the list; and

upon determining the object matches one of the substantially stationary objects in the stored list, adding time the object is substantially stationary before leaving the scene, time between the object leaving the scene and appearing from behind the other object, and time during which the object is substantially stationary after appearing from behind the other object, to obtain the time the object has remained at the observed location in the scene.

16. The system of claim 13, the operations further comprising, updating one or both of the stationary and non-stationary count maps based on the observed stationary and/or non-stationary behavior of the object.

17. The system of claim 16, wherein the updating includes updating pixel count value(s) in area(s) covered by the object in the stationary count map or the non-stationary count map if the object remained stationary or non-stationary, respectively.