ANOMALOUS STATIONARY OBJECT DETECTION AND REPORTING

• US 20140002647A1
• Filed: 06/28/2013
• Published: 01/02/2014
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: XU, Gang, COBB, Wesley Kenneth
• Granted Patent: US 9,723,271 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

Techniques are disclosed for analyzing a scene depicted in an input stream of video frames captured by a video camera. The techniques include receiving data for an object within the scene and determining whether the object has remained substantially stationary within the scene for at least a threshold period. If the object is determined to have remained stationary for at least the threshold period, a rareness score is calculated for the object to indicate a likelihood of the object being stationary to the observed degree at the observed location. The rareness score may use a learning model to take into account previous stationary and/or non-stationary behavior of objects within the scene. In general, the learning model may be updated based on observed stationary and/or non-stationary behaviors of the objects. If the rareness score meets reporting conditions, the stationary object event may be reported.

Claims

1. A computer-implemented method for analyzing a scene depicted in an input stream of video frames captured by a video camera, the method comprising:

determining, via one or more processors, whether an object has remained at an observed location in the scene for at least a threshold period;

determining whether the object has remained substantially stationary during the threshold period;

if the object is determined to have remained substantially stationary and remained within the scene for at least the threshold period, determining a rareness score for the object, wherein the rareness score indicates a likelihood of the object being substantially stationary at the observed location, and wherein the rareness score is determined based at least on a learned model of previous object behavior within the scene; and

upon determining the rareness score exceeds a threshold, generating an alert.

2. The method of claim 1, further comprising, updating the learned model based on the observed stationary and/or non-stationary behavior of the object.

3. The method of claim 2, wherein the updating includes updating pixel count value(s) in area(s) covered by the object in a stationary count map or a non-stationary count map if the object remained stationary or non-stationary, respectively.

4. The method of claim 1, wherein the rareness score has form

5. The method of claim 1, wherein determining whether the object has remained substantially stationary includes determining if a bounding box of the object in each of a given number video frames overlap for at least the threshold period.

6. The method of claim 1, further comprising:

storing a list of substantially stationary objects no longer visible in the scene;

upon determining the object appears from behind a second object, comparing the object to the substantially stationary objects in the list; and

upon determining the object matches one of the substantially stationary objects in the stored list, adding time the object is substantially stationary before leaving the scene, time between the object leaving the scene and appearing from behind the second object, and time during which the object is substantially stationary after appearing from behind the other object, to obtain the time the object has remained at the observed location in the scene.

7. The method of claim 1, further comprising:

storing a list of objects previously associated with rareness scores that exceeded the threshold; and

if the object is determined to have remained stationary and remained within the scene for at least the threshold period, comparing pixel locations of the object with pixel locations of objects in the stored list,

wherein the alert is generated or the object is reported only if pixel locations of the object do not match the pixel locations of any of the objects in the stored list.

8. A non-transitory computer-readable storage medium storing instructions, which when executed by a computer system, perform operations for analyzing a scene depicted in an input stream of video frames captured by a video camera, the operations comprising:

determining, via one or more processors, whether an object has remained at an observed location in the scene for at least a threshold period;

determining whether the object has remained substantially stationary during the threshold period;

if the object is determined to have remained substantially stationary and remained within the scene for at least the threshold period, determining a rareness score for the object, wherein the rareness score indicates a likelihood of the object being substantially stationary at the observed location, and wherein the rareness score is determined based at least on a learned model of previous object behavior within the scene; and

upon determining the rareness score exceeds a threshold, generating an alert.

9. The computer-readable storage medium of claim 8, the operations further comprising, updating the learned model based on the observed stationary and/or non-stationary behavior of the object.

10. The computer-readable storage medium of claim 9, wherein the updating includes updating pixel count value(s) in area(s) covered by the object in a stationary count map or a non-stationary count map if the object remained stationary or non-stationary, respectively.

11. The computer-readable storage medium of claim 8, wherein the rareness score has form

12. The computer-readable storage medium of claim 8, wherein determining whether the object has remained substantially stationary includes determining if a bounding box of the object in each of a given number video frames overlap for at least the threshold period.

13. The computer-readable storage medium of claim 8, the operations further comprising:

storing a list of substantially stationary objects no longer visible in the scene;

upon determining the object appears from behind a second object, comparing the object to the substantially stationary objects in the list; and

upon determining the object matches one of the substantially stationary objects in the stored list, adding time the object is substantially stationary before leaving the scene, time between the object leaving the scene and appearing from behind the second object, and time during which the object is substantially stationary after appearing from behind the other object, to obtain the time the object has remained at the observed location in the scene.

14. The computer-readable storage medium of claim 8, the operations further comprising:

storing a list of objects previously associated with rareness scores that exceeded the threshold; and

if the object is determined to have remained stationary and remained within the scene for at least the threshold period, comparing pixel locations of the object with pixel locations of objects in the stored list,

wherein the alert is generated or the object is reported only if pixel locations of the object do not match the pixel locations of any of the objects in the stored list.

15. A system, comprising:

a processor; and

a memory, wherein the memory includes an application program configured to perform operations for analyzing a scene depicted in an input stream of video frames captured by a video camera, the operations comprising;

determining, via one or more processors, whether an object has remained at an observed location in the scene for at least a threshold period,

determining whether the object has remained substantially stationary during the threshold period,

if the object is determined to have remained substantially stationary and remained within the scene for at least the threshold period, determining a rareness score for the object, wherein the rareness score indicates a likelihood of the object being substantially stationary at the observed location, and wherein the rareness score is determined based at least on a learned model of previous object behavior within the scene, and

upon determining the rareness score exceeds a threshold, generating an alert.

16. The system of claim 15, the operations further comprising, updating the learning model based on the observed stationary and/or non-stationary behavior of the object.

17. The system of claim 16, wherein the updating includes updating pixel count value(s) in area(s) covered by the object in a stationary count map or a non-stationary count map if the object remained stationary or non-stationary, respectively.

18. The system of claim 15, wherein the rareness score has form

19. The system of claim 15, wherein determining whether the object has remained substantially stationary includes determining if a bounding box of the object in each of a given number video frames overlap for at least the threshold period.

20. The system of claim 15, the operations further comprising:

storing a list of substantially stationary objects no longer visible in the scene;

upon determining the object appears from behind a second object, comparing the object to the substantially stationary objects in the list; and

upon determining the object matches one of the substantially stationary objects in the stored list, adding time the object is substantially stationary before leaving the scene, time between the object leaving the scene and appearing from behind the second object, and time during which the object is substantially stationary after appearing from behind the other object, to obtain the time the object has remained at the observed location in the scene.