Systems and methods for behavioral based alarms

• US 10,140,832 B2
• Filed: 10/19/2016
• Issued: 11/27/2018
• Est. Priority Date: 01/26/2016
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Stout, Arthur, Distelzweig, John, Messely, Pieter, Janssens, Koen, Nguyen, Vu
• Primary Examiner(s): Chakraborty, Raj
• Application Number:
  US15/297,891
• Publication Number:
  US 20170213436A1
• Time in Patent Office: 769 Days
• Field of Search: 382103
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 21/0423 detecting deviation from an expected pattern of behaviour or schedule

Abstract

Various embodiments of the present disclosure may include an imaging system. The imaging system may include a controller that is adapted to determine behavior typical of an object within an environment. The imaging system may additionally monitor the object within the environment and detect when the object behaves anomalously within the environment. Upon detection of anomalous behavior, the controller may output an alarm and/or a response message.

Claims

1. A system comprising:

a first camera configured to acquire a plurality of first image frames of at least a first scene and output first image data; and

a controller configured to communicate with the first camera and configured to;

receive first period data, wherein the first period data is first image data associated with a first time period,

process the first period data to generate processed first period data of binary images,

analyze the processed first period data to determine a presence of an object of interest within at least one image frame of the processed first period data and highlight the object of interest within the at least one image frame of the processed first period data,

analyze movement of the highlighted object of interest within the at least one image frame of the processed first period data to determine a historical movement pattern of the object within the first scene,

store the historical movement pattern,

receive, after the storing the historical movement pattern, second period data, wherein the second period data is first image data associated with a second time period,

process the second period data to generate processed second period data of binary images,

analyze the processed second period data to determine a presence of the object of interest within at least one image frame of the processed second period data and highlight the object of interest within the at least one image frame of the processed second period data,

analyze movement of the highlighted object of interest within the at least one image frame of the processed second period data to determine a recent movement pattern of the object,

compare the recent movement pattern of the object to the stored historical movement pattern of the object,

determine, responsive to the comparison, that an anomalous trigger condition has been met, and

output an alarm or response message in response to determining that the anomalous trigger condition has been met.

2. The system of claim 1, further comprising a second camera communicatively connected to the controller and configured to acquire a plurality of second image frames of at least the first scene and output second image data, wherein the controller is further configured to:

receive the second image data;

analyze the second image data to determine the presence of the object of interest within at least one image frame of the processed first image data and/or the second image data, wherein generating the historical movement pattern of the object within the first scene comprises generating the historical movement pattern with the first image data and/or the second image data.

3. The system of claim 2, wherein the first camera is a thermal camera and the second camera is a visual and/or depth camera.

4. The system of claim 1, wherein the first camera is a thermal, visual, and/or depth camera.

5. The system of claim 1, wherein the historical movement pattern is associated with a presence of the object of interest within at least a plurality of image frames.

6. The system of claim 1, wherein the historical movement pattern comprises a plurality of movement paths and a likelihood associated with each of the plurality of movement paths.

7. The system of claim 1, wherein the controller is further configured to:

communicate a response message requesting a human response;

determine that no acceptable response is received responsive to the response message; and

cause an alarm to be triggered.

8. The system of claim 1, wherein comparing the recent movement pattern of the object to the historical movement pattern of the object comprises comparing a location of the object to the historical movement pattern.

9. The system of claim 1, wherein the controller is further configured to:

analyze the processed first period data to determine a first size of the object within the processed first period data; and

analyze the processed second period data to determine a second size of the object within the processed second period data, wherein comparing the recent movement pattern of the object to the historical movement pattern of the object comprises determining a posture of the object by comparing an aspect ratio of the second size to the first size.

10. The system of claim 1, wherein comparing the recent movement pattern of the object to the historical movement pattern of the object comprises determining that the object is behind an obstacle.

11. The system of claim 1, wherein comparing the recent movement pattern of the object to the historical movement pattern of the object comprises determining a dwell time of the object and comparing the dwell time to the historical movement pattern.

12. The system of claim 1, wherein generating the historical movement pattern of the object within the first scene comprises determining one or more environmental features of the first scene and comparing the recent movement pattern of the object to the historical movement pattern of the object comprises determining a location of the object relative to the one or more environment features.

13. The system of claim 1, wherein the object is a person.

14. The system of claim 1, wherein the highlighting the object of interest comprises one or more of rendering the object in a pixel value different from a pixel value for rendering a background within the first scene, rendering a box around an outside dimension of the object, and rendering the object in a line format.

15. A method comprising:

receiving first period data, wherein the first period data is first image data of a first scene from a first camera associated with a first time period;

processing the first period data to generate processed first period data of binary images;

analyzing the processed first period data to determine a presence of an object of interest within at least one image frame of the processed first period data and highlighting the object of interest within the at least one image frame of the processed first period data;

analyzing movement of the highlighted object of interest within the at least one image frame of the processed first period data to determine a historical movement pattern of the object within the first scene;

storing the historical movement pattern;

receiving, after the storing the historical movement pattern, second period data, wherein the second period data is first image data associated with a second time period;

processing the second period data to generate processed second period data of binary images;

analyzing the processed second period data to determine a presence of the object of interest within at least one image frame of the processed second period data and highlighting the object of interest within the at least one image frame of the processed second period data;

analyzing movement of the highlighted object of interest within the at least one image frame of the processed second period data to determine a recent movement pattern of the object;

comparing the recent movement pattern of the object to the stored historical movement pattern of the object;

determining, responsive to the comparison, that an anomalous trigger condition has been met; and

outputting an alarm or response message in response to determining that the anomalous trigger condition has been met.

16. The method of claim 15, further comprising:

receiving second image data of the first scene from a second camera;

analyzing the second image data to determine the presence of the object of interest within at least one image frame of the processed first image data and/or the second image data, wherein generating the historical movement pattern of the object within the first scene comprises generating the historical movement pattern with the first image data and/or the second image data.

17. The method of claim 15, wherein the historical movement pattern is associated with a presence of the object of interest within at least a plurality of image frames.

18. The method of claim 15, wherein the historical movement pattern comprises a plurality of movement paths and a likelihood associated with each of the plurality of movement paths.

19. The method of claim 15, further comprising:

communicating a response message requesting a human response;

determining that no acceptable response is received responsive to the response message; and

causing an alarm to be triggered.

20. The method of claim 15, further comprising:

analyzing the processed first period data to determine a size of the object, wherein comparing the recent movement pattern of the object to the historical movement pattern of the object comprises

determining a posture of the object by comparing an aspect ratio of the object to the determined size of the object.