Investigation generation in an observation and surveillance system

• US 9,786,113 B2
• Filed: 03/14/2014
• Issued: 10/10/2017
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: James Carey
• Original Assignee: James Carey
• Inventors: Carey, James
• Primary Examiner(s): Tsai, Tsung-Yin
• Application Number:
  US14/213,548
• Publication Number:
  US 20140267735A1
• Time in Patent Office: 1,306 Days
• Field of Search: None
• CPC Class Codes: G06Q 20/20 Point-of-sale [POS] network systems
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07F 9/026 for alarm, monitoring and auditing in vending machines or means for indication, e.g. when empty
• CPC Class Codes: G07G 3/003 Anti-theft control theft alarms G08B13/00
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The present disclosure is directed to systems and methods for generating investigations of user behavior. In an example embodiment, the system includes a video camera configured to capture video of user activity, a video analytic module to perform real-time video processing of the captured video to generate non-video data from video, and a computer configured to receive the video and the non-video data from the video camera. The computer includes a video analytics module configured to analyze one of video and non-video data to identify occurrences of particular user behavior, and an investigation generation module configured to generate an investigation containing at least one video sequence of the particular user behavior. In some embodiments, the investigation is generated in near real time. The particular user behavior may be defined as an action, an inaction, a movement, a plurality of event occurrences, a temporal event and/or an externally-generated event.

Claims

1. A system to generate real-time investigations of user behavior comprising:

a video camera configured to capture video of user activity; and

a computer configured to receive video data from the video camera, the computer including;

a video analytics module configured to;

perform real-time video processing of the captured video to generate non-video data from the captured video, wherein the non-video data identifies one or more occurrences of particular user behavior occurring at one or more corresponding behavior occurrence times;

a point-of-sale (POS) module configured to;

obtain POS data from a POS system, and

identify, based on the obtained POS data, an event that corresponds to completion of a POS transaction at a POS transaction completion time;

a matching module configured to search, based on the one or more corresponding behavior occurrence times, a portion of the non-video data generated based on video captured by the video camera within a predetermined time window relative to the POS transaction completion time, to identify, among the one or more occurrences of particular user behavior, a predetermined user behavior having a temporal match with the POS transaction; and

an investigation generation module configured to, in response to the identification of the predetermined user behavior having a temporal match with the POS transaction, generate an investigation containing at least one video sequence of the particular user behavior.

2. The system according to claim 1, wherein the investigation is generated in near real time.

3. The system according to claim 1, wherein the investigation generation module assigns externally-generated data to the investigation.

4. The system according to claim 3, wherein the computer receives the externally-generated data from the POS system and the externally-generated data includes information associated with at least one POS transaction.

5. The system according to claim 1, wherein the investigation generation module is configured to simultaneously manage and populate a plurality of investigations.

6. The system according to claim 1, wherein the particular user behavior is defined as at least one of an action, an inaction, a movement, a plurality of event occurrences, a temporal event or an externally-generated event.

7. The system according to claim 1, wherein the particular user behavior is defined by a model of the particular user behavior and the video analytics module further includes:

a comparator module configured to compare the model of the particular user behavior and the non-video data.

8. A system to generate real-time investigations of user behavior comprising:

a video camera configured to capture video of user activity; and

a computer configured to receive video data from the video camera, the computer including;

a video analytics module configured to;

perform real-time video processing of the captured video to generate non-video data from the captured video, wherein the non-video data identifies one or more occurrences of particular user behavior occurring at one or more corresponding behavior occurrence times;

a point-of-sale (POS) module configured to;

obtain POS data from a POS system, and

identify, based on the obtained POS data, an event that corresponds to completion of a POS transaction at a POS transaction completion time;

a matching module configured to search, based on the one or more corresponding behavior occurrence times, a portion of the non-video data generated based on video captured by the video camera within a predetermined time window relative to the POS transaction completion time, to identify, among the one or more occurrences of particular user behavior, a predetermined user behavior having a temporal match with the POS transaction; and

an investigation generation module configured to, in response to the identification of the predetermined user behavior having a temporal match with the POS transaction, assign a video sequence related to the identified occurrence of particular user behavior to an investigation.

9. The system according to claim 8, wherein the investigation is generated in near real time.

10. The system according to claim 8, wherein the investigation generation module assigns externally-generated data to the investigation.

11. The system according to claim 8, wherein the computer receives the externally-generated data from the POS system and the externally-generated data includes information associated with at least one POS transaction.

12. The system according to claim 8, wherein the investigation generation module is configured to simultaneously manage and populate a plurality of investigations.

13. The system according to claim 8, wherein the particular user behavior is defined as at least one of an action, an inaction, a movement, a plurality of event occurrences, a temporal event or an externally-generated event.

14. The system according to claim 8, wherein the particular user behavior is defined by a model of the particular user behavior and the video analytics module further includes:

a comparator module configured to compare the model of the particular user behavior and the non-video data.

15. A method of observing behavior comprising the steps of:

receiving video data from a camera;

generating non-video data from the video data wherein the non-video data includes non-video data related to user behavior;

identifying a particular user behavior;

identifying, based on the video data, at least one occurrence of the particular user behavior referenced within the video data and occurring at at least one corresponding behavior occurrence time;

obtaining point-of-sale (POS) data from a POS system;

identifying, based on the obtained POS data, an event that corresponds to completion of a POS transaction at a POS transaction completion time;

searching, based on the at least one corresponding behavior occurrence time, a portion of the non-video data generated based on video captured by the camera within a predetermined time window relative to the POS transaction completion time, to identify, among the at least one occurrence of the particular user behavior, a predetermined user behavior having a temporal match with the POS transaction; and

generating an investigation related to the particular user behavior, in response to the identification of the predetermined user behavior having a temporal match with the POS transaction.

16. The method according to claim 15, wherein the identifying the particular user behavior further includes:

defining the particular user behavior as at least one of an action, an inaction, a movement, a plurality of event occurrences, a temporal event, or an externally-generated event.

17. A non-transitory computer-readable medium comprising software for monitoring a point of sale (POS) transaction, which software, when executed by a computer system, causes the computer system to perform operations comprising a method of:

receiving video data from a camera;

generating non-video data from the video data, wherein the non-video data includes non-video data related to user behavior;

identifying a particular user behavior;

identifying, based on the video data, at least one occurrence of the particular user behavior referenced within the non-video data and occurring at at least one corresponding behavior occurrence time;

obtaining point-of-sale (POS) data from a POS system;

identifying, based on the obtained POS data, an event that corresponds to completion of a POS transaction at a POS transaction completion time;

searching, based on the at least one corresponding behavior occurrence time, a portion of the non-video data generated based on video captured by the camera within a predetermined time window relative to the POS transaction completion time, to identify, among the at least one occurrence of the particular user behavior, a predetermined user behavior having a temporal match with the POS transaction; and

generating an investigation related to the identified occurrence of the particular user behavior, in response to the identification of the predetermined user behavior having a temporal match with the POS transaction.

18. The medium as set forth in claim 17, wherein the investigation includes video of the occurrence of the particular user behavior contained within the non-video data.

19. The medium as set forth in claim 17, wherein the POS data includes transaction data relating to at least one transaction, and wherein the method further includes:

identifying the particular user behavior temporally related to the at least one transaction; and

providing data related to the at least one transaction to the investigation.

20. The medium as set forth in claim 17, wherein the method further includes:

defining the particular user behavior as at least one of an action, an inaction, a movement, a plurality of event occurrences, a temporal event or an externally-generated event.

21. A non-transitory computer-readable medium comprising software for monitoring user behavior, which software, when executed by a computer system, causes the computer system to perform operations comprising a method of:

receiving non-video data from a camera wherein the non-video data includes user behavioral information data,

identifying a particular user behavior;

identifying at least one occurrence of the particular user behavior referenced within the non-video data and occurring at at least one corresponding behavior occurrence time;

obtaining point-of-sale (POS) data from a POS system;

identifying, based on the obtained POS data, an event that corresponds to completion of a POS transaction at a POS transaction completion time;

searching, based on the at least one corresponding behavior occurrence time, a portion of the non-video data generated based on video captured within a predetermined time window relative to the POS transaction completion time, to identify, among the at least one occurrence of the particular user behavior, a predetermined user behavior having a temporal match with the POS transaction;

identifying video of the identified occurrence of the particular user behavior; and

generating, in response to the identification of the predetermined user behavior having a temporal match with the POS transaction, an investigation related to the identified occurrence of the particular user behavior, the investigation including the identified video.

22. The medium as set forth in claim 21, wherein the method further includes:

defining the particular user behavior as at least one of an action, an inaction, a movement, a plurality of event occurrences, a temporal event or an externally-generated event.

23. The medium as set forth in claim 22, wherein the particular user behavior is defined by a model of the particular user behavior and the identifying the at least one occurrence of the particular user behavior further includes:

comparing the model of the particular user behavior to the non-video data.

24. The medium as set forth in claim 21, wherein the POS data includes transaction data relating to at least one POS transaction, and wherein the method further includes:

identifying an occurrence of the particular user behavior within the non-video data related to the at least one POS transaction.

25. The medium as set forth in claim 21, wherein the generating the investigation further includes:

providing data related to the at least one POS transaction in the investigation.