Detecting contextual trends in digital video content

• US 9,996,749 B2
• Filed: 05/27/2016
• Issued: 06/12/2018
• Est. Priority Date: 05/29/2015
• Status: Active Grant

• Current Assignee: Accenture Global Solutions Limited (Accenture PLC)
• Original Assignee: Accenture Global Solutions Limited (Accenture PLC)
• Inventors: Bataller, Cyrille, Astrom, Anders
• Primary Examiner(s): Azarian, Seyed
• Application Number:
  US15/167,576
• Publication Number:
  US 20160350596A1
• Time in Patent Office: 746 Days
• Field of Search: 382100, 382103, 382106-107, 382155, 382162, 382168, 382173, 382181, 382190, 382209, 382224, 382232, 382238, 382254, 382274, 382276, 382286-295, 382305, 382312, 707706, 340540, 600301, 702 2, 705 3
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/53 : Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/166 : using acquisition arrangements
• CPC Class Codes: G06V 40/173 : face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G08B 13/19602 : Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19608 : Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

Computerized methods and systems, including computer programs encoded on a computer storage medium, may detect events shown within digital video content captured by one or more video cameras during a prior time period, and predict an occurrence of an additional event during a future time period based on time-varying patterns among the detected events. For example, a computing system may detect events shown within digital video content captured by one or more video cameras, and may establish a predictive model that identifies one or more time-varying patterns in event parameter values that characterize the detected events within the prior time period. Based on an outcome of the predictive model, the computing system may determine an expected value of one of the event parameters during the second time period.

Claims

1. A system, comprising:

one or more computers; and

one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising;

detecting events shown within image frames of digital video content captured by one or more video cameras, the detected events being associated with corresponding event parameters and detection times within a first time period, wherein the video cameras are disposed at corresponding geographic locations and the detected events are associated with corresponding ones of the geographic locations;

applying a predictive model that includes a machine learning algorithm to values of the event parameters, the predictive model identifying a time-varying pattern in the detected events within the first time period;

based on an outcome of applying the predictive model to the values of the event parameters, determining a plurality of expected occurrences of additional events during a second time period, the additional events being associated with corresponding ones of the geographic locations and corresponding additional event parameters, and the second time period occurring after the first time period; and

transmitting, over a data communications network, data identifying at least a subset of the expected occurrences of the additional events to a communications device having a display, the communications device being configured to present a representation of each expected occurrence in the subset to a user through an interface of the display.

2. The system of claim 1, wherein:

the one or more computers further perform the operation of receiving a portion of the digital video content from the one or more video cameras; and

the one or more video cameras comprise at least one of a pan-zoom-tilt video camera or a camera having a fixed field-of-view.

3. The system of claim 1, wherein the one or more computers further perform the operation of receiving a portion of the digital video content from a video management system.

4. The system of claim 1, wherein the one or more computers further perform the operation of applying one or more video analytics to portions of the digital video content, the applied video analytics comprising at least one of an image acquisition process, an object detection process, an object classification process, an object recognition process, an event detection process, or an object tracking process.

5. The system of claim 4, wherein detecting the events comprises, based on an outcome of the applied video analytics, detecting at least one of the events and establishing the value of at least one of the event parameters.

6. The system of claim 1, wherein applying predictive model comprises:

applying at least one of the machine learning algorithm or a data mining algorithm to data identifying the values of the event parameters; and

establishing the existence of the time-varying pattern based on an outcome of the at least one machine learning algorithm or data mining algorithm.

7. The system of claim 1, wherein the determining comprises:

applying the machine learning algorithm to data identifying the time-varying pattern in the detected events within the first time period and data identifying the second time period; and

predicting the expected occurrence of the additional event based on the outcome of the machine learning algorithm; and

establishing a value for the corresponding additional event parameter based on the outcome of the machine learning algorithm.

8. The system of claim 1, wherein the operations further comprise:

based on the outcome of applying the predictive model to the values of the event parameters, determining expected occurrences of a plurality of additional events during the second time period, the additional events being associated with corresponding additional event parameters; and

transmitting data identifying at least a subset of the expected occurrences to the communications device.

9. The system of claim 1, wherein:

the detected events comprise at least one of queued individuals, individuals entering a restricted area through a corresponding entrance, or vehicles entering the restricted area through a corresponding entrance; and

the event parameters comprise at least one of a number of the queued individuals, a number of the individuals that enter the restricted area through the corresponding entrance, or a number of the vehicles that enter the restricted area through the corresponding entrance.

10. The system of claim 1, wherein:

the one or more computers further perform the operation of receiving a request to obtain the data identifying the expected occurrence of the additional event from the communications device, the request identifying at least one time during the second time period;

the determining comprises determining the expected occurrence of an additional event at the at least one time during the second time period; and

the transmitting comprises transmitting the data identifying the expected occurrence of the additional event in response to the received request, the data comprising the value of the additional parameter value.

11. A computer-implemented method, comprising:

detecting, by at least one processor, events shown within image frames of digital video content captured by one or more video cameras, the detected events being associated with corresponding event parameters and detection times within a first time period, wherein the video cameras are disposed at corresponding geographic locations and the detected events are associated with corresponding ones of the geographic locations;

applying, by the at least one processor, a predictive model that includes a machine learning algorithm to values of the event parameters, the predictive model identifying a time-varying pattern in the detected events within the first time period;

based on an outcome of applying the predictive model to the values of the event parameters, determining, by the at least one processor, a plurality of expected occurrences of additional events during a second time period, the additional events being associated with corresponding ones of the geographic locations and corresponding additional event parameters, and the second time period occurring after the first time period; and

transmitting, by the at least one processor and over a data communications network, data identifying at least a subset of the expected occurrences of the additional events to a communications device having a display, the communications device configured to present a representation of each expected occurrence in the subset to a user through an interface of the display.

12. The method of claim 11, wherein:

the method further comprises receiving a portion of the digital video content from the one or more video cameras; and

the one or more video cameras comprise at least one of a pan-zoom-tilt video camera or a camera having a fixed field-of-view.

13. The method of claim 11, further comprising applying one or more video analytics to portions of the digital video content, the applied video analytics comprising at least one of an image acquisition process, an object detection process, an object classification process, an object recognition process, an event detection process, or an object tracking process.

14. The method of claim 13, wherein detecting the events comprises, based on an outcome of the applied video analytics, detecting at least one of the events and establishing the value of at least one of the event parameters.

15. The method of claim 11, wherein applying the predictive model comprises:

applying at least one of the machine learning algorithm or a data mining algorithm to data identifying the values of the event parameters; and

establishing the existence of the time-varying pattern based on an outcome of the at least one machine learning algorithm or data mining algorithm.

16. The method of claim 11, wherein the determining comprises:

applying the machine learning algorithm to data identifying the time-varying pattern in the detected events within the first time period and data identifying the second time period; and

based on the outcome of the machine learning algorithm, predicting the expected occurrence of the additional event and establishing a value for the corresponding additional event parameter.

17. The method of claim 11, wherein:

the detected events comprise at least one of queued individuals, individuals entering a restricted area through a corresponding entrance, or vehicles entering the restricted area through a corresponding entrance; and

the event parameters comprise at least one of a number of the queued individuals, a number of the individuals that enter the restricted area through the corresponding entrance, or a number of the vehicles that enter the restricted area through the corresponding entrance.

18. One or more non-transitory computer-readable media storing instructions that, when executed by at least one processor of a client device, cause performance of operations comprising:

detecting events shown within image frames of digital video content captured by one or more video cameras, the detected events being associated with corresponding event parameters and detection times within a first time period, wherein the video cameras are disposed at corresponding geographic locations and the detected events are associated with corresponding ones of the geographic locations;

applying a predictive model that includes a machine learning algorithm to values of the event parameters, the predictive model identifying a time-varying pattern in the detected events within the first time period;

based on an outcome of applying the predictive model to the values of the event parameters, determining a plurality of expected occurrences of additional events during a second time period, the additional events being associated with corresponding ones of the geographic locations and corresponding additional event parameters, and the second time period occurring after the first time period; and

transmitting, over a data communications network, data identifying at least a subset of the expected occurrences of the additional events to a communications device having a display, the communications device configured to present a representation of each expected occurrence in the subset to a user through an interface of the display.