Automatic extraction of secondary video streams

• US 10,614,311 B2
• Filed: 11/22/2016
• Issued: 04/07/2020
• Est. Priority Date: 03/28/2006
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Fortress Corporation
• Inventors: Scanlon, W. Andrew, Chosak, Andrew J., Clark, John I. W., Cutting, Robert A., Lipton, Alan J., Myers, Gary W.
• Primary Examiner(s): Lazaro, David R
• Assistant Examiner(s): Henry, Mariegeorges A
• Application Number:
  US15/358,780
• Publication Number:
  US 20170132471A1
• Time in Patent Office: 1,232 Days
• Field of Search: 348143, 348169, 348159, 348172
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

The automatic generation of one or more secondary video streams based on an input primary video stream may use analytics performed on the video to provide information on targets, events, and/or areas of interest to permit the one or more secondary video streams to concentrate on one or more targets, events and/or areas of interest.

Claims

1. A system to automatically generate a secondary video stream based on an incoming primary video stream, the system comprising:

a computer configured to;

perform video analytics on the primary video stream to generate one or more analysis results, the one or more analysis results including information relating to first and second targets of interest;

detect the first and second targets of interest using the analysis results, and determine the first and second targets of interest are moving targets;

automatically extract a first secondary video stream from the primary video stream, the first secondary video stream capturing at least a portion of each of the first and second targets of interest in motion, the first secondary stream having a field of view smaller than a field of view of the primary video stream;

track the first and second targets of interest; and

display the first secondary stream including the at least the portion of each of the first and second targets of interest in motion.

2. The system of claim 1, wherein the computer is configured to perform video analytics by down-sampling a higher resolution primary video stream to obtain a lower resolution primary video stream and the computer is configured to perform video analytics on the lower resolution primary video stream.

3. The system of claim 2, wherein the computer is configured to extract the first secondary video stream from the higher resolution primary video stream.

4. The system according to claim 1, wherein the computer is configured to detect foreground objects by determining regions in the primary video stream containing newly-modified objects.

5. The system of claim 1, wherein the field of view of the first secondary video stream is a portion of the field of view of the primary video stream.

6. The system of claim 1, further comprising a video camera, wherein the primary video stream is obtained directly from an image sensor of the video camera.

7. The system of claim 1, wherein the computer is configured to detect the detected first and second targets of interest in the primary video stream and to determine a first direction of motion of the detected first target of interest and a second direction of motion of the detected second target of interest, in the primary video stream.

8. The system of claim 7, wherein the computer is configured to automatically extract the first secondary video stream in response to the determined first and second directions of motion.

9. The system of claim 1, wherein the computer is configured to determine a relationship between the detected first and second targets of interest by performing video analytics on the primary video stream, wherein the first secondary video stream is generated to include the first and second targets of interest in response to the determined relationship between the detected first and second targets of interest.

10. The system of claim 1, wherein the computer is configured to generate the first secondary video stream to include first and second targets of interest in response to an analysis of movement of the first and second targets of interest in the performing of video analytics on the primary video stream.

11. The system of claim 1, wherein the computer is configured to automatically extract the first secondary video stream in order to track the detected first and second targets of interest.

12. The system of claim 1, wherein the computer is configured to automatically extract the first secondary video stream to keep each of the detected first and second targets of interest at a particular position in the first secondary video stream.

13. The system of claim 1, wherein the computer is configured to analyze the first secondary video stream to detect a first target of interest and to automatically extract a second secondary video stream in response to detecting the first target of interest.

14. The system according to claim 1, wherein the computer comprises:

a vision module configured to receive the primary video stream; and

an inference module configured to obtain results from said vision module.

15. The system according to claim 14, wherein the vision module is configured to perform target detection.

16. The system according to claim 15, wherein the vision module is configured to perform at least one operation selected from the group consisting of classifying at least one target and tracking at least one target.

17. The system according to claim 14, wherein the vision module is configured to perform camera motion compensation.

18. The system according to claim 14, wherein the vision module is configured to identify one or more foreground portions in the primary video stream, to group the one or more foreground portions into one or more targets, and to generate at least one video primitive describing at least one characteristic of at least one of said one or more targets.

19. The system according to claim 18, wherein the vision module is configured to track said one or more targets and to classify said one or more targets.

20. The system according to claim 14, wherein the inference module is configure to apply at least one rule to said results obtained from said vision module, the at least one rule selected from the group consisting of:

a target criterion, an event criterion, and an area of interest criterion.

21. The system of claim 1, wherein the computer is configured to maintain a list of one or more currently active targets of interest.

22. The system according to claim 1, wherein the computer is configured to receive configuration information from a user interface to determine how to extract the first secondary video stream.

23. A computer-implemented method of automatically generating a secondary video stream based on an incoming primary video stream, comprising:

performing video analytics on the primary video stream to generate one or more analysis results, the one or more analysis results including information relating to first and second targets of interest;

detecting the first and second targets of interest using the analysis results, and determining the first and second targets of interest are moving targets;

automatically extracting a first secondary video stream from the primary video stream, the first secondary video stream capturing at least a portion of each of the first and second targets of interest in motion, the first secondary video stream having a smaller than a field of view of the primary video stream;

tracking the first and second targets of interest; and

displaying the first secondary video stream including the at least the portion of each of the first and second targets of interest in motion.