Generating a summary video sequence from a source video sequence

• US 10,192,119 B2
• Filed: 05/23/2017
• Issued: 01/29/2019
• Est. Priority Date: 05/23/2016
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Axis AB (Canon Inc.)
• Inventors: Ljungberg, Christian, Nilsson, Erik
• Primary Examiner(s): Shibru, Helen
• Application Number:
  US15/602,222
• Publication Number:
  US 20170337429A1
• Time in Patent Office: 616 Days
• Field of Search: 386278, 386290, 386241, 386287
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G11B 27/02 Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A method for generating a summary video sequence from a source video sequence is disclosed. The method comprises: identifying, in the source video sequence, event video sequences, wherein each event video sequence comprises consecutive video frames in which one or more objects of interest are present; extracting, from video frames of one or more event video sequences of the event video sequences, pixels depicting the respective one or more objects of interest; while keeping spatial and temporal relations of the extracted pixels as in the source video sequence, overlaying the extracted pixels of the video frames of the one or more event video sequences onto video frames of a main event video sequence of the event video sequences, thereby generating the summary video sequence. A video processing device configured to generate the summary video sequence is also disclosed.

Claims

1. A method for generating a summary video sequence from a source video sequence, the method comprising:

identifying, in the source video sequence, event video sequences, wherein each event video sequence comprises consecutive video frames in which one or more objects of interest are present;

extracting, from video frames of one or more event video sequences of the event video sequences, pixels depicting the respective one or more objects of interest; and

while keeping spatial and temporal relations of the extracted pixels as in the source video sequence, overlaying the extracted pixels of the video frames of the one or more event video sequences onto video frames of a main event video sequence acting as a background sequence, the main event video sequence being one of the event video sequences, thereby generating the summary video sequence.

2. The method according to claim 1, wherein the overlaying is subjected to a rule of keeping the number of objects of interest being present in each frame of the summary video sequence below a predetermined maximum number.

3. The method according to claim 1, wherein the overlaying is subjected to a rule of keeping a number of temporal and spatial collisions between objects of interests from different event video sequences below a predetermined maximum number.

4. The method according to claim 1, further comprising marking the objects of interest in the summary video sequence, wherein objects of interest from a first event video sequence are marked differently than objects of interest from a second event video sequence, wherein the first and second event video sequences are different event video sequences.

5. The method according to claim 4, wherein the marking comprises an outline enclosing pixels of the object of interest, wherein the outlines of objects of interest from the first event video sequence are different from the outlines of objects of interest from the second event video sequence.

6. The method according to claim 1, wherein the overlaying is performed by blending pixel data from the extracted pixels of the video frames of the one or more event video sequences with pixel data of video frames of the main event video sequence.

7. The method according to claim 1, further comprising identifying objects of interest in the source video sequence by motion detection and/or object type detection.

8. The method according to claim 1, wherein the one or more event video sequences are subsequent to the main event video sequence in the source video sequence.

9. The method according to claim 1, further comprising simultaneously generating a plurality of sub source video sequences, wherein the source video sequence comprises the plurality of sub source video sequences, wherein each event video sequence comprises a plurality of sub event video sequences, wherein each of the plurality of sub event video sequences comprises video frames from a corresponding one of the sub source video sequences, wherein the identifying a specific event video sequence is performed as long as there are one or more objects of interest in at least one of the sub source video sequences.

10. The method according to claim 1, wherein the act of overlaying is performed by overlaying the extracted pixels of the video frames of the one or more event video sequences onto complete video frames of the main event video sequence.

11. The method according to claim 1, wherein each event video sequence comprises consecutive video frames of the source video sequence in which one or more objects of interest are present, wherein one of the event video sequences comprises at least two objects of interest, and wherein pixels depicting the at least two objects of interest are extracted from video frames of the one or more event video sequences comprising at least two objects of interest.

12. A non-transitory computer readable recording medium having computer readable program code recorded thereon which when executed on a device having processing capability is configured to perform the steps of:

identifying, in a source video sequence, event video sequences, wherein each event video sequence comprises consecutive video frames in which one or more objects of interest are present;

extracting, from video frames of one or more event video sequences of the event video sequences, pixels depicting the respective one or more objects of interest; and

while keeping spatial and temporal relations of the extracted pixels as in the source video sequence, overlaying the extracted pixels of the video frames of the one or more event video sequences onto video frames of a main event video sequence acting as a background sequence, the main event video sequence being one of the event video sequences, thereby generating the summary video sequence.

13. A video processing device configured to generate a summary video sequence from a source video sequence, the video processing device comprising:

a memory configured to store the source video sequence; and

a processing unit configured to;

identify, in the source video sequence, event video sequences, wherein each event video sequence comprises consecutive video frames in which one or more objects of interest are present;

extract, from video frames of one or more event video sequences of the event video sequences, pixels depicting the respective one or more objects of interest; and

while keeping spatial and temporal relations of the extracted pixels as in the source video sequence, overlaying the extracted pixels of the video frames of the one or more event video sequences onto video frames of a main event video sequence acting as a background video sequence, the main event video sequence being one of the event video sequences, thereby generating the summary video sequence.

14. The video processing device according to claim 13, wherein each event video sequence comprises consecutive video frames of the source video sequence in which one or more objects of interest are present, wherein one or more of the event video sequences comprises at least two objects of interest, and wherein the object extractor module is configured to extract pixels depicting the at least two objects of interest from video frames of the one or more event video sequences comprising at least two objects of interest.