GENERATING A SUMMARY VIDEO SEQUENCE FROM A SOURCE VIDEO SEQUENCE

• US 20170337429A1
• Filed: 05/23/2017
• Published: 11/23/2017
• Est. Priority Date: 05/23/2016
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Axis AB (Canon Inc.)
• Inventors: Ljungberg, Christian, Nilsson, Erik
• Granted Patent: US 10,192,119 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G11B 27/02 Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A method for generating a summary video sequence from a source video sequence is disclosed. The method comprises: identifying, in the source video sequence, event video sequences, wherein each event video sequence comprises consecutive video frames in which one or more objects of interest are present; extracting, from video frames of one or more event video sequences of the event video sequences, pixels depicting the respective one or more objects of interest; while keeping spatial and temporal relations of the extracted pixels as in the source video sequence, overlaying the extracted pixels of the video frames of the one or more event video sequences onto video frames of a main event video sequence of the event video sequences, thereby generating the summary video sequence. A video processing device configured to generate the summary video sequence is also disclosed.

Claims

1. A method for generating a summary video sequence from a source video sequence, the method comprising:

identifying, in the source video sequence, event video sequences, wherein each event video sequence comprises consecutive video frames in which one or more objects of interest are present;

extracting, from video frames of one or more event video sequences of the event video sequences, pixels depicting the respective one or more objects of interest; and

while keeping spatial and temporal relations of the extracted pixels as in the source video sequence, overlaying the extracted pixels of the video frames of the one or more event video sequences onto complete video frames of a main event video sequence of the event video sequences, thereby generating the summary video sequence.

2. The method according to claim 1, wherein the overlaying is subjected to a rule of keeping the number of objects of interest being present in each frame of the summary video sequence below a predetermined maximum number.

3. The method according to claim 1, wherein the overlaying is subjected to a rule of keeping a number of temporal and spatial collisions between objects of interests from different event video sequences below a predetermined maximum number.

4. The method according to claim 1, further comprising marking the objects of interest in the summary video sequence, wherein objects of interest from a first event video sequence are marked differently than objects of interest from a second event video sequence, wherein the first and second event video sequences are different event video sequences.

5. The method according to claim 4, wherein the marking comprises an outline enclosing pixels of the object of interest, wherein the outlines of objects of interest from the first event video sequence are different from the outlines of objects of interest from the second event video sequence.

6. The method according to claim 1, wherein the overlaying is performed by blending pixel data from the extracted pixels of the video frames of the one or more event video sequences with pixel data of video frames of the main event video sequence.

7. The method according to claim 1, further comprising identifying objects of interest in the source video sequence by motion detection and/or object type detection.

8. The method according to claim 1, wherein the one or more event video sequences are subsequent to the main event video sequence in the source video sequence.

9. The method according to claim 1, further comprising simultaneously generating a plurality of sub source video sequences, wherein the source video sequence comprises the plurality of sub source video sequences, wherein each event video sequence comprises a plurality of sub event video sequences, wherein each of the plurality of sub event video sequences comprises video frames from a corresponding one of the sub source video sequences, wherein the identifying a specific event video sequence is performed as long as there are one or more objects of interest in at least one of the sub source video sequences.

10. A non-transitory computer readable recording medium having computer readable program code recorded thereon which when executed on a device having processing capability is configured to perform the steps of:

identifying, in a source video sequence, event video sequences, wherein each event video sequence comprises consecutive video frames in which one or more objects of interest are present;

extracting, from video frames of one or more event video sequences of the event video sequences, pixels depicting the respective one or more objects of interest; and

while keeping spatial and temporal relations of the extracted pixels as in the source video sequence, overlaying the extracted pixels of the video frames of the one or more event video sequences onto complete video frames of a main event video sequence of the event video sequences, thereby generating a summary video sequence.

11. A video processing device configured to generate a summary video sequence from a source video sequence, the video processing device comprising:

an event video sequence identifier module configured to identify, in the source video sequence, event video sequences, wherein each event video sequence comprises consecutive video frames in which one or more objects of interest are present;

an object extractor module configured to extract, from video frames of one or more event video sequences of the event video sequences, pixels depicting the respective one or more objects of interest; and

a video frame renderer module configured to, while keeping spatial and temporal relations of the extracted pixels as in the source video sequence, overlaying the extracted pixels of the video frames of the one or more event video sequences onto complete video frames of a main event video sequence of the event video sequences, thereby generating the summary video sequence.