Space-time video montage
First Claim
Patent Images
1. A method at least partially implemented by a computing device, the method comprising:
- identifying one or more arbitrary space-time volumes representing one or more informative video portions of at least one input video data sequence;
segmenting the one or more informative video portions to generate one or more volumetric saliency blobs, each of the one or more volumetric saliency blobs comprising a high saliency video portion;
dilating the one or more volumetric saliency blobs using respective one or more mask volumes to simulate spread of respective high saliency video portions of the one or more volumetric saliency blobs on respective surrounding portions of the one or more volumetric saliency blobs to form one or more volumetric saliency layers; and
generating a video summary montage of the at least one input video data sequence based on the one or more volumetric saliency layers.
2 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods for space-time video montage are described. In one aspect, one or more arbitrary space-time volumes representing informative video portion(s) of at least one input video data sequence are identified. A video summary representing a montage of the at least one input video data sequence is generated for presentation to user from the one or more arbitrary space-time volumes.
-
Citations
20 Claims
-
1. A method at least partially implemented by a computing device, the method comprising:
-
identifying one or more arbitrary space-time volumes representing one or more informative video portions of at least one input video data sequence; segmenting the one or more informative video portions to generate one or more volumetric saliency blobs, each of the one or more volumetric saliency blobs comprising a high saliency video portion; dilating the one or more volumetric saliency blobs using respective one or more mask volumes to simulate spread of respective high saliency video portions of the one or more volumetric saliency blobs on respective surrounding portions of the one or more volumetric saliency blobs to form one or more volumetric saliency layers; and generating a video summary montage of the at least one input video data sequence based on the one or more volumetric saliency layers. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A computing device comprising:
-
one or more processors; and memory coupled to the processor, the memory storing computer-program instructions executable by the one or more processors, the computer-program instructions when executed by the one or more processors performing operations comprising; extracting visually informative space-time portions from video frames of an input video data sequence, the informative space-time portions including spatio-temporal saliency measuring salient texture of the visually informative space-time portions on each of the video frames; segmenting the visually informative space-time portions to obtain volumetric saliency layers, each volumetric saliency layer including a single saliency portion of the visually information space-time portions; positioning at least a subset of the volumetric saliency layers into a 3-D video volume to maximize saliency of pixels in the 3-D video volume; and merging data associated with the at least subset of the volumetric saliency layers in the 3-D video volume to regulate continuity of high-saliency portions of the pixels and provide color coherence at boundaries between respective ones of pixels in the volumetric saliency layers, wherein the 3-D video volume represents a video summary montage of the input video data sequence. - View Dependent Claims (17, 18, 19)
-
-
20. A computing device comprising:
-
a processor; and a memory coupled to the processor, the memory comprising computer-program instructions executable by the processor, the computer-program instructions when executed by the processor for performing operations comprising; receiving an input video data sequence; assigning saliency values to each pixel of the input video data sequence to obtain a saliency volume, the saliency volume comprising one or more spatio-temporal informative video portions of the input video data sequence; segmenting the one or more spatio-temporal informative video portions to generate one or more volumetric saliency blobs, each of the one or more volumetric saliency blobs comprising a set of pixels representing a high saliency video portion; dilating the one or more volumetric saliency blobs using respective one or more mask volumes to simulate spread of respective high saliency video portions on respective surrounding portions to form at least a first volumetric saliency layer and a second volumetric saliency layer; for the first volumetric saliency layer, assigning positive saliency values to locations corresponding to the high saliency portions of the first volumetric saliency layer, and assigning negative values to locations corresponding to the high saliency portions of the second volumetric saliency layer, wherein the negative values are used to reduce the importance of the high saliency portions of the second volumetric saliency layer in the first volumetric saliency layer; positioning at least a subset of information associated with the first and second volumetric saliency layers into a 3-D video volume to maximize saliency of pixels in the 3-D video volume; merging data associated with the at least a subset of the information in the 3-D video volume to regulate continuity of high-saliency portions of the pixels and provide color coherence at boundaries between respective ones of pixels in the one or more volumetric saliency layers; and presenting the 3-D volume as a video summary of the input video data sequence.
-
Specification