Detection of motion activity saliency in a video sequence
First Claim
1. A method comprising:
- generating, using a processor, a motion vector field including a motion vector for each of a plurality of pixels in a video;
partitioning the motion vector field into a plurality of grid blocks;
binning motion vectors assigned to each of the plurality of grid blocks into a plurality of orientation ranges;
calculating a grid block weight for each of the plurality of grid blocks based on a distance to a center of the video;
calculating, using the processor, a plurality of motion scores, wherein each of the plurality of motion scores is for a different frame of the video and is based on the grid block weight based on the distance to the center of the video and a count of motion vectors for at least one of the orientation ranges;
identifying a frame having the highest motion score for the video, the identifying based on comparing the motion scores for different frames of the video; and
generating data indicative of a thumbnail image for the frame of the video having the highest motion score.
1 Assignment
0 Petitions
Accused Products
Abstract
A streaming video or a stored video is analyzed to identify salient features. The salient features are the more interesting portions of the video because salient features include the most motion. A motion vector field including a motion vector for multiple pixels in the video is generated. The motion vector field is partitioned into grid blocks, and each of the grid blocks is divided into a set of orientation ranges. The vectors of the motion vector field for each grid block are binned into the orientation ranges. A motion score is calculated based on a count of motion vectors for the orientation ranges. The section of the video having higher or the highest motion score is designated as a salient feature. Among other applications, salient features may be used for thumbnail images, highlight reels, or video editing.
-
Citations
17 Claims
-
1. A method comprising:
-
generating, using a processor, a motion vector field including a motion vector for each of a plurality of pixels in a video; partitioning the motion vector field into a plurality of grid blocks; binning motion vectors assigned to each of the plurality of grid blocks into a plurality of orientation ranges; calculating a grid block weight for each of the plurality of grid blocks based on a distance to a center of the video; calculating, using the processor, a plurality of motion scores, wherein each of the plurality of motion scores is for a different frame of the video and is based on the grid block weight based on the distance to the center of the video and a count of motion vectors for at least one of the orientation ranges; identifying a frame having the highest motion score for the video, the identifying based on comparing the motion scores for different frames of the video; and generating data indicative of a thumbnail image for the frame of the video having the highest motion score. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. An apparatus comprising:
-
at least one processor; and at least one memory including computer program code for one or more programs;
the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least perform;calculating motion vector fields for time ranges of frames in a video, wherein each of the motion vector fields includes motion vectors that describe movement of pixels in the video; partitioning the motion vectors into a plurality of grid blocks; sorting the motion vectors within each of the plurality of grid blocks into a plurality of orientation ranges; calculating a grid block weight for each of the plurality of grid blocks based on a distance to a center of the video; applying the grid block weight to motion vectors as weighted motion vectors; comparing the weighted motion vectors in pairs of subsequent time ranges to calculate a motion score for each pair of subsequent time ranges; comparing the motion scores for each of the subsequent time ranges to select a frame having the highest motion score for the video; and generating data indicative of a thumbnail image to represent the video, the data indicative of a thumbnail image generated using the frame having the highest motion score for the video. - View Dependent Claims (9, 10, 11, 12, 13, 14, 15)
-
-
16. A non-transitory computer readable medium including instructions that when executed are operable to:
-
collect a video comprising a series of video frames; generate motion vectors for tracking movement in the series of video frames, wherein each of the motion vectors is associated with one of a plurality of grid blocks forming the series of video frames; calculate a grid block weight for each of the plurality of grid blocks based on a distance to a center of the video frames; apply the grid block weight to motion vectors as weighted motion vectors; sort weighted motion vectors assigned to each of the plurality of grid blocks into a plurality of orientation ranges; calculate a motion score based on a count of weighted motion vectors for at least one of the orientation ranges; perform a comparison of motion scores for each video frame of the series of video frames, wherein the motion scores are calculated from the weighted motion vectors; and select one or more of the video frames to represent the video as a thumbnail image or a highlight reel, the selection based on the comparison of the motion scores, the selected one or more of the video frames having the highest motion score for the video. - View Dependent Claims (17)
-
Specification