Method for detecting visual saliencies of video image based on spatial and temporal features
First Claim
1. A method for detecting visual saliencies of a video image based on spatial and temporal features, characterized in that, the method comprises the following steps:
- 1) selecting a current image from a video, dividing the current image into L non-overlapping square image blocks, each of the image blocks containing K2 pixels;
2) vectorizing each pixel in each of the image blocks into a column vector, each value in the column vectors being a R value, a G value and a B value of a RGB value of the pixel such that the column vector has a length of 3K2 values;
3) jointing column vectors of all image blocks in a row direction, to form a value matrix of the current image which has 3K2 rows and L columns;
4) performing a dimension decreasing operation on the value matrix of the current image by utilizing a principal component analysis algorithm;
5) calculating a spatial visual saliency of each image block in the current image with decreased dimensions, adding the spatial visual saliency to a position corresponding a position of the image block in the current image, to constitute a two-dimension spatial feature saliency map of the current image;
6) calculating a temporal saliency of each image block in the current image with decreased dimensions according to a block matching method, adding the temporal visual saliency to a position corresponding a position of the image block in the current image, to constitute a two-dimension temporal feature saliency map of the current image; and
7) integrating the two-dimension spatial feature saliency map and the two-dimension temporal feature saliency map, to obtain a spatiotemporal feature saliency map.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention relates to a method for detecting visual saliencies of a video image based on spatial and temporal features, including: dividing an input image into image blocks and vectorizing the image blocks; decreasing dimensions of each image block through principal component analysis; calculating a dissimilarity between each image block and each of the other image blocks; calculating a visual saliency of each image block by combining a distance between image blocks, to obtain a spatial feature saliency map; imposing a central bias on the spatial feature saliency map; calculating a motion vector of each image block, extracting a temporal visual saliency of the current image by combining motion vectors of previous two frames, to obtain a temporal feature saliency map; integrating the spatial feature saliency map and the temporal feature saliency map to obtain a spatiotemporal feature saliency map, and smoothing the spatiotemporal feature saliency map to obtain a resulted image finally reflecting a saliency of each region on the current image. In the present invention, a saliency map integrating the temporal features and the spatial features, so that saliencies in different regions in a video may be predicted more accurately.
-
Citations
11 Claims
-
1. A method for detecting visual saliencies of a video image based on spatial and temporal features, characterized in that, the method comprises the following steps:
-
1) selecting a current image from a video, dividing the current image into L non-overlapping square image blocks, each of the image blocks containing K2 pixels; 2) vectorizing each pixel in each of the image blocks into a column vector, each value in the column vectors being a R value, a G value and a B value of a RGB value of the pixel such that the column vector has a length of 3K2 values; 3) jointing column vectors of all image blocks in a row direction, to form a value matrix of the current image which has 3K2 rows and L columns; 4) performing a dimension decreasing operation on the value matrix of the current image by utilizing a principal component analysis algorithm; 5) calculating a spatial visual saliency of each image block in the current image with decreased dimensions, adding the spatial visual saliency to a position corresponding a position of the image block in the current image, to constitute a two-dimension spatial feature saliency map of the current image; 6) calculating a temporal saliency of each image block in the current image with decreased dimensions according to a block matching method, adding the temporal visual saliency to a position corresponding a position of the image block in the current image, to constitute a two-dimension temporal feature saliency map of the current image; and 7) integrating the two-dimension spatial feature saliency map and the two-dimension temporal feature saliency map, to obtain a spatiotemporal feature saliency map. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method for detecting visual saliencies of a video image based on spatial and temporal features, characterized in that, the method comprises the following steps:
-
1) acquiring a plurality of frames of images of the video with a predetermined time interval, selecting a current image from the frames of images, dividing the current image into L non-overlapping square image blocks, each of the image blocks containing K2 pixels; 2) vectorizing each pixel in each of the image blocks into a column vector, each value in the column vectors being one of a R value, a G value and a B value of a RGB value of the pixel such that the column vector has a length of 3K2 values; 3) jointing column vectors of all image blocks in a row direction, to form a value matrix of the current image which has 3K2 rows and L columns; 4) performing a dimension decreasing operation on the value matrix of the current image by utilizing a principal component analysis algorithm; 5) calculating a spatial visual saliency of each image block in the current image with decreased dimensions, adding the spatial visual saliency to a position corresponding a position of the image block in the current image, to constitute a two-dimension spatial feature saliency map of the current image; 6) calculating a temporal saliency of each image block in the current image with decreased dimensions according to a block matching method, adding the temporal visual saliency to a position corresponding a position of the image block in the current image, to constitute a two-dimension temporal feature saliency map of the current image; and 7) integrating the two-dimension spatial feature saliency map and the two-dimension temporal feature saliency map, to obtain a spatiotemporal feature saliency map.
-
Specification