Content-Aware Image and Video Resizing by Anchor Point Sampling and Mapping
First Claim
1. A computer-readable medium carrying one or more sequences of instructions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to resize a set of images comprising pixels by performing the steps comprising:
- calculating a saliency value for at least some of the pixels in the set of images;
using the saliency values to select a set of anchor pixels, the set of anchor pixels being a subset of the pixels in the set of images;
defining a set of linear constraints comprising linear constraints based upon pixel pairs selected from the set of anchor pixels;
using the set of linear constraints to compute a set of mappings of the set of anchor pixels in the set of images to a set of target pixels in a set of target images, the set of target images having at least one dimension that is different from the set of images; and
warping the set of images to the set of target images based at least in part upon the set of mappings.
3 Assignments
0 Petitions
Accused Products
Abstract
Aspects of the present invention include systems and methods for resizing a set of images, which may comprises one or more images, while preserving the important content. In embodiments, the saliency of pixels in the set of images is determined using one or more image features. A small number of pixels, called anchor points, are selected from the set of images by saliency-based sampling. The corresponding positions of these anchor points in the set of target images are obtained using pixel mapping. In embodiments, to prevent mis-ordering of pixel mapping, an iterative approach is used to constrain the mapped pixels to be within the boundaries of the target image/video. In embodiments, based on the mapping of neighboring anchor points, other pixels in the target are inpainted by back-projection and interpolation. The combination of sampling and mapping greatly reduces the computational cost yet leads to a global solution to content-aware image/video resizing.
58 Citations
21 Claims
-
1. A computer-readable medium carrying one or more sequences of instructions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to resize a set of images comprising pixels by performing the steps comprising:
-
calculating a saliency value for at least some of the pixels in the set of images; using the saliency values to select a set of anchor pixels, the set of anchor pixels being a subset of the pixels in the set of images; defining a set of linear constraints comprising linear constraints based upon pixel pairs selected from the set of anchor pixels; using the set of linear constraints to compute a set of mappings of the set of anchor pixels in the set of images to a set of target pixels in a set of target images, the set of target images having at least one dimension that is different from the set of images; and warping the set of images to the set of target images based at least in part upon the set of mappings. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A computer-implemented method for resizing a set of input images by performing the steps comprising:
-
[a] selecting a set of pixels from the set of input images; [b] defining a set of linear constraints based upon pixel pairs from the set of pixels; [c] using the set of linear constraints to compute a set of mappings of the set of pixels in the set of input images to a set of target images, wherein the set of target images has at least one dimension that is different from the set of images; [d] responsive to the mapping resulting in pixels in a pixel pair having a different ordering in the set of target images than in the image, altering the set of linear constraints by assigning an increased weight to a linear constraint in the set of linear constraints associated with the pixel pair and returning to step [c]; and [e] using the set of mappings to warp the set of input images to the set of target images. - View Dependent Claims (11, 12, 13, 14, 15)
-
-
16. A system for resizing a set of images comprising pixels, the system comprising:
-
a saliency calculator, coupled to receive the set of images, that calculates a saliency value for at least some of the pixels in a set of images; an anchor point selector, coupled to receive the saliency values, that uses the saliency values to selects a set of anchor pixels, which is a subset of the pixels in the set of images; an anchor point mapper, coupled to receive the set of anchor pixels, that defines a set of linear constraints comprising linear constraints based upon pixel pairs selected from the set of anchor pixels and that uses the set of linear constraints to compute a set of mappings of the set of anchor pixels in the set of images to a set of target pixels in a set of target images, the set of target images having at least one dimension that is different from the set of images; and a warper, coupled to receive the set of mappings and the set of input images, that warps the set of images to the set of target images based at least in part upon the set of mappings. - View Dependent Claims (17, 18, 19, 20, 21)
-
Specification