MULTI-OPERATOR MEDIA RETARGETING

• US 20100328352A1
• Filed: 06/24/2009
• Published: 12/30/2010
• Est. Priority Date: 06/24/2009
• Status: Active Grant

• Current Assignee: Ariel Shamir, Michael Rubinstein, The Interdisciplinary Center Herzliya
• Original Assignee: Ariel Shamir, Michael Rubinstein, The Interdisciplinary Center Herzliya
• Inventors: Rubinstein, Michael, Shamir, Ariel
• Granted Patent: US 8,400,473 B2
• Time in Patent Office: Days
• US Class Current: 345/661
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G06T 3/4092 Image resolution transcoding, e.g. by using client-server architectures
• CPC Class Codes: G09G 2340/0442 Handling or displaying different aspect ratios, or changing the aspect ratio
• CPC Class Codes: G09G 5/373 for modifying the size of the graphic pattern

Abstract

A method for image processing includes receiving in a computerized image processor a source image having a source size and a definition of a target size that is different from the source size, and defining a plurality of different sequences of two or more different image retargeting operations that when applied to the source image, generates respective target images of the target size. The different sequences are automatically tested, using the image processor, by measuring respective similarities of the respective target images to the source image. One of the sequences is selected responsively to the similarities, and the selected one of the sequences is applied to the source image, using the image processor, in order to generate a target image.

Claims

1. A method for image processing, comprising:

receiving in a computerized image processor a source image having a source size and a definition of a target size that is different from the source size;

defining a plurality of different sequences of two or more different image retargeting operations that when applied to the source image, generate respective target images of the target size;

automatically testing, using the image processor, the different sequences by measuring respective similarities of the respective target images to the source image;

selecting one of the sequences responsively to the similarities; and

applying the selected one of the sequences to the source image, using the image processor, in order to generate a target image.

2. The method according to claim 1, wherein the image retargeting operations are selected from a group of the operations consisting of scaling, cropping, seam carving, non-homogeneous scaling, image warping, and segmentation and recomposition.

3. The method according to claim 1, wherein defining the plurality of the different sequences comprises recording the sequences in a dynamic programming table representing a multi-dimensional space, and wherein selecting the one of the sequences comprises choosing a path through the space corresponding to the selected one of the sequences.

4. The method according to claim 1, wherein automatically testing the different sequences comprises computing a respective measure of a distance between the source image and each of the target images.

5. The method according to claim 4, wherein computing the respective measure of the distance comprises applying a bidirectional warping operation to the source and target images.

6. A method for video processing, comprising:

receiving in a computerized image processor a source video comprising a series of source frames having a source size and a definition of a target size that is different from the source size;

segmenting the series into at least two key source frames and a plurality of secondary source frames between the key source frames;

retargeting each of the key source frames to the target size, by the image processor, using respective first sequences of image retargeting operations comprising operators and operator parameters;

creating one or more second sequences of the image retargeting operations for application to the secondary source frames by interpolating the operator parameters between the operator parameters of the first sequences; and

retargeting the secondary source frames by applying the image retargeting operations in the second sequences to the secondary source frames.

7. A method for image processing, comprising:

presenting on a display of a computer a source image having a source image size and a plurality of on-screen user interface controls that are respectively associated with different image retargeting operations;

receiving user inputs to the computer defining a target image size and relative amounts of the different image retargeting operations to be applied to the source image, responsively to manipulation by the user of the user interface controls; and

retargeting the source image by applying the different image retargeting operations in the defined amounts so as to generate a target image of the target image size.

8. The method according to claim 7, wherein the image retargeting operations comprise at least three different image retargeting operations.

9. The method according to claim 8, wherein the at least three different image retargeting operations are selected from a group of the operations consisting of scaling, cropping, seam carving, non-homogeneous scaling, image warping, and segmentation and recomposition.

10. The method according to claim 8, wherein the on-screen user interface controls comprise at least two sliders on a single track, such that movement of one of the sliders along the single track by the user causes the computer to increase a first proportion of one of the image retargeting operations while decreasing a second proportion of at least one other of the image retargeting operations.

11. The method according to claim 7, wherein receiving the user inputs comprises defining an order in which the image retargeting operations are to be applied by the computer responsively to a sequence of the user inputs.

12. The method according to claim 11, wherein the order of the retargeting operations defines a path in a multi-dimensional retargeting space from the source image size to the target image size.

13. The method according to claim 11, wherein the user interface controls comprise multiple sliders, each slider corresponding to a different one of the image retargeting operations.

14. A computer program product for image processing, the computer program product comprising:

a computer usable medium having computer usable code embodied therewith, the computer usable program code comprising;

computer usable code configured for receiving in a computer a source image having a source size and a definition of a target size that is different from the source size;

computer usable code configured for defining a plurality of different sequences of two or more different image retargeting operations that when applied to the source image, generate respective target images of the target size;

computer usable code configured for automatically testing the different sequences by measuring respective similarities of the respective target images to the source image;

computer usable code configured for selecting one of the sequences responsively to the similarities; and

computer usable code configured for applying the selected one of the sequences to the source image in order to generate a target image.

15. A computer program product for video processing, the computer program product comprising:

a computer usable medium having computer usable code embodied therewith, the computer usable program code comprising;

computer usable code configured for receiving a source video comprising a series of source frames having a source size and a definition of a target size that is different from the source size;

computer usable code configured for segmenting the series into at least two key source frames and a plurality of secondary source frames between the key source frames;

computer usable code configured for retargeting each of the key source frames to the target size, by the image processor, using respective first sequences of image retargeting operations comprising operators and operator parameters;

computer usable code configured for creating one or more second sequences of the image retargeting operations for application to the secondary source frames by interpolating the operator parameters between the operator parameters of the first sequences; and

computer usable code configured for retargeting the secondary source frames by applying the image retargeting operations in the second sequences to the secondary source frames.

16. A computer program product for image processing, the computer program product comprising:

a computer usable medium having computer usable code embodied therewith, the computer usable program code comprising;

computer usable code configured for presenting on a display of a computer a source image having a source image size and a plurality of on-screen user interface controls that are respectively associated with different image retargeting operations;

computer usable code configured for receiving user inputs to the computer defining a target image size and relative amounts of the different image retargeting operations to be applied to the source image, responsively to manipulation by the user of the user interface controls; and

computer usable code configured for retargeting the source image by applying the different image retargeting operations in the defined amounts so as to generate a target image of the target image size.

17. An image processing apparatus, comprising:

a memory; and

an image processor, which is configured to receive into the memory a source image having a source size and a definition of a target size that is different from the source size,

define a plurality of different sequences of two or more different image retargeting operations that when applied to the source image, generate respective target images of the target size,

automatically test the different sequences so as to measure respective similarities of the respective target images to the source image,

select one of the sequences responsively to the similarities; and

apply the selected one of the sequences to the source image, in order to generate a target image.

18. A video processing apparatus, comprising:

a memory; and

an image processor, which is configured to receive into the memory a source video comprising a series of source frames having a source size and a definition of a target size that is different from the source size;

segment the series into at least two key source frames and a plurality of secondary source frames between the key source frames;

retarget each of the key source frames to the target size, using respective first sequences of image retargeting operations comprising operators and operator parameters;

create one or more second sequences of the image retargeting operations for application to the secondary source frames by interpolating the operator parameters between the operator parameters of the first sequences; and

retarget the secondary source frames by applying the image retargeting operations in the second sequences to the secondary source frames.

19. An image processing apparatus, comprising:

a display; and

an image processor, which is configured to present on the display a source image having a source image size and a plurality of on-screen user interface controls that are respectively associated with different image retargeting operations;

receive user inputs defining a target image size and relative amounts of the different image retargeting operations to be applied to the source image, responsively to manipulation by the user of the user interface controls; and

retarget the source image by applying the different image retargeting operations in the defined amounts so as to generate a target image of the target image size.