Method for Retargeting Images

• US 20080219587A1
• Filed: 03/06/2007
• Published: 09/11/2008
• Est. Priority Date: 03/06/2007
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Avidan, Shmuel, Shamir, Ariel
• Granted Patent: US 7,747,107 B2
• Time in Patent Office: Days
• US Class Current: 382/276
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12

Abstract

A method for content-aware image retargeting generates an energy image from a source image according to an energy function. From the energy image, one or more seams are determined according to a minimizing function such that each seam has a minimal energy. Each seam is applied to the source image to obtain a target image that preserves content and a rectangular shape of the source image.

Claims

1. A method for content-aware image retargeting, comprising:

generating an energy image from a source image according to an energy function;

determining, from the energy image, one or more seams according to a minimizing function such that each seam has a minimal energy; and

applying each seam to the source image to obtain a target image that preserves content and a rectangular shape of the source image.

2. The method of claim 1, in which pixels in the source image are indexed in a vertical dimension by indices n and in a horizontal dimension by indices m, and each minimal energy seam extends from one edge of the source image to an opposite edge of the image, and a number of pixels in each seam corresponds to the indices of the dimension in which the seam is oriented, and the seams include vertical seams and horizontal seams.

3. The method of claim 2, further comprising:

applying the seams in an optimal order according to an objective function.

4. The method of claim 1, in which a particular seem is applied repeatedly.

5. The method of claim 1, in which the applying changes a size of the source image.

6. The method of claim 5, in which the size of the source image is increased.

7. The method of claim 5, in which the size of the source image is decreased.

8. The method of claim 1, in which the applying changes an aspect ratio of the source image.

9. The method of claim 1, in which the applying removes an object from the source image.

10. The method of claim 1, in which the applying enlarges the content of the source image while preserving a size of the source image.

11. The method of claim 1, in which the generating of the energy map is performed in an intensity domain of the source image.

12. The method of claim 1, in which the generating of the energy map is performed in a gradient domain of the source image.

13. The method of claim 1, in which the energy function is a gradient magnitude.

14. The method of claim 1, in which the energy function is entropy.

15. The method of claim 1, in which the energy function is according to visual saliency.

16. The method of claim 1, in which the energy function is according to eye-gaze movement.

17. The method of claim 3, further comprising:

storing the optimal order to enable changing a size of the source image in real time.

18. The method of claim 2, in which pixels to neighboring pixels to have a low energy, and pixels dissimilar to neighboring pixels have a high energy.

19. The method of claim 1, in which the energy function is

20. The method of claim 1, in which the applying maximizes energy of the target image.

21. The method of claim 3, in which the vertical seam is:

s^x={s_i^x}_i=1ⁿ={(x(i), i)}_i=1ⁿ, s.t. ∀

i,|x(i)=x(i−

1)|≦

1,

where x is a mapping x;

[1, . . . , n]→

[1, . . . , m], and the horizontal seam is;

s^y={s_j^y}_j=1^m={(j,y(j))}_j=1^m, s.t. ∀

j|y(j)−

y(j−

1)|≦

1,

where y is a mapping y;

[1, . . . , m]→

[1, . . . , n].

22. The method of claim 21, in which the pixels of the seam are eight-connected.

23. The method of claim 21, in which the pixels of the seams are piece-wise connected.

24. The method of claim 21, in which the pixels of the seams are disconnected.

25. The method of claim 21, the pixels of the vertical seam {s_i} 101, are

I_s={I(s_i)}_i=1ⁿ={I(x(i), i)}_i=1ⁿ,

and the energy of the vertical seam is

E(s)=E(I_s)=Σ

_i=1ⁿe(I(s_i)),

and the optimal minimal energy seam s* is

26. The method of claim 25, in which the optimal minimal energy seam s* is found using dynamic programming.

27. The method of claim 1, in which the energy function is according to user input.

28. The method of claim 1, in which the energy function is according to abject detection.

29. The method of claim 2, further comprising:

constructing horizontal and vertical index seam maps T in eluding entries T(r, c).

30. The method of claim 3, in which the optimal order of horizontal and vertical seam removal is computed using