Method and apparatus for modifying an image by using a saliency map based on color frequency

• US 8,582,881 B2
• Filed: 03/24/2010
• Issued: 11/12/2013
• Est. Priority Date: 03/26/2009
• Status: Expired due to Fees

• Current Assignee: TP Vision Holding BV (TPV Technology Co., Ltd. (f/k/a Nanjing Huadong Electronics Information & Technology))
• Original Assignee: TP Vision Holding BV (TPV Technology Co., Ltd. (f/k/a Nanjing Huadong Electronics Information & Technology))
• Inventors: Damkat, Chris, De Haan, Gerard, Hofman, Paul Michiel, Kapadia, Dipti
• Primary Examiner(s): Do, Anh Hong
• Application Number:
  US13/256,845
• Publication Number:
  US 20120002107A1
• Time in Patent Office: 1,329 Days
• Field of Search: 382/162, 382/165, 382/168, 382/169, 382/170, 382/132, 382/133, 345/582, 600/109, 600/128, 600/160, 600/476
• US Class Current: 382/170
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12

Abstract

A method for modifying an image being defined by a matrix of pixels comprises: •calculating (1) a color frequency distribution of the matrix of pixels; •defining (3) for each pixel an energy value as a weighted function of the color frequency of the pixel, so as to define an image saliency map; •transforming (5) the image, said transformation being based on the pixel energy values. A computer software product to apply the method, an apparatus and a TV set are also disclosed.

Claims

1. Method for modifying an image, said image being defined by a matrix of pixels, comprising:

calculating a color frequency distribution of said matrix of pixels;

defining for each pixel an energy value as a weighted function of the color frequency of said pixel, so as to define a saliency map of said image;

transforming said image, said transformation being based on said pixel energy values.

2. Method according to claim 1, wherein said transformation transforms said image into a retargeted image having a different aspect ratio.

3. Method according to claim 1, wherein each pixel energy value is weighted in inverse proportion of the color frequency of said pixel.

4. Method according to claim 3, wherein the pixel energy value is weighted by a predetermined parameter when the color frequency is below a predetermined threshold.

5. Method according to claim 1, wherein the pixel energy value is weighted with respect to the spatial position of said pixel in said image.

6. Method according to claim 5, wherein the pixel energy value is weighted by the inverse of the distance of the pixel from the image center.

7. Method according to claim 1, wherein a smoothed image is calculated by applying a low pass filter to said image and the pixel energy value is weighted by a gradient function applied to the corresponding pixel of the smoothed image.

8. Method according to claim 7, wherein the low pass filter is a 2D Gaussian smoothing function.

9. Method according to claim 8, wherein the smoothing is comprised between 4 and 6.

10. Method according to claim 7, wherein, each pixel comprising at least a luminance value and a chrominance value, the gradient function is based on said pixel luminance.

11. Method according to claim 7, wherein, each pixel comprising at least a luminance value and a chrominance value, the gradient function is based on said pixel chrominance.

12. Method according to claim 2, wherein the transformation of said image into a retargeted image comprises:

selecting a seam of least energy value among a set of seams wherein a seam is a continuous path of pixels from one side to the opposite side of the image and its energy value is equal to the sum of the energy value of its pixel;

suppressing or duplicating said selected seam in said image to modify its aspect ratio.

13. Computer software product stored on a recording media and comprising a set of instructions to enable a computer to practice the method according to claim 1 when the computer executes said set of instructions.

15. TV set comprising a display with a first aspect ratio and an input device for receiving at least a source image, wherein said TV further comprises an apparatus according to claim 13 for modifying said source image into a target image having the first aspect ratio to be displayed.

14. Apparatus for modifying an image, said image being defined by a matrix of pixels, comprising:

a first calculator of a color frequency distribution of said image;

connected to

a second calculator of an energy value for each pixel, said energy value being a weighted function of the color frequency of said pixel, so as to define a saliency map of said image;

connected to

an image modifier for transforming said image, said transformation being based on said pixel energy value.