SYSTEM, METHOD AND A COMPUTER READABLE MEDIUM FOR PROVIDING AN OUTPUT IMAGE

• US 20110199536A1
• Filed: 04/17/2008
• Published: 08/18/2011
• Est. Priority Date: 04/23/2007
• Status: Active Grant

• Current Assignee: Ramot at Tel Aviv University Limited (Tel Aviv University)
• Original Assignee: Ramot at Tel Aviv University Limited (Tel Aviv University)
• Inventors: Cohen-or, Daniel, Wolf, Lior, Guttman, Moshe
• Granted Patent: US 8,718,333 B2
• Time in Patent Office: Days
• US Class Current: 348/441
• CPC Class Codes: G06T 15/40 Hidden part removal
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G06V 10/32 Normalisation of the pattern dimensions
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

A method for providing an output image, the method includes: determining an importance value for each input pixels out of multiple input pixels of an input image; applying on each of the multiple input pixels a conversion process that is responsive to the importance value of the input pixel to provide multiple output pixels that form the output image; wherein the input image differs from the output image.

Claims

1. A method for providing an output image, the method comprising:

determining an importance value for each input pixels out of multiple input pixels of an input image;

applying on each of the multiple input pixels a conversion process that is responsive to the importance value of the input pixel to provide multiple output pixels that form the output image;

wherein the input image differs from the output image.

2. The method according to claim 1 wherein the determining comprises determining an importance input pixel mask.

3. The method according to claim 1 wherein the determining is responsive to motion associated with each of the multiple input pixels.

4. The method according to claim 3 wherein the determining is responsive to a saliency score of the input pixels.

5. The method according to claim 3 wherein the determining is responsive to an inclusion of an input pixel within an input image area that represents a face of a person.

6. The method according to claim 3 wherein the determining is responsive to an inclusion of an input pixel within an input image area that represents an object of interest.

7. The method according to claim 1 wherein a distance between output image representations of a pair of adjacent input pixels is responsive to an importance of at least one of pair of adjacent input pixels.

8. The method according to claim 1 further comprising compensating for camera induced motion.

9. The method according to claim 1 wherein the applying comprises applying an optimal mapping between the input image to the output image.

10. The method according to claim 9 wherein the applying comprises solving a set of sparse linear equations.

11. The method according to claim 1 wherein the applying is responsive to at least one of the following constraints:

each input pixel is mapped to an output pixel that is located at substantially a fixed distance from its left and right neighbors;

each input pixel is mapped to an output pixel located to substantially a similar location to which upper and lower input pixels are mapped;

an input pixel is mapped to an output pixel located substantially at a same location as an output pixel to which the same input pixel at a previous image was mapped; and

size and shape of the output image.

12. The method according to claim 1 wherein the applying comprises re-sizing.

13. The method according to claim 1 wherein the applying comprises altering an aspect ratio.

14. The method according to claim 1 wherein the input image belongs to an input image sequence and wherein the applying is responsive to a relationship between the input image and at least one other input of the input image sequence.

15. The method according to claim 1 wherein the determining is responsive to a saliency score of the input pixels;

wherein the saliency score is computed by applying a wavelet decomposition process.

16. The method according to claim 1 wherein the determining is responsive to a saliency score of the input pixels;

wherein the saliency score is computed by locating the coarsest diagonal high frequency frame in which the percentage of wavelet coefficients having values below a first threshold is below a second threshold.

17. The method according to claim 1 wherein the determining is responsive to a saliency score of the input pixels;

wherein the saliency score is computed by applying a wavelet decomposition process;

wherein the wavelet decomposition process is followed by thresholding a diagonal high frequency image to generate a binary frame;

re-scaling the binary frame; and

smoothing the re-scaled binary frame.

18. The method according to claim 1 wherein the applying is preceded by applying a data reduction stage;

and applying on each of the results of the data reduction stage a conversion process that is responsive to the importance value of the results to provide converted results.

19. The method according to claim 1 comprising generating motion based saliency of a pixel in response to the amount of motion.

20. The method according to claim 1 comprising:

determining an importance value for each input pixels out of multiple input pixels of a group of input images; and

applying on each of the multiple input pixels a conversion process that is responsive to the importance value of the input pixel to provide multiple output pixels that form group of output images;

wherein inputs image differs from output images.

21. The method according to claim 20 comprising:

generating a combined saliency matrix that includes multiple saliency matrices, each representative of a saliency of multiple pixels of an input image; and

applying the conversion process on elements of the combined saliency matrix.

22. The method according to claim 20 comprising:

applying a data reduction stage; and

applying on each of the results of the data reduction stage a conversion process that is responsive to the importance value of the results to provide converted results.

23. The method according to claim 20 comprising generating a panning effect.

24. A device for providing an output image, the device comprising:

a memory unit adapted to store an input image and

a processor, adapted to;

determine an importance value for each input pixels out of multiple input pixels of an input image and apply on each of the multiple input pixels a conversion process that is responsive to the importance value of the input pixel to provide multiple output pixels that form the output image;

wherein the input image differs from the output image.

25. The device according to claim 24 wherein the processor is adapted to determine an importance input pixel mask.

26. The device according to claim 24 wherein the processor is adapted to determine an importance of an input pixel in response to motion associated with each of the multiple input pixels.

27. The device according to claim 26 wherein the processor is adapted to determine an importance of an input pixel in response to a saliency score of the input pixels.

28. The device according to claim 26 wherein the processor is adapted to determine an importance of an input pixel in response to an inclusion of an input pixel within an input image that represents a face of a person.

29. The device according to claim 26 wherein the processor is adapted to determine an importance of an input pixel in response to an inclusion of an input pixel within an input image that represents an object of interest.

30. The device according to claim 26 wherein the processor is adapted to determine a distance between output image representations of a pair of adjacent input pixels in response to an importance of at least one of pair of adjacent input pixels.

31. The device according to claim 24 wherein the processor is adapted to compensate for camera induced motion.

32. The device according to claim 24 wherein the processor is adapted to perform an optimal mapping between the input image to the output image.

33. The device according to claim 32 wherein the processor is adapted to solve a set of sparse linear equations.

34. The device according to claim 24 wherein the processor is adapted to apply a conversion process in response to at least one of the following constraints:

each input pixel is mapped to an output pixel that is located at substantially a fixed distance from its left and right neighbors;

each input pixel is mapped to an output pixel located to substantially a similar location to which upper and lower input pixels are mapped;

an input pixel is mapped to an output pixel located substantially at a same location as an output pixel to which the same input pixel at a previous image was mapped; and

size and shape of the output image.

35. The device according to claim 24 wherein the processor is adapted to apply re-sizing.

36. The device according to claim 24 wherein the processor is adapted to alter an aspect ratio.

37. The device according to claim 24 wherein the input image belongs to an input image sequence and wherein the processor is adapted to apply a conversion process in response to a relationship between the input image and at least one other input of the input image sequence.

38. The device according to claim 24 wherein the processor is adapted to determine in response to a saliency score of the input pixels;

wherein the saliency score is computed by applying a wavelet decomposition process.

39. The device according to claim 24 wherein the processor is adapted to determine in response to a saliency score of the input pixels;

wherein the saliency score is computed by locating the coarsest diagonal high frequency frame in which the percentage of wavelet coefficients having values below a first threshold is below a second threshold.

40. The device according to claim 24 wherein the processor is adapted to determine in response to a saliency score of the input pixels;

wherein the saliency score is computed by applying a wavelet decomposition process;

wherein the wavelet decomposition process is followed by thresholding a high frequency image to generate a binary frame;

re-scaling the binary frame and smoothing the re-scaled binary frame.

41. The device according to claim 24 wherein the processor is adapted to apply a data reduction stage;

and applying on each of the results of the data reduction stage a conversion process that is responsive to the importance value of the results to provide converted results.

42. The device according to claim 24 wherein the processor is adapted to determine in response to generate motion based saliency of a pixel in response to the amount of motion.

43. The device according to claim 24 wherein the processor is adapted to:

determine in response to an importance value for each input pixels out of multiple input pixels of a group of input images; and

apply on each of the multiple input pixels a conversion process that is responsive to the importance value of the input pixel to provide multiple output pixels that form group of output images;

wherein inputs image differs from output images.

44. The device according to claim 43 wherein the processor is adapted to generate a combined saliency matrix that includes multiple saliency matrices, each representative of a saliency of multiple pixels of an input image;

and applying the conversion process on elements of the combined saliency matrix.

45. The device according to claim 43 wherein the processor is adapted to apply a data reduction stage;

and apply on each of the results of the data reduction stage a conversion process that is responsive to the importance value of the results to provide converted results.

46. The device according to claim 43 wherein the processor is adapted to generate a panning effect.

47. A computer readable medium that stores instructions for:

determining an importance value for each input pixels out of multiple input pixels of an input image; and

applying on each of the multiple input pixels a conversion process that is responsive to the importance value of the input pixel to provide multiple output pixels that form the output image;

wherein the input image differs from the output image.

48. The computer readable medium of claim 47 that stores instructions for determining an importance input pixel mask.

49. The computer readable medium of claim 47 that stores instructions for determining an importance value in response to motion associated with each of the multiple input pixels.

50. The computer readable medium of claim 47 that stores instructions for determining an importance value in response to a saliency score of the input pixels.

51. The computer readable medium of claim 50 that stores instructions for determining an importance value in response to an inclusion of an input pixel within an input image that represents a face of a person.

52. The computer readable medium of claim 50 that stores instructions for determining an importance value in response to an inclusion of an input pixel within an input image that represents an object of interest.

53. The computer readable medium of claim 50 that stores instructions for determining an importance value in response to applying a conversion process such that a distance between output image representations of a pair of adjacent input pixels is responsive to an importance of at least one of pair of adjacent input pixels.

54. The computer readable medium of claim 47 that stores instructions for determining is responsive to a saliency score of the input pixels;

wherein the saliency score is computed by applying a wavelet decomposition process.

55. The computer readable medium of claim 47 that stores instructions for determining is responsive to a saliency score of the input pixels;

wherein the saliency score is computed by locating the coarsest high frequency frame in which the percentage of wavelet coefficients having values below a first threshold is below a second threshold.

56. The computer readable medium of claim 47 that stores instructions for determining is responsive to a saliency score of the input pixels;

wherein the saliency score is computed by applying a wavelet decomposition process;

wherein the wavelet decompression process is followed by thresholding a high frequency image to generate a binary frame;

re-scaling the binary frame and smoothing the re-scaled binary frame.

57. The computer readable medium of claim 56 that stores instructions for applying a data reduction stage;

and applying on each of the results of the data reduction stage a conversion process that is responsive to the importance value of the results to provide converted results.

58. The computer readable medium of claim 47 that stores instructions for generating motion based saliency of a pixel in response to the amount of motion.

59. The computer readable medium of claim 47 that stores instructions for:

determining an importance value for each input pixels out of multiple input pixels of a group of input images; and

applying on each of the multiple input pixels a conversion process that is responsive to the importance value of the input pixel to provide multiple output pixels that form group of output images;

wherein inputs image differs from output images.

60. The computer readable medium of claim 59 that stores instructions for generating a combined saliency matrix that includes multiple saliency matrices, each representative of a saliency of multiple pixels of an input image;

and applying the conversion process on elements of the combined saliency matrix.

61. The computer readable medium of claim 59 that stores instructions for applying a data reduction stage;

and applying on each of the results of the data reduction stage a conversion process that is responsive to the importance value of the results to provide converted results.

62. The computer readable medium of claim 59 that stores instructions for generating a panning effect.

63. A method for providing an output image, the method comprising:

receiving an input frame and a feature mask defined by a rough selection of the features;

applying a mapping process to provide the output image;

wherein the mapping process differentiates between pixels of features included in the feature mask and between other pixels;

wherein the applying comprises solving a sparse equation set.

64. The method according to claim 63 wherein the mapping process applies a similarity transformation on pixels of features included in the feature mask.

65. The method according to claim 63 wherein the mapping process allows pixels of features included in the feature mask to slightly change.

66. A device for providing an output image, the device comprising:

a memory unit adapted to store an input image and a feature mask defined by a rough selection of the features;

a processor, adapted to;

apply a mapping process to provide the output image;

wherein the mapping process differentiates between pixels of features included in the feature mask and between other pixels;

wherein the applying comprises solving a sparse equation set.

67. The device according to claim 66 wherein the processor applies a similarity transformation on pixels of features included in the feature mask.

68. The device according to claim 66 wherein the processor applies allows pixels of features included in the feature mask to slightly change.

69. A computer readable medium that stores instructions for:

receiving an input frame and a feature mask defined by a rough selection of the features;

applying a mapping process to provide the output image;

wherein the mapping process differentiates between pixels of features included in the feature mask and between other pixels;

wherein the applying comprises solving a sparse equation set.

70. The computer readable medium of claim 69 that stores instructions for applying a similarity transformation on pixels of features included in the feature mask.

71. The computer readable medium of claim 69 that stores instructions for allowing pixels of features included in the feature mask to slightly change.