Thumbnail manipulation using fast and aspect ratio zooming, compressing and scaling

• US 6,545,687 B2
• Filed: 01/05/1998
• Issued: 04/08/2003
• Est. Priority Date: 01/09/1997
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Long, Timothy M., Scott, Paul Q., Coleman, Nicolas L., Antill, Richard M., Thorp, Jeremy D. M.
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US09/002,883
• Publication Number:
  US 20020000998A1
• Time in Patent Office: 1,919 Days
• Field of Search: 345/433, 345/439, 345/438, 345/629, 345/643, 345/645, 345/634, 345/637, 345/638
• US Class Current: 345/629
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06T 3/4084 in the transform domain, e.g. fast Fourier transform [FFT] domain scaling
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets

Abstract

A method and apparatus manipulates thumbnail images as used in image-based browsing file management systems. Zooming in and out of thumbnail images can be performed without a continued need to decompress a true image thus providing for faster operation. Pixel interpolation and/or replication are used to generate intermediate images that are displayed to deliver to the user a perception of a transitory zoom yet are of sufficient detail to maintain user orientation. Aspect ratio zooming of thumbnail containment areas is also disclosed which facilitates ease of browsing. The compression of thumbnail type images using a discrete wavelet transform facilitates the fast zoom of thumbnails and their associated containment areas.

Claims

1. A method of zooming an image on a display device, said method comprising the successive steps of:

displaying a first image at a first predetermined size;

stretching or shrinking the first image to one or more predetermined image sizes;

successively displaying at least one modified image, each one of said modified image being related to a respective one of said stretched or shrunken images; and

thereafter displaying a second image related to a zoomed version of said first image at a second predetermined image size different from said first predetermined image size.

2. A method according to claim 1, wherein said first representation of said image is stored on a memory storage means in an encoded format and said step of displaying said first representation further includes a decoding of said image for display.

3. A method according to claim 2, wherein the encoded format is a compressed image format.

4. A method according to claim 3, wherein the compressed image format is a wavelet coding technique.

5. A method according to claim 4, wherein said wavelet coding technique comprises the SWEET method.

6. A method according to claim 3, wherein the compressed image format is achieved by a JPEG coding technique.

7. A method according to claim 6, wherein said single image is stored as a single compressed image and wherein the first and second images are extracted from the single compressed image.

8. A method according to claim 1, wherein said image is stored on a memory storage means in an encoded format and said step of displaying said further representation further includes a decoding of said further representation of said image for display.

9. A method according to claim 8, wherein said image is coded into said encoded format by a method of representing a digital image to provide a coded representation, said method comprising the steps of:

transforming said digital image to derive a plurality of coefficients, each coefficient represented by a predefined bit sequence;

selecting a portion of said plurality of coefficients as a region;

(a) scanning the significance of each bitplane of said selected region from a most significant bitplane towards a least significant bitplane and providing a first token in said coded representation for each insignificant bitplane until a significant bitplane is determined, wherein a second token is provided in said coded representation for said significant bitplane;

(b) partitioning said selected region into two or more subregions having a predetermined form, and setting each of said subregions as said selected region;

(c) repeating steps (a) and (b) commencing from said significant bitplane until said selected region has a predetermined size, wherein said coefficients of said selected region are coded and provided in said coded representation.

10. A method according to claim 9, wherein said transforming step comprises applying a discrete wavelet transform to said digital image.

11. A method according to claim 10, wherein said portion comprises a subband of said plurality of coefficients.

12. A method according to claim 9, wherein said region comprises said entire plurality of coefficients.

13. A method according to claim 9, wherein said first and second tokens comprise bit values of 0 and 1, respectively.

14. A method according to claim 9, wherein said subregions are equally sized.

15. A method according to claim 14, wherein said subregions are square.

16. A method according to claim 9, wherein said predetermined size of said subregion is a 1×

1 coefficient.

17. A method according to claim 16, wherein said 1×

1 coefficient is encoded by outputting bits of said corresponding bit sequence beginning with the respective significant bitplane.

18. A method according to claim 17, wherein only bits of said corresponding bit sequence above a predetermined minimum bit level are output in said coded representation.

19. A method according to claim 9, wherein, in step (c), steps (a) and (b) are repeated otherwise until each bitplane of said selected region has been scanned.

20. A method according to claim 19, wherein each bitplane of said selected region above a minimum bit level has been scanned.

21. A method according to claim 1, wherein the first and second image are different resolutions of a single image.

22. A method according to claim 1, wherein said stretching or shrinking comprises adjusting the size of the first image by manipulating pixels of the first image.

23. A method according to claim 22, wherein said stretching comprises mathematically manipulating values of groups of pixels to provide at least one pixel value, and replicating said at least one pixel value.

24. A method according to claim 22, wherein said shrinking comprises mathematically manipulating groups of pixels to provide at least one pixel value, and assigning said one pixel value to a limited number of pixels smaller than that contained in said groups.

25. A method according to claim 24 wherein said assigning comprises discarding pixels from said image.

26. A method of zooming an image on a display device, said method comprising the successive steps of:

(a) decompressing a compressed true image to a first representation of said image at a predetermined size, and displaying said first representation;

(b) if the predetermined size is smaller than a desired image size then

(i) stretching the representation to a larger image size and displaying said stretched representation; and

(ii) repeating step (b)(i) until said desired image size is reached;

(c) if the predetermined size is larger than a desired image size then

(i) shrinking the representation to a smaller image size and displaying said shrunk representation; and

(ii) repeating step (c)(i) until said desired image size is reached, and

(d) decompressing the compressed true image to a further zoomed representation at said desired image size being different from the predetermined size and displaying said further representation.

27. A method according to claim 26, wherein step (d) occurs substantially simultaneously during steps (a) (b) and (c).

28. Apparatus for zooming an image for display on a display device, said apparatus comprising:

means for displaying a first representation of said image at a first predetermined size;

means for stretching or shrinking said first representation to at least one further predetermined size;

means for successively displaying said stretched or shrunken representation at each said further predetermined size, and

means for displaying a second representation of said image related to a zoomed version of said first image, at a second predetermined size being different from said first predetermined image size.

29. Apparatus according to claim 28, wherein means for displaying said second representation operates substantially simultaneously with the other said means.

30. A computer program product comprising a computer readable medium having a computer program recorded thereon for zooming images for display on a display device, said computer program product comprising:

means for displaying a first representation of said image at a first predetermined size;

means for stretching or shrinking said first representation to at least one further predetermined size;

means for successively displaying said stretched or shrunken representation at each said further predetermined size, and

means for displaying a second representation of said image related to a zoomed version of said first image at a second predetermined size being different from said first predetermined image size.

31. A method of zooming thumbnails of digital images for use in image browsing, said method comprising the steps of:

laying out said thumbnails in at least one containment area having the same aspect ratio as an available display area of a display means; and

scaling said thumbnails contained in said at least one containment area to fit within the available area of said containment area.

32. A method according to claim 31, further comprising the step of increasing the scale of said thumbnails if increased detail of said thumbnails is required for assessment.

33. A method according to claim 32, wherein said thumbnails are scaled to be as large as possible and fit within said available display area.

34. A method according to claim 31, further comprising the step of organising said thumbnails into two or more groups in respective containment areas.

35. A method according to claim 34, wherein said groups have a hierarchical structure.

36. A method according to claim 34, wherein one or more groups laid out in respective containment areas comprises two or more subgroups of thumbnails laid out in respective subcontainment areas, each subcontainment area having the same aspect ratio as the parent containment area and said displayable area.

37. A method according to claim 31, wherein one or more of said digital images is generated using a digital image capture device.

38. A method according to claim 31, wherein said scaling comprises zooming said thumbnails according to a method comprising the steps of:

displaying a first representation of said image at a first predetermined size, said first representation having a first predetermined resolution;

stretching or shrinking the first representation to provide one or more modified representations;

successively displaying said one or more modified representations; and

thereafter displaying a further representation of said image at a second predetermined size, said further representation having a further predetermined resolution;

wherein said further predetermined size and resolution are different to said first predetermined size and resolution.

39. Apparatus for zooming thumbnails of digital images for use in image browsing, said thumbnails being displayable on a display means, said apparatus comprising:

means for laying out said thumbnails in at least one containment area having the same aspect ratio as an available display area of a display means; and

means for scaling said thumbnails contained in said at least one containment area to fit within the available area of said containment area.

40. Apparatus according to claim 39, further comprising:

a computer;

a computer readable medium having recorded thereon a computer program for zooming said digital images, said computer program carrying out the steps of;

laying out said thumbnails in at least one containment area having the same aspect ratio as an available display area of a display means; and

scaling said thumbnails contained in said at least one containment area to fit within the available area of said containment area, wherein

said computer program can be loaded into and run by said computer to implement said apparatus.

41. A computer program product having a computer readable medium having a computer program recorded thereon for zooming thumbnails of digital images for use in image browsing, said computer program product comprising:

means for laying out said thumbnails in at least one containment area having the same aspect ratio as an available display area of a display means; and

means for scaling said thumbnails contained in said at least one containment area to fit within the available area of said containment area.

42. A method of scaling thumbnails representing digital images, said method comprising the step of:

encoding a digital image using a hierarchical representation to provide a thumbnail.

43. A method according to claim 42, further comprising the step of decoding said hierarchical representation of said digital image at a predetermined one of a plurality of scales to provide said thumbnail having a desired size.

44. A method according to claim 43, wherein the scale of said decoded hierarchical representation is equal to or greater than the desired size of said thumbnail.

45. A method according to claim 43, further comprising the step of downsampling the decoded hierarchical representation to provide the desired-size thumbnail if the scale of said decoded hierarchical representation is greater than the desired size of said thumbnail.

46. A method according to claim 43, further comprising the step of upsampling the decoded hierarchical representation to provide the desired-size thumbnail if the scale of said decoded hierarchical representation is less than the desired size of said thumbnail.

47. A method according to claim 43, further comprising the step of displaying said thumbnail.

48. A method according to claim 43, further comprising the step of caching said hierarchical representation and/or said decoded hierarchical representation.

49. A method according to claim 48, further comprising the step of caching intermediate size or scaled versions of said decoded hierarchical representation.

50. A method according to claim 42, further comprising the step of decoding said hierarchical representation of said digital image at two different adjacent scales of a plurality of scales to provide said thumbnail having a desired size.

51. A method according to claim 50, further comprising the step of interpolating between the two decoded hierarchical representations having different adjacent scales to provide the desired-size thumbnail if the scale of one of said two decoded hierarchical representations is greater than the desired thumbnail size and the other of said two decoded hierarchical representations is less than the desired thumbnail size.

52. A method according to claim 42, wherein said encoding step includes the step of applying a hierarchical discrete wavelet transform to said image.

53. A method according to claim 52, wherein the encoding comprises the steps of:

transforming said digital image to derive a plurality of coefficients, each coefficient represented by a predefined bit sequence;

selecting a portion of said plurality of coefficients as a region;

(a) scanning the significance of each bitplane of said selected region from a most significant bitplane towards a least significant bitplane and providing a first token in said coded representation for each insignificant bitplane until a significant bitplane is determined, wherein a second token is provided in said coded representation for said significant bitplane;

(b) partitioning said selected region into two or more subregions having a predetermined form, and setting each of said subregions as said selected region;

(c) repeating steps (a) and (b) commencing from said significant bitplane until said selected region has a predetermined size, wherein said coefficients of said selected region are coded and provided in said coded representation.

54. A method according to claim 53, wherein said transforming step comprises applying a discrete wavelet transform to said digital image.

55. A method according to claim 54, wherein said portion comprises a subband of said plurality of coefficients.

56. A method according to claim 53, wherein said region comprises said entire plurality of coefficients.

57. A method according to claim 53, wherein said first and second tokens comprise bit values of 0 and 1, respectively.

58. A method according to claim 53, wherein said subregions are equally sized.

59. A method according to claim 58, wherein said subregions are square.

60. A method according to claim 53, wherein said predetermined size of said subregion is a 1×

1 coefficient.

61. A method according to claim 60, wherein said 1×

1 coefficient is encoded by outputting bits of said corresponding bit sequence beginning with the respective significant bitplane.

62. A method according to claim 61, wherein only bits of said corresponding bit sequence above a predetermined minimum bit level are output in said coded representation.

63. A method according to claim 53, wherein, in step (c), steps (a) and (b) are repeated otherwise until each bitplane of said selected region has been scanned.

64. A method according to claim 63, wherein each bitplane of said selected region above a minimum bit level has been scanned.

65. A method according to claim 52, wherein said encoding comprises the steps of:

decomposing said image using a subband transform to provide a number of subbands;

for each subband, selecting said subband as an initial region and performing the following sub-steps;

(a) checking if a current bit level of the selected region is significant;

(b) if said current bit level is significant, outputting a first token in a coded representation and partitioning said selected region into a number of equally sized sub-regions, wherein each sub-region is processed as said selected region in turn;

(c) if said current bit level is insignificant, outputting a second token in said coded representation and selecting the next lower bit level of said selected region as said current bit level;

(d) repeating steps (a) to (c) until said current bit level is less than a specified minimum bit level, or said selected region has a predetermined size and coefficients of said selected region are coded in said coded representation.

66. A method according to claim 65, wherein said coefficients of said selected region are coded by representing each coefficient by the bits between the corresponding current bit level and the minimum bit level.

67. A method according to claim 65, wherein said subband transform comprises a discrete wavelet transform.

68. A method according to claim 52, wherein said encoding comprises the steps of:

a) dividing the digital image into a plurality of blocks;

b) applying a subband transform to each of said blocks to provide a plurality of AC subband regions, at one or more resolutions, and a DC subband region for each block;

c) selecting said DC subband region as a selected region and performing the following sub-steps;

ca) checking if a current bitplane of the selected region is significant;

cb) if said current bitplane is significant, outputting a first token in a coded representation and partitioning said selected region into a number of sub-regions, wherein each sub-region is processed as the selected region in turn;

cc) if said current bitplane is insignificant, outputting a second token in the coded representation and selecting the next lower bitplane of the selected region as the current bitplane;

cd) repeating sub-steps ca) to cc) until said current bitplane is less than a specified minimum bitplane, or said selected region has a predetermined size and coefficients of said selected region are coded in said coded representation;

d) selecting substantially all uncoded AC subband regions as a remaining region of each block and scanning the significance of each bitplane of the remaining region from a most significant bitplane towards a least significant bitplane, and outputting the second token for each insignificant bitplane until a significant bitplane is determined;

e) setting one or more AC subband regions, of a current resolution level, as a selected region and performing the sub-steps ca) to cd);

f) repeating step e) until substantially all AC subbands of the current resolution level have been encoded; and

g) repeating steps d) to f) until all AC subbands of each block have been encoded.

69. Apparatus for scaling thumbnails representing digital images, said apparatus comprising:

means for storing a thumbnail, said thumbnail comprising a hierarchically encoded representation of a digital image; and

means for decoding said hierarchical representation of said digital image at at least a predetermined one of a plurality of scales to provide said thumbnail having a desired size.

70. Apparatus according to claim 69, wherein the scale of said decoded hierarchical representation is equal to or greater than the desired size of said thumbnail.

71. Apparatus according to claim 69, further comprising means for downsampling the decoded hierarchical representation to provide the desired-size thumbnail if the scale of said decoded hierarchical representation is greater than the desired size of said thumbnail.

72. Apparatus according to claim 69, further comprising means for upsampling the decoded hierarchical representation to provide the desired-size thumbnail if the scale of said decoded hierarchical representation is less than the desired size of said thumbnail.

73. Apparatus according to claim 69, wherein said decoding means decodes said hierarchical representation of said digital image at two different adjacent scales of a plurality of scales to provide said thumbnail having a desired size.

74. Apparatus according to claim 73, further comprising means for interpolating between the two decoded hierarchical representations having different adjacent scales to provide the desired-size thumbnail if the scale of one of said two decoded hierarchical representations is greater than the desired thumbnail size and the other of said two decoded hierarchical representations is less than the desired thumbnail size.

75. Apparatus according to claim 69, further comprising display means for displaying said thumbnail.

76. Apparatus according to claim 69, further comprising means for encoding said digital image to provide said hierarchically encoded representation.

77. Apparatus according to claim 76, wherein said encoding means includes means for applying a hierarchical discrete wavelet transform to said image.

78. Apparatus according to claim 77, wherein encoding comprises SWEET encoding.

79. Apparatus according to claim 69, further comprising means for caching said hierarchical representation and/or said decoded hierarchical representation.

80. Apparatus according to claim 79, further comprising means for caching intermediate size or scaled versions of said decoded hierarchical representation.

81. A computer program product comprising a computer readable medium having a computer program recorded thereon for scaling thumbnails representing digital images, said computer program product comprising:

means for storing a thumbnail, said thumbnail comprising a hierarchically encoded representation of a digital image; and

means for decoding said hierarchical representation of said digital image at at least a predetermined one of a plurality of scales to provide said thumbnail having a desired size.

82. The computer program product according to claim 81, wherein the scale of said decoded hierarchical representation is equal to or greater than the desired size of said thumbnail.

83. The computer program product according to claim 81, further comprising means for downsampling the decoded hierarchical representation to provide the desired-size thumbnail if the scale of said decoded hierarchical representation is greater than the desired size of said thumbnail.

84. The computer program product according to claim 81, further comprising means for upsampling the decoded hierarchical representation to provide the desired-size thumbnail if the scale of said decoded hierarchical representation is less than the desired size of said thumbnail.

85. The computer program product according to claim 81, wherein said decoding means decodes said hierarchical representation of said digital image at two different adjacent scales of a plurality of scales to provide said thumbnail having a desired size.

86. The computer program product according to claim 85, further comprising means for interpolating between the two decoded hierarchical representations having different adjacent scales to provide the desired-size thumbnail if the scale of one of said two decoded hierarchical representations is greater than the desired thumbnail size and the other of said two decoded hierarchical representations is less than the desired thumbnail size.

87. The computer program product according to claim 81, further comprising display means for displaying said thumbnail.

88. The computer program product according to claim 81, further comprising means for ending said digital image to provide said hierarchically encoded representation.

89. The computer program product according to claim 88, wherein said encoding means includes means for applying a hierarchical discrete wavelet transform to said image.

90. The apparatus according to claim 89, wherein encoding comprises SWEET encoding.

91. A method according to claim 81, further comprising means for caching said hierarchical representation and/or said decoded hierarchical representation.

92. A method according to claim 91, further comprising means for caching intermediate size or scaled versions of said decoded hierarchical representation.

93. An image-based computerized file browsing system comprising:

a storage device for retaining a plurality of images, at least a group of which are stored in an encoded format;

a processor for running an operating system incorporating a file management arrangement wherein true images retained in said storage device are depicted as thumbnail representations when viewed via said file management arrangement; and

image manipulation means for manipulating said thumbnail representations so as to be alterable in size and such that only a limited number of size alterations result from decoding from said encoded format, the remainder arising from pixel manipulation of intermediate sized images sized between a starting size and an ending size.

94. A system according to claim 93, wherein said file management arrangement groups certain ones of said representations into containment areas whereby an aspect ratio of said containment areas corresponds to that of a display device upon which said images and representations are reproduced.

95. A system according to claim 94, wherein representations within a containment area can be treated as a single representation when the display size of the containment area is being altered.

96. A system according to claim 93, wherein said images are encoded according to a hierarchical coding scheme that provides for incremental decoding of different resolutions of an image and said image manipulation arrangement decodes said thumbnail representations via one of said increments when performing a size alternation.