Image transmission for low bandwidth with region of interest

• US 6,882,755 B2
• Filed: 10/19/2001
• Issued: 04/19/2005
• Est. Priority Date: 10/19/2001
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Said, Amir, Ray, Nirmalya, Mukherjee, Debargha, Silverstein, D. Amnon, Chen, Mei
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Lu, Tom Y.
• Application Number:
  US10/039,994
• Publication Number:
  US 20030077002A1
• Time in Patent Office: 1,278 Days
• Field of Search: 382/282, 382/284, 382/232, 382/313
• US Class Current: 382/282
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/36 Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/25825 involving client display capabilities, e.g. screen resolution of a mobile phone optimising the visualisation of content during browsing in the Internet G06F16/9577; processing of terminal status or physical abilities in wireless networks H04W8/22; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/440245 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/440272 for performing aspect ratio conversion
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region

Abstract

A system and method of transmitting an image adapted to a first display area size, such as a standard computer screen having a particular pixel-by-pixel resolution, to an apparatus having a smaller second display area size by scaling the image through coefficient cropping its frequency domain representation. When the cropped frequency coefficients are used to display an image within the second smaller display area, a scaled version of the image is displayed. In a specific embodiment, frequency domain coefficients are cropped such that a region of interest of the image is scaled down less than the remainder of the image when displayed in the second smaller display area. Scaling in this manner provides the user with a readable region of interest with the remainder of the image scaled down so as to facilitate easy image navigation by the user.

Claims

1. A method of transmission of an image adapted to a first display area to an apparatus having a smaller second display area, said method comprising:

dividing the image into a plurality of regions each having a corresponding block of data;

converting each block into a frequency domain representation comprising an array of coefficients;

cropping frequency coefficients of a portion of the arrays so as to cause corresponding regions of the image to be scaled when converted into a spatial domain representation dependent on the amount of cropping wherein cropped blocks of coefficients are obtained;

transmitting coefficient data including the cropped blocks and un-cropped arrays of frequency coefficients to the apparatus for displaying a scaled version of the image in the second display area.

2. The method as described in claim 1 further comprising:

identifying a region of interest within the scaled version of the image wherein at least one of the plurality of blocks of data corresponds to the identified region of interest;

wherein the frequency coefficient array of at least one block of data corresponding to the region of interest is cropped less than the coefficient arrays corresponding to the remaining regions of the scaled version of the image.

3. The method as described in claim 1 further comprising compressing the coefficient data prior to transmitting and decompressing prior to displaying in the second display area.

4. The method as described in claim 2 further comprising:

storing the coefficient data prior to transmitting to the apparatus;

identifying a new region of interest within the image;

determining new coefficient data dependent on the new region of interest;

determining a sub-set of coefficient data representing the difference between the previously transmitted coefficient data and the new coefficient data; and

transmitting this sub-set of coefficient data to the apparatus for combining with coefficient data previously transmitted to the apparatus so as to provide image data for redisplaying the scaled version of the image in the second display area dependent on the new region of interest.

5. The method as described in claim 4 wherein identifying a new region of interest comprises selecting the new region through a user interface.

6. The method as described in claim 1 wherein each block has an associated scale factor dependent on the size of the first display area, the size of the second display area and the size of the block wherein each block is cropped according to its associated scale factor.

7. The method as described in claim 6 further comprising selecting a scale factor for each of the blocks by adjusting the size of each of the plurality of regions.

8. The method as described in claim 1 wherein the apparatus is a handheld device.

9. The method as described in claim 1 further comprising transmitting coefficient data in a perceptually significant sequence by transmitting low frequency coefficients corresponding to the entire image then transmitting higher frequency coefficients corresponding to the ROI and then transmitting the remainder of the coefficient data that is required.

10. A system for transmitting an image adapted to a first display area, said system comprising:

image divider for dividing the image into a plurality of regions each having a corresponding block of data;

image data converter for converting each block into a frequency domain representation comprising an array of coefficients;

frequency coefficient cropper for cropping frequency coefficients of a portion of the arrays so as to cause corresponding regions of the image to be scaled when converted into a spatial domain representation dependent on the amount of cropping wherein cropped blocks of coefficients are obtained;

data transmitter for transmitting coefficient data including the cropped blocks and un-cropped arrays of coefficients to the apparatus for displaying a scaled version of the image in a second smaller display area.

11. The system as described in claim 10 wherein the apparatus comprises a user interface for selecting a region of interest (ROI) within the second display area and for transmitting a ROI pointer to the coefficient cropper wherein the selected ROI has at least one corresponding array of coefficients and wherein in response to the ROI pointer, the coefficient cropper crops at least one ROI coefficient array less than the arrays corresponding to the remaining of the plurality of regions.

12. The system as described in claim 10 further comprising an encoder for encoding the coefficient data prior to transmitting to the apparatus and the apparatus comprising a decoder for decoding encoded coefficient data.

13. The system as described in claim 10 wherein the apparatus further comprises means for driving the second display area dependent on the transmitted coefficient data to display the scaled version of the image.

14. A system as described in claim 10 wherein the display driver comprises a means for converting the coefficient data into a spatial domain representation.

15. The system as described in claim 13 wherein the apparatus comprises a first means for storing transmitted coefficient data including previously received coefficient data and newly received coefficient data and the display driver further comprises a means for rendering the scaled version of the image within the second display screen, wherein the rendering means selects coefficients from the first storage means for displaying the scaled version of the image in response to the region of interest pointer.

16. The system as described in claim 14 wherein the system further includes a second means for storing all coefficient data previously transmitted to the apparatus and comprising means for determining a subset of coefficients corresponding to the difference between previously transmitted coefficient data and new coefficient data corresponding to a new ROI pointer received from the apparatus.

17. The system as described in claim 16 wherein the first and second storage means are synchronized under a guaranteed delivery protocol.