System and method for transmitting a digital image over a communication network

• US 6,314,452 B1
• Filed: 08/31/1999
• Issued: 11/06/2001
• Est. Priority Date: 08/31/1999
• Status: Expired due to Term

• Current Assignee: IDX Investment Corporation (General Electric Company)
• Original Assignee: Rtimage, Ltd.
• Inventors: Dekel, Shai, Ovsiankin, Alexander
• Primary Examiner(s): Meky, Moustafa M.
• Application Number:
  US09/386,264
• Time in Patent Office: 798 Days
• Field of Search: 709/201, 709/203, 709/217, 709/218, 709/219, 709/212, 709/213, 709/206, 709/208
• US Class Current: 709/203
• CPC Class Codes: H04N 19/122 Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
• CPC Class Codes: H04N 19/129 Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/1883 the unit relating to sub-band structure, e.g. hierarchical level, directional tree, e.g. low-high [LH], high-low [HL], high-high [HH]
• CPC Class Codes: H04N 19/36 Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/635 characterised by filter definition or implementation details
• CPC Class Codes: H04N 19/647 using significance based coding, e.g. Embedded Zerotrees of Wavelets [EZW] or Set Partitioning in Hierarchical Trees [SPIHT]
• CPC Class Codes: H04N 19/96 Tree coding, e.g. quad-tree coding

Abstract

The imaging system that is described below is an image streaming system which is different from traditional compression systems. It eliminates the necessity to store a compressed version of the original image, by streaming ROI data using the original stored image. The imaging system of the present invention also avoids the computationally intensive task of compression of the full image. Instead, once a user wishes to interact with a remote image, the imaging server performs a fast preprocessing step in near real time after which it can respond to any ROI requests also in near real time. When a ROI request arrives at the server, a sophisticated progressive image encoding algorithm is performed, but not for the full image. Instead, the encoding algorithm is performed only for the ROI. Since the size of the ROI is bounded by the size and resolution of the viewing device at the client and not by the size of the image, only a small portion of the full progressive coding computation is performed for a local area of the original image. This local property is also true for the client. The client computer performs decoding and rendering only for ROI and not the full image. This real time streaming or Pixels-On-Demand™ architecture requires different approaches even to old ideas. For example, similarly to some prior art, the present imaging system is based on wavelets. But while in other systems wavelet bases are selected according to their coding abilities, the choice of wavelet bases in the present imaging system depends more on their ability to perform well in the real time framework. The system of the present invention supports several modes of progressive transmission: by resolution, by accuracy and by spatial order.

Claims

1. A system for transmitting a digital image over a communication network, comprising:

(a) an image storage device for storing an original digital image at an original resolution;

(b) a client computer coupled to the communication network, wherein the client computer generates at least one request for interaction with the original digital image and at least one subsequent request, each identifying a region of interest at a selected resolution within the original digital image;

(c) a server computer coupled to the communication network and the storage device, the server computer performing the steps of;

(i) pre-processing the original digital image in response to the receipt of the first request, the pre-processed digital image having a resolution lower than or equal to the original resolution;

(ii) for each subsequent request received from the client computer;

(A) compressing the region of interest associated with each request, based upon;

(1) the pre-processed digital image, and

(2) further processing of portions of the original digital image at the selected resolution of the region of interest identified in the subsequent request; and

(B) progressively transmitting the compressed region of interest to the client computer via the communication network.

2. The system of claim 1, wherein the client computer progressively performs the steps of:

(i) decompressing the compressed region of interest transmitted from the server computer; and

(ii) rendering the decompressed region of interest.

3. The system of claim 2, wherein the client computer renders the decompressed region of interest onto a video display.

4. The system of claim 2, wherein the client computer renders the decompressed region of interest onto a printer.

5. The system of claim 2, wherein the client computer performs the initial step of storing the compressed region of interest in a cache associated with the client computer.

6. The system of claim 5, wherein the client computer only requests that portion of a region of interest that was not previously stored in the cache.

7. The system of claim 1, wherein the region of interest is transmitted from the server computer to the client computer using encoded data blocks having X, Y, resolution and accuracy dimensions.

8. The system of claim 1, wherein the client computer may request the digital image from the server computer in a progressive mode selected from the list of:

resolution, accuracy and spatial.

9. The system of claim 1, wherein the pre-processing step performed by the server comprises the sub-steps of:

(A) generating a lower resolution digital image from the original digital image, reading the original digital image from the image storage device;

(B) color transforming the lower resolution digital image to a compression color space;

(C) transforming the compression color space digital image into wavelet coefficients;

(D) performing quantization on the wavelet coefficients; and

(E) storing the quantified wavelet coefficients in a storage device associated with the server computer.

10. The system of claim 9, wherein the transforming sub-step converts the digital image from its original color space to a compression color space, and the client computer converts the digital image from the compression color space to a rendering color space.

11. The system of claim 1, wherein the pre-processing step utilizes constrained memory with an order of magnitude of the square root of the original digital image size.

12. The system of claim 1, wherein the compressing step utilizes the non-zero-trees type of progressive encoding technique.

13. The system of claim 1, wherein in the compressing step, the server computer fully compresses the region of interest.

14. The system of claim 1, wherein in the compressing step, the server partially compresses the region of interest.

15. The system of claim 1, wherein the progressive transmitting step comprises the sub-steps of:

(1) transmitting the compressed region of interest within the selected digital image at a relatively low resolution; and

(2) transmitting the compressed region of interest within the selected digital image at successively higher resolutions.

16. The system of claim 1, wherein the progressive transmitting step comprises the sub-steps of:

(1) transmitting the compressed region of interest within the selected digital image at a relatively low level of accuracy; and

(2) transmitting the compressed region of interest within the selected digital image at successively higher levels of accuracy.

17. The system of claim 1, wherein the progressive transmitting step comprises the sub-steps of:

(1) transmitting a first band of the compressed region of interest within the selected digital image in spatial order starting from the top; and

(2) repeating the transmission of the rest of the bands of the compressed region of interest within the selected digital image in spatial order.

18. A system for transmitting a digital image over a communication network, comprising:

(a) an image storage device for storing an original digital image;

(b) a client computer coupled to the communication network, wherein the client computer generates at least one request, each request identifying a region of interest within the original digital image;

(c) a server computer coupled to the communication network and the storage device, the server computer performing the steps of;

(i) receiving a first request from the client computer;

(ii) pre-processing the original digital image in response to the receipt of the first request;

(iii) for each subsequent request received from the client computer;

(A) compressing the region of interest associated with each subsequent request, based upon the pre-processed digital image; and

(B) progressively transmitting the compressed region of interest to the client computer via the communication network.

19. The system of claim 18, wherein the client computer progressively performs the steps of:

(i) decompressing the compressed region of interest transmitted from the server computer; and

(ii) rendering the decompressed region of interest.

20. The system of claim 19, wherein the client computer renders the decompressed region of interest onto a video display.

21. The system of claim 19, wherein the client computer renders the decompressed region of interest onto a printer.

22. The system of claim 19, wherein the client computer performs the initial step of storing the compressed region of interest in a cache associated with the client computer.

23. The system of claim 22, wherein the client computer only requests that portion of a region of interest that was not previously stored in the cache.

24. The system of claim 18, wherein the region of interest is transmitted from the server computer to the client computer using encoded data blocks having X, Y, resolution and accuracy dimensions.

25. The system of claim 18, wherein the client computer may request the digital image from the server computer in a progressive mode selected from the list of:

resolution, accuracy and spatial.

26. The system of claim 18, wherein the pre-processing step performed by the server comprises the sub-steps of:

(A) reading the original digital image from the image storage device;

(B) color transforming the lower resolution digital image to a compression color space;

(C) transforming the compression color space digital image into wavelet coefficients;

(D) performing quantization on the wavelet coefficients; and

(E) storing the quantified wavelet coefficients in a storage device associated with the server computer.

27. The system of claim 26, wherein the transforming sub-step converts the digital image from its original color space to a compression color space, and the client computer converts the digital image from the compression color space to a rendering color space.

28. The system of claim 18, wherein the pre-processing step utilizes constrained memory with an order of magnitude of the square root of the original digital image size.

29. The system of claim 18, wherein the compressing step utilizes the non-zero-trees type of progressive encoding technique.

30. The system of claim 18, wherein in the compressing step, the server computer fully compresses the region of interest.

31. The system of claim 18, wherein in the compressing step, the server partially compresses the region of interest.

32. The system of claim 18, wherein the progressive transmitting step comprises the sub-steps of:

(1) transmitting the compressed region of interest within the selected digital image at a relatively low resolution; and

(2) transmitting the compressed region of interest within the selected digital image at successively higher resolutions.

33. The system of claim 18, wherein the progressive transmitting step comprises the sub-steps of:

(1) transmitting the compressed region of interest within the selected digital image at a relatively low level of accuracy; and

(2) transmitting the compressed region of interest within the selected digital image at successively higher levels of accuracy.

34. The system of claim 18, wherein the progressive transmitting step comprises the sub-steps of:

(1) transmitting a first band of the compressed region of interest within the selected digital image in spatial order starting from the top; and

(2) repeating the transmission of the rest of the bands of the compressed region of interest within the selected digital image in spatial order.

35. The system of claim 18, wherein the pre-processing step performed by the server comprises the sub-steps of:

(A) reading the original digital image from the image storage device;

(B) color transforming the original digital image to a compression color space;

(C) transforming the compression color space digital image into wavelet coefficients;

(D) performing quantization on the wavelet coefficients; and

(E) storing the quantified wavelet coefficients in a storage device associated with the server computer.

36. A system for transmitting a digital image over a communication network, comprising:

(a) an image storage device for storing an original digital image;

(b) a client computer coupled to the communication network, wherein the client computer generates at least one request, each request identifying a region of interest within the original digital image;

(c) a server computer coupled to the communication network and the storage device, the server computer performing the steps of;

(i) receiving a first request from the client computer;

(ii) pre-processing the original digital image in response to the receipt of the first request;

(iii) for each subsequent request received from the client computer;

(A) compressing the region of interest associated with each subsequent request, based upon the pre-processed digital image; and

(B) transmitting the compressed region of interest to the client computer via the communication network.

37. A server computer for transmitting a digital image to a client computer over a communication network, the server computer being coupled to an image storage device for storing an original digital image at an original resolution, wherein the client computer generates at least one request for interaction with the original digital image and at least one subsequent request, each identifying a region of interest at a selected resolution within the original digital image, the server computer comprising:

(a) means for pre-processing the original digital image in response to the receipt of the first request, the pre-processed digital image having a resolution lower than or equal to the original resolution;

(b) means, for each subsequent request received from the client computer, for;

(i) compressing the region of interest associated with each request, based upon;

(1) the pre-processed digital image, and

(2) further processing of portions of the original digital image at the selected resolution of the region of interest identified in the subsequent request; and

(ii) progressively transmitting the compressed region of interest to the client computer via the communication network.

38. A server computer for transmitting a digital image to a client computer over a communication network, the server computer being coupled to an image storage device for storing an original digital image, wherein the client computer generates at least one request, each request identifying a region of interest within the original digital image, the server computer comprising:

(a) means for receiving a first request from the client computer;

(b) means for pre-processing the original digital image in response to the receipt of the first request;

(c) means, for each subsequent request received from the client computer, for;

(i) compressing the region of interest associated with each subsequent request, based upon the pre-processed digital image; and

(ii) progressively transmitting the compressed region of interest to the client computer via the communication network.

39. A method for transmitting a digital image from an image storage device to a client computer over a communication network, comprising the steps of:

(a) storing an original digital image at an original resolution in the image storage device;

(b) generating at the client computer at least one request for interaction with the original digital image and at least one subsequent request, each request identifying a region of interest at a selected resolution within the original digital image;

(c) pre-processing the original digital image in response to the receipt of the first request, the pre-processed digital image having a resolution lower than or equal to the original resolution;

(d) for each subsequent request received from the client computer;

(i) compressing the region of interest associated with each request, based upon;

(1) the pre-processed digital image, and

(2) further processing of portions of the original digital image at the selected resolution of the region of interest identified in the subsequent request; and

(ii) progressively transmitting the compressed region of interest to the client computer via the communication network.

40. A method for transmitting a digital image to a client computer over a communication network, comprising the steps of:

(a) storing an original digital image;

(b) generating at the client computer at least one request, each request identifying a region of interest within the original digital image;

(c) receiving a first request from the client computer;

(d) pre-processing the original digital image in response to the receipt of the first request;

(e) for each subsequent request received from the client computer;

(i) compressing the region of interest associated with each subsequent request, based upon the pre-processed digital image; and

(ii) progressively transmitting the compressed region of interest to the client computer via the communication network.

41. A computer-readable media for a server computer, programmed to cause the server computer to transmit a digital image to a client computer over a communication network, the server computer being coupled to an image storage device for storing an original digital image at an original resolution, wherein the client computer generates at least one request for interaction with the original digital image and at least one subsequent request, each identifying a region of interest at a selected resolution within the original digital image, the computer-readable media programmed to performed the steps of:

(a) pre-processing the original digital image in response to the receipt of the first request, the pre-processed digital image having a resolution lower than or equal to the original resolution;

(b) for each subsequent request received from the client computer;

(i) compressing the region of interest associated with each request, based upon;

(1) the pre-processed digital image, and

(2) further processing of portions of the original digital image at the selected resolution of the region of interest identified in the subsequent request; and

(ii) progressively transmitting the compressed region of interest to the client computer via the communication network.

42. A computer-readable media for a server computer, programmed to cause the server computer to transmit a digital image to a client computer over a communication network, the server computer being coupled to an image storage device for storing an original digital image, wherein the client computer generates at least one request, each request identifying a region of interest within the original digital image, the computer-readable media programmed to perform the steps of:

(a) receiving a first request from the client computer;

(b) pre-processing the original digital image in response to the receipt of the first request;

(c) for each subsequent request received from the client computer;

(i) compressing the region of interest associated with each subsequent request, based upon the pre-processed digital image; and

(ii) progressively transmitting the compressed region of interest to the client computer via the communication network.