Foveated image coding system and method for image bandwidth reduction

• US 6,252,989 B1
• Filed: 12/23/1997
• Issued: 06/26/2001
• Est. Priority Date: 01/07/1997
• Status: Expired due to Term

• Current Assignee: Board of Regents of the University of Texas System (University of Texas System)
• Original Assignee: Board of Regents of the University of Texas System (University of Texas System)
• Inventors: Kortum, Philip, Geisler, Wilson
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): Mariam, Daniel G.
• Application Number:
  US08/997,109
• Time in Patent Office: 1,281 Days
• Field of Search: 382/232, 382/239, 382/248, 382/250, 382/166, 382/236, 382/240, 382/244, 345/202, 348/384.1, 348/387.1, 348/395, 348/398, 348/403, 348/390.1, 358/426
• US Class Current: 382/232
• CPC Class Codes: H04N 19/102 characterised by the element, parameter or selection affected or controlled by the adaptive coding
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/64 characterised by ordering of coefficients or of bits for transmission

Abstract

A foveated imaging system, which can be implemented on a general purpose computer and greatly reduces the transmission bandwidth of images has been developed. This system has demonstrated that significant reductions in bandwidth can be achieved while still maintaining access to high detail at any point in an image. The system is implemented with conventional computer, display, and camera hardware. It utilizes novel algorithms for image coding and decoding that are superior both in degree of compression and in perceived image quality and is more flexible and adaptable to different bandwidth requirements and communications applications than previous systems. The system utilizes novel methods of incorporating human perceptual properties into the coding the decoding algorithms providing superior foveation. One version of the system includes a simple, inexpensive, parallel pipeline architecture, which enhances the capability for conventional and foveated data compression. Included are novel applications of foveated imaging in the transmission of pre-recorded video (without eye tracking), and in the use of alternate pointing devices for foveation.

Claims

1. A method for forming a compressed image, the method comprising:

acquiring an image;

reducing the image with a Foveated Laplacian pyramid algorithm to form a plurality of down-sampled images, the down-sampled images decreasing in resolution from a highest to a lowest resolution;

designating a subset of each of the plurality of down-sampled images to form a plurality of subset images; and

processing the lowest resolution down-sampled image with the plurality of subset images to form the compressed image.

2. The method of claim 1, wherein the processing comprises real-time processing.

3. The method of claim 1, wherein the processing comprises:

expanding the plurality of subset images to form a plurality of expanded subset images; and

subtracting the plurality of expanded subset images from the plurality of down-sampled images to form a plurality of difference images.

4. The method of claim 3, wherein the processing further comprises:

thresholding the plurality of difference images to form a plurality of threshold images;

quantizing the plurality of threshold images;

compressing the plurality of threshold images using a lossless compression algorithm to form a plurality of compression images;

compressing a down-sampled image using a lossless compression algorithm to form a compressed down-sampled image; and

transmitting the compressed down-sampled image with the plurality of compression images.

5. The method of claim 1, wherein the designating corresponds to one or more fixation points.

6. The method of claim 1, wherein the designating corresponds one or more screen coordinates of a pointing device.

7. The method of claim 6, wherein the pointing device comprises a mouse.

8. The method of claim 1, wherein the compressed image comprises a video image.

9. A method for forming a compressed image in real-time, the method comprising:

acquiring an image;

reducing the image with a Foreated Laplacian pyramid algorithm to form a plurality of down-sampled images, the down-sampled images decreasing in resolution from a highest to a lowest resolution;

designating a subset of each of the plurality of down-sampled images to form a plurality of subset images;

expanding the plurality of subset images to form a plurality of expanded subset images;

subtracting the plurality of expanded subset images from the plurality of down-sampled images to form a plurality of difference images; and

processing the lowest resolution down-sampled image with the plurality of difference images to form the compressed image.

10. The method of claim 9, wherein the processing comprises:

thresholding the plurality of difference images to form a plurality of threshold images;

quantizing the plurality of threshold images;

compressing the plurality of threshold images using a lossless compression algorithm to form a plurality of compression images; and

transmitting the lowest resolution down-sampled image with the plurality of compression images.

11. The method of claim 9, wherein the designating corresponds to one or more fixation points.

12. The method of claim 11, wherein the pointing device comprises a mouse.

13. The method of claim 9, wherein the designating corresponds one or more screen coordinates of a pointing device.