Multi-point predictive foveation for bandwidth reduction of moving images

• US 7,894,682 B2
• Filed: 07/02/2008
• Issued: 02/22/2011
• Est. Priority Date: 04/15/2002
• Status: Expired due to Fees

• Current Assignee: Gamehancement LLC (Jeffrey M. Gross)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Sullivan, Marc Andrew, Kortum, Philip Ted
• Primary Examiner(s): Couso; Jose L
• Application Number:
  US12/166,861
• Publication Number:
  US 20090010555A1
• Time in Patent Office: 965 Days
• Field of Search: 382232-233, 382/236, 382238-240, 382243-252, 348/384.1, 348/387.1, 348/390.1, 375/240, 375/240.1
• US Class Current: 382/239
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

A scene of an image sequence is transmitted that has been compressed using a number of foveation zones, each foveation zone being weighted based on a probability of a viewer looking at a corresponding portion of the first scene.

Claims

1. A method comprising:

receiving an image sequence that is compressed based on a plurality of foveation zones; and

decompressing, by a processor, the received image sequence;

wherein a first foveation zone of the plurality of foveation zones corresponds to a first view location and a second foveation zone of the plurality of foveation zones corresponds to a second view location; and

wherein, during a portion of the image sequence, the first and second view locations occur concurrently.

2. The method of claim 1, further comprising:

displaying the decompressed image sequence.

3. The method of claim 1, further comprising:

compressing the image sequence based on the plurality of foveation zones; and

transmitting the compressed image sequence.

4. The method of claim 3, wherein compressing the image sequence based on the plurality of foveation zones comprises:

determining view locations within the plurality of foveation zones for a sample of viewers.

5. The method of claim 3, wherein compressing the image sequence based on the plurality of foveation zones comprises:

estimating a probability of a viewer looking at the first view location and the second view location based at least in part on a scene type occurring during the portion of the image sequence where the first and second view locations occur concurrently; and

weighting at least the first and second foveation zones based on the probability.

6. The method of claim 1 wherein each of the plurality of foveation zones is definable by a continuous falloff function.

7. The method of claim 1 wherein each of the plurality of foveation zones is associated with a corresponding half-resolution constant based on a probability of a viewer looking at the view location corresponding to the foveation zone.

8. The method of claim 7 wherein the probability of a viewer looking at the first view location corresponding to the first foveation zone is greater than the probability of a viewer looking at the second view location corresponding to the second foveation zone, and wherein the half-resolution constant of the first foveation zone is greater than the half-resolution constant of the second foveation zone.

9. The method of claim 1 wherein each of the plurality of foveation zones is definable by a discrete function.

10. The method of claim 1 wherein each of the plurality of foveation zones is associated with a corresponding fixed resolution based on a probability of a viewer looking at a view location corresponding to the foveation zone.

11. A computer-readable storage medium comprising computer-readable instructions to direct a processor to perform acts of:

receiving an image sequence that is compressed based on a plurality of foveation zones;

wherein a first foveation zone of the plurality of foveation zones corresponds to a first view location and a second foveation zone of the plurality of foveation zones corresponds to a second view location; and

wherein during a portion of the image sequence, the first and second view locations occur concurrently.

12. The computer-readable storage medium of claim 11, further comprising computer-readable instructions to direct the processor to perform acts of:

decompressing the received image sequence; and

displaying the decompressed image sequence.

13. A computer apparatus comprising a processor responsive to computer-readable instructions stored in a computer readable medium, the instructions including instructions to receive an image sequence that is compressed based on a plurality of foveation zones, wherein a first foveation zone of the plurality of foveation zones corresponds to a first view location and a second foveation zone of the plurality of foveation zones corresponds to a second view location;

wherein during a portion of the image sequence, the first and second view locations occur concurrently.

14. The apparatus of claim 13, wherein the instructions further include instructions to decompress the received image sequence and display the decompressed image sequence.

15. The apparatus of claim 14, wherein the instructions further include instructions to store the decompressed image sequence.

16. The apparatus of claim 13, wherein the instructions further include instructions to store the compressed image sequence.

17. The apparatus of claim 13, wherein the compressed image sequence is received via a digital subscriber line (DSL) network.

18. The apparatus of claim 13, wherein the compressed image sequence is received via a digital cable television transmission network.

19. The apparatus of claim 13, wherein the compressed image sequence is received via a satellite transmission network.

20. A computer system comprising a processor having access to a computer-readable medium including computer executable instructions, embedded on the computer-readable medium, to decompress an image sequence that is compressed based on a plurality of foveation zones and display the decompressed image sequence;

wherein a first foveation zone of the plurality of foveation zones corresponds to a first view location and a second foveation zone of the plurality of foveation zones corresponds to a second view location; and

wherein during a portion of the image sequence, the first and second view locations occur concurrently.

21. A method comprising:

decompressing, by a processor, an image sequence that is compressed based on a plurality of foveation zones; and

displaying the decompressed image sequence;

wherein a first foveation zone of the plurality of foveation zones corresponds to a first view location and a second foveation zone of the plurality of foveation zones corresponds to a second view location; and

wherein during a portion of the image sequence, the first and second view locations occur concurrently.

22. A computer-readable storage medium comprising a set of instructions for directing a processor to perform acts of:

decompressing an image sequence that is compressed based on a plurality of foveation zones; and

displaying the decompressed image sequence;

wherein a first foveation zone of the plurality of foveation zones corresponds to a first view location and a second foveation zone of the plurality of foveation zones corresponds to a second view location; and

wherein during a portion of the image sequence, the first and second view locations occur concurrently.