Digital image compression with spatially varying quality levels determined by identifying areas of interest

• US 20020141650A1
• Filed: 03/29/2001
• Published: 10/03/2002
• Est. Priority Date: 03/29/2001
• Status: Active Grant

• Current Assignee: Electronics For Imaging Incorporated
• Original Assignee: Electronics For Imaging Incorporated
• Inventors: Keeney, Richard A., Olson, Thor A.
• Granted Patent: US 7,027,655 B2
• Time in Patent Office: Days
• US Class Current: 382/239
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/198 including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value
• CPC Class Codes: H04N 19/60 using transform coding

Abstract

Methods and systems for compression of digital images (still or motion sequences) are provided wherein predetermined criteria may be used to identify a plurality of areas of interest in the image, and each area of interest is encoded with a corresponding quality level (Q-factor). In particular, the predetermined criteria may be derived from measurements of where a viewing audience is focusing their gaze (area of interest). In addition, the predetermined criteria may be used to create areas of interest in an image in order to focus an observer'"'"'s attention to that area. Portions of the image outside of the areas of interest are encoded at a lower quality factor and bit rate. The result is higher compression ratios without adversely affecting a viewer'"'"'s perception of the overall quality of the image.

Claims

1. A method of digital image compression, comprising:

identifying a plurality of areas of interest in the digital image;

encoding the identified areas of interest at a first quality level; and

encoding unidentified areas of the image at a second and lower quality level than the identified areas.

2. A method in accordance with claim 1, further comprising:

creating a quantization map based on the identified areas of interest, wherein;

the encoding is performed based on the quantization map.

3. A method in accordance with claim 1, wherein the digital image is a single still frame.

4. A method in accordance with claim 1, wherein the digital image is one of a sequence of images in a digital motion picture.

5. A method in accordance with claim 4, wherein:

areas of interest are identified only for selected images in the sequence of images; and

areas of interest for a remainder of images in the sequence are extrapolated from the identified areas of interest.

6. A method in accordance with claim 1, wherein the areas of interest are identified by tracking the eye gaze point of one or more viewers as the one or more viewers view the image.

7. A method in accordance with claim 1, wherein the areas of interest are identified by one or more viewers using a pointing device to designate the areas of interest on a display of the image.

8. A method in accordance with claim 1, wherein the areas of interest are identified by a group of viewers.

9. A method in accordance with claim 8, wherein the group of viewers comprises a representative audience made up of people likely to view the image.

10. A method in accordance with claim 8, wherein a histogram is used to determine the most popular areas of interest.

11. A method in accordance with claim 1, wherein the areas of interest are identified in real time during a live transmission of the image.

12. A method in accordance with claim 1, wherein the digital image is a spatially representative version of the image to be encoded.

13. A method in accordance with claim 1, further comprising:

assigning values to each area of interest based on the amount of interest in that area, first values being assigned to areas with higher interest and second values being assigned to areas of lower interest; and

encoding each area of interest at a quality level corresponding to the assigned value, said areas with said first values being encoded at higher quality levels than said areas with said second values.

14. A method in accordance with claim 1, wherein said encoding is performed to provide a gradual transition in quality between an identified area of interest and an unidentified area.

15. A method in accordance with claim 1, wherein the encoding is performed using a block discrete cosine transform (DCT).

16. A method in accordance with claim 15, wherein the quality level for blocks of pixels is adjusted for the areas of interest through the use of a quantization scale factor encoded for each block of pixels.

17. A method in accordance with claim 15, wherein the quality levels of the unidentified areas are adjusted downward by one of:

(i) truncating one or more DCT frequency coefficients;

(ii) setting to zero one or more DCT frequency coefficients;

or (iii) otherwise discarding one or more DCT frequency coefficients, on a block by block basis.

18. A method in accordance with claim 1, wherein the encoding is performed using a wavelet transform.

19. A method in accordance with claim 1, wherein the quality level for the unidentified areas is adjusted downward by pre-filtering the image with a spatially varying spatial frequency filter prior to encoding.

20. A method in accordance with claim 1, further comprising:

sampling the identified areas of interest at a higher spatial resolution than the unidentified areas; and

encoding the identified areas of interest in one or more additional data streams.

21. A method in accordance with claim 20, wherein:

the additional data stream(s) are encoded at a first quality level; and

a data stream which contains the unidentified areas is encoded at a second quality level.

22. A method in accordance with claim 20, wherein:

the additional data stream(s) are encoded using a first method; and

a data stream containing the unidentified areas is encoded using a second method.

23. A method in accordance with claim 1, wherein the areas of interest are identified while the image is in transit.

24. A method in accordance with claim 1, wherein the areas of interest are identified while the image is partially displayed.

25. A method in accordance with claim 1, wherein the quality level of the unidentified areas of the image is reduced for security purposes.

26. A method in accordance with claim 1, wherein one of a constant bit rate or a constant compression ratio is maintained.

27. A method in accordance with claim 1, wherein:

the identified areas of interest are transmitted according to level of interest, so that areas with a higher level of interest are transmitted first with successively lower interest level areas transmitted successively thereafter; and

the image is built up as it is received starting with the areas of highest interest.

28. A method in accordance with claim 1, wherein identified areas of interest from multiple images are statistically recorded.

29. A method in accordance with claim 28, wherein the multiple images are from multiple sources.

30. A method in accordance with claim 1, wherein the quality levels of certain image areas are enhanced to create areas of interest.

31. A method in accordance with claim 30, wherein the enhanced areas are image areas containing at least one of a product and a name of a product.

32. A system for digital image compression, comprising:

a digital image display;

means for identifying a plurality of areas of interest in a digital image provided by said display; and

an encoder, wherein the encoder encodes the identified areas of interest at a first quality level and encodes unidentified areas of the image at a second and lower quality level than the identified areas.

33. A system in accordance with claim 32, further comprising a quantization map created based on said identified areas of interest, wherein:

the encoding is performed based on the quantization map.

34. A system in accordance with claim 32, wherein the digital image is a single still frame.

35. A system in accordance with claim 32, wherein the digital image is one of a sequence of images in a digital motion picture.

36. A system in accordance with claim 35, wherein:

areas of interest are identified only for selected images in the sequence of images; and

areas of interest for a remainder of images in the sequence are extrapolated from the identified areas of interest.

37. A system in accordance with claim 32, wherein the means for identifying areas of interest comprises one or more eye tracking mechanisms for tracking the eye gaze point of one or more viewers as the one or more viewers view the image.

38. A system in accordance with claim 32, wherein the means for identifying areas of interest comprises a pointing device for one or more viewers to designate the areas of interest on the image display.

39. A system in accordance with claim 32, wherein the areas of interest are identified by a group of viewers.

40. A system in accordance with claim 39, wherein the group of viewers comprises a representative audience made up of people likely to view the image.

41. A system in accordance with claim 39, wherein a histogram is used to determine the most popular areas of interest.

42. A system in accordance with claim 32, wherein the areas of interest are identified in real time during a live transmission of the image.

43. A system in accordance with claim 32, wherein the digital image is a spatially representative version of the image to be encoded.

44. A system in accordance with claim 32, wherein:

values are assigned to each area of interest based on the amount of interest in that area, first values being assigned to areas with higher interest and second values being assigned to areas of lower interest; and

each area of interest is encoded at a quality level corresponding to the assigned value, said areas with said first values being encoded at higher quality levels than said areas with said second values.

45. A system in accordance with claim 32, wherein said encoding is performed to provide a gradual transition in quality between an identified area of interest and an unidentified area.

46. A system in accordance with claim 32, wherein the encoding is performed using a block discrete cosine transform (DCT).

47. A system in accordance with claim 46, wherein the quality level for blocks of pixels is adjusted for the areas of interest through the use of a quantization scale factor encoded for each block of pixels.

48. A system in accordance with claim 46, wherein the quality levels of the unidentified areas are adjusted downward by one of:

(i) truncating one or more DCT frequency coefficients;

(ii) setting to zero one or more DCT frequency coefficients;

or (iii) otherwise discarding one or more DCT frequency coefficients, on a block by block basis.

49. A system in accordance with claim 32, wherein the encoding is performed using a wavelet transform.

50. A system in accordance with claim 32, further comprising:

a spatially varying spatial frequency filter, wherein the quality level for the unidentified areas is adjusted downward by pre-filtering the image using the spatial frequency filter prior to encoding.

51. A system in accordance with claim 32, wherein:

the identified areas of interest are sampled at a higher spatial resolution than the unidentified areas; and

the identified areas of interest are encoded in one or more additional data streams.

52. A system in accordance with claim 51, wherein:

the additional data stream(s) are encoded at a first quality level; and

a data stream which contains the unidentified areas is encoded at a second quality level.

53. A system in accordance with claim 51, wherein:

the additional data stream(s) are encoded using a first method; and

a data stream containing the unidentified areas is encoded using a second method.

54. A system in accordance with claim 32, wherein the areas of interest are identified while the image is in transit.

55. A system in accordance with claim 32, wherein the areas of interest are identified while the image is partially displayed.

56. A system in accordance with claim 32, wherein the quality level of the unidentified areas of the image is reduced for security purposes.

57. A system in accordance with claim 32, wherein one of a constant bit rate or a constant compression ratio is maintained.

58. A system in accordance with claim 32, wherein:

the identified areas of interest are transmitted according to level of interest, so that areas with a higher level of interest are transmitted first with successively lower interest level areas transmitted successively thereafter; and

the image is built up as it is received starting with the areas of highest interest.

59. A system in accordance with claim 32, wherein identified areas of interest from multiple images are statistically recorded.

60. A system in accordance with claim 59, wherein the multiple images are from multiple sources.

61. A system in accordance with claim 32, wherein the quality levels of certain image areas are enhanced to create areas of interest.

62. A system in accordance with claim 61, wherein the enhanced areas are image areas containing at least one of a product and a name of a product.

63. A method of digital image compression, comprising:

identifying a plurality of areas of interest in the digital image by tracking the eye gaze point of one or more viewers as the one or more viewers view the image;

encoding the identified areas of interest at a first quality level; and

encoding unidentified areas of the image at a second and lower quality level than the identified areas.

64. A system for digital image compression, comprising:

a digital image display device for displaying a digital image;

one or more eye tracking mechanisms for tracking the eye gaze of one or more viewers as the one or more viewers view the digital image in order to identify a plurality of areas of interest in the digital image; and

an encoder, wherein the encoder encodes the identified areas of interest at a first quality level and encodes unidentified areas of the image at a second and lower quality level than the identified areas.