Digital image compression with spatially varying quality levels determined by identifying areas of interest

• US 7,027,655 B2
• Filed: 03/29/2001
• Issued: 04/11/2006
• Est. Priority Date: 03/29/2001
• Status: Expired due to Fees

• Current Assignee: Electronics For Imaging Incorporated
• Original Assignee: Electronics For Imaging Incorporated
• Inventors: Olson, Thor A., Keeney, Richard A.
• Primary Examiner(s): Bali, Vikram
• Assistant Examiner(s): LAROSE, COLIN M
• Application Number:
  US09/821,104
• Publication Number:
  US 20020141650A1
• Time in Patent Office: 1,839 Days
• Field of Search: 382/243, 382/239, 375/240.08, 375/240.1, 725/9, 725/10, 725/14, 725/16
• US Class Current: 382/239
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/198 including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value
• CPC Class Codes: H04N 19/60 using transform coding

Abstract

Methods and systems for compression of digital images (still or motion sequences) are provided wherein predetermined criteria may be used to identify a plurality of areas of interest in the image, and each area of interest is encoded with a corresponding quality level (Q-factor). In particular, the predetermined criteria may be derived from measurements of where a viewing audience is focusing their gaze (area of interest). In addition, the predetermined criteria may be used to create areas of interest in an image in order to focus an observer'"'"'s attention to that area. Portions of the image outside of the areas of interest are encoded at a lower quality factor and bit rate. The result is higher compression ratios without adversely affecting a viewer'"'"'s perception of the overall quality of the image.

Claims

1. A method of digital image compression, comprising:

identifying a plurality of areas of interest in a subset of digital images in a sequence of related images;

extrapolating areas of interest for a remainder of images in the sequence from the identified areas of interest;

using a histogram to determine most popular identified area of interest; and

encoding the most popular identified areas of interest at a first quality level and unidentified areas of the image at a second and lower quality level than the most popular identified areas in order to produce a single compressed copy of each image which can be decoded at a decoder, wherein;

a size of the identified areas of interest corresponds to an angular coverage of an average human fovea at a predetermined viewing distance;

the areas of interest are identified by a group of viewers; and

said group of viewers comprises a statistically representative subset of an intended audience for the sequence of related images in order to predict areas of interest of the intended audience.

2. A method in accordance with claim 1, further comprising creating a quantization map based on the identified areas of interest, wherein the encoding is performed based on the quantization map.

3. A method in accordance with claim 1, wherein the areas of interest are identified by tracking the eye gaze point of said viewers as the viewers view the image.

4. A method in accordance with claim 1, wherein the areas of interest are identified by said viewers using a pointing device to designate the areas of interest on a display of the image.

5. A method in accordance with claim 1, wherein the sequence of related images comprises a digital motion picture.

6. A method in accordance with claim 1, wherein the areas of interest are identified in real time during a live transmission of the image.

7. A method in accordance with claim 1, wherein the digital image is a spatially representative version of the image to be encoded.

8. A method in accordance with claim 1, further comprising:

assigning values to each area of interest based on the amount of interest in that area, first values being assigned to areas with higher interest and second values being assigned to areas of lower interest; and

encoding each area of interest at a quality level corresponding to the assigned value, said areas with said first values being encoded at higher quality levels than said areas with said second values.

9. A method in accordance with claim 1, wherein said encoding is performed to provide a gradual transition in quality between an identified area of interest and an unidentified area.

10. A method in accordance with claim 1, wherein the encoding is performed using a block discrete cosine transform (DCT).

11. A method in accordance with claim 10, wherein the quality level for blocks of pixels is adjusted for the areas of interest through the use of a quantization scale factor encoded for each block of pixels.

12. A method in accordance with claim 10, wherein the quality levels of the unidentified areas are adjusted downward by one of:

(i) truncating one or more DCT frequency coefficients;

(ii) setting to zero one or more DCT frequency coefficients;

or (iii) otherwise discarding one or more DCT frequency coefficients, on a block by block basis.

13. A method in accordance with claim 1 wherein the encoding is performed using a wavelet transform.

14. A method in accordance with claim 1, wherein the quality level for the unidentified areas is adjusted downward by pro-filtering the image with a spatially varying spatial frequency filter prior to encoding.

15. A method in accordance with claim 1, wherein the areas of interest are identified while the image is in transit.

16. A method in accordance with claim 1, wherein the areas of interest are identified while the image is partially displayed.

17. A method in accordance with claim 1, wherein the quality level of the unidentified areas of the image is reduced for security purposes.

18. A method in accordance with claim 1, wherein one of a constant bit rate or a constant compression ratio is maintained.

19. A method in accordance with claim 1, wherein:

the identified areas of interest are transmitted according to level of interest, so that areas with a higher level of interest are transmitted first with successively lower interest level areas transmitted successively thereafter; and

the image is built up as it is received starting with the areas of highest interest.

20. A method in accordance with claim 1, wherein identified areas of interest from multiple images are statistically recorded.

21. A method in accordance with claim 1, further comprising enhancing the quality levels of certain unidentified areas to artificially create additional areas of interest from said certain unidentified areas in order to draw a viewer'"'"'s attention to said additional areas of interest.

22. A method in accordance with claim 21, wherein the additional areas of interest are image areas containing at least one of a product and a name of a product.

23. A system for digital image compression, comprising:

a digital image display;

means for a group of viewers to identify a plurality of areas of interest in a subset of digital images in a sequence of related images provided by said display;

a means for extrapolating area of interest for a remainder of images in the sequence from the identified areas of interest;

means for using a histogram to determine most popular identified area of interest; and

an encoder, wherein the encoder encodes the most popular identified areas of interest at a first quality level and unidentified areas of the image at a second and lower quality level than the most popular identified areas in order to produce a single compressed copy of each image which can be decoded at a decoder wherein;

a size of the identified areas of interest corresponds to an angular coverage of an average human fovea at a predetermined viewing distance; and

the group of viewers comprises a statistically representative subset of an intended audience for the sequence of related images in order to predict areas of interest of the intended audience.

24. A system in accordance with claim 23, further comprising a quantization map created based on said identified areas of interest, wherein the encoding is performed based on the quantization map.

25. A system in accordance with claim 23, wherein the means for identifying areas of interest comprises eye tracking mechanisms for tracking the eye gaze point of the viewers as the viewers view the image.

26. A system in accordance with claim 23, wherein the means for identifying areas of interest comprises pointing devices for designating the areas of interest on the image display.

27. A system in accordance with claim 23, wherein the sequence of related images comprises a digital motion picture.

28. A system in accordance with claim 23, wherein the areas of interest are identified in real time during a live transmission of the image.

29. A system in accordance with claim 23, wherein the digital image is a spatially representative version of the image to be encoded.

30. A system in accordance with claim 23, wherein:

values are assigned to each area of interest based on the amount of interest in that area, first values being assigned to areas with higher interest and second values being assigned to areas of lower interest; and

each area of interest is encoded at a quality level corresponding to the assigned value, said areas with said first values being encoded at higher quality levels than said areas with said second values.

31. A system in accordance with claim 23, wherein said encoding is performed to provide a gradual transition in quality between an identified area of interest and an unidentified area.

32. A system in accordance with claim 23, wherein the encoding is performed using a block discrete cosine transform (DCT).

33. A system in accordance with claim 32, wherein the quality level for blocks of pixels is adjusted for the areas of interest through the use of a quantization scale factor encoded for each block of pixels.

34. A system in accordance with claim 32, wherein the quality levels of the unidentified areas are adjusted downward by one of:

(i) truncating one or more DCT frequency coefficients;

(ii) setting to zero one or more DCT frequency coefficients;

or (iii) otherwise discarding one or more DCT frequency coefficients, on a block by block basis.

35. A system in accordance with claim 23, wherein the encoding is performed using a wavelet transform.

36. A system in accordance with claim 23, further comprising:

a spatially varying spatial frequency filter, wherein the quality level for the unidentified areas is adjusted downward by pre-filtering the image using the spatial frequency filter prior to encoding.

37. A system in accordance with claim 23, wherein the areas of interest are identified while the image is in transit.

38. A system in accordance with claim 23, wherein the areas of interest are identified while the image is partially displayed.

39. A system in accordance with claim 23, wherein the quality level of the unidentified areas of the image is reduced for security purposes.

40. A system in accordance with claim 23, wherein one of a constant bit rate or a constant compression ratio is maintained.

41. A system in accordance with claim 23, wherein:

the identified areas of interest are transmitted according to level of interest, so that areas with a higher level of interest are transmitted first with successively lower interest level areas transmitted successively thereafter; and

the image is built up as it is received starting with the areas of highest interest.

42. A system in accordance with claim 23, wherein identified areas of interest from multiple images are statistically recorded.

43. A system in accordance with claim 23, wherein the quality levels of certain unidentified areas are enhanced to artificially create additional areas of interest from said certain unidentified areas in order to draw a viewer'"'"'s attention to said additional areas of interest.

44. A system in accordance with claim 43, wherein the additional areas of interest are image areas containing at least one of a product and a name of a product.

45. A method of digital image compression, comprising:

identifying a plurality of areas of interest in each digital image in a digital motion picture by tracking the eye gaze point of a plurality of viewers as the viewers view each image;

creating a histogram of the identified areas of interest to determine the most popular identified areas of interest; and

encoding the most popular identified areas of interest of each image at a first quality level and remaining areas of each image at a second and lower quality level than the most popular identified areas in order to produce a single compressed copy of each image which can be decoded at a standard decoder;

wherein the plurality of viewers comprises a statistically representative subset of an intended audience for the motion picture in order to predict areas of interest of the intended audience.

46. A system for digital image compression, comprising:

a digital image display device for displaying a related sequence of digital images comprising a digital motion picture;

a plurality of eye tracking mechanisms for tracking the eye gaze of a plurality of viewers as the viewers view each digital image in the digital motion picture order to identify a plurality of areas of interest in each digital image;

means for creating a histogram of the identified areas of interest to determine the most popular identified areas of interest; and

an encoder, wherein the encoder encodes the most popular identified areas of interest of each image at a first quality level and encodes remaining areas of each image at a second and lower quality level than the most popular identified areas in order to produce a single compressed copy of each image which can be decoded at a standard decoder;

wherein the plurality of viewers comprises a statistically representative subset of an intended audience for the motion picture in order to predict areas of interest of the intended audience.