Image compression method and apparatus

• US 10,657,674 B2
• Filed: 06/14/2017
• Issued: 05/19/2020
• Est. Priority Date: 06/17/2016
• Status: Active Grant

• Current Assignee: Immersive Robotics Pty., Ltd.
• Original Assignee: Immersive Robotics Pty., Ltd.
• Inventors: Fitzgerald, Daniel Liam, Lucas, Timothy Simon
• Primary Examiner(s): Sajous, Wesner
• Application Number:
  US16/310,756
• Publication Number:
  US 20190378305A1
• Time in Patent Office: 1,070 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence

Abstract

A method of compressing image data from one or more images forming part of digital reality content, the method including obtaining pixel data from the image data, the pixel data representing an array of pixels within the one or more images; determining a position of the array of pixels within the one or more images relative to a defined position, the defined position being at least partially indicative of a point of gaze of the user; and compressing the pixel data at least partially in accordance the determined position so that a degree of compression depends on the determined position of the array of pixels.

Claims

1. A method of compressing image data representing one or more images, the method including:

a) obtaining pixel data from the image data, the pixel data representing an array of pixels within the one or more images;

b) applying a transformation to the pixel data to determine a set of frequency coefficients indicative of frequency components of the array of pixels;

c) selecting one of a plurality of bit encoding schemes, wherein each of the plurality of bit encoding schemes selectively encodes different frequency coefficients with respective different numbers of bits to provide a different degree of compression and wherein the bit encoding scheme is selected at least in part based on;

i) a desired degree of compression; and

,

ii) a position of the array of pixels in the one or more images;

d) selectively encoding at least some of the frequency coefficients using the selected bit encoding scheme to thereby generate a set of encoded frequency coefficients, wherein the bit encoding scheme defines the number of bits used to encode each of the frequency coefficients so that when the frequency coefficients are selectively encoded;

i) at least some of the encoded frequency coefficients are encoded with a different numbers of bits; and

,

ii) at least one frequency coefficient is discarded so that the set of encoded frequency coefficients is smaller than the set of frequency coefficients; and

,

e) generating compressed image data using the encoded frequency coefficients.

2. A method according to claim 1, wherein the method includes selecting the bit encoding scheme based on at least one of:

a) a transmission bandwidth of a wireless communications link used to transmit the compressed image data to a display device;

b) a transmission quality of service of a wireless communications link used to transmit the compressed image data to a display device;

c) a lens arrangement of a display device;

d) movement of a display device;

e) image display requirements of a display device;

f) a target display resolution of a display device.

3. A method according to claim 1, wherein the frequency coefficients are selectively encoded so that at least one of:

a) a number of bits used to encode each frequency coefficient is defined irrespective of the value of the respective frequency coefficient;

b) a smaller number of bits are used to encode frequency coefficients corresponding to higher frequencies;

c) a progressively smaller number of bits are used to encode frequency coefficients corresponding to progressively higher frequencies;

d) at least one frequency coefficient is discarded corresponding to higher frequencies.

4. A method according to claim 1, wherein the method includes applying a scaling factor to at least some of the frequency coefficients so that scaled frequency coefficients are encoded and wherein at least one of:

a) different scaling factors are applied to at least some frequency coefficients;

b) the same scaling factor is applied to each frequency coefficient; and

,

c) the scaling factor is used to reduce a magnitude of each frequency coefficient.

5. A method according to claim 1, wherein the frequency components are arranged in a plurality of levels and wherein each bit encoding scheme defines a respective number of bits to be used to encode the frequency coefficients in each of the plurality of levels.

6. A method according to claim 1, wherein the method includes:

a) determining a point of gaze of an observer of the one or more images;

b) selectively encoding frequency coefficients at least partially in accordance with the point of gaze by;

i) determining a distance between the point of gaze and a position of the array of pixels in the one or more images; and

,

ii) selectively encoding frequency coefficients in accordance with the distance so that less frequency coefficients are encoded for greater distances.

7. A method according to claim 1, wherein the image data defines a plurality of channels, and wherein the method includes selectively encoding frequency coefficients for each channel in parallel by;

a) converting the RGB channels into YCbCr channels; and

,

b) transforming the YCbCr channels by selectively encoding more frequency coefficients for the Y channel than the Cb or Cr channels.

8. A method according to claim 1, wherein the method includes obtaining pixel data from image data by:

a) buffering image data corresponding to a next n−

1 rows of pixels of the image;

b) buffering image data for a next n pixels of the next row of pixels;

c) obtaining pixel data for a next n×

n block of pixels from the buffered image data;

d) repeating steps b) and c) until pixel data has been obtained from all of the n rows of pixels; and

,

e) repeating steps a) to d) until pixel data has been obtained from each row of pixels of the image.

9. A method according to claim 1, wherein the method includes:

a) selectively encoding frequency coefficients in parallel; and

,

b) generating compressed image data at least in part by parallel to serial byte encoding.

10. A method according to claim 1, wherein the digital reality is at least one of:

a) augmented reality;

b) virtual reality; and

,

c) mixed reality.

11. A method according to claim 1, wherein the method is used for displaying image data in a wearable digital reality headset by receiving the compressed image data from a computing device via at least one of:

a) a communication network; and

,

b) a wireless communications link.

12. A method according to claim 1, wherein the method is used for at least one of:

a) transmitting virtual reality video data; and

,

b) wirelessly transmitting virtual reality video data.

13. A method of decompressing compressed image data representing one or more images, the method including:

a) obtaining compressed image data;

b) determining a selected one of a plurality of bit encoding schemes, wherein each of the plurality of bit encoding schemes selectively encodes different frequency coefficients with respective different numbers of bits to provide a different degree of compression, wherein the bit encoding scheme was selected at least in part based on;

i) a desired degree of compression; and

,

ii) a position of the array of pixels in the one or more images;

c) decoding the encoded frequency coefficients in accordance with the selected bit encoding scheme by;

i) determining a set of encoded frequency coefficients from the compressed image data in accordance with the selected bit encoding scheme;

ii) performing bit decoding of the encoded frequency coefficients in accordance with the selected bit encoding scheme to thereby generate a set of frequency coefficients, wherein at least one frequency coefficient is generated so that the set of encoded frequency coefficients is smaller than the set of frequency coefficients; and

,

iii) applying an inverse transformation to the set of frequency coefficients to determine pixel data representing an array of pixels within the one or more images.

14. A method according to claim 13, wherein the bit encoding scheme is selected based on at least one of:

a) a transmission bandwidth of a wireless communications link used to transmit the compressed image data to a display device;

b) a transmission quality of service of a wireless communications link used to transmit the compressed image data to a display device;

c) a lens arrangement of a display device;

d) movement of a display device;

e) image display requirements of a display device;

f) a target display resolution of a display device.

15. Apparatus for compressing image data representing one or more images, the apparatus including at least one electronic encoder processing device that:

a) obtains pixel data from the image data, the pixel data representing an array of pixels within the one or more images;

b) applies a transformation to the pixel data to determine a set of frequency coefficients indicative of frequency components of the array of pixels;

c) selecting one of a plurality of bit encoding schemes, wherein each of the plurality of bit encoding schemes selectively encodes different frequency coefficients with respective different numbers of bits to provide a different degree of compression and wherein the bit encoding scheme is selected at least in part based on;

i) a desired degree of compression; and

,

ii) a position of the array of pixels in the one or more images;

d) selectively encodes at least some of the frequency coefficients using the selected bit encoding scheme to thereby generate a set of encoded frequency coefficients, wherein the bit encoding scheme defines the number of bits used to encode the frequency coefficients, and wherein the frequency coefficients are selectively encoded so that;

i) at least some of the encoded frequency coefficients have different numbers of bits; and

,

ii) at least one frequency coefficient is discarded so that the set of encoded frequency coefficients is smaller than the set of frequency coefficients; and

,

e) generates compressed image data using the encoded frequency coefficients.

16. Apparatus according to claim 15, wherein the at least one electronic encoder processing device selects the bit encoding scheme based on at least one of:

a) a transmission bandwidth of a wireless communications link used to transmit the compressed image data to a display device;

b) a transmission quality of service of a wireless communications link used to transmit the compressed image data to a display device;

c) a lens arrangement of a display device;

d) movement of a display device;

e) image display requirements of a display device;

f) a target display resolution of a display device.

17. Apparatus according to claim 15, wherein the apparatus includes an encoder in wireless communication with a decoder allowing image data to be transferred between the encoder and decoder as compressed image data, wherein the encoder is at least one of coupled to and part of a suitably programmed processing system and the decoder is at least one of coupled to and part of a wearable display device.

18. Apparatus according to claim 15, wherein the at least one encoder processing device includes:

a) a suitably programmed field programmable gate array;

b) an Application-Specific Integrated Circuit; and

,

c) a Graphics Processing Unit.

19. Apparatus according to claim 15, wherein the pixel data defines a plurality of channels, and wherein the apparatus includes:

a) a respective processing device for each channel; and

,

b) a parallel processing device for processing each channel in parallel.

20. Apparatus for decompressing compressed image data representing one or more images, the apparatus including at least one electronic decoder processing device that:

a) obtains compressed image data;

b) determines a selected one of a plurality of bit encoding schemes, wherein each of the plurality of bit encoding schemes selectively encodes different frequency coefficients with respective different numbers of bits to provide a different degree of compression, wherein the bit encoding scheme was selected at least in part based on;

i) a desired degree of compression; and

,

ii) a position of the array of pixels in the one or more images;

c) decodes the encoded frequency coefficients in accordance with the selected bit encoding scheme by;

i) determining a set of encoded frequency coefficients from the compressed image data in accordance with the selected bit encoding scheme that defines the number of bits used in each encoded frequency coefficient;

ii) performing bit decoding of the encoded frequency coefficients in accordance with the selected bit encoding scheme to thereby generate a set of frequency coefficients, wherein at least one frequency coefficient is generated so that the set of encoded frequency coefficients is smaller than the set of frequency coefficients; and

,

iii) applying an inverse transformation to the set of frequency coefficients to determine pixel data representing an array of pixels within the one or more images.