Image encoding method, image decoding method, moving-image compression method, moving-image expansion method, image encoding program, image decoding device, image encoding device, image encoding/decoding system, and extended image compression/expansion system

• US 20070133889A1
• Filed: 12/27/2006
• Published: 06/14/2007
• Est. Priority Date: 06/29/2004
• Status: Active Grant

• Current Assignee: Olympus Corporation
• Original Assignee: Olympus Corporation
• Inventors: Horie, Kenichi, Sakata, Seiichiro
• Granted Patent: US 7,848,580 B2
• Time in Patent Office: Days
• US Class Current: 382/232
• CPC Class Codes: H04N 19/102 characterised by the element, parameter or selection affected or controlled by the adaptive coding
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

The present image encoding method and the like represents a pixel block prior to performing spatial frequency transformation in processing such as JPEG or the like as a matrix, calculates sum data sc (i) of pixel values in a column i in the matrix and sum data sr (j) of pixel values in a row j in the matrix, calculates nc (i) and nr (j) such that the sc (i) and sc (j) are arrayed in descending order respectively, and when a predetermined condition that permutation is considered reasonable is satisfied, performs permutation of the matrix, generates header information Ph (n) including the nc (i) and nr (j) necessary for subjecting the matrix to permutation in the original state, and adds this to the header of a compressed image file such as JPEG or the like.

Claims

1. An image encoding method comprising:

applying a spatial frequency transformation to an image data to obtain transform coefficients;

encoding the transform coefficients as coded data;

generating a header portion of the coded data comprising information related to the image data;

wherein, prior to applying the spatial frequency transformation, applying to the image data a reversible spatial transformation adapted to the image data; and

adding into the header portion header information necessary for inverse transforming the reversible spatial transformation.

2. The image encoding method according to claim 1,

wherein the encoding of the transform coefficients reduces, in general, data amount of high-frequency transform coefficients more than data amount of low-frequency transform coefficients;

and

wherein the reversible spatial transformation generally suppresses the high-frequency transform coefficients.

3. The image encoding method according to claim 2,

wherein the encoding of the transform coefficients applies, in general, larger quantization steps for quantizing the high-frequency transform coefficients than for quantizing the low-frequency transform coefficients;

and

the reversible spatial transformation is selected adaptively in response to the image data so as to suppress the values of the high-frequency coefficients.

4. The image encoding method according to claim 1,

wherein the image data is partitioned into or comprises a plurality of image blocks;

and wherein

the reversible spatial transformation and the spatial frequency transformation are applied to at least one of the image blocks.

5. The image encoding method according to claim 4,

wherein the reversible spatial transformation includes a plurality of block-wise reversible spatial transformations, each of which is applied to a respective one of the image blocks;

the spatial frequency transformation is applied to the image blocks;

the added header information includes information necessary for inverse transforming the block-wise reversible spatial transformations; and

wherein

the block-wise reversible spatial transformations are adapted to the respective image blocks.

6. The image encoding method according to claim 5,

wherein, prior to applying a block-wise reversible spatial transformation to a respective image block, deciding whether the block-wise reversible spatial transformation is to be applied to the respective image block;

applying, if the decision is affirmative, the block-wise reversible spatial transformation to the image block.

7. The image encoding method according to claim 4,

wherein the at least one of the image blocks is expressible as a matrix M, and wherein

the reversible spatial transformation is expressible as an operation including at least one of row-wise and column-wise permutations of the matrix M.

8. The image encoding method according to claim 4,

wherein the at least one of the image blocks is expressible as a matrix M, and wherein

the reversible spatial transformation is expressible as a linear matrix operation

M→

AMB,

where A and B represent invertible matrices.

9. The image encoding method according to claim 8,

wherein one of the matrix A and matrix B is a matrix for performing row-wise permutation of the matrix M by matrix operation, the other is a matrix for performing column-wise permutation of the matrix M by matrix operation.

10. The image encoding method according to claim 7,

wherein the row-wise permutation is characterized in that rows of the matrix M will be permuted such that sums of values of matrix elements in respective rows are arranged in descending or ascending order in the column direction;

and

wherein the column-wise permutation is characterized in that columns of the matrix M will be permuted such that sums of values of matrix elements in respective columns are arranged in descending or ascending order in the row direction.

11. The image encoding method according to claim 4,

wherein the transform coefficients include a DC coefficient in proportion to a mean value of the at least one image block, and

whereby the reversible spatial transformation keeps the DC coefficient unchanged.

12. An image encoding method according to claim 5,

wherein the image blocks are expressible as matrices;

the block-wise reversible spatial transformation to be applied to at least one of the image blocks is selected from a plurality of operations comprising a permutation operation and a reversible low-pass filtering operation; and

wherein

the added header information includes information necessary for deciding which of the operations has been selected for the at least one image block.

30. The image encoding method according to claim 1,

wherein the reversible spatial transformation is adjustable with at least one adjustment parameter;

and

wherein the header information comprises the adjustment parameter.

31. The image encoding method according to claim 30,

wherein the image data is partitioned into or comprises a plurality of image blocks;

and wherein

the reversible spatial transformation and the spatial frequency transformation are applied to at least one of the image blocks.

32. The image encoding method according to claim 31,

wherein the at least one of the image blocks is expressible as a matrix M, and wherein

the reversible spatial transformation is expressible as one of linear matrix operations

M→

AM,
M→

MB, and
M→

AMB,

where A and B represent invertible matrices.

33. The image encoding method according to claim 32,

wherein the matrix A and matrix B are matrices for effecting spatial low-pass filtering to the image block represented by the matrix M.

34. The image encoding method according to claim 30,

wherein the reversible spatial transformation effects spatial low-pass filtering to the image data;

and

wherein the adjustment parameter includes a filter adjustment parameter for adjusting a filter strength of the spatial low-pass filtering.

35. The image encoding method according to claim 34, further comprising:

quantizing the transform coefficients using a quantization parameter, which is included into the header portion, wherein;

the quantization parameter is adjusted in response to the filter adjustment parameter.

36. The image encoding method according to claim 30,

wherein the transform coefficients include a DC coefficient in proportion to a mean value of the image data, and

whereby the reversible spatial transformation keeps the DC coefficient unchanged.

37. An image encoding method according to claim 30, further comprising:

selecting, prior to encoding, encoding modes; and

selecting the adjustment parameter based on criteria corresponding to a selected one of the encoding modes.

38. An image encoding method according to claim 35, further comprising:

selecting, prior to encoding, encoding modes; and

selecting both the adjustment parameter and the quantization parameter based on criteria corresponding to a selected one of the encoding modes.

39. An image encoding method according to claim 38,

wherein the encoding modes include a high compression mode and a high quality mode.

40. An image encoding method according to claim 34,

wherein the reversible spatial transformation effects spatial low-pass filtering having a selectable directional feature;

and

the header information further comprises information related to the directional feature, which has been actually selected.

13. An image encoding method applicable to a JPEG encoding process, which includes

a) partitioning of an image into a plurality of 8×

8 image blocks;

b) applying a discrete cosine transform to each of the 8×

8 image blocks to thereby obtain DCT coefficients including a DC coefficient indicative of a mean value of the 8×

8 image block, low-frequency AC coefficients and high-frequency AC coefficients;

c) quantizing the DCT coefficients using a quantization table;

d) encoding the quantized DCT coefficients using run-length coding and a subsequent Huffman coding; and

e) generating a header portion,

the method comprising;

applying a reversible spatial transformation to at least one 8×

8 image block prior to applying the discrete cosine transform, whereby the reversible spatial transformation is selected adaptively in response to the at least one 8×

8 image block;

extracting spatial transformation parameters necessary for inverse transforming the spatial transformation; and

adding header information indicative of the spatial transformation parameters into the header portion.

14. The image encoding method according to claim 13,

wherein the adaptive reversible spatial transformation is characterized in that

the DC coefficient remains unchanged whether or not the spatial transformation is applied;

and

amplitudes of the low-frequency AC coefficients are increased by applying the spatial transformation.

15. An image decoding method for decoding encoded image data, which comprises a header portion and a coded data portion, comprising:

reading out the header portion of the encoded image data to extract information related to an image data;

decoding the coded data to obtain transform coefficients in spatial frequency domain;

applying an inverse spatial frequency transformation to the transform coefficients to thereby obtain a second image data;

the decoding method further comprising

extracting information necessary for inverse transforming a reversible spatial transformation;

applying the inverse transformation of the reversible spatial transformation to the second image data to obtain the image data.

16. The image decoding method according to claim 15, further comprising:

identifying, prior to extracting, whether or not the information necessary for inverse transforming is present in the header portion; and

if the information is identified, applying the inverse transformation to the second image data to obtain the image data; and

if the information is not identified, outputting the second image data without performing the inverse transformation as the image data.

17. A moving-image compression method for encoding moving image data comprising image frames in a time-series-like relation using intra-frame compression based on image data within one image frame and inter-frame compression based on multiple image frames,

wherein the intra-frame compression includes partitioning an image frame into image blocks;

applying a spatial frequency transformation to the image blocks to obtain transform coefficients; and

generating a header portion comprising information related to the image frame, the intra-frame compression further comprising;

applying a reversible spatial transformation to at least one of the image blocks prior to applying the spatial frequency transformation; and

adding into the header portion header information comprising spatial transformation parameters necessary for inverse transforming the reversible spatial transformation.

18. The moving-image compression method according to claim 17, further comprising:

deciding, prior to applying the reversible spatial transformation, whether or not the reversible spatial transformation is performed.

19. A moving-image compression method for encoding moving image data comprising image frames in a time-series-like relation using intra-frame compression based on image data within one image frame and inter-frame compression based on multiple image frames,

wherein the inter-frame compression includes partitioning an image frame into pixel blocks;

generating an image block by obtaining difference between at least one of the pixel blocks and a corresponding pixel block related to an image frame to be compared with;

applying a spatial frequency transformation to the image block to obtain transform coefficients; and

generating a header portion comprising information related to the image frame, the inter-frame compression further comprising;

applying a reversible spatial transformation to at least one of the image blocks prior to applying the spatial frequency transformation; and

adding into the header portion header information comprising spatial transformation parameters necessary for inverse transforming the reversible spatial transformation.

20. The moving-image compression method according to claim 19, further comprising:

deciding, prior to applying the reversible spatial transformation, whether or not the reversible spatial transformation is performed.

21. The moving-image compression method according to claim 19,

wherein the header portion comprises information related to a group including multiple image frames;

and wherein the header information comprises the spatial transformation parameters related to a plurality of image frames from the group.

22. A moving-image decoding method for decoding encoded moving-image data, which comprises a header portion and a coded data portion, to obtain a moving-image data comprising:

reading out the header portion to extract information related to an image data related to an image frame;

decoding the coded data portion to obtain transform coefficients related to the image frame;

applying an inverse spatial frequency transformation to the transform coefficients to thereby obtain a second image data;

the decoding method further comprising;

extracting header information necessary for inverse transforming a reversible spatial transformation from the header portion;

applying the inverse transformation of the reversible spatial transformation to the second image data to obtain the image data related to the image frame.

23. An image encoding program, which can be executed by a computing device, comprising operation instructions for:

applying a spatial frequency transformation to an image data to obtain transform coefficients;

encoding the transform coefficients as coded data;

generating a header portion of the coded data comprising information related to the image data;

wherein, prior to applying the spatial frequency transformation, applying to the image data a reversible spatial transformation adapted to the image; and

adding into the header portion header information necessary for inverse transforming the reversible spatial transformation.

24. An image decoding program for decoding encoded image data comprising a header portion and a coded data portion, which can be executed by a computing device, comprising operation instructions for:

reading out the header portion of the encoded image data to extract information related to an image data;

decoding the coded data to obtain transform coefficients;

applying an inverse spatial frequency transformation to the transform coefficients to thereby obtain second image data;

the decoding program further comprising operation instructions for;

extracting information necessary for inverse transforming a reversible spatial transformation;

applying the inverse transformation of the reversible spatial transformation to the second image data to obtain the image data.

25. An image encoding device comprising:

a spatial transforming unit for applying to an image data a reversible spatial transformation adapted to the image data;

a spatial frequency transformation unit for applying a spatial frequency transformation to the image data to thereby obtain transform coefficients;

a header portion generator for creating a header portion comprising information related to the image data; and

a header information addition unit for adding information comprising spatial transformation parameters necessary for inverse transforming the spatial transformation to the header portion.

26. An image decoding device for decoding encoded image data comprising a header portion and a coded data portion comprising:

an inverse spatial frequency transformation unit for applying to transform coefficients resulting from the encoded image data an inverse spatial frequency transformation to thereby obtain second image data;

a header interpreter for reading out spatial transformation parameters from the header portion; and

an inverse spatial transformation unit for applying to the second image data an inverse spatial transformation based on the spatial transformation parameters to thereby obtain an image data.

27. An image encoding-decoding system comprising:

an image encoding device comprising a spatial transforming unit for applying to an image data a reversible spatial transformation adapted to the image data, a spatial frequency transformation unit for applying a spatial frequency transformation to the image data to thereby obtain transform coefficients, a header portion generator for creating a header portion comprising information related to the image data, and a header information addition unit for adding information comprising spatial transformation parameters necessary for inverse transforming the spatial transformation to the header portion; and

an image decoding device for decoding encoded image data comprising a header portion and a coded data portion comprising an inverse spatial frequency transformation unit for applying to transform coefficients resulting from the encoded image data an inverse spatial frequency transformation to thereby obtain second image data, a header interpreter for reading out spatial transformation parameters from the header portion, and an inverse spatial transformation unit for applying to the second image data an inverse spatial transformation based on the spatial transformation parameters to thereby obtain an image data;

wherein the image decoding device is capable of decoding an encoded image data encoded via the image encoding device.

28. An image encoding-decoding system comprising:

a first image encoding device comprising;

1a) dividing unit for partitioning an original image data into image blocks;

1b) spatial frequency transformation unit for acquiring transform coefficients by applying to the image blocks a spatial frequency transformation;

1c) quantizer for quantizing the transform coefficients to generate integral values of transform coefficients;

1d) encoder for encoding these integral values of transform coefficients; and

1e) encoded image file generating means for generating a first encoded image file by combining a header portion with a data obtained based on results of the encoder;

a second image encoding device for generating a second encoded image file obtained by adding to the first image encoding device;

2a) spatial transformation unit for applying a reversible spatial transformation identifiable with spatial transformation parameters to at least one of the image blocks prior to the spatial frequency transformation being performed; and

2b) header information adding unit for adding the spatial transformation parameters into the header portion;

a first image decoding device comprising;

3a) header reader for reading out a header portion from the first encoded image file;

3b) decoder for acquiring integral values of transform coefficients by decoding codes obtained from the first encoded image file;

3c) inverse quantizer for de-quantizing the integral values of transform coefficients generated by the decoder to thereby obtain transform coefficients;

3d) inverse spatial frequency transformation unit for acquiring an image data in units of image block by applying inverse spatial frequency transformation to the transform coefficients from the inverse quantizer; and

3e) rendering means for generating an original image data from the image data in units of image blocks; and

a second image decoding device for acquiring an original image data from the second encoded image file obtained by adding to the first image decoding device;

4a) additional header information reader for reading out the spatial transformation parameters from the header portion; and

4b) inverse spatial transformation unit for inverse transforming the spatial transformation based on the spatial transformation parameters read out by this additional header information reader;

wherein in the event that an encoded image file to be decoded is the second encoded image file, the first image decoding device is able, without error, to restore data different from the original image data by ignoring spatial transformation parameters described in the header portion of the second compressed image file;

and wherein in the event that spatial transformation parameters cannot be read out from the header portion by the additional header information reader, the second image decoding device restores an original image data by processing decoding by a portion given by the first image decoding device alone without processing the inverse spatial transformation unit.

29. The image encoding-decoding system according to claim 28, further comprising:

an image capturing device having the second image encoding device; and

a computer having the first image decoding device.