Image Processing Apparatus, Image Processing Method, and Program

• US 20090219404A1
• Filed: 02/18/2009
• Published: 09/03/2009
• Est. Priority Date: 02/19/2008
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kobayashi, Seiji
• Application Number:
  US12/388,065
• Publication Number:
  US 20090219404A1
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/1883 the unit relating to sub-band structure, e.g. hierarchical level, directional tree, e.g. low-high [LH], high-low [HL], high-high [HH]
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/619 the transform being operated outside the prediction loop
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/635 characterised by filter definition or implementation details
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

An image processing apparatus includes a time subband splitting unit configured to generate a lower-frequency subband split signal formed of low frequency components at a frame rate lower than a frame rate of an image signal and a higher-frequency subband split signal formed of high frequency components by performing a subband splitting process in a time direction on the image signal; a first encoding unit configured to compress the lower-frequency subband split signal; and a second encoding unit configured to compress the higher-frequency subband split signal, wherein the first encoding unit and the second encoding unit perform different encoding processes.

Claims

1. An image processing apparatus comprising:

time subband splitting means for generating a lower-frequency subband split signal formed of low frequency components at a frame rate lower than a frame rate of an image signal and a higher-frequency subband split signal formed of high frequency components by performing a subband splitting process in a time direction on the image signal;

first encoding means for compressing the lower-frequency subband split signal; and

second encoding means for compressing the higher-frequency subband split signal,

wherein the first encoding means and the second encoding means perform different encoding processes.

2. The image processing apparatus according to claim 1, wherein the first encoding means includes encoding processing means having a compression efficiency higher than the second encoding means.

3. The image processing apparatus according to claim 1, wherein the second encoding means includes encoding processing means having a circuit scale smaller than the first encoding means.

4. The image processing apparatus according to claim 1, wherein the time subband splitting means is a wavelet converter in the time direction, and performs a process for dividing a frame signal into low-frequency components and high-frequency components by performing a Haar transform among a plurality of frames adjacent in the time direction.

5. The image processing apparatus according to claim 1, wherein the time subband splitting means receives an image signal at P frames per second (P is an integer) as the image signal and generates a lower-frequency subband split signal and a higher-frequency subband split signal at (P/Q) frames/second (Q is an integer of 2 or more) by performing a subband splitting process in the time direction.

6. The image processing apparatus according to claim 5, wherein the first encoding means and the second encoding means perform an encoding process at (P/Q) frames per second.

7. The image processing apparatus according to claim 1, further comprising:

recording means for recording lower frequency stream data output from the first encoding means and higher frequency stream data output from the second encoding means;

first decoding means for receiving lower-frequency stream data recorded in the recording means and performing a process for decompressing low-frequency components;

second decoding means for receiving higher-frequency stream data recorded in the recording means and performing a process for decompressing high-frequency components; and

time subband combining means for generating a combined image signal at a frame rate higher than a frame rate of image signals, which are the decoding results of the first decoding means and the second decoding means, by performing a subband combining process in the time direction on the image signals, which are the decoding results of the first decoding means and the second decoding means.

8. The image processing apparatus according to claim 7, wherein the time subband combining means is an inverse wavelet converter in the time direction.

9. The image processing apparatus according to claim 1,

wherein the time subband splitting means generates a plurality of higher frequency subband splitting signals as the higher frequency subband splitting signals, and

wherein the frame rates of the lower frequency subband splitting signal and the plurality of higher frequency subband splitting signals are frame rates that are determined in accordance with the total number of the lower frequency subband splitting signal and the plurality of higher frequency subband splitting signals.

10. The image processing apparatus according to claim 9,

wherein the second encoding means includes a plurality of different encoding means for performing a compression process on each of the plurality of higher frequency subband splitting signals.

11. The image processing apparatus according to claim 1,

wherein the time subband splitting means generates one lower frequency subband splitting signal by performing a process for adding signal values of corresponding pixels of N (N≧

2) image frames continuous with respect to time, which are contained in the image signal, and generates N−

1 higher frequency subband splitting signals by performing processes for adding and subtracting signal values of corresponding pixels of N (N≧

2) image frames continuous with respect to time, which are contained in the image signal, and

wherein each of the N−

1 higher frequency subband splitting signals is a signal that is calculated by differently setting a combination of image frames for which an addition process and a subtraction process are performed.

12. The image processing apparatus according to claim 1, further comprising:

an image-capturing element configured to obtain an image signal by photoelectric conversion,

wherein the time subband splitting means generates a lower-frequency subband split signal at a frame rate lower than a frame rate of a signal from the image-capturing element and a higher-frequency subband split signal by performing a process on the image signal from the image-capturing element.

13. An image processing apparatus comprising:

first decoding means for receiving lower-frequency stream data recorded in recording means and performing a process for decompressing low-frequency components;

second decoding means for receiving higher-frequency stream data recorded in the recording means and performing a process for decompressing high-frequency components; and

time subband combining means for generating a combined image signal at a frame rate higher than a frame rate of the image signals, which are the decoding results of the first decoding means and the second decoding means, by performing a subband combining process in the time direction on the image signals, which are decoding results of the first decoding means and the second decoding means.

14. The image processing apparatus according to claim 13, wherein the first decoding means performs decoding of encoded data having a compression efficiency higher than that of the second decoding means.

15. The image processing apparatus according to claim 13,

wherein the second decoding means receives a plurality of different items of higher-frequency stream data recorded in the recording means and generates a plurality of different image signals of high frequency components, and

wherein the time subband combining means generates a combined image signal at a high frame rate by performing a subband combining process in the time direction on an image signal that is a decoding result of the first decoding means and a plurality of different image signals that are decoding results of the second decoding means.

16. The image processing apparatus according to claim 13,

wherein the time subband combining means generates a combined image signal at a frame rate determined in accordance with the total number of the image signal of low frequency components, which are generated by the first decoding means, and the image signals of high frequency components, which are generated by the second decoding means.

17. The image processing apparatus according to claim 13,

wherein the second decoding means includes a plurality of different decoding means for performing a decompression process on each of a plurality of items of higher frequency stream data recorded in the recording means.

18. An image processing method comprising the steps of:

generating a lower-frequency subband split signal formed of low frequency components at a frame rate lower than a frame rate of an image signal and a higher-frequency subband split signal formed of high frequency components by performing a subband splitting process in a time direction on the image signal;

compressing the lower-frequency subband split signal; and

compressing the higher-frequency subband split signal,

wherein the step of compressing the lower-frequency subband split signal and the step of compressing the higher-frequency subband split signal perform different encoding processes.

19. An image processing method comprising the steps of:

receiving lower-frequency stream data and performing a process for decompressing low-frequency components;

receiving higher-frequency stream data and performing a process for decompressing high-frequency components; and

generating a combined image signal at a frame rate higher than the frame rate of a first image signal by performing a subband combining process in the time direction on the basis of the first image signal obtained by a process for decompressing the lower-frequency stream data and a second image signal obtained by a process for decompressing the higher-frequency stream data.

20. A program for causing a computer to perform an information processing method, the information processing method comprising the steps of:

generating a lower-frequency subband split signal formed of low frequency components at a frame rate lower than a frame rate of an image signal and a higher-frequency subband split signal formed of high frequency components by performing a subband splitting process in a time direction on the image signal;

compressing the lower-frequency subband split signal; and

compressing the higher-frequency subband split signal,

wherein the step of compressing the lower-frequency subband split signal and the step of compressing the higher-frequency subband split signal perform different encoding processes.

21. A program for causing a computer to perform an information processing method, the information processing method comprising the steps of:

receiving lower-frequency stream data and performing a process for decompressing low-frequency components;

receiving higher-frequency stream data and performing a process for decompressing high-frequency components; and

generating a combined image signal at a frame rate higher than the frame rate of a first image signal by performing a subband combining process in the time direction on the basis of the first image signal obtained by a process for decompressing the lower-frequency stream data and a second image signal obtained by a process for decompressing the higher-frequency stream data.

22. An image processing apparatus comprising:

a time subband splitting unit configured to generate a lower-frequency subband split signal formed of low frequency components at a frame rate lower than a frame rate of an image signal and a higher-frequency subband split signal formed of high frequency components by performing a subband splitting process in a time direction on the image signal;

a first encoding unit configured to compress the lower-frequency subband split signal; and

a second encoding unit configured to compress the higher-frequency subband split signal,

wherein the first encoding unit and the second encoding unit perform different encoding processes.

23. An image processing apparatus comprising:

a first decoding unit configured to receive lower-frequency stream data recorded in a recording unit and perform a process for decompressing low-frequency components;

a second decoding unit configured to receive higher-frequency stream data recorded in the recording unit and perform a process for decompressing high-frequency components; and

a time subband combining unit configured generate a combined image signal at a frame rate higher than a frame rate of the image signals, which are the decoding results of the first decoding unit and the second decoding unit, by performing a subband combining process in the time direction on the image signals, which are decoding results of the first decoding unit and the second decoding unit.