High quality image and audio coding apparatus and method depending on the ROI setting

• US 7,702,513 B2
• Filed: 09/08/2003
• Issued: 04/20/2010
• Est. Priority Date: 09/19/2002
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kishi, Hiroki
• Primary Examiner(s): Hudspeth; David R
• Assistant Examiner(s): SHAH, PARAS D
• Application Number:
  US10/656,375
• Publication Number:
  US 20040057514A1
• Time in Patent Office: 2,416 Days
• Field of Search: 704/500, 704/230, 704/228, 382/243, 382/232, 386/112, 386/109
• US Class Current: 704/500
• CPC Class Codes: H04N 19/127 Prioritisation of hardware or computational resources
• CPC Class Codes: H04N 19/129 Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/645 by grouping of coefficients into blocks after the transform
• CPC Class Codes: H04N 19/647 using significance based coding, e.g. Embedded Zerotrees of Wavelets [EZW] or Set Partitioning in Hierarchical Trees [SPIHT]
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

Disclosed are an image processing apparatus and an image processing method capable of changing a quality of audio data according to a degree of significance of an image. The image processing apparatus includes an image encoding unit for encoding image data inputted, an audio data encoding unit for encoding audio data inputted together with the image data, a significant scene setting unit for setting to encode a part of region of the image with a high image quality, and an audio data encoding setting unit for setting to process the audio data with a high quality in accordance with the setting by the significant scene setting unit.

Claims

1. An image and audio processing apparatus comprising:

an input unit configured to input image data and audio data corresponding thereto;

an image encoder configured to encode the image data input by said input unit;

a first audio encoder configured to encode the audio data input by said input unit by an audio encoding method for encoding general audio data;

a second audio encoder configured to encode the audio data input by said input unit, by another audio encoding method which is suitable for encoding speech data;

an image encoding setting unit configured to set the encoding for said image encoder to encode the image data so that a partial region in each of frame images included in the image data is encoded with a high image quality;

an audio encoding unit including said first encoder and said second encoder, said audio encoding unit being configured to operate said first audio encoder and said second audio encoder in accordance with whether said image encoding setting unit sets the encoding to make the partial region in each of frame images the high image quality, such that (1) the audio data is encoded by said first audio encoder to provide audio encoded data irrespectively of whether said image encoding setting unit effects the setting of the encoding, and (2) the audio data is encoded by each of said first audio encoder and said second audio encoder in a time period corresponding to the encoding set by said image encoding setting unit so that one of the respective audio encoded data provided by said first audio encoder and said second audio encoder is selected during the time period; and

a data integration unit configured to integrate the encoded audio data provided by said audio encoding unit with encoded image data encoded by said image encoder in accordance with whether said image encoding setting unit sets the encoding for said image encoder, so as (a) to integrate, with the encoded image data encoded by said image encoder, a selected one of the respective encoded audio data provided by said first audio encoder and said second audio encoder in a case that said image encoding setting unit effects the setting of the encoding, and (b) to integrate, with the encoded image data encoded by said image encoder, the encoded audio data provided by said first audio encoder in a case that said image encoding setting unit does not effect the setting of the encoding, thereby outputting the integrated encoded data.

2. An apparatus according to claim 1, wherein said image encoding setting unit makes the setting so as to encode a region, with the high image quality, including an arbitrary object in the image data.

3. An apparatus according to claim 2, wherein said image encoding setting unit makes ROI setting of the region including the arbitrary object, and

wherein said image encoding unit executes ROI encoding.

4. An apparatus according to claim 1, wherein said image encoding setting unit makes the setting so as to encode the partial region of the image data with the high image quality in accordance with a user'"'"'s instruction for designating an object displayed on a display screen.

5. An apparatus according to claim 4, wherein said image encoding setting unit makes the ROI setting in accordance with the user'"'"'s instruction, and wherein said image encoder executes the ROI encoding.

6. An apparatus according to claim 1, wherein said data integration unit selects encoded audio data having higher quality by comparing audio quality of the encoded audio data provided by said first audio encoder and audio quality of the encoded audio data provided by said second audio encoder, and integrates the selected encoded audio data with the encoded image data.

7. An image and audio processing method comprising:

an input step of inputting image data and audio data corresponding thereto;

an image encoding step of encoding the image data input by said input step;

a first audio encoding step of encoding audio data input by said input step, by an audio encoding method for encoding general audio data;

a second audio data encoding step of encoding the audio data input by said input step, by another audio encoding method which is suitable for encoding speech data;

an image encoding setting step of setting the encoding in said image encoding step to control an image quality of encoded image data in accordance with a partial region in each of frame images included in the image data;

an audio encoding step of performing said first audio encoding step and said second audio encoding step in accordance with whether said image encoding setting step sets the encoding to make the partial region in each of frame images the high image quality, such that (1) the audio data is encoded by said first audio encoding step to provide audio encoded data irrespectively of whether said image encoding setting step effects the setting of the encoding, and (2) the audio data is encoded by each of said first audio encoding step and said second audio encoding step in a time period corresponding to the encoding set in said image encoding setting step so that one of the respective audio encoded data provided by said first audio encoding step and said second audio encoding step is selected during the time period;

a data integration step of integrating the encoded audio data provided in said audio encoding step with encoded image data encoded in said image encoding step in accordance with whether the encoding is set in said image encoding setting step, so as to (a) integrate with the encoded image data encoded in said image data encoding step, a selected one of the respective encoded audio provided in said first audio encoding step and said second audio encoding step in a case that said image encoding setting step effects the setting of the encoding, and (b) to integrate, with the encoded image data encoded in said image encoding step, the encoded audio data provided in said first audio encoding step in a case that said image encoding setting step does not effect the setting of the encoding, thereby outputting the integrated encoded data.

8. A method according to claim 7, wherein said image encoding setting step includes setting so as to encode a region, with the high image quality including an arbitrary object in the image data.

9. A method according to claim 8, wherein said image encoding setting step includes making ROI setting of the region including the arbitrary object, and

wherein said image encoding step includes executing ROI encoding.

10. A method according to claim 7, wherein said image encoding setting step includes setting so as to encode the partial region of the image data with the high image quality in accordance with a user'"'"'s instruction for designating an object displayed on a display screen.

11. A method according to claim 10, wherein said image encoding setting step includes making the ROI setting in accordance with the user'"'"'s instruction, and

wherein said image encoding step includes executing the ROI encoding.

12. A method according to claim 7, wherein said data integration step includes selecting encoded audio data having higher quality by comparing audio quality of the encoded audio data provided in said first audio encoding step and audio quality of the encoded audio data provided in said second audio encoding step, and integrating the selected encoded audio data with the encoded image data.

13. A storage medium storing a program executable by a data processing apparatus, said program including program codes for realizing an image processing method described in claim 7.