Image processing apparatus and method for processing images with different scalabilites

• US 6,650,783 B2
• Filed: 01/14/1999
• Issued: 11/18/2003
• Est. Priority Date: 01/14/1998
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Hamanaka, Akiyoshi
• Primary Examiner(s): Chen, Wenpeng
• Application Number:
  US09/229,895
• Publication Number:
  US 20020090138A1
• Time in Patent Office: 1,769 Days
• Field of Search: 382/240, 382/232, 382/236, 382/239, 348/14.12, 348/14.08, 375/240.16, 375/240.03, 375/240.12, 708/203
• US Class Current: 382/240
• CPC Class Codes: H04N 19/37 with arrangements for assigning different transmission priorities to video input data or to video coded data

Abstract

A single base layer is formed by reducing information of input image data with at least a plurality of scalable factors, and a plurality of enhancement layers having information associated with a plurality of scalable factors are formed by using the base layer, thereby minimizing the amount of data to be transmitted and allowing the receiving side to arbitrarily select one of images with large information amounts based on various scalable factors. A system suited for image selection on the receiving side is also constructed.

Claims

1. An image processing apparatus comprising:

(a) a base layer generator, arranged to generate a base layer by reducing information of input image data with a plurality of kinds of scalable factors;

(b) an enhancement layer generator, arranged to generate a plurality of kinds of enhancement layers of different kinds of scalable factors, wherein each of the plurality of kinds of enhancement layers is generated by using both the common information of input image data and the base layer; and

(c) an encoder, arranged to encode the base layer and the plurality of kinds of enhancement layers and to transmit the encoded base layer and the encoded plurality of kinds of enhancement layers.

2. An apparatus according to claim 1, wherein said encoder serially processes the base layer and the plurality of kinds of enhancement layers.

3. An apparatus according to claim 1, wherein said encoder divides a luminance signal and color signal included in each of the layers into blocks each consisting of a plurality of pixels in vertical and horizontal directions, and forms macroblocks by combining predetermined numbers of luminance signal blocks and color signal blocks.

4. An apparatus according to claim 1, wherein the plurality of enhancement layers include a spatial enhancement layer having information for increasing a spatial resolution and a time enhancement layer for increasing a time resolution.

5. An apparatus according to claim 4, wherein the base layer is obtained by decreasing the spatial resolution and time resolution of the input image data, the spatial enhancement layer is formed from data obtained by decreasing the time resolution of the input image data and the base layer, and the time enhancement layer is formed from data obtained by decreasing the spatial resolution of the input image data and the base layer.

6. An apparatus according to claim 1, further comprising a selector arranged to selectively output the base layer, the plurality of kinds of enhancement layers, or the input image data.

7. An image processing method comprising the steps of:

(a) generating a base layer by reducing information of input image data with a plurality of kinds of scalable factors;

(b) generating a plurality of kinds of enhancement layers of different kinds of scalable factors, wherein each of the plurality of kinds of enhancement layers is generated by using both the common information of input image data and the base layer;

(c) encoding the base layer and the plurality of kinds of enhancement layers; and

(d) transmitting the encoded base layer and the encoded plurality of kinds of enhancement layers.

8. A method according to claim 7, wherein the encoding step comprises serially processing the base layer and the plurality of enhancement layers.

9. An image processing apparatus comprising:

(a) a reconstruction unit, arranged to extract encoded data of a base layer obtained from input data by reducing information of image data by using a plurality of kinds of scalable factors, and decode the encoded data of the base layer to reconstruct an image signal of the base layer; and

(b) a forming unit, arranged to extract encoded data of a plurality of kinds of enhancement layers of different kinds of scalable factors respectively from the input data, and to decode the encoded data of the plurality of kinds of enhancement layers each, to reconstruct a plurality of image signals according to the plurality of kinds of enhancement layers, wherein each of the plurality of kinds of enhancement layers is generated by using both the common information of input image data and the base layer.

10. An apparatus according to claim 9, wherein said forming unit serially processes the base layer and the plurality of kinds of enhancement layers.

11. An apparatus according to claim 9, wherein the data is encoded in units of blocks each including a plurality of pixels in vertical and horizontal directions, and said forming unit outputs data, input in units of blocks, in units of lines.

12. An apparatus according to claim 9, wherein said forming unit forms a plurality of image signals having a larger information amount than the reconstructed image signal of the base layer by using the plurality of kinds of enhancement layers and the base layer, decoded by said decoder.

13. An apparatus according to claim 12, further comprising a selector arranged to selectively output the image signal according to the base layer and the plurality of image signals having the larger information amount.

14. An apparatus according to claim 13, wherein said selector effects selection by manual operation.

15. An apparatus according to claim 13, wherein said selector comprises an image display unit, and said image display unit can display an image associated with the image signal of the base layer.

16. An apparatus according to claim 15, wherein said selector further comprises a pointing device arranged to designate a position on said display unit, and when an image associated with the image signal of the base layer is displayed, said pointing device designates a position on the image to select the plurality of image signals having the larger information amount so as to display the image signals.

17. An apparatus according to claim 15, wherein said display unit can simultaneously display image signals of a plurality of base layers from said reconstruction unit, and can select an image signal for displaying a motion image from the plurality of image signals.

18. An image processing method comprising the steps of:

(a) extracting encoded data of a base layer obtained from input data by reducing information of image data by using a plurality of kinds of scalable factors;

(b) decoding the encoded data of the base layer to reconstruct an image signal of the base layer;

(c) extracting encoded data of a plurality of kinds of enhancement layers of different kinds of scalable factors respectively from the input data; and

(d) decoding the encoded data of the plurality of kinds of enhancement layers each, to reconstruct a plurality of image signals according to the plurality of kinds of enhancement layers, by using the reconstructed image signal of the base layer, wherein each of the plurality of kinds of enhancement layers is generated by using both the common information of input image data and the base layer.

19. A method according to claim 18, further comprising a selection step of selectively outputting the image signal of the base layer and the plurality of image signals according to the plurality of kinds of enhancement layers.

20. An image processing method comprising the steps of:

(a) inputting a first image signal reconstructed from data of a common base layer obtained by reducing information of image data by using a plurality of kinds of scalable factors, and second and third image signals each of which is larger in information amount associated with a respective different one of the plurality of kinds of scalable factors than the image signal of the common base layer, reconstructed from data of a plurality of kinds of enhancement layers having information associated with the different one of the plurality of kinds of scalable factors and the data of the common base layer,

respective data of the plurality of kinds of enhancement layers being formed by using both the common image data and the base layer;

(b) selecting the first, second, or the third image signal; and

(c) displaying an image corresponding to the selected image signal.

21. A method according to claim 20, wherein the selection step comprises executing said selecting step by manual operation.

22. A method according to claim 20, wherein, in said selecting step, when an image associated with the first image signal is displayed, an image associated with the second or third image signal can be selected to be displayed by designating a position on the corresponding image using a pointing device for designating a position on a display screen.

23. A method according to claim 22, wherein a plurality of first image signals are input, images associated with the plurality of first image signals can be simultaneously displayed, and an image signal to be displayed as a motion image is selected from the plurality of image signals.

24. A computer-readable recording medium storing a program for executing the steps of:

(a) inputting a first image signal reconstructed from data of a common base layer obtained by reducing information of image data by using a plurality of kinds of scalable factors, and second and third image signals each of which is larger in information amount associated with a respective different one of the plurality of kinds of scalable factors than the image signal of the common base layer reconstructed from data of a plurality of kinds of enhancement layers having information associated with the different one of the plurality of kinds of scalable factors and the data of the common base layer,

respective data of the plurality of kinds of enhancement layers being formed by using both the common image data and the base layer;

(b) selecting the first, second, or the third image signal; and

(c) displaying an image corresponding to the selected image signal.

25. A computer-readable recording medium storing a program for executing the steps of:

(a) generating a base layer by reducing information of input image data with a plurality of kinds of scalable factors;

(b) generating a plurality of kinds of enhancement layers of different kinds of scalable factors respectively by using the base layer, wherein each of the plurality of kinds of enhancement layers is generated by using both the common information of input image data and the base layer;

(c) encoding the base layer and the plurality of kinds of enhancement layers; and

(d) transmitting the encoded base layer and the encoded plurality of kinds of enhancement layers.

26. A computer-readable recording medium storing a program for executing the steps of:

(a) extracting encoded data of a base layer obtained by reducing information of image data by using a plurality of kinds of scalable factors, from input data;

(b) decoding the encoded data of the base layer to reconstruct an image signal of the base layer;

(c) extracting encoded data of a plurality of kinds of enhancement layers of different kinds of scalable factors respectively from the input data; and

(d) decoding the encoded data of the plurality of kinds of enhancement layers, to reconstruct a plurality of image signals according to the plurality of kinds of enhancement layers, by using the reconstructed image signal of the base layer in common to the different kinds of scalable factors,

wherein each of the plurality of kinds of enhancement layers is generated by using both the common information of image data and the base layer.