Image processing apparatus

• US 7,142,725 B2
• Filed: 10/01/2003
• Issued: 11/28/2006
• Est. Priority Date: 04/09/1992
• Status: Expired due to Fees

• Current Assignee: Olympus Optical Company Ltd. (Olympus Corporation)
• Original Assignee: Olympus Optical Company Ltd. (Olympus Corporation)
• Inventors: Mori, Takeshi, Komiya, Yasuhiro, Iba, Yoichi, Oura, Koutatsu, Nagasaki, Tatsuo, Sawaki, Ryoichi
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): Strege, John B.
• Application Number:
  US10/677,072
• Publication Number:
  US 20040062454A1
• Time in Patent Office: 1,154 Days
• Field of Search: 382/284, 382/294, 382/300, 348/383, 348/750, 353/30
• US Class Current: 382/284
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 1/19505 Scanning picture elements spaced apart from one another in at least one direction
• CPC Class Codes: H04N 1/19515 in two directions
• CPC Class Codes: H04N 1/19584 Combination of arrays
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 2209/048 having several pick-up sensors
• CPC Class Codes: H04N 23/15 Image signal generation with circuitry for avoiding or correcting image misregistration
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors

Abstract

An image processing apparatus, in which an object image focused by a lens is split into a plurality of images by means of a light splitting section. These images are converted into image data items by a plurality of imaging devices which are arranged with their imaging area overlapping in part. The image data items are stored temporarily in an image storing section. A displacement detecting section detects displacement coefficients (rotation angle R and parallel displacement S) from the image signals representing the mutual overlap region of two images which are to be combined and which are represented by two image data items read from the image storing section. The position of any specified pixel of the image displayed is identified by the pixel signal generated by the corresponding pixel of any imaging device. An interpolation section performs interpolation on the pixel values of the imaging device, thereby correcting the values of the other pixels of the image displayed and ultimately generating interpolated image signals. The interpolated image signals are combined with the image signals produced by the imaging device, whereby a display section displays a high-resolution image.

Claims

1. An image display apparatus for displaying a continuous image by using a plurality of images having an overlapping area, comprising:

a plurality of displaying means for displaying the images corresponding to the plurality of image signals;

interpolation arithmetic means for correcting a geometric displacement between the images displayed by said plurality of displaying means;

brightness correcting means for correcting brightness of the overlapping area of the images displayed by said plurality of displaying means; and

image picking-up means for picking up the images displayed by said plurality of displaying means;

wherein when displaying the images, said plurality of displaying means use the plurality of image signals which are subjected to displacement correction of said interpolation arithmetic means and brightness correction of said brightness correcting means;

wherein the interpolation arithmetic means detects a positional relationship between the images displayed by said plurality of displaying means based on the images picked-up by said image picking-up means, and corrects the displacement between the images obtained by said plurality of displaying means by using a correction coefficient calculated from the positional relationship; and

wherein the image picking-up means picks up the images displayed by said plurality of displaying means by partially using an optical system of said plurality of displaying means.

2. An image display apparatus for displaying a continuous image by using a plurality of image signals which correspond to a plurality of images having an overlapping area, comprising:

a plurality of displaying means for displaying the images corresponding to the plurality of image signals;

interpolation arithmetic means for correcting a geometric displacement between the images displayed by said plurality of displaying means;

brightness correcting means for correcting brightness of the overlapping area of the images displayed by said plurality of displaying means;

image picking-up means for picking up the images displayed by said plurality of displaying means;

a frame memory for storing a total image signal as total image data which corresponds to a total image required to be displayed by said plurality of displaying means; and

a plurality of image memories for storing image signals as image data which respectively correspond to the images to be displayed by said plurality of displaying means;

wherein when displaying the images, said plurality of displaying means use the plurality of image signals which are subjected to displacement correction of said interpolation arithmetic means and brightness correction of said brightness correcting means;

wherein the interpolation arithmetic means detects a positional relationship between the images displayed by said plurality of displaying means based on the images picked-up by said image picking-up means, and corrects the displacement between the images obtained by said plurality of displaying means by using a correction coefficient calculated from the positional relationship; and

wherein divided image signals are stored as divided image data in said plurality of image memories, after the divided image signals are read out from the frame memory storing the total image signal, on the basis of a positional relationship between the images displayed by said plurality of displaying means, the positional relationship being determined based on the images picked-up by said image picking-up means.

3. An image display apparatus for displaying a plurality of image signals produced from a plurality of image signals which have overlap areas such that the images are combined into a single continuous image, said image display apparatus comprising:

an interpolation calculating means for performing interpolation on respective ones of the plurality of image signals which produce the overlap areas, whereby any adjacent two of the plurality of images are matched to overlap each other;

a brightness converting means for correcting in brightness the image signals producing the overlap areas, whereby joints between the plurality of images are made inconspicuous;

a plurality of memories for storing, as image data, the plurality of image signals after the image signals producing the overlap areas are interpolated by the interpolation calculating means and corrected by the brightness converting means;

a plurality of displaying means for reading the image data stored in the plurality of memories, and for displaying the plurality of images as said single continuous image; and

an image-pickup means including an image-pickup device for picking up image signals corresponding to an image displayed by the plurality of display means;

wherein the interpolation calculating means performs the interpolation on the respective ones of the plurality of image signals by using a coefficient calculated based on positional relations of the plurality of display means, which are calculated from the image signals corresponding to the image displayed by the plurality of display means which are picked up by the image-pickup means.

4. An image display apparatus according to claim 3, further comprising an image dividing means for dividing an image desired to be displayed into the plurality of image signals.

5. An image display apparatus according to claim 3, wherein said brightness converting means multiplies a weighting coefficient from a weighting coefficient calculator by using a multiplier.

6. An image display apparatus according to claim 3, wherein said image-pickup means shares a part of an optical system of the displaying means.

7. An image display apparatus according to claim 3, wherein said image-pickup means comprises a camera specifically for detecting displacement, provided independently of the image displaying apparatus, and said camera performs an image pickup operation in one of an entire projection area and a part of the projection area.

8. An image display apparatus according to claim 3, wherein said image-pickup means picks up a subject which has a good correlation, and detects displacement.

9. An image display apparatus according to claim 3, wherein the image-pickup means operates to pick up a reference image displayed by the displaying means, the interpolation calculating means does not perform interpolation and the brightness converting means does not perform conversion.

10. An image display apparatus for dividing and processing an image to be displayed on a screen, comprising:

an image dividing means for dividing the image into a plurality of image signals;

an interpolation calculating means for correcting displacement between the plurality of image signals;

a brightness converting means for correcting a brightness between the plurality of image signals;

a plurality of displaying means for displaying the plurality of image signals corrected by the interpolation calculating means and brightness converting means; and

an image-pickup means including an image-pickup device for picking up an image displayed by the displaying means,

wherein said interpolation calculating means obtains a plurality of positional relations of the image displayed by the displaying means from the image signals picked up by the image-pickup means, and interpolates displacement data using a displacement coefficient calculated in accordance with the positional relations.