Processing of image obtained by multi-eye camera

• US 6,549,650 B1
• Filed: 09/10/1997
• Issued: 04/15/2003
• Est. Priority Date: 09/11/1996
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Mori, Katsuhiko, Iijima, Katsumi, Yano, Kotaro, Sakimura, Takeo, Ishikawa, Motohiro, Kurahashi, Sunao
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): Bali, Vikkram
• Application Number:
  US08/926,817
• Time in Patent Office: 2,043 Days
• Field of Search: 382/154, 382/291, 348/42, 348/43, 348/46, 348/47, 348/49, 348/50, 348/51, 348/52, 348/53, 348/54, 348/55, 348/56, 348/57, 348/58, 348/59, 348/60, 348/705, 348/706, 345/419, 345/139, 345/435, 345/113, 345/114, 345/115, 345/116, 356/12, 356/13
• US Class Current: 382/154
• CPC Class Codes: G02B 30/24 involving temporal multiplexing, e.g. using sequentially activated left and right shutters
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/133 Equalising the characteristics of different image components, e.g. their average brightness or colour balance
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/289 Switching between monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 13/31 using parallax barriers
• CPC Class Codes: H04N 13/337 using polarisation multiplexing
• CPC Class Codes: H04N 13/341 using temporal multiplexing
• CPC Class Codes: H04N 13/359 Switching between monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0088 : Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image

Abstract

There is disclosed a binocular camera which can realize panoramic view and stereoscopic view during image sensing. There is also disclosed a binocular camera which has two image sensing optical systems, a circuit for synthesizing right and left sensed parallax image signals to a panoramic image or a three-dimensional image, and a display for displaying the synthesized image signal.

Claims

1. An image sensing apparatus comprising:

an image sensing means for sensing a plurality of images for forming a three-dimensional image and outputting a plurality of image signals representing the images having a predetermined three-dimensional data format;

an input means for inputting a three-dimensional image signal having a predetermined three-dimensional data format different from the predetermined three-dimensional data format of said plurality of image signals outputted by said image sensing means, from an external device;

a selecting means for selectively outputting said plurality of image signals outputted from said image sensing means and said three-dimensional image signal;

a display means for displaying the selected output of said selecting means; and

a converting means for converting the selected output of said selecting means to a display format conforming to a characteristic of said display means.

2. The apparatus according to claim 1, wherein the three-dimensional image signal includes right and left two-dimensional image data.

3. The apparatus according to claim 1, wherein the three-dimensional image signal is descriptive data for generating a three-dimensional image.

4. The apparatus according to claim 1, wherein at least one of said image sensing means comprises an adapter device for inputting said three-dimensional image signal.

5. The apparatus according to claim 1, further comprising output means for outputting said plurality of image signals input from said image sensing means, and a synchronization signal synchronized with each of said plurality of image signals.

6. An image sensing method comprising:

a step of selecting one of a plurality of image synthesis methods different from each other in quality of synthesized image; and

a step of synthesizing two, right and left images sensed by an image sensing optical system to a single synthesized image in accordance with the selected image synthesis method,

wherein the plurality of image synthesis methods include at least a first synthesis method for synthesizing images while giving priority to a synthesis speed.

7. The method according to claim 6, wherein the plurality of image synthesis methods further include a second synthesis method for synthesizing images while giving priority to image quality of the synthesized image.

8. The method according to claim 7, wherein the first synthesis method synthesizes the two, right and left sensed images by giving a predetermined overlapping amount, and

the second synthesis method corrects right-and-left differences of luminance levels and color information, and trapezoidal distortions of the two, right and left sensed images, detects an overlapping region between the two images, and synthesizes the two images using an overlapping amount calculated based on the overlapping region.

9. The method according to claim 6, wherein the selection step includes the step of selecting the first synthesis method in a through display mode and selecting the second synthesis method in recording and reproduction modes.

10. The method according to claim 6, wherein the synthesized image is a panoramic synthesized image.

11. The method according to claim 6, wherein the synthesized image is an image for stereoscopic view.

12. The method of claim 6, wherein the plurality of image synthesis methods are also different from each other in image overlap determination.

13. An image sensing apparatus comprising:

a synthesis means having a plurality of synthesis methods for generating a single synthesized image from two, right and left images sensed by an optical system, the plurality of synthesis methods being different from each other in quality of synthesized image; and

,

a switching means for switching the plurality of synthesis methods,

wherein the plurality of image synthesis methods include at least a first synthesis method for synthesizing the images while giving priority to a synthesis speed.

14. The method according to claim 13, wherein the plurality of synthesis methods further include a second synthesis method for synthesizing images while giving priority to image quality of the synthesized image.

15. The apparatus according to claim 14, wherein the first synthesis method synthesizes the two, right and left sensed images by giving a predetermined overlapping amount, and

the second synthesis method corrects right-and-left differences of luminance levels and color information, and trapezoidal distortions of the two, right and left sensed images, detects an overlapping region between the two images, and synthesizes the two images using an overlapping amount calculated based on the overlapping region.

16. The apparatus according to claim 13, wherein said switching means selects the first synthesis method in a through display mode, and selects the second synthesis method in recording and reproduction modes.

17. The apparatus according to claim 13, wherein the synthesized image is a panoramic synthesized image.

18. The apparatus according to claim 13, wherein the synthesized image is an image for stereoscopic view.

19. The image sensing apparatus of claim 13, wherein the plurality of synthesis methods are also different from each other in image overlap determination.

20. A storage medium that stores a program which is executed by a computer and controls an image sensing apparatus, comprising:

a synthesis program code means for describing a plurality of different synthesis methods for generating a single synthesized image from two, right and left images sensed by an optical system, the plurality of synthesis methods being different from each other in quality of synthesized image; and

,

a switching program code means for switching the plurality of synthesis methods,

wherein the plurality of image synthesis methods include at least a first synthesis method for synthesizing the images while giving priority to a synthesis speed.

21. The medium according to claim 20, wherein the plurality of synthesis methods further include a second synthesis method for synthesizing images while giving priority to the quality of synthesized image.

22. The medium according to claim 21, wherein the first synthesis method synthesizes the two, right and left sensed images by giving a predetermined overlapping amount, and

the second synthesis method corrects right-and-left differences of luminance levels and color information, and trapezoidal distortions of the two, right and left sensed images, detects an overlapping region between the two images, and synthesizes the two images using an overlapping amount calculated based on the overlapping region.

23. The medium according to claim 20, wherein said switching program code means selects a program of the first synthesis method in a through display mode and selects a program of the second synthesis method in recording and reproduction modes.

24. The medium according to claim 20, wherein the synthesized image is a panoramic synthesized image.

25. The medium according to claim 20, wherein the synthesized image is an image for stereoscopic view.

26. The storage medium of claim 20, wherein the synthesis methods are also different from each other in image overlap determination.

27. An image sensing method, comprising:

a step of selecting one of a plurality of image synthesis methods different from each other in quality of synthesized image; and

a step of synthesizing two, right and left images sensed by two, right and left image sensing optical systems to a single synthesized image in accordance with the selected image synthesis method;

wherein the plurality of image synthesis methods are different from each other in image overlap determination.

28. An image sensing apparatus, comprising:

synthesis means having a plurality of synthesis methods for generating a single synthesized image from two, right and left images sensed by two, right and left image sensing optical systems, the plurality of synthesis methods being different from each other in quality of synthesized image; and

switching means for switching the plurality of synthesis methods;

wherein the plurality of image synthesis methods are different from each other in image overlap determination.

29. A storage medium that stores a program which is executed by a computer and controls an image sensing apparatus, comprising:

synthesis program code means for describing a plurality of different synthesis methods for generating a single synthesized image from two, right and left images sensed by two, right and left image sensing optical systems, the plurality of synthesis methods being different from each other in quality of synthesized image; and

switching program code means for switching the plurality of synthesis methods;

wherein the plurality of image synthesis methods are different from each other in image overlap determination.