Compound eye image pickup device utilizing plural image sensors and plural lenses

• US 6,236,748 B1
• Filed: 07/31/1995
• Issued: 05/22/2001
• Est. Priority Date: 08/02/1994
• Status: Expired due to Term

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: Mori, Katsuhiko, Iijima, Katsumi, Yano, Kotaro, Kondo, Toshiaki
• Primary Examiner(s): MEHTA, BHAVESH M
• Application Number:
  US08/509,142
• Time in Patent Office: 2,122 Days
• Field of Search: 382/154, 382/153, 382/284, 382/285, 382/294, 358/540, 358/450, 348/42, 348/47, 348/48, 348/578, 348/580, 348/239
• US Class Current: 382/154
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0088 Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0096 Synchronisation or controlling aspects

Abstract

For facilitating the search of corresponding points in plural images obtained from plural image pickup systems, in obtaining left and right images from left and right image pickup systems and searching two corresponding points in the two images by a corresponding point extraction unit, the search range thereof is determined according to the phototaking parameters of the image pickup systems. The paired corresponding points, thus extracted, are synthesized, in a synthesis/conversion unit, into a panoramic image or a high definition image. Thus achieved are a reduction in the search time and an improvement in the precision of search.

Claims

1. An image pickup device comprising:

plural image pickup systems;

a search unit adapted to search paired corresponding points between plural images obtained from said plural image pickup systems in a search area set in the plural images; and

a search range control unit adapted to detect an overlapping area between the plural images according to phototaking parameters of the plural images and to determine the range of the search area set in the overlapping area of the plural images to be searched by said search unit.

2. An image pickup device according to claim 1, wherein said search range control unit determines the search range by adding, to a basic search range determined according to said phototaking parameters, a marginal range based on the individual differences of said plural image pickup systems.

3. An image pickup device according to claim 1, wherein said search unit is based on a template matching method in which a template of an image is moved in another image and a corresponding point is extracted at a position where the correlation becomes maximum.

4. A compound-eye image pickup device according to claim 1, wherein said phototaking parameters are information on the convergence angle.

5. An image pickup device according to claim 4, wherein said phototaking parameters comprise information relating to the focal length.

6. An image pickup device according to claim 1, wherein said search range control unit determines the search range by converting images, taken in the presence of a convergence angle, into images taken without such convergence angle.

7. An image pickup device comprising:

plural image pickup means;

memory means for respectively storing image information taken by said plural image pickup means;

means for determining a convergence angle of said image pickup means;

area determination means for determining, based on the convergence angle, an area for searching corresponding points in the plural images stored in said memory means; and

search and controlling means for correcting the area determined by said area determination means based on error information in phototaking parameters of said image pickup means and for setting the corrected area as a search range.

8. An image pickup device according to claim 7, wherein said search is based on a template matching method in which a template of an image is moved in another image and a corresponding point is extracted at a position where the correlation becomes maximum.

9. An image pickup device according to claim 8, wherein said area determination means is adapted to determine the search area by converting images, taken in the presence of a convergence angle, into images taken without such convergence angle.

10. An image pickup method for use in an image pickup device including plural image pickup systems, comprising:

a step of searching paired corresponding points between plural images obtained from said plural image pickup systems in a search area set in said plural images; and

a step of detecting an overlapping area between the plural images according to phototaking parameters of the plural images and determining the range of the search area set in the overlapping area of the plural images to be searched by said search step.

11. A method according to claim 10, wherein said determining step is adapted to determine the search range by adding, to a basic search range determined according to the phototaking parameters, a marginal range based on the individual difference of the plural image pickup systems.

12. A method according to claim 10, wherein said search step is based on a template matching method in which a template of an image is moved in another image and a corresponding point is extracted at a position where the correlation becomes maximum.

13. A method according to claim 10, wherein the phototaking parameters comprise information relating to the convergence angle.

14. A method according to claim 13, wherein the phototaking parameters comprise information relating to the focal length.

15. A method according to claim 10, wherein said determining step determines the search range by converting images, taken in the presence of a convergence angle, into images taken without such convergence angle.

16. An image pickup method for use in an image pickup device including plural image pickup means, comprising:

a step of respectively storing image information taken by the plural image pickup means into memory means;

a step of determining a convergence angle of the image pickup means,

a step of determining, based on the convergence angle, an area for searching corresponding points in the plural images stored in the memory means; and

a step of correcting the area determined by said area determination step based on error information in phototaking parameters of the image pickup means and for setting the correction area as a search range.

17. A method according to claim 16, wherein the search is based on a template matching method in which a template of an image is moved in another image and a corresponding point is extracted at a position where the correlation becomes maximum.

18. A method according to claim 17, wherein said area determining step is adapted to determine the search area by converting images, taken in the presence of a convergence angle, into images taken without such convergence angle.

19. A recording medium for computer-readably storing an image pickup method for use in an image pickup device including plural image pickup systems, said method comprising:

a step of searching paired corresponding points between plural images obtained from said plural image pickup systems in a search area set in the plural images; and

a step of detecting an overlapping area between the plural images according to phototaking parameters of the plural images and determining the range of the search area set in the overlapping area of the plural images to be searched by said searching step.

20. A recording medium according to claim 19, wherein said determining step determines the search range by adding, to a basic search range determined according to the phototaking parameters, a marginal range based on the individual difference of the plural image pickup systems.

21. A recording medium according to claim 19, wherein said search step is based on a template matching method in which a template of an image is moved in another image and a corresponding point is extracted at a position where the correlation becomes maximum.

22. A recording medium according to claim 19, wherein the phototaking parameters comprise information relating to the convergence angle.

23. A recording medium according to claim 22, wherein the phototaking parameters comprise information relating to the focal length.

24. A recording medium according to claim 19, wherein said determining step determines the search range by converting images, taken in the presence of a convergence angle, into images taken without such convergence angle.

25. A recording medium for computer-readably storing an image pickup method for use in an image pickup device including plural image pickup means, said method comprising:

a step of respectively storing image information taken by the plural image pickup means into memory means into memory means;

a step of determining a convergence angle of the image pickup means;

a step of determining, based on the convergence angle, an area for searching correspoding points in the plural images stored in the memory means; and

a step of correcting the area determined by said determination step based on error information in phototaking parameters of the image pickup means and for setting the corrected area as a search range.

26. A recording medium according to claim 25, wherein the search is based on a template matching method in which a template of an image is moved in another image and a corresponding point is extracted at a position where the correlation becomes maximum.

27. A recording medium according to claim 26, wherein said area determination step is adapted to determine the search area by converting images, taken in the presence of a convergence angle, into images taken without such convergence angle.