Method and apparatus for obtaining three dimensional distance information stereo vision

• US 4,792,694 A
• Filed: 04/17/1986
• Issued: 12/20/1988
• Est. Priority Date: 04/17/1985
• Status: Expired due to Term

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Funabashi, Motohisa, Shioya, Makoto, Nishiya, Takushi
• Primary Examiner(s): Nelms, David C.
• Assistant Examiner(s): Ruoff, Jessie
• Application Number:
  US06/853,231
• Time in Patent Office: 978 Days
• Field of Search: 250/558, 358/88, 358/3, 356/12, 356/13, 356/14
• US Class Current: 250/558
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method of forming a three-dimensional stereo vision is disclosed in which, in order to prevent the erroneous detection of an object point in the conventional method using two image focusing lens systems, three image points of an object point formed by logically and/or physically selecting three image focusing lens systems and three image sensing surfaces from a multiplicity of image points on an image sensing surface by using the fact that a positional relation among three image points of the same object point is similar to the positional relation among the three image focusing lens systems, and the positional information of three selected image points on the image sensing surface is used for obtaining three-dimensional distance information of the object point.

Claims

1. A method, of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image point of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein said correlating of the image points comprises allocating the object images formed on said respective image sensing areas to respective imaginary planes which are superimposed with a spatial relationship corresponding to the spatial relationship of said image focusing systems, and detecting those image points on the imaginary planes which lie on the same straight line.

4. A method according to claim 1, wherein each of said image sensing areas are formed on the same one-dimensional image sensing surface.

5. A method according to claim 1, wherein single image focusing system and a single image sensing surface are fixed successively at least at three different positions and the image of said object system is formed on said image sensing surface so as to produce at least three object images by using the formed image data on said image sensing surface.

6. A method according to claim 1, wherein a single image focusing system is fixed successively at different positions so that the image of said object system can be formed on each of at least three image sensing surfaces.

7. A method according to claim 1, wherein said image focusing systems are arranged on a straight line.

8. A method according to claim 1, wherein said image focusing systems are arranged in a plane surface.

9. A method according to claim 1, wherein said image focusing systems are arranged in a curved surface.

10. A method according to claim 1, wherein said image focusing systems are arranged on a straight line at regular intervals.

2. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein correlating of the image points comprises detecting image points in respective image sensing areas which satisfy the relationship that the ratio of the distance between first and second image points of the same object point in respective first and second image sensing areas to the distance between said second and a third image point of the same object point in respective second and third sensing areas is equal to the ratio of the distance between first and second focusing system to the distance between said second and a third focusing system.

3. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an objection at least one image sensing surface by using at least three image focusing systems so that at least three object images of the object are formed on said image sensing surface;

determining correspondence between image points of said images formed on said image sensing surface and object points on said object by using a geometrical relationship in which a shape is formed by joining said at least three image focusing systems, which shape moves toward said image sensing surface in parallel while maintaining a similar form so as to form a similar shape disposed between said shape and said image sensing surface, each of the image points corresponding to the object points existing on points of intersection of said image sensing surface and straight lines passing through vertices of said shape and the corresponding vertices of said similar shape; and

obtaining three-dimensional distance information of said object points by using information relating to the positions of at least two of said image points formed on said image sensing surface corresponding to a certain object point and information relating to the geometrical positional relationship between said two corresponding image points and said image focusing systems.

25. A method according to claim 3, wherein when said at least three image focusing systems are arranged on a line and said line is parallel with said image sensing surface, and wherein determination of said correspondency is made by using the geometrical relationship that a ratio between distances of adjacent image focusing systems is the same as the ratio of distances of image points corresponding to the certain object point.

26. A method according to claim 3, wherein said at least three image focusing systems are arranged on a common plane and said plane is perpendicular to said image sensing surface.

27. A method according to claim 3, wherein said at least three image focusing systems are arranged on a common plane parallel with said image sensing surface so that all of the image points corresponding to a certain object point exist on vertices of a figure on said image sensing surface, said figure being similar to said shape formed on said plane.

11. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein said image focusing systems are arranged in a plane surface so as to form a regular polygon.

12. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein said image focusing systems are arranged on a straight line at random.

13. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein said image focusing systems are arranged in a plane surface so as to form a ordinary polygon.

14. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein said image focusing systems are arranged in a plane surface parallel to said image sensing surfaces.

15. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between aaid image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein said image focusing systems are arranged in a plane surface which is oblique with respect to said image sensing surfaces.

16. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein a plurality of operations corresponding to a range of vertical azimuth angles for correlating points are performed in parallel.

17. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points ff an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein a plurality of operations corresponding to a range of horizontal azimuth angles and a range of vertical azimuth angles for correlating points are performed in parallel.

18. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein corresponding points are determined in a limited range of vertical azimuth angles and in a limited range of horizontal azimuth angles.

19. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein a plurality of limited ranges of vertical azimuth angles and a plurality of limited ranges of horizontal azimuth angles are specified in a desired order, and corresponding points are obtained in each specified, limited range of vertical azimuth angles and in each specified, limited range of horizontal azimuth angles.

20. A method of obtaining three-dimensional distance information, comprising the steps of:

forming images of an object system on an image sensing surface by using at least three image focusing systems so that at least three object images of the object system are formed on respective image sensing areas;

correlating the image points of an object point which are formed on said respective image sensing areas by said image focusing systems by using a relationship which exists between the geometrical positional relationships among said image points and the geometrical positional relationships among said image focusing systems themselves and between said image focusing systems and said image sensing areas; and

obtaining the three-dimensional distance information of said object point by using information relating to the positions of at least two of the corresponding image points on the image sensing areas and information relating to the geometrical positional relationship between said two corresponding image points and the image focusing systems, wherein image points corresponding to a non-existing object point are removed on the basis of brightness information obtained on said image sensing areas.

21. An apparatus for obtaining three-dimensional distance information, comprising:

three color filtering means corresponding to three primary colors and spaced apart from one another, for dividing light from an object system into three light means;

an image focusing system for forming the image of an object system by using said light means of said object system received from said color filtering means;

image sensing means formed of a color image sensor, said color image sensor being provided with three kinds of color filters corresponding to said three color filtering means in such a manner that the three color filters are repeatedly mounted on pixels which form said color image sensor, in a horizontal direction, to detect said three light beams separately, said pixels being scanned in a horizontal direction;

optical deflection means for making the optical axes of said light beams from said three color filtering means parallel to the axis of said image focusing system, for juxtaposing said optical axes of said 1ight beams in a direction parallel to a horizontal pixel layer of said color image sensor so that said optical axes deviate from one another, and for providing the same optical path length between each of said three color filtering means and the image sensing surface of said image sensing means;

horizontal addressing means for taking out information from pixels included in a horizontal pixel layer of said color image sensor in such a manner that a plurality of pixel groups each including three pixels which are provided with said three color filters, are successively addressed in a predetermined order, and the ouputs of said three pixels of each pixel group are simultaneously taken out;

detection means for delivering ON-information when said three light beams from said three color filtering means have a brightness value within a predetermined range, at those three pixels of a horizontal pixel layer which are addressed by said horizontal addressing means, and for delivering OFF-information when at least one of said three light beams does not have a brightness value within said predetermined range at one of said three pixels;

two-dimensional distance information calculation means for obtaining two-dimensional distance information which includes the distance from said image sensing surface to said object system in the direction of depth and at least one horizontal azimuth angle, from the horizontal address information on said color image sensor; and

three-dimensional distance information calculation means for obtaining three-dimensional distance information which includes said two-dimensional distance information of said object system and a vertical azimuth angle, by using the vertical address information on said color image sensor.

22. An apparatus according to claim 21, further comprising a brightness correction system for effecting brightness correction for outputs of three pixels which are included in one pixel group and provided with said three color filters.

23. An apparatus according to claim 21, wherein a contour extracting element is disposed in front of said image sensing surface.

24. An apparatus according to claim 21, wherein three polarizers and three analyzers for detecting said three light beams separately, perform substantially the same function as said three color filtering means and said three color filters, respectively.

28. An apparatus for obtaining three-dimensional distance information, comprising:

three color filtering means corresponding to three primary colors and spaced apart from one another, for dividing light from an object system into three light means;

an image focusing system for forming the image of an object system by using said light means of said object system received from said color filtering means;

image sensing means formed of a color image sensor, said color image sensor being provided with three kinds of color filters corresponding to said three color filtering means in such a manner that the three color filters are repeatedly mounted on pixels which form said color image sensor, in a horizontal direction, to detect said three light beams separately, said pixels being scanned nn a horizontal direction;

optical deflection means for making the optical axes of said light beams from said three color filtering means parallel to the axis of said image focusing system, for juxtaposing said optical axes of said light beams in a direction parallel to a horizontal pixel layer of said color image sensor so that said optical axes deviate from one another, and for providing the same optical path length between each of said three color filtering means and the image sensing surface of said image sensing means;

horizontal addressing means for taking out information from pixels included in a horizontal pixel layer of said color image sensor in such a manner that a plurality of pixel groups each including three pixels which are provided with said three color filters, are successively addressed in a predetermined order, and the outputs of said three pixels of each pixel group are simultaneously taken out;

detection means for delivering ON-information when said three light beams from said three color filtering means have a brightness value within a predetermined range, at those three pixels of a horizontal pixel layer which are addressed by said horizontal addressing means, and for delivering OFF-information when at least one of said three light beams does not have a brightness value within said predetermined range at one of said three pixels;

two-dimensional distance information calculation means for obtaining two-dimensional distance information which includes two horizontal azimuth angles, from the horizontal address information on said color image sensor; and

three-dimensional distance information calculation means for obtaining three-dimensional distance information which includes said two-dimensional distance information of said object system and a vertical azimuth angle, by using the vertical address information on said color image sensor.