Method and apparatus for measuring position and orientation of an object based on a sequence of projected points

• US 5,446,798 A
• Filed: 02/24/1994
• Issued: 08/29/1995
• Est. Priority Date: 06/20/1989
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Kawakami, Susumu, Morita, Toshihiko
• Primary Examiner(s): Moore, David K.
• Assistant Examiner(s): Johns, Andrew W.
• Application Number:
  US08/201,082
• Time in Patent Office: 551 Days
• Field of Search: 382/1, 382/44, 382/45, 356/12, 356/20, 356/21, 356/22, 250/206.1, 250/206.2, 250/206.3, 348/135, 348/142
• US Class Current: 382/154
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

A method for measuring a position and an orientation of an object. A sequence of movement points which line up on an intersection line of a predetermined projection surface and a plane which contains a predetermined projection center corresponding to a position of a camera, is obtained by obtaining projected points respectively generated for a plurality of positions of the camera, onto a point on the predetermined projection surface where the camera position is deemed as the projection center. Then, a sequence of reference points which line up on an intersection line of a plane containing the center of projection and the predetermined projection surface, shares a point with the sequence of movement points, and has the same cross ratio (inharmonic ratio) as a cross ratio of the sequence of movement points, is generated. An intersection point of intersection lines which are respectively intersection lines of the predetermined projection surface and planes each containing the center of projection and a pair of corresponding points one of which belongs to the sequence of movement points, and the other of which belongs to the sequence of reference points, is obtained. Then, geometrical information on the image element of the object when using the projection center in the three-dimensional space as a reference position is obtained based on the position of the above intersection point.

Claims

1. A method for obtaining geometrical information on an object, comprising the steps of:

a) moving a camera in a predetermined direction with regard to an object subject to measurement;

b) taking a plurality of images of the object by the camera at a plurality of camera positions;

c) extracting a plurality of contour images respectively from the plurality of images taken in step b) by the camera, where each of the plurality of contour images contains at least one image element of the object;

d) projecting at least one image element in each of the plurality of contour images of the object onto a projected point on a predetermined projection surface, for each of the at least one image element, from a predetermined projection center corresponding to a camera center, to obtain, for each of the at least one image element, a plurality of projected points on a predetermined projection surface as a sequence of movement points (x^t, t=0, 1, 2, . . .) which line up on a first intersection line at which the predetermined projection surface intersects with a first plane containing the predetermined projection center, where the predetermined projection surface is one of a plane and a surface of the second order;

e) generating as a sequence of reference points (τ

^t, t=0, 1, 2, . . .), a plurality of points which line up on a second intersection line of the predetermined projection surface and a second plane containing the projection center, so that the sequence of reference points (τ

^t, t=0, 1, 2, . . .) and the sequence of movement points (x^t, t=0, 1, 2, . . .) share a point, and the sequence of reference points (τ

^t, t=0, 1, 2, . . .) is not identical with the sequence of movement points (x^t, t=0, 1, 2, . . .), and has the same cross ratio (inharmonic ratio) as a cross ratio of the sequence of movement points (x^t, t=0, 1, 2, . . .);

f) obtaining a first intersection point of a plurality of third intersection lines at which a plurality of third planes respectively intersect the predetermined projection surface, the plurality of third planes each containing the projection center, one movement point in the sequence of movement points (x^t, t=0, 1, 2, . . .), and one reference point, corresponding to the movement point, in the sequence of reference points (τ

^t, t=0, 1, 2, . . .); and

g) obtaining geometrical information on the object with respect to the camera position, based on the position of the first intersection point to achieve environment recognition.

2. A method according to claim 1, wherein, in said step c) the sequence of movement points (x^t, t=0, 1, 2, . . .) is obtained as a plurality of intersection points at which a plurality of straight lines intersects the predetermined projection surface, and the straight lines passing through the projection center and being parallel to respective directions of shooting by the camera at the plurality of camera positions.

3. A method according to claim 1, wherein, in said step e) the geometrical information on the object comprises a distance from an initial position of the camera to a point, corresponding to the image element of the object.

4. A method according to claim 1, wherein, in said step d), the sequence of movement points (x^t, t=0, 1, 2, . . .) and the sequence of reference points (τ

^t, t=0, 1, 2, . . .) share a common point as their beginning points (x⁰ and τ

⁰).

5. A method according to claim 4, wherein the plurality of positions of the camera line up in the predetermined direction with arbitrary pitches, and

wherein said step d) for generating the sequence of reference points (τ

^t, t=0, 1, 2, . . .) includes the substeps of;

(i) generating an auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitches as the arbitrary pitches on a first straight line (lT) passing through an arbitrary position on a second straight line passing through the projection center and the beginning point (x⁰ ) of the sequence of movement points (x^t, t=0, 1, 2, . . .), where the beginning point (T⁰) of the auxiliary sequence of reference points (T^t, t=0, 1, 2, . . .) is the arbitrary point; and

(ii) projecting each point of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

6. A method according to claim 5, wherein the first straight line (lT) passing through the arbitrary position on the second straight line passing through the projection center and the beginning point (x⁰ ) of the sequence of movement points (x^t, t=0, 1, 2, . . .), is perpendicular to the second straight line passing through the projection center and the beginning point (τ

⁰) of the sequence of reference points (τ

^t, t=0, 1, 2, . . .).

7. A method according to claim 5, further comprising the steps of:

g) defining a first point at infinity (x∞

) of the sequence of movement points as an intersection point of the predetermined projection surface and a first straight line passing through the projection center and being parallel to a direction of shooting the object by the camera when the camera is moved to infinity in the predetermined direction; and

h) defining a second point at infinity (τ

∞

) of the sequence of reference points (τ

^t, t=0, 1, 2, . . .) as an intersection point of the predetermined projection surface and a second straight line passing through the projection center and being parallel to the straight line (lT) passing through the arbitrary position on the other straight line passing through the projection center and the beginning point (x⁰) of the sequence of movement points (x^t, t=0, 1, 2, . . .);

wherein said step f) comprises obtaining a distance r from an initial position of the camera to a point, corresponding to the image element of the object by

space="preserve" listing-type="equation">r=R.sup.0 ·

|sin(γ

-α

)|/|sinα

.vertline.

where R⁰ denotes a distance from the projection center to the beginning point (T⁰) of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .), γ

denotes an angle between a direction to the first point at infinity (x∞

) of the sequence of movement points and a direction to the second point at infinity (τ

∞

) of the sequence of reference points (τ

^t, t=0, 1, 2, . . .) at the projection center, and α

denotes an angle between a direction to the first intersection point and a direction to the point at infinity (τ

∞

) of the sequence of reference points (τ

^t, t=0, 1, 2, . . .) at the projection center.

8. A method according to claim 4, wherein the plurality of positions of the camera line up in the predetermined direction with an equal pitch (Δ

X); and

wherein the sequence of reference points (τ

^t, t=0, 1, 2, . . .) being generated by;

(i) generating an auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitch as the equal pitch (Δ

X) on a first straight line (lT) passing through an arbitrary position on a second straight line passing through the projection center and the beginning point (x⁰ ) of the sequence of movement points (x^t, t=0, 1, 2, . . .), where the beginning point (T⁰) of the auxiliary sequence of reference points (T^t, t=0, 1, 2, . . .) is the arbitrary point; and

(ii) projecting each point of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

9. A method according to claim 1, further comprising the steps of:

g) defining a first point at infinity (x∞

) of the sequence of movement points as an intersection point of the predetermined projection surface and a straight line passing through the projection center and being parallel to a direction of shooting the object by the camera when the camera is moved to infinity in the predetermined direction; and

h) defining a second point at infinity (τ

∞

) of the sequence of reference points (τ

^t, t=0, 1, 2, . . .) which corresponds to the first point at infinity (x∞

) of the sequence of movement points, and which is the same point as the first point at infinity (x∞

) of the sequence of movement points.

10. A method according to claim 9, wherein the plurality of positions of the camera line up in the predetermined direction with arbitrary pitches, and

wherein said step d for generating the sequence of reference points (τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

(i) generating an auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitches as the arbitrary pitches on an arbitrary straight line (lT) parallel to another straight line passing through the projection center and the first point at infinity (x∞

) of the sequence of movement points, where the beginning point (T⁰) of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .) is the arbitrary position, and a step for projecting each point of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

11. A method according to claim 10, wherein the arbitrary position (T⁰) is determined so that a straight line passing through the arbitrary position (T⁰) and the projection center is perpendicular to the parallel arbitrary straight line (lT).

12. A method according to claim 10, wherein, in said step f), a distance r from an initial position of the camera to a point corresponding to the image element of said object is obtained by

13. A method according to claim 9, wherein said plurality of positions of the camera line up in the predetermined direction with an equal pitch (Δ

X), and

wherein said step d) for generating said sequence of reference points (τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

(i) generating an auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitch as the equal pitch (Δ

X) on an arbitrary straight line (lT) parallel to another straight line passing through the projection center and the first point at infinity (x∞

) of the sequence of movement points, where the beginning point (T⁰) is the arbitrary position; and

ii) projecting each point of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

14. A method according to claim 1, wherein the predetermined projection surface onto which the at least one image element is projected in the projection step is a plane.

15. A method according to claim 1, wherein the predetermined projection surface onto which the at least one image is projected in the projection step is a sphere, and the projection center is the center of the sphere.

16. A method according to claim 1, wherein the predetermined projection surface onto which the at least one image is projected in the projection step is a circular cylinder, and the projection center is a point on an axis of the circular cylinder.

17. A method for obtaining geometrical information on an object, comprising the steps of:

a) moving a camera in a predetermined direction with regard to an object subject to measurement; and

b) taking a plurality of images of the object by the camera at a plurality of camera positions, respectively, by moving a camera in a predetermined direction;

c) extracting a plurality of contour images respectively from the plurality of images taken in said step b) by the camera, where each of the plurality of contour images contains at least one image element of the object;

d) projecting the at least one image element in each of the plurality of contour images of the object onto a projected point on a predetermined projection surface, for each of the at least one image element, from a predetermined projection center corresponding to a camera center, to obtain, for each of the at least one image element, a plurality of projected points on a predetermined projection surface as a sequence of movement points (x^t, t=0, 1, 2, . . .) which line up on a first intersection line at which the predetermined projection surface intersects with a first plane containing the predetermined projection center, and where the predetermined projection surface is one of a plane and a surface of the second order;

e) generating as a first sequence of reference points (τ

^t, t=0, 1, 2, . . .), a plurality of points which line up on a second intersection line of the predetermined projection surface and a second plane containing the projection center, so that the first sequence of reference points (τ

^t, t=0, 1, 2, . . .) and the sequence of movement points (x^t, t=0, 1, 2, . . .) share a point, and the sequence of reference points (τ

^t, t=0, 1, 2, . . .) is not identical with the sequence of movement points (x^t, t=0, 1, 2, . . .), and has the same cross ratio (inharmonic ratio) as a cross ratio of the sequence of movement points (x^t, t=0, 1, 2, . . .);

f) generating as a second sequence of reference points (-τ

^t, t=0, 1, 2, . . .), a plurality of points which line up on the second intersection line, so that the second sequence of reference points (-τ

^t, t=0, 1, 2, . . .) and the first sequence of reference points (τ

^t, t=0, 1, 2, . . .) share a beginning point (τ

⁰), and the second sequence of reference points (-τ

^t) has the same cross ratio (inharmonic ratio) as the cross ratio of the first sequence of reference points (τ

^t, t=0, 1, 2, . . .);

g) obtaining a first intersection point of a plurality of third intersection lines at which a plurality of third planes respectively intersects the predetermined projection surface, the plurality of third planes each containing the projection center, one movement point in the sequence of movement points (x^t, t=0, 1, 2, . . .) and one reference point, corresponding to the movement point in the first sequence of reference points (τ

^t, t=0, 1, 2, . . .); and

h) obtaining a second intersection point of a plurality of fourth intersection lines at which a plurality of fourth planes respectively intersects the predetermined projection surface, the plurality of fourth planes each containing the projection center, one movement point in the sequence of movement points (x^t, t=0, 1, 2, . . .) and one reference point, corresponding to the movement point in said second sequence of reference points (-τ

^t, t=0, 1, 2, . . .); and

i) obtaining geometrical information on the object with respect to the camera position, based on the positions of the first and second intersection points to achieve environment recognition.

18. A method for measuring a position and an orientation of an object according to claim 17, wherein, in said step c), said sequence of movement points (x^t, t=0, 1, 2, . . .) is obtained by obtaining intersection points of the predetermined projection surface and straight lines passing through the projection center and being parallel to respective directions of shooting by the camera at the plurality of positions of the camera corresponding to the movement of the camera in the predetermined direction.

19. A method according to claim 17, wherein, in said step f) said geometrical information on the object comprises a distance from an initial position of the camera to a point corresponding to the image element of the object.

20. A method according to claim 17, wherein, in said step d), said sequence of movement points (x^t, t=0, 1, 2, . . .) and the sequence of reference points (τ

^t, t=0, 1, 2, . . .) share a common point as their beginning points (x⁰ and τ

⁰).

21. A method according to claim 20, wherein the plurality of positions of the camera line up in the predetermined direction with arbitrary pitches,

wherein said first reference-point-sequence generating step d) for generating the first sequence of reference points (τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

(i) generating a first auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitches as the arbitrary pitches on a straight line (lT) passing through an arbitrary position on another straight line passing through the projection center and the beginning point (x⁰) of the sequence of movement points (x^t, t=0, 1, 2, . . .), where the beginning point (T⁰) of the first auxiliary sequence of reference points (T^t, t=0, 1, 2, . . .) is the arbitrary point; and

(ii) projecting each point of the first auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center, and

wherein said second reference-point-sequence generating step e) for generating a second sequence of reference points (-τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

(i) generating a second auxiliary sequence of points (-T^t, t=0, 1, 2, . . .) which line up in the direction opposite to the first sequence of reference points (τ

^t, t=0, 1, 2, . . .) with the same pitches as the arbitrary pitches on the straight line (lT) passing through the arbitrary position on the other straight line, where the beginning point (T⁰) of the second auxiliary sequence of reference points (-T^t, t=0, 1, 2, . . .) is the arbitrary point; and

(ii) projecting each point of the second auxiliary sequence of points (-T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

22. A method according to claim 21, wherein the straight line (lT) passing through an arbitrary position on the other straight line passing through the projection center and the beginning point (x⁰) of the sequence of movement points (x^t, t=0, 1, 2, . . .) is perpendicular to the straight line passing through the projection center and the beginning point (x⁰) of the sequence of movement points (x^t, t=0, 1, 2, . . .).

23. A method according to claim 21, further comprising the steps of:

i) defining a first point at infinity (x∞

) of the sequence of movement points as an intersection point of the predetermined projection surface and a first straight line passing through the projection center and being parallel to a direction of shooting the object by the camera when the camera is moved to infinity in the predetermined direction;

j) defining a second point at infinity (τ

.sup.∞

) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) is defined as an intersection point of the predetermined projection surface and a straight line passing through the projection center and being parallel to the straight line (lT) passing through the arbitrary position on a second straight line passing through the projection center and the beginning point (x⁰) of the sequence of movement points (x^t, t=0, 1, 2, . . .); and

k) determining a third reference point on a fifth intersection line of the predetermined projection surface and a fifth plane containing the projection center and the first and second intersection points so that an angle γ

between a direction to the third reference point and the direction to the second point at infinity (τ

.sup.∞

) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center on the predetermined projection surface, is defined as

space="preserve" listing-type="equation">γ

-cot.sup.-1 ((cotα

+cotβ

/2)

where α

denotes an angle between a direction to the first intersection point and a direction to the second point at infinity (τ

.sup.∞

) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center, β

denotes an angle between a direction to the second intersection point and the direction to the second point at infinity (τ

.sup.∞

) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center;

in said geometrical information obtaining step h), a distance r from an initial position of the camera to a point, corresponding to the image element, of the object is obtained by

space="preserve" listing-type="equation">r-R.sup.0 ·

|sin(γ

-α

|/|sinα

.vertline.,

where R⁰ denotes a distance from the projection center to the beginning point (T⁰) of the first auxiliary sequence of points (T^t, t=0, 1, 2, . . .).

24. A method according to claim 23, further comprising the steps of:

l) selecting a point of the contour image which is extracted from the image taken in the initial camera position of the plurality of camera positions, and

wherein in said projection step c), the selected point of contour is regarded as the image element to be projected onto the predetermined projection surface; and

m) obtaining a distance to a point on the object corresponding to the point of the contour image selected in the point selecting step in said geometrical information obtaining step f), assuming that a direction of the point at infinity of the sequence of movement points is equal to a known direction (v) of the movement of the camera.

25. A method according to claim 24, further comprising the steps of:

n) determining a direction to the third reference point by executing said geometrical information obtaining step h), wherein the image element is the selected point of the contour image; and

o) comparing the direction of the third reference point with the direction of the movement of the camera which is known in advance.

26. A method according to claim 23, further comprising the steps of:

l) selecting a plurality of points of the contour image which is extracted from the image taken in the initial camera position of the plurality of camera positions; and

m) determining the direction to the third reference point for each of the plurality of points of the contour image, to obtain a plurality of third reference points, by executing said step f), where each of the plurality of selected points of contour is regarded as the image element; and

n) obtaining the direction of the movement of the camera by obtaining an average of the determined directions to the plurality of third reference points.

27. A method according to claim 23, further comprising before execution of said first reference-point-sequence generating step d):

o) extracting a contour line segment (l) from each contour image extracted in said contour image extracting step b); and

wherein in said projection step c), an intersection point of the predetermined projection surface and a normal line, passing through the projection center of a sixth plane containing the projection center and the contour line segment (l), is obtained as a projected point (s) of each contour line segment for each of a plurality of images, said projection step c) comprising the substep of;

(i) selecting a beginning point (s⁰) of a sequence of movement points (s^t, t=0, 1, 2, . . . , t is a frame number), where the projected points (s) which are obtained from the plurality of images, are regarded as the points of the sequence of movement points (x^t, t=0, 1, 2, . . .);

wherein in said geometrical information obtaining step f) the beginning point (s⁰) of the sequence of movement points is made to be the same as the beginning point of the first and second sequence of reference points;

wherein said geometrical information obtaining step f) comprising the substeps of;

i) obtaining a third intersection point (Ss) of the predetermined projection surface and a normal line, passing through the projection center, of a seventh plane containing a trace of the sequence of movement points (s^t, t=0, 1, 2, . . .); and

ii) obtaining an orientation (_sh⁰) of a perpendicular line which is dropped from the initial position of the camera to a portion of the object, by obtaining an exterior product of a first vector being directed to the beginning point (s⁰), and a second vector being directed to the third intersection point (Ss) of the predetermined projection surface and the normal line, passing through the projection center of the seventh plane containing the trace of the sequence of movement points.

28. A method according to claim 23, further comprising before execution of said geometrical information obtaining step f) the steps of:

i) extracting a contour line segment (l) from each of the contour image extracted in said contour image extracting step b); and

wherein in said projection step c), an intersection point of the predetermined projection surface and a normal line, passing through the projection center of a sixth plane containing the projection center and the contour line segment (l), is obtained as a projected point (s) of each contour line segment for each of a plurality of images, where the projected points (s) obtained from the plurality of images are regarded as the points of the sequence of movement points, said projection step c) comprising the substep of;

i) selecting a beginning point (s⁰) of a sequence of movement points (s^t, t=0, 1, 2, . . . , t is a frame number), where the projected points (s) which are obtained from the plurality of images are regarded as the points of the sequence of movement points (x^t, t=0, 1, 2, . . .);

wherein in said geometrical information obtaining step h) said beginning point (s⁰) of the sequence of movement points is made to be the same as the beginning point of the first and second sequence of reference points, said geometrical information obtaining step h) comprising the substeps of;

(i) obtaining a third intersection point (Ss) of the predetermined projection surface and a normal line passing through the projection center of a seventh plane containing a trace of said sequence of movement points (s^t, t=0, 1, 2, . . .); and

(ii) obtaining a distance h from the initial camera position to a foot of a perpendicular line which is dropped to a portion of the object corresponding to the contour line segment which is subject to the measurement, by equations h=r·

sin π

, π

=cos^-1 (v·

Ss) using the distance r, where v denotes a first vector being directed in a known direction of the movement of the camera, Ss denotes a second vector being directed to the third intersection point (Ss) and (v·

Ss) denotes an inner product of the first and second vectors.

29. A method according to claim 28, further comprising a step for confirming that a relative angle between the direction of said third reference point (c) and the direction of the movement of the camera, which is known in advance, is equal to 90 degrees.

30. A method according to claim 23, further comprising the step of:

l) selecting a plurality of contour line segments which are subject to the measurement from the image which is obtained in an initial camera position;

wherein in said projection step c), the plurality of selected contour line segments are each regarded as one of the at least one image element to be projected onto the predetermined projection surface, and the sequence of movement points is obtained for each of the plurality of selected contour line segments to produce a plurality of sequences of movement points; and

wherein said geometrical information obtaining step h) comprises;

(i) obtaining a third intersection point (Ss) of the predetermined projection surface and a normal line passing through the projection center of a sixth plane containing a trace of each of the plurality of sequences of movement points (s^t, t=0, 1, 2, . . .) to produce a plurality of third intersection points (Ss); and

(ii) determining a direction of the movement of the camera as a direction of a fourth intersection point of a plurality of sixth intersection lines at which the predetermined projection surface intersect with a plurality of seventh planes respectively perpendicular to directions to the determined plurality of reference points (c).

31. A method according to claim 21, further comprising the steps of:

i) defining a first point at infinity (x.sup.∞

) of the sequence of movement points as an intersection point of the predetermined projection surface and a first straight line passing through the projection center and being parallel to a direction of shooting the object by the camera when the camera is moved to infinity in the predetermined direction;

j) defining a second point at infinity (τ

.sup.∞

) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) as an intersection point of the predetermined projection surface and a straight line passing through the projection center and being parallel to the straight line (lT) passing through the arbitrary position on a second straight line passing through the projection center and the beginning point (x⁰) of the sequence of movement points (x^t, t=0, 1, 2, . . .); and

k) determining a third reference point on a fifth intersection line of the predetermined projection surface and a fifth plane containing the projection center and the first and second intersection points so that an angle γ

between a direction to the third reference point and the direction to the second point at infinity (τ

.sup.∞

) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center on the predetermined projection surface, is defined as

space="preserve" listing-type="equation">γ

=cot.sup.-1 ((cotα

+cotβ

)/2)

where α

denotes an angle between a direction to the first intersection point and a direction to the second point at infinity (τ

.sup.∞

) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center, β

denotes an angle between a direction to the second intersection point and the direction to the second point at infinity (τ

^t) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center; and

wherein

in said step h), a distance r from an initial position of the camera to a point, corresponding to the image element of the object is obtained by

space="preserve" listing-type="equation">r=R.sup.0 ·

|sin(γ

-α

|/|sinα

.vertline.,

where R⁰ denotes a distance from the projection center to the beginning point (T⁰) of the first auxiliary sequence of points (T^t, t=0, 1, 2, . . .).

32. A method according to claim 20, wherein the plurality of positions of the camera line up in the predetermined direction with an equal pitch (Δ

X),

wherein said first reference-point-sequence generating step d) for generating the first sequence of reference points (τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

i) generating a first auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitch as the equal pitch (Δ

X) on a straight line (lT) passing through an arbitrary position on another straight line passing through the projection center and the beginning point (x⁰) of the sequence of movement points (x^t, t=0, 1, 2, . . .), where the beginning point (T⁰) of the first auxiliary sequence of reference points (T^t, t=0, 1, 2, . . .) is the arbitrary point; and

(ii) projecting each point of the first auxiliary sequence of points (T^t t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center; and

wherein said second reference-point-sequence generating step e) for generating the second sequence of reference points (-τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

(i) generating a second auxiliary sequence of points (-T^t, t=0, 1, 2, . . .) which line up in the direction opposite to the first sequence of reference points (τ

^t, t=0, 1, 2, . . .) with the same pitch as the equal pitch (Δ

X) on the straight line (lT) passing through the arbitrary position on the other straight line, where the beginning point (T⁰) of the second auxiliary sequence of reference points (-T^t, t=0, 1, 2, . . .) is the arbitrary point; and

(ii) projecting each point of the second auxiliary sequence of points (-T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

33. A method according to claim 17, further comprising the steps of:

i) defining a first point at infinity (x.sup.∞

) of the sequence of movement points as an intersection point of the predetermined projection surface and a first straight line passing through the projection center and being parallel to a direction of shooting the object by the camera when the camera is moved to infinity in the predetermined direction; and

j) defining a second point at infinity (τ

.sup.∞

) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) corresponding to the point at infinity (x.sup.∞

) of the sequence of movement points, is the same point as the point at infinity (x.sup.∞

) of the sequence of movement points.

34. A method according to claim 33, wherein the plurality of positions of the camera line up in the predetermined direction with arbitrary pitches,

wherein said first reference-point-sequence generating step d) for generating the first sequence of reference points (τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

(i) generating a first auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitches as the arbitrary pitches on a second straight line (lT) parallel to a third straight line passing through the projection center and the first point at infinity (x.sup.∞

) of the sequence of movement points, where the beginning point (T⁰) of the first auxiliary sequence of reference points (T^t, t=0, 1, 2, . . .) is an arbitrary point on the second straight line (lT); and

(ii) projecting each point of the first auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center, and

wherein said second reference-point-sequence generating step e) for generating the second sequence of reference points (-τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

(i) generating a second auxiliary sequence of points (-T^t, t=0, 1, 2, . . .) which line up in the opposite direction with the same pitches as the arbitrary pitches on the second straight line (lT) parallel to the third straight line passing through the projection center and the first point at infinity (x.sup.∞

) of the sequence of movement points, where the beginning point (T⁰) of the second auxiliary sequence of reference points (-T^t, t=0, 1, 2 . . .) is the arbitrary point on the second straight line (lT); and

(ii) projecting each point of the second auxiliary sequence of points (-T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

35. A method according to claim 34, wherein the beginning point (T⁰) is determined so that the third straight line passing through the beginning point (T⁰) and the projection center, is perpendicular to the second straight line (lT).

36. A method according to claim 34, further comprising the step of:

i) determining a third reference point on a fifth intersection line of the predetermined projection surface and a fifth plane containing the projection center and the first and second intersection points so that an angle γ

between a direction to the beginning point (x⁰) of the sequence of reference points (x^t, t=0, 1, 2, . . .) and the direction to the beginning point (τ

⁰) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center on the predetermined projection surface is defined as

space="preserve" listing-type="equation">γ

=cot.sup.-1 ((cotα

+cotβ

)/2);

where α

denotes an angle between a direction to the first intersection point and a direction to the beginning point (τ

⁰) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center, β

denotes an angle between a direction to the second intersection point and the direction to the beginning point (τ

⁰) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center;

wherein in said geometrical information obtaining step h), a distance r from an initial position of the camera to a point corresponding to the image element of the object which is subject to the measurement is obtained by

space="preserve" listing-type="equation">r=R.sup.0 ·

|sinβ

|/|sin(γ

-β

).vertline.,

where R⁰ denotes a distance from the projection center to the beginning point (T⁰) of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .).

37. A method according to claim 36, wherein in said projection step c), more than one point of the contour image extracted in the contour extracting step being regarded as the image element, and being projected onto the predetermined projection surface;

and

wherein in said geometrical information obtaining step h), a distance to more than one point on the object corresponding to more than one point of the contour image is obtained assuming that a direction of the point at infinity of the sequence of movement points is equal to a known direction (v) of the movement of the camera.

38. A method according to claim 36, further comprising before execution of said step f) the steps of:

i) extracting a contour line segment (l) from each contour image extracted in the contour image extracting step;

wherein in said projection step c), an intersection point of the predetermined projection surface and a normal line, passing through the projection center, of a fifth plane containing the projection center and the contour line segment (l), is obtained as a projected point (s) of each contour line segment for each of a plurality of images, where the projected points (s) obtained from the plurality of images, are regarded as the points of the sequence of movement points and further comprising the substep of;

(i) selecting one movement point of the sequence of movement points as the first point at infinity (s.sup.∞

) of the sequence of movement points (s^t, t=0, 1, 2, . . . , t is a frame number), where the projected points (s) which are obtained from the plurality of images are regarded as the points of the sequence of movement points (x^t, t=0, 1, 2, . . .); and

wherein in said geometrical information obtaining step h) said first point at infinity (s.sup.∞

) of the sequence of movement points is made to be the same as the point at infinity of the first and second sequence of reference points, said geometrical information obtaining step comprising the substeps of;

(i) obtaining a third intersection point (Ss) of the predetermined projection surface and a normal line, passing through the projection center, of a sixth plane containing a trace of the sequence of movement points (s^t, t=0, 1, 2, . . .);

(ii) a distance obtaining step for obtaining a distance h from the initial camera position to a foot of a perpendicular line which is dropped to a portion of the object corresponding to the contour line segment which is subject to the measurement, by equations h=r·

sin π

, and π

=cos^-1 (v·

Ss) using the distance r, where v denotes a first vector being directed in a direction of the movement of the camera, Ss denotes a second vector being directed to the third intersection point (Ss), and (v·

Ss) denotes an inner product of the first and second vectors.

39. A method according to claim 36, further comprising before execution of said first reference-point-sequence generating step d) the substep of:

(i) extracting a contour line segment (l) [(l)] from each of the contour image extracted in the contour image extracting step; and

wherein in said projection step c), an intersection point of the predetermined projection surface and a normal line, passing through the projection center of a fifth plane containing the projection center and the contour line segment (l), is obtained as a projected point (s) of each contour line segment for each of a plurality of images, said projection step c) comprising the substep of;

(i) selecting one movement point of the sequence of movement points as the first point at infinity (s.sup.∞

) of the sequence of movement points (s^t, t=0, 1, 2, . . . , t is a frame number), where the projected points (s) which are obtained from the plurality of images, are regarded as the points of the sequence of movement points (x^t, t=0, 1, 2, . . .); and

wherein in said geometrical information obtaining step h) said first point at infinity (s.sup.∞

) of the sequence of movement points is made to be the same as the second point at infinity of the first and second sequence of reference points, said geometrical information obtaining step comprising the substeps of;

(i) obtaining a third intersection point (Ss) of the predetermined projection surface and a normal line, passing through the projection center of a sixth plane containing a trace of the sequence of movement points (s^t, t=0, 1, 2, . . .); and

(ii) an orientation obtaining step for obtaining an orientation (Sh⁰) of a perpendicular line which is dropped from the initial position of the camera to a portion of the object, by obtaining an exterior product of a first vector being directed to the third reference point (c=s⁰), and a second vector being directed to the third intersection point (Ss) of the predetermined projection surface and the normal line, passing through the projection center of the sixth plane containing the trace of the sequence of movement points.

40. A method according to claim 34, further comprising the step of:

i) determining a third reference point on a fifth intersection line of the predetermined projection surface and a fifth plane containing the projection center and the first and second intersection points so that an angle γ

between a direction to the beginning point (x⁰) of the sequence of reference points (x^t, t=0, 1, 2, . . .) and the direction to the beginning point (τ

⁰) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center on the predetermined projection surface is defined as

space="preserve" listing-type="equation">γ

=cot.sup.-1 ((cotα

+cotβ

)/2),

where α

denotes an angle between a direction to the first intersection point and a direction to the beginning point (τ

⁰) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center, and β

denotes an angle between a direction to the second intersection point and the direction to the beginning point (τ

⁰) of the first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at the projection center; and

wherein

in said step h), a distance r from an initial position of the camera to a point, corresponding to the image element of the object which is subject to the measurement is obtained by

space="preserve" listing-type="equation">r=R.sup.0 ·

|sinα

|/|sin(γ

-α

).vertline.,

where R⁰ denotes a distance from the projection center to the beginning point (T⁰) of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .).

41. A method according to claim 33, wherein the plurality of positions of the camera line up in the predetermined direction with an equal pitch (Δ

X);

wherein said first reference-point sequence generating step d) comprises the substeps of;

(i) generating a first auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitch as the equal pitch (Δ

X) on a second straight line (lT) parallel to a third straight line passing through the projection center and the first point at infinity (x.sup.∞

) of the sequence of movement points, where the beginning point (T⁰) of the second auxiliary sequence of reference points (T^t, t=0, 1, 2, . . .) is an arbitrary point on the second straight line (lT);

(ii) projecting each point of the first auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center; and

wherein said second reference-point sequence generating step e) includes a step for generating a second auxiliary sequence of points (-T^t, t=0, 1, 2, . . .) which line up in the opposite direction with the same pitch as the equal pitch (Δ

X) on the second straight line (lT) parallel to the third other straight line passing through the projection center and the first point at infinity (x.sup.∞

) of the sequence of movement points, where the beginning point (T⁰) of the first auxiliary sequence of reference points ((T^t, t=0, 1, 2, . . .) is the arbitrary point on the second straight line (lT); and

(iii) projecting each point of the second auxiliary sequence of points (-T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

42. A method according to claim 17, wherein the predetermined projection surface is a plane.

43. A method according to claim 17, wherein the predetermined projection surface is a sphere, and said projection center is the center of the sphere.

44. A method according to claim 17, wherein the predetermined projection surface is a circular cylinder, and the projection center is a point on an axis of the circular cylinder.

45. A method for obtaining geometrical information on an object, comprising the steps of:

a) moving a camera in a predetermined direction with regard to an object subject to measurement; and

b) taking a plurality of images of the object by the camera at a plurality of camera positions, respectively, by moving a camera in a predetermined direction;

c) extracting a plurality of contour images respectively from the plurality of images taken in said step b) by the camera, where each of the plurality of contour images contains at least one image element of the object;

d) projecting at least one image element in each of the plurality of contour images of the object onto a projected point on a predetermined projection surface, for each of the at least one image element, from a predetermined projection center corresponding to a camera center, to obtain, for each of the at least one image element, a plurality of projected points on a predetermined projection surface as a sequence of movement points (x^t, t=0, 1, 2, . . .) which line up on a first intersection line at which the predetermined projection surface intersects a first plane containing the predetermined projection center, where the predetermined projection surface is one of a plane and a surface of the second order;

e) obtaining, as a reference point for plane projection, a first intersection point of the predetermined projection surface and a straight line which passes through the projection center and is perpendicular to an arbitrary projection plane which contains the projection center;

f) obtaining a sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .), comprising the substeps of;

(i) obtaining a plurality of second planes each containing the projection center, the reference point for plane projection, and one movement point of the sequence of movement points (x^t, t=0, 1, 2, . . .);

(ii) obtaining a plurality of second intersection lines at which the plurality of second planes respectively intersect the predetermined projection surface; and

(iii) obtaining, as the sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .), a plurality of second intersection points at which the plurality of second intersection lines respectively intersect the predetermined projection surface;

g) generating, as a sequence of reference points (τ

^t, t=0, 1, 2, . . .), a plurality of points which line up on a third intersection line of the predetermined projection surface and a third plane containing the projection center, so that the sequence of reference points (τ

^t, t=0, 1, 2, . . .) and the sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .) share a point, and the sequence of reference points (τ

^t, t=0, 1, 2, . . .) is not identical with either of the sequence of movement points (x^t, t=0, 1, 2, . . .) and the sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .), and has the same cross ratio (inharmonic ratio) as a cross ratio of the sequence of projected movement points (x^t, t=0, 1, 2, . . .);

h) obtaining a third intersection point of a plurality of third intersection lines at which a plurality of fourth planes respectively intersects the predetermined projection surface, the plurality of fourth planes each containing the projection center, one projected movement point in the sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .) and one reference point corresponding to the projected movement point in the sequence of reference points (τ

^t, t=0, 1, 2, . . .); and

i) obtaining geometrical information which receives the third intersection point, on the object with respect to the camera position, based on the position of the third intersection point to achieve environment recognition.

46. A method according to claim 45, wherein, in said step c) the sequence of movement points (x^t, t=0, 1, 2, . . .) is obtained as a plurality of intersection points at which a plurality of straight lines intersects with the predetermined projection surface, and the straight lines passing through the projection center and being parallel to respective directions of shooting by the camera at the plurality of camera positions.

47. A method according to claim 45, wherein, in said step h) the geometrical information on the object comprises a distance from an initial position of the camera to a plane projected point, where the plane projected point is obtained by projecting a point, corresponding to the image element, of the object, onto the projection plane.

48. A method according to claim 45, wherein, in said step d), the sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .) and the sequence of reference points (τ

^t, t=0, 1, 2, . . .) share a common point as their beginning points (x'"'"'⁰ and τ

⁰).

49. A method according to claim 48, wherein the plurality of positions of the camera line up in the predetermined direction with arbitrary pitches, and

said step e) for generating the sequence of reference points (τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

(i) generating an auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitches as the arbitrary pitches on a first straight line (lT) passing through an arbitrary position on a second straight line passing through the projection center and the beginning point (x'"'"'⁰) of the sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .), where the beginning point (T⁰) of the auxiliary sequence of reference points (T^t, t=0, 1, 2, . . .) is the arbitrary point; and

(ii) projecting each point of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

50. A method according to claim 49, wherein the first straight line (lT) passing through the arbitrary position on the second straight line passing through the projection center and the beginning point (x'"'"'⁰) of the sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .) is perpendicular to the second straight line passing through the projection center and the beginning point (τ

⁰) of the sequence of reference points (τ

^t, t=0, 1, 2, . . .).

51. A method according to claim 49, further comprising the steps of:

i) defining a first point at infinity (x'"'"'.sup.∞

) of the sequence of projected movement points as an intersection point of the predetermined projection surface and a first straight line passing through the projection center and being parallel to a direction of shooting the object by the camera when the camera is moved to infinity in the predetermined direction; and

j) defining a second point at infinity (τ

.sup.∞

) of the sequence of reference points (τ

^t, t=0, 1, 2, . . .) as an intersection point of the predetermined projection surface and a second straight line passing through the projection center and being parallel to the straight line (lT) passing through the arbitrary position on the other straight line passing through the projection center and the beginning point (x'"'"'⁰) of the sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .);

and wherein in said step h), a distance r from an initial position of the camera to a plane projected point is obtained by

space="preserve" listing-type="equation">r=R.sup.0 ·

|sin(γ

-α

)|/|sinα

.vertline.,

where the plane projected point is obtained by projecting a point, corresponding to the image element of the object, onto the projection plane, R⁰ denotes a distance from the projection center to the beginning point (T⁰) of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .), γ

denotes an angle between a direction to the first point at infinity (x'"'"'.sup.∞

) of the sequence of projected movement points and a direction to the second point at infinity (τ

.sup.∞

) of the sequence of reference points (τ

^t, t=0, 1, 2, . . .) at the projection center, and α

denotes an angle between a direction to the first intersection point and a direction to the point at infinity (τ

.sup.∞

) of the sequence of reference points (τ

^t, t=0, 1, 2, . . .) at the projection center.

52. A method according to claim 48, wherein the plurality of positions of the camera line up in the predetermined direction with an equal pitch (Δ

X);

wherein the sequence of reference points (τ

^t, t=0, 1, 2, . . .) is generated comprising the steps of;

i) generating an auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitch as the equal pitch (Δ

X) on a first straight line (lT) passing through an arbitrary position on a second straight line passing through the projection center and the beginning point (x'"'"'⁰) of the sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .), where the beginning point (T⁰) of the auxiliary sequence of reference points (T^t, t=0, 1, 2, . . .) is the arbitrary point; and

j) projecting each point of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

53. A method according to claim 45, further comprising the steps of:

i) defining a first point at infinity (x'"'"'.sup.∞

) of the sequence of projected movement points as an intersection point of the predetermined projection surface and a straight line passing through the projection center and being parallel to a direction of shooting the object by the camera when the camera is moved to infinity in the predetermined direction; and

j) defining a second point at infinity (τ

.sup.∞

) of the sequence of reference points (τ

^t, t=0, 1, 2, . . .) which corresponds to the first point at infinity (x'"'"'.sup.∞

) of the sequence of projected movement points, and which is the same point as the first point at infinity (x'"'"'.sup.∞

) of the sequence of projected movement points.

54. A method according to claim 53, wherein the plurality of positions of the camera line up in the predetermined direction with arbitrary pitches, and

wherein said step f) for generating said sequence of reference points (τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

(i) generating an auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitches as the arbitrary pitches on an arbitrary straight line (lT) parallel to another straight line passing through the projection center and the first point at infinity (x'"'"'.sup.∞

) of the sequence of projected movement points, where the beginning point (T⁰) of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .) is the arbitrary position; and

(ii) projecting each point of the auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

55. A method according to claim 54, wherein the arbitrary position (T⁰) is determined so that a straight line passing through the arbitrary position (T⁰) and the projection center is perpendicular to the parallel arbitrary straight line (lT).

56. A method according to claim 54, wherein, in said step h), a distance r from an initial position of the camera to a plane projected point is obtained by

57. A method according to claim 56, further comprising the steps of:

i) selecting a point of the contour image which is extracted from the image taken in the initial camera position of the plurality of camera positions, and in said projection step c), the selected point of contour is regarded as the image element to be projected onto the predetermined projection surface; and

wherein in said geometrical information obtaining step c), a distance to a point on the object corresponding to the point of the contour image selected in the point selecting step is obtained determining a fourth reference point (w) for plane projection, and assuming that a direction of the point at infinity of the sequence of projected movement points is equal to a known direction (v) of the movement of the camera.

58. A method according to claim 56, further comprising the steps of:

i) extracting a contour line segment (l) from each of the contour images extracted in said step b); and

wherein in said projection step c), a fourth intersection point of the predetermined projection surface and a normal line, passing through the projection center of a sixth plane containing the projection center and the contour line segment (l), is obtained as a projected point (s) of each contour line segment for each of a plurality of images, where the projected points (s) obtained from the plurality of images are regarded as the points of the sequence of movement points;

wherein said step h) further comprises the substeps of;

(i) determining the fourth reference point (w) for plane projection in a direction perpendicular to a known direction of the movement of the camera regarding the projection center; and

(ii) obtaining the point at infinity of the sequence of projected movement points by an exterior product of a first vector directed in the known direction (v) of the camera and a second vector directed to the fourth reference point (w) for plane projection, the first point at infinity (s'"'"'.sup.∞

) of the sequence of projected movement points is regarded to be the same as the second point-at-infinity of the first and second sequence of reference points, and the distance from the initial camera position to a fifth intersection point at which a line segment, corresponding to the contour line segment on the object or a straight line containing the line segment, intersects with the predetermined projection plane in the three-dimensional space, is obtained by the distance r.

59. A method according to claim 58, further comprising, in said step h), after executing the operation of obtaining the point at infinity (s.sup.∞

) of a sequence of movement points (s^t, t=0, 1, 2, . . . , t is a frame number), the substep of;

i) rotating the beginning point of the sequence of projected movement points by 90 degrees around the direction of the fourth reference point (w) for plane projection, to obtain a direction from the initial camera position to the fifth intersection point of the line segment and the predetermined projection plane.

60. A method according to claim 59, further comprising the steps of:

i) executing the operations of said geometrical information obtaining step h) for a plurality of fourth reference points (w) for plane projection which are perpendicular to the known direction of the movement of the camera, to obtaining the distance r and direction to the line segment or the straight line containing the line segment on the object for each of the plurality of the fourth reference points (w) for plane projection; and

j) determining positions and orientations of the line segment on the object in the three-dimensional space based on the distances r and the directions of the intersection points of the line segment and the predetermined projection plane, where the intersection points are obtained in the plural projection plane step for the plurality of fourth reference points (w) for plane projection.

61. A method according to claim 53, wherein the plurality of positions of the camera line up in the predetermined direction with an equal pitch (Δ

X), and

wherein said step f) for generating the sequence of reference points (τ

^t, t=0, 1, 2, . . .) comprises the substeps of;

(i) generating an auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitch as the equal pitch (Δ

X) on an arbitrary straight line (lT) parallel to another straight line passing through the projection center and the first point at infinity (x'"'"'.sup.∞

) of the sequence of projected movement points, where the beginning point (T⁰) is the arbitrary position; and

(iii) projecting each point of the auxiliary sequence of points (Tt, t=0, 1, 2, . . .) onto the predetermined projection surface from the projection center.

62. A method according to claim 45, wherein the predetermined projection surface is a plane.

63. A method according to claim 45, wherein the predetermined projection surface is a sphere, and the projection center is the center of the sphere.

64. A method according to claim 45, wherein the predetermined plane or surface of the second class is a circular cylinder, and the projection center is a point on an axis of the circular cylinder.

65. An apparatus comprising:

a camera, for taking a plurality of images of an object which is subject to measurement, at a plurality of camera positions, respectively, by moving said camera in a predetermined direction; and

data processing means for processing image data which is taken by said camera, said data processing means comprising;

a contour image extracting circuit, operatively connected to said camera, for extracting a plurality of contour images respectively from said plurality of images taken by said camera, where each of said plurality of contour images contains at least one image element of said object;

a projection circuit, operatively connected to said contour image extracting circuit, for projecting said at least one image element in each of said plurality of contour images of said object onto a projected point on a predetermined projection surface, for each of said at least one image element, from a predetermined projection center corresponding to a camera center, to obtain, for each of said at least one image element, a plurality of projected points on a predetermined projection surface as a sequence of movement points (x^t, t=0, 1, 2, . . .) which line up on a first intersection line at which said predetermined projection surface intersects with a first plane containing said predetermined projection center, said predetermined projection surface is one of a plane and a surface of the second order;

a reference-point-sequence generating circuit for generating, as a sequence of reference points (τ

^t, t=0, 1, 2, . . .), a plurality of points which line up on a second intersection line of said predetermined projection surface and a second plane containing said projection center, so that said sequence of reference points (τ

^t, t=0, 1, 2, . . .) and said sequence of movement points (x^t, t=0, 1, 2, . . .) share a point, and said sequence of reference points (τ

^t, t=0, 1, 2, . . .) is not identical with said sequence of movement points (x^t, t=0, 1, 2, . . .), and has the same cross ratio (inharmonic ratio) as a cross ratio of said sequence of movement points (x^t, t=0, 1, 2, . . .);

an intersection-point obtaining circuit for obtaining a first intersection point of a plurality of third intersection lines at which a plurality of third planes respectively intersecting the predetermined projection surface, said plurality of third planes each containing said projection center, one movement point in said sequence of movement points (x^t, t=0, 1, 2, . . .), and one reference point, corresponding to the movement point, in said sequence of reference points (τ

^t, t=0, 1, 2, . . .); and

a geometrical information obtaining circuit, operatively connected to said intersection-point-obtaining circuit, for obtaining geometrical information on said object with respect to said camera position, based on the position of said first intersection point.

66. An apparatus comprising:

a camera, for taking a plurality of images of an object which is subject to the measurement, at a plurality of camera positions, respectively, by moving said camera in a predetermined direction; and

data processing means for processing image data which is taken by said camera, said data processing means comprising;

a contour image extracting circuit, operatively connected to said camera, for extracting a plurality of contour images respectively from said plurality of images taken by said camera, where each of said plurality of contour images contains at least one image element of said object;

projection means for projecting said at least one image element in each of said plurality of contour images of said object onto a projected point on a predetermined projection surface, for each of said at least one image element, from a predetermined projection center corresponding to a camera center, to obtain, for each of said at least one image element, a plurality of projected points on a predetermined projection surface as a sequence of movement points (x^t, t=0, 1, 2, . . .) which line up on a first intersection line at which said predetermined projection surface intersects with a first plane containing said predetermined projection center, where said predetermined projection surface is one of a plane and a surface of the second order;

a first reference-point-sequence generating circuit for generating, as a first sequence of reference points (τ

^t, t=0, 1, 2, . . .), a plurality of points which line up on a second intersection line of said predetermined projection surface and a second plane containing said projection center, so that said first sequence of reference points (τ

^t, t=0, 1, 2, . . .) and said sequence of movement points (x^t, t=0, 1, 2, . . .) share a point, and said sequence of reference points (τ

^t, t=0, 1, 2, . . .) is not identical to the sequence of movement points (x^t, t=0, 1, 2, . . .), and has the same cross ratio (inharmonic ratio) as a cross ratio of said sequence of movement points (x^t, t=0, 1, 2, . . .);

a second reference-point-sequence generating circuit for generating, as a second sequence of reference points (-τ

^t, t=0, 1, 2, . . .), a plurality of points which line up on said second intersection line, so that said second sequence of reference points (-τ

^t, t=0, 1, 2, . . .) and said first sequence of reference points (τ

^t, t=0, 1, 2, . . .) share a beginning point (τ

⁰), and said second sequence of reference points (-τ

^t, t=0, 1, 2, . . .) has the same cross ratio (inharmonic ratio) as the cross ratio of said first sequence of reference points (τ

^t, t=0, 1, 2, . . .);

a first intersection-point obtaining circuit for obtaining a first intersection point of a plurality of third intersection lines at which a plurality of third planes respectively intersecting said predetermined projection surface, said plurality of third planes each containing said projection center, one movement point in said sequence of movement points (x^t, t=0, 1, 2, . . .), and one reference point, corresponding to the movement point, in said first sequence of reference points (τ

^t, t=0, 1, 2, . . .);

a second intersection-point obtaining circuit for obtaining a second intersection point of a plurality of fourth intersection lines at which a plurality of fourth planes respectively intersecting the predetermined projection surface, said plurality of fourth planes each containing said projection center, one movement point in said sequence of movement points (x^t, t=0, 1, 2, . . .), and one reference point, corresponding to the movement point, in said second sequence of reference points (-τ

^t, t=0, 1, 2, . . .); and

a geometrical information obtaining circuit, operatively connected to said first and second intersection-point obtaining circuits, for obtaining geometrical information on the object with respect to said camera position, based on the positions of said first and second intersection points.

67. An apparatus according to claim 66, wherein in said first reference-point-sequence generating circuit, said sequence of movement (x^t, t=0, 1, 2, . . .) and said sequence of reference points (τ

^t, t=0, 1, 2, . . .) share a common point as their beginning points (x⁰ and τ

⁰).

68. An apparatus according to claim 67, wherein said plurality of camera positions line up in said predetermined direction with arbitrary pitches,

wherein said first reference-point-sequence generating circuit generates said sequence of reference points (τ

^t, t=0, 1, 2, . . .) by means for generating a first auxiliary sequence of points (T^t, t=0, 1, 2, . . .) which line up with the same pitches as said arbitrary pitches on a straight line (lT) passing through an arbitrary position on another straight line passing through said center of projection and said beginning point (x⁰) of the sequence of movement points (x^t, t=0, 1, 2, . . .), where the beginning point (T⁰) of said first auxiliary sequence of reference points (T^t, t=0, 1, 2, . . .) is said arbitrary point, and means for projecting each point of said first auxiliary sequence of points (T^t, t=0, 1, 2, . . .) onto said predetermined plane or surface of the second order from said center of projection, and

wherein said second reference-point-sequence generating circuit generates said second sequence of reference points (-τ

^t, t=0, 1, 2, . . .) by means for generating a second auxiliary sequence of points (-T^t, t=0, 1, 2, . . .) which line up in the direction opposite to said first auxiliary sequence of reference points (T^t, t=0, 1, 2, . . .) with the same pitches as said arbitrary pitches on said straight line (lT) passing through said arbitrary position on said other straight line, where the beginning point (T⁰) of said second auxiliary sequence of reference points (-T^t, t=0, 1, 2, . . .) is said arbitrary point, and means for projecting each point of said second auxiliary sequence of points (-T^t, t=0, 1, 2, . . .) onto said predetermined plane or surface of the second order from the center of projection.

69. An apparatus according to claim 68, further comprising:

first point-at-infinity defining means for defining a first point at infinity (x.sup.∞

) of said sequence of movement points as an intersection point of said predetermined projection surface and a first straight line passing through said projection center and being parallel to a direction of shooting said object by said camera when said camera is moved to infinity in said predetermined direction; and

second point-at-infinity defining means for defining a second point at infinity (τ

.sup.∞

) of said first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) as an intersection point of said predetermined projection surface and a straight line passing through said projection center and being parallel to said straight line (lT) passing through said arbitrary position on a second straight line passing through said projection center and said beginning point (x⁰) of the sequence of movement points (x^t, t=0, 1, 2, . . .); and

reference point determining means for determining a third reference point on a fifth intersection line of said predetermined projection surface and a fifth plane containing said projection center and said first and second intersection points so that an angle γ

between a direction to the third reference point and the direction to said second point at infinity (τ

.sup.∞

) of said first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at said projection center on said predetermined projection surface, is defined as

space="preserve" listing-type="equation">γ

=cot.sup.-1 ((cotα

+cotβ

)/2),

where α

denotes an angle between a direction to said first intersection point and a direction to said second point at infinity (τ

.sup.∞

) of said first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at said projection center, and β

denotes an angle between a direction to said second intersection point and the direction to said second point at infinity (τ

.sup.∞

) of said first and second sequences of reference points (τ

^t and -τ

^t, t=0, 1, 2, . . .) at said projection center;

wherein in said geometrical information obtaining circuit, a distance r from an initial camera position to a point, corresponding to the image element of said object is obtained by

space="preserve" listing-type="equation">r=R.sup.0 ·

|sin(γ

-α

)|/|sinα

.vertline.,

where R⁰ denotes a distance from said projection center to said beginning point (T⁰) of said first auxiliary sequence of points (T^t, t=0, 1, 2, . . .).

70. An apparatus according to claim 69, further comprising:

contour line segment extracting means for extracting a contour line segment (l) from each contour image extracted by the contour image extracting means; and

said projection means obtaining, by an intersection point of said predetermined projection surface and a normal line, passing through said projection center, of a sixth plane containing the projection center and the contour line segment (l), a projected point (s) of each contour line segment for each of a plurality of images, where said projected points (s) obtained from said plurality of images are regarded as the points of said sequence of movement points, said projection means comprising;

beginning point selecting means for selecting a beginning point (s⁰) of a sequence of movement points (s^t, t=0, 1, 2, . . . , t is a frame number), where said projected points (s) are obtained from said plurality of images and are regarded as the points of said sequence of movement points (x^t, t=0, 1, 2, . . .);

said geometrical information obtaining circuit obtaining said beginning point (s⁰) of the sequence of movement points made to be the same as said beginning point of said first and second sequence of reference points, said geometrical information obtaining means comprising;

an intersection point obtaining means for obtaining a third intersection point (Ss) of said predetermined projection surface and a normal line, passing through said projection center of a seventh plane containing a trace of said sequence of movement points (s^t, t=0, 1, 2, . . .); and

distance obtaining means for obtaining a distance h from the initial camera position to a foot of a perpendicular line which is dropped to a portion of the object corresponding to said contour line segment which is subject to the measurement, by equations h=r·

sin π

, and π

=cos^-1 (v·

Ss) using the distance r, where v denotes a first vector being directed in a known direction of the movement of said camera, Ss denotes a second vector being directed to said third intersection point (Ss), and (v·

Ss) denotes an inner product of the first and second vectors.

71. An apparatus according to claim 70, wherein

said camera comprises a moving device for moving said camera;

said apparatus further comprising;

a monitor device for displaying an image;

a superimposer for controlling the superimposition of an image of points of said contour image on the image which is taken by said camera, and displaying the superimposed image on said monitor device;

a pointing device for receiving information on a designated position, the information on said designated position being manually input to said monitor device;

contour point display means for generating an image of the points to be displayed on said monitor device according to outputs of said contour extracting means and said contour point candidate selecting means; and

contour line segment selecting means for selecting a contour line segment nearest to said designated position pointed to by said pointing device, among the contour line segments contained in said contour image supplied from said contour extracting means, said projected point (s) obtained by said projection means for the contour line segment selected by said contour line segment selecting means;

wherein in said beginning point selecting means, said beginning point (s⁰) can be selected by inputting through said pointing device a position of one of the projected points of the contour line segment (l) corresponding to the plurality of camera positions and being displayed on said monitor device, said projected point (s) of the selected line segment is regarded as the beginning point (s⁰).

72. An apparatus according to claim 69, further comprising:

point selecting means for selecting a point of said contour image which is extracted from the image taken in the initial camera position of said plurality of camera positions, said selected point of contour is regarded as said image element to be projected onto the predetermined projection surface; and

wherein

a distance to a point on said object corresponding to said point of said contour image selected by said point selecting means is obtained in said geometrical information obtaining means assuming that a direction of said point at infinity of said sequence of movement points is equal to a known direction (v) of the movement of said camera.

73. An apparatus according to claim 72, wherein

said camera comprises a moving device for moving said camera;

and wherein said apparatus further comprises;

a monitor device for displaying an image;

a superimposer for controlling superimposition of an image of points of one of said contour images on the image which is taken by said camera, and displaying the superimposed image on said monitor device;

a pointing device for receiving information on a designated position, the information on said designated position being manually input to said monitor device;

contour point display means for generating an image of the points to be displayed on said monitor device, according to outputs of said contour image extracting means and said contour point candidate selecting means;

means for obtaining a distance to the selected point of one of said contour images in a three-dimensional space regarding the selected point as said image element;

contour point three-dimensional measuring means for obtaining the position of the point of contour which is selected by the contour point candidate selecting means, based on the points of said one of said contour images obtained during the movement of said camera.

74. An apparatus according to claim 73, further comprising:

means for determining a direction to said third reference point wherein said selected point of contour is regarded as said image element; and

means for comparing said direction of said third reference point with the direction of movement of said camera which is known in advance.

75. An apparatus according to claim 72, wherein

said camera further comprises a moving device for moving said camera;

said apparatus further comprising;

a monitor device for displaying an image;

a superimposer for controlling the superimposition of an image of points of said one of said contour images on said image which is taken by said camera, and displaying the superimposed image on said monitor device;

a pointing device for receiving information on a designated position, the information on the designated position being manually input to said monitor device;

contour point candidate selecting means for selecting a point nearest to said designated position pointed to by said pointing device, among the points of the contour supplied from said contour image extracting means;

contour point display means for generating an image of the points to be displayed on the monitor device, according to outputs of said contour image extracting means and said contour point candidate selecting means;

plural point selecting means for selecting a plurality of points of said contour images which are subject to the measurement from said image which is obtained at the initial position of said camera;

direction determining means for determining the direction to said third reference point for each of said plurality of points of said contour image, where each of said plurality of selected points of said contour image is regarded as said image element; and

movement direction obtaining means for obtaining the direction of the movement of said camera by obtaining an average of the determined directions to the third reference points obtained by said reference point determining means for the plurality of points selected by said plural point selecting means.

76. An apparatus according to claim 72, further comprising means for confirming that a relative angle between the direction of said third reference point (c) and the direction of the movement of said camera, which is known in advance, is equal to 90 degrees.

77. An apparatus according to claim 69, wherein

said camera comprises a moving device for moving said camera;

said apparatus further comprising;

a monitor device for displaying said image;

a superimposer for controlling the superimposition of an image of points of said contour image on said image which is taken by said camera, and displaying the superimposed image on said monitor device;

a pointing device for receiving information on a designated position, the information on said designated position being manually input to said monitor device;

contour line segment selecting means for selecting a contour line segment nearest to said designated position pointed to by said pointing device, among the contour line segments contained in said contour image supplied from the contour image extracting means;

contour point displaying means for generating an image of the points to be displayed on said monitor device, according to outputs of said contour image extracting means and the contour point candidate selecting means;

contour line segment extracting means for extracting a contour line segment (l) from each of said contour images extracted in said contour image extracting means; and

projected point obtaining means for obtaining as a projected point of the contour line segment an intersection point of the predetermined projection surface and a normal line, passing through the projection center of a plane containing said projection center and a contour line segment (l);

beginning point selecting means for selecting the beginning point (s⁰) of the sequence of movement points (s^t, t=0, 1, 2, . . . , t is a frame number) of the projected points (s) of the contour line segment, where said projected points (s) which are obtained from said plurality of images, are regarded as the points of said sequence of movement points (x^t, t=0, 1, 2, . . .);

said geometrical information obtaining circuit comprising;

an intersection point obtaining means for obtaining an intersection point (Ss) of said predetermined projection surface and a normal line, passing through said center of projection, of a plane containing a trace of said sequence of movement points (s^t, t=0, 1, 2, . . .); and

an orientation obtaining circuit for obtaining the orientation (sh⁰) of a perpendicular line which is dropped from said camera to the line segment which is subject to the measurement, by an exterior product of said intersection point (Ss) of said predetermined plane or surface of the second order, and a normal line, passing through said center of projection, of a plane containing a trace of said sequence of movement points and said reference point (c=s⁰), wherein said beginning point (s⁰) of the sequence of movement points is made to be the same as said beginning point of said first and second sequence of reference points.

78. An apparatus comprising:

a camera, for taking a plurality of images of an object which is subject to the measurement, at a plurality of camera positions, respectively, by moving said camera in a predetermined direction; and

a data processor, operatively connected to said camera, for processing image data which is taken by said camera, said data processor comprising;

a contour image extracting circuit, operatively connected to said camera, for extracting a plurality of contour images respectively from said plurality of images taken by said camera, where each of said plurality of contour images contains at least one image element of said object;

a projector for projecting said at least one image element in each of said plurality of contour images of said object onto a projected point on a predetermined projection surface, for each of said at least one image element, from a predetermined projection center corresponding to a camera center, to obtain, for each of said at least one image element, a plurality of projected points on a predetermined projection surface as a sequence of movement points (x^t, t=0, 1, 2, . . .) which line up on a first intersection line at which said predetermined projection surface intersects with a first plane containing said predetermined projection center, where said predetermined projection surface is one of a plane and a surface of the second order;

a plane-projection-reference-point obtaining circuit for obtaining, as a reference point for plane projection, a first intersection point of said predetermined projection surface and a straight line which passes through said projection center and is perpendicular to an arbitrary projection plane which contains said projection center;

a plane projection circuit for obtaining a sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .), by obtaining a plurality of second planes each containing said projection center, said reference point for plane projection, and one movement point of the sequence of movement points (x^t, t=0, 1, 2, . . .), by obtaining a plurality of second intersection lines at which said plurality of second planes respectively intersect with said predetermined projection surface, and by obtaining, as said sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .), a plurality of second intersection points at which said plurality of second intersection lines respectively intersect with said predetermined projection surface;

a reference-point-sequence generating circuit for generating, as a sequence of reference points (τ

^t, t=0, 1, 2, . . .), a plurality of points which line up on a third intersection line of said predetermined projection surface and a third plane containing said projection center, so that said sequence of reference points (τ

^t, t=0, 1, 2, . . .) and said sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .) share a point, and said sequence of reference points (τ

^t, t=0, 1, 2, . . .) is not identical with either of said sequence of movement points (x^t, t=0, 1, 2, . . .) and said sequence of projected movement points (x'"'"'^t, t=0, 1, 2, . . .), and has the same cross ratio (inharmonic ratio) as a cross ratio of said sequence of projected movement points (x^t, t=0, 1, 2, . . .);

an intersection-point obtaining circuit for obtaining a third intersection point of a plurality of third intersection lines at which a plurality of fourth planes respectively intersecting the predetermined projection surface, said plurality of fourth planes each containing said projection center, one projected movement point in said sequence of projected movement points (x'"'"'^t t, t=0, 1, 2, . . .), and one reference point, corresponding to the projected movement point, in said sequence of reference points (τ

^t, t=0, 1, 2, . . .); and

a geometrical information obtaining circuit, operatively connected to said intersection-point obtaining circuit, for obtaining geometrical information on said object with respect to said camera position, based on the position of said third intersection point.

79. A method of measuring a position and orientation of an object, comprising the steps of:

(a) moving a camera in a predetermined direction with regard to an object subject to measurement;

(b) projecting movement points from the image of the object based on positions of the camera onto a plane or second order surface at an intersection with a plane formed by the positions of the camera and a center of projection;

(c) generating reference points, each corresponding to one of said movement points, on the plane or second order surface at an intersection with a plane containing the center of projection, said reference points sharing a point with said movement points and having a same cross harmonic ratio as said movement pints;

(d) obtaining an intersection point of lines on the plane or second order surface that connect corresponding pairs of said movement points and said reference pints; and

(e) obtaining the position and orientation of the object based on the positions of the camera, the center of projection and said intersection point to achieve environment recognition.

80. A method according to claim 79, wherein the surface of the second order is a sphere and the center of projection is a center of the sphere.

81. A method according to claim 79, wherein the surface of the second order is a circular cylinder and the center of projection is a point on the axis of the circular cylinder.

82. A method according to claim 79, wherein the surface of the second order is a sphere and the center of projection is a center of the sphere.

83. A method according to claim 79, wherein the surface of the second order is a circular cylinder and the center of projection is a point on the axis of the circular cylinder.

84. A method of measuring a position and orientation of an object, comprising the steps of:

(a) moving a camera to various positions with regard to an object subject to measurement;

(b) projecting movement points from an image of an object based on the positions of a camera onto a plane or second order surface of an intersection with a plane formed by the positions of the camera and a center of projection;

(c) generating a first sequence of reference points, each corresponding to one of said movement points, on the plane or second order surface at an intersection with a plane containing the center of projection, said reference points sharing a point with said movement points and having a same cross harmonic ratio as said movement points;

(d) generating a second sequence of reference points, each corresponding to one of said movement points, on the plane or second order surface at an intersection with a plane containing the center of projection, said reference points sharing a point with said movement points and having a same cross harmonic ratio as said movement points, said sequence of reference points lining up in the opposite direction to said first sequence of reference points;

(e) obtaining a first intersection point of lines on the plane or second order surface that connect corresponding pairs of said movement points and said first sequence of reference points;

(f) obtaining a second intersection point of lines on the plane or second order surface that connect corresponding pairs of said movement points and said second sequence of reference points; and

(g) obtaining the position and orientation of the object based on the positions of the camera, the center of projection and said first and second intersection points to achieve environment recognition.