Three-dimensional measuring apparatus and method, image pickup apparatus, and apparatus and method for inputting image

• US 6,038,074 A
• Filed: 05/19/1998
• Issued: 03/14/2000
• Est. Priority Date: 05/20/1997
• Status: Expired due to Term

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Kitaguchi, Takashi, Murata, Norihiko
• Primary Examiner(s): Ben, Loha
• Application Number:
  US09/081,020
• Time in Patent Office: 665 Days
• Field of Search: 359/618, 356/2, 356/139, 356/140, 356/147, 356/149, 356/376, 356/379, 128/916, 128/661.09
• US Class Current: 359/618
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: Y10S 128/916 Ultrasound 3-D imaging

Abstract

Images of an object viewed from a plurality of points of view is input through an image inputting unit. Change of orientation of the image inputting means, occurring when the images of the object are input, is detected. Change of position of point of view, occurring when the images of the object are input, is detected. Characteristic points are extracted from the image which is input previously, each of the characteristic points corresponding to a respective one of objects points on the object. Corresponding points are extracted from the image which is input currently, each of the corresponding points corresponding to a respective one of the characteristic points. The three-dimensional position of each of the object points is calculated, based on the change of orientation and the change of position of point of view of the image inputting unit and the data of the characteristic points and the corresponding points.

Claims

1. A three-dimensional-position measuring method, comprising the steps of:

a) inputting images of an object viewed from a plurality of points of view through image inputting means;

b) detecting change of orientation of said image inputting means, occurring when said step a) is performed;

c) detecting change of position of point of view, occurring when said step a) is performed;

d) extracting characteristic points from the image which is input in said step a) previously, each of said characteristic points corresponding to a respective one of objects points on said object;

e) extracting corresponding points from the image which is input in said step a) currently, each of said corresponding points corresponding to a respective one of said characteristic points; and

d) calculating the three-dimensional position of each of said object points, based on the change of orientation and the change of position of point of view of said image inputting means and said characteristic points and said corresponding points.

2. A three-dimensional-position measuring apparatus, comprising:

image inputting means for inputting images of an object viewed from a plurality of points of view;

means for detecting change of orientation of said image inputting means, occurring when said image inputting means inputs the images viewed from the plurality of points of view;

means for detecting change of position of point of view, occurring when said image inputting means inputs the images from the plurality of points of view;

means for extracting characteristic points from the image which is input by said image inputting means previously, each of said characteristic points corresponding to a respective one of objects points on said object;

means for extracting corresponding points from the image which is input by said image inputting means currently, each of said corresponding points corresponding to a respective one of said characteristic points; and

means for calculating the three-dimensional position of each of said object points, based on the change of orientation and the change of position of point of view of said image inputting means and said characteristic points and said corresponding points.

3. A three-dimensional-position measuring apparatus, comprising:

an image pickup portion which picks up images of an object from a plurality of points of view;

a motion detecting portion which detects change of orientation of said image pickup portion occurring when said image pickup portion picks up the images from the plurality of points of view;

said image detecting portion also detecting change of position of point of view occurring when said image pickup portion picks up the images from the plurality of points of view;

a correspondence relationship extracting portion which extracts characteristic points from the image which is picked up by said image pickup portion previously, each of said characteristic points corresponding to a respective one of objects points on said object;

said correspondence relationship extracting portion also extracting corresponding points from the image which is picked up by said image pickup portion currently, each of said corresponding points corresponding to a respective one of said characteristic points; and

a three-dimensional-position calculating portion which calculates the three-dimensional position of each of said object points based on the change of orientation and the change of position of point of view of said image pickup portion and said characteristic points and said corresponding points.

4. A three-dimensional-shape measuring method, comprising the steps of:

a) inputting images of an object to be measured viewed from different first and second points of view through image inputting means;

b) extracting a plurality of characteristic points from the input image of said object viewed from the first point of view;

c) extracting corresponding points from the input image of said object viewed from the second point of view, the corresponding points corresponding to the characteristic points, respectively;

d) inputting the gravity direction with respect to the orientation of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

e) inputting the rotation angle about the gravity direction of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

f) detecting the orientation of said image inputting means, for each point of view from which said image inputting means views said object, so as to input the image of said object, using the gravity direction and the rotation angle about the gravity direction;

g) calculating vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image inputting means;

h) calculating a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight; and

i) calculating the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight, so as to obtain the three-dimensional shape of said object.

5. A three-dimensional-shape measuring method, comprising the steps of:

a) inputting images of an object to be measured viewed from different first and second points of view through image inputting means;

b) extracting a plurality of characteristic points from the input image viewed from the first point of view;

c) extracting corresponding points from the input image viewed from the second point of view, the corresponding points corresponding to the characteristic points, respectively;

d) inputting the gravity direction with respect to the orientation of said image inputting means for each point of view from which said image inputting means views said object, so as to input the image of said object;

e) inputting the rotation angle about the gravity direction of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

f) detecting the orientation of said image inputting means, for each point of view from which said image inputting means views said object, so as to input the image of said object, using the gravity direction and the rotation angle about the gravity direction;

g) calculating vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image inputting means;

h) calculating a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight;

i) repeating correction of the positions of the corresponding points using the translational-motion direction vector, calculation of the vectors of lines of sight using the corrected positions of the corresponding points and calculation of the translational-motion direction vector using the calculated vectors of lines of sight, until the amount of change of the positions of the corresponding points when the positions of the corresponding points are corrected becomes less than a fixed value or the number of the repetitions reaches a fixed value; and

j) calculating the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight, finally obtained in said step i), so as to obtain the three-dimensional shape of said object.

6. A three-dimensional-shape measuring method, comprising the steps of:

a) inputting images of an object to be measured viewed from different first and second points of view through image inputting means;

b) extracting a plurality of characteristic points from the input image viewed from the first point of view;

c) extracting a plurality of candidates of each corresponding point from the input image viewed from the second point of view, each corresponding point corresponding to a respective one of the characteristic points;

d) inputting the gravity direction with respect to the orientation of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

e) inputting the rotation angle about the gravity direction of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

f) detecting the orientation of said image inputting means, for each point of view from which said image inputting means views said object, so as to input the image of said object, using the gravity direction and the rotation angle about the gravity direction;

g) calculating vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image inputting means;

h) calculating a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight;

i) repeating correction of the position of each corresponding point among the candidates using the translational-motion direction vector, calculation of the orientation of said image inputting means for the second point of view using the corrected corresponding points, calculation of the vectors of lines of sight for the second point of view using the thus-calculated orientation of said image inputting means and calculation of the translational-motion direction vector using the thus-calculated vectors of lines of sight for the second point of view and the vectors of lines of sight for the first point of view, until the amount of change of the positions of the corresponding points when the positions of the corresponding points are corrected becomes less than a fixed value or the number of the repetitions reaches a fixed value; and

j) calculating the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight finally obtained in said step i), so as to obtain the three-dimensional shape of said object to be measured.

7. A three-dimensional-shape measuring method, comprising the steps of:

a) inputting images of an object to be measured viewed from different first and second points of view through image inputting means;

b) extracting a plurality of characteristic points from the input image viewed from the first point of view;

c) extracting corresponding points from the input image viewed from the second point of view, the corresponding points corresponding to the characteristic points, respectively;

d) inputting the gravity direction with respect to the orientation of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

e) inputting the rotation angle about the gravity direction of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

f) detecting the orientation of said image inputting means, for each point of view from which said image inputting means views said object, so as to input the image of said object, using the gravity direction and the rotation angle about the gravity direction;

g) calculating vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image inputting means;

h) calculating a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight;

i) repeating detection of erroneous correspondences between characteristic points and corresponding points, respectively, using the translational-motion direction vector, removal of the characteristic points and the corresponding points, the correspondences therebetween having been determined to be erroneous, respectively, and calculation of the translational-motion direction vector using the remaining characteristic points and corresponding points, until it is determined that the translational-motion direction vector has converged; and

j) calculating the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight, finally obtained in said step i), so as to obtain the three-dimensional shape of said object to be measured.

8. A three-dimensional-shape measuring apparatus, comprising:

image inputting means for inputting images of an object to be measured viewed from different first and second points of view;

correspondence relationship extracting means for extracting a plurality of characteristic points from the input image viewed from the first point of view,

said correspondence relationship extracting means also extracting corresponding points from the input image viewed from the second point of view, the corresponding points corresponding to the characteristic points, respectively;

gravity-direction detecting means for detecting the gravity direction with respect to the orientation of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

means for detecting the rotation angle about the gravity direction of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

orientation calculating means for calculating the orientation of said image inputting means, for each point of view from which said image inputting means views said object, so as to input the image of said object, using the gravity direction and the rotation angle about the gravity direction;

means for calculating vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image inputting means;

translational-motion calculating means for calculating a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight; and

three-dimensional-position calculating means for calculating the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight, so as to obtain the three-dimensional shape of said object.

9. The three-dimensional-shape measuring apparatus, according to claim 8, wherein said gravity-direction detecting means comprises three-axis acceleration sensors.

10. The three-dimensional-shape measuring apparatus, according to claim 8, wherein said gravity-direction detecting means detects the gravity direction using the input images.

11. The three-dimensional-shape measuring apparatus, according to claim 8, wherein said means for detecting the rotation angle about the gravity direction comprises magnetism sensors.

12. The three-dimensional-shape measuring apparatus, according to claim 8, wherein said means for detecting the rotation angle about the gravity direction comprises magnetism sensors and the gravity-direction detecting means.

13. The three-dimensional-shape measuring apparatus, according to claim 8, wherein:

said correspondence relationship extracting means further calculates the values of weights for the respective characteristic points and corresponding points; and

said translational-motion calculating means calculates the translational-motion direction vector in consideration of the values of the weights for the respective characteristic points and corresponding points.

14. A three-dimensional-shape measuring apparatus, comprising:

image inputting means for inputting images of an object to be measured viewed from different first and second points of view;

correspondence relationship extracting means for extracting a plurality of characteristic points from the input image viewed from the first point of view,

said correspondence relationship extracting means also extracting corresponding points from the input image viewed from the second point of view, the corresponding points corresponding to the characteristic points, respectively;

gravity-direction detecting means for detecting the gravity direction with respect to the orientation of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

means for inputting the rotation angle about the gravity direction of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

orientation calculating means for calculating the orientation of said image inputting means, for each point of view from which said image inputting means views said object, so as to input the image of said object, using the gravity direction and the rotation angle about the gravity direction;

means for calculating vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image inputting means;

translational-motion calculating means for calculating a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight,

said means for calculating vectors of lines of sight correcting the positions of the corresponding points using the translational-motion direction vector, and calculating the vectors of lines of sight using thus-corrected positions of the corresponding points;

convergence-signal generating means for generating a convergence signal when the amount of change of the positions of the corresponding points when the positions of the corresponding points are corrected becomes less than a fixed value or the number of repetitions reaches a fixed value during the repetitions of correction of the positions of the corresponding points using the translational-motion direction vector, calculation of the vectors of lines of sight using the corrected positions of the corresponding points and calculation of the translational-motion direction vector using the thus-calculated vectors of lines of sight; and

three-dimensional-position calculating means for calculating the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight, obtained when said convergence-signal generating means generates the convergence signal, so as to obtain the three-dimensional shape of said object to be measured.

15. The three-dimensional-shape measuring apparatus, according to claim 14, wherein said convergence-signal generating means generates the convergence signal when the amount of the change of the translational-motion direction vector comes to be less than a fixed value during the repetitive calculation of the translational-motion direction vector.

16. The three-dimensional-shape measuring apparatus, according to claim 14, wherein said gravity-direction detecting means comprises three-axis acceleration sensors.

17. The three-dimensional-shape measuring apparatus, according to claim 14, wherein said gravity-direction detecting means detects the gravity direction using the input images.

18. The three-dimensional-shape measuring apparatus, according to claim 14, wherein said means for detecting the rotation angle about the gravity direction comprises magnetism sensors.

19. The three-dimensional-shape measuring apparatus, according to claim 14, wherein said means for detecting the rotation angle about the gravity direction comprises magnetism sensors and the gravity-direction detecting means.

20. The three-dimensional-shape measuring apparatus, according to claim 14, wherein:

said correspondence relationship extracting means further calculates the values of weights for the respective characteristic points and corresponding points; and

said translational-motion calculating means calculates the translational-motion direction vector in consideration of the values of the weights for the respective characteristic points and corresponding points.

21. A three-dimensional-shape measuring apparatus, comprising:

image inputting means for inputting images of an object to be measured viewed from different first and second points of view;

correspondence relationship extracting means for extracting a plurality of characteristic points from the input image viewed from the first point of view,

said correspondence relationship extracting means also extracting a plurality of candidates of each corresponding point from the input image viewed from the second point of view, each corresponding point corresponding to a respective one of the characteristic points;

gravity-direction detecting means for detecting the gravity direction with respect to the orientation of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

means for detecting the rotation angle about the gravity direction of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

orientation calculating means for calculating the orientation of said image inputting means, for each point of view from which said image inputting means views said object, so as to input the image of said object, using the gravity direction and the rotation angle about the gravity direction;

means for calculating vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image inputting means;

translational-motion calculating means for calculating a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight,

said means for calculating vectors of lines of sight correcting each corresponding point among the candidates thereof using the translational-motion direction vector and calculating the vectors of lines of sight using the corrected corresponding points and the characteristic points;

convergence-signal generating means for generating a convergence signal when the amount of change of the positions of the corresponding points when the positions of the corresponding points are corrected becomes less than a fixed value or the number of repetitions reaches a fixed value during the repetitions of correction of the positions of the corresponding points using the translational-motion direction vector, calculation of the orientation of said image inputting means for the second point of view using the corrected corresponding points, calculation of the vectors of lines of sight for the second point of view using the thus-calculated orientation of said image inputting means and calculation of the translational-motion direction vector using the thus-calculated vectors of lines of sight for the second point of view and the vectors of lines of sight for the first point of view; and

three-dimensional-point calculating means for calculating the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight obtained when said convergence-signal generating means generates the convergence signal, so as to obtain the three-dimensional shape of said object.

22. The three-dimensional-shape measuring apparatus, according to claim 21, wherein said convergence-signal generating means generates the convergence signal when the amount of the change of the translational-motion direction vector comes to be less than a fixed value during the repetitive calculation of the translational-motion direction vector.

23. The three-dimensional-shape measuring apparatus according to claim 21, wherein said convergence-signal generating means detects the convergence of the translational-motion direction vector using an error curve which expresses the changes of the translational-motion direction vector T during the repetitive calculation of the translational-motion direction vector.

24. The three-dimensional-shape measuring apparatus, according to claim 21, wherein said gravity-direction detecting means comprises three-axis acceleration sensors.

25. The three-dimensional-shape measuring apparatus, according to claim 21, wherein said gravity-direction detecting means detects the gravity direction using the input images.

26. The three-dimensional-shape measuring apparatus, according to claim 21, wherein said means for detecting the rotation angle about the gravity direction comprises magnetism sensors.

27. The three-dimensional-shape measuring apparatus, according to claim 21, wherein said means for detecting the rotation angle about the gravity direction comprises magnetism sensors and the gravity-direction detecting means.

28. The three-dimensional-shape measuring apparatus, according to claim 21, wherein:

said correspondence relationship extracting means further calculates the values of weights for the respective characteristic points and corresponding points; and

said translational-motion calculating means calculates the translational-motion direction vector in consideration of the values of the weights for the respective characteristic points and corresponding points.

29. A three-dimensional-shape measuring apparatus, comprising:

image inputting means for inputting images of an object to be measured viewed from different first and second points of view;

correspondence relationship extracting means for extracting a plurality of characteristic points from the input image viewed from the first point of view,

said correspondence relationship extracting means also extracting corresponding points from the input image viewed from the second point of view, the corresponding points corresponding to the characteristic points, respectively;

gravity-direction detecting means for detecting the gravity direction with respect to the orientation of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

means for detecting the rotation angle about the gravity direction of said image inputting means for each point of view from which said image inputting means views said object so as to input the image of said object;

orientation calculating means for calculating the orientation of said image inputting means, for each point of view from which said image inputting means views said object, so as to input the image of said object, using the gravity direction and the rotation angle about the gravity direction;

means for calculating vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image inputting means;

translational-motion calculating means for calculating a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight;

erroneous correspondence detecting means for detecting erroneous correspondences between characteristic points and corresponding points, respectively, using the translational-motion direction vector;

erroneous correspondence removing means for removing of the characteristic points and the corresponding points, the correspondences therebetween having been determined to be erroneous, respectively;

convergence-signal generating means for generating a convergence signal when it is determined that the translational-motion direction vector has converged during repetitions of the detection of the erroneous correspondences between characteristic points and corresponding points, respectively, using the translational-motion direction vector, the removal of the characteristic points and the corresponding points, the correspondences therebetween having been determined to be erroneous, respectively, and the calculation of the translational-motion direction vector using the remaining characteristic points and corresponding points; and

three-dimensional-position calculating means for calculating the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight, obtained when the convergence signal is generated, so as to obtain the three-dimensional shape of said object.

30. The three-dimensional-shape measuring apparatus according to claim 29, wherein said convergence-signal generating means detects the convergence of the translational-motion direction vector using an error curve which expresses the changes of the translational-motion direction vector T during the repetitive calculation of the translational-motion direction vector.

31. The three-dimensional-shape measuring apparatus, according to claim 29, wherein said gravity-direction detecting means comprises three-axis acceleration sensors.

32. The three-dimensional-shape measuring apparatus, according to claim 29, wherein said gravity-direction detecting means detects the gravity direction using the input images.

33. The three-dimensional-shape measuring apparatus, according to claim 29, wherein said means for detecting the rotation angle about the gravity direction comprises magnetism sensors.

34. The three-dimensional-shape measuring apparatus, according to claim 29;

wherein said means for detecting the rotation angle about the gravity direction comprises magnetism sensors and the gravity-direction detecting means.

35. The three-dimensional-shape measuring apparatus, according to claim 29, wherein:

said correspondence relationship extracting means further calculates the values of weights for the respective characteristic points and corresponding points; and

said translational-motion calculating means calculates the translational-motion direction vector in consideration of the values of the weights for the respective characteristic points and corresponding points.

36. A three-dimensional-shape measuring apparatus, comprising:

an image pickup portion which picks up images of an object to be measured from different first and second points of view;

a correspondence relationship extracting portion which extracts a plurality of characteristic points from the image picked-up from the first point of view,

said correspondence relationship extracting portion also extracting corresponding points from the image picked up from the second point of view, the corresponding points corresponding to the characteristic points, respectively;

a gravity-direction detecting portion which detects the gravity direction with respect to the orientation of said image pickup portion for each point of view from which said image pickup portion picks up the image of said object;

a portion for detecting rotation angles about the gravity direction which detects the rotation angle about the gravity direction of said image pickup portion for each point of view from which said image pickup portion picks up the image of said object;

an orientation calculating portion which calculates the orientation of said image pickup portion, for each point of view from which said image pickup portion picks up the image of said object, using the gravity direction and the rotation angle about the gravity direction;

a portion for calculating vectors of lines of sight which calculates vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image pickup portion;

a translational-motion calculating portion which calculates a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight; and

a three-dimensional-position calculating portion which calculates the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight, so as to obtain the three-dimensional shape of said object.

37. A three-dimensional-shape measuring apparatus, comprising:

an image pickup portion which picks up images of an object to be measured from different first and second points of view;

a correspondence relationship extracting means for extracting a plurality of characteristic points from the image picked up from the first point of view,

said correspondence relationship extracting portion also extracting corresponding points from the image picked up from the second point of view, the corresponding points corresponding to the characteristic points, respectively;

a gravity-direction detecting portion which detects the gravity direction with respect to the orientation of said image pickup portion for each point of view from which said image pickup portion picks up the image of said object;

a portion for detecting rotation angles about the gravity direction which detects the rotation angle of said image pickup portion about the gravity direction for each point of view from which said image pickup portion picks up the image of said object;

an orientation calculating portion which calculates the orientation of said image pickup portion, for each point of view from which said image pickup portion picks up the image of said object, using the gravity direction and the rotation angle about the gravity direction;

a portion for calculating vectors of lines of sight which calculates vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image pickup portion;

a translational-motion calculating portion which calculates a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight,

said portion for calculating vectors of lines of sight correcting the positions of the corresponding points using the translational-motion direction vector, and calculating the vectors of lines of sight using thus-corrected positions of the corresponding points;

a convergence-signal generating portion which generates a convergence signal when the amount of change of the positions of the corresponding points when the positions of the corresponding points are corrected becomes less than a fixed value or the number of repetitions reaches a fixed value during the repetitions of correction of the positions of the corresponding points using the translational-motion direction vector, calculation of the vectors of lines of sight using the corrected positions of the corresponding points and calculation of the translational-motion direction vector using the thus-calculated vectors of lines of sight; and

a three-dimensional-position calculating portion which calculates the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight, obtained when said convergence-signal generating means generates the convergence signal, so as to obtain the three-dimensional shape of said object to be measured.

38. A three-dimensional-shape measuring apparatus, comprising:

an image pickup portion which picks up images of an object to be measured from different first and second points of view;

a correspondence relationship extracting portion which extracts a plurality of characteristic points from the image picked up from the first point of view;

said correspondence relationship extracting portion also extracting a plurality of candidates of each corresponding point from the image picked up from the second point of view, each corresponding point corresponding to a respective one of the characteristic points;

a gravity-direction detecting portion which detects the gravity direction with respect to the orientation of said image pickup portion for each point of view from which said image pickup portion picks up the image of said object;

a portion for detecting rotation angles about the gravity direction which detects the rotation angle of said image pickup portion about the gravity direction for each point of view from which said image pickup portion picks up the image of said object;

an orientation calculating portion which calculates the orientation of said image pickup portion, for each point of view from which said image pickup portion picks up the image of said object, using the gravity direction and the rotation angle about the gravity direction;

a portion for calculating vectors of lines of sight which calculates vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image pickup portion;

a translational-motion calculating portion which calculates a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight;

said portion for calculating vectors of lines of sight correcting each corresponding point among the candidates thereof using the translational-motion direction vector and calculating the vectors of lines of sight using the data of the corrected corresponding points and the characteristic points;

a convergence-signal generating portion which generates a convergence signal when the amount of change of the positions of the corresponding points when the positions of the corresponding points are corrected becomes less than a fixed value or the number of repetitions reaches a fixed value during the repetitions of correction of the positions of the corresponding points using the translational-motion direction vector, calculation of the orientation of said image pickup portion for the second point of view using the data of the corrected corresponding points, calculation of the vectors of lines of sight for the second point of view using the thus-calculated orientation of said image pickup portion and calculation of the translational-motion direction vector using the thus-calculated vectors of lines of sight for the second point of view and the vectors of lines of sight for the first point of view; and

a three-dimensional-point calculating portion which calculates the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight obtained when said convergence-signal generating means generates the convergence signal, so as to obtain the three-dimensional shape of said object.

39. A three-dimensional-shape measuring apparatus, comprising:

an image pickup portion which picks up images of an object to be measured from different first and second points of view;

a correspondence relationship extracting portion which extracts a plurality of characteristic points from the image picked up from the first point of view;

said correspondence relationship extracting portion also extracting corresponding points from the image picked up from the second point of view, the corresponding points corresponding to the characteristic points, respectively;

a gravity-direction detecting portion which detects the gravity direction with respect to the orientation of said image pickup portion for each point of view from which said image pickup portion picks up the image of said object;

a portion for detecting rotation angles about the gravity direction which detects the rotation angle of said image pickup portion for each point of view from which said image pickup portion picks up the image of said object;

an orientation calculating portion which calculates the orientation of said image pickup portion, for each point of view from which said image pickup portion picks up the image of said object, using the gravity direction and the rotation angle about the gravity direction;

a portion for calculating vectors of lines of sight which calculates vectors of lines of sight for the first and second points of view, respectively, the vectors of lines of sight passing through the characteristic points and corresponding points, respectively, using the characteristic points and corresponding points, and the orientations of said image pickup portion;

a translational-motion calculating portion which calculates a translational-motion direction vector extending from said first point of view to said second point of view, using the vectors of lines of sight;

an erroneous correspondence detecting portion which detects erroneous correspondences between characteristic points and corresponding points, respectively, using the translational-motion direction vector;

an erroneous correspondence removing portion which removes the characteristic points and the corresponding points, the correspondences therebetween having been determined to be erroneous, respectively;

a convergence-signal generating portion which generates a convergence signal when it is determined that the translational-motion direction vector has converged during repetitions of the detection of the erroneous correspondences between characteristic points and corresponding points, respectively, using the translational-motion direction vector, the removal of the characteristic points and the corresponding points, the correspondences therebetween being determined to be erroneous, respectively, and the calculation of the translational-motion direction vector using the remaining characteristic points and corresponding points; and

a three-dimensional-position calculating portion which calculates the three-dimensional positions of object points on said object, using the translational-motion direction vector and the vectors of lines of sight, obtained when the convergence signal is generated, so as to obtain the three-dimensional shape of said object.

40. An image pickup apparatus, comprising:

an image pickup portion which picks up images of an object plane to be input from a plurality of points of view in a manner in which a portion of each picked-up image corresponds to a portion of another picked-up image;

a motion detecting portion which detects change of the orientation and change of the position of point of view of said image pickup portion, occurring when said image pickup portion picks up the images of said object plane from the respective points of view;

a correspondence relationship extracting portion which extracts characteristic points from one of the picked-up images, the characteristic points corresponding to object points on said object plane, respectively,

said corresponding relationship extracting portion also extracting corresponding points from another one of the picked-up images, which is picked up immediately after said one of the picked-up images from which the characteristic points are extracted is picked up, the corresponding points corresponding to the characteristic points, respectively;

a three-dimensional-position measuring portion which calculates the three-dimensional positions of the object points, based on the detected change of the orientation, the detected change of the position of point of view, the data of the extracted characteristic points and corresponding points;

a plane calculating portion which assumes that the calculated positions of the object points are present on the same plane and calculates an equation of the plane which fits the calculated positions of the object points, based on the calculated three-dimensional positions of the object points;

a projection calculating portion which performs a calculation so as to virtually project the respective images picked up by said image pickup portion on the same arbitrary virtual image plane so as to synthesize the respective images picked up from the plurality of points of view on the virtual image plane, based on the detected change of the orientation and the detected change of the position of the point of view, and the calculated equation of the plane.

41. The image pickup apparatus, according to claim 40, further comprising an adjusting portion which performs fine adjustment of the position, at which each picked-up image is to be projected, and the scale of each picked-up image to be projected, in a manner in which the cross-correlation value between the corresponding portions of the respective picked-up images, which portions are superimposed on the virtual image plane, is maximized when the respective images picked up by said image pickup portion are projected on the virtual image plane.

42. An image pickup apparatus, comprising:

an image pickup portion which picks up images of an object plane to be input from a plurality of points of view in a manner in which a portion of each picked-up image corresponds to a portion of another picked-up image;

an orientation detecting portion which detects the orientation of said image pickup portion, in which said image pickup portion picks up the image of said object plane from each point of view;

a correspondence relationship extracting portion which extracts characteristic points from one of the picked-up images, the characteristic points corresponding to object points on said object plane, respectively,

said corresponding relationship extracting portion also extracting corresponding points from another one of the picked-up images, which is picked up immediately after said one of the picked-up images from which the characteristic points are extracted is picked up, the corresponding points corresponding to the characteristic points, respectively;

a translational-motion detecting portion which detects a translational-motion direction vector for the respective object points, based on the detected orientations, and the extracted characteristic points and corresponding points;

a three-dimensional-position measuring portion which calculates the three-dimensional positions of the respective object points, based on the detected orientations, the data of the extracted characteristic points and corresponding points, and the detected translational-motion direction vector;

a plane calculating portion which assumes that the calculated positions of the object points are present on the same plane and calculates an equation of the plane which fits the calculated positions of the object points, based on the calculated three-dimensional positions of the object points;

a projection calculating portion which performs a calculation so as to virtually project the respective images picked up by said image pickup portion on the same arbitrary virtual image plane so as to synthesize the respective images picked up from the plurality of points of view on the virtual image plane, based on the detected orientations, the detected translational-motion direction vector, and the calculated equation of the plane.

43. The image pickup apparatus, according to claim 42, wherein said orientation detecting portion is provided with a magnetism detecting portion which detects magnetism in one of two-axis directions and three-axis directions, orthogonal to each other.

44. The image pickup apparatus, according to claim 42, wherein said orientation detecting portion is provided with a magnetism detecting portion which detects the magnetism in one of two-axis directions and three-axis directions, orthogonal to each other, and a gravity-direction detecting portion which detects the gravity direction.

45. The image pickup apparatus, according to claim 42, wherein said orientation detecting portion is provided with an angular-speed detecting portion which detects the rotation angular speeds of said image pickup portion about one of two-axis directions and three-axis directions, orthogonal to each other, respectively.

46. The image pickup apparatus, according to claim 42, further comprising an adjusting portion which performs fine adjustment of the position, at which each picked-up image is to be projected, and the scale of each picked-up image to be projected, in a manner in which the cross-correlation value between the corresponding portions of the respective picked-up images, which portions are superimposed on the virtual image plane, is maximized when the respective images picked up by said image pickup portion are projected on the virtual image plane.

47. An image inputting apparatus, comprising:

image inputting means for inputting images of an object plane to be input viewed from a plurality of points of view in a manner in which a portion of each input image corresponds to a portion of another input image;

motion detecting means for detecting change of the orientation and change of the position of point of view of said image inputting means, occurring when said image inputting means inputs the images of said object plane viewed from the respective points of view;

correspondence relationship extracting means for extracting characteristic points from one of the input images, the characteristic points corresponding to object points on said object plane, respectively,

said corresponding relationship extracting means also extracting corresponding points from another one of the input images, which is input immediately after said one of the input images from which the characteristic points are extracted is input, the corresponding points corresponding to the characteristic points, respectively;

three-dimensional-position measuring means for calculating the three-dimensional positions of the object points, based on the detected change of the orientation, the detected change of the position of point of view, the data of the extracted characteristic points and corresponding points;

plane calculating means for assuming that the calculated positions of the object points are present on the same plane and calculating an equation of the plane which fits the calculated positions of the object points, based on the calculated three-dimensional positions of the object points;

projection calculating means for performing a calculation so as to virtually project the respective images input by said image inputting means on the same arbitrary virtual image plane so as to synthesize the respective images, viewed from the plurality of points of view, so as to be input, on the virtual image plane, based on the detected change of the orientation and the detected change of the position of the point of view, and the calculated equation of the plane.

48. The image inputting apparatus, according to claim 47, further comprising adjusting means for performing fine adjustment of the position, at which each input image is to be projected, and the scale of each input image to be projected, in a manner in which the cross-correlation value between the corresponding portions of the respective input images, which portions are superimposed on the virtual image plane, is maximized when the respective images input by said image inputting means are projected on the virtual image plane.

49. An image inputting apparatus, comprising:

image inputting means for inputting images of an object plane to be input viewed from a plurality of points of view in a manner in which a portion of each input image corresponds to a portion of another input image;

orientation detecting means for detecting the orientation of said image inputting means, in which said image inputting means inputs the image of said object plane viewed from each point of view;

correspondence relationship extracting means for extracting characteristic points from one of the input images, the characteristic points corresponding to object points on said object plane, respectively,

said corresponding relationship extracting means also extracting corresponding points from another one of the input images, which is input immediately after said one of the picked-up images from which the characteristic points are extracted is input, the corresponding points corresponding to the characteristic points, respectively;

translational-motion detecting means for detecting a translational-motion direction vector for the respective object points, based on the detected orientations, and the extracted characteristic points and corresponding points;

three-dimensional-position measuring means for calculating the three-dimensional positions of the respective object points, based on the detected orientations, the extracted characteristic points and corresponding points, and the detected translational-motion direction vector;

plane calculating means for assuming that the calculated positions of the object points are present on the same plane and calculating an equation of the plane which fits the calculated positions of the object points, based on the calculated three-dimensional positions of the object points;

projection calculating means for performing a calculation so as to virtually project the respective images input by said image inputting means on the same arbitrary virtual image plane so as to synthesize the respective images, viewed from the plurality of points of view so as to be input, on the virtual image plane, based on the detected orientations, the detected translational-motion direction vector, and the calculated equation of the plane.

50. The image inputting apparatus, according to claim 49, further comprising adjusting means for performing fine adjustment of the position, at which each input image is to be projected, and the scale of each input image to be projected, in a manner in which the cross-correlation value between the corresponding portions of the respective input images, which portions are superimposed on the virtual image plane, is maximized when the respective images input by said image inputting means are projected on the virtual image plane.

51. An image inputting method, comprising the steps of:

a) inputting images of an object plane to be input viewed from a plurality of points of view in a manner in which a portion of each input image corresponds to a portion of another input image;

b) detecting change of the orientation and change of the position of point of view, occurring when said step a) is performed;

c) extracting characteristic points from one of the input images, the characteristic points corresponding to object points on said object plane, respectively;

d) extracting corresponding points from another one of the input images, which is input immediately after said one of the input images from which the characteristic points are extracted is input, the corresponding points corresponding to the characteristic points, respectively;

e) calculating the three-dimensional positions of the object points, based on the detected change of the orientation, the detected change of the position of point of view, the extracted characteristic points and corresponding points;

f) assuming that the calculated positions of the object points are present on the same plane and calculating an equation of the plane which fits the calculated positions of the object points, based on the calculated three-dimensional positions of the object points; and

g) performing a calculation so as to virtually project the respective images input in said step a) on the same arbitrary virtual image plane so as to synthesize the respective images, viewed from the plurality of points of view, so as to be input, on the virtual image plane, based on the detected change of the orientation and the detected change of the position of the point of view, and the calculated equation of the plane.

52. The image inputting method, according to claim 51, further comprising the step h) performing fine adjustment of the position, at which each input image is to be projected, and the scale of each input image to be projected, in a manner in which the cross-correlation value between the corresponding portions of the respective input images, which portions are superimposed on the virtual image plane, is maximized when the respective images input in said step a) are projected on the virtual image plane.

53. An image inputting method, comprising the steps of:

a) inputting images of an object plane to be input viewed from a plurality of points of view in a manner in which a portion of each input image corresponds to a portion of another input image;

b) detecting the orientations, in which said step a) is performed;

c) extracting characteristic points from one of the input images, the characteristic points corresponding to object points on said object plane, respectively;

d) extracting corresponding points from another one of the input images, which is input immediately after said one of the picked-up images from which the characteristic points are extracted is input, the corresponding points corresponding to the characteristic points, respectively;

e) detecting a translational-motion direction vector for the respective object points, based on the detected orientations, and the data of the extracted characteristic points and corresponding points;

f) calculating the three-dimensional positions of the respective object points, based on the detected orientations, the extracted characteristic points and corresponding points, and the detected translational-motion direction vector;

g) assuming that the calculated positions of the object points are present on the same plane and calculating an equation of the plane which fits the calculated positions of the object points, based on the calculated three-dimensional positions of the object points;

h) performing a calculation so as to virtually project the respective images input in said step a) on the same arbitrary virtual image plane so as to synthesize the respective images, viewed from the plurality of points of view, so as to be input, on the virtual image plane, based on the detected orientations, the detected translational-motion direction vector, and the calculated equation of the plane.

54. The image inputting method, according to claim 53, further comprising the step i) performing fine adjustment of the position, at which each input image is to be projected, and the scale of each input image to be projected, in a manner in which the cross-correlation value between the corresponding portions of the respective input images, which portions are superimposed on the virtual image plane, is maximized when the respective images input in said step a) are projected on the virtual image plane.