Image pickup system

• US 20020175994A1
• Filed: 05/20/2002
• Published: 11/28/2002
• Est. Priority Date: 05/25/2001
• Status: Active Grant

• Current Assignee: Minolta Corporation Limited (Konica Minolta Inc.)
• Original Assignee: Minolta Corporation Limited (Konica Minolta Inc.)
• Inventors: Fujiwara, Koji, Fujii, Eiro, Sakakibara, Kuniteru
• Granted Patent: US 7,423,666 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

The present invention provides a three-dimensional chart, a parameter acquiring method and an information processing device that are used for carrying out camera calibration in a wide range with high precision. For this reason, a plurality of unit graphic forms the sizes of which are coded by using different cross-ratios are placed on side faces of a three-dimensional chart. The cross-ratios of unit graphic forms are calculated from a picked-up image of the three-dimensional chart, and by collating these with actual values, the position and orientation of the image-pickup point are determined. The shifting average of the heights of the unit graphic forms is made approximately proportional to the distance from the apex so that it is possible to reduce the limitation of image-pickup distances. A movable camera is attached to a subject-use camera, and upon picking up an image of the subject, an image of the three-dimensional chart is also picked up simultaneously. Based upon the position and orientation of the movable camera found out by the above-mentioned sequence of image-pickup processes and the relative position and orientation between the two cameras that have been preliminarily found, it is possible to determine the position and orientation of the subject-use camera.

Claims

1. An image pickup system, which comprises:

(a) a movable image-pickup apparatus; and

(b) a reference object having a known shape in a three-dimensional space,

said movable image-pickup apparatus including;

(a-1) an image-pickup device for acquiring an image of said reference object as first image data;

(a-2) a driver which rotates and drives said image-pickup device with respect to a fixed portion; and

(a-3) a controller which controls said driver means in such a manner that, when said movable image-pickup apparatus is moved, said image-pickup device can pick up an image of said reference object.

2. The image pickup system in accordance with claim 1, further comprising:

(c) a calibration-use information processing device,

wherein said movable image-pickup apparatus comprises;

(a-4) a first communication device for carrying out data communications with said calibration-use information processing device, said calibration-use information processing device comprising;

(c-1) a second communication device for carrying out data communications with said movable image-pickup apparatus; and

(c-2) a first calculator which, based upon said first image data received from said second communication device, calculates a first parameter used for calibration for said movable image-pickup apparatus that depends on a relative position and a relative orientation of a image-pickup position of said movable image-pickup apparatus and said reference object.

3. The image pickup system according to claim 2, further comprising:

(d) a subject-use image-pickup apparatus for acquiring an image of a subject as second image data,

wherein said movable image-pickup apparatus further comprises;

(a-5) a securing member which secures said fixed portion to another object;

(a-6) an element for finding a rotation angle from a reference orientation of said image-pickup device that has been rotated by said driver;

(a-7) an image-pickup controller which is attached to said subject-use image-pickup apparatus by said securing member, and allows said image-pickup device to acquire said first image data in cooperation with said second image data acquiring process of said subject-use image-pickup apparatus, said subject-use image-pickup apparatus comprises;

(d-1) a third communication device for transmitting said second image data to said calibration-use information processing device, and said calibration-use information processing device further comprises;

(c-3) a second calculator which, based upon said second image data obtained by image-picking up said reference object, calculates a second parameter used for calibration for said subject-use image-pickup apparatus that depends on a relative position and a relative orientation of an image-pickup position of said subject-use image-pickup apparatus and said reference object;

(c-4) a third calculator which, based upon said first parameter, said second parameter and said rotation angle, calculates a third parameter used for calibration for said subject-use image-pickup apparatus that depends on a relative position and a relative orientation between an image-pickup position in a reference orientation of said image pickup device and an image-pickup position of said subject-use image-pickup apparatus; and

(c-5) a fourth calculator which, based upon said first parameter and said third parameter, calculates a fourth parameter used for calibration for said subject-use image-pickup apparatus that depends on a relative position and a relative orientation between an image-pickup position of said subject-use image-pickup apparatus and said reference object.

4. The image pickup system according to claim 2, wherein said movable image-pickup apparatus further comprises:

(a-8) a securing member which secures said fixed portion to another object;

(a-9) an element for finding a rotation angle from a reference orientation of said image-pickup device that has been rotated by said driver;

(a-10) an element for acquiring reference relative data that depends on a relative position and a relative orientation between an image-pickup position in a reference orientation of said image pickup device and said fixed portion; and

(a-11) an element which, based upon said rotation angle and said reference relative data, calculates a fifth parameter that depends on a relative position and a relative orientation between an image-pickup position of said image pickup device and said fixed portion,

wherein said calibration-use information processing device calculates a six parameter that depends on a relative position and a relative orientation between a position of said fixed portion and said reference object based upon said fifth parameter received from said second communication device and said first parameter.

5. The image pickup system according to claim 4, wherein said movable image-pickup apparatus is attachable to a human body by said securing member.

6. An image pickup system comprising:

(a) a reference object that is placed with a predetermined positional relationship with respect to a subject; and

(b) a portable camera system,

wherein said camera system comprises;

(b-1) a measuring device;

(b-2) a movable image-pickup apparatus which has an image-pickup device that is capable of relatively rotating with respect to said measuring device so as to pick up an image of said reference object while tracing said image thereof;

(b-3) a detection device for detecting an amount of rotation shift of said measuring device by using an orientation sensor; and

(b-4) a controller which, based upon said amount of rotation shift detected by said detection device, carries out a rotation driving control on said image pickup device by using said movable image-pickup apparatus so that a tracing control is carried out on said reference object,

wherein based upon results of picked-up image of said reference object by said movable image-pickup apparatus, a calibration-use parameter of said measuring device is calculated.

7. The image pickup system according to claim 6, wherein said detection device comprises a gyro sensor for detecting an amount of rotation shift of said measuring device as said orientation sensor, and said first controller carries out said rotation driving control based upon said amount of rotation shift from said gyro sensor.

8. The image pickup system according to claim 6, wherein said detection device further comprises:

a parallel shifting amount detector for detecting an amount of parallel shift of said measuring device through an integrating process of acceleration, and said controller carries out said rotation driving control based upon said amount of parallel shift.

9. The image pickup system according to claim 6, wherein said calibration-use parameter is calculated by using said amount of rotation shift together with said results of picked-up image of said reference object by said movable image-pickup apparatus.

10. The image pickup system according to claim 6, wherein said controller carries out a tracing control so as to make a target point on said reference object coincident with a specific position in an view angle in said image-pickup device.

11. The image pickup system according to claim 6, wherein said reference object is a three-dimensional chart having faces of a pyramid shape on which a predetermined pattern is displayed.

13. The camera system according to claim 12, wherein said detection device comprises a gyro sensor for detecting an amount of rotation shift of said measuring device as said orientation sensor, and said first controller carries out said rotation driving control based upon said amount of rotation shift from said gyro sensor.

14. The camera system according to claim 12, wherein said detection device further comprises:

a parallel shifting amount detector for detecting an amount of parallel shift of said measuring device through an integrating process of acceleration, and said controller carries out said rotation driving control based upon said amount of parallel shift.

16. The movable image-pickup device according to claim 15 further comprising:

a securing member which attaches said fixed portion to a subject-use image-pickup device that acquires an image of a subject as second image data; and

an image-pickup controller which allows said image pickup device to acquire said first image data in synchronism with an acquiring process of said second image data of said subject-use image-pickup device.

17. The movable image-pickup device according to claim 15 which further comprises:

a securing member which secures said fixed portion to another object; and

an element for finding a rotation angle from a reference orientation of said image-pickup device that has been rotated by said driver, wherein a parameter that depends on a relative position and a relative orientation between the image-pickup position of said image pickup device and said fixed portion is calculated, based upon said rotation angle and a positional relationship between said image pickup device in said reference orientation and said fixed portion.

18. The movable image-pickup device according to claim 17, which is attachable to a human body by said securing member.

20. The measuring device according to claim 19, further comprising:

a second image-pickup device which picks up a two-dimensional image of an object,

wherein said display device selectively displays an image picked up by said first image-pickup device and an image picked up by said second image-pickup device.

21. The measuring device according to claim 19, wherein said first image-pickup device has a variable image-pickup direction.

22. The measuring device according to claim 21, wherein said first image-pickup device comprises a controller which controls an image-pickup direction so as to automatically trace said reference object.

26. The measuring device according to claim 25, wherein an image is picked up with at least two of orientation-detecting-use reference objects being held within a view angle of said reference-object pickup-use camera, and based upon respective orientation-detecting-use reference objects contained in said image, said first calculator finds a relative positional relationship of said at least two of orientation-detecting-use reference objects.

27. The measuring device according to claim 25, wherein each of images of said plurality of orientation-detecting-use reference objects are picked up by said reference-object pickup-use camera with an orientation of said reference-object pickup-use camera being altered, and based upon said each of images and said orientation of said reference-object pickup-use camera at the time of picking up said each of images, said first calculator finds a relative positional relationship of said plurality of orientation-detecting-use reference objects.

28. The measuring device according to claim 25, wherein said reference-object pickup-use camera is secured to said measuring device in a manner so as to alter an image-pickup direction thereof.

29. The measuring device according to claim 28, wherein said controller comprises an image-pickup direction controller that controls said image-pickup direction of said reference-object pickup-use camera so as to allow said reference-object pickup-use camera to pick up an image of at least one of said orientation-detecting-use reference objects.

30. The measuring device according to claim 25, wherein said measuring device is a device for measuring a three-dimensional shape of a subject so that said three-dimensional shapes obtained measurements of several times are composed based upon respective orientations at time of said measurements.

12. A portable camera system comprising:

(a) a measuring device;

(b) a movable image-pickup apparatus which has an image-pickup device that is capable of relatively rotating with respect to said measuring device so as to pick up an image of a reference object that is placed with a predetermined positional relationship with respect to a subject while tracing said image thereof;

(c) a detection device for detecting an amount of rotation shift of said measuring device by using an orientation sensor; and

(d) a controller which, based upon said amount of rotation shift detected by said detection device, carries out a rotation driving control on said image pickup device by using said movable image-pickup apparatus so that a tracing control is carried out on said reference object,

wherein based upon results of picked-up image of said reference object by said movable image-pickup apparatus, a calibration-use parameter of said measuring device is calculated.

15. A movable image-pickup device comprising:

an image pickup device which acquires an image of a reference object having a known three-dimensional shape;

a driver which drives said image pickup device with respect to a fixed portion; and

a controller which, in the case when said movable image-pickup device is shifted, controls said driver so as to allow said image pickup device to pick up an image of said reference object.

19. A measuring device, which measures an object, comprising:

a first image-pickup device for picking up an image of a reference object having a known three-dimensional shape;

an orientation detector which calculates an orientation of said measuring device based upon a reference object image picked up by said first image-pickup device; and

a display device which displays an image picked up by said first image-pickup device.

23. A measuring device, which measures an object, comprising:

a first image-pickup device which picks up an image of a reference object having a known three-dimensional shape;

an orientation detector which calculates an orientation of said measuring device based upon a reference object image picked up by said first image-pickup device;

a detector which detects whether or not said reference object is in an operable state for an image-pickup process by said first image-pickup device; and

an informing device for informing results of detection by said detector.

24. A measuring device, which measures an object, comprising:

a first image-pickup device which picks up an image of a reference object having a known three-dimensional shape;

an orientation detector which calculates an orientation of said measuring device based upon a reference object image picked up by said first image-pickup device;

a detector which detects whether or not said reference object is in an operable state for an image-pickup process by said first image-pickup device; and

an inhibition element which, when said detector has detected that said reference object is not in said operable state for said image-pickup process, inhibits a measuring operation in said measuring device.

25. A measuring device, which measures a subject in a state where a plurality of orientation-detecting-use reference objects, each having a known shape, are placed on a periphery of said subject, comprising:

a reference-object pickup-use camera;

a first calculator which, based upon images of said plurality of orientation-detecting-use reference objects preliminarily picked up by said reference-object pickup-use camera, calculates relative positional relationships among said plurality of orientation-detecting-use reference objects;

a controller which controls said reference-object pickup-use camera so as to pick up an image in synchronism with a measuring operation of said measuring device; and

a second calculator which, based upon images of said orientation-detecting-use reference objects picked up by said reference-object pickup-use camera during the measuring process, calculates an orientation of said measuring device.

31. A measuring device, which measures a subject in a state where a plurality of orientation-detecting-use reference objects, each having a known shape, are placed on a periphery of said subject, comprising:

a subject-image pickup-use camera for picking up an image of said subject;

a reference-object pickup-use camera;

a first calculator which, based upon images of said plurality of orientation-detecting-use reference objects that have been preliminarily picked up by said reference-object pickup-use camera and/or said subject-image pickup-use camera, calculates relative positional relationships among said plurality of orientation-detecting-use reference objects;

a controller which controls said reference-object pickup-use camera so as to pick up an image in synchronism with an image pickup operation of said subject-image pickup-use camera; and

a second calculator which, based upon images of said orientation-detecting-use reference objects picked up by said reference-object pickup-use camera during a measuring process, calculates an orientation of said measuring device,

wherein predetermined measuring calculations are carried out based upon an image picked up by said subject-image pickup-use camera and an orientation at that time.

32. A measuring device, which measures a subject in a state where a plurality of orientation-detecting-use reference objects, each having a known shape, are placed on a periphery of said subject, comprising:

a movable camera which is designed so as to alter an image-pickup orientation; and

a controller which controls said image-pickup orientation of said movable camera so that said movable camera is allowed to pick up an image of a proximate reference object that is closest to said movable camera among said plurality of orientation-detecting-use reference objects at time of measuring operation by said measuring device.

33. The measuring device according to claim 32, further comprising:

a calculator which, based upon respective orientation-detection-use reference objects contained in an image of said plurality of orientation-detection-use reference objects preliminarily picked up by said movable camera, finds a relative positional relationship of said plurality of orientation-detection-use reference objects,

wherein, during said measuring operation, based upon an image obtained by allowing said movable camera to pick up a desired orientation-detection-use reference object among said plurality of orientation-detection-use reference objects, said controller finds a relative position and a relative orientation of said movable camera with respect to said desired orientation-detection-use reference object, and further specifies said proximate reference object among said plurality of orientation-detection-use reference objects based upon said relative position and said relative orientation of said movable camera with respect to said desired orientation-detection-use reference object and said relative positional relationship.

34. The measuring device according to claim 32, wherein said controller controls said image-pickup orientation so as to allow said movable camera to automatically trace said proximate reference object.

35. The measuring device according to claim 34, wherein a marker is placed on a predetermined position of each of said plurality of orientation-detection-use reference objects, and said controller controls said image-pickup orientation so as to direct image components of said marker contained in said image picked up by said movable camera to a predetermined position in said image so that said movable camera is allowed to automatically trace said proximate reference object.

36. The measuring device according to claim 33, wherein measured data at the time of measuring said subject is generated based upon a relative position and a relative orientation between said movable camera and said proximate reference object and said relative positional relationship with respect to said proximate reference object and the other orientation-detection-use reference objects.

37. The measuring device according to claim 32, further comprising:

a subject-use camera for picking up a two-dimensional image of said subject.

39. The device according to claim 38, wherein each of said plurality of measuring device calibration-use reference objects has an external appearance element indicating a direction, and respective orientations of said plurality of measuring device calibration-use reference objects are determined by using said external appearance element indicating said direction.

38. A device, which finds calibration-use parameters of a measuring device, comprising:

(a) an observer which observes at least one measuring-device-calibration-use reference object of a plurality of measuring device calibration-use reference objects that have same external features and are placed in a space in a dispersed manner so as to be distinguished in mutual orientations from external features independent of observing positions, from an observing position related to said measuring device;

(b) a specifying element which specifies a relative position and a relative orientation between said at least one measuring-device-calibration-use reference object and said measuring device based upon images obtained from said observer;

(c) an orientation detector which detects an orientation of said measuring device in a space to accommodate said subject by using an angle sensor attached to said measuring device; and

(d) an identifying element which identifies said measuring-device-calibration-use reference object observed by said observer by collating a detected orientation of said measuring device within said space and an orientation of each of said plurality of measuring-device-calibration-use reference objects within said space that have been preliminarily found.

40. A device, which finds a calibration-use parameter of a camera, comprising:

an identifying element which extracts at least one unit graphic form from a chart image obtained by observing a camera calibration-use three-dimensional chart having a plurality of charts formed on a plurality of side faces from an observing position as a target unit graphic form, and identifies said target unit graphic form among said plurality of charts and among a plurality of unit graphic forms; and

a calculator which, based upon a relationship between an actual size that has been preliminarily specified with respect to said target unit graphic form identified and an observed size of said target unit graphic form viewed from said observing position, calculates said calibration-use parameter of said camera that depends on a relative position and a relative orientation between said observing position and said camera calibration-use three-dimensional chart.

41. A camera-calibration-use three-dimensional chart comprising:

a conical body having a bottom face and a plurality of side faces; and

a plurality of charts that are displayed on each of said plurality of side faces, each of said plurality of charts having a set of a plurality of unit graphic forms having respectively known sizes,

wherein each of said plurality of unit graphic forms has a geometrical shape that is coded so as to be identified by an observation from said observing position with respect to both of inside of said set and said plurality of charts, independent of relative positions and relative orientations between each of unit graphic forms and said observing position.

42. The camera-calibration-use three-dimensional chart according to claim 41, wherein said plurality of unit graphic forms have increased sizes as a distance from said bottom face of said conical body becomes shorter.

43. The camera-calibration-use three-dimensional chart according to claim 42, wherein said conical body has a pyramid shape and said plurality of charts are coded so as to be mutually recognizable by an observation from said observing position independent of a relative position and a relative orientation between each of said plurality of charts and said observing position.

44. The camera-calibration-use three-dimensional chart according to claim 43, wherein said plurality of unit graphic forms are a plurality of trapezoid shapes formed by crossing between a plurality of first straight lines in parallel with said bottom face of said conical body and a plurality of second straight lines that extend radially from a position corresponding an apex of said conical body, with cross ratios of sizes of said plurality of trapezoid shapes being coded.

45. The camera-calibration-use three-dimensional chart according to claim 44, wherein a marker is placed on said apex of said conical body.

46. The camera-calibration-use three-dimensional chart according to claim 44, wherein said marker includes a light-emitting element.

47. The camera-calibration-use three-dimensional chart according to claim 41, wherein adjacent unit graphic forms among said plurality of unit graphic forms are painted in mutually different brightness or hues in a separated manner.

48. A camera-calibration-use three-dimensional chart comprising:

a conical body having a bottom face and a side face; and

a chart that is displayed along a circumferential portion of said side face, said chart having a set of a plurality of unit graphic forms having respectively known sizes,

wherein each of said plurality of unit graphic forms has a geometrical shape that is coded so as to be identified by an observation from an observing position with respect to inside of said set, independent of relative positions and relative orientations between each of said plurality of the unit graphic forms and said observing position.

49. A method for measuring a subject in a state where a plurality of orientation-detecting-use reference objects having a known shape are placed on a periphery of said subject, comprising the steps of:

(a) preliminarily picking up images of said plurality of orientation-detecting-use reference objects by a reference-object pickup-use camera and calculating relative positional relationships among said plurality of orientation-detecting-use reference objects based upon said images;

(b) picking up an image of at least one of said plurality of orientation-detecting-use reference objects by using said reference-object pickup-use camera in synchronism with a measuring operation of a measuring device; and

(c) calculating an orientation of said measuring device, based upon images of said orientation-detecting-use reference objects picked up by said reference-object pickup-use camera during a measuring process.

50. A method for measuring a subject in a state where a plurality of orientation-detecting-use reference objects having a known shape are placed on a periphery of said subject, comprising the steps of:

(a) preliminarily picking up images of said plurality of orientation-detecting-use reference objects by a reference-object pickup-use camera and/or a subject-use camera and calculating relative positional relationships among said plurality of orientation-detecting-use reference objects based upon said images;

(b) picking up an image of at least one of said plurality of orientation-detecting-use reference objects by using said reference-object pickup-use camera in synchronism with an image-pickup operation of said subject-use camera;

(c) calculating an orientation of said measuring device, based upon images of said orientation-detecting-use reference objects picked up by said reference-object pickup-use camera during a measuring process; and

(d) carrying out a predetermined measuring calculation based upon an image picked up by said subject-use camera and an orientation thereof at that time.

51. A method for measuring a subject, comprising the steps of:

(a) placing a plurality of orientation-detecting-use reference objects having a known shape on a periphery of said subject;

(b) finding a relative positional relationship among said plurality of orientation-detecting-use reference objects from images obtained by image-picking up said plurality of orientation-detecting-use reference objects;

(c) specifying a proximate reference object that is closest to a movable camera that is designed so as to alter an image-pickup orientation, among said plurality of orientation-detecting-use reference objects, based upon said relative positional relationship;

(d) allowing said movable camera to automatically trace said proximate reference object when a measuring operation of said subject is carried out; and

(e) picking up an image of said proximate reference object by said movable camera in synchronism with said measuring operation.

52. The method according to claim 51, wherein said step (d) includes the step of:

(d-1) in an image obtained by said movable camera, altering an image-pickup orientation of said movable camera so as to direct an image component of a marker placed at a predetermined position of each of said plurality of orientation-detection-use reference objects to a predetermined position in an image so that said movable camera is allowed to automatically trace said proximate reference object.

53. The method according to claim 51, further comprising the step of:

(f) generating measured data obtained upon measuring said subject based upon a relative position and a relative orientation between said movable camera and said proximate reference object and said relative positional relationship between said proximate reference object and another orientation-detection-use reference object.

54. A method of acquiring calibration-use parameters of a camera, comprising the steps of:

(a) placing a camera-calibration-use three-dimensional chart in a space to accommodate a subject;

(b) observing at least one chart of a plurality of charts contained in said camera-calibration-use three-dimensional chart from an observing position related to said camera;

(c) extracting at least one unit graphic form contained in an observed chart as a target unit graphic form, and identifying said target unit graphic form among said plurality of charts as well as a plurality of unit graphic forms; and

(d) specifying a calibration-use parameter of said camera that depend on a relative position and a relative orientation between said observing position and said camera-calibration-use three-dimensional chart, based upon a relationship between an actual size that has been preliminarily specified with respect to said target unit graphic form identified and an observed size of said target unit graphic form viewed from said observing position.

55. A method of acquiring calibration-use parameters of a measuring device, comprising the steps of:

(a) placing a plurality of measuring-device-calibration-use reference objects that have same external features in a space to accommodate a subject in a dispersed manner with different orientations so as to be distinguished in mutual orientations from external features independent of observing positions;

(b) observing at least one measuring-device-calibration-use reference object of said plurality of measuring-device-calibration-use reference objects from an observing position related to said measuring device;

(c) specifying a relative position and a relative orientation between said at least one measuring-device-calibration-use reference object and said measuring device based upon images obtained from said step (b);

(d) detecting an orientation of said measuring device in a space to accommodate said subject by using an angle sensor attached to said measuring device; and

(e) identifying said measuring-device-calibration-use reference object observed by said step (b) by collating a detected orientation of said measuring device within said space and an orientation of each of said plurality of measuring-device-calibration-use reference objects within said space that have been preliminarily found.

56. The method according to claim 55, wherein each of said plurality of measuring-device-calibration-use reference objects has an external appearance element indicating a direction.

57. The method according to claim 56, wherein respective orientations of said plurality of measuring-device-calibration-use reference objects are determined by using said external appearance element indicating said direction.

58. The method according to claim 56, wherein with respect to said plurality of measuring-device-calibration-use reference objects, directions indicated by external appearance elements are not in parallel with each other.

59. The method according to claim 58, wherein with respect to said plurality of measuring-device-calibration-use reference objects, said directions indicated by said external appearance elements are set so as to intersect each other at one point.

60. A computer software product for measuring a subject from an image obtained by picking up an image of said subject in a state where a plurality of orientation-detection-use reference objects having a known shape are placed on a periphery of said subject, allowing a computer to execute the steps of:

(a) finding relative positional relationships among said plurality of orientation-detecting-use reference objects, based upon images obtained by picking up said plurality of orientation-detecting-use reference objects by a reference-object pickup-use camera; and

(b) calculating an image-pickup orientation during said image-pickup operation, based upon images of said orientation-detecting-use reference objects picked up by said reference-object pickup-use camera in synchronism with an image-pickup operation of said subject.

61. A computer software product for measuring a subject from an image obtained by picking up an image of said subject in a state where a plurality of orientation-detection-use reference objects having a known shape are placed on a periphery of said subject, allowing a computer to execute the steps of:

(a) finding relative positional relationships among said plurality of orientation-detecting-use reference objects, based upon images obtained by picking up said plurality of orientation-detecting-use reference objects by a reference-object pickup-use camera and/or a subject-use camera; and

(b) calculating an orientation of said subject-use camera during an image-pickup operation, based upon images of said orientation-detecting-use reference objects picked up by said reference-object pickup-use camera in synchronism with said image-pickup operation of said subject by said subject-use camera, and

(c) carrying out a predetermined measuring calculation based upon an image picked up by said subject-use camera and an orientation of said subject-use camera.

62. A computer software product for finding a camera-calibration-use parameter from an image obtained by observing at least one chart from an observing position related to a camera among a plurality of charts that are included in camera- calibration-use three-dimensional charts that are placed within a space for accommodating a subject, allowing a computer to execute the steps of:

(a) extracting at least one unit graphic form contained in an observed chart as a target unit graphic form, and identifying said target unit graphic form among said plurality of charts as well as a plurality of unit graphic forms; and

(b) specifying calibration-use parameters of said camera that depend on a relative position and a relative orientation between said observing position and said camera-calibration-use three-dimensional chart, based upon a relationship between an actual size that has been preliminarily specified with respect to said target unit graphic form identified and an observed size of said target unit graphic form viewed from said observing position.

63. A computer software product for finding a measuring-device-calibration-use parameter from an image obtained by observing at least one measuring-device-calibration-use reference object from an observing position related to the measuring device among a plurality of measuring-device-calibration-use reference objects that have same external appearance feature, and are placed in a dispersed manner with different orientations so as to be distinguished in mutual orientations based upon external appearance features independent of observing positions, allowing a computer to execute the steps of:

(a) specifying a relative position and a relative orientation between said at least one measuring-device-calibration-use reference object and said measuring device based upon images obtained from observed images;

(b) detecting an orientation of said measuring device in a space in which said plurality of measuring-device-calibration-use reference objects are placed in a dispersed manner from results of detection by an angle sensor attached to said measuring device; and

(c) identifying said measuring-device-calibration-use reference object obtained by an observation by collating a detected orientation of said measuring device and an orientation of each of said plurality of measuring-device-calibration-use reference objects within said space that have been preliminarily found.