Computer aided work support system and method

• US 5,647,009 A
• Filed: 10/19/1993
• Issued: 07/08/1997
• Est. Priority Date: 10/20/1992
• Status: Expired due to Term

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Kamejima, Kohji, Aoki, Toshiyuki, Hamada, Tomoyuki
• Primary Examiner(s): Couso, Jose L.
• Application Number:
  US08/137,973
• Time in Patent Office: 1,358 Days
• Field of Search: 382/1-8, 382/45, 382/46, 382/47, 382/57, 364/512, 364/550, 348/86, 348/94, 348/95, 395/135, 395/137, 395/138
• US Class Current: 382/100
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

A computer aided work support system having a measuring unit for measuring the position and orientation of the measuring unit itself; an image pickup unit for picking up an object to be measured; an unit for generating an estimated image of the object in accordance with information of the form, position, and orientation of the object, the position and orientation of the measuring unit, the position and orientation of the image pickup unit relative to the measuring unit, a plurality of image magnification factors, and the position of the optical axis of the image pickup unit on the screen of a display unit, the display unit displaying the picked-up image and estimated image of the object in a superposed manner at the same time; and a modifying unit for modifying the information of the previous positions and orientations of the object and image pickup unit in accordance with a displacement amount of the estimated image from the picked-up image.

Claims

1. A computer aided work support system comprising:

measuring means for measuring position and orientation of said measuring means in a coordinate system defined with respect to a reference point which is used to determine position and orientation of a second object to be measured and a position of a point on a reference first object which is used to estimate part of image pickup parameters;

image pickup means for picking up images of said first object and said second object, said image pickup means and said measuring means having a fixed spatial relationship therebetween;

storage means for storing information of form, position, and orientation of one of said first and second objects, and image pickup parameters such as position and orientation of said measuring means, relative position and orientation of said image pickup means to said measuring means, a plurality of image magnification factors, and a position of an optical axis of said image pickup means on a screen of display means;

input means for inputting the form, position, and orientation of one of said first and second objects into said storage means;

calculation means for calculating position and orientation of said measuring means, relative position and orientation of said image pickup means to said measuring means, and position of the optical axis of said image pickup means on the screen of said display means, respectively for each of said plurality of image magnification factors, in accordance with the relative position of said image pickup means to said measuring means;

means for generating an estimated image of one of said first and second objects in accordance with calculation results by said calculation means and said information stored in said storage means;

said display means for displaying an image picked up by said image pickup means and said estimated image in a superposed manner at the same time on the screen of said display means; and

means for detecting a displacement amount of said estimated image from said picked-up image displayed on said display means, converting said displacement amount into a change amount of the position and orientation of one of said second object and said image pickup means, and adding said change amount to data of the position and orientation of one of said second object and said image pickup means stored in said storage means.

2. A computer aided work support system according to claim 1, further comprising means for generating a work specification in accordance with the information of the form, position, and orientation of said second object stored in said storage means.

3. A computer aided work support system according to claim 1, further comprising:

working means for performing a work in accordance with inputted information;

means for determining operation information for said working means in accordance with the information of the form, position, and orientation of said second object stored in said storage means and an instruction from an operator; and

means for transferring said operation information to said working means.

4. A computer aided work support system according to claim 1, further comprising means for storing an image picked up by said image pickup means.

5. A computer aided work support system according to claim 1, further comprising:

cargo transport means for loading, unloading, and transporting a cargo;

means for determining operation information for said cargo transport means in accordance with the information of the form, position, and orientation of said second object stored in said storage means and an instruction from an operator; and

means for transferring said operation information to said cargo transport means.

6. A computer aided work support system comprising:

image pickup means for picking up the image of an object;

storage means for storing information of form, three-dimensional position, and three-dimensional orientation of said object, three-dimensional position and three-dimensional orientation of said image pickup means, an image magnification factor, and a position of an optical axis of said image pickup means on a screen of display means;

means for generating an estimated image of said object in accordance with information stored in said storage means;

said display means for displaying an image picked up by said image pickup means and said estimated image in a superposed manner at the same time on the screen of said display means; and

means for detecting a signal entered by an operator, converting said signal into a change amount of the three-dimensional position and the three-dimensional orientation of one of said object and said image pickup means, and adding said change amount to data of the three-dimensional position and the three-dimensional orientation of said object stored in said storage means.

7. A computer aided work support system comprising:

measuring means for measuring position and orientation of said measuring means in a coordinate system defined with respect to a reference point which is used to determine position and orientation of an object to be measured;

image pickup means for picking up an image of said object, said image pickup means and said measuring means having a fixed spatial relationship therebetween;

storage means for storing information of form, position, and orientation of said object, relative position and orientation of said image pickup means to said measuring means, an image magnification factor, and a position of an optical axis of said image pickup means on a screen of display means;

means for generating an estimated image of said object in accordance with information stored in said storage means;

said display means for displaying the image picked up by said image pickup means and said estimated image in a superposed manner at the same time on the screen of said display means; and

means for detecting a signal entered by an operator, converting said signal into a change amount of the position and orientation of one of said object and said image pickup means, and adding said change amount to data of the position and orientation of said object stored in said storage means.

8. A computer aided work support system comprising:

image pickup means for picking up an image of an object to be measured;

means for generating an estimated image of said object in accordance with position and orientation of said image pickup means, an image magnification factor, and a position of an optical axis of said image pickup means on a screen of display means;

storage means for storing information of position and orientation of said object, position and orientation of said image pickup means, said image magnification factor, and a position of an optical axis of said image pickup means on the screen of said display means;

information modifying means for modifying the position and orientation of said object stored in said storage means;

means for generating an edge pattern in accordance with an image picked up by said image pickup means;

means for calculating a potential in accordance with said edge pattern; and

means for calculating a correction amount for position and orientation of said object in accordance with said estimated image and said potential, and adding said correction amount to the position and orientation of said object stored in said storage means if said information modifying means has not modified information stored in said storage means.

9. A computer aided work support method comprising the steps of:

picking up an image of an object to be measured by using image pickup means;

measuring position and orientation of measuring means in a coordinate system defined with respect to a reference point which is used to determine position and orientation of said object at a time when said image is picked up by said image pickup means[,]by using said measuring means itself while maintaining a fixed spatial relationship between said image pickup means and said measuring means;

generating an estimated image of said object in accordance with design information of said object stored in advance in storage means, an image pickup parameter to be used for displaying said object on a screen of display means, and information of the position and orientation of said measuring means obtained at said measuring step;

displaying said estimated image and the image of said object picked up by said image pickup means on the screen of said display means in a superposed manner at the same time;

moving said estimated image by inputting a signal to information modifying means to superpose said estimated image upon the image of said object, and measuring position and orientation of said object; and

storing measured data of the position and orientation of said object in said storage means.

10. A computer aided work support method according to claim 9, further comprising the step of generating a work specification in accordance with said design information and the information of the position and orientation of said object obtained at said measuring step.

11. A computer aided work support method according to claim 9, further comprising the steps of:

determining operation information of working means in accordance with said design information and the position and orientation of said object obtained at said measuring step; and

performing a work by inputting said operation information to said working means.

12. A computer aided work support method according to claim 9, further comprising the step of using the measured data of the position and orientation of said object as the design data for an architecture.

13. A computer aided work support method according to claim 9, wherein said estimated image generating step includes a step of entering, as said image pickup parameter, information of relative position and orientation of said image pickup means to said measuring means, an image magnification factor of a whole image pickup system, and a position of an optical axis of said image pickup means on a screen of said display means.

14. A computer aided work support method according to claim 9, wherein said displaying step in a superposed manner includes a step of selecting a motion direction of said estimated image by using said information modifying means.

15. A computer aided work support method comprising the steps of:

displaying an image of an object picked up by image pickup means;

generating an estimated image of said object;

displaying said estimated image and said picked-up image on a screen of display means at the same time;

moving said estimated image by inputting a signal to information modifying means to superpose said estimated image upon said picked-up image; and

measuring three-dimensional position and three-dimensional orientation of said object from an amount of motion of said estimated image superposed on said picked-up image.