SYSTEM, OPERATION CELL, METHOD, PRODUCT MANUFACTURING METHOD, AND MARKER FOR LOCATING OPERATION POSITION

• US 20080177417A1
• Filed: 11/29/2007
• Published: 07/24/2008
• Est. Priority Date: 01/24/2007
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: KASUGA, Toshinori, YAMANISHI, Hirokazu
• Granted Patent: US 8,761,921 B2
• Time in Patent Office: Days
• US Class Current: 700/213
• CPC Class Codes: G05B 19/41805 characterised by assembly
• CPC Class Codes: G05B 19/41875 characterised by quality surveillance of production
• CPC Class Codes: G05B 2219/31027 Computer assisted manual assembly CAA, display operation, tool, result
• CPC Class Codes: G05B 2219/37567 3-D vision, stereo vision, with two cameras
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
• CPC Class Codes: Y10T 29/49764 with testing or indicating
• CPC Class Codes: Y10T 29/49769 Using optical instrument [excludes mere human eyeballing]

Abstract

An operation position locating system locates a position on a product where an operation is performed in a manufacturing process of the product including one or more operations to be performed. The system includes an operation completion coordinates detection unit and an operation position locating unit. The operation completion coordinates detection unit processes a plurality of images obtained by capturing from a plurality of viewpoints a range including an operation completion position where an operation is completed when the operation is completed, thereby detecting a set of operation completion coordinates representing the operation completion position. The operation position locating unit obtains a set of operation coordinates representing an operation position where the operation is to be performed on each operation, and locates the operation position corresponding to the operation completion position based on the set of operation completion coordinates and the one or more obtained sets of operation coordinates.

Claims

1. An operation position locating system which locates a position on a product where an operation is performed in a manufacturing process of the product including one or more operations to be performed using a tool, comprising:

an operation completion coordinates detection unit processing a plurality of images obtained by capturing from a plurality of viewpoints a range including an operation completion position as a position on which an operation is completed when the operation is completed, thereby detecting a set of operation completion coordinates representing the operation completion position; and

an operation position locating unit obtaining a set of operation coordinates representing an operation position as a position on which the operation is to be performed on each of the one or more operations, and locating the operation position corresponding to the operation completion position based on the set of operation completion coordinates and the one or more obtained sets of operation coordinates.

2. The operation position locating system according to claim 1, wherein

the operation completion coordinates detection unit detects a position and a direction of the tool, and calculates the set of operation completion coordinates based on the detected position and direction of the tool.

3. The operation position locating system according to claim 2, wherein

the position and the direction of the tool are detected by detecting a tool marker attached to the tool by image recognition.

4. The operation position locating system according to claim 3, wherein

the tool marker comprises a base unit and a plurality of projections extended from the base unit;

the base unit has high reflectance and the plurality of projections have low reflectance, or the base unit has low reflectance and the plurality of projections have high reflectance; and

each of the plurality of the projections takes a form of a substantially radial symmetry about an axis common among the plurality of projections when the tool marker is attached to the tool.

5. The operation position locating system according to claim 1, wherein

the operation completion coordinates are represented by a first coordinate system;

the operation coordinates are represented by a second coordinate system based on the product;

the operation position locating system further comprises a parameter determination unit detecting a position and direction represented by the first coordinate system of the product or an object having a predetermined positional relation with the product, thereby determining a parameter value required to coordinate transformation between the first coordinate system and the second coordinate system; and

the operation position locating unit transforms coordinates using the parameter to represent the operation completion position and the operation position by a same coordinate system, and locates the operation position corresponding to the operation completion position.

6. The operation position locating system according to claim 5, wherein

the coordinate transformation is a transformation from the first coordinate system to the second coordinate system.

7. The operation position locating system according to claim 5, wherein:

the operation is performed with the product fixed to an operation table at least one of whose position and direction can be adjusted, and which is provided with an operation table marker;

the range captured in the plurality of images includes the operation table;

the parameter determination unit detects a position and direction of the operation table represented by the first coordinate system by detecting the operation table marker on the plurality of images by image recognition, and determines the parameter value based on the position and direction of the operation table.

8. The operation position locating system according to claim 7, wherein:

the operation table marker has a base unit having an attachment surface to the operation table and a projection extending from the base unit;

the base unit has high reflectance and the projection has low reflectance, or the base unit has low reflectance and the projection has high reflectance;

a plurality of projections correspond to the operation table as a result of attaching a plurality of the operation table marker to the operation table, or as a result of providing a plurality of projections to the operation table marker; and

the position and the direction of the operation table are detected based on at least one of a difference in appearance of the plurality of projections, and an arrangement pattern of the plurality of projections.

9. The operation position locating system according to claim 1, further comprising

a display unit displaying operation information about the operation corresponding to the located operation position by using a character, a symbol, an image, sound, or a combination thereof.

10. The operation position locating system according to claim 1, wherein

process information indicating which operation of the operations to be performed has been completed is managed based on a result of locating the operation position.

11. The operation position locating system according to claim 1, wherein:

the operation is a screwing operation;

the tool is a torque driver capable of detecting that a specified torque has been reached;

the torque driver outputs a completion signal when a torque applied to the screw has reached the specified torque;

the operation completion coordinates detection unit receives the completion signal and detects the set of operation completion coordinates upon receipt of the completion signal.

12. The operation position locating system according to claim 1, wherein:

the operation is to be performed on a range having linear or planar expansion;

the operation completion coordinates detection unit receives an operation in-progress signal output one or more times during progress of the operation, and detects a set of operation in-progress coordinates indicating a position in which the operation is being performed when the operation in-progress signal is received; and

the operation position locating unit further locates the operation position corresponding to the set of operation in-progress coordinates.

13. An operation position locating system which locates a position on a product where an operation is performed in a manufacturing process of the product including one or more operations to be performed using a tool, comprising:

an operation completion coordinates detection unit detecting a set of operation completion coordinates in which an operation completion position as a position where the operation is completed is represented by a first coordinate system;

a parameter determination unit detecting a position and direction represented by the first coordinate system of the product or an object having a predetermined positional relation with the product, thereby determining a parameter value required to coordinate transformation between the first coordinate system and a second coordinate system based on the product;

an operation position locating unit obtaining a set of operation coordinates in which an operation position as a position where the operation is to be performed is represented by the second coordinate system on each of the one or more operations, transforming coordinates using the parameter to represent the operation completion position and the operation position by a same coordinate system, and locating the operation position corresponding to the operation completion position.

14. The operation position locating system according to claim 13, wherein

the operation completion coordinates detection unit detects the operation completion coordinates using output of a sensor attached to the tool and capable of performing a three-dimensional measurement.

15. The operation position locating system according to claim 13, wherein

the parameter determination unit detects the position and the direction of the product or the object represented in the first coordinate system using a sensor capable of performing a three-dimensional measurement.

16. An operation cell corresponding to a process including one or more operations to be performed using a tool in a manufacturing process of a product including one or more processes, comprising:

an operation completion coordinates detection unit processing a plurality of images obtained by capturing from a plurality of viewpoints a range including an operation completion position as a position on which an operation is completed when the operation is completed, thereby detecting a set of operation completion coordinates representing the operation completion position; and

an operation position locating unit obtaining a set of operation coordinates representing an operation position as a position on which the operation is to be performed on each of the one or more operations, and locating the operation position corresponding to the operation completion position based on the set of operation completion coordinates and the one or more obtained sets of operation coordinates.

17. A method performed in a manufacturing process of a product, the manufacturing process including one or more operations to be performed using a tool, comprising:

processing a plurality of images obtained by capturing from a plurality of viewpoints a range including an operation completion position as a position on which an operation is completed when the operation is completed;

detecting a set of operation completion coordinates representing the operation completion position based on a result of processing the plurality of images;

obtaining a set of operation coordinates representing an operation position as a position on which the operation is to be performed on each of the one or more operations; and

locating the operation position corresponding to the operation completion position based on the set of operation completion coordinates and the one or more obtained sets of operation coordinates.

18. The method according to claim 17, wherein

the product is manufactured in the manufacturing process while locating each operation position for each operation.

19. A computer-readable storage medium storing a program used to direct a computer to locate a position on a product where an operation is performed in a manufacturing process of the product including one or more operations to be performed using a tool, comprising:

a step of processing a plurality of images obtained by capturing from a plurality of viewpoints a range including an operation completion position as a position on which an operation is completed when the operation is completed, thereby detecting a set of operation completion coordinates representing the operation completion position;

a step of obtaining a set of operation coordinates representing an operation position as a position on which the operation is to be performed on each of the one or more operations; and

a step of locating the operation position corresponding to the operation completion position based on the set of operation completion coordinates and the one or more obtained sets of operation coordinates.