QUALITY ASSURED MANUFACTURING

• US 20150253766A1
• Filed: 03/05/2015
• Published: 09/10/2015
• Est. Priority Date: 03/06/2014
• Status: Abandoned Application

• Current Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Original Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Inventors: PETTERSSON, Bo, SIERCKS, Knut, ZEBHAUSER, Benedikt
• Application Number:
  US14/639,889
• Publication Number:
  US 20150253766A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G05B 19/41805 characterised by assembly
• CPC Class Codes: G05B 19/4183 characterised by data acquisition, e.g. workpiece identification
• CPC Class Codes: G05B 19/4187 by tool management
• CPC Class Codes: G05B 2219/31027 Computer assisted manual assembly CAA, display operation, tool, result
• CPC Class Codes: G05B 2219/31304 Identification of workpiece and data for control, inspection, safety, calibration
• CPC Class Codes: G05B 2219/37567 3-D vision, stereo vision, with two cameras
• CPC Class Codes: G05B 2219/40557 Tracking a tool, compute 3-D position relative to camera
• CPC Class Codes: G05B 2219/45127 Portable, hand drill
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

The invention relates to a tool location system for a handheld tool in a manufacturing environment, wherein the tool is equipped with a camera for capturing images. According to the invention, the location system comprises numerical evaluator means, which is built to determine a tool-location of the tool by a simultaneous location and mapping (SLAM) navigation calculation based on the images from the camera. Therein the tool-location is determined with respect to a global coordinate system in relation to the manufacturing environment from portions of the images which comprise a view of the manufacturing environment and the tool-location is determined with respect to a local coordinate system in relation to the workpiece from portions of the images which comprise a view of the workpiece.

Claims

1. A tool location system for a handheld tool in a manufacturing environment where a workpiece is manufactured, the tool comprising:

a camera for capturing images; and

a numerical evaluator means configured to determine a tool-location of the tool by a simultaneous location and mapping (SLAM) navigation calculation based on the images from the camera;

wherein the tool-location is determined with respect to a global coordinate system in relation to the manufacturing environment from portions of the images which comprise a view of the manufacturing environment, and

wherein the tool-location is determined with respect to a local coordinate system in relation to the workpiece from portions of the images which comprise a view of the workpiece,

2. The system according to claim 1, wherein the multiple manufacturing stations is located within a car body, an airplane, or a machinery production.

3. The system according to claim 1, wherein the local coordinate system is moving with respect to the global coordinate system in an assembly line arrangement.

4. The system according to claim 1, wherein:

the determination of the tool-location involves a matching of the SLAM navigation with known spatial information of the workpiece and/or the environment, of spatial information in form of CAD or pointcloud data.

5. The system according to claim 1, wherein:

the evaluator means is configured to establish a seamless transition of the tool-location between the local coordinate system and the global coordinate system, with a live-updated referencing of the local coordinate system with respect to the global coordinate system.

6. The system according to claim 1, wherein:

the tool comprise an inertial measurement unit (IMU) and the determination of the tool-location involves information gathered from the IMU used in an inertial navigation algorithm.

7. The system according to claim 1, wherein:

the manufacturing environment and/or the workpiece comprises multiple cooperative and or actively light emitting targets built to be identified in the camera image, contrast faces, reflectors or LEDs.

8. The system according to claim 1, wherein:

the tool comprises a communication interface, which is built to exchange its localization and/or image information with a manufacturing management system and/or other tools.

9. The system according to claim 8, wherein

the tool comprises a sensor means for sensing a work process parameter of a tools operation, and

the work process parameter is provided together with the corresponding tool-location as tool-surveillance data by the communication interface, wherein the tool-surveillance data comprises an image from the camera depicting a work process result of the tools operation.

10. The system according to claim 1, wherein:

the tool is a mounting tool selected from the list consisting of a screwdriver tool, a wrench tool, a clamping tool, and a rivet-tool, driven by electricity or compressed air, and

the camera has a field of view comprising at least a part of an operating range of the tool.

11. The system according to claim 1, wherein:

the workpiece is movable and is also equipped with a camera for SLAM navigation of the workpiece-location in the global coordinate system, wherein the workpiece-location and the tool-location are combined, wherein a referencing of the global coordinate system and the local coordinate system is established,

wherein a back-referencing between the tool and the workpiece navigation is established by the location of the tool with the camera at the workpiece and the location of the workpiece with the camera at the tool.

12. A tool surveillance system for a manufacturing environment, wherein the manufacturing environment comprises stationary items and movable items, wherein the tool surveillance system comprises:

a workpiece to be manufactured and/or parts thereof,

a tool to be handled by a human worker, and

a tool location system according to claim 1 for the tool.

13. The tool surveillance system claim 12, wherein the manufacturing environment is equipped with:

multiple global cameras for imaging the manufacturing environment, and

an image processing means built to identify/detect the stationary and movable items in the manufacturing environment and to determine their global positional coordinates in a global environment coordinate system, in at least three dimensions, and

wherein the global positional coordinates are matched with the determined tool-location, wherein a back-referencing between the tool and the environment navigation is established by the location of the tool with the camera at the environment and the location of the environment with the camera at the tool.

17. The method according to claim 12, wherein

a sensing of a work process parameter by a sensor means at the tool and providing the tool-location together with the corresponding work process parameter as tool-surveillance data by a communication interface, with providing the surveillance data to a manufacturing management system for managing the manufacturing and/or documenting the manufacturing of the workpiece, together with an image from the camera of a work process result at the corresponding tool-location.

14. A method for locating a handheld tool in a manufacturing environment where a workpiece is manufactured, wherein the tool is equipped with a camera for capturing images, the method comprising:

determining a tool-location of the tool by a SLAM navigation calculation based on the images from the camera, that further comprises;

determining the tool-location with respect to a global coordinate system in relation to the manufacturing environment from portions of the image which comprise a view of the manufacturing environment, and

determining the tool-location with respect to a local coordinate system in relation to the workpiece from portions of the image which comprise a view of the workpiece;

wherein the local coordinate system is moving with respect to the global coordinate system.

15. The method according to claim 14, wherein:

in an environment with multiple manufacturing stations of a car body, airplane or machinery production, in an assembly line arrangement,

16. The method according to claim 14, wherein:

determining the tool-location involves a matching of the SLAM navigation with known spatial information of the workpiece and/or the environment, of spatial information in form of CAD or pointcloud data.

18. One or more non-transitory computer-readable media storing one or more programs that are configured, when executed, to cause one or more processors to execute a SLAM navigation algorithm based on images from a camera at a tool in a manufacturing environment in which a workpiece is manufactured, with a moving workpiece in an assembly line configuration, wherein a location of the tool is established:

with respect to a global coordinate system relative to the environment by portions of the images depicting the environment and also

with respect to a local coordinate system relative to the workpiece by portions of the image depicting the workpiece.