Time of flight camera for welding machine vision

• US 10,363,632 B2
• Filed: 06/24/2015
• Issued: 07/30/2019
• Est. Priority Date: 06/24/2015
• Status: Active Grant

• Current Assignee: Illinois Tool Works Inc.
• Original Assignee: Illinois Tool Works Inc.
• Inventors: Hsu, Christopher
• Primary Examiner(s): Evans, Geoffrey S
• Application Number:
  US14/749,057
• Publication Number:
  US 20160375524A1
• Time in Patent Office: 1,497 Days
• CPC Class Codes: B23K 26/03 Observing, e.g. monitoring, the workpiece
• CPC Class Codes: B23K 26/044 Seam tracking
• CPC Class Codes: B23K 31/02 relating to soldering or welding dip or wave soldering in the manufacture of printed circuits H05K3/34
• CPC Class Codes: B23K 37/00 Auxiliary devices or processes, not specially adapted to a procedure covered by only one of the preceding main groups eye-shields for welders worn on the operator's body or carried in the hand A61F9/00 , i.e. A61F9/02; applicable to metal-working machines other than soldering, welding, or flame-cutting machines B23Q; laser protective screens B23K26/706; protective shields for other welding methods F16P1/06
• CPC Class Codes: B23K 9/095 Monitoring or automatic control of welding parameters
• CPC Class Codes: B23K 9/127 Means for tracking lines during arc welding or cutting copying in general B23Q35/00

Abstract

A machine-vision-assisted welding system comprises welding equipment, a time of Flight (ToF) camera operable to generate a three-dimensional depth map of a welding scene, digital image processing circuitry operable to extract welding information from the 3D depth map, and circuitry operable to control a function of the welding equipment based on the extracted welding information. The welding equipment may comprise, for example, arc welding equipment that forms an arc during a welding operation, and a light source of the ToF camera may emit light whose spectrum comprises a peak that is centered at a first wavelength, wherein the first wavelength is selected such that a power of the peak is at least a threshold amount above a power of light from the arc at the first wavelength.

Claims

1. A machine-vision-assisted welding system comprising:

welding equipment;

a first Time of Flight (ToF) camera operable to generate a three-dimensional (3D) depth map of a welding scene, wherein the first ToF camera is configured to capture a 3D map of an actual bead stack up;

digital image processing circuitry operable to extract welding information from said 3D depth map and compare the 3D map of the actual bead stack-up to a desired weld geometry; and

circuitry operable to control a function of said welding equipment based on said extracted welding information and said comparison.

2. The machine-vision-assisted welding system of claim 1, wherein said welding equipment comprises an automation device.

3. The machine-vision-assisted welding system of claim 1, wherein said welding equipment is operated by a human operator.

4. The machine-vision-assisted welding system of claim 1, comprising:

an image sensor comprising a plurality of pixels, each of which is operable to convert light energy incident on it to an electrical signal; and

circuitry operable to, for each one of said pixels, measure time required for light to travel from a light source of said ToF camera to said welding scene and back to said one of said pixels.

5. The machine-vision-assisted welding system of claim 1, wherein:

said welding equipment comprises arc welding equipment that forms an arc during a welding operation; and

a light source of said first ToF camera is configured to emit light whose spectrum comprises a peak that is centered at a first wavelength, wherein said first wavelength is selected such that a power of said peak, as measured at a particular pixel of an imager of said first ToF camera after reflection off of said welding scene, is at least a threshold amount above a power of light from said arc at said first wavelength, as measured at said particular pixel of said imager of said first ToF camera.

6. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information characterizes activity of one or both of a human and a machine in said welding scene; and

said function comprises activity tracking, cycle time improvement, and workflow streamlining.

7. The machine-vision-assisted welding system of claim 6, wherein said activity comprises one or more of:

welding operations, pre-welding operations and post-welding operations.

8. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information comprises activity of one or both of a human and a machine in said welding scene; and

said function comprises real-time assessment of risk of personal injury.

9. The machine-vision-assisted welding system of claim 1, wherein said digital image processing circuitry uses data from sources other than said first ToF camera to interpret said 3D depth map.

10. The machine-vision-assisted welding system of claim 1, comprising a second ToF camera, wherein said first ToF camera and said second ToF camera differ in one or more of:

field of focus, mounting location, wavelength of light used, and targets viewed.

11. The machine-vision-assisted welding system of claim 1, comprising a non-ToF camera configured to capture visible-spectrum images, wherein said information from said depth map and visible-spectrum images are combined to provide an augmented reality user interface.

12. The machine-vision-assisted welding system of claim 1, comprising a non-ToF camera configured to capture infrared images, and being configured to use said infrared images to provide a temperature overlay on said 3D depth map.

13. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information comprises identification of one or more of;

welding equipment, welding consumables, and welding operator; and

said function comprises one or more of;

verification of compliance with a welding procedure specification (WPS), welder performance qualification (WPQ), procedure qualification record (PQR), and generation of alerts for error prevention.

14. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information identifies a sequence of welds being made on a workpiece in said welding scene; and

said function comprises one or both of;

indication, via a user interface of said welding equipment, of a location of a next weld in said sequence; and

one or both of generation of an alert and triggering of a lock out for prevention of wrong weld sequence.

15. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information comprises frequency and/or amplitude of surface oscillations of a weld puddle; and

said function comprises controlling said welding equipment based on said frequency and/or amplitude of said surface oscillations.

16. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information comprises joint fit-up condition of a workpiece in said welding scene; and

said function comprises adaptation of welding parameters used by said welding equipment based on said fit-up condition.

17. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information comprises actual joint seam location; and

said function comprises guiding a robot of said welding equipment to ensure a welding tool center point follows said joint seam.

18. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information indicates maintenance condition of said welding equipment; and

said function comprises one or both of;

preventative maintenance service; and

condition-based maintenance service tracking and alert generation.

19. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information indicates remaining stock of welding consumables; and

said function comprises generation of an alert to replenish a supply of said welding consumables.

20. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information indicates an identity and position of a welding workpiece in said welding scene; and

said function comprises automatic selection of a weld program to be used for welding said workpiece, based on said identity and position of said welding workpiece.

21. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information indicates a position and orientation of a welding torch in said welding scene; and

said function comprises automatic generation of a weld program to be used by an automated robot for welding said workpiece.

22. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information indicates an identity and position of a welding workpiece in said welding scene; and

said function comprises automatic selection of a weld program to be used for welding said workpiece, based on said identity and position of said welding workpiece.

23. The machine-vision-assisted welding system of claim 1, wherein:

said extracted welding information indicates the dimensions of previous welds in a multi-pass weld or clad of the welding workpiece in said welding scene; and

said function comprises automatic planning of the remaining one or more passes to be used for multi-pass welding of the said workpiece, based on a previous one or more welding passes on said welding workpiece.

24. The machine-vision-assisted welding system of claim 1, wherein:

said welding information indicates hand gestures made by a worker in said welding scene; and

said function comprises controlling said welding equipment based on said hand gestures.

25. The machine-vision-assisted welding system of claim 1, wherein said function comprises acquisition of 3D depth data of said welding scene for machine learning and predictive analytics.

26. The machine-vision-assisted welding system of claim 1, wherein:

said welding information comprises one or more of;

weld presence, weld size, weld length, weld location, workpiece distortion, and weld appearance, and

said function comprises post-welding visual inspection of weld quality.