Sensor assisted head mounted displays for welding

• US 10,448,692 B2
• Filed: 03/04/2016
• Issued: 10/22/2019
• Est. Priority Date: 03/06/2015
• Status: Active Grant

• Current Assignee: Illinois Tool Works Inc.
• Original Assignee: Illinois Tool Works Inc.
• Inventors: Hsu, Christopher
• Primary Examiner(s): Hoang, Phi
• Application Number:
  US15/061,360
• Publication Number:
  US 20160260261A1
• Time in Patent Office: 1,327 Days
• CPC Class Codes: A42B 3/225 with full face protection, e.g. for industrial safety applications
• CPC Class Codes: A61F 9/06 Masks, shields or hoods for welders safety devices for welding in general F16P1/00
• CPC Class Codes: A61F 9/067 with variable transmission
• CPC Class Codes: B23K 9/0953 using computing means
• CPC Class Codes: B23K 9/0956 using sensing means, e.g. optical
• CPC Class Codes: B23K 9/1087 Arc welding using remote control
• CPC Class Codes: B23K 9/173 and of a consumable electrode
• CPC Class Codes: B23K 9/32 Accessories earthing connections H01R
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 3/40 : Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

Sensor assisted head mounted displays for welding are disclosed. Disclosed example head mounted devices include an optical sensor, an augmented reality controller, a graphics processing unit, and a semi-transparent display. The optical sensor collects an image of a weld environment. The augmented reality controller determines a simulated object to be presented in a field of view, a position in the field of view, and a perspective of the simulated object in the field of view. The graphics processing unit renders the simulated object based on the perspective to represent the simulated object being present in the field of view and in the weld environment. The display presents the rendered simulated object within the field of view based on the position. At least a portion of the weld environment is observable through the display and the lens when the display is presenting the rendered simulated object.

Claims

1. A welding interface device, comprising:

an optical sensor to collect an image of a weld environment comprising;

an illuminator to output a radiation at a first wavelength outside of an arc radiation spectrum;

a time-of-flight sensor to collect the image of the weld environment at the first wavelength; and

a bandpass filter to mitigate light at wavelengths other than the first wavelength;

an augmented reality controller to, based on the image of the weld environment and first instructions that correspond to a weld operation in the weld environment, determine a simulated object to be presented in a field of view, a position of the simulated object in the field of view, and a perspective of the simulated object in the field of view;

a graphics processing unit to render the simulated object as a three-dimensional stereographic image based on the perspective to represent the simulated object being present in the field of view and in the weld environment;

a semi-transparent near-eye display to present the rendered simulated object within the field of view based on the position determined by the augmented reality controller, at least a portion of the weld environment being observable through the semi-transparent display when the display is presenting the rendered simulated object; and

a body, the optical sensor, the graphics processing unit, and the display being attached to the body, the body dimensioned to, when worn by a wearer, enable the wearer to further wear a welding helmet over the body, the welding helmet having a viewing port;

wherein the augmented reality controller is to;

based on additional images collected by the optical sensor, determine an interaction by a wearer of the welding user interface device with the simulated object; and

determine a response to the interaction with the simulated object by the wearer.

2. The welding user interface device as defined in claim 1, further comprising a lens attached to the body, the lens to reduce an intensity of light occurring in the weld environment, the lens being arranged to provide the field of view to a wearer of the welding user interface device when the welding user interface device is worn by the wearer, the display being positioned between the lens and the wearer of the welding user interface device when worn by the wearer.

3. The welding user interface device as defined in claim 1, wherein the augmented reality controller is to render a view of the simulated object when viewing of a second object present in the weld environment is obscured to the field of view.

4. The welding user interface device as defined in claim 1, wherein the augmented reality controller is to generate the simulated object to identify a next weld to be performed based on a current weld being performed and a specified weld sequence.

5. The welding user interface device as defined in claim 1, wherein the simulated object includes at least one of a weld pool, a finished weld bead, an electrode, an electrical arc, a laser path, a shielding gas path, a powder path, a weld tool, a weld bead, a weld joint, a weld fixture, a weldment or parts to be fit assembled onto the weldmk.mt after welding.

6. The welding user interface device as defined in claim 1, wherein the simulated object comprises an electrical arc, the graphics processing unit to render the simulated object to be at least partially opaque and the position corresponding to a portion of the field of view in which an actual electrical arc is present in the weld environment, the display to display the rendered simulated object to reduce an intensity of light from the actual electrical arc that is observable to a wearer of the welding user interface device.

7. The welding user interface device as defined in claim 1, wherein the graphics processing unit is to render a zoomed view of a portion of the field of view, the augmented reality controller to determine the portion of the field of view based on at least one of the image of the weld environment or input from a gaze tracker.

8. The welding user interface device as defined in claim 7, wherein the augmented reality controller determines the position of the zoomed view of the portion of the field of view as a corner of the field of view.

9. The welding user interface device as defined in claim 1, wherein the optical sensor is to collect a second image of the weld environment, the augmented reality controller is to update the position and the perspective of the simulated object based on the second image of the weld environment, the graphics processing unit to render the simulated object based on the update to the position and the perspective by the augmented reality controller.

10. The welding user interface device as defined in claim 1, further including a communications controller to communicate a disable command in response to identifying a deviation from the first instructions, the disable command to cause at least one of a welding torch, a welding power source, or a welding wire feeder to be disabled.

11. The welding user interface device as defined in claim 1, wherein the augmented reality controller is to determine a status of a weld being performed and compare the status to the first instructions, the augmented reality controller to determine the position and the perspective of the simulated object based on the comparison.

12. The welding user interface device as defined in claim 1, wherein the image of the weld environment is a three-dimensional depth map.

13. The welding user interface device as defined in claim 1, further comprising a communications interface to receive data representative of a weld being performed, the graphics processing unit to render a graphic representing the data and the display to present the graphic.

14. The welding user interface device as defined in claim 1, wherein the response to the interaction by the wearer comprises:

the augmented reality controller to update the position and the perspective of the simulated object; and

the graphics processing unit to render the simulated object based on the update to the position and the perspective by the augmented reality controller.

15. The welding user interface device as defined in claim 1, wherein the response to the interaction by the wearer comprises:

the augmented reality controller to determine a second simulated object to be presented in the field of view, a position of the second simulated object in the field of view, and a perspective of the second simulated object in the field of view; and

the graphics processing unit to render the second simulated object as a three-dimensional stereographic image based on the perspective to represent the second simulated object being present in the field of view and in the weld environment.

16. The welding user interface device as defined in claim 1, wherein the interaction by a wearer of the welding user interface device with the simulated object comprises a gesture.

17. The welding user interface device as defined in claim 1, wherein the viewing port of the welding helmet comprises an auto-darkening lens, and wherein the augmented reality controller is configured to determine the simulated object to be presented in the field of view through the auto-darkening lens, and wherein the optical sensor is configured to collect images of the weld environment through the auto-darkening lens.

18. The welding user interface device as defined in claim 1, further comprising an inertial measurement unit to provide at least one of movement information or orientation information corresponding to the field of view, the augmented reality controller to determine at least one of the position of the simulated object or the perspective of the simulated object based on the at least one of the movement information or the orientation information.

19. The welding user interface device as defined in claim 1, wherein the first instructions correspond to an electrical arc being present in the field of view, the augmented reality controller to determine the simulated object to he at least one of an electrode, a torch, a weld puddle, a weld bead, or a seam to be welded based on the electrical arc reducing visibility of the weld environment.

20. The welding user interface device as defined in claim 1, wherein the first instructions correspond to a portion of a subsequent welding operation that is to be performed after a current welding operation is complete, the augmented reality controller to determine the simulated object to correspond to a location of the subsequent welding operation on a workpiece.