System and method providing arc welding training in a real-time simulated virtual reality environment using real-time weld puddle feedback

• US 8,747,116 B2
• Filed: 07/10/2009
• Issued: 06/10/2014
• Est. Priority Date: 08/21/2008
• Status: Active Grant

• Current Assignee: Lincoln Global Incorporated (Lincoln Electric Holdings Incorporated)
• Original Assignee: Lincoln Global Incorporated (Lincoln Electric Holdings Incorporated)
• Inventors: Zboray, David Anthony, Bennett, Matthew Alan, Wallace, Matthew Wayne, Hennessey, Jeremiah, Lenker, Zachary Steven, Lundell, Andrew Paul, Dudac, Yvette Christine, Dana, Paul, Preisz, Eric A.
• Primary Examiner(s): Gishnock, Nikolai A
• Assistant Examiner(s): CRENWELGE, RANDALL NEAL
• Application Number:
  US12/501,257
• Publication Number:
  US 20100062405A1
• Time in Patent Office: 1,796 Days
• Field of Search: 35/9, 35/13, 35/37, 434/234, 434/260, 396/374, 219/130.21, 706/45
• US Class Current: 434/260
• CPC Class Codes: G09B 19/24 Use of tools

Abstract

A real-time virtual reality welding system including a programmable processor-based subsystem, a spatial tracker operatively connected to the programmable processor-based subsystem, at least one mock welding tool capable of being spatially tracked by the spatial tracker, and at least one display device operatively connected to the programmable processor-based subsystem. The system is capable of simulating, in virtual reality space, a weld puddle having real-time molten metal fluidity and heat dissipation characteristics. The system is further capable of displaying the simulated weld puddle on the display device in real-time.

Claims

1. A virtual reality welding system comprising:

a programmable processor-based subsystem;

a spatial tracker operatively connected to said programmable processor-based subsystem;

at least one mock welding tool capable of being spatially tracked by said spatial tracker, said spatial tracker comprising a tactilely resistive tip to simulate resistive feedback that occurs during welding thereby driving feedback for use in adjusting or maintaining welding procedure, said mock welding tool emitting simulated particles onto a surface of a texture map; and

at least one display device operatively connected to said programmable processor-based subsystem, said display device providing visual cues during welding to provide immediate feedback by displaying overlaid colors or indicators for each tracked parameter based upon predefined limits or tolerances for said tracked parameter, wherein said system simulates, in a virtual reality space, a weld puddle having real-time molten metal fluidity and heat dissipation characteristics, and displaying said simulated weld puddle on said at least one display device in real-time, said virtual reality space comprising a welding element displacement map grid, in which said map grid defines surface resolution with changeable channel parameter values assigned to each portion of said grid, said values dynamically changing in real-time during a simulated welding process, said real-time molten metal fluidity and heat dissipation characteristics of said simulated weld puddle provide real-time visual feedback to a user of said mock welding tool when displayed on said at least one display device, said at least one display device comprising a user-viewable face-mounted display device, allowing said user to adjust or maintain a welding technique in real-time in response to said real-time visual feedback, and wherein said real-time molten metal fluidity and heat dissipation characteristics of said simulated weld puddle are illustrated on said texture map comprising channels of information which comprise at least a first channel which stores a displacement value for any liquefied metal at a wexel location;

a second channel which stores a value giving a magnitude of heat at the wexel location;

a third channel which stores a displacement value for a solidified metal at the wexel location; and

a fourth channel which stores logical information about the wexel, including a representation regarding slag at the wexel location, and wherein when said particles make contact with the surface of the texture map, said particles interact with each other and the texture map and build up in real-time, wherein more heat is added the nearer the wexel is to an emitter and heat is modeled in dependence on distance and an amount of time that heat is input from the emitter, and said weld puddle is rendered in virtual reality space for wexels having enough heat as wherever said heat exceeds a threshold value, the wexel at that wexel location liquefies, causing the displacement value for said liquefied metal to raise up for those wexel locations, and wherein said displacement is determined by sampling the highest particles at each wexel location, and further wherein as the emitter moves on along the weld trajectory, the wexel locations left behind cool with heat removed from wexel locations at a defined rate, wherein when a cooling threshold is reached, the wexel at that wexel location solidifies.

2. The system of claim 1 wherein

said at least one display device includes at least one user-viewable face-mounted display device capable of being spatially tracked by said spatial tracker.

3. The system of claim 1 wherein

said programmable processor-based subsystem includes at least one graphic processing unit (GPU).

4. The system of claim 1 wherein

said simulated weld puddle comprises, in said virtual reality space, a plurality of temporary virtual dynamic fluid particles overlaying a portion of a virtual surface displacement map.

5. The system of claim 1 wherein

said programmable processor-based subsystem supports implementation of data parallel algorithms.

6. The system of claim 1 further comprising

a welding user interface simulating a real-world welding power source user interface in real time.

7. The system of claim 3 wherein

said real-time molten metal fluidity and heat dissipation characteristics of said simulated weld puddle are generated by a physics model operating on said at least one GPU.

8. The system of claim 1 wherein

said system simulates in said virtual reality space, a weld bead having a real-time weld bead wake characteristic resulting from a real-time fluidity-to-solidification transition of said simulated weld puddle as said simulated weld puddle is moved, and displaying said simulated weld bead on said at least one display device.

9. The system of claim 1 further comprising a welding coupon having at least one surface and simulating a real-world part to be welded, wherein said at least one surface of said welding coupon is simulated in said virtual reality space as a double displacement layer including a solid displacement layer and a puddle displacement layer, wherein said puddle displacement layer is capable of modifying said solid displacement layer.

10. A method of training using a virtual reality welding system, said method comprising:

moving a first mock welding tool with respect to a welding coupon in accordance with a first welding technique, said first mock welding tool emitting simulated particles onto a surface of a texture map;

tracking said first mock welding tool in three-dimensional space using said virtual reality welding system, and wherein said step of tracking further comprises calculating a real-time molten metal fluidity and heat dissipation characteristics of a simulated weld puddle are illustrated on a texture map comprising channels of information which comprise at least a first channel which stores a displacement value for any liquefied metal at the wexel location;

a second channel which stores a value giving the magnitude of heat at the wexel location;

a third channel which stores a displacement value for a solidified metal at the wexel location; and

a fourth channel which stores logical information about the wexel, including a representation regarding slag at the wexel location;

viewing a display of said virtual reality welding system showing a real-time virtual reality simulation of said first mock welding tool and said welding coupon in a virtual reality space as said simulated first mock welding tool deposits a first simulated weld bead material onto at least one simulated surface of said simulated welding coupon by forming a simulated weld puddle in a vicinity of a simulated arc emitting from said simulated first mock welding tool, said display providing visual cues during welding to provide immediate feedback by displaying overlaid colors or indicators for each tracked parameter based upon predefined limits or tolerances for said tracked parameter;

viewing, on said display, first real-time molten metal fluidity and heat dissipation characteristics of said first simulated weld puddle, said simulated weld puddle comprising a welding element displacement map grid, in which said map grid defines surface resolution with changeable channel parameter values assigned to each portion of said grid, said values dynamically changing in real-time during a simulated welding process;

using a tactilely resistive tip on said first mock welding tool to simulate resistive feedback that occurs during welding thereby driving feedback for use in adjusting or maintaining welding procedure; and

modifying, in real-time, at least one aspect of said first welding technique in response to viewing said first real-time molten metal fluidity and heat dissipation characteristics of said first simulated weld puddle, and wherein when said particles make contact with the surface of the texture map, said particles interact with each other and the texture map and build up in real-time, wherein more heat is added the nearer a wexel is to the emitter and heat is modeled in dependence on distance and the amount of time that heat is input from the emitter, and said weld puddle is rendered in virtual reality space for wexels having enough heat as wherever said heat exceeds a threshold value, the wexel map liquefies, causing the displacement value for said liquefied metal to raise up for those wexel locations, and wherein said displacement is determined by sampling the highest particles at each wexel location, and further wherein as the emitter moves on along a weld trajectory, the wexel locations left behind cool with heat removed from wexel locations at a defined rate, wherein when a cooling threshold is reached, the wexel map solidifies.

11. The method of claim 10 further comprising:

moving a second mock welding tool with respect to said welding coupon in accordance with a second welding technique;

tracking said second mock welding tool in three-dimensional space using said virtual reality welding system;

viewing said display of said virtual reality training system showing said real-time virtual reality simulation of said second mock welding tool, said welding coupon, and said first simulated weld bead material in said virtual reality space as said simulated second mock welding tool deposits a second simulated weld bead material merging with said first simulated weld bead material by forming a second simulated weld puddle in the vicinity of a simulated arc emitting from said simulated second mock welding tool;

viewing, on said display, second real-time molten metal fluidity and heat dissipation characteristics of said second simulated weld puddle; and

modifying, in real-time, at least one aspect of said second welding technique in response to viewing said second real-time molten metal fluidity and heat dissipation characteristics of said second simulated weld puddle.

12. The method of claim 10 further comprising:

moving said first mock welding tool with respect to said mock welding coupon in accordance with a second welding technique;

continuing to track said first mock welding tool in three-dimensional space using said virtual reality welding system;

viewing said display of said virtual reality training system showing said real-time virtual reality simulation of said first mock welding tool, said welding coupon, and said first simulated weld bead material in said virtual reality space as said simulated first mock welding tool deposits a second simulated weld bead material merging with said first simulated weld bead material by forming a second simulated weld puddle in the vicinity of said simulated arc emitting from said simulated first mock welding tool;

viewing, on said display, second real-time molten metal fluidity and heat dissipation characteristics of said second simulated weld puddle; and

modifying, in real-time, at least one aspect of said second welding technique in response to viewing said second real-time molten metal fluidity and heat dissipation characteristics of said second simulated weld puddle.

13. The method of claim 10 wherein

said display is presented on a user-viewable face mounted display device.

14. The method of claim 10 wherein

said display is presented on an observer display device.

15. A welding simulation comprising:

means for creating a simulated real-time weld puddle, said means emitting simulated particles onto a surface of a texture map;

means for simulating a real-time molten metal fluidity characteristic of said simulated weld puddle; and

means for simulating a real-time heat dissipation characteristic of said simulated weld puddle;

means for displaying said simulated weld puddle with at least said simulated molten metal fluidity and heat dissipation characteristics on a user-viewable face-mounted display device;

means for providing tactile resistive feedback that occurs during welding thereby driving feedback for use in adjusting or maintaining welding procedure; and

means for providing visual cues during welding to provide immediate feedback by displaying overlaid colors or indicators for each tracked parameter based upon predefined limits or tolerances for said tracked parameter, and wherein said means for providing visual cues includes

real-time molten metal fluidity and heat dissipation characteristics of said simulated weld puddle which are illustrated on a texture map comprising channels of information which comprise at least

a first channel which stores a displacement value for any liquefied metal at the wexel location;

a second channel which stores a value giving the magnitude of heat at the wexel location;

a third channel which stores a displacement value for a solidified metal at the wexel location; and

a fourth channel which stores logical information about the wexel, including a representation regarding slag at the wexel location, and wherein

when said particles make contact with the surface of the texture map, said particles interact with each other and the texture map and build up in real-time, wherein more heat is added the nearer a wexel is to the emitter and heat is modeled in dependence on distance and the amount of time that heat is input from the emitter, and said weld puddle is rendered in virtual reality space for wexels having enough heat as wherever said heat exceeds a threshold value, the wexel map liquefies, causing the displacement value for said liquefied metal to raise up for those wexel locations, and wherein said displacement is determined by sampling the highest particles at each wexel location, and further wherein as the emitter moves on along the weld trajectory, the wexel locations left behind cool with heat removed from wexel locations at a defined rate, wherein when a cooling threshold is reached, the wexel map solidifies.

16. The welding simulation of claim 15 further comprising

means for simulating a real-time weld bead wake characteristic of a simulated weld in response to a welding technique of a user.

17. The welding simulation of claim 15 further comprising

means for simulating an undercut state of a simulated weld caused by a welding technique of a user.

18. The welding simulation of claim 15 further comprising

means for simulating a porosity state of a simulated weld caused by a welding technique of a user.

19. The welding simulation of claim 15 further comprising

means for simulating a burnthrough state of a simulated weld caused by a welding technique of a user.

20. A virtual reality welding system comprising:

a programmable processor-based subsystem;

a spatial tracker operatively connected to said programmable processor-based subsystem;

at least one mock welding tool emitting simulated particles onto a surface of a welding element map and configured to be spatially tracked by said spatial tracker; and

at least one display device operatively connected to said programmable processor-based subsystem;

wherein said system is configured to simulate, in a virtual reality space, a weld puddle having real-time molten metal fluidity and heat dissipation characteristics, and display said simulated weld puddle on said at least one display device in real-time; and

wherein said weld puddle is represented in the virtual reality space as a portion of said welding element map comprising a plurality of welding elements each having changeable channel parameter values that dynamically change in real time during a simulated welding process and represent at least a molten metal fluidity displacement and a solid metal displacement, based upon a cooling threshold value for said transition between said molten metal to a solid metal.

21. The system of claim 20 wherein

said at least one display device includes at least one user-viewable face-mounted display device capable of being spatially tracked by said spatial tracker.

22. The system of claim 20 wherein

said simulated weld puddle comprises, in said virtual reality space, a plurality of temporary virtual dynamic fluid particles overlaying a portion of a virtual surface displacement map.

23. The system of claim 20 further comprising

a welding user interface simulating a real-world welding power source user interface in real time.

24. The system of claim 23 wherein

said real-time molten metal fluidity and heat dissipation characteristics of said simulated weld puddle are generated by a physics model operating on at least one GPU.

25. The system of claim 20 wherein

said system simulates in said virtual reality space, a weld bead having a real-time weld bead wake characteristic resulting from a real-time fluidity-to-solidification transition of said simulated weld puddle as said simulated weld puddle is moved, and displaying said simulated weld bead on said at least one display device.

26. The system of claim 20 further comprising a welding coupon having at least one surface and simulating a real-world part to be welded, wherein said at least one surface of said welding coupon is simulated in said virtual reality space as a double displacement layer including a solid displacement layer and a puddle displacement layer, wherein said puddle displacement layer is capable of modifying said solid displacement layer.

27. A virtual reality welding system comprising:

a programmable processor-based subsystem;

a spatial tracker operatively connected to said programmable processor-based subsystem;

at least one mock welding tool emitting simulated particles onto a surface and configured to be spatially tracked by said spatial tracker; and

at least one display device operatively connected to said programmable processor-based subsystem,

wherein said system is configured to simulate, in a virtual reality space, a weld puddle created by those particles having sufficient heat, and having real-time molten metal fluidity and heat dissipation characteristics, and display said simulated weld puddle on said at least one display device in real-time; and

wherein

said weld puddle is represented in the virtual reality space as a puddle displacement layer overlaid onto a solid displacement layer that is modified in real time by the puddle displacement layer, based upon a cooling threshold value for said transition between said molten metal to said solid metal.

28. The system of claim 27 wherein

said at least one display device includes at least one user-viewable face-mounted display device capable of being spatially tracked by said spatial tracker.

29. The system of claim 27 wherein

said simulated weld puddle comprises, in said virtual reality space, a plurality of temporary virtual dynamic fluid particles overlaying a portion of a virtual surface displacement map.

30. The system of claim 27 further comprising

a welding user interface simulating a real-world welding power source user interface in real time.

31. The system of claim 30 wherein

said real-time molten metal fluidity and heat dissipation characteristics of said simulated weld puddle are generated by a physics model operating on at least one GPU.

32. The system of claim 27 wherein

said system simulates in said virtual reality space, a weld bead having a real-time weld bead wake characteristic resulting from a real-time fluidity-to-solidification transition of said simulated weld puddle as said simulated weld puddle is moved, and displaying said simulated weld bead on said at least one display device.

33. The system of claim 27 further comprising a welding coupon having at least one surface and simulating a real-world part to be welded, wherein said at least one surface of said welding coupon is simulated in said virtual reality space as a double displacement layer including a solid displacement layer and a puddle displacement layer, wherein said puddle displacement layer is capable of modifying said solid displacement layer.

34. A virtual reality welding system comprising:

a programmable processor-based subsystem;

a spatial tracker operatively connected to said programmable processor-based subsystem;

at least one mock welding tool emitting simulated particles onto a surface and configured to be spatially tracked by said spatial tracker; and

at least one display device operatively connected to said programmable processor-based subsystem; and

wherein

said system is configured to simulate, in a virtual reality space, a weld puddle created by those particles having sufficient heat, and having real-time molten metal fluidity and heat dissipation characteristics, and display said simulated weld puddle on said at least one display device in real-time; and

wherein

said weld puddle is represented in the virtual reality space as a plurality of temporally temporary virtual dynamic fluid particles interacting with a portion of a welding element displacement map in real time to maintain a processing load of said programmable processor-based subsystem to be substantially constant during a simulated welding process.

35. The system of claim 34 wherein

said at least one display device includes at least one user-viewable face-mounted display device capable of being spatially tracked by said spatial tracker.

36. The system of claim 34 wherein

said simulated weld puddle comprises, in said virtual reality space, a plurality of temporary virtual dynamic fluid particles overlaying a portion of a virtual surface displacement map.

37. The system of claim 34 further comprising

a welding user interface simulating a real-world welding power source user interface in real time.

38. The system of claim 37 wherein

said real-time molten metal fluidity and heat dissipation characteristics of said simulated weld puddle are generated by a physics model operating on at least one GPU.

39. The system of claim 34 wherein

said system simulates in said virtual reality space, a weld bead having a real-time weld bead wake characteristic resulting from a real-time fluidity-to-solidification transition of said simulated weld puddle as said simulated weld puddle is moved, and displaying said simulated weld bead on said at least one display device.

40. The system of claim 34 further comprising a welding coupon having at least one surface and simulating a real-world part to be welded, wherein said at least one surface of said welding coupon is simulated in said virtual reality space as a double displacement layer including a solid displacement layer and a puddle displacement layer, wherein said puddle displacement layer is capable of modifying said solid displacement layer.

41. A virtual reality welding system comprising:

a programmable processor-based subsystem;

a spatial tracker operatively connected to said programmable processor-based subsystem;

at least one mock welding tool emitting simulated particles onto a surface and configured to be spatially tracked by said spatial tracker; and

at least one display device operatively connected to said programmable processor-based subsystem; and

wherein

said system is configured to simulate, in a virtual reality space, a weld puddle having real-time molten metal fluidity and heat dissipation characteristics, and display said simulated weld puddle on said at least one display device in real-time; and

wherein

said weld puddle is represented in the virtual reality space by a displacement map and a particle system in which particles in said particle system interact and collide with said displacement map, each of said particles being virtual dynamic particles and provide the liquid behavior of said weld puddle, and further wherein

a texture map is created in which each element of said map is a wexel comprised of at least four channels of information which defines a weldable surface in said virtual reality space, each channel containing at least four floating point numbers in which said channels comprise;

a first channel which stores a displacement value for any liquefied metal at the wexel location;

a second channel which stores a value giving the magnitude of heat at the wexel location;

a third channel which stores a displacement value for a solidified metal at the wexel location; and

a fourth channel which stores logical information about the wexel, including a representation regarding slag at the wexel location; and

when said particles make contact with the surface of the texture map, said particles interact with each other and the texture map and build up in real-time, wherein more heat is added the nearer the wexel is to the emitter and heat is modeled in dependence on distance and the amount of time that heat is input from the emitter, and said weld puddle is rendered in virtual reality space for wexels having enough heat as wherever said heat exceeds a threshold value, the wexel at that wexel location liquefies, causing the displacement value for said liquefied metal to raise up for those wexel locations, and wherein said displacement is determined by sampling the highest particles at each wexel location, and further wherein as the emitter moves on along the weld trajectory, the wexel locations left behind cool with heat removed from wexel locations at a defined rate, wherein when a cooling threshold is reached, the wexel at that wexel location solidifies.

42. The system of claim 41 wherein

said particles stream from an emitter and make first contact with a surface of a simulated welding coupon where said surface is defined by said wexel map;

and further wherein

said particles interact with each other and said wexel map and build up in real-time; and

further wherein

more heat is added the nearer said wexel is to the emitter.

43. The system of claim 42 wherein

heat is modeled in dependence on a distance from an arc point and an amount of time that said heat is input from said arc, and further wherein

visuals are driven by said heat wherein said weld puddle is rendered in virtual reality space for wexels having a sufficient quantity of heat.

44. The system of claim 43 wherein

said wexel map liquefies when it reaches a heating threshold, thereby causing said first channel to increase for said liquefied wexel map locations;

and further wherein

said wexel map solidifies when said liquefied wexel map locations go below a cooling threshold and a value of said first channel is converted into a value for said third channel.